Worklog for task "Research parametric synthesis of Russian sounds and transitions between them"

New Progress: Structural Sound Representation and the Human ↔ AI Loop

Several encouraging signs of the current prototype have been recorded.

1. We Are No Longer Working with Waveforms, but with a Compact Program

The current synthesizer operates on individual initial state variables and separate scenarios on a shared timeline. This is fundamentally different from directly storing thousands or tens of thousands of samples.

Despite the model's coarseness, a relatively small number of parameters already yields sounds that a human sometimes recognizes as letters, phonemic fragments, or syllables. This does not yet prove the model's sufficiency for natural speech, but it shows that the parametric space is not completely chaotic.

2. The Human Effectively Acts as a Perceptual Validator

The AI does not have a reliable standard for how natural the synthesis sounds. Therefore, the current working loop looks like this:

expected sound → scenario → synthesis → human listens → evaluates quality → AI suggests parameter changes → new synthesis.

Here, the human provides the final perceptual feedback, while the AI helps analyze structural differences and navigate the parameter space.

3. Using a Real Recording as a Local Reference

When searching for the sound Kha, a recording of the word/phrase KuKhaRe was used, where the desired segment is actually present. The AI received:

• a log of the synthetic variant;
• a log of the real recording;
• information on where the target fragment is located within the recording.

Next, it compares structural features and suggests which scenario parameters can be changed to get closer to the desired section.

This does not provide an instantaneous result, but it looks like a working human-AI loop for approximating the target sound.

4. Perhaps the Current Logs Are Too Heavily Compressed

There is a suspicion that the quality of the analytical logs is insufficient for precise comparison. If a human hears differences that the AI cannot infer from the log, the bottleneck may lie not only in the generator, but also in the analysis resolution.

It is necessary to separately check how the quality of the AI's suggestions changes when log compression is reduced and the number of temporal/spectral features is increased.

5. Accidental Real Sounds Found Are Also Knowledge

During the search for Kha, one of the variants was perceived as Va.

This is useful for two reasons:

1. the found sound itself may be needed later;
2. the transition from the previous "hissing" to the perceived Va shows the direction in the parametric space.

Such accidental finds must be saved along with parameter changes, because they help map the space of possible sounds.

6. The Space Still Remains in the Domain of Speech-Like Sounds

Despite the potential universality of AudioContext, when changing the parameters of the current voice-oriented generator, we still mainly encounter sounds that are perceived as vocal/phonetic or close to them, rather than random real sounds like barking, bird singing, or hoof beats.

This does not prove universality, nor does it prove the completeness of human speech coverage. A more cautious conclusion:

the current generator architecture already contains a strong inductive bias towards the voice-like sound space.

This can be a useful property because the search is performed not across the entire possible audio space, but within a narrower region.

7. Refined Practical Goal

Now the main question is not only "what parameters are needed for speech synthesis," but:

how compactly can the required region of human speech be described without reverting to thousands of samples.

The extreme points are already clear:

• waveform / μ-law — high quality and universality, but thousands of values per second;
• VoiceScenario — compactness and controllability, but currently unknown quality limits.

Further work should explore the gap between them and the dependency:

quality ↑ ↔ complexity/size of representation ↑.

A possible path is to gradually add only those mechanisms that provide a noticeable perceptual gain while preserving the model's structural nature.