Worklog for task "Research parametric synthesis of Russian sounds and transitions between them"

Research Progress

The core architectural approach has been established and experimentally verified: human sound is described as a continuous scenario of modifying a universal DSP state over time, rather than as a special letter/phoneme renderer.

Implemented

• VoiceScenario with initial and a shared timeline.
• set and animate commands, including cubic bezier.
• Continuous state with parameters for excitation, periodicity, noise, F0, source shape, three universal resonance channels, and final level.
• A new command for a single parameter starts moving from its current value and replaces any previous future automation.
• The synthesizer accepts both a single VoiceScenario and VoiceScenario[].
• The array of scenarios unfolds into a single timeline without resetting the source phase or DSP state between elements.
• This made it possible to assemble Privet as [PRI_SCENARIO, VET_SCENARIO] rather than creating a separate word scenario.
• The noise source was migrated to a seeded PRNG for reproducible PCM.
• Synthetic sound passes through the same VoiceLogger as live recordings.

Experimental Cycle

The following scheme is used:

live recording → log → analysis → scenario → synthesis → same logger → comparison → adjustment

The current logger measures RMS, peak, ZCR, spectral centroid, spectral flatness, dominant frequency, and several energy bands. It is still rough: analysis at 128 samples yields a frequency step of about 375 Hz, so it needs further improvement.

Results for Individual Sounds

A

The sustained A has already turned out relatively natural. This confirmed that timeline + periodic source + universal resonances are generally suitable for some stable voice states.

S

After several iterations, it turned out that subjective "hissing" cannot be reduced to an excess of high frequencies. A live, strong S simultaneously has a high ZCR, a high spectral centroid, and intense high-frequency turbulence. Boosting only the overall amplitude does not solve the source quality problem.

Sh

Sh is treated as a continuous noise state close to S, but with a lower and broader spectral energy distribution.

T

T confirmed the presence of short-term events that cannot be stretched as a stable state. A live T is characterized by a very short high-frequency burst and rapid decay; in several recordings, the main peak often turned out to be around 15–16 kHz. This was successfully expressed using the same timeline model without a special T type.

Composite Scenarios

The scenarios Pri and Vet were created, and their composition via an array of scenarios was tested. They are already recognizable, but the sound quality remains distinctly computer-generated.

After comparison with a live Privet, it was revealed that:

• the initial scenarios were greatly stretched in time;
• manual deep oscillations of sourceLevel for R sound mechanical;
• the final T in one of the variants reached peak = 1, i.e., hit a hard clamp;
• the scenarios were shortened to more conversational durations and the final T was attenuated.

Current DSP Limitations

The main risk now lies not in the scenario architecture, but in the quality of the sound engine itself. Possible missing universal mechanisms:

• a more realistic glottal source;
• spectral tilt;
• aspiration / breathiness;
• broadband/direct noise path;
• a more physical turbulence model;
• antiresonances;
• nasal tract;
• source-filter interaction;
• a richer vocal tract model;
• possibly jitter/shimmer and other small variations;
• clipping control instead of a hard clamp that hides the causes.

Architectural Conclusions

• letters, phonemes, and syllables should remain metadata, not DSP ontology;
• stable, transitional, and plosive phenomena can currently be expressed using a single timeline model;
• scenario composition seems to be the right direction;
• generative TTS is not suitable as a research subject, since the task requires explicit programming of new sounds, including previously unseen ones;
• Pink Trombone and similar physical models are useful as references, but their quality shows that a single continuous tract model is not enough.

Next Step

A separate related research task cmtzxmokp085eqw0qtb7vg7c2 has been created regarding high-quality programmatic human voice synthesis in the browser. It aims to find browser-native/browser-capable projects and DSP approaches that qualitatively reproduce at least a few human sounds or syllables without generative TTS, and to understand which universal mechanisms should be ported to the current engine.