Worklog for task "Research parametric synthesis of Russian sounds and transitions between them"

New Findings on Synthesizer Architecture and Reversibility

An analysis of the full VoiceScenario -> PCM pipeline has been conducted.

1. Synthesis is Entirely in Our Code

It has been clarified that there is no external hidden audio engine generating the PCM. The final audio track array is entirely computed inside synthesizeScenario:

VoiceScenario -> VoiceControlState(t) -> excitation -> resonators -> value -> Float32Array.

At each sample, the code independently calculates the final value and writes it to output[sampleIndex]. The browser's AudioBufferSourceNode then simply plays back the ready-to-use PCM.

This means the entire DSP can be instrumented and analyzed at every step.

2. Control Parameters and Measured Acoustic Metrics are Different Levels

After fixing the scenario representation, it became clear that even nearly constant control parameters produce high-frequency changes in the measured audio. This is expected.

Reasons:

• a constant f0Hz determines the rate of change of voicePhase, not a constant waveform value;
• a constant noiseLevel controls the amplitude of a random source that changes every sample;
• resonators have an internal stateful state (low, band) and continue to change dynamically even with constant parameters;
• RMS, spectralFlatness, and dominantFrequency depend on multiple control parameters and the internal DSP state simultaneously.

Therefore, control curve != measured acoustic curve.

3. An Additional Layer Hidden from the Scenario Has Been Found: Internal DSP State

Between VoiceControlState and PCM, there are internal states:

• voicePhase;
• resonator1/2/3.low;
• resonator1/2/3.band;
• the state of the deterministic PRNG;
• active automations.

It is precisely this layer that explains why nearly horizontal control lines can lead to a complex high-frequency waveform and uneven acoustic metrics.

4. Next Research Direction

We need to move from the question "which log metric corresponds to a parameter" to the system identification task:

how does a change in each VoiceControlState parameter affect the waveform and measurable acoustic features?

Proposed experiment: a parameter sweep for each control parameter while keeping the others fixed, collecting the waveform + spectral/temporal characteristics. This will allow building a control -> acoustic features map and understanding which parameters are actually observable from the audio, which depend on combinations of parameters, and which are completely invisible to the current logger.

The basic theory of PCM was also discussed: a constant unsigned 8-bit PCM level (e.g., 255,255,255,...) does not produce a stable tone because there is no periodic change; transitions are primarily what is heard. This emphasizes that an individual PCM value is an instantaneous amplitude, while frequency/timbre/loudness are properties of the sequence's structure over time.