Time-Varying Spectra

• In most synthesis contexts, the desired sound is not a steady, periodic signal. Rather, the spectral content of most sounds varies over time, both in terms of amplitudes and frequencies.

• In this case, it is necessary to develop a method for estimating the parameters of each spectral component over time.

• Additive synthesis is most often used to produce sounds that are quasi-periodic (having a clear sense of pitch), with a limited number of spectral components that are nearly harmonic over their duration.

• Early work producing high-quality synthesized tones using additive synthesis was conducted by Jean-Claude Risset at AT&T Bell Laboratories in the 1960s Risset and Mathews (1969).

• Another approach to the additive synthesis of musical instrument tones was reported by Grey (1974). Figure 3 plots the analyzed and approximated amplitude functions of the first 16 harmonics of a violin tone over time and frequency.

  Figure 3: (left) Analyzed amplitude functions for first 16 harmonics from a real violin tone; (right) Corresponding amplitude functions approximated with linear line segments (Grey, 1974).

  

• Grey (1974) reported parameters for a wide range of wind instrument tones, including for B$\flat$ and E$\flat$ clarinets, the oboe, bassoon, tenor saxophone, trumpet, english horn, french horn and flute.

• An inharmonic bell sound synthesis algorithm from Risset Risset and Mathews (1969) is diagrammed in Fig. 4 (and implemented as audio example D7 in the Pd documentation).

  Figure 4: Risset bell instrument algorithm and parameters (Risset and Mathews, 1969).

  

• One of the earliest music compositions created using an additive approach was Kenneth Gaburo's “Lemon Drops”, which was made on an early electronic music instrument called the “harmonic tone generator” at the University of Illinois at Urbana-Champaign.

• The Synclavier, developed in the 1980s by New England Digital, was one of the earliest digital music instruments to include a bank of oscillators for additive synthesis.

  Figure 5: The Synclavier by New England Digital.

  

• Additive approaches can be used to synthesize nearly any sound, allowing for precise control of spectral content.

• That said, it can be challenging to automatically parameterize the time-varying evolution of each spectral component over the duration of a sound. Further, this parameterization cannot generally be transposed to all pitches of an instrument sound. And attack-like sounds are generally not well approximated using sinusoids.

• Additive synthesis parameters can be analyzed by splitting a time-domain signal into blocks or “frames”, each of which is processed using the FFT (referred to as the Short-Time Fourier Transform (STFT)).

• The STFT provides a means for joint time-frequency analysis, as roughly demonstrated in the exstft.m Matlab script.