l09-synth

8/12/2019 l09-synth

1/29

Eng. 100: Music Signal Processing

DSP Lecture 9

Music synthesis

Announcements: Final Exam: Tue. Dec. 18, 4:00-6:00PM.

No office hours Fri. Nov. 9 (will reschedule closer to finals)1

8/12/2019 l09-synth

2/29

Outline

Part 1: MATLABloops

Part 2: Music synthesis via Fourier series

Part 3: Other music synthesis techniques

Part 4: Project 3 Q/A

2

8/12/2019 l09-synth

3/29

Part 1: MATLABloops

3

8/12/2019 l09-synth

4/29

MATLABanonymous functions

Tedious way to write a 4-note song in MATLAB:

sound(0.9*cos(2*pi*440*[0:1/8192:0.5]))

sound(0.9*cos(2*pi*660*[0:1/8192:0.5]))

sound(0.9*cos(2*pi*880*[0:1/8192:0.5]))

sound(0.9*cos(2*pi*660*[0:1/8192:1.0]))

Using ananonymous functioncan simplify and clarify:

S = 8192;

playnote = @(f,d) sound(0.9*cos(2*pi*f*[0:1/S:d]));

playnote(440, 0.5)

playnote(660, 0.5)

playnote(880, 0.5)

playnote(660, 1.0)

Simpler and easier to see the key elements of the song (notes and duration).

Easier to make global changes (such as amplitude 0.9).

But still tedious if the song is longer than 4 notes...

4

8/12/2019 l09-synth

5/29

MATLABloops

Using a forloop is the most concise and elegant:

S = 8192;

playnote = @(f,d) sound(0.9*cos(2*pi*f*[0:1/S:d]));

fs = [440 660 880 660];

ds = [0.5 0.5 0.5 1.0];

for index=1:numel(fs)

playnote(fs(index), ds(index))

end

Here is another loop version that sounds better:

S = 8192;

note = @(f,d) 0.9*cos(2*pi*f*[0:1/S:d]);

fs = [440 660 880 660];

ds = [0.5 0.5 0.5 1.0];

x = [ ] ;

for index=1:numel(fs)

x = [x note(fs(index), ds(index))];

end

sound(x, S)

5

play

8/12/2019 l09-synth

6/29

Part 2: Music synthesis via Fourier series

6

8/12/2019 l09-synth

7/29

Additive Synthesis: Mathematical formula

Simplified version of Fourier series for monophonic audio:

x(t) =K

k=1

ckcos2

k

Tt

No DC term for audio: c0= 0.

Phase unimportant for monophonic audio, so k= 0.

Example:

x(t) = 0.5cos(2400t) +0.2cos(2800t) +0.1cos(22000t)

7

8/12/2019 l09-synth

8/29

Example: Why we might want harmonics

0 0.001 0.002 0.003 0.004 0.005 0.006 0.007 0.008 0.009 0.011

0.5

0

0.5

1

t

x(t),y(t)

x

y

y(t) = 0.5cos(2400t)x(t) = 0.5cos(2400t) +0.2cos(2800t) +0.1cos(22000t)

% fig_why1: example of additive synthesis

S = 44100;

N = 0 . 5 * S; % 0.5 sec

t = [0:N-1]/S; % time samples: t = n/Sy = 0 . 5 * cos(2*pi * 400 * t);

x = y + 0 . 2 * cos(2*pi * 800 * t) + 0.1 * cos(2*pi * 2000 * t);

plot(t, x, -, t, y, --), legend(x, y)

xlabel t, ylabel x(t), y(t), axis([0 0.01 -1 1])

Same fundamental period, same pitch, different timbre.8

play play

8/12/2019 l09-synth

9/29

MATLAB implementation: Simple

Example: x(t) = 0.5cos(2400t) +0.2cos(2800t) +0.1cos(22000t)

A simple MATLABversion looks a lot like the mathematical formula:

S = 44100;

N = 0 . 5 * S; % 0.5 sec

t = [0:N-1]/S; % time samples: t = n/S

x = 0 . 5 * cos(2 * pi * 400 * t) ...

+ 0.2 * cos(2 * pi * 800 * t) ...

+ 0.1 * cos(2 * pi * 2000 * t);

There are many hidden forloops above. Where???

MATLABsaves us from the tedium of writing out those loops,

thereby making the syntax look more like the math.In traditional programming languages like C,

one would have to code all those loops.

This simple implementation is still somewhat tedious,particularly for signals having more harmonics.

9

8/12/2019 l09-synth

10/29

C99 implementation

#include

void my_signal(void){

float S = 44100;

int N = 0.5 * S; // 0.5 sec

float x[N]; // signal samples

for (int n=0; n < N; ++n)

{float t = n / S;

x[n] = 0.5 * cos(2 * M_PI * 400 * t)

+ 0.2 * cos(2 * M_PI * 800 * t)

+ 0.1 * cos(2 * M_PI * 2000 * t);

}

}

10

8/12/2019 l09-synth

11/29

Example revisited: Square wave

0 5 10 15

1

0

1

sin(t)

0 5 10 15

1

0

1

sin(t) + sin(3t)/3

0 5 10 15

1

0

1

sin(t) + sin(3t)/3 + sin(5*t)/5

0 5 10 15

1

0

1

sin(t) + ... + sin(15*t)/15

0 5 10 15

1

0

1

sin(t) + ... + sin(49*t)/49

0 5 10 15

1

0

1

sin(t) + ... + sin(999*t)/999

t

x(

t)

Many dozens of harmonics to get a good square wave approximation.

11

8/12/2019 l09-synth

12/29

MATLABimplementation: Loop over harmonics

Example: x(t) = 0.5cos(2400t) +0.2cos(2800t) +0.1cos(22000t)

S = 44100;

N = 0.5 * S; % 0.5 sec


c = [0.5 0.2 0.1]; % amplitudes

f = [ 1 2 5 ] * 400; % frequencies

x = 0 ;for k=1:length(c)

x = x + c ( k ) * cos(2 * pi * f(k) * t);

end

I think this version is the easiest to read and debug.

It looks the most like the Fourier series formula: x(t) =K

k=1

ckcos2k

Tt .

In fact it is a slight generalization. In Fourier series, the frequencies are multiples: k/T. In this code, the frequencies can be any values we put in the farray.

12

8/12/2019 l09-synth

13/29

Example: Square wave via loop, with sin

S = 44100;

N = 0.5 * S; % 0.5 sect = [0:N-1]/S; % time samples: t = n/S

c = 1 ./ [1:2:15]; % amplitudes

f = [1:2:15] * 494; % frequencies

x = 0 ;

for k=1:length(c)

x = x + c ( k ) * sin(2 * pi * f(k) * t);end

0 0.001 0.002 0.003 0.004 0.005 0.006 0.007 0.008 0.009 0.011

0.5

0

0.5

1

t

x(t)

Music example, circa 1978:13

play

play

8/12/2019 l09-synth

14/29

Example: Square wave via loop, with cos

S = 44100;

N = 0.5 * S; % 0.5 sect = [0:N-1]/S; % time samples: t = n/S

c = 1 ./ [1:2:15]; % amplitudes

f = [1:2:15] * 494; % frequencies

x = 0 ;

for k=1:length(c)

x = x + c ( k ) * cos(2 * pi * f(k) * t);end

0 0.001 0.002 0.003 0.004 0.005 0.006 0.007 0.008 0.009 0.01

2

1

0

1

2

t

x(t)

14

play

8/12/2019 l09-synth

15/29

MATLAB implementation: Concise

We can avoid writing any for loops (and reduce typing) byusing the following more concise (i.e., tricky) MATLABversion:

S = 44100;

N = 0.5 * S; % 0.5 sec


c = [0.5 0.2 0.1]; % amplitudes

f = [ 1 2 5 ] * 400; % frequenciesz = cos(2 * pi * f * t);

x = c * z;

cis 13 fis 31 tis 1N

zis ??xis ??

Where are the (hidden) loops in this version???

Use this approach or the previous slide in Project 3.15

8/12/2019 l09-synth

16/29

Part 3: Other music synthesis techniques

16

8/12/2019 l09-synth

17/29

Frequency Modulation (FM) synthesis

In 1973, John Chowning of Stanford invented the use of frequency modula-tion (FM) as a technique for musical sound synthesis:

x(t) =A sin(2f t+Isin(2gt)),

whereIis themodulation indexand f andgare both frequencies.(Yamaha licensed the patent for synthesizers and Stanford made out well.)

This is a simple way to generate complicated periodic signals that are richin harmonics.

However, finding the value of I that gives a desired effect requires experi-

mentation.

17

8/12/2019 l09-synth

18/29

FM example 1: Traditional

S = 44100;

N = 1 . 0 * S;


I = 7; % adjustable

x = 0 . 9 * sin(2*pi*400*t + I * sin(2*pi*400*t));

Very simple implementation, yet harmonically very rich spectrum:

0 400 1200 2000 4000 80000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

frequency [Hz]

spectrum

18

play

8/12/2019 l09-synth

19/29

FM example 2: Time-varying

Time-varying modulation index: x(t) =A sin2f t+ I(t)

sin(2gt)

.

Simple formula / implementation can make remarkably intriguing sounds.

S = 44100;

N = 1 . 0 * S;


I = 0 + 9*t/max(t); % slowly increase modulation indexx = 0 . 9 * sin(2 * pi * 400 * t + I .* sin(2*pi*400*t));

What is the most informative graphical representation? ??

19

play

8/12/2019 l09-synth

20/29

FM example 2: Time-varying

Time-varying modulation index: x(t) =A sin2f t+ I(t)

sin(2gt)

.

Simple formula / implementation can make remarkably intriguing sounds.

S = 44100;

N = 1 . 0 * S;


I = 0 + 9*t/max(t); % slowly increase modulation indexx = 0 . 9 * sin(2 * pi * 400 * t + I .* sin(2*pi*400*t));

What is the most informative graphical representation? ??

Spectrogram of FM signal with timevarying modulation

t [sec]

frequency[Hz]

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

400

1200

2000

4000

20

play

8/12/2019 l09-synth

21/29

Nonlinearities

Another way to make signals that are rich in harmonics is to use a nonlinearfunction such asy(t) =x9(t).

S = 44100;

N = 1 . 0 * S;


b = 0.2; % adjustable saturation point

x = cos(2*pi*400*t);

y = x.9;

0 0.001 0.002 0.003 0.004 0.005 0.006 0.007 0.008 0.009 0.011

0.5

0

0.5

1

t [sec]

x(t)

0 0.001 0.002 0.003 0.004 0.005 0.006 0.007 0.008 0.009 0.011

0.5

0

0.5

1

t [sec]

y(t)

0 400 1200 2000 4000 80000

0.2

0.4

0.6

frequency [Hz]

spectrumo

fy

21play

8/12/2019 l09-synth

22/29

Nonlinearities in amplifiers

High quality audio amplifiers are designed to be very close to linear be-cause any nonlinearity will introduce undesired harmonics.

Quality amplifiers have a specified maximum total harmonic distortion(THD) that quantifies the relative power in the output harmonics for apure sinusoidal input.

A formula for THD is:

THD=c22+c2

3+c2

4+

c21

100%

What is the best possible value for THD???

On the other hand, electric guitarists often deliberately operate their am-plifiers nonlinearly to induce distortion, thereby introducing more harmon-

ics than produced by a simple vibrating string. pure:

distorted:

22

play

play

8/12/2019 l09-synth

23/29

Envelope of a musical sound

23

E l l T i hi tl

8/12/2019 l09-synth

24/29

Envelope example: Train whistle

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.61

0.5

0

0.5

1

x(t)

train whistle signal

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.60

0.2

0.4

0.6

0.8

1

t [sec]

envelope(t)

train whistle envelope

24play

E l l Pl k d it

8/12/2019 l09-synth

25/29

Envelope example: Plucked guitar

0 0.5 1 1.5 2 2.5 3 3.5 4 4.51

0.5

0

0.5

1

x(t)

guitar signal

0 0.5 1 1.5 2 2.5 3 3.5 4 4.50

0.2

0.4

0.6

0.8

1

t [sec]

envelope(t)

plucked guitar envelope

25play

Envelope implementation

8/12/2019 l09-synth

26/29

Envelope implementation

S = 44100;

N = 1 * S;

t = [0:N-1]/S;c = 1 ./ [1:2:15]; % amplitudes

f = [1:2:15] * 494; % frequencies

x = 0 ;

for k=1:length(c)

x = x + c ( k ) * sin(2 * pi * f(k) * t);

end

env = (1 - exp(-80*t)) .* exp(-3*t); % fast attack; slow decay

y = e n v .* x;

0 0.2 0.4 0.6 0.8 11

0.5

0

0.5

1

t

x(t)

0 0.2 0.4 0.6 0.8 11

0.5

0

0.5

1

t

y(t)

26

play play

Attack Decay Sustain Release (ADSR)

8/12/2019 l09-synth

27/29

Attack, Decay, Sustain, Release (ADSR)

These refer to the time durations of 4 components of the enve-lope as controlled in many synthesizers.

27

Summary

8/12/2019 l09-synth

28/29

Summary

There are numerous methods for musical sound synthesis

Additive synthesis provides complete control of spectrum Other synthesis methods provide rich spectra with simple

operations (FM, nonlinearities)

Time-varying spectra can be particularly intriguing

Signal envelope (time varying amplitude) also affects soundcharacteristics

Ample room for creativity and originality!

28

8/12/2019 l09-synth

29/29

Part 4: Project 3 Q/A?

29

Documents

l09-synth