Signals, Instruments, and Systems – W5

Introduction to Signal Processing – Convolution, Sampling, Reconstruction

1

Motivation from Week 1 Lecture

Sensing

Processing Communication

Processing

Mobility

In-situ instrument

Visualization

Storing

Transportation channel Base station

Highlighted blocks are those mainly leveraging the content of this lecture.

2

Convolution

3

Convolution

( ) ∫∞

∞−

−⋅=∗ τττ dtgftgf )()()(

• For each value of t:1. Flip (reflect) g2. Shift g by t3. Multiply f and g4. Integrate over τ

• Note that the result does not depend on τ!

∫∞

∞−

−⋅

−⋅−→−

−→

τττ

ττττ

ττ

dtgf

tgftgg

gg

)()( 4)

)()( 3))()( 2)

)()( 1)

4

[Matlab demo “cconvdemo”]

http://users.ece.gatech.edu/mcclella/matlabGUIs/5

Examples

=-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

0.2

0.4

0.6

0.8

1

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

0.2

0.4

0.6

0.8

1

*-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

0.2

0.4

0.6

0.8

1

1.2

1.4

6

Examples

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

0.2

0.4

0.6

0.8

1

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

0.2

0.4

0.6

0.8

1

*-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

0.2

0.4

0.6

0.8

1

=

7

Examples

=-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

0.2

0.4

0.6

0.8

1

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

0.2

0.4

0.6

0.8

1

*-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

0.2

0.4

0.6

0.8

1

8

Examples

=-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

0.2

0.4

0.6

0.8

1

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

0.2

0.4

0.6

0.8

1

*-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

0.2

0.4

0.6

0.8

1

9

Examples

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

0.2

0.4

0.6

0.8

1

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

0.2

0.4

0.6

0.8

1

*-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

0.2

0.4

0.6

0.8

1

=

10

Convolution in Time and Frequency Domains

))(ˆˆ()(ˆ)()()(

)(ˆ)(ˆ)(ˆ))(()(

ξξ

ξξξ

gfhtgtfth

gfhtgfth

∗=⇔⋅=

⋅=⇔∗=

Time domain Frequency domain

Ord

inar

yfr

eque

ncy

))((21)()()()(

)()()())(()(

ωπ

ω

ωωω

GFHtgtfth

GFHtgfth

∗=⇔⋅=

⋅=⇔∗=

Ang

ular

freq

uenc

y

11

Convolution Properties

)()()(

)()()(

)()(

agfgafgfa

hfgfhgf

hgfhgf

fggf

∗=∗=∗

∗+∗=+∗

∗∗=∗∗

∗=∗

tionmultiplica scalar ity withAssociativ

vityDistributi

ityAssociativ

ityCommutativ

12

Discrete Convolution

( )

( ) ∑

∫

∞

−∞=

∞

∞−

−⋅=∗

−⋅=∗

mmngmfngf

dtgftgf

][][][

)()()( τττ

• Similar to the continuous version• The integral becomes an infinite sum• Matlab, operating on a computer (i.e. a digital device) can only emulate

continuity and therefore use the discrete version with an adjustable discretization level (in time and amplitude)

13

[Matlab demo “dconvdemo”]

http://users.ece.gatech.edu/mcclella/matlabGUIs/14

Sampling

15

• Digital– Discrete– Digital world

Analog – Digital

• Analog– Continuous– Real world

0 100 200 300 400 500 600 700 800 900 1000-4

-2

0

2

4

6

8

10

Time

Sig

nal A

mpl

itude

Time0 100 200 300 400 500 600 700 800 900 1000

-4

-2

0

2

4

6

8

10

Sig

nal A

mpl

itude

16

Train of Dirac Impulses

17

∑∞

−∞=

−=n

nTttx )()( δ

Train of Dirac Impulses

18

∑

∫

∑

∞

−∞=

−

−

∞

−∞=

−=

==↔

−=

n

T

T

tjk

n

Tk

TX

Tdtetx

Tatx

nTttx

πωδπω

δ

ω

22)(

1)(1)(

)()(

2/

2/

0

Time domain Frequency domain

F()

Sampling in Time Domain

19

∑∑

∑∞

−∞=

∞

−∞=

∞

−∞=

−=−==

−=

nnp

n

nTtnTxnTttxtptxtx

nTttp

tx

)()()()()()()(

)()(

)(

δδ

δ Train of Dirac pulses

T = sampling periodf = 1/T = sampling

frequency

Sampling in Frequency Domain

20

∑

∑

∑

∞

−∞=

∞

−∞=

∞

−∞=

−=

−=

−=

=

=

kS

kSp

kS

p

p

kXT

kXT

X

kT

P

PXX

tptxtx

)(1

)(*)(1)(

)(2)(

)(*)(21)(

)()()(

ωω

ωωδωω

ωωδπω

ωωπ

ω

Tsπω 2= Samplingfrequency

Convolution propertyTime domain

Frequency domain

Sampling a Band-Limited Signal

21

∑∞

−∞=

−=

=

kS

p

kXT

PXX

)(1

)(*)(21)(

ωω

ωωπ

ω

∑∞

−∞=

−=k

SkTP )(2)( ωωδπω

X(ω) → spectrum of signal x(t) with highest frequency < ωm

Sampling frequency ωs > 2 ωm

Sampling a Band-Limited Signal

22

X(ξ) → spectrum of signal x(t) with highest frequency < 2 kHz

Sampling frequency: 5 kHz > 2 * 2 kHz

Sampling a Band-Limited Signal

23

X(ξ) → spectrum of signal x(t) with highest frequency < 3 kHz

Sampling frequency: 5 kHz < 2 * 3 kHz

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

-1.5

-1

-0.5

0

0.5

1

1.5

Time [s]

Original Signal

)52sin(2.0)22sin(4.0)2sin()( ttttf ⋅+⋅+= πππ24

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

-1.5

-1

-0.5

0

0.5

1

1.5

Time [s]

Too Few Samples (1Hz)

→ Data is lost25

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

-1.5

-1

-0.5

0

0.5

1

1.5

Time [s]

Too Many Samples (100 Hz)

→Redundant data→Increase of data size

26

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

-1.5

-1

-0.5

0

0.5

1

1.5

Time [s]

Minimal Possible Sampling (> 10 Hz)

27

Nyquist–Shannon Theorem• If a function x(t) contains no frequencies higher than

B Hz, it is completely determined by giving its coordinates at a series of points spaced 1/(2B) seconds apart.

• Sampling frequency must be at least two times greater than the maximal signal frequency

28

Sampling in Practice

• Sampling frequency two times greater than maximal frequency is the limit

• If possible, try to use a sampling frequency 10 times greater than the maximal frequency

• Audio CD, sampling at 44.1 kHzMaximal hearable frequency: 20 kHz

29

Signal Reconstruction

30

31

spectrum of original signal

spectrum of reconstructedsignal

)(tx x

( )∑+∞

−∞=

−=n

nTttp δ)(

)(txr)(ωH

spectrum of sampled signalsampling frequency ωs > 2 ωm

filtering filter cut-off frequency ωm < ωc < (ωs –ωm)

If there is no overlap between the shifted spectra the signal xr(t) can beperfectly reconstructed from x(t)

)(txp

)()()( ωωω HXX pr =

32

spectrum of original signal

spectrum of reconstructedsignal

)(tx x

( )∑+∞

−∞=

−=n

nTttp δ)(

)(txr)(ωH

spectrum of sampled signalsampling frequency fs < 2 fm

filtering filter cut-off frequency fcfm < fc < (fs –fm)

If there is overlap between the shifted spectra the signal xr(t) cannot beperfectly reconstructed from x(t)

)(txp

)()()( ωωω HXX pr =

)(tx x

( )∑+∞

−∞=

−=n

nTttp δ)(

)(txr

Time Domain Interpretationof Signal Reconstruction

ωm < ωc < (ωs –ωm)

( ) ( )

( ) ( )

( ) ( )( )( )nTtnTtTnTx

nTthnTx

thnTtnTx

thtxtx

c

n

n

n

pr

−−

=

−=

−=

=

∑

∑

∑

∞

−∞=

∞

−∞=

∞

−∞=

πω

δ

sin

)(*

)(*)()(

( )t

tTth cπωsin)( = with

F-1()

The lowpass filter interpolates the samples assuming x(t) contains no energy at frequencies > ωc(ωc =cutoff frequency) 33

34From Prof. A. S. Willsky, Signals and Systems course

Signal Reconstruction Methods

∗

−⋅=

−⋅=

∑

∑∞

−∞=

∞

−∞=

sns

s

s

n

TtnTtnxtx

TnTtnxtx

sinc

sinc

)(][)(

][)(

δ

• Signal has to be band limited (i.e. Fourier transform for frequencies greater than B equal 0)

• The sampling rate must exceed twice the bandwidth, 2B

BTs 2

1<

35

Bfs 2>or

1. Zero-order hold2. First-order hold – linear interpolation 3. Whittaker-Shannon interpolation (bandlimited interpolation):

(Alternative equivalent formulation)

36From Prof. A. S. Willsky, Signals and Systems course

Aliasing

37

No Problems in Reconstruction

38

Reconstruction Problems

Alias

Overlapping

39

Harmonics

Fundamental Frequency

40

-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5-1

-0.5

0

0.5

1

Time [s]

1Hz2Hz3Hz4Hz

Harmonics

41

La-Tone (440 Hz) sampled at 44.1 kHz (CD standard)

42

La-Tone (440 Hz) sampled at 4 kHz without filtering

43

La-Tone (440Hz) sampled at 4 kHz filtered at 2 kHz

44

Aliasing Audio Examples

Original sound Aliases 4 kHz Correct sampling 4 kHz

45

Moiré Pattern

46

47

Aliasing Video Examples

http://www.youtube.com/watch?v=jHS9JGkEOmA

https://www.youtube.com/watch?v=R-IVw8OKjvQ

Conclusion

48

Take Home Messages

When you sample a signal …

1. Make sure you know what the maximum frequency fmax is or enforce it through a low-pass filter

2. Make sure you sample at fs > 2 fmax(Nyquist rule)

49

Books:• Ronald W. Schafer and James H. McClellan

“DSP First: A Multimedia Approach”, 1998• A. Oppenheim and A. S. Willsky with S. Hamid,

“Signals and Systems”, Prentice Hall, 1996.

50

Additional Literature – Week 5

Slide Number 1Motivation from Week 1 LectureSlide Number 3Convolution[Matlab demo “cconvdemo”]ExamplesExamplesExamplesExamplesExamplesConvolution in Time and Frequency DomainsConvolution PropertiesDiscrete Convolution[Matlab demo “dconvdemo”]Slide Number 15Analog – DigitalTrain of Dirac ImpulsesTrain of Dirac ImpulsesSampling in Time DomainSampling in Frequency DomainSampling a Band-Limited SignalSampling a Band-Limited SignalSampling a Band-Limited SignalOriginal SignalToo Few Samples (1Hz)Too Many Samples (100 Hz)Minimal Possible Sampling (> 10 Hz)Nyquist–Shannon TheoremSampling in PracticeSlide Number 30Slide Number 31Slide Number 32Time Domain Interpretation�of Signal ReconstructionSlide Number 34Signal Reconstruction Methods�Slide Number 36Slide Number 37No Problems in ReconstructionReconstruction ProblemsHarmonicsHarmonicsLa-Tone (440 Hz) sampled at 44.1 kHz (CD standard)La-Tone (440 Hz) sampled at �4 kHz without filteringLa-Tone (440Hz) sampled at �4 kHz filtered at 2 kHzAliasing Audio ExamplesMoiré PatternSlide Number 47Slide Number 48Take Home MessagesAdditional Literature – Week 5