Lecture 16 Outline: Linear Convolution, Block-by Block

Convolution, FFT/IFFTl Announcements:

l HW 4 posted, due tomorrow at 4:30pm. No late HWs as solutions will be available immediately.

l Midterm details on next pagel HW 5 will be posted Fri, due following Fri May 18l Guest lecture Wed on AR/VR (Gordon Weitzstein)

l Linear Convolution from Circular

l Block-by-block convolution: Overlap/Add

l FFT/IFFT and its Complexity

Midterm Detailsl Time/Location: Friday, May 11, 1:30-2:50pm in this room.

l Please confirm special arrangements by tomorrow if you have made them.

l Open book and notes – you can bring any written material you wish to the exam. Calculators and electronic devices not allowed.

l Will cover all class material from Lectures 1-13.

l Practice MT posted, worth 25 extra credit points for “taking” it.l Can be turned in any time up until you take the exam (send scanned version to TAs, or

give them a hard copy in OHs/section)l Solutions given when you turn in your answers l In addition to practice MT, we will also provide additional practice problems/solns

l MT Review in class today

l Discussion Section May 8, 4:30-6 (MT review and practice problems)

l Regular OHs for me/TAs this week (no new HW this week)

Review of Last Lecture: Key DFT Properties

l Circular Time Shift ® DFT mutiplication with exponential

l Circular Frequency Shift ® multiplication in time by exponential

l Circular convolution in time is multiplication in frequency

l Multiplication in time is circular convolution in frequency

( )( )[ ] [ ] [ ]kXekXWmnxkm

Njkm

NN

p2-

=«-

[ ] [ ] ( )( )[ ]Nln

Njln

N lkXnxenxW -«=-p2

[ ] ( )( )[ ] [ ] [ ]kXkXmnxmxDFTN

mN 21

1

021 «-å

-

=

[ ] [ ] [ ] [ ]kXkXnxnx N 2121 !«×

Computing Circular Convolution;Circular vs. Linear Convolution

l Computing circular convolution:l Linearly convolve and

l Place sequences on circle in opposite directions, sum up all pairs, rotate outer sequence clockwise each time increment

l Matlab Command:

l Circular versus Linear Convolution

[ ] [ ] [ ] [ ]ïî

ïíì

-££-= å-

=

otherwise0

10~~1

021

21Nnmnxmxnxnx

N

mN![ ]nx1~ [ ]nx2~

[ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ]13223100 212121210241 xxxxxxxxnxnx n +++==! [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ]23320110 212121211241 xxxxxxxxnxnx n +++=

=!

[ ] ][21 nxnx =

n0 3

1

1 2

[ ] [ ]nxnx 21 4

n0 3

4

1 2

x1[n] * x2[n]

n0 3

1

1 2 64 5

1

2 2

3 3

4

y = ifft(fft(x1,N).*fft(x2,N))

Linear Convolution using Circular Convolution

l Want to linearly convolve two sequencesl e.g. to obtain the output of a filter to an input sequence

● x[n] has length L, h[n] has length P, y[n] has length L+P-1

l Obtain linear convolution as follows:l Zero pad x[n] by appending P-1 zeros to get xzp[n]; 0£n£L+P-2l Zero pad h[n] by appending L-1 zeros to get hzp[n]; 0£n£L+P-2l Both sequences are of length M=L+P-1, same as y[n]l Take circular convolution of zero padded sequences l This yields the linear convolution:

● Zeros padding removes circular effect

[ ] [ ] [ ] [ ] [ ] [ ] [ ]mhmnxmnhmxnhnxnyP

m

L

måå-

=

-

=

-=-==1

0

1

0

*

, the same as the linear convolution .

Example: Linear from Circularl Linear Convolution

l Linear from Circular with Zero Padding

][2 nx

n0 3

1

1 2

x1[n] * x2[n]

n0 3

1

1 2 64 5

2

3

4

n0 3

1

1 2 4 5

[ ]nx1

87

1

2

3

4 4

][,2 nx zp

n0 3

1

1 2

0 3

1

1 2 4 5

[ ]nx zp,1

L=4

P=6

M=L+P-1=9

64 5 7 8

n6 7 8

[ ] [ ] 4|21 6 =nnxnx

n0 3

4

1 2 54

1 1

1

1

00

0

0

0

1 0

0

0

11

1

1

1

n=0

1 1

1

1

00

0

0

0

1 1

0

0

01

1

1

1

n=1

[ ] [ ]nxnx zpzp ,2,1 9

Block Convolution using Overlap Methods

l Want block-by-block linear convolution for long sequencesl Uses fixed hardware. Has fixed delay/complexity

l Goal: compute linear convolution of x[n] and y[n]l x[n] very long, h[n] has length Pl Want to break x[n] into shorter blocks and compute portions of

y[n] block-by-block.

l Overlap-Add Methodl Breaks x[n] into non-overlapping segments of length L:

l Convolve each segment with h[n] and sum: l These convolutions computed using DFT (zero padding):

[ ] [ ] [ ] [ ] [ ] [ ] [ ]mhmnxmnhmxnhnxnyP

mmåå-

=

¥

=

-=-==1

00

*

[ ] [ ]å¥

=

=0r

r nxnx

[ ] [ ] [ ] [ ] [ ]å¥

=

==0

**r

r nhnxnhnxny

[ ] [ ] ( )îíì -+££

=otherwise0

11 LrnrLnxnxr

Overlap-Save(Not responsible for this topic)

l Breaks x[n] into segments of length L>P, each segment overlapping with previous one at P-1 points

l Perform L-point circular convolution of each segment with zero-padded filter h[n] (using DFT):

l Identify portion of each circular convolution that corresponds to a linear convolution, and save it. l First P – 1 points are unusable, while the remaining L – P + 1

points correspond to a linear convolution.

l Thus, we save L – P + 1 points from each circular convolution.

l Because first P – 1 points are unusable, the input segments must overlap at P – 1 points.

[ ] [ ]îíì -££-

=otherwise0

11p, LnPnyny r

r

[ ] ( )[ ]å¥

=

-++--=0

11r

r PPLrnyny

Blocklength Choice(ppt slides only)

l In overlap methods, several factors affect the choice of the block length Ll a shorter block length minimizes latency.l a shorter block length minimizes memory required for

performing the DFTs, multiplication, and inverse DFT.

l Given P (length of h[n]), there is an optimal block length L that minimizes complexity. l L too short, complexity increased by overhead of

adjacent block overlap l L too long, complexity increased because DFT

complexity increases with the block length l In practice, set block length so that DFT blocklength is

an integer power of 2 (required for FFTs)

FFT and IFFT Algorithms(ppt slides only)

l FFT computes the DFT of a sequence, IFFT computes the inverse DFT:l DFT as matrix operation: N2 complex multipliesl Complexity of FFT and IFFT same:

l FFT/IFFT breaks down a DFT with N2 complex multiplies into many smaller DFTs with N multipliesl Not responsible for details of how this is done (pp. 107-113 of reader, end of chapter 4)

l Reduces complexity of computing N-point DFT or IDFT from N2 complex multiplies to .5Nlog2N

N N2 NN22 log

NN

N22

2

log

16 256 32 8.0

128 16,384 448 36.6

1,024 1,048,576 5,120 204.8

8,192 67,108,864 53,248 1260.3

- In 1994 Strang described the FFT as "the most important numerical algorithm of our lifetime”- Included in Top 10 Algorithms of 20th Century by IEEE Journal of Computing in Science and Engineering

[ ] [ ] knN

N

n

WnxkX å-

=

=1

0

[ ] [ ] knN

N

k

WkXN

nx --

=å=1

0

1

[ ]{ } [ ]{ }( )**1 1 kXDFTN

kXDFT =-

Main Points

l Can compute linear convolution using circular convolution by zero padding

l For linear convolution of long sequences, computation is done in L-length blocks using overlap-add or overlap-savel Methods are very similar, differ in where overlap is introducedl Choice of L optimizes tradeoff in latency, memory, and complexity

l The FFT and IFFT drastically reduce the complexity of the DFT/IDFT computationl These algorithms are responsible for the widespread use of digital

signal processing in today’s electronic devices