MULTIRATE DIGITAL SIGNAL PROCESSING

Signal processing can be enhanced by changing sampling rate:

• Up-sampling before D/A conversion in order to relax

requirements of analog antialiasing filter. Cf. audio CD,

where the sampling frequency 44.1 kHz is increased fourfold

to 176.4 kHz.

• Need to connect systems in digital (audio) signal processing

which operate at different sampling rates.

• Decomposition of a signal intoM components corresponding

to various frequency bands. If original signal is sampled at

the sampling frequency fs (with frequency band of width

45

fs/2), the components then contain a frequency band of

width 12fs/M , and can be represented using the sampling

rate fs/M . This allows for:

- parallel signal processing with processors operating at lower

sampling rates.

- data compression in subband coding, by representing

different frequency band components with different word

lengths (e.g., in speech processing).

- implementation of high-performance filtering operations

with a very narrow transition band. Decomposing

the signal into passband, stopband and transition band

components, each component can be processed at a lower

rate, and the transition band will be less narrow. This

results in significant simpler filter complexity.

46

Basic multirate signal processing operations

Sampling rate conversion

Decimation

-- ↓M y(m)x(n)

y(m) = x(mM) (every Mth element of x)

Matlab routines:

y=downsample(x,M)

[ y=decimate(x,M) applies antialiasing lowpass filtering followed by

downsampling ]

47

Spectrum of decimated signal:

For example, for M = 2,

- spectrum of {x(n)} consists of frequencies in [0, π],

- spectrum of {y(m)} consists of frequencies in [0, π/2], given

by

Y (ω) =1

2(X(ω/2) +X(π − ω/2)∗)

The second term is due to frequency folding, factor 1/2 is due to sampling

rate conversion

This implies that if x is bandlimited; X(ω) = 0, π/2 ≤ ω ≤ π,

then

Y (ω) =1

2X(ω/2)

48

Expansion

-- ↑ L y(m)x(n)

y(m) =

{x(m/L), for m = 0, L, 2L, . . .

0, otherwise

Expansion followed by interpolation is equivalent to sampling

rate increase by factor L.

Matlab routine:

y=upsample(x,L)

49

Spectrum of expanded signal:

For example, for M = 2,

- spectrum of {x(n)} consists of frequencies in [0, π]

- spectrum of {y(m)} consists of frequencies in [0, π], given by

Y (ω) =

{X(2ω), for 0 ≤ ω ≤ π/2

X(2(π − ω))∗ [= Y (π − ω)∗], for π/2 ≤ ω ≤ π

It is seen that Y (ω) is uniquely defined by its value in frequency

band [0, π/2]

Correct interpolation is achieved by a low-pass filter which

eliminates the frequencies in the band π/2 ≤ ω ≤ π.

50

Sampling rate conversion by non-integer factors

Sampling rate conversion with factor F = L/M can be

performed by applying:

- expansion by factor L

- low-pass filtering (corresponding to interpolation)

- decimation by factor M

Matlab routine:

y=resample(x,L,M)

51

Analysis and synthesis filter banks

Simple Analysis filter bank:

H1(z)x ↓ 2- - -

xD1

H2(z) -- ↓ 2 -xD2

- H1(z): low-pass filter to extract subband [0, π/2]

- H2(z): high-pass filter to extract subband [π/2, π]

- subbands components can be processed using

half the sampling rate

52

Analysis filter bank with three subbands:

H1(z)

H1(z)

x ↓ 2

↓ 2

- -

-

-

-

xD1

H2(z)

H2(z)

-

-

-

-

↓ 2

↓ 2

-

-

xD2 xD21

xD22

- xD1: subband component [0, π/2]

- xD2: subband component [π/2, π]

- xD21: subband component [π/2, 3π/4]

- xD22: subband component [3π/4, π]

53

Simple multirate signal processing system

Analysis filter bank Synthesis filter bank

H1(z)x ↓ 2- - - -

xD1s s s ↑ 2 -G1(z)xE1 --

y

H2(z) -- ↓ 2 - -xD2s s s ↑ 2 -G2(z)

xE2

6

k++

- subband components xD1, xD2 are processed independently at

lower sampling frequency

- processed components are upsampled to original sampling

frequency

- synthesis filters G1(z), G2(z) remove alias components

- filtered components are combined back to give the processed

signal y

54

Subband decomposition

Decomposition into low-frequency component and high-

frequency component

H1(z)x ↓ 2- - -

xD1

H2(z) -- ↓ 2 -xD2

- H1(z): low-pass filter to extract subband [0, π/2]

- H2(z): high-pass filter to extract subband [π/2, π]

55

Brickwall filters:

- H1 ideal low-pass filter

- H2 ideal high-pass filter

Frequency

response

-

6

ωππ/20

H1(ejω) H2(e

jω)

56

Real filters: frequency bands cannot be exactly separated

- H1 real low-pass filter

- H2 real high-pass filter

Frequency

response

-

6

ωππ/20���������������T

TTTTTTTTTTTTTT

H1(ejω) H2(e

jω)

57

Solution:

Construct H1,H2, G1, G2 so that perfect reconstruction is

achieved (up to a constant and time delay):

y(n) = Kx(n− P )

Analysis filter bank Synthesis filter bank

H1(z)x x1 ↓ 2- - -

xD1 ↑ 2 -v1 G1(z)

xE1 --y

H2(z) --x2 ↓ 2 -

xD2 ↑ 2 -v2 G2(z)

xE2

6

k++

58

Here:{xD1(n)} = {x1(0), x1(2), x1(4), . . .}

and

{v1(n)} = {xD1(0), 0, xD1(1), 0, xD1(2), 0, . . .}= {x1(0), 0, x1(2), 0, x1(4), 0, . . .}

But as

x1(±z) = x1(0)± x1(1)z−1 + x1(2)z

−2 ± x1(3)z−3 + · · ·

we have

v1(z) = v1(0) + v1(1)z−1 + v1(2)z

−2 + v1(3)z−3 + · · ·

= x1(0) + x1(2)z−2 + x1(3)z

−4 + · · ·

=1

2(x1(z) + x1(−z))

Similarly: v2(z) =12 (x2(z) + x2(−z))

59

We have the relations

v1(z) =1

2

(H1(z)x(z) +H1(−z)x(−z)

)v2(z) =

1

2

(H2(z)x(z) +H2(−z)x(−z)

)and

y(z) = xE1(z) + xE2(z)

= G1(z)v1(z) +G2(z)v2(z)

=⇒

y(z) =1

2

(G1(z)H1(z) +G2(z)H2(z)

)x(z)

+1

2

(G1(z)H1(−z) +G2(z)H2(−z)

)x(−z)

60

Here x(−z) is an alias component

(Fourier transform= x(−ejω) = x(ej(ω−π)))

Alias avoided if:

G1(z)H1(−z) +G2(z)H2(−z) = 0

=⇒G1(z) = 2H2(−z)G2(z) = −2H1(−z)

Scaling factor 2 introduced to compensate factor 1/2 in

expression for y(z)

61

Typically low- and high-pass filters H1,H2 are selected so that

H2(z) = H1(−z)

=⇒H2(e

jω) = H1(−ejω) = H1(ej(π−ω))∗

=⇒Frequency responses are mirror images about the quadrature

frequency π/2 = 2π/4 (quadrature mirror filters)

It follows that

G1(z) = 2H2(−z) = 2H1(z)

G2(z) = −2H1(−z) = −2H2(z)

62

Introducing G1, G2 and H2(z) = H1(−z) into

y(z) =1

2

(G1(z)H1(z) +G2(z)H2(z)

)x(z)

+1

2

(G1(z)H1(−z) +G2(z)H2(−z)

)x(−z)

gives

y(z) =(H1(z)

2 −H1(−z)2)x(z)

=⇒ Condition for perfect signal recovery

H1(z)2 −H1(−z)2 = Kz−P

63

Haar filters

Only quadrature mirror FIR filters which satisfy perfect signal

recovery:

H1(z) =1

2

(1 + z−1

), G1(z) = 1 + z−1

H2(z) =1

2

(1− z−1

), G2(z) = −1 + z−1

giving y(z) = z−1x(z), or y(n) = x(n− 1)

Alternatively, when noncausal filters can be used,

H1(z) =1

2(z + 1) , G1(z) = 1 + z−1

H2(z) =1

2(z − 1) , G2(z) = −1 + z−1

giving y(z) = x(z), i.e., y(n) = x(n)

64

Subband coding and multiresolution analysis

Pyramid algorithm:

H1(z) H1(z) H1(z)x ↓ 2 ↓ 2 ↓ 2- - - -- - -

xL xLL xLLL

H2(z) H2(z) H2(z)- - -- - -↓ 2 ↓ 2 ↓ 2- - -xH xLH xLLH

- x: frequency band 0, π], length N

- xH: frequency band [π/2, π], length N/2

- xLH: frequency band [π/4, π/2], length N/4

- xLLH: frequency band [π/8, π/4], length N/8

- xLLL frequency band [0, π/8], length N/8

65

Example (N = 8):

-

6

Time

Frequency

N00

π

π/2

π/4

xH

xLH

xLLH

xLLL

Time-frequency resolution

Low-frequency components xLLL, xLLH have high frequency

resolution and low time resolution

High-frequency component xH has low frequency resolution

and high time resolution

66

The pyramid algorithm generates a transform of input signal

{x(n)} = {x(0), x(1), . . . , x(N − 1)} to the signal transform

(when using three stages as above)

xHaar = [xLLL, xLLH, xLH, xH]

where

xLLL = {xLLL(0), . . . , xLLL(N/8− 1)}

xLLH = {xLLH(0), . . . , xLLL(N/8− 1)}

xLH = {xLH(0), . . . , xLH(N/4− 1)}

xH = {xH(0), . . . , xH(N/2− 1)}

67

Note: sequences x and xHaar have both length N

Inverse transform:

Use filters G1, G2 to reconstruct

- xLL from xLLH and xLLL

- xL from xLH and xLL

- x from xH and xL

68

Signal compression using multiresolution

Signal can be compressed by discarding small elements of xHaar:

xHaar(m) =

{xHaar(m), if |xHaar(m)| > d

0, if |xHaar(m)| ≤ d

where d > 0 is a specified threshold.

- Resolution in both time and frequency domain

- Fourier transform gives only frequency domain resolution

69

EXAMPLE

{x(n)} = {37, 35, 28, 28, 58, 18, 21, 15}

{xHaar(m)} = {30,−2,−4,−10,−1, 0,−20,−3}

1 2 3 4 5 6 7 8

20

30

40

50

60

(a)

1 2 3 4 5 6 7 8

20

30

40

50

60

(b)

1 2 3 4 5 6 7 8

20

30

40

50

60

(c)

(a): d = 2, (b): d = 3, (c): d = 4

70

Wavelets

Haar wavelets

Recall that the Fourier transform expresses a signal {x(n)} in

terms of frequency components ej2πkn/N .

In a similar way, the Haar transform xHaar can be expressed as

an expansion of the signal x in terms of a function set.

For the cases with three stages (resolution levels), we can showthat

x(n) =

N/23−1∑m=0

xLLL(m)ϕH(2−3n−m)−N/23−1∑m=0

xLLH(m)ψH(2−3n−m)

−N/22−1∑m=0

xLH(m)ψH(2−2n−m)−N/2−1∑m=0

xH(m)ψH(2−1n−m)

71

Here:

ψH(t) =

1, 0 ≤ t < 1/2

−1, 1/2 ≤ t < 1

0, otherwiseand

ϕH(t) =

{1, 0 ≤ t < 1

0, otherwise

- ψH(2−in) is defined for 0 ≤ n < 2i (dilated, or stretched,

version of ψH(n))

- ψH(2−in −m) is defined for m2i ≤ n < (m + 1)2i (dilated

and translated version of ψH(n))

72

Examples

(a): ψH(t); (b): ψH(2−2t); (c): ψH(2−2t− 1)

−2 0 2 4 6 8 10 12

−1

0

1

(a)

−2 0 2 4 6 8 10 12

−1

0

1

(b)

−2 0 2 4 6 8 10 12

−1

0

1

(c)

73

Expansion of x in terms of ϕH(2−in−m) and ψH(2−in−m)

is a wavelet expansion.

ϕH(t) and ψH(t): Haar wavelets

74

Generalization

Discrete wavelet expansion of {x(n)} of length N and Jresolution levels has the form

x(n) =

N/2J−1∑m=0

XDWT (0,m)ϕ(2−Jn−m)+

J∑i=1

N/2i−1∑m=0

XDWT (i,m)ψ(2−in−m)

- ψ(2−in) is defined for 0 ≤ n < 2i (dilated, or stretched,

version of ψ(n))

- ψ(2−in −m) is defined for m2i ≤ n < (m + 1)2i (dilated

and translated version of ψ(n))

ϕ(t): father wavelet

ψ(t): mother wavelet

ψ(2in−m): daughter wavelets

75

Daubechies wavelets

Main restriction of Haar wavelets: Haar filter does not provide

good separation of frequency bands.

For more powerful wavelet transforms, we should require that:

• H1(z),H2(z) are FIR filters, which give good frequency

separation

• H1(z),H2(z) give a filter bank which corresponds to a

wavelet expansion for some wavelet functions ϕ(t) and ψ(t).

• filter bank should have the perfect reconstruction property

76

In 1988 Ingrid Daubechies found a class of filters satisfying the

conditions:

Instead of using quadrature mirror filters, H1,H2 should be

related according to

H2(z) = −zMH1(−z−1)

where M= order of H1(z)

Elimination of alias components gives condition for synthesis

filters G1, G2 :

G1(z) = z−MH2(−z)G2(z) = −z−MH1(−z)

77

Daubechies filters are defined only for odd values of M .

Perfect reconstruction requirement gives condition:

1

2

(G1(z)H1(z) +G2(z)H2(z)

)= 1

Solution of order M = 2m− 1,m = 1, 2, . . . is given by

H1(z) =√2

(1 + z

2

)m

S(z)

where S(z) should satisfy

|S(ejω)|2 =m−1∑k=0

(m+ k − 1

k

)sin2k

(ω2

)

78

Case m = 1:

Daubechies wavelet is equivalent to the Haar wavelet (scaled

by factor√2)

Case m = 2:

S(z) =1−

√3

2+

√3 + 1

2z

and

H1(z) =√2

{1−

√3

8+

3−√3

8z +

3 +√3

8z2 +

1 +√3

8z3

}= −0.129409522551 + 0.224143868042z

+0.836516303738z2 + 0.482962913145z3

79

Efficient algorithms exist for calculating higher-order

Daubechies wavelets, and coefficients have been tabulated.

Daubechies wavelets form are an example of orthogonal

wavelets, because the associated filters H1,H2, G1, G2 have a

certain orthogonality property. Other orthogonal filter families

are the symlets and the Coiflets.

Another important type of wavelets are the biorthogonal

wavelets, for which the associated filters satisfy a

biorthogonality condition.

An important family of biorthogonal wavelets are the CDF

(Cohen-Daubechies-Feauveau) wavelets, which are used in the

JPEG 2000 standard.

80

Wavelet transform of 2-dimensional signal {x(n,m)}

- first perform 1-dimensional wavelet transform of each row to

give XDWT,row(n, k) (=DWT of nth row of x(n,m))

- then perform 1-dimensional wavelet transform of each column

to give the result XDWT(l, k) (=DWT of kth column of

XDWT,row(n, k))

Application: image compression and denoising

81

Signal processing with wavelets

Time-frequency resolution makes wavelets powerful for data

denoising and compression. A standard approach is to use

threshold-based methods:

Hard thresholds:

Given a wavelet transform XDWT(i,m), define:

XDWT(i,m) =

{XDWT(i,m), if |XDWT(i,m)| ≥ d(i)

0, if |XDWT(i,m)| < d(i)

Threshold d(i) ≥ 0 may be different for different resolution

levels i. Optimal value of d(i) which depend of signal variance

and length have been presented in the literature.

82

Soft thresholds:

Given a wavelet transform XDWT(i,m), define:

XDWT(i,m) = sgn(XDWT(i,m))|XDWT(i,m)− d(i)|

if |XDWT(i,m)| ≥ d(i),

and

XDWT(i,m) = 0, if |XDWT(i,m)| < d(i)

Threshold d(i) ≥ 0 may be different for different resolution

levels i. Optimal value of d(i) which depend of signal variance

and length have been presented in the literature.

83

Hard and soft thresholds can be applied to both data

compression and denoising.

JPEG 2000 image compression standard uses biorthogonal

CDF wavelets (cf. above) and quantization of the wavelet

components to achieve data compression

84