CMLab

What is a Digital Filter?

• In our technical society we often measure a continuously varying (analog) quantity.eg. Blood pressure, earthquake displacement, population of a city, waves falling on a beach, and the prob. of death

• All these measurement varying with time; we regard them as functions of time: u(t) in mathematical notation.

1

CMLab

• For technical reasons, instead of the signal u(t), we usually record equally spaced samples un of the function u(t).

• The sampling theorem gives the conditions on the signal that justify this sampling process.

• Moreover, when the samples are taken they are not recorded with infinite precision but are rounded off (sometimes chopped off) to comparatively few digits.

2

CMLab

• This procedure is often called quantizing the samples.

• It is these quantized samples that are available for the processing that we do. We do the processing in order to understand.

• What the function samples Un reveal about the underlying phenomena that gave rise to the observations, and Digital filters are the main processing tool.

3

CMLab

• It is necessary to emphasize that the samples are assumed to be equally spaced; any error or noise is in the measurement Un.Notation:{Un or U(n)}: the sequence of numbers of a set of equally spaced measurements of some quantity u(t), where n is an integer and t is a continuous variable.

4

CMLab

• The simplest kinds of filters are the Nonrecursive Filters; they are defined by the linear formula

∑∞

−∞=−=

kknkn UCy

The coeffs. Ck are the coefficiients of the filters, the Un-k are the input data, and the yn

are the output.

5

CMLab

Sum of products

∑−=

−=N

Nkknkn UCy

:convolution

Non-recursive digital filter

6

CMLab

When we use not only data values to compute the output values yn but use other values of the output, we have a formula of the form

Where both the ck and the dk are constants.In practice, the number of products that can be computed must be finite, that is

∑∑∞

∞−−

∞

∞−− += knkknkn yducy

∑∑=

−=

− +=M

kknk

N

kknkn yducy

107

CMLab

8

CMLab

• Filters that use only past computed values and the current value of the data are called causal, for if time is the independent variable, they do not react to future events but only past ones (causes).

• Note, however, that more and more often all the data of an experiment is recorded on a magnetic tape or other storage medium before any data processing is done. In such cases the restriction to causal filters is plainly foolish.

9

CMLab

The coeffs. ck and dk of the filter are assumed to be constants. Such filters are called time-invariant filters and are the filters most used in practice.Time-varying filters are occasionally useful – adaptive signal processing (adaptive filters) .

In practice, all computation must be done with finite-length (finite-precision) numbers.The process of quantization effects not only the input numbers, but it may affect all the internal (to the filter) arithmetic that is done.

10

CMLab

• Consequently, these are round off errors in the finite output numbers yn. It is often convenient to think in terms of infinite precision arithmetic and perfect input data un; but in the end we must deal with reality.- Finite wordlength effects!

11

CMLab

Noise amplification in a filter:Let un be the ”true”’ measurement with added noise εn whose expected value is zero. The condition of zero mean isE{εn}=0And it implies that there is no “bias” in the measurement, only local, random error. Let this noise εn have a variance σ2.

What is the corresponding noise in the out put of a nonrecursive filter (assuming that the arithmetic we do does not increase)?

12

CMLab

• To compute this, we make the additional assumption (which is often, but not always, true) that in making the measurements un+ εn the errors εn are uncorrelated.

• This assumption in mathematical notation is

• A nonrecursive filter is defined by the formula

• For the variance calculation, we begin with 13

⎩⎨⎧

≠=

=⋅nmnm

E mn ,0,

}{2σ

εε

∑ ∑∑−= −=

−−−−=

−− =+=+=N

Nk

N

Nkknkknknkn

N

Nkknknkn ucEucyEucy }){(}{ ,)( εε

})]()({[ 2n

N

Nkknknk yEucE −+∑

−=−− ε

CMLab

But this is (using different summation indices to keep things clear)

Since E{εn}=0 and, for ,Multiplying out and applying the operator E to the nleaves only the terms

Thus,the sum of the sequence of the coefficients of a filter

measures the noise amplification of the filtering process.

]}][{[}{ 2 ∑∑∑−=

−−=

−−=

− =N

Nmmnm

N

Nkknk

N

Nkknk ccEcE εεε

∑∑∑=

==N

Nkkkkk ccEc

-

222222 }{ σσε

0}{, =≠ mnEnm εε

14

CMLab

Geometric progressions

A geometric progression is a sequence of n values

The special case of symmetrically arranged terms beginning at a=z-m ,and going on for 2m+1 terms gives

differential (Δ) w.r.t. z and set a=1, we get

)(1

)1()1(][

,,,,

12

12

Δ−−

=++++= −

−

KL

L

zzazzzanS

azazazan

n

n

)'(11

)1(1],[21

21

)21(

21

11211 Δ

−

−=

−−

=−−

=++++++=−−

+−++−+−−+−− KLL

zz

zzzzz

zzzzzzzmmS

mmmmmmmmmm

2

1)2(2

)1()1(1)1(321

zznnzznzz

nnn

−−+−

=−++++−

−L

15

CMLab

• In many data-processing problems there is no natural original, and therefore an arbitrary point is selected as the origin (typically, for a time signal, n=0 is the time we set t=0, which is arbitrary).

• From the addition formula of trigonometrysin(x+y)=sinxcosy+cosxsinycos(x+y)=cosxcosy-sinxsiny

• It is an easy exercise in trigonometry to see that, when x=x’+h

• Asinx+Bcosx becomes A’sinx’+B’cosx’• Where we have A’=Acosh-Bsinh; B’=Asinh+Bcosh

16

CMLab

Invariance under translation

Squaring each of these experssions and adding them, we obtain A’2+B’2=A2+B2

⇒Under the operation of translation the pair of functions cosx and sinx constitutes the eigenfunctions, for when placed into the operation of translation by the amount h, they again emerge.

Euler identities: cosx+jsinx = ejx; cosx-jsinx=e-jx

⇒

⇒ The complex exponentials are “eigenfunctions” of translation

)(21sin );(

21cos jxjxjxjx ee

jxeex −− −=+=

17

CMLab

• In the complex exponential notation the eigenfunction property is much more simply expressed than in the trigonometric notation,

• Where is the eigenvalue

• And is independent of the variable t.• Thus the are “eigenfunctions ” of translation.

)()()( )(

tueeehtu tjhjhtj

ωλ

ωωω

=⋅==+ +

)(ωλhje ωωλ =)(

hje ω 18

CMLab

• In general, – (Where c can be a complex number) and the invariant is

• Linear Systems:The complex exponential functions and are eigenfunctions for LTI systems.

tjcetu ω=)(

2

*

))((

))((

ccc

ecce

cecetjtj

tjtj

=⋅=

= − ωω

ωω

tje ω tje ω−

{ } tjtj eeL ωω ωλ )(= 19

CMLab • For nonrecursive filters of the form

The substitutionor

Produces

Where: transfer function

Fourier Transform: DTFT

∑−=

−=N

Nkknkn ucy

tjetu ω=)( njenu ω=)(

njN

Nk

kjk

njn eecey ωωω ωλ )(== ∑

−=

−

∑−=

−=N

Nk

kjkec ωωλ )(

20

CMLab • For recursive filter we need to assume that both the input and output are of the form

and• Where the input coeff. ,and output coeff. may

be complex numbers.

: Transfer function

njIn eAu ω= nj

on eAy ω=

IA oA

∑∑ −− +=M

knk

N

knkn yducy10

∑

∑

∑∑

−

−

−−

−=

+=⇒

Mkj

k

Nkj

k

I

o

Mknj

ko

Nknj

kInj

o

ed

ec

AA

edAecAeA

1

0

1

)(

0

)(

1 ω

ω

ωωω

21

CMLab

This is a transfer function because, when you multiply the input by this ratio you get the output:you ‘transfer’ from the input to the output!

22

CMLab

• It is worth nothing that the exponential function is also the eigenfunction that is appropriate for the calculus operations– Of differentiation

– Of integration

• The exponential is also the eigenfunction for differencing , since

tjtj ejedtd ωω ω)(=

tjtj ej

dte ωω

ω)1(=∫

( ) tjjtjtjtj eeeee ωωωωω ]1[)1( −=−=Δ +23

CMLab

• The expression occurs frequently and it is often easier to make a notational change and write

• Hence for nonrecursive filter

: (mod ( ))For recursive filter

Z-transformation

ωje

∑−=

−=N

Nk

kk zc)(ωλ

ωjez−

∑

∑−

−

−= M

kk

Nk

k

I

o

zd

zc

AA

1

0

1

24

ze j =ω

CMLab

25

Some basic Applications of Digital filters :

• Least-square fitting of polynomialsGiven M data points (tm, um), m=1,2,…,M wish to fit (approximate) these data, in some sense, by a polynomial u=u(t) of degree N where M≧N+1

Principle of least square : the sum of the squares of the residuals is the least

u(t)

t

• All the errors are in the vertical direction

• The differences between the polynomial and the original data are called the residuals

CMLab

26

Ex: consider a set of 5 equally spaced data points.tm=m for m=-2, -1, 0, 1, 2um unspecifiedfit the above data by a straight line u=A+Btin least-square sense.

In the case of smoothing, we use midpoint of the line as the smoothed value   in place of the original point u0. In this case, only A is needed. From (1), we have  

( ) ( )[ ]

( )

( )

[ ]1 1, 1, 1, 1,51 :window running

averagemoving : 51 general,In

equation normal 0,

0,

,min

2

2

2

2

2

∑

∑

−=−

−=

=

⎪⎪⎭

⎪⎪⎬

⎫

=∂

∂

=∂

∂

+−=

mmnn

mm

uu

BBAF

ABAF

BmAuBAF

CMLab

27

From the frequency point of view :S’pose the input function is one of the eigenfunctions in the complex form eiωt of frequency ω.Since the system is linear, the same function will emerge at the output except that it is multiplied by its eigenvalue H(ω).

CMLab

28

( ) [ ]

[ ]

( )( )2sin5

25sin

2cos2cos2151

151 22

ω

ω

ωω

ω ωωωω

=

++=

++++= −− iiii eeeeH

In general :

: transfer function

( ) ( )[ ]

( )( )( )2

212

1

sinsin

121

...12

1

ω

ωωω

ω

ω

+

−−−

+=

++++

=

N

iNNiiN

N

eeeN

H

CMLab

29

the more terms used , the more rapid are the wiggles of H(w), and the more the envelope of the wiggles is squeezed toward the frequency axis.

These smoothing formulas are the same as the running average in statistics!Note that these transfer functions are periodic and symmetric about ω=0 and ω= π (f=1/2). The periodicity about f=1/2 reflect the Aliasing effect, while the periodicity about f=0 arises from the symmetry of the filter coefficients.

CMLab

30

• Least-squares Quadratics and QuarticsInstead of a straight line, fit by a quardratic (or a cubic) u(t)=A+Bt+Ct2

( ) ( )[ ][ ][ ][ ][ ]

]31217123[351 general,In

36,9,44,69,84,89,84,69,44,9,361121,14,39,54,59,54,39,14,219

2,3,6,7,6,3,273,12,17,12,35

,,min

2112

4291

2311

211

351

22

++−− −+++−=

−−=−−=

−−=−−=

++−=⇒ ∑

nnnnnn

m

uuuuuu

mmmm

CmBmAuCBAF

By differentiating w.r.t the variables A, B and C, we obtain the normal equations:

Where {·} means “sum” of all values of the quantities enclosed in the braces

By selecting the particular set of equally spaced data points –m, -(m-1), … , 0, … (m-1), m, the sum over all odd power are all zero.

CMLab

31

• The effect of the higher-degree polynomial is a higher degree of tangency at ω=0, as compared with the straight line case. That is, the curves are more flat around DC.

• The use of more terms in the smoothing formula makes the curve come down rapidly and to be slightly lower in the subsequent wiggles.

• Theorem: The more powers of t that we fit in the time domain, the higher the tangent at ω=0 in the frequency domain.

HW: please find corresponding H(ω) and ck for N=5, 7, 9, 11 if f(t)=A+Bt+Ct2+Dt3+Et4

CMLab

32

• Modified least squares :Smoothing Window for 2N+1 points

modified Smoothing Windows for 2N+1 points: reduce the two end values to one-half their assigned values

( ) ( )( )

( ) ( )22

2

222

2

212

cossinsin

21

)sin(2)]sin(cos)sin(cos)cos([sin

cossin

sin21

ωω

ω

ωωω

ω

ω

ωωω

ωωω

⎭⎬⎫

⎩⎨⎧

=

−+=

⎭⎬⎫

⎩⎨⎧

−=+

NN

NNNN

NN

HN

CMLab

33

• The curve will come down more rapidly• The main lobe is slightly wider.★ the end constants can be chosen as a parameter

• The brackets are the same as before except with 2N in place of 2N+1. This means that the zeros of the transfer function are spaced slightly farther apart, at π/N instead of at π/(N+1). This means that the main lobe is slightly wider.

• The change of the division from 2N+1 to 2N also means that the term in the brackets would end at a slightly higher value.

To ensure that the end of the interval at f=0.5, the transfer function shall be zero.

• The extra   factor means that the curve will come down some what more rapidly.

• Smoothing   removal of high frequencies in the signal

CMLab

34

• Differences and DerivativesThe difference operator: can be shown to “amplify “ small errors!

the operator annihilates a polynomial Pn(x) of degree n in X; that is

[ ] ( )[ ][ ]nK

nK

nnn

uu

uuu1

1−

+

ΔΔ=Δ

−=Δ

1+Δn

( ) ( ) 01 =Δ + xPnn

CMLab

35

from frequency point of view :( )

[ ]( )[ ]

[ ] [ ] ( )[ ] tiKiKKtiK

tii

tii

tititi

eeie

eie

eeeee

ωωωω

ωωω

ωω

ωωω

22

22

1

sin2

sin2

1

=Δ

=

−=

−=Δ +

CMLab

36

Since , so the amplification at frequency w is contained in the factor

; △decreases the amplitude of any frequency; there is an amplification

12 ==ωiKei K

( )[ ]K2sin2 ω

πωω

π

π

<≤<≤

3

30

CMLab

37

high-pass behavior of the difference operator Δk

CMLab

38

Differences are also used to approximate derivatives.• Central difference formula

from the formula (with h=1)( ) ( ) titi

nnn

eituetuhuuu

ωω ω=′=

−=′ −+

so , set

...(1) 2

11

ωωω sin2iee ii =− −

CMLab

39

The ratio of the calculated to the true answer (which is iω) is :

ω=0, R=1ω≠0, |R|<1 ⇒ the formula underestimates the value of the

derivatives for all other freqs. in the Nyquest interval

For the 2nd derivate :

ωω

ωω sin

2sin2

truecalculated

==Δi

iR

( ) ( ) ( ) ( )( ) 2

2

2sin

121

⎥⎦

⎤⎢⎣

⎡=

−+−+=′′

ω

ω

R

tutututu

Which is the square of the previous case but contracted by a factor of 2 in the independent variable ω

CMLab

40

Linear processing of smoothing:

Spencer’s smoothing formula:15-point: 21-point:

[ ][ ],...57,60,57,47,33,18,6,2,5,5,3,1

3,6,5,3,21,46,67,74,67,46,21,3,5,6,3

3501

3201

−−−−−

−−−−−−

not informative

Smoothing: keep the value at zero frequency correct but, in general, decrease the amount of any higher frequency that may be existed in the function being smoothed

CMLab

41

plot the logs of the numbers |H(ω)| 20 log|ratio| = decibel units (dB)20 dB = factor of 10

As shown in the above figure, the curves are not too informative, since they are as small at the high frequencies that we cannot see how good or bad they are.It is therefore better to plot the logs of the numbers |H(ω)|

The smoothing formulas generally remove some frequencies and let others pass.

40db 10060db 1000

CMLab

42

• Missing Data and Interpolation :The reasons or situations for the occurrence of

“missing data” in a long record of data :• the measurements may never have been made;

they may have been misrecorded and thus later removed

• the formula used to compute successive values of the function may have involved an indeterminate form, such as at x=0, and the computer refused to divide by zero.

xxsin

CMLab

43

Usually, an interpolation formula based on the assumption that the data locally is a polynomial of some odd degree.

This is equivalent to the assumption that the next higher-order difference is zero.

11 -11 -2 1

1 -3 3 -11 -4 6 -4 1

Difference table △un=un+1-unn f(n) △f(n) △2f(n) △3f(n)-3 0-2 0 0-1 0 10 1 -21 0 12 0 03 0 04 0

0011000

1-3-3-10

CMLab

44

For instance, k=4

note: In general case the sum of the sequences of the binomial coeffs. of order N is . Hence for the 2k th difference formula, the noise amplification is

[ ]211261

211224

440464

++−−

++−−−

−++−=⇒=+−+−=Δ

nnnnn

nnnnnn

uuuuuuuuuuu

NNC 2

( )( ) 1222

2242 −≅−

Δ kC

CCNk

k

kk

kk

Aπ

31.1,3,2

5.0,1

1817

==

==

==

A

A

A

NkNkNk

The noise amplification factor of this formula is  

CMLab

45

If set then ( ) [ ]ωωω

ω

2coscos431 −=

=H

eu nin

CMLab

46

H(0)=1 However, for high freqs, the value is not too good, particularly for very high freqs.

• Negative values on the graph of the transfer function imply a change in sign.The above figure points out the danager of interpolating a missing value when the data is noisy, which means that the data has numerous high frequencies.

Which implies that we obtain the right answer for ω=0

CMLab

47

Interpolation midpoint values : linear interpolation gives

if 4 adjacent points are used and make the formula exact for all cubics gives the formula: (proof:HW)

[ ]( ) ( )ωω ω

23

281

4

161

coscos9

9923

21

21

23

−=

−++−= −−++

H

uuuuu nnnnn

Please plot H2(ω) and H4(ω) and see what are their behaviors in the high and low frequency ranges respectively.

( ) ( ) ( )221 cos

21

21

ωω =→+= −+ Huuu nnn

CMLab

48

• A Class of Nonrecursive Smoothing FiltersDesign of filters : Symmetric Filters

H(0)=1 exact at dc(lowest freq.)H(π)=0 no highest freq. get through

( ) cbaHeu

aubucubuauyni

n

nnnnnn

++=⇒=

++++= ++−−

ωωω

ω

cos22cos2let

2112

L.P.

CMLab

49

Since H(π)=0 the factor [cos ω+1] had to occur Now one can select a filter that approximately meets the specific requirement.

To convert a low-pass filter to a high-pass filter, we need only to compute un-yn as the new filter!!

( ) [ ] ( )[ ]1coscos142022

122

81

21

41

−++=

−==

⇒⎭⎬⎫

=+−=++

⇒

aaHac

bcbacba

ωωω

CMLab

50

for Ex.1, if we require ( ) 43

81

21

21 ,221 =−=⇒−+−== caaaH π

Ex.2

if we set ( )

[ ]( ) 2

121

2

1141

21

21

21

22

cos2

,022

+=++=⇒

==⇒−+−==

+−

ωω

π

Huuuy

caaaH

nnnn

[ ]

( ) [ ][ ]2cos1cos

262

21

43

2cos

42cos

211281

−+−=++−=

−+++−=⇒ ++−−

ωωω

ωω

H

uuuuuy nnnnnn

CMLab

51

Ex. 3. S’pose that we try to do as well as possible in the neighborhood of zero freq.( )( )

( )161

21

02

32

0

3

3

80

0

10

−=⇒−−==

=

=

=

=

aadHd

ddHH

ω

ω

ωω

ωω

CMLab

52

Ex: we pick our filter form as

[ ]

( ) [ ][ ]3cos1cos

4104

41

85

2cos

82cos

2112161

−+−=++=

−+++−=⇒ ++−−

ωω

ωω

wH

uuuuuy nnnnnn

( )( ) ( )

[ ] ]1,1,1[0cos)(11cos)(1

2cos)( and

0,1 and2cos2;

21

1121

21

32

31

21

361

21

21

21

11

31

61

=++=⇒⎩⎨⎧

=+=+=+=+

⇒+=⇒==⇒

==

+=++=

+−

==

−+

nnnn

ff

nnnn

uuuy

fHba

fHfHbfaHaubuauy

π

π

πω

πω

CMLab

53

• Integration : Recursive FiltersTrapezoid Rule (using y0=0)

[ ]( )

( ) ( )

( ) [ ][ ] 2

221

0

121

1

sin2cos

11 where

form theof is

0 , Set

ω

ω

ω

ω

ω

ω

ω

ω

ieeA

eAty

yetu

uuyy

i

i

ti

tinnnn

=−+

=

==

++= ++

CMLab

54

The true answer for integration of is tie ω [ ] tii e ωω1

( ) 0 , 1 , 0At

21 =====

RfR

πωω

, which gives the quadratic shape of the ratio near ω=0

which is the natural boundary, for it is where Aliasing begins

( )( )[ ]

...1

sincos

truecalculated

72012

2

221

42 ++−=

⎥⎦

⎤⎢⎣

⎡==Δ

ωω

ω

ωω

ωω

ωi

AR

CMLab

55

Simpson’s Rule :

ω=0 , R=1and has a tangency through the 3rd derivative, and rises initially like a quartic until at ω=πThe denominator becomes zero => R ∞

( )1131

11 4 −+−+ +++= nnnnn uuuyy

( ) ( )( ) ( )

[ ][ ]

( )[ ] L++=+

==

−++

= −

−

1801cos2

34

4

sin31

ωωω

ω

ωω

ω

ω

ωω

ωω

i

ii

ii

HR

eeeeH

CMLab

56

• Midpoint integration formula (using y0=0)

Simpon’s formula amplifies the upper part of the Nyquest interval (the high freqs), whereas trapezoid rule damps them out.

This expression begins at 1 and slowly rises as   increases,   .

( )2

2

1

sin

21

ω

ω

=

+= ++

R

uyy nnn

CMLab

57

• Leo Tick Formula ( )1111 3584.02832.13584.0 −+−+ +++= nnnnn uuuhyy

This formula was designed to have a transfer function that is as close to unity as we make it throughout the lower half of the Nyquest interval while still involving only three consecutive terms

1.57= 

Tick’s integration rule was designed to be as accurate as it could be for lower half of the Nyquestinterval,  

CMLab

• Simpson’s formula amplifies the upper part of Nyquist interval (the higher frequencies) where as the trapezoid rule damps them out.

• In the presence of noise. Simpson’s formula is more dangerous to use than are the trapezoid or midpoint formulas. But when there is relatively little high freq. In the function being integrated, then the flatness of Simpson’s formula for low freqs. show why it is superior.

Good Reference for Window Selection:“On the use of windows for Harmonic Analysis with the DFT”, Proc. IEEE, vol. 66, pp.58-83, Jan. 1978.