Fourier series, Discrete Time Fourier Transform and Characteristic functions

Fourier series, Discrete Time Fourier Transform and Characteristic functions

Fourier proposed in 1807

A periodic waveform f(t) could be broken down into an infinite series of simple sinusoids which, when added together, would construct the exact form of the original waveform.

Consider the periodic function

T = Period, the smallest value of T that satisfies the above equation.

Fourier series

To be described by the Fourier Series the waveform f(t) must satisfy the following mathematical properties:

1. f(t) is a single-value function except at possibly a finite number of points.

2. The integral for any t0.

3. f(t) has a finite number of discontinuities within the period T.4. f(t) has a finite number of maxima and minima within the

period T.

0

0

( )t T

tf t dt

Fourier series: conditions

T 2T 3T

t

f(t)

T

ntb

T

nta

atf

N

nn

N

nn

2sin

2cos

2)(

11

0

DC Part Even Part Odd Part

T is a period of all the above signals

Let ω0=2π/T.

)sin()cos(2

)( 01

01

0 tnbtnaa

tfN

nn

N

nn

Fourier series: synthesis

A Fourier Series is an accurate representation of a periodic signal (when N ∞) and consists of the sum of sinusoids at the fundamental and harmonic frequencies.

The waveform f(t) depends on the amplitude and phase of every harmonic components, and we can generate any non-sinusoidal waveform by an appropriate combination of sinusoidal functions.

Fourier series: definition

-0.5

0

0.5

1

1.5

Call a set of functions {ϕk} orthogonal on an interval a < t < b if it satisfies

Example

0

m =1

n = 2

-π π

Orthogonal functions

Define ω0=2π/T.

0 ,0)cos(2/

2/ 0 mdttmT

T0 ,0)sin(

2/

2/ 0 mdttmT

T

nmT

nmdttntm

T

T 2/

0)cos()cos(

2/

2/ 00

nmT

nmdttntm

T

T 2/

0)sin()sin(

2/

2/ 00

We now prove this one

Orthogonal set of sinusoidal functions

dttntmT

T 2/

2/ 00 )cos()cos(

0

)]cos()[cos(2

1coscos

dttnmdttnmT

T

T

T

2/

2/ 0

2/

2/ 0 ])cos[(2

1])cos[(

2

1

2/

2/00

2/

2/00

])sin[()(

1

2

1])sin[(

)(

1

2

1 T

T

T

Ttnm

nmtnm

nm

Case 1: m ≠ n

0

0

Proof or orthogonality

dttntmT

T 2/

2/ 00 )cos()cos(

0

dttmT

T 2/

2/ 02 )(cos

2/

2/

00

2/

2/

]2sin4

1

2

1T

T

T

T

tmm

t

2

T

]2cos1[2

1cos2

dttmT

T 2/

2/ 0 ]2cos1[2

1

nmT

nmdttntm

T

T 2/

0)cos()cos(

2/

2/ 00

Proof or orthogonalityCase 2: m = n

Define ω0=2π/T.

,3sin,2sin,sin

,3cos,2cos,cos

,1

000

000

ttt

ttt

an orthonormal set.an orthonormal set.

Orthogonal set of sinusoidal functions

Decomposition

dttfT

aTt

t

0

0

)(2

0

,2,1 cos)(2

0

0

0

ntdtntfT

aTt

tn

,2,1 sin)(2

0

0

0

ntdtntfT

bTt

tn

)sin()cos(2

)( 01

01

0 tnbtnaa

tfn

nn

n

Example (Square Wave)

112

200

dta

,2,1 0sin1

cos2

200

nntn

ntdtan

,6,4,20

,5,3,1/2)1cos(

1 cos

1sin

2

200

n

nnn

nnt

nntdtbn

π 2π 3π 4π 5π-π-2π-3π-4π-5π-6π

f(t)1

112

200

dta

,2,1 0sin1

cos2

200

nntn

ntdtan

,6,4,20

,5,3,1/2)1cos(

1 cos

1sin

2

100

n

nnn

nnt

nntdtbn

π 2π 3π 4π 5π-π-2π-3π-4π-5π-6π

f(t)1

Example (Square Wave)

ttttf 5sin

5

13sin

3

1sin

2

2

1)(

112

200

dta

,2,1 0sin1

cos2

200

nntn

ntdtan

,6,4,20

,5,3,1/2)1cos(

1 cos

1sin

2

100

n

nnn

nnt

nntdtbn

π 2π 3π 4π 5π-π-2π-3π-4π-5π-6π

f(t)1

Example (Square Wave)

-0.5

0

0.5

1

1.5

ttttf 5sin

5

13sin

3

1sin

2

2

1)(

When series is truncated

Harmonics

T

ntb

T

nta

atf

nn

nn

2sin

2cos

2)(

11

0

DC Part Even Part Odd Part

T is a period of all the above signals

)sin()cos(2

)( 01

01

0 tnbtnaa

tfn

nn

n

Harmonics

tnbtnaa

tfn

nn

n 01

01

0 sincos2

)(

Tf

22 00Define , called the fundamental angular frequency.

0 nnDefine , called the n-th harmonic of the periodic function.

tbtaa

tf nn

nnn

n

sincos2

)(11

0

Harmonicstbta

atf n

nnn

nn

sincos2

)(11

0

)sincos(2 1

0 tbtaa

nnnn

n

12222

220 sincos2 n

n

nn

nn

nn

nnn t

ba

bt

ba

aba

a

1

220 sinsincoscos2 n

nnnnnn ttbaa

)cos(1

0 nn

nn tCC

Amplitudes and Phase Angles

)cos()(1

0 nn

nn tCCtf

20

0

aC

22nnn baC

n

nn a

b1tan

harmonic amplitude phase angle

Complex Form of the Fourier Series

Complex Exponentials

Complex Form of the Fourier Series

tnbtnaa

tfn

nn

n 01

01

0 sincos2

)(

tjntjn

nn

tjntjn

nn eeb

jeea

a0000

11

0

22

1

2

1

0 00 )(2

1)(

2

1

2 n

tjnnn

tjnnn ejbaejba

a

1

000

n

tjnn

tjnn ececc

)(2

1

)(2

12

00

nnn

nnn

jbac

jbac

ac

Complex Form of the Fourier Series

1

000)(

n

tjnn

tjnn ececctf

1

10

00

n

tjnn

n

tjnn ececc

n

tjnnec 0

)(2

1

)(2

12

00

nnn

nnn

jbac

jbac

ac

Complex Form of the Fourier Series

2/

2/

00 )(

1

2

T

Tdttf

T

ac

)(2

1nnn jbac

2/

2/ 0

2/

2/ 0 sin)(cos)(1 T

T

T

Ttdtntfjtdtntf

T

2/

2/ 00 )sin)(cos(1 T

Tdttnjtntf

T

2/

2/

0)(1 T

T

tjn dtetfT

2/

2/

0)(1

)(2

1 T

T

tjnnnn dtetf

Tjbac )(

2

1

)(2

12

00

nnn

nnn

jbac

jbac

ac

Complex Form of the Fourier Series

n

tjnnectf 0)(

dtetfT

cT

T

tjnn

2/

2/

0)(1

)(2

1

)(2

12

00

nnn

nnn

jbac

jbac

ac

If f(t) is real,*nn cc

nn jnnn

jnn ecccecc

|| ,|| *

22

2

1|||| nnnn bacc

n

nn a

b1tan

,3,2,1 n

00 2

1ac

Complex Frequency Spectra

nn jnnn

jnn ecccecc

|| ,|| *

22

2

1|||| nnnn bacc

n

nn a

b1tan ,3,2,1 n

00 2

1ac

|cn|

n

amplitudespectrum

ϕn

n

phasespectrum

Example

2

T

2

T TT

2

d

t

f(t)A

2

d

TdnT

dn

T

Adcn

sin

82

5

1

T ,

4

1 ,

20

1

0

T

dTd

Example

40π 80π 120π-40π 0-120π -80π

A/5

5ω0 10ω0 15ω0-5ω0-10ω0-15ω0

TdnT

dn

T

Adcn

sin

Example

40π 80π 120π-40π 0-120π -80π

A/10

10ω0 20ω0 30ω0-10ω0-20ω0-30ω0

Example

dteT

Ac

d tjnn

0

0

d

tjnejnT

A

00

01

00

110

jne

jnT

A djn

)1(1

0

0

djnejnT

A

2/0

sindjne

TdnT

dn

T

Ad

TT d

t

f(t)A

0

)(1 2/2/2/

0

000 djndjndjn eeejnT

A

Discrete-time Fourier transform Until this moment we were talking continuous periodic functions.

However, probability mass function is a discrete aperiodic function. One method to find the bridge is to start with a spectral

representation for periodic discrete function and let the period become infinitely long.

T 2T 3T

t

f(t)

0

continuous

discrete

Discrete-time Fourier transform

We will take a shorter but less direct approach. Recall Fourier series

A spectral representation for the continuous periodic function f(t)

Consider now, a spectral representation for the sequence cn, -∞ < n < ∞

• We are effectively interchanging the time and frequency domains.• We want to express an arbitrary function f(t) in terms of complex

exponents.

Discrete-time Fourier transform

To obtain this, we make the following substitutions in

This is the inverse transform

discrete continuous

Discrete-time Fourier transform

To obtain the forward transform, we make the same substitution in

Discrete-time Fourier transform

Putting everything together

Sufficient conditions of existence

Properties of the Discrete-time Fourier Transform

Initial value

Homework: Prove it

Characteristic functions Determining the moments E[Xn] of a RV can be difficult.

An alternative method that can be easier is based on

characteristic function ϕX(ω).

There are particularly simple results for the characteristic

functions of distributions defined by the weighted sums of

random variables.

37

Characteristic functions

The function g(X) = exp(jωX) is complex but by defining

E[g(X)] = E[cos(ωX) + jsin(ωX)] = E[cos(ωX)] + jE[jsin(ωX)],

we can apply formula for transform RV and obtain

for those integers not included in SX.

Characteristic functions

The definition is slightly different than the usual Fourier

transform(the discrete time Fourier transform), which uses the

function exp(-jωk) in its definition.

As a Fourier transform it has all the usual properties.

The Fourier transform of a sequence is periodic with period of 2π.

Finding moments using CFTo find moments, let’s differentiate the sum “term by term”

Carrying out the differentiation

So that

Repeated differentiation produces the formula for the nth moment as

Moments of geometric RV: exampleSince the PMF for a geometric RV is given by pX[k] = (1 - p)k-1p for k = 1,2,…, we have that

but since |(1-p) exp(jω)| < 1, we can use the result

For z a complex number with |z| < 1 to yield the CF

Note that CF is periodic with period 2π.

Moments of geometric RV: example Let’s find the mean(first moment) using CF.

• Let’s find the second moment using CF and then variance.

Where D = exp(-jω) - (1-p). Since D|ω=0 = p, we have that

Expected value of binomial PMF

Binomial theorema b

By finding second moment we can find variance

Properties of characteristic functions Property 1. CF always exists since

Proof

Property 2. CF is periodic with period 2π. Proof: For m an integer

since exp(j2πmk) = 1 for mk an integer.

Properties of characteristic functions

Property 3. The PMF may be recovered from the CF. Given the CF, we may determine the PMF using

Proof: Since the CF is the Fourier transform of a sequence (although its definition uses a +j instead of the usual -j), it has an inverse Fourier transform. Although any interval of length 2π may be used to perform the integration in the inverse Fourier transform, it is customary to use [-π, π].

Fourier transform

Properties of characteristic functions Property 4. Convergence of characteristic functions guarantees

convergence of PMFs (Continuity theorem of probability).

If we have a sequence of CFs φnX(ω) converge to a given CF, say

φX(ω), then the corresponding sequence of PMF, say pnX[k], must

converge to a given PMF say pX[k].

The theorem allows us to approximate PMFs by simpler ones if we can show that the CFs are approximately equal.

Application example of property 4

Recall the approximation of the binomial PMF by Poisson PMF under

the conditions that p 0 and M ∞ with Mp = λ fixed. To show this using the CF approach we let Xb denote a binomial RV.

And replacing p by λ/M we have

as M ∞.

Application example of property 4

For Poisson RV XP we have that

Since φXb(ω) φXp(ω) as M ∞, by property 4 we must have that

pXb[k] pXp[k] for all k. Thus, under the stated conditions the

binomial PMF becomes the Possion PMF as M ∞.

Practice problems1. Prove that the transformed RV

has an expected value of 0 and a variance of 1.2. If Y = aX + b, what is the variance of Y in terms of the variance of X?3. Find the characteristic function for the PMF pX[k] = 1/5, for k = -2,-1,0,1,2.4. A central moment of a discrete RV is defined as E[(X – E[X])n] , for n positive integer. Derive a formula that relates the central moment to the usual (raw) moments. 5. Determine the variance of a binomial RV by using the properties of the CF. Assume knowledge of CF for binomial RV.

Homework1. Apply Fourier series to the following functions on (0; 2π)

a.

b.

c.

2. Find the second moment for a Poisson random variable by using the characteristic function exp [λ(exp(jω)-1)].3. A symmetric PMF satisfies the relationship pX[ -k] = pX[k] for k = …,-1,0,1,…. Prove that all the odd order moments, E[Xn] for n odd, are zero.