Fourier Series and Characteristic functions

Fourier proposed in 1807

A periodic waveform f(t) could be broken down into an infinite series of simple sinusoids which, when added together, would construct the exact form of the original waveform.

Consider the periodic function

T = Period, the smallest value of T that satisfies the above equation.

To be described by the Fourier Series the waveform f(t)must satisfy the following mathematical properties:

1. f(t) is a single-value function except at possibly a finite number of points.

2. The integral for any t0.

3. f(t) has a finite number of discontinuities within the period T.

4. f(t) has a finite number of maxima and minima within the period T.

0

0

( )t T

tf t dt

Synthesis

T 2T 3Tt

f(t)

Tntb

Tntaatf

N

nn

N

nn

2sin2cos2

)(11

0

DC Part Even Part Odd Part

T is a period of all the above signals

Let ω0=2π/T.

)sin()cos(2

)( 01

01

0 tnbtnaatfN

nn

N

nn

A Fourier Series is an accurate representation of a periodic signal (when N ∞) and consists of the sum of sinusoids at the fundamental and harmonic frequencies.

The waveform f(t) depends on the amplitude and phase of every harmonic components, and we can generate any non-sinusoidal waveform by an appropriate combinationof sinusoidal functions.

Definition

Orthogonal Functions

Call a set of functions {ϕk} orthogonal on an interval a < t < b if it satisfies

Example

0m =1n = 2

-π π

Orthogonal set of Sinusoidal Functions

Define ω0=2π/T.0 ,0)cos(

2/

2/ 0 mdttmT

T0 ,0)sin(

2/

2/ 0 mdttmT

T

nmTnm

dttntmT

T 2/0

)cos()cos(2/

2/ 00

nmTnm

dttntmT

T 2/0

)sin()sin(2/

2/ 00

nmdttntmT

T and allfor ,0)cos()sin(

2/

2/ 00

We now prove this one

Proof

dttntmT

T 2/

2/ 00 )cos()cos(0

)]cos()[cos(21coscos

dttnmdttnmT

T

T

T

2/

2/ 0

2/

2/ 0 ])cos[(21])cos[(

21

2/

2/00

2/

2/00

])sin[()(

121])sin[(

)(1

21 T

T

T

Ttnm

nmtnm

nm

m ≠ n0

0

Proof

dttntmT

T 2/

2/ 00 )cos()cos(

0

dttmT

T 2/

2/ 02 )(cos

2/

2/0

0

2/

2/

]2sin4

121

T

T

T

T

tmm

t

m = n

2T

]2cos1[21cos2

dttmT

T 2/

2/ 0 ]2cos1[21

nmTnm

dttntmT

T 2/0

)cos()cos(2/

2/ 00

Orthogonal set of Sinusoidal Functions

Define ω0=2π/T.

,3sin,2sin,sin,3cos,2cos,cos

,1

000

000

tttttt

an orthonormal set.

Decomposition

dttfT

aTt

t

0

0

)(20

,2,1 cos)(20

0

0

ntdtntfT

aTt

tn

,2,1 sin)(20

0

0

ntdtntfT

bTt

tn

)sin()cos(2

)( 01

01

0 tnbtnaatfn

nn

n

Example (Square Wave)

1122

00

dta

,2,1 0sin1cos22

00

nntn

ntdtan

,6,4,20

,5,3,1/2)1cos(1 cos1sin

22

00

nnn

nn

ntn

ntdtbn

π 2π 3π 4π 5π-π-2π-3π-4π-5π-6π

f(t)1

1122

00

dta

,2,1 0sin1cos22

00

nntn

ntdtan

,6,4,20

,5,3,1/2)1cos(1 cos1sin

21

00

nnn

nn

ntn

ntdtbn

π 2π 3π 4π 5π-π-2π-3π-4π-5π-6π

f(t)1

Example (Square Wave)

ttttf 5sin

513sin

31sin2

21)(

1122

00

dta

,2,1 0sin1cos22

00

nntn

ntdtan

,6,4,20

,5,3,1/2)1cos(1 cos1sin

21

00

nnn

nn

ntn

ntdtbn

π 2π 3π 4π 5π-π-2π-3π-4π-5π-6π

f(t)1

Example (Square Wave)

-0.5

0

0.5

1

1.5

ttttf 5sin

513sin

31sin2

21)(

When series is truncated

Harmonics

Tntb

Tntaatf

nn

nn

2sin2cos2

)(11

0

DC Part Even Part Odd Part

T is a period of all the above signals

)sin()cos(2

)( 01

01

0 tnbtnaatfn

nn

n

Harmonics

tnbtnaatfn

nn

n 01

01

0 sincos2

)(

Tf

22 00Define , called the fundamental angular frequency.

0 nnDefine , called the n-th harmonic of the periodic function.

tbtaatf nn

nnn

n

sincos2

)(11

0

Harmonics

tbtaatf nn

nnn

n

sincos2

)(11

0

)sincos(2 1

0 tbtaannn

nn

12222

220 sincos2 n

n

nn

nn

nn

nnn t

ba

btba

abaa

1

220 sinsincoscos2 n

nnnnnn ttbaa

)cos(1

0 nn

nn tCC

Amplitudes and Phase Angles

)cos()(1

0 nn

nn tCCtf

20

0aC

22nnn baC

n

nn a

b1tan

harmonic amplitude phase angle

Complex Form of the Fourier Series

Complex Exponentials

Complex Form of the Fourier Series

tnbtnaatfn

nn

n 01

01

0 sincos2

)(

tjntjn

nn

tjntjn

nn eebjeeaa

0000

11

0

221

2

1

0 00 )(21)(

21

2 n

tjnnn

tjnnn ejbaejbaa

1

000

n

tjnn

tjnn ececc

)(21

)(212

00

nnn

nnn

jbac

jbac

ac

Complex Form of the Fourier Series

1

000)(

n

tjnn

tjnn ececctf

1

10

00

n

tjnn

n

tjnn ececc

n

tjnnec 0

)(21

)(212

00

nnn

nnn

jbac

jbac

ac

Complex Form of the Fourier Series

2/

2/0

0 )(12

T

Tdttf

Tac

)(21

nnn jbac

2/

2/ 0

2/

2/ 0 sin)(cos)(1 T

T

T

Ttdtntfjtdtntf

T

2/

2/ 00 )sin)(cos(1 T

Tdttnjtntf

T

2/

2/0)(1 T

T

tjn dtetfT

2/

2/0)(1)(

21 T

T

tjnnnn dtetf

Tjbac )(

21

)(212

00

nnn

nnn

jbac

jbac

ac

Complex Form of the Fourier Series

n

tjnnectf 0)(

dtetfT

cT

T

tjnn

2/

2/0)(1

)(21

)(212

00

nnn

nnn

jbac

jbac

ac

If f(t) is real,*nn cc

nn jnnn

jnn ecccecc

|| ,|| *

22

21|||| nnnn bacc

n

nn a

b1tan

,3,2,1 n

00 21 ac

Complex Frequency Spectra

nn jnnn

jnn ecccecc

|| ,|| *

22

21|||| nnnn bacc

n

nn a

b1tan ,3,2,1 n

00 21 ac

|cn|

n

amplitudespectrum

ϕn

n

phasespectrum

Example

2T

2T

TT2d

t

f(t)A

2d

dteTAc

d

d

tjnn

2/

2/0

2/

2/0

01

d

d

tjnejnT

A

2/

0

2/

0

0011 djndjn ejn

ejnT

A

)2/sin2(10

0

dnjjnT

A

2/sin10

021

dnnT

A

TdnT

dn

TAd

sin

TdnT

dn

TAdcn

sin

82

51

T ,

41 ,

201

0

T

dTd

Example

40π 80π 120π-40π 0-120π -80π

A/5

5ω0 10ω0 15ω0-5ω0-10ω0-15ω0

TdnT

dn

TAdcn

sin

Example

40π 80π 120π-40π 0-120π -80π

A/10

10ω0 20ω0 30ω0-10ω0-20ω0-30ω0

Example

dteTAc

d tjnn

00

dtjne

jnTA

00

01

00

110

jne

jnTA djn

)1(10

0

djnejnT

A

2/0

sindjne

TdnT

dn

TAd

TT d

t

f(t)A

0

)(1 2/2/2/

0

000 djndjndjn eeejnT

A

Discrete-time Fourier transform• Until this moment we were talking continuous periodic functions.

• However, probability mass function is a discrete aperiodic function. • One method to find the bridge is to start with a spectral

representation for periodic discrete function and let the period become infinitely long.

T 2T 3Tt

f(t)

0

continuous

discrete

Discrete-time Fourier transform• We will take a shorter but less direct approach. Recall Fourier series

A spectral representation for the continuous periodic function f(t)

Consider now, a spectral representation for the sequence cn, -∞ < n < ∞

• We are effectively interchanging the time and frequency domains.• We want to express an arbitrary function f(t) in terms of complex

exponents.

Discrete-time Fourier transform• To obtain this, we make the following substitutions in

This is the inverse transform

discrete continuous

Discrete-time Fourier transform

To obtain the forward transform, we make the same substitution in

Discrete-time Fourier transform• Putting everything together

• Sufficient conditions of existence

Properties of the Discrete-time Fourier Transform

Initial value

Homework: Prove it

Characteristic functions• Determining the moments E[Xn] of a RV can be difficult.• An alternative method that can be easier is based on

characteristic function ϕX(ω).

• The function g(X) = exp(jωX) is complex but by defining E[g(X)] = E[cos(ωX) + jsin(ωX)] = E[cos(ωX)] + jE[jsin(ωX)], we can apply formula for transform RV and obtain

for those integers not included in SX.

Characteristic functions• The definition is slightly different than the usual Fourier

transform, called the discrete time Fourier transform, which uses the function exp(-jωk) in its definition.

• As a Fourier transform it has all the usual properties.• The Fourier transform of a sequence is periodic with period of 2π.

Characteristic functions• CF for finding moments.• Note that we can differentiate the sum “term by term”

• Carrying out the differentiation

• So that

• In fact, repeated differentiation produces the formula for the nth moment as

Moments of geometric RV: example• Since the PMF for a geometric RV is given by pX[k] = (1 - p)k-1p

for k = 1,2,…, we have that

but since |(1-p) exp(jω)| < 1, we can use the result

For z a complex number with |z| < 1 to yield the CF

Note that CF is periodic with period 2π.

Moments of geometric RV: example• Let’s find the mean(first moment) using CF.

• Let’s find the second moment using CF and then variance.

Where D = exp(-jω) - (1-p). Since D|ω=0 = p, we have that

Expected value of binomial PMF

binomial theorem

a b

• By finding second moment we can find variance

Properties of characteristic functions• Property 1. CF always exists since• Proof

• Property 2. CF is periodic with period 2π.• Proof: For m an integer

since exp(j2πmk) = 1 for mk an integer.

Properties of characteristic functions• Property 3. The PMF may be recovered from the CF.• Given the CF, we may determine the PMF using

• Proof: Since the CF is the Fourier transform of a sequence (although its definition uses a +j instead of the usual -j), it has an inverse Fourier transform. Although any interval of length 2π may be used to perform the integration in the inverse Fourier transform, it is customary to use [-π, π].

Properties of characteristic functions• Property 4. Convergence of characteristic functions

guarantees convergence of PMFs (Continuity theorem of probability).• If we have a sequence of CFs φn

X(ω) converge to a given CF, say φX(ω), then the corresponding sequence of PMF, say pn

X[k], must converge to a given PMF say pX[k].

• The theorem allows us to approximate PMFs by simpler ones if we can show that the CFs are approximately equal.

Application example of property 4• Recall the approximation of the binomial PMF by Poisson PMF under

the conditions that p 0 and M ∞ with Mp = λ fixed.• To show this using the CF approach we let Xb denote a binomial RV.

And replacing p by λ/M we have

as M ∞.

Application example of property 4

• For Poisson RV XP we have that

• Since φXb(ω) φXp(ω) as M ∞, by property 4 we must have that pXb[k] pXp[k] for all k. Thus, under the stated conditions the binomial PMF becomes the Possion PMF as M ∞.

Practice problems

1. Prove that the transformed RV

has an expected value of 0 and a variance of 1.2. If Y = ax + b, what is the variance of Y in terms of the variance of X?3. Find the characteristic function for the PMF pX[k] = 1/5, for k = -2,-1,0,1,2.4. A central moment of a discrete RV is defined as E[(X – E[X])2] , for n positive integer. Derive a formula that relates the central moment to the usual (raw) moments. 5. Determine the variance of a binomial RV by using the properties of the CF. Assume knowledge of CF for binomial RV.

Homework

1. Apply Fourier series to the following functions on (0; 2π)

a.

b.

c.2. Find the second moment for a Poisson random variable by using the characteristic function exp [λ(exp(jω)-1)].3. A symmetric PMF satisfies the relationship pX[ -k] = pX[k] for k = …,-1,0,1,…. Prove that all the odd order moments, E[Xn] for n odd, are zero.