Part 3. Part 3. Spectrum Spectrum …classweb.ece.umd.edu › enee630.F2012 › slides ›...

ENEE630 PartENEE630 Part--33

Part 3. Part 3. Spectrum Spectrum EstimationEstimation3.1 3.1 Classic Methods for Spectrum EstimationClassic Methods for Spectrum Estimation

Electrical & Computer EngineeringElectrical & Computer Engineering

University of Maryland, College Park

Acknowledgment: ENEE630 slides were based on class notes developed by Profs. K.J. Ray Liu and Min Wu. The slides were made by Prof. Min Wu,

ith d t f M W i H Ch C t t i @ d d

UMD ENEE630 Advanced Signal Processing (ver.1211)

with updates from Mr. Wei-Hong Chuang. Contact: minwu@umd.edu

LogisticsLogistics

Last Lecture: lattice predictorcorrelation properties of error processes©

– correlation properties of error processes– joint process estimator in lattice– inverse lattice filter structureat

inverse lattice filter structure

Today: S t ti ti b k d d l i l th d63

– Spectrum estimation: background and classical methods

Homework setUM

UMD ENEE630 Advanced Signal Processing (ver.1211) Nonparametric spectral estimation [2]

Summary of Related Readings on PartSummary of Related Readings on Part--IIII

2.1 Stochastic Processes and modelingHaykin (4th Ed) 1.1-1.8, 1.12-1.14 Hayes 3.3 – 3.7 (3.5); 4.7

2.2 Wiener filteringHaykin (4th Ed) Chapter 2Hayes 7.1, 7.2, 7.3.1

2 3 2 4 Li di ti d L i D bi i2.3-2.4 Linear prediction and Levinson-Durbin recursionHaykin (4th Ed) 3.1 – 3.3Hayes 7.2.2; 5.1; 5.2.1 – 5.2.2, 5.2.4– 5.2.5

2.5 Lattice predictorHaykin (4th Ed) 3.8 – 3.10

UMD ENEE630 Advanced Signal Processing (ver.1211) Parametric spectral estimation [3]

Hayes 6.2; 7.2.4; 6.4.1

Summary of Related Readings on PartSummary of Related Readings on Part--IIIIII

Overview Haykins 1.16, 1.10

3 1 Non-parametric method3.1 Non-parametric methodHayes 8.1; 8.2 (8.2.3, 8.2.5); 8.3

3 2 P t i th d3.2 Parametric methodHayes 8.5, 4.7; 8.4

3.3 Frequency estimationHayes 8.6

Review – On DSP and Linear algebra: Hayes 2.2, 2.3

– On probability and parameter estimation: Hayes 3.1 – 3.2

Spectrum Estimation: BackgroundSpectrum Estimation: Background

Spectral estimation: determine the power distribution in frequency of a random process©

frequency of a random process– E.g “Does most of the power of a signal reside at low or high

frequencies?” “Are there resonances in the spectrum?”

Applications:– Needs of spectral knowledge in spectrum domain non-causal

Wi fil i i l d i d ki b f i30 S

Wiener filtering, signal detection and tracking, beamforming, etc.– Wide use in diverse fields: radar, sonar, speech, biomedicine,

geophysics, economics, …

g p y , ,

Estimating p.s.d. of a w.s.s. process is equivalent to estimate autocorrelation at all lags

Spectral Estimation: ChallengesSpectral Estimation: Challenges

When a limited amount of observation data are available– Can’t get r(k) for all k and/or may have inaccurate estimate of r(k)©

Can t get r(k) for all k and/or may have inaccurate estimate of r(k)– Scenario-1: transient measurement (earthquake, volcano, …) – Scenario-2: constrained to short period to ensure (approx.)

stationarity in speech processing

Mkknunukr 10][][1)(ˆ

Ob d d t h b t d b iCP

MkknunukN

,1,0 ],[][)(

Observed data may have been corrupted by noise

Spectral Estimation: Major ApproachesSpectral Estimation: Major Approaches

Nonparametric methods– No assumptions on the underlying model for the data

p y g– Periodogram and its variations (averaging, smoothing, …)– Minimum variance method

Parametric methods– ARMA, AR, MA models

M i t th d30 S

– Maximum entropy method

Frequency estimation (noise subspace methods)For harmonic processes that consist of a sum of sinusoids orC

– For harmonic processes that consist of a sum of sinusoids or complex-exponentials in noise

High-order statistics

Spectral Estimation: Major ApproachesSpectral Estimation: Major Approaches

Nonparametric methods– No assumptions on the underlying model for the data

p y g– Periodogram and its variations (averaging, smoothing, …)– Minimum variance method

Parametric methods– ARMA, AR, MA models

M i t th d30 S

– Maximum entropy method

Frequency estimation (noise subspace methods)For harmonic processes that consist of a sum of sinusoids orC

– For harmonic processes that consist of a sum of sinusoids or complex-exponentials in noise

High-order statistics

Example of Speech SpectrogramExample of Speech Spectrogramu

Figure 3 of SPM May’98 Speech Survey

“Sprouted grains and seeds are used in salads and dishes such as chop suey”W

1.0 2.0 3.0 4.0 0.0

Time (sec)

“S t d”

(“Sprouted”

0.1 0.3 0.5

fricative stopconsonantglide vowel stopconsonantvowel

Section 3.1 Classical Nonparametric MethodsSection 3.1 Classical Nonparametric Methods©

3) Recall: given a w.s.s. process {x[n]} with

)(]][][[]][[

krknxnxEmnxE x

The power spectral density (p.s.d.) is defined as

)(]][][[ krknxnxE

fkjekrfP 2)()(

As we can take DTFT on a specific realization of a random process,What is the relation between the DTFT of a specific signal and the

C ):2or ( f

What is the relation between the DTFT of a specific signal and the p.s.d. of the random process?

Section 3.1 Classical Nonparametric MethodsSection 3.1 Classical Nonparametric Methods©

3) Recall: given a w.s.s. process {x[n]} with

)(]][][[]][[

krknxnxEmnxE x

The power spectral density (p.s.d.) is defined as

)(]][][[ krknxnxE

fkjekrfP 2)()(

As we can take DTFT on a specific realization of a random process,What is the relation between the DTFT of a specific signal and the

C ):2or ( f

What is the relation between the DTFT of a specific signal and the p.s.d. of the random process?

Ensemble Average of Squared Fourier MagnitudeEnsemble Average of Squared Fourier Magnitude

p.s.d. can be related to the ensemble average of the squared Fourier magnitude |X()|2

q g | ( )|

121)(Consider

fnjM enx

a 12 Mn

mnfjemxnx )(2][][1

i.e., take DTFT on (2M+1) samples and CP

emxnxM

][][12

, ( ) pexamine normalized squared magnitude

Note: for each frequency f, PM(f) is a random variable

Ensemble Average of Squared Fourier MagnitudeEnsemble Average of Squared Fourier Magnitude

p.s.d. can be related to the ensemble average of the squared Fourier magnitude |X()|2

q g | ( )|

121)(Consider

fnjM enx

a 12 Mn

mnfjemxnx )(2][][1

emxnxM

][][12

i.e., take DTFT on (2M+1) samples and

C , ( ) pexamine normalized squared magnitude

Note: for each frequency f, PM(f) is a random variable

Ensemble Average of PEnsemble Average of PMM(f)(f)©

mnfjM emnr

MfPE )(2)(

121)]([

u © Mn MmM 12

fkjekrkMM

22)()12(

Mfkjekr

k22)(1

a MkM 212

fkj ekrkM

2 )(12

Now, what if M goes to infinity?

Ensemble Average of PEnsemble Average of PMM(f)(f)©

mnfjM emnr

MfPE )(2)(

121)]([

u © Mn MmM 12

fkjekrkMM

22)()12(

Mfkjekr

k22)(1

MkM 212

fkj ekrkM

2 )(12

Now, what if M goes to infinity?

P.S.D. and Ensemble Fourier MagnitudeP.S.D. and Ensemble Fourier Magnitude©

3) If the autocorrelation function decays fast enough s.t.

)for rapidly 0 (i.e., )(

kr(k)krkk

fPekrfPE )()()]([lim 2

p.s.d.

)( ][12

1lim)(2

P.S.D. and Ensemble Fourier MagnitudeP.S.D. and Ensemble Fourier Magnitude©

3) If the autocorrelation function decays fast enough s.t.

)for rapidly 0 (i.e., )(

kr(k)krkk

a then

fPekrfPE )()()]([lim 2

p.s.d.

Thus )( ][12

1lim)(2

3.1.1 Periodogram Spectral Estimator3.1.1 Periodogram Spectral Estimator©

3) (1) This estimator is based on (**)

Given an observed data set {x[0], x[1], …, x[N-1]},the periodogram is defined as

a p g21

2PER ][1)(

fnjenxN

3.1.1 Periodogram Spectral Estimator3.1.1 Periodogram Spectral Estimator©

3) (1) This estimator is based on (**)

Given an observed data set {x[0], x[1], …, x[N-1]},the periodogram is defined as

2PER ][1)(

fnjenxN

63 0nN

An Equivalent Expression of PeriodogramAn Equivalent Expression of Periodogram20

04) The periodogram estimator can be given in terms of )(kr

NkekrfP 2

)1(PER )()(

0for )()( ];[][1)(1

kkrkrknxnxN

where0n

– The quality of the estimates for the higher lags of r(k) may be poorer since they involve fewer terms of lag products in theM

poorer since they involve fewer terms of lag products in the averaging operation

Exercise: to show this from the periodogram definition in last page

An Equivalent Expression of PeriodogramAn Equivalent Expression of Periodogram20

04) The periodogram estimator can be given in terms of )(kr

NkekrfP 2

)1(PER )()(

where 0for )()( ];[][1)(1

kkrkrknxnxN

– The quality of the estimates for the higher lags of r(k) may be poorer since they involve fewer terms of lag products in theM

630 0n

poorer since they involve fewer terms of lag products in the averaging operation

p g p g

(2) Filter Bank Interpretation of Periodogram(2) Filter Bank Interpretation of Periodogram©

For a particular frequency of f0:21

2 ][1)( 0

kfj kfP

20PER ][)( 0

kfj kxeN

kxknhN

;0,1),...,1(for )2exp(1][ 0 Nnnfj

63 where

otherwise0

– Impulse response of the filter h[n]: a windowed version of a

complex exponential

(2) Filter Bank Interpretation of Periodogram(2) Filter Bank Interpretation of Periodogram©

For a particular frequency of f0:21

2 ][1)( 0

kfj kfP

20PER ][)( 0

kfj kxeN

kxknhN

;0,1),...,1(for )2exp(1][ 0 Nnnfj

Nnh where

– Impulse response of the filter h[n]: a windowed version of a

C otherwise0

complex exponential

Frequency Response of h[n]Frequency Response of h[n]©

3) )()1(exp)(i)(sin)( 0

0 ffNjffNffNfH

)()(p)(sin

ffjffN

sinc-like function centered at f0:

H(f) is a bandpass filterC f i f30

rea 0:

– Center frequency is f0

– 3dB bandwidth 1/N

Frequency Response of h[n]Frequency Response of h[n]©

3) )()1(exp)(i)(sin)( 0

0 ffNjffNffNfH

)()(p)(sin

ffjffN

sinc-like function centered at f0:

H(f) is a bandpass filterC f i f30

rea 0:

– Center frequency is f0

– 3dB bandwidth 1/N

Periodogram: Filter Bank PerspectivePeriodogram: Filter Bank Perspective

Can view the periodogram as an estimator of power spetrum that has a built-in filterbank©

p– The filter bank ~ a set of bandpass filters– The estimated p.s.d. for each frequency f0 is the power of one

p q y 0 poutput sample of the bandpass filter centering at f0

0PER ][][)(

kxknhNfP

Periodogram: Filter Bank PerspectivePeriodogram: Filter Bank Perspective

Can view the periodogram as an estimator of power spetrum that has a built-in filterbank©

p– The filter bank ~ a set of bandpass filters– The estimated p.s.d. for each frequency f0 is the power of one

p q y 0 poutput sample of the bandpass filter centering at f0

0PER ][][)(

kxknhNfP

E.g. White Gaussian ProcessE.g. White Gaussian Process[Lim/Oppenheim Fig.2.4] Periodogram of zero-mean white Gaussian noise using N-point data record: N=128, 256, 512, 1024

The random fluctuation (measured by variance) of the i d d t d ith i i N

periodogram does not decrease with increasing N periodogram is not a consistent estimator

(3) How Good is Periodogram for Spectral Estimation?(3) How Good is Periodogram for Spectral Estimation?00

?)( p.s.d. will, If PER fPPN

Estimation: Tradeoff between bias and varianced by

For white Gaussian process, we can show that at fk = k/N

Performance of Periodogram: SummaryPerformance of Periodogram: Summary

The periodogram for white Gaussian process is an unbiased estimator but not consistent

– The variance does not decrease with increasing data length– Its standard deviation is as large as the mean (equal to the quantity

to be estimated)ated

to be estimated)

Reasons for the poor estimation performance– Given N real data points, the # of unknown parameters {P(f0), … 30

p , p { ( 0),P(fN/2)} we try to estimate is N/2, i.e. proportional to N

Similar conclusions can be drawn for processes withCP

Similar conclusions can be drawn for processes with arbitrary p.s.d. and arbitrary frequencies– Asymptotically unbiased (as N goes to infinity) but inconsistent

3.1.2 Averaged Periodogram3.1.2 Averaged Periodogram

As one solution to the variance problem of periodogram– Average K periodograms computed from K sets of data records

g p g p

PERPER AV )(1)(K

V )()(m

where 212

PER ][1)(

fnjenxfP

0PER ][)(

m enxL

And the K sets of data records are

... ;10 ],[ ];1[ ..., ],0[{ 100 LnnxLxx}10 ],1[{ 1 LnnxK

}],[{ 1K

Performance of Averaged PeriodogramPerformance of Averaged Periodogram

– If K sets of data records are uncorrelated with each other,we have: ( fi = i/L )

( fi )

)(1 2ifP

KK for 0Varand,Var i.e., KK for 0Var and ,Var i.e.,

KK for 0Varand,Var i.e., ,,i.e., consistent estimatei.e., consistent estimate

,,i.e., consistent estimate

Performance of Averaged PeriodogramPerformance of Averaged Periodogram

– If K sets of data records are uncorrelated with each other,we have : ( fi = i/L )

( fi )

)(1 2ifP

KK for 0Var and ,Var i.e.,i e consistent estimate

i.e., consistent estimate

Practical Averaged PeriodogramPractical Averaged Periodogram

Usually we partition an available data sequence of length Ninto K non overlapping blocks each block has length L (i e N=KL)©

– into K non-overlapping blocks, each block has length L (i.e. N=KL)

i.e. 1 ..., ,1 ,0 ,][][ LnmLnxnxm

Since the blocks are contiguous, the K sets of data records 30 S

a 1...,,1,0 Km

gmay not be completely uncorrelated– Thus the variance reduction factor is in general less than K

Periodogram averaging is also known as the Bartlett’s method

Averaged Periodogram for Fixed Data SizeAveraged Periodogram for Fixed Data Size

Given a data record of fixed size N, will the result be better if we segment the data into more and more subrecords?

We examine for a real-valued stationary process:

1 1 )(K m

a )(ˆ)(1)( )0(PER

PERPER AV fPEfPK

identical stat. mean for all mNote

2)0()0(PER )(ˆ)(ˆ

LfljelrfP

lnxnxlr1

)0( ][1)(ˆ

an equivalent expression to definition in terms of x[n]

lnxnxL

][)( of x[n]

Mean of Averaged PeriodogramMean of Averaged Periodogram©

Mean of Averaged Periodogram (cont’d)Mean of Averaged Periodogram (cont’d)©

3) fkrkwfPE )}(][{DTFT)](ˆ[ PER AV 1

multiplication in time

dPfW )()(21

convolution in frequency

Biased estimate (both averaged and regular periodogram)– The convolution with the window function w[k] lead to the mean of30

rea )( fP

– The convolution with the window function w[k] lead to the mean of the averaged periodogram being smeared from the true p.s.d

Asymptotic unbiased as L

– To avoid the smearing, the window length L must be large enough so that the narrowest peak in P(f) can be resolved

This gives a tradeoff between bias and variance

gSmall K => better resolution (smaller smearing/bias) but larger variance

Mean of Averaged Periodogram (cont’d)Mean of Averaged Periodogram (cont’d)©

3) fkrkwfPE )}(][{DTFT)](ˆ[ PER AV 1

multiplication in time

dPfW )()(21

convolution in frequency

Biased estimate (both averaged and regular periodogram)– The convolution with the window function w[k] lead to the mean of30

rea )( fP

– The convolution with the window function w[k] lead to the mean of the averaged periodogram being smeared from the true p.s.d

Asymptotic unbiased as L

– To avoid the smearing, the window length L must be large enough so that the narrowest peak in P(f) can be resolved

This gives a tradeoff between bias and variance

gSmall K => better resolution (smaller smearing/bias) but larger variance

NonNon--parametric Spectrum Estimation: Recapparametric Spectrum Estimation: Recap

Periodogram– Motivated by relation between p.s.d. and squared magnitude of DTFT

of a finite-size data record– Variance: won’t vanish as data length N goes infinity ~ “inconsistent”– Mean: asymptotically unbiased w r t data length N in generalMean: asymptotically unbiased w.r.t. data length N in general

equivalent to apply triangular window to autocorrelation function(windowing in time gives smearing/smoothing in freq. domain)

unbiased for white Gaussian (flat spectrum) unbiased for white Gaussian (flat spectrum)

Averaged periodogram– Reduce variance by averaging K sets of data record of length L eachReduce variance by averaging K sets of data record of length L each– Small L increases smearing/smoothing in p.s.d. estimate thus higher

bias equiv. to triangular windowing

Windowed periodogram: generalize to other symmetric windows

Case Study on NonCase Study on Non--parametric Methodsparametric Methods Test case: a process consists of narrowband components

(sinusoids) and a broadband component (AR)– x[n] = 2 cos(1 n) + 2 cos(2 n) + 2 cos(3 n) + z[n]

where z[n] = a1 z[n-1] + v[n], a1 = 0.85, 2 = 0.1 /2 = 0 05 /2 = 0 40 /2 = 0 421/2 = 0.05, 2/2 = 0.40, 3/2 = 0.42

– N=32 data points are available periodogram resolution f = 1/32

Examine typical characteristics of various non-parametric pspectral estimators

(Fig.2.17 from Lim/Oppenheim book)

3.1.3 Periodogram with Windowing3.1.3 Periodogram with Windowing

Review and Motivation

The periodogram estimator can be given in terms of )(k

u © The periodogram estimator can be given in terms of )(kr

2PER )()(

NfkjekrfP

a )1( Nk

where )()( ];[][1)(1

krkrknxnxN

– The higher lags of r(k), the poorer estimates since the estimates

)()(][][)(0nN 0for k

g g ( ) pinvolve fewer terms of lag products in the averaging operation

Solution: weigh the higher lags less

– Trade variance with bias

WindowingWindowing Use a window function to weigh the higher lags less

w(0)=1 preserves variance r(0)

Effect: periodogram smoothing – Windowing in time Convolution/filtering the periodogram

g g p g– Also known as the Blackman-Tukey method

Common Lag WindowsCommon Lag Windows Much of the art in non-parametric spectral estimation is in

choosing an appropriate window (both in type and length)

Table 2.1 common lag window (from Lim-Oppenheim book)

Discussion: Estimate r(k) via Time AverageDiscussion: Estimate r(k) via Time Average Normalizing the sum of (N-k) pairs

by a factor of 1/N ? v.s. by a factor of 1/(N-k) ?Biased (low variance) Unbiased (may not non-neg. definite)

• Hints on showing the non-negative definiteness: using )(kr

definiteness: using to construct correlation matrix

)(1 kr

HW#8)(For 2 kr

Discussion: Estimate r(k) via Time AverageDiscussion: Estimate r(k) via Time Average Normalizing the sum of (N-k) pairs

by a factor of 1/N ? v.s. by a factor of 1/(N-k) ?Biased (low variance) Unbiased (may not non-neg. definite)

• Hints on showing the non-negative definiteness: using )(kr

definiteness: using to construct correlation matrix

)(1 kr

HW#8)(For 2 kr

3.1.43.1.4 Minimum Variance Spectral Estimation Minimum Variance Spectral Estimation (MVSE)(MVSE)

Recall: filter bank perspective of periodogram– The periodogram can be viewed as estimating the p s d byThe periodogram can be viewed as estimating the p.s.d. by

forming a bank of narrowband filters with sinc-like response– The high sidelobe can lead to “leakage” problem:

large output power due to p.s.d outside the band of interest

MVSE designs filters to minimize the leakage from out-of-MVSE designs filters to minimize the leakage from out ofband spectral components– Thus the shape of filter is dependent on the frequency of interest

d d t d tiand data adaptive(unlike the identical filter shape for periodogram)

– MVSE is also referred to as the Capon spectral estimator

Main Steps of MVSE MethodMain Steps of MVSE Method

1. Design a bank of bandpass filters Hi(f) with center frequency fi so thati

– Each filter rejects the maximum amount of out-of-band power– And passes the component at frequency fi without distortion

2. Filter the input process { x[n] } with each filter in the filter bank and estimate the power of each output processbank and estimate the power of each output process

3. Set the power spectrum estimate at frequency fi to be the power estimated above divided by the filter bandwidth

Formulation of MVSEFormulation of MVSE

The MVSE designs a filter H(f) for each f f i t t ffrequency of interest f0

minimize the output power

dffPfH )()( 221

minimize the output power

dffPfH )()(21

subject to 1)( 0 fH

(i.e., to pass the components at f0 w/o distortion)

Formulation of MVSEFormulation of MVSE

The MVSE designs a filter H(f) for each f f i t t ffrequency of interest f0

minimize the output powerminimize the output power

dffPfH )()( 221

subject to

dffPfH )()(21

1)( 0 fH

(i.e., to pass the components at f0 w/o distortion)

Deriving MVSE SolutionsDeriving MVSE Solutions

Output Power From H(f) filterOutput Power From H(f) filter

From the filter bank perspective of periodogram:0

2][)(Nn

fnjenhfH

dffPelhekh fljfkj )(][][0

)(2)(][][ klfj dfefPlhkh

ffNlNk

)(][][)1(2

)1( )1( 2

1 )(][][Nk Nl

dfefPlhkh

)(][][ klrlhkhEquiv. to filter r(k) with { h(k) h*(-k) } and evaluate at

)1( )1(

)(][][Nk Nl

klrlhkh

and evaluate at output time k=0

Output Power From H(f) filterOutput Power From H(f) filter

From the filter bank perspective of periodogram:0

2][)(Nn

fnjenhfH

dffPelhekh fljfkj )(][][0

NlNk)(][][

)(2)(][][ klfj dfefPlhkh

Equiv. to filter r(k) with { h(k) h*(-k) } and evaluate at

)1( )1( 2

1 )(][][Nk Nl

dfefPlhkh

)(][][ klrlhkh

and evaluate at output time k=0and evaluate at output time k=0

)1( )1(

)(][][Nk Nl

klrlhkh

MatrixMatrix--Vector Form of MVSE FormulationVector Form of MVSE Formulation

Define

The constraint can be written in vector form as 1ehH

)( 0fH

Thus the problem becomes

hRh THmin subject to 1ehH

MatrixMatrix--Vector Form of MVSE FormulationVector Form of MVSE Formulation

Define

The constraint can be written in vector form as 1ehH

)( 0fH

Thus the problem becomes

hRh THmin subject to 1ehH

Solving MVSESolving MVSE )1(2Re ehhRhJ HTHdef

Use Lagrange multiplier approach for solving the constrained optimization problem

– Define real-valued objective function s.t. the stationary condition can be derived in a simple and elegant way based on the theorem for complex derivative/gradient operatorsp g p

)1()1(min*

,ehehhRhJ HHTH

)1()1( * heehhRh HHTH

00either * ehRJ T 1

00either

ehRJTTH

00 ehRehR THT 1 and eh

Solution to MVSESolution to MVSE *

,)1()1(min ehehhRhJ HHTH

)( 1 0or H*

)( )(0 0or 1*

eRhehRJ TT

The optimal filter and its output power:

T 1 RTH 11

Bring () into ():

Filter’s output power:

eRe TH 11

1eRRhhRh TTHTH

MVSE: SummaryMVSE: Summary

If choosing the bandpass filters to be FIR of length q, its 3dB-b.w. is approximately 1/q

Thus the MVSE ismatrix ncorrelatio is ˆ qqR

eReqfP

TH 1MVˆ

)2exp(

(i.e. normalize by filter b.w.)

))1(2exp( qfj

MVSE is a data adaptive estimator and provides improved resolution and reduced variance over periodogram– Also referred to as “High-Resolution Spectral Estimator”

Also referred to as High Resolution Spectral Estimator– Doesn’t assume a particular underlying model for the data

MVSE vsMVSE vs. . PeriodogramPeriodogram

MVSE is a data adaptive estimator and provides improved resolution and reduced variance over periodogramp g

Periodogram MVSEEquivalent Bandpass Filter

hFilter is “universal” data-independent

Filter adapts to observation data via R

E i l t

Equivalent spectrum estimate eRe

qTH 1ˆ

)( fP eReq TH ˆ

Recall: Case Study on NonRecall: Case Study on Non--parametric Methods parametric Methods Test case: a process consists of narrowband components

(sinusoids) and a broadband component (AR)– x[n] = 2 cos(1 n) + 2 cos(2 n) + 2 cos(3 n) + z[n]

where z[n] = a1 z[n-1] + v[n], a1 = 0.85, 2 = 0.1 /2 = 0 05 /2 = 0 40 /2 = 0 421/2 = 0.05, 2/2 = 0.40, 3/2 = 0.42

– N=32 data points are available periodogram resolution f = 1/32

Examine typical characteristics of various non-parametric pspectral estimators

(Fig.2.17 from Lim/Oppenheim book)

Ref. on Derivative and Gradient Operators for Ref. on Derivative and Gradient Operators for C lC l V i bl F tiV i bl F tiComplexComplex--Variable FunctionsVariable Functions

Ref: D.H. Brandwood, “A complex gradient operator and its application i d ti th ” i IEE P l 130 P t F d H 1in adaptive array theory,” in IEE Proc., vol. 130, Parts F and H, no.1, Feb. 1983. (downloadable from IEEEXplorer)

– Solving constrained optimization ith l l d bj ti f ti f l i blwith real-valued objective function of complex variables,

subject to constraint function of complex variables As seen in minimum variance spectral estimation and other As seen in minimum variance spectral estimation and other

array/statistical signal processing context.

UMD ENEE630 Advanced Signal Processing (ver.1211) Discussions [64]

ReferenceReferenceby M

ReferenceReference

Recall: Filtering a Random ProcessRecall: Filtering a Random Process

ChiChi--Squared DistributionSquared Distribution

ChiChi--Squared Distribution (cont’d)Squared Distribution (cont’d)

Periodogram of White Gaussian ProcessPeriodogram of White Gaussian Process

See proof in Appendix 2 1 in Lim Oppenheim Book:

See proof in Appendix 2.1 in Lim-Oppenheim Book:- Basic idea is to examine the distribution of real and imaginary part of the DFT, and take the magnitude

Preview & WarmPreview & Warm upupby M

Preview & WarmPreview & Warm--upup

Model or Not?Model or Not? Implicit assumption by classical methods

– Classical methods use Fourier transform on either windowed d t i d d t l ti f ti (ACF)data or windowed autocorrelation function (ACF)

– Implicitly assume the unobserved data or ACF outside the window are zero => not true in reality

– Consequence of windowing: smeared spectral estimate(leading to low resolution)

If prior knowledge about the process is available If prior knowledge about the process is available– Can use prior knowledge and select a good model to

approximate the process– Usually need to estimate fewer model parameters (than non-

parametric approaches) using the limited data points we have– The model may allow to better describe the process outside

The model may allow to better describe the process outside the window (instead of assuming zeros)

General Procedure of Parametric MethodsGeneral Procedure of Parametric Methods

Select a model (based on prior knowledge)

Estimate the parameters of the assumed model

Obtain the spectral estimate implied by the model (with the estimated parameters)

Spectral Estimation using AR, MA, ARMA ModelsSpectral Estimation using AR, MA, ARMA Models

Physical insight: the process is generated/approximated by filtering white noise with an LTI filter of rational transfer func H(z)

Use observed data to estimate a few lags of r(k)– Larger lags of r(k) can be implicitly extrapolated by the model

Relation between r(k) and filter parameters {ak} and {bk}PARAMETER EQUATIONS f S ti 2 1 2(6) i thi– PARAMETER EQUATIONS from Section 2.1.2(6) review this

– Solve the parameter equations to obtain filter parameters– Use the p.s.d. implied by the model as our spectral estimatep p y p

Deal with nonlinear parameter equationsTry to “convert” or relate them to AR models that has linear equations

– Try to convert or relate them to AR models that has linear equations

Review: Parameter EquationsReview: Parameter Equations

Yule-Walker equations (for AR process)

ARMA model MA model

Spectrum Spectrum Estimation with Estimation with AR ModelingAR Modeling

• Use Levinson-Durbin recursion and solve for

– Approximate the observed data sequence {x[0] x[N]} with an ARApproximate the observed data sequence {x[0], …, x[N]} with an AR model (consider real-valued process here for simplicity)

– Use biased ACF estimate here to ensure nonnegative definiteness and ll i th bi d ti t (di idi b N k)

smaller variance than unbiased estimate (dividing by N-k)

MA MA Spectral EstimationSpectral Estimation

An MA(q) modelqq

kk zbzBknvbnx

00)( ][][

can be used to define an MA spectral estimator2

22 1)(ˆ q

fkjbfP

Recall: ( ) f f { ( )} f

22MA 1)(

fkjkebfP

(1) The problem of solving for bk given {r(k)} is to solve a set of nonlinear equations;

(2) An MA process can be approximated by an AR process of

( ) p pp y psufficiently high order.

Basic Idea to Avoid Solving Nonlinear EquationsBasic Idea to Avoid Solving Nonlinear Equations

Consider two processes:

Process#1: we observed N samples and need to perform Process#1: we observed N samples, and need to perform spectral estimate– We first model it as a high-order AR process, generated by 1/A(z) filter

Process#2: an MA-process generated by A(z) filter– Since we know A(z), we can know process#2’s autocorrelation function;

– We model process#2 as an AR(q) process => the filter would be 1/B(z)

Complex Exponentials in Additive NoiseComplex Exponentials in Additive Noise

UMD ENEE630 Advanced Signal Processing Frequency estimation [78]

Correlation Matrix for the ProcessCorrelation Matrix for the Process

– Determine autocorrelation function

– Rs = ? Rw = ? Rx = ?

– Rank of correlation matrices?

E[ x( ) w( ) ] = E[x( )] E[w( )] = 0

E[ x( ) w( ) ] = E[x( )] E[w( )] = 0this crosscorr term vanish because of uncorrelated *and* zero mean for either x( ) or w( ).

Part 3. Part 3. Spectrum Spectrum …classweb.ece.umd.edu › enee630.F2012 › slides ›...

Documents

TV WHITE SPACE AS PART OF THE FUTURE SPECTRUM …

Multi-rate Signal Processingclassweb.ece.umd.edu/enee630.F2012/slides/part-1_sec1_2_handoutPreLec.pdf · Single-rate processing: the digital samples before and after processing correspond

Part 3. Spectrum Estimation - UMD ECE Class Sitesclassweb.ece.umd.edu/enee630.F2012/slides/part-3_sec2... · 2012. 11. 28. · ENEE630 PartENEE630 Part--33 Part 3. Spectrum Estimation

Part 3Part 3. Spectrum . Spectrum EstimationEstimation 3.1 ...classweb.ece.umd.edu/.../part-3_sec1_handoutPreLec.pdf · Part 3Part 3. Spectrum . Spectrum EstimationEstimation 3.1

Quick Start Guide - RefPay - Part of the NCAA Spectrum

Part III Spectrum Estimation 3.1 Classic Methods for ...ECE792-41 Statistical Methods for Signal Analytics Part III Spectrum Estimation 3.1 Classic Methods for Spectrum EstimationElectrical

Multi-rate Signal Processing 4. Multistage Implementations ...classweb.ece.umd.edu/enee630.F2012/slides/part-1_sec4_5_handout.pdfAcknowledgment: ENEE630 slides were based on class

The Disabled Throwing Shoulder: Spectrum of Pathology Part

Serving Children with Autism Spectrum Disorder (Part 1)

Serving Children with Autism Spectrum Disorder: Part 2

Parametric Signal Modeling and Linear Prediction Theory 1. …classweb.ece.umd.edu/enee630.F2012/slides/part-2 sec1_v3... · 2012-10-15 · 1 Discrete-time Stochastic Processes Appendix:

An Introduction to Radiofrequency Radiation. What is Radiofrequency Radiation (RF)? Part of the electromagnetic spectrum Part of the electromagnetic spectrum

Part-II Parametric Signal Modeling and Linear Prediction ...classweb.ece.umd.edu/enee630.F2012/slides/part-2... · 2 Discrete Wiener Filter Appendix: Detailed Derivations 2.0 Preliminaries

BHECN webinar series The Schizophrenia Spectrum: Part 2 ...€¦ · BHECN webinar series The Schizophrenia Spectrum: Part 2-Practice October 2017 Will Spaulding, Ph.D. Professor of

Serving Children with Autism Spectrum Disorder (Part 2)

Part 3. Spectrum Estimation 3.3 Subspace 3.3 Subspace ...classweb.ece.umd.edu/enee630.F2012/slides/part-3_sec3_F12_slidesAll.pdfPisarenko Method for Frequency Estimation (1973) This

Electromagnectic Spectrum Chapter 1 section 2. Electromagnetic Spectrum In 1852, James Clerk Maxwell proved that visible light was part of a larger spectrum

PART 3A - CT Compliance Manual - Spectrum Enterprises

Radio Communication Geoff Partridge. The Radio Spectrum Radio Spectrum refers to the part of the electromagnetic spectrum corresponding to radio frequencies

Nephrocytes are part of the spectrum of filtration