MORE ON ML ESTIMATION UNDER MISSPECIFIED NUMBERS OF SIGNALS

Pei-Jung Chung

Institute for Digital CommunicationsJoint Research Institute for Signal & Image Processing

School of Engineering and Electronics, The University of Edinburgh, UKpeijung.chung@gmail.com

ABSTRACT

The maximum likelihood (ML) approach for estimating direction ofarrival (DOA) plays an important role in array processing. Its con-sistency and efficiency have been well established in literature. Acommon assumption is that the number of signals is known. In manyapplications, this information is not available and needs to be esti-mated. However, the estimated number of signals does not neces-sarily equal the true number of signals. Therefore, it is important toknow whether the ML estimator provides any relevant informationabout the true parameters. Previous study on the ML estimattion un-der misspecified numbers of signals have focused on the asymptoticproperties. In this work, we investigate the impact of misspecifica-tion on estimation performance and show that the covariance matrixgrows monotonically with increasing degree of mismatch. Finally,we carry out numerical experiments under various cases of misspec-ification and further validate our analysis.

1. INTRODUCTION

The problem of estimating direction of arrival (DOA) plays a keyrole in array processing. Among existing methods, the maximumlikelihood (ML) approach has the best asymptotic properties. It isalso known to be robust against small sample numbers, signal coher-ence and closely located sources.

The asymptotic properties of the ML approach have been wellstudied in the literature [5, 6]. Therein, it is implicitly assumed thatthe number of signals is known in advance. In many applications,this information is not available and the number of signals needsto be determined via an additional procedure. However, the esti-mated number of signals does not necessarily coincide with the trueone. Thus, it is crucial to know whether the ML estimator providesany relevant information about DOA parameters under a misspeci-fied number of signals. In particular, we want to know (1) whetherthe ML estimator converges to any meaningful limit and (2) how amisspecified number of signals affects its performance.

In [2], we showed that the ML estimator under a misspecifiednumber of signals converges to a well defined limit. When signalsources are well separated, the ML estimator converges to the trueparameters. In [3], we applied the general theory of misspecifiednonlinear regression models and derived a compact expression for

P.-J. Chung acknowledges support of her position from the ScottishFunding Council and their support of the Joint Research Institute with theHeriot-Watt University as a component part of the Edinburgh Research Part-nership.

the asymptotic covariance matrix. In this work, we study the im-pact of misspecification on estimation accuracy. More specifically,we shall make analytical comparison between different degrees ofmisspecification and show that the covariance matrix grows with in-creasing degree of mismatch.

This paper is outlined as follows. We give a brief descriptionof the signal model in the next section. Section 3 reviews someuseful results. We study the impact of misspecification in Section4. Numerical results are presented and discussed in Section 5. Ourconcluding remarks are given in Section 6.

Notations

Notations used throughout this paper are summarized here.

1. (·)T denotes matrix transpose. (·)H denotes Hermitian trans-pose.� denotes element-wise multiplication.

2. In is an n × n identity matrix.

3. sp(A) denotes the subspace spanned by A’s columns.Re(A) denotes the real part of A.

4. D = [d′

1 · · ·d′

m], d′

i = ∂d(θi)/∂θi.

5. For any m, Csm = E[ sm(t)sm(t)H ] ,where sm = [ s1(t), . . . , sm(t) ]T .

For m0 < m, Cs0 = E[ sm(t)sm(t)H ] ,where sm(t) = [ sm0+1(t), . . . , sm(t) ]T .

6. H0 = H(θ0).

For m < m0, H0 = [H0 H0].

H0 = [d(θ1) · · ·d(θm)], H0 = [d(θm+1) · · ·d(θm0)].

2. PROBLEM FORMULATION

Consider an array of n sensors receiving narrow band signals emittedby m far-field sources located at θm = [ θ1,. . ., θm ]T . The arrayoutput x(t) can be expressed as

x(t) = H(θm)s(t) + n(t), t = 1, . . . , T (1)

83

1-4244-0882-2/07/$20.00 c©2007 IEEE

where the ith column of the matrix

H(θm) = [d(θ1) · · ·d(θi) · · · d(θm)] (2)

d(θi) is the steering vector associated with the signal arriving fromthe direction θi. The n × m matrix H(θm) is assumed to be fullrank. The complex signal waveform s(t) = [s1(t),. . ., sm(t)]T isrealization of an independent stationary process. The noise vectorn(t) is independent, identically complex normally distributed withzero mean and covariance matrix σ2

In, where σ2 is an unknownnoise spectral parameter.

Based on the observations {x(t)}Tt=1 and a pre-specified num-

ber of signals, m, the conditional ML estimate for θm is obtained byminimizing the negative concentrated likelihood function [1]

θm(T ) = arg minθm

tr[P ⊥Cx ] (3)

where P⊥ = I −P and P is the projection matrix into the column

space of H(θm). Cx = 1

T

T

t=1x(t)x(t)H represents the sample

covariance matrix.

To investigate the asymptotic behavior of θm(T ), we furtherassume that for large T , Cx converges to the true covariance matrix

Cx0 = H(θ0)Cs0H(θ0)H + σ

2In (4)

where the steering matrix H(θ0) is computed at the true parameterθ0 = [θ1, · · · , θm0

]T and m0 denotes the true number of signals.The signal covariance matrix Cs0 is given by E[s0(t)s0(t)

H ] wheres0(t) represents the m0×1 signal vector. For simplicity, we consideronly uncorrelated signals, i.e. Cs0 is a diagonal matrix.

3. ASYMPTOTIC PROPERTIES OF ML ESTIMATORUNDER MISSPECIFIED NUMBERS OF SIGNALS

It is well established that the ML estimator converges to the true pa-rameter θ0 with increasing sample size [5, 6] when the number ofsignals, m, is correctly specified. In practice, m is usually unknownand needs to be estimated via an additional step. As in any esti-mation problems, the estimate m does not necessarily coincide withthe true number of signals m0. Hence, whether the ML estimateprovides any useful information about θ0 with a misspecified m iscrucial to practical and theoretical considerations.

In previous works [2, 3], we applied the general theory of mis-specified nonlinear least regression models [7] to study the asymp-totic behavior of the ML estimator. In particular, we showed thatfor a misspecified m, the ML estimator θm(T ) converges to a welldefined limit θ

∗

m that minimizes the ensemble average of the con-centrated likelihood function

l(θm) = E tr[P ⊥

mx(t)x(t)H ] . (5)

The signal subspace computed at θ∗

m is related to the true signal sub-space as follows.

Property 1 (a) For m < m0,

sp(H(θ∗

m)) ⊂ sp(H0) (6)

and the elements of θ∗

m coincide with m elements of θ0 for widelyseparated sources.

(b) For m > m0,

sp(H(θ∗

m)) ⊃ sp(H0) (7)

and θ∗

m has m0 components equal to those of θ0 for widely sepa-rated sources. The remaining (m − m0) components of are unpre-dictable.

Proof Details can be found in [2]. �

In addition to the consistency property, the ML estimator is alsocharacterized by asymptotic normality. As pointed out in (b) ofProperty 1, in the case of m > m0, some of the elements of θ

∗

m

are unpredictable even for large T . The covariance matrix does notseem to be a proper measure for this case. Hence we will restrict ourdiscussion to the case in which m < m0 in the following discussion.

Property 2 Suppose m < m0 and that the ML estimator θm(T )

(3) converges to θ∗

m . Then√

T (θm(T ) − θ∗

m) is asymptoticallynormally distributed with zero mean and the covariance matrix

C(m) = A−1

m BmA−1

m , (8)

where

Am = 2Re (DHP

⊥D) � Csm

−2Re (DHH0Cs0H

H

0 D) � (HHH)−1

, (9)

Bm = 2σ2Re (DH

P⊥

D) � (Csm + σ2(HH

H)−1)

+2Re (DHH0Cs0H

H

0 D)�(Csm+σ2(HH

H)−1) .

(10)

Proof Details can be found in [3]. �

Remark The minor error of the second term of the matrix Bm pre-sented in Theorem 1 of [3] has been corrected in (10).

4. IMPACT OF MODEL MISSPECIFICATION

Property 1 and 2 summarize the asymptotic properties of the MLestimator under misspecified numbers of signals. Despite misspeci-fication, the ML estimator still provides useful information about thetrue parameter θ0. When θ

∗

m contain elements of θ0, it is interest-ing to know how does misspecification of m affect the performanceof ML estimation ? In the following, we will make analytical com-parison between covariance matrices associated with various degreeof mismatch.

The following result plays an important role in our analysis.

Lemma 1 Let

C = A−1

BA−1

, (11)

CΔ = (A −ΔA)−1(B + ΔB)(A − ΔA)−1, (12)

where ΔA and ΔB are positive definite matrices. Suppose A−ΔA

is also positive definite, then

84 Proc. of the 2007 15th Intl. Conf. on Digital Signal Processing (DSP 2007)

CΔ > C (13)

in the positive definite sense.

Proof Since ΔA and ΔB are positive definite,

CΔ = (A − ΔA)−1(B + ΔB)(A − ΔA)−1

> (A − ΔA)−1B (A − ΔA)−1

> A−1

B (A − ΔA)−1

> A−1

B A−1 = C .

The above inequality leads to (13). �

Let C(m) denote the covariance matrix of ML estimator whenno misspecification is present, i.e. the true number of signals is m.According to [6], C(m) can be expressed as

C(m) = A−1

m BmA−1

m , (14)

Am = 2Re (DHP

⊥D) � Csm , (15)

Bm = 2σ2Re (DH

P⊥

D) � (Csm + σ2(HH

H)−1) .

(16)

Using Lemma 1, we can readily show that the variance of theML estimator increases due to misspecified numbers of signals.

Lemma 2 Suppose the assumed number of signals m < m0. Thenthe covariance matrix of the mismatch case, C(m), is related to thatof the no mismatch case, C(m) as follows:

C(m) < C(m) . (17)

Proof Compare (9) with (15), we have

Am = Am − ΔA (18)

where

ΔA = 2Re (DHH0Cs0H

H

0 D) � (HHH)−1

. (19)

Since the matrices (DHH0Cs0H

H

0 D) and (HHH)−1 are posi-

tive definite, the resulting Hadamard product is also positive definite[4]. Therefore, ΔA is positive definite.

Similarly, from (10) and (16), we have

Bm = Bm + ΔB (20)

where

ΔB = 2Re (DHH0Cs0H

H

0 D)�(Csm+σ2(HH

H)−1) . (21)

Since (DHH0Cs0H

H

0 D) and (Csm+σ2(HHH)−1) are positive

definite, the resulting Hadamard product is also positive definite [4].Thus, ΔB is positive definite.

By Lemma 1, for positive definite ΔA and ΔB , we have thedesired inequality:

C(m) = (Am −ΔA)−1(Bm + ΔB)(Am −ΔA)−1

> A−1

m BmA−1

m = C(m).

�

In the above proof, we have implicitly assumed that C(m) iscomputed at θ

∗m. Lemma 2 implies that the asymptotic variance of

the ith component, θi(T ), is larger than that in the no mismatch case.

In addition to the comparison with the no mismatch case, we alsoconsider different degrees of mismatch. To emphasize the degree ofmismatch, we use θ

∗m|m0

and C(m|m0) to denote the limiting pointand covariance matrix when the true number of signals is m0. Weshow in Lemma 3 that a larger mismatch leads to a larger covariancematrix.

Lemma 3 Suppose M0 > m0 > m and θ∗m|M0

= θ∗m|m0

. Then

C(m|m0) < C(m|M0) . (22)

Proof According to (8),

C(m|M0) = A−1

m|M0Bm|M0

A−1

m|M0, (23)

where Am|M0and Bm|M0

are defined in (9) and (10), respectively.As M0 > m0, it can be easily verified that

Am|M0= Am|m0

− Δ′A, (24)

Bm|M0= Bm|m0

+ Δ′B , (25)

where Δ′A and Δ′

B are positive definite. By Lemma 3, we have thedesired result (22).

�

Remark Combining Lemma 2 and 3, we establish the following in-equality

C(m) < C(m|m0) < C(m|M0), (26)

which implies that the covariance matrix grows with increasing de-gree of misspecification.

5. SIMULATION

We confirm our analysis by numerical experiments. In particular,we consider two cases of mismatch. The number of signals m isassumed to be 2. The array outputs are simulated by m0 = 3 andm0 = 4 signals. For comparison, the data is also generated withm0 = 2. The empirical variances of the ML estimates and com-pared with theoretical values.

In the first experiment, the true number of signals, m0 = 3. Auniform linear array of 15 sensors with inter-element spacings of halfa wavelength is employed. Three uncorrelated narrow band signalsare generated by three sources located at θ0 = [−30◦ 12◦ 80◦] witha SNR difference [3 2 0] dB. In the second experiment, m0 = 4, weadd a signal located at −50◦ and θ0 = [−30◦ 12◦ 80◦ − 50◦]. TheSNR difference is given by [3 2 0 0] dB. The Signal to Noise Ratio(SNR) of the weakest signals runs from −10 to 10 dB in a 2 dB step.The number of snapshots is T = 200. We perform 200 trials foreach experiment.

The empirical variances of the first component of the ML esti-mator, θ1, are depicted in Fig 1. As expected, the case of no mis-match m0 = 2 has the lowest variances among all three considered

Proc. of the 2007 15th Intl. Conf. on Digital Signal Processing (DSP 2007) 85

−10 −8 −6 −4 −2 0 2 4 6 8 1010

−2

10−1

100

SNR (dB)

square

rootofvar(

θ 1)

ML estimate under misspecified number of signals: m=2,θ1

m0=2

m0=3

C2|3

m0=4

C2|4

Fig. 1. The square root of empirical variance of θ1. m = 2, m0 = 3,4. SNR = [−10 : 2 : 10] dB, number of snapshots T = 200. Empiricalvariance, −◦ : m0 = 2, −. : m0 = 3, · · · : m0 = 4. Theoretical value,− : m0 = 3, −− : m0 = 4.

scenarios. The variances associated with m0 = 3 are lower thanthose associated with m0 = 4. With increasing SNR the differencebetween the three curves becomes larger. The theoretical values ob-tained from (8) are also presented in Fig 1. Over the entire SNRrange, the predicted values are very close to the empirical variances.The behavior of the second component, θ2, is similar to that pre-sented in Fig 1.

In summary, the empirical variance of the ML estimator growswith increasing degree of mismatch and can be well predicted byeq (8). The predicted values are more accurate than the previousformula presented in [3]. Both the empirical and theoretical valuesconfirm analytical results presented in Section 4.

6. CONCLUSION

We investigate the performance of conditional ML estimation undermisspecified numbers of signals. While previous anaysis [2, 3] fo-cused on the asymptotic properties of the ML estimator, we studythe impact of misspecification on estimation performance. In par-ticular, we make analytical comparison between asymptotic covari-ance matrices in the following three cases: 1) m = m0 wherethe number of signals is correctly specified, 2) m < m0 and 3)m < M0 where m0 < M0. We prove that the covariance matri-ces associated with 1), 2) and 3) are related through the inequality:C(m) < C(m|m0) < C(m|M0) . This implies that misspeci-fication increases the variance of the ML estimator in a monotonicmanner. This relation is confirmed by numerical experiments overa wide range of SNRs. Both empirical and theoretical variances arewell predicted by our analysis.

7. REFERENCES

[1] Johann F. Bohme. Statistical array signal processing of mea-sured sonar and seismic data. In Proc. SPIE 2563 AdvancedSignal Processing Algorithms, pages 2–20, San Diego, Jul 1995.

[2] P.-J. Chung. ML estimation under misspecified number of sig-nals. In Proc. of the 39th Asilomar Conference on Signals, Sys-tems, and Computers, Pacific Grove, CA, USA, Oct 30 - Nov 2,2005.

[3] Pei-Jung Chung. Performance analysis of ML estimation undermisspecified numbers of signals. In IEEE Proc. Sensor Arrayand Multichannel Signal Processing Workshop, Boston, USA,July 2006.

[4] Roger A. Horn and Charles R.Johnson. Matrix Analysis. 1985,Cambridge University Press.

[5] D. Kraus. Approximative Maximum-Likelihood-Schatzung undverwandte Verfahren zur Ortung und Signalschatzung mit Sen-sorgruppen. Dr.–Ing. Dissertation, Faculty of Electrical En-gineering, Ruhr–Universitat Bochum, Shaker Verlag, Aachen,1993.

[6] Petre Stoica and Arye Nehorai. Performance study of condi-tional and unconditional direction–of–arrival estimation. IEEETrans. ASSP, 38(10):1783–1795, October 1990.

[7] Halbert White. Consequences and detection of misspecifiednonlinear regression models. Journal of American StatisticalAssociation, 76(374):419–433, 1981.

86 Proc. of the 2007 15th Intl. Conf. on Digital Signal Processing (DSP 2007)