30
A CFAR Adaptive Subspace Detector for Second Order Gaussian Signals Yuanwei Jin and Benjamin Friedlander Abstract In this paper we study the problem of detecting subspace signals described by the Second Order Gaussian (SOG) model in the presence of noise whose covariance structure and level are both unknown. Such a detection problem is often called Gauss-Gauss problem in that both the signal and the noise are assumed to have Gaussian distributions. We propose adaptive detectors for the SOG model signals based on a single observation and multiple observations. With a single observation, the detector can be derived in a manner similar to that of the generalized likelihood ratio test (GLRT), but the unknown covariance structure is replaced by sample covariance matrix based on training data. The proposed detectors are constant false alarm rate (CFAR) detectors. As a comparison, we also derive adaptive detectors for the First Order Gaussian (FOG) model based on multiple observations under the same noise condition as for the SOG model. With a single observation, the seemingly ad hoc CFAR detector for the SOG model is a true GLRT in that it has the same form as the GLRT CFAR detector for the FOG model. We give an approximate closed form of the probability of detection and false alarm in this case. Furthermore, we study the proposed CFAR detectors and compute the performance curves. Keywords Adaptive detection, CFAR detector, Generalized LikelihoodRatio Test (GLRT), Second Order Gaussian Model, First Order Gaussian Model Y. Jin is with the University of California at Santa Cruz, CA 95060 USA (e-mail: [email protected]). B. Friedlander is with the University of California at Santa Cruz, CA 95060 USA (e-mail: [email protected]). This work was supported by the Office of Naval Research, under grant no. N00014-01-1-0075.

A CFAR Adaptive Subspace Detector for Second Order

  • Upload
    others

  • View
    7

  • Download
    0

Embed Size (px)

Citation preview

Page 1: A CFAR Adaptive Subspace Detector for Second Order

A CFAR Adaptive Subspace Detector for

Second Order Gaussian Signals

Yuanwei Jin and Benjamin Friedlander

Abstract

In this paper we study the problem of detecting subspace signals described by the Second Order Gaussian

(SOG) model in the presence of noise whose covariance structure and level are both unknown. Such a detection

problem is often called Gauss-Gauss problem in that both the signal and the noise are assumed to have Gaussian

distributions. We propose adaptive detectors for the SOG model signals based on a single observation and multiple

observations. With a single observation, the detector can be derived in a manner similar to that of the generalized

likelihood ratio test (GLRT), but the unknown covariance structure is replaced by sample covariance matrix based

on training data. The proposed detectors are constant false alarm rate (CFAR) detectors. As a comparison, we also

derive adaptive detectors for the First Order Gaussian (FOG) model based on multiple observations under the same

noise condition as for the SOG model.

With a single observation, the seemingly ad hoc CFAR detector for the SOG model is a true GLRT in that

it has the same form as the GLRT CFAR detector for the FOG model. We give an approximate closed form of

the probability of detection and false alarm in this case. Furthermore, we study the proposed CFAR detectors and

compute the performance curves.

Keywords

Adaptive detection, CFAR detector, Generalized Likelihood Ratio Test (GLRT), Second Order Gaussian Model,

First Order Gaussian Model

Y. Jin is with the University of California at Santa Cruz, CA 95060 USA (e-mail: [email protected]).B. Friedlander is with the University of California at Santa Cruz, CA 95060 USA (e-mail: [email protected]).This work was supported by the Office of Naval Research, under grant no. N00014-01-1-0075.

Page 2: A CFAR Adaptive Subspace Detector for Second Order

2

I. INTRODUCTION

Detecting subspace signals in the presence of noise is a common problem in multi-dimensional signal

processing. By a low rank subspace signal, we mean that each observation of the signal waveform can be

modelled as some linear combination of � basis vectors or “modes” where � is the subspace rank. When

we say that the signal � obeys the linear subspace model ������� , we are saying that the vector ��� �actually lies in a � -dimensional subspace of �� which we denote ����� . The subspace ����� is the range of

the transformation � . It is spanned by the columns of the matrix � . These columns comprise a basis for

the subspace, and the elements of ������������������� � � !#" are the coordinates of � with respect to this basis.

Depending on the statistical property of the coordinate vector � , we have different signal models.

For a large class of detection problems, the signal of interest is modelled as Second Order Gaussian

(SOG) model, of which the signal coordinate vector � has a Gaussian distribution, i.e.:

�%$'&��)(+*-,/.0. ! *1,�.0.��2 �435� (1)

Such a problem is usually called Gauss-Gauss problem in that both the signal and the noise are assumed

to have Gaussian distributions. One example of this type of model is the underwater acoustic source

generated by, for instance, ships or submarines. The acoustic signal is random in nature and is spread in

frequency and space.

For another type of detection problem, the signal of interest is modelled as the First Order Gaussian

(FOG) model, which means that the signal coordinate vector � is deterministic but unknown. This kind

of signal model typically arises in radar where the received radar echoes are described as unknown but

deterministic signals corrupted by noise (see e.g. [28], [31], [20], [29]).

For the FOG model, various forms of detection problem have been discussed in the statistics literature

and in the signal processing field. The commonly used generalized likelihood ratio (GLR) detector has

been extensively studied by many authors (see e.g. Trees[32], Kelly [19], Steinhardt [3], Burgess [5] and

Kirsteins [20]). Among others, Scharf [30], Kruat [22], and Friedlander [31] have investigated low rank

properties of the subspace signals and designed the non-adaptive and adaptive matched subspace filters.

In addition, the theory of invariance in hypothesis testing is exploited by many authors (see e.g. Scharf

[30], Bose and Steinhardt [3]) which leads to detectors that share a natural set of invariance with respect to

scaling, transformation and rotation. The detection problem of this type of model has a wide application

in active radar and sonar, space-time adaptive processing (STAP), mobile communication systems, and

many other multi-sensor or time series applications (see e.g. [6], [7], [12], [26], [31]).

DRAFT

Page 3: A CFAR Adaptive Subspace Detector for Second Order

3

Some radar applications of the SOG model have be addressed by several authors (see e.g. Raghavan

[26], [27] and Gini [10]). Nevertheless, unlike the well studied first order Gaussian models, the detection

problem for the second order Gaussian model does not seem to be well understood. Recently, a general

second order signal model is presented by McWhorter, et al (see e.g. [24], [25]). They study the matched

subspace detectors (MSDs) in presence of white noise based upon a single observation. They discover

that the GLRT for the detection of a FOG model signal is identical to the GLRT for the SOG model signal.

The fact that the MSD for the FOG model has the same form as that for SOG model does not occur

by coincidence. The difference between the FOG model and the SOG model lies in different statistical

descriptions of the unknown coordinate vector � in a known signal subspace. Intuitively, with a single

observation, there is insufficient information to estimate the unknown coordinate vector � , therefore it

is a natural result that the MSD detectors for the two models are identical. However the situation may

change when multiple observations are used. Detection of signals on the basis of multiple observations

is of interest in many applications, such as adaptive radar (see [8], [16]). In general, the detector based

on multiple observations is not a straightforward extension of that based on a single observation. Hence

further investigation on this issue is needed. In addition, prior work on non-adaptive subspace detec-

tors motivates us to study adaptive subspace detectors for the SOG model and the FOG model, and to

investigate the connections between the adaptive detectors for these two models.

We study in this paper the adaptive detectors of a subspace signal based upon a single and multiple

observations. We begin with the SOG signals. The detectors are derived based upon the GLRT principle

assuming that the covariance is known. After the test statistics are derived, the maximum likelihood

estimate of the covariance matrix based on the secondary data is inserted in place of the known covariance.

The resulting test statistic are CFAR detectors. In order to gain insights into the SOG detection problem,

we also derive adaptive detectors for the FOG model based upon multiple observations under the same

noise condition as for the SOG model. The proposed CFAR adaptive detectors for the SOG and the FOG

model signals based upon single and multiple observations, to best of our knowledge, appear to be new.

One important piece of work in this paper is that, for the SOG model, we are able to derive a closed form

expression of the approximate distribution function of the detection statistic for the false alarm rate ( 6 798 )

and the detection probability ( 6;: ). The form of approximation, verified by Monte Carlo simulation,

appears to have quite good accuracy. This approximate closed form simplifies the computation of the

detection threshold. In addition, the closed form reveals that the test statistic derived for the second order

Gaussian has a central <>= distribution.

DRAFT

Page 4: A CFAR Adaptive Subspace Detector for Second Order

4

By comparing the detectors for the SOG model and the FOG model, we observe that the derived CFAR

detectors for both models have the same form under a single observation assumption. In fact, the detector

for the FOG model on the basis of a single observation is proven to be a GLRT detector (see Kraut [21]).

This observation suggests that, in this case, the derived adaptive detector for SOG model is the true GLRT

detector although not proven mathematically. Our results extend the results on non-adaptive matched

subspace detectors (see e.g. [25]) to adaptive subspace detectors and expand the range of applications of

adaptive subspace detectors for the FOG model to that for the SOG model.

The remainder of paper is organized as follows. Section II introduces the subspace detection problem

for the SOG model and the FOG model. Section III computes the CFAR adaptive detector for the SOG

model and provides analytical and quantitative performance results. Section IV discusses the derived

detectors for the SOG and the FOG models and their connections. We conclude the paper in section V.

A. Notations�?� !#" transpose�?� !A@ Hermitian transposeB C �ED expected valueFHG C �ED traceIKJML�N CPO D diagonal matrix whose diagonal is the vectorOQ � Q

determinant of matrix �R � R vector (matrices) Frobenius normSUTidentity matrix of order V

II. FORMULATION OF THE DETECTION PROBLEM

LetCPOXW DY��Z� be a sequence of statistically independent, stationary, complex Gaussian distributed

random data vectors for [\� C^] *������_*-`aD . In an array processing application,OZW

typically represents a

snapshot (sample of the sensor outputs at time [ ) collected from an array of 6 sensors, and it is assumed

thatCPObW D is the superposition of the signal of interest and the noise. We define the random data matrixc �ed O �P* O ��*������f* Obgih �\ �j3 g as a concatenation of all available data. We consider the data

CPOZW D as

the primary data for detection.

The general signal model we consider in this paper is described as follows,OXW �k��� WmlonHW �p � (2)

where �q��;�j35� is the known signal subspace. � W �p9� is distributed as &r��s9*-,t.0. ! . ,/.0. is a positive

DRAFT

Page 5: A CFAR Adaptive Subspace Detector for Second Order

5

definite matrix of dimension �2uv� . In other words, the signal covariance matrix is given as

,/wx�y�/,/.0.K� @(3)

n�Wis the noise data vector and is distributed as

n;W $'&r��s9*�zH, !, where z denotes the noise level.

The CFAR requirement of a candidate algorithm ensures that the false alarm rate may be prescribed

at a given value independent of the correlation properties between the various noise components. In the

nomenclature of the nonadaptive detection literature, “CFAR” is with respect to noise level or variancez in the test data. In the adaptive detection literature (see, e.g.,[29]), “CFAR” is with respect to the

noise covariance matrix , , assumed to be uniform over test and training data. However, if we allow the

noise level to vary between training and test data with covariance matrix zb, and , , respectively, we

then mean “CFAR” with respect to both the shared noise covariance matrix structure , and independent

scaling z of the noise in the test data. This generalizes the meaning of “CFAR” in both the nonadaptive

and adaptive detection literature, where “CFAR” is respect to a shared covariance matrix or presumed

gain factor between test data and training data, respectively [22].

Our goal is to determine the existence of a signal in the received data matrixc

. Posing the problem as

a hypothesis test, we let the null hypothesis be that the data is signal free and the alternative hypothesis

be that the data contains a signal. We also assume that one has access to secondary data, or the “signal

free” dataCP{XT *|V}� ] *������f*�~ Q {9T $\&r��s9*-, ! D . Hence, the SOG detection problem we consider in the

paper can be formulated as

�>��� ���� OXW $'&r�)(+*�zb, ! *�[�� ] *������f*-`{9T $'&��)(+*-, ! * Ve� ] *������_*�~ (4)

and � � � �� � OXW $o&r�)(+*-,/w l zH, ! *�[>� ] *������_*-`{9T $'&r�)(+*-, ! * Ve� ] *������_*�~ (5)

Notice that z is the scale factor accounting for the power mismatch between the primary and secondary

data. The arbitrary scaling between the test data and training data is of significance in practical situation.z�� ]can be considered as an idealized condition. In fact, we hope that the false alarm rate is insensitive

to z when it deviates from unity [21].

As a comparison, we also study the detection problem for the FOG signals. Without knowing the

distribution of the unknown vector � W, we assume that this vector at each sample time [ is deterministic

DRAFT

Page 6: A CFAR Adaptive Subspace Detector for Second Order

6

but unknown. The detection problem for the FOG model is formulated as follows:

�>��� �� � OXW $'&r�)(+*�zb, ! *�[�� ] *������f*-`{ T $'&��)(+*-, ! * Ve� ] *������_*�~ (6)

and � � � ���� OXW $o&r���/� W *�zH, ! *�[>� ] *������_*-`{ T $'&r�)(+*-, ! * Ve� ] *������_*�~ (7)

In the next section, we will derive adaptive detectors for the SOG signals.

III. ADAPTIVE SUBSPACE DETECTOR FOR SOG MODEL WITH ` OBSERVATIONS

According to the Neyman-Pearson criterion, the optimum solution to the above hypothesis testing

problem (4) and (5) is the likelihood ratio test. However, for the case under consideration, it cannot be

employed since the total ignorance of the parameters

,�*-,�.0.X*�z (8)

is assumed. A possible way to cope with the aforementioned a priori uncertainty is to resort to the GLRT,

which is tantamount to replacing the unknown parameters by their maximum likelihood estimates under

each hypothesis [23]. In other words, the GLRT is to be derived from� � c *|� ! ��� L5�����m� �;� �^� ����� �_� c *|� !� L5� �E�x� ��� ��� � c *|� ! (9)

where� � � �_� c *|� !

is the joint densities under�/�

and� � .

Unfortunately, it has been well known that when both the interference and signal covariance are un-

known, the GLRT detector is intractable [24], [3], [26]. In order to circumvent this drawback we resort

to an ad hoc two-step design procedure: First we derive the GLRT detector assuming the covariance is

known. After the test statistic is derived, the maximum likelihood estimate of the covariance matrix based

upon the secondary data is inserted in place of the known covariance matrix. The resulting detectors have

the desirable CFAR property.

A. The Derivation of the CFAR Adaptive Detectors

Before we proceed, we define a pre-whitening filter ,��m�� , where ,��� ,��� is the Cholesky factorization

of , [13]. Then the pre-whitened measurement �%��,1�m�� O and the disturbance becomes white noise, �m�� n $�&���sX*�z S !. In reality, we will replace this filter by its estimate from the training data, by means

of either maximum likelihood estimate or some forms of reduced rank processing. Let us define several

DRAFT

Page 7: A CFAR Adaptive Subspace Detector for Second Order

7

notations before we begin the derivation of the GLRT detector. We use the notation �, w �k, �m�� , w , �m��to represent the whitened signal covariance matrix. Thus the detection problem can be formulated as the

following simple hypothesis test, �>��� � W $'&��)(+*�z S !� � � � W $'&��)(+* �,tw l z S ! (10)

The proposed CFAR subspace detectors are summarized in Table I. With `�� ]single data snapshot,

the GLRT is given as (see Appendix I.B for details)

� �)� ! � � @i ¢¡ �� @  �£¡ � (11)

where ¤q�¥, � �� � , ¢¡

is the projection operator on the subspace ¤ while  £¡ � S =  �¡

is its nulling

projector. This detector is also called generalized energy detector in that the detection statistic is basically

the ratio of signal power projected onto the signal subspace to the noise power projected onto its null

space. Taking into account the pre-whitening filter, we obtain the following test

� � O ! � O @ , �m��   �x¦ ��5§ , �m�� OO @ ,2�m�� � S =   � ¦ �� § ! ,2�m�� O (12)

With `©¨ ]data snapshots, the proposed CFAR detector is given as below (see Appendix I.C),

� �«ª ! �­¬ � @W  ¢¡ � W¬ � @W   £¡ � W (13)

Or taking into account the pre-whitening filter

� � c ! � ¬ O @W , � ��   � ¦ ��5§ , � �� OXW¬ O @W , � �� � S =   � ¦ �� § ! , � �� OXW (14)

It should be noted that this is not a GLRT detector in a strict sense. In fact, the GLRT when `®¨ � is

given by (see Appendix I.A) � �«ª ! � � FbG d ,/¯ h ! �� FbG d  °£¡ ,/¯ h ! � � � Q �, ¯ Q � � (15)

where ,/¯1� �g ¬ gW�± � � W � @W , and �,/¯Y�²�)¤ @ ¤ ! � �� ¤ @ ,/¯�¤a�)¤ @ ¤ ! � �� . When � � ], it is easy to

see that the detector (13) and (15) are equivalent. However, the proposed CFAR detector (13) generally

outperforms the GLRT (15). In fact, it can be shown that the proposed CFAR detector can be obtained

through a maximization based on a loosened condition (see Appendix I.C). We also see that for the SOG

signal of rank � , the GLRT detectors take different forms with different ` . This is because the detection

DRAFT

Page 8: A CFAR Adaptive Subspace Detector for Second Order

8

statistic shall depend on signal power distribution along each dimension for the SOG model. When there

are ` ¨ � data snapshots, the total signal power can be resolved onto each dimension of the signal

subspace, and the signal-to-noise powers along each dimension of the signal subspace are accounted

for. With only `³� ]data snapshot and without a priori knowledge of ,�.0. , resolving signal onto each

dimension of the signal subspace is intractable, hence only the total signal power projected onto the whole

signal subspace is accounted for.

TABLE I

CFAR SUBSPACE DETECTORS FOR SOG MODEL

Sample Size ´ is known ´ is unknownµ�¶1· ¸X¹»ºP¼½¶'¾?¿½À+Á+¾¾ ¿ À+ÂÁ ¾ ¸b¹»º�¼Ã¶ ¾«¿ÃÀ�ÄÁ ¾¾ ¿ ÀÅ ÄÁ ¾Eq.(11), (GLRT) (GLRT)µyÆ2· ¸X¹ÈÇ�¼K¶�É ¾«¿ÊÃÀÅÁ˾ ÊÉ ¾ ¿Ê ÀÅÂÁ ¾ Ê ¸X¹ÈÇ�¼Ã¶ É ¾ ¿Ê½À;ÄÁ ¾É ¾ ¿Ê ÀÅ ÄÁ ¾ Ê

Eq. (13)µyÆ�Ì ¸X¹ÈÇ�¼K¶ Í�Î�ÏÑÐ Ò½ÓÕÔ�ÖE×ÍØÎ�ÏÑÐ À ÂÁ Ò Ó Ô Ö ×^Ù�Ú½Û?Ü´ Ý Û ¦ � ¸b¹ÈÇü½¶ ÍØÎ�ÏÑÐ Ò½Ó#Ô�Ö�×ÍØÎ ÏÈÐ À  ÄÁ Ò Ó Ô Ö ×�Ù�Ú½ÛßÞ Ü´ Ý Û ¦ �Eq. (15), (GLRT)

If the noise covariance matrix were known, then we would use the detector described by (12) and (14).

In general, the covariance matrix is unknown and must be estimated by using adaptive techniques. In

this paper, we use an ad hoc procedure by substituting the unknown covariance matrix with its maximum

likelihood estimate based on the secondary data. The resulting detector when `à� ]is given as follows,

� � O *|� ! � O @âá � ��  �ã ¦ �� § á � �� OO @ á �m�� � S =   ã ¦ �� § !�á �m�� O (16)

whereá

is the maximum likelihood estimate of the noise covariance matrix from the secondary data, i.e.,

á � ]~äåT ± � {9Ti{ @T

(17)

and æ¤ç� á �m�� � , æ�,/¯t�è�éæ¤ @ æ¤ ! �m�� æ¤ @ ,/¯xæ¤a�éæ¤ @ æ¤ ! �m�� . Similarly, when ` ¨ ], the corresponding

detector is given as follows,

� � c *|� ! � ¬ O @W á �m��  �ã ¦ �� § á �m�� OXW¬ O @W á � �� � S =   ã ¦ �� § !�á � �� OXW (18)

So far, we have not specified the structure of the covariance matrix , . The adaptive detector uses the

covariance matrix estimate to calculate the adaptive weights and then filter the primary data. Additional

DRAFT

Page 9: A CFAR Adaptive Subspace Detector for Second Order

9

noise residue exists at the detector output becauseáëê��, . This residue degrades the detection perfor-

mance. It is well known that (see e.g. [28]) the number of data samples to obtain sufficient estimation

accuracy requires ~ ì�í�6 . Certainly, if , is a structured covariance matrix, for instance, ,­�¥,Yî lï � S ,where ,�î is a low rank matrix of rank ð , we shall use ML estimate which incorporates the structure of, (see e.g. [2],[20]) or other reduced rank processing schemes (see e.g. Goldstein [12], Haimovich

[15], Kirsteins [20], Gau [9], Guerci [14]). With a modification based on Anderson’s [1] early work, the

maximum likelihood estimate of the structured , for the case when the number of secondary samples~ ¨�ð can be obtained. We skip the derivation for brevity purpose. Let the eigen-decomposition ofá

beá �¥ñ IKJML�N d òË�P*������f*-ò � h ñ @. The ML estimate of , is given as follows,ó,ô�¥ñ IKJßL�N d òK�X�����_*-ò�õö * ó ï � *������_* óï � h ñ @

(19)

where the noise estimate is given as follows,

æï � � ]6k=ø÷ðùZú�û � ä � � �åü ± õö?ý � ò ü (20)

C ò � *�������ò õö D are the ÷ð largest eigenvalues ofá

, ÷ð is the estimate of the rank of interference, or it can also

be interpreted as the upper bound of the rank ð . ñ is the maximum likelihood estimate of the eigenvector

matrix of the sample covariance matrixá

. Certainly, a priori knowledge of the structure of , loosens

the sample size ~ requirement for the estimation of , . However, a thorough study of impact of , on

performance when , has a particular structure is out of the scope of this paper.

Equivalently, the CFAR detector (16) can also be written as

�;þ � O *|� ! � O @ÿá �m��  °ã ¦ ���§ á �m�� OO @ á � � O (21)

due to the fact that� þ � O *|� !

is a monotonic function of equation (16), or�;þ � O *|� ! � O @¢á � � �1��� @ÿá � � � ! � � � @ÿá � � OO @ á � � O (22)

We immediately recognize that this detector has the same form as the GLRT CFAR adaptive subspace

detector for the FOG model [21]. Hence we have a conjecture that this seemingly ad hoc CFAR detector

is a true GLRT.

B. Decision Thresholds

A closed form of the test’s probability of false alarm and detection is usually difficult to obtain. How-

ever, we find that when the signal subspace � is known, an approximate closed form of the 6m798 and 6 :DRAFT

Page 10: A CFAR Adaptive Subspace Detector for Second Order

10

is achievable for the proposed CFAR detectors (11) and (13) (see appendix II). This approximate closed

form reveals an insight as how the designed detector differs from the one designed for the FOG model

although the two bear the same structure. That is, the test statistic for the SOG model has a central <>=distribution while the test statistic for the first order Gaussian model has a non-central <>= distribution

[31]. We write < � ä � � ä � � to denote the central < -distribution with degrees of freedom of ��~k�P*�~ � ! , the

detection statistics under�/�

and� � are given as follows

� � c � �>� ! $ �6ø= � u < � � g � � � g � � � � �M� (23)

� � c � � � ! $ � ���9�� � � � u < ��� � � � � � (24)

where � � * � � *�� � * � � are defined in equation (100).

Equation (23) is a function of the signal subspace rank (� ), the total dimension ( 6 ) and the number of

observations ( ` ). It is independent of noise structure ( , ) and level ( z ). This clearly indicates that the

proposed detector (11) and (13) are constant false alarm rate (CFAR) detectors.

C. Performance Results

In this section, the computer simulation is conducted to verify the analytical result presented in the

previous sub-section. The simulation setting is chosen to be the underwater acoustic scenario such that

the change of the signal subspace rank is caused by a change of the signal spatial angular spread (see [17]

for details). Nevertheless this particular choice of simulation setup can be extended to other scenarios in

principle. We use a uniform linear array of 6 ���( sensors with half-wavelength unit apart. We consider

the detector performance when the number of training samples is sufficient. The choice ~ �yí�6 is made

since this condition provides a reasonable accuracy for estimation of the covariance matrix of noise , .

We will study the performance of the detector with ` � ]and ` ��í�( snapshots respectively, which

represents the cases of single snapshot and multiple snapshots detection.

In Fig. 1 - 3, the detection performance vs. the SNR value for different dimension of signal subspace

is depicted. We calculate the detection probability under ` � ]and `²� í�( snapshots for false alarm

rate of 6�7981� (�Ø( ] and 6�798Y�¥(�Ø(0( ] respectively. In these figures, the symbols denote the Monte Carlo

trial results while the lines denote the theoretical results. The three trial cases show that the theoretical

results match with the Monte Carlo results quite well. However, cautions should be taken when using this

approximation if a precise detection probability is required.

Fig. 1 and 2 show that for a single snapshot detection, there is a cross-over point on the receiver

DRAFT

Page 11: A CFAR Adaptive Subspace Detector for Second Order

11

operating characteristic (ROC) curves. It demonstrates that within a certain SNR range, the increase

of the dimension of subspace improves the detection performance. However, when multiple snapshots

are used for detection as is shown in Fig. 3, the increase of the rank of subspace reduces the detection

performance. The results are consistent with the results reported in [17] and [26]. The explanation is

as follows. The performance of a detector depends on both SNR and the shape of probability density

functions. The degrees of freedom (DOF) of the pdfs depend on the subspace signal rank (� ) and the

number of test data snapshots ( ` ), as is shown in equation (23) and (24). When the degrees of freedom

are small and with a certain SNR range, the increase of DOF improves the detection performance due to

a favorable change of the shape of probability density function. However, further increase of the DOF

brings down the detection performance, as is shown in Fig. 3 where large number of observations is used.

It is clear that the performance decreases when the signal subspace rank � increases.

It is interesting to see that the derived CFAR detector with � � ](or � ��b�� ! ) and ` � ]

becomes� � O ! � ����� � ¦ ��� ���f� � �� � � ¦ � � . This is the test statistic of the conventional minimum variance distortionless

response (MVDR) beamformer �ô�¥, � � �b�� ! for subspace signal with � � ], scaled by

O @ , � � O . Fig.

4 depicts the performance of the derived CFAR detector with `à� ] * � � ]and the conventional MVDR

beamformer. It is expected that as � increases, the performance of the conventional MVDR beamformer

degrades very quickly. This is due to an increasing loss of signal-to-noise ratio as the signal subspace

opens up and the MVDR fails to capture the signal energy with a single main beam.

In Fig. 5, we show that when ` ¨ � , the proposed CFAR detector outperforms the GLRT detector

under several testing scenarios. No claim to optimality is made for the GLRT. This is one example of a

technique that is superior.

Finally, we study the performance of the CFAR detectors to the mismatched signals when there exists

signal subspace rank ambiguity between the subspaces spanned by the signal components and the noise

components. We discuss Fig. 6-7, the detection curves under mismatched conditions. Here the true signal

subspace rank is � ��� , the estimated subspace rank � þ þ �ëd ] *��Å*��Å*��4*�� h . When � ¨ � þ þ , signal components

captured by the adaptive detector are lost; on the other hand, when ���\� þ þ, more noise components are

projected onto the signal subspace. The performance drop-off for the mismatched signals is evident in

both scenarios.

IV. DISCUSSIONS

In this section, we compare non-adaptive and adaptive detectors for the FOG model and the SOG

model based on `à� ]and `è¨ ]

observations. The detectors for FOG models are listed in Table II (see

DRAFT

Page 12: A CFAR Adaptive Subspace Detector for Second Order

12

−14 −12 −10 −8 −6 −4 −2 0 2 4 60

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

SNR [dB]

Det

ectio

n P

roba

bilit

y

p=5

p=3

p=1

Fig. 1. ��� vs. SNR for Gaussian random signals confined in subspace with !#"%$'&)(+*,(.-0/ dimension. �21435"687 6 &9(+:;"<& . The symbols square ( = ), cross ( > ) and star ( ? ) denote the Monte Carlo trial results for !#"<& ,!@"5* and !A"B- respectively, while the lines denote the corresponding analytical results.

−14 −12 −10 −8 −6 −4 −2 0 2 4 60

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

SNR [dB]

Det

ectio

n P

roba

bilit

y

p=3

p=5

p=1

Fig. 2. ��� vs. SNR for Gaussian random signals confined in subspace with !#"%$'&)(+*,(.-0/ dimension. �21435"687 6)6 &9(C:D"E& . The symbols square ( = ), cross ( > ) and star ( ? ) denote the Monte Carlo trial results for !F"�& ,!@"5* and !A"B- respectively, while the lines denote the corresponding analytical results.

Appendix III for details).

TABLE II

CFAR SUBSPACE DETECTORS FOR FOG MODEL SIGNALS

Sample Size ´ is known ´ is unknownµ�¶Y·GLRT, Eq.(25) GLRT, Eq.(26)µ¥Æp·GLRT, Eq.(117) Eq.(118)

DRAFT

Page 13: A CFAR Adaptive Subspace Detector for Second Order

13

−18 −17 −16 −15 −14 −13 −12 −11 −10 −9 −80

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

SNR [dB]

Det

ectio

n P

roba

bilit

y

p=1

p=3

p=5

Fig. 3. ��� vs. SNR for Gaussian random signals confined in subspace with !#"%$'&)(+*,(.-0/ dimension. �21435"687 6)6 &9(C:G"DH 6 . The symbols square ( = ), star ( ? ) and triangle ( I ) denote the Monte Carlo trial results for!@"J& , !A"B* and !A"5- respectively, while the lines denote the corresponding analytical results.

−14 −12 −10 −8 −6 −4 −2 0 2 4 60

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

SNR [dB]

Det

ectio

n P

roba

bilit

y

CFAR, p=1MVDR,p=1CFAR, p=3MVDR,p=3CFAR, p=5MVDR,p=5

Fig. 4. Performance of the CFAR detector and the MVDR beamformer detector. �2� vs. SNR for Gaussian random

signals confined in subspace with !A"K$L&9(+*8(C-M/ dimension. � 143 " 6,7 696 &9(C:%"J& .In [25], it has been shown that GLRT for the FOG model is identical to the GLRT for the SOG model

for the non-adaptive case ( , is known). In our notation, the GLRT detector has the form� � O ! � O @ , � � �1��� @ , � � � ! � � � @ , � � OO @ , � � O (25)

For the adaptive detector, it has been shown that, when `è� ], the adaptive subspace detector (ASD)

is sample-matrix version of the CFAR matched subspace detector (MSD) [22]. The resulting detector

takes the form as � � O *|� ! � O @ á � � �1��� @ á � � � ! � � � @ á � � OO @ á � � O (26)

DRAFT

Page 14: A CFAR Adaptive Subspace Detector for Second Order

14

−18 −16 −14 −12 −10 −8 −6 −4 −20

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

SNR [dB]

Det

ectio

n P

roba

bilit

yProposed CFAR, p=1Proposed CFAR, p=3Proposed CFAR, p=5GLRT, p=1GLRT, p=3GLRT, p=5

Fig. 5. Performance comparison of CFAR detector (13) and (14). �N143O" 687 6)6 &)(+:P"KH 6 . The rank ! is set to be!@"Q$'&)(+*,(.-0/ .

−14 −12 −10 −8 −6 −4 −2 0 2 4 60

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

SNR [dB]

Det

ectio

n P

roba

bilit

y

p"=1p"=3p"=5p"=7p"=9

Fig. 6. CFAR test with mismatch. �R� vs. SNR for Gaussian random signals confined in subspace with !S"K- .��143T" 6,7 696 &9(C:%"J& . The estimate of ! is given as !�U UV"J$L&9(C*,(C-,(.WX(+YM/This observation leads to a conjecture that CFAR detector (22) may be GLRT although it has not been

proven mathematically.

The GLRTs for the SOG and the FOG models when , is known take different form for ` ¨ ]. This

is understandable in that different statistical characteristics of the data for two models generally lead to

different detector structures.

V. CONCLUSION

We have proposed the adaptive subspace detectors for the Second Order and the First Order Gaussian

models based upon multiple observations. The proposed tests are CFAR tests and are invariant to arbitrary

DRAFT

Page 15: A CFAR Adaptive Subspace Detector for Second Order

15

−16 −15 −14 −13 −12 −11 −10 −9 −8 −7 −6 −50

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

SNR [dB]

Det

ectio

n P

roba

bilit

y

p"=1p"=3p"=5p"=7p"=9

Fig. 7. CFAR test with mismatch. �R� vs. SNR for Gaussian random signals confined in subspace with !S"K- .��143T" 6,7 696 &9(C:%"ZH 6 . The estimate of ! is given as ![U UV"K$L&9(C*,(.-X(.WX(+Y\/ .scaling of the training data and the test data. It is interesting that, with a single observation, the proposed

CFAR detector for the Second Order Gaussian model has the same form as the GLRT for the First Order

Gaussian model. This leads us to a conjecture, although not proven mathematically, that the proposed

CFAR detector are the true GLRT detectors. With multiple observations, the GLRT test is not optimal in

the Neyman-Pearson sense the the proposed CFAR test has a probability of detection that is higher than

that of the GLRT for the test scenarios.

There are several open questions that deserve further investigation. For instance, the detection per-

formance is affected by the estimation errors of the noise covariance. The two dominant factors in the

estimation errors are the number of secondary data samples and the knowledge of principal eigenvalues

and eigenvectors. The unavailability of sufficient number of secondary data for large aperture arrays and

the unknown effective rank of the noise covariance structure in practical situations have profound impact

on detection performance. Furthermore, the problem becomes more complicated when the noise level is

time varying. The understanding of the optimal or various suboptimal detector structures and their sensi-

tivities to different kinds of noise covariances is of great importance from both theoretical and practical

standpoint.

VI. ACKNOWLEDGEMENT

The authors would like to thank the anonymous reviewers for their helpful suggestions that consider-

ably improved the quality of this paper.

DRAFT

Page 16: A CFAR Adaptive Subspace Detector for Second Order

16

REFERENCES

[1] T.W. Anderson, An Introduction To Multivariate Statistical Analysis, Second Edition, 1971 [page 58]

[2] G. Bienvenu and L. Kopp, “Optimality of high resolution array processing using the eigen-system approach”, IEEE Trans-

actions On Acoustics, Speech, and Signal Processing, vol. ASSP-31, no.5, October 1983.

[3] S. Bose and A.O. Steinhardt, “A maximal invariant framework for adaptive detection with structured and unstructured

covariance matrices”. IEEE Transactions on Signal Processing, vol.43, (no.9), p.2164-75. Sept. 1995.

[4] G.E.P.Box, “Some Theorems on Quadratic Forms Applied in the Study of Analysis of Variance Problems, I. Effect of

Inequality of Variance in the One-Way Classification,” Annals of Mathematical Statistics, vol. 25, no. 2, pp. 290–302, June

1954.

[5] K.A. Burgess and B.D. Van Veen, “Subspace-based adaptive generalized likelihood ratio detection”, IEEE Transactions

on Signal Processing, vol. 44, no.4, April 1996.

[6] E. Conte, A. De Maio, and G. Ricci, “GLRT-based adaptive detection algorithms for range-spread targets”. IEEE Transac-

tions on Signal Processing, vol.49, (no.7), p.1336-48, July 2001.

[7] E. Conte, A. De Maio, and G. Ricci, “CFAR detection of distributed targets in non-Gaussian disturbance”, IEEE Transac-

tions on Aerospace and Electronic Systems, vol. 38, no. 2, April 2002.

[8] J.V. DiFranco and W.L. Rubin, Radar Detection, Prentice-Hall, Englewood Cliffs, N.J., 1968.

[9] Y. Gau and I.S. Reed, “An imporved reduced-rank CFAR space-time adaptive radar detection algorithm”, IEEE Transac-

tions on Signal Processing, vol.46, no. 8, July 1998.

[10] R. Gini and A. Farina, “Matched subspace CFAR detection of hovering helicopters”, IEEE Transactions on Aerospace and

Electronic Systems, vol. 35, no. 4, Oct. 1997.

[11] C.H. Gierull and B. Balaji, “Minimal sample support space-time adaptive processing with fast subspace techniques”, IEE

Proc.-Radar Sonar Navig., vol. 149, no. 5, October 2002.

[12] J.S. Goldstein and I. S. Reed, “Reduced rank adaptive filtering”, IEEE Transactions on Signal Processing, vol.45, no.2,

Feb. 1997, pp.492-496.

[13] G.H. Golub and C.F. Van Loan, Matrix Computations, 2nd ed. Baltimore, MA: Johns Hopkins Univ. Press, 1989.

[14] J.R. Guerci, J.S. Goldstein and I.S. Reed, “Optimal and adaptive reduced rank STAP”, IEEE Transactions on Aerospace

and Electronic Systems, vol.36, no.2, April 2000.

[15] A.M. Haimovich, “The eigencaceler” adaptive radar by eigenanalysis methods”, IEEE Transactions on Aerospace and

Electronic Systems, vol.32, April 1996, pp.532-542.

[16] C.W. Helstrom, Satistical Theory of Signal Detection, Second edition, Pergamon Press, 1968.

[17] Y. Jin and B. Friedlander, “Adaptive array processing for distributed sources”, IEEE Transactions on Signal Processing

(2004 to appear)

[18] T. Kailath, Linear System, Englewood Cliffs, NJ: Prentice-Hall,1980, Appendix, A.25, 658.

[19] E.J. Kelly, “An adaptive detection algorithm”, IEEE Transactions on Aerospace and Electronic Systems, vol. AES-22, no.

1, pp.115-127. March 1986

[20] I.P. Kirsteins and D.W. Tufts, “Adaptive detection using low rank approximation to a data matrix”, IEEE Transactions On

Aerospace and Electronic Systems, vol. 30, no. 1, January 1994.

DRAFT

Page 17: A CFAR Adaptive Subspace Detector for Second Order

17

[21] S. Kraut and L.L. Scharf, “The CFAR adaptive subspace detector is a scale-invariant GLRT”, IEEE Transactions on Signal

Processing, vol. 47, no. 9, September 1999.

[22] S. Kraut, L.L. Scharf and L.T. McWhorter, “Adaptive Subspace Detectors”, IEEE Transactions on Signal Processing, vol.

49, no. 1, January 2001.

[23] E. Lehmann, Testing Statistical Hypotheses. New York: Wiley-Interscience, 2nd edition 1986, pp.284-286

[24] L.T. McWhorter and M. Clark, “Matched subspace detectors and classifiers”, Mission Research Technical Report:

MRC/MRY-R-073, August 2001.

[25] L.T. McWhorter and L.L. Scharf, “Matched subspace detectors for stochastic signals”, September 2001 (unpublished

manuscript)

[26] R.S. Raghavan, N. Pulsone and D.J. McLaughlin “Performance of the GLRT for adaptive vector subspace detection”. IEEE

Transactions on Aerospace and Electronic Systems, vol.32, (no.4), p.1473-87. Oct. 1996.

[27] R.S. Raghavan, H.E. Qiu and D.J. McLaughlin, “CFAR detection in clutter with unknown correlation properties”, IEEE

Transactions on Aerospace and Electronic Systems, vol.31, no.2, April 1995.

[28] I. Reed, J. Mallet and L. Brennan, “Rapid convergence rate in adaptive arrays”, IEEE Transactions on Aerospace and

Electronic Systems, AES-10, Nov. 1974.

[29] F. Robey, D.R. Fuhrmann, E.J. Kelly and R. Nitzberg, “A CFAR adaptive matched filter detector”, IEEE Transactions on

Aerospace and Electronic Systems, vol. 28, no.1 pp.208-216. Jan. 1992.

[30] L.L. Scharf, Statistical Signal Processing, Addison-Wesley, 1991.

[31] L.L. Scharf and B. Friedlander, “Matched Subspace Detection,” IEEE Transactions on Signal Processing, vol. 42, no. 8,

pp. 2146–2157, August 1994.

[32] H.L. Van Trees, Part I: Detection, Estimation , and Modulation Theory, New York: Wiley, 1968.

APPENDIX

I. DERIVATION OF GLRT WHEN , IS KNOWN FOR SOG MODELS

In this appendix, The GLRT detector when , is given is derived. Let ¤��k, �m�� � and � W �k, �m�� OXWdenote the pre-whitened signal subspace and test data respectively. We derive the GLRT test for the

following binary hypothesis test:

� � � � W $'&r�)(+*�z S ! *�[>� ] *������_*-` (27)

and

� � � � W $'&��)(+*-¤�,�.0.½¤ @ l z S ! *�[�� ] *������f*-` (28)

The GLRT test takes the form of

� �«ª ! ��� L5�����j� �^� ���^� �_�«ª !� L5�Ã�E���^� � �«ª ! (29)

DRAFT

Page 18: A CFAR Adaptive Subspace Detector for Second Order

18

where ªa�ëd �Ë�_*������P*-� gihis the whitened data matrix. Next, we would follow McWhorter and Clark’s [24]

derivation to derive the GLRT test of this hypothesis problem. To proceed, we present the signal model

in a different form. Let �,�.0.��ë�)¤ @ ¤ ! �� ,�.0.X�)¤ @ ¤ ! �� ��]#^_] @(30)

where ] is the unitary matrix, and ^k� I½JßL�N da`^�P*�`_��*������_*�` � h is the ��uv� diagonal matrix, and

�¤ �¥¤o�)¤ @ ¤ ! � �� ]��ëd �b �P*������_* �b � h (31)

It is straight forward to see that the columns of �¤ are a set of orthonormal vectors. Furthermore, let�¤ £denote the null space of �¤ of dimension �Yu �)6 = � ! , and it is assumed that each column of �¤ £

is

orthonormal. Thus we have

�¤ £ @ d �¤ �¤ £ h �ëd�s ��c �Vd � 3 d S �ec �Vd � 3 �ec �Vd � h (32)

The covariance matrix ¤�,t.0.ˤ @ l z Scan then be written as

z Sjl ¤2,/.0.ˤ @ � d �¤ �¤ £ h fg z S l ^ ss z Shi d �¤ �¤ £ h @

(33)

It follows that j z Sjl ¤�,�.0.ˤ @lk � � � d �¤ �¤ £ h fg ��z S l ^ ! � � ss �� Shi d �¤ �¤ £ h @

� �¤a��z Sjl ^ ! � � �¤ @ l �� �¤ £ �¤ £ @� ¬ � î ± � �� ýRm+n  poq n l �� � S =  ¢¡ !

(34)

where   o¡ �  ¢¡ �¥¤a�)¤ @ ¤ ! � � ¤ @  oq n � �b î �b @î (35)

After decomposing ,t.0. into ^ and �¤ , we then obtain an equivalent GLRT as follows,

� �«ª ! � � L5� ��rb� o¡ � ��� � �P�«ª !� L5� �E��� � � �«ª ! (36)

Under hypothesis���

, the likelihood function is given as� � �«ª ! � s gW�± � � � �)� W !� s gW�± �ut � � Q z S Q � �wv �[x C = �� � @W � W D� t � g �zy ��N{ g � v �[x C = �� ¬ gW�± � � @W � W D (37)

DRAFT

Page 19: A CFAR Adaptive Subspace Detector for Second Order

19

or its log-likelihood function as| � �«ª ! �ë= 6¢`~}�� t =%`16�}��ËzY= ]zgåW�± � � W � @W (38)

It is straightforward to calculate the maximum likelihood estimate of z ,

æz1� ¬ gW�± � � @W � W`16 (39)

The compressed likelihood function under���

is then

� � �«ª � æz ! ��� ]tl� � g � ` � g � �)`Y6 ! g �Q� ]¬ gW�± � � @W � W�� g �

(40)

The likelihood function under� � can be written as� �P�«ª ! � s gW�± � � �_�)� W !� s gW�± � t � � Q z Sjl ¤2,/.0.ˤ @ Q � � v �[x C = � @W ��z S l ¤�,�.0.ˤ @ ! � � � W D� y �� { � g y ��N{ � � � � � g s � î ± �l� �� ýRm+nw� gv �[x�� = �� ¬ gWU± � � @W � S =  ¢¡ ! � W = ¬ gW�± � ¬ � î ± ��� �� c��� n � �� ýRmCn�� (41)

and the log-likelihood as| � �«ª ! � = 6¢`~}�� t =%`\�)6ø= � ! }��Kz�= ` ¬ � î ± � }��X��z l `Pî != �� ¬ gW�± � � @W � S =  ¢¡ ! � W = ¬ � î ± � ¬ gWU± ��� �� c��� n � �� ýRmCn (42)

The maximum likelihood estimate ofC ` î D is obtained by maximizing this function. Toward that end,� | �f�«ª !� ` î �r= ` ]z l ` î l � ]z l ` î � � gåW�± � � @W  poq n � W � ( (43)

This implies that

z l `�îb� ]`gåW�± � � @W  poq n � W (44)

Plugging (44) into (42), we obtain that| �_�«ª � æ^ ! � = 6¢`~}�� t =%`\�)6ø= � ! }��KzY=%` ¬ � î ± � }�� �g ¬ gWU± � � @W   oq n � W= �� ¬ gW�± � � @W � S =  ¢¡ ! � W =�` � (45)

or equivalently, | �f�«ª � æ^ ! � = 6¢`~}�� t =%`16�}��+zY=%` ¬ � î ± � }�� ���� ��.� � � �� c��� n � ��= �� ¬ gW�± � � @W � S =  ¢¡ ! � W =%` � (46)

DRAFT

Page 20: A CFAR Adaptive Subspace Detector for Second Order

20

Let � î denote � î � �g ¬ gW�± � � @W   oq nÕ� Wz (47)

If we call z an artificial noise, then it is reasonable to call � î the signal-to-noise ratio resolved onto the

one-dimensional subspace � �b î�� , and�� �å î ± � � î � �g ¬ gW�± � � @W  �¡ � Wz (48)

is the signal-to-noise ratio resolved onto the � = dimensional subspace �)¤1� .Maximizing

| �f�«ª � æ^ !over z leads to� | �f�«ª !� z �ë= `\�)6k= � !z l ]z �

gåW�± � � @W   £¡ � W �¥( (49)

or

æz1� ]`'�)6ø= � !gåW�± � � @W � S =  ¢¡ ! � W (50)

Hence, the compressed likelihood function becomes| �_�«ª � æ^v* æz ! �r= 6¢`~}�� t = `Y6@}�� ¬ gW�± � � @W   £¡ � W`\�)6k= � ! =%` �å î ± � }�� æ� î =%`16 (51)

where æ� î denote

æ� î � �)6ø= � ! ¬ gW�± � � @W   oq n«� W¬ gW�± � � @W   £¡ � W (52)

and æ�

� �å î ± � æ� î � �)6k= � ! ¬ gW�± � � @W  ¢¡ � W¬ gW�± � � @W   £¡ � W (53)

Equation (48) and (53) are equation (52) and (48) with z being replaced by its maximum likelihood

estimate in (50) respectively.

Up to this point, the remaining question is how to maximize the term = ¬ � î ± � }�� æ� î or equivalentlys � î ± � ����n . We consider the following three cases: (1) large number of data snapshots `ç¨ � , (2) single

snapshot `à� ]and (3) a postulated solution for `©¨ ]

.

A. Large data record `è¨ �In this case, large number of test data samples are used. Let ,�¯o� �g ¬ gW�± � � W � @W . Notice that,

equation (52) can be further written as follows,

DRAFT

Page 21: A CFAR Adaptive Subspace Detector for Second Order

21

æ� î � � � � � ��X�¡  cN¢£ � ÝC¤ �b @î ,/¯ �b î� � � � � ��X�¡  c ¢£ � ÝC¤+¥ @î �)¤ @ ¤ ! � �� ¤ @ ,/¯�¤a�)¤ @ ¤ ! � �� ¥ î� � � � � ��X�¡  cN¢£ � Ý ¤+¥ @î �, ¯ ¥ î (54)

where �,/¯ �ë�)¤ @ ¤ ! � �� ¤ @ ,/¯�¤a�)¤ @ ¤ ! � �� (55)

When ` ¨ � , �, ¯ will be a full rank matrix of size �aut� . An eigen-decomposition of �, ¯ yields the

following, �,�¯i�yñ IKJßL�N d �¦ ��*������_* �¦ � h ñ @(56)

Asymptotically ( `¨§ª© )

�,/¯«§ �)¤ @ ¤ ! �m�� ¤ @ �)¤ @ ,/.0.ˤ @ l z S ! ¤o�)¤ @ ¤ ! �m�� �E]%�¬^ l z S ! ] @(57)

Hence, ñY�E]1* �¦ î ��` î l z;* as `�§ª© (58)

Assuming �¦ �iìy�����P*Pì �¦ � , it is straightforward to see that,

�¦ ��­ ¥ @î �,/¯ ¥ î ­ �¦ � (59)

Due to the constraint that ]��ëd ¥ �P*������f* ¥ � h is a set of orthogonal basis, we conclude that

� L5� �®î ± �]æ� î � �®î ± �

FHG d   £¡ , ¯ h�)6ø= � ! �¦ î (60)

Hence the compressed the likelihood function has the form of

� � �«ª � æz * æ^ * æ�¤ ! � � ]t � g � ` � g � ` g �J� 6ø= �¬ gW�± � � @W   £¡ � W � g � � �®î ± �

FHG d   £¡ ,�¯ h�)6ø= � ! �¦ î ! g (61)

Hence the GLRT detection statistic is given as� �«ª ! � ¯ � �e°²± ��Å� �rb� �o¡ �¯+³ �´°²± ����� � � �� ! � �)6ø= � l æ�! � s � î ± � �X�¡  cN¢£ � Ý ¤� � � � � oµ n � g� � � þ �«ª ! � g (62)

DRAFT

Page 22: A CFAR Adaptive Subspace Detector for Second Order

22

where � þ �«ª ! � � �� ! �m�)6k= � l æ�! � s � î ± � �X�C  cN¢£ � ÝC¤� � � � � oµ n� � � � �� ! � � �X�+  � Ý.¤�X�+  c ¢£ � Ý.¤ � � � �X�C  cN¢£ � Ý.¤ ��¶· o� Ý ·� � � � �� ! � � �X�+  � Ý ¤ ��¸� �X�+  cN¢£ � Ý ¤ � ¸ ¦ ¶ Q �, ¯ Q � � (63)

Notice that� �«ª !

is a monotonic function of� þ �«ª !

. Further simplification of� þ �«ª !

by ignoring the

constant term yields � �«ª ! � � FbG d ,/¯ h ! �� FbG d   £¡ ,/¯ h ! � � � Q �, ¯ Q � � (64)

It is easy to see that this is a CFAR detector. Taking into account the pre-whitening filter, we obtain the

following test

� � c ! � � FbG d , � � ,º¹ h ! �� FHG d  �£� ¦ �� § , � �� , ¹ , � �� h ! � � � Q � @ , � � , ¹ , � � �Y��� @ , � � � ! � � Q � � (65)

B. `}� ]It should be noted that with large number of data sample, we are able to estimate the signal subspace

matrix ,/.0. , in other words, the energy distribution within the signal subspace is tractable. However,

with a single data snapshot � , we are no longer able to do that. Interestingly, in this case the GLRT takes

another form.

To proceed, based on equation (44) we notice that

æ` î � � L5� �)(+*-� @   o� n ��=az !(66)

which leads to the following � î � � @  �o� n �z � ] l æ`Pîz � ] l æ`�îz ì ](67)

and�

� �å î ± � � î � � @   ¡ �z ì � (68)

Here � î and

�should not be confused with these defined in (47) and (48). Similarly

æ� î � �)6k= � ! � @   o� n �� @   £¡ � ì ](69)

and æ�

� �å î ± � æ� î � �)6ø= � ! � @  �¡ �� @   £¡ � ì � (70)

DRAFT

Page 23: A CFAR Adaptive Subspace Detector for Second Order

23

Equation (67) and (68) imply that one can rotate each individual �b î such that equation (67) is always met

while the sum constraint simply implies that the signal-to-noise ratios in the individual subspaces much

account for all of the signal-to-noise ratio in the overall subspace.

Let| �f�)� � æ^v* æz !

denote equation (51) with `�� ]. Maximization of

| �f�)� � æ^ * æz !over �¤ is equivalent to

maximizing s � î ± � ����n with the constraint of (70), we obtain

� L5� �®î ± �]æ� î � � J � �®î ± � æ� î � æ

�=�� � = ] !

(71)

The equality occurs where � = ]of æ� î is equal to

]and the remaining æ� î is æ

�=\� � = ] !

. This constrained

maximization can be obtained by the simple although tedious inductive reasoning. The proof is provided

as follows.

Theorem 1: For real numbersC � î ì ] *�»i� ] *������ � D , with a constant constraint ¬ � î ± � � î �

�� ì � ,

then

� J � �®î ± � � î ��� =�� � = ] !

(72)

The equality occurs where � = ]of � î is equal to

]and the remaining � î is

�� =�� � = ] !

.

Proof: We conduct the proof by utilizing the inductive reasoning. For � �eí , we have the � �pì ],� �i�

��m= � �iì ]

, thus equation� � � �i� � �f� � � = � � ! �r=�� � �j=��í ! � l

���¼ (73)

is a parabolic with the maximum at � ���¾½ �� ì ]. Since

] ­ � � ­ ���= ]

. Hence we know that

the minimum value occurs at either � � � ]or � � �

��â= ]

. In other words, � �����ÿ= ]

or � ��� ]respectively. Thus we prove this case. Next we assume that the argument is true for � ��V , then for� � V ly]

, we have

� J � T ý �®î ± � � î � � J �9� T®î ± � � î ! � T ý � � � J �X� T®î ± � � î ! ��T ý � =

�T ! � �

�T =�V ly] ! �

�T ý �x=

�T !

(74)

The above equation means that given

�T ý � , we like to find

�T

such that the product of all the � î is

minimized.

��T =�V l¥] ! �

�T ý � =

�T ! �r= � � T =

�T ý � l V­= ]í � � l �

�T ý �j=%V l¥]í � �

(75)

The above function is a parabolic function with its maximum at ½R¿ÁÀ � ý T � �� ì\V . To obtain the minimum,

we only need to check the following numbers

�T �­V or

�T �

�T ý � = ]

. In fact,

�T ��V and

DRAFT

Page 24: A CFAR Adaptive Subspace Detector for Second Order

24�T �

�T ý � = ]

produce the same function value. When

�T � V , we have� � �������Å� � T � � � � T � ] * � T ý �x�

�T ý � =�V (76)

or when

�T �

�T ý � = ]

, we have� � �r�����Å� � T � � � ] * � T ��T ý � =%V�* � T ý � � ]

(77)

Both sequences satisfy the conditions set by the theorem. We thus complete the proof.

This result gives rise to the maximum likelihood estimator of �¤ . Geometrically, the subspace � �¤Y� is

rotated so that the all the signal energy is resolved onto one-dimensional subspace �b � while the artificial

noise power is resolved evenly onto subspace �)¤1� .The remaining question is whether a corresponding choice of �¤ exists and why should the maximum

likelihood estimator of the subspace �¤ work like this? The answer is that with a single observation, the

total energy � @  ¢¡ � may not be resolved into the signal subspace along each dimension explicitly. Thus

the maximum likelihood estimation places all the signal energy and one unit of noise in one dimension� �b � , and the remaining � = ]units of noise power in the remaining � = ]

coordinates based on theorem]

(see also [25]). This is different than the case where `©¨ � where the total signal energy can be resolved

along each dimension of the signal subspace.

Hence the compressed likelihood function takes the form as� �_�)� � æz * æ^ * æ�¤ ! � � ]tÂ� � ` � � � 6k= �� @   £¡ � � � � æ�

= � ly] ! � � (78)

The GLRT detection statistic is then given as follows,� �)� ! � ¯ � � � ± ��+� �rb� �o¡ �¯+³ � � ± ��^�� � �� ! � �)6k= � l æ�! � ��½H� � ý � (79)

Furthermore, taking the derivative of log-function of� �)� ! leads toÃMÄ ûMÅ � � �à �½ � Ãà �½ C =i6�}��Ë6 l 6@}��X�)6k= � l æ

�! =B}��X� æ

�= � l¥] ! D� 6 �� � � ý �½ = ��½H� � ý �� � æ

�= � ! � � �� � � � ý �½ ��� �½�� � ý � �ì (

(80)

The last line is due to the fact that æ�

ì � . Thus function (79) is a monotonic function of æ�

, an equivalent

detection statistic is given below � þ �)� ! � ]6k= � æ�

� � @i ¢¡ �� @  �£¡ � (81)

DRAFT

Page 25: A CFAR Adaptive Subspace Detector for Second Order

25

Taking into account the pre-whitening filter, we obtain the detector

� � O ! � O @ , � ��   � ¦ �� § , � �� OO @ , �m�� � S =   � ¦ �� § ! , �m�� O (82)

or equivalently � � O ! � O @ , � � �1��� @ , � � � ! � � � @ , � � OO @ , � � O (83)

C. A postulated CFAR detector for ` observations

A postulated CFAR detector utilizing ` observations can be described as follows

� � c ! � ¬ gW�± � O @W , �m��   � ¦ �� § , �m�� OXW¬ gWU± � O @W ,p�m�� � S =   � ¦ �� § ! ,p�m�� OXW (84)

In fact, this detector can be obtained if we are allowed to loosen the condition (52) to be æ� î ì ], (53) to beæ

�ì � and to utilize theorem

]. Consequently, the maximization results in a greater likelihood function

value in equation (51) than that given in equation (61). Not surprisingly, it turns out that this detector has

better performance than the GLRT (64) proposed earlier.

II. CALCULATION OF PROBABILITY OF FALSE ALARM AND DETECTION

In this appendix, we derive the probability of false alarm and detection for the detector (84) assuming

that � is known. As we can see, the detection statistic is the ratio of two nonnegative quadratic forms.

In [17], we have shown that the approximation to the distribution of a positive quadratic form by a

scaled Chi-squared distribution is fairly satisfactory. We then attempt to approximate the ratio of two

independent quadratic forms by Chi-squared distributions in the numerator and the denominator. Cautions

should be taken regarding to the mutual independence of the numerator and the denominator. After the

interference nulling filter, the received signal is separated into two subspaces, the signal subspace and its

null space. It is assumed that the target signal is independent of noise, therefore the signal component and

the residue noise component projected onto the signal subspace are independent of the noise component

in the null space. The numerical simulations (see e.g. Fig. 1-3) show that the accuracy of the simple

approximation is quite good. This method may be usefully employed to supplement the accurate (but less

suggestive) exact methods.

In what follows, we write Æ � ä to denote the central Chi-squared distribution with ~ degrees of free-

dom. Similarly, we write < � ä � � ä � � to denote the central < -distribution with degrees of freedom of

DRAFT

Page 26: A CFAR Adaptive Subspace Detector for Second Order

26

��~��P*�~ � ! , and � 7XÇ �.È Ç � �ÊÉ ! to be the probability density function (PDF) of a central F-distribution with

degrees of freedom �ÊË+�P*�Ë0� ! .The false alarm rate is computed as follows,

6ÍÌÏÎ1�EÐ G C_� � c � �>� ! ¨JѽD (85)

Let � W �¥, � �� OXW , then under���

, � W $'&r�)(+*�z S !, we have

� �«ª � � � ! � ¬ � @W , �� , �m��   � ¦ �� § , �m�� , �� � W¬ � @W ,��� ,2�m��  �£� ¦ �� § ,��� ,2�m�� � W � ¬ � @W   � ¦ �� § � W� @W   £� ¦ �� § � W (86)

The rank of the projection matrix  �x¦ �� § is � . It is shown in [30] that the quadratic form � @W   �x¦ �� § � W

is central Chi-squared distributed with í � degrees of freedom if and only if  � ¦ �� § is idempotent,

i.e.,  � ¦ �� § �   �� ¦ �� § . In our case

  � ¦ �� § is a projection matrix, so this condition is satisfied and� @W   � ¦ �� § � W $ÒÆ �� � . Obviously � @W   £� ¦ ���§ � W $ÓÆ �� � � � � � . Taking the ` independent and identically

distributed snapshots into account, the detection statistic under�2�

has the following distribution

� � c � �>� ! $ Æ �� � gÆ �� � � � � � g � �6k= � u < � � � g � � � � � � � g �(87)

while the probability density function under�t�

is � � �� � 7 � � � � � � � � � � �� É ! by the Jacobian transformation,

where the density function

� 7 Í ¶ È ¸ ¦ ¶ Ö �ÊÉ ! � ñ �)6_Ô�í !ñ � � Ô�í ! ñ d»�)6�= � ! Ô�í h � �6 = � ! �9Õ � É �9Õ �d ] l �� � � É h �VÕ � (88)

Similarly the detection probability is computed as follows

6ÍÖ%�EÐ G C_� � c ! ¨JѽDâ��Ð G �� � ¬ O @W , � ��   � ¦ �� § , � �� OXW¬ O @W , �m��   £� ¦ �� § , �m�� OXW ¨KÑØ×aÙÚ (89)

The PDF of detection probability is more complicated. We will employ the following theorem which is

an straightforward extension of Theorem 2.1 of [4]. The theorem is given as follows,

Theorem 2. If Û denotes a column vector of � random variables �é�P*-����*������_*-� � having expectation zero

and distributed in a complex multi-normal distribution with 6 u 6 variance-covariance matrix Ü , and ifÝ �¥� @_Þ � in any real quadratic form of rank ß ­ 6 , thenÝ

is distributed like a quantityà � áåü ± ��â ü Æ �� (90)

DRAFT

Page 27: A CFAR Adaptive Subspace Detector for Second Order

27

where each Æ �� variate is distributed independently of every other and â ’s are the ß real non-zero eigen-

values of the matrix ]e�EÜ Þ .

Proof: This theorem is a straightforward extension of Theorem 2.1 of [4] from real numbers to complex

numbers. end

Based on this theorem, we know that the numerator of� � c � � � ! is distributed as a weighted sum of a

chi-squared distribution, i.e., å O @W , � ��   �x¦ �� § , � �� OXW $ �åü ± �4ã ü Æ �� g (91)

where the ã ü are the eigenvalues of the matrix �),pw l zb, ! ,2�m��   � ¦ �� § ,p�m�� . Let¦ �¬ä !

denote the

eigenvalues of the matrix ä , and notice that¦ �|�), w l zH, ! , � ��   � ¦ �� § , � �� ! � ¦ �), � �� �), w l zb, ! , � ��   � ¦ �� § !� ¦ �|��z S l , � �� , w , � �� !   � ¦ �� § !� ¦ �|��z S l ,/w�, � � !   � ¦ �� § ! (92)

In the above, we use the fact that, given two 6 u 6 matrices � and ä , the eigenvalues of matrix �Tä are

identical to those of matrix ä� (see e.g. [18]). Therefore, the eigenvalues of , �m�� , w , �m�� are identical

to those of , � � , w (which are also equal to the eigenvalues of the matrix , w , � � ).It is worth noticing that the eigenvalues of , w , � � are essential for the Gauss-Gauss problem we are

studying here. Given a signal vectorO w $\&r�)(+*-, w ! , the random variable � O @w , � � O w represents the

signal-to-interference-plus-noise ratio. Hence

EC åKD¢� E

CPO @w , � � O w D�� FHG d , w , � � h (93)

Specifically ifC ¦ î � »i� ] *������ � D denote the eigenvalues of the matrix , w , � � , then E

C åKD�� ¬ � î ± � ¦ î is

the average signal to interference plus noise (SINR) confined within the signal subspace.

Carrying out the similar operations on denominator, we haveå O @W , � ��   £� ¦ �� § , � �� OXW $� � � � �åü ± �zâ ü Æ �� g (94)

The eigenvalue set ¦ �|�), w l zb, ! , �m��   £� ¦ �� § , �m�� ! � C â ��* â ��*M´´»* â � � � D (95)

Next we notice that a weighted chi-squared distribution can be approximated by a scaled chi-squared

distributed given by the following theorem [4].

DRAFT

Page 28: A CFAR Adaptive Subspace Detector for Second Order

28

Theorem 3 æ �¥� @ Þ �°� áåü ± � ¦ ü Æ �ç�è (96)

is distributed approximately as � Æ �� where� � ¬ Ë ü ¦ �ü¬ Ë ü ¦ ü * L � I �/� � ¬ Ë ü ¦ ü ! �¬ Ë ü ¦ �ü (97)

Based on the above theorem, we haveå O @W , �m��   � ¦ �� § , �m�� OXW $ � � Æ �� � (98)

and å O @W , �m��   £�x¦ �� § , �m�� OXW $ � �0Æ �� � (99)

where � � � � ¶è � �9é �è� ¶è � �9é è �é�ç� í � � ¶è � �9é è � �� ¶è � � é �è� � � � ¸ ¦ ¶è � �ëê �è� ¸ ¦ ¶è � ��ê è �K� � í � � ¸ ¦ ¶è � �ìê è � �� ¸ ¦ ¶è � ��ê �è (100)

Hence the detection statistic under� � has the following approximate distribution

� � O � � � ! $ � �wÆ �� �� �0Æ �� � � � ���é�� �\�K� u < �e� � � � � � (101)

while the probability density function takes the form� � � �� � � � � 7,í �.È í � � � � � �� � � � É ! by Jacobian transformation.

Hence, the probability of detection and false alarm can be approximated as follows,

6 Ì�ÎBîïKðñ � 7 Í ¶ È ¸ ¦ ¶ Ö �¬Û ! òòÛ (102)

6ÍÖ îï ðó ³ó � ñ � 7 Í í �CÈ í � Ö �¬Û ! ò8Û (103)

where ã � � �� � � , and ã � � � � � �� � � �III. ADAPTIVE DETECTORS FOR FIRST ORDER MODEL WITH ` OBSERVATIONS

The detection problem is formulated in equation (6) and (7). In general, the GLRT detector is to be

derived from � � c *|� ! � � L5� ���x� . � � ��� � � � c *|� !� L5� �E�x� ��� � � � c *|� ! (104)

where� � � �_� c *|� !

is the joint densities under�/�

and� � respectively for the above detection problem.

DRAFT

Page 29: A CFAR Adaptive Subspace Detector for Second Order

29

However, we soon find out that the GLRT for general ` is intractable except for the case ` � ].

Hence, we utilize an ad hoc approach assuming , is known. Pre-filtering , � �� on dataObW

and subspace� , we have the following � W �y, � �� OXW and ¤ �¥, � �� � , thus the detection problem becomes�>��� � W $'&r�)(+*�z S ! *�[>� ] *������_*-` (105)

and � � � � W $\&��)¤�� W *�z S ! *�[�� ] *������f*-` (106)

Under� � , the joint density is given as� � �«ª ! � s gWU± � � � �)� W !� s gWU± � �� ¸ � ¸ v ��x C = �� �)� W = ¤2� W ! @ �)� W = ¤2� W ! D� y �� �N{ � g v ��x C = �� ¬ W �)� W =%¤�� W ! @ �)� W = ¤2� W ! D (107)

The maximum likelihood estimate of z is given as

æz�� 6¢`¬ W �)� W =%¤2� W ! @ �)� W =%¤2� W ! (108)

Hence the compressed density function under� � is given� �_�«ª � æz ! � ]��` t ! � g � 6¢`¬ W �)� W =%¤2� W ! @ �)� W =%¤�� W ! � � g

(109)

It is easy to verify that the density under�t�

is� � �«ª � æz ! � � �_�«ª � æz !�Q . � ± � � ]��` t ! � g � 6¢`¬ W � @W � W � � g(110)

The intermediate GLRT is then� �«ª ! � � �_�«ª � æz !��� �«ª � æz ! � � ¬ W � @W � W¬ W �)� W =%¤�� W ! @ �)� W =%¤�� W ! � � g(111)

or equivalently � �«ªm* æz ! � ¬ W � @W � W¬ W �)� W = ¤2� W ! @ �)� W = ¤2� W ! (112)

Finally maximizing� �«ªm* æz !

over the unknown vector � Wis equivalent to minimizing the quadratic

form �)� W =%¤2� W ! @ �)� W =%¤�� W !, yielding

�)� W =%¤�� W ! @ �)� W =%¤�� W ! � R �)¤ @ ¤ ! �� d»�)¤ @ ¤ ! � � ¤ @ � W = � W_h R � l � @W   £¡ � W (113)

Hence, æ� W �ë�)¤ @ ¤ ! � � ¤ @ � W (114)

DRAFT

Page 30: A CFAR Adaptive Subspace Detector for Second Order

30

We then obtain that

� �«ªx* æz *Ëæ� W ! � ¬ W � @W � W¬ W � @W � W = ¬ W � @W  ¢¡ � W (115)

Or equivalently, � �«ª ! � ¬ W � @W  ¢¡ � W¬ W � @W � W (116)

Taking the pre-whitening filter into consideration, we have

� � c ! � ¬ W O @W , � ��   � ¦ �� § , � �� OXW¬ W O @W , � � OXW (117)

Replacing , with its ML estimateá

from training data, we have

� � c *|� ! � ¬ W O @W á � � �1��� @ á � � � ! � � � @ á � � OXW¬ W O @W á � � OXW (118)

Particularly when `à� ], we have

� � O *|� ! � O @ÿá � � �1��� @âá � � � ! � � � @¢á � � OO @ á � � O (119)

DRAFT