Object recognition based on impulse restoration with use ...ds/Papers/AGWS98.pdf · Object recognition based on impulse restoration with use of the expectation-maximization algorithm

Abu-Naser et al. Vol. 15, No. 9 /September 1998 /J. Opt. Soc. Am. A 2327

Object recognition based on impulse restorationwith use of the

expectation-maximization algorithm

Ahmad Abu-Naser, Nikolas P. Galatsanos,* and Miles N. Wernick

Department of Electrical and Computer Engineering, Illinois Institute of Technology, Chicago, Illinois 60616

Dan Schonfeld

Department of Electrical Engineering and Computer Science (m/c 154), University of Illinois at Chicago,851 South Morgan Street, 1120 SEO, Chicago, Illinois 60607-7053

Received October 6, 1997; revised manuscript received March 17, 1998; accepted May 11, 1998

It has recently been demonstrated that object recognition can be formulated as an image-restoration problem.In this approach, which we term impulse restoration, the objective is to restore a delta function that indicatesthe detected object’s location. We develop solutions based on impulse restoration for the Gaussian-noise case.We propose a new iterative approach, based on the expectation-maximization (EM) algorithm, that simulta-neously estimates the background statistics and restores a delta function at the location of the template. Weuse a Monte Carlo study and localization-receiver-operating-characteristics curves to evaluate the performanceof this approach quantitatively and compare it with existing methods. We present experimental results thatdemonstrate that impulse restoration is a powerful approach for detecting known objects in images severelydegraded by noise. Our numerical experiments point out that the proposed EM-based approach is superior toall tested variants of the matched filter. This result demonstrates that accurate modeling and estimation ofthe background and noise statistics are crucial for realizing the full potential of impulse restoration-based tem-plate matching. © 1998 Optical Society of America [S0740-3232(98)01309-X]

OCIS codes: 100.5010, 100.1830, 100.3010, 100.3020.

1. INTRODUCTIONTemplate matching, which consists of the identificationand localization of known objects (templates) in a scene, isan important image-processing task with application innovelty detection, motion estimation, target recognition,industrial inspection, and sensor fusion. The classicalapproach to this problem is the matched filter (MF),which is optimal in the sense that it is the filter for whichthe signal-to-noise ratio of the output is maximized.1

However, the MF is known to have several importantdrawbacks, including that (1) it is very sensitive to smallvariations of the object, (2) it can produce very broad cor-relation peaks that do not localize the object well,2 and (3)it fails in the presence of background areas of high inten-sity because these areas produce high peaks in the out-put.

Attempts to overcome these limitations have led inves-tigators to propose a number of modifications to the clas-sical MF, among the most important being the phase-onlyMF (POMF) and its variants (see, for example, Refs. 3, 4,and 5). It has been widely recognized that the phase ofthe Fourier transform is more important for describing animage than is its magnitude (see, for example, Ref. 6).Both POMF’s and symmetric phase-only MF’s (SPOMF’s)have been proposed and have been shown to provide bet-ter discrimination ability and noise robustness than theMF (see, for example, Refs. 2, 3, and 7 and the referenceswithin).

0740-3232/98/092327-14$15.00 ©

Filters in use in joint transform correlators have beenproposed that further improve the capabilities of the MF.8

These filters are based on normalization of the correlationsignal by the background power spectrum. Methods forestimation of the power spectrum of the background aregiven in Ref. 8.

A different way of viewing the design of template-matching filters has recently emerged. In this new ap-proach the problem of template matching is framed as oneof image restoration, in which the image to be restored isan impulse at the location of the object to be detected. Toour knowledge the first explicit application of this prin-ciple was reported in Ref. 9, in which the use of expansionmatching, i.e., self-similar nonorthogonal basis decompo-sition, was proposed. More recently, the relationship be-tween linear-minimum-mean-square-error (LMMSE) im-pulse restoration and template matching was recognizedand impulse-restoration filters were proposed fortemplate-matching applications.10–12 In these papers itwas demonstrated that impulse restoration is a very pow-erful approach to template matching and has the poten-tial to provide better localization, class discrimination,and error tolerance than the MF and its variants. Inde-pendently, the relationship between template matchingand impulse restoration was recognized in previouspapers.13–15 In these papers LMMSE-based impulse res-toration was used to find the displacement vector fieldfrom noisy and/or blurred image sequences.

1998 Optical Society of America

2328 J. Opt. Soc. Am. A/Vol. 15, No. 9 /September 1998 Abu-Naser et al.

A critical shortcoming of the earlier work on impulserestoration has been the assumption that the backgroundand/or noise statistics are known, which is not the case inpractice. In realistic applications, one is required to es-timate these statistics from the observed image. Withoutaccurate estimates, the performance of impulse-restoration methods is severely compromised. In Ref. 8,three different approaches to estimate the backgroundpower spectrum were introduced. In Refs. 13–15, one-step periodogram-based approaches were proposed to es-timate the background statistics.

In this paper we propose a new technique for templatematching based on the approach of restoring an impulseindicating the object location. We describe anexpectation-maximization (EM) algorithm16 that simulta-neously estimates the impulse along with the statistics ofthe noise and background. This algorithm is similar inphilosophy to the application of EM to the image-restoration problem in Ref. 17. A major advantage of theimpulse-restoration viewpoint is that it permits the sub-stantial base of knowledge gained in the image-restoration field to be brought to bear on the template-matching problem.

Another important issue not often addressed in theobject-recognition literature is the quantitative evalua-tion of task performance. Frequently figures of merit areused that only indirectly measure the ability of an algo-rithm to detect and locate objects with accuracy. Suchperformance measures include the mean square error(MSE) between the filter output and the desired impulseand the peak-to-sidelobe ratio of the filter output.18 Al-though these measures certainly have some relationshipto detection performance, they provide only indirect evi-dence.

To make quantitative evaluations of the performance ofvarious techniques, we use an extension of the receiver-operating-characteristics (ROC) curve, called the localiza-tion ROC (LROC) curve.19 The ROC curve, which plotsthe probability of detection versus the probability of falsealarm for the continuum of possible decision thresholds,is a comprehensive description of the detection perfor-mance of an algorithm (see, for example, Ref. 20). TheLROC curve plots the probability of detection and correctlocalization of an object versus the false-alarm probabil-ity; thus it also measures the ability of the detection algo-rithm to locate objects correctly. The LROC curve thor-oughly and directly describes the detection andlocalization capability of an algorithm, and thus it ismuch more informative than criteria such as MSE orpeak-to-sidelobe ratio.

The remainder of this paper is organized as follows. InSection 2 we review the impulse-restoration formulationfor the template-matching problem. In Section 3 we in-troduce a maximum-likelihood (ML) approach based onthe EM algorithm that simultaneously estimates thebackground statistics and restores an impulse at the loca-tion of the template. In Section 4 we briefly review theLROC curve. In Section 5 we present numerical experi-ments and LROC-based comparisons demonstrating theproposed impulse-restoration template-matching algo-rithm. Finally, in Section 6 we present our conclusionsand suggestions for future work.

2. BACKGROUNDA. Object Recognition As Impulse RestorationIn this section the formulation of object recognition as animpulse-restoration problem is reviewed. For notationalsimplicity, images are represented throughout the paperby one-dimensional signals. Consider the observed im-age g(n), n 5 0, 1,..., N8 2 1 in which the known objectf (n) is located at the unknown position n0 . One canthen represent the image by

g~n ! 5 f ~n 2 n0! 1 b~n !, (1)

where b(n) is the combined background and noise. Thegoal of template matching is to find the location n0 of theknown object f (n) within the image g(n). Equation (1)can be rewritten as

g~n ! 5 f ~n ! * d ~n 2 n0! 1 b~n !, (2)

where d (n 2 n0) represents a discrete impulse functionlocated at n0 and * denotes convolution.

Using a matrix-vector notation and circular convolu-tions, we write Eq. (2) as

g 5 Fd 1 b, (3)

where g, b, and d are N 3 1 vectors representing g(n),b(n), and d (n), respectively. In the rest of this paper,lowercase boldface letters represent vectors and upper-case italic letters represent matrices. In Eq. (3) d is avector having zeros everywhere except in element n0 ,which has a value of 1. The matrix F is an N 3 N circu-lant matrix constructed from the template f (n) such thatFd represents the convolution f (n)* d (n 2 n0). In gen-eral, the equivalence of Eqs. (2) and (3) can be guaranteedby choosing N > N8.21 Note that if each image vector isobtained by lexicographic ordering of a two-dimensionalimage, then F is a block-circulant matrix.

In this formulation it becomes apparent that the prob-lem of detecting and locating the template can be viewedas a restoration problem in which d (n 2 n0) is the signalto be restored and the template f (n) is the point-spreadfunction.

B. Linear-Minimum-Mean-Square-Error FormulationsAn LMMSE framework can be used to estimate d fromthe observation g in Eq. (3). We will assume signals dand b to have zero mean with covariance matrices Cd andCb , respectively. Moreover, we assume signals d andb to be stationary; thus Cb is an N 3 N circulant covari-ance matrix with eigenvalues equal to the power-spectrum coefficients Sb(k), k 5 0, 1,..., N 2 1. Since dis a discrete impulse function, Cd 5 I, where I isthe N 3 N identity matrix and Sd (k) 5 1/N,k 5 0 , 1,..., N 2 1. We also make the assumption thatb and d are uncorrelated.

In previous work, LMMSE estimates of d have beensought. To find the LMMSE estimate for d, one mini-mizes with respect to d the Bayesian mean square error(BMSE),


BMSE~ d ! 5 E@~d 2 d !2#, (4)

while constraining the estimator to be linear. One canfind the LMMSE estimate for the linear observationmodel in Eq. (3) by using the orthogonality principle,which gives the following well-known solution:22

d 5 ~FtCb21F 1 Cd

21!21FtCb21g

5 ~FtCb21F 1 I !21FtCb

21g, (5)

where the superscript t denotes the transpose of a matrix.Note that d cannot be determined with Eq. (5) in the ab-sence of accurate knowledge of Cb .

Because F and Cb are assumed circulant, Eq. (5) can bewritten in the discrete Fourier transform (DFT) domain1

as

D~k ! 5F* ~k !G~k !

uF~k !u2 1 NSb~k !, k 5 0, 1,..., N 2 1, (6)

where D(k) is the kth DFT component of d, F(k) andG(k) represent the kth DFT components of the templateand the observed image, respectively, Sb(k) is the ktheigenvalue of the covariance matrix of the background,and * denotes conjugation. In the rest of this paper, in-dexed capital letters represent DFT quantities. Clearly,the computation of D(k) with Eq. (6) requires knowledgeof Sb(k), the power spectrum of the combined backgroundand noise. Filters identical to the LMMSE estimator inEq. (6) were used in Ref. 11; in Ref. 9 the background wasassumed white with variance s b

2; i.e., Sb(k) 5 s b2,

k 5 0, 1,..., N 2 1. But neither Ref. 9 nor Ref. 11 of-fered a systematic approach for estimating s b

2 or Sb(k).In Ref. 14 the following periodogram estimate for Sb(k)

was used for impulse restoration:

Sb~k ! 5uG~k !u2 2 uF~k !u2

N, k 5 0, 1,..., N 2 1. (7)

In Ref. 23 it was proved that this is equivalent to the MLestimate of Sb(k); therefore we refer to the use of Eqs. (6)and (7) as the LMMSE-ML method. By definition thepower spectrum Sb(k) cannot be negative. However, ow-ing to inconsistencies between the data and the assumedmodel, due mainly to the assumption that the signal andthe background are uncorrelated, it was observed that theperiodogram estimator yields a negative value for thepower spectrum at many frequencies. These negativevalues were observed to degrade the performance of theLMMSE estimate. So the following constrained ML(CML) estimate of Sb(k) was introduced in Ref. 23:

Sb~k ! 5 maxH 0,uG~k !u2 2 uF~k !u2

N J ,

k 5 0, 1,..., N 2 1. (8)

The main difficulty in the impulse-restoration problemarises from the fact that in most practical applications thepower spectrum Sb(k) of the background in Eq. (6) is un-known and must be estimated. A solution is offered inthe following section.

3. MAXIMUM-LIKELIHOOD IMPULSERESTORATIONThe purpose of impulse restoration is to obtain an esti-mate of d that is as close as possible to the true impulse d.Restoration filters of the kind described in Section 2 re-quire knowledge of the background statistics Sb(k) to ac-complish this goal.

In this section we derive a new iterative restoration fil-ter that does not require advance knowledge of Sb(k).This filter estimates a set of unknown parameters that isdefined by

u 5 $d, Cb%, (9)

where d denotes the impulse and Cb represents the back-ground covariance matrix.

The ML estimator of u is given by

uML 5 arg maxu

log p~g u u! (10)

where p(g u u) is the likelihood function of g given u. Wenow assume the background to obey a Gaussian probabil-ity density function (PDF) with zero mean and covariancematrix Cb ; i.e., b ; N(0, Cb). We further assume thatd ; N(0, Cd ). Assuming d and b to be stationary sig-nals, Cb is an N 3 N circulant covariance matrix, the ei-genvalues of which are the power-spectrum coefficientsSb(k), k 5 0, 1, ..., N 2 1. Because d is a delta function,Cd 5 I, where I is the N 3 N identity matrix. This im-plies that Sd (k) 5 1/N, k 5 0, 1, ..., N 2 1. We alsomake the assumption that b and d are uncorrelated;therefore, because they are Gaussian, they are also inde-pendent.

In this formulation the number of parameters to be es-timated is 2N : N power-spectrum coefficients of b and Nelements of d. Therefore the number of parameters farexceeds the number of observations N (the number of pix-els in the observed image). In an effort to reduce thenumber of parameters to be estimated, we model thebackground as a low-order two-dimensional autoregres-sive (AR) process with causal support,1 i.e.,

b~n ! 5 b~n ! * a~n ! 1 v~n !, (11)

where a(n) represents the background AR-model coeffi-cients that minimize E@v(n)2#. The modeling error is azero-mean Gaussian random process with covariance ma-trix Cv 5 s v

2I. Using a matrix-vector notation and cir-cular convolutions, we write Eq. (11) as

b 5 Ab 1 v. (12)

Thus,

Cb 5 s v2~I 2 A !21~I 2 A !2t, (13)

where A is a circulant matrix containing the model pa-rameters a(n) such that the matrix multiplication isequivalent to convolution. Using the AR model for thebackground, we can rewrite our set of unknown param-eters as u 5 $d, A%. Hence the likelihood function be-comes

p~ guu! 5 N(Fd, FtF 1 s v2~I 2 A !21~I 2 A !2t).

(14)


By substituting Eq. (14) into Eq. (10), we obtain the MLestimate of u as

uML 5 arg maxu

$2log~ uFtF 1 Cbu!

2 ~g 2 Fd !t@FtF 1 Cb#21~g 2 Fd !%, (15)

where Cb is the covariance matrix of b as in Eq. (13).Unfortunately, this is a complicated nonlinear functionthat cannot be optimized directly. Therefore we proposeto employ the EM algorithm to obtain a solution for Eq.(15). The EM algorithm is a general iterative method tocompute ML estimates if the observed data can be re-garded as incomplete.16

We rewrite the matrix-vector form of the imaging equa-tion (1) as follows:

g 5 Fd 1 b 5 @F; I#F dbG 5 Hz, (16)

where we have defined z 5 @d t, bt# t and I is an N 3 Nidentity matrix.

In applying the EM algorithm, one defines a set of com-plete data and a set of incomplete data.16 Here the vec-tor z represents the complete data, consisting of the back-ground b and the impulse d ; the observation vector grepresents the incomplete data. As its name suggests,the EM algorithm consists of two steps: the expectationstep (E step) and the maximization step (M step). In theE step, one computes the conditional expectation of thelikelihood function of the complete data parameterized bythe observed data g and the current estimate of the rel-evant parameters. In the M step this expectation ismaximized. The EM algorithm can be expressed as thealternating iterative computation of the following equa-tions:

E step:

Q~u; u~l !! 5 E$ln@ p~z ; u!#ug, u~l !% (17)

M step:

u ~l11 ! 5 arg maxu

@Q~u; u~l !!#. (18)

We will assume the complete data and the observationto be stationary Gaussian random signals. The Gaussianassumption results in linear equations for the E step andthe M step. Moreover, the stationarity that we assumeyields circulant covariance matrices. Using the diagonal-ization properties of the DFT for circulant matrices, wecan represent the EM algorithm in the DFT domain as aset of scalar equations.17

By evaluating the conditional expectation of thecomplete-data likelihood function as shown in AppendixA, the E step can be written, for k 5 0, 1, ..., N 2 1, as

Md u g~l ! ~k ! 5

F* ~k !G~k !

uF~k !u2 1 NSb~l !~k !

, (19)

Mb u g~l ! ~k ! 5

NSb~l !~k !G~k !


, (20)

Sb u g~l ! ~k ! 5

uF~k !u2Sb~l !~k !


, (21)

where F(k) and G(k) represent the kth DFT componentsof the template and the observed image, respectively;Md u g

(l) (k) and Mb u g(l) (k) represent the kth DFT components

of the conditional mean of the impulse and the back-ground, respectively; Sb u g

(l) (k) is the kth eigenvalue of theconditional covariance matrix of the background param-eterized by the observation; and Sb

(l)(k) is the kth eigen-value of the autocorrelation matrix of the backgroundbased on the AR model. The index l is the iteration in-dex of the algorithm.

In Appendix A we show that the M step is obtained bytaking the partial derivative of the expectation of the like-lihood function with respect to the background AR-modelparameters that are contained in matrix A and setting itto zero. This leads to the set of normal equations

Rb~l !a ~l11 ! 5 Rb

~l !1 5 r b~l ! , (22)

where Rb(l) is an N 3 M matrix containing the first M col-

umns of the N 3 N (N . M) conditional autocorrelationmatrix of the background given the observation Rb

(l) ,where M is of the order of the AR model and 15 @1, 0, ..., 0#t. Thus the normal equations can be writ-ten as

r b~l ! 5 Rb

~l !a ~l11 !, (23)

where r b(l) is an N 3 1 vector representing the inverse

DFT of Sb(l)(k) and a 5 @a(1), a(2),..., a(M)#. In Ap-

pendix A we show that the DFT of r b(l) is found by

Sb~l !~k ! 5 Sb u g

~l ! ~k ! 11N

uMb u g~l ! ~k !u2,

k 5 0, 1,..., N 2 1. (24)

Consequently Sb(l)(k) is the kth eigenvalue of the circu-

lant matrix Rb(l) .

If we take Eq. (13) to the DFT domain, the AR-modeledpower spectrum of the background b is given by

Sb~l !~k ! 5

~sv~l !!2

u1 2 A ~l !~k !u2 ,

k 5 0, 1,..., N 2 1. (25)

4. LOCALIZATION-RECEIVER-OPERATING-CHARACTERISTICS CURVEThe ROC curve is a comprehensive way to describe thedetection performance of a human or machine observer.The ROC curve is a plot of the probability of correct de-tection PD versus the probability of false alarm PF for thecontinuum of possible decision thresholds. Thus it sum-marizes the range of trade-offs between missed detectionsand false alarms. In a comparison of two detection algo-rithms, the one having a strictly higher ROC curve can besaid to be superior in the sense that for any specifiedfalse-alarm probability, the detection probability is higher(likewise, for a specified detection probability, the false-alarm rate is lower).

The limitation of the ROC curve in characterizing anobject-recognition algorithm is that it captures only theability of the algorithm to decide correctly whether an ob-ject is present somewhere in a given scene. Thus it fails


to account for errors in localization, which can be quitesignificant. For example, if the algorithm mistakenlyidentifies a nontarget object as a target, this is treated bythe ROC curve as a correct decision as long as a targethappens to be present somewhere else in the scene.

The localization LROC curve remedies this shortcom-ing by taking into account localization performance aswell as detection performance. An LROC curve is a plotof the probability of detection and correct localization ver-sus the probability of false alarm. Thus a decision is saidto be correct only if the object is both detected and locatedcorrectly by the object-recognition algorithm.

The experimental method we used to compute LROCcurves is described in the following section.

5. EXPERIMENTAL RESULTSIn this section we describe numerical experiments thatcompare the proposed EM-based impulse-restoration fil-ter with previous implementations of the LMMSEimpulse-restoration filter and different types of MF’s.Monte Carlo studies were performed to evaluate the per-formance of the different algorithms. Multiple syntheticimages were generated by randomly varying the object lo-cations and noise realizations.

We compared the phase-only matched filter3 (POMF),given by

D~k ! 5F* ~k !G~k !

uF~k !u, k 5 0, 1, ..., N 2 1, (26)

the symmetric phase-only matched filter2 (SPOMF), givenby

D~k ! 5F * ~k !G~k !

uF~k !uuG~k !u, k 5 0, 1, ..., N 2 1, (27)

and previous LMMSE filters given in Eq. (6) that use theML and the CML estimates of the background statisticsdefined in Eqs. (7) and (8), respectively.

We also evaluated the performance of the MF variantsdescribed in Ref. 8. The first approach in Ref. 8 modelsthe background as

b~n ! 5 g~n ! 2 f ~n 2 n0!, (28)

where b(n) represents the background, g(n) representsthe observation image, and f (n 2 n0) is the template atthe location of the target in the observation. Since thelocation of the target is not normally known a priori, auniform distribution of the target location is assumed.Using this assumption, one obtains the following form forthe power spectrum of the background:

Sb~k ! 5 uG~k !u2 1 uF~k !u2. (29)

We will refer to this filter in the experiments as MF–Model(1). Notice that this is identical to an LMMSE fil-ter in which the power spectrum of the observation isused as an approximation of the background power spec-trum. The second MF approach in Ref. 8 models thebackground as

b~n ! 5 w~n 2 n0!g~n !, (30)

where w(n 2 n0) is a window function centered at the lo-cation of the object that has a value of 0 within the tem-plate and 1 elsewhere. Again by assuming a uniform dis-tribution of the target location, the method in Ref. 8approximates the background power spectrum as

Sb~k ! 5 uG~k !u2* uW~k !u2, (31)

Fig. 1. Examples of the backgrounds used by the exact LMMSE filter. The object of interest is the tank. The exact LMMSE filter isunrealistic. It was studied only to determine a theoretical upper bound on the performance.


where * denotes convolution and W(k) represents the kthDFT component of the window function. We will refer tothis filter as MF–Model(2).

The third MF approach in Ref. 8 uses the power spec-trum of the observation as an estimate for the background

Fig. 2. 2 3 2 autoregressive image model with causal support.

Fig. 3. Object templates.

power spectrum. This approach is based on the assump-tion that the size of the template is much smaller thanthe scene. This matched filter turns out to be identical inform to the LMMSE ML filter defined in Subsection 2.B.

To establish the upper performance bound of theimpulse-restoration template-matching approach, weimplemented an exact LMMSE filter that assumes thatperfect knowledge of the noise and background statisticswas also used. Of course, such information is neveravailable in practice. The exact LMMSE filter servesonly as a basis for judging the other more realistic ap-proaches. The true statistics were provided only to theexact LMMSE filter. For purposes of implementing theexact LMMSE filter, the actual background was deter-mined on the basis of the imaging model of Eq. (3) by plac-ing zeros at the location of the object (see, for example,Fig. 1). The sum of the periodogram power-spectrum es-timate of the true background, plus the known variance ofadded white noise, was used as the background statisticsfor the implementation of the exact LMMSE filter in Eq.(6).

As stated before, the EM algorithm uses an AR modelfor the background. A 2 3 2 AR image model withcausal support as shown in Fig. 2 was used to model thebackground. In our experience it was observed that theperformance of the EM algorithm is not very sensitive tothe order of the AR model in this application.

A. Experiment 1: Localization-Receiver-Operating-Characteristics AnalysisIn this experiment, LROC analysis was used to quantifythe performance of various algorithms in detecting and lo-cating objects within a scene. Test scenes of dimension256 3 256 were generated by embedding the three objectsshown in Fig. 3 at random locations within a backgroundterrain image. These scenes were then degraded by blurand white Gaussian noise to represent the effects of theimaging system. The scenes were blurred with aGaussian-shaped point-spread function with full width athalf-maximum (FWHM) of 3 pixels. White Gaussiannoise with s 5 20 was used, where s represents the stan-dard deviation. Examples of the noisy blurred scenesused in the experiment are shown in Fig. 4.

In each experimental trial, an object-recognition filterwas applied to the generated scene. The peak of the out-put was taken to be the restored impulse, and the value atthe peak was used as a decision variable x. If x exceededa decision threshold T, then the object was said to bepresent at the location of the peak; otherwise, the objectwas said to be absent. The performance of the algorithmfor various thresholds is summarized by the LROC curvethat plots the probability of detection and correct localiza-tion (PDL) versus the probability of false alarm (PF).These probabilities were obtained by numerical evalua-tion of the following integrals:

PDL 5 ET

`

p~x u H1!dx

PF 5 ET

`

p~x u H0!dx, (32)


Fig. 4. Examples of noisy test scenes with the object of interest present (in this case, tank). The noise level is s 5 20. In each re-alization the objects are placed randomly within the scene.

Fig. 5. Examples of noisy scenes with the object of interest absent (in this case, the tank). The noise level is s 5 20.

where p(xuHj) is the conditional PDF of x given hypoth-esis Hj . The null hypothesis H0 is that the object is ab-sent; the alternative hypothesis H1 is that the object ispresent.

In this study the conditional PDF’s p(xuH1) andp(xuH0) were obtained by the following procedure. Twosets of 100 images were generated each with a differentnoise realization and with different random object loca-tions. In one set of images, all three objects were presentin each scene (see, for example, Fig. 4); in the other set,the object of interest was absent (see, for example, Fig. 5).

In the case where the object of interest was present, eachalgorithm was applied to every image, and the magnitudex of the peak value of the restored impulse was recorded,provided that the peak correctly indicated the target loca-tion. The normalized histogram of these values was thenused as p(xuH1). To find p(xuH0), we used the set of im-ages in which the object of interest was absent. Againthe algorithms were applied to each image, but the mag-nitude of the peak value of the restored impulse was re-corded regardless of its location. The normalized histo-gram of the recorded peak values of the restored impulse


Fig. 6. Conditional PDF’s for s 5 20 when the tank is the object of interest.


obtained was used as p(xuH0). Different points of theLROC curves were obtained by varying the decisionthreshold T and computing the integrals in Eq. (32).

In Fig. 6 we show the conditional PDF’s p(xuH0) andp(xuH1) for the exact LMMSE, EM-AR, MF-Model(1),MF-Model(2), LMMSE-CML, SPOMF, LMMSE-ML, andPOMF filters for the case in which the tank was the objectof interest. These figures illustrate that good discrimi-nating capability results from nonoverlapping p(xuH0)and p(xuH1). For example, the hypotheses are almostperfectly distinguished by the exact LMMSE if the deci-sion threshold T is 175.

The LROC curves are shown in Fig. 7. The LROCcurves in Fig. 7 indicate that the proposed EM-based ap-proach with AR modeling performs the best overallamong the template-matching filters tested. Othermethods worked better in some cases but were inconsis-tent and so did not perform as well overall.

B. Experiment 2: Robustness ComparisonIn this experiment we compare the robustness of the pro-posed algorithms as the noise increases, using the 256

3 256 Lena image and the 21 3 21 template, both shownin Fig. 8. White additive noise was used to degrade thescene. Twenty different noise levels, with noise standarddeviation ranging from 10 to 100 in increments of 10,were used to degrade the scene (see, for example, Fig. 9).For each noise level the experiment was repeated 50times with different noise realizations. The probabilityPDL was found as PDL 5 N / 50, where N is the number oftimes that the maximum of the output of the restoreddelta was positioned correctly at the object location.

In Fig. 10 plots of PDL versus the noise standard devia-tion are shown for the exact LMMSE, EM-AR, MF-Model(1), MF-Model(2), LMMSE-CML, SPOMF, LMMSE-ML, and POMF algorithms. This plot shows that theproposed EM-AR filter is the best among all filters that donot require prior knowledge of the location of the object.In Fig. 11 the restored impulses for the exact LMMSE,EM-AR, MF-Model(1), MF-Model(2), LMMSE-CML,SPOMF, LMMSE-ML, and POMF algorithms are shownfor a specific noise realization when the standard devia-tion is 30. Notice that the proposed EM-AR filter givesthe closest approximation to an ideal delta function

Fig. 7. LROC curves for s 5 20. Overall, the EM-AR algorithm outperforms all but the exact (unrealistic) LMMSE filter, which makesuse of the unknown object location. Other methods did outperform the EM-AR algorithm in some specific situations but were incon-sistent and did not perform as well overall.


among all filters that do not require prior knowledge ofthe location of the object. A metric MSE for the restoredimpulse was used to compare different algorithms. MSEis defined by

MSE 51

N (i51

N

@ d~i ! 2 d ~n0!#2, (33)

where d (i) is the normalized restored impulse as shownin Fig. 11 and n0 represents the location of the object. InTable 1 the MSE values for the restored impulses aretabulated for different filters for the case of s 5 30.These values represent the average MSE over all the 50cases for s 5 30. The table shows that the EM-AR filtergives the lowest average MSE among all filters that donot require prior knowledge of the location of the object.

Fig. 8. Lena scene and the template.

6. CONCLUSIONS AND FUTURE WORKIn this paper restoration-based template-matching filterswere used to detect objects within a noisy, blurred scene.The main challenge involved in the impulse-restorationapproach is good estimation of the background statistics.To address this, we proposed a new iterative approachbased on the EM algorithm that simultaneously esti-mates the background statistics and restores a delta func-tion at the location of the template. Our algorithm mod-els the background as an AR process with causal support.

Our numerical experiments verified how important it isto the success of the template-matching impulse-restoration approach that the background statistics be ac-curately estimated. The proposed EM-AR approach wassuperior to the variants of the matched filter that wetested. When the background statistics are not esti-mated accurately, impulse-restoration-based templatematching cannot realize its full potential. Our numericalexperiments also show that the advantage of the proposedapproach over previous methods increases with thestrength of the noise in the scene.

We are currently studying the application of this ap-proach to photon-limited images and to the more difficultproblems of object rotation and scaling.

APPENDIX A: THE EXPECTATION-MAXIMIZATION ALGORITHMLet us assume the complete-data set z to obey the follow-ing Gaussian likelihood function:

p~z ; u! 5 u2p Czu21/2 exp~212 z tCz

21z!, (A1)

in which u is the set of the unknown parameters in Eq.(9).

Taking the logarithm of Eq. (A1), we get

ln@ p~z ; u!# 5 212 @ln~ u2p Czu! 1 z tCz

21z#

5 K 212 @ln~ uCzu! 1 z tCz

21z#. (A2)

Then the expected value of the logarithm of the condi-tional PDF required by the E step of the EM algorithm inEq. (17) is given by

Q~u; u~l !! 5 K 212 $E@ln~ uCzu!ug ; u~l !#

1 E~g tCz21zug ; u ~l !!

5 K 2 Y~u; u ~l !!. (A3)

It is easy to show that

Y~u ; u~l !! 5 ln~ uCzu! 1 trace~Cz21Rzug

~l ! !, (A4)

with

Rz u g~l ! 5 E~zz tug ; u~l !! 5 Cz u g

~l ! 1 mz u g~l ! ~mz u g

~l ! !t. (A5)

Thus we can write

Y~u ; u~l !! 5 ln~ uCzu! 1 trace~Cz21Cz u g

~l ! ! 1 ~mz u g~l ! !tCz

21mz u g~l ! .

(A6)Because of the definition of z in Eq. (16) and since d and bare assumed uncorrelated, we get


Cz 5 F I0

0Cb

G , Czug 5 FCdug0

0Cbug

G ,mz u g 5 @~md u g!t, ~mb u g!t# t. (A7)

Using the definition of Cb in Eq. (13), we can write Eq.(A6) asY~u ; u~l !! 5 lnusv

2~I 2 A !21~I 2 A !2tu

1 traceFCd u g~l ! 1

1

sv2 ~I 2 A !t~I 2 A !Cb u g

~l ! G1 ~m d u g

~l ! !tm d u g~l ! 1

1

sv2 ~m b u g

~l ! !t~I 2 A !t

3 ~I 2 A !m b u g~l ! . (A8)

Since we assume an AR model with causal support, A is astrictly upper-triangular matrix. Hence

lnu~I 2 A !21~I 2 A !2tu 5 ln 1 5 0. (A9)Thus we can write

Y~u ; u ~l !! 5 traceH Cd u g 11

sv2 ~I 2 A !t@Cb u g

~l !

1 m b u g~l ! ~mb u g

~l ! !t#~I 2 A !J1 ~md u g

~l ! !tmd u g~l ! 1 2N ln sv . (A10)

A is a circulant matrix containing the model parameters,and Cb u g

(l) is a circulant matrix, since we assume station-arity. For circulant matrices it is easy to show that

Fig. 9. Examples of noisy Lena scenes for different noise levels. Clockwise from top left, s 5 5, s 5 30, s 5 50, and s 5 100.


trace$XtAX% 5 N~x tAx!, (A11)

trace~Xaa tX t! 5 trace~Axx tA ! 5 atX tXa, (A12)

where X and A are N 3 N circulant matrices and x and aare N 3 1 vectors representing the first columns of X andA, respectively. Hence we obtain

Y~u ; u~l !! 5 traceH Cd u g 11

sv2 ~1 2 a8!t

3 @N • Cb u g~l ! 1 Mb u g

~l ! ~Mb u g~l ! !t#~1 2 a8!J

1 ~m d u g~l ! !tm d u g

~l ! 1 2N ln sv , (A13)

in which 1 5 @1 0...0# t; a8 5 @a(1)...a(M), 0, 0,..., 0#t,where M is the order of the AR model, a(k) is the kth pa-rameter of the AR model, and Mbug is an N 3 N circulantmatrix constructed by circular shifting of m b u g .

Letting

Rb~l ! 5 Cb u g

~l ! 11N

Mb u g~l ! ~Mb u g

~l ! !t 1 2N ln sv , (A14)

we can write

Y~u ; u~l !! 5 traceFCd u g 1N

s v2 ~1 2 a8!t

3 Rb~l !~1 2 a 8!G 1 ~m d u g

~l ! !tmd u g~l ! .

(A15)

Using the diagonalization properties of the DFT for cir-culant matrices, from Eq. (A14) we can write

Sb~l !~k ! 5 Sb u g

~l ! ~k ! 11N

uMb u g~l ! ~k !u2

for k 5 0, 1,..., N 2 1, (A16)

where Sb(l)(k) is the kth eigenvalue of Rb

(l) , Sb u g(l) (k) is the

kth eigenvalue of the covariance matrix of the back-ground parameterized by the observation obtained by Eq.

Fig. 10. Plot of the probability of correct localization as a func-tion of noise standard deviation.

(21), and Mb u g(l) (k) represents the kth eigenvalue of the

matrix Mb u g(l) , found as the kth DFT component of the vec-

tor m d u g .

1. M stepTaking the partial derivative of Y(u ; u (l)) in Eq. (A15)with respect to a and setting it to zero, we obtain

Rb~l !a8~l11 ! 5 Rb

~l !1 5 r b~l ! , (A17)

Fig. 11. Examples of the restored impulse for s 5 30.


so the AR parameters are found by solving a set of normalequations. Since a8 contains M , N nonzero unknowncoefficients, we calculate the least-squares estimate ofa (l11):

a ~l11 ! 5 @~Rb~l !!tRb

~l !#21~Rb~l !!tr b

~l ! , (A18)

where Rb(l) is an N 3 M matrix containing the first M col-

umns of Rb(l) .

From Eq. (13) the updated equation of the power spec-trum of the background b under the AR model is thengiven by

Sb~l !~k ! 5

~sv~l !!2

u1 2 A ~l !~k !u2 , k 5 0, 1,..., N 2 1,

(A19)

where sv is the standard deviation of the driving noise ofthe AR model. It is calculated from rb and a,24 accordingto

~sv~l !!2 5 rb

~l !~0 ! 2 (k51

M

a~l !~k !rb~l !~k !, (A20)

and A (l)(k) represents the kth DFT component of a8(l),where a8(l) is a version of a (l) zero padded to length N.

2. E stepWhen the vectors z and g are related by Eq. (16) it can beshown22 that the conditional mean is given by

m z u g 5 Cz Ht~HCzHt!21g

5 F Ft

CbG~FFt 1 Cb!21g 5 Fm d u g

m b u gG . (A21)

In the DFT domain Eq. (A21) gives, for k 5 0, 1, ..., N2 1,

Md u g~l ! ~k ! 5

F* ~k !G~k !


, (A22)

Mb u g~l ! ~k ! 5

NSb~l !~k !G~k !


. (A23)

For Gaussian z and g with g 5 Hz it can be shown22 thatthe conditional covariance

Cz u g 5 FCd/g 0

0 Cb/gG 5 Cz 2 Cz Ht~HCz Ht!21HCz .

(A24)

Thus,

Cb/g 5 Cb 2 Cb~FFt 1 Cb!21Cb . (A25)

Table 1. Mean-Square-Error Values for DifferentFilters for s 5 30

Filter MSE

Exact LMMSE 4.4 3 1023

EM-AR 1.6 3 1022

LMMSE-CML 4.2 3 1022

SPOMF 5.5 3 1022

LMMSE-ML 1.2 3 1021

POMF 1.3 3 1021

In the DFT domain Eq. (A25) becomes

Sb u g~k ! 5 Sb~k ! 2Sb

2~k !

1N uF~k !u2 1 Sb~k !

. (A26)

Thus from Eq. (A26) we obtain

Sb u g~l ! ~k ! 5

uF~k !u2Sb~l !~k !


for k 5 0, 1,..., N 2 1. (A27)

In this EM formulation the conditional mean md u g isequivalent to the LMMSE estimate for d in Eq. (6). Sincethe AR model is assumed for the background statistics,Sb

(l)(k) in Eqs. (A22), (A23) and (A26) is replaced bySb

(l)(k), which yields Eqs. (19), (20), and (21), respectively.

*Address correspondence to Nikolas P. Galatsanos atthe address on the title page, at [email protected], or attelephone 312-567-5259.

REFERENCES1. A. K. Jain, Fundamentals of Digital Image Processing

(Prentice-Hall, Englewood Cliffs, N.J., 1989).2. Q. Chen, M. Defrise, and F. Decorninck, ‘‘Symmetric phase-

only matched filtering of Fourier–Mellin transforms for im-age registration and recognition,’’ IEEE Trans. PatternRecogn. Mach. Intell. 12, 1156–1198 (1994).

3. J. L. Horner and P. D. Gianino, ‘‘Phase-only matched filter-ing,’’ Appl. Opt. 23, 812–816 (1984).

4. J. L. Horner and R. L. Leger, ‘‘Pattern recognition with bi-nary phase-only matched filters,’’ Appl. Opt. 23, 812–816(1985).

5. F. M. Dickey and B. D. Hansche, ‘‘Quad-phase correlationfilters for pattern recognition,’’ Appl. Opt. 28, 1611–1613(1989).

6. A. V. Oppenheim and J. S. Lim, ‘‘The importance of phasein signals,’’ Proc. IEEE 69, 529–541 (1981).

7. O. K. Ersoy and M. Zeng, ‘‘Nonlinear matched filtering,’’ J.Opt. Soc. Am. A 6, 636–648 (1989).

8. L. P. Yaroslavsky and E. Marom, ‘‘Nonlinearity optimiza-tion in nonlinear joint transform correlators,’’ Appl. Opt.36, 4816–4822 (1997).

9. J. Ben-Arie and K. R. Rao, ‘‘A novel approach to templatematching by nonorthogonal image expansion,’’ IEEE Trans.Circuits Syst. Video Technol. 3, 71–84 (1993).

10. C. R. Chatwin R. K. Wang, and R. C. D. Young, ‘‘Assess-ment of a Wiener filter—synthetic discriminant function foroptical correlation,’’ J. Optics Lasers Eng. 22, 33–51 (1995).

11. E. Marom and H. Inbar, ‘‘New interpretations of Wiener fil-ters for image recognition,’’ J. Opt. Soc. Am. A 13, 1325–1330 (1996).

12. B. Javidi, F. Parchekani, and G. Zhang, ‘‘Minimum-mean-square error filters for detecting a noisy target in back-ground noise,’’ Appl. Opt. 35, 6964–6975 (1996).

13. M. Choi, N. Galatsanos, and D. Schonfeld, ‘‘Imagerestoration-based template-matching with application tomotion estimation,’’ in Visual Communications Image Pro-cessing ’96, R. Ansari and M. Smith, eds., Proc. SPIE 2727,375–386 (1996).

14. M. Choi, N. Galatsanos, and D. Schonfeld, ‘‘On the relationof image restoration and template-matching: applicationto block-matching motion estimation,’’ in Proceedings of theIEEE International Conference on Acoustics, Speech, and


Signal Processing 1996 (Institute of Electrical and Elec-tronics Engineers, Piscataway, N.J., 1996), Vol. IV, pp.2112–2115.

15. M. Choi, N. Galatsanos, and D. Schonfeld, ‘‘Image restora-tion based template-matching for multi-channel restorationof image sequences,’’ Presented at the 1996 ASILOMARConference, Pacific Grove, Calif., November 1996.

16. A. P. Dempster, N. M. Laird, and D. B. Rubin, ‘‘Maximum-likelihood from incomplete data,’’ J. R. Statist. Soc. B 39,1–38 (1977).

17. A. K. Katsaggelos and K.-T. Lay, Digital Image Restoration(Springer-Verlag, Berlin, 1991).

18. B. V. Kumar and L. Hassebrook, ‘‘Performance measuresfor correlation filters,’’ Appl. Opt. 29, 2997–3006 (1990).

19. S. J. Starr, C. E. Metz, L. B. Lusted, and D. J. Goodenough,

‘‘Visual detection and localization of radiographic images,’’Radiol. 116, 533–538 (1975).

20. H. L. Van Trees, Detection Estimation, and ModulationTheory: Part I (Wiley, New York, 1968).

21. H. Andrews and B. Hunt, Digital Image RestorationPrentice-Hall, Englewood Cliffs, N.J., 1977).

22. S. Kay, Fundamentals of Statistical Signal Processing, Es-timation Theory (Prentice-Hall, Englewood Cliffs, N.J.,1993).

23. A. Abu-Naser, ‘‘Object recognition based on impulse resto-ration for images in Gaussian noise,’’ Master’s thesis (Illi-nois Institute of Technology, Chicago, Ill., 1996).

24. John G. Proakis and Dimitris G. Manolakis, Digital SignalProcessing, Principles, Algorithms, and Applications(Prentice-Hall, Englewood Cliffs, N.J., 1996).

Documents

Object recognition based on impulse restoration with use ...ds/Papers/AGWS98.pdf · Object recognition based on impulse restoration with use of the expectation-maximization algorithm