1186 OPTICS LETTERS / Vol. 16, No. 15 / August 1, 1991

Normalized correlation for pattern recognition

Fred M. Dickey and Louis A. Romero

Sandia National Laboratories, Albuquerque, New Mexico 87185

Received April 16, 1991

The normalization of the correlation filter response effects intensity invariance. We discuss the implications ofa normalization based on the Cauchy-Schwarz inequality for the discrimination problem. It is shown that nor-malized phase-only and synthetic discriminant functions do not provide the discrimination/recognitionobtained with the classical matched filter.

The technique of using correlation filters for opticalpattern recognition has received considerablestudy.1 3 The volume of literature on the subjecthas become quite large and would be difficult to sur-vey adequately in a short paper. However, the ref-erences in the papers cited in this Letter offerextensive coverage of the field.

Several performance measures have been sug-gested for correlation filters3; however, little atten-tion has been given to the normalization of the filteroutput. Goodman suggested that the matched filterbe normalized by using the Cauchy-Schwarz in-equality.4 The Cauchy-Schwarz inequality can beapplied to arbitrary correlation filters. This nor-malization achieves intensity invariance.

The use of correlation filters in pattern recogni-tion is essentially an inner product between twofunctions: the object and reference functions.These functions may be considered vectors in aHilbert space. The normalization of the innerproduct (correlation integral) defines a unique anglebetween the reference and the object function. It isthis angle that provides a measure of similarity be-tween the object and reference functions.

The normally computed correlation function is

c(y - X0) =2J h*(x)Sh(X)f(X + y - xo)dX , (1)

where f(x) is an input(object) function, h*(-x) is afilter impulse response,

Sh(x) = ox E support of h(x)otherwise

and x0 is a coordinate that maximizes the integral.In what follows, we frequently use

Sh(x)h(x) = h(x), (2)

Sh 2(x) = Sh(x). (3)

Application of the Cauchy-Schwarz inequality toEq. (1) gives

c(y - x0) c f f(x + y - x0) 2Sh(x)dx J h(x) dx.

The preceding suggests a normalized correlationfunction given by

(y -Xo)1. ~~~~~~~~~2J h*(x)f(x + y - xo) dx

< 1.

J f(x + y - x0) Sh(x)dx h(X) dx(5)

It is a further property of the Cauchy-Schwarz in-equality that the equality in relation (5) is obtainedat y = xo = O if and only if

h(x) = Af(X)Sh(X). (6)

It is the form of relation (5) and the A in Eq. (6) thateffects intensity invariance. Thus it is the classicalmatched filter associated with white noise thatmaximizes the normalized correlation given by rela-tion (5). If the filter used to identify an object func-tion f(x) is not the matched filter, a normalizedcorrelation value of less than one is obtained. Inthis case, there are many functions different fromf(x) that give the same correlation value. The dif-ference (e.g., the mean-square difference) betweenthese functions and the object function must ap-proach zero as the normalized correlation functionapproaches one.

For any filter other than the matched filter, thenormalized correlation is less than one when the ob-ject function for which the filter was made is theinput function. Further, there is always a functionthat gives a normalized correlation value of one.This function is just the filter impulse response.For an arbitrary filter, there are many functionsthat produce a correlation value equal to or greaterthan that produced by the object function.

Unless there are additional constraints on theset of admissible functions (e.g., limited charac-ter recognition), the preceding suggests that thematched filter provides the best recognition/discrimination performance. This does not excludethe use of preprocessing to enhance the systemrecognition/discrimination performance. Prepro-cessing is used to define the objects or features of theobjects to be recognized. The correlation filtershould then be designed to address the preprocessed

0146-9592/91/151186-03$5.00/0 C 1991 Optical Society of America

August 1, 1991 / Vol. 16, No. 15 / OPTICS LETTERS 1187

image set. In no case is it conceivable that a corre-lation filter should be designed to produce a smallnormalized correlation response.

It is interesting to apply the normalization tospecific filter types such as the phase-only filter(POF) and synthetic discriminant functions (SDF's).Phase-only-type filters are, perhaps, best treated inthe frequency domain. Application of Parseval's re-lation to relation (5) gives

6(y - x0) =r 2J H*(v)F(v)exp[i2Trrv(y - xo)] dv

It is instructive to consider the SDF for the train-ing set of f (x) that is constrained by

and

f h*(x)fi(x)dx2 = 1

f 2ffi*(x)fj(x)dx = ij fi(X) dx

(13)

(14)

for 1 < i, j ' N. For square integrable functions,h(x) can be written as

h(x) = 2 o tit) ++(X), (15)

f[F(v) 0 F(V)]*Sh(V)exp[-i2wv(y - xo)]dvf H(V)I dv

(7)

where capital letters or the tilde is used denoteFourier transforms and 0 denotes correlation.

For object functions of finite extent and the POFdefined by

H*(P) = IF(7,)I v E f., (8)

where f fi(x)4(x)dx = 0, ljail = 1, and Ilf(x)112 =f If(x)2 dx.

Then for the element fi(x) of the training set as theinput, the peak normalized correlation response de-termined by relation (5) for any element of the train-ing set is

C(0) =

[ 1lf (l2+g(X)| ]li(X)112

N m axklj If(x)11where fl is the support of the POF, the peak value ofEq. (7) is given by

[J IF(V) dv]

c(0) =

B f [F(v) 0 F(v)]*Sh(v) di,

In this equation B is given by

B = J dv (10)

and provides a measure of the spectral content(bandwidth) of the filter.

For bounded square integrable functions5 f(x),

IF(v)l = 0(1/v) as v -+ °oo. (11)

If we apply Eq. (11) to Eq. (9) we get the limit

c(0) -0 as B -oo. (12)

This is just the condition that allows extremely largevalues for the unnormalized correlation of Eq. (1).For discontinuous functions, the numerator of Eq. (9)diverges as B increases.

The discussion following relation (5) and the lim-its expressed in expression (12) suggest that thespectral support (bandwidth) for POF's should not bearbitrarily large if discrimination is considered.Equivalently, the unnormalized correlation responseshould not be made arbitrarily large. This is clearlythe case when random noise is considered.

SDF's6 are filter functions designed to give aspecified response for a set of images. A typical setof images, commonly called the training set, is a setof images of an object made from different rotations(orientations) of the object. One application ofSDF's is to achieve rotation (orientation) invariancein optical correlators used for pattern recognition.

(16)

If the training set consists of in-plane rotations, theright-hand side of inequality (16) is just 1/N. Notethat some undesired function (the matched function)will give a maximum value of one. The result isthat SDF's made for training sets with a large num-ber of elements produce a correspondingly small nor-malized correlation. This suggests, as in the casefor POF's, that recognition/discrimination is beingdegraded as N increases.

In summary, we have suggested the normalizationof correlation filter by applying the Cauchy-Schwarzinequality to the filter output. This normalizationachieves intensity invariance. Recognition/discrimination is effected by the fact that the equal-ity in the Cauchy-Schwarz inequality is obtained ifand only if the filter is matched to the input func-tion. Application of the normalization to POF's andSDF's showed that they generally do not provide therecognition/discrimination capabilities of thematched filter and suffer from the fact that thereare always other functions that give a larger correla-tion response than that of the object function. Thiscan be worsened by poor design of the POF or SDF,such as incorporation of excessive bandwidth or toomany elements in the training set. Although theanalysis was presented in one-dimensional form, itis easily generalized to two dimensions.

The authors thank Jeff Mason for arguing theneed for normalization. This study was performedat Sandia National Laboratories and was supportedby the U.S. Department of Energy under contractDE-AC04-76DP00789.

References

1. D. L. Flannery and J. L. Horner, Proc. IEEE 77, 1511(1989).

1188 OPTICS LETTERS / Vol. 16, No. 15 / August 1, 1991

2. F M. Dickey, B. V K. V Kumar, L. A. Romero, andJ. M. Connelly, Opt. Eng. 29, 994 (1990).

3. B. V K. V Kumar and L. Hassebrook, Appl. Opt. 29,2997 (1990).

4. J. W. Goodman, Introduction to Fourier Optics(McGraw-Hill, New York, 1968), Chap. 7, p. 179.

5. N. Bleistein and R. A. Handelsman, AsymptoticExpansion of Integrals (Dover, New York, 1986),Chap. 3, p. 80.

6. Z. Bahri and B. V K. V Kumar, J. Opt. Soc. Am. A 5,562 (1988).