Title Goes Here Correlation Pattern Recognitionkumar/DowdSeminar.pdf · 1 Vijayakumar Bhagavat ula Vijayakumar Bhagavatula Title Goes Here Correlation Pattern Recognition December

1

Vijayakumar Bhagavatula


Title Goes HereCorrelation Pattern Recognition

December 10, 2003

2


Outline

! Correlation pattern recognition! Pattern recognition examples! Book! Demos

3


18-794 Pattern Recognition Theory

! Speech recognition! Optical character recognition (OCR)! Fingerprint recognition! Face recognition! Automatic target recognition! Biomedical image analysis

Objective: To provide the background and techniques needed for pattern classification

For advanced UG and starting graduate students

Example Applications:

4


Pattern Recognition Methods

Feature ExtractionInput

Classifier Class

! Statistical methods (e.g., Bayes decision theory)! Machine learning methods! Artificial neural networks! Correlation filters

Most approaches are based in image domain whereas significant advantages exist in spatial frequency domain.

6


Example Feature-based Matching

Minutiae

Minutiae Extraction

Input Image

Minutiae

Orientation Field

Region of Interest

Thinned Ridges

Extracted RidgesRidge Ending

Ridge Bifurcation

Orientation Estimation

Fingerprint Locator

Ridge Extraction

Thinning f

Minutiae Extraction

! Features based on intuition & experience

! Significant preprocessing needed

! Sensitive to occlusions

7


Correlation Pattern Recognition

! Normalized correlation between r(x) and s(x) between -1 and +1; reaches +1 if and only if r(x) = s(x).

! Problem: Reference patterns rarely have same appearance! Solution: Find the pattern that is consistent (i.e., yields large

correlation) among the observed variations.

( ) ( )

( ) ( )2 21 1

r x s x dx

r x dx s x dx

! r(x) test pattern! s(x) reference pattern

8


Pattern Variability

! Facial appearance may change due to illumination! Fingerprint image may change due to plastic deformation

9


Pattern Locations

! Desired Pattern can be anywhere in the input scene.! Multiple patterns can appear in the scene.! Pattern recognition methods must be shift-invariant.

11


Cross-Correlation Function

! Determine the cross-correlation between the reference and test images for all possible shifts

!When the target scene matches the reference image exactly, output is the autocorrelation of the reference image.

! If the input r(x) contains a shifted version s(x-x0) of the reference signal, the correlator will exhibit a peak at x=x0.

! If the input does not contain the reference signal s(x), the correlator output will be low

! If the input contains multiple replicas of the reference signal, resulting cross-correlation contains multiple peaks at locations corresponding to input positions.

( ) ( ) ( )c r x s x dx =

12


Cross-Correlation Via Fourier Transforms

InputScene

FT

CorrelationFilter

IFTCorrelationOutput

ReferenceIm age s(x)

FilterDesign

r(x)

R(f)

H (f)

c()

! Fourier transforms can be done digitally or optically

13


ToInput SLM

FourierLens

FourierLens

Correlationpeaks for objects

ToFilter SLM

CCD Detector

Laser Beam

FourierTransform

InverseFourierTransform

Optical Correlator

SLM: Spatial Light ModulatorCCD: Charge-Coupled Detector

14


Correlation Filters

M atchNo M atch

DecisionTest Image

IFFT Analyze

Correlation output

FFT

Correlation Filter

Filter Design . . .Training Images

TrainingRecognition

15


Peak to Sidelobe Ratio (PSR)

meanPeak

PSR

=

1. Locate peak1. Locate peak

2. M ask a sm all 2. M ask a sm all pixel regionpixel region

3. Com pute the m ean and 3. Com pute the m ean and in a in a bigger region centered at the peakbigger region centered at the peak

! PSR invariant to constant illumination changes

! Match declared when PSR is large, i.e., peak must not only be large, but sidelobes must be small.

16


Train on 3, 7, 16, Train on 3, 7, 16, --> Test on 10.> Test on 10.

18


Using sam e Filter trained before,

Perform cross-correlation on cropped-face shown on left.

19


CO RRELATIO N FILTERS ARE SHIFT-INVARIANT

Correlation output is shifted down by the sam e am ount of the shifted face im age, PSR rem ains SAM E!

20


Using SO M EO NE ELSES Filter, . Perform cross-correlation on cropped-face shown on left.

As expected very low PSR.

21


Automatic Target Recognition Example

22


Correlation Plane Contour M ap Correlation Plane Contour M ap

Correlation Plane SurfaceCorrelation Plane Surface

M 1A1 in the open M 1A1 near tree line

SAIP ATR SDF Correlation Perform ance for Extended Operating

Conditions

Courtesy: Northrop Grum m an

Adjacent trees cause some correlation noise

24


Biometric Verification Examples

25


Facial Expression Database

! Facial Expression Database (AMP Lab, CMU)! 13 People! 75 images per person! Varying Expressions! 64x64 pixels! Constant illumination

! 1 filter per person made from 3 training images

26


PSRs for the Filter Trained on 3 Images

Response to Training Images Response to

Faces Images from Person A

M ARGIN OF SEPARATION

Response to 75 face images of the other 12 people=900 PSRs

PSR

28


PIE Database Illumination Variations

! Simulations using 65 people from the Pose, Illumination and Expression (PIE) Database.

! Each person (with and without background lighting) has 21/22 face images respectively at frontal view.

29


49 Faces from PIE Database illustrating the variations in illum ination

30


Training Image selection

! We used three face images to synthesize a correlation filter ! The three selected training images consisted of 3 extreme

cases (dark left half face, normal face illumination, dark righthalf face).

n = 3 n = 7 n = 16

33


Reject Reject

AuthenticateAuthenticateThresholdThreshold

EER using Filter with Background illumination

36


Iris Verification

! High-quality iris images yield low error rates

! Correlation filters yield zero verification errors for the 9 iris images

! Challenge is to acquire high-quality iris images

Source: National Geographic Magazine

Source: Dr. J. Daugmans web site

37


Features of Correlation Filters

! Shift-invariant; no need for centering the test image! Graceful degradation! Can handle multiple appearances of the reference image in

the test image! Closed-form solutions based on well-defined metrics

48


Motivation for the Book

! Most pattern recognition researchers are not able to take advantage of the power of correlation filters because of the diverse background needed! Signals and systems

! Probability theory and random variables

! Linear algebra! Optical processing

! Digital signal processing

! Detection and estimation theory

! Goal of the book: To provide the background and techniques for correlation pattern recognition and illustrate with applications.

49


Book Chapters

! Introduction! Mathematical background ! Signals and systems! Detection theory! Basic correlation filters! Advanced correlation filters! Optics basics! Optical correlators! Application examples

50


Book Status

! Co-authors! Dr. AbhijitM ahalanobis, Lockheed M artin

! Dr. Richard Juday, NASA Johnson Space Center (Retired)

! All nine chapters written! References and final editing being done! To be published by Cambridge University Press! Should come out in late 2004

Documents

Title Goes Here Correlation Pattern Recognitionkumar/DowdSeminar.pdf · 1 Vijayakumar Bhagavat ula Vijayakumar Bhagavatula Title Goes Here Correlation Pattern Recognition December