Likelihood Models for Template MatchingUsing the PDF Projection Theorem

Arasanathan Thayananthan

Ramanan Navaratnam

Dr. Phil Torr

Prof. Roberto Cipolla

ProblemProblem

The correct template

The minimum chamfer score

Chamfer score 3.49 3.07

OverviewOverview

1. Problem Motivation

2. PDF Projection Theorem

3. Likelihood Modelling for Chamfer Matching

4. Experiments

5. Conclusion

MotivationMotivation

Template matching widely used in computer vision

Similarity measures are obtained from matching a template to a new image e.g. chamfer score, cross-correlation, etc.

A likelihood value need to be calculated from the similarity measures.

Chamfer score 3.58

Likelihood ?

MotivationMotivation

Is the similarity measure alone enough to calculate the likelihood ?

What are the probabilities of matching to a correct image and an incorrect image at this specific matching measure ?

Feature LikelihoodFeature Likelihood

Feature likelihood distributions, obtained by matching the templates to the real images they represent

They differ according to the shape and scale of the templates.

Feature LikelihoodsFeature Likelihoods

chamfer 6.0

likelihood 0.14 likelihood 0.03

Clutter LikelihoodsClutter Likelihoods

Clutter likelihood distributions are obtained by matching the template to the background clutter

Likelihood RatiosLikelihood Ratios

The ratio of the feature and clutter likelihood provides a robust likelihood measure.

Likelihood Ratio Tests (LRT) are often used in many classification problems

Jones & Ray [99], skin-colour classification

Sidenbladh & Black [01], limb-detector

Modelling the likelihoodModelling the likelihood

Need a principled framework for modelling the likelihood for template matching

Probability Distribution Function Projection Theorem ( Baggenstoss [99]) provides such a framework

OverviewOverview

1. Problem Motivation

2. PDF Projection Theorem

3. Likelihood Modelling for Chamfer Matching

4. Experiments

5. Conclusion

PDF Projection TheoremPDF Projection Theorem

Provides a mechanism to work in raw data space, I, instead of extracted feature space, z.

This is done by projecting the PDF estimates from the feature space back to the raw data space

PDF Projection TheoremPDF Projection Theorem

Neyman-Fisher factorisation states that if is a sufficient statistic for H, p(I|H) can be factored as

Applying Eq(1) for a hypothesis, H, and a reference Hypothesis, H0,

PDF Projection TheoremPDF Projection Theorem

Image space, I

I

PDF Projection TheoremPDF Projection Theorem

Image space, I Feature space, z

I z

PDF Projection TheoremPDF Projection Theorem

Image space, I Feature space, z

I z

Class-specific featuresClass-specific features

PDF Projection Theorem extends to class-specific features

Each hypothesis or class can have its feature set

Yet, we get consistent and comparable raw image likelihoods

Reference hypothesis H0 remains the same for all hypothesis

Class-specific featuresClass-specific features

I

OverviewOverview

1. Problem Motivation

2. PDF Projection Theorem

3. Likelihood Modelling for Chamfer Matching

4. Experiments

5. Conclusion

Chamfer MatchingChamfer Matching

Input image Canny edges

Distance transform Template

Chamfer MatchingChamfer Matching

We apply PDF projection Theorem to model likelihood in a chamfer matching scheme

Each template chooses its own subset of edge features, zj

Chamfer MatchingChamfer Matching

A common reference hypothesis is chosen for all templates

p(zj|H0) provides the probability of template matching to any image.

Difficulty is in learning p(zj|Hj) and p(zj|H0) for each template Tj

Learning the PDFsLearning the PDFs

Time-consuming to obtain real images for learning the PDFs

Software like “Poser” can create “near” real images

Becoming popular for learning image statistics e.g. Shakhnarovich [03]

For each template Tj, we learn p(zj|Hj) and p(zj|H0) from synthetic images.

Learning the PDFsLearning the PDFs

Example learning images for the template

For learning the feature likelihood p(zj|Hj)

For learning the reference likelihood p(zj|H0)

OverviewOverview

1. Problem Motivation

2. PDF Projection Theorem

3. Likelihood Modelling for Chamfer Matching

4. Experiments

5. Conclusion

ExperimentsExperiments

35 hand templates from a 3D hand model with 5 gestures at 7 different scales

Hypothesis, Hj, is that the image contains a hand pose similar to Template Tj, (in scale and gesture).

The distributions p(zj|Hj) and p(zj|H0) were learned off-line for each template.

ExperimentsExperiments

Aim of the experiment is to compare the matching performances of1. Zj, the chamfer score obtained by matching

the template Tj to the image

2. P(zj|Hj), the feature likelihood of Template Tj

3. P(I|Hj), the data likelihood value using the PDF projection theorem.

ExperimentsExperiments

Template matching on 1000 randomly created synthetic images.

Each synthetic image contains a hand pose similar in scale and pose to a randomly chosen template.

Three ROC curves were obtained for each matching measure.

ResultsResults

ResultsResults

PDF ProjectionTheorem

Chamfer

Chamfer score 4.96 4.06

feature likelihood 14.59 x 10-2 8.62 x 10-2

reference likelihood 88.69 x 10-5 383.32 x 10-5

data likelihood 0.164 x 103 0.022 x 103

ResultsResults

PDF ProjectionTheorem

Chamfer

Chamfer score 4.96 4.06

feature likelihood 14.59 x 10-2 8.62 x 10-2

reference likelihood 88.69 x 10-5 383.32 x 10-5

data likelihood 0.164 x 103 0.022 x 103

ResultsResults

PDF ProjectionTheorem

Chamfer

Chamfer score 4.96 4.06

feature likelihood 14.59 x 10-2 8.62 x 10-2

reference likelihood 88.69 x 10-5 383.32 x 10-5

data likelihood 0.164 x 103 0.022 x 103

ResultsResults

PDF ProjectionTheorem

Chamfer

Chamfer score 4.96 4.06

feature likelihood 14.59 x 10-2 8.62 x 10-2

reference likelihood 88.69 x 10-5 383.32 x 10-5

data likelihood 0.164 x 103 0.022 x 103

ResultsResults

PDF ProjectionTheorem

Chamfer

Chamfer score 3.49 3.07

feature likelihood 24.94x 10-2 27.88 x 10-2

reference likelihood 4.73 x 10-5 24.7 x 10-5

data likelihood 5.27 x 103 1.126 x 103

ResultsResults

PDF ProjectionTheorem

Chamfer

Chamfer score 3.49 3.07

feature likelihood 24.94x 10-2 27.88 x 10-2

reference likelihood 4.73 x 10-5 24.7 x 10-5

data likelihood 5.27 x 103 1.126 x 103

ResultsResults

PDF ProjectionTheorem

Chamfer

Chamfer score 3.49 3.07

feature likelihood 24.94x 10-2 27.88 x 10-2

reference likelihood 4.73 x 10-5 24.7 x 10-5

data likelihood 5.27 x 103 1.126 x 103

ResultsResults

PDF ProjectionTheorem

Chamfer

Chamfer score 3.49 3.07

feature likelihood 24.94x 10-2 27.88 x 10-2

reference likelihood 4.73 x 10-5 24.7 x 10-5

data likelihood 5.27 x 103 1.126 x 103

ResultsResults

PDF ProjectionTheorem

Chamfer

Chamfer score 3.72 3.54

feature likelihood 13.15 x 10-2 20.5 x 10-2

reference likelihood 8.5 x 10-5 108.0 x 10-5

data likelihood 1.547 x 103 0.191 x 103

ConclusionConclusion

Depending on raw matching score is less reliable in template matching

PDF Projection theorem provides a principled framework for modelling the likelihood in raw image data space.

Consistent and comparable likelihoods obtained through PDF projection theorem improves the efficiency of template matching scheme