Visual Similarity Measures for Images

8/3/2019 Visual Similarity Measures for Images

1/45

Similarity and Difference

Pete Barnum

January 25, 2006

Advanced Perception


2/45

Visual Similarity

Color Texture


3/45

Uses for

Visual Similarity Measures

Classification Is it a horse?

Image Retrieval Show me pictures of horses.

Unsupervised segmentation Which parts of the image are grass?


4/45


5/45

CumulativeH

istogram

Normal

Histogram

Cumulative

Histogram

Slides from Dave Kauchak


6/45

Joint vs MarginalH

istograms

Images from Dave Kauchak


7/45

Joint vs MarginalH

istograms

Images from Dave Kauchak


8/45

Adaptive Binning


9/45

Clusters (Signatures)


10/45

Higher Dimensional

Histograms

Histograms generalize to any number

of features

Colors

Textures

Gradient

Depth


11/45

Distance Metrics

x

y

x

y

-

-

-

= Euclidian distance of 5 units

= Grayvalue distance of 50 values

= ?


12/45

Bin-by-bin

Good!

Bad!


13/45

Cross-bin

Good!

Bad!


14/45

Distance Measures Heuristic

Minkowski-form

Weighted-Mean-Variance (WMV)

Nonparametric test statistics G 2 (Chi Square) Kolmogorov-Smirnov (KS)

Cramer/von Mises (CvM)

Information-theory divergences Kullback-Liebler (KL)

Jeffrey-divergence (JD) Ground distance measures

Histogram intersection

Quadratic form (QF)

Earth Movers Distance (EMD)


15/45

Heuristic

Histogram Distances

Minkowski-form distance Lp

Special cases:

L1: absolute, cityblock, orManhattan distance

L2: Euclidian distance

Lg: Maximum value distance

p

i

p

JifIifJID

/1

),(),(),(

!



16/45

MoreH

euristic Distances

!

r

rr

r

rJIJI

JIDrr

WW

WW

QW

QQ),(


Weighted-Mean-Variance

Only includes minimal information about

the distribution


17/45

Nonparametric Test Statistics

G2 Measures the underlying similarity of two

samples

? A 2/;;,

;,

2

JifIififif

ifIifJID

i

!

!

Images from Kein Folientitel


18/45


Kolmogorov-Smirnov distance Measures the underlying similarity of two samples

Only for 1D data


19/45


Kramer/von Mises Euclidian distance

Only for 1D data


20/45

Information Theory

Kullback-Liebler Cost of encoding one distribution as another


21/45

Information Theory

Jeffrey divergence Just like KL, but more numerically stable


22/45

Ground Distance

Histogram intersection Good for partial matches


23/45

Ground Distance

Quadratic form Heuristic

JIt

JIJID ffAff, !



24/45

Ground Distance

Earth Movers Distance


!

ji

ij

ji

ijij

g

dg

JID

,

,,


25/45

Summary



26/45

Moving Earth


27/45

Moving Earth


28/45

Moving Earth

=


29/45

The Difference?

=

(amount moved)


30/45

The Difference?

=

(amount moved) * (distance moved)


31/45

Linear programming

m clusters

n clusters

P

Q All movements

(distance moved) * (amount moved)


32/45

Linear programming

m clusters

n clusters

P

Q

(distance moved) * (amount moved)


33/45

Linear programming

m clusters

n clusters

P

Q

* (amount moved)


34/45

Linear programming

m clusters

n clusters

P

Q


35/45

Constraints

m clusters

n clusters

P

Q

1. Move earth only from P to Q

P

Q


36/45

Constraints

m clusters

n clusters

P

Q

2. Cannot send more earth than

there is

P

Q


37/45

Constraints

m clusters

n clusters

P

Q

3. Q cannot receive more earth

than it can hold

P

Q


38/45

Constraints

m clusters

n clusters

P

Q

4. As much earth as possible must

be moved

P

Q


39/45

Advantages

Uses signatures

Nearness measure without

quantization

Partial matching

A true metric


40/45

Disadvantage

High computational cost

Not effective for unsupervised

segmentation, etc.


41/45

Examples

Using

Color (CIE Lab)

Color + XY

Texture (Gabor filter bank)


42/45

Image Lookup


43/45

Image Lookup

L1 distance

Jeffrey divergence

2 statistics

Quadratic form distance

Earth Mover Distance


44/45

Image Lookup


45/45

Concluding thought

-

-

-

= it depends on the application

Documents

Visual Similarity Measures for Images