Computer Vision November 2003 L1.1© 2003 by Davi Geiger

Shape Representation and SimilarityOcclusion Articulation

Shape similarities should be preserved under occlusions and articulation.

Two equal stretching along contours can affect shape similarity differently.

Shapes can be represented by contours, or by a set of interior points, or other ways. What is the shape representation that can be best used for shape similarity?

Stretching Stretching

Computer Vision November 2003 L1.2© 2003 by Davi Geiger

Shape Axis (SA)

SA-Tree

Shape

Symmetry Axis Representation

Computer Vision November 2003 L1.3© 2003 by Davi Geiger

A variational shape representation model based on self-

similarity of shapes.

For each shape contour, first compute its shape axis then derive a unique shape-axis-tree (SA-tree) or shape-axis-forest (SA-forest) representation.

Shape Axis (SA) SA-TreeShape Contour

Shape Representation via Self-SimilarityBased on work of Liu, Kohn and Geiger

Computer Vision November 2003 L1.4© 2003 by Davi Geiger

Use Use twotwo different parameterizations to compute the different parameterizations to compute the represention of a shape.represention of a shape.

Construct a cost functional to measure the Construct a cost functional to measure the goodnessgoodness of a of a match between the two parameterizations.match between the two parameterizations.

The cost functional is decided by the self-similarity criteria The cost functional is decided by the self-similarity criteria of of symmetrysymmetry..

The Insight

counterclockwise clockwise

Computer Vision November 2003 L1.5© 2003 by Davi Geiger

Two different parameterizations:Two different parameterizations:

counterclockwisecounterclockwise

clockwiseclockwise

When the curve is closed we haveWhen the curve is closed we have

Matching the curves is described as the match of functionsMatching the curves is described as the match of functions

Parameterized Shapes

}10:)({1 ssx

}10:)(~{2 ttx )1()(~: txtxnote

(1)x~(0)x~ and )1()0( xx

))((x~ ))(x(or[0,1];)()( tsts

Computer Vision November 2003 L1.6© 2003 by Davi Geiger

A Global Optimization Approach We seek “good” matches over the possible We seek “good” matches over the possible

correspondencescorrespondences

[0,1]where))((x~ ))(x(or)()( tsts

)1()(~: txtxnote

Computer Vision November 2003 L1.7© 2003 by Davi Geiger

Co-Circularity

Mirror Symmetry

Similarity criteria: Symmetry

)('))((~)()(~))((~

)())(()()())((

where

)())((~))((

))((~)((

)())((~))((

))((~)((

of valuessmall give shoulderror y thisIntuitivel

ror.measure/ersymmetry

a define nowcan but we symmetry,perfect

havelonger nomay we~ tangent extra the

Given y.circularit-osymmetry/cmirror defines

~point another and, tangent its ,point One

ttd

dt

dt

txd

d

txd

ssd

ds

ds

sdx

d

sdx

bd

txd

d

sdxtxsx

ad

txd

d

sdxtxsx

xx

)(~ t

)(~ tx

)(~ t

)(~ t

)(~ tx)(~ t

)(sx)(s

)(~ tx

)(sx)(s

)(~ tx

Computer Vision November 2003 L1.8© 2003 by Davi Geiger

nt vectorsunit tange oriented are

))((~and))((where),()( escorresponc possible allover

)(),(,))((~)),((x~)),(()),(x( ))(t,)E(s(

Minimize1

0

tstσs

dtsttssF

A Global Optimization Approach

Computer Vision November 2003 L1.9© 2003 by Davi Geiger

Constraints on the form of F

).( and)( maps of choice on the

not and)()(match on theonly dependscost that theassuring

~)~(),~(,~,x~,,x

~,~,~,x~,,x~~,,~,x~,,x

)(),(,~,x~,,x

)(~ :ationparametriz of changeany under invarianceobtain To

)(),(,~,x~,,x)(),(,~,x~,,x Condition Scaling 2.

roles equivalentplay ~

and i.e.,

)(),(,,x,~,x~)(),(,~,x~,,xmatch theofSymmetry 1.

1

0~

1

0

1

0

1

0

ts

ts

dtsF

dd

d

d

dt

d

d

d

dsFd

d

d

d

dt

d

dsF

dtsF

tsFtsF

stFtsF

Computer Vision November 2003 L1.10© 2003 by Davi Geiger

Constraints on the form of F (cont.)

. )(amount known andt independen

shape aby cost thechange and matching thepreserve will

factorcommon by the ~ sand scaling Thus, ).(function somefor

)(),(,,~,x~,,x)()(),(,~,x~,,x

Invariance Scaling 4.

cost affect thenot willations transformrigid that so

)(),(,,~,x~,,x)(),(,~,x~,,x

InvarianceRotation 3.

g

g

tsFgtsF

Partial

tsFtsRRRRF

Computer Vision November 2003 L1.11© 2003 by Davi Geiger

1

02

2

1

0

1

0

)(

)(')(

)('))((~)())(())((~))((

)(')(

)('))((~)())(())((~)((

)(),(,~,x~,,x

as defined is encecorrespondany for cost symmetry The

).)( giveslly automatica(which )( mappings possible allover

cost theminize we)2constraint()(zation parameteriarbitrary any for where

)(),(,~,x~,,xmin)~

,(

as measuresymmetry thewrite We

d

ts

ttsstxsx

ts

ttsstxsx

dtsF

stt

s

dtsFS t

Similarity criteria: Symmetry

Computer Vision November 2003 L1.12© 2003 by Davi Geiger

Similarity criteria: Symmetry II

(a)

(b)

dFstE JumpCost )()similarity-self())(),((1

0

solution theof cost) (jump nsbifurcatiofor cost a add we(d), and

(c) see parts,object for nsbifurcatio allow andty irregulari shape todues

nbifurcatio ofcreation theavoid To small. be should parameter The

)(')(

)(')())((~))((

)(')(

)('))((~)())(())((~))((

)(')(

)('))((~)())(())((~)((

)(),(,~,x~,,x

solutions (b)over (a) bias topoints, closed

between encescorrespond bias to term thirda add We

1

0

22

2

2

1

0

c

d

ts

tstxsxc

ts

ttsstxsx

ts

ttsstxsx

dtsF

(c)

(d)

Computer Vision November 2003 L1.13© 2003 by Davi Geiger

Structural propertiesStructural propertieso SymmetricSymmetrico Parameterization IndependentParameterization Independent

Geometrical propertiesGeometrical propertieso Translation invariantTranslation invarianto Rotation invariantRotation invarianto Scale “almost” invarianceScale “almost” invariance

Self-Similarity propertiesSelf-Similarity properties

Summary: Cost Functional/Energy Density

Computer Vision November 2003 L1.14© 2003 by Davi Geiger

A Dynamic Programming Solution

Computer Vision November 2003 L1.15© 2003 by Davi Geiger

A Dynamic Programming

SolutionThe recurrence is on the “square boxes”. Each box is either

linked to one inside square box or linked to

a pair of inside and well aligned square

boxes (leading to bifurcations).

Assumption, that will not be needed later: The final state is a

match (0,0).

…?

!!

!!

!!

bifurcation

bifurcationt

s

1-s

1-t

tt

1-tts+1/N

t+1/N

!!

! !! !

Computer Vision November 2003 L1.16© 2003 by Davi Geiger

)],(),1

[(tt],1

BestCost[smin

)],()1

,[(]1

ts,BestCost[smin

Jumpt]tt,-BestCost[1tt],BestCost[smin

min

t],BestCost[s

}1

1,...,1

{tt

}1

1,...,1

{ss

}1,1

1,...,1

,{

tsttN

sFN

tsN

tssFN

Ns

Nt

Nt

Ns

sN

sN

tttt

t

1-s

tt

t+1/N …?

!!

!

!

!!!

s 1-t1-tts+1/N

A Dynamic Programming Solution

s

t

Computer Vision November 2003 L1.17© 2003 by Davi Geiger

A Dynamic Programming Solution

A few more steps …

iteration direction

s

t

Starting states

(matching candidates).

All corresponding to

“self-matches”

Computer Vision November 2003 L1.18© 2003 by Davi Geiger

A General Dynamic Programming Solution

No need to force a match at (0,0) or any particular node/match.

s

t

Extended Graph

Starting states

(matching candidates).

All corresponding to

“self-matches”

Ending states

Extend the parameters s and t such that negative values x s,t=1+x.

x

x

Computer Vision November 2003 L1.19© 2003 by Davi Geiger

A Dynamic Programming Solution

Computer Vision November 2003 L1.20© 2003 by Davi Geiger

Experimental Results for Closed Shapes

Shape Axis

SA-Tree

Shape

Computer Vision November 2003 L1.21© 2003 by Davi Geiger

Experimental Results for Open Shapes(the first and last points are assumed to be a match)

Shape Axis (SA)

SA-Tree

Shape

Computer Vision November 2003 L1.22© 2003 by Davi Geiger

Shape Axis (SA)

SA-Forest

Experimental Results for Open Shapes

Computer Vision November 2003 L1.23© 2003 by Davi Geiger

Matching Trees with deletions and merges for articulation and occlusions

Computer Vision November 2003 L1.24© 2003 by Davi Geiger

Experimental Results for Open Shapes

Computer Vision November 2003 L1.25© 2003 by Davi Geiger

Convexity v.s. Symmetry

White Convex Region Black Convex Region