Rock, paper, and scissors Joint extrinsic and intrinsic similarity of non-rigid shapes

1Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

Rock, paper, and scissors

Joint extrinsic and intrinsic similarity of non-rigid shapes

Alex Bronstein, Michael Bronstein, Ron Kimmel

Department of Computer ScienceTechnion – Israel Institute of Technology

2Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

Extrinsic vs intrinsic similarity

Intrinsic similarity

Are the shapes congruent? Do the shapes have the

same

metric structure?

Extrinsic similarity

Rock, paper, and scissors: is the hand similar to a rock? Is it similar to

another posture of a hand?

The answer depends on the definition of similarity.

3Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

Extrinsic similarity

Can be expressed as a distance between two shapes and

Find a rigid motion bringing the shapes into best alignment

Misalignment is quantified using the Hausdorff distance

or some of its variants

Computed using ICP algorithms

4Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

Extrinsic similarity – limitations

Extrinsically similar Extrinsically dissimilar

Suitable for nearly rigid shapes Unsuitable for non-rigid

shapes

5Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

Intrinsic similarity

Compare the intrinsic geometries of two shapes

Intrinsic geometry is expressed in terms of geodesic distances

Geodesic distances are computed using Dijkstra’s shortest path

algorithm or fast marching

Euclidean distance

Geodesic distance

6Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

Intrinsic similarity – canonical forms

Embed intrinsic geometries of and into a common metric space

Minimum-distortion embeddings and computed using

multidimensional scaling (MDS) algorithms

Compare the images and as rigid shapes

A. Elad, R. Kimmel, CVPR 2001

7Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

Intrinsic similarity – GMDS

Find the minimum distortion embedding of one shape into the other

The minimum distortion is the measure of intrinsic dissimilarity

Computed using the generalized MDS

BBK, PNAS 2006

8Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

Intrinsic similarity – limitations

Intrinsically dissimilar

Intrinsically similar

Suitable for near-isometric

shape deformations

Unsuitable for deformations

modifying shape topology

9Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

Extrinsically dissimilarIntrinsically similar

Extrinsically similarIntrinsically dissimilar

Extrinsically dissimilarIntrinsically dissimilar

THIS IS THE SAME SHAPE!

Desired result:

10Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

Joint extrinsic and intrinsic similarity

Combine intrinsic and extrinsic similarities into a single criterion

Find a deformation of whose intrinsic geometry is similar to

and extrinsic geometry is more similar to

defines the relative importance of intrinsic and extrinsic criteria

is a collection of optimal tradeoffs between intrinsic and

extrinsic criteria

Can be formalized using the notion of Pareto optimality

11Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

Intrinsic similarity

Ext

rinsi

c si

mila

rity

12Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

Computation of joint similarity

Hybridization of ICP and GMDS in L2 formulation for robustness

Fix correspondence between and for intrinsic similarity

where is precomputed and

are computed at each iteration

Closest-point distance for extrinsic similarity

where are the closest points to in

More details in the paper

13Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

Numerical example – dataset

= topology changeData: tosca.cs.technion.ac.il

14Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

Numerical example – tradeoff curves

Dissi

mila

r

Simila

r

15Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

Numerical example – intrinsic similarity

= topology-preserving no topology changes

16Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

Numerical example – intrinsic similarity

= topology change= topology-preserving

17Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

Numerical example – extrinsic similarity

= topology change= topology-preserving

18Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

Numerical example – joint similarity

= topology change= topology-preserving

19Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

Numerical example – ROC curves

20Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

Numerical example – shape morphing

Stronger intrinsic similarity (smaller λ)

Stronger extrinsicsimilarity (larger λ)

21Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

Conclusion

Extrinsic similarity is insensitive to topology changes, but sensitive to

non-rigid deformations

Intrinsic similarity is insensitive to nearly-isometric non-rigid

deformations, but sensitive to topology changes

Joint similarity is insensitive to both non-rigid deformations and topology

changes

Can be used to produce near-isometric morphs

22Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

References

A. M. Bronstein, M. M. Bronstein, A. M. Bruckstein, R. Kimmel, Analysis of two-dimensional non-rigid shapes, IJCV, to appear.

A. M. Bronstein, M. M. Bronstein, R. Kimmel, Rock, Paper, and Scissors: extrinsic vs. intrinsic similarity of non-rigid shapes, Proc. ICCV, (2007).

I. Eckstein, J. P. Pons, Y. Tong, C. C. J. Kuo, and M. Desbrun, Generalized surface flows for mesh processing, Proc. SGP, (2007).

M. Kilian, N. J. Mitra, and H. Pottmann, Geometric modeling in shape space, Proc. SIGGRAPH, vol. 26, (2007).

A. M. Bronstein, M. M. Bronstein, A. M. Bruckstein, R. Kimmel, Paretian similarity for partial comparison of non-rigid objects, Proc. SSVM, pp. 264-275, 2007.

A. M. Bronstein, M. M. Bronstein, R. Kimmel, Calculus of non-rigid surfaces for geometry and texture manipulation, IEEE TVCG, Vol. 13/5, pp. 902-913, (2007).

A. M. Bronstein, M. M. Bronstein, R. Kimmel, Generalized multidimensional scaling: a framework for isometry-invariant partial surface matching, PNAS, Vol. 103/5, pp. 1168-1172, (2006).

23Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

References

F. Mémoli and G. Sapiro, A theoretical and computational framework for isometry invariant recognition of point cloud data, Foundations of Computational Mathematics 5 (2005), 313-346.

N. J. Mitra, N. Gelfand, H. Pottmann, and L. Guibas, Registration of point cloud data from a geometric optimization perspective, Proc. SGP, (2004), pp. 23-32.

A. Elad, R. Kimmel, On bending invariant signatures for surfaces, Trans. PAMI 25 (2003), no. 10, 1285-1295.

P. J. Besl and N. D. McKay, A method for registration of 3D shapes, Trans. PAMI 14 (1992), 239-256.

Y. Chen and G. Medioni, Object modeling by registration of multiple range images, Proc. Conf. Robotics and Automation, (1991).

E. L. Schwartz, A. Shaw, and E. Wolfson, A numerical solution to the generalized mapmaker's problem: flattening nonconvex polyhedral surfaces, Trans. PAMI 11 (1989), 1005-1008.

24Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes

Shameless advertisement

COMING SOON…

Published by Springer Verlag

To appear in early 2008

Approximately 320 pages

Over 50 illustrations

Color figures

Book website

tosca.cs.technion.ac.il