Shape-Based Retrieval of Articulated3D Models Using Spectral Embedding
GrUVi Lab, School of Computing Science
Simon Fraser University, Burnaby, BC Canada
Varun Jain and Hao Zhang{vjain,haoz}@cs.sfu.ca
Problem OverviewShape Retrieval
Applications Computer aided design Game design Shape recognition Face recognition
Database
Output
User Interface
Outline
Problem OverviewRetrieval Problem
Methods
Spectral Embeddings
Shape Descriptors
Results
Future Work
Acknowledgements
…
Query
…
Shape Retrieval
How? Using Correspondence
Efficiency??
Using Global Descriptors
For 3D shapes• Fourier Descriptors• Light Field• Spherical Harmonics• Skeletal Graph Matching
Outline
Problem OverviewRetrieval Problem
Methods
Spectral Embeddings
Shape Descriptors
Results
Future Work
Acknowledgements
Non-rigid Non-rigid transformations??transformations??•StretchingStretching•Articulation (bending)Articulation (bending)
Shape Retrieval
Our MethodNormalize non-rigid transformations
• Construct affinity matrix• Spectral embedding
Use global shape descriptors• Light Field Descriptor (LFD)• Spherical Harmonics Descriptor (SHD)• Eigenvalues??
Outline
Problem OverviewRetrieval Problem
Methods
Spectral Embeddings
Shape Descriptors
Results
Future Work
Acknowledgements
Shape Retrieval
Advantages of Our Method
Handles shape articulation (best performance for articulated shapes).
Flexibility of affinity matrices
Robustness of affinity matrices
Outline
Problem OverviewRetrieval Problem
Methods
Spectral Embeddings
Shape Descriptors
Results
Future Work
Acknowledgements
Spectral EmbeddingsAffinity matrixOutline
Problem Overview
Spectral EmbeddingsBasics
Problems
Solutions
Shape Descriptors
Results
Future Work
Acknowledgements
>>:4
¢¢¢
n
An£
n = n
8>>>>>>>><
>>>>>>
2
66666666
z }| {a11 a12
¢¢¢a1n
a21
¢¢¢......
...ai1 ai2
¢¢¢ain
......
...an1 an2 ann
3
777777775
aaii jj == eeggeeoodd (( ii ;; jj )) 22
22¾¾22
Spectral EmbeddingsEigenvalue decomposition:
Scaled eigenvectors:
Jain, V., Zhang, H.: Robust 3D Shape Correspondence in the Spectral Domain. Proc. Shape Modeling International 2006.
Outline
Problem Overview
Spectral EmbeddingsBasics
Problems
Solutions
Shape Descriptors
Results
Future Work
Acknowledgements
A = E ¤E T
k-dimensional spectral embedding coordinates of ith point of P
Uk =Ek¤12k =n
8>>>>>>>>>><
>>>>>>>>>>:
2
66666666664
kz }| {~u1 ~u2 ¢¢¢ ~uk
......
...ui1 ui2 ¢¢¢ uik...
......
3
77777777775
¯̄¯̄¯̄¯̄¯̄¯̄¯̄¯̄
¯̄¯̄¯̄¯̄¯̄¯̄¯̄¯̄
¯̄¯̄¯̄¯̄¯̄¯̄¯̄¯̄
Spectral EmbeddingsExamples of 3D embeddings
Outline
Problem Overview
Spectral EmbeddingsBasics
Problems
Solutions
Shape Descriptors
Results
Future Work
Acknowledgements
¯̄¯̄¯̄¯̄¯̄¯̄¯̄¯̄
¯̄¯̄¯̄¯̄¯̄¯̄¯̄¯̄
U3 = n
8>>>>>>>>>><
>>>>>>>>>>:
2
66666666664
~u2 ~u3 ~u4
......
...ui2 ui3 ui4...
......
3
77777777775
EigenValue Descriptor (EVD):
Use deviation in projected data as descriptor:
Our shape descriptor:
¾i =³
ku i k2
n
´ 12
=p
¸ ir P
i¸ i
Spectral EmbeddingsOutline
Problem Overview
Spectral EmbeddingsBasics
Problems
Solutions
Shape Descriptors
Results
Future Work
Acknowledgements
f¾1;¾2; : : : ;¾kg
Why use eigenvalues??
Spectral EmbeddingsOutline
Problem Overview
Spectral EmbeddingsBasics
Problems
Solutions
Shape Descriptors
Results
Future Work
Acknowledgements
AA==
22
666666666666666644
aa1111 aa1122 ¢¢¢¢¢¢ aa11nn
aa2211 ¢¢¢¢¢¢......
............
aaii11 aaii22 ¢¢¢¢¢¢ aaiinn......
............
aann11 aann22 ¢¢¢¢¢¢ aannnn
33
777777777777777755
BB ==
22
666666666666666644
bb1111 bb1122 ¢¢¢¢¢¢ bb11nnbb2211 ¢¢¢¢¢¢......
............
bbii11 bbii22 ¢¢¢¢¢¢ bbiinn......
............
bbnn11 bbnn22 ¢¢¢¢¢¢ bbnnnn
33
777777777777777755
RR == AAAATT
RREE == EE ¤¤
RR == AA22
Spectral Embeddings
Problems:
Geodesic distance computation
Efficiency of geodesic distance computation & eigendecomposition:
Outline
Problem Overview
Spectral EmbeddingsBasics
Problems
Solutions
Shape Descriptors
Results
Future Work
Acknowledgements
OO((nn22 llooggnn)) ++ OO((kknn22))
Spectral Embeddings
Geodesics using Structural Graph: Add edges to make mesh connected Geodesic distance ≈ Shortest graph distance
Problem: Unwanted (topology modifying) edges!
Solution: Add shortest possible edges.
Choice of graph to take edges from: p-nearest neighbor (may not return connected
graph) p-edge connected [Yang 2004]
Outline
Problem Overview
Spectral EmbeddingsBasics
Problems
Solutions
Shape Descriptors
Results
Future Work
Acknowledgements
Spectral EmbeddingsEfficiency with Nyström approximation
Outline
Problem Overview
Spectral EmbeddingsBasics
Problems
Solutions
Shape Descriptors
Results
Future Work
Acknowledgements
AAnn££nn ==nn
88>>>>>>>>>>>>>>>><<
>>>>>>>>>>>>>>>>::
22
666666666666666644
nnzz }}|| {{aa1111 aa1122 ¢¢¢¢¢¢ aa11nn
aa2211 ¢¢¢¢¢¢......
............
aaii11 aaii22 ¢¢¢¢¢¢ aaiinn......
............
aann11 aann22 ¢¢¢¢¢¢ aannnn
33
777777777777777755
EE ll££ ll ==
22
666644
............
~~uu11 ¢¢¢¢¢¢ ~~uu ll......
......
33
777755
¯̄̄̄̄̄¯̄̄̄̄̄¯̄̄̄̄̄¯̄̄̄̄̄¯̄̄̄̄̄¯̄
¯̄̄̄̄̄¯̄̄̄̄̄¯̄̄̄̄̄¯̄̄̄̄̄¯̄̄̄̄̄¯̄
EE nn££ 33==nn
88>>>>>>>>>>>><<
>>>>>>>>>>>>::
22
66666666666644
............
......~~uu22 ~~uu33 ~~uu44......
............
33
77777777777755
¯̄̄̄¯̄̄̄¯̄̄̄¯̄̄̄¯̄̄̄¯̄̄̄¯̄̄̄¯̄̄̄
¯̄̄̄¯̄̄̄¯̄̄̄¯̄̄̄¯̄̄̄¯̄̄̄¯̄̄̄¯̄̄̄
eexxttrraappoollaattiioonnOO((llnn))
mmaaxx--mmiinnssuubbssaammpplliinnggOO((llnn llooggnn))
AAll££ ll ==ll
88>>>><<
>>>>::
22
666644
nnzz }}|| {{aa1111 aa1122 ¢¢¢¢¢¢ aa11nn
aa2211 aa2222 ¢¢¢¢¢¢ aa22nn......
............
aall11 aall22 ¢¢¢¢¢¢ aallnn
33
777755
eeiiggeennddeeccoommppoossiittiioonn
OO((ll33))
Shape Descriptor
Global Shape Descriptors
Light Field Descriptor (LFD)
Spherical Harmonics Descriptor (SHD)
Our Similarity Measure (EVD):
Outline
Problem Overview
Spectral Embeddings
Shape Descriptors
Results
Future Work
Acknowledgements
SSiimmCCoosstt((PP;;QQ)) ==1122
kkXX
ii ==11
££pp¸̧ ii ¡¡
pp°°ii
¤¤22
pp¸̧ ii ++
pp°°ii
Experimental DatabaseMcGill 3D Articulated Shapes Databasehttp://www.cim.mcgill.ca/~shape/benchMark/
Outline
Problem Overview
Spectral Embeddings
Shape Descriptors
Results
Future Work
Acknowledgements
ResultsPrecision-Recall plot for McGill databasePrecision-Recall plot for McGill database
Outline
Problem Overview
Spectral Embeddings
Shape Descriptors
Results
Future Work
Acknowledgements
ResultsMcGill articulated shape databaseMcGill articulated shape database
Outline
Problem Overview
Spectral Embeddings
Shape Descriptors
Results
Future Work
Acknowledgements
Non-robustness of geodesic distances
Non-robustness to outliers
Limitations & Future WorkOutline
Problem Overview
Spectral Embeddings
Shape Descriptors
Results
Future Work
Acknowledgements
Acknowledgements
McGill 3D Shape Benchmark.
Phil Shilane (LFD & SHD implementations).
Outline
Problem Overview
Spectral Embeddings
Shape Descriptors
Results
Future Work
Acknowledgements