Shape-Based Retrieval of Articulated3D Models Using Spectral Embedding

GrUVi Lab, School of Computing Science

Simon Fraser University, Burnaby, BC Canada

Varun Jain and Hao Zhang{vjain,haoz}@cs.sfu.ca

Problem Overview

Problem OverviewShape Retrieval

Applications Computer aided design Game design Shape recognition Face recognition

Database

Output

User Interface

Outline

Problem OverviewRetrieval Problem

Methods

Spectral Embeddings

Shape Descriptors

Results

Future Work

Acknowledgements

…

Query

…

Shape Retrieval

How? Using Correspondence

Efficiency??

Using Global Descriptors

For 3D shapes• Fourier Descriptors• Light Field• Spherical Harmonics• Skeletal Graph Matching

Outline

Problem OverviewRetrieval Problem

Methods

Spectral Embeddings

Shape Descriptors

Results

Future Work

Acknowledgements

Non-rigid Non-rigid transformations??transformations??•StretchingStretching•Articulation (bending)Articulation (bending)

Shape Retrieval

Our MethodNormalize non-rigid transformations

• Construct affinity matrix• Spectral embedding

Use global shape descriptors• Light Field Descriptor (LFD)• Spherical Harmonics Descriptor (SHD)• Eigenvalues??

Outline

Problem OverviewRetrieval Problem

Methods

Spectral Embeddings

Shape Descriptors

Results

Future Work

Acknowledgements

Shape Retrieval

Advantages of Our Method

Handles shape articulation (best performance for articulated shapes).

Flexibility of affinity matrices

Robustness of affinity matrices

Outline

Problem OverviewRetrieval Problem

Methods

Spectral Embeddings

Shape Descriptors

Results

Future Work

Acknowledgements

Spectral Embeddings

Spectral EmbeddingsAffinity matrixOutline

Problem Overview

Spectral EmbeddingsBasics

Problems

Solutions

Shape Descriptors

Results

Future Work

Acknowledgements

>>:4

¢¢¢

n

An£

n = n

8>>>>>>>><

>>>>>>

2

66666666

z }| {a11 a12

¢¢¢a1n

a21

¢¢¢......

...ai1 ai2

¢¢¢ain

......

...an1 an2 ann

3

777777775

aaii jj == eeggeeoodd (( ii ;; jj )) 22

22¾¾22

Spectral EmbeddingsEigenvalue decomposition:

Scaled eigenvectors:

Jain, V., Zhang, H.: Robust 3D Shape Correspondence in the Spectral Domain. Proc. Shape Modeling International 2006.

Outline

Problem Overview

Spectral EmbeddingsBasics

Problems

Solutions

Shape Descriptors

Results

Future Work

Acknowledgements

A = E ¤E T

k-dimensional spectral embedding coordinates of ith point of P

Uk =Ek¤12k =n

8>>>>>>>>>><

>>>>>>>>>>:

2

66666666664

kz }| {~u1 ~u2 ¢¢¢ ~uk

......

...ui1 ui2 ¢¢¢ uik...

......

3

77777777775

¯̄¯̄¯̄¯̄¯̄¯̄¯̄¯̄

¯̄¯̄¯̄¯̄¯̄¯̄¯̄¯̄

¯̄¯̄¯̄¯̄¯̄¯̄¯̄¯̄

Spectral EmbeddingsExamples of 3D embeddings

Outline

Problem Overview

Spectral EmbeddingsBasics

Problems

Solutions

Shape Descriptors

Results

Future Work

Acknowledgements

¯̄¯̄¯̄¯̄¯̄¯̄¯̄¯̄

¯̄¯̄¯̄¯̄¯̄¯̄¯̄¯̄

U3 = n

8>>>>>>>>>><

>>>>>>>>>>:

2

66666666664

~u2 ~u3 ~u4

......

...ui2 ui3 ui4...

......

3

77777777775

EigenValue Descriptor (EVD):

Use deviation in projected data as descriptor:

Our shape descriptor:

¾i =³

ku i k2

n

´ 12

=p

¸ ir P

i¸ i

Spectral EmbeddingsOutline

Problem Overview

Spectral EmbeddingsBasics

Problems

Solutions

Shape Descriptors

Results

Future Work

Acknowledgements

f¾1;¾2; : : : ;¾kg

Why use eigenvalues??

Spectral EmbeddingsOutline

Problem Overview

Spectral EmbeddingsBasics

Problems

Solutions

Shape Descriptors

Results

Future Work

Acknowledgements

AA==

22

666666666666666644

aa1111 aa1122 ¢¢¢¢¢¢ aa11nn

aa2211 ¢¢¢¢¢¢......

............

aaii11 aaii22 ¢¢¢¢¢¢ aaiinn......

............

aann11 aann22 ¢¢¢¢¢¢ aannnn

33

777777777777777755

BB ==

22

666666666666666644

bb1111 bb1122 ¢¢¢¢¢¢ bb11nnbb2211 ¢¢¢¢¢¢......

............

bbii11 bbii22 ¢¢¢¢¢¢ bbiinn......

............

bbnn11 bbnn22 ¢¢¢¢¢¢ bbnnnn

33

777777777777777755

RR == AAAATT

RREE == EE ¤¤

RR == AA22

Spectral Embeddings

Problems:

Geodesic distance computation

Efficiency of geodesic distance computation & eigendecomposition:

Outline

Problem Overview

Spectral EmbeddingsBasics

Problems

Solutions

Shape Descriptors

Results

Future Work

Acknowledgements

OO((nn22 llooggnn)) ++ OO((kknn22))

Spectral Embeddings

Geodesics using Structural Graph: Add edges to make mesh connected Geodesic distance ≈ Shortest graph distance

Problem: Unwanted (topology modifying) edges!

Solution: Add shortest possible edges.

Choice of graph to take edges from: p-nearest neighbor (may not return connected

graph) p-edge connected [Yang 2004]

Outline

Problem Overview

Spectral EmbeddingsBasics

Problems

Solutions

Shape Descriptors

Results

Future Work

Acknowledgements

Spectral EmbeddingsEfficiency with Nyström approximation

Outline

Problem Overview

Spectral EmbeddingsBasics

Problems

Solutions

Shape Descriptors

Results

Future Work

Acknowledgements

AAnn££nn ==nn

88>>>>>>>>>>>>>>>><<

>>>>>>>>>>>>>>>>::

22

666666666666666644

nnzz }}|| {{aa1111 aa1122 ¢¢¢¢¢¢ aa11nn

aa2211 ¢¢¢¢¢¢......

............

aaii11 aaii22 ¢¢¢¢¢¢ aaiinn......

............

aann11 aann22 ¢¢¢¢¢¢ aannnn

33

777777777777777755

EE ll££ ll ==

22

666644

............

~~uu11 ¢¢¢¢¢¢ ~~uu ll......

......

33

777755

¯̄̄̄̄̄¯̄̄̄̄̄¯̄̄̄̄̄¯̄̄̄̄̄¯̄̄̄̄̄¯̄

¯̄̄̄̄̄¯̄̄̄̄̄¯̄̄̄̄̄¯̄̄̄̄̄¯̄̄̄̄̄¯̄

EE nn££ 33==nn

88>>>>>>>>>>>><<

>>>>>>>>>>>>::

22

66666666666644

............

......~~uu22 ~~uu33 ~~uu44......

............

33

77777777777755

¯̄̄̄¯̄̄̄¯̄̄̄¯̄̄̄¯̄̄̄¯̄̄̄¯̄̄̄¯̄̄̄

¯̄̄̄¯̄̄̄¯̄̄̄¯̄̄̄¯̄̄̄¯̄̄̄¯̄̄̄¯̄̄̄

eexxttrraappoollaattiioonnOO((llnn))

mmaaxx--mmiinnssuubbssaammpplliinnggOO((llnn llooggnn))

AAll££ ll ==ll

88>>>><<

>>>>::

22

666644

nnzz }}|| {{aa1111 aa1122 ¢¢¢¢¢¢ aa11nn

aa2211 aa2222 ¢¢¢¢¢¢ aa22nn......

............

aall11 aall22 ¢¢¢¢¢¢ aallnn

33

777755

eeiiggeennddeeccoommppoossiittiioonn

OO((ll33))

Shape Descriptor

Shape Descriptor

Global Shape Descriptors

Light Field Descriptor (LFD)

Spherical Harmonics Descriptor (SHD)

Our Similarity Measure (EVD):

Outline

Problem Overview

Spectral Embeddings

Shape Descriptors

Results

Future Work

Acknowledgements

SSiimmCCoosstt((PP;;QQ)) ==1122

kkXX

ii ==11

££pp¸̧ ii ¡¡

pp°°ii

¤¤22

pp¸̧ ii ++

pp°°ii

Results

Experimental DatabaseMcGill 3D Articulated Shapes Databasehttp://www.cim.mcgill.ca/~shape/benchMark/

Outline

Problem Overview

Spectral Embeddings

Shape Descriptors

Results

Future Work

Acknowledgements

ResultsPrecision-Recall plot for McGill databasePrecision-Recall plot for McGill database

Outline

Problem Overview

Spectral Embeddings

Shape Descriptors

Results

Future Work

Acknowledgements

ResultsMcGill articulated shape databaseMcGill articulated shape database

Outline

Problem Overview

Spectral Embeddings

Shape Descriptors

Results

Future Work

Acknowledgements

Non-robustness of geodesic distances

Non-robustness to outliers

Limitations & Future WorkOutline

Problem Overview

Spectral Embeddings

Shape Descriptors

Results

Future Work

Acknowledgements

Acknowledgements

McGill 3D Shape Benchmark.

Phil Shilane (LFD & SHD implementations).

Outline

Problem Overview

Spectral Embeddings

Shape Descriptors

Results

Future Work

Acknowledgements

Thank You! for you attention