Randomized Algorithms for Comparing and Understanding 3D Geometry Niloy J. Mitra Department of Electrical Engineering Advisor: Leonidas J. Guibas Associate

Randomized Algorithms for Comparing and Understanding 3D Geometry

Niloy J. MitraDepartment of Electrical Engineering

Advisor: Leonidas J. Guibas Associate Advisor: Marc Levoy

Comparing and Understanding 3D Geometry

Need for Digital 3D Models

games

movies

architectural design

medicine

Google earth

manufacturing


Creating Geometry: 3D Modelers


Capturing Geometry: 3D Scanners

laser scanner 3D geometry entertainment


Shape Acquisition


Shape Acquisition


Shape Acquisition

• Partial similarity between shapes


Shape Acquisition

• Partial similarity between shapes


Shape Acquisition

• Partial similarity between shapes• Efficient shape retrieval for partial queries


Geometry Processing

[Funkhouser et al. `05]

[Katz and Tal `04]

[Gelfand et al. `05]

[Sharf et al.`04]


Model Organization and Retrieval

[Kazdhan et al. `04]


Partial Shape Similarity

Self-similarity of an object symmetry

partial similarity


Total vs Partial Matching• Total matching is easy

PCA (Principal Component Analysis) Axes


Total vs Partial Matching• Partial matching is difficultWhich region matches which other region(s)?Space of rigid transforms rotation + translation

Brute force approach not feasible

Instead of exhaustive searching, use local geometry to guide where to search

Easy to verify a transform


ContributionsAlgorithms to:• Identify and extract similar (symmetric) patches of different

size• Estimate partial shape similarity between models without

explicitly aligning them

Properties:• Scalable and parallel• Theoretical error bounds• Output sensitive depends on complexity of solution and

not on the complexity of model(s)


Outline• Introduction• Related Work• Symmetry Detection• Probabilistic Fingerprints• Conclusion and Future Work


Related Work: Global Alignment

[Huber and Hebert`01] [Li and Guskov `05] [Gelfand et al. `05]

• Feature based alignment• Combinatorial search, need multiple objects together


Related Work: Desc. Based Align.

[Kazhan et al. `03][Osada et al. `02]


• Descriptor based alignment • Fails for partial similarity


Related Work: Geometric Hashing

[Gal and Cohen-Or`05][Wolfson and Rigoutsos`97]


• Descriptor based alignment • Fails for partial similarity

• Geometric hashing• Tradeoff memory for time


Related Work: Symmetry Detection

[Podolak et al. `06]

brute force : O(n6)

[Loy and Eklundh `06]

Hough transform on feature points

[Thrun and Wegbreit `05]

Shape from symmetry




Symmetry in Nature“Symmetry is a complexity-reducing concept [...]; seek it everywhere.”

- Alan J. Perlis

"Females of several species, including […] humans, prefer symmetrical males." - Chris Evan


Partial Symmetry Detection (SIGGRAPH 2006)

GivenObject/shape (represented as point cloud, mesh, ... )

Identify and extract similar (symmetric) patches of different size across different resolutions

Goal


Partial Symmetry

Transform Types:

• Reflection

• Rotation + Translation

• Uniform Scaling


Reflective Symmetry


Reflective Symmetry : A Pair Votes


Reflective Symmetry : Voting Continues


Reflective Symmetry : Voting Continues


Reflective Symmetry : Largest Cluster

• Height of cluster ! size of patch

• Spread of cluster ! approximation level


Pipeline


Pruning: Local Signatures• Local signature invariant under transforms• Signatures disagree points don’t correspond

Use (1, 2) for curvature based pruning


Reflection: Normal-based Pruning


Point Pair Pruning

all pairs curvature based curvature + normal based


Transformations• Reflection point-pairs• Rigid transform more information

Robust estimation of principal curvature frames [Cohen-Steiner et al. `03]


Mean-Shift ClusteringKernel:• Type radially symmetric hat• Radius


Verification• Clustering gives a good guess• Verify build symmetric patches• Locally refine solution using ICP algorithm [Besl and McKay `92]


Random Sampling• Height of clusters related to symmetric region size• Random samples !

larger regions likely to be detected earlier• Output sensitive


AnalysisAssumptions:• Smooth surface -sampled• No noise

Relates number(n’) of random samples to: • Size of symmetric patch (p)• Confidence (1-)• Sampling spacing, kernel radius, continuity of signature

Tools for Analysis:• Signature continuity• Chernoff bound


Compression: Chambord






Approximate Symmetry: Dragon

correction fieldUNITS: fraction of bounding box diagonal

detected symmetries


Limitations

• Cannot differentiate between small sized symmetries and comparable noise

[Castro et al. `06]


Articulated Motion: Horses

registration symmetry detection between two objects




Partial Shape Similarity (SGP 2006)

Are two shapes similar in parts?

Efficient tests require compact signatures• database query • online setting


partial similarity

Probabilistic Fingerprints

probabilisticfingerprint

probabilisticfingerprint

comparecompact

independent


Insight Partial matching ! difficult problemTotal matching ! easy problem

Reduce partial matching ! many small total matching problems

Results in few false positives ! quick to verify and discard


Input Shapes


Sample Points


Shingles: Overlapping Patches


Shingles: Overlapping Patches


Bag of Patches: Ordering Discarded


Pipeline


Pipeline: Uniform Sampling• Uniform spacing use [Turk`92]• Sample spacing


Pipeline: Shingle Generation• Shingles: overlapping unordered patches• Shingle radius: •


Pipeline: Signatures• Stable signatures• Invariant to rigid transforms

• Spin-images• Shape

unordered high-dimensional point set with rigid transform factored out


• Similarity/resemblance• Defined wrt. signatures

• Compare two bag of points in high-dim space• No alignment required• Brute force evaluation impractical

Pipeline: Resemblance


How to Compare Point Sets

• Compare two point sets no need to align• Don’t have red and blue points together


Reduce Sample Size

• Randomly sample red points• Randomly sample blue points• still need to solve for correspondence

independently


Min-hashing [Broder`97]

Each of m random ‘experts’• Has an ordering of space-boxes• Selects the point that lies in lowest ordered box

2 3


Min-hashing [Broder`97]

Each of m random ‘experts’• Has an ordering of space-box• Selects the point that lies in lowest ordered box

2 3

1 1


Pipeline: Min-hashing [Broder`97]

Feature selection by random experts• ‘Features’ only useful for correspondence

• Need not have any visual importance• Reduces set comparison to element-wise comparison


Applications: Adaptive Features


Applications: Adaptive Features

merged scan


Applications: Shape Space

• Partial similarity• Articulated motion


Applications: Database Retrieval


• Pre-processing time in seconds:

• Query time → 15 msec/model • Fingerprint size → 10kb

model # vertices uniform sampling

spin image Rabin hash min-hash

skull 54k 0.8 7.5 0.05 4.5

Caesar 65k 1.4 7.3 0.08 10.3

bunny 121k 1.8 13.8 0.04 2.9

horse 8k 0.7 5.7 0.05 7.3

Statistics


Limitations• Fails if resemblance is small• How to handle uniform scaling?• Stability of spin-images




Conclusion• Simple probabilistic framework

• Local evidence → global reasoning• Geometric information for guidance• Complexity of problem, not complexity of model

• Symmetry information → High level model understanding

• Possible to compare two shapes using very compact fingerprints without aligning the models

• Local reasoning → possible false positives → verification


Future Works• Continuous scanning, assembly, hole filing• Extension to deformable, time varying models• Understanding of high dimensional data• Online transmission, authentication, and security

Data courtesy: Prof. B. Chen


CollaboratorsNatasha Gelfand, Stanford

Joachim Giesen, MPII Saarbrücken

Markus Gross, ETH Zürich

Leonidas Guibas, Stanford

An Nguyen, Stanford

Mark Pauly, ETH Zürich

Helmut Pottmann, Vienna University of Technology


AcknowledgementsGunnar CarlsonLeonidas GuibasJean-Claude LatombeMarc LevoyMark Pauly

Mike Cammarano, Billy Chen, Milton Chen , Kayvon Fatahalian, Gaurav Garg, Eran Guendelman, Daniel Horn, Mike Houston, Jeff Klingner, David Koller, Manu Kumar, Ren Ng, John Owens, Doantam Phan, Marie Ringel, Pradeep Sen, Eino-Ville Talvala, Vaibhav Vaish, Ron Yeh

Emilio Antunez, Qing Fang, Natasha Gelfand, Olaf A. Hall-Holt, Kyle Heath, Rachel Kolodny, Nikola Milosavljevic, An Nguyen, Steve Oudot, Maksim Ovsjanikov, Daniel Russel, Aneesh Sharma, Jaewon Shin, Primoz Skraba, Michael Wand, Yusu Wang, Danny Yang, Afra Zomorodian

Pierre Alliez, Mario Botsch, Pat Hanrahan, Michael Hoffer, Rajiv Motwani, Richard Keiser, Doo Young Kwon, Bob Sumner, Martin Wicke

Manuela Cavegn, Heather Gentner, John Gerth, Ada Glucksman, Hoa Nguyen

Joseph W. and Hon Mai Goodman Stanford Graduate FellowshipCargo, Darpa, ITR, NIH, and NSF funding


Acknowledgements

Stanford Cricket Club

Stanford Outing Club

Stanford Climbing Wall

Stanford Alpine Club

Kamran Ahsan, Abhishek Bapna, Akanksha Bapna, Indrahit Bhattacharya, Gaurav Chandra, Anirban Dasgupta, Anupam Datta, Amal Ekbal, Gaurav Garg, Mahesh Hardikar, Sara Kalantari, Uma Kelkar, Neha Kumar, Subhasish Mitra, Shoubhik Mukhopadhyay, Subha Nabar, Anindya Pathak, Inam Ur-Rehman, Mitul, Saha, Debasis Sahoo, Sriram Sankaranarayanan, Arjun Singh, Padma Sundaram, Vaibhav Vaish, gsb-SIE fellows, climbing buddies, cricket club folks, … many I missed


Acknowledgements

ParentsBrother

Devasree


Acknowledgements


thank you!

Documents

Randomized Algorithms for Comparing and Understanding 3D Geometry Niloy J. Mitra Department of Electrical Engineering Advisor: Leonidas J. Guibas Associate