EMANUELE RODOLÀ<RODOLA@DSI .UNIVE. IT>

A Game-Theoretic Perspective on Registration and Recognition of

3D Shapes

2

Surface registration

The aim is to rigidly align (“register”) two or more 3D surfaces so as to attain automatic assemblage of range data (demo)

3

Surface registration

Typically a 2-step process: Coarse motion estimation Refinement

4

Coarse alignment

Knowledge of the acquisition processMarker-basedRANSAC-based DARCESPROSAC variantsPCA / 4PCS / Genetic

5

Refinement

Given a “good enough” initial alignment, it is possible to refine the registration iteratively.

This is usually done by establishing pointwise correspondences among the two surfaces, and using them to estimate the rigid transformation.

(demo chef)

6

A Game-Theoretic approach

We cast the registration problem to an inlier selection scenario:

We are given a set of candidate correspondences (strategies)

Then we look for a robust set of inlier correspondences wrt some notion of “rigidity”

Finally we can estimate the rigid transformation between the two surfaces

7

Enforcing rigidity

Given a model mesh M, a data mesh D, and a set of candidate correspondences (or strategies):

We define a rigidity-enforcing payoff function giving a measure of compatibility among strategies:

We wish to bring global information into the matching process by favoring sets of point-associations that are mutually compatible with a single rigid transformation. We do this by operating at a local level.

22211

2121babaλe,bb,,aaπ,Sπ:S

DMSSba ,,

8

The evolutionary process

The search for a solution is performed by simulating the evolution of a natural selection process. The choice of an actual selection process is not crucial and can be driven mostly by considerations of efficiency and simplicity.

9

Building the strategies set

It is not practical to deal with all the surface points from both the surfaces, i.e. we restrict to

Moreover, the isolation of interest points can help to avoid false correspondences

DMS

10

Surface descriptors

Surface vertices can be described using information at the vertex and of a local patch around it.

o Spin Images (ref. axis)o Integral Invariants (no ref.)o Point Signatures (ref. frame)o Signatures of Histograms (stable ref. frame)o And many more

11

Surface Hashes

To fully exploit our inlier selection method, we need descriptors with the following properties:

o high repeatabilityo weak distinctiveness

We introduce the Surface Hashes (demo):

o Normal Hasho Integral Hasho Mixed

12

Interest point detection

Relevance-based samplingClustering (via a Matching Game)Simple threshold on the Integral Hash

13

Enhancing the framework

The set of strategies can now be greatly reduced:

The descriptor prior gives better candidatesThe least likely correspondences are prunedThe selection process converges more rapidlyMuch lower memory requirements

Memory is the bottleneck!

14

Object-in-clutter

Focus is on recognition rather than alignment

We now have a known, usually complete model to match against an incomplete and cluttered scene. (demo)

15

Object-in-clutter

A good point selection strategyRobust descriptors wrt to occlusion and clutter

16

Object-in-clutter

17

Future directions

Scale invariance could be attained by taking into account the geodesic path between strategies

Non-rigid registration

18

Questions?