Lecture 20: Structure from Motion

Announcements

Proposals due today or Wednesday if you need an extra day or two

I will schedule project meetings next week

Today

We've talked about finding corresponding points in images

If we know the projection matrices of the cameras, we can recover the locations of points in space

What if we only know the correspondences? Known as “Structure from Motion”

Today

Today we will be talking about solving the problem under a simplifying assumption

To understand the assumption, we'll first talk about a simplified model of imaging

The equation of projection

(Image from Slides by Forsyth)

We know:

so

The equation of projection

(Image from Slides by Forsyth)

We know:

so

Makes things hard!

Weak perspective Issue

perspective effects, but not over the scale of individual objects

collect points into a group at about the same depth, then divide each point by the depth of its group

Adv: easy Disadv: wrong

(Image from Slides by Forsyth)

Effectively dividing by a constant z

Affine Model

The projection equation can be written as

No division! Okay approximation when variation in depth is

small relative to the overall depth of the object

3D Coordinate

Basic problem Given n fixed points observed by m affine

cameras we can say that for each point

For large enough m and n this is solvable Up to an ambiguity If M and P are a solution, so is

2x4 matrix

Invertible 3x3 matrix

Affine Structure and Motion from Two Images

Projection equations

Leads to condition

Take advantage of affine ambiguity (see text), we can rewrite this as

Which is

One equation per set of correspondences Can solve with 4 sets of corresponding (u,v)

and (u',v') Given new correspondence, solve

What if I have multiple images?

Basic Equations

If I stack the m instances across cameras

Since I'm tracking multiple points

Can stack these into a matrix

(from Forsyth and Ponce)

Side Trip: SVD

(from Forsyth and Ponce)

More properties of SVD

Back to Recovering Structure and Motion

D is a product of a 2mx3 matrix and 3xn matrix Rank 3

So, using SVD

Using the SVD

If

We claim that

Results (Tomasi and Kanade '92)

Input