Lecture 8Feature Matching and Stereo Vision

Slides by:Clark F. Olson

Jean PonceLinda G. Shapiro

2

Shape from single images

There are many image cues from which we can determine the shape of objects in the scene.

What can we determine from a single image?

3

Shape from X

Methods for extracting shape in computer vision include:

• Silhouette• Shading• Texture• Focus• Structured lighting• Motion• Stereo

4

Stereo Vision

Stereo vision is the ability to infer the 3D structure of a scene using two (or more) images from different viewpoints.

Two fundamental problems:• Correspondence problem

Which points are the projection of the same scene element?• Reconstruction problem

Given a correspondence (and the camera geometry) what is the 3D location of the observed object?

For precise measurements, calibration is necessary.

5

• Epipolar Plane

• Epipoles • Epipolar Lines

• Baseline

Epipolar Geometry

6

• Potential matches for p have to lie on the corresponding epipolar line l’.

• Potential matches for p’ have to lie on the corresponding epipolar line l.

Epipolar Constraint

7

Stereo Reconstruction

If we know the corresponding points in the two images, we can determine where the point is relative to the cameras.

This assumes that we know precisely where the cameras are relative to each other!

8

Binocular Fusion

Usually, we don’t know in advance which points correspond to each other.

If we get the correspondence wrong, then we will compute the wrong point.

Finding the correspondences is an important problem.

9

Reconstruction

If our correspondences are precisely correct, then the rays from image points through the optical centers will intersect. In practice, this is rare.Solution: find the point that minimizes the distance to the two rays.

10

Rectification

A useful trick: If we reproject the images carefully (onto a common plane parallel to the baseline) then the epipolar lines are horizontal. We will assume that this has been done.

11

Rectification Example

Input left image:Not rectified

Input right image:Not rectified

After rectification

The red lines show corresponding scanlines.

12

Stereo Disparity

left image right image

3D point

Disparity: the difference in image location of the same 3Dpoint when projected under perspective to two different cameras.

d = xleft - xright

13

Correspondence

How do we determine correspondences between (rectified) images?

Usually by comparing small image patches.

?Left image Right image

14

Correspondence

Far away points move little and look similar.Nearby points move a lot and look dissimilar.(Assumes rectified images.)

15

Dense Stereo

Dense stereo information can be computed by asking for each pixel in the left image: what is the corresponding pixel in the right image?

Left image

Right image

Color-coded range map

Range map after pruning likely mismatches