Upload
ashlie-austin
View
232
Download
1
Tags:
Embed Size (px)
Citation preview
Computer Vision
Stereo Vision
Bahadir K. Gunturk 2
Pinhole Camera
Bahadir K. Gunturk 3
Review: Perspective Projection
' ' 'x y f
x y z
Bahadir K. Gunturk 4
Stereo scene pointscene point
optical centeroptical center
image planeimage plane
p p’
p p’
Bahadir K. Gunturk 5
Stereo Constraints
X1
Y1
Z1
O1
Image plane
Focal plane
M
p p’
Y2
X2
Z2O2
Epipolar Line
Epipole
Bahadir K. Gunturk 6
A Simple Stereo System
Zw=0
LEFT CAMERA
Left image:reference
Right image:target
RIGHT CAMERA
Elevation Zw
disparity
Depth Z
baseline
Bahadir K. Gunturk 7
Stereo View
Left View Right View
Disparity
Bahadir K. Gunturk 8
Stereo Disparity The separation between two matching objects
is called the stereo disparity.
Bahadir K. Gunturk 9
Parallel Cameras
ZT
fZxxTlr
OOll OOrr
PP
ppll pprr
TT
ZZ
xxll xxrr
ff
T is the stereo baseline
rlxx
TfZ
rlxxd Disparity:
Bahadir K. Gunturk 10
Correlation Approach
For Each point (xl, yl) in the left image, define a window centered at the point
(xl, yl)LEFT IMAGE
Bahadir K. Gunturk 11
Correlation Approach
… search its corresponding point within a search region in the right image
(xl, yl)RIGHT IMAGE
Bahadir K. Gunturk 12
Correlation Approach
… the disparity (dx, dy) is the displacement when the correlation is maximum
(xl, yl)dx(xr, yr)RIGHT IMAGE
Bahadir K. Gunturk 13
Maximize Cross correlation
Minimize Sum of Squared Differences
Comparing Windows ==??
ff gg
Bahadir K. Gunturk 14
Feature-based correspondence Features most commonly used:
Corners Similarity measured in terms of:
surrounding gray values (SSD, Cross-correlation) location
Edges, Lines Similarity measured in terms of:
orientation contrast coordinates of edge or line’s midpoint length of line
Bahadir K. Gunturk 15
Feature-based Approach
For each feature in the left image…
LEFT IMAGE
corner line
structure
Bahadir K. Gunturk 16
Feature-based Approach
Search in the right image… the disparity (dx, dy) is the displacement when the similarity measure is maximum
RIGHT IMAGE
corner line
structure
Bahadir K. Gunturk 17
Correspondence Difficulties Why is the correspondence problem difficult?
Some points in each image will have no corresponding points in the other image.(1) the cameras might have different fields of view.
(2) due to occlusion.
A stereo system must be able to determine the image parts that should not be matched.
Bahadir K. Gunturk 18
Structured Light Structured lighting
Feature-based methods are not applicable when the objects have smooth surfaces (i.e., sparse disparity maps make surface reconstruction difficult).
Patterns of light are projected onto the surface of objects, creating interesting points even in regions which would be otherwise smooth.
Finding and matching such points is simplified by knowing the geometry of the projected patterns.
Bahadir K. Gunturk 19
Stereo results
Ground truthScene
Data from University of Tsukuba
(Seitz)
Bahadir K. Gunturk 20
Results with window correlation
Estimated depth of field Ground truth
(Seitz)
Bahadir K. Gunturk 21
Results with better method
A state of the art methodBoykov et al., Fast Approximate Energy Minimization via Graph Cuts,
International Conference on Computer Vision, September 1999.
Ground truth
(Seitz)
Bahadir K. Gunturk 22
Other constraints
It is possible to put some constraints. For example: smoothness. (Disparity usually doesn’t
change too quickly.)
Bahadir K. Gunturk 23
Parameters of a Stereo System Intrinsic Parameters
Characterize the transformation from camera to pixel coordinate systems of each camera
Focal length, image center, aspect ratio
Extrinsic parameters Describe the relative
position and orientation of the two cameras
Rotation matrix R and translation vector T
pl
pr
P
Ol Or
Xl
Xr
Pl Pr
fl fr
Zl
Yl
Zr
Yr
R, T
Bahadir K. Gunturk 24
Applications
courtesy of Sportvision
First-down line
Bahadir K. Gunturk 25
ApplicationsVirtual advertising
courtesy of Princeton Video Image