Upload
gordon-wade
View
214
Download
0
Embed Size (px)
Citation preview
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Martin Bujňá[email protected]
Supervisor : RNDr. Martin Samuelčík
On-line Structure from Motion
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Aim
• Reconstruction of sparse 3D model of the scene using video sequence
• Input : video sequence captured with standard hand-held camera
• As small as possible user interactions
• Output :– Sparse 3D model– Camera calibration parameters
• Intrinsic
• Extrinsic
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Feature detection
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Feature detection
• Harris based detector– Removing features that can be
interchanged with its neighborhood
– Introducing feature orientation
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Features matching
• Find the nearest (distance) similar (correlation) feature in the next frame
• Matching ambiguities– two neighboring frames don’t
differ a lot – „outliers“ are effectively
removed in further processes (up to 30% of bad pairs is expected)
??
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Guided searching
• F-matrix : two view geometry– Restricts searching for
matching pair to 1D search region in the second frame
– Using “stereo” principle• Epipolar line ordering constrain• Ratio of corresponding
segments is near 1 for small motion between two frames
?x
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Guided searching
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Projective reconstruction• Canonical camera pair from
F-matrix– F-matrix calculated using
RANSAC paradigm– 4 free parameters calculated
so that camera pair satisfies calibration conditions
• Principal point at image center• Aspect ration of 1• Zero skew• Unknown varying focal length
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Projective reconstruction
• sparse 3D reconstruction obtained from camera pair– Intersection of sightlines passing
through known feature pairs
• Quasi calibration– Angles are near to “calibrated”
case• Estimation of precision of the 3D
point (size of area where 3D point can be put)
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Structure from motion (SfM)
• Merging camera pairs– Merge using common 2D-
3D feature points– Search for projective
transformation H • Find H or H-1 depending
on quality of 3D points• All measurements are
performed in 2D• RANSAC paradigm
– Linear solution• New more precise SfM
– Recalculating structure– Recalculating camera
motionNew pair
Scene
4x4 homography H
H-1
3px noise added to images
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Self-Calibration
**
projection
constraints
• Pinhole camera :– Expressed by 3x4 matrix– Can be decomposed into calibration
3x3 (K), rotation 3x3 (R) and translation (T) 3x1 matrix
• Self-calibration– Means searching for calibration
parameters – only from input images– Math : Dual Absolute Quadrics(Ω*)
• KKT = P(H Ω* HT)PT
KKT = (PH) Ω* (HTPT)
– Linear solution with one cubical constrain
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Experiment
• Detection and correction of lens radial distortion– Polynomial model :
X1 = (1 + a(x02+y0
2))* x0
Y1 = (1 + a(x02+y0
2))* y0
– Search for ‘a’ using F-matrix
• minimizing distance of matching feature points from their corresponding epipolar lines
– Numerical minimization • simulated annealing
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research
Conclusion
• Contribution– New feature tracker– Enhanced guided matching – Sequential SfM without need of searching for initial
frame– Lens radial distortion correction
• Results– Video
• Current state– video
© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research