Jakob Engel Semi-Dense Visual Odometry for a Monocular Camera 1

Semi-Dense Visual Odometry for a Monocular CameraJakob Engel, Jürgen Sturm, Daniel Cremers

Intl. Conf. on Computer Vision (ICCV) 2013

Monocular Video Camera Motion and Scene Geometry

Computer Vision Group

Technical University of MunichJakob Engel

Jakob Engel Semi-Dense Visual Odometry for a Monocular Camera 2

Visual Odometry

Camera: 752x480 @ 30fps, global shutter, monochrome, 130° fovhttp://youtu.be/LZChzEcLNzI

Jakob Engel Semi-Dense Visual Odometry for a Monocular Camera 3

Applications

Augmented / Virtual Reality Robotics

• requires camera pose to render objects

• requires scene geometry e.g.for physical interaction

• requires camera pose to control robot position

• requires scene geometry e.g.to avoid obstacles

http://youtu.be/eznMokFQmpc

Jakob Engel Semi-Dense Visual Odometry for a Monocular Camera 4

• small, light-weight• cheap• low power comsumption • versatile (scale-ambivalent)• easy to calibrate

Why Monocular?

http://youtu.be/AWSUMGJKt0U

Jakob Engel Semi-Dense Visual Odometry for a Monocular Camera 6

Input Video640x480 @ 30Hz

Semi-Dense Depth Map

• pixel depth

• depth variance

• image

TrackingEstimate pose of new frame

relative to depth map

Overview

Jakob Engel Semi-Dense Visual Odometry for a Monocular Camera 8

Tracking

Camera position?

Jakob Engel Semi-Dense Visual Odometry for a Monocular Camera 10

Directly minimize photometric error:

reference back-warpedimage to track

sum over all pixel

camera pose

per-pixel depth

Tracking

-> Iteratively reweighted least-squares optimization (LM Algorithm)

Jakob Engel Semi-Dense Visual Odometry for a Monocular Camera 11

Input Video640x480 @ 30Hz

Semi-Dense Depth Map

MappingPropagate and update

depth map (stereo)

• pixel depth

• depth variance

• image

TrackingEstimate pose of new frame

relative to depth map

Overview

Jakob Engel Semi-Dense Visual Odometry for a Monocular Camera 13

1. Propagation

2. Stereo-update

3. Regularization

Mapping

Extract & MatchFeatures

(SIFT / SURF / ...)

Input Images

Jakob Engel Semi-Dense Visual Odometry for a Monocular Camera 15

Track:min. reprojection error

(point distances)

Map:est. feature-parameters

(3D points / normals)

abstract images to feature observations

Input Images

Track:min. photometric error

(intensity difference)

Map:est. per-pixel depth

(semi-dense depth map)

keep full image

So, what‘s new?Keypoint-Based Semi-Dense (direct)

can only use & reconstruct corners can use & reconstruct whole image

Jakob Engel Semi-Dense Visual Odometry for a Monocular Camera 16

...and why do that?

+accuracy – +information – +parallelizable

Jakob Engel Semi-Dense Visual Odometry for a Monocular Camera 17

Large-Scale Semi-Dense Monocular SLAM J. Engel, T. Schöpps, D. Cremers, submitted to ECCV 2014

Future Work

patent owned by TU Munich

Video no online yet.Will be published shortly on youtube channel cvprtum