Vision-based SLAM

  • View
    18

  • Download
    0

Embed Size (px)

DESCRIPTION

Vision-based SLAM. Simon Lacroix Robotics and AI group LAAS/CNRS, Toulouse With contributions from: Anthony Mallet, Il-Kyun Jung, Thomas Lemaire and Joan Sola. Perceive data In a volume Very far Very precisely. 1024 x 1024 pixels 60º x 60º FOV  0.06 º pixel resolution - PowerPoint PPT Presentation

Text of Vision-based SLAM

  • Vision-based SLAMSimon LacroixRobotics and AI groupLAAS/CNRS, Toulouse

    With contributions from:Anthony Mallet, Il-Kyun Jung,Thomas Lemaire and Joan Sola

  • Benefits of vision for SLAM ?Cameras : low cost, light and power-savingImages carry a vast amount of informationA vast know-how exists in the computer vision community

  • The way humans perceive depth0. A few words on stereovision Very popular in the early 20th century AnaglyphsPolarizationRed/Blue

  • Principle of stereovisionIn 2 dimensions (two linear cameras):

  • Principle of stereovisionIn 3 dimensions (two usual matrix cameras):Establish the geometry of the system (off line)Establish matches between the two images, compute the disparityOn the basis of the matches disparity, compute the 3D coordinates

  • Geometry of stereovisionxyzxyzOlOr

  • Geometry of stereovision

  • Geometry of stereovisionxyzxyzOlOr

  • Stereo images rectificationGoal: transform the images so that epipolar lines are parallelInterest: computational cost reduction of the matching process

  • Dense pixel-based stereovisionProblem: For each pixel in the left image, find its correspondant in the right imageLeft lineRight line???

    363792876896490990

    357496396586301975

  • Dense pixel-based stereovision

  • Outline0. A few words on stereovision0-bis. Visual odometry

  • Visual odometry principle

  • Visual odometryFairly good precision (up to 1% on 100m trajectories)But:Depends on odometry (to track pixels)No error model available

  • Visual odometryApplied on the Mars Exploration Rovers50 % slip

  • Outline0. A few words on stereovision0-bis. Visual odometry1. Stereovision SLAM

  • What kind of landmarks ?Interest points = sharp peaks of the autocorrelation function

    Harris detector (precise version [Schmidt 98])Auto-correlation matrix: Principal curvatures defined by the two eigen values of the matrix (s: scale of the detection)

  • Landmarks : interest points

  • Interest points stabilityInterest point repeatability

    Interest point similarity : resemblance measure of the two principal curvatures of repeated points= 70% (7 repeated points out of 10 detected points)Maximum point similarity: 1

  • Interest points stabilityRepeatability and point similarity evaluation:Evaluated with known artificial rotation and scale changes

  • Interest points matchingPrinciple: combine signal and geometric information to match groups of points [Jung ICCV 01]

  • Consecutive imagesLarge viewpoint changeSmall overlapLandmark matching results

  • 1.5 scale change3.0 scale changeLandmark matching results

  • Landmark matching results (Ced)Detected pointsMatched pointsAn other example

  • Stereovision SLAM

    Landmark detectionRelative observations (measures)Of the landmark positionsOf the robot motionsObservation associationsRefinement of the landmark and robot positions

    Vision : interest points

    StereovisionVisual motion estimation Interest points matchingExtended Kalman filter

  • Dense stereovision actually not requiredIP matching applied on stereo frames (even easier !)

  • Dense stereovision actually not requiredIP matching applied on stereo frames (even easier !)

  • Visual motion estimation

  • Stereovision SLAM

    Landmark detectionRelative observations (measures)Of the landmark positionsOf the robot motionsObservation associationsRefinement of the landmark and robot positions

    Vision : interest points OK

    Stereovision OKVisual motion estimation OKInterest points matching OKExtended Kalman filter

  • Seting up the Kalman filterSystem state:

    System equation:

    Observation equation:

    Prediction: motion estimates

    Landmark discovery: stereovision

    Observation : matching + stereovision

  • Error estimatesStereovision errors Dense stereovision: empirical analysisSparse stereo: disparity error deduced from the matching errors

    Observation errorsCombination of stereovision and matching errors

    Prediction errors1st order development of the Jacobian of the function minimized by the least square estimate

  • Error estimates (1) Errors on the disparity estimatesempirical study: s Errors on the 3D coordinates : Maximal errors : 0.4 m baseline: 1.2 m baseline: Online estimation of the errorsStereovision error:

  • Error estimates (2) Interest point matching error (not miss-matching)

    - Correlation surface built thanks to rotation and scale adaptive correlation,fitted with a Gaussian distributionGaussian distributionCorrelation surface Combination of matching and stereo error - Driven by 8 neighbor 3D points and projecting one sigma covariance ellipse to 3D surface

  • Error estimates (3)Visual motion estimation error Propagating the uncertainty of 3D matching points set to optimal motion estimate

    - 3D matching points set

    - Optimal motion estimate

    - Cost function

    Covariance of the random perturbation u : propagation using Taylor series expansion of the Jacobian of the cost function around

  • Results70m loop, altitude from 25 to 30m, 90 stereo pair processed Landmark error ellipses (x40)Trajectory and landmarksPosition and attitude variances

  • Results70m loop, altitude from 25 to 30m, 90 stereo pair processed

    Frame 1/90ReferenceReferenceStd. Dev.VMEresultVMEAbs.errorSLAMresultSLAMStd. Dev.SLAMAbs. error6.190.1811.935.746.010.160.182.310.664.001.691.420.550.89-105.940.06-105.520.41-106.030.080.09tx3.17m0.26m5.31m2.14m3.13m0.09m0.04mty0.61m0.07m2.01m1.40m0.26m0.19m0.35mtz-1.52m0.04m-3.25m1.73m-1.51m0.03m0.01m

  • Results (Ced)270m loop, altitude from 25 to 30m, 400 stereo pairs processed, 350 landmark mappedLandmark error ellipses (x30)Trajectory and landmarksPosition and attitude variances

  • Results (Ced)270m loop, altitude from 25 to 30m, 400 stereo pairs processed, 350 landmark mapped

    Frame 1/400ReferenceReferenceStd. Dev.VMEresultVMEAbs.errorSLAMresultSLAMStd. Dev.SLAMAbs. error-0.120.87-0.130.01-3.680.383.562.871.14-4.997.865.540.401.64105.440.23101.823.62104.320.191.12tx-4.73m0.57m5.45m10.38m-3.98m0.21m0.95mty0.14m0.46m3.04m2.90m-2.16m0.22m2.12mtz3.89m0.15m19.81m15.94m3.46m0.11m0.43m

  • Application to ground roverslandmark uncertainty ellipses (x5)110 stereo pairs processed, 60m loop

  • Application to ground rovers110 stereo pairs processed, 60m loop

    Frame 1/100ReferenceReferenceStd. Dev.VMEresultVMEAbs.errorSLAMresultSLAMStd. Dev.SLAMAbs. error0.520.312.752.230.880.980.36 0.360.25-0.110.470.720.740.36 -0.140.161.892.031.241.841.38tx-0.012m0.010m0.057m0.069m-0.077m0.069m0.065mty-0.243m0.019m-1.018m0.775m-0.284m0.064m0.041mtz0.019m0.015m0.144m0.125m0.018m0.019m0.001m

  • Application to indoor robotsAbout 30 m long trajectory, 1300 stereo image pairs

  • Application to indoor robots10 timesCov. ellipseAbout 30 m long trajectory, 1300 stereo image pairs

  • Application to indoor robotsAbout 30 m long trajectory, 1300 stereo image pairs

  • Application to indoor robotsTwo rotation angles(Phi, Theta) and Elevation must be zero About 30 m long trajectory, 1300 stereo image pairs

  • Outline0. A few words on stereovision0-bis. Visual odometry Stereovision SLAM Monocular (bearing-only) SLAM

  • Bearing-only SLAM Generic SLAM

    Landmark detectionRelative observations (measures)Of the landmark positionsOf the robot motionsObservation associationsRefinement of the landmark and robot positions

    Stereovision SLAM

    Vision : interest points

    StereovisionVisual motion estimation Interest points matchingExtended Kalman filter

  • Bearing-only SLAMGeneric SLAM

    Landmark detectionRelative observations (measures)Of the landmark positionsOf the robot motionsObservation associationsRefinement of the landmark and robot positions

    Monocular SLAM

    Vision : interest points

    Multi-view stereovisionINS, Motion model, GPSInterest points matchingParticle filter + extended Kalman filter

  • Landmark observationsObservation filter Gaussian particles1. Landmark initialisation2. Landmark observations

  • Bearing-only SLAM

  • Bearing-only SLAMOverview of the whole algorithm

  • Comparison stereo / bearing-only

  • Looking forward / looking sidewardsstereovisionbearing-only

  • Using panoramic vision

  • Data association is still an issueView-based qualitative navigation can help to focus the search

  • View-based navigationLocal characteristics histograms based on gaussian derivatives Indexing with global attributesColor HistogramsTexture histogramsLocal Characteristics Histograms Family (LCHF)

  • View-based navigationEmpirical relation between image distance and cartesian distance

  • Closing the loop1. Image processing at each image acquisition

  • Closing the loop2. SLAM processes at each image acquisition

  • Closing the loop

  • Outline0. A few words on stereovision0-bis. Visual odometry Stereovision SLAM Monocular (bearing-only) SLAM Bearing-only SLAM using line segments

  • Using line segments

  • Initializing line segments landmarksIn 2 dimensions:

  • Bearing-only SLAM with line segments

  • Bearing-only SLAM with line segments

  • Summary0. A few words on stereovision0-bis. Visual odometry Stereovision SLAM Monocular (bearing-only