Combining Visual and Spatial Appearance for Loop Closure

Detection in SLAM

Kin Leong Ho, Paul NewmanOxford University Robotics Research Group

Motivation

• Loop Closing – the task of deciding whether a vehicle has returned to a previously visited area

• Popular approaches – nearest neighbour statistical gate, joint compatibility test

Image Loop Closure

• Closing loops with visually salient features to avoid dependence on global position estimate

Closing the loop

MSER detector

Saliency detector

Image Feature Extraction Process

Demonstration of wide-baseline stability of visually salient features under perspective distortion and variation in illumination conditions

Matching Performance

Similar posters found in the environment.

[Newman,Ho ICRA2005]

Query Image Tentative Match

Tentative MatchTentative Match

Results from Image Retrieval System

Limitations of Image Matching

- Repetitive visual artifacts in urban environments such as posters, signs and wall pattern

- False triggering of loop closure event based solely on image matching

Query Image Tentative MatchTentative Match

Incorporating Spatial Information

-Spatial information can be used to disambiguate visually confusing locations

Spatial Descriptors

• Reduced a laser scan patch into a set of descriptor

• Describe curvature of shape

• Describe complexity of shape

• Describe spatial configuration of laser scan

Segmentation

• Laser scan is divided into smaller but sizeable segments

• Segments are formed due to break in boundary or occlusions

Original Laser Scan Set of Descriptors

Cumulative Angular Function

• A plot of the cumulative change in turning angle versus the arc length of the segment

• Invariant to rotation and translation

Arc length of Segment

TurningAngle

Entropy of CAF

• A measure of complexity of segment

• Weight descriptors to prefer between complex versus simple shapes

CAF Histogram of Turning Angle

Inter-Segment Descriptors

• Extract critical points: Critical points are points along a segment where there are sharp changes in cumulative angular function

• Distances and relative

orientations between critical points form links between segments

Descriptor Comparison 1

• Angular function disparity – minimum error between two cumulative angular functions

Descriptor Comparison 2

• entropy disparity – Kullback-Leiber distance

Edge Comparison

• Matching of links• Links that are

matched are coloured in black

• Links that are not matched are coloured in blue

Spatial Similarity Score

•Shape similarity metric comprises of two parts: shape similarity and spatial similarity

Results from Spatial Retrieval System

More Results

Query Image

MSER Detector

Saliency Detector

SelectedRegions

SIFTDescriptor

ImageDatabase

Similarity Measure

LaserDescriptor

Laser ScanDatabase

Similarity Measure

Combined Similarity

Scores

Segmentation

QueryLaser Scan

Visual Similarity Matrix

Spatial Similarity Matrix

Combined Similarity Matrix

Demonstration

Issues• Setting of threshold values• Principled way of combining similarity scores• At present limited to planar environments

Current Extensions

• Removal of repetitive images by spectral decomposition• Successful Application to 3D laser mapping and SLAM

Questions

Thank you!