Registration of Colored 3D Point Clouds with a Kernel-based Extension to the

Normal Distributions Transform

Benjamin Huhle1, Martin Magnusson2, Wolfgang Straßer1, Achim J. Lilienthal2

1 WSI/GRIS, University of Tübingen, Germany2 AASS, Dept. of Technology, Örebro University,

Sweden

05/23/2008 ICRA '08, Pasadena

Benjamin Huhle - WSI/GRIS, Tübingen

Motivation

2

• point cloud registration for Localization & Mapping

• Problems– geometric features

(structure) required– small field-of-view– noise

• additional color data available!

Outline of the talk

3

• Related Work– Normal Distributions Transform (NDT)– Vision-aided registration (SIFT-Features)– Combined approach (SIFT-Features+NDT)

• Color-NDT– straightforward approach fails– Kernel-based Color-NDT

• Experiments– mobile robot with time-of-flight camera

Normal Distributions Transform (NDT)

4

• Biber & Straßer, 2003– cell grid– approximate point

distributions– multiple overlapping

grids– optimization using

analytical (2nd order) derivatives from: Biber, 2003

3D Normal Distributions Transform

5

• in 3D

• comparison with ICP: Magnusson et al., 2007

Vision-Aided RegistrationAndreasson & Lilienthal, 2007

6

• robust registration with image features (SIFT)– feature detection in images– lookup of 3D coordinates

• challenges:– noise– dynamic environments from: Andreasson et al., 2007

Combined Energy ApproachHuhle, Jenke & Straßer, 2008

7

• Sum of NDT score and feature distances

• must favor features (small )

Ad-hoc approach to using color with NDT

8

• colored point cloud: [x,y,z,r,g,b]• 6D color–space distribution

toy example:• 2D position• 1 color-

dimension (hue)

• model is 3D normal distribution

Single-Mode Color–Space Distribution

9

• conditional distributions of 2 test-points

Single-Mode Color–Space Distribution

10

• ... for a test-point with different color

Kernel-Based Color-NDT

11

• Gaussian mixture-model in color-space (EM-Algorithm)

• components are weighting kernels for point distributions

• use 3 kernels

• for each kernel: compute spatial Normal Distribution using– weighted mean– weighted covariance

Toy Example revisited

12

• perfect match of model and data

Toy Example revisited

13

• ... test point with different color

Registration with Color-NDT

14

• optimize score (mixture model of normal distributions)

• Newton's method• translation + rotation vector (6D parameters)

Experiments

15

• Time-of-flight camera – PMDTec 19k– 160x120 pixels– 30° fov– significant noise level

• additional color camera

Results

16

• 21 incrementally registered frames• odometry as initial poses

Results using combined approach

17

SIFT only

• 11 frames

combinedSIFT+NDT

combinedSIFT+Color-NDT

Results using combined approach

18

SIFT only

• 11 frames

combinedSIFT+NDT

combinedSIFT+Color-NDT

Results using combined approach

19

SIFT only

• 11 frames

combinedSIFT+NDT

combinedSIFT+Color-NDT

Results using combined approach

20

SIFT only

• 11 frames

combinedSIFT+NDT

combinedSIFT+Color-NDT

Conclusion

21

• Color-NDT:– more robust/stable– more weight on Color-NDT score in combined

approach– can fix inaccuracies of SIFT registration

• towards registration of low-end sensor data with integrated use of color and depth data

Thank you for your attention!

22

• Peter Biber and Wolfgang Straßer. The Normal Distributions Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2743–2748, 2003

• Martin Magnusson, Achim Lilienthal and Tom Duckett, Scan Registration for Autonomous Mining Vehicles Using 3D-NDT. Journal of Field Robotics, 24:10, 2007, pp. 803-827

• Henrik Andreasson and Achim J. Lilienthal, Vision Aided 3D Laser Based Registration. Proceedings of the 3rd European Conference on Mobile Robots (ECMR), 2007

• Benjamin Huhle, Philipp Jenke and Wolfgang Straßer. On-the-Fly Scene Acquisition with a Handy Multi-Sensor System. Int. J. of Intelligent Systems Technologies and Applications, to appear, 2008