Probabilistic Techniques for Mobile Robotics Course Content · 1 Probabilistic Techniques for Mobile Robotics John Leonard MIT Department of Mechanical Engineering MIT Center for

1

Probabilistic Techniques for Mobile RoboticsJohn Leonard

MIT Department of Mechanical EngineeringMIT Center for Ocean Engineering

MIT CSAIL

J. Leonard MIT 2.166, Fall 2008

Course Content

Book: Probabilistic Robotics, by Thrun, Burgard, and FoxUnderlying theoretical foundation is Bayesian Statistical InferencePrerequisites: probability, linear algebra, and programming experienceFocus will be on implementing key algorithms


Course Philosophy

"What we have to learn to do, we learn by doing." - Aristotle

A good way to learn probabilistic algorithms for robotics is to implement them

Since simulation is “doomed to succeed”, I will ask you to implement these algorithms using

real data, on a real robotJ. Leonard MIT 2.166, Fall 2008

Course Organization

5 Research AssignmentsRA1: introduction to robot programmingRA2: localization using beaconsRA3: Gaussian SLAMRA4: advanced data association techniquesRA5: laser scan matching

1 Final Project

No problem sets

No exams

2

Course StaffLectures: Prof. John Leonard ([email protected])

TA: Jacques Leedekerken ([email protected])

Help with MOOS robot programming: Dr. Mike Benjamin ([email protected])

Help with robots and software infrastructure: Andrew Patrikalakis ([email protected])

RA1 introduction to robot programming

Implement some simple robot behaviors, with inspiration from the subsumption architecture Using MOOS and C++If you’ve never done this before, we will give you lots of helpHands-on intro session tentatively scheduled for Friday 1pm in room 1-115


Outline (1)

Why Probabilistic Robotics?

Representations for Probabilistic Robotics

State Estimation in Probabilistic Robotics

Conclusion

Autonomous Underwater and Sea Surface Vehicles

3

MIT DARPA Urban Challenge Team


What has caused our recent progress in Robotics Research?

Better robot platforms?

Better computers?

Better sensors?

Better software tools?

Better algorithms?


Example: Robot NavigationWhere am I?

Where am I going?

How should I get there?



LocalizationMappingSimultaneous Localization and Mapping

Where am I going?


4

SICK laser scanner180 range returns,

one per degree, at 5-75 Hz

Polaroid sonar ring12 range returns,

one per 30 degrees, at ~4 Hz

Common range-and-bearing sensors

Other possibilities: Stereo/monocular vision; Robot itself (stall, bump sensing)

Robot

Robot

(+ servoed rotation)


Navigate by Dead Reckoning

J. Leonard MIT 2.166, Fall 2008 J. Leonard MIT 2.166, Fall 2008

Localization:Given a Map, where is the robot?

Olle Wijk, KTH, 2002

5

Mapping: Given the robot position, build a map

Example: Helicopter with GPS and a SICK laser scanner (Thrun & Haehnel)

Scan Alignment and 3-D Surface Modeling with a Helicopter Platform, FSR 2003Sebastian Thrun, Mark Diel, Dirk Haehnel

Mapping: Given the robot position, build a map


Simultaneous Localization and Mapping (SLAM):

Build a map while concurrently localizing the robot

The SLAM Summer School (Stockholm, Aug 2002)

6

Hugh Durrant-Whyte

Sebastian Thrun

SLAM aka Concurrent Mapping and Localization

Goal:enable one or more mobile robots to build a map of a large-scale unknown environment, while concurrently using that map to navigate

Technical challenges:Uncertainty

NoiseAmbiguityNavigation error

feature detection and modelingdata associationstate estimation

Sonar data (MIT Bldg. 5)

Laser Data (AAAI 2002)

Explore and return (Newman, Leonard, Neira, and Tardos 2002)

View from the robotStarting positionOverhead view of scene (MIT Lobby 7)

Final adjustmentReturn to homeCML softwarehttp://oe.mit.edu/~jleonard/icra2002.mpg

Explore and return (Newman, Leonard, Neira, and Tardos 2002)

7




Where am I going?ExplorationTask Planning





Where am I going?ExplorationTask Planning

How should I get there?Motion PlanningObstacle Avoidance

Team MIT – DARPA Urban Challenge

Excited by DARPAs missionPlus, civilian applications

Our first time out—Didn’t know what would workTried a little bit of everythingMake sure we have a big sandbox to play in

Land Rover LR3 (Talos)

Blade cluster10 blades each with two 2.33GHz dual-core processors 40 coresTotal # of processes > 40

A lot of sensorsApplanix IMU/GPS12 SICK LidarsVelodyne (~64 Lidars)15 radars5 cameras

ACPower consumption~5500WInternal generator

8

System Architecture / Planning

Perception

NavigatorMDF

Goal

Trajectory

Steer, gas/brake

Drivable surface,Lane markings, Obstacles, andVehicle Tracks

Local map

Grid MapMotionPlanner

Vehicle Controller

Vehicle

Vehicle State Estimator

SensorsSensorsSensorsSensorsSensorsSensorsSensorsRNDF

Progress

Key Contributors:

Jon How

Emilio Frazzoli

Yoshia Kuwata

Gaston Fiore

Justin Teo

David Moore

Stefan Campbell

Sertac Karaman

System Architecture / Perception

Perception

NavigatorMDF

Goal

Trajectory

Steer, gas/brake

Drivable surface,Lane markings, Obstacles, andVehicle Tracks

Local map

Grid MapMotionPlanner

Vehicle Controller

Vehicle

Vehicle State Estimator

SensorsSensorsSensorsSensorsSensorsSensorsSensorsRNDF

Progress

Key Contributors for Perception System:

Seth Teller

Luke Fletcher

Ed Olson

Albert Huang

David Moore

Matt Antone (BAE)

Olivier Koch

Key Algorithms:

o Obstacles

o Hazards

o Lane Finding

SoftwareWrote lots of codeC ~150k LOCPython ~10k LOCJava ~6k LOC

Some of it’s worth keepingSome of it other people might actually find useful

libcamLCMprocmanethcanvarious modules

All our race data is now online (plus viewing tools) http://grandchallenge.mit.edu

LCM

Lightweight Communications and MarshallingInterprocess communications (IPC) framework

Simple APIRun-time type safety

Publish/SubscribeUDP multicastFast, scalable

C, Java, Python bindings

Publicly available on google code

9

Major Subsystems

Navigator

MDF

Drivability Map

MotionPlanner

Controller

Vehicle

RNDF

SICK

Velodyne

CamerasRoad paintdetectionLanes

Staticobstacles

Curbs

Positioning IMU, GPS,Odometry

RadarsVehicles

Cameras

5 Firewire CamerasPoint Grey Firefly MV

720x480 8bpp Bayer @ 22.8 fps

~40 MB/s (2.5 GB/min)Lots of data!

Separate network

Purpose: lane detection

SICK Lidars12 SICKs

5 pushbrooms (ground-looking)7 skirts (obstacle-looking)

180 measurements, 1° spacing @ 75 Hz (dithered)

~70m range, ~1cm accuracy

Primary function:Obstacles, vehicles, curbs, road hazards

Velodyne (3D Laser Scanner)Tons of data

64 lasers360° horizontal FOV+2° -24° vertical FOV15 Hz1M points/sec

Good, but noisy!

10

Velodyne DataFalse colored by height

Radars

Delphi ACC radarsRange + bearing + closing rateNarrow beam width

Need 16 to cover ~288 degreesGood far field car detectors

GPS/IMU Navigation

GPS• 3 antennas• Absolute heading

from standstill

Odometry• High precisionexternal mountwheel encoder

• compare to 64 count forLR3 builtinodometry

IMU• Linearaccelerations

• Angularvelocities

• Fuse withGPS + odometry

Major Subsystems

Navigator

MDF

Drivability Map

MotionPlanner

Controller

Vehicle

RNDF

SICK

Velodyne


Staticobstacles

Curbs


RadarsVehicles

11

Obstacle Detection Curb Detection

Vehicles Lane Estimation

Road paintdetectors

CurbDetectors

Lane centerline estimator

Lane tracker

RNDF

12

Playback speed: 2x

Visual Lane Finding (PhD Thesis of Albert Huang)

“Multi-Sensor Lane Finding in Urban Road Networks”, Albert Huang, David Moore, Matthew Antone, Edwin Olson, Seth Teller, RSS 2008

Major Subsystems

Navigator

MDF

Drivability Map

MotionPlanner

Controller

Vehicle

RNDF

SICK

Velodyne


Staticobstacles

Curbs


RadarsVehicles

Drivability Maps

Two mapsObstaclesLanes

Cost function for motion planner

Planning System Design

Several planning challenges:Many different driving scenariosDynamic and uncertain environmentNumerous physical and logical constraintsComplex and unstable vehicle dynamics

Approach: extension of Rapidly-exploring Random Tree (RRT)

Planning on a closed-loop systemBiased samplingSafety guarantees

Navigator

Drivability Map

MotionPlanner

Controller

Car

Goal

Divider infeasible

13

DARPA Urban Challenge ResultsSurvived difficult elimination rounds and finished 4th

Key challenge: coping with uncertainty in sensing and action need more progress in probabilistic algorithms

5335

11

6

Site VisitInvited to NQE

QualifiedFinished

CMU Stanford Virginia Tech

89Initially

MIT J. Leonard MIT 2.166, Fall 2008

Outline (2)




Conclusion


Choices for Map Representation

Features (Landmarks)Occupancy GridsPoses and Laser ScansParticlesTopologicalHybrid metric/topological

1. Move2. Sense3. Associate measurements with known features4. Update state estimates for robot and previously mapped features5. Find new features from unassociated measurements6. Initialize new features

Feature-based SLAM: Points

14

http://www.informatik.uni-bremen.de/~ufrese/slamvideos1_e.html http://www.informatik.uni-bremen.de/~ufrese/slamvideos1_e.html


Features (Landmarks)Occupancy GridsPoses and Raw Laser ScansParticlesTopologicalHybrid metric/topological

Representation: Occupancy Grids

Regular grid.Free and occupied spaceProbability of presence of obstacle surfaceBayesian Update with new measurementsAssumes independancebetween grid elements

15

Representation: Occupancy Grids

Images courtesy of Olle Wijk, KTH


Features (Landmarks)Occupancy GridsPoses and Raw Laser ScansParticlesTopologicalHybrid metric/topological

Representation: Poses and Scans

Robot scans, moves, scans againShort-term odometry errorcauses misregistration of scansScan matching is the process ofbringing these scans into alignment

Ground truth (unknown)

1 2 1 2

Scan from pose 1 Scan from pose 2Olsen 2005

SLAM based onLaser scan matching

16

Vision-based SLAM using Poses and Images (R. Eustice)

Real-time fusion of “zero-drift” camera measurements with navigation

sensor data to close-the-loop on dead-reckoned error.

Seafloor Imagery

I1I2I3

IN

IN-1IN-2

I4I5I6

I7I8

Camera Constraints

Visually Augmented Navigation Estimate

Navigation Sensors(Doppler Velocity Sensor, Compass, …)

+

Eustice, Pizarro, and Singh, ICRA 2004

Results for100 Image Sequence

Temporal “Break”

DR / VAN VAN

Evolution of XY Pose Uncertainty

Loop-closing event

Eustice, Pizarro, and Singh, ICRA 2004


Outline (3)




Conclusion

Why is Probabilistic Robotics difficult?

“The key scientific and technological issue in robotics is that of coping with uncertainty ... In fact, the uncertainty is such that one of the most challenging activities for a mobile robot is simply going from point A to point B.'’

Tomas Lozano-Perez, 1990

17

Forms of uncertainty in robotic mapping

Loop closingNavigation drift

Global

Data association

Sensor noiseLocal

DiscreteContinuousUncertainty:

Scale:

What tools and techniques are available for SLAM?

Examples from the literature …Recursive least-squares/EKF Probabilistic data associationMultiple hypothesis trackingHough transformSequential Monte Carlo methods (particle filters) Random Sample Concensus(RANSAC)Bundle adjustment Variable state dimension filter

Hartley and Zisserman, © 2001

(courtesy of Udo Frese) (courtesy of Udo Frese)

18

(courtesy of Udo Frese)

Basic Linear-Gaussian SLAM (Smith, Self, and Cheeseman, 1987)

1. Move2. Sense3. Associate measurements with known features4. Update state estimates for robot and previously

mapped features5. Find new features from unassociated measurements6. Initialize new features

Feature-based SLAM

Hans Jacob Feder, MIT ME PhD 1999

Example: Land robot using laser scanner data

MIT Indoor Track

19

64 hurdles, 4 tennis courts

Example: Land robot using laser scanner data Example: Land robot using laser scanner dataGround Truth Map and Dead-Reckoning

Example: Land robot using laser scanner dataVideo of SLAM Measurement Processing

Comparison of Mapped Target Locations (blue circles) with Ground Truth (red stars)

20

Vision-based Simultaneous Localization and Mapping Joint work with Ryan Eustice & Hanu Singh, MIT/WHOI Joint Program

Remotely operated vehicle survey of RMS Titanic (Data Courtesy of Robert Ballard)

Input: 866 Images + Doppler & Inertial data

Output: Texture-mapped model and triangulated scene structure

“Exactly Sparse Delayed-State Filters for View-Based SLAM”, R. Eustice, H. Singh and J. Leonard. Accepted for publication in IEEE Transactions on Robotics, 2006.


Outline (4)

What is Probabilistic Robotics?



Conclusion

Documents

Probabilistic Techniques for Mobile Robotics Course Content · 1 Probabilistic Techniques for Mobile Robotics John Leonard MIT Department of Mechanical Engineering MIT Center for