Odin’s JourneyDevelopment of Team Victor Tango’s Autonomous

Vehicle for the DARPA Urban Challenge

December 9th, 2007

Dennis HongAssistant ProfessorRobotics & Mechanisms Laboratory (RoMeLa)dhong@vt.edu

Jesse HurdusGraduate StudentUnmanned Systemsjhurdus@vt.edu

Presentation Outline

Introduction

System Overview

Software

Qualifying and Final Race

The DARPA Grand Challenge

DARPA Grand Challenge I

Cliff was a ClubCarconverted in only a few months and on a shoestring budget.

One of three vehicles to finish the Qualifying course without hitting anything

Completed about 100 yds in the desert before a drive belt failed

DARPA Grand Challenge II

Virginia Tech had two entries in the 2005 Grand Challenge

Cliff was rebuilt for another try at the desert

Rocky was built to test new software algorithms

DARPA Grand Challenge II: ‘The Desert’

Finished 8th and 9th, both vehicles traversed about 40 miles of the courseRocky had a generator failure, Cliff an engine shutdown

Final course was 130 miles long with a 10 hour time limit

DARPA Announces Urban Challenge!

Race set for November 3rd, 2007

1st Place: $2,000,0002nd Place: $1,000,0003rd Place: $500,000

$1,000,000 research contracts to given to 11 teams, including - Stanford University- Carnegie Mellon University- MIT- Cornell University- California Institute of Technology- Honeywell Aerospace- Raytheon- Virginia Tech!!

Team VictorTango was formed

Advisory Board+

The Urban Challenge addressed city driving

Navigate approximately 60 miles of roads in 6 hours

Obey California State driving regulations and interact with moving vehicles

Master many driving behaviors:• Stop, check, and go• Vehicle following• Lane changing• U-turns• Parking

All of this must be completely autonomous

1. Route Network Definition File (RNDF):Defines road segments and lane widths, zones, parking spots, stop lines, and possible waypoints throughout the course

2. Mission Definition File (MDF):Defines the current vehicle mission, which includes active checkpoints to achieveand speeds limits to obey

building

stop line

waypoints

checkpoints 1

4

32

5

A Priori data was supplied in two course definition files

Odin was VictorTango’s Urban Challenge entry

VT System: Modified 2005 Ford Hybrid Escape (Odin)

Control and Monitoring:• Electronic Control of

Steering, Throttle, Shift• Linear Brake Actuator• E-Brake, CAN Bus

Three computers on board:• Windows Quad Core Server• Linux Quad Core Server• NI RIO (FPGA & Controller)

Power Distribution

Uses the 330 VDC and 12 VDC from the Ford Escape Hybrid battery pack

2 kW of power available for added components

Continuous runtime with the vehicle charging capabilities

Sensor Configuration Overview

100m

Sensor Ranges

NovAtel Propack GPS/IMU provides localizationSICK laser range finders perform road and small obstacle detectionIBEO ALASCA laser scanners provide obstacle detectionCameras are used for vision line finding and object classification

Entire Software Architecture

Driving BehaviorsProduce target points for high-

level maneuvers (Lane Change, Passing, Intersection

Precedence, Parking)

OptimalRoute(Exits)

Motion PlannerAchieve target points, Follow

roads, Avoid obstacles

Vehicle InterfaceControl vehicle speed and curvature, signals, E-stop

Route PlannerDetermine list of exits to

achieve checkpoints. Replan to save time or upon request.

Road DetectionLocate drivable area, lanes,

and stop lines

Object ClassificationLocate and classify dynamic

and static obstacles

Dynamic Obstacle Predictor Predict likely paths of dynamic

obstacles, based on traffic rules

LocalizationCompute global position, local

position, and velocity state

User InterfaceProvide interface to set MDF,

RNDF, and Vehicle Mode

RequestReplan

Target Points,Max Speed,Lane Mgmt,

Direction

PointsAchievable?

Curvature,Speed

SteeringAngle

Blockage/Traffic

Updates

TurnSignals

Vehicle-SpecificDBW Signals

Drive-by-Wire (DBW)

PlatformSpecs

Control Mode

Stop Line Position

Emergency Stop

Status

Monocular Vision

IBEO LIDAR

GPS/INS

SICK LIDAR

Sensor InterfaceAcquire, timestamp, and

log sensor data

Sensor InterfaceAcquire, timestamp, and

log sensor data

Sensor InterfaceAcquire, timestamp, and

log sensor data

Sensor Dataand Specs

Sensor Dataand Specs

Sensor Dataand Specs

Health MonitorTrack health of all software

components

StatusLane Position

Local Position andVelocity State

Static Obstacles

Obstacle Predictions

Drivable Area

Dynamic Obstacles

Steering Angleand Odometry

JAUS Communication

Software uses JAUS conventions and messages

Software uses standard defined messages as well as custom messages

Software can be moved without reconfiguration

Node 1

Node 2

NodeManager

NodeManager

Component A

Component B

Component C

Perception Software Overview

Raw data is taken in from each of the sensing systems

Perception Modules

Localization

Object Classification

Road Detection

Sensors Planning Modules

Perception Messages

Drivable Area

Lane Position

Dynamic Obstacle Predictor

Position,Velocity

Dynamic Obstacle Predictions

Static Obstacles

Perception Modules process the raw data looking for specific types of informationPerception Messages are transmitted via JAUS to Planning ModulesPerception Modules output sensor independent Perception Messages

Obstacle Classification & Prediction

IBEO Alasca laser scanner returns segmented obstacles as well as raw scan data

Classification software uses additional velocity and size constraints for classification

Classification results verified with vision

Prediction software uses lane data and history to predict future actions

Road DetectionRNDF Based:• Cubic spline interpolation

generated between RNDF points• Georeferenced imagery allowed

for tweaking in complex areas

Vision Based:• Initial processing based on color • Road coverage is formed with

morphological operations

LIDAR Based:• Curb detection algorithm uses

statistical methods• Tracks around low visibility

corners and brief occlusions

Planning Software Overview

The Route Planner component focuses on mission level goals, specifically which roads are traveled.

The Driving Behaviors component is responsible for following the rules of the road, such as when to proceed at an intersection.

Motion Planning controls the vehicle path, avoiding obstacles and setting travel speed.

Vehicle Interface executes the motion commands

Route Planning

Route planner chooses roads to travel and produces sequence of exits

Minimizes total travel time using A* search

Receives updates:– Blockages– Traffic speed

A Planned Route

Driving Behaviors

Driving behaviors makes decisions concerning the rules of the road

Multiple behaviors are integrated based off the current situation

Directs motion planning with target points and a behavior profile

Classified Situation:Intersection Stop Checkpoint

DisabledVehicle

Odin

Route Driver

Passing Driver

Classified Situation:Standard Road

1. My turn?2. Intersection clear?

Special Considerations in Driving Behaviors

State machines within behaviors are used to deal with noisy perception data

A winner takes all strategy is used to deal with competing behaviors

Passing Driver

RouteDriver

Blockage Driver

Behavior Integrator

Desired Lane

Desired Speed

Desired Turn Signal

Motion Planning

Motion Planning

Motion planning produces vehicle commands to reach goal points and decides if path is blocked

Speed limiter software monitors stop points and other vehicles

Searches achievable trajectory space to plan final travel path

Speed Limiter

Trajectory Search

Vehicle Interface

Cost Map Creation

Cost map stores cost of area around vehicle

Vehicle is at bottom center of map

Costs from– Obstacles– Lane

Cost increases with obstacle proximity Vehicle

Obstacles

Lane Center

Lane Boundary

Trajectory Search

X

Search algorithm used to evaluate sequences of actions

For each action, vehicle motion is simulated

Vehicle motion is checked against cost map and goal

Trajectory SearchSearch algorithm used to

evaluate sequences of actions

For each action, vehicle motion is simulated

Vehicle motion is checked against cost map and goal

Actions are combined into a final path

Vehicle Modeling• Simple bicycle model

used• Model used to

determine vehicle path for cost map

φ

L

Vehicle Parameters• Wheel Base• Understeer

Coefficient

State Variables• Heading• Steering Angle• Position (x,y)

Inputs• Forward Velocity• Steering Rate

Motion Planning Example

Software Starts in Simulation

Simulation provides many advantages– Controlled environment– Division of labor– Visual Overlays

Simulation step increases team safety

Simulation visualization also used during field testing

Simulated 4-way Stop

Dynamic Obstacle Prediction

Odin ready for competition

National Qualification Event: Oct. 2007, Victorville, CA

35 teams traveled to Victorville for NQE

DARPA presented 3 courses to test the teams: A, B, & C

Course requirements were unknown until arrival at the site

The Competition

Many different approaches to the problem were represented

- Massachusetts Institute of Technology

- Case- Western- Osh-Kosh- Insight Racing

NQE Course A tested traffic behaviors

Odin was first up and performed well

…but not all teams did as well

NQE B tested navigation and static obstacles

Odin started well, but had difficulties

Odin encountered some unexpected situations

‘The Gauntlet’ caused the most difficulty– Unnecessary blind spot

check– Lane widths too thin– Had to pass in-between

lanes

Follow-up Testing in Practice Area

NQE C tested intersections and blockages

Odin was first vehicle to fully complete NQE C

Odin was quickly selected for the finals

Second of six teams to receive early selection

Five more teams selected after further NQE runs

DARPA fell short of 20 team target

Urban Challenge Final Event

Practice start revealed interesting complications

So many robots had never been put so close together!– Sensor interference– GPS outages– Safety radio system failures– Odin had no problems– DARPA changed start procedure

November 3rd, 2007 – DARPA Urban Challenge

Bot-on-Bot Interactions were common

Bot-on-Bot Interactions were common

After almost 6 long hours, Odin crossed the finish

The next day, the places were announced

VictorTango took 3rd place, winning $500,000

Tartan Racing was 1st, Stanford 2nd

Only 6 of the 11 teams completed the course

The Celebration

Questions?

Reference Slides

Dynamic Obstacle Filtering

Motion Planning

Target Points, Speed, Lane

Vehicle Interface

Speed, Curvature,

Acceleration

Navigation

Parking

ZONE

Lane

Segment

U-turn

SEGMENT

Intersection

Segment

Zone Speed LimiterMonitors dynamic obstacles and stop lines to manage desired speed

Pause?

MOTIONPLANNING

Driving Behaviors

Object Data

Road Detection

Driving Behaviors

Vehicle Localization

Achievable Trajectory SearchDetermines fastest path through the given environment based on:• Drivable area (lane/zone/segment)• Static obstacles• Speed• Vehicle dynamics

INPUTS

OUTPUTSTarget PointsAchievable?

Computing Systems

Only three computers on board• Windows Quad Core Server

– Vision processing• Linux Quad Core Server

– Multiple virtual machines• NI RIO

– Integrated FPGA and controller– Controls drive by wire

components

GPS / INS

Novatel SPAN System consisting of:• Novatel Propack LB plus GPS• Honeywell HG1700 IMU

OmniStar HP corrections

Position further refined by localization software filter incorporating:

• Wheel speeds• Stop line location

Sick LIDAR

Single plane LIDAR common in robotics

Mounted on roof pointed downwards

Used to profile terrain• Road Detection• Negative Obstacle Detection

IBEO LIDAR

Using IBEO’s A0 scanner as well as latest XT Fusion System

Multi-plane and Multi-pulse technology

System includes processor to fuse sensors and classify obstacles

Cameras

Odin uses a pair of industrial firewire cameras

Only monocular image processing techniques used (no stereo)

Vision used for road detection, lane segmentation and obstacle classification

Lane Finding

Lanes are determined by analyzing detected road area and lane markings

A priori lane information is used for initial lane position estimate

Line/curve fitting is used to find road markings from camera images

Example 1: Cutting Off Traffic

• Problem narrowed to a bug in the IBEO ECU software

• To avoid object overflow, a region of interest is used

• This region of interestwas not workingproperly

• Rectangular ROIs were chosen

OdinCommanded ROI

Actual ROI

Odin also encountered a GPS problem

Drove off course on to curb– Determined to be GPS “pop”

problem– 10m pop due to unknown

phenomena– Other teams encountered

similar problems– No solution found during

eventNQE B run otherwise successful

Site Visit: July 29, Blacksburg, VA

DARPA down selected from 86 to 36 teams based on Site Visit performances

Tested Skills:– Basic navigation– Intersection behavior– Passing disabled vehicles

Bot-on-Bot Interactions were common