Safe Long-Range Travel for Planetary Rovers through Forward...

Safe Long-Range Travel for Planetary Rovers through

Forward Sensing

12th ESA Workshop on Advanced Space Technologies for Robotics & Automation

Yashodhan Nevatia Space Applications Services

May 16th, 2013

2

This research project ‘Forward Acquisition of Soil and Terrain

data for Exploration Rover (FASTER)’, running from 2011 to

2014, is supported by the European Commission through the

SPACE theme of the FP7 Programme under Grant Agreement

284419

Roland Sonsalla

Martin Frische

Thomas Voegele

Thilo Kaupisch*

Yang Gao

Said Al-Milli

Rajeev Kandiyil

Chakravarthini Saaj

Marcus Matthews

Brian Yeomans

Elie Allouis

Yashodhan Nevatia

Jeremi Gancet

Francois Bulens

Stephen Ransom

Lutz Richter

Krzysztof Skocki

*Thilo Kaupisch was affiliated with DFKI at the time part of the work reported here was performed, and is currently affiliated with DLR eV.

3

Motivation

• Advances in autonomy for terrestrial robots

DARPA Grand Challenges (2004, 2005, 2007)

Google Autonomous Car

Etc…

• Planetary rovers operations

Minimal risk

Largely planned by operators so far

► Analysis, simulation based on telemetry

► Short turn around time

► Limited to line of sight operations

Still dangerous

► MER Spirit

• Future missions require greater rover capabilities!

ExoMars – mission length of ~ 9 months

Mars Sample Fetch Rover - ~ 15 km traversal

Photos: NASA/JPL-Caltech

4

Objectives

• Autonomous operations allowing safe long range traversal

Forward sensing of terrain trafficability

Allow detection of hazards with minimal risk to mission rover

Components as mission ‘add on’

• System components

Scout rover

Soil sensors

► Primary rover sensors

► Scout rover sensors

► Remote sensing

Cooperative autonomy

5

Operation Concept

• Primarily to pure traversal operations

Minimal or no science during traverse

• Ground Planning Phase

Identification and specification of traverse command

• On Board Command Procedures

Global Path Planning

Waypoint Traversal

6

Ground Planning Phase

• Preparation for a single traverse command

• Identification of target location

Can be more than one sol away

> 200 m

• Identification of potential paths

Using orbiter data

Path as set of waypoints

► Straight line between waypoints

► Estimated (directional) cost of traversal

Multiple, interconnected paths

• Paths encoded as a directional graph

Waypoints as nodes, costs as edge weights

7

Global Path Planning

Start

Traverse

O

E D

B

C

A

G

Current Location

(Start)

Target

Location

8

Global Path Planning

Start

Traverse

Find best path

O

E D

B

C

A

G

• Simple graph search

• Best path: (O, A, C, E, G)

9

Global Path Planning

Start

Traverse

Find best path Waypoint

Traversal

Path

found

Y

O

E D

B

C

A

G

• Sub procedure

• Traversal to next waypoint

10

Global Path Planning

Start

Traverse

Find best path

Waypoint

Traversal

Path

found

Success

Target

reached

N

Y

Y

O

E D

B

C

A

G

Update edge cost

11

Global Path Planning

Start

Traverse

Find best path

Waypoint

Traversal

Path

found

Success

Target

reached

N

Y

Y

O

E D

B

C

A

G

Update edge cost

12

Global Path Planning

Start

Traverse

Find best path

Waypoint

Traversal

Path

found

Success

Remove edge

Target

reached

N

N Y

O

E D

B

C

A

G

Y

Update edge cost

13

Global Path Planning

Start

Traverse

Find best path

Waypoint

Traversal

Path

found

Success

Remove edge

Target

reached

N

N Y

O

E D

B

C

A

G

Y

Update edge cost

14

Global Path Planning

Start

Traverse

Find best path

Waypoint

Traversal

Path

found

Success

Remove edge

Target

reached

N

N Y

O

E D

B

C

A

G

Y

Update edge cost

15

Global Path Planning

Start

Traverse

Find best path

Waypoint

Traversal

Path

found

Success

Remove edge

Target

reached

N

N Y

O

E D

B

C

A

G

Y

Update edge cost

16

Global Path Planning

Start

Traverse

Find best path

Waypoint

Traversal

Path

found

Success

Remove edge

Target

reached

N

N Y

O

E D

B

C

A

G

Y

Update edge cost

17

Global Path Planning

Start

Traverse

Find best path

Waypoint

Traversal

Path

found

Success

Remove edge

Target

reached

N

N Y

O

E D

B

C

A

G

Y

Update edge cost

18

Global Path Planning

Start

Traverse

Find best path

Waypoint

Traversal

Path

found

Success

Remove edge

Target

reached

N

N Y

O

E D

B

C

A

G

Traverse

success

Y

Y

Update edge cost

19

Global Path Planning

Start

Traverse

Find best path

Waypoint

Traversal

Path

found

Success

Remove edge

Target

reached

N

N Y

Y

O

E D

B

C

A

G

Traverse

fail

N

Traverse

success

Y

Update edge cost

20

Waypoint Traversal

• Similar to behaviours proposed by

Volpe et. al. [1]

‘Motion-to-Goal’ and ‘Boundary-Following’

• Rovers turn towards the next waypoint

• Build elevation map

Waypoint

21

Waypoint Traversal

• Similar to behaviours proposed by

Volpe et. al. [1]

‘Motion-to-Goal’ and ‘Boundary-Following’

• Rovers turn towards the next waypoint

• Build elevation map

• Remote soil sensing input for rocks

Waypoint

22

Waypoint Traversal

• Similar to behaviours proposed by

Volpe et. al. [1]

‘Motion-to-Goal’ and ‘Boundary-Following’

• Rovers turn towards the next waypoint

• Build elevation map

• Remote soil sensing input for rocks

• Artificial target if needed

Waypoint

23

Waypoint Traversal

• Similar to behaviours proposed by

Volpe et. al. [1]

‘Motion-to-Goal’ and ‘Boundary-Following’

• Rovers turn towards the next waypoint

• Build elevation map

• Remote soil sensing input for rocks

• Artificial target if needed

• Path planning

Waypoint

24

Waypoint Traversal

• Similar to behaviours proposed by

Volpe et. al. [1]

‘Motion-to-Goal’ and ‘Boundary-Following’

• Rovers turn towards the next waypoint

• Build elevation map

• Remote soil sensing input for rocks

• Artificial target if needed

• Path planning

• No path – Obstacle circumnavigation

Rovers allowed to rotate slightly

Waypoint

25

Waypoint Traversal

• Scout begins ‘forward sensing’

Path updates at scout location as needed (eg. hazard detection)

Extension of elevation map on reaching end of path

Constrained to within line of sight of primary rover

• Primary rover traversal

Path update from primary rover location on detection of hazard

Scout returns to primary rover location

26

Waypoint Traversal Failure

• No path found, rovers at waypoint

No new nodes needed

A

B

27

Waypoint Traversal Failure

• No path found, rovers at waypoint

No new nodes needed

• Hazard detection by scout

Addition of two connected nodes

Scout to primary, or vice versa

A

B

A1

A2

28

Waypoint Traversal Failure

• No path found, rovers at waypoint

No new nodes needed

• Hazard detection by scout

Addition of two connected nodes

Scout to primary, or vice versa

• Hazard detection by primary rover

Single new node

Scout to primary rover

A

B

A1

29

Waypoint Traversal Failure

• No path found, rovers at waypoint

No new nodes needed

• Hazard detection by scout

Addition of two connected nodes

Scout to primary, or vice versa

• Hazard detection by primary rover

Single new node

Scout to primary rover

• New nodes are connected

Last waypoint

Other nodes in line of sight? A

B

A1

C

30

Cooperative Autonomy & Software

• E3 level of autonomy

Adaptive mission operations on-board

• Partial support for E4 level of autonomy

Operation concept provides for goal based traversal

• Scout Rover

Minimal autonomy

► Path following

► Return to primary

► Proprioception

• Primary rover

Subsystem architecture based on GenoM with ROS modules

31

Primary Rover Software Architecture

32

Representative Subsystems

• Task Execution Controllers

Simple on board procedure engines

Predefined procedures

• Health Management

Offline analysis based fault detection and recovery

• Data Management

Simplified data repository supporting pull/push

• Communication

Wireless communication with scout

Simulated communication with ground

33

Representative Subsystems

• Locomotion Controller

Implements drive commands

TCP interface for Bridget Controller

• Sensor Managers

Point Grey XB3 Stereo Camera

XSens MTi IMU

34

SSS Software Chain

• Interface with soil sensing system

Traversability classification

Based on outputs of multiple sensors

• Rock tracking

via Remote Sensing

35

Task Planner

• Planning and monitoring high level tasks

• Symbolic planner

Hierarchical Task Networks

Domain configurable, multi agent framework [2]

• Generation of interleaved plans

Multiple, concurrent action sequences

Query functional modules for cost estimates

Re-planning, including contingency actions

• Validation of plans with regard to resources

36

Scout Localization

• Tracking 6DOF scout pose in images

• Combined with scout proprioception

• SURF feature based:

3D descriptor database

Generation of image-model

correspondences

• Marker based:

ARToolkit for benchmarking

Custom marker configurations

Single Marker Multi Marker

37

GNC: Localization

• Wheel and/or Inertial Odometry

• Visual Odometry

• Rock Tracking

Output from Remote sensing

Sparse feature SLAM being investigated

• Map matching

Detailed local elevation map built by rover

Low resolution orbiter height map

Bounds for odometry error

38

GNC: Mapping & Path Planning

• Elevation maps based on stereo camera images

Filtering

Iterative Closest Point for 3D data fusion

• D* based local path planning, trajectory fitting

Primary Rover

Stereo Images

Scout Rover

Stereo Images

Point Cloud

Point Cloud

Scout

Localization

Point Cloud

Merging (ICP)

DEM Generation Path Planning

Remote Soil

Sensing

39

System Validation

Simulation

Autonomy

Field Trials

Integrated

Field Trials

• Rapid development & testing

• Gazebo

Multi-robot outdoor simulation

Open Dynamics Engine

DARPA Robotics Challenge

• HiRISE imagery from MRO

40

System Validation

Simulation

Autonomy

Field Trials

Integrated

Field Trials

• Iterative validation of algorithms

• Preliminary integration with scout

• Realistic datasets

• Clearpath Husky A200

Beez Quarry, Belgium

41

System Validation

Simulation

Autonomy

Field Trials

Integrated

Field Trials

• Integrated system validation

Soil sensors

Bridget locomotion breadboard

• Field trials at Stevenage

Second half of 2014

Photo: PRoVisG Field Trial, 2011

Thank you!

Yashodhan Nevatia

Work Package Leader Space Applications Services

yn@spaceapplications.com

Chakravarthini Saaj

Technical Manager University of Surrey

c.saaj@surrey.ac.uk

Thomas Vögele

Project Coordinator German Research Center for

Artificial Intelligence

thomas.voegele@dfki.de

https://www.faster-fp7-space.eu/

43

References

[1] Volpe, R., Estlin, T., Laubach, S., Olson, C., & Balaram, J. (2000). Enhanced

mars rover navigation techniques. In Proc. IEEE International Conference on

Robotics and Automation, 2000 (ICRA'00), IEEE, Vol. 1, pp. 926-931.

[2] R. Kandiyil and Y. Gao, “A Generic Doman Configurable Planner using HTN for

Autonomous Multi-Agent Space System,” in Proc. 11th International

Symposium on Artificial Intelligence, Robotics and Automation in Space, Turin,

Italy, 2012.

44

MSR Mission Outline

Mission Surface Phases Activities Duration Units

Post Landing Checkout

Comms establishment & HK status 4 sols

Initial Post landing checkout 4 sols

Egress 4 sols

Post-landing checkout duration 12 sols

Preparation for Departure

from Landing Point

SFR commissioning 5 sols

Landing Site local exploration 2 sols

Pre-excursion duration (sols) 7 sols

Sample Cache Acquisition

Identification of target location 1 sols

travel to target location 3 sols

Verification & confirmation of target 2 sols

Sample Acquisition 1 sols

Sample Cache Acquisition Duration 7 sols

Cache transfer to MSR Lander

Identification of target location 1 sols

travel to target location 2 sols

cache Transfer 1 sols

Cache Transfer to MSR Duration 4 sols

Contingencies

Conjunction Allocation (no ops) 20 sols

Dust storm or operational contingency 20 sols

Total Contingency 40 sols

Total Mission Operations duration 70 sols

Nominal Mission Duration 180 sols

Duration left for traverse 110 sols

Traverse Distance 15 km

Traverse Margin 1.3

Minimum Rover speed 177 m/sol

45

Scout Forward Sensing

1

2 3

• Three scenarios considered with different parameters

46

Scout Forward Sensing

• Measurement time of 60 s

• Step pattern is too slow to achieve desired daily traverse

• Scenario 3 chosen as baseline

Most plausible in terms of mission size and complexity

turns

47

Extended Operations

• Exploration or scientific tasks

Very useful, but considered out of scope

• Long Term Scouting

beyond range of Primary Rover sensors/communication

delays in case of wrong trafficability from scout sensors

• Increased requirements for Scout

Fully autonomous operation

► Self localization and task planning

Increased power requirements

• Increased mission complexity

Relative localization between scout path and primary rover

Recharging of scout batteries

48

Remote Sensing

• Semantic feature detection rather than pixel-based feature

detection

• Three approaches are currently being investigated:

o Supervised machine learning classifiers (SVM & Boosted

LDA):

-1 1

Positive Patch

F(x) = 1

Negative Patch

F(x) = -1

Stage-1

(Learning)

Stage-2

Classification

49

Remote Sensing

Stage-1 Stage-2

• Blob Detection, classification, and tracking

• Saliency Detection, classification, and tracking