Bounded Optimal Coordination for Human-Robot Teams

Bounded Optimal Coordination for

Human-Robot Teams

G. Ayorkor Mills-TetteyThesis Proposal: 12th December 2008

Committee: Anthony Stentz, M. Bernardine Dias,

Michael Trick, Nicola Muscettola

2

INTRODUCTION RELATED WORK PROBLEM STATEMENT TECHNICAL APPROACH WORK TO DATE CONCLUSION

Motivation

INTRODUCTION

[IMAGE CREDITS]

3


Example Problem

INTRODUCTION

Compute:• Allocation of tasks to agents• Schedule for each agent• Route for each agent

Maximizing: (rewards for completed tasks)- (travel costs for agents’ routes)- (Cost for agents’ idle time)

4


Example Problem

INTRODUCTION

Compute:• Allocation of tasks to agents• Schedule for each agent• Route for each agent

Maximizing: (rewards for completed tasks)- (travel costs for agents’ routes)- (Cost for agents’ idle time)

5


Problem Features Tasks

Spatially distributed Multi-step Time constraints Location choice Precedence constraints Simultaneity constraints Proximity constraints

Agents Heterogeneous Capacity constraints

Locations Capacity constraints Mutual exclusion constraints

INTRODUCTION

6


Thesis

Goal: compute bounded optimal, anytime solution to complex coordination problem

Allocation of tasks to agents Schedule for each agent Route for each agent

Characteristics of application domains: Bounded optimality is important “Small” teams: 10s of agents, 10s-100s of tasks Planning time in minutes (or hours) acceptable Centralized planning acceptable

INTRODUCTION

7


Related WorkProblem Features Vehicle

RoutingMulti-Robot Task Allocation

Spatial distribution

Multi-step

Time constraints

Location choice

Precedence constraints

Simultaneity constraints

Proximity constraints

Heterogeneity

Capacity constraints

Capacity constraints

Mutual exclusion constraints

Tasks

Agents

Locations

RELATED WORK

Optimal algorithms Approximate algorithms

8


Vehicle Routing Problem

Transportation of passengers / distribution of goods between depots and final users

RELATED WORK

iD

j

l

k

9



Mathematical models – mixed integer linear programming 3-index models (e.g. Cordeau, 2006) 2-index / set-partitioning models

(e.g. Savelsbergh & Sol, 1998)

RELATED WORK

iD

j

l

kaijx a

rxrIndicates if an agent a

traverses the edge from node i to node j

Indicates if an agent a performs route r

10



Exact approaches Mathematical Programming

Heuristic Approaches Construction and improvement heuristics Tabu Search, Genetic Algorithms, Simulated

Annealing, etc

RELATED WORK

11



Problem features not addressed: Precedence & simultaneity constraints with

penalization of waiting/idle time Location choice for tasks Capacity constraints on locations Proximity constraints Mutual exclusion constraints

RELATED WORK

12


Multi-Robot Task Allocation

In a multi-robot system, which robot should execute which task?

RELATED WORK

13



Market-Based Approaches

RELATED WORK

A victim needs to be rescued at location (4, 2)

I can do it for $80

It will cost me $54 $73

$101

14



Related problem (Jones, 2007) Planning, allocating and scheduling agent interactions for

team tasks with precedence and simultaneity constraints Solution approach: Market-based approach with

combinatorial bidding and tiered auctions Limitations for this thesis:

Does not include all problem features

No optimality guarantees

RELATED WORK

15



Related problem (Koes, 2005) Planning and Execution for Multirobot Coordination Solution approach: Mathematical programming (constraint

optimization) Limitations for this thesis:

Does not include all problem features Relies on off-the-shelve solvers, solution algorithm not

customized to problem domain

RELATED WORK

16


Problem Statement

Given A set of tasks (could be multi-step) A team of heterogeneous agents A set of locations A set of constraints (compatibility, precedence,

simultaneity, proximity, mutual exclusion)

Problem is to determine optimal: Task allocation Schedule Routes

PROBLEM STATEMENT

17


Technical Approach

Mathematical programming To guarantee bounded optimality

Specifics: Create a set-partitioning formulation of the

problem, with side constraints Design a branch-and-price algorithm to solve it

TECHNICAL APPROACH

18


Technical Approach: Overview

Background Set-partitioning formulation Branch-and-bound Branch-and-price

Application to thesis problem Set-partitioning formulation with side constraints Formulation of pricing sub-problem

TECHNICAL APPROACH

19


ExampleSet-Partitioning Formulation

1P

B

3D

3P

1D

2D

2P

4D

A

4P

TECHNICAL APPROACH

20


Set-Partitioning Formulation Example

1P

B

3D

3P

1D

2D

2P

4D

A

4P

TECHNICAL APPROACH

Possible route

r0

21



1P

B

3D

3P

1D

2D

2P

4D

A

4P

TECHNICAL APPROACH

Possible route

r1

22



1P

B

3D

3P

1D

2D

2P

4D

A

4P

TECHNICAL APPROACH

Possible route

r2

23



1P

B

3D

3P

1D

4D

4P r0 r1 r2 r3 r4 r5

A: 0 0 1

B: 0 0 0 0 1

Each agent assigned to at most 1 route:

r0 r1 r2 r3 r4 r5

1: 0 0 0 = 1

2: 0 0 = 1… … … … … … …

Each task assigned to exactly 1 route:

TECHNICAL APPROACH

Set Partitioning Formulation:

Arx1

Arx2

Arx3

Brx0

Brx4

Brx0

Arx3

Brx4

Arx1

Arx2

Arx3

r0

r1

r2

r3

r4

2DA

2P

Arx5

r1 Arx5

24


Set of all agents


r0 r1 r2 r3 r4 r5

A: 0 0 1

B: 0 0 0 0 1

Each agent assigned to at most 1 route:

r0 r1 r2 r3 r4 r5

1: 0 0 0 = 1

2: 0 0 = 1… … … … … … …

Each task assigned to exactly 1 route:

Arx1

Arx2

Arx3

Brx0

Brx4

Brx0

Arx3

Brx4

Arx1

Arx2

Arx3

TECHNICAL APPROACH

Set Partitioning Formulation:

kr

krx 1

1 Mk r

kr

kir

k

x

(for each agent, k)

(for each task, i)

Set of all feasible routes for agent k

Indicates if request i is on route r

Arx5

Arx5

25


Branch-and-Bound

TECHNICAL APPROACH

P

P1 P2

20.67

22 21

min:Ar

Ar

Br

Br

Ar

Ar

Ar

Ar

Ar

Ar

Br

Br xcxcxcxcxcxc

554433221100

subject to: 15321 A

rAr

Ar

Ar xxxx

140 B

rBr xx

…

}1,0{,,,,,543210A

rBr

Ar

Ar

Ar

Br xxxxxx

)0,,,,,0( 32

31

31

31x

)0,0,0,0,1,1(x)1,1,0,0,0,0(x

Ar

Br

Ar

Ar

Ar

Br xxxxxx

54321071598615

route costs

01A

rx 11A

rx

26


Solution

1P

B

3D

3P

1D

2D

2P

4D

A

4P

10B

rx

11A

rx

TECHNICAL APPROACH

r0

r1

27


Branch-and-Price

But, there might be too many routes to enumerate explicitly. What to do? Column generation at each node of branch-and-

bound tree:

TECHNICAL APPROACH

P

P1 P2

P11 P12

At each node:• Compute the linear programming (LP)

relaxation with a subset of the routes• Solve a pricing sub-problem to find a

profitable route to include• Re-compute the LP relaxation• Repeat until no profitable routes

r0 r1 r2.

5.45.24.7r3.

r0 r1 r2. r3. r0 r1 r2. r3.

5.3 5.04.8r4.

4.9

r5.

28


Background:Branch-and-Price

Pricing sub-problem for this example:

TECHNICAL APPROACH

kNi r

kikir

kr rMkvuc

k

,min

Cost of route r for agent k

Cost modification for each task (node) on the route

Cost modification for entire route

Dual variables

Shortest route problem in a graph with modified node costs!

1 iff task i is on route rk

29


Recap Thesis Problem

Problem Features: Tasks:

Agents: Locations:

Technical Approach: Branch-and-price algorithm on set-partitioning

formulation with side constraints

TECHNICAL APPROACH

Spatially distributed, Multi-step, Location choice, Time windows,Precedence, Simultaneity, Mutual exclusion, Proximity constraints

Heterogeneous, Capacity constraints

Capacity constraints, Mutual exclusion constraints

30


Example Problem

1

A

2

3a

4a

3b

4b

B

C

Silo 1

Silo 2

Field 1

Field 2

5

3b

4b 5

6

6D

TECHNICAL APPROACH

Harvest field 1

Harvest field 2

Move field 1 grain to silo

Move field 2 grain to silo

Monitor field 1 grain unloading


1

2

3a 3b

4a 4b

5

6

31


Example Problem

Task Subtasks Compatible locations

Compatible agents

Harvest field 1 Field 1

Harvest field 2 Field 2

3. Transport field 1 grain Load

Unload

Field 1

Silo 1, Silo 2

4. Transport field 2 grain Load

Unload

Field 2

Silo 1, Silo 2



A

A

B C

B C

3a

3b

4a

4b

1

2

5

6

D

D

TECHNICAL APPROACH

32


Example ProblemPrecedence constraints:

Harvest field 1 before Load grain from field 1

Harvest field 2 before Load grain from field 2

Simultaneity constraints:

Monitor field 1 grain unloading concurrent with Unload field 1 grain

Monitor field 2 grain unloading concurrent with Unload field 2 grain

Proximity constraints:

Monitor field 1 grain unloading collated with Unload field 1 grain

Monitor field 2 grain unloading collated with Unload field 2 grain

1

2

3a

3b

3a

4b

5

6

5

6

3b

4b

TECHNICAL APPROACH

33


Example Problem:Feasible Route

1

2

3.1

4.1

B

C

A

3.2

4.2

Silo 1

Silo 2

5

3.2

4.2 5

6

6D

TECHNICAL APPROACH

Field 1

Field 2

34



1

2

3.1

4.1

B

C

A

3.2

4.2

Silo 1

Silo 2

5

3.2

4.2 5

6

6D

TECHNICAL APPROACH

Field 1

Field 2

35



1

2

3.1

4.1

B

C

A

3.2

4.2

Silo 1

Silo 2

5

3.2

4.2 5

6

6D

TECHNICAL APPROACH

Field 1

Field 2

36



1

2

3.1

4.1

B

C

A

3.2

4.2

Silo 1

Silo 2

5

3.2

4.2 5

6

6D

TECHNICAL APPROACH

Field 1

Field 2

37


Set Partitioning Formulation with Side Constraints

Objective Function

1 route per agent 1 route per task

Valid start time for taskValid arrival time for agent for task

Valid idle time for agent for task

Synchronization constraints



Non-overlapping constraints


Location capacity constraints

Maximize:

Subject to:

TECHNICAL APPROACH

Sid

e co

nstr

aint

sS

tand

ard

set-

part

ition

ing

form

ulat

ion

38


Objective Function

Maximize

Rewards for completed tasks

Travel costs for selected routes

Costs for agents’ idle/waiting time

TECHNICAL APPROACH

39


Objective Function

Maximize

Mj Kk r

kr

kjrj

k

xr

Kk r

kr

Rkr

k

xc

Ni l Kk

lki

Wk

i

wc

Rewards for completed tasks

Travel costs for selected routes

Costs for agents’ idle/waiting time

Route selection variable: 1 if agent k is assigned route r

Idle/waiting time of agent k for subtask i at location l

TECHNICAL APPROACH

40


Set Partitioning Formulation with Side Constraints

Objective Function

1 route per agent 1 route per task

Valid start time for taskValid arrival time for agent for task

Valid idle time for agent for task

Synchronization constraints



Non-overlapping constraints


Location capacity constraints

Maximize:

Subject to:

TECHNICAL APPROACH

Sid

e co

nstr

aint

sS

tand

ard

set-

part

ition

ing

form

ulat

ion

41


Pricing Sub-problemTECHNICAL APPROACH

Reward for each completed task

∆cost for entire route

∆cost for each subtask

linear ∆cost for each subtask

∆cost for each subtask in a precedence/simultaneity constraint

∆cost for each subtask at a capacity-constrained location

∆cost for each subtask in a non-overlapping constraint

∆cost for each subtask at a location with mutual-exclusion

Find route (sequence of subtask/location pairs) that maximizes:

+

route

-Travel cost-

-

-

--

--

Subject to constraints:

Each route includes only 1 location per subtask

All subtasks of a given task are on the same route

∆reward for each completed task -

∆cost for each subtask in a proximity constraint

42


Work to Date

Implemented branch-and-bound algorithm for simplified version of thesis problem Includes precedence constraints Does not include other side constraints Explicitly enumerates all routes – does not involve

column generation

WORK TO DATE

43


Example Problem: 12 single-step tasks, 5 agents, max 3 tasks/agent

WORK TO DATE

Tasks

Agents

44


Example Problem: Optimal Solution 12 single-step tasks, 5 agents, max 3 tasks/agentNo precedence constraints

WORK TO DATE

0 5 10 15 time

Timeline:

45


Example Problem: Optimal Solution12 single-step tasks, 5 agents, max 3 tasks/agent2 precedence constraints: t4<t8, t9<t8

WORK TO DATE

0 5 10 15 time

Timeline:

46


Example Problem: Optimal Solution12 single-step tasks, 5 agents, max 3 tasks/agent4 precedence constraints: t4<t8, t9<t8, t8<t5, t5<t3

WORK TO DATE

0 5 10 15 time

Timeline:

Waiting/idle time

47


Quantitative Results

WORK TO DATE

48


Bounded Optimality:Example with 4 precedence constraints

# Iterations0 100 200 300 400 500

30,000

25,000

20,000

15,000

10,000

5,000

0

So

luti

on

Val

ue

Legend

Best solution found

Upper bound

Found optimal solution

Proved optimality of solution

49


Overview of Remaining Work

Develop branch-and-price algorithm Extend implementation to include all side constraints Develop algorithms to solve pricing sub-problem Implement column generation

Develop heuristic algorithms for comparison Evaluate branch-and-price against heuristic

algorithms Solution quality Solution time

WORK TO DATE

50


Schedule

CONCLUSION

Dec ’08 – Jan ‘09 Formulate benchmark problems

Jan ‘09 Implement omitted side constraints

Feb – Apr ’09 Design and implement algorithms for pricing sub-problem

Apr – May ’09 Implement column generation

Jun – Jul ’09 Evaluate branch-and-price algorithm, develop comparison heuristic approaches

Aug – Sep ’09 Refine algorithms and continue evaluation

Oct – Nov ’09 Write thesis

Dec ’09 Defend Thesis

51


Contributions

First branch-and-price approach to optimal robotic team coordination

Set-partitioning formulation of novel complex coordination problem

First optimal task allocation approach enabling cooperation in human-robot teams

CONCLUSION

52


Acknowledgements

Sponsors: Qatar National Research Fund (QNRF) NASA Jet Propulsion Laboratory (JPL)

Thesis Committee: Anthony Stentz, M. Bernardine Dias,

Michael Trick, Nicola Muscettola rCommerce Lab Members Friends and Family

CONCLUSION

Questions?

54


Image Credits

“Motivation” Slide: Construction site:

http://www.freefoto.com/images/13/12/13_12_2_prev.jpg Combine harvester:

http://www.freefoto.com/images/07/37/07_37_65_prev.jpg Tractor-trailer:

http://www.freefoto.com/images/07/37/07_37_70_prev.jpg Disaster response (with people & cranes):

http://anitokid.blogspot.com/2008/05/china-earthquake-leaves-students.html

Earthquake: http://www.cemeteryspot.com/blog/?p=180

Extra slides

56



Transportation of passengers / distribution of goods between depots and final users

Some variants: Capacitated vehicle routing with time-windows Multi-depot vehicle routing problems Pickup and delivery / dial-a-ride

RELATED WORK

iD

j

l

k

57



Gerkey and Matarić’s (2004) categorization of MRTA problems

RELATED WORK

ST

MT

SR

MR

IA TA

Single versus multi task robots

Instantaneous versus time-extended assignment

Single versus multi robot tasks

Our problem:

(ST-SR-TA or ST-MR-TA) + inter-task constraints+ location choice

58



1P

B

3D

3P

1D

2D

2P

4D

A

4P

TECHNICAL APPROACH

Possible route

r3

59



1P

B

3D

3P

1D

2D

2P

4D

A

4P

TECHNICAL APPROACH

Possible route

r4

60



1P

B

3D

3P

1D

2D

2P

4D

A

4P

TECHNICAL APPROACH

Possible route

r5

61



2

3.1

4.1

B

C

A

1

3.2

4.2

Silo 1

Silo 2

5

3.2

4.2 5

6

6D

TECHNICAL APPROACH

Field 1

Field 2

62



2

3.1

4.1

B

C

A

1

3.2

4.2

Silo 1

Silo 2

5

3.2

4.2 5

6

6D

TECHNICAL APPROACH

Field 1

Field 2

63



1

2

3.1

4.1

B

C

A

3.2

4.2

Silo 1

Silo 2

5

3.2

4.2 5

6

6D

TECHNICAL APPROACH

Field 1

Field 2

64



1

2

3.1

4.1

B

C

A

3.2

4.2

Silo 1

Silo 2

5

3.2

4.2 5

6

6D

TECHNICAL APPROACH

Field 1

Field 2

65



1

2

3.1

4.1

B

C

A

3.2

4.2

Silo 1

Silo 2

5

3.2

4.2 5

6

6D

TECHNICAL APPROACH

Field 1

Field 2

66


Dual Variables

67


Details of Pricing Sub-problem

Documents

Bounded Optimal Coordination for Human-Robot Teams