April 21, 2023

Decentralized Mission Planning for Heterogeneous Human-Robot Teams

Sameera Ponda

Prof. Jonathan How

Department of Aeronautics and AstronauticsMassachusetts Institute of Technology

2

Motivation

• Modern day complex missions involve networked teams of heterogeneous agents executing several tasks simultaneously:

– Unmanned aerial vehicles (UAVs) – target tracking, surveillance– Human operators – classify targets, monitor status– Ground convoys – rescue operations

• Key Research Questions:– How can we coordinate team behavior to improve mission performance? – How should planning strategies evolve as we acquire more information?

3

Problem Statement

• Goal: Automate task allocation to improve mission performance – Spatial and temporal synchronization – Reduce costs and improve efficiency

• Key Technical Challenges:– Combinatorial decision problem – computationally intractable (NP-hard) – Complex agent modeling & constraints (stochastic, non-linear, time-varying)– Limited resources (bandwidth, fuel, etc)– Dynamic networks and communication constraints – Unknown and dynamic environments

Agent2

Task2

Agent1 Agent4

Agent6

Agent3

Agent5

Task1

Task3

Task7

Task6 Task5

Task4

Task9

Task8

4

Planning Approaches

• Optimal solution methods are computationally intractable for large problems– Typically use approximation methods

• Centralized Planning approach– Mission Control Center (MCC) plans & distributes tasks for all agents– High bandwidth, slow reaction, resource intensive

• Recent research in Decentralized Planning – Individual agents make their own plans and coordinate with each other– Faster reaction to local information changes– Trade-off between communication and computation

• Key Questions:– What quantities should the agents agree upon?

• Information / tasks & plans / objectives / constraints– How do we ensure that the planning is robust to inaccurate information and models?

Agent1

Agent4

Agent2

Agent3

Agent4

Agent2

Agent3

MCC

Agent1

5

Consensus-Based Bundle Algorithm

• Decentralized task allocation approach called Consensus-Based Bundle Algorithm (CBBA) [Choi, Brunet, How 2009]

– CBBA iterates between 2 phases: Bidding & Consensus

• Core features of CBBA:– Polynomial-time decentralized algorithm with provably good approximate solutions– Consensus on task assignments, not information – guaranteed real-time convergence

even with inconsistent information

Phase 2: Consensus

(all agents)

All agents consistent?

Yes

No

Phase 1: Build Bundle & Bid on Tasks (individual agents)

3

2

N

1

Key extensions to CBBA:1) Temporal constraints – Time-windows of validity for tasks

2) Connectivity issues and constraints

3) Planning for teams with Humans-in-the-loop

6

CBBA with Time-Windows

• In realistic missions, task scores often depend on arrival times and have associated time-windows of validity:

• Issue: Planning algorithms usually involve time discretization– Extra planning dimension – computationally intractable!

• CBBA extended to include time-windows– Solution does not discretize time!– Preserves convergence properties– Planner decides arrival times, producing

task schedules for agents

• Embedded CBBA with Time-Windows

into a real-time system architecture

Arrival Time Arrival Time

Time-critical Peak-timeFlat

Arrival Time

e.g. monitor status, security shifts

e.g. rescue ops, target tracking

e.g. rendezvous, special ops

7

CBBA with Time-Windows

• CBBA successfully used in real-time fight test environments– Cooperative search, acquisition, and track (CSAT)– Coordination of agents under dynamic network topologies

• Further information available online at: http://acl.mit.edu/projects/cbba.html

8

Connectivity: Network Challenges

• As agents move around in the environment, expect varying network topologies– Limited communication radius between agents– Potential broken comm links and/or disconnected networks

• Main issue: Planner cannot converge with a disconnected network, leading to conflicting assignments

• Developed two solution approaches:– CBBA with Relays – Creates relay tasks to ensure connectivity– CBBA with Network Handling Protocols – Protocols to adjust task lists prior to planning

Task1

Agent2

Task2

Agent1

Agent4

Agent6

Agent3

Agent5

Task3

Task7Task6 Task5

Task4

Task9

Task8

Disconnected Network

9

Connectivity: CBBA with Relays

• Extended CBBA to include relay tasks – (Published in GlobeComm 2010)

– Employs underutilized agents as relays– Key feature: Agents use bid info to predict

network structure at select times– Guarantees connectivity– Computationally efficient - converges in real-time

• CBBA with Relays improves team performance and network connectivity

Relay Task

10

Connectivity: Network Protocols

• If preventing disconnects is too conservative: Network Handling Protocols to adjust task lists for agents prior to planning – (Published in ACC 2010)

• Local Adjustment improves mission performance

with low bandwidth and computation requirements

Baseline (no adjustment)

Central Adjustment

Local Adjustment

All tasks available to all agents

MCC distributes tasks to networks at each replan

MCC distributes new tasks to closest agents (once per task)

Low Bandwidth High Bandwidth

Low Bandwidth

Low Computation

High Computation

Low Computation

Conflicting Assignments – lower mission scores and wasted fuel

Guaranteed Deconfliction – higher mission scores and lower fuel consumption

11

Planning for Human-Robot Teams

• Most modern missions involve human-robot teams– Human operators perform several tasks

(e.g. supervisory, target classification, monitoring)– Need to coordinate robotic agents and operators

• Main Issue: Operator performance is stochastic– Heterogeneous operator capabilities (“slow” vs. “fast”)– Robustness to uncertainty in team performance

• Recent research has explored modeling operators using probabilistic distributions – [Cummings et al ‘10]

• Key Challenge: Incorporate uncertainty into planner to increase robustness

Log-Normal Distribution for Operator Target Identification

Figure from [D. Southern, Masters Thesis, 2010]

Predator UAV Operations – Associated Press

Planning for Human-Robot Teams

• Consider a time-critical mission with operators performing target classification

• As expected vs. actual service times differ, planner performance degrades– Adding a margin of conservatism can mitigate this problem– Tradeoff between late penalties and number of tasks assigned

• Simulation Observations:– Performance is best when expected & actual

are close (ridge line)– Steeper drop for overestimating (optimistic)

vs. underestimating (conservative)– Conservative Planning performs better

than Optimistic Planning

• Developing a Robust Planning Framework– Explicitly embed PDFs of plan parameters– Adapt as estimates improve

0

0.2

0.4

0.6

0.8

1

0

0.2

0.4

0.6

0.8

1

50

100

150

200

250

300

350

400

450

500

550

Actual OperatorPercentile

Mission Performance

Expected OperatorPercentile

Mis

sio

n S

core

Optimistic Planning

ConservativePlanning

Time

Ta

sk

sAgent Schedule

Late!

12

Planning for Human-Robot Teams

• Currently performing Human-in-the-loop experiments at Cornell

– CBBA used to allocate targets to agents (MIT)– Image processing and sensor fusion used to

update target PDFs (Cornell)– Human-in-the-loop for target classification and

PDF updates through HRI (Cornell)

13

14

Conclusions

• Explored strategies to coordinate team behavior to improve mission performance

• Extended the Consensus-Based Bundle Algorithm (CBBA) to address the demands of more realistic multi-agent mission planning

– Included task time-windows of validity– Addressed connectivity issues and communication constraints– Explored planning for heterogeneous human-robot teams

• Current research and expected thesis contributions:

1) Robust decentralized planning framework• Embed distributions of parameters into planner• Preserve computational tractability and scalability

(e.g. avoid discretization, explore efficient sampling techniques)

2) Flexible planner structure that adapts to dynamic uncertainty representations• Modular uncertainty representations (Nonparametric Bayesian models, etc)• Modify planning strategy without recomputing all scenarios

3) Efficient strategies for information consensus to improve planner performance• Decide what information and when to share (e.g. hyperparameter consensus) • Cooperative decentralized strategies to update global distributions