1

Cooperative Control of Distributed AutonomousVehicles in Adversarial Environments

2.5 Year MURI Research ReviewNovember 18, 2003

2

Team vs Team Siege

• Combat decisions

• Communication decisions

• Real-time computations

• Multivehicle (re)grouping

• Trajectory execution

• As well as…

– Above: Resource allocation

– Below: Servoloops

– Parallel: Human intervention

3

Lots of Issues & Approaches

•Hierarchy

•Distributed processing

•Distributed information

•Constrained communication

•Uncertain evolution

•Adversarial elements

•Multi-tier imposition

•Aesthetic aggregation

•Task decomposition

•Fixed structure imposition

•Discrete/continuous interaction

•Verification

•Stochastic estimation

•Hypothesis falsification

•Minimax allocation

•Game formulation

•Combinatorial optimization

•Embedded simulation

•Multi-hop communication

•Receding horizon implementation

4

Core MURI Research

• Our approach:

Extract & distill essential elements with well formulated subproblems.

• Develop core theory & understand limitations/trade-offs.

• Develop supporting computational algorithms.

• Illustrate & motivate new directions in test-bed examples.

• Recognize traceable and transportable implications.

5

Dimensions of Cooperative Control

• Distributed control & computation

The defining feature of cooperative control problems.

• Adversarial Interactions

• Uncertain Evolution

• Complexity Management

6

Evolution of Dimensions

Year 2

• Distributed control & computation

• (Virtual) Hierarchy

• Adversary & Uncertainty

• Finite-state representations

Proposal & Year 1

• Scalability, modeling & reduction

• High level planning

• Low level execution

• Communications

Year 2.5

• Distributed control & computation

• Adversarial Interactions

• Uncertain Evolution

• Complexity Management

7

Today’s Agenda

Distributed Control & Computation

Murray, Caltech & Klavins, UW

Adversarial Interactions

Speyer, UCLA

Uncertain Evolution

Dahleh, MIT

Complexity Management

D’Andrea, Cornell

Discuss on-going work across universities in context of dimension.

• Explore multiple facets of research challenge.

• Recognize multiple dimensionality.

8

Cross Dimensional Threads

• Explore multiple facets of research challenge.

• Recognize multiple dimensionality.

Enemy Models

Coordinating Actions

Constructive Algorithms

Roboflag Drill

9

• RoboFlag Drill problem with semi-intelligent targets• Encode vehicle dynamics, obstacle avoidance, target

intelligence, and group objective as a mixed integer linear program (MILP). Solving the MILP gives the optimal group strategy.

Enemy Model 1/5MILP Methods for Multi-Vehicle Systems

10

Enemy Model 2/5Linear-Programming-Based Multi-vehicle Path Planning

with Adversaries

Objective:

• Minimize the number of adversaries that enter a protected area.

• Explore the utility of Linear Programming for trajectory planning.

• Represent Enemy as “probabilistic diffusion”

Potential Advantages:

• Reduce complexity with LP’s

(versus mixed integer LP’s)

• Allow 2-sided optimization

(versus “scripted” adversaries)

11

Enemy Model 3/5Probability Map of the Environment with Moving Opponents

010

2030

4050

60

0

10

20

30

40

0

0.5

1

010

2030

4050

60

0

10

20

30

40

0

0.2

0.4

010

2030

4050

60

0

10

20

30

40

0

0.5

1

PDF of Each Opponent• Map Building

010

2030

4050

60

0

10

20

30

40

0

0.1

0.2

• Path Planning- Find a sequence of cells connecting the origin and the destination using Dijkstra algorithm- Plan a path considering the centers of the sequence of cells as waypoints

12

Enemy Model 4/5Linear Quadratic Gaussian (LQG) Differential Games with

Different Information Patterns

SolutionEvader Filter Solution has following structure.A reduced-order dynamic filter in a subspace orthogonal to the pursuer’s control input.Remaining estimated states are constructed via algebraic equations

Control strategies appear similar to deterministic LQG GameSolution is linear and finite dimensional.No linearity assumption on the strategies is made.

ProblemTwo player zero-sum LQG pursuit-evasion game

Linear dynamics and Gaussian process and sensor noiseQuadratic cost function: Q(u,v)Evader u makes noisy partial measurements of the state.Pursuer v knows the state perfectly.

Pursuer (Perfect measurement of Engagement states.)

Evader (Partial measurement Of Engagement states.)

State SpaceTo describe

Dynamic Motion of the Adversaries

13

Conventional Methods: Agents “chase” other agent behaviors

Alternative: Distributed feedback stabilization

Enemy Model 5/5Distributed Convergence to Nash Equilibria

Can individual agents reach strategic equilibrium without declaration of their intentions?

Utility: Strategic robustness…Adaptation vs fragile planning

Game Theory Literature: It can’t be done (40yrs)

Standard “Counterexample”: Anti-coordination Game

P1 wants to deviate from P2

P2 wants to deviate from P3

P3 wants to deviate from P1

Each player only has 2 moves…all can’t be satisfied

P1

P2 P3

14

Coordinating Actions 1/6Consensus in Networks with Mobile Agents and Switching

Topology

• Approach

– Design cooperative control protocols for networks of mobile agents and analyze their convergence, performance, and robustness properties.

• Accomplishments

– Theory for agreement protocols in networks of mobile agents with switching communications topology

– Analysis of speed of reaching consensus in a group of vehicles/agents based on second eigenvalue of graph Laplacian

Formation switching using balanced graphs

Attitude Alignment for Large Collections of Vehicles

15

Objective 1Objective 1 - - Want to design:

1ˆ,Pr qkk rfqqD

noisedistkk gxxE &2

2ˆ

Objective 2 -Objective 2 - Given rq, the rate of qk, find suitable D and f such that:

that parallels the classical Kalman filtering performance analysis, where:

Estim

atorE

stimator

Uk

Yk

knk qq ˆ,,ˆ Estimates of Estimates of switching switching sequencesequence

Given:Given:

System 1System 1System 1

System MSystem MSystem Mqk qkqk

Switching systemSwitching system

UkYk

M,,1kq

Coordinating Actions 2/6Mode Estimation of Switching Linear Systems

16

Coordinating Actions 3/6Observation of CCL-like Programs

Problem: Determine state of communications protocol used by a group of robots given their physical movements.

Assumptions: Protocol and motion control are described in CCL like language.

Results:

•Definitions of observability, etc. for CCL programs

•Construction and analysis of an observer that converges when the system is "weakly" observable

•Construction of an efficient observer for Roboflag drill in particular.

17

Coordinating Actions 4/6Adaptive Languages in Uncertain Environments

• Elements:

– Symbol grounding

– Language learning

– Language evolution

18

Coordinating Actions 5/6Adaptive Models in Interactive Markov Chains

• Start with two dynamically coupled systems with centralized objective.

– Each subsystem makes simplified model of other.

– Each subsystem designs local optimal controller based on modified cost.

– After simulation/experience, subsystems revise models.

• Will it converge? What is performance?

• Anticipate FP proof

Scheme Centralized Coordinated Decentralized

Miss/Thousand 21 59 173

19

Coordinating Actions 6/6Communications under Bandwidth Limitations

CentralCommand

Vehicle

Local Control

Vehicle

Local Control

Vehicle

Local Control

Wireless digital link

Generates: references and control signals.

Has access to informationabout the vehicles and adversarial environment.

Vehicle

Local Control

Strategy and path planning

20

Constructive Algorithms 1/3Flocking with Obstacle Avoidance

Stability of flocks is formalized. A flock contains a, b, and g agents with specific

tasks:a: maintains a distance d from an a agent.b: repels an a agent and exists if a exists.g: behaves like an a agent but is fixed.

Split/Rejoin and Squeezing maneuvers w/ local information.

Consensus under switching topology addressed for directed graphs.

21

Constructive Algorithms 2/3Decomposition Methods

Decomposition

• We use trajectory generation and obstacle avoidance primitives to pose cooperative planning problems such as the target assignment problem (ex. RoboFlag Drill).

• Problems are effectively reduced to combinatorial optimization problems

Complexity

• We show the target assignment problem is NP-hard.

Greedy

Branch & Bound algorithm

• Form a search tree and explore using upper and lower bounds to prune branches.

• Upper bound is computed using greedy cost to go algorithm thus you can stop at any point in your search and use the best feasible solution found from the greedy algorithm.Multi-level MPC algorithm

• For semi-intelligent targets.

• Run each level of the hierarchy in an MPC framework at rate governed by the complexity of the level. RTG > ROA > RBB

Branch & Bound

Steps

Jub

Jopt

22

P(k1,k2) := { initializers guard1:rule1

guard2:rule2

...

}

S(k1,k2):=P(k1,k2)+C(k1+1) sharing y,u

"soup" of guarded commands

composition = union

non-shared variables remain local to component programs

Constructive Algorithms 3/3CCL: Computation and Control Language

• CCL Interpreter

• Formal programming language for control and computation. Interfaces with libraries in other languages.

• Automated Verification

• CCL encoded in the Isabelle theorem prover; basic specs verified semi-automatically. Investigating various model checking tools.

• Formal Results

• Formal semantics in transition systems and temporal logic. RoboFlag drill formalized and basic algorithms verified.

CCL Protocol forDecentralized

Target Allocation

23

Roboflag Drill

• MILP Planning.

• Hierarchical decomposition.

• Model reduction.

• LP planning.

• Adaptive representations.

• CCL protocols.

• CCL observers.

24

Dimensions as Specific Core Challenges

Distributed Control & Communication

Adversarial Interaction

Uncertain Evolution Complexity Management

Decision makers (DM’s) are self-interested.

Do not have access to other DM’s intentions.

Do not have access to other DM’s information.

Static utility functions for dynamic underlying systems.

P1

P2 P3

25

Proposal: Expected Insights

• How to address scalability through modeling & decomposition.

• How to address computational complexity in hierarchical designs.

• How to develop reliable multi-layered cooperative strategies.

• How to counter adversarial actions with constrained communications.

• How to integrate local optimizations for collective performance.

• How to synchronize cooperating elements through modeling and ID.

• How to exploit neurological models to design cooperating elements.

• How to achieve reliable communications in hierarchical structures.

• How to derive adaptive languages for autonomous operations.

26

Today’s Agenda

Distributed Control & Computation

Murray, Caltech & Klavins, UW

Adversarial Interactions

Speyer, UCLA

Uncertain Evolution

Dahleh, MIT

Complexity Management

D’Andrea, Cornell

Discuss on-going work across universities in context of dimension.

• Explore multiple facets of research challenge.

• Recognize multiple dimensionality.