IPAM, 3 Mar 06R. M. Murray, Caltech1 Cooperative Control of Multi-Vehicle Systems Richard M. Murray Control and Dynamical Systems California Institute

IPAM, 3 Mar 06 R. M. Murray, Caltech 1

Cooperative Control of Multi-Vehicle Systems

Richard M. Murray

Control and Dynamical Systems

California Institute of Technology

Domitilla Del Vecchio (U Mich) Bill Dunbar (UCSC) Alex

Fax (NGC) Eric Klavins (U Wash)

Reza Olfati-Saber (Dartmouth)

Vijay Gupta Zhipu Jin Demetri Spanos

Abhishek Tiwari Yasi Mostofi

Cedric Langbort (CMI/UIUC)


Arbiter

computersfor each vehicle

Humans(2-3 per team)

Example: RoboFlag (D’Andrea, Cornell)

Robot version of “Capture the Flag”• Teams try to capture flag of opposing

team without getting tagged• Mixed initiative system: two humans

controlling up to 6-10 robots• Limited BW comms + limited sensing

D’Andrea & MACC 2003


QuickTime™ and aCinepak decompressor

are needed to see this picture.

RoboFlag Demonstration

Integration of computer science, communications, and control• Time scales don’t allow standard abstractions to isolate disciplines• Example: how do we maintain a consistent, shared view of the field?

Higher levels of decision making and mixed initiative systems• Where do we put the humans in the loop? what do we present to them?• Example: predict “plays” by the other team, predict next step, and react

Hayes et alACC 2003

Red Team view

Flagcarrier

Taggedrobot (blue)

Obstacle


Outline

I. Cooperative Control subproblems

II. Information Flow for Consensus and Formation Control

III. Performance and Robustness

IV. Distributed receding horizon control

V. Protocols for cooperative control


Context

Cooperative control of large collections of vehicles• Vehicles linked by task rather than dynamics• Distributed computation “consensus” required• Point to point communications; limited BW, noisy links• Dynamic and uncertain environments

Avenues of research• Stability and performance of interconnected systems• Robustness and reconfiguration of vehicle teams• Control over networks (“packet-based control theory”)• Higher-level decision making and autonomy

Model system for other applications• Air traffic control• Electric power networks• Congestion and router control

Integrated control,communications,

computation


RoboFlag Subproblems

Goal: develop systematic techniques for solving subproblems• Cooperative control and graph Laplacians• Distributed receding horizon control• Verifiable protocols for consensus and control

1. Formation control

• Maintain positions to guard defense zone

2. Distributed estimation

• Fuse sensor data to determine opponent location

3. Distributed consensus

• Assign individuals to tag incoming vehicles

Implement and testas part of annual RoboFlag competition


Information Flow in Vehicle Formations

Example: satellite formation• Blue links represent sensed

information• Green links represent

communicated information

Sensed information• Local sensors can see some subset of nearby

vehicles• Assume small time delays, pos’n/vel info only

Communicated information• Point to point communications (routing OK)• Assume limited bandwidth, some time delay• Advantage: can send more complex

information

Topological features• Information flow (sensed or communicated)

represents a directed graph• Cycles in graph information feedback loops

Question: How does topological structure of information flow affectstability of the overall formation?


Sample Problem: Formation Stabilization

Goal: maintain position relative to neighbors• “Neighbors” defined by graph• Assume only sensed data for now• Assume identical vehicle dynamics, identical

controllers?

Example: hexagon formation• Maintain fixed relative spacing between left and

right neighbors

Can extend to more sophisticated “formations”• Include more complex spatia-temporal constraints

( )i

i j i j ijj N

e w y y h∈

= − −∑relativeposition

weightingfactor

offset

1 2

3

45

6

1 2

3

45

6


Graph Laplacian

Construction of (weighted) Laplacian

A = adjacency matrix

D = diagonal matrix, weighted by outdegree

Properties of Laplacian• Row sum equal 0 (stochastic matrix)• All eigenvalues are non-negative, with at

least one zero eigenvalue (from row sum)• Multiplicity of 0 as an eigenvalue is equal

to the number of strongly connected compo-nents of the graph

• All eigenvalues lie in a circle of radius one centered at 1 + 0i (Perron Frobenius)

• For bidirectional (eg, undirected) graphs, eigenvalues are all real, in [0,2]

1 11 0 0 0

2 20 1 0 0 1 0

1 1 10 0 1

3 3 30 0 0 1 1 0

1 1 10 0 1

3 3 31 1

0 0 0 12 2

L

⎡ ⎤− −⎢ ⎥⎢ ⎥

−⎢ ⎥⎢ ⎥

− − −⎢ ⎥⎢ ⎥=

−⎢ ⎥⎢ ⎥⎢ ⎥− − −⎢ ⎥⎢ ⎥

− −⎢ ⎥⎢ ⎥⎣ ⎦

1L I D A−= −


Mathematical Framework

Analyze stability of closed loop

• Interconnection matrix, L, is the Laplacian of the graph

• Stability of closed loop related to eigenstructure of the Laplacian

ˆ ( )K s ˆ( )P s

yL I⊗

y

h

e u


Stability Condition

Theorem The closed loop system is (neutrally) stable iff the Nyquist plot of the open loop system does not encircle -1/i(L), where i(L) are the nonzero eigenvalues of L.

Example

ˆ ( )K s ˆ( )P s

yL I⊗

y

h

e u

2( )

seP s

s

τ−

= ( ) d pK s K s K= +

Fax and MurrayIFAC 02, TAC 04


Spectra of Laplacians

0,1 =

Unidirectionaltree

2 ( 1) /1 i j Ni e π −= −

Cycle

[0,2] ∈

Undirected graph

1 0, 2N = =

Periodicgraph


Example Revisited

Example

• Adding link increases the number of three cycles (leads to “resonances”)• Change in control law required to avoid instability• Q: Increasing amount of information available decreases stability (??)• A: Control law cannot ignore the information add’l feedback inserted

2( )

seP s

s

τ−

= ( ) d pK s K s K= +

x

x

x

x


QuickTime™ and aMicrosoft Video 1 decompressorare needed to see this picture.

Improving Performance through Communication

Baseline: stability only• Poor performance due to interconnection

Method #1: tune information flow filter• Low pass filter to damp response• Improves performance somewhat

Method #2: consensus + feedforward• Agree on center of formation, then move• Compensate for motion of vehicles by

adjusting information flow



Fax and MurrayIFAC 02


Special Case: Consensus

Consensus: agreement between agents using information flow graph• Can prove asymptotic convergence to single value if graph is connected

• If wij = 1/(in-degree) + graph is balanced (same in-degree for all nodes) all agents converge to average of initial condition

Variations and extensions (Jadbabaie, Leonard, Moreau, Morse, Olfati-Saber, Xiao, …)

• Switching (packet loss, dropped links, etc)• Time delays, plant uncertainty• Nearest neighbor graphs, small world networks, optimal weights• Nonlinear: potential fields, passive systems, gradient systems

-1

x

x

x

x


Open Problems: Design of Information Flow (graph)

How does graph topology affect location of eigenvalues of L?• Would like to separate effects of topology from agent dynamics

• Possible approach: exploit for of characteristic polynomial

-1

x

x

x

x


Performance

Look at motion between selected vehicles

Jin and MurrayCDC 04

G1 - Control G2 - Performance

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.


Robustness

What happens if a single node “locks up”

Different types of robustness (Gupta, Langbort & M)• Type I - node stops communicating (stopping failure)• Type II - node communicates constant value• Type III - node computes incorrect function (Byzantine failure)

Related ideas: delay margin for multi-hop models (Jin and M)• Improve consensus rate through multi-hop, but create sensitivity to

communcations delay

• Single node can change entirevalue of the consenus

• Desired effect for “robust”behavior: xI = /N

x1(0) = 4

x2(0) = 9

x3(0) = 6 x4(t) = 0

X5(t) = 6

x6(t) = 5

Gupta, Langbort and MurrayCDC 06



Goal: develop systematic techniques for solving subproblems• Cooperative control and graph Laplacians• Distributed receding horizon control• Verifiable protocols for consensus and control









Optimization-Based Control

Task:• Maintain equal

spacing of vehicles around circle

• Follow desired trajectory for center of mass

Parameters:• Horizon: 2 sec• Update: 0.5 sec

Local MPC + CLF• Assume neighbors follow

straight lines

Global MPC + CLF

Dunbar and MIFAC 02



Individual optimization:

Theorem. Under suitable assumptions, vehicles are stable and converge to global optimal solution.

Pf Detailed Lyapunov calculation (Dunbar thesis)

Main Idea: Assume Plan for Neighbors

stat

e

timet0 t0+d

z3(t0)

z3*(t;t0) z3

k(t)

What 2 assumes

What 3 does

Compatibility constraint:• each vehicle transmits

plan to neighbors• stay w/in bounded path

of what was transmitted


Example: Multi-Vehicle Fingertip Formation

1

3

2

4

qref

d31


Simulation Results



Goal: develop systematic techniques for solving subproblems• Distributed receding horizon control• Packet-based, distributed estimation• Verifiable protocols for consensus and control









P(k1,k2) := { initializers guard1:rule1

guard2:rule2

...

}

S(k1,k2):=P(k1,k2)+C(k1+1) sharing y,u

"soup" of guarded commands

composition = union

non-shared variables remain local to component programs

CCL: Computation and Control LanguageFormal Language for Provably Correct Control Protocols

CCL Interpreter

Formal programming lang-uage for control and comp-utation. Interfaces with libraries in other languages.

Automated Verification

CCL encoded in the Isabelle theorem prover; basic specs verified semi-automatically. Investigating various model checking tools.

Formal Results

Formal semantics in transition systems and temporal logic. RoboFlag drill formalized and basic algorithms verified.

CCL Protocol forDecentralized

Target Allocation



Example: RoboFlag Drill KlavinsCDC, 03

Things that we can prove (so far)• Semi-automated proofs (Isabelle)• Avoidance: no two robots collide• Self-stabilization: if attackers are

far enough away, defenders self-stabilize before attackers arrive

Next steps• Implement CCL on MVWT• Improved reasoning toolbox

Sample drill• N on N attack, w/

replenishment• Random initial

assignments• Switching proto-

col to avoid collisions


Observation of CCL Programs

Del Vecchio & Klavins, CDC'03

Problem: Determine state of communications protocol used by a group of robots given their physical movements.

Assumptions: Protocol and motion control are described in CCL like language.

Results: • Defns of observability, etc. for CCL programs• Construction and analysis of observer that

converges when the system is "weakly" observable

• Construction of an efficient observer for Roboflag drill in particular

• Everything specified in CCL


Summary: Cooperative Control of Multi-Vehicle Systems







Integration of computer science, communications, and control• Mixture of techniques from computer science, communications, control• Increased need for reasoning at higher levels of abstraction (strategy)

Documents

IPAM, 3 Mar 06R. M. Murray, Caltech1 Cooperative Control of Multi-Vehicle Systems Richard M. Murray Control and Dynamical Systems California Institute