Decentralised Coordination of Mobile Sensors

Decentralised Coordination of Mobile Sensors 

School of Electronics and Computer ScienceUniversity of Southamptonrs2@ecs.soton.ac.uk

Ruben Stranders, Alessandro Farinelli, Francesco Delle Fave, Alex Rogers, Nick Jennings

2

This presentation focuses on coordinating mobile sensors for information gathering tasks

Sensor Architecture

Decentralised Control using Max-Sum

Model

Value

Coordinate

Problem Formulation

3

This presentation focuses on coordinating mobile sensors for information gathering tasks

Sensor Architecture

Decentralised Control using Max-Sum

Model

Value

Coordinate

Problem Formulation

Mobile sensor platforms are becoming the de facto means of establishing situational awareness

“3D”Dull

DirtyDangerous

Know what is happening

Predict what will happen

and understand the impact on the mission

Currently, there is a strong trend toward making these platforms fully autonomous and cooperative

“Auto target engage by 2049…”

(My focus was on less nightmarish scenarios….)

Individual remote controlled vehicles

Teams of autonomous vehicles

The key challenge is to coordinate a team of sensors to gather information about some features of an environment

Sensors

Feature:• moving target• spatial phenomena (e.g. temperature)

We focus on three well known information gathering domains: (1) Pursuit Evasion PE

We focus on three well known information gathering domains: (2) Patrolling P

We focus on three well known information gathering domains: (3) Monitoring Spatial Fields SF

The sensors operate in a constrained environment

No centralised control

The sensors operate in a constrained environment

LimitedCommunication

The aim of the sensors is to collectively maximise the value of the observations they take

Paths leading to areas already explored- Low value

The aim of the sensors is to collectively maximise the value of the observations they take

Paths leading to unexplored areas- High value

The aim of the sensors is to collectively maximise the value of the observations they take

As a result, the target is detected faster

PE P+

The aim of the sensors is to collectively maximise the value of the observations they take

As a result, the predictive variance is minimised

SF

16

This presentation focuses on coordinating mobile sensors for information gathering tasks

Sensor Architecture

Decentralised Control using Max-Sum

Model

Value

Coordinate

Problem Formulation

17

This presentation focuses on coordinating mobile sensors for information gathering tasks

Sensor Architecture

Decentralised Control using Max-Sum

Model

Value

Coordinate

Problem Formulation

To solve this coordination problem, we had to address three challenges

1. How to model the problem?2. How to value potential samples?3. How to coordinate to gather

samples of highest value?

The three central challenges are clearly reflected in the architecture of our sensing agents

Samples sent toneighbouring agents

Samples received fromneighbouring agents

Information processing

Model of Environment

Outgoing negotiation messages

Incomingnegotiation messages

Value of potential samples Action

Selection

Move

Samples from own sensor

SensingAgent

Rawsamples

Model

Value

Coordinate

Samples sent toneighbouring agents

Samples received fromneighbouring agents

Information processing

Model of Environment

Outgoing negotiation messages

Incomingnegotiation messages

Value of potential samples Action

Selection

Move

Samples from own sensor

SensingAgent

Rawsamples

Model

Each sensor builds its own belief map containing all the information gathered about the target

Map of the probability distribution over the target’s position

The map is dynamically updated by fusing the new observation gathered

PE P+

The sensors model the spatial fields using Gaussian Processes

Weak Strong

Spatial Correlations SF

The sensors model the spatial fields using Gaussian Processes

Weak Strong

Temporal Correlations SF

Samples sent toneighbouring agents

Samples received fromneighbouring agents

Information processing

Model of Environment

Outgoing negotiation messages

Incomingnegotiation messages

Value of potential samples Action

Selection

Move

Samples from own sensor

SensingAgent

Rawsamples

Value

The value of a set of observations is equal to the probability of detecting the target

High probability

Low probability

High value: - target might be there

Low value:- Target is probably

somewhere else

PE P+

The value of a sample is based on how much it reduces uncertainty

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

8

PredictionConfidence IntervalCollected Sample

High entropyHigh value: - Strong uncertainty reduction

Low entropyLow value: - Small uncertainty reduction

SF

The sensor agents coordinate using the Max-Sum algorithm

Samples sent toneighbouring agents

Samples received fromneighbouring agents

Information processing

Model of Environment

Outgoing negotiation messages

Incomingnegotiation messages

Value of potential samples Action

Selection

Move

Samples from own sensor

SensingAgent

Rawsamples

Coordinate

To decompose the utility function we use the concept of incremental utility value

)(1Y )( 12

YY )( 213YYY

1U 2U 3U

)()()(),,( 211321 321YYYYYYf YYY

)(1

1i

jjY Y

i

The key problem is to maximise the social welfare of the team of sensors in a decentralised way

M

iYi

1

1-i

1jj)Y(maxarg

xSocial welfare:

Mobile Sensors

The key problem is to maximise the social welfare of the team of sensors in a decentralised way

),,( 3211 pppU

),( 212 ppU

),( 323 ppU

Variable encode paths

),,( 3211 pppU

),( 212 ppU

),( 323 ppU

Variable encode paths of finite length

Coordinating over all paths is infeasible: it results in a combinatorial explosion for increasing path length

Thus, we apply receding horizon control

),,( 3211 pppU

),( 212 ppU

),( 323 ppU

Clusters

Our solution: we cluster the neighborhood of each sensor

(now each variable represent a path to the Center of each cluster) Most informative is chosen!

This presentation focuses on coordinating mobile sensors for information gathering tasks

Sensor Architecture

Decentralised Control using Max-Sum

Model

Value

Coordinate

Problem Formulation

This presentation focuses on coordinating mobile sensors for information gathering tasks

Sensor Architecture

Decentralised Control using Max-Sum

Model

Value

Coordinate

Problem Formulation

35

We can now use Max-Sum to solve the social welfare maximisation problem

Complete Algorithms

DPOPOptAPOADOPT

Communication Cost

Iterative AlgorithmsBest Response (BR)

Distributed Stochastic Algorithm (DSA)

Fictitious Play (FP)

Max-SumAlgorithm

Optimality

The input for the Max-Sum algorithm is a graphical representation of the problem: a Factor Graph

Variable nodes Function nodes

1p

2p

3p

1U

2U

3U

Agent 1Agent 2

Agent 3

Max-Sum solves the social welfare maximisation problem by local computation and message passing

1p

2p

3p

1U

2U

3U

Variable nodes Function nodes

Agent 1Agent 2

Agent 3

Max-Sum solves the social welfare maximisation problem by local computation and message passing

jiadjk

iikiji prpq\)(

)()(

ijadjk

kjkjjiiij pqUprj \)(\p

)()p(max)(

From variable i to function j

From function j to variable i

In acyclic factor graphs, the messages converge to the marginal utility functions

)( iij pr A B

)( iji pq

)p(max)(B\p j

kkiiij Upr

j

)p(max)(A\p j

kkiiij Upq

j

In acyclic factor graphs, the messages converge to the marginal utility functions

)( iij pr A B

)( iji pq

In such cases, maximising the marginal utility functions is equivalent to maximising the global objective function

Max-Sum is optimal on acyclic factor graphs

To use Max-Sum, we encode the mobile sensor coordination problem as a factor graph

1p

2p

3p

1U

2U

3U

Sensor 1Sensor 2

Sensor 3

Sensor 1

Sensor 2

Sensor 3

To use Max-Sum, we encode the mobile sensor coordination problem as a factor graph

1p

2p

3p

1U

2U

3U

Sensor 1Sensor 2

Sensor 3

Sensor 1

Sensor 2

Sensor 3

Paths to the most informativepositions

To use Max-Sum, we encode the mobile sensor coordination problem as a factor graph

1p

2p

3p

1U

2U

3U

Sensor 1Sensor 2

Sensor 3

Sensor 1

Sensor 2

Sensor 3

Local Utility Functions• Measure value of observations

along paths

ijadjk

kjkjjiiij xqUxrj \)(\

)()(max)( xx

Unfortunately, the straightforward application of Max-Sum is too computationally expensive

jiadjk

iikiji xrxq\)(

)()(From variable i to function j

From function j to variable i

ijadjk

kjkjjiiij xqUxrj \)(\

)()(max)( xx

Unfortunately, the straightforward application of Max-Sum is too computationally expensive

jiadjk

iikiji xrxq\)(

)()(From variable i to function j

From function j to variable i

Bottleneck!

ijadjk

kjkjjiiij xqUxrj \)(\

)()(max)( xx

Therefore, we developed two general pruning techniques that speed up Max-Sum

Goal: Make as small as possible

ijadjk

kjkjjiiij xqUxrj \)(\

)()(max)( xx

Therefore, we developed two general pruning techniques that speed up Max-Sum

Goal: Make as small as possible

1. Try to prune the action spaces of individual sensors

2. Try to prune joint actions

ix

ij \x

The first pruning technique prunes individual actions by identifying dominated actions

The first pruning technique prunes individual actions by identifying dominated actions

1. Neighbours send bounds

↑ [2, 2]↓ [1, 1]

↑ [5, 6]↓ [0, 1]

↑ [1, 2]↓ [3, 4]

The first pruning technique prunes individual actions by identifying dominated actions

↑ [2, 2]↓ [1, 1]

↑ [5, 6]↓ [0, 1]

↑ [1, 2]↓ [3, 4]

2. Bounds are summed

↑ [8, 10]↓ [4, 7]

The first pruning technique prunes individual actions by identifying dominated actions

2. Bounds are summed

↑ [8, 10]↓ [4, 7]

↓ [4, 7]↑ [8, 10]

The first pruning technique prunes individual actions by identifying dominated actions

3. Dominated actions are pruned

[8, 10][4, 7]

X

ijadjk

kjkjjiiij xqUxrj \)(\

)()(max)( xx

We developed two general pruning techniques that speed up Max-Sum

Goal: Make as small as possible

1. Try to prune the action spaces of individual sensors

2. Try to prune joint actions

ix

ij \x✔

ijadjk

kjkjjiiij xqUxrj \)(\

)()(max)( xx

Sensor 1 Sensor 2 Sensor 3

The second pruning technique reduces the joint action space because exhaustive enumeration is too costly

Sensor 1 Sensor 2 Sensor 3

ijadjk

kjkjjiiij xqUxrj \)(\

)()(max)( xx

The second pruning technique reduces the joint action space because exhaustive enumeration is too costly

132 \)(},{

11 )()(max)(xjadjk

kjkjjxx

j xqUxr x

132 \)(},{

11 )()(max)(xjadjk

kjkjjxx

j xqUxr x

Sensor 1 Sensor 2 Sensor 3

The second pruning technique reduces the joint action space because exhaustive enumeration is too costly

),,(max)( 32},{

132

xxUr jxx

j

The second pruning technique reduces the joint action space because exhaustive enumeration is too costly

),,(max)( 32},{

132

xxUr jxx

j

),,,(),,,(max jj UU

),,,(),,,( jj UU...),,,(),,,( jj UU

The second pruning technique prunes the joint action space using Branch and Bound

Sensor 1

Sensor 2

Sensor 3

[7, 13][0, 4] [2, 6]

Sensor 1

Sensor 2

Sensor 3

The second pruning technique prunes the joint action space using Branch and Bound

[7, 13][0, 4] [2, 6]XXSensor 1

Sensor 2

Sensor 3

The second pruning technique prunes the joint action space using Branch and Bound

The second pruning technique prunes the joint action space using Branch and Bound

9 10 7 8

[7, 13][0, 4] [2, 6]XXSensor 1

Sensor 2

Sensor 3

The second pruning technique prunes the joint action space using Branch and Bound

9 10 7 8

[7, 13][0, 4] [2, 6]XX

X X XO

Sensor 1

Sensor 2

Sensor 3

The two pruning techniques combined prune 95% of the action space with 6 neighbouring sensors

2 2.5 3 3.5 4 4.5 5 5.5 60

25

50

75

100

Number of neighbouring sensors

% o

f joi

nt a

ction

s pru

ned

Our Algorithm outperforms state-of-the-art approaches by up to 52% for Pursuit Evasion PE

Our Algorithm outperforms state-of-the-art approaches by up to 44% for Patrolling P

Avg.

Roo

t Mea

n Sq

uare

d Er

ror

Our Algorithm reduces Root Mean Squared Error of predictions up to 50% compared to Greedy

Our Al-gorithm

Greedy Random Fixed0.0

0.2

0.4

0.6

0.8

1.0

SF

In conclusion, our algorithm is effective for a broad range of information gathering problems

1. Decentralised + robust

2. General

3. Effective and efficient

For future work, we intend to extend our approach to compute solutions with a guaranteed approximation ratio for any planning horizon

In conclusion, our algorithm is effective for a broad range of information gathering problems

1. Decentralised

2. General

3. Effective and efficient

QUESTIONS?