Switch-and-Navigate: Controlling Data Ferry Mobility for Delay-Bounded Messages Liang Ma, Ting He +, Ananthram Swami §, Kang-won Lee + and Kin K. Leung

Switch-and-Navigate:Controlling Data Ferry Mobility for

Delay-Bounded Messages

Switch-and-Navigate:Controlling Data Ferry Mobility for

Delay-Bounded Messages

Liang Ma*, Ting He+, Ananthram Swami§, Kang-won Lee+ and Kin K. Leung*

*Imperial College London, UK+IBM T.J. Watson Research Center, USA

§Army Research Laboratory, USA

2

Introduction

Problem Formulation

Local Control: Navigate

Agenda

2

3

4

5

Global Control: Switch

1

Comparison and Simulation Results

6 Conclusion

3

IntroductionProblem DescriptionGoalMethodContributions

Problem DescriptionProblem Description

4

d21D1

D4 D3

D2

d14

d43

d32Permanently partitioned networks

Permanently partitioned networks

IntroductionProblem DescriptionGoalMethodContributions

5

d21D1

D4 D3

D2

d14

d43

d32Goal:

Deliver delay-constrained

messages among disconnected

domains

Goal:

Deliver delay-constrained

messages among disconnected

domains

IntroductionProblem descriptionGoalMethodContributions

6

d21D1

D4 D3

D2

d14

d43

d32

Method:

Relay messages by a designated

communication node (data ferry)

Method:

Relay messages by a designated

communication node (data ferry)


7

2

General inter-domain distances

Single data ferry mobility control

Features


1

Finite message lifetime

8

Introduction

Problem Formulation


Agenda

3

4

5



6 Conclusion

22

1

9

d21D1

D4 D3

D2

d14

d43

d32

Problem Formulation

Assumptions and Partial ObservationSAN StructureControl Objective

gateway

gateway

Assumptions & Partial Observation

10

Problem Formulation


Partition each domain into cells

Gateway~Markovian mobility, transition P

Data ferry: inter-domain distance dij (in #slots), intra-domain distance 1 (slot)

Constant #messages generated at gateways each slot, with finite lifetime lmax

Partition each domain into cells

Gateway~Markovian mobility, transition P

Data ferry: inter-domain distance dij (in #slots), intra-domain distance 1 (slot)

Constant #messages generated at gateways each slot, with finite lifetime lmax

The exact gateway location is unknown at

slot t

The exact gateway location is unknown at

slot t

Control data ferry among domains with

partial observations

Control data ferry among domains with

partial observations

d21D1

D4 D3

D2

d14

d43

d32d13

11

d21D1

D4 D3

D2

d14

d43

d32Start Point d13

Global controlGlobal control

Local controlLocal control

Problem Formulation


Switch-and-Navigate Structure (POMDP)

12

=1=1Control policyControl policy

No. of messages delivered within lifetime at t

No. of messages delivered within lifetime at t

Discount factorDiscount factor

Discounted effective throughput


Problem Formulation


Control Objective

(1)(1)

13

Introduction

Problem Formulation


Agenda

2

4

5



6 Conclusion

3

1

14

Local Control Bellman EquationMyopic Local Control

The optimal policy of the navigation controller is the solution to the value iteration (T is the control duration):

The optimal policy of the navigation controller is the solution to the value iteration (T is the control duration):

00 11 22 33 T-1T-1 TTtime

time

Value iteration:Value iteration:

(2)(2)

Optimal policy

15

Local Control Bellman EquationMyopic Local Control

Distribution of gateway location (belief b) is updated every slot

Distribution of gateway location (belief b) is updated every slot

Until the gateway is finally foundUntil the gateway is finally found

Suppose the data ferry knows the transition matrix Pq in each domain

Suppose the data ferry knows the transition matrix Pq in each domain

(3)(3)

Myopic Local Policy (T=1)

16

Introduction

Problem Formulation


Agenda

2

3

5



6 Conclusion

4

1

17

D1D2

D3

D4

d14d13

d21

d43

d32obstacle

t t+1action...

switch...

one round

Global Control

Buffer States UpdateMyopic Global PolicyTwo-step Global PolicyApproximations

Gateway buffer state G

Gateway buffer state G Ferry buffer state

F

Ferry buffer state F

G11 G12 G13 … G1(L-1) G1L

G21 G22 G23 … G2(L-1) G2L

G31 G32 G33 … G3(L-1) G3L

G41 G42 G43 … G4(L-1) G4L

F11 F12 F13 … F1(L-1) F1L

F21 F22 F23 … F2(L-1) F2L

F31 F32 F33 … F3(L-1) G3L

F41 F42 F43 … F4(L-1) G4L

0

0

0

0

0

0

0

0

duration between 2 consecutive contacts is a round

duration between 2 consecutive contacts is a round

Before observationBefore observation

Global Control: Selecting the next domain to serve

18

Global Control


t t+1action...

switch...

one round

where Rj is the identity matrix except row j is 0.where Rj is the identity matrix except row j is 0.

(4)(4)

(5)(5)

After observationAfter observation

19

Global Control


Value Iteration for Global ControlValue Iteration for Global Control

where denotes the no. of delivered messages when a contact occurs, is the First Contact Time in domain j, is the total no. of rounds in the global control,

where denotes the no. of delivered messages when a contact occurs, is the First Contact Time in domain j, is the total no. of rounds in the global control,

t t+1action...

switch...

current round Future roundsFuture rounds

(8)(8)

Myopic Global Policy

(7)(7)

=1=1

20

Global Control


Future roundsFuture rounds

t t+1action...

switch...

current round predict the next roundpredict the next round

(9)(9)

Two-step Global Policy

=2=2

21

Global Control


For computational efficiency, Approximate the belief by the steady-state distribution Approximate the First Contact Time (FCT) by the average FCT

Original policies:

MY: Myopic policy

TS: Two-step policy

Steady-state-based approximations:

S-MY: Steady-state based myopic

S-TS: Steady-state based two-step policy

Further approximations:

S-TSA2: Average FCT is used in the 2nd step

S-TSA1,2: Average FCT is used in both steps

Approximations of Global Policies

22

Introduction

Problem Formulation


Agenda

2

3

4 Global Control: Switch


6 Conclusion

55

1

23

Choose some way-points and waits at each of them for a fixed no. of slots Connect the way-points to form the shortest closed path through TSP algorithms

Comparison & Simulation Results

OPWPSimulation Results

SAN vis-à-vis Predetermined Control: OPWP

Homogeneous domain settings:Homogeneous domain settings:

Heterogeneous domain settings:Heterogeneous domain settings:


OPWPSimulations

Suppose the gateways follow 2-D localized random walk model.Suppose the gateways follow 2-D localized random walk model.

Simulation settings

25



OPWPSimulations

HomogeneousHomogeneousHeterogeneousHeterogeneous

Simulation Results

26

Message loss ratio


OPWPSimulations

HomogeneousHomogeneousHeterogeneousHeterogeneous

Simulation Results

27

Introduction

Problem Formulation


Agenda

2

3

4

5



Conclusion66

1

28

Consider more practical constraints (constrained message delays, general inter-domain distances)

Consider more practical constraints (constrained message delays, general inter-domain distances)

Propose a hierarchical framework for controlling data ferry in highly partitioned networks

Propose a hierarchical framework for controlling data ferry in highly partitioned networks

The two-step policies and the approximations outperform the state of the art (optimized predetermined policy)

The two-step policies and the approximations outperform the state of the art (optimized predetermined policy)

Conclusions

29

Thank you!

Q & A

30

Choose some way-points and waits at each of them for a fixed no. of slots Connect the way-points to form the shortest closed path through TSP algorithms


OPWPSimulation Results

SAN vis-à-vis Predetermined Control: OPWP

Documents

Switch-and-Navigate: Controlling Data Ferry Mobility for Delay-Bounded Messages Liang Ma*, Ting He +, Ananthram Swami §, Kang-won Lee + and Kin K. Leung*

Switch-and-Navigate: Controlling Data Ferry Mobility for Delay-Bounded Messages Liang Ma, Ting He +, Ananthram Swami §, Kang-won Lee + and Kin K. Leung