Networks: Design,Analysisand Optimization · Stochastic modeling and analysis. • Randomness is inherent in the network, for example the behavior of Internet users when they connect,

Networks: Design, Analysis andOptimization

Urtzi Ayesta

NETWORKS Grouphttp://www.bcamath.org/ayesta

BCAM-INRIA WorkshopSeptember 23, 2010

Motivation

• Internet is today the fundamental component of the

worldwide communication infrastructure

• Critical role in education, entertainment, business, social

life and tourism

• The use of both the Internet and wireless services has

experienced an explosive growth. Anticipate further

expansion in future, boosted by convergence of wireless and

internet access

• Need for the development and analysis of mathematical

models to predict and control the QoS in communication

systems

Objectives

We aim at contributing with both fundamental and applied

results:

• To develop mathematical methods and software tools

for the performance evaluation, optimization and control of

communication networks.

• Optimization and enhancement of communication

networks (protocols, architectures, applications)

Methodology

Stochastic modeling and analysis.

• Randomness is inherent in the network, for example the

behavior of Internet users when they connect, what they

download, when they disconnect is random

• Specifically: Queueing theory, stochastic scheduling

theory, optimal stochastic control, and asymptotic regimes

Game theory. The study of optimal decentralized decision

making

European context

• INRIA: CQFD (F. Dufour), TREC (F. Baccelli),

MAESTRO (Ph. Nain), RAP (Ph. Robert), PLANETE

(W. Dabbous), RESO (P. Goncalves), DYONISOS (G.

Rubino), MESCAL (B. Gaujal)

European context





• CNRS: LAAS, LIX (Polytechnique), LIAFA, LaBRI

European context






• Europe: EURANDOM (Eindhoven), CWI (Probability

and Stochastic Networks), EPFL, University of Cambridge,

UPMC (LIP6), ...

European context






• Europe: EURANDOM (Eindhoven), CWI (Probability

and Stochastic Networks), EPFL, University of Cambridge,

UPMC (LIP6), ...

• Industry: Technicolor (Paris), TNO (Delft), Microsoft

Lab (Cambridge), Telefonica (Barcelona), France Telecom

R&D (Issy-les-Moulineaux and Sophia Antipolis), T-LABS

(Berlin)

Research directions

• (Nearly) Optimal Stochastic Control

Research directions


• Asymptotic Analysis: Heavy-Traffic, Fluid limit

Research directions



• Queueing Games

Research directions



• Queueing Games

• Stochastic coupling and sample-path techniques

Research directions



• Queueing Games

• Stochastic coupling and sample-path techniques

and other techniques like

• Mean Field Limit, Large deviations and Differential

Traffic Theory

Scheduling in Wireless

• In each time slot, the base station

selects a customer to serve

• Channel conditions vary due to

fading and interference effects

• In each channel condition, the prob-

ability of completing the job in one

time slot is different

Which user to choose?

Channel quality varies over multiple time scales

Which user to choose?


=⇒ Capacity scales with number of users

=⇒ The greedy control that selects always the user with

instantaneous higher capacity performs very poorly

Multi armed bandit problem

• A sequential decision problem where at each time slot the agent

must choose one of K available options

• Depending on the chosen action, the agent receives a payoff at

the end of the time slot

• Goal: maximize the present value of the future payoffs, choosing

the right sequence of actions

Opportunistic Scheduling

Serve a user who has a “good” channel condition with

respect to its own statistics

Proportional Fair Algorithm: At time t selects the user

with highest:feasible current rate

average rate

Opportunistic Scheduling

Serve a user who has a “good” channel condition with

respect to its own statistics

Proportional Fair Algorithm: At time t selects the user

with highest:feasible current rate

average rate

=⇒ Little mathematical understanding concerning the

properties of opportunistic schedulers

Problem Formulation

• Time is slotted.

• K classes of jobs, a new class- k user arrives with

probability λk

• Nk = {1, 2, ..., Nk} set of possible states for customer of

type k

• ∀n ∈ Nk, qk,n is the probability for customer k of being at

state n and µk,n is departure probability for customer k,

if served, when it is at state n

• 0 ≤ µk,1 ≤ µk,2 ≤ · · · ≤ µk,Nk≤ 1

• Independence in the state evolution history, independence

between different customer’s current states

MDP formulation

• A = {0, 1}: Action space.

• The expected one-period reward earned by customer k at state n,

depending if it is served or not, will be given by

R1k,n = −(1− µk,n) R0

k,n = −1

• Xk(·): state process of customer k and ak(·): action process of

customer k

Objective:

lim supT→∞

1

T

T−1∑

t=0

Eπ0

[

∑

k∈K

Ra(t)k,X(t)

]

subject to∑

k∈K

ak(t) = 1, for all t ∈ T

Relaxation

• We relax the constraint: serve one customer on average.

∑

k∈K

ak(t) = 1 =⇒ lim supT→∞

1

T

T−1∑

t=0

Eπ

0

[

∑

k∈K

ak(t)

]

= 1

We obtain the next relaxed problem:

lim supT→∞

1

T

T−1∑

t=0

Eπ0

[

∑

k∈K

Ra(t)k,X(t)

]

subject to lim supT→∞

1

T

T−1∑

t=0

Eπ

0

[

∑

k∈K

ak(t)

]

= 1

Solution: Potential Improvement rule

Relaxed problem can be approached using Lagrangian methods.

∑

k∈K

lim supT→∞

1

T

T−1∑

t=0

Eπ0

[

Ra(t)k,X(t)

− νak(t)]

− ν


Relaxed problem can be approached using Lagrangian methods.

∑

k∈K

lim supT→∞

1

T

T−1∑

t=0

Eπ0

[

Ra(t)k,X(t)

− νak(t)]

− ν

Theorem: Let

νk,n =µk,n

∑

m>n

qk,m(µk,m − µk,n)for n 6= Nk, νk,Nk

= ∞

Then:

• If ν ≤ νk,n, it is optimal to serve customer k under state n ∈ Nk;

• If ν ≥ νk,n, it is optimal not to serve customer k under state

n ∈ Nk


• We construct a feasible policy for the original problem,

using the optimal solution of the relaxed problem:

• Potential Improvement rule: Give service at time t to

job k∗ (t) such that:

k∗(t) := argmaxk∈K

µk,n∑

m>n

qk,m(µk,m − µk,n)

• Not necessarily optimal for the original problem

Scheduling disciplines

• µ-index: νµk,n := µk,n for n ∈ Nk

• Score Based index (T.Bonald, 2004):

νSBk,n :=∑n

m=1 qk,m, for n ∈ Nk

• Relatively Best index (Qualcomm 3G standard, 2000):

νRBk,n :=

µk,n

Nk∑

m=1

qk,mµk,m

, for n ∈ Nk.

• Potential Improvement index:

νPIk,n =µk,n

∑

m>n

qk,m(µk,m − µk,n)for n 6= Nk, νPIk,Nk

= ∞

Simulations

0.5 0.60 0.70 0.80 0.90 10

20

40

60

80

ρ

Mea

n nu

mbe

r of

jobs

in th

e sy

stem

CµRBSBPI

How good is the heuristic?

Questions:

• Stability

• Optimal in some sense?

How good is the heuristic?

Questions:

• Stability

• Optimal in some sense?

Fluid limit: Y r(t) := Xr(rt)r

=⇒ we characterize the Maximum stability condition and

the set of policies that achieve maximum stability

Fluid Limit

Theorem: For a given policy f inducing a partially increasing vector

field with uniform limits drift, we have:

limr→∞

P( sup0≤s≤t

|Y f,r(s)− yf (s)| ≥ ε) = 0, for all ε > 0,

with yf (t) a piece-wise linear function

0 20 40 60 80 100 1200

0.5

1

1.5

2

2.5

t

Y 1 r

( t

) +

Y 2 r

( t

)

PIcµRBSB, PB

Stability

Theorem: A policy f inducing a partially increasing drift

with uniform limits is stable if Tfl < ∞ for all l, where

Tfl = inf{t ≥ 0, s.t. yl(t) = 0}

Stability

Theorem: A policy f inducing a partially increasing drift

with uniform limits is stable if Tfl < ∞ for all l, where

Tfl = inf{t ≥ 0, s.t. yl(t) = 0}

Theorem: The maximum stability condition is

K∑

k=1

λk

µk,Nk

< 1.

Any policy that prefers a customer in its best state over any

other customer is maximum stable

=⇒ PI has maximum stability region

Asymptotic Fluid Optimality

min

∫ D

0

K∑

k=1

ckxk(t),

xk(t) = xk(0) + λkt−

Nk∑

n=1

µk,n

∫ t

0uk,n(v)dv,

xk(t) ≥ 0, k = 1, . . . ,K

such that for all v ≥ 0,

K∑

k=1

Nk∑

n=1

uk,n(v) ≤ 1, uk,n(v) ≥ 0, for all k, n


Lemma. Assume µ1,N1≥ µ2,N2

≥ . . . ≥ µK,NK. The fluid

control that solves the fluid control problem is as follows: Let

l = argmin{k : xk(t) > 0}. Then

u∗k,Nk(t) =

λk

µk,Nk

, for k < l, u∗l,Nl(t) = 1−

l−1∑

i=1

λi

µi,Ni


Lemma. Assume µ1,N1≥ µ2,N2

≥ . . . ≥ µK,NK. The fluid

control that solves the fluid control problem is as follows: Let

l = argmin{k : xk(t) > 0}. Then

u∗k,Nk(t) =

λk

µk,Nk

, for k < l, u∗l,Nl(t) = 1−

l−1∑

i=1

λi

µi,Ni

Lemma. For any policy f and D > 0 we have

lim infr→∞

E

(

∫ D

0

K∑

k=1

Yf,rk (t)dt

)

≥

∫ D

0

K∑

k=1

x∗k(t)dt

Theorem. The PI policy is asymptotically fluid optimal

Conclusions

• PI: A heuristic simple to implement

• Characterize the stability region, the set of stable policies, and set

of asymptotic fluid optimal policies

• PI: Maximum stability region and is asymptotically fluid optimal

Conclusions

• PI: A heuristic simple to implement

• Characterize the stability region, the set of stable policies, and set

of asymptotic fluid optimal policies

• PI: Maximum stability region and is asymptotically fluid optimal

U. Ayesta, M. Erausquin, P. Jacko, A Modeling Framework for Optimizing

the Flow-Level Scheduling with Time-Varying Channels. to appear

Performance Evaluation 2010

U. Ayesta, M. Erausquin, M. Jonckheere, I.M. Verloop, Opportunistic

scheduling in wireless systems: stability and asymptotic optimality, submitted

for publication

U. Ayesta, P. Jacko, Method for selecting a transmission channel within a

time division multiple access (TDMA) communication system, BCAM,

Application Number 10380060.3, April 2010

Perspectives

• Combination of Deterministic and Stochastic techniques

Perspectives


• Performance Evaluation+Game Theory = Queueing

Games

Perspectives



Games

• Bio-Inspired Networks, Swarm Intelligence

Perspectives



Games

• Bio-Inspired Networks, Swarm Intelligence

• New applications: Clean-Slate Internet, Energy

minimization, electrical networks, power grid

Networks: Design, Analysis andOptimization

Urtzi Ayesta

NETWORKS Grouphttp://www.bcamath.org/ayesta

BCAM-INRIA WorkshopSeptember 23, 2010

NET group at BCAM

• U. Ayesta (Ikerbasque)

• J. Anselmi (postdoc)

• P. Jacko (postdoc)

• I.M. Verloop (postdoc)

• M. Erauskin (PhD, co-supervised with E. Ferreira)

• A. Izagirre (internship)

Solution to Relaxed Formulation

Relaxed problem can be approached using Lagrangian

methods.

lim supT→∞

1

T

T−1∑

t=0

(

Eπ0

[

Ra(t)X(t)

]

− ν Eπ

0

∑

k∈K

ak(t)

)

− ν

We decompose this problem in K subproblems:

lim supT→∞

1

T

T−1∑

t=0

(

Eπ0

[

Rak(t)k,X(t)

− νak(t)])

− ν

We solve K subproblems, and we obtain the joint optimal

policy for the relaxed problem combining them.

Wireless Channel


Class of policies

Best Rate (BR) The BR policies are such that whenever

there are users present that are currently in their best channel

condition, i.e., in state Nk, only such users are served.

Best Rate Priority (BRB) A BRP policy is a BR policy

where ties are broken using the cµ-rule, that is, based on

ckµk,Nk.

Capacity scales with number of users

Simulations

0 1000 2000 3000 40000

50

100

150

200

250

300

350

Time

Num

ber

of jo

bs in

the

syst

ems

CµRBSBPI

Fluid Limit

Theorem: For a given policy f inducing a partially increasing vector

field with uniform limits drift, we have:

limr→∞

P( sup0≤s≤t

|Y f,r(s)− yf (s)| ≥ ε) = 0, for all ε > 0,

with yf (t) a piece-wise linear function

y1

y2

(0, 0)

(y1(0), y2(0))

Performance in overload

0 2000 4000 6000 8000 10000 120000

10

20

30

40

t

Y1r (t

)+Y

1r (t)

PI

Cµ

RB

SB

Documents

Networks: Design,Analysisand Optimization · Stochastic modeling and analysis. • Randomness is inherent in the network, for example the behavior of Internet users when they connect,