1

Mean field models of interacting objects:

fluid equations,independence assumptions and pitfalls

Jean-Yves Le Boudec

EPFL

October 2009

2

AbstractWe consider a generic model of N interacting objects, where eachobject has a state and interaction between objects is Markovian,i.e. the evolution of the system depends only on the collection ofstates at any point in time. This is quite a general modelingframework, which was successfully applied to model many forms ofcommunication protocols. When the number of objects N is large, oneoften uses simplifying assumptions called "mean fieldapproximation", "fluid approximation", "fixed point method" or"decoupling assumption". In this tutorial we explain the meaning ofthese four concepts and show that the first two, namely mean fieldapproximation and fluid approximation, are generally valid. However,we also show that the last two, namely fixed point method anddecoupling assumption, require more care, as they may not be valid,even in simple cases. We give sufficient conditions under which theyare valid. We illustrate the concepts with the analysis of the802.11 WiFi protocol.

This slide show is available on my home page under “talks” or directly at http://ica1www.epfl.ch/PS_files/lebSlides.htm

3

Contents

A Simple Model of Interacting Objects

Fluid Approximation

Mean Field Approximation, Fast Simulation

Stationary Regime and the Decoupling Assumption

The Fixed Point Method

Useful Extensions

4

A Simple Model of Interacting ObjectsTime is discrete

N objects

Object n has state Xn(t) 2 {1,…,I}

(X1(t), …, XN(t)) is Markov

Objects can be observed only through their state

N is large, I is small

Called “Mean Field Interaction Models” in the Performance Evaluation community (But mean field has other meanings in physics, see later)

Example 1: N wireless nodes, state = retransmission stage k

Example 2: N wireless nodes, state = k,c (c= node class)

Example 3: N wireless nodes, state = k,c,x (x= node location)

5

Example: 2-step malware propagation

Mobile nodes are either`S’ Susceptible`D’ Dormant`A’ Active

3 states

N nodes

Nodes meet pairwise (bluetooth)

Possible interactions:

1. RecoveryD -> S

2. Mutual upgrade D + D -> A + A

3. Infection by activeD + A -> A + A

4. RecoveryA -> S

5. Recruitment by DormantS + D -> D + D

6. Direct infectionS -> A

6

Details of the ExampleTo specify the previous model entirely, ( = to be able to simulate it), we need to specify the transition matrixIn a compact form, we define probas of each type of transition.

1. RecoveryD -> S

2. Mutual upgrade D + D -> A + A

3. Infection by activeD + A -> A + A

4. RecoveryA -> S

5. Recruitment by Dormant

S + D -> D + D6. Direct infection

S -> A

Simulation algorithm:At every time step

Pick one case with prob as given in table

sum of probs is less than 1, possible to do nothing at one step(case 1) Pick one node uniformly at random among all nodes that are in state ‘D’(case 2) Pick one pair of nodes uniformly at random among all pairs nodes that are in state ‘D’(case 3) Pick one node among ‘A’ nodes, and one among ‘D’ nodes, each uniformly at randometcS, D, A are the numbers of nodes in state `S’, `D’, `A’

A(t)Proportion of nodes In state i=2

7

Simulation Runs, N=1000 nodesNode 1

Node 2

Node 3

D(t)Proportion of nodes In state i=1

State = DState = AState = S

8

Sample Runs with N = 1000

Simplified Analysis 1 :Decoupling Assumption (Transient Regime)

9

1. RecoveryD -> S

2. Mutual upgrade D + D -> A + A

3. Infection by activeD + A -> A + A

4. RecoveryA -> S

5. Recruitment by Dormant

S + D -> D + D6. Direct infection

S -> A

Simplified Analysis 2Decoupling Assumption (Stationary

Regime)

Solve for (D,A,S)

Has a unique solution

10

1. RecoveryD -> S

2. Mutual upgrade D + D -> A + A

3. Infection by activeD + A -> A + A

4. RecoveryA -> S

5. Recruitment by Dormant

S + D -> D + D6. Direct infection

S -> A

11

Issues

When is decoupling assumption valid ?

How to formulate the ODE ?

Is stationary regime of ODE an approximation of stationary regime of original system ?

12

Contents

A Simple Model of Interacting Objects

Fluid Approximation

Mean Field Approximation, Fast Simulation

Stationary Regime and the Decoupling Assumption

The Fixed Point Method

Useful Extensions

13

Scaling Assumptions

We want to simplify the model for large N, we need scaling assumptions

Let WN(t) be the (random) number of objects that do a transition at time slot t when there are N objects

Informally, the main scaling assumptions are:The expectation of WN(t) tends to a constant as N growsThe second moment of WN(t) remains bounded as N grows

i.e., for large N, the probability that this object makes a transition is O(1/N)

This is equivalent to a time scale assumption: the time slot duration is O(1/N)

14

Formal Statements

Definition: Occupancy MeasureMN

i(t) = fraction of objects in state i at time tExample: MN(t) = (D(t), A(t), S(t))

Definition: drift = expected change to MN(t) in one time slot

The scaling assumptions are:

15

Writing the Drift Without Error

Drift = sum over all transitions of

proba of transition£Delta to the system state MN(t)

Can be automated

http://icawww1.epfl.ch/IS/tsed

16

Example

17

Fluid Approximation Theorem

Under the scaling assumption:

stochastic system MN(t) can be approximated by fluid limit (t), solution of the ODE:

Rescaled drift of MN(t)

18

Example

19

Fluid limitN = +1

Stochastic system

N = 1000

20

Fluid Approximation Theorem

Definition: Re-Scaled Occupancy measure

[Benaïm, L] :

21

Computing the Mean Field Limit

Compute the drift of MN and its limit over intensity

22

Contents

A Simple Model of Interacting Objects

Fluid Approximation

Mean Field Approximation, Fast Simulation

Stationary Regime and the Decoupling Assumption

The Fixed Point Method

Useful Extensions

23

Propagation of Chaos

Convergence to an ODE implies “propagation of chaos” [Sznitman, 1991]

This says that, for large N, any k objects are ≈ independentJustifies Decoupling Assumption (transient)

mean field limit

24

Mean Field Independence

At any time tk nodes are asymptotically independent

Thus for large t :Prob (node n is dormant) ≈ 0.3Prob (node n is active) ≈ 0.6 Prob (node n is susceptible) ≈ 0.1

25

Fast Simulation Result

A stronger result than propagation of chaos – does not require exchangeability

[Tembine, L et al], 2009

Assume we know the state of object n at time 0; we can approximate its evolution by

Replacing all other objects collectively by the ODE

The state of object n is a jump process, with transition matrix driven by the ODE

PNi,j (m) is the transition probability for one object, given that the state of the system

is m

Note: Knowing the transition matrix PN (m) is not enough to be able to simulate (or

analyze) the system with N objectsBecause there may be simultaneous transitions of several objects (on the example, up to 2)

However, the fast simulation says that, in the large N limit, we can consider one (or k) objects as if they were independent of the other N-k

(XN1(t/N), MN(t/N)) can be approximated by the process (X1(t), m(t)) where m(t)

follows the ODE and X1(t) is a jump process with time-dependent transition matrix A(m(t)) where

Example

26

The state of one object is a jump process with transition matrix:

where m = (D, A, S) depends on time (is solution of the ODE)

27

AN

Example

28

pdf of node 1

pdf of node 2

pdf of node 3

occupancy measure(t)

ODEs

Let pNj(t|i) be the probability that a node that starts in state i is

in state j at time t:

The fast simulation result says that

With the ODEs:

Computing the Transition Probability

PNi,j (m) is the transition probability for one object, given that

the state if m

30

31

The Mean Field Approximation

Common in Physics

Consists in pretending that XNm(t), XN

n(t) are independent in the time evolution equation

It is asymptotically true for large N, at fixed time t, for our model of interacting objects

Also called “decoupling assumption” (in computer science)

32

Contents

A Simple Model of Interacting Objects

Fluid Approximation

Mean Field Approximation, Fast Simulation

Stationary Regime and the Decoupling Assumption

The Fixed Point Method

Useful Extensions

33

Stationary Regime

Original system (stochastic):(XN(t)) and (MN(t)) are Markov, finite state space, discrete timeAssume either one is irreducible, thus has a unique stationary proba N

For large N, how does N relate to the stationary regime of the ODE ?

Law of MN(t) N

(t) ???

t -> +1

N -> +1

Assume (H) the ODE has a unique stable point m* to which all trajectories converge

Theorem Under (H)

i.e.

(1) m* is the limit of N for large N [N = stat. prob. of (X1N(t),

…, XNN(t) ]

and

(2) decoupling assumption holds in stationary regime

34

Under (H), Decoupling Assumption Holds In Stationary Regime

35

Example

In stationary regime:

Prob (node n is dormant) ≈ 0.3Prob (node n is active) ≈ 0.6 Prob (node n is susceptible) ≈ 0.1

Nodes m and n are independent

We are in the good case: the diagram commutes

Law of MN(t) N

(t) m*

t -> +1

N -> +1

t -> +1

N -> +1

36

Counter-Example

The ODE does not converge to a unique attractor (limit cycle)

Assumption H does not hold; does the decoupling assumption still hold ?

Same as beforeExcept for one

parameter value

h = 0.1 instead of 0.3

37

Decoupling Assumption Does Not Hold HereIn Stationary Regime

In stationary regime, m(t) = (D(t), A(t), S(t)) follows the limit cycle

Assume you are in stationary regime (simulation has run for a long time) and you observe that one node, say n=1, is in state ‘A’

It is more likely that m(t) is in region R

Therefore, it is more likely that some other node, say n=2, is also in state ‘A’

This is synchronization

R

38

Numerical Example

Mean of Limit of N = pdf of one node in stationary regime

Stationary point of ODE

pdf of node 2 in stationary regime, given node 1 is D

pdf of node 2 in stationary regime, given node 1 is S

pdf of node 2 in stationary regime, given node 1 is A

Simplified Analysis 2Decoupling Assumption (Stationary

Regime)

Solve for (D,A,S)Has a unique solution

10

1. RecoveryD -> S

2. Mutual upgrade D + D -> A + A

3. Infection by activeD + A -> A + A

4. RecoveryA -> S

5. Recruitment by Dormant

S + D -> D + D6. Direct infection

S -> A

40

Where is the Catch ?

Fluid approximation and fast simulation result say that nodes m and n are asymptotically independent

But we saw that nodes may not be asymptotically independent

… is there a contradiction ?

The Diagram Does Not Commute

For large t and N:

where T is the period of the limit cycle

42

Generic Result for Stationary Regime

Original system (stochastic):(XN(t)) is Markov, finite, discrete timeAssume it is irreducible, thus has a unique stationary proba N

Let N be the corresponding stationary distribution for MN(t), i.e.

P(MN(t)=(x1,…,xI)) = N(x1,…,xI) for xi of the form k/n, k integer

Theorem

Birkhoff Center: closure of set of points s.t. m2 (m)Omega limit: (m) = set of limit points of orbit starting at m

43

Here: Birkhoff center = limit cycle fixed point

The theorem says that the stochastic system for large N is close to the Birkhoff center,

i.e. the stationary regime of ODE is a good approximation of the stationary regime of stochastic system

ExampleAt fixed t, the large N limit is deterministic (Dirac)

In stationary regime it is not

Stationary regime is periodicSampling algorithm for the stationary regime

pick t uniformly at random in a periodSet m = (t)

QuizMN(t) is a Markov chain on E={(a, b, c) ¸ 0, a + b + c =1, a, b, c multiples of 1/N}

E (for N = 200)

A. MN(t) is periodic, this is why there is a limit cycle for large N.

B. For large N, the stationary proba of MN tends to be concentrated on the blue cycle.

C. For large N, the stationary proba of MN tends to a Dirac.

D. MN(t) is not ergodic, this is why there is a limit cycle for large N.

Take Home Message

For large N the decoupling assumption holds at any fixed time t

It holds in stationary regime under assumption (H)ODE has a unique global stable point to which all trajectories converge

Otherwise the decoupling assumption may not hold in stationary regime

It has nothing to do with the properties at finite NIn our example, for h=0.3 the decoupling assumption holds in stationary regimeFor h=0.1 it does not

Study the ODE !

46

47

Contents

A Simple Model of Interacting Objects

Fluid Approximation

Mean Field Approximation, Fast Simulation

Stationary Regime and the Decoupling Assumption

The Fixed Point Method

Useful Extensions

48

The Fixed Point Method

Commonly used to model protocol performance, finite state machines etc

When valid, works as followsNodes 1…N each have a state in {1,2,…,I}Assume N is large and therefore nodes are independent (decoupling assumption)Let i be the proba that any given node n is in state I. Write the equilibrium equations using the independenceCan often be cast as a fixed point equation for , solved numerically by iteration

The Fixed Point Method Consists in Finding the Stationary Points of the ODE

The transition matrix for one object depends on the occupancy measure, assumed equal to

This is the same as

or

For large N it is the same as the drift

Thus is a fixed point of the ODE

This is justified if assumption (H) holds, otherwise not

50

Existence and Unicity of a Fixed Point are not Sufficient for Validity of Fixed Point Method

Essential assumption is

(H) () converges to a unique m*

It is not sufficient to find that there is a unique stationary point, i.e. a unique solution to F(m*)=0

Counter Example on figure(XN(t)) is irreducible and thus has a unique stationary probability

There is a unique stationary point ( = fixed point ) (red cross)

F(m*)=0 has a unique solutionbut it is not a stable equilibrium

The fixed point method would say hereProb (node n is dormant) ≈ 0.1Nodes are independent

… but in realityWe have seen that nodes are not independent, but are correlated and synchronized

51

Mean of limit of N = pdf of one node in stationary regime

Stationary point of ODE

52

Correct Use of Fixed Point Method

Verify scaling assumption

Write ODE

Study stationary regime of ODE, not just fixed point

Verify assumption (H), i.e. there is a unique attractor to which all solutions converge

53

Example: Bianchi’s Formula

Example: 802.11 single cell

mi = proba one node is in backoff stage I= attempt rate = collision proba

Solve for Fixed Point:

54

Bianchi’s Formula requires analysis of ODEThe fixed point solution satisfies “Bianchi’s Formula” [Bianchi]

Another interpretation of Bianchi’s formula [Kumar, Altman, Moriandi, Goyal]

= nb transmission attempts per packet/ nb time slots per packet

assumes collision proba remains constant from one attempt to next

(H) true in single cell system [Bordenave,McDonald,Proutière] for q0< ln 2 and K= 1 [Sharma, Ganesh, Key] and for K=1

Method to porve (H) uses majorization by linear systems of ODE

Unclear what happens for heterogeneous systems

55

Contents

A Simple Model of Interacting Objects

Fluid Approximation

Mean Field Approximation, Fast Simulation

Stationary Regime and the Decoupling Assumption

The Fixed Point Method

Useful Extensions

56

Useful Extensions

Systems with a global resource that is updated at every transition [Bordenave,McDonald,Proutière], [Benaïm, L]

Multiclass systems [Banerjee et al ] [Chaintreau, L., Ristanovic]

Spatial model : XNn(t) = ( i, c) where c

= location on grid

Continuous state space (instead of discrete) [Chaintreau, L., Ristanovic]

XNn(t) = ( a, c) where a 2 [0, 1)

57

Conclusion

Convergence to Mean Field:

We have found a simple framework, easy to verify, as general as can beMultiple synchronized transitions are possible in the model

Under very large assumptions, first order properties can be predicted by the limit ODE

But for stationary regime one needs to study the ODE for itself

original system being ergodic does not imply ODE converges to a fixed point

Decoupling assumption holds (in stationary regime) if ODE has a unique attractor to which all trajectories converge

Otherwise, it may not hold

58

[Benaïm, L] “A Class Of Mean Field Interaction Models for Computer and Communication Systems”, Performance Evaluation, April 2008

[Tembine, L et al] Hamidou Tembine , Jean Yves Le Boudec, Rachid ElAzouzi, Eitan Altman, "Mean Field Asymptotic of Markov Decision Evolutionary Games and Teams", May 2009 " in proceedings of GameNets

[L,Mundinger,McDonald]

[Benaïm,Weibull]

[Sharma, Ganesh, Key]

[Bordenave,McDonald,Proutière]

References

59

[Chaintreau, L., Ristanovic] A. Chaintreau, J.-Y. Le Boudec and N. Ristanovic, “The age of Gossip: Spatial Mean Field Regime”, ACM Sigmetrics 2009

[Banerjee et al ] N Banerjee, MD Corner, D Towsley, BN Levine, “Relays, base stations, and meshes: enhancing mobile networks with infrastructure”, Mobicom 2008

Sznitman]

[Bianchi]

[Kumar, Altman, Moriandi, Goyal]

Solution to Quiz

The only correct answer is B.

MN is aperiodic (there are cycles of length 3 and 4)