Optimal Capacity Sharing of Networks with Multiple Overlays

Zheng Ma, Jiang Chen, Yang Richard Yang and Arvind Krishnamurthy

Yale UniversityUniversity of Washington

{zhengma,criver,yry}@cs.yale.edu {arvind}@cs.washington.edu

Presented by Zheng MaJun 19, 2006

2006-6-19 2

Introduction Overlay networks are becoming widely deployed:

P2P applications: e.g., BitTorrent, PPlive VoIP applications: e.g., Skype Testbeds: e.g., Planetlab, Emulab

http://www.cachelogic.com

2006-6-19 3

Example of Overlays

The overlay O1 is trying to find the max flow from node 1 to node 5. There is a TCP flow from node 2 to node 5, which could be viewed as an overlay with only 1 link.

How to model their behavior when they share the network resource?

Topology of Overlay O1

2006-6-19 4

State of Art: Resource Allocation of Multiple Overlays

No congestion control Network collapse Using UDP to probe available bandwidth is possible but

the packets may be dropped by the network if you don’t react to the network feedback correctly.

ISP will limit the rate.

Use TCP at each overlay link e.g. Skype and BitTorrent use TCP on each overlay link

with the hope that it will share network resource fairly and efficiently.

If the flow rate on each link is controlled by TCP without coordinating with other links of the same overlay application, we refer to such a scheme as flow-level rate control.

Is this enough?NO!

2006-6-19 5

Talk Outline

Introduction Problem statement Design of distributed algorithm for capacity

sharing of multiple overlays Case study: overlay maximum flow problem Evaluation: simulation results Related works and conclusion

2006-6-19 6

Problem Statement

Network model: Physical : G = (V,L,C), node set V, link set L, with capacity

C={ Cl }.

Overlay: Gi = (Hi ,Ei) : node set Hi overlay link set Ei

Each overlay link has rate xe -- control variables. Mapping between overlay link and a physical path: Ale=1 if e goes link l in physical network, otherwise 0.

So the capacity constraint at physical network is Each overlay may have application constraints, e.g., flow

conservation constraint Fhe=1 if e=(h,v), Fhe=-1 if e=(u,h), otherwise Fhe=0

Utility function: Each overlay has a utility function Ui which is strictly concave.

The input to Ui is an aggregation function fi applied to

fi is differentiable, application-specified. For overlay maximal flow problem:

The overlay i is trying to maximize:

CAx

0Fx

iei Eexx },{

))(( iii xfU

)(

)(DINeeii xxf

2006-6-19 7

System Problem

Capacity sharing of multiple overlays If the system design objective is to maximize the sum of the

utilities of all overlays, we can write down the system optimization problem as:

When all overlays are single end-to-end flows, the above formulation is reduced to that of Frank Kelly’s framework.

Reminder: we call a rate control mechanism in overlay network flow-level rate control if each control variable xe is controlled by TCP or other transport protocol without coordinating within the overlay.

A rate control mechanism is overlay flows control if the overlay will coordinate the control of all xe.

.,0

))((1

e

xfUn

iiii

e xover

0Fx

C Axtosubject

maximize :P

2006-6-19 8

Example 1: Unfair Sharing with TCP Using Only Flow-level Rate Control

)log())((

)log())((

)(

)(

),,,(

225

145

125111

225

145

12511

2252

145

125

134

132

1131

xxfU

xxxfU

xxf

xxxf

xxx

xxxxxx

tcptcptcp

tcptcp

tcp

The system optimal is x1=(1,0,1,0,1), x2=1,total utility

0

With only flow-level rate control:

x1=(1,1/3,2/3,1/3,2/3), x2=1/3, total utility -0.48

Topology of Overlay O1

1

1 1

1/31/3

2/32/3

1

2006-6-19 9

Example 2: Sub-optimal Capacity Sharing Among Multiple Overlays

245222

145

125111

24522

145

12511

245

224

214

232

2312

145

125

134

132

1131

))((

))((

)(

)(

),,,,(

),,,,(

xxfU

xxxfU

xxf

xxxf

xxxxxx

xxxxxx

The system optimal is x1=(1,1,0,1,0), x2=(0,1,0,1,1), total

utility 2

With only flow-level rate control:

x1=(1/3,0,1/3,0,1/3), x2=(1/3,1/3,1/3,1/3,2/3), total

utility 1

1

1

1

1

1

1

1/3

1/31/3

1/3

1/3

1/3

1/3

2/3

Overlay O1

Overlay O2

2006-6-19 10

Our Contributions

The traditional flow-level rate control is not enough for resource allocation among multiple overlays. It may reach sub-optimal equilibrium.

We propose overlay flows control to coordinate the rate flow to solve the problem by controlling flows in an overlay network coordinatively.

Key Idea: to solve the overlay utility maximization system problem in a distributed way. We don’t require the knowledge of the underlay networks (i.e. A and C in the physical network). Instead we use a “try and back off” approach.

.,0

))((1

e

xfUn

iiii

e xover

0Fx

C Axtosubject

maximize :P

2006-6-19 11

Algorithmic Design in P is not strictly concave. We use Proximal Minimization method to make the

objective function strictly concave.

.,0)()(21))(( 1211exbxkxfUniiieeniEeniiiiie x over 0Fx C Ax tosubject maximize :P1

))((iiixfU

B={be} is the introduced auxiliary variables. In P1, it is fixed. Iterative process: Solve P1 and obtain optimal solution X, set B=X, and solve P1

again.

.,0

)()(2

1))((

1

2

11

e

xbxk

xfUn

iiiee

n

i Ee

n

iiii

i

e x over

0Fx

C Ax tosubject

maximize :P1

))(( iii xfU

2006-6-19 12

A Price Based Approach

P1 can be solved by a price based approach. Lagrangian form:

Llll

Eeeee

n

iii

n

iii

cpqxx

xFcxApxpxL

i

))()((

)()()(),,(

1

1

Ll

llee pAqPath Price

Link Price

Maximizer

Application price

Hh

hehe F

Node Price

2006-6-19 13

Case Study: Overlay Maximum Flow

Rate adaptation and price calculation Link Price Update, we can use queuing delay as an

approximation

Node Price Update

Overlay Flows Rate Adaptation

Convergence We used Lyapunov stability theory to prove the

algorithm is globally asymptotically stable.

),( ),(

)()()()1(hue vhe

eehh txtxtth

))()(())((1

)))((())(()1( ' ttqbtxkx

ftxfUtxktxe eeee

e

iiieee i

)(

)()()1(lEe

lell ctxtptpl

2006-6-19 14

Evaluation: Convergence

Simulation setup: BRITE topology generator. All experiments showed a similar

result. Use the algorithm for overlay maximum flow. Results for example 1 and example 2.

Convergence results

Overlay 1

TCP flow

Overlay 1

Overlay 2

2006-6-19 15

Evaluation: Dynamics

Simulation setup: In example 1, add more TCP flows between node 2 and node

5 at different time. The algorithm can react to the change and converge to the

fair share quickly. One could consider our algorithm as a generalization of

protocol compliance requirements: e.g. TCP friendliness.

TCP flow

2006-6-19 16

Related Work

Coexistence of multiple overlays (focusing on cost or delay) Selfish routing effects (Qiu et al. SIGCOMM’03). Interaction of multiple overlay routing (Jiang et al.

Performance’05). Can overlays inadvertently step on each other?

(Keralapura et al. ICNP’05). Overlay networks

Overlay networks with linear capacity constraints. (Zhu et al. IWQoS’05)

Transport protocol design Fast TCP: motivation, architecture, algorithms,

performance. (Wei et al. TON’07)

2006-6-19 17

Conclusion and Future Work

Our contributions: Define the problem of optimal capacity sharing of multiple

overlays. Show that flow-level rate control cannot achieve system-wide

optimality. Develop a framework to use overlay flows rate control to solve

the problem in distributed way and show its convergence and effectiveness.

The protocol can be implemented by measuring end-to-end queuing delay at overlay level. This is a try-band-back-off approach similar to TCP Vegas and FAST TCP.

Future work: Convergence of the algorithm in other setups. Utility function design for overlay networks, implementing

different types of fairness among overlays. Consider other popular overlay applications like network coded

overlay multicast.

2006-6-19 18

The End

Thanks! Questions?

More information: Google “zheng ma”

2006-6-19 19

Backup Slides

Non-triviality of overlay maximum flow algorithm

1

3

2 5

4

6

7

Overlay maximum flow problem is non-trivial even for a single overlay. i.e. we can’t use traditional max flow algorithm by measuring available bandwidth on overlay level.

In above topology, each link is overlay link, all underlay physical links has unit capacity. Suppose (2,4), (4,5) and (4,6) share a physical link. The max flow algorithm will try to push 1 unit traffic at each overlay link. (2,4) (4,5) and (4,6) will get 1/3 each, no more bandwidth available, no augmenting path. The max flow rate is 2/3. However, by sending 1 unit traffic on (1,3)(3,4)(4,6)(6,7), we get max flow 1.