Congestion control for Multipath TCP (MPTCP)Damon WischikCostin RaiciuAdam GreenhalghMark Handley

THE ROYALSOCIETY

2Packet switching ‘pools’ circuits.Multipath ‘pools’ links : it is Packet Switching 2.0.TCP controls how a link is shared.How should a pool be shared? What is TCP 2.0?

Two circuits A link Two separate links

A pool of links

3In a BCube data center, can we use multipath to get higher throughput?

Initially, there is

one flow.

4In a BCube data center, can we use multipath to get higher throughput?

Initially, there is

one flow.

A new flow starts. Its

direct route collides with the first flow.

5In a BCube data center, can we use multipath to get higher throughput?

Initially, there is

one flow.

A new flow starts. Its

direct route collides with the first flow.

But it also has longer routes

available, which don’t collide.

6How can a wireless device use two channels simultaneously, without hurting other users of the network?

How should it balance its traffic, when the channels have very different characteristics?

wifi path: high loss, small RTT

3G path: low loss, high RTT

7

What were our design goals for MPTCP congestion control?

I will propose design goals for Multipath TCP, and illustrate them in simple scenarios.

I will show experimental results for our MPTCP algorithm in these simple scenarios.

8What is the MPTCP protocol?MPTCP is a replacement for TCP which lets you use multiple paths simultaneously.

TCP

IP

user spacesocket API

MPTCP MPTCP

addr1addr2addr

The sender stripes packets across paths

The receiver puts the packets in the correct order

9Design goal 1:Multipath TCP should be fair to regular TCP at shared bottlenecks

To be fair, Multipath TCP should take as much capacity as TCP at a bottleneck link, no matter how many paths it is using.

Strawman solution:Run “½ TCP” on each path

A multipath TCP flow with two subflows

Regular TCP

10Design goal 2:MPTCP should use efficient paths

Each flow has a choice of a 1-hop and a 2-hop path. How should split its traffic?

12Mb/s

12Mb/s

12Mb/s

11Design goal 2:MPTCP should use efficient paths

If each flow split its traffic 1:1 ...

8Mb/s

8Mb/s

8Mb/s

12Mb/s

12Mb/s

12Mb/s

12Design goal 2:MPTCP should use efficient paths

If each flow split its traffic 2:1 ...

9Mb/s

9Mb/s

9Mb/s

12Mb/s

12Mb/s

12Mb/s

13Design goal 2:MPTCP should use efficient paths

If each flow split its traffic 4:1 ...

10Mb/s

10Mb/s

10Mb/s

12Mb/s

12Mb/s

12Mb/s

14Design goal 2:MPTCP should use efficient paths

If each flow split its traffic ∞:1 ...

12Mb/s

12Mb/s

12Mb/s

12Mb/s

12Mb/s

12Mb/s

15Design goal 2:MPTCP should use efficient paths

12Mb/s

12Mb/s

12Mb/s

12Mb/s

12Mb/s

12Mb/s

Theoretical solution (Kelly+Voice 2005; Han, Towsley et al. 2006) MPTCP should send all its traffic on its least-congested paths. Theorem. This will lead to the most efficient allocation possible, given a network topology and a set of available paths.

16MPTCP chooses efficient paths in a BCube data center, hence it gets high throughput.

Initially, there is

one flow.

A new flow starts. Its

direct route collides with the first flow.

But it also has longer routes

available, which don’t collide.

MPTCP shifts its traffic away from the congested link.

17MPTCP chooses efficient paths in a BCube data center, hence it gets high throughput.

We ran packet-level simulations of BCube (125 hosts, 25 switches, 100Mb/s links) and measured average throughput, for three traffic matrices.

For two of the traffic matrices, MPTCP and ½ TCP (strawman) were as good. For one of the traffic matrices, MPTCP got 19% higher throughput.

0

50

100

150

200

250

300

½ TCPMPTCP

perm. traffic matrix

sparse traffic matrix

local traffic matrix

throughput [Mb/s]

18Design goal 3:MPTCP should be fair compared to TCP

Design Goal 2 says to send all your traffic on the least congested path, in this case 3G. But this has high RTT, hence it will give low throughput.

cd

wifi path: high loss, small RTT

3G path: low loss, high RTT

Goal 3a. A Multipath TCP user should get at least as much throughput as a single-path TCP would on the best of the available paths.

Goal 3b. A Multipath TCP flow should take no more capacity on any link than a single-path TCP would.

19

wifi through-put [Mb/s]

3G through-put [Mb/s]

time [min]

User in his office, using wifi and 3G

Going downstairsIn the kitchen

MPTCP gives fair throughput.

20

wifi through-put [Mb/s]

3G through-put [Mb/s]

time [min]

3G has lower loss rate. Design Goal 2 says to shift traffic onto 3G ...

But, today, TCP over 3G was only getting 0.4Mb/s, so don’t take more than that …

But, today, TCP over wifi was getting 2.2Mb/s, so user is entitled to this much…

MPTCP gives fair throughput.

MPTCP sends0.4Mb/s over 3G, and the remaining 1.8Mb/s over wifi.

21

wifi through-put [Mb/s]

3G through-put [Mb/s]

time [min]

MPTCP gives fair throughput.

0

0.5

1

1.5

2

2.5

½ TCP

MPTCP

We measured throughput , for both ½ TCP (strawman) and MPTCP, in the office.

½ TCP is unfair to the user, and its throughput is 25% worse than MPTCP.

22

Design goals

Goal 1. Be fair to TCP at bottleneck linksGoal 2. Use efficient paths ...Goal 3. as much as we can, while being fair to TCPGoal 4. Adapt quickly when congestion changesGoal 5. Don’t oscillate

How does MPTCP achieve all this?

redundant

23

How does TCP congestion control work?Maintain a congestion window w.

• Increase w for each ACK, by 1/w

• Decrease w for each drop, by w/2

24

How does MPTCP congestion control work?

Maintain a congestion window wr, one window for each path, where r ∊ R ranges over the set of available paths.

• Increase wr for each ACK on path r, by

• Decrease wr for each drop on path r, by wr /2

25

How does MPTCP congestion control work?

Maintain a congestion window wr, one window for each path, where r ∊ R ranges over the set of available paths.

• Increase wr for each ACK on path r, by

• Decrease wr for each drop on path r, by wr /2

Design goal 3:At any potential bottleneck S that path r might be in,look at the best that a single-path TCP could get,and compare to what I’m getting.

26

How does MPTCP congestion control work?

Maintain a congestion window wr, one window for each path, where r ∊ R ranges over the set of available paths.

• Increase wr for each ACK on path r, by

• Decrease wr for each drop on path r, by wr /2

Design goal 2:We want to shift traffic away from congestion.

To achieve this, we increase windows in proportion to their size.

27MPTCP is a control plane for multipath transport.

What problem is it trying to solve?

28At a multihomed web server, MPTCP tries to share the ‘pooled access capacity’ fairly.

100Mb/s

100Mb/s

2 TCPs @ 50Mb/s

4 TCPs@ 25Mb/s

29At a multihomed web server, MPTCP tries to share the ‘pooled access capacity’ fairly.

100Mb/s

100Mb/s

2 TCPs @ 33Mb/s

1 MPTCP @ 33Mb/s

4 TCPs@ 25Mb/s

30At a multihomed web server, MPTCP tries to share the ‘pooled access capacity’ fairly.

100Mb/s

100Mb/s

2 TCPs @ 25Mb/s

2 MPTCPs @ 25Mb/s

4 TCPs@ 25Mb/s

The total capacity, 200Mb/s, is shared out evenly between all 8 flows.

31At a multihomed web server, MPTCP tries to share the ‘pooled access capacity’ fairly.

100Mb/s

100Mb/s

2 TCPs @ 22Mb/s

3 MPTCPs @ 22Mb/s

4 TCPs@ 22Mb/s

The total capacity, 200Mb/s, is shared out evenly between all 9 flows.

It’s as if they were all sharing a single 200Mb/s link. The two links can be said to form a 200Mb/s pool.

32At a multihomed web server, MPTCP tries to share the ‘pooled access capacity’ fairly.

100Mb/s

100Mb/s

2 TCPs @ 20Mb/s

4 MPTCPs @ 20Mb/s

4 TCPs@ 20Mb/s

The total capacity, 200Mb/s, is shared out evenly between all 10 flows.

It’s as if they were all sharing a single 200Mb/s link. The two links can be said to form a 200Mb/s pool.

33At a multihomed web server, MPTCP tries to share the ‘pooled access capacity’ fairly.

100Mb/s

100Mb/s

5 TCPs

First 0, then 10 MPTCPs

15 TCPs

time [min]

throughput per flow [Mb/s]

We confirmed in experiments that MPTCP nearly manages to pool the capacity of the two access links.Setup: two 100Mb/s access links, 10ms delay, first 20 flows, then 30.

34At a multihomed web server, MPTCP tries to share the ‘pooled access capacity’ fairly.

100Mb/s

100Mb/s

5 TCPs

First 0, then 10 MPTCPs

15 TCPs

MPTCP makes a collection of links behave like a single large pool of capacity —i.e. if the total capacity is C, and there are n flows, each flow gets throughput C/n.

35Open question:

Can we make a data center behave like a simple ‘capacity pool’?

What topologies and path choices would make it behave like a simple capacity pool?

What is the pooled capacity?

36

Further questionsHow much of the Internet can be pooled? What are the implications for network operators?

How should we fit multipath congestion control to CompoundTCP or CubicTCP?

Is it worth using multipath for small flows?

37

Conclusion• Multipath is Packet Switching 2.0

It lets you share capacity between links.• MPTCP is TCP 2.0

It is a control plane to harness the flexibility of multipath.It is traffic engineering, done by end-systems, and it works in milliseconds.

• We formulated design goals and test scenarios for how multipath congestion control should behave.

• We designed, implemented and evaluated MPTCP, a TCP-like congestion control algorithm that achieves these goals.

38

Related workon multipath congestion controlpTCP , CMT over SCTP, and M/TCP

that meets goal 1 (fairness at shared bottleneck)mTCP, ≈ R-MTP

and goal 2 (choosing efficient paths)Honda et al. (2009), ≈ Tsao and Sivakumar (2009)

and goal 5 (non-oscillation)Kelly and Voice (2005), Han et al. (2006)

and goal 3 (fairness)and goal 4 (rapid adjustment)(none)