1

COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.

Michael Scharf, Thomas-Rolf BannizaOctober 2011

MULTIPATH TRANSPORTCHALLENGES AND SOLUTIONS

Supported by the German-Labproject NETCOMP funded bythe German Federal Ministry ofEducation and Research (BMBF).

2

COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.

MOTIVATIONTCP-BASED MULTIPATH TRANSPORT

• Challenge: Multipath transport in the Internet

• Aggregation of capacity for terminalswith more than one interface (e. g., LTE+Wifi)

• Extension of Transmission Control Protocol (TCP)to deal with multiple subflows

� “Network MIMO” is a missing feature

• Potential solution: Multipath TCP

• New TCP-based multipath transport protocolwith several subflows

• Resource pooling of different paths

• Assumption: At least one multi-addressedendpoint (multi-homed device)

• Backward compatible to current TCP,in particular same socket API

� Ongoing IETF standardization in MPTCP WG

Terminal Server

Multipathtransport

Subflow

Subflow

Multipath transfer

Initial setup

Single path transfer

Subflow setup

Resource pooling

Multipath transfer

Terminal (multihomed) Server

3

COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.

A MULTIPATH TRANSPORT SHIM LAYER MULTIPATH TRANSPORT DESIGN SPACE

Option encoding

• MPTCP: IETF solution

• TCP extension adding multipath transport capability for unmodified applications

• New TCP options, consuming large part of scarce TCP option space

• Tight integration in TCP/IP stack

� draft-ietf-mptcp-multiaddressed

Shim layer

• MCTCP: Multi-Connection TCP

• Hybrid approach combining a minimal TCP extension and an app protocol

• Simple and extensible type-length-value (TLV) encoding whenever possible

• Shim-layer implementation possible

� draft-scharf-mptcp-mctcp

TCP

IP

Socket API(extended)TCP

IP

MCTCP

ApplicationSocket API

Kernelspace

User orkernel space

Userspace

IP IP

MPTCP

ApplicationSocket API

Kernelspace

Userspace

TCPTCP

4

COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.

A MULTIPATH TRANSPORT SHIM LAYER MPTCP MESSAGE SEQUENCE DIAGRAM

� Payload encoding outside initial subflow (used also for fallback)

Host A

addr. a1

Host B

addr. b1

Host A

addr. a2

Initial subflow� Unmodified bytestream

Data transfer over initial subflow:byte stream, identically to standard TCP

Single connectionmode

MPCAP

MPCAP

JOIN

JOIN

Data transfer over coupled subflow #1:data in TLV encoded messages

Follow-up subflow #1(“1st coupled connection”)� TLV encoding

JOIN

JOIN

Data transfer over coupledsubflow #2: data in

TLV encoded messages

Follow-up subflow #2(“2nd coupled connection”)� TLV encoding

Multi connectionmode

Initial subflow idleas a fallback

Changeover

Host B

addr. b2

Address announce msg.

5

COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.

A MULTIPATH TRANSPORT SHIM LAYER COMPARISON OF MPTCP AND MCTCP

SimpleDifficultExtensibility

FallbackFallbackDropped/striped options in SYNs

Option encoding(MPTCP)

Shim layer (MCTCP)

Signaling connection setup TCP options TCP options

Signaling during session TCP options TCP payload

Dopped/striped options outside SYNsFallback or failure

Not affected

Protocol helpers (e. g., NAPT) Simple Difficult

6

COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.

LINUX IMPLEMENTATIONCOMPLEXITY AND LESSONS LEARNT

• Minimal Linux 2.6.32 kernel patch: 800 lines of code

• Shim layer library: 5000 lines of code (baseline protocol)

• Challenges for shim approach: Memory copies, flow control, blocking calls

TCP(coupled connection)

TCP(coupled connection)

TCP(initial connection)

Application

TCP(coupled connections)

Application

Listensocket

Ker

nel

sp

ace

Use

r sp

ace MCTCP

send

Standardsocket

interface

Socketinterface

withMCTCP

extensions

Tokenhandling

Sche-duler

Addressandpath

mgmt.

Con

g. c

ontr

olco

uplin

g

Control msg. processing

Sockets

Controlmsg. Shim layer

library

MCTCPreceive

Tokenhandling

Cong.control

Addressandpath

mgmt.

Tokenhandling

TCP(initial connection)

TCP(coupled connections)

TLVparser

Cong.control

Sockets

Reassembly

MCTCPreceive

MCTCPsend

segm.

Tokenhandling

Tokenhandling

7

COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.

PERFORMANCE RESULTSEVALUATION METHODOLOGY

• Testbed setup

• Two computers with Linux Ubuntu 10.04 running patched kernel

• Two disjoint 10 Mbit/s Ethernet links, optionally also Wireless Local Area (WLAN)

• Delay and packet loss by Linux “netem” tool

• TCP stack parameters

• CUBIC congestion control, unless mentioned otherwise

• Socket buffer size set to 262144B to prevent limitations by the TCP flow control

• Other TCP stack parameters are set to the default values

• Applications

• Simple client and server programs written in C

• Also tests with a real video streaming application (VLC player) and a HTTP server

Client

withMCTCP

Server

withMCTCP

WireSharkLoad monitor

Path 1

Path 2

TCP

IP

TCP

IP

MCTCP

Application

Protocol stack

8

COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.

PERFORMANCE RESULTSEFFICIENCY OF RESOURCE POOLING

� MCTCP dynamically pools the available bandwidth of several paths

Shifting ofthe load to better path

9

COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.

PERFORMANCE RESULTSDEALING WITH LINK OUTAGE

� Compensation of path failures transparent to applications

Link failurehandling

10

COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.

PERFORMANCE RESULTSPERFORMANCE OVER CONGESTED LINKS

� Possible support of a coupled congestion control for fairness

Coupled congestion control: Fall-backto singlepath TCPfor high loss rate

Usage of one path only

Usage of both paths

11

COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.

PERFORMANCE RESULTSCONGESTED REVERSE PATH

� No significant impact by reverse path congestion even with DATA ACKs

Impact only for loss packet rates larger

than 10%

12

COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.

PERFORMANCE RESULTSBELL LABS OPEN DAY DEMO 2010

Server

Terminal

Ethernet EthernetWifi

� Multipath HTTP-based video streaming without interruptions

13

COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.

MULTIPATH CHALLENGESMULTIPATH: QUO VADIS?

• Multipath use cases

• Multi-interfaced wireless devices: Vendor support?

• Data centers: Benefit?

• What else? Benefit only for several roughly equal paths...

• Protocol design issues

• MPTCP specification is still a moving target

• Additional complexity (checksums etc.), TCP option space limitation

• Currently one major MPTCP implementation only

• Fundamental questions

• Multiple addresses ≠ multiple paths

• Simple, robust algorithms for many degrees of freedom

14

COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.

MULTIPATH CHALLENGESIMPACT ON CAPACITY SHARING?

• Question: Would multipath fundamentally affect capacity sharing?

(assuming widespread deployment)

• Well, not necessarily...

• Multiple paths are seldom

• No benefit for short flows, bulk data download does it today (e. g., P2P)

• Even without congestion control coupling still somehow similar to single-path TCP

• Capacity sharing is mostly a network task

• Scheduling policies

• Per-subscriber queues

• Deep packet inspection (DPI) in border/edge routers

• ...

15

COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.

CONCLUSION AND OUTLOOKMULTIPATH CHALLENGES AND SOLUTIONS

Conclusion

• Multi-Connection TCP (MCTCP): A multipath transport shim layer

• Simple, extensible and robust

• Alternative to a Multipath TCP solution only using option encoding

• Evaluation of MCTCP

• Limited implementation complexity

• Efficient resource pooling of several paths, including congestion control coupling

• Multipath transport: Quo vadis?

Outlook

• Performance comparison with other multipath transport protocols

• More complex setups (e. g., larger topologies, German-Lab platform)

Reference: M. Scharf, T.-R. Banniza, “MCTCP: A Multipath Transport Shim Layer”, Proc. IEEE Globecom, 2011

16

COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.