Multipath TCP - DiVA portal938436/FULLTEXT01.pdf · Combining multiple radio access technologies, ... 1 Introduction 1 ... Each gray dot is a test result. The

Linköpings universitetSE–581 83 Linköping

+46 13 28 10 00 , www.liu.se

Linköping University | Department of Computer ScienceMaster thesis, 30 ECTS | Datateknik

2016 | LIU-IDA/LITH-EX-A--16/015--SE

Multipath TCP– Performance in a LTE Environment

Axel Pyk

Supervisor : Vengatanathan KrishnamoorthiExaminer : Niklas Carlsson

http://www.liu.se

Upphovsrätt

Detta dokument hålls tillgängligt på Internet – eller dess framtida ersättare – under 25 årfrån publiceringsdatum under förutsättning att inga extraordinära omständigheter uppstår.Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstakakopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och förundervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva dettatillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. Föratt garantera äktheten, säkerheten och tillgängligheten finns lösningar av teknisk och admin-istrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman iden omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sättsamt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sam-manhang som är kränkande för upphovsmannenslitterära eller konstnärliga anseende elleregenart. För ytterligare information om Linköping University Electronic Press se förlagetshemsida http://www.ep.liu.se/.

Copyright

The publishers will keep this document online on the Internet – or its possible replacement– for a period of 25 years starting from the date of publication barring exceptional circum-stances. The online availability of the document implies permanent permission for anyone toread, to download, or to print out single copies for his/hers own use and to use it unchangedfor non-commercial research and educational purpose. Subsequent transfers of copyrightcannot revoke this permission. All other uses of the document are conditional upon the con-sent of the copyright owner. The publisher has taken technical and administrative measuresto assure authenticity, security and accessibility. According to intellectual property law theauthor has the right to be mentioned when his/her work is accessed as described above andto be protected against infringement. For additional information about the Linköping Uni-versity Electronic Press and its procedures for publication and for assurance of documentintegrity, please refer to its www home page: http://www.ep.liu.se/.

c© Axel Pyk

http://www.ep.liu.se/

http://www.ep.liu.se/

Abstract

§ The market penetration of mobile access devices with multiple network interfaceshas increased dramatically over the last few years. As a consequence, the quest for awidespread multi-path transport protocol that takes advantage of all available interfaces si-multaneously to increase data throughput and improve robustness, has received consider-able attention. One prominent protocol introduced by the IETF is Multipath TCP (MPTCP).MPTCP is an extension to the predominant single-path transport protocol, the TransportControl Protocol (TCP) that enables multihomed devices to aggregate available resourcestransparently to the applications.

Combining multiple radio access technologies, like LTE and Wi-Fi, with diverse char-acteristics in terms of transmission rates and fluctuations opens for novel challenges thatmay disrupt and even harm the data throughput. Therefore MPTCP must take path hetero-geneity into account. For MPTCP to supersede single-path TCP it is required that MPTCPalways achieve at least the throughput of the best individual TCP path.

This thesis investigates if MPTCP with uncoupled congestion control fulfills this con-dition, and if so, how much it improves the throughput. By examining the protocol in a de-terministic emulated environment defined by the characteristics of LTE, we conclude twokey factors impacting the outcome: the download size and the difference in characteristicsbetween the paths. Our experiments show that MPTCP overall fulfills this task, especiallyduring path homogeneity with near aggregated results. But we also show that MPTCP maydecrease data throughput with 16 % compared to TCP during path heterogeneity. HenceMPTCP does not always fulfill the goal of throughput. We therefore conclude further intel-ligence is needed for the packet scheduling mechanism to avoid throughput degradationin the initial phase of a transmission.

Acknowledgments

Thank you to my supervisor Vengatanathan Krishnamoorthi, PhD student, and examinerNiklas Carlsson, Associate Professor, at Linköping University for the help and support dur-ing this whole process. I also thank Rasmus Axén, Thomas Walldeen and Helena Westlinderat Ericsson, Linköping, for being most accommodating and helpful in my work of progress.I would also like to thank the responsive technician of the prototype implementation at Eric-sson R&D, Stockholm, for providing detail information.

v

Contents

Abstract iii

Acknowledgments v

Contents vi

List of Figures viii

List of Tables xi

1 Introduction 1

2 Motivation 3

3 Background and Theory 73.1 Internet’s challenges for MPTCP . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.2 Bandwidth aggregating solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.3 Long Term Evolution (LTE) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.4 Transmission Control Protocol (TCP) . . . . . . . . . . . . . . . . . . . . . . . . . 203.5 Multipath TCP (MPTCP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

4 Methodology 334.1 UE experimental setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344.2 Server experimental setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354.3 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

5 Results 395.1 Aggressive retransmit feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395.2 HTTP GET experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425.3 Bulk transfer experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455.4 Varying characteristics experiments . . . . . . . . . . . . . . . . . . . . . . . . . 48

6 Discussion 536.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 546.2 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 556.3 The work in a wider context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

7 Conclusions 57

A Terminology and acronyms 59

B Tables 61

C Figures 67

vi

Bibliography 69

List of Figures

2.1 shows the global smartphone data traffic per region and month in exabytes. . . . . 4

3.1 illustrates a modified 5-layer version of the TCP/IP model with descriptions andexample techniques. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.2 illustrates the principle of encapsulation used in network communication systems. 83.3 displays an incomplete illustration of the hourglass architecture of the Internet. . . 103.4 illustrates the initial design of the Internet where the transport layer provided

end-to-end functionality. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.5 illustrates how a middlebox violates the end-to-end argument. Falsely indicating

content is delivered to destination by acknowledging data to the sender. Due tothe middlebox the transport layer does not fulfill end-to-end principle. . . . . . . . 11

3.6 illustrates the partitioning of the current transport layer into three different layersby the TNG model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

3.7 illustrates the TNG architecture with end-to-end at transport layer. . . . . . . . . . 133.8 illustrates an incomplete overview of the Evolved Packet System (EPS) including

bearers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.9 illustrates the E-UTRAN user plane protocol stack. . . . . . . . . . . . . . . . . . . . 163.10 illustrates three different allocation schemes. OFDMA is used in LTE downlink

and SC-FDMA is used in LTE uplink. . . . . . . . . . . . . . . . . . . . . . . . . . . 163.11 illustrates the generic radio frame of LTE. . . . . . . . . . . . . . . . . . . . . . . . . 173.12 illustrates a block diagram of the physical layer of LTE when transmitting a bit

stream. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.13 illustrates the LTE Quality of Service (QoS) topology. . . . . . . . . . . . . . . . . . 193.14 illustrates the TCP header structure in solid with accompanying data part dotted. . 203.15 illustrates the three-way handshake exchange. . . . . . . . . . . . . . . . . . . . . . 213.16 illustrates the TCP flow control. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.17 shows the impact of packet error probability p on the maximum data rate of a TCP

transmission with packet RTT = 10ms & 60ms and 1460 octets MSS. . . . . . . . . . 243.18 illustrates the four-way handshake process used when terminating a TCP connec-

tion using the finished (FIN) and acknowledged (ACK) flags. . . . . . . . . . . . . 253.19 illustrates the relationship between standard TCP (left) and MPTCP (right) . . . . . 263.20 shows the MPTCP data sequence numbering. . . . . . . . . . . . . . . . . . . . . . . 283.21 illustrates a blocking scenario during path heterogeneity leading to throughput

degradation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293.22 shows the mandatory MPTCP option field format. . . . . . . . . . . . . . . . . . . . 313.23 illustrates the connection sequence of a MPTCP capable operation followed by a

MPTCP join connection operation, where . . . . . . . . . . . . . . . . . . . . . . . . 31

4.1 shows a blueprint of the experimental setup with flow of data. . . . . . . . . . . . . 33

viii

4.2 shows the architecture of the simulated receiving side user equipment in the ex-perimental setup as a stack. On the top are the applications (wget and netperf)and in the bottom the network interface controllers. Traffic comes in through theNIC moves up the stack to destined application, then down the stack and outthrough the NIC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

4.3 shows the server side architecture in the experimental setup as a stack with theapplications at the top and network interface controller at the bottom. . . . . . . . 36

5.1 shows the throughput T in two separate 20 second TCP transmissions. TCP1 allo-cate 40 Mbit/s stationary. TCP2 alternate allocated bandwidth between 40 and 1Mbit/s with 0.2 Hz. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

5.2 shows the throughput T for two MPTCP subflows; Subflow1 and Subflow2, andthe MPTCP goodput G when the aggressive retransmit feature is off. Subflow1allocate 40 Mbit/s stationary. Subflow2 alternate allocated bandwidth between 40and 1 Mbit/s with 0.2 Hz. This is the same characteristics scenario as Figure 5.1.The arrows mark the areas where Subflow1 is affected by the changes in Subflow2and flow starvation occurs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

5.3 shows the throughput T for two MPTCP subflows; Subflow1 and Subflow2,and the MPTCP goodput G when the aggressive retransmit feature is activated.Subflow1 allocate 40 Mbit/s stationary. Subflow2 alternate allocated bandwidthbetween 40 and 1 Mbit/s with 0.2 Hz. This is the same characteristics scenario asFigure 5.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

5.4 shows the average download times as a function of file size when using eitherregular TCP (red squares) and MPTCP (blue circles) for the best-case scenario.Both lines are equipped with a dashed linear fit. The graph is equipped with azoomed area displaying file sizes 32KB to 512KB. . . . . . . . . . . . . . . . . . . . . 42

5.5 shows the MPTCP rate in percentage compared to the rate the best TCP path giventhe test scenario in Figure 5.4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

5.6 shows the average download times as a function of file size when using eitherregular TCP (red squares) and MPTCP (blue circles) for the worst-case scenario.Both lines are equipped with a dashed linear fit. The graph is equipped with azoomed area displaying file sizes 32KB to 512KB. . . . . . . . . . . . . . . . . . . . . 43

5.7 shows the MPTCP rate in percentage compared to the rate the best TCP path giventhe test scenario in Figure 5.6. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

5.8 shows the average download times as a function of file size between 32KB and4MB when using either MPTCP or regular TCP. Red squares indicate the bestdownload time when using regular TCP, blue circles indicate the worst attaineddownload time when using MPTCP and green cross indicates the best downloadtime when using MPTCP. Both lines are equipped with a dashed linear fit. Grayedarea indicate MPTCP upper- and lower bound. Each gray dot is a test result. Thegraph is equipped with a zoomed area for sizes 32 to 512KB. . . . . . . . . . . . . . 44

5.9 shows the normalized gain compared to the best TCP path as a function of twosubflow RTTs when BW1 = 40 Mbit/s and BW2 = 8 Mbit/s. The markers indicatethe three different packet error probabilities. . . . . . . . . . . . . . . . . . . . . . . 45

5.10 shows the normalized gain compared to the best TCP path as a function of twosubflow RTTs when BW1 = BW2 = 40 Mbit/s. The markers indicate three differ-ent packet error probabilities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

5.11 shows the normalized gain compared to the best TCP path as a function of twosubflow RTTs for two test scenarios. The markers indicate the three differentpacket error probabilities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

5.12 shows the throughput T for Subflow1 and Sublow2, the RTT for Subflow2 and theMPTCP goodput in the scenario without the bufferbloat phenomena. At T = 0 thebandwidth drop from 40 Mbit/s to 256 Kbit/s. . . . . . . . . . . . . . . . . . . . . . 48

5.13 shows the throughput T for Subflow1 and Sublow2, the RTT for Subflow2 and theMPTCP goodput in the scenario with the bufferbloat phenomena. At T = 0 thebandwidth drop from 40 Mbit/s to 256 Kbit/s. It is reset after 5 seconds. . . . . . . 49

5.14 shows the RTT of Subflow2 when either exposed (solid blue) or unexposed(dashed red) to the bufferbloat phenomena. . . . . . . . . . . . . . . . . . . . . . . . 49

5.15 shows the MPTCP transmission rate for both subflows and the goodput forcingthe bufferbloat phenomenon when changing the bandwidth of Subflow2 between40 and 1 Mbit/s with a frequency of 2Hz. Subflow1 allocate stationary 40 Mbit/sand RTT = 10ms and p = 0%. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

5.16 shows the results from the packet loss experiment. Both subflows allocate 40Mbit/s and ∆RTT = 50ms. At T = 0 the packet error probability changes from0% to 0.1% and p is reset at T = 5. The lines indicate the average rate during theinterval. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

C.1 shows the MPTCP resource utilization percentage as a function of two subflowRTTs when BW1 = 40 Mbit/s and BW2 = 40 Mbit/s. The markers indicate threedifferent packet error probabilities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67



List of Tables

B.1 LTE QoS Class Identifiers (QCI). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61B.3 The MPTCP option subtypes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62B.5 Details the average improvement when using MPTCP compared the best con-

stituent path with TCP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62B.6 Details the resulting throughput in Mbit/s from the bulk experiment when

Subflow1 allocate 40 Mbit/s stationary and Subflow2 allocate 8 Mbit/s. Viewed inthe table are the two different packet latencies used in the bulk experiment; to theleft a latency difference of 0ms (10ms, 10ms), and to the right a 50ms difference(10ms, 60ms). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

B.7 Details the resulting throughput in Mbit/s from the bulk experiment whenSubflow1 allocate 40 Mbit/s stationary and Subflow2 allocate 40 Mbit/s. Viewedin the table are the two different packet latencies used in the bulk experiment; tothe left a latency difference of 0ms (10ms, 10ms), and to the right a 50ms difference(10ms, 60ms). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

B.8 Details the resulting throughput in Mbit/s from the bulk experiment whenSubflow1 allocate 40 Mbit/s stationary and Subflow2 allocate 1 Mbit/s. Viewed inthe table are the two different packet latencies used in the bulk experiment; to theleft a latency difference of 0ms (10ms, 10ms), and to the right a 50ms difference(10ms, 60ms). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

B.9 The goals of MPTCP architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64B.11 The specifications of the experimental setup hardware. . . . . . . . . . . . . . . . . 65

xi

1 Introduction

In recent years, a variety of Radio Access Technologies (RATs) with diverse transmission fea-tures have enabled network operators to offer services to customers with varying demands.Examples include short- and long ranged RATs such as IEEE 802.11, commonly known asWi-Fi, and Long Term Evolution (LTE), which forms the basis for the fourth generation mo-bile broadband (4G). RATs like these are responses to increasing demands for mobile networkresources, due to an on-going influx of bandwidth intensive services, increased availabilityof low-price portable devices and the evolution of Internet of Things (IoT)1. According tothese factors and more, it is forecast that mobile data traffic will eight-fold between 2014 and2020 globally [14]. Because todays’ radio resources already are severely limited one visiondiscussed by the telecommunication industry are wireless networks that allow an integrationof different RATs to form heterogeneous wireless networks [41].

A Heterogeneous Wireless Network (HWN) allows network operators to combine advan-tages and thereby overcome disadvantages of different RATs [11]. They allow operators todeliver high-quality services to customers with smartphones, tablets and other multi-RATportable devices. A multi-RAT device is a unit equipped with multiple controllers henceable to communicate using different radio technologies. Benefits of HWNs include addi-tional capacity, better coverage and better bandwidth. Depending on desired characteristics,subscribers with multi-RAT devices can switch between or combine RATs in overlappingcoverage areas to meet their application requirements. By creating a single logical link sepa-rated over multiple resource, the principle of bandwidth aggregation is achievable. It allowsresource maximization and increased redundancy without any changes at the applicationlevel. Multi-RAT devices are potentially capable of aggregating bandwidth since they areequipped with multiple network controllers [65].

The most used transport protocol within the core of Internet is the Transmission ControlProtocol (TCP), standardized in 1981 in RFC 793 [43]. TCP was designed and introducedalong with the Internet Protocol (IP) [42] to ensure reliability, avoid congestion collapse andfacilitate fair resource sharing in a network. They are commonly known as the TCP/IP pro-tocol suite. The generic term TCP/IP usually means anything and everything related to thespecific protocols of TCP and IP. It can include other protocols, applications, and even the

1Internet of Things is a network of interconnecting physical objects, or “things", to offer advanced exchange ofinformation between devices, systems and services. This is to achieve greater value to the customers by exchangingdata with the manufacturer, operator and/or other connected devices [59].

1

network medium [55]. Although TCP has been mostly unchanged in the past thirty years,TCP still fulfills its purpose eminently. Over the same time period, applications and servicescommunicating over the Internet have evolved exponentially and sandwiched TCP/IP be-tween an influx of application layer protocols and link layer protocols, making developmentat the transport layer difficult. This is further explained in Chapter 3.

Because modern networks aim for heterogeneity with the ability to combine resourcesfrom multiple technologies, this has been a subject to solve for the research community pastdecade. Today’s networks are restricted by TCP’s inability to operate over multiple inter-faces. In order to solve this issue the Internet Engineering Task Force (IETF) published RFC6824 [18] in January 2013. RFC 6824 details an experimental protocol standard called Multi-path TCP (MPTCP) with a set of extensions to TCP. These extensions enable TCP to operateacross multiple paths simultaneously. MPTCP operates by transparently opening multipleTCP connections to a MPTCP capable host with the goal of improving throughput and re-silience to network failures.

Aggregating different technologies’ resources with dissimilar characteristics in terms ofbandwidth, latencies and packet error probabilities opens the possibility of new disruptiveobjectives like synchronization, that may degrade the throughput of a multi path transmis-sion [65]. One of the functional goals of MPTCP quoted from RFC 6182 is “to meet theminimum performance incentives for deployment, a Multipath TCP connection over mul-tiple paths should achieve no worse throughput than a single TCP connection over the bestconstituent path." This thesis investigates if it is possible to fulfill this goal practically withproposed design and asymmetric characteristics. We perform a performance evaluation onMPTCP to determine how MPTCP impacts application layer throughput in short- and bulktransfers given dissimilar transmission characteristics compared to regular TCP. Experimentswere conducted in a closed experimental setup to create reproducible test-scenarios with min-imal influence from external artifacts.

Our results conclude that MPTCP attains a rate close to the aggregated rates of the con-stituent links during path homogeneity in bulk transfers. With increasing differences duringpath heterogeneity the gain decreases until the rate is near equal the rate of the best TCP con-nection, thus MPTCP more or less fulfills its goal in bulk transfers. Our key findings includethat MPTCP can degrade the performance with up to „ 16% compared to the best TCP path.Especially web-browsing most likely is to be harmed by an introduction of MPTCP. Hencethe download size is a key factor if uncoupled MPTCP is advantageous over TCP. This opensthe discussion on how and when to introduce MPTCP ubiquitously.

The remainder of this thesis is structured as follows: Chapter 2 details the motivation ofthis thesis. Chapter 3 provides a brief background about the problems behind the develop-ment of MPTCP with the next generation architecture. This is followed by an overview oftransport layer bandwidth aggregation, LTE radio access technology and its characteristicsaccording to its class-based QoS. The reminder of Chapter 3 details the theory of TCP andMPTCP, in terms of design, goals and theory. Chapter 4 details the methodology of the the-sis. The results from the experiments are detailed in Chapter 5, followed by the discussionincluding future work in Chapter 6. The conclusions is presented in Chapter 7.

2

2 Motivation

Since the introduction of the Internet Protocol (IP) and Transmission Control Protocol (TCP)in the beginning of the eighties, the Internet1 has reached unbelievable proportions. As aresult of the World Wide Web (WWW) the Internet moved into family homes during thenineties hence established it globally. Twenty years later the number of active users today onthe Internet is estimated to be approximately three billion, which is one third of the world’spopulation [63]. Today’s interconnectivity between people and services makes Internet akey in most industrial peoples’ daily life. Social medias like Facebook and Twitter activeusers ranging from hundreds of millions to billions per month. Services earlier dominatedby physical retail e.g. audio and video are moving from the streets to the cloud with on-demand streaming services like Netflix, Spotify, HBO, etc. Thus the transition to the cloud ofaudio-/video services entail Internet to supersede the CD-/DVD era.

Driven by fixed broadband in the past, network operators today provide high-speed con-nections with mobile standards like Wideband Code Division Multiple Access (WCDMA),High Speed Packet Access (HSPA) and Long Term Evolution (LTE). Providing peak rates at1 Gbit/s stationary and 100 Mbit/s in motion, mobile broadband and LTE are able to play acomplementary or even replacing role to fixed broadband [2]. With on-going infrastructuraldevelopment of mobile broadband and increased availability of low-price smartphones, peo-ple are able to change from cellular phones to smartphones, especially in the developingworld. Thus the number of mobile broadband subscribers is forecast to grow nearly threetimes from 2.9 to 8.4 billions between 2014 and 2020 [15]. The largest portion of the growth isrelated to LTE (3.1 billion).

Along with the increasing number of mobile-broadband subscribers and services movingto the cloud, applications become more network intensive. The evolution of portable devicescommits access networks to deliver larger amounts of data to provide higher image-, music-and video qualities. Ericsson forecast 45% annual growth in the video segment between2014 and 2020. With such growth video will comprise the largest segment of all segments(e.g. audio, social networking, web browsing, etc.) of mobile data traffic in 2020 (55%) [15].Given these factors, the global average data traffic per smartphone and month 2020 is pro-jected to be four times more than 2014, 4 GB compared to 1 GB. Besides this, the growing

1Internet can be defined as world’s largest meta-network. It is a global system of interconnected computernetworks without centralized governance. Interconnected networks are implemented by a common network modelreferred as the Internet protocol suite or TCP/IP-model referring to the core protocols of the Internet [62].

3

phenomenon referred as the Internet of Things require enormous amounts of traffic whenbillions of devices communicate.

All these factors together increase the demand for mobile network resources. Since theamount of radio frequencies is already constrained and therefore the amount of resourceslimited, this will be a challenge for the network operators. Figure 2.1 is referenced fromEricsson Mobility Report February 2015 [15] and displays the global data traffic per monthin exabytes separated in geographical regions for the smartphone segment. Viewed in thefigure the global mobile data traffic is projected to grow 40% annually leading to a 8-foldincrease in six years. According to Ericsson 70% of mobile broadband data traffic will befrom smartphones by the end of 2020 [15].

6 ERICSSON MOBILITY REPORT MWC EDITION FEBRUARY 2015

Smartphone traffic dominatesGlobal mobile traffic (monthly ExaBytes)

25

0

5

10

15

20

20202014 2017

Data: mobile PCs, tablets and mobile routersData: smartphones

Voice

0

5

10

15

20

2014 2017 2020

Smartphone data traffic per region (monthly ExaBytes)

Latin America

North America

Asia Pacific

Middle East and Africa

Central and Eastern Europe

Western Europe

Total mobile data traffic is expected to rise at a compound annual growth rate (CAGR) of around 40 percent

The rising number of smartphone subscriptions and increasing data consumption per subscriber are driving mobile data traffic growth. This will result in an 8-fold increase in traffic by the end of 2020. The growth in data traffic between 2019 and 2020 will be greater than the total sum of all mobile data traffic up to the end of 2013.

There are large differences in subscribers’ data consumption patterns between networks, markets and subscriber segments. Factors such as data plans, user device capabilities and network performance all impact data consumption per subscriber.

70% of mobile data traffic will be from

smartphones by the end of 2020

In 2020, smartphones alone will generate five times the total mobile

traffic of today

Asia Pacific will generate 50 percent of smartphone traffic by the end of 2020

Monthly smartphone data consumption per active subscription in Asia Pacific (3.2 GB) will only be 50 percent of that in North America (6.0 GB) and Western Europe (6.5 GB). However, the Asia Pacific region will have the largest share of total smartphone traffic in 2020, due to subscription growth.

X5

Source: Ericsson Mobility Report February 2015

Figure 2.1 shows the global smartphone data traffic per region and month in exabytes.

When the Internet evolved from the ARPANET2 backbone, multi-path was not a primaryconcern and therefore was not included in the initial design. With the increased demandfor network resources, multi-connectivity alternatives can offload demands by moving traf-fic to other less congested paths and improve performance by combining resources. Poolingresources is one of the most established principles to enhance performance in terms of ro-bustness and throughput. In order to understand the challenges in extending the Internetmodel to enable multi-path, it is therefore necessary to understand the architecture behind it.In the next chapter we detail why earlier multipath transport protocols have not succeededand how the new design used in Multipath TCP, make the protocol with most potential.

The first ubiquitous operating system released with Multipath TCP (MPTCP) includedwas Apple iOS 7 in September 2013 [24]. However the implementation of MPTCP in iOS isrestricted to their Siri application, to enhance robustness and not throughput. From an end-user’s perspective, the quality of a network connection is often determined by the applicationlevel throughput. There are essentially three fundamental factors impacting the throughputof a network transmission:

• Bottleneck bandwidth defines the amount of resources available to transfer information.

• Packet latency defines the time it takes a packet to travel from one end-point to another.Round-Trip Time (RTT) refers to the duration between a packet sent and an acknowl-edgment received, indicating the packet was successfully received at destination.

2Advanced Research Projects Agency NETwork (ARPANET) was developed by U.S. Department of Defenseagency Defense Advanced Research Projects Agency (DARPA) during the Cold War and completed in 1969 [5].

4

• Packet error probability defines the amount of packets lost in transmission or corruptat destination.

To understand the impact with an introduction of MPTCP to multi-RAT devices, there isa need to understand how the throughput of MPTCP in practice depends on different trans-mission characteristics in different scenarios. Other studies exist in the area, most of themare simulation- and measurement-based. Raiciu et al. [47], [66] shows that MPTCP improveload distribution in data centers compared to standard TCP in combination with randomizedflow-level load balancing. Chen et al. [7], [6] show that MPTCP is robust in achieving closeto the best single path TCP connection, where the download size is a key factor to the gainof MPTCP compared to TCP and that MPTCP supports mobility without breaking existingconnections. Paasch et al. [40] studied the mobile/Wi-Fi handover performance of MPTCPwhen the Wi-Fi interface goes down. They conclude MPTCP is robust during mobile/Wi-Fihandovers.

Measurement studies have also reported download speed differences that can be lever-aged by multi-homed clients. For example, Linder et al. [32] use crowd-based measurementsto characterize the download speeds seen by different technologies and operators. In theirwork they identify many locations with significant download speeds differences betweenoperators. With no operator consistently outperforming the other operators and the bestoperator often varying between locations, their data suggests that there may be significantadvantages for multi-homed users running a multi-path transport protocol such as MPTCP.In contrast to their work, this thesis focuses on the potential performance improvement seenby such clients.

This Master’s Thesis presents an initial and comprehensive study using a performance-focused prototype implementation to evaluate an introduction of MPTCP. Based on the fun-damental functional goal of MPTCP mentioned earlier the following questions were defined:

a) Does an introduction of MPTCP degrade end-user throughput compared to best pathof regular TCP in path heterogeneity?

b) How responsive is MPTCP to varying link characteristics?

c) Does MPTCP utilize given resources?

Our goal is to determine if MPTCP fulfills its goal in short-burst traffic and bulk trans-fers. In our results we compare the gain, resource utilization and responsiveness of MPTCPwith situations related to LTE. The MPTCP stack was provided by Ericsson Research & De-velopment department and later applied in a closed network environment with the abilityto emulate desired characteristics. Based on the class-based QoS of LTE, we design a set ofexperiments with characteristics ranging from good to bad. To determine attained gain whenintroducing MPTCP we use the best case scenario with single path TCP as reference value inthe process of evaluation. In the second part we stress-test the protocol by performing rapidchanges in the characteristics and monitoring how the protocol reacts.

5

6

3 Background and Theory

This chapter discuss the model design of the Internet and why it is difficult to implementnew protocols like Multipath TCP. The chapter then describe the benefits of bandwidth ag-gregation, the technology of LTE with the background of radio access technologies. Finally itdiscuss the Transmission Control Protocol and the intended protocol; Multipath TCP.

The design of modern communication systems can be illustrated with a variety of models, e.g.the four layered TCP/IP model, the seven layered OSI model [49], AppleTalk, InternetworkPacket Exchange/Sequenced Packet Exchange (IPX/SPX), etc. We use a five-layered stackmodel, based on the four layers in the TCP/IP model including the physical layer from theOSI model [29]. The model is shown below in Figure 3.1.

Physical Layer Media, Signal and Binary Transmission

Data Link Layer MAC and LLC (Physical Addressing)

Network Layer Path Determination and IP (Logical Addressing)

Transport Layer End-to-End Connection and Reliability

Application Layer Network Process to Application HTTP, HTTPS, DNS, SSL, FTP, SMTP ...

TCP, UDP, SCTP, UDP-Lite ...

IP, IPSec, ICMP, ARP, DHCP ...

Ethernet, LTE, UMTS, WiMAX, HSPA, WiFi, Bluetooth, IP ...

NR

5

4

3

2

1

Extended TCP/IP Model Internet Protocols

Ho

stN

etw

ork

Figure 3.1 illustrates a modified 5-layer version of the TCP/IP model with descriptions andexample techniques.

The goal of these models is to characterize and standardize the internal functionality ofcommunication systems according to a stack. Our stack is composed of five independentlayers, each providing a service to the layer above. Structuring communications accordingto these models form a flexible, robust and interoperable architecture that allows systems tocommunicate using a broad range of transmission mediums and services without any hard-ware or software requirements. By allowing the principle of protocol multiplexing, multipleinstances per protocol are allowed to operate simultaneously within the same infrastructureover a single network link.

7

The fundamental principle behind this layered model is the principle of encapsulation.Encapsulation creates abstraction between the services at different layers. When data is sentfrom an application, logically higher protocols is further encapsulated at each layer with ad-ditional header (and footer) information included to direct it towards its destination.

Link Layer

Transport Layer

Network Layer

Application LayerDATA

TCP DATA

MESSAGE

TCP HEADER

IP HEADER

FRAME HEADER

SEGMENT

DATAGRAM

IP DATA

FRAME DATA FRAME FOOTER

FRAME

Figure 3.2 illustrates the principle of encapsulation used in network communication systems.

The application layer is the highest layer closest to the end-users through the network ap-plications. It provides process-to-process services between two or more entities in a com-munication system across a TCP/IP internet. Usually network applications rely on existingnetwork protocols to provide intercommunications between hosts. A network protocol is asystem of rules that defines a syntax, semantics and synchronization standard of transmittedmessages to provide a service. Example includes common protocols like the File Transfer Pro-tocol (FTP), Hypertext Transfer Protocol (HTTP), Simple Mail Transfer Protocol (SMTP) andmany more. There are no requirement for a network application to use an existing networkprotocol. Instead they are able to send information directly to the endpoint of the applicationlayer using the network socket Application Programming Interface (API). The socket API al-lows direct access to the network. A network socket is a process end-point, defined by a IPaddress, that determines the destination host, and a port number directing the informationto the correct process.

The transport layer is the layer closest to the network and defines the edge between thehost and the network. It provides host-to-host transport services to applications using differ-ent transport layer protocols. This is to hide the complexity of the network from the upperlayers. Depending on desired behaviour by the applications, the transport layer provides awide variety of services e.g. connection-oriented communication, reliability, flow and con-gestion control. Connection-oriented communication is a mode that states that a transmis-sion session must be established before information can be transferred and that delivery isalways error-corrected and in-order. TCP is a connection-oriented protocol. The oppositeis connection-less communication, found in stateless protocols e.g. User Datagram Protocol(UDP) and IP. Within computer networking, reliability refers to the sender’s assurance thatthe content is successfully delivered to the destination.

The network layer defines the entry point of the network. It provides global interconnec-tivity with the task to route packets across networks toward multiple remote destinations.The most used interconnecting network layer protocol is the Internet Protocol. It providesconnection-less packet routing, host addressing and packet forwarding. To receive informa-tion from an interconnected unit a device must have at least one unique IP-address withinthat particular network. To interconnect multiple networks, devices named routers or gate-ways direct packets towards their destination using information in their routing tables orrouting policies. A network may be a home or small office network that routes to Internet

8

3.1. Internet’s challenges for MPTCP

through a router connected to a Internet Service Provider (ISP). Larger business or ISP net-works use enterprise routers to connect up to powerful core routers in the conglomerationof multiple, redundant networks owned by numerous companies, referred as the Internetbackbone.

The data link layer focus on local delivery between adjacent nodes within a subnetwork.Nodes interconnected by network devices operating at the physical- or data link layer, e.g.hubs, bridges and switches. It provide services like physical addressing, packet switching,error detection and correction for information sent at the physical layer, flow and QoS control,frame synchronization and media arbitration. The data link layer can be subdivided intotwo sublayers. The Logical Link Control (LLC) sublayer provides node-to-node flow- anderror-management control using ARQ protocols, as in the transport layer. The underlyingMedia Access Control (MAC) sublayer acts as an interface between the LLC and the physicallayer. It provides addressing within a subnetwork using unique MAC-addresses and channelaccess control mechanisms to detect and avoid interference between nodes sharing the samemedium.

The physical layer is the lowest layer in the network model and refers to the basic hard-ware communication technologies used within a transmission. It defines the means of trans-mitting raw bits rather than logical data packets over a physical link between two networknodes. The bits are converted to a physical signal that is transmitted over hardware trans-mission mediums, e.g. electrical connectors, electromagnetic waves.

3.1 Internet’s challenges for MPTCP

Numerous multi-path transport protocols already have been published. Some based on TCP[20], [23], [33], [50] and others based on the Stream Control Transmission Protocol (SCTP)[26], [31]. Still TCP is the most used transport protocol and it is restricted to operate over asingle path. This section clarifies the origin of the problem in more detail.

When networks’ extensibility at both high and low levels have evolved, the core of the In-ternet has become trapped in the middle. This has resulted in a hourglass design illustratedin Figure 3.3. On top of the already complex design is the current version of the transportlayer providing a widespread selection of functionality: identifying application end-pointsvia port numbers [44], [42], end-to-end congestion control and fairness [30], utilizing alter-nate end-to-end paths [58] and providing reliable in-order delivery [42], [58]. The approachof merging much functionality into a single layer has made the transport layer difficult toevolve. Thus there are only two transport layer protocols working ubiquitously: TCP and theUser Datagram Protocol (UDP) [44].

To obtain neutral networks, network devices e.g. routers, bridges and hubs, are designedto operate between network layers in the initial design of Internet, illustrated in Figure 3.4.Switches and hubs would only operate at data link- and physical layers to interconnect nodes.Flexibility was a main factor in the initial design of the Internet. Intermediate systems shouldnot care about the transferred content stated by the end-to-end principle, only provide in-terconnected hop-to-hop services using the IP protocol. During the time computer networkshave evolved, a variety of extensions affecting the behaviour of the networks have been in-troduced.

Kurose et al. [45] published an analysis that showed that approximately half of the IP-traffic carrying information in the options field were discarded due to middleboxes, raisingserious dependability issues. The provision of the optional header field allows protocols tobe extended with further functionality without affecting the standard. Network componentmanufacturers implemented such drop modes, e.g. Cisco’s “ACL IP Options Selective Drop"

9


[9], to prevent their routers’ CPU from overloading. Although IP options were fully standard-ized in RFC 791 [42], the demand for improved throughput, monitoring and security madethe Internet irregular.

Application Layer

Transport Layer

Network Layer

Data Link Layer

Firefox HotmailGoogle

Mail FileZilla Transmission

HTTP SMTP FTP P2P BitTorrent

TCP UDP

IP

Starcraft 2 Skype

RTP

Ethernet IEEE 802 LTEPPP

Physical LayerOpticalfiber

IEEE 802.adCoaxialcable

Twistedpair OFDM

Figure 3.3 displays an incomplete illustration of the hourglass architecture of the Internet.

Figure 3.4 illustrates the initial design of the Internet where the transport layer provided end-to-end functionality.

Middlebox

A middlebox is defined as any intermediary device performing functionality other than stan-dard functions of a network router on the path between a source host and destination host[39]. By this definition middleboxes include a variety of devices, e.g. firewalls, NATs, applica-tion layer gateways, performance enhancing proxies, traffic normalizers, etc. The functional-ities of such devices are essential in todays access networks. Examples include functionalityto prevent malicious users from accessing confidential information and causing damage tothe network, routing and improving performance. The proliferation of mobile devices is setto further expand the range of middlebox applications when new markets are faced with newchallenges and possibilities.

The fundamental principle behind Internet and large distributed communication net-works is the principle of the end-to-end argument. Quoted from Saltzer et al. [51] in theirarticle about system design:

"The principle, called the end-to-end argument, suggests that functions placedat low levels of a system may be redundant or of little value when compared withthe cost of providing them at that low level."

10


The end-to-end argument implies neutral networks where communications protocol op-erations should be defined to occur at the end-points of the communication system ratherthan in intermediary nodes. Provided they can be implemented completely and correctly inthe end hosts. This is to avoid implementing intelligence beyond bit error recovery, securityusing encryption, duplicate message suppression, recovery from system crashes and deliveryacknowledgment in the networks. This is necessary because any network, however carefullydesigned will subject to failures of transmission at some statistically determined rate. Hencethe best solution is to place responsibility for the integrity of communication to the end sys-tems.

An example how middleboxes violate the end-to-end principle including TCP is illus-trated in Figure 3.5. When a middlebox receives data it sends an acknowledgment to thesender. The sender is than falsely indicated the transmission successfully transmitted to thehost, but the data is than sent towards the destination from the middlebox and lost. Theoutcome of the incident is the sender falsely thinks the content is delivered. This has left theapplication layer as the first end-to-end layer.

Figure 3.5 illustrates how a middlebox violates the end-to-end argument. Falsely indicatingcontent is delivered to destination by acknowledging data to the sender. Due to the middle-box the transport layer does not fulfill end-to-end principle.

The problem in introduction of multi-path communication arises due to the uncertainty ofhow already deployed middlebox applications behaves to these new standards. Past intro-ductions of transport layer protocols have been unsuccessful as middleboxes performs pro-cessing at higher layers thus terminates the new protocols and preventing widespread dis-semination. Examples include SCTP [58] and Datagram Congestion Control Protocol (DCCP)[16]. Network component manufactures spread diversity of functionality throughout net-works when implementing proprietary solutions in middleboxes that override standards. In-stead of concentrating them at the end-systems. They may damage the end-to-end nature ofthe Internet. Without any global standard how middleboxes should behave to new protocolsand act upon failures, no one can predict how middleboxes will behave towards MPTCP.

In 2012 the first large-scale study of middlebox deployment was studied by Sherry et al.[54]. Their study is based on surveyed data of 57 enterprise networks. They concluded thenumber of middleboxes operating on networks is almost as large as the number of routers.This emphasizes the importance of middleboxes in networks and considering middleboxeswhen designing new protocols to introduce centralized standards.

As the middlebox-problem grew, the number of unsuccessful transport protocols provid-ing simultaneous use of multiple paths increased. Thereby IETF and the research communityrealized they had to adapt. Rather then introducing a new protocol, they had to reuse anexisting one and design MPTCP around the middlebox-problem.

11


Middleboxes effect on TCP Options field

To successfully introduce a transport protocol at global level, it is necessary to design theprotocol to be accepted by already deployed intermediate devices in the networks. Honda etal. [22] published an extensive study on 142 networks in 24 countries, where they probedthe networks for middleboxes violating the end-to-end principle. Their study showed that atleast 25% of paths interfered with traffic communication at the transport layer in some way,firewalls excluded. But their study also concluded that it is possible to extend TCP using theoptions field, as long such design is robust and adaptive.

Next generation transport layer model

To solve the middlebox-problem, mentioned in Section 3.1 (page 9), and reapply the end-to-end argument to the transport layer, Ford et al. [19] presented a new design model expandingthe transport layer called Transport Next Generation (TNG). TNG was invented with thegoal to regain end-to-end connectivity to the transport layer. Their approach suggested anew layering model, factoring the transport layer into three separate layers, illustrated inFigure 3.6.

Figure 3.6 illustrates the partitioning of the current transport layer into three different layersby the TNG model.

The three layers can be divided into two categories: application-oriented and network-oriented. The logically highest layer, supersedes current transport layer and provides limitedapplication-oriented functionality to protect and support the end-to-end communication forhigher-level layers, e.g. reliability, ordering and error recovery. The two lower network-oriented layers provide network functionality and operate segment-by-segment in the trans-mission. The Endpoint layer is a extension of the network layer and provides shared func-tionality among transport protocols e.g. application end-point identification via port number.The Flow Regulation layer, in the future called Flow layer, provides congestion control andperformance-related mechanisms.

The most important benefit when refactoring the functionality of the transport layer intoapplication- and network-oriented layers, is intermediate devices are allowed to traverse thenetwork-oriented layers without affecting the transport layer. This preserves the end-to-endprinciple at the transport layer, illustrated in Figure 3.7. TNG also benefit deployments of newtransport protocols by facilitating the transaction when moving new protocols from user-space libraries into operating system kernels. Additionally TNG extends the flexibility forapplications to decide which transport protocol to use rather than opening new sessions perspecific protocol. Similarity of functionality is found today at the Session- and Presentationlayers in the OSI model.

It may seem optimistic to introduce a major architectural change as TNG in Internet, butit is possible to achieve incremental deployment. By utilizing existing protocols in the lowerlayers, e.g. UDP at the endpoint layer or TCP at the flow regulation layer, it provides imme-diate deployment and a basis for long-term evolution.

12

3.2. Bandwidth aggregating solutions

Figure 3.7 illustrates the TNG architecture with end-to-end at transport layer.

Due to the large diversity within network architectures, the problem has been unsolvedfor decades. But the promising extension of MPTCP, an solution may be in reach as long itdoes not impact the current situation within the networks. This is not included in the scopeof this thesis.

3.2 Bandwidth aggregating solutions

In this section we further detail benefits of transport layer bandwidth aggregation and theaggregating approach used in MPTCP.

According to an overview published by Ramaboli et al. [48], bandwidth aggregating solu-tions can be divided into two approaches:

a) Non-adaptive solutions do not adjust their resource allocation and traffic schedules.This means links send data whenever possible, expecting the data to be successfullytransferred and upon failure handles the consequences. The downside is non-adaptivesolutions may be impacted by the event of changes in characteristics.

b) Adaptive solutions corrects the pitfalls of non-adaptive practices. By listening to thebehaviour of the network and adapting the transmission accordingly. They are morecomplex and harder to implement.

MPTCP solution: The MPTCP standard defined in RFC 6824 [18] can be classified as a non-adaptive solution. The protocol does not include intelligent interface selection and trafficdistribution mechanisms to reduce the occurrence of packet reordering at the destination inan heterogeneous network. To handle reordering, MPTCP employs a buffer to store out-of-order segments before sending them in-order to the application.

Benefits of aggregating heterogeneous RATs

As mentioned in the beginning of Chapter 3, the primary benefits of aggregating resourcesare increased bandwidth and improved failure robustness. When combining different RATsthere are also a couple of additional benefits.

a) Aggregating resources of several radio access technologies allows multi-RAT devices toimprove bandwidth without the need for infrastructural developments [48].

b) Using more than one RAT, traffic can be distributed evenly or unevenly over multi-ple computing resources depending on different conditions. Hence congestion may beavoided and efficient usage of network resources allowed. By moving load balancingfrom the Internet Service Providers (ISPs) to the end-systems resource utilization can beoptimized globally, not only locally.

13

3.3. Long Term Evolution (LTE)

c) Transport layer protocols provides full congestion detection and avoidance of the linksand direct contact to the destination. This makes it possible to leverage TCP handshakesdirectly thus bootstrap subflows quickly. Thereby will transport layer protocols achievebetter load balancing than application layer protocols [48].

3.3 Long Term Evolution (LTE)

This section details factors impacting the end-user experience in Radio Access Technologiesand why the physical layer within Radio Access Networks (RANs) often is the main bottle-neck.

The theoretical limit information can be transmitted through any analog communicationchannel, regardless of the technology, is fundamentally limited by two parameters: theamount of allocated bandwidth and the signal-to-noise ratio between transmitter and receiverusing a single antenna. This is known as the Shannon–Hartley channel capacity [53]:

C = BW ˚ log2(1 +SN). (3.1)

Here, C denotes the channel capacity which defines the theoretical upper information rate(excluding error correcting codes) of clean (or arbitrarily low bit error rate) data that canbe sent through an analog communication channel in bits/s. The channel capacity is af-fected by the amount of allocated bandwidth BW of the channel in hertz, the average signalpower S in watts and the average noise or interference power N in watts. The ratio S

N iscommonly referred as the signal-to-noise ratio (SNR). Due to the occurrence of interferencein radio communication systems compared to wired, it is necessary to include it using thesignal-to-interference-plus-noise ratio (SINR). With encapsulation overhead excluded fromthe channel capacity the theoretical rate of a radio transmission is referred as the peak rate.

Shown in the Shannon–Hartley equation the capacity depends on quality of a radio signal.The reduction in power density a electromagnetic wave is subjected to when it propagatesthrough the environment is referred as the signal path loss.

To handle larger differences in SNR there have been an extensive drive for the develop-ment of new adaptive modulation schemes in RATs compared to wired access technologies.Two prominent radio technologies that are used are, spatial multiplexing, also known asmultiple-input multiple-output (MIMO) and adaptive modulation. MIMO provides the abil-ity the achieve link aggregation at the physical layer and adaptive modulation enhances thethroughput by adapting the robustness of the digital encoding (error rate coding) accordingto the signal quality, thus balances the throughput with error resilience.

History

When further development of 3G standards became costly due to inherent design limitationsin UMTS, 3GPP initiated a work item termed “3G Long Term Evolution - Evolved PacketSystem RAN", referred as LTE. The LTE work item was commenced to redesign both thecore network and the radio networks to create a new highly flexible standard. This standardwould lay the groundwork in the quest to achieve the requirements for the fourth genera-tion (4G) mobile access service. These requirements are issued by the RadiocommunicationSector of the International Telecommunication Union and is referred as International MobileTelecommunications-Advanced, or IMT-Advanced. IMT-Advanced require a nominal datarate of 100 Mbit/s during high speed vehicular mobility (120 to 350 km/h) and 1 Gbit/s sta-tionary. IMT-Advanced systems shall also be able to achieve a user plane latency of less than10 ms in unloaded conditions [25].

The development of a new global standard was divided into two work items: LTE target-ing the radio access network and System Architecture Evolution (SAE), targeting the packet

14


core network. The output from the work items is embodied in the Evolved Universal Ter-restrial Radio Access Network (E-UTRAN) and the Evolved Packet Core (EPC) standards,together referred as the Evolved Packet System (EPS). In contrast to earlier standards, theLTE standard was designed only to be built with a packet switched domain, thus removingthe circuit switched domain. This reduced the number of network elements involved in LTE.

Architecture

The LTE access network, also referred as the E-UTRAN, is the access part to the SAE core net-work and comprises a network of base stations, named Evolved Node B (eNodeB), illustratedin Figure 3.8. E-UTRAN operates without a centralized intelligent controller and distributesits intelligence amongst the base stations. This enhances the connection set-up and reducethe time when performing a handover.

UE

LTE-Uu Serving Gateway(S-GW)

PDN Gateway(P-GW) Internet

S5S1-U

Mobility Management

Entity(MME)

S1-MME

Radio Bearer S1 Bearer S5/S8 Bearer

External Bearer

EPS Bearer

end-to-end service

E-UTRAN (LTE) SAE

Figure 3.8 illustrates an incomplete overview of the Evolved Packet System (EPS) includingbearers.

The Mobility Management Entity (MME) is the control-node of E-UTRAN and handles thebearer activation/deactivation process, the paging of idle mode UEs and choosing a SGW fora UE. The Serving Gateway (S-GW) routes user data packets and acts as an anchor duringboth inter-eNodeB handovers and vertical handovers between LTE and other 3GPP tech-nologies. The Packet Data Network (PDN) Gateway (PDW) connects SAE to external accessnetworks and is the point of exit and entry of traffic for the UE. The PDW also is responsiblefor IP address allocation for the UE, as well as QoS enforcement by filtering downlink packetsinto different QoS-bearers.

To successfully transmit data between the PDW, through the eNodeB to the UE, IP packetsare encapsulated and tunneled with different tunneling protocols across different interfaces.Between the SAE interfaces (S1 and S5/S8) the 3GPP-specific tunneling protocol called theGPRS Tunneling Protocol (GTP) is used, for further information see [1].

The E-UTRAN user plane protocol stack in shown in gray in Figure 3.9, with the sublayersPacket Data Convergence Protocol (PDCP), Radio Link Control (RLC) and Medium AccessControl (MAC). They are terminated in the eNodeB on the network side. For further detailsregarding the protocols can be found in Chapter 4 of [52].

15


GTP-U

UDP/IP

L2

L1

GTP-U

Application

IP

RLC

MAC

L1

UE

RLC

MAC

L1

LTE-Uu

PDCP PDCP

Relay

UDP/IP

L2

L1

GTP-U

UDP/IP

L2

L1

Relay

GTP-U

UDP/IP

L2

L1

IP

S1-U S5/S8eNodeB S-GW P-GW

Figure 3.9 illustrates the E-UTRAN user plane protocol stack.

Technology

The key element of LTE is the technology used as signal bearer, called Orthogonal Frequency-Division Multiplex (OFDM). In LTE OFDM is used with two adapted access schemes depend-ing on downlink operations (eNodeB to UE) or uplink operations (UE to eNodeB):

a) Orthogonal Frequency-Division Multiple Access (OFDMA) in downlink operations.

b) Single-Carrier Frequency-Division Multiple Access (SC-FDMA) in uplink operations.

OFDM is a commonly used modulation format found in other wireless standards, e.g.Wi-Fi. An OFDM signal of LTE comprises a maximum of 2048 different close-spaced sub-carriers, allocating 15 kHz each. These sub-carriers are orthogonal to each other to create atransmission scheme with the ability to support parallel channel operations. Thus transmit-ted data is spread across the multiple carriers of a OFDM signal with reduced data rate percarrier.

Figure 3.10 illustrates the difference between the allocation schemes OFDM, OFDMA andSC-FDMA. Each row refers to a sub-carrier and the colors refers to data transmitted by aspecific user per time unit. In OFDM all frequency resources (sub-carriers) are allocated toprovide data for a single user per time unit. OFDMA enables dynamic allocation in bothtime- and frequency domain, thus multiple users can allocate bandwidth per time unit.

TIME

FRE

QU

EN

CY

(sub

-car

rier

s)

(a) OFDM

TIME

FREQ

UEN

CY(sub-carriers)

(b) OFDMA (downlink)

TIME

FREQ

UEN

CY(sub-carriers)

(c) SC-FDMA (uplink)

Figure 3.10 illustrates three different allocation schemes. OFDMA is used in LTE downlinkand SC-FDMA is used in LTE uplink.

This increases the efficiency and interference resilience. SC-FDMA is beneficial in LTE up-link operations as it lowers peak-to-average power ratio and thereby benefits the UEs trans-

16


mit power efficiency compared to OFDM and OFDMA. The decision to use OFDM modula-tion scheme with the support of both Frequency-Division Duplex (FDD) and Time-DivisionDuplex (TDD) yields the flexibility of the LTE standard. In FDD the downlink and uplink areseparated by frequency and in TDD they are separated in time.

The generic radio frame of LTE has a time duration of 10ms, divided into 10 sub-frames.One sub-frame consist of two slots of length 0.5ms, illustrated in Figure 3.11.

Each slot consists of seven or six OFDM symbols with a guard bands between. A guardband, marked with light gray in Figure 3.11, is a space included between symbols withpurpose of preventing interference and multipath propagation. Depending on the numberof symbols used per slot, they are fitted with either short/normal Cyclic Prefixes (CPs) orlong/extended CPs. The CP is a portion of the end of sample of the time domain block,copied to the beginning of the symbol. CPs along with guard bands are the overhead in anOFDM system, as they does not carry any useful information.

#0 #1 #2 #18 #19

One Radio Frame - 10ms

Slot - 0.5ms

#0 #1 #2 #3 #4 #5 #6

OFDM Symbols in a Slot

Cyclic Prefix

OFDM Symbol

Sub-Frame - 1ms

Figure 3.11 illustrates the generic radio frame of LTE.

Twelve OFDM slots are then grouped into a so-called resource block with total size of12 ˚ 15 kHz = 180 kHz in frequency domain and 0.5ms in time domain. A user is allocateda number of resource blocks. Either in time-frequency domain during downlink operationswith OFDMA or in frequency domain during uplink operations with SC-FDMA, shown inFigure 3.10 b) and c). The more resource blocks a user allocate and the higher the modulationformat used, the higher the bit rate.

Figure 3.12 show a block diagram of the physical layer in LTE. When viewing the blockdiagram it is notable that SC-FDMA can be considered as an OFDM system with a DiscreteFourier Transform (DFT) mapper by the extra DFT component viewed in gray.

A OFDM transmitter includes a baseband modulator, sub-carrier mapping, Inverse FastFourier Transform (IFFT), parallel to serial conversion, Cyclic Prefix (CP) addition, PulseShaping (PS), Digital to Analog Conversion (DAC) followed by a Radio Frequency (RF) mod-ulator.

The baseband modulator transforms the data into different modulation formats. Here theadaptive modulation balances the robustness of the format against the SNR. LTE providesseveral modulation formats: Quadrature Phase-Shift Keying (QPSK) and 16 or 64 Quadra-ture Amplitude Modulation (16QAM/64QAM). The modulated symbols are then mapped tosub-carriers. It is possible to use different modulation formats over multiple sub-carriers asthe interference level on the carriers can vary with time. The Inverse Fast Fourier Transform(IFFT) transforms the modulated sub-carriers in frequency domain to time domain samples.Cyclic prefixes are then applied to the samples before transforming them from digital to ana-

17


log signals. OFDM receiver then reverts the process according to the scenario in the lowerrow of Figure 3.12 to decode the binary stream.

Mod Symbol Mapping

Bit Stream

S-t

o-P

DFTSub-carrierMapping IFFT

P-t

o-S Add

CP/PS DAC/RF

1, 2, 4 or 8 layers

- S-to-P: Serial to Parallel- P-to-S: Parallel to Serial- CP: Cyclic Prefix- PS: Pulse Shaping

1, 2, 4 or 8 antenna

ports

RF/ADCRemove

CP

S-t

o-P

FFTSub-carrierDemapping/Equalization

IDFTP

-to-

SDetection & Equalizing

Bit Stream

Channel

- DFT: Discrete Fourier Transform- IDFT: Inverse Discrete Fourier Transform- FFT: Fast Fourier Transform- IFFT: Inverse Fast Fourier Transform

- DAC: Digital to Analog Conversion- RF: Radio Frequency SIGNAL- ADC: Analog to Digital Conversion

Both OFDMA and SC-FDMA

Only SC-FDMA

Figure 3.12 illustrates a block diagram of the physical layer of LTE when transmitting a bitstream.

The difference between SC-FDMA and OFDM is the transformation to frequency domainby a Discrete Fourier Transform (DFT) before going through the standard OFDM modulation.Shown as dark gray in Figure 3.12.

LTE is able to deploy and allocate spectrum from 1.4 MHz up to 20 MHz,t1.4, 3, 5, 10, 15, 20uMHz. Combined with spatial multiplexing of four layers (4ˆ 4 MIMO),LTE release 8 frozen in December 2008, supported peak data rates up to 300 Mbit/s in thedownlink and 75 Mbit/s in the uplink using single-antenna (1 ˆ 1 MIMO). Since LTE re-lease 10, frozen in April 2011, also referred as LTE-Advanced, the standard fulfills the IMT-Advanced requirements of 4G with 8ˆ 8 MIMO [10]. These values defines the theoreticallimit an compatible UE can obtain in lab environment with ideal radio signal quality and noerror rate coding, which is not applicable in real-life scenarios. With 5/6 error rate coding thepeak rate decreases to „200 Mbit/s.

Quality of Service (QoS)

To cope with the demands from subscribers and to attain better end-user experience, EPSimplements a sophisticated class-based QoS and defines priorities for certain customers andservices during times of high congestion in the network. In the EPS architecture, QoS is im-plemented using a set of bearers between a UE and the PDN gateway, illustrated in Figure 3.8.A bearer is basically a virtual concept that defines how data should be treated when it travelsacross the network, e.g. “UE A will always achieve at least 512 Kbit/s download speed whileUE B is not guaranteed any bit rate and might face extremely bad download speed at timeswith high congestion". LTE QoS can be divided into two types of EPS bearers: Dedicated andDefault, creating a topology architecture illustrated in Figure 3.13.

A LTE QoS Class Indicator (QCI) defines the IP level packet characteristics, the table defin-ing the classifiers is reproduced below from Table 6.1.7 of 3GPP TS 23.203 (v11.3.0).

Default bearer When LTE UE attaches to the network, it is assigned a default bearer whichmaps the UE to IP address and remains attached as long as the UE is connected. Defaultbearers provide best effort service and do not guarantee any bit rate. Therefore QoS classidentifiers (QCIs) 5-9 (non-GBR) can be assigned to a default bearer, see Table B.1 (page 61).The other parameters (APN-/UE-AMBR etc.) specify the upper rate of a subscriber, for fur-ther information see A.1.1.4 in 3GPP TS 23.203 V11.6.0 [3].

18


Dedicated bearer The dedicated bearer can be divided into a non-guar-anteed bit rate (non-GBR) or a guaranteed bit rate (GBR) service. A dedicated bearer is applied on top of a defaultbearer to provide an dedicated tunnel to one or more services. For high priority services, likeVoLTE, dedicated bearers are essential. To achieve the mapping between the services and thededicated bearer it uses Traffic Flow Templates (TFTs) compared to the default bearer. A TFTgives special treatment to a specific service. Dedicated GBR bearer map to QCI 1-4, where asnon-GBR map to QCI 5-9.

Figure 3.13 illustrates the LTE Quality of Service (QoS) topology.

Transmission characteristics

There are several main factors impacting the end-user experience, especially in short-bursttraffic. First, the duration time it takes for a UE to access the radio and the core networkresources. Here the benefit of LTE’s flat IP architecture becomes obvious. The number ofconnection states is reduced to only two states compared to previous four states in HSPA.This is possible as the number of network elements involved in the access network path isfewer. Thus the control plane latency requirement of IMT-Advanced is to be lower than50ms to establish an connection with the access network and post-establishment less than5ms latency. This characteristic is comparable with Wi-Fi technologies such as IEEE 802.11n-2009 and 802.11ac.

Bufferbloat is a phenomena that lead to large differences in RTT due to existence use ofbuffers in networks. The bufferbloat phenomena is especially common in radio access net-works where large buffers are used to compensate the bandwidth bottleneck of the air in-terface that occur when the SNR increases. During an transmission, the RTT can multiplyhundreds of times as the buffers grow thus may lead to flow starvation. Flow starvationoccurs when a flow is restricted from continuing the transmission because it is denied nec-essary resources to process its work. This may occur if a flow is waiting for the last packetsent to be acknowledged, either sent on its own flow or another flow. Thus the flow is starv-ing and the throughput is degraded. The occurrence of bufferbloat is less prominent in mostwired/Wi-Fi networks. Chen et al. [8] present a study regarding the impact of the bufferbloatphenomenon on MPTCP in wireless networks. Their study revealed that MPTCP in somecases suffered to severe performance degradation due to the occurrence of the bufferbloatphenomenon when combining resource from both cellular and Wi-Fi networks. To furtherunderstand how MPTCP behaves when exposed to the bufferbloat phenomena, one of ourexperiments include a scenario that exposes MPTCP to the phenomena.

19

3.4. Transmission Control Protocol (TCP)

3.4 Transmission Control Protocol (TCP)

Since MPTCP is an extension to TCP this section details the TCP header stucture, the flow-and congestion control, and the impact on the TCP throughput of packet error probability.This section of followed by the details regarding the theory of MPTCP.

Illustrated in Figure 3.3, there are two prominent transport protocols used ubiquitously:the connection-oriented protocol TCP [43] and the connection-less protocol UDP [44]. Aconnection-oriented communication session is established before any data can be transmit-ted and the stream of data sent by the protocol will be delivered ordered and error-checked atthe destination port if the transmission succeeds. But providing such functionality on top ofpotentially unreliable datagram services in lower-level protocols comes with a price in termsof increased overhead and latency compared to using UDP.

The TCP protocol data unit is referred as a TCP segment. Figure 3.14 illustrates the struc-ture of a TCP segment, divided into two blocks:

a) A TCP header block that directs the segment to its destination.

b) A TCP data block carrying the application layer payload.

To successfully provide its functionalities, the TCP header is structured with ten manda-tory fields, resulting in a minimum header length of 20 octets or 160 bits (the optional fieldexcluded). Given these fields, TCP achieves the ability to exchange information bidirectionalbetween the hosts.

32

64

96

128

160

Offset in bits

Source port

Sequence number

Destination port

Acknowledgment number (if ACK set)

Data offset Reserved 0 0 0 Window Size

Checksum Urgent pointer (if URG set)

Flags

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 11 2 3

Data

TCP Optional Field (if Data offset > 5)

MSS

Figure 3.14 illustrates the TCP header structure in solid with accompanying data part dotted.

When the transport protocol receives a stream of information to be transmitted from theupper-layer application, it is fragmented into payload blocks, with the size of the MaximumSegment Size (MSS) in octets. The Maximum Transmission Unit (MTU) of the outgoing inter-connecting service for that particular link derives the MSS. In LTE the MTU is 1428 octetsand 1500 in Ethernet. Depending on the interconnecting protocol, IP version 4 (IPv4) [42]or IP version 6 (IPv6) [12], the IP header is 20 resp. 40 octets. Therefore the MSS of LTE is1428´ 20´ 20 = 1388 resp. 1428´ 40´ 20 = 1368 octets leading to an overhead of „ 2.8%resp. „ 4.2%. In comparison to Ethernet the MSS is 1460 resp. 1440 octets.

Connection initiation

In order to exchange the starting sequence number two entities plan to use, the procedure toestablish a TCP connection is a handshake process involving an exchange of three messages,referred as a three-way handshake. The three-way handshake was first introduced 1978 by

20


Carl Sunshine et al. [60]. This sequence is displayed in Figure 3.15. The sequence of mes-sages are identified by utilizing the synchronize flag (SYN) and the acknowledgment flag(ACK). Initially the initiator sends a TCP segment to listener with the SYN flag set carryingthe initial sequence number. The listener then responds to the SYN operation using a TCPsegment with the SYN and the ACK flags set. The response also carries the listener’s initialsequence number with the acknowledgment number set to the initiator’s sequence numberincremented. To finalize the three-way handshake the initiator responds to the SYN/ACKwith a TCP segment with the ACK flag set, carrying the listener’s sequence number incre-mented as acknowledgment number.

The reason for TCP not to use a default sequence number e.g. 0, when establishing aconnection is to protect against two incarnations of the same connection reusing the samesequence number too soon. That is if a segment from an earlier incarnation of a connectionmight interfere with a later incarnation of a current connection.

Initiator Listener

SYN (SEQNR X)

SYN+ACK (SEQNR Y, ACKNR X+1)

ACK (ACKNR Y+1)

Not

est

ablis

hed

Not established

Established

Est

ablis

hed

Figure 3.15 illustrates the three-way handshake exchange.

After the three-way handshake procedure is completed the connection is allowed to initi-ate transmitting information to the destination host.

Sliding window error control

To ensure end-to-end reliability and sequentiality on top of a unreliable service, TCP usesliding-window Automatic Repeat reQuest (ARQ) error-control techniques. Each octet ofdata is assigned a sequence number. The TCP sequence number field carries the sequencenumber of the first data octet. The next segment is assigned sequence number seqi =seqi´1 + ni´1 + 1 where n is number of data octets. The acknowledgement number indicatesthe expected next segment sequence number. Timeouts indicate when a segment is markedas lost and an action is necessary to ensure reliability. Before forwarding the information tothe application segments must be rearranged in-order.

Three common ARQ retransmission schemes are:

a) Stop-and-Wait ARQ sends and acknowledgments one packet a time before moving tothe next packet. Therefore it is the simplest and most limited.

b) Go-Back-N (GBN) ARQ sends cumulative acknowledgments indicating the last seg-ment received by the receiver and retransmits every unit from the last sequence numberupon retransmit.

c) Selective Repeat (SR) ARQ only retransmits assigned units and transmitted packets areindividually ACKed by the receiver.

21


Generally TCP use a mixture of GBN and SR. TCP acknowledgments are cumulative.Hence correctly received out-of-order segments are not individually acknowledged by thereceiver. The TCP sender only maintain the smallest sequence number of a transmitted butunacknowledged segment and the sequence number of the next segment to be sent. When asegment is lost the sender is indicated by ACKs carrying the same sequence number from thereceiver. Thus a duplicate ACK are triggered. The sender use the information it get from theacknowledgements to prematurely detect if a segment is lost before the retransmission time-out triggers. But TCP does not know whether a duplicate ACK is caused by a lost segment orby a segment arriving out-of-order when routed through the network. Therefore it waits for asmall number of duplicate ACKs to be received. If three or more duplicate ACKs are receivedin a row, it is a strong indication that a segment has been lost. TCP then performs a retrans-mission of what appears to be the missing segment, without waiting for a retransmissiontimer to expire. This is referred as a TCP Fast Retransmit [57].

Flow control

The upper rate limit of a transmission is the rate the receiver is able to process the receiveddata. All traffic sent above that rate is wasted network resources. To prevent such eventfrom occurring TCP uses a sliding window flow control protocol where the receiver notifiesthe transmitter how much additional data starting with the acknowledgment number, thereceiver is willing to receive in octets before the buffer overruns and starts dropping packets,see Window Size in Figure 3.14. This window is referred as the receiving window (rwnd).The mechanism is illustrated in Figure 3.16:

Figure 3.16 illustrates the TCP flow control.

The scenario shows a sender initially transmitting information with size 2 Kb to a receiverwith a 4 Kb buffer, whereupon the receiver respond with an acknowledgment and its avail-

22


able space in the buffer (rwnd). When the buffer is full, rwnd = 0 and the sender is blockeduntil a new acknowledgment is received notifying the sender available space in the window[43].

Congestion control

With the upper rate limit defined, the sending side aims to obtain maximum throughputwithout harming other on-going transfers and avoiding congestion collapse. Congestion col-lapse is a phenomena that occurs when senders’ calculated RTT is not updated fast enoughcompared to the actual RTT. If the actual RTT exceeds a sender’s RTO, the sender will be-gin to resend already sent segments. Eventually all available buffers inside the network willbe full and packets dropped. This phenomena was first documented in 1986 [36]. TCP usea host-driven, window-based congestion control mechanism that adjust the rate of the out-going packets using a congestion control window (cwnd). Initially the cwnd has send smallwindow size but growing fast to accelerate the rate until a stable rate is obtained. Thus avoidsother transfers from being damaged and TCP obeys the goal of resource fairness. But it is im-portant a network transmission also obtain good bandwidth allocation. Therefore is TCPbased on a control law referred as additive-increase multiplicative-decrease (AIMD) whichmakes hosts additively increase rate while the network is uncongested and multiplicativelydecrease when congestion occurs. This protects fairness amongst bandwidth allocations in anetwork. The size of cwnd is depends on which algorithms used in a transmission. In ourexperiments we use the TCP NewReno congestion control algorithm.

TCP NewReno

The TCP NewReno congestion control can be divided into four congestion control algorithms:slow-start, congestion avoidance, fast retransmit and fast recovery [57]. TCP Fast Retransmitis detailed in Section 3.4.

Slow-start defines the initial part of a TCP transmission; after the three-way handshake ora retransmission timeout. The idea with slow-start is for cwnd to reach near the maximumrate in a fair amount of time as an additive increase would make the acceleration too slow.

TCP NewReno uses the simple slow-start algorithm first defined by Jacobsson, 1988 [27]and the TCP Tahoe algorithm; for each acknowledgment achieved, increase cwnd with one.This means cwnd will increase exponentially per round-trip time, visible in equation 3.2:

cwndi+1 = cwndi ˚ 2 per RTT. (3.2)

When a packet loss event occurs (at least one packet loss during a RTT), the slow-startphase ends and the slow-start threshold size (ssthresh) is set to the cwnd divided by two:

ssthresh =cwnd

2. (3.3)

Congestion avoidance is followed by slow-start and starts when cwnd ą ssthresh. Dur-ing the congestion avoidance cwnd increase less aggressive compared to slow-start, seen inequation 3.4, which roughly corresponds to one packet per RTT:

cwndi+1 = cwndi +1

cwndiper ACK. (3.4)

When a packet loss event occur the cwnd is halved and set the ssthresh and continueswith congestion avoidance.

23


The flow control and the congestion control are the crucial mechanisms determining the ob-tained data rate for the end-user. Therefore the throughput T correspond to the sender win-dow size with the equation:

T »W

RTTpackets/s =

W ˚MSSRTT

bytes/s, (3.5)

where W = min(cwnd, rwnd).

Fast Recovery collaborates with the Fast Retransmit algorithm, see Section 3.4, in an algo-rithm called Fast Retransmit/Fast Recovery Algorithm [57]. Because the receiver only cangenerate duplicate ACKs when another segment is received, it means there is still data flow-ing in the network. To not reduce the flow abruptly by going into slow start, the fast recoveryalgorithm perform congestion avoidance, but not slow start.

Packet error probability impact on throughput

Packet error probability has a significant impact on the throughput of TCP, especially duringthe congestion avoidance algorithm. This can be illustrated by the Mathis model [34], visiblein equation 3.6;

T =MSSRTT

b

32

?p. (3.6)

Here, T denotes the maximum throughput, depending on the maximum segment size, MSS,the round-trip time RTT and the packet error probability p. The Mathis model predicts thebandwidth of a sustained TCP connection subjected to light to moderate packet error prob-ability. The model assumes connections avoids retransmission timeouts and are not limitedby the receiver window as well as have a continuous send flow.

Figure 3.17 plot the predicted throughput T using Mathis model with the packet er-ror probability range 0.01%´ 0.1%, the maximum bandwidth allocation refers to the maxi-mum available bandwidth in the experimental test-cases per technology with accompanyinground-trip times.

Figure 3.17 shows the impact of packet error probability p on the maximum data rate of aTCP transmission with packet RTT = 10ms & 60ms and 1460 octets MSS.

24


Viewing the figure it is notable an TCP transmission with packet error probability 0.01%and round-trip time 60ms is limited to „23 Mbit/s independent on available bandwidth.Since our experimental setup use Ethernet and IPv4 the MSS is 1460 octets.

Connection termination

To implement a controlled termination process TCP use a four-way handshake using thefinished (FIN) flag and the acknowledged (ACK) flag, displayed in Figure 3.18. Both sidescan at any time independently terminate the connection with the other side. This meansone side may terminate its connection to the opposite side making the connection simplex.That side can only receive information and will be open until the other side terminates theconnection.

Initiator Listener

FIN

ACK

Est

ablis

hed E

stablishedC

losedC

lose

d

FIN

ACK

Figure 3.18 illustrates the four-way handshake process used when terminating a TCP connec-tion using the finished (FIN) and acknowledged (ACK) flags.

25

3.5. Multipath TCP (MPTCP)

3.5 Multipath TCP (MPTCP)

In this section we detail the multi-path extension to TCP, referred as Multipath TCP (MPTCP)[18]. Initially the overall design of MPTCP followed by the technical details, including afunctional decomposition of the protocol.

MPTCP is a set of features applied on top of TCP to extend the single path protocol to providesimultaneous use of multiple paths. This has been a goal desired for decades, but it hasproven to be hard to implement due to the vast difference of legacy devices deployed in theInternet. The idea of pooling resources’ from multiple paths contribute to two main objectivesto the end-user perspective:

a) Improved throughput thanks to bandwidth aggregation in the occurrence of multiplepaths to destination.

b) Increased robustness by redundancy against network failure and redirecting traffic inthe event of link failure.

The first objective defines a fundamental goal of multipath protocols and this thesis hypoth-esis basis. Below this goal is quoted for MPTCP by Ford et al. in RFC 6182, “ArchitecturalGuidelines for Multipath TCP Development":

“To meet the minimum performance incentives for deployment, a MultipathTCP connection over multiple paths should achieve no worse throughput than asingle TCP connection over the best constituent path." [17]

It is not unjustified to describe MPTCP as the next generation transport layer protocol.Compared to its fellow competitors, MPTCP aims to achieve widespread deployment by be-ing applicable in legacy applications without any changes. This allows MPTCP to supersedeTCP transparently for both higher and lower layers relying on standard TCP. Given an ex-tended API additional control is provided for applications interested in MPTCP’s features,but for non-MPTCP-aware applications, the protocol behaves as normal TCP.

Protocol stack architecture

MPTCP’s architectural design follows the Transport-Next-Generation (TNG) decompositionmodel mentioned in Section 3.1 (page 12). By designing the architecture according to the TNGmodel and reusing the TCP protocol as the transport protocol in the Flow- and Endpoint lay-ers, MPTCP can traverse already configured and deployed middleboxes immediately as longthey do not alter the optional header field. This is the main factor to succeed a widespreaddeployment of MPTCP compared to SCTP and other multipath protocols. The additionalMPTCP-layer applied on top of the network-oriented layers results in a reintroduction of theimportant end-to-end principle to the transport layer. The relationship between the regu-lar TCP and MPTCP’s two layered architecture is illustrated in Figure 3.19. The dotted linedescribes the tightly coupled relation between the extension and regular TCP.

Figure 3.19 illustrates the relationship between standard TCP (left) and MPTCP (right)

26


For every network address, the semantic layer initiate a ordinary TCP session to the des-tination, referred as a MPTCP subflow. With a number of subflows greater than one, band-width aggregation is achievable thus should result in an improved goodput to the applica-tion. This refers to goal a), mentioned in the beginning of this section. After connectionestablishment the subflows are prioritized in descending order according to the lowest RTTduring the initiation.

The MPTCP semantic layer (associated with the transport layer in TNG) handles connection-oriented functionality and arrangement of MPTCP data segments. Every MPTCP connectionhas a connection-level sequence number (DATA SEQ), mapping subflow-segments (regularTCP segments) to MPTCP-segments. This makes it possible for connection-level segmenta-tion and reassembly at destination, further mentioned in Section 3.5.

A summary of the semantic layer’s functionality follows:

a) Application-oriented functionality to handle operations from higher layers throughMPTCP’s socket API and preserving backward compatibility to regular TCP.

b) Path management identification and initiation to determine and initialize usable sub-flows between two hosts in a MPTCP connection.

c) Packet scheduling of the byte stream received from the application into MPTCP seg-ments and scheduled on available subflows. To achieve sufficient throughput dependsthe packet-scheduler heavily upon information about the availability of the subflows.

d) Congestion control coordinating the subflows to ensure fairness against other transmis-sions.

e) A subflow interface to provide the communication link between each subflow and thepacket-scheduler, providing information to the scheduler and managing subflow trans-missions.

The MPTCP subflow layer (associated with the network-oriented layers in TNG (Flow- and End-point)) handles the transport of assigned segments to the destination using standard TCP. Themain difference between regular TCP and MPTCP’s subflow layer is the coupled congestioncontrol, explained in Section 3.5, and the new sequencing structure in the optional headerfield.

These functions fit together as followed:The path managing functionality explores after initializing the connection to the first addressfor additional paths to destination host. The packet scheduling then receives messages des-tined for the network from the application through the API. Segments it to connection-levelsegments and applies a connection-level sequence number before sending them to a subflowthrough the subflow interface. The subflow then applies its own sequence number, ACKs,and passes it to the network. The receiver subflow reorders the data into correct order beforepassing it to the receiver’s packet scheduling function. The packet scheduling then performsconnection-level reordering before passing the data to the application.

Sequence numbering

MPTCP use two levels of sequence numbers, illustrated in Figure 3.20: First an absoluteconnection-level data sequence number (DATA SEQ) and second a relative subflow sequence

27


numbering, (SUBFLOW SEQ). DATA SEQ enables segmentation and reassembly at destina-tion with the ability to retransmit data segments on different subflows or send same datasimultaneously on multiple subflows for resilience or efficiency purposes.

After the DATA SEQ is applied and the segment sent to a subflow, the SUBFLOW SEQ isapplied which enables reordering and retransmissions at the subflow-level. MPTCP choosesnot to use the TCP header sequence number as firewalls undertaking Initial Sequence Num-ber (ISN) Randomization change the sequence number, thus harming the data sequence map-ping of the MPTCP subflow.

Figure 3.20 shows the MPTCP data sequence numbering.

For MPTCP to fully satisfy the end-to-end argument, it is crucial to ensure deliveryof transmitted content at the end-host. Under ideal conditions would MPTCP be able torely on subflow acknowledgments, sent by the receiver to notify delivery to determine if aconnection-level segment has been successfully transmitted thus fulfill the end-to-end prin-ciple. But due to the middlebox-problem, discussed in Section 3.1 (page 10), MPTCP cannotensure delivery by TCP acknowledgments. To solve this MPTCP use both connection-leveland subflow-level acknowledgments to ensure robustness and reliability. The transmissionsof TCP ACKs for a subflow are entirely handled at the subflow-level to support TCP seman-tics and generate retransmissions. The connection acknowledgment indicates when to moveforward the connection-level sequence space.

Buffers

As mentioned in Section 3.2 the protocol standard of MPTCP referenced in RFC 6824 [18] isclassified as a non-adaptive protocol without any intelligence projecting changes in the char-acteristics. To handle the unavoidable occurrence of segments arriving out-of-order duringpath heterogeneity, MPTCP use a connection level receive buffer shared amongst the sub-flows.

The buffer stores segments until they are in-order and requested by the application. Withmore significant differences in throughput larger buffers are required. Otherwise the bufferdensity in the buffer increase making the receive window to shrink. This may lead to a flowstarvation where one subflow has to wait for another subflow. Thus the throughput of theMPTCP connection may decrease or even halt. This scenario is illustrated in Figure 3.21.

In the scenario illustrated in Figure 3.21 the server initially sends segments 1-4 using RAT1with low RTT towards the UE, in the same time RAT2 pulls data 5-7 and sends it towards theUE. After one RTT RAT1 has available space in its congestion window and pulls data 8-11.The problem now arises as the UE will be delivered data 1-4 as it is in-order, but has to wait 10more milliseconds for data 5-7 to arrive even though data 8-11 already is present. Therefore,high reorder buffer density make the receive window to become smaller which degradesthroughput.

28


Figure 3.21 illustrates a blocking scenario during path heterogeneity leading to throughputdegradation.

In standard TCP, the optimal receive buffer size is twice the Bandwidth Delay Product(BDP), where the BDT = BW ˚RTT, (BW is the bandwidth and the RTT is the Round-TripTime). The first BDP allows support for reordering of segments where the second BDP allowsthe connection to continue during TCP fast retransmit.

The worst-case scenario in multipath communication occurs when a subflow with highestRTT or longest retransmission timeout (RTO) experiences a timeout. Then MPTCP has tobuffer data from all subflows for the duration of the RTO. To cope with worst-case scenarioMPTCP’s receive and send buffers need to be at least the size (S):

Smin = 2 ˚ÿ

i

(BWi) ˚max(RTT) bits, (3.7)

where max(RTT) is the largest RTT across the subflows. However no more than 12.5 Mb.As some middleboxes alter the TCP-level receive window it is important MPTCP chooses thelargest advertised window size.

Aggressive retransmit The main reason free space may occur after pulling data from boththe retransmission queue and the connection-level send buffer is the connection is limitedby the receiving side MPTCP buffer. This means no new data can be sent. With aggressiveretransmit activated the spare network resources is used to resend segment sent on a othersubflow avoid building up the receive buffer. The scenario illustrated in Figure 3.21, wouldresult in RAT1 resending data 5-7 which would arrive 25ms before the arrival of the segmentsby RAT2.

29


Congestion control

The congestion control of MPTCP determines how the protocol would behave in the momentof congestion and will be crucial for the throughput.

To achieve fairness amongst TCP and MPTCP transmissions, Raiciu, et al. [46] in RFC6356 suggests a coupled congestion control algorithm for multipath transport protocols. Acoupled algorithm is necessary if a MPTCP transmission with two subflows should allocatethe same amount of resources as a single TCP connection in a bottleneck. Each MPTCP sub-flow maintains its own congestion window and retransmission scheme during data transfers,beginning with slow-start and followed by congestion avoidance, see Section 3.4 (page 23).The difference is in the increase phase in AIMD, where the congestion windows are coupledaccording to the algorithm below:

cwndi+1 = cwndi + min(α

ř

cwnd,

1cwndi

) per ACK on subflow i, (3.8)

where α =max( cwndi

RTT2i)

(ř

icwndiRTTi

)2˚

ÿ

i

cwndi. (3.9)

As α is updated once per RTT or when a packet loss event occurs, the coupled conges-tion algorithm lags in responsiveness. In combination with Nagle’s Algorithm and delayedACKs the RTT is falsely estimated in timely manner. Nagle’s algorithm works by combininga number of small outgoing messages, and sending them all at once [36]. This is to reduceoverhead. Delayed acknowledgements tries to improve effiency by not sending acknowl-edgements immediately. Instead the receiver waits for data to be sent on which it the ACKcan be piggybacked. On top of that the RTT values in α are smoothened values (SRTT) withthe consequence SRTT will always lag behind the true RTT during drastic changes, henceimpacts the data rate of MPTCP.

Congestion control implementation in experimental setup

To clarify if MPTCP may fulfill the goal of throughput, the scheduling mechanism in theprototype stack is focused on performance rather than fairness. Therefore we do not use thesuggested congestion control in RFC 6824. Instead the congestion control implemented in theexperimental setup use an uncoupled TCP Reno congestion control, mentioned in Section 3.4(page 23). This means the subflows use TCP Reno and pull data from the MPTCP send bufferwhenever they have available space in their congestion window. This result in a self-balancedMPTCP congestion control with the downside that it is not fair against regular TCP.

Technical details

Given the background and architecture of MPTCP, the following section discuss the technicaldetails of MPTCP. Including a functional decomposition and walk though of each step of aMPTCP connection.

MPTCP operates by transmitting predefined structural instructions, encapsulated within theTCP header optional field, see Figure 3.14. To identify an option field as a MPTCP-operationthe initial part of all instructions is watermarked with a composition according to the formatillustrated in Figure 3.22.

The Internet Assigned Numbers Authority (IANA) has assigned an unique numericaltype, termed “Kind", to MPTCP in their “TCP Option Kind Numbers" registry [61] which alsoinclude other TCP extensions. The 8-bit field “Kind", is identical for every MPTCP-operation,and when set to 0x1E associates a message in a TCP header option field with an MPTCP op-tion format. Second in every MPTCP option format is the 8-bit field “Length", which specifies

30


the length of the optional header field in octets and is used for verification. It is followed bya field, termed “Subtype", identifying individual MPTCP operations of variable lengths by anew sub-registry entitled “MPTCP Option Subtypes" [35], detailed in Table B.3 (page 62). Bythis approach, further TCP extensions can be introduced and permits MPTCP from interfer-ing with other extensions. Depending on requested operations these subtypes have differentstructures, defined in RFC 6824 [18].

Figure 3.22 shows the mandatory MPTCP option field format.

Since the transmitted data is carried by subflows using TCP are the procedure to establisha MPTCP connection according to TCP’s three-way handshake (SYN, SYN/ACK, ACK). Eachpacket involved in the initial handshake contains the MultiPath capable TCP option, referredas MP_CAPABLE, to notify corresponding destination MPTCP is the preferred communica-tion protocol. Additional subflows are added to the connection using a MultiPath join TCPoption exchange, referred as MP_JOIN. This sequence is illustrated in Figure 3.23.

Figure 3.23 illustrates the connection sequence of a MPTCP capable operation followed by aMPTCP join connection operation, where

HMACA = HMAC(Key = KeyA + KeyB, msg = NonceA + NonceB),

HMACB = HMAC(Key = KeyB + KeyA, msg = NonceB + NonceA), and

+ describes a concatenation functionality.

31


Each host is identified by an unique 64-bit identification key that requires to be fully randomkey recommended by the IETF randomness security standard [4]. The opponents MPTCP keyis used to generate a token that identify which MPTCP connection a joining subflow wants tojoin. For authentication in the process of establishment joining subflow exchange Hash-basedMessage Authentication Codes (HMACs) generated using the exchanged keys and randomnumbers (nonces). The cryptographic algorithm used is negotiated in the MP_CAPABLEoperation. The prototype implementation used in the experimental setup used HMAC-SHA1[56].

If the end-host is MPTCP incompatible, source host is notified by a empty SYN/ACKresponse and then resuming the connection using regular TCP instead of MPTCP.

Additional subflows

Each MPTCP endpoint detects and maintains a list of available IP addresses associated withall interfaces of a terminal. Thus additional paths may exist increasing the available band-width between hosts. MPTCP allows a host to announce additional addresses (and option-ally ports) on which a host can be reached to the opponent host. This supports mobility andadaptivity for the connection and done by adding the MPTCP ADD_ADDR TCP option for-mat to a packet [35]. Thereby multiple addresses can be announced in a single packet, as longthere is available space. Due to the proliferation of NATs, it is reasonably likely that one hostmay attempt to advertise private addresses, which is undesirable. With the authenticationand security check in the MP_JOIN operation, the join to the announced address will fail andterminated. If an interface becomes invalid during the connection lifetime (e.g. if the interfacegoes down), the affected host announce the opponent peer so that it remove subflows relatedto this address (or address list) from its availability list and terminate any subflows currentlyusing that address.

Sequencing

To ensure reliable in-order delivery of a single data stream transmitted over one or more sub-flows, the destination must be able to map the information received from a subflow to itscorrect place in the data sequence space and than acknowledge the transfer. For both oper-ations (data sequence mapping and data acknowledgment) MPTCP use a signal referred asData Sequence Signal (DSS) TCP option format. Depending on flags set in the format eitheror both operations can be signaled in the option. The DSS carries information how the recip-ient application map data from a subflow sequence space maps to information in a MPTCPdata sequence space, and the data ACK acknowledges receipt of data at the connection level.This is expressed in terms of starting sequence numbers for the subflow and the data level,and a length of bytes for which this mapping is valid. To reduce overhead the DSS optiondoes not need to be included in every MPTCP packet, as long as the subflow sequence spacein that packet is covered by a mapping known at the receiver. So a sender must include a DSSoption with data sequence mapping in every segment until one of the sent segments has beenacknowledged with a DSS option containing a data acknowledgment. Once acknowledgedthe sender can exclude the DSS to minimize overhead.

To fulfill the end-to-end principle earlier discussed in Section 3.5 (page 27), the data ac-knowledgment operation proves that the data, and all required MPTCP signaling, has beenreceived and accepted by the remote end. The sender cannot free data from the MPTCP sendbuffer until data has been acknowledged by both a data acknowledgment received on anysubflow and at the subflow level by all subflows on which the data was sent.

For further information about MPTCP, see RFC 6182 and RFC 6824.

32

4 Methodology

This chapter details the methodology followed in this thesis and the experimental setup withaccompanying test software.

All tests were conducted on an isolated experimental setup, illustrated in Figure 4.1, basedon two Ubuntu 12.04 desktop computers interconnected with Ethernet through a 1 Gbit/sswitch. The technical specification of the setup is defined in Table B.11 on page 65. To pre-vent overrunning buffers and reducing the computational power in the setup, all NICs werelimited to 100 Mbit/s. The default configuration of Ubuntu’s TCP autotuning functionalitywas chosen to create an uniform underlying environment. This allowed full supervision ofMPTCP with minimal impact by the computational power of the devices. The only buffer wechose to modify for each test, was the MPTCP receive buffer of MPTCP to the recommendedbuffer size according to RFC 6824, see Section 3.5 (page 28).

To simulate changes in the downlink bandwidth shaping was performed at Host A, in thefuture referred as the User Equipment (UE). The server-side to the UE simulates a MPTCP-capable server, in the future referred as the server, generates downlink data flow directedfrom the server to the UE.

Host AUE

Host BServer

Ethernet Switch

A1

A2

B1

Data flow

Figure 4.1 shows a blueprint of the experimental setup with flow of data.

The test setup was constructed to simulate any type of device, for instance a smartphonewith two technologies (e.g. LTE and Wi-Fi) downloading content from a server. Any givenlink characteristics in terms of available bandwidth, latency, packet loss rate and jitter, couldbe triggered to see how MPTCP adapts to certain events and if the overall goodput could

33

4.1. UE experimental setup

be degraded in comparison to the best consistent path using regular TCP. As reference tothe MPTCP-test the same test was performed using single-path technology with regular TCP,separately using one interface at the time. Since the main reason is not to emulate real LTEor Wi-Fi, rather to understand the behaviour of MPTCP and how adaptive the protocol isto changing characteristics, a lowest RTT of 10ms was decided in dialogue with experts inthe area which still is applicable to real-life scenarios. A RTT of 10ms allowed the testbed toprovide detailed information without being effected by technical bottlenecks like write speedto disk, available RAM and CPU usage exceeding 40%.

The experimental setup was then extended with a set of Python- and Bash-scripts to cre-ate a controlled environment. With the completed setup each test could be configured andexecuted accordingly.

4.1 UE experimental setup

To emulate desired network characteristics, the default traffic control utility in Linux TrafficControl, referred as TC. TC was used in conjunction with the network emulation extension tothe software, named NetEm [37]. NetEm performed latency- and jitter-regulation with desiredpacket error probability where as the bandwidth shaping was performed by a token bucketfilter in TC. NetEm and the usage of TC is limited to egress direction, but by redirectingingress traffic to a pseudo-device referred as an intermediate functional block manipulatingincoming traffic in possible. Placing NetEm and the TBF on outgoing traffic from the IFB-device does this.

The accuracy of a emulation softwares depends heavily on the time source used by thesoftware. In terms of NetEm and TC both uses the kernel timer’s interrupt frequency, there-fore we modified the maximum available frequency in the Linux kernel to 1000 Hz in theexperimental setup. This is to get a more accurate result.

Architecture

The architecture stack of the UE experimental setup is illustrated in Figure 4.2. Incoming traf-fic from the switch arrives to a network interface controller then redirected to an intermediatefunctional block pseudo-device where desired characteristics is emulated using TC. TC usescheduling queuing disciplines, so called qdiscs to create desired characteristics.

First in the sequence we apply the network emulator NetEM qdisc to apply packet errorprobability p and packet RTT with a 2 ˚ BDP = 2 ˚max(BW) ˚max(RTT) first-in first-outqueue. Second is the token bucket filter qdisc with a same queue size, emulating desired bitrate.

The experimental setup had two monitors at different layers:

a) Capturing incoming traffic using dumpcap on both NICs [13].

b) Self-scripted socket monitor using Linux ss command to obtain information regardingeach TCP subflow. The socket monitor gathers information about the TCP window sizeand scale, buffer allocation, and content size.

For the two experimental types performed we equipped the experimental setup with twodifferent test softwares:

a) wget to download files of different sizes using HTTP GET from a web-server duringthe short-transfer experiment [64].

b) netperf to generate a bulk transfer from the server to the UE [38]. The executiondefinition of netperf is displayed in Listing 4.1.

34

4.2. Server experimental setup

Figure 4.2 shows the architecture of the simulated receiving side user equipment in the ex-perimental setup as a stack. On the top are the applications (wget and netperf) and in thebottom the network interface controllers. Traffic comes in through the NIC moves up thestack to destined application, then down the stack and out through the NIC.

Listing 4.1: Netperf executed with following parameters

netperf -t TCP_MAERTS -c -C -H 10.42.0.1 -p 5000 -l X

4.2 Server experimental setup

The MPTCP capable server was equipped with a TCP server at the application layer to en-able exchange of monitored information. Pre-testing operations from the test client at the UEare sent to enable the TCP monitor to be sent to the UE after the test is executed. The serverside was also equipped with different self-implemented TCP socket monitor to monitor con-gestion window size (cwnd), send buffer information and timing information (RTT, RTO andRTT variation).

Architecture

Illustrated in Figure 4.3 is the architecture stack of the MPTCP capable server. Dependingon test traffic is sent either from the netserver process (bulk experiments) or the apache web-server process (short-transfer experiments). The TCP layer uses Linux kernel 3.2’s defaultconfiguration and the MPTCP layer the default configuration of the prototype implementa-tion which is 3 MB of memory both at sending and receiving.

35

4.2. Server experimental setup

Network interface controller

MPTCP Capable server

MPTCPFIFO STATIC QUEUES (3 MB)

TCPFIFO DYNAMIC QUEUES according to Linux kernel version 3.2 default

TCP SOCKET MONITOR

Incoming traffic Outgoing traffic

Application layerTCP TEST SERVER netserver apache2

INGRESS EGRESS

Figure 4.3 shows the server side architecture in the experimental setup as a stack with theapplications at the top and network interface controller at the bottom.

Input example to the experimental application

Listing 4.2 below displays an input example for a test case to the UE experimental setup.

Listing 4.2: Input example

renorexmit 0

eth0 20mbit 0% 20ms 2mseth1 80mbit 0% 20ms 2ms10

for 2eth1 0mbit10

eth1 80mbit10

rof

Description:

1: Defines the congestion control algorithm to be used during the test is TCP Reno in thiscase, see Section 3.4 (page 23).

2: Defines if to use the experimental aggressive behaviour of MPTCP, discussed in Sec-tion 3.5.

3-4: Defines the behaviour of the two interfaces at the UE:

a) Mbit/s available downlink bandwidth at device.b) Drop loss rate in percentage.c) Round-trip time with accompanying jitter in milliseconds.

5: Defines how many seconds to run with current configuration.

6-11: Loop twice, first configure interface eth1, with accompanying address 10.42.0.11, to 0Mbit/s available bandwidth and run for 10 seconds. After 10 seconds reconfigure sameinterface with 80 Mbit/s allowed bandwidth.

36

4.3. Experiments

4.3 Experiments

The test set consist of five experiments, designed to subject MPTCP to bad scenarios but stillrealistic to be applicable on LTE. Initially we execute two test with non-alternating character-istics to see the improvement of MPTCP compared to regular TCP. Additionally, we performthree tests where each metric alternating with different frequencies.

To avoid confusion between transport and application layer throughput this thesis denotetransport layer throughput as throughput T and application layer throughput as goodput G.The goodput describes the amount useful information delivered by a network to a certainapplication per unit of time. Bottleneck bandwidth is denoted by BW, packet round-trip timeis denoted by RTT and packet error probability is denoted by p. To create a static realisticvariance we define the first-in first-out jitter as 20% of RTT.

Stationary experiment

The first category consist of two of the five experiments. The stationary experiments aims toexpose the improvement in goodput of MPTCP compared to TCP. The experiments will alsodistinguish how much difference between the links is possible to remain stable.The experiments are:

a) A HTTP GET experiment using wget to download files with following sizes:t32KB, 64KB, 128KB, 256KB, 512KB, 1MB, 2MB, 4MBu and measure the download timeper file.

b) A bulk-transfer experiment executed for t = 20 seconds using netperf.

The transmission characteristics during the experiments will be unchanged but modifiedbetween each experiment to cover many possible conditions. The experiments consist of18 (3x2x3) test cases iterated ten times. The extracted value used in the results were theaverage values of the tests to get a clear indication on how MPTCP behaves under differentcharacteristics.

The configuration of each metric set was defined as followed:

a) The bandwidth allocation of the two subflows was stated to be a fundamental factorand three different allocations were defined.

1. BW1 = 40 Mbit/s BW2 = 40 Mbit/s



The allocations were decided to be applicable bandwidths and not high enough to im-pact the test.

b) The decision of a standard RTT = 10ms was taken in all tests, to reduce the compu-tational power in the test environment. Two RTTs were chosen for the experiments,the first with path homogeneity (best case) and second with path heterogeneity (worstcase).

1. RTT1 = 10ms RTT2 = 10ms

2. RTT1 = 10ms RTT2 = 60ms

37

4.3. Experiments

c) Referring to Table B.1 (page 61), three packet error probabilities were decided.

1. p1 = 0% p2 = 0%

2. p1 = 0% p2 = 0.01%

3. p1 = 0.01% p2 = 0.1%

The last p test case is included to get an indicator of a worst case scenario of higherpacket error probability which for instance may occur with Wi-Fi.

Notable is with latency 60ms and packet error probability 0.01% and 0.1% the rate of TCPis limited by Mathis model with MSS 1460 octets to „23.8 resp. „7.5 Mbit/s. The cases withlower RTTs or loss rates can see larger throughputs. The result will be applicable to LTEalthough LTE MSS is 1388 octets and Ethernet 1460, see Section 3.4.

Given these 18 fullfactor combinations (3x2x3) we will get an indication about the improve-ment using MPTCP compared to TCP.

Dynamic experiments

Given the results from the first experiment, we further subject MPTCP to alternating char-acteristics, where we vary one metric at the time. The second experimental category aimsto further expose MPTCP to radio-alike conditions but under controlled circumstances so anreliable reference value of TCP can be provided. The category therefore is divided into threesubcategories related bandwidth, latency and packet error probability. To expose the proto-col to high frequency changes the Hz was decided to t0.2, 2u and each test were executed for20 seconds. The alternating metrics follows the pattern from the first category experiments,configured as followed:

a) The bandwidth-alternating experiment changes the available bandwidth according tothe ratios ∆BW with the frequency of Hz.

b) The latency-alternating experiment changes the latency according to ∆RTT with thefrequency of Hz to see if the variance impacts MPTCP.

c) The experiment alternating packet error probability changes according to ∆p.

To exclude eventual error factors, each test was repeated several times. The overall averagewas used in the analysis for the comparison between the TCP and MPTCP. The results fromthe experiments are presented in next chapter.

38

5 Results

This chapter details the results given by our experiments performed in the experimental testsetup. Throughout this chapter we will use the b.c.p. to refer to the best constituent path.This path is defined as the path with highest average throughput for a given characteristicsusing regular TCP.

5.1 Aggressive retransmit feature

To illustrate the difference between the experimental aggressive-retransmit feature enabledversus disabled, Figure 5.1 display an experiment configured with characteristics: RTT1 =RTT2 = 10ms, p1 = p2 = 0%, BW1 = 40 Mbit/s and BW2 = 40 Mbit/s Ñ 1 Mbit/s @ 0.2Hz(alternate between 40 Mbit/s and 1 Mbit/s every 5 sec.).

Figure 5.1 shows the throughput T in two separate 20 second TCP transmissions. TCP1 allo-cate 40 Mbit/s stationary. TCP2 alternate allocated bandwidth between 40 and 1 Mbit/s with0.2 Hz.

39

5.1. Aggressive retransmit feature

TCP1 has the highest averaged throughput, hence it is the b.c.p. transmitting at an aver-age rate of 38.5 Mbit/s. Thus TCP1 is our reference. TCP2 displays the slower, alternatingpath with 19.3 Mbit/s average included to show the transmission characteristics. Both areequipped with lines for the average throughput.

Without aggressive retransmit

Figure 5.2 implement the same characteristics as Figure 5.1 but instead use MPTCP and bothpaths simultaneously with AR inactivated. This means the characteristics of Subflow1 corre-sponds to the characteristics TCP1 and the same goes for Subflow2 to TCP2. What is notableis the flow starvation that occurs in Subflow1, demonstrated with arrows in Figure 5.2 com-pared to TCP1 in Figure 5.1.

The underutilization occurs when the bandwidth in Subflow2 rapidly changes, thus theround-trip-time becomes longer at that particular subflow. The receiver waits for segmentstransmitted over Subflow2 and the buffer density increases. Thus the available space inMPTCP’s receiver window decreases until Subflow1 has to wait for available space and flowstarvation occurs. This means Subflow2 impact the utilization of Subflow1 and limit the good-put of the MPTCP connection. This can be seen during the time intervals: 2.5 ă t ď 5,7.5 ă t ď 10, 12.5 ă t ď 15 and 17.5 ă t ď 20.

Figure 5.2 shows the throughput T for two MPTCP subflows; Subflow1 and Subflow2, andthe MPTCP goodput G when the aggressive retransmit feature is off. Subflow1 allocate 40Mbit/s stationary. Subflow2 alternate allocated bandwidth between 40 and 1 Mbit/s with 0.2Hz. This is the same characteristics scenario as Figure 5.1. The arrows mark the areas whereSubflow1 is affected by the changes in Subflow2 and flow starvation occurs.

In the interval after the impact of a change in bandwidth allocation 7.5 ă t ď 10, Gavg «

6.8 Mbit/s. This means a low utilizing percentage of only„ 16.5% of its 41 Mbit/s allocation.During the interval 10 ă t ď 12.5 both links allocate 40 Mbit/s each, Gavg « 75.7 Mbit/s,which gives an „ 94.6% utilization.

40

5.1. Aggressive retransmit feature

With aggressive retransmit

Figure 5.3 implements the same scenario as Figure 5.2 where MPTCP use both paths simulta-neously, but in this case the aggressive retransmit feature is activated. Notable is immediatelythe difference in utilization between Subflow1 in Figure 5.3 compared to Figure 5.2.

With the aggressive retransmit feature activated the goodput during the same impact-interval as before: 7.5 ă t ď 10 is Gavg « 38.1 Mbit/s of its 41 Mbit/s allocation. This resultin a „ 92.9% utilization, a significant improvement. In the interval 10 ă t ď 12.5 Gavg haslowered from 75.7 to 71.2 Mbit/s. The difference is the MPTCP retransmission percentagehas increased from 0.282% to 7.32%. For RTT1 = 10ms and RTT2 = 60ms, the percentage islower (3.12%). Hence AR may affect the maximum rate due to the amount of resources lostwhen the number of retransmissions increases. Especially with uniform round-trip times.

Figure 5.3 shows the throughput T for two MPTCP subflows; Subflow1 and Subflow2, andthe MPTCP goodput G when the aggressive retransmit feature is activated. Subflow1 allocate40 Mbit/s stationary. Subflow2 alternate allocated bandwidth between 40 and 1 Mbit/s with0.2 Hz. This is the same characteristics scenario as Figure 5.1.

Given the result from the experiment it is notable the aggressive retransmit feature iscrucial to avoid flow starvation. Therefore following results always have the aggressive re-transmit feature activated.

41

5.2. HTTP GET experiment

5.2 HTTP GET experiment

In the HTTP GET Experiment files of various sizes ranging from 32 KB to 4 MB was down-loaded using a HTTP GET request from a webserver. This is to simulate a web-browsingscenario. Each file was downloaded using the 18 different combinations in characteristics,defined in Section 4.3. Initially each file was downloaded using the interfaces and single pathTCP individually, as in the aggressive retransmit experiment. The best (lowest) downloadedtime was recorded to be used as our reference in the analysis. After that the file was down-loaded using MPTCP with both interfaces simultaneously. We initially compare the best-casescenario versus the worst, followed up by an average result given the 18 cases.

Figure 5.4 shows the download times as a function of the file size when using either regu-lar TCP and MPTCP for the best-case scenario. Hence the scenario with path homogeneity.The best-case scenario indicates the maximum improvement of MPTCP compared to the bestdownload time when using TCP. Given the results it is notable MPTCP improve the UE per-formance regardless on file size when MPTCP is exposed to uniform characteristics in thesubflows. This is showed in the graph by the consistency of lower download durations.

Figure 5.4 shows the average download times as a function of file size when using eitherregular TCP (red squares) and MPTCP (blue circles) for the best-case scenario. Both lines areequipped with a dashed linear fit. The graph is equipped with a zoomed area displaying filesizes 32KB to 512KB.

Figure 5.5 display the best case results compared to the best TCP path in percentage.MPTCP achieves a maximum rate improvement between 50 to 85 % compared to TCP dur-ing good conditions. Due to TCP slow-start, TCP and MPTCP mostly are limited by thedifference in round-trip time when downloading smaller files. Thus best case scenario havepath homogeneity RTT1 = 10ms, RTT2 = 60ms and BW1 = BW2 = 40 Mbit/s whenever therate is limited by the available bandwidth.

The average improvement when downloading files between 32KB and 4MB with similarcharacteristics indicates MPTCP improve the UE performance by „ 77%. The smallest file

42


size, 32KB, attains an maximum improvement percentage of 48.3%. The fact it is the small-est file size is expected as the congestion window has less time to expand and most of thetransmission is transmitted over the primary subflow.

Figure 5.5 shows the MPTCP rate in percentage compared to the rate the best TCP path giventhe test scenario in Figure 5.4.

Figure 5.6 display the results when MPTCP is exposed to bad scenarios with large dif-ferences in characteristics between the subflows: BW1 = 40 Mbit/s, BW2 = 1 Mbit/s andRTT1 = 10ms, RTT2 = 60ms. Given the results it is notable MPTCP may not be as advanta-geous during this scenario compared to path homogeneity. The results are displayed in per-centage in Figure 5.7. It appears MPTCP have it hard to achieve as the best TCP link duringpath heterogeneity. This means MPTCP may harm the UE performance when characteristicsdifferences are large.

Figure 5.6 shows the average download times as a function of file size when using eitherregular TCP (red squares) and MPTCP (blue circles) for the worst-case scenario. Both linesare equipped with a dashed linear fit. The graph is equipped with a zoomed area displayingfile sizes 32KB to 512KB.

43


Our results indicate a small degradation up to 16% compared to TCP when MPTCP isexposed to dissimilar characteristics. Between 256KB and 512KB there seems to be a little bitlarger degradation.

Figure 5.7 shows the MPTCP rate in percentage compared to the rate the best TCP path giventhe test scenario in Figure 5.6.

The overall MPTCP performance from the HTTP GET experiment is displayed in Fig-ure 5.8. The graph is equipped with the upper- and lower bound area in gray including adashed line indicating a linear fit to the average result. The small dots indicate a test resultper characteristics scenario. Table B.5 details the overall performance improvement from the18 test cases of the experiment. Notable is the average download time per characteristicsscenario MPTCP appears to improve the download time per file size. Compared to the pathheterogeneity (worst case) scenario in Figure 5.6, the average value looks more promising.

Figure 5.8 shows the average download times as a function of file size between 32KB and4MB when using either MPTCP or regular TCP. Red squares indicate the best download timewhen using regular TCP, blue circles indicate the worst attained download time when usingMPTCP and green cross indicates the best download time when using MPTCP. Both linesare equipped with a dashed linear fit. Grayed area indicate MPTCP upper- and lower bound.Each gray dot is a test result. The graph is equipped with a zoomed area for sizes 32 to 512KB.

44

5.3. Bulk transfer experiment

5.3 Bulk transfer experiment

This experiment was conducted by using 18 different characteristics, ranging from bad togood that performed a 20 seconds bulk transfer. The first two bulk transfers was executed byusing each interface and single path TCP individually. The best average TCP goodput is usedas a reference in our analysis. The third bulk transfer was performed using MPTCP with bothinterfaces simultaneously. The figures y-axis show the normalized gain ą 1 or degradationă 1 when using MPTCP and two subflows compared to the best path with regular TCP.

Figure 5.9 display the results when BW1 = 40 Mbit/s, BW2 = 8 Mbit/s as a function of thetwo defined RTTs:

1. RTT1 = RTT2 = 10ms to the left in the figure.

2. RTT1 = 10ms and RTT2 = 60ms to the right in the figure.

These RTTs are combined with the three defined packet error probabilities: p1 = p2 =0% marked with red squares, p1 = 0%, p2 = 0.01% marked with green circles and p1 =0.01%, p2 = 0.1% marked with blue triangles. The results can be found in detail in Table B.6,located in Appendix Chapter B. By studying the result it is indicated packet error probabilityhas a small impact on MPTCP due to the results are clustered together. This conforms onlyunless the packet error probability does not limit the rate of transfer by Mathis formula, seeSection 3.4.

Figure 5.9 shows the normalized gain compared to the best TCP path as a function of twosubflow RTTs when BW1 = 40 Mbit/s and BW2 = 8 Mbit/s. The markers indicate the threedifferent packet error probabilities.

Test case RTT1 = RTT2 = 10ms contributes to an improvement of 1.179 times increasedtransmission rate when using MPTCP compared to the best TCP path. The maximum achiev-able gain of 48 Mbit/s compared to the maximum 40 Mbit/s using TCP is 1.2 times improve-ment. This clearly indicates the benefits of MPTCP. Test case RTT1 = 10ms, RTT2 = 60msattain an improvement of 1.159 times compared to TCP’s b.c.p.

45


Figure 5.10 display the results for the same scenario as above but in this case BW1 = BW2 =40 Mbit/s. The results are detailed in Table B.7, located in Appendix Chapter B. At first glanceit is notable the results are not clustered as previous graph. In the test case RTT1 = RTT2 =10ms the initial two packet error probabilities: p1 = p2 = 0% and p1 = 0%, p2 = 0.01%appears to cluster, whereas p1 = 0.01%, p2 = 0.1% deviants from the others. If we assumep1 = 0%, p2 = 0% is correct when RTT1 = 10ms, RTT2 = 60ms based on earlier results in theprevious test, we can assume the other two deviants for some reason.

One reason they deviant is due the restriction in rate described by Mathis formula, men-tioned in Section 3.4 (page 24). The maximum rate of a TCP transmission is limited by thepacket error probability. By studying Mathis formula it becomes clear. The theoretical max-imum attainable rate when p = 0.1% and RTT = 10ms is „ 36.9 Mbit/s. Subflow2 withp = 0.1% and RTT = 10ms has an average rate of „ 32 Mbit/s. Hence the gain is limited byMathis formula and not clustered, resulting in a „ 1.78 gain compared to the best TCP path.The same applies to the two deviating test cases when RTT1 = 10ms, RTT2 = 60ms. Subflow2has an average rate of 7.4 Mbit/s with p = 0.1% and 23.4 Mbit/s when p = 0.01%.

Hence the throughput inevitably will be limited by Mathis formula in some cases, the re-sults indicate MPTCP performs well during path homogeneity in terms of bandwidth, with-out being affected by packet error rate and packet latency. The improved goodput of theMPTCP transmission result in a near aggregated rate based on the two subflows.

Figure 5.10 shows the normalized gain compared to the best TCP path as a function of twosubflow RTTs when BW1 = BW2 = 40 Mbit/s. The markers indicate three different packeterror probabilities.

Figure 5.11 (a) and (b) display the results with the same test scenario as before but BW1 = 40Mbit/s, BW2 = 1 Mbit/s in (a) and BW1 = 80 Mbit/s, BW2 = 1 Mbit/s in (b). The testscenario (b) was included as a verification. The results are detailed in Table B.8 in AppendixChapter B. Both graphs display MPTCP’s gain during large differences in bandwidth.

Again the markers are clustered as before. What is crucial is, in both graphs the gainis near the best TCP path. This tells us, it does not matter which interface an UE use, itwould always achieve as the best constituent TCP path when using MPTCP in a bulk transfer.Showed in Section 5.1 (page 39), there are a small extra degradation when round-trip times

46


are uniform, or near uniform. This is viewed in RTT1 = RTT2 = 10ms for both BW1 = 40Mbit/s, BW2 = 1 Mbit/s and BW1 = 80 Mbit/s, BW2 = 1 Mbit/s.

(a) BW1 = 40 Mbit/s and BW2 = 1 Mbit/s

(b) BW1 = 80 Mbit/s and BW2 = 1 Mbit/s

Figure 5.11 shows the normalized gain compared to the best TCP path as a function of twosubflow RTTs for two test scenarios. The markers indicate the three different packet errorprobabilities.

47

5.4. Varying characteristics experiments

5.4 Varying characteristics experiments

This section display the results from the experiments with varying characteristics to furtherunderstand the performance of MPTCP.

Bufferbloat experiment

The first experiment test how MPTCP handles large variances in round-trip time. Since thebufferbloat phenomenon is commonly encountered in RANs, large differences may occur.The test case was defined as followed: Initial characteristics is best case scenario. At T = 0the bandwidth drops from 40 Mbit/s to 256 Kbit/s. The first test case is executed withoutforcing the bufferbloat phenomenon. The second test force bufferbloat to occur by not adapt-ing the buffers according to equation 3.7 when the rate suddenly drops. This will lead toexcess memory in the network where segments are queued before being passed towards thedestination. This will make the RTT in Subflow2 increase rapidly. After 5 seconds the rate isreset. Our goal is to see if MPTCP achieves to remain stable during such occurrence.

Figure 5.12 display the results when not forcing bufferbloat. This can be seen by the slowincrease when T ą 5. This slope occurs because the buffers are adjusted to the recommendedbuffer size during the interval 0 ă T ă 5. Thus no extra buffers are needed and the latencyremains stable, seen in the figure. When the buffer grow and the connection may allocatemore bandwidth TCP enters the additive increase state, resulting in a slow stable increase.This increase will always be as fast as the increase in regular TCP due to the uncoupledcongestion control.

Figure 5.12 shows the throughput T for Subflow1 and Sublow2, the RTT for Subflow2 and theMPTCP goodput in the scenario without the bufferbloat phenomena. At T = 0 the bandwidthdrop from 40 Mbit/s to 256 Kbit/s.

48


Figure 5.13 display the results when executing the test case that forces the bufferbloat phe-nomena. Notable is the RTT of Subflow2 increase rapidly from 15ms to more than 1800ms,detailed in Figure 5.14. Still MPTCP succeeds to remain a stable transmission rate during theimpact.

Figure 5.13 shows the throughput T for Subflow1 and Sublow2, the RTT for Subflow2 and theMPTCP goodput in the scenario with the bufferbloat phenomena. At T = 0 the bandwidthdrop from 40 Mbit/s to 256 Kbit/s. It is reset after 5 seconds.

Figure 5.14 shows the RTT of Subflow2 when either exposed (solid blue) or unexposed(dashed red) to the bufferbloat phenomena.

49


Figure 5.15 display how MPTCP behaves when changing the bandwidth between 40 and 1Mbit/s with a frequency of 2Hz. This might be seen as a phenomena not occurring that oftenin real environments. Still MPTCP appears to react fast to changes, but notable is the reducedmaximum goodput. This is due to a higher MPTCP retransmission percentage (15.6%) be-cause of the aggressive retransmit feature. Thanks to aggressive retransmit the connectiondoes not stall compared to Figure 5.2 in Section 5.1.

Figure 5.15 shows the MPTCP transmission rate for both subflows and the goodput forcingthe bufferbloat phenomenon when changing the bandwidth of Subflow2 between 40 and 1Mbit/s with a frequency of 2Hz. Subflow1 allocate stationary 40 Mbit/s and RTT = 10msand p = 0%.

Given these results MPTCP appears to be robust against large variances in round-triptime. Due to the fact the throughput is related to the round-trip time, see Section 3.4 (page23) for explanation, this experiment also conclude MPTCP appears to be robust against rapidchanges in bandwidth.

50


Packet Loss experiment

The second experiment test how MPTCP handles large variances in packet error proba-bility. Since RATs are exposed to environmental attenuating factors, rapid changes in pmay occur. The test case was defined as followed: Both subflows allocate 40 Mbit/s andRTT1 = 10ms, RTT2 = 60ms between the subflows. At T = 0 the packet error probabilitychanges from 0% to 0.1% in both subflows.

Figure 5.16 display the results from the experiment. The graph is equipped with two linesindicating the average rate. Dashed line for MPTCP and solid for the best TCP path. Giventhe results it is notable MPTCP still improves the performance compared to TCP. Thus rapidchanges in packet error probability appear not harm MPTCP when using uncoupled TCPReno. Also notable is the slow additive-increase increment in Subflow2 when T ą 5.

Figure 5.16 shows the results from the packet loss experiment. Both subflows allocate 40Mbit/s and ∆RTT = 50ms. At T = 0 the packet error probability changes from 0% to 0.1%and p is reset at T = 5. The lines indicate the average rate during the interval.

51


52

6 Discussion

This thesis has completed a number of experiments to evaluate the impact of an introduc-tion by the MPTCP protocol with uncoupled congestion control for LTE subscribers. Withthroughput as a primary metric for LTE subscribers, the goal of this thesis was to determineif MPTCP fulfills its goal of performance to never achieve no less throughput than the rate ofthe slowest TCP path.

Experiments were conducted in a closed controlled environment using a prototype im-plementation of uncoupled MPTCP. By emulating desired characteristics in a deterministicenvironment the results are based on real measurements, hence they are not simulated. Toeliminate outliers each test were performed multiple times and the average value used in theresults. As reference in the analysis the best constituent TCP path was used.

By examining the protocol with uncoupled congestion control the results provide an up-per limit behaviour about MPTCP. This means the results declare how a performance-focusedversion of the protocol behaves given different scenarios. Furthermore the design used in theexperiments do not fulfill goal 8 in Table B.9 as uncoupled congestion control may harm otherTCP connections.

The experiments were defined given the LTE QoS Class Indicators (QCIs) table, repro-duced in Table B.1. Hence experiments ranging from good to bad were defined with increas-ing differences in bandwidth, latency and packet error probability. To enumerate realisticscenarios each experiment used a first-in first-out jitter of 20% of the RTT. To successfully ex-pose MPTCP to scenarios with uniform underlying resources TCP buffers were controlled byUbuntu’s autotuning functionality. But the MPTCP receive buffer was stationary with the rec-ommended buffer size according to RFC 6824 [18]. The average MPTCP buffer size of all testsis„ 3KB. This technique reduce the risk of external factors like buffer lag affecting the results.In real life scenarios the UE’s operating system most likely will have an adaptive buffer size,allowing it to expand up to megabytes in size thus avoiding the throughput degradation; forinstance Ubuntu’s autotuning allow up to 3 MB buffers as default which would allow thedifference in transfer rate to expand up to ten times compared to our test-cases. But this is aoperating system design question as mobile devices have limited amount of RAM comparedto desktops, thus with large enough buffers at the UE the design of MPTCP in theory worksand in most cases also works in practice. But one has to remember, with the presence of bufferbloat phenomenon, the difference in round-trip time between the RATs becomes larger thusrequires even larger buffers from the UEs.

53

6.1. Results

6.1 Results

Given the results it is notable uncoupled MPTCP performs well during path homogeneity.But may have some troubles when the rate between the subflows increase. This may be dueeither by limitations in available bandwidth, higher packet error probability, see Mathis for-mula in Section 3.4 (page 24) or latency. Therefore in some cases the best path using TCPachieves better throughput than MPTCP and therefore MPTCP do not fulfill its goal to al-ways achieve better or equal throughput than the best consistent path with TCP. Hence weconclude MPTCP may not be recommended during path heterogenity with significant dif-ferences, especially in mobile devices. This is due to the inevitable occurrence of increasedbuffers and power consumption. Significant difference in our experiments are declared toa 40 : 1 ratio with 40 Mbit/s on one subflow and 1 Mbit/s on the other. Our results showMPTCP contribute to a throughput degradation up to 16% compared to the best TCP path.

Hence there are two notable key factors impacting the outcome of MPTCP:

a) The size of data information to be transmitted.

b) The throughput-difference between subflows regardless on round-trip time, bandwidthand packet error probability.

This is supposed the buffers are not limiting the transfer. They may be excluded since thebehaviour of a limited buffer behaves the same in MPTCP and TCP.

MPTCP in web browsing

To determine how MPTCP impact end-user experience during web-browsing we performedan experiment where we downloaded files with sizes ranging from 32KB to 4MB from awebserver using HTTP GET. The average amount of data transmitted in a TCP connectionwhen loading a average web page is 54KB per TCP transmission. This is what we referas short-burst traffic, see definition in Terminology and acronyms. Putting the results fromour HTTP GET experiment, in perspective to short-burst traffic, it is indicated that MPTCPusing uncoupled congestion control improve the performances with„ 65% compared to TCPduring path homogeneity. But during path heterogeneity with a 40:1 difference in bandwidthallocation, 50ms RTT difference and packet error p1 = 0.01%, p2 = 0.1%, MPTCP appears todegrade the performance up to „ 16%. Average gain from our eighteen scenarios was a 20%improvement.

This makes it hard to conclude if MPTCP is recommended for short-burst traffic. Mostsmall transactions are done within a few round-trip times. This means after the initial subflowis set, the second interface must be awaken. Hence battery consumption will inevitable in-crease thus harming the UE. Furthermore requested buffers will harm the UE memory usage.Something some mobile devices struggle with. Given our test we differently can conclude,MPTCP does not fulfill its goal with a difference-ratio of 40 : 1 in achievable throughput(40mbps, 1mbps).

MPTCP in bulk transfers

With download size factor excluded in bulk-transfers, MPTCP significantly improves the per-formance during path homogeneity. Resulting in a near aggregated transfer rate. The exper-iments used up to 80 : 1 ratio between two subflows and still achieved close to the best TCPpath. Only „ 2% slower than TCP. This was verified by the BW1 = 80 Mbit/s, BW2 = 1Mbit/s experiment with similar results. We conclude uncoupled MPTCP achieves an goodaverage resource utilization percentage of 94.1%. Only 3% from the theoretical limit.

Further experiments conclude uncoupled MPTCP’s ability to adapt to changing character-istics. Shown in the bufferbloat experiment MPTCP is robust to large changes in round-trip

54

6.2. Methodology

time without impacting the throughput. The same goes for large changes in bandwidth andpacket error probability. Thus uncoupled MPTCP is concluded as a robust protocol. Thiswas expected due to the decoupled design of the subflows making them independent of eachother.

Our results are in line with other studies. Most conclude MPTCP performs well duringpath homogeneity such as in data centers and appears to be robust. But further intelligenceis needed during path heterogeneity [6], [40], [47], [7], [40].

6.2 Methodology

Since the experiments were conducted in a controlled environment using real measurementsit provides a good indication how a performance focused implementation of MPTCP impactsLTE. By implementing the deterministic setup using common softwares and hardwares theexperiments are easy to replicate and the results should provide high reliability. Howevera controlled environment may contribute to a good indication, it is necessary to do real de-terministic measurements. To verify our results it would be interesting to perform the testsusing LTE and Wi-Fi on different smartphone and tablet devices in a laboratory environment.This was an extended goal to this thesis but unfortunately there was not enough time.

We focused on performance and resource utilization during this thesis, not battery con-sumption and memory utilization. Since the experimental setup was two desktop computersand not handhold devices where the amount of resources are more constrained it is necessaryto further investigate how MPTCP impacts the UE in terms of the gain. Aspects that wouldbe especially interesting is to investigate how MPTCP impacts battery consumption and theamount of memory.

There is always some factor of errors when emulating characteristics that may harm thedeterminism of a non-real-time system. We worked hard to minimize these factors by fine-tuning the environment. But one have to remember there always is a small factor of error.

6.3 The work in a wider context

Most portable devices today are equipped with multiple network controllers, e.g. LTE, Wi-Fi, bluetooth, etc. But the single path restriction of regular TCP results in unused resourcesthat multi-RAT user equipments already own and have access to. With on-going infrastruc-tural development of mobile broadband and increased availability of low-price smartphones,people are able to change from cellular phones to smartphones, especially in the developingworld. Thus the number of mobile broadband subscribers is forecast to grow nearly threetimes from 2.9 to 8.4 billions between 2014 and 2020 [15]. LTE is the standard bearer of thisadvancing. Since the amount of radio frequencies is already constrained and therefore theamount of resources limited, this will be a challenge for the network operators. MPTCP al-lows for global load balancing from the UEs to offload congested access networks or combineresources they already have access to. This may allow users to use applications they earlierwas unable to use or replace expensive subscriptions with multiple cheaper ones. Howeverthe inevitable increase in energy consumption an introduction of MPTCP would add shouldbe considered, both at the user side and the system side.

55

6.3. The work in a wider context

56

7 Conclusions

The potential of MPTCP to be next generation TCP is evident without any doubt. But thesimple design of MPTCP proposed in RFC 6824 has its flaws. Concluded in the report is thesimple design and lack of intelligence in current design of MPTCP may not be ideal. Espe-cially during short burst traffic and path heterogeneity. By including an intelligent modulethat adapts to the environment, buffer density may be minimized. Thus the degradationavoided.

An introduction today of uncoupled MPTCP within data centers most certainly wouldimprove performance with near aggregated results. There the occurrence of path homogene-ity are more frequent and memory/battery usage are not a problem. Not compared to mobiledevices. But today the gain when using MPTCP may not be that obvious to recommend in-troducing MPTCP in multi-RAT devices. This is due to the inevitable occurrence of increasedbattery consumption, increased memory usage and the possibility of a degradation with tosignificant path heterogeneity.

From our experiments it is notable MPTCP fully utilize given resources. For large fluctu-ations our experimental feature referred as aggressive retransmit was crucial to avoid flowstarvation. It improve robustness by resending unacknowledged information more rapidly.The drawback is it may affect throughput a small fraction when the percentage retransmittedpackages increase.

Also concluded is the responsiveness of uncoupled MPTCP make it a robust protocol incases of bufferbloat phenomena. This is a crucial benefit to multi-RAT devices due to thefluctuations in power of radio signals.

To further understand how MPTCP impact LTE future work is necessary. By movingthe experiments to a isolated lab environment equipped with cellular technologies and realuser equipments a better understanding of the protocol is achievable. After that live mass-measurement experiments may be executed.

Furthermore is the importance to investigate and implement intelligence to the schedulingmechanism. This is to reduce buffers and improve performance during short transfers. Thereexists a couple of proposed designs trying to improve the intelligence of MPTCP [28].

Interesting future work could also include evaluating the impact that running MPTCPmay have on the utilization of multi-core web servers [21] and the energy consumption of theUEs.

57

58

A Terminology and acronyms

Host A host (computer) is the consumer of communication services andgenerally communicates with other hosts by executing applicationsprograms on behalf of a user.

Multi-RAT device Is a device equipped with multiple network interface controllersconnected to different Radio Access Technologies.

Multi-homed device A device is said to be multi-homed if it has multiple interfaces, eachequipped with its own unique IP address.

API An Application Programming Interface (API) specifies a set offunctionalities for operating with a software component such as asocket.

MSS The Maximum Segment Size (MSS) specifies the largest amount ofdata, specified in octets, a TCP segment encapsulates.

MTU The Maximum Transmission Unit (MTU) specifies the maximumsize of a network layer (IP) protocol data unit in octets.

UE Refers to User Equipment.b.c.p. Is an acronym for the ’best constituent path’.

59

60

B Tables

Table B.1: LTE QoS Class Identifiers (QCI).

QCI Bearer Type Priority RTT (ms) p (%) Description

1

GBR

2 100 1e´2 VoIP call2 4 150

1e´3 Video call3 3 50 Real time gaming4 5 300

1e´6Video streaming

5

Non-GBR

1 100 IMS Signaling6 6 300 Video, TCP

7 7 100 1e´3 Voice, Video (Live), InteractiveGaming

8 8300 1e´6 Video, TCP

91 9

RTT - Packet Round-Trip Time p - Packet Error Probability1 Default QCI in the default bearer for non-privileged subscribers.

61

Table B.3: The MPTCP option subtypes.

Value Symbol Description

0x0 MP_CAPABLE Initiates the MPTCP connection by determining if destina-tion host is multipath capable.

0x1 MP_JOIN Joins a already established MPTCP connection.

0x2 DSS Data Sequence Signal (Data ACK and Data Sequence Map-ping).

0x3 ADD_ADDR Allows hosts to announce additional addresses (and option-ally ports).

0x4 REMOVE_ADDR Announce removal of addresses.

0x5 MP_PRIO Change Subflow Priority.

0x6 MP_FAIL Change Subflow Priority.

0x7 MP_FASTCLOSE Fast Close.

See RFC 6824 and the Internet Assigned Numbers Authority (IANA) “TCP Option KindNumbers" registry for further details [18] [61].

Table B.5: Details the average improvement when using MPTCP compared the best con-stituent path with TCP.

File size MPTCP (ms) TCP (ms) MPTCP ImpactImprovement(+) / Degradation(´)

32KB 21.553 26.586 23.9%

64KB 28.536 33.104 19.5%

128KB 42.859 50.464 21.7%

256KB 75.507 79.253 11.3%

512KB 126.101 134.841 13.6%

1MB 228.434 251.690 14.6%

2MB 412.450 473.370 19.1%

4MB 765.818 911.900 24.8%

Table B.6: Details the resulting throughput in Mbit/s from the bulk experiment whenSubflow1 allocate 40 Mbit/s stationary and Subflow2 allocate 8 Mbit/s. Viewed in the ta-ble are the two different packet latencies used in the bulk experiment; to the left a latencydifference of 0ms (10ms, 10ms), and to the right a 50ms difference (10ms, 60ms).

10ms, 10ms 10ms, 60msPacket error probability Subflow1 Subflow2 Subflow1 Subflow2

0%, 0% 37.294 7.401 37.696 7.5070%, 0.01% 37.378 7.388 37.78 7.5180.01%, 0.1% 37.353 7.38 37.535 6.295

62



0%, 0% 36.656 37.588 37.674 37.3750%, 0.01% 36.713 37.008 37.611 23.4250.01%, 0.1% 37.368 31.927 37.562 7.492



0%, 0% 37.016 0.19612 38.525 0.971610%, 0.01% 36.664 0.19343 38.391 0.972330.01%, 0.1% 36.991 0.19638 38.180 0.97573

63

Table B.9: The goals of MPTCP architecture.

Functionality goals

Goal 1 MPTCP must be able to support simultaneous use of multiple paths.Goal 2 A MPTCP connection using multiple paths must provide a throughput no less

than the throughput of the best constituent path.Goal 3 A MPTCP connection using multiple paths must support interchangeable use

of paths, to ensure improved resilience of the connection and provide no lessresilient than regular TCP.

Application compatibility goals

Goal 4 MPTCP must provide the same service model as TCP.Goal 5 MPTCP must be in some degree backward compatibility with the API of regular

TCP to supersede transparently.

Network compatibility goals

Goal 6 MPTCP must be able to traverse already deployed and configured middleboxesto minimize the impact of the middlebox-problem.

Goal 7 If a MPTCP connection encounters a path with insurmountable incompatibilities,MPTCP must revert to regular TCP.

Goal 8 A MPTCP connection may not under not circumstances harm or displace otherconnections of any type at shared bottlenecks.

Security goal

Goal 9 MPTCP must provide as equal or higher degree of security than regular TCP.

64

Table B.11: The specifications of the experimental setup hardware.

UE MPTCP-capable server

CPU Intel Core 2 Duo E84003GHz x2

Intel Core 2 Duo E65502.33GHz x2

RAM 3GB 1GB

OS Ubuntu Desktop 12.04.05 (precise) 64bit

Kernel Custom kernel based on Linux 3.2.50

Kconfig Fixed interrupt timer freqeuncy:1000hz

Tickless system, interrupt timer fre-quency: 250hz

NIC eth0IP-address 10.42.0.10Intel 82567LM-3 1Gbit/sLimited to 100Mbit/s

eth1IP-address 10.42.0.11Intel 82541PI 1Gbit/sLimited to 100Mbit/s

eth0IP-address 10.42.0.1Intel 82566DM-2 1Gbit/sLimited to 100Mbit/s

Duplex Full

txqueuelen 1000

RX / TX 256 octets

MPTCP-buf Read: 2 ˚ř

(BW) ˚max(RTT), Write: 6Mb

TCP-buf Read + write: Linux 3.2 default configuration

65

66

C Figures

Figure C.1 shows the MPTCP resource utilization percentage as a function of two subflowRTTs when BW1 = 40 Mbit/s and BW2 = 40 Mbit/s. The markers indicate three differentpacket error probabilities.

67



68

Bibliography

[1] 3GPP Technical Specification 23.402. Architecture enhancements for non-3GPP accesses (Re-lease 8). Tech. rep. URL: www.3gpp.org.

[2] 3GPP TR 36.913 version 8.0.1 Release 8. LTE; Requirements for further advancements forEvolved Universal Terrestrial Radio Access (E-UTRA). Tech. rep. URL: www.3gpp.org.

[3] 3rd Generation Partnership Project. Oct. 15, 2014. URL: www.3gpp.org.

[4] D. Eastlake 3rd, J. Schiller, and S. Crocker. Randomness Requirements for Security. RFC4086 (Best Current Practice). Internet Engineering Task Force, June 2005.

[5] Janet Ellen Abbate. “From ARPANET to Internet: A history of ARPA-sponsored com-puter networks, 1966–1988”. In: (1994).

[6] Yung-Chih Chen, Yeon-sup Lim, Richard J Gibbens, Erich M Nahum, Ramin Khalili,and Don Towsley. “A Measurement-based Study of Multipath TCP Performance overWireless Networks”. In: Proceedings of the 2013 conference on Internet measurement confer-ence. ACM. 2013, pp. 455–468.

[7] Yung-Chih Chen, Erich M Nahum, Richard J Gibbens, Don Towsley, and Y sup Lim.“Characterizing 4G and 3G Networks: Supporting Mobility with Multi-Path TCP”. In:University of Massachusetts Amherst, Tech. Rep (2012).

[8] Yung-Chih Chen and Don Towsley. “On Bufferbloat and Delay Analysis of MultiPathTCP in Wireless Networks”. In: ().

[9] America Cisco Headquarters. Security Configuration Guide: Access Control Lists, Cisco IOSXE Release 3S. May 8, 2015. URL: www.cisco.com/c/en/us/td/docs/ios-xml/ios/sec_data_acl/configuration/xe-3s/sec-data-acl-xe-3s-book.pdf.

[10] Erik Dahlman, Stefan Parkvall, and Johan Skold. 4G: LTE/LTE-advanced for mobile broad-band. Academic Press, 2013.

[11] Aleksandar Damnjanovic, Juan Montojo, Yongbin Wei, Tingfang Ji, Tao Luo, MadhavanVajapeyam, Taesang Yoo, Osok Song, and Durga Malladi. “A survey on 3GPP hetero-geneous networks”. In: Wireless Communications, IEEE 18.3 (2011), pp. 10–21.

[12] S. Deering and R. Hinden. Internet Protocol, Version 6 (IPv6) Specification. RFC 2460 (DraftStandard). Updated by RFCs 5095, 5722, 5871, 6437, 6564, 6935, 6946, 7045, 7112. Inter-net Engineering Task Force, Dec. 1998.

69

www.3gpp.org

www.3gpp.org

www.3gpp.org

www.cisco.com/c/en/us/td/docs/ios-xml/ios/sec_data_acl/configuration/xe-3s/sec-data-acl-xe-3s-book.pdf



Bibliography

[13] dumpcap - Dump network traffic. Feb. 19, 2015. URL: www.wireshark.org/docs/man-pages/dumpcap.html.

[14] Ericsson. Ericsson Mobility Report, On the pulse of the Networked Society. Tech. rep. June2014.

[15] February Ericsson. Ericsson Mobility Report, On the pulse of the Networked Society. Tech.rep. 2015.

[16] G. Fairhurst. The Datagram Congestion Control Protocol (DCCP) Service Codes. RFC 5595(Proposed Standard). Updated by RFC 6335. Internet Engineering Task Force, Sept.2009.

[17] A. Ford, C. Raiciu, M. Handley, S. Barre, and J. Iyengar. Architectural Guidelines for Multi-path TCP Development. RFC 6182 (Informational). Internet Engineering Task Force, Mar.2011.

[18] A. Ford, C. Raiciu, M. Handley, and O. Bonaventure. TCP Extensions for Multipath Oper-ation with Multiple Addresses. RFC 6824 (Experimental). Internet Engineering Task Force,Jan. 2013.

[19] Bryan Ford and Janardhan R Iyengar. “Breaking Up the Transport Logjam.” In: HotNets.2008, pp. 85–90.

[20] Thomas J Hacker, Brian D Athey, and Brian Noble. “The end-to-end performance ef-fects of parallel TCP sockets on a lossy wide-area network”. In: Parallel and DistributedProcessing Symposium., Proceedings International, IPDPS 2002, Abstracts and CD-ROM.IEEE. 2001, 10–pp.

[21] Raoufehsadat Hashemian, Diwakar Krishnamurthy, Martin Arlitt, and Niklas Carls-son. “Improving the scalability of a multi-core web server”. In: Proceedings of the 4thACM/SPEC International Conference on Performance Engineering. ACM. 2013, pp. 161–172.

[22] Michio Honda, Yoshifumi Nishida, Costin Raiciu, Adam Greenhalgh, Mark Handley,and Hideyuki Tokuda. “Is it still possible to extend TCP?” In: Proceedings of the 2011ACM SIGCOMM conference on Internet measurement conference. ACM. 2011, pp. 181–194.

[23] Hung-Yun Hsieh and Raghupathy Sivakumar. “pTCP: An end-to-end transport layerprotocol for striped connections”. In: Network Protocols, 2002. Proceedings. 10th IEEE In-ternational Conference on. IEEE. 2002, pp. 24–33.

[24] iOS: Multipath TCP Support in iOS 7. Feb. 19, 2015. URL: www.support.apple.com/us-en/HT201373.

[25] ITU-R. Requirements related to technical performance for IMT-Advanced ratio interface(s).Tech. rep. Report ITU-R M.2134, 2008.

[26] Janardhan R Iyengar, Paul D Amer, and Randall Stewart. “Concurrent multipath trans-fer using SCTP multihoming over independent end-to-end paths”. In: Networking,IEEE/ACM Transactions on 14.5 (2006), pp. 951–964.

[27] Van Jacobson. “Congestion avoidance and control”. In: ACM SIGCOMM Computer Com-munication Review. Vol. 18. 4. ACM. 1988, pp. 314–329.

[28] Han Ah Kim, Bong-hwan Oh, and Jaiyong Lee. “Improvement of MPTCP Performancein heterogeneous network using packet scheduling mechanism”. In: Communications(APCC), 2012 18th Asia-Pacific Conference. IEEE. 2012, pp. 842–847.

[29] James F Kurose and Keith W Ross. Computer Networking: A top-down approach featuringthe Internet. Vol. 2. Addison-Wesley Reading, 2001.

[30] James F Kurose and Keith W Ross. Computer Networking: A top-down approach featuringthe Internet. Vol. 2. Addison-Wesley Reading, 2001.

70

www.wireshark.org/docs/man-pages/dumpcap.html

www.wireshark.org/docs/man-pages/dumpcap.html

www.support.apple.com/us-en/HT201373

www.support.apple.com/us-en/HT201373

Bibliography

[31] Jianxin Liao, Jingyu Wang, and Xiaomin Zhu. “cmpSCTP: An extension of SCTP tosupport concurrent multi-path transfer”. In: Communications, 2008. ICC’08. IEEE Inter-national Conference on. IEEE. 2008, pp. 5762–5766.

[32] Tova Linder, Pontus Persson, Anton Forsberg, Jakob Danielsson, and Niklas Carlsson.“On Using Crowd-sourced Network Measurements for Performance Prediction”. In:Proc. IEEE/IFIP WONS. Cortina d’Ampezzo, Italy: IEEE, Jan. 2016.

[33] Luiz Magalhaes and Robin Kravets. “Transport level mechanisms for bandwidth ag-gregation on mobile hosts”. In: Network Protocols, 2001. Ninth International Conferenceon. IEEE. 2001, pp. 165–171.

[34] Matthew Mathis, Jeffrey Semke, Jamshid Mahdavi, and Teunis Ott. “The macroscopicbehavior of the TCP congestion avoidance algorithm”. In: ACM SIGCOMM ComputerCommunication Review 27.3 (1997), pp. 67–82.

[35] MPTCP Option Subtypes. Oct. 26, 2014. URL: www.iana.org.

[36] J. Nagle. Congestion Control in IP/TCP Internetworks. RFC 896. Internet Engineering TaskForce, Jan. 1984.

[37] NetEm - Linux Network Emulator. Dec. 11, 2014. URL: www.manpages.ubuntu.com/manpages/utopic/man8/tc-netem.8.html.

[38] Netperf 2.6.0. Dec. 11, 2014. URL: netperf.org.

[39] Norbert Niebert, Andreas Schieder, Jens Zander, and Robert Hancock. Ambient net-works: co-operative mobile networking for the wireless world. John Wiley & Sons, 2007.

[40] Christoph Paasch, Gregory Detal, Fabien Duchene, Costin Raiciu, and Olivier Bonaven-ture. “Exploring mobile/WiFi handover with Multipath TCP”. In: Proceedings of the 2012ACM SIGCOMM workshop on Cellular networks: operations, challenges, and future design.ACM. 2012, pp. 31–36.

[41] Domenico Porcino and Walter Hirt. “Ultra-wideband radio technology: potential andchallenges ahead”. In: Communications Magazine, IEEE 41.7 (2003), pp. 66–74.

[42] J. Postel. Internet Protocol. RFC 791 (INTERNET STANDARD). Updated by RFCs 1349,2474, 6864. Internet Engineering Task Force, Sept. 1981.

[43] J. Postel. Transmission Control Protocol. RFC 793 (INTERNET STANDARD). Updated byRFCs 1122, 3168, 6093, 6528. Internet Engineering Task Force, Sept. 1981.

[44] J. Postel. User Datagram Protocol. RFC 768 (INTERNET STANDARD). Internet Engineer-ing Task Force, Aug. 1980.

[45] R Fonseca, G Porter, R. H. Katz, S. Shenker. IP Options are not an option. Tech. rep. Elec-trical Engineering and Computer Sciences University of California at Berkeley, 2005.

[46] C. Raiciu, M. Handley, and D. Wischik. Coupled Congestion Control for Multipath Trans-port Protocols. RFC 6356 (Experimental). Internet Engineering Task Force, Oct. 2011.

[47] Costin Raiciu, Sebastien Barre, Christopher Pluntke, Adam Greenhalgh, Damon Wis-chik, and Mark Handley. “Improving datacenter performance and robustness withMultipath TCP”. In: ACM SIGCOMM Computer Communication Review. Vol. 41. ACM.2011, pp. 266–277.

[48] Allen L Ramaboli, Olabisi E Falowo, and Anthony H Chan. “Bandwidth aggregationin heterogeneous wireless networks: A survey of current approaches and issues”. In:Journal of Network and Computer Applications 35.6 (2012), pp. 1674–1690.

[49] ITUTX Recommendation. “200 (1994)| ISO/IEC 7498-1: 1994”. In: Informationtechnology–Open Systems Interconnection–Basic Reference Model: The basic model ().

71

www.iana.org

www.manpages.ubuntu.com/manpages/utopic/man8/tc-netem.8.html

www.manpages.ubuntu.com/manpages/utopic/man8/tc-netem.8.html

netperf.org

Bibliography

[50] Kultida Rojviboonchai, Toru Osuga, and Hitoshi Aida. “RM/TCP: Protocol for reliablemulti-path transport over the internet”. In: Advanced Information Networking and Appli-cations, 2005. AINA 2005. 19th International Conference on. Vol. 1. IEEE. 2005, pp. 801–806.

[51] Jerome H Saltzer, David P Reed, and David D Clark. “End-to-end arguments in systemdesign”. In: ACM Transactions on Computer Systems (TOCS) 2.4 (1984), pp. 277–288.

[52] Stefania Sesia, Issam Toufik, and Matthew Baker. LTE: the UMTS long term evolution.Wiley Online Library, 2009.

[53] Claude Elwood Shannon. “A mathematical theory of communication”. In: ACM SIG-MOBILE Mobile Computing and Communications Review 5.1 (2001), pp. 3–55.

[54] Justine Sherry, Shaddi Hasan, Colin Scott, Arvind Krishnamurthy, Sylvia Ratnasamy,and Vyas Sekar. “Making middleboxes someone else’s problem: network processingas a cloud service”. In: ACM SIGCOMM Computer Communication Review 42.4 (2012),pp. 13–24.

[55] T.J. Socolofsky and C.J. Kale. TCP/IP tutorial. RFC 1180 (Informational). Internet Engi-neering Task Force, Jan. 1991.

[56] Data Encryption Standard. Federal Information Processing Standards Publication (FIPSPUB) 46-3, National Bureau of Standards, Gaithersburg, MD (1999).

[57] W. Stevens. TCP Slow Start, Congestion Avoidance, Fast Retransmit, and Fast Recovery Al-gorithms. RFC 2001 (Proposed Standard). Obsoleted by RFC 2581. Internet EngineeringTask Force, Jan. 1997.

[58] R. Stewart. Stream Control Transmission Protocol. RFC 4960 (Proposed Standard). Up-dated by RFCs 6096, 6335, 7053. Internet Engineering Task Force, Sept. 2007.

[59] ITU Strategy and Policy Unit. “ITU Internet Reports 2005: The internet of things”. In:Geneva: International Telecommunication Union (ITU) (2005).

[60] Carl A Sunshine and Yogen K Datal. “Connection management in transport protocols”.In: Computer Networks (1976) 2.6 (1978), pp. 454–473.

[61] TCP Option Kind Numbers. Oct. 26, 2014. URL: www.iana.org.

[62] Jonathan S Turner and David E Taylor. “Diversifying the internet”. In: Global Telecom-munications Conference, 2005. GLOBECOM’05. IEEE. Vol. 2. IEEE. 2005, 6–pp.

[63] International Telecommunication Union. The World in 2014: ICT Facts and Figures. Tech.rep. 2014.

[64] wget - Linux manual page. Dec. 11, 2014. URL: www.linux.die.net/man/1/wget.

[65] Damon Wischik, Mark Handley, and Marcelo Bagnulo Braun. “The resource poolingprinciple”. In: ACM SIGCOMM Computer Communication Review 38.5 (2008), pp. 47–52.

[66] Damon Wischik, Costin Raiciu, Adam Greenhalgh, and Mark Handley. “Design, Imple-mentation and Evaluation of Congestion Control for Multipath TCP.” In: NSDI. Vol. 11.2011, pp. 8–8.

72

www.iana.org

www.linux.die.net/man/1/wget

Documents

Multipath TCP - DiVA portal938436/FULLTEXT01.pdf · Combining multiple radio access technologies, ... 1 Introduction 1 ... Each gray dot is a test result. The