Layer 4 TCP Performance in carrier transport networks · 2012-11-07 · Layer 4 TCP Performance in carrier transport networks Throughput is not always equal to bandwidth Roland Stooss

Layer 4 TCP Performancein carrier transport networksThroughput is not always equal to bandwidth

Roland Stooss

JDSU Deutschland GmbHSenior ConsultantData/IP Analysis SolutionsMühleweg 5, D-72800 Eningen u. A.Tel. +49 (0)7121/86 2138Email: [email protected]

© 2011 JDSU. All rights reserved. Layer 4 TCP Performance in carrier transport networ ks2

Network

Datalink

Physical1

23

OSI Model: Responsibility and recommended tests

Customer’sIT departmentresponsibility

NetworkProvider’sresponsibility

Sharedresponsibility

Transport4

Application

Presentation

Session5

6

7HTTP, FTP, Email, etc.

TCP

IP

Ethernet

RFC2544and/or

Y.1564

RFC6349

Network

Datalink

Physical1

23

Transport4

Application

Presentation

Session5

6

7


Throughput is not always equal to available bandwid th

Layer 2/3 throughput (bandwidth) test

� Across test link� 100 % available

Layer 4 throughput test (upload)� Back-to-back� 100 % available

Layer 4 throughput test (upload)

� Across test link� 40 % throughput -> Why?

Layer 4/TCP …”really” outside of the responsibility of the carrier.

100%

100%

40% ?

40% ?

LAN

Typical customer complains• Bad application responds times• Total throughput is not like total

available bandwidth -> Why?


Upload

TCP throughput – the limiting factors … (1)

Window Size

PerformanceBandwidthPerformance

Retransmission Timer

Latency Error Rate

Sender can use full bandwidth and transmit at 100 % .� Thus the transmit buffer is quickly used up.Latency is too large.� Therefore the ACKs of the receiver are coming too la te.� Sender may already start sending unnecessary retran smissions.Cause� Window Size does not fit to bandwidth and latency.� Only one TCP session with high available bandwidth.

Window Size

Bandwidth Latency

… even if there is no JAM in the network

Max. Segment Size


Upload

Error rateLatencyBandwidth

Window Size

PerformancePerformance

Retransmission TimerPolicing/Shaping

MTU

TCP throughput – the limiting factors … (2)

Window Size

PerformancePerformance

Retransmission Timer Window SizeResponsibility with carrier Responsibility with customer

Error rateBandbreiteBandbreiteBandwidth LaufzeitLaufzeitLatency

Policing/Shaping

MTU

� MTU: Customer configured too small MTU or carrier has packet size limitations because of used encapsulation method (MPLS)

� Bandwidth: Customer may ordered too low BW or carrier doesn’t deliver BW� Latency: Customer didn’t fix RTT in SLA. Carrier selected long path

� Policing/Shaping: At the critical rate limitation interfaces (in customers or carriers network) Basic policing is extreme critical for TCP performance.

Shared responsibility:


Why does a TCP traffic get less throughput?

� Performance of TCP client and server and Round Trip Time– As higher the performance and as lower the RTT as faster the Slow Start

TCP Traffic rate

t

Available BandwidthRate limit by policing600

1000

� High performance TCP client and low RTT [1]:� Session could go above available BW. It always runs in extreme congestion. Final

averaged throughput is very low� High performance TCP client and large RTT [2]:

� Session can’t go above available BW, therefore no congestion and stable traffic rate.� Multiple TCP sessions with possible cumulated TCP traffic above BW may not reach

the BW, because of congestion effect.

[1]

[2]


Where might be the network problems for TCP?

LAN CE PE LANCE

� At carrier access with rate limit function– Most critical point, especially when Traffic Policy instead of Traffic Shaping is used

� At customers LAN to WAN CE interface− Total LAN rates are cumulated higher than WAN access interface (1 GigE)

Access

1GigE SDH/OTN �Fixed BW

Metro/Core

600Mb

� At carrier metro/core SDH/OTN– This is mainly an end-to-end fixed and 100% available bandwidth– No further congestion situations (loss, delay) -> fine for TCP traffic

PEIP/MPLSEth/VLAN �Dynamic BW

� At carrier metro/core IP/MPLS or Eth(VLAN)– This is packet based switching with dynamic and possible over booked BW– Congestion situations can occur (loss events and increasing delay)-> can be stress for TCP sessions. Probably less li ke with policing at the access.


Carrier Access – When is their rate limit?

� Multiple services with different CIRs has to be con sidered as point of rate limit like single service with rate limit.

� Recommended test application to detect TCP performa nce problems-> RFC6349 Frame Work for TCP Throughput testing

� Not really helpful – RFC2544 and/or Y.1564

Total BW 100%, but each service has its “CIR” rate limit

Single service rate limit

� The carrier access port is limiting the accepted traffic to the available transport rate.

� The carrier access port is limiting the accepted traffic per service to their available service rate (CIR)

� Total access bandwidth might be 100%.


Turn-up Related Problem RFC2544 Y.1564 RFC 6349

Single Service, Layer 2/3 SLA Issues (loss, jitter, etc.)

N/A

Multi-service, Layer 2/3 SLA Issues (service prioritization, loss, jitter, etc.)

N/A

Demonstrate the effect of End customer TCP Window size on throughput (CPE issue)

Inadequate device buffers to handle bursty applications

Policing effects to TCPperformance

Lack of TCP Testing is a Service Activation Gap


Application aware turn -up testing workflow

Application level testing

RFC-6349 / TrueSpeed™

Application level testing

RFC-6349 / TrueSpeed™

Experience Your Network Like Your

Customer Do

Class of Service verification

ITU-T Y.1564SAMComplete

Class of Service verification

ITU-T Y.1564SAMComplete

Prove L2/3 KPIs for differentiated

services

Traditional L2/3 Turn-up

RFC-2544

Traditional L2/3 Turn-up

RFC-2544

Verify basic performance for single services


Quality of testing =

Reliability + Repeatabilityindependent of individual skill level

Automated test routines help ensure quality testing

TrueSpeed� automated RFC-6349 test in <5 min


TCP Performance Tests – What has to be tested?� Can the maximum packet size be used?

– Path MTU (IP level) should be 1500 bytes = 1460 bytes MSS on TCP level

� What is the best Round Trip Time (RTT)?– Measured with low load to avoid influence of filled buffers

� Show influence of TCP Window Size– Run 4 TCP sessions with different WS values and compare actual measured versus

ideal TCP throughput.– This helps to explain the TCP WS influence and will already show critical problems.

� Run a typical TCP session below the Bottleneck Band width– Test duration based on bytes to transfer instead of fixed test time.– Compare actual versus ideal rate and transfer time – Graphical Results to correlate possible TCP performance issues due to

retransmissions and/or congestive RTT.

� Run multiple TCP sessions to force congestion situa tion– This test will conduct multiple TCP connection transfers to test whether the link fairly

shares (traffic shaping) or unfairly shares (traffic policing) the bandwidth

(MTU: Maximum Transfer Unit; MSS: Maximum Segment S ize)

New RFC 6349


Why does TCP traffic not reach the full bandwidth?

Few main factors if rate limit is in place:� TCP load is mainly based on large file transfers an d less short sessions

– Short sessions are not all active at the same time, they “better” share the bandwidth– With large file transfers a bad throughput is more likely recognized.

� How is the performance of TCP client/server and how is Round Trip Time– High-performance servers or “old & weak” PCs– As higher the performance and as lower the RTT as faster the Slow Start

� Possible “total” TCP rates are higher then available transport rate.– The TCP session(s) may run into congestion– As larger the difference as more critical (E.g. 100Mbps interface to 2 Mbps transport)

� TCP window size is larger then ingress buffer of th e network port– During slow start the buffer is quickly filled and results in packet loss– If there is also large RTT the sessions runs into Dup Ack situation

� Policing function at the network access port– Is the throughput rate done in slots of available bandwidth

Why do customer only sometimes complain about not getting the committed throughput?


TCP performance and congestion in the metro/core

AccessCore

� Congestion in the core is different to congestion ( policy based rate limitation) at customer access− This is packet based switching with dynamic and possible over booked BW.

Congestion may cause loss events and increasing delay.

� Recommended test application to detect TCP performa nce problems-> RFC6349 Frame Work for TCP Throughput testing-> Layer 2/3 traffic test with focus on packet dela y and delay variation− PDV shows short burst situations and PD shows longer high load and

overload situations, which may result in loss.


Carrier Access – Where and how to limit the rate?� Who is managing the bandwidth limitation – customer or/and carrier?� Essential criteria: Access rate = or > as transpor t rate

NothingPolicingShapingPolicingShaping

NothingNothingPolicingPolicingNothing

CPE CE

Customer ProviderN x Access rate Access rate Transport rate

� Access rate even as transport rate

NothingPolicingNothingShapingShaping

PolicingPolicingShapingShapingPolicing

� Access rate larger as transport rate

The above examples are only exemplary and not compl ete. They shall only show the complexity


Why layer 4 TCP full state test stream emulation?

� Customer requires proof of the transfer capacity wi th „real“ layer 4 TCP load (According RFC 6349)– Mostly as second step after failed self-test with laptop file transfer. – Quite meaningfully, if the connection goes through a packet oriented (Ethernet, PBB,

MPLS) core network.

– UDP displaces TCP traffic. This is not detected with pure L3 or state less L4 load.– Low packet loss events can extremely reduce the TCP throughput if there is also

high RTT.– To determine the best TCP window size configuration for max. throughput. – Performance tests across full state fire walls.

� When layer 4 TCP state less load is enough?– When the transport link has fixed bandwidth and no rate limit at the access

(Fiber, optical channel, SDH/OTN)

– Only RTT and TCP Windowsize is influencing the final throughput


Estimated, ideal max TCP throughput

� The test application is to supply additional inform ation, which points the protocol specific throughput limits out.– What is the max. throughput for a TCP session, which can be expected?– Depends on the standard TCP window size values (8/16/32/64 kbyte)

� The table shows the max. throughput rates, dependent on the TCP window size, RTT and number of TCP sessions.

� Note: Calculation is based on ideal network situations and is done with statically test load

Calculation based on ideal network situations

Documents

Layer 4 TCP Performance in carrier transport networks · 2012-11-07 · Layer 4 TCP Performance in carrier transport networks Throughput is not always equal to bandwidth Roland Stooss