Estimating Available Bandwidthwith pathload and abget

Professor Nelson Fonsecanfonseca@ic.unicamp.br

Estimators - Professor Nelson Fonseca 2

Outline

Introduction Measuring available bandwidth SLoPS method

pathload abget

Interrupt coalescence Examples Conclusions References

Estimators - Professor Nelson Fonseca 3

Introduction

Estimators - Professor Nelson Fonseca 4

Bandwidth Bandwidth defines how much data

(bits) can be transferred between two points in a time interval (seconds)

Related terms: nominal capacity, capacity, narrow link, available bandwidth, tight link, bulk transfer capacity (BTC), ...

Estimators - Professor Nelson Fonseca 5

Available Bandwidth Important to traffic engineering Important to take scheduling decisions

(grids and clouds) Important to take decisions about links

upgrade

Estimators - Professor Nelson Fonseca 6

Measuring availablebandwidth

Estimators - Professor Nelson Fonseca 7

Varies with timeui : utilization of link i in a time interval ( 0 <= ui <= 1 ) Available bandwidth in link i:

Available bandwidth in path (Avail-bw): minimum Ai

Available Bandwidth: unutilized capacity

)1( uiCiAi

Calculating Available Bandwidth

Estimators - Professor Nelson Fonseca 8

τ defines sampling/averaging timescale Average avail-bw in τ

Does not tell how avail-bw varies Variation range gives more information

t

A(t)

τ

T

Available Bandwidth:time varying metric

Estimators - Professor Nelson Fonseca 9

Can be measured at each link from interface utilization data using SNMP

MRTG (Multi Router Traffic Grapher) graphs: 5-minute averages

But users do not normally have access to SNMP data And MRTG graphs give only per-hop avail-bandwidth

Measuring per-hop available bandwidth

Estimators - Professor Nelson Fonseca 10

Blast path with UDP packets Intrusive Carter & Crovella: cprobe (Infocom 1996)

Packet train dispersion does not measure available bandwidth (Dovrolis et.al. Infocom’01)

Measure throughput of large TCP transfer TCP throughput depends on network buffer

Ribeiro et.al. : Delphi (ITC’00) Correct estimation when queuing occurs only at single

link Assumes that cross traffic can be modelled by MWM

model

Measuring path Available Bandwidth

Estimators - Professor Nelson Fonseca 11

SloPS method

Estimators - Professor Nelson Fonseca 12

Examine One-Way Delay (OWD) variations of a fixed rate stream Relate rate to avail-bw

OWD: Di = Tarrive-T = Tarrive - Tsend + Clock_Offset(S,R) SLoPS uses relative OWDs, ΔDi = Di+1 – Di-1 (independent of clock

offset) With a stationary & fluid model for the cross traffic, and FIFO

queues: If R > min Ai, then ΔDi > 0 for I = 1…N Else ΔDi = 0 for for I = 1…N

sendS R RR

Main Idea

Estimators - Professor Nelson Fonseca 13

Periodic Stream: K packets, size L bytes, rate R = L/T

If R>A, OWDs gradually increase due to self-loading of stream

Illustration

Estimators - Professor Nelson Fonseca 14

For some rate R Increasing trend in OWDs R > Avail-bw No trend in OWDs R < Avail-bw

Trend in real data

Estimators - Professor Nelson Fonseca 15

At sender: Send periodic stream n with rate Rn At receiver: Measure OWDs Di for i=1…K At receiver: Notify sender of trend in OWDs At sender: If trend is :

increasing (i.e. Rn >A ) repeat with Rn+1 < Rn non-increasing (i.e. Rn <A ) repeat with Rn+1>Rn

Selection of Rn+1 : Rate adjustment algorithm Terminate if Rn+1 – Rn < ω ω : resolution of final estimate

Iterative Algorithm in SloPS

Estimators - Professor Nelson Fonseca 16

Grey region: Rate R not clearly greater or smaller than Avail-bw during the duration of stream

Rate R is within variation range of avail-bw

If things were black and white...

Estimators - Professor Nelson Fonseca 17

Increasing trend R > variation range of Avail-bw No trend R < variation range of Avail-bw Grey trend R inside variation range

Big Picture

Estimators - Professor Nelson Fonseca 18

Grey region

Increasing trend :Rmax = R(n)R(n+1) = (Gmax + Rmax)/2

Non-increasing trend:Rmin = R(n)R(n+1) = (Gmax +Rmin)/2

Grey region & R(n) > Gmax:

Gmax = R(n) R(n+1) = (Gmax + Rmax )/2

Grey region & R(n) < Gmin:

Gmin = R(n)R(n+1) = (Gmin + Rmin )/2

Terminate if:(Rmax – Gmax) && (Rmin– Gmin) <

Rmax > A

Rmin < A

Gmax

Gmin

Varia

tion

Rang

e

Rate Adjustment Algorithm

Initial Gmin = 0Initial Gmax = Average throughput of a small initial transfer of UDP data

Estimators - Professor Nelson Fonseca 19

pathload (1)

Implements SLoPS Estimates end-to-end available bandwidth Based on one-way delay trend of periodic streams Reports a range of avail-bw

Corresponds to variation, measured in stream duration

According the developers, the interference is almost nonexistent

Estimators - Professor Nelson Fonseca 20

pathload (2) Version 1.3.2 (2006) Features

Open Source Two processes (one at sender, one at receiver) No need to root privileges No need to define several parameters

pathload2 released recently (Apr 2009) Installation

Download from http://www.cc.gatech.edu/fac/Constantinos.Dovrolis/bw-est/pathload

.tar.gz Uncompress in a Linux operating system ./configure && make

Estimators - Professor Nelson Fonseca 21

pathload (3) Two connections

TCP – control UDP – data stream

Execution order At sender:

pathload_snd Waits receiver connection via TCP port 55002

At receiver: pathload_rcv -s <sender IP address|sender

hostname> Connects to sender at TCP port 55002 Sender sends UDP data to receiver at UDP port

55001

Estimators - Professor Nelson Fonseca 22

pathload (4)

Problems Static ports User needs access to the two end points Ports must be opened in firewalls

Estimators - Professor Nelson Fonseca 23

abget (1)

Also implements SloPS Solves some pathload problems

Runs only in one end-point Can make the estimations using TCP ports already

opened in firewalls (e.g. 80)

Estimators - Professor Nelson Fonseca 24

abget (2) Version 1.0 (2007) Features

Open Source One process Need root privileges Can estimate both upstream and downstream

availability Installation

Install libpcap Download from http://www.ics.forth.gr/~papadog/abget/abget-1.0.tar.gz Uncompress in a Linux operating system ./configure && make && make install

Estimators - Professor Nelson Fonseca 25

TCP connections Both control and data stream abget uses TCP control mechanisms

Execution order (at user end-point) Downstream

abget -d -f <file location> <web server> Retrieves the file and control the rate simulating the

ACKs Upstream

abget -u <web server>

abget (3)

Estimators - Professor Nelson Fonseca 26

Downstream abget emulates TCP by sending fake ACKs to the

server (It captures the received packets and bypasses the default TCP implementation)

The advertise window of the server is limited to 1 MSS

Each ACK “requests” 1 MSS By defining the ACKs period, it is possible to define

the downstream rate To force the server to send at rate R, abget sends

ACKs at T=MSS/R

abget (4)

Estimators - Professor Nelson Fonseca 27

Process illustration (downstream)

abget (5)

Estimators - Professor Nelson Fonseca 28

Upstream abget sends an HTTP request in many overlapping

segments. It repeats for many iterations with different rates in

order to estimate a variation range for the available bandwidth in that path.

abget (6)

Estimators - Professor Nelson Fonseca 29

abget (7)

Problems Many parameters Results not so precise as pathload root privileges

Estimators - Professor Nelson Fonseca 30

Interrupt Coalescence

Estimators - Professor Nelson Fonseca 31

Interrupt Coalescence

1 Network card event != 1 Interrupt Enabled to avoid overload to the operating

system Should be disabled to avoid incorrect

estimations With interrupt coalescence: more packets in a time

interval

Estimators - Professor Nelson Fonseca 32

pathload and abget

By reading the source code, pathload avoids interrupt coalescence. abget doesn't

But each manufacturer implements a different method

Estimators - Professor Nelson Fonseca 33

Examples

Estimators - Professor Nelson Fonseca 34

Examples 1 and 2 - Scenario

Estimators - Professor Nelson Fonseca 35

Examples 1 – Precision at 10Mbps

Estimators - Professor Nelson Fonseca 36

Examples 1 – Overhead at 10Mbps

Estimators - Professor Nelson Fonseca 37

Examples 2 – Precision at 1Gbps

Estimators - Professor Nelson Fonseca 38

Examples 2 – Overhead at 1Gbps

Estimators - Professor Nelson Fonseca 39

Conclusions

Estimators - Professor Nelson Fonseca 40

Conclusions Available Bandwidth estimators must

estimate using a range instead a average Interrupt coalescence must be disabled pathload and abget

Unchanged Clock offset pathload gives more precise results abget runs only in one end-point abget uses HTTP (no problem with firewalls) pathload doesn't need root privileges

pathload is ideal if there is access to the two end-points. Abget is ideal if there isn't.

Estimators - Professor Nelson Fonseca 41

References http://www.cc.gatech.edu/fac/Constantinos.Dovrolis/pathload.html Official pathload website

http://www.measurementlab.net/measurement-lab-tools#pathload2 pathload2

http://www.ics.forth.gr/~papadog/abget/ Official abget website

http://www.caida.org/tools/taxonomy/performance.xml CAIDA (The Cooperative Association for Internet Data Analysis) website about network performance tools

http://www.29west.com/docs/THPM/latency-interrupt-coalescing.htmlInterrupt coalescence and GNU/Linux