2009. 3.17

Alok Shriram and Jasleen Kaur

Presented by Moonyoung Chung

Empirical Evaluation of Techniques for Measuring Available Bandwidth

Outline

Introduction– Available Bandwidth

– ABETs

Related Work Motivation and Goal Experimental Framework Experimental Results

– Accuracy

– Cost

Conclusion

Infocom '07 2

Available Bandwidth (AB) End-to-End AB:

– minimum unused capacity of path.

– varies with time

Infocom '07 3

Narrow LinkTight Link or Available Bandwidth (AB)

100 Mbps

10 Mbps2500 Mbps

1200 Mbps

1000 Mbps

950 Mbps

Tight link: minimum avail-bw link

ui : utilization of link i in time interval t ( 0 ≤ ui ≤ 1 )

Available bandwidth in link i:

Available bandwidth in path (Avail-bw):

)u-(1C A iii

)u-(1 C min A min A ii0..Hi

i0..Hi

• Applications

– Network monitoring

– Congestion control

– Design of transport protocol

– Streaming applications

Methodology for AB estimation

Infocom '07 4

Tool Probes Inference Metric

Pathload Equi-spaced One-way delay

Pathchirp Exp-Spaced Dispersion

Spruce Packet-Pair Dispersion

IGI Packet Train Dispersion

Iperf Tcp-Stream Receiving Rate

Cprobe Packet Train Receiving Rate

1. Design of Probes

2. Inference Logic

End-to-End Path

Feedback for successive Iterations

Send probe packet(s) into the network and measure a response

closed-loop tools

closed-loop tools

Packet Pair [Jacobson ’88, Keshav ’91]

– Spruce

Packet Train

– Pathload, PathChirp, IGI, Cprobe

Algorithmic Techniques

dispersion

T1 T0Narrow Link

Tn+1 Tn

Gap

Gap Gap

t

t

Infocom '07 5

one way delaydispersionreceiving rate

cross traffic

Implementation techniques High time-stamping accuracy

– variations in end-to-end delays, in the sub-millisecond rage

1. OS support

• for detecting and discarding probe streams that ap-pear to not have been time-stamped accurately

2. Collect observations from several probe streams before converging on a robust estimate of AB

Infocom '07 6

Questions

Infocom '07 7

Which algorithmic technique performs the best? To what extend does current implementation tech-

nology limit tool performance?

– How well would tools perform if technology ad-vances in the future?

Related Works a small subset of ABETs

– Robeiro et. al. [6] compares PathChirp to Pathload and TOPP

– Hu et. al. [4] compares IGI/PTR to Pathload and Iperf

– Strauss et. al. [7] compares Spruce to Pathload and IGI

only simple network and traffic scenarios biased by current implementation technology ignores two key quantities: MT and SI

Infocom '07 8

Motivation and Goal Motivation

– Existing evaluations are either non-comprehensive and bi-ased, or are affected by implementation issues

Goal– Conducting an extensive experimental study of existing

techniques for measuring AB

Approach– Evaluation independent of current implementation tech-

nology

• simulation environments

– Evaluation against diverse probing and network conditions

• Gigabyte network path/ diverse conditionsInfocom '07 9

Evaluating Conditions

1. dynamic traffic load

2. measurement timescales (MT)– the time-scale at which AB is observed.

3. sampling intensities (SI)– the duration for which the AB is sampled per unit time.

4. number of bottleneck links

5. location of bottleneck

Both MT and SI impact the accuracy and variability of the AB sampled by an ABET. [Shriram et al. 2006]

Infocom '07 10

Definition– Timescale at which we observe the AB process

– the duration of a single probe stream = length of the probe stream

Measurement Timescale (MT)

Infocom '07 11Time

AB

MT

Packet Pair Packet Train

MT MT

MT effect on AB

Infocom '07 12

Smaller MT expect more variability, expect lower accuracy

Sampling Intensity (SI) Definition

– the duration for which the AB is sampled per unit time.

– the number of probes

– the product of the MT and the number of probe streams sent per unit time

Infocom '07 13

Higher SI expect better accuracy

ABET Implementation Tools:

– Pathload, PathChirp, Spruce, IGI, Fast-IGI, and Cprobe

Implement in the NS-2 network simulation envi-ronment

Incorporating MT Incorporating Si

– Open-loop tools : Cprobe, Spruce, PathChirp

• SI = MT/(MT+G)

• G: the gap between successive probe-streams

– Closed-loop tools: Pathload, IGI, Fast-IGI

• The construction of a probe-stream is determined by the delays experienced by the previous probe-stream.

• RTT instead of SIInfocom '07 14

Performance Metrics Accuracy-related

– AB estimation error: the estimated AB – the actual AB

(* actual AB = the number of bits that traverse the link during the tool run/the tool run-time)

Cost-related– run time: the time taken by a tool to return an estimate

– probing overhead: the total amount of network probe traffic sent by the tool in order to arrive at a single esti-mate of AB

– intrusiveness: the average bit-rate of a tool (overhead/runtime)

Impact on responsive cross-traffic– probe traffic on the response time of ongoing TCP connec-

tionsInfocom '07 15

Single Bottleneck Topology with a Single Bottleneck Link

– Tool Traffic: traffic by ABETs

– Cross Traffic: traffic with a constant bit-rate (CBR)

Infocom '07 16

link capacity = 1Gbpslink delay = 1mssufficient buffer

Validation of ABET Implementation

Infocom '07 17

Pathload, Spruce quite accurate pathChirp slightly higher Cprobe based on Receiving Rate poor IGI -> R-IGI good

Cross wit a constant bit-rate (CBR)

Cprobe based on Receiving Rate poor

Dynamic Traffic Load Trace used for evaluation

– Collect five 1-hour packet traces from four different Inter-net links

Infocom '07 18

for 1 Gbps links

Single Bottleneck Tool errors with default parameters

Infocom '07 19

95-per-centiles

average

5-per-centiles

Accuracy The average estimation errors are higher with dynamic cross-

traffic than with CBR. Pathload, PathChirp, Fast-IGI have similar average error. R-IGI has lower error. Spruce has higher error.

Variability The estimation errors vary widely around the average. least for Pathload quite high for Spruce and PathChirp

MT=1msMT=1msMT=10msMT=0.5msMT=10ms

Impact of MT IPLS-CLEV: Impact of MT (SI=0.1, RTT=60ms)

Infocom '07 20

Increasing the MT improves the accuracy of all ABETs• The gain are negligible beyond an MT of 50ms.

MT impact on PathChirp is lower. Spruce now is the most accurate (it was the least with default

settings)

Impact of SI IPLS-CLEV: Impact of SI (MT=10ms)

Infocom '07 21

SI and RTT has a negligible impact on the accuracy

open-looped tools

Bottleneck Location

Infocom '07 22

Different tight and narrow links

tight linknarrow

link

cross traffic: IPLS-CLEV(410Mbps), IPLS-KSCY(530Mbps)

Bottleneck Location : Result MT=50ms, SI=0.1

Infocom '07 23

The error of PathChirp and Spruce increases by a factor of 2-3.

Other ABETs are not impacted much.

Single Bottleneck

Spruce

PathChirp

Different tight and narrow links

IPLS-CLEV

Multiple bottlenecks

Infocom '07 24

Single narrow link: two tight link

tight link

narrow link

IPLS-KSCYIPLS-CLEV

Multiple bottlenecks : Result MT=50ms, SI=0.1

Infocom '07 25

Single narrow link: two tight link

PathChirp and Spruce further degrades and the most in-accurate.

The accuracy of the others are not significantly impacted.

Single Bottleneck

Spruce

PathChirp

IPLS-CLEVIPLS-CLEV, IPLS-

KSCY

Overhead

Infocom '07 26

Spruce

PathChirp

Fast-IGI

R-IGI

Pathload

PathChirp, R-IGI, Fast-IGI have the least over-head. Overhead increase with MT.

SI and RTT has no impact on the overhead.

Run-time

Infocom '07 27

Spruce is the fastest tool. Pathload is the slowest tool.

Spruce

Pathload

Increase the MT a proportional increase in the runtime.

Intrusiveness Non-intrusiveness: cross traffic should not be affected

Infocom '07 28

All closed-loop tools are quite intrusive. Spruce has the highest value of intrusiveness. PathChirp is the most non-intrusive tool

Responsive Cross-Traffic Responsive cross-traffic

– TCP uses congestion-control mechanism to reduce the data sending rate on detecting network congestion.

• queuing delays

• losses on the subsequent packet transmissions

How adversely do these tools impact the perfor-mance of applications that rely on such responsive transport protocols? – Tmix: traffic-generation tool that incorporate the respon-

sive behavior of TCP

Infocom '07 29

link capacity = 1Gbpsaverage traffic = 300Mbpsbuffer size = 100 MSS-sized packets

Impact on Responsive Cross-Traffic CDF of response times with default parameters

Infocom '07 30

PathChirp has no noticeable impact on the response time.

Pathload and Fast-IGI can significantly impact on the response times.

CD

F o

f con

necti

on

s

no tool & PathChirp

Pathload

Fast-IGI

Conclusion Conduct a comprehensive empirical evaluation of existing al-

gorithmic techniques used for measuring end-to-end AB. Key Observations

– Accuracy

• The accuracy can be improved by using an MT of 50ms.

• SI and path RTT have negligible impact on the accuracy.

• While Spruce is the most accurate for paths with a single bot-tleneck link, its accuracy worsens for paths for multiple bot-tleneck links.

– Cost

• PathChirp has the lowest overhead, and it has no impact on the response times of TCP.

• Spruce is the fastest, but has the highest value of intrusive-ness.

• The cost of Pathload, R-IGI, and Fast-IGI seems to be highest.

– Responsive Cross-traffic

• If an application needs to run an ABET repeatedly on a given internet path, it should use PathChirp.

Infocom '07 31