1

BWdetail: A bandwidth tester with detailed

reporting Masters of Engineering

Project Presentation

Mark McGinley April 19, 2007Advisor: Malathi Veeraraghavan

Acknowledgement: This work was carried out under the sponsorship of NSF ITR-0312376, NSF ANI-0335190, and DOE DE-FG02-04ER25640 grants.

2

Outline

Problem Statement Related Work Overview of BWdetail Features and Uses Demonstration Conclusion

3

Problem Statement

Growing eScience traffic

Ter

abyt

es /

mo

nth

ESnet Monthly Accepted Traffic, January, 2000 – June, 2006Ref. 1

4

Problem Statement Scientific research efforts such as Atlas

characterized by: Large datasets Geographically distributed

collaborators 1800 physicists, 150 universities

and laboratories

Supporting high speed transfers of datasets often requires system tuning, diagnostics

[2]

5

Outline

Problem Statement Related Work Overview of BWdetail Features and Uses Demonstration Conclusion

6

Related Work Iperf from NLANR

Successor to ttcp, similar to nttcp, nuttcp

Web100 Host system software (autotuning) Lack of instrumentation/tools for

diagnosing performance issues Iperf100

Web100-based Iperf

7

Related Work

Nettest Incorporates Iperf

Net100 Leverages Web100 to eliminate

“wizard gap” with autotuning tcpdump

8

Related Work

Iperf

9

Related Work

tcpdump

10

Related Work

BWdetail

11

Related Work

Iperf Iperf100 Bwdetail

Bandwidth estimation

Buffer tuning

Parameter tuning

Cwnd, etc  

No kernel modifications    

12

Outline

Problem Statement Related Work Overview of BWdetail Features and Uses Demonstration Conclusion

13

Overview of BWdetail

Bandwidth estimation ability of Iperf

Instrumentation capabilities similar to Web100, without kernel modification

14

Overview of BWdetail

Software architecture C program TCP_INFO struct associated with each

socket Kernel parameters read/written using

getsockopt(), setsockopt(), sysctl(), system()

15

Outline

Problem Statement Related Work Overview of BWdetail Features and Uses Demonstration Conclusion

16

Features and Uses

Data transfer for goodput estimation Memory-to-memory is most common

usage Disk/memory-to-disk/memory Memory transfer can either be a

specific size or duration Disk transfer using a file of specific

size, to eliminate caching effects

17

Features and Uses

Effect of caching 1 GB disk-to-disk successive transfer

0

100

200

300

400

500

600

700

800

900

1000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

seconds

Mbps Transfer 1

Transfer 2

18

Features and Uses

Multi-threaded client Client uses separate threads to send

data and monitor flow

Sendfile(), send loop Monitoring loop

Transfers data, records goodput from application’s perspective

Records fields from TCP_INFO, particularly cwnd and RTT

19

Features and Uses

Tunable parameters from command line Socket buffer size TCP congestion control algorithm CTCP-enabled Web100 capable SACK, Nagle, no metrics save

20

Features and Uses

Adjustable reporting granularity Client and server report goodput over

some small, configurable interval (100ms)

Client reports cwnd, RTT, etc, every smaller, configurable interval (5ms)

21

Features and Uses

Applications Research regarding transport

protocols CTCP/Reno/BIC Cwnd and RTT measurements

Improving performance on a path Records relevant kernel parameters Ongoing recording of cwnd, RTT, goodput Change parameters, repeat

22

Outline

Problem Statement Related Work Overview of BWdetail Features and Uses Demonstration Conclusion

23

Demonstration Results

BIC

0

200400

600

8001000

1200

14001600

1800

0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57

seconds

# pa

cket

s

cwnd

24

Outline

Problem Statement Related Work Overview of BWdetail Features and Uses Demonstration Conclusion

25

Conclusion

Available bandwidth estimation Instrumentation such as cwnd

without requiring kernel modification or privileged access

26

Thank you

Questions?

27

References [1] W.E. Johnston, ESnet4: Networking for the Future of DOE

Science, http://www.es.net/ESnet4/ESnet4-Internet2Members-Dec-06-2006-1.ppt, Dec. 5, 2006

[2] The Atlas Experiment, www.atlas.ch

28

Limitations

Does not replace tcpdump tcpdump handles content

Needed to detect retransmissions, Ethernet PAUSE frames, verification

Incorporation of libpcap capabilities would degrade the non-interference status of BWdetail

BWdetail is interested in statistics, not content

29

Limitations

BWdetail does not replace an expert Still requires knowledge to tune, does

not provide autotuning Use of TCP_INFO struct presents

portability issues

30

Impact of monitoring

Minimal, even at ~1 Gbps

880

890

900

910

920

930

940

950

0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0

Seconds

Mbps

100ms

25

5

1

31

RTT

BIC

0

20

40

60

80

100

120

140

160

0 3 6 10 13 16 19 22 25 29 32 36 39 42 45 48 52 55 58

seconds

ms

RTT