1

Monitoring Internet connectivity of Research

and Educational Institutions

Les Cottrell – SLAC/Stanford UniversityPrepared for the workshop on “Developing Country Access to On-line

Scientific Publishing", 4-5 October 2002, Trieste, Italy http://www.slac.stanford.edu/grp/scs/net/talk/ictp-02.html

Partially funded by DOE/MICS Field Work Proposal on Internet End-to-end Performance Monitoring (IEPM), also supported by IUPAP

2

Outline• Measurement project initially created for HEP to

measure performance for various collaborations, extended to other physics & science collaborations– Value for planning, trouble shooting, setting expectations,

comparisons, setting SLAs etc.

• Methodology• Results

– Round trip times

– Loss

• Summary

3

Measurement Architecture• Uses existing ubiquitous Internet ping infrastructure, no tools to install• Hierarchical vs. full mesh, each monitoring site chooses remote sites• Lightweight –

– low network impact (100bits/s/path)– no special machines– trivial to add monitored sites

• Runs continuously since 1995

WWWWWW

ArchiveArchive

MonitoringMonitoringMonitoringMonitoring MonitoringMonitoring

RemoteRemote

RemoteRemoteRemoteRemote

RemoteRemote

HEPNRC

Reports & Data

CacheMonitoringMonitoring

SLAC Ping

HTTP

ArchiveArchive

1 monitor hostremote host pair

4

PingER Measurement Methodology• Measurement host admin choose remote hosts of interest

– sends 21 pings each 30 mins to each chosen remote host

– Records RTT, loss, jitter, unreachable, out of order …

– Records data in local cache

• Archive host gathers data from measurements hosts regularly (at least daily)– Archives, analyzes and generates reports from data

– Make reports and data publicly available via the web

• Requirements:– Remote host: need a host accessible to pings, and a contact in case host does

not respond (almost no effort)

– Monitoring host: a low end host to make measurements, file space for cache, admin to install toolkit, choose remote hosts, build configuration file, respond to archivers in case unable to get data & keep it running (<<10% FTE)

– Archive site: probably about 20% of an FTE

5

PingER deployment• Measurements from

– 34 monitors in 14 countries– Over 600 remote hosts– Over 72 countries – Over 3300 monitor-remote site pairs– Measurements go back to Jan-95– Reports on RTT, loss, reachability, jitter, reorders, duplicates …

• Countries monitored– Contain 78% of world population– 99% of online users of Internet– Mainly A&R sites

Monitoring Sites

Remote Sites

6

User interface 1/2

Choose: metric, monitoring site(s), remote sites(s), time granularityShows colored values by time, allows downloading of results for further analysis

7

User interface 2/2: PingER Group History Table

8

PingWorld

Java applet at http://jas.freehep.org/demos/PingWorld/

9

Performance Results

Examples of the type of information that the monitoring can provide

10

History - Round Trip Time (RTT)• Improving by 10-20%

year• More direct paths• Replacing satellites

with land lines – Satellite >~550ms

• Faster lines & network equipment

• Lower limit speed of light in fiber

Typical lower limit today ~ distance/(0.3 * (0.6 * c))

Speed of light in fiber

11

RTT to world from US• Note large

number of satellite links (> 600ms dark red)

• Note reduction by Aug 2002

Jan 2000

Aug 2002

12

Impact of loss on applications• Email

– fairly insensitive to quality, may be delayed but keeps retrying for days and eventually gets through

• Web– usually has human but expectations are

low, performance often more limited by server, human present so can retry

• Bulk file transfer– unattended, if > 10-12% loss

connections can time out

• Interactive telnet, voice– very time & loss sensitive– E.g. telnet/ssh loss of > 3% severely

impacts typing ability, interactive voice sensitive at lower losses

Importance of loss/perform

ance

13

History - Loss• Loss more

critical than RTT

• Losses cause timeouts of typically seconds

• 40-50% improve/yr

• Best networks below 0.1%

• Russia, SE Europe, China several years behind

14

History – Loss Quality

• Fewer sites have v. poor to dreadful performance• More have good performance (< 1%)

15

Loss to world from USUsing year 2000, fraction of world’s population/country fromwww.nua.ie/surveys/how_many_online/

16

Losses: World by region, Jan ‘02• <1%=good, <2.5%=acceptable, < 5%=poor, >5%=bad

• Russia, S America bad

• Balkans, M East, Africa, S Asia, Caucasus poor

Monitored Region \ Monitor Country

BR (1)

CA (2)

DK (1)

DE (1)

HU (1)

IT (3)

JP (2)

RU (2)

CH (1)

UK (3)

US (16) Avg Region

Avg -(H

Avg NA + WEU + JP Pairs

COM 0.2 0.3 0.3 0.2 COM 0.27 23

Canada 1.8 1.6 0.3 0.5 9.0 0.3 1.4 21.7 0.7 0.7 0.5 3.5 Canada 0.74 126

US 0.4 2.6 0.2 0.3 8.0 0.1 1.4 13.8 0.3 1.3 0.9 2.7 US 0.88 2149

C America 0.9 0.9 C America 0.89 19

Australasia 0.8 1.8 1.3 Australasia 1.30 18

E Asia 1.2 3.5 1.0 1.1 9.0 0.9 2.0 5.2 1.5 1.4 1.5 2.6 E Asia 1.61 215

Europe 0.4 5.6 0.3 0.5 5.4 0.4 1.3 15.5 1.1 1.0 1.0 2.9 Europe 1.38 852

NET 1.7 6.2 1.0 1.3 8.0 1.6 3.6 21.9 0.7 0.8 0.9 4.3 NET 2.00 85

FSU- 4.5 0.5 9.8 0.5 1.6 11.2 4.3 1.2 2.0 4.0 FSU- 2.09 48

Balkans 3.8 3.8 Balkans 3.83 109

Mid East 4.6 1.4 3.0 8.5 2.8 3.2 11.8 2.0 2.5 2.1 4.2 Mid East 2.70 57

Africa 5.8 1.5 12.0 1.2 4.2 11.9 2.0 1.9 2.5 4.8 Africa 2.72 45

Baltics 5.3 0.8 2.3 7.7 2.2 3.5 10.8 4.8 2.1 3.9 4.3 Baltics 3.12 67

S Asia 1.6 7.3 0.1 3.1 9.2 3.0 3.9 17.9 1.5 3.1 3.0 4.9 S Asia 3.12 97

Caucasus 3.2 3.2 Caucasus 3.22 19

S America 24.1 11.3 0.6 0.9 6.7 12.9 7.7 23.0 9.3 1.1 6.6 9.5 S America 6.30 203

Russia 35.9 24.1 22.2 13.4 23.8 21.7 13.6 0.7 8.7 24.1 12.7 18.3 Russia 17.57 91

Avg 7.5 6.9 2.8 2.4 9.8 3.7 3.9 13.8 3.1 3.2 2.8 4.4 Avg 3.16

Pairs 64 144 54 67 70 203 190 114 209 192 1990 Pairs

17

History - Throughput quality improvements from US

TCPBW < MSS/(RTT*sqrt(loss)) (1)

(1) Macroscopic Behavior of the TCP Congestion Avoidance Algorithm, Matthis, Semke, Mahdavi, Ott, Computer Communication Review 27(3), July 1997

80% annual improvement ~ factor 10/4yr

~Factor 100 improvement in 8 years

18

Detailed example of improvementsIncrease of bandwidth by factor of 460 in 6 years, more than kept pace - factor of 50 times improvement in loss

Note valleys when students on vacation

19

Summary - results• Internet A&R connectivity performance is improving

– RTT 10-20%/yr, loss 50%/yr, throughput 80%/yr– Reduced use of satellites, mainly use for new hard to get to areas

(e.g. S. Russian Republics)

• China, S.E. Europe, Russia rate of change keeps up but several years behind

• India, S. America performance is where N. America & W. Europe were 4 – 5 years ago

• Africa limited continuous results (UCT & Wits. no longer respond): Uganda losses in last 2 years reduced from10% to 3%, RTT fairly constant at 800ms.

• Improvements need constant investments to understand & improve

20

Summary - PingER• Lightweight (100bps/host pair, 21 pings/30mins per pair)• Very useful for inter-regional and poor links• Easy to deploy (uses ubiquitous Internet ping

infrastructure), however pings can be blocked• Easy to deploy for monitoring of sites in developing

countries– Remote sites ~ no effort (provide contact & host)

– Monitoring site small effort:1 day to download software set up & configure, (shared host) choose remote hosts to monitor, make data available for upload, check working, ongoing respond to emails.

– SLAC would be willing to assist

– Data public so anyone can do analysis/presentation of data

– Provide me (business card or email cottrell@slac.stanford.edu) with contact and name of host to be monitored

21

Help• Looking for better hosts to monitor & contacts in:

– Albania, Armenia, Austria, Azerbaijan

– Macedonia*, Turkey*, Yugoslavia

– Columbia*, Venezuela*, Cuba, Mexico*

– Pakistan*

– Africa (apart from Egypt, Uganda & South Africa, n.b. according to http://www3.sn.apc.org/africa/afrmain.htm all 54 countries in Africa now have Internet access in capitals)

– Note there are a few countries (about 5% of the world’s countries) that do not have full Internet connections and pay dearly by the byte.

• A couple of years ago these included: Afghanistan, Western Sahara, Christmas Island, S. Georgia, Marshall Islands, Myanmar, Montserrat, N. Korea, Pitcairn, St Vincente & Grenadines

22

More Information• IEPM/PingER home site:

– www-iepm.slac.stanford.edu/• What we do, coverage, how to download (free) software, requirements for

hosts, results, data download etc.

• Java demonstration from your computer– http://jas.freehep.org/demos/PingWorld/