Internet Monitoring and Results for the Digital Divide

Les CottrellSLAC, Aziz RehmatullahNIIT, Jerrod WilliamsSLAC, Akbar KhanNIIT

Presented at the Internationmal ICFA Workshop on Grid Activities Within Large Scale International Collaborations

Sinaia, Romania October 13-18, 2006 http://www.slac.stanford.edu/grp/scs/net/talk06/digital-divide-oct06-romania.ppt

Introduction• PingER project originally (1995) for measuring

network performance for US, Europe and Japanese HEP community

• Extended this century to measure Digital Divide

• Last year added monitoring sites in S. Africa, Pakistan & India

• Will report on network performance to these regions from US and Europe – trends, comparisons

• Plus early results within and between these regions

PingER Methodology

Internet

10 ping request packets each 30 mins

RemoteHost(typicallya server)

Monitoring host

>ping remhost

Ping response packets

Measure Round Trip Time & Loss

Data Repository @ SLAC

On

ce a Day

PingER coverage• ~120 countries (99%

world’s connected population), 35 monitor sites in 14 countries

• New monitoring sites in Cape Town, Rawalpindi, Bangalore

• Monitor 25 African countries, contain 83% African population

Finding Hosts• Best via contacts• Also use Google to provide hosts for country (eg .ly)

– Found 844 hosts => 702 unique names => 600 ping– 88 unique IP addresses– 6 in Libya according to Geo IP Tool www.geoiptool.com/

• Verified with TULIP geolocator– Locates hosts using RTT from multiple landmarks to

target– Also see Octant for US

www.cs.cornell.edu/~bwong/octant/

TULIP geolocator• www.slac.stanford.edu/comp/net/wan-mon/tulip/

– Java applet (needs Java Webstart)– Friendly client, easy vizualization– Need landmarks around world

LandmarksTarget

Enter target

Pings min/avg/max from landmarks to targets

Traceroutes from landmarks to targets

Minimum RTT from US

• Maps show increased coverage• Min RTT indicates best possible, i.e. no queuing• >600ms probably geo-stationary satellite• Between developed regions min-RTT dominated by distance

– Little improvement possible• Only a few places still using satellite, mainly Africa & Central Asia

– E. African Submarine System (EASSy)

2000 2006

Effect of Losses• Losses critical, cause multi-second timeouts

• Typically depend on a bad link, so ~distance independent

• > 4-6% video-conf irritating, non-native language speakers unable to communicate

• > 4-5% irritating for interactive telnet, X windows

• >2.5% VoIP annoying every 30 seconds or so

• Burst losses of > 1% slightly annoying for VoIP

Losses from SLAC to world

>=12%>=5% <12%

>=2.5% < 5%>=1% < 2.5%

< 1%

• # hosts monitored increased seven-fold• Increase in fraction with good loss

– Despite adding more hosts in developing world

Loss Improvement by Population

• Loss by country weighted by population of country

Unreachability from SLAC• All pings of a set fail ≡ unreachable• Shows fragility, ~ distance independent• Developed regions US, Canada, Europe, Oceania lead

– Factor of 10 improvement in 8 years• Africa, S. Asia followed by L. America worst off

World thruput seen from US

Behind Europe6 Yrs: Russia, Latin America 7 Yrs: Mid-East, SE Asia10 Yrs: South Asia11 Yrs: Cent. Asia12 Yrs: Africa

South Asia, Central Asia, and

Africa are in Danger of Falling

Even Farther Behind

Throughput ~1460Bytes /(RTT*sqrt(loss))

S. Asia & Africa from US• Data v. noisy but there are noticeable trends• Separate India & Pak: India may be gaining• Africa & Pakistan are falling behind

Compare to US residence• Sites in many countries have bandwidth< US residence

– “10 Meg is Here”, www.lightreading.com/document.asp?doc_id=104415

• Africa: $5000/Mb/mo– Ghana

$10K/mo/2Mbps– Togo

$450/mo/128Kbps

India to India• Monitoring host in Bangalore from Oct ’05

– Too early to tell much, also need more sites, have some good contacts

• 3 remote hosts (need to increase): – R&E sites in Mumbai, Pune & Hyderabad– Government site in AP

• Lot of difference between sites, Gov. site sees heavy congestion

0

50

100

150

200

250

300

350

400

9-O

ct

8-N

ov

8-D

ec

7-J

an

6-F

eb

RT

T m

s

Govt of. APHyderabad, APMumbai, Maharashtra

Average - Minimum RTT from Banglore

Pakistan to Pakistan• 3 monitoring sites in Islamabad/Rawalpindi

– NIIT via NTC, NIIT via Micronet, NTC (PERN supplier)– All monitor 7 Universities in ISB, Lahore, KHI, Peshawar

• Careful: many University sites have proxies in US & Europe

• Minimum RTTs: best NTC 6ms, NIIT/NTC 10ms - extra 4ms for last mile, NIIT/Micronet 60ms – slower links different routes

• Queuing = Avg(RTT)-Min(RTT) – NIIT/NTC heavily congested

• 200-400ms queuing

– Better when students holiday– NIIT/Micronet & NTC OK– Outages show fragility

Avg - Min RTT from 3 monitoring Sites in Pakistan to Pakistan

0

50

100

150

200

250

300

350

400

450

500

1-Dec 11-Dec 21-Dec 31-Dec 10-Jan 20-Jan 30-Jan

RT

T m

s

Median NIIT N2Median NTCMedian NIIT N4

The PingER Project:http://www-iepm.slac.stanford.edu/pinger/

NIIT

Holiday

Pakistan Network Fragility

NIIT/Micronet

NIIT/NTC

NTC

NIIT/NTC heavily congested

Other sites OK

NIIT outage

Remote host outages

Pakistan International fragility

• Infrastructure appears fragile• Losses to QEA & NIIT are 3-8% averaged over month

RT

T m

s

Los

s %

Feb05 Jul05Fiber cut off Karachi causes 12 day outage Jun-Jul ’05, Huge losses of confidence and business

Another fiber outage, this time of 3 hours!Power cable dug up by excavators of Karachi Water & Sewage Board

• Typically once a month losses go to 20%

Routing from S Africa

• Seen from ZA• Only Botswana & Zimbabwe are direct• Most go via Europe or USA• Wastes costly international bandwidth

Many systemic factors:Electricity, Import duties,Skills, disease915M people 14% world population, 2.2% of world Internet users

Satellites vs Terrestrial• Terrestrial links via SAT3 & SEAMEW (Mediterranean)• Terrestrial not available to all within countries

PingER min-RTT measurements fromS. African TENET monitoring station EASSy fibre for E. Africa

Will it share sorry experience of SAT3 for W. Africa?

2006

Between Regions• Red ellipses show

within region• Blue = min(RTT)• Red = min-avg RTT• India/Pak green

ellipses• ZA heavy congestion

– Botswana, Argentina, Madascar, Ghana, BF

• India better off than Pak

Overall (Aug 06)• ~ Sorted by Average throughput• Within region performance better (black ellipses)• Europe, N. America, E. Asia generally good• M. East, Oceania, S.E. Asia, L. America acceptable• Africa, C. Asia, S. Asia poor

UNDP Human Development Index (HDI)

• A long and healthy life, as measured by life expectancy at birth

• Knowledge, as measured by the adult literacy rate (with two-thirds weight) and the combined primary, secondary and tertiary gross enrolment ratio (with one-third weight)

• A decent standard of living, as measured by GDP per capita.

UNDP Technology

Achievement Index (TAI)

• Creation of technology (e.g. patents, royalties); • diffusion of recent innovations (Internet

hosts/capita, high & medium tech export); • Diffusion of old innovations (log phones/capita,

log of electric consumption/capita); • Human skills (years of schooling, enrollment in

tertiary level in science, math & engineering).

• Less coverage (50 countries vs. 96 HDI )

• Linear fit (both variables technology related)

• Better fit, fewer outliers

• S Asia better thruput vs TAI

Why does it matter: Science

• Scientists cannot collaborate as equal partners unless they have connectivity to share data, results, ideas etc.

• Distance education needs good communication for access to libraries, journals, educational materials, video, access to other teachers and researchers.

Why does it matter: Business•G8 specifically pledged support for African higher education and research by “Helping develop skilled professionals for Africa's private and public sectors, through supporting networks of excellence between African's and other countries' institutions of higher education and centres of excellence in science and technology institutions” G8 specifically pledged support for African higher education and research by “Helping develop skilled professionals for Africa's private and public sectors, through supporting networks of excellence between African's and other countries' institutions of higher education and centres of excellence in science and technology institutions”

•Saturating western markets•High growth IT markets: BRIC•NOT business as usual

– New business models– Distinct needs– Dearth of distribution channels

Traditional MNCBusiness Model >$20K per year

75 to 100 million people

Some MNCs

>$1,500 - 20K per year1.5 to 1.75 billion people

Local Firms

<$1,500 per year4 billion people

Future Opportunity?

Prahalad and Hart

Karen CoppockRDVP, Stanford

What can we do?• The worldwide science and education community is in a unique

position to facilitate persistent, non-threatening dialog and increased cooperation between nations that have often been at odds.

• Has a track record:– first permanent Internet connection to mainland China[1]; – initiating the "Silk Road" satellite system[2] to bring connectivity to central Asia; – upgrading connectivity to Brazil; leading the installation and demonstrating the

first 622 Mbps connection to India; – the efforts of the International Committee for Future Accelerators (ICFA)

Standing Committee on Inter-regional Connectivity (SCIC[3]);– and the free eJournals delivery service[4] of the Abdus Salam International

Centre for Theoretical Physics (ICTP) etc • [1] “Networking with China”, R. L. A. Cottrell, C. Granieri, L. Fan, R. Xu, Y. Karita,

CHEP04, Japan, also SLAC-PUB-6478, Aug 1994• [2] See http://www.silkproject.org/• [3] See http://cern.ch/icfa-scic/• [4] See http://www.ejds.org/

• Extend PingER coverage, contacts for more monitoring & remote sites, jerrodw@slac.stanford.edu, cottrell@slac.stanford.edu

Need contacts, can you help?• Need PingER monitoring sites in Africa (only have S. Africa)• Need PingER remote sites in:

– Africa:• All central African countries• E. Africa: Ethiopia, Somalia, Tanzania, Zimbabwe• N Africa: Libya• W Africa: Gambia, Guinea, Ivory Coast, Liberia, Sierra Leone, Togo• S Africa: Swaziland

– L America• Bolivia, Chile, Columbia, Costa Rica, Cuba, Honduras, Nicaragua, Paraguay

– Mid East• Iraq, Palestine, Syria

– SE Asia• Cambodia, Myanmar, Singapore, Thailand, Vietnam

– S. Asia• Bangladesh

• Need TULIP landmark/traceroute servers (especially in Africa):– www.slac.stanford.edu/comp/net/wan-mon/traceroute-srv.html – Email: cottrell@slac.stanford.edu

Conclusions• S. Asia and Africa ~ 10 years behind and falling

further behind creating a Digital Divide within a Digital Divide

• India appears better than Africa or Pakistan• Last mile problems, and network fragility• Decreasing use of satellites, still needed for many

remote countries in Africa and C. Asia– EASSy project will bring fibre to E. Africa

• Growth in # users 2000-2005 400% Africa, 4000% Pakistan networks not keeping up

• Need more sites in developing regions– Especially Africa, S. America

• Funding for operations would help (0.5 FTE)

More information• Acknowledgements:

– Harvey Newman and ICFA/SCIC for a raison d’etre, ICTP for contacts and education on Africa, NIIT/Pakistan, Maxim Grigoriev (FNAL), Warren Matthews (GATech) for ongoing code development for PingER, USAID MoST/Pakistan for development funding, SLAC for support for ongoing management/operations

• PingER– www-iepm.slac.stanford.edu/pinger

• Human Development– http://www.gapminder.org/

Prolog: Internet growth• Internet use, performance & coverage exploding

– > 1Billion users– In 2004 users in China 6 => 78Million– Traffic through Amsterdam increased fourfold in 2005– CERN-US connection 9.6kbsp ’85 to 10Gbits/s today– Typical backbone bandwidths (including transoceanic)

2.5 – 10Gbits for developed world

Prolog: New Technologies

• The transition to the use of "dense wavelength division multiplexing" (DWDM) to support multiple optical links on a single fiber has made these links increasingly affordable, and this has resulted in a substantially increased number of these links coming into service.

• At the end nodes the commoditization of Gigabit and 10 Gigabit Ethernet, new buses, and faster cpus are driving performance higher and costs lower.

Prolog: Developing world• The Global Ring Network for Advanced Applications Development

(GLORIAD[5]) project is providing high speed connectivity especially for Russia and China 10GBps around globe by Mar ’07);

• The Trans-Eurasia Information Network (TEIN2[6]) is improving the connectivity of the Asia Pacific region;

• The Latin America Cooperation of Advanced Networks (CLARA[7]) and the Western Hemisphere Research and Education Networks (WHREN[8]) Links Interconnecting Latin America (LILA) projects are bringing Gbits/s to Latin America;

• EUMEDConnect[9] is improving connectivity to the Mediterranean; • The East African Submarine System (EASSy[10]) is bringing fibre to

the E. coast of Africa; • Four Southern African National Research and Education Networks

(NRENS) in Kenya, Malawi, Mozambique, Rwanda and South Africa have come together to found the Ubuntunet[11] Alliance for Research and Education Networking with the goal of delivering Gigabits/s connectivity to their countries and the rest of the world.