HENP Networks and Grids for Global VOs: from Vision to Reality HENP Networks and Grids for Global VOs: from Vision to Reality Harvey B. Newman Harvey B

HENP HENP Networks and Grids for Networks and Grids for Global VOs: from Vision to Reality Global VOs: from Vision to Reality

Harvey B. NewmanHarvey B. Newman California Institute of TechnologyCalifornia Institute of Technology

GNEW2004GNEW2004March 15, 2004March 15, 2004

ICFA and Global Networks ICFA and Global Networks for HENPfor HENP

ICFA and Global Networks ICFA and Global Networks for HENPfor HENP

National and International Networks, with sufficient National and International Networks, with sufficient (rapidly increasing) capacity and seamless end-to-end (rapidly increasing) capacity and seamless end-to-end capability, are essential forcapability, are essential for The daily conduct of collaborative work in bothThe daily conduct of collaborative work in both

experiment and theory experiment and theory Detector development & construction Detector development & construction

on a global scale on a global scale Grid systems supporting analysis involving Grid systems supporting analysis involving

physicists in all world regions physicists in all world regions The conception, design and implementation of The conception, design and implementation of

next generation facilities as “global networks” next generation facilities as “global networks” ““Collaborations on this scale would never have been Collaborations on this scale would never have been

attempted, if they could not rely on excellent networks”attempted, if they could not rely on excellent networks”

The Challenges of Next Generation Science in the Information Age

The Challenges of Next Generation Science in the Information Age

Flagship Applications Flagship Applications High Energy & Nuclear Physics, AstroPhysics Sky Surveys:High Energy & Nuclear Physics, AstroPhysics Sky Surveys:

TByte to PByte “block” transfers at 1-10+ Gbps TByte to PByte “block” transfers at 1-10+ Gbps eVLBI:eVLBI: Many real time data streams at 1-10 Gbps Many real time data streams at 1-10 Gbps BioInformatics, Clinical Imaging:BioInformatics, Clinical Imaging: GByte images on demand GByte images on demand

HEP Data Example: HEP Data Example: From Petabytes in 2004, ~100 Petabytes by 2008, From Petabytes in 2004, ~100 Petabytes by 2008,

to ~1 Exabyte by ~2013-5. to ~1 Exabyte by ~2013-5. Provide results with rapid turnaround, coordinating Provide results with rapid turnaround, coordinating

large but limited computing and data handling resources,large but limited computing and data handling resources,over networks of varying capability in different world regionsover networks of varying capability in different world regions

Advanced integrated applications, such as Data Grids, Advanced integrated applications, such as Data Grids, rely on seamless operation of our LANs and WANsrely on seamless operation of our LANs and WANs With reliable, quantifiable high performanceWith reliable, quantifiable high performance

Petabytes of complex data explored and analyzed by Petabytes of complex data explored and analyzed by 1000s of globally dispersed scientists, in hundreds of teams1000s of globally dispersed scientists, in hundreds of teams

Four LHC Experiments: The Four LHC Experiments: The Petabyte to Exabyte Petabyte to Exabyte

ChallengeChallengeATLAS, CMS, ALICE, LHCBATLAS, CMS, ALICE, LHCB

Higgs + New particles; Quark-Gluon Plasma; CP ViolationHiggs + New particles; Quark-Gluon Plasma; CP Violation

Tens of PB 2008; To 1 EB by ~2015Tens of PB 2008; To 1 EB by ~2015 Hundreds of TFlops To PetaFlops Hundreds of TFlops To PetaFlops

6000+ Physicists & Engineers; 60+ Countries;

250 Institutions

LHC Data Grid HierarchyLHC Data Grid Hierarchy

Tier 1

Tier2 Center

Online System

CERN Center PBs of Disk;

Tape Robot

FNAL CenterIN2P3 Center INFN Center RAL Center

InstituteInstituteInstituteInstitute

Workstations

~100-1500 MBytes/sec

2.5-10 Gbps

0.1 to 10 GbpsTens of Petabytes by 2007-8.An Exabyte ~5-7 Years later.Physics data cache

~PByte/sec

~10 Gbps

Tier2 CenterTier2 CenterTier2 Center

~2.5-10 Gbps

Tier 0 +1

Tier 3

Tier 4

Tier2 Center Tier 2

Experiment

CERN/Outside Resource Ratio ~1:2Tier0/( Tier1)/( Tier2) ~1:1:1

Emerging Vision: A Richly Structured, Global Dynamic System

ICFA Standing Committee on Interregional Connectivity (SCIC)

ICFA Standing Committee on Interregional Connectivity (SCIC)

Created by ICFA in July 1998 in VancouverCreated by ICFA in July 1998 in Vancouver CHARGE: CHARGE:

Make recommendations to ICFA concerning the connectivity Make recommendations to ICFA concerning the connectivity between between the Americasthe Americas, Asia and Europe, Asia and Europe As part of the process of developing theseAs part of the process of developing these

recommendations, the committee should recommendations, the committee should Monitor traffic Monitor traffic Keep track of technology developmentsKeep track of technology developments Periodically review forecasts of future Periodically review forecasts of future

bandwidth needs, and bandwidth needs, and Provide early warning of potential problemsProvide early warning of potential problems

Representatives: Major labs, ECFA, ACFA, North Representatives: Major labs, ECFA, ACFA, North and South American Physics Community and South American Physics Community

SCIC in 2003-2004 http://cern.ch/icfa-scic SCIC in 2003-2004 http://cern.ch/icfa-scic

Strong Focus on the Digital Divide Since 2002Strong Focus on the Digital Divide Since 2002

Three 2004 Reports; Presented to ICFA Feb. 13Three 2004 Reports; Presented to ICFA Feb. 13

Main Report: “Networking for HENP”Main Report: “Networking for HENP” [H. Newman et al.] [H. Newman et al.] Includes Brief Updates on Monitoring, the Digital Divide Includes Brief Updates on Monitoring, the Digital Divide

and Advanced Technologies [*]and Advanced Technologies [*] A World Network Overview (with 27 Appendices): A World Network Overview (with 27 Appendices):

Status and Plans for the Next Few YearsStatus and Plans for the Next Few Years of National and Regional Networks, of National and Regional Networks, and Optical Network Initiatives and Optical Network Initiatives

Monitoring Working Group ReportMonitoring Working Group Report [L. Cottrell] [L. Cottrell] Digital Divide in RussiaDigital Divide in Russia [V. Ilyin] [V. Ilyin]

[*] [*] Also See the 2003 SCIC Reports of the Advanced Also See the 2003 SCIC Reports of the Advanced Technologies and Digital Divide Working GroupsTechnologies and Digital Divide Working Groups

ICFA Report: Networks for HENPICFA Report: Networks for HENPGeneral Conclusions (1)General Conclusions (1)


Bandwidth Usage Continues to Grow by 80-100% Per YearBandwidth Usage Continues to Grow by 80-100% Per Year Current generation of 2.5-10 Gbps backbones and major Int’l links Current generation of 2.5-10 Gbps backbones and major Int’l links

used by HENP arrived in the last 2 Years [US+Europe+Japan+Korea]used by HENP arrived in the last 2 Years [US+Europe+Japan+Korea] Capability Increased from ~4 to several hundred times, Capability Increased from ~4 to several hundred times,

i.e. much faster than Moore’s Law i.e. much faster than Moore’s Law This is a direct result of the continued precipitous fall This is a direct result of the continued precipitous fall

of network prices for 2.5 or 10 Gbps links in these regions of network prices for 2.5 or 10 Gbps links in these regions Technological progress may drive BW higher, unit price lowerTechnological progress may drive BW higher, unit price lower

More wavelengths on a fiber; Cheap, widespread Gbit EthernetMore wavelengths on a fiber; Cheap, widespread Gbit Ethernet Grids may accelerate this growth, and the demand for Grids may accelerate this growth, and the demand for

seamless high performanceseamless high performance Some regions are moving to owned or leased dark fiber Some regions are moving to owned or leased dark fiber The rapid rate of progress is confined mostly to the US, Europe, The rapid rate of progress is confined mostly to the US, Europe,

Japan , Korea, and the major TransAtlantic and Pacific routes Japan , Korea, and the major TransAtlantic and Pacific routes This may worsen the problem of the This may worsen the problem of the Digital DivideDigital Divide

History of Bandwidth Usage – One Large Network; One Large Research Site

History of Bandwidth Usage – One Large Network; One Large Research Site

ESnet Monthly Accepted Traffic 1/90-1/04

0

50

100

150

200

250

300

Jan,

90Oc

t, 90

Jul,9

1Ap

r, 92

Jan,

93Oc

t,93

Jul, 9

4Ap

r, 95

Jan,

96Oc

t, 96

Jul, 9

7Ap

r, 98

Jan,

99Oc

t, 99

Jul, 0

0Ap

r, 01

Jan,

02Oc

t, 02

Jul, 0

3

TByte

/Mon

th

SLAC Traffic ~300 Mbps; ESnet Limit

Growth in Steps: ~ 10X/4 YearsProjected: ~2 Terabits/s by

~2014

ESnet Accepted Traffic 1/90 – 1/04Exponential Growth Since ’92;

Annual Rate Increased from 1.7 to 2.0X Per Year In the Last 5 Years

Internet Growth in the World At Large

Internet Growth in the World At Large

Derived throughputs in kbits/s for Aug '03August-03 AU CA EDU GOV CH DE DK HU IT UK JP NET ORG RU SU Avg

N America 234 716 9346 1993 322 344 305 329 337 405 197 3231 4121 278 148 1487Australasia 7833 72 258 224 134 132 164 279 194 96 939Balkans 149 233 276 365 1111 1544 939 4745 1093 1148 155 321 351 670 346 896Europe 148 234 316 380 1662 1714 1049 1840 1596 2538 164 390 367 720 298 895Baltics 126 183 265 331 644 1009 1693 851 896 804 149 298 313 1050 213 588Russia 129 170 220 225 508 593 912 545 587 561 321 285 244 809 768 458E Asia 142 196 219 209 143 163 157 152 147 181 3559 205 219 163 157 401M East 125 187 168 234 227 505 386 453 645 572 125 243 283 331 185 311L America 131 150 220 191 165 162 152 159 164 175 117 198 186 139 100 161S Asia 73 179 112 121Caucasus 87 87Central Asia 64 68 66Africa 40 56 27 44 32 31 38Avg 1002 205 898 418 455 754 699 1023 563 727 598 548 698 520 257 496

Amsterdam Internet Exchange Point Example

75-100% Growth Per Year

10 G

20 G

Avg

5 Minute

Max

Some Growth Spurts;Typically In Summer-

Fall

The Rate of HENP Network Usage Growth (~100% Per Year) is Similar to the World at Large

Bandwidth Growth of Int’l HENP Bandwidth Growth of Int’l HENP Networks (US-CERN Example)Networks (US-CERN Example)

Bandwidth Growth of Int’l HENP Bandwidth Growth of Int’l HENP Networks (US-CERN Example)Networks (US-CERN Example)

Rate of Progress >> Moore’s Law. Rate of Progress >> Moore’s Law. (US-CERN Example)(US-CERN Example) 9.6 kbps Analog9.6 kbps Analog (1985) (1985) 64-256 kbps Digital (1989 - 1994) [X 7 – 27]64-256 kbps Digital (1989 - 1994) [X 7 – 27] 1.5 Mbps Shared (1990-3; IBM) [X 160]1.5 Mbps Shared (1990-3; IBM) [X 160] 2 -4 Mbps2 -4 Mbps (1996-1998) [X 200-400] (1996-1998) [X 200-400] 12-20 Mbps (1999-2000) [X 1.2k-2k]12-20 Mbps (1999-2000) [X 1.2k-2k] 155-310 Mbps155-310 Mbps (2001-2) [X 16k – 32k] (2001-2) [X 16k – 32k] 622 Mbps622 Mbps (2002-3) [X 65k] (2002-3) [X 65k] 2.5 Gbps 2.5 Gbps (2003-4) [X 250k] (2003-4) [X 250k] 10 Gbps 10 Gbps (2005) [X 1M] (2005) [X 1M]

A factor of ~1M over a period of 1985-2005 A factor of ~1M over a period of 1985-2005 (a factor of ~5k during 1995-2005)(a factor of ~5k during 1995-2005)

HENP has become a leading applications driver, HENP has become a leading applications driver, and also a co-developer of global networks; and also a co-developer of global networks;

Note 10 Gbps

Connections to Poland,

Czech Republic, Hungary

Pan-European Multi-Gigabit Backbone (33 Countries)

February 2004

Planning Underway for “GEANT2” (GN2), Multi-Lambda Backbone, to Start In 2005

Core Capacity on Western European NRENs 2001-2003

15 European NRENs have made a step up to 1, 2.5 or 10 Gbps core capacity in the last 3

years

100M

1G

10G

Log Scale

TERENA Compendiumwww.terena.nl

Int’l Circuit ~ 5-10 GbpsDomestic Circuit 30 – 100 Mbps

SuperSINET 10 Gbps

SuperSINET 10 Gbps IP; Tagged VPNs Additional 1 GbE Inter-University Waves For HEP 4 X 2.5 Gb to NY; 10 GbE Peerings: to Abilene, ESnet and GEANT

SuperSINET in JAPAN: Updated Map Oct. 2003SuperSINET in JAPAN: Updated Map Oct. 2003

SURFNet5 in the Netherlands Fully Optical 10G IP NetworkSURFNet5 in the Netherlands Fully Optical 10G IP Network

Fully dual stack:Fully dual stack: IPv4 + IPv6 IPv4 + IPv6

65% of Customer 65% of Customer Base Connects Base Connects with Gigabit with Gigabit EthernetEthernet

Germany: 2003, 2004, 2005Germany: 2003, 2004, 2005

GWIN Connects 550 Universities, Labs, Other InstitutionsGWIN Connects 550 Universities, Labs, Other Institutions

GWIN: Q4/03GWIN: Q4/03 GWIN: Q4/04GWIN: Q4/04PlanPlan

XWIN: Q4/05XWIN: Q4/05((Dark Fiber Option)Dark Fiber Option)

Abilene - Upgrade Completed!Abilene - Upgrade Completed!

1660 km of Dark Fiber CWDM Links, up to 112 km.

1 to 4 Gbps (GbE)

August 2002: First NREN in Europe to establish Int’l GbE Dark Fiber Link, to Austria

April 2003 to Czech Republic.

Planning 10 Gbps Backbone; dark fiber link to Poland this year.

PROGRESS in SE Europe (Sk, Pl, Cz, Hu, …)

Romania: Improved to 34 to 155 Mbps in 2003; GEANT-Bucharest Link Improved: 155 to 622 Mbps.

Note: Inter-City Links Were Only 2 to 6 Mbps in 2002

Romania: Improved to 34 to 155 Mbps in 2003; GEANT-Bucharest Link Improved: 155 to 622 Mbps.

Note: Inter-City Links Were Only 2 to 6 Mbps in 2002

Timişoara

GEANT connection

RoEduNet2004

Plans for IntercityDark Fiber Backbone

Australia (AARnet): SXTransport Project in 2004

Australia (AARnet): SXTransport Project in 2004

Connect Major Australian Universities to 10 Gbps BackboneConnect Major Australian Universities to 10 Gbps Backbone Two 10 Gbps Research Links to the USTwo 10 Gbps Research Links to the US Aarnet/USLIC Collaboration on Net R&D Starting SoonAarnet/USLIC Collaboration on Net R&D Starting Soon

Connect Telescopesin Australia, Hawaii (Muana

Kea)

GLORIAD: Global Optical Ring (US-Russia-China)

GLORIAD: Global Optical Ring (US-Russia-China)

“ “Little Gloriad” (OC3) Launched January 12; to OC192 Little Gloriad” (OC3) Launched January 12; to OC192

Also Important for Intra-Russia Connectivity;Education and Outreach

ITER Distributed Ops.;Fusion-HEP Cooperation

National Lambda Rail (NLR)National Lambda Rail (NLR)

15808 Terminal, Regen or OADM siteFiber route

NLRComing Up Now; Initially 4 10G

WavelengthsFull Footprint

by ~4Q04 Internet2 HOPI Initiative (w/HEP) To 40 10G Waves in Future

Transition beginning now to optical, multi-wavelength Community owned or leased

fiber networks for R&E

PITPIT

PORPOR

FREFRE

RALRAL

WALWAL

NASNASPHOPHO

OLGOLGATLATL

CHICHI

CLECLE

KANKAN

OGDOGDSACSAC BOSBOSNYCNYC

WDCWDC

STRSTR

DALDAL

DENDEN

LAXLAX

SVLSVL

SEASEA

SDGSDG

JACJAC

18 State Dark Fiber Initiatives

In the U.S. (As of 3/04)

California (CALREN), Colorado (FRGP/BRAN)Connecticut Educ. Network,Florida Lambda Rail, Indiana (I-LIGHT), Illinois (I-WIRE), Md./DC/No. Virginia (MAX),Michigan, Minnesota, NY + New England (NEREN), N. Carolina (NC LambdaRail), Ohio (Third Frontier Net) Oregon, Rhode Island (OSHEAN), SURA Crossroads (SE U.S.), Texas,Utah, Wisconsin

FiberCO

The Move to Dark Fiber is Spreading

CA*net4 (Canada)Two 10G Waves Vancouver- Halifax

CA*net4 (Canada)Two 10G Waves Vancouver- Halifax

Interconnects Interconnects Regional Nets at 10GRegional Nets at 10G

Acts as Parallel Acts as Parallel Discipline-Oriented Discipline-Oriented Nets; 650 SitesNets; 650 Sites

Connects to US Nets Connects to US Nets at Seattle, Chicago at Seattle, Chicago and NYCand NYC

Third Nat’l Lambda Third Nat’l Lambda Later in 2004Later in 2004

User Controlled User Controlled Light Path SoftwareLight Path Software(UCLP) for “Lambda(UCLP) for “LambdaGrids”Grids”

Calgary

ReginaWinnipeg

OttawaMontreal

Toronto

Halifax

St. John’s

Fredericton

Charlottetown

Chicago

Seattle

New York

Edmonton

Saskatoon

Vancouver

Calgary

ReginaWinnipeg

OttawaMontreal

Toronto

Halifax

St. John’s

Fredericton

Charlottetown

Chicago

Seattle

New York

Edmonton

Saskatoon

Vancouver

SURFNet6 in the Netherlands 3000 km of Owned Dark FiberSURFNet6 in the Netherlands

3000 km of Owned Dark Fiber

40M Euro Project40M Euro ProjectScheduled Scheduled

Start Mid-2005;Start Mid-2005;Support Hybrid GridsSupport Hybrid Grids

* 9/01 105 Mbps 30 Streams: SLAC-IN2P3; 102 Mbps 1 Stream CIT-CERN9/01 105 Mbps 30 Streams: SLAC-IN2P3; 102 Mbps 1 Stream CIT-CERN 5/20/02 450-600 Mbps SLAC-Manchester on OC12 with ~100 Streams5/20/02 450-600 Mbps SLAC-Manchester on OC12 with ~100 Streams 6/1/02 290 Mbps Chicago-CERN One Stream on OC12 6/1/02 290 Mbps Chicago-CERN One Stream on OC12 9/02 850, 1350, 1900 Mbps Chicago-CERN 1,2,3 GbE Streams, 2.5G Link9/02 850, 1350, 1900 Mbps Chicago-CERN 1,2,3 GbE Streams, 2.5G Link 11/02 [LSR] 930 Mbps in 1 Stream California-CERN, and California-AMS11/02 [LSR] 930 Mbps in 1 Stream California-CERN, and California-AMS FAST TCP 9.4 Gbps in 10 Flows California-ChicagoFAST TCP 9.4 Gbps in 10 Flows California-Chicago 2/03 [LSR] 2.38 Gbps in 1 Stream California-Geneva (99% Link Use)2/03 [LSR] 2.38 Gbps in 1 Stream California-Geneva (99% Link Use) 5/03 [LSR] 0.94 Gbps IPv6 in 1 Stream Chicago- Geneva5/03 [LSR] 0.94 Gbps IPv6 in 1 Stream Chicago- Geneva TW & SC2003 [LSR]: 5.65 Gbps (IPv4), 4.0 Gbps (IPv6) GVA-PHX (11 kkm)TW & SC2003 [LSR]: 5.65 Gbps (IPv4), 4.0 Gbps (IPv6) GVA-PHX (11 kkm) 3/043/04 [LSR] 6.25 Gbps (IPv4) in 8 Streams LA-CERN [LSR] 6.25 Gbps (IPv4) in 8 Streams LA-CERN

HEP is Learning How to Use Gbps Networks Fully: Factor of ~500 Gain in Max. Sustained TCP Thruput

in 4 Years, On Some US+Transoceanic Routes

HEP is Learning How to Use Gbps Networks Fully: Factor of ~500 Gain in Max. Sustained TCP Thruput

in 4 Years, On Some US+Transoceanic Routes

Transatlantic Ultraspeed TCP TransfersTransatlantic Ultraspeed TCP TransfersThroughput Achieved: X50 in 2 yearsThroughput Achieved: X50 in 2 years

Juniper, HP

Level(3)Telehous

e

Terabyte Transfers by the Caltech-CERN Team: Across Abilene (Internet2) Chicago-LA,

Sharing with normal network traffic Oct 15: 5.64 Gbps IPv4 Palexpo-L.A. (10.9 kkm)

Peaceful Coexistence with a Joint Internet2-Telecom World VRVS Videoconference

Nov 18: 4.00 Gbps IPv6 Geneva-Phoenix (11.5 kkm)March 2004: 6.25 Gbps in 8 Streams (S2IO Interfaces)

Nov 19: 23+ Gbps TCP: Caltech, Nov 19: 23+ Gbps TCP: Caltech, SLAC, CERN, LANL, UvA, ManchesterSLAC, CERN, LANL, UvA, Manchester

http://www.cern.ch/

HENP Major Links: Bandwidth Roadmap in Gbps

HENP Major Links: Bandwidth Roadmap in Gbps

Year Production Experimental Remarks

2001 0.155 0.622-2.5 SONET/SDH

2002 0.622 2.5 SONET/SDH DWDM; GigE Integ.

2003 2.5 10 DWDM; 1 + 10 GigE Integration

2005 10 2-4 X 10 Switch; Provisioning

2007 2-4 X 10 ~10 X 10; 40 Gbps

1st Gen. Grids

2009 ~10 X 10 or 1-2 X 40

~5 X 40 or ~20-50 X 10

40 Gbps Switching

2011 ~5 X 40 or

~20 X 10

~25 X 40 or ~100 X 10

2nd Gen Grids Terabit Networks

2013 ~Terabit ~MultiTbps ~Fill One Fiber

Continuing the Trend: ~1000 Times Bandwidth Growth Per Decade;A new DOE Science Network Roadmap: Compatible

HENP Lambda Grids:Fibers for Physics

HENP Lambda Grids:Fibers for Physics

Problem: Extract “Small” Data Subsets of 1 to 100 Terabytes Problem: Extract “Small” Data Subsets of 1 to 100 Terabytes from 1 to 1000 Petabyte Data Storesfrom 1 to 1000 Petabyte Data Stores

Survivability of the HENP Global Grid System, with Survivability of the HENP Global Grid System, with hundreds of such transactions per day (circa 2007-8)hundreds of such transactions per day (circa 2007-8)requires that each transaction be completed in a requires that each transaction be completed in a relatively short time. relatively short time.

Example: Take 800 secs to complete the transaction. ThenExample: Take 800 secs to complete the transaction. Then Transaction Size (TB)Transaction Size (TB) Net Throughput (Gbps)Net Throughput (Gbps) 1 101 10 10 10010 100 100 1000 (Capacity of 100 1000 (Capacity of

Fiber Today) Fiber Today) Summary: Providing Switching of 10 Gbps wavelengthsSummary: Providing Switching of 10 Gbps wavelengths

within ~2-4 years; and Terabit Switching within 5-8 years within ~2-4 years; and Terabit Switching within 5-8 years would enable “10G Lambda Grids with Terabyte transactions”,would enable “10G Lambda Grids with Terabyte transactions”,to fully realize the discovery potential of major HENP programs, to fully realize the discovery potential of major HENP programs, as well as other data-intensive research.as well as other data-intensive research.



Reliable high Reliable high End-to-end PerformanceEnd-to-end Performance of networked applications such as of networked applications such as large file transfers and Data Grids is required. Achieving this requires:large file transfers and Data Grids is required. Achieving this requires:

Removing local, last mile, and nat’l and int’l bottlenecks Removing local, last mile, and nat’l and int’l bottlenecks end-to-end, whether technical or political in origin. end-to-end, whether technical or political in origin.While National and International backbones have reached 2.5 to 10 Gbps While National and International backbones have reached 2.5 to 10 Gbps speeds in many countries, the bandwidths speeds in many countries, the bandwidths across borders, the countrysideacross borders, the countrysideor the city may be much less. or the city may be much less. This problem is very widespread in our community, with examples This problem is very widespread in our community, with examples stretching from China to South America to the Northeastern U.S. stretching from China to South America to the Northeastern U.S. Root causes for this vary, from lack of local infrastructure to Root causes for this vary, from lack of local infrastructure to unfavorable pricing policies. unfavorable pricing policies.

Upgrading campus infrastructures.Upgrading campus infrastructures. These are still not designed to support Gbps data transfers in most of HEP These are still not designed to support Gbps data transfers in most of HEP centers. One reason for the under-utilization of National and International centers. One reason for the under-utilization of National and International backbones, is the lack of bandwidth to groups of end-users inside the campus. backbones, is the lack of bandwidth to groups of end-users inside the campus.

End-to-end monitoring extending to all regions serving our community.End-to-end monitoring extending to all regions serving our community. A coherent approach to monitoring that allows physicists throughout A coherent approach to monitoring that allows physicists throughout our community to extract clear, unambiguous and inclusive information our community to extract clear, unambiguous and inclusive information is a prerequisite for this. is a prerequisite for this.



We must Remove Firewall Bottlenecks We must Remove Firewall Bottlenecks [Also at some Major HEP Labs][Also at some Major HEP Labs]

Firewall systems are so far behind the needsFirewall systems are so far behind the needs that they that they won’t match the data flow of Grid applications. The maximum won’t match the data flow of Grid applications. The maximum throughput measured across available products is limited to a throughput measured across available products is limited to a few X 100 Mbps !few X 100 Mbps !

It is urgent to address this issue by designing new architectures It is urgent to address this issue by designing new architectures that eliminate/alleviate the need for conventional firewallsthat eliminate/alleviate the need for conventional firewalls. For . For example, Point-to-point provisioned high-speed circuits as example, Point-to-point provisioned high-speed circuits as proposed by emerging Light Path technologies could remove proposed by emerging Light Path technologies could remove the bottleneck. the bottleneck.

With endpoint authentication [as in Grid AAA systems],With endpoint authentication [as in Grid AAA systems], for example, for example, the point-to-point paths are private, intrusion resistant circuits, the point-to-point paths are private, intrusion resistant circuits, so they should be able to bypass firewalls if the endpoint sitesso they should be able to bypass firewalls if the endpoint sites trust each other. trust each other.

The original Computational and Data Grid concepts are The original Computational and Data Grid concepts are largely stateless, open systems: known to be scalable largely stateless, open systems: known to be scalable

Analogous to the WebAnalogous to the WebThe classical Grid architecture had a number of implicit The classical Grid architecture had a number of implicit

assumptionsassumptions The ability to locate and schedule suitable resources,The ability to locate and schedule suitable resources,

within a tolerably short time (i.e. resource richness) within a tolerably short time (i.e. resource richness) Short transactions with relatively simple failure modesShort transactions with relatively simple failure modes

HENP Grids are HENP Grids are Data Intensive & Resource-ConstrainedData Intensive & Resource-Constrained 1000s of users competing for resources at dozens of sites1000s of users competing for resources at dozens of sites Resource usage governed by local and global policiesResource usage governed by local and global policies Long transactions; some long queuesLong transactions; some long queues

HENP HENP Stateful, End-to-end Monitored and Tracked ParadigmStateful, End-to-end Monitored and Tracked Paradigm Adopted in OGSA [Now WS Resource Framework]Adopted in OGSA [Now WS Resource Framework]

HENP Data Grids, and Now HENP Data Grids, and Now Services-Oriented GridsServices-Oriented Grids

The Move to OGSA and then The Move to OGSA and then Managed Integration SystemsManaged Integration Systems

Incr

ease

d f

un

ctio

nal

ity,

stan

dar

diz

atio

n

Time

Customsolutions

Open GridServices Arch

GGF: OGSI, …(+ OASIS, W3C)

Multiple implementations,including Globus Toolkit

Web services + …

Globus Toolkit

Defacto standardsGGF: GridFTP, GSI

X.509,LDAP,FTP, …

App-specificServices~Integrated Systems~Integrated Systems

Stateful; ManagedWeb Services

Resrc Framwk

Managing Global Systems: Dynamic Managing Global Systems: Dynamic Scalable Services ArchitectureScalable Services Architecture

MonALISA: http://monalisa.cacr.caltech.edu

““Station Server” Station Server” Services-enginesServices-engines at sites host many at sites host many“Dynamic Services”“Dynamic Services”Scales to Scales to

thousands of thousands of service-Instancesservice-Instances

Servers autodiscover Servers autodiscover and interconnect and interconnect dynamically to form dynamically to form a robust fabric a robust fabric

Autonomous agentsAutonomous agents

42Caltech GAE Team

Grid Analysis EnvironmentGrid Analysis Environment

Analysis Clients talk standard protocols to the “Grid Services Web Server”, a.k.a. the Clarens data/services portal, with a simple Web service API

The secure Clarens portal hides the complexity of the Grid Services from the client

Key features: Global Scheduler, Catalogs, Monitoring, and Grid-wide Execution service;Clarens servers forma Global Peer to peer Network

SchedulerCatalogs

Analysis Client

Grid ServicesWeb Server

ExecutionPriority

Manager

Grid WideExecutionService

DataManagement

Fully-ConcretePlanner

Fully-AbstractPlanner

Analysis Client

AnalysisClient

Virtual Data

ReplicaApplicationsMonitoring

Partially-AbstractPlanner

Metadata

HTTP, SOAP, XML/RPC

CLARENS: Web Services Architecture

Integrated hybrid experimental network, leveraging Transatlantic Integrated hybrid experimental network, leveraging Transatlantic R&D network partnerships; packet-switched + dynamic optical paths R&D network partnerships; packet-switched + dynamic optical paths 10 GbE across US and the Atlantic: NLR, DataTAG, TransLight, 10 GbE across US and the Atlantic: NLR, DataTAG, TransLight,

NetherLight, UKLight, etc.; Extensions to Japan, Taiwan, BrazilNetherLight, UKLight, etc.; Extensions to Japan, Taiwan, Brazil End-to-end monitoring; Realtime tracking and optimization; End-to-end monitoring; Realtime tracking and optimization;

Dynamic bandwidth provisioningDynamic bandwidth provisioning Agent-based services spanning all layers of the system, from the Agent-based services spanning all layers of the system, from the

optical cross-connects to the applications.optical cross-connects to the applications.

National Lambda RailNational Lambda Rail

UltraLight Collaboration:UltraLight Collaboration:http://ultralight.caltech.eduhttp://ultralight.caltech.edu

Caltech, UF, FIU, UMich, SLAC,FNAL,MIT/Haystack,CERN, UERJ(Rio), NLR, CENIC, UCAID,Translight, UKLight, Netherlight, UvA, UCLondon, KEK, Taiwan

Cisco, Level(3)

Flagship Applications(HENP, VLBI, Oncology, …)

Grid/Storage Management

Network Protocols &Bandwidth Management

En

d-t

o-e

nd

Mo

nit

ori

ng

Inte

llig

en

t A

ge

nts

En

d-t

o-e

nd

Mo

nit

ori

ng

Inte

llig

en

t A

ge

nts

Application Frameworks

Distributed CPU & Storage

Network Fabric

Grid Middleware

Flagship Applications(HENP, VLBI, Oncology, …)

Grid/Storage Management

Network Protocols &Bandwidth Management

En

d-t

o-e

nd

Mo

nit

ori

ng

Inte

llig

en

t A

ge

nts

En

d-t

o-e

nd

Mo

nit

ori

ng

Inte

llig

en

t A

ge

nts

Application Frameworks

Distributed CPU & Storage

Network Fabric

Grid Middleware

PITPIT

PORPOR

FREFRE

RALRAL

WALWAL

NASNASPHOPHO

OLGOLG ATLATL

CHICHICLECLE

KANKAN

OGDOGDSACSAC NYCNYC

WDCWDC

STRSTR

DALDAL

DENDEN

LAXLAX

SVLSVL

SEASEA

SDGSDG

JACJAC

GLIF: Global Lambda Integrated Facility

DWDM SURFnet

lambda service path

IP service path

10 Gbit/s

SURFnet10 Gbit/s

SURFnet10 Gbit/s

IEEAF10 Gbit/s

DwingelooASTRON/JIVE

DwingelooASTRON/JIVE

PragueCzechLight

PragueCzechLight

2.5 Gbit/s

NSF10 Gbit/s

LondonUKLightLondonUKLight

StockholmNorthernLightStockholm

NorthernLight

2.5 Gbit/s

New YorkMANLANNew YorkMANLAN

TokyoWIDETokyoWIDE

10 Gbit/s

10Gbit/s

10 Gbit/s

2x10 Gbit/s

IEEAF10 Gbit/s

2x10 Gbit/s

10 Gbit/s

2.5 Gbit/s

2.5 Gbit/s

TokyoAPANTokyoAPAN

GenevaCERN

GenevaCERN

ChicagoChicago AmsterdamAmsterdam

“GLIF is a World Scale Lambda based Lab for Application & Middleware development, where Grid applications ride on dynamically configured networks based on optical wavelengths ...

GLIF will use the Lambda network to support data transport for the most demanding e-Science applications, concurrent with the normal best effort Internet for commodity traffic.”

10 Gbps Wavelengths For R&E Network Development Are

Proliferating, Across Continents and Oceans

SCIC Report 2004 The Digital DivideSCIC Report 2004 The Digital Divide

As the pace of network advances continues to accelerate, As the pace of network advances continues to accelerate, the gap between the economically “favored” regions and the gap between the economically “favored” regions and the rest of the world is in danger of widening. the rest of the world is in danger of widening.

We must therefore workWe must therefore work to Close the Digital Divideto Close the Digital Divide To make Physicists from All World Regions Full Partners To make Physicists from All World Regions Full Partners

in Their Experiments; and in the Process of Discoveryin Their Experiments; and in the Process of Discovery This is essential for the health of our global This is essential for the health of our global

experimental collaborations, our plans for future experimental collaborations, our plans for future projects, and our field.projects, and our field.

SCIC Monitoring WG PingER (Also IEPM-BW) SCIC Monitoring WG PingER (Also IEPM-BW)

Measurements fromMeasurements from 33 monitors in 12 countries33 monitors in 12 countries 850 remote hosts in 100 850 remote hosts in 100

Countries; 3700 monitor-Countries; 3700 monitor-remote site pairsremote site pairs

Measurements go back to ‘95Measurements go back to ‘95 Reports on link reliability, Reports on link reliability,

qualityquality Aggregation in Aggregation in

“affinity groups”“affinity groups”

Countries monitoredCountries monitored Contain 78% of Contain 78% of

world populationworld population 99% of Internet users 99% of Internet users

Monitoring Sites

Affinity Groups (Countries)

Anglo America (2), Latin America (14), Europe (24), S.E. Europe (9), Africa (21), Mid East (7), Caucasus (3), Central Asia (8), Russia

includes Belarus & Ukraine (3), S. Asia (7), China (1) and Australasia (2).

SCIC Monitoring WG - Throughput Improvements 1995-2004

Bandwidth of TCP < MSS/(RTT*Sqrt(Loss)) (1)

(1) 1) Matthis et al., Computer Communication Review 27(3), July 1997Matthis et al., Computer Communication Review 27(3), July 1997

60% annual improvement

Factor ~100/10 yr

Progress: but Digital Divide is Mostly Maintained

Some Regions ~5-10 Years

Behind

SE Europe, Russia, Central

Asia May be Catching Up

(Slowly);

India Ever-Farther Behind

APAN Link Information 2004.1.05 [email protected] Network Bandwidth(Mbps) AUP/Remark

AU-US AARNet 310 to 2 X 10 Gbps R&E + CommodityCN-HK CERNET/HARNET 2 R&ECN-HK CSTNET 155 R&ECN-JP CERNET 10 R&ECN-JP CERNET 45 Native IPv6CN-UK CERNET 45 R&ECN-US CERNET 10 Research CN-US CERNET 200 R&ECN-US CSTNET 155 R&EHK-US HARNET 45 R&EHK-TW HARNET/TANET 10 R&EJP-ID AI3(ITB) 2/1.5 R&EJP-KR APII 2Gbps R&EJP-MY AI3(USM) 1.5/0.5 R&EJP-PH AI3(ASTI) 1.5/0.5 R&EJP-PH MAFFIN 2 Research JP-SG AI3(SICU) 1.5/0.5 R&EJP-TH AI3(AIT) 1.5/0.5 R&EJP-TH SINET(ThaiSarn) 2 R&EJP-US TransPac 10 Gbps R&EJP-VN AI3(IOIT) 1.5/0.5 R&E

Inhomogeneous Bandwidth Distributioin Latin America. CAESAR Report (6/02)Inhomogeneous Bandwidth Distributioin Latin America. CAESAR Report (6/02)

Int’l Links0.071 Gbps

Used

4,236 Gbps Capacity to Latin America

Need to Pay Attentionto End-point connections

DAI: State of the WorldDAI: State of the World


Digital Access Index Top Ten +Digital Access Index Top Ten +

Pakistan 0.03 0.54 0.41 0.2 0.01 0.24

Work on the Digital Dividefrom Several PerspectivesWork on the Digital Dividefrom Several Perspectives

Work on Policies and/or Pricing: pk, in, br, cn, SE Europe, …Work on Policies and/or Pricing: pk, in, br, cn, SE Europe, … Share Information: Comparative Performance and BW Pricing Share Information: Comparative Performance and BW Pricing Exploit Model Cases: e.g. Poland, Slovakia, Czech RepublicExploit Model Cases: e.g. Poland, Slovakia, Czech Republic Find Ways to work with vendors, NRENs, and/or Gov’tsFind Ways to work with vendors, NRENs, and/or Gov’ts

Inter-Regional Projects Inter-Regional Projects GLORIAD, Russia-China-US Optical RingGLORIAD, Russia-China-US Optical Ring South America: CHEPREO (US-Brazil); EU ALICE ProjectSouth America: CHEPREO (US-Brazil); EU ALICE Project Virtual SILK Highway Project (DESY): FSU satellite linksVirtual SILK Highway Project (DESY): FSU satellite links

Help with Modernizing the Infrastructure Help with Modernizing the Infrastructure Design, Commissioning, DevelopmentDesign, Commissioning, Development Provide Tools for Effective Use: Monitoring, CollaborationProvide Tools for Effective Use: Monitoring, Collaboration

Participate in Standards Development; Open ToolsParticipate in Standards Development; Open Tools Advanced TCP stacks; Grid systemsAdvanced TCP stacks; Grid systems

Workshops and Tutorials/Training SessionsWorkshops and Tutorials/Training Sessions Example: ICFA Digital Divide and HEPGrid Workshop in RioExample: ICFA Digital Divide and HEPGrid Workshop in Rio

Raise General Awareness of the Problem; Approaches to SolutionsRaise General Awareness of the Problem; Approaches to Solutions WSIS/RSIS; Trieste WorkshopWSIS/RSIS; Trieste Workshop

International Connectivity Costs in the Different European Market Segments

Market segment

Number of Countries

Cost Range

Liberal Market with transparent pricing

8 1-1.4

Liberal Market with less transparent pricing structure

7 1.8-3.3

Emerging Market without transparent pricing

3 7.5-7.8

Traditional Monopolist market

9 18-39

Dai Davies SERENATE Workshop Feb. 2003

Ratio to 114 If Include Turkey, Malta;Correlated with the Number of Competing Vendors

www.serenate.org

Virtual Silk HighwayVirtual Silk Highway Hub Earth Station at DESY with

access to the European NRENs and the Internet via GEANT Providing International

Internet access directly National Earth Station at each

Partner site Operated by DESY, providing

international access Individual uplinks, common

down-link, using DVB Currently 4 Mbps Up; 12

Down; for ~$ 350k/Year

The SILK Highway Countriesin Central Asia & the Caucasus

Note: Satellite Links are a Boon to the Region, but Unit Costs are Very High compared to Fiber.

There is a Continued Need for Fiber Infrastructure

A Series of Strategic Studies into the Future of Research and Education Networking in Europe

A significant “divide” exists in Europe – the worst countries [Macedonia, B-H, Albania, etc.] are 1000s of times worse off than the best. Also many of the 10 new EU members are ~5X worse off

than the 15 present members. If there is one single technical lesson from SERENATE it is that

transmission is moving from the electrical domain to optical. When there’s good competition users can still lease traditional

communications services (bandwidth) on an annual basis. But: Without enough competition prices go through the roof.

The more you look at underlying costs the more you see the need for users to get access to fibre.

Our best advice has to be “If you’re in a mess, you must get access to fibre”.

From Summary and Conclusions by D.O. Williams, CERN

See www.serenate.org

Dark Fiber in Eastern Europe Poland: PIONIER Network

Dark Fiber in Eastern Europe Poland: PIONIER Network

ŁÓDŹ

TORUŃ

P OZNAŃ

B YDGOS ZCZ

OLS ZTYN

B IAŁYS TOK

GDAŃS K

KOS ZALIN

S ZCZECIN

ZIELONAGÓRA

WROCŁAW

CZĘS TOCHOWA

KRAKÓW RZES ZÓW

LUB LIN

KIELCE

P UŁAWY

RADOM

KATOWICEGLIWICE

B IELS KO-B IAŁA

OP OLE

GUB IN WARS ZAWA

CIES ZYN

S IEDLCE

P IO NIE R n o d e sIn s ta lle d fib e r

F ib e rs p la n n e d in 2 0 0 4P IO NIE R n o d e s p la n n e d in 2 0 0 4

2650 km Fiber 2650 km Fiber ConnectingConnecting

16 MANs; 5200 km 16 MANs; 5200 km and 21 MANs by 2005 and 21 MANs by 2005

Support Support Computational GridsComputational Grids

Domain-Specific Domain-Specific Grids Grids

Digital LibrariesDigital Libraries Interactive TVInteractive TV

Add’l Fibers for Add’l Fibers for e-Regional Initiatives e-Regional Initiatives

CESNET Dark Fiber Case Study (Czech Republic): Within ReachCESNET Dark Fiber Case Study (Czech Republic): Within Reach

Case Study ResultCase Study ResultWavelength ServiceWavelength Service

Vs. Fiber Lease:Vs. Fiber Lease:

Cost Savings of Cost Savings of 50-70% Over 4 Years50-70% Over 4 Years

for Long 2.5G for Long 2.5G or 10G Linksor 10G Links

For Example: 4 X 10G For Example: 4 X 10G Over 300 km Over 300 km

for 14k Euro/Month for 14k Euro/Month

2513 km 2513 km Leased Fibers Leased Fibers (Since 1999) (Since 1999)

1 x 2,5G Leased 1 x 2,5G

(EURO/Month)about 150km (e.g. Ústí n.L. - Liberec) 7,000

about 300km (e.g. Praha - Brno) 8,000

***

4 x 2,5G Leased 4 x 2,5G



***

1 x 10G Leased 1 x 10G



***

4 x 10G Leased 4 x 10G



***

2 x booster 24dBm, 2 x DCF, DWDM 10G2 x (booster +In-line + preamplifier), 6 x DCF, DWDM 10G

Leased fibre with own equipment (EURO/Month)

5 000 *7 000 **

2 x booster 18dBm 2 x booster 27dBm + 2 x preamplifier + 6 x DCF


12 000 *14 000 **

8 000 **

2 x booster 21dBm, 2 x DCF2 x (booster 21dBm + in-line + preamplifier) + 6 x DCF

2 x booster 24dBm, DWDM 2,5G2 x (booster +In-line + preamplifier), 6 x DCF, DWDM 2,5G


5 000 *


8 000 *11 000 **

NEWS: Bulletin: ONE TWOWELCOME BULLETIN General InformationRegistrationTravel Information Hotel RegistrationParticipant List How to Get UERJ/Hotel Computer AccountsUseful Phone Numbers ProgramContact us: Secretariat Chairmen

CLAF CNPQ FAPERJ UERJ

SPONSORS

HEPGRID and Digital Divide Workshop

UERJ, Rio de Janeiro, Feb. 16-20 2004

Theme: Global Collaborations, Grids and Their Relationship to the Digital

Divide

For the past three years the SCIC has focused on understanding and seeking the means of reducing or eliminating the Digital Divide, and

proposed to ICFA that these issues, as they affect our field of High

Energy Physics, be brought to our community for discussion.

This led to ICFA’s approval, in July 2003, of the Digital Divide and H

EP Grid Workshop. http://www.uerj.br/lishep2004

TutorialsC++Grid TechnologiesGrid-Enabled

AnalysisNetworksCollaborative Systems

All Sessions andTutorials Available

Live Via VRVS

http://www.uerj.br/lishep2004

World Summit on the Information Society(WSIS): Geneva 12/2003 and Tunis in 2005 World Summit on the Information Society

(WSIS): Geneva 12/2003 and Tunis in 2005 The UN General Assembly adopted in 2001 a resolution The UN General Assembly adopted in 2001 a resolution

endorsing the organization of the endorsing the organization of the World Summit on the World Summit on the Information Society (WSIS),Information Society (WSIS), under UN Secretary-General, under UN Secretary-General, Kofi Annan, with the ITU and host governments taking Kofi Annan, with the ITU and host governments taking the lead role in its preparation. the lead role in its preparation.

GOAL: To Create an Information Society:GOAL: To Create an Information Society: A Common Definition was adoptedA Common Definition was adoptedin the “Tokyo Declaration” of January 2003:in the “Tokyo Declaration” of January 2003:“… One in which highly developed ICT networks, equitable “… One in which highly developed ICT networks, equitable and ubiquitous access to information, appropriate content and ubiquitous access to information, appropriate content in accessible formats and effective communication can in accessible formats and effective communication can help people achieve their potential”help people achieve their potential”

Kofi Annan Challenged the Scientific Community to Help (3/03) Kofi Annan Challenged the Scientific Community to Help (3/03) CERN and ICFA SCIC have been quite active in the WSIS in CERN and ICFA SCIC have been quite active in the WSIS in

Geneva (12/2003) Geneva (12/2003)

Role of Science in the Information Society. Palexpo, Geneva 2004

Role of Science in the Information Society. Palexpo, Geneva 2004

CERN SIS Forum, and CERN SIS Forum, and CERN/Caltech Online CERN/Caltech Online

StandStand Visitors:Visitors:

Kofi AnnanKofi Annan, UN Sec’y General , UN Sec’y General John H. MarburgerJohn H. Marburger, ,

Science Adviser to US PresidentScience Adviser to US PresidentIon IliescuIon Iliescu, President of , President of

Romania; Romania; and Dan Nica, Minister and Dan Nica, Minister of ICTof ICT

Jean-Paul Hubert, Ambassador Jean-Paul Hubert, Ambassador of Canada in Switzerland of Canada in Switzerland

Carlo Lamprecht, Pres. of Carlo Lamprecht, Pres. of Economic Dept. of Canton Economic Dept. of Canton

de Genevade Geneva … …

Role of Sciences in Information Society. Palexpo, Geneva 2003

Role of Sciences in Information Society. Palexpo, Geneva 2003

Demos at the CERN/Caltech Demos at the CERN/Caltech RSIS RSIS Online StandOnline Stand

Distance diagnosis and surgery using Distance diagnosis and surgery using Robots with “haptic” feedback Robots with “haptic” feedback (Geneva-Canada)(Geneva-Canada)

World Scale multisite multi-protocol World Scale multisite multi-protocol videoconference with VRVSvideoconference with VRVS

(Europe-US-Asia-South America) (Europe-US-Asia-South America) Music Grid: live performance with Music Grid: live performance with

bands at St. John’s, Canada and the bands at St. John’s, Canada and the Music Conservatory of Geneva on Music Conservatory of Geneva on stagestage

Advanced network and Grid-enabled Advanced network and Grid-enabled analysis demonstrationsanalysis demonstrations

Monitoring very large scale Grid farms Monitoring very large scale Grid farms with MonALISAwith MonALISA

VRVS on WindowsVRVS on WindowsVRVS (Version 3)VRVS (Version 3)Meeting in 8 Time Meeting in 8 Time

ZonesZones

28k registered hosts28k registered hostsUsers in 103 CountriesUsers in 103 Countries

2-3X Growth/Year2-3X Growth/Year

Networks, Grids and HENPNetworks, Grids and HENP Network backbones and major links used by HENP experimentsNetwork backbones and major links used by HENP experiments

are advancing rapidly are advancing rapidly To the 10 G range in < 2 years; much faster than Moore’s LawTo the 10 G range in < 2 years; much faster than Moore’s Law Continuing a trend: a factor ~1000 improvement per decade; Continuing a trend: a factor ~1000 improvement per decade;

a new HENP (and DOE) Roadmap a new HENP (and DOE) Roadmap HENP is learning to use long distance 10 Gbps networks effectively HENP is learning to use long distance 10 Gbps networks effectively

2003-2004 Developments: to 6 Gbps flows over 11 kkm2003-2004 Developments: to 6 Gbps flows over 11 kkm Transition to community-owned and operated R&E networks Transition to community-owned and operated R&E networks

is beginning (ca, nl, us, pl, cz, sk …) or considered (de, ro, …) is beginning (ca, nl, us, pl, cz, sk …) or considered (de, ro, …) Removing Regional, Last Mile, Local Bottlenecks and Removing Regional, Last Mile, Local Bottlenecks and

Compromises in Network Quality are nowCompromises in Network Quality are now On the critical path, in all world regionsOn the critical path, in all world regions

Digital Divide: Network improvements are especially neededDigital Divide: Network improvements are especially neededin SE Europe, Much of Asia, So. America; and Africain SE Europe, Much of Asia, So. America; and Africa

Work on These Issues in Concert with Internet2, Terena, APAN, Work on These Issues in Concert with Internet2, Terena, APAN, AMPATH; DataTAG, the Grid projects & the GGFAMPATH; DataTAG, the Grid projects & the GGF

Computing Model Progress CMS Internal Review of Software and Computing

Some Extra Slides Some Extra Slides

FollowFollow

ICFA and International Networking ICFA and International Networking

ICFA Statement on Communications in Int’l HEPICFA Statement on Communications in Int’l HEPCollaborations of October 17, 1996Collaborations of October 17, 1996 See See http://www.fnal.gov/directorate/icfa/icfa_communicaes.htmlhttp://www.fnal.gov/directorate/icfa/icfa_communicaes.html

““ICFA urges that all countries and institutions wishing ICFA urges that all countries and institutions wishing to participate even more effectively and fully in to participate even more effectively and fully in international HEP Collaborations should:international HEP Collaborations should: Review their operating methods to ensure they Review their operating methods to ensure they

are fully adapted to remote participation are fully adapted to remote participation Strive to provide the necessary communications Strive to provide the necessary communications

facilities and adequate international bandwidth” facilities and adequate international bandwidth”

NREN Core Network Size (Mbps-km):http://www.terena.nl/compendium/2002

NREN Core Network Size (Mbps-km):http://www.terena.nl/compendium/2002

Logarithmic Scale

1k

100k

100

100M

10M

1M

10kRo

It

PlGrIr

Ukr

Hu Cz

Es

Nl

Fi

Ch

Lagging

In Transition

Leading

Advanced

Network Readiness Index:How Ready to Use Modern ICTs [*]?

Network Readiness Index:How Ready to Use Modern ICTs [*]?

NetworkReadiness

Index

Environment

Readiness

Usage

Market

Infrastructure

Political/Regulatory

Individual Readiness

Gov’t Readiness

Business Readiness

Individual Usage

Gov’t Usage

Business Usage

(FI)

(FI)

(SG)

(US)(IC)

(SG)

(US)

(SG)

(US)

(FI)

(FI)

(DE)

(KR)

( ): Which Country is First

From the 2002-2003 Global InformationTechnology Report. See http://www.weforum.org

Throughput vs Net Readiness Index Throughput vs Net Readiness Index NRI from Center for Int’l Development, Harvard NRI from Center for Int’l Development, Harvard http://www.cid.harvard.edu/cr/pdf/gitrr2002_ch02.pdfhttp://www.cid.harvard.edu/cr/pdf/gitrr2002_ch02.pdf

Improved correlation (0.21 Improved correlation (0.21 0.41) by Using derived throughput ~0.41) by Using derived throughput ~ MSS / (RTT * sqrt(loss)); fit an exponential MSS / (RTT * sqrt(loss)); fit an exponential

Interesting Outliers: Slovakia, Hungary, Portugal. LithuaniaInteresting Outliers: Slovakia, Hungary, Portugal. Lithuania

Intern

et for all fo

cusA

&R

focus

NRI TopsNRI TopsFinland Finland 5.925.92US US 5.795.79SingaporeSingapore 5.745.74Sweden Sweden 5.585.58Iceland Iceland 5.515.51Canada Canada 5.445.44UKUK 5.355.35Denmark Denmark 5.335.33TaiwanTaiwan 5.315.31GermanyGermany 5.295.29Netherlnd Netherlnd 5.285.28Israel Israel 5.225.22Switz’land Switz’land 5.185.18Korea Korea 5.105.10

http://www.cid.harvard.edu/cr/pdf/gitrr2002_ch02.pdf

UltraLight UltraLight

http://ultralight.caltech.edu Serving the major LHC experiments; developments Serving the major LHC experiments; developments broadly applicable to other data-intensive programs broadly applicable to other data-intensive programs ““Hybrid” packet-switched and circuit-switched, Hybrid” packet-switched and circuit-switched,

dynamically managed optical networkdynamically managed optical network Global services for system managementGlobal services for system management Trans-US wavelength riding on NLR: LA-SNV-CHI-JAXTrans-US wavelength riding on NLR: LA-SNV-CHI-JAX Leveraging advanced research & production networksLeveraging advanced research & production networks

USLIC/DataTAG, SURFnet/NLlight, UKLight, USLIC/DataTAG, SURFnet/NLlight, UKLight, Abilene, CA*net4 Abilene, CA*net4

Dark fiber to CIT, SLAC, FNAL, UMich; Florida Light RailDark fiber to CIT, SLAC, FNAL, UMich; Florida Light Rail Intercont’l extensions: Rio de Janeiro, Tokyo, TaiwanIntercont’l extensions: Rio de Janeiro, Tokyo, Taiwan

Flagship Applications with a diverse traffic mixFlagship Applications with a diverse traffic mix HENP: TByte to PByte “block” data transfers at 1-10+ HENP: TByte to PByte “block” data transfers at 1-10+

GbpsGbps eVLBI: Real time data streams at 1 to several GbpseVLBI: Real time data streams at 1 to several Gbps

User Requirements

In ALL countries and in ALL disciplines researchers are eagerly anticipating improved networking tools. There is no divide on the demand-side. Sciences, such as particle physics, which make heavy use of advanced networking, must help to break down any divide on the supply-side, or else declare themselves elitist and irrelevant to researchers in essentially all developing countries.

Connectivity pricing and competition

In some locations the price of connectivity is (really) unreasonably high

Linked (obviously) to how competitive the market is Strong competition on routes between various key European

cities, and between major national centres Less competition effectively none as you move to countries

with de facto monopoly or simply to parts of countries where operators see little reason to invest.

While some expensive routes are where you would expect, others are much more surprising (at first sight), like Canterbury and Lancaster (UK) and parts of Brittany (F)

Understanding transmission costs and DIY solutions

Own trenching only makes sense in very special cases. Say 1-30 km. Even then look for partners.

Maybe useful (as a threat) over longer distances in countries with crazy pricing

Now possible to lease (short- or long-term) fibres on many routes in Europe [0.5 to 2 KEuro/km Typ.]

Transmission costs jump at ~200 km [below which you can operate with “Nothing In Line” (NIL), above which you need amplifiers] and ~800 km [above which you need signal regenerators]

Possibly leading to some new approaches in GEANT-2 implementation

Jensen, ICTP

Typ. 0-7 bpsPer Person

Limited by many externalsystemic

factors: Electricity;

Import Duties;Education; Trade

restrictions

Jensen, ICTP

Progress in Africa ?



RENATER3 in France2.5 G Backbone (Since 10/2002)

RENATER3 in France2.5 G Backbone (Since 10/2002)

650 Sites, Most Connect 650 Sites, Most Connect Through MANs Through MANs

< 50 Direct Connected< 50 Direct Connected GEANT Connection to GEANT Connection to

10G in March 2004 10G in March 2004 IN2P3 InvestigatingIN2P3 Investigating

Dark Fiber to CERNDark Fiber to CERN

APANAPANAPAN Link Map

Access PointExchange PointCurrent Status2004 (plan)

USA

Europe

AU

ID

VN

LK

KR

TH

MY

SG

TW

RU

PH

CN

HK

JP

APAN Members

ASEM Member:China, Japan, Korea, Malaysia, Philippines, Singapore, Thail and

Non-ASEM Member:

Australia, Taiwan, Hong Kong, Sri Lanka

Applying APAN Membership:

Bangladesh, Nepal

Courtesy: Prof. Kilnam Chon (APAN Chair)

AMPATH Miami-RNP Brazil

AMPATH Miami-REUNA (Chile)

Note: CMS-Tier1 (Brazil);ALMA (Chile)

Brazil: RNP in Early 2004Brazil: RNP in Early 2004

NREN Core Network Size (Mbps-km):Last Year

http://www.terena.nl/compendium/2002

NREN Core Network Size (Mbps-km):Last Year

http://www.terena.nl/compendium/2002Logarithmic Scale

1k

100k

100

100M

10M

1M

10kRo

It

PlGrIr

Ukr

Hu Cz

Es

Nl

Fi

Ch

Lagging

In Transition

Leading

Advanced

Relative Cost of Connectivity Compared with Number of Suppliers

0

5

10

15

20

25

30

35

40

45

0 2 4 6 8 10 12 14

Number of suppliers

Relative cost

Trend Line

Multipliers for Differing Circuit Speeds

0

0.2

0.4

0.6

0.8

1

1.2

050010001500200025003000

Speed In MBps

Multiplier

PingER derived throughput vs. the ITU Digital Access Index (DAI) for PingER countries

monitored from the U.S.

PingER derived throughput vs. the ITU Digital Access Index (DAI) for PingER countries

monitored from the U.S.

NRNs’ Bandwidth in Latin AmericaCountry Organization Existing REN? National connections External

CapacityNumber of

Connected SitesConnection

to USInternet2

Argentina RETINA yes 256Kbps – 34 Mbps 59 Mbps 56 yes

Bolivia BOLnet yes 64 – 128 Kbps 1.5 Mbps 18 no

Brazil RNPyes

2 – 30 Mbps (backbone up to 622 Mbps

202 Mbps369

yes

Chile REUNAyes 155 Mbps

45 Mbps via AMPATH

18yes

Colombia RedCETCol Not known Not known Not known Not known Not known

Costa Rica CRNet yes 32 – 512 Kbps Not known 34 no

Cuba RedUniv University Network

19.2 Kbps – 2 MbpsNot known

23no

Ecuador FUNDACYT In planning no

El Salvador CONACYT In planning no

Guatemala Not known Non-existent no

Honduras HONDUnet Not known no

Nicaragua Not known no

Panama PANNET/SENACYT

University/Gov. Network

256 – 512 Kbps 1.54 Mbps 11 no

Peru CONCYTEC In planning no

Uruguay RAU yes 64 Kbps to 1 Mbps 6 Mbps 46 no

Venezuela REACCIUN yes 45 Mbps via AMPATH

78 January 2003

Source:

CAESAR - Review of Develop-ments in Latin America

Undersea Optical InfrastructureSubmarine Fiber-Optic Cable

SystemTotal Bandwidth Capacity (GB)

Americas 1 .560

Americas II 2.5

South American Crossing 1,280

Columbus II .560

Columbus III 2.5

Telefonica’s Emergia 1,920

ARCOS 960

Maya-1 60

360 Americas 10

The total aggregate bandwidth capacity Latin America and Caribbean region is estimated at 4,236 GB

It is generally accepted that once a technology is perceived as having broad utilitarian value, price as a % of per capita income, is the main driver of penetration

C. Casasus, CUDI (Mexico); CANARIE

Penetration of telecommunications in low income countries is further inhibited by at least 3 factors…

Low income per capitaLess competition. Higher prices

from monopoliesFewer applications. No broad

utilitarian value


Income vs. penetration given the price of a technology

Per capitaIncome

% Penetration

High income countries

Low

High

Low High

Low income countries


Telecom monopolies have even higher prices in low income countries

Fewer entrants. Less competition No unbundling Price cap regulation creates cross

subsidies between costumer groups. Large customers (inelastic) subsidize

small costumers (elastic). High bandwidth services are very expensive

Inefficient ROW regulation Inefficient spectrum policies


Virtual Silk Highway Project Managed by DESY and Partners

Virtual SILK Highway Project (from 11/01): Virtual SILK Highway Project (from 11/01): NATO ($ 2.5 M) and Partners ($ 1.1M) NATO ($ 2.5 M) and Partners ($ 1.1M) Satellite Links to South Caucasus Satellite Links to South Caucasus and Central Asia (8 Countries) and Central Asia (8 Countries)

In 2001-2 (pre-SILK) BW 64-512 kbpsIn 2001-2 (pre-SILK) BW 64-512 kbpsProposed VSAT to get 10-50 X BWProposed VSAT to get 10-50 X BW for same cost for same cost

See See www.silkproject.orgwww.silkproject.org[*] [*] Partners: CISCO, DESY. GEANT, Partners: CISCO, DESY. GEANT, UNDP, US State Dept., Worldbank, UNDP, US State Dept., Worldbank, UC London, Univ. Groenigen UC London, Univ. Groenigen

Note: NATO Science for Peace Program

Documents

HENP Networks and Grids for Global VOs: from Vision to Reality HENP Networks and Grids for Global VOs: from Vision to Reality Harvey B. Newman Harvey B