Networks, Grids and the Digital Divide in HEP and Global e-Science Networks, Grids and the Digital Divide in HEP and Global e-Science Harvey B. Newman.

  • Published on

  • View

  • Download


  • Networks, Grids and the Digital Divide in HEP and Global e-Science Harvey B. Newman ICFA Workshop on HEP Networking, Grids, and Digital Divide Issues for Global e-Science Daegu, May 23 2005

  • TOTEMpp, general purpose; HILHCb: B-physicsALICE : HI pp s =14 TeV L=1034 cm-2 s-1 27 km Tunnel in Switzerland & FranceLarge Hadron Collider CERN, Geneva: 2007 Start CMSAtlasHiggs, SUSY, Extra Dimensions, CP Violation, QG Plasma, the Unexpected5000+ Physicists 250+ Institutes 60+ Countries

  • LHC Data Grid Hierarchy:Developed at CaltechTier 1Online System CERN Center PBs of Disk; Tape RobotFNAL CenterIN2P3 Center INFN Center RAL Center InstituteInstituteInstituteInstitute Workstations~150-1500 MBytes/sec~10 Gbps1 to 10 GbpsTens of Petabytes by 2007-8, at ~100 Sites. An Exabyte ~5-7 Years later.Physics data cache~PByte/sec10 - 40 Gbps~1-10 GbpsTier 0 +1Tier 3Tier 4Tier 2ExperimentCERN/Outside Resource Ratio ~1:2 Tier0/( Tier1)/( Tier2) ~1:1:1Emerging Vision: A Richly Structured, Global Dynamic System

  • ICFA and Global Networks for Collaborative ScienceNational and International Networks, with rapidly increasing capacity and end-to-end capability are essential, forThe daily conduct of collaborative work in both experiment and theoryExperiment development & construction on a global scaleGrid systems supporting analysis involving physicists in all world regions The conception, design and implementation of next generation facilities as global networksCollaborations on this scale would never have been attempted, if they could not rely on excellent networks

  • Challenges of Next Generation Science in the Information AgeFlagship Applications High Energy & Nuclear Physics, AstroPhysics Sky Surveys: TByte to PByte block transfers at 1-10+ GbpsFusion Energy: Time Critical Burst-Data Distribution; Distributed Plasma Simulations, Visualization, Analysis eVLBI: Many (quasi-) real time data streams at 1-10 Gbps BioInformatics, Clinical Imaging: GByte images on demandAdvanced integrated Grid applications rely on reliable, high performance operation of our LANs and WANsAnalysis Challenge: Provide results with rapid turnaround, over networks of varying capability in different world regionsPetabytes of complex data explored and analyzed by 100s-1000s of globally dispersed scientists, in 10s-100s of teams

  • Huygens Space Probe Lands on Titan - Monitored by 17 telescopes in Au, Jp, CN, USIn October 1997, the Cassini spacecraft left Earth to travel to SaturnOn Christmas day 2004, the Huygens probe separated from Cassini On 14 January 2005 it started its descent through the dense (methane, nitrogen) atmosphere of Titan (speculated to be similar to that of Earth billions of years ago)The signals sent back from Huygens to Cassini were monitored by 17 telescopes in Australia, China, Japan and the US to accurately position the probe to within a kilometre (Titan is ~1.5 billion kilometres from Earth)Courtesy G. McLaughlin

  • Australian eVLBI data sent over high speed links to the NetherlandsThe data from two of the Australian telescopes were transferred to the Netherlands over the SXTransport and IEEAF links, and CA*net4 using UCLP, and were the first to be received by JIVE (Joint Institute for VLBI in Europe), the correlator siteThe data was transferred at an average rate of 400Mbps (note 1Gbps was available) The data from these two telescopes were reformatted and correlated within hours of the end of the landingThis early correlation allowed calibration of the data processor at JIVE, ready for the data from other telescopes to be addedSignificant intl collaborative effort: 9 OrganizationsG. McLaughlin, D. Riley

  • ICFA Standing Committee on Interregional Connectivity (SCIC)Created in July 1998 in Vancouver ; Following ICFA-NTFCHARGE: Make recommendations to ICFA concerning the connectivity between the Americas, Asia and Europe As part of the process of developing these recommendations, the committee should Monitor traffic on the worlds networks Keep track of technology developmentsPeriodically review forecasts of future bandwidth needs, and Provide early warning of potential problemsCreate subcommittees as needed to meet the chargeRepresentatives: Major labs, ECFA, ACFA, North and South American UsersChair of the committee reports to ICFA twice per year

  • SCIC in 2004-2005 2005 Reports, Presented to ICFA Today Main Report: Networking for HENP [H. Newman et al.]Includes Updates on the Digital Divide, World Network Status; Brief updates on Monitoring and Advanced Technologies [*]18 Appendices: A World Network Overview Status and Plans for the Next Few Years of Natl & Regional Networks, and Optical Network Initiatives Monitoring Working Group Report [L. Cottrell] Also See: SCIC Digital Divide Report [A. Santoro et al.] SCIC 2004 Digital Divide in Russia Report [V. Ilyin]TERENA ( 2004 Compendium

  • SCIC Main Conclusion for 2002-5The disparity among regions in HENP could increase even more sharply, as we learn to use advanced networks effectively, and we develop dynamic Grid systems in the most favored regions We must therefore take action, and work to Close the Digital DivideTo make Scientists in All World Regions Full Partners in the Process of Frontier Discoveries This is essential for the health of our global experimental collaborations, for our field, and for international collaboration in many fields of science.

  • NEWS: Bulletin: ONE TWO WELCOME BULLETIN General Information Registration Travel Information Hotel RegistrationParticipant List How to Get UERJ/Hotel Computer AccountsUseful Phone Numbers ProgramContact us: Secretariat Chairmen CLAF CNPQ FAPERJ UERJ SPONSORSHEPGRID and Digital Divide Workshop UERJ, Rio de Janeiro, Feb. 16-20 2004Theme: Global Collaborations, Grids and Their Relationship to the Digital DivideFor the past three years the SCIC has focused on understanding and seeking the means of reducing or eliminating the Digital Divide. It proposed to ICFA that these issues, as they affect our field, be brought to our community for discussion. This led to ICFAs approval, in July 2003, of the Digital Divide and HEP Grid Workshop. Review of R&E Networks; Major Grid Projects Perspectives on Digital Divide Issues by Major HEP Experiments, Regional Representatives Focus on Digital Divide Issues in Latin America; Relate to Problems in Other Regions

    See http://www.lishep.uerj.brTutorialsC++Grid TechnologiesGrid-Enabled AnalysisNetworksCollaborative Systems A. Santoro

  • International ICFA Workshop on HEP Networking, Grids, and Digital Divide Issues for Global e-ScienceMay 23-27, 2005Daegu, Korea

    Dongchul SonCenter for High Energy Physics Harvey NewmanCalifornia Institute of Technology Focus on Asia-Pacific Also Latin America, Middle East, AfricaApproved by ICFA August 2004


    International ICFA Workshop on HEP Networking, Grids and Digital Divide Issues for Global e-Science

    Workshop Goals Review the current status, progress and barriers to effective use of major national, continental and transoceanic networksReview progress, strengthen opportunities for collaboration, and explore the means to deal with key issues in Grid computing and Grid-enabled data analysis, for high energy physics and other fields of data intensive scienceExchange information and ideas, and formulate plans to develop solutions to specific problems related to the Digital Divide, with a focus on the Asia Pacific region, as well as Latin America, Russia and Africa Continue to advance a broad program of work on reducing or eliminating the Digital Divide, and ensuring global collaboration, as related to all of the above aspects.

  • PingER: World View from SLAC, CERNS.E. Europe, Russia: Catching upLatin Am., China: Keeping upIndia, Mid-East, Africa: Falling BehindC. Asia, Russia, SE Europe, L. America, M. East, China: 4-7 yrs behindIndia, Africa: 7-8 yrs behindLatin AmericaLatin AmericaR. Cottrell

  • Connectivity to Africa Internet Access: More than an order of magnitude lower than the corresponding percentages in Europe (33%) & N. America (70%).

    Digital Divide: Lack of Infrastructure, especially interior, high prices (e.g. $ 4-10/kbps/mo.); Gray satellite bandwidth market Intiatives: EUMEDCONNECT (EU-North Africa); GEANT: 155 Mbps to S. Africa; Nectarnet (Ga. Tech); IEEAF/I2 NSF-Sponsored Initiative

  • Sample Bandwidth Costs for African UniversitiesSample size of 26 universities Average Cost for VSAT service: Quality, CIR, Rx, Tx not distinguishedBandwidth prices in Africa vary dramatically; are in general many times what they could be if universities purchase in volumeRoy Steiner Internet2 2004 WorkshopAvg. Unit Cost is 40X US Avg.; Cost is Several Hundred Times, Compared to Leading Countries











    IBAUD Target$3.00















  • Asia Pacific Academic Network ConnectivityBetter North/South Linkages within Asia Needed JP- TH link: 2Mbps 45Mbps in 2004.Connectivity to US from JP, KO, AU is Advancing Rapidly. Progress in the Region, and to Europe is Much Slower APAN Status 7/2004D. Y. Kim

  • Some APAN Links G. McLaughlin

  • Digital Divide Illustrated by Network Infrastructures: TERENA NREN Core Capacity Core capacity goes up in Large Steps: 10 to 20 Gbps; 2.5 to 10 Gbps; 0.6-1 to 2.5 Gbps Current In Two YearsSE Europe, Medit., FSU, Middle East: Less Progress Based on Older Technologies (Below 0.15, 1.0 Gbps): Digital Divide Will Not Be ClosedSource: TERENA

  • Long Term Trends in Network Traffic Volumes: 300-1000X/10Yrs SLAC Traffic ~400 Mbps; Growth in Steps (ESNet Limit): ~ 10X/4 Years.Projected: ~2 Terabits/s by ~2014July 2005: 2x10 Gbps links: one for production and one for researchESnet Accepted Traffic 1990 2004 Exponential Growth Since 92; Annual Rate Increased from 1.7 to 2.0X Per Year In the Last 5 YearsW. JohnstonL. CottrellProgress in StepsW. Johnston10 Gbit/s FNAL: 10 to 20 (+40) Gbps by Fall 2005

  • CANARIE (Canada) Utilization Trends and UCLPv2

    GbpsNetwork Capacity LimitJan 05Demand for Customer Empowered Nets (CENs) is exceeding our wildest expectationsNew version of UCLP will allow easier integration of CENs into E2E nets for specific communities &/or disciplinesUCLPv2 will be based on SOA, web services and workflow to allow easy integration into cyber-infrastructure projectsThe Network is no longer a static fixed facility but can be orchestrated with different topologies, routing etc to meet specific needs of end usersW. St. Arnaud

  • National Lambda Rail (NLR): www.nlr.netNLRInitially 4-8 10G WavelengthsTo 40 10G Waves in FutureUltralight, Internet2 HOPI, Cisco Research & UltraScience Net Initiatives w/HEP Atlantic & Pacific WaveTransition beginning now to optical, multi-wavelength Community owned or leased dark fiber networks for R&E Initiatives in: nl, ca, jp, uk, kr; pl, cz, sk, pt, ei, gr, sb/mn + 30 US States (Ca, Il, Fl, In, )

  • GEANT2 Hybrid ArchitectureGlobal Connectivity10 Gbps + 3x2.5 Gbps to North America2.5 Gbps to Japan622 Mbps to South America45 Mbps to Mediterranean countries 155 Mbps to South AfricaWill be Improved in GEANT2

    Cooperation of 26 NRENsImplementation on dark fiber, IRU Asset, Transmission & Switching EquipmentLayer 1 & 2 switching, the Light PathPoint to Point (E2E) Wavelength servicesLHC in Europe: N X 10G T0-T1 Overlay NetH. Doebbling

  • SXTransport: Au-US 2 X 10GAARNet has dual 10Gbps circuits to the US via Hawaii, dual 622Mbps commodity links G. McLaughlin

  • JGN2: Japan Gigabit Network (4/04 3/08)20 Gbps Backbone, 6 Optical Cross-ConnectsJGN2Connection services at the optical level: 1 GbE and 10GbE Optical testbeds: e.g. GMPLS Interop. TestsY. Karita

  • APAN-KR : KREONET/KREONet2 IIKREONET11 Regions, 12 POP CentersOptical 2.5-10G Backbone; SONET/SDH, POS, ATMNational IX ConnectionKREONET2Support for Next Gen. Apps:IPv6, QoS, Multicast; Bandwidth Alloc. ServicesStarLight/Abilene ConnectionInternational LinksGLORIAD Link to 10G to Seattle Aug. 1 (MOST)US: 2 X 622 Mbps via CA*Net; GbE via TransPACJapan: 2 GbpsTEIN to GEANT: 155 MbpsSuperSIREN (7 Res. Institutes)Optical 10-40G BackboneCollaborative Environment SupportHigh Speed Wireless: 1.25 GD. Son

  • The Global Lambda Integrated Facility for Research and Education (GLIF) Architecting an international LambdaGrid infrastructure Virtual organization supports persistent data-intensive scientific research and middleware development on LambdaGrids Many 2.5 - 10G Links Across the Atlantic and Pacific Peerings: Pacific & Atlantic Wave; Seattle, LA, Chicago, NYC, HK

  • Internet 2 Land Speed Record (LSR)Product of transfer speed and distance using standard Internet (TCP/IP) protocols.Single Stream 7.5 Gbps X 16 kkm with Linux: July 2004IPv4 Multi-stream record with FAST TCP: 6.86 Gbps X 27kkm: Nov 2004IPv6 record: 5.11 Gbps between Geneva and Starlight: Jan. 2005Concentrate now on reliable Terabyte-scale file transfers Disk-to-disk Marks: 536 Mbytes/sec (Windows); 500 Mbytes/sec (Linux) Note System Issues: PCI-X Bus, Network Interface, Disk I/O Controllers, CPU, DriversNB: Computing Manuf.s Roadmaps for 2006: One Server Pair to One 10G LinkNov. 2004 Record Network Internet2 LSRs: Blue = HEP7.2G X 20.7 kkmThrouhgput (Petabit-m/sec) S. Ravot

  • SC2004 Bandwidth Record by HEP: High Speed TeraByte Transfers for Physics Caltech, CERN SLAC, FNAL, UFl, FIU, ESNet, UK, Brazil, Korea; NLR, Abilene, LHCNet, TeraGrid; DOE, NSF, EU, ; Cisco, Neterion, HP, NewiSys, Ten 10G Waves, 80 10GbE Ports, 50 10GbE NICsAggregate Rate of 101 Gbps1.6 Gbps to/from Korea2.93 Gbps to/from BrazilUERJ, USP Monitoring NLR, Abilene, LHCNet, SCINet, UERJ, USP, Intl R&E Nets and 9000+ Grid Nodes Simultaneously I. Legrand

  • SC2004 KNU Traffic: 1.6 Gbps to/From Pittsburgh Via Transpac (LA) and NLR Monitoring in DaeguCourtesy K. Kwon

  • SC2004: 2.93 (1.95 + 0.98) Gbps Sao Paulo Miami Pittsburgh (Via Abilene)Brazilian T2+T3 HEPGrid: Rio + Sao Paolo Also 500 Mbps Via Red CLARA, GEANT (Madrid)Madrid & GEANTGEANT (SURFNet)J. Ibarra

  • HENP Bandwidth Roadmap for Major Links (in Gbps)Continuing Trend: ~1000 Times Bandwidth Growth Per Decade; HEP: Co-Developer as well as Application Driver of Global Nets












    SONET/SDHDWDM; GigE Integ.




    DWDM; 1 + 10 GigEIntegration



    2-4 X 10

    ( Switch;( Provisioning


    2-4 X 10

    ~10 X 10; 40 Gbps

    1st Gen. ( Grids


    ~10 X 10or 1-2 X 40

    ~5 X 40 or~20-50 X 10

    40 Gbps (Switching


    ~5 X 40 or

    ~20 X 10

    ~25 X 40 or ~100 X 10

    2nd Gen ( GridsTerabit Networks




    ~Fill One Fiber

  • Evolving Quantitative Science Require...


View more >