ASCR/ESnet Network Requirements an Internet2 Perspective

2009 ASCR/ESnet Network Requirements WorkshopApril 15/16, 2009Richard Carlson -- Internet2<rcarlson@internet2.edu>

Basic Premise

• Large science projects have become, or are becoming, global collaborations • Multiple investigators• Multiple physical locations• High-Performance infrastructure required

to support scientific investigations and collaborations

Reframing the Science needs Question

• Conclusion based request• I need 5 Gbps of bandwidth

• Science based request• I need to move 2 TBytes in 1 hour• I will move this amount of data once per

day

• Users need ways to make effective complaints

Current Network Traffic Flows

• DoE-SC major funding source for U.S. scientific research

• DoE operates specialized instruments and facilities (labs)

• Majority of U.S. users located at Universities and other non-Lab sites

Current Network Architecture

• ESnet interconnects DoE Labs and peers with national and international partners

• Internet2 interconnect regional/state networks and peers with national and international partners

• Regional/State networks connect to University campus networks

Internet2 Background

• Member organization formed in 1996• 30 Regional/State network connectors• 211 Universities• 11 Corporate partners• 44 Affiliate members• 60 International partners

Internet2 Management

• Member focused, Member led organization• 18 member Board of Trustees• 4 Advisory Councils• Architecture & Operations• Applications, Middleware, and Services• Research • External Relations

• All councils have CIO, Scientific Researcher, Regional Network, Industry, and Appointed representatives

Internet2 Strategic Planning Initiative

• 2008 – Community lead initiative to develop new strategic plan • http://www.internet2.edu/strategicplanning/• 4 major goals, 8 strategic statements, 28

action items

• 2009 – Strategic implementation plans begin developed

Internet2 Infrastructure

Nodes

Internet2 Collaboration Model

• Direct support to member institutions

• Indirect support for most funded member lead research projects

• Direct collaboration on specific funded research projects

End-to-End Performance Initiative

• perfSONAR• International measurement infrastructure

collaboration with ESnet, GEANT, and RNP• Currently being developed and deployed

• Measurement tools• OWAMP – One-Way Delay• NDT – Advanced host/network diagnostics• BWCTL – Advanced throughput measurement

Middleware and Security

• Shibboleth• Multi-institution single sign-on (SSO)

service

• Grouper• Groups Management toolkit

• COmanage• Identity-oriented requirements of common

collaboration tools

Current Control Plane Developments• Intra-domain

• Currently on the Ciena platform, to be deployed on the Infinera platform in the future

• Have an Installed working configuration on the Cienas

• Inter-domain• Working with the DICE (Dante, Internet2, Canarie/LHC,

ESnet) group• Working on Topology, Path Computation and

Scheduling, Signaling, and Lookup Services (think a circuit based DNS)

• Web service based communications• Face-to-face meeting every three months and biweekly

calls

Photo by Steven S. Wallace

Conclusions

• Internet2 is a “Member focused, Member led” organization

• Works closely with DOE, NSF and other Fed agencies to support community

• Strong collaborations with partners and peers

• Comprehensive set of services to support high-performance scientific collaborations

• Supplemental Information

Using the Network

• Application’s performance should meet your expectations!• 100 MB/s is reasonable• 1000 MB/s is possible

• If they don’t you should complain!• But you need to complain effectively if you

want the problem fixed

Web Service based E-NNI

• Routing• Topology Exchange• Domain Abstraction• Varying levels of dynamic information

• Resource Scheduling• Multi-Domain path computation techniques• Resource identification, reservation, confirmation

• Signaling• path setup, service instantiation

• Host Lookup Service• Uses DNS pointers

Dynamic Circuit NetworksWeb Service Control Plane Interfaces

• Web Services provides a mechanism to deal with heterogeneous control planesinspired by the standards bodies work on control plane protocols, but

not just recreating that work at the web service level

Ethernet/L2SC(Dataplane)

SONET/TDM(Dataplane)

Router(MPLS)/PSC(Dataplane)

GMPLS(I-NNI)

MPLS(I-NNI)

Management System(I-NNI)

Inter-Domain Controller (IDC)

WS E-NNIWS E-NNI

WS UNI WS UNI

IDC

IDCWS I-NNI IF

WS I-NNI IF WS I-NNI IF

Shared IP Network

• Juniper routers at 9 locations• 10 Gbps links interconnect routers• 20 Gbps on multiple links as required

• Router proxy allows access to current config’s and statistics

• Internet2 Observatory provides detailed utilization information

Network Research Example - Phoebus

• Phoebus - TCP data flows• File transfers over long distance segments

not requiring congestion control• Lead by Martin Swaney at the University of

Delaware

Nodes

Dynamic Circuit Network

• 21 Ciena Core Director multiplexor sites

• Dedicated circuits between 2 points• site-to-site, cluster-to-cluster, or host-to-

host• One end-point may be non-Internet2 site• Bandwidth steps from 51 Mbps to 10 Gbps• Short duration (minutes to weeks)• Long duration (weeks to years)

Nodes

Dynamic Circuit Network

• Similar to the IP network model, but using different protocols

• Connections are one armed• Physical connection• Control Plane Protocols

• Create Circuits in seconds for periods of hours to days

• Only through the Ciena equipment at the start, eventually evolving to the full platform