Internet2, Abilene, and GigaPoPs:Una vista de los EE UU

Steve Corbató corbato@internet2.edu

Director, Backbone Network Infrastructure

University Corp. for Advanced Internet Development

Jornadas Técnicas RedIRIS 2000Murcia, España15 de noviembre, 2000

2

Advanced U.S. research university connectivity requirements• Research testbed

• configurable, breakable, measurable infrastructure• serving computer science research and advanced engineering• traditional province of DARPA

• Advanced service/application deployment network • standards based, 7x24 operation expectation (vBNSAbilene)

• National education intranet• interconnecting all K-20 educational institutions/networks to

enable applications and services unavailable over the commercial Internet

• Commercial entities (high performance connectivity)• providers of content of value to EDU (e.g., Nexis-Lexis, Akamai)• EDU-related startups (genomics, IP networking)

3

Unique features of Internet2 environment• Per capita available bandwidth O(10-100) higher than over the commercial Internet

• TCP flows of 0.4 Gbps and higher possible

• Active advanced service deployment efforts• Native multicast most widely deployed

• Commitment to open network management and active measurement

• Emphasis on end-to-end (e2e) performance measurment and assurance

• Collaborative relationship with GigaPoPs and research university campus technical communities

4

Outline

• Internet2 project status

• Abilene network status• Advanced services

• International Transit Network (ITN)

• Changes to the Abilene Program

• Future infrastructure plans• New technologies

• Internet2 End-to-End Performance Initiative – e2e• Campus networks

5

Membership update

•179+ universities

•70+ corporations

•40+ affiliated research organizations

•30+ International MoU partners

6

Internet2 ActivitiesApplications

• Advanced applications with focus on research and education

Middleware• Interoperability across Internet2 institutions

Network Technologies• New network technologies across Internet2 networks

Network Infrastructure• Pre-commercial, production network infrastructure

Partnerships/Tech Transfer• Ensure this technology becomes available beyond Internet2

7

ApplicationsDiscipline focus

• Health Sciences• Arts & Humanities

Support• Portable MPEG2 video conferencing• Portable SGI O2 for other demos• Portable Access Grid Node

Workshops• Survivors of the Shoah/Visual History

Foundation: digital media

www.internet2.edu/apps

8

High Definition Television/IPStudio grade (200 Mbps UDP/stream)

Collaboration of University of Washington (Research Channel and Pacific/Northwest GigaPoP) and Sony

Raw HDTV/IP (1.5 Gbps) coming soon (Tektronix)…

9

Throughput - Linear

10

Middleware

Applications Needs:• Scalable, interoperable authentication and authorization (digital

libraries)• Grid computation resources using Globus security, location and

allocation of resources, scheduling etc. plugged into campus middleware infrastructures

• Common authentication and storage (next generation portals)

www.internet2.edu/middleware

Abilene Network

12

Qwest acquisition of U S WEST

13

Abilene – November, 2000 • Inflection point in network development

• OC-48c (2.5 Gbps) IP-over-SONET backbone

• 53 current and pending connections in 32 states

– OC-48c connections: Seattle, Atlanta, SC2000

• 175 participants in 47 states and D.C.

• Ongoing strong partnership

– Cisco, Nortel, Qwest, Indiana Univ., ITECs (NC and OH)

• Increasing backbone utilization • Characteristic exponential growth

• O(OC-12c) peak utilization on some links

• Traffic doubling time: 7 months

14

Seattle

Kansas City

Denver

Cleveland

New York

Atlanta

Houston

Abilene Backbone – autumn 2000

Sacramento

Los Angeles

Denver

Indianapolis

Washington

15

16

Backbone developments• New router node (#11) in Washington DC

• Northern California router node is moving from Sacramento to Sunnvyale

• Houston-Atlanta link upgraded to OC-48c

• Only one link (Seattle-Sacramento) remains at OC-12c

• Increasingly distributed international peering

• Peering with CUDI in Los Angeles (CENIC) and soon in El Paso (UTEP)

• Abilene International Transit Network (ITN) in production

• Collaboration with STARTAP and CA*NET3

• Backbone and connections were unaffected by Qwest’s acquisition of USWEST this summer

17

Abilene International Peering

APAN/TransPAC, Ca*net3, CERN, CERnet, IUCC, NORDUnet, RENATER, REUNA2, SURFnet, SingAREN, SINET, TAnet2 , (ANSP, HARnet?)

OC12 NYCMTEN-155*,

JANET, NORDUnet,

SURFnet CA*net3(HEAnet)

STTLCA*net3, (AARnet)

SNVA(SINET, GEMNET)

LOSASingAREN, SINET(HARNET?)

AmPATH(REUNA2, RNP2, RETINA?)

OC3-12UT El Paso(CUDI)

CALREN2CUDI

* ARNES, BELNET, CESnet, DFN, GRNET, HEAnet, RESTENA, SWITCH, HUNGARNET, GARR-B, POL-34, RCCN, RedIRIS

18

Abilene International Transit Network

• ITN concept developed as some international NRNs moved their U.S. circuit terminations closer to the landing points

• By default, Abilene international peers are ITN participants• Non-participating international peers must indicate this intent to

UCAID

• Abilene ITN service commenced October 23, 2000

• ITN service currently does not extend to U.S. federal research and education networks

• ESnet, NISN, NREN, DREN, DARPA Supernet

• As an Internet2 backbone, vBNS has become an ITN participant

19

Current Abilene ITN participants (15)

• CA*Net3

• CERNET

• CUDI

• DFN (via DANTE)

• IUCC

• JANET

• NORDUnet

• RENATER

• REUNA2

• SINET

• SingAREN

• SURFnet

• TANET2

• TransPAC

• vBNS

20

Abilene ITN implementation

• BGP communities for international peers• 11537:2501 – ITN participants• 11537:2500 – non-ITN participants

• Abilene ITN participants are tagged with ITN community on ingress (route map: TRANSIT-in)

• Abilene ITN participants receive all routes with 11537:2501 community (route map: TRANSIT-out)

21

BGP considerations for ITN participants

22

ITN collaboration

•Abilene, CA*Net3, and STARTAP are actively collaborating on international peering and transit issues in North America

•In particular, Abilene and CA*Net3 have agreed to provide transit among their respective ITN participants

• Abilene requires a Memorandum of Agreement and an Interconnection Agreement with the foreign NRN directly connected to CA*Net3

23

Abilene program changes

• General end of new OC-3c Abilene backbone connections• Existing and pending OC-3c connections will be unaffected

• Revised fee structure for OC-12c and OC-48c connections• Objective is to incent bandwidth upgrades for existing connectors• Overall price reduction is ~15%• Encourage Packet-over-SONET (POS) connections

• Expansion to serve the broader educational community

24

Revised Abilene annual connection fees

Previous New

OC-3c $110,000 ($110,000) SONET & ATM

OC-12c $320,000 $270,000 SONET

$280,000 ATM/1 PVC &

1 BGP peering

$290,000 ATM

OC-48c $495,000 $430,000 SONET

25

Sponsored Education Group Participation• Effective January 15, 2001, a networked aggregate of educational institutions may gain access to Abilene as a Sponsored Education Group Participant.

• designed primarily to accommodate existing and emerging state-based education networks

• reflects modified Abilene CoU

• This new class of Abilene participation supplements the existing classes of Member Participant, Collaboration Site, and Sponsored Participant

• Applications will be accepted commencing 1/12/2000

26

Advanced service deployment• Multicast

• fully native deployment using current inter-domain protocols

• PIM-Sparse, MBGP, MSDP

• support for source specific multicast (SSM) in place

• IPv6 • overlay testbed in production

• Quality of Service (QoS)• QBONE via IETF Differentiated Services protocols

• Abilene premium service testing starting

•Measurement and Network Management

• active probes (Surveyor) deployed in all nodes

• open network management stance

27

Building the Network of the Future – I • Upgrade Abilene to a leading-edge optical transport capability

• Transition to OC-192c (10 Gbps) over DWDM backbone

• Explore optical interconnection options with international networks (e.g., CA*Net4) & leading GigaPoPs

• Establish a rapidly configurable, breakable Internet2 national testbed for the computer science research and the advanced network engineering communities

• Interconnection and limited peering with the DARPA Supernet

• Overlay networks (server-based)

• Option of limited-term dedicated capacity via MPLS tunnels or DWDM λ’s

28

Network of the Future – II

• Continue the vital role of Abilene as a reliable platform for the development of innovative applications and the deployment of advanced services

• IP remains the common bearer service

• Position Abilene as a critical component of the Internet2 End-to-End (e2e) Initiative – in particular, its vital role within the U.S. research infrastructure

• Emphasis on pro-active measurement and open network management

• Increasing dependence on Abilene for next generation science

29

Project NEPTUNE

30

Network of the Future - III

• Collaborate with the GigaPoPs on higher bandwidth attachments and the facilitation of international peering

• Creative, localized solutions often required above OC-3c

• Increasingly distributed set of international landing points and termination options

31

GigaPoPs

•Gigabit-per-second Point of Presence• OC-12 connectivity or greater (GbEth)

•Regional aggregation point• high performance (HPNSPs)• commodity (NSPs)

•Economies of scale• Member sites• Value added services shared by members

•Not limited to research universities

•(High-speed) local traffic stays local

32

Leading Regional Gigapops

MREN (Chicago)

CalREN2 (California)

SoX (Atlanta/Southeast)

Great Plains Network (Kansas City)

P/NW Gigapop (Seattle)

NYSERNET (New York)

MAX (Washington D.C.)

NOX (Boston)

Front Range (Denver)

33

CalREN-2 Topology Plan (5/98)

Fresno

Internet2(vBNS )

Internet2(vBNS )

USC, ISI, UCLA, UCI, Caltech,CSU, JPL, UCR, UCSB

UCB, UCD, UCSF,UCSC, Stanford, UCOP

UCOP, Oakland

UC Berkeley

Monterey

Stanford

UC Davis

UCSD & SDSC

UC Riverside

Corporation for Education Network Initiatives in California

CalREN-2 Network

UC Santa Barbara

UC Irvine

UCLA

ISI

USC

UC San Francisco

UC Santa Cruz

JPL

Merced

Northern CaliforniaGigaPOP

Southern CaliforniaGigaPOP

San Diego GigaPOP (Future)

Sacramento

(Planned but not funded)

OC-12c

OC-48

CSU

UC San Diego

Caltech

Planned N-SInterconnect

With thanks to David Wasley

34

Importance of carrier hotels• Facilities where multiple telecommunications carriers have presences

• Orginally exchanged just voice traffic• Often attract other related businesses – ISPs, Web hosting• Westin Bldg (Seattle), One Wilshire (LA), 60 Hudson (NYC)

• Very active area of capital investment in the U.S.

• Leading GigaPoPs establishing presences

• Essential distinguishing factors• Riser capacity• Fiber interconnection room (‘meet-me room’) • Power – production and backup• Ease and speed of construction for tenant buildout• Local exchange point for peering among ISP tenants

• Dark fiber from the local campuses is very enabling

35

Internet2 End-to-End Performance Initiative• Extending focus from connectivity to performance experienced by end user

• e2e will address all obstacles to performance• Application readiness and tuning• End system operating systems and networking support• Local Area Network and campus backbone upgrades• Outreach to end users and campus support teams

• Widely distributed, pro-active measurement

• Performance Evaluation and Response Teams (PERTs)• Hybrid of Network Operations Center and Applications Support Team

• Phased approach• 10-15 campuses and GigaPoPs self-select for first stage• Campus network/applications workshop – UCSD – December, 2000

36

End-to-End (e2e) PerformanceInitiativeHuman to Human Collaboration Experience

• Application• Operating system• Host IP stack • Network card• Local Area Network (LAN)• Campus backbone network• Campus connection to regional network/GigaPoP• GigaPoP connection to Internet2 national backbone

37

Defining E2E Success Metrics• Selecting set of appropriate core applications and services

• TCP applications – e.g., Web, file transfer• Internet-based telephony (VoIP)• Internet-based videoconferencing

– Multiple technologies with distinct service levels

• Pervasive multicast for multimedia and data distribution

• Scope• How broadly across the campus network should e2e be

supported?

• Timing• How quickly can these goals be met?• End user expectation management essential

38

Summary

• Second wave of backbone development concluding

• Advanced service deployment proceeding

• Utilization growing

• Moving to develop next generation backbone and to incent GigaPoP/campus connectivity upgrades

• Focusing on assuring e2e performance

• www.internet2.edu/abilene

39

Upcoming Meetings

APAN/Internet2/NLANR/TransPAC• 28-31 January 2001, Hawaii

Internet2 Member Meeting• 7-10 March 2001, Washington, DC

www.internet2.edu