42
MCNC Grid Computing and Networking Services (GCNS) North Carolina Statewide Grid Western Carolina University John Killebrew Director - NCREN

NCREN Director on Grid Computing and Networking

Embed Size (px)

Citation preview

Page 1: NCREN Director on Grid Computing and Networking

MCNC Grid Computing and Networking Services (GCNS)

North Carolina Statewide Grid

Western Carolina University

John KillebrewDirector - NCREN

Page 2: NCREN Director on Grid Computing and Networking

GCNS UNIVERSE

UNC-OP

e-NC

RENCI

ITS

Universities

GeneralAssembly

Industry

RTI RDI

MCNC GCNS

SURA

Internet2 NLRServices and Relationships

DukeNet

ERCWC

WinstonNet

SERNet

Page 3: NCREN Director on Grid Computing and Networking

NCREN Video ServicesMCNC GCNS

• Broadcast quality interactive video conferencing• 20 sites continuous presence audio and video• Full bridging interoperability among

Broadcast Quality video sitesH.320 private line sitesH.320 dial (ISDN) sitesH.323 Video-over-IP sitesMPEG2 Video-over-IP sites

• Live monitoring of all scheduled Conferences

• Satellite Links•Downlinks for any/all campuses•Uplinks via Microspace from any/all campuses

Page 4: NCREN Director on Grid Computing and Networking

MCNC Network Access and Services

• Direct Access to– NCREN– Internet2 (Abilene)– Research networks (NLR, NCNI)– National Labs– NC Grid, MCNC Enterprise Grid

• Grid and Data Center Offerings– Co-location hosting services– Managed hosting services– Managed WAN hosting– Use of Grid Compute & Data resources– NEW! Grid Technology Evaluation Cntr

•benchmarking, interoperability, integration, and training

Page 5: NCREN Director on Grid Computing and Networking

“Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations”

Ian Foster & Carl Kesselman, 2002

What is a Grid ?

I don’t know, but…

“Context, Not Products, Defines Grid Computing” Gartner, 2004

Page 6: NCREN Director on Grid Computing and Networking

This is a Grid !

An IT Utility

A Grid Middleware (the glue)

And Networked DistributedResources

Page 7: NCREN Director on Grid Computing and Networking

The North Carolina Economy Challenge

400000

450000

500000

550000

600000

650000

700000

750000

800000

850000

1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002

Manufacturing

Information, FinancialActivities, Professionaland Business Services

Employment by North American Industry Classification System (NAICS)

Source: The Employment Security Commission of NC, Labor Market Information

Page 8: NCREN Director on Grid Computing and Networking

• Access: transparent, remote, secure, wireless• Sharing: enable collaboration over the network• Failover: migrate/restart applications automatically• On Demand: get resources, when you need them• Productivity: more work done in shorter time• Virtualization: access compute services, not servers• Heterogeneity: platforms, OSs, devices, software• Resource Utilization: increase from 20% to 80+%• Virtual Organizations: build & dismantle on the fly

Grid BenefitsDepartment, Enterprise, and Global Grids

Page 9: NCREN Director on Grid Computing and Networking

Grid Computing: Competitive Advantage for N.C.

• Rapidly process and solve problems beyond an individual organization's capacity

• Leverage very powerful but inexpensive networked resources

• Connect and share computing resources seamlessly - regardless of system type or physical location

• Reduce IT costs• Access disparate data across multiple domains• Enhance Education, R&D and Business processes• Create innovative products and services• Reduce design time and time to market• Increase quality (reduce defects)• Accelerate product development (e.g. drugs)

Education, Research,Government, Industry

Technology

Page 10: NCREN Director on Grid Computing and Networking

However, There is Still a Long Way to Go !

Grids are over-hyped: currently, they promise much more

than they can really offer.

Grid technology is far from mature and complete.

Grid standards are (mostly) still missing.

Grids are very complex IT infrastructures.

Grids bring new challenges: sharing resources, loosing

direct control, security, intellectual property, legal, social,

political issues . . .Therefore, MCNC - GCNS !

Page 11: NCREN Director on Grid Computing and Networking

CiscoEPA

Our Foundation for Grid: NCREN

4-7 MCNC-owned Clusters distributed throughout the stateLocations still under evaluation

Internet Internet

2NLR

Internet Internet

2NLR

InternetInternet

Existing: Blend of owned and leased fiber and circuits moving toward resilient rings powered by Cisco routers

Planned: Strong focus on owned and leased fiber, Lambda, and few circuits, in resilient rings powered by Cisco routers and Wave Division Multiplexers

Page 12: NCREN Director on Grid Computing and Networking

The GridThe next IT Infrastructure of North Carolina

2002 2003 2004 2005

NC BioGrid

• Proving ground for Grid• Successful prototype apps• Catalyst for collaboration• International recognition

Page 13: NCREN Director on Grid Computing and Networking

2002 2003 2004 2005

NC BioGrid

MCNC Enterprise Grid

• Cluster and SMP resources • Research platform for GTEC• Core component in NCGrid• Revenue generation

The GridThe next IT Infrastructure of North Carolina

Page 14: NCREN Director on Grid Computing and Networking

2002 2003 2004 2005

NC BioGrid

MCNC Enterprise Grid

NC Grid Initiative

• State-wide partnership• Leverage lessons learned• Grid education & training resource• Enable first mover applications

The Grid The next IT Infrastructure of North Carolina

Page 15: NCREN Director on Grid Computing and Networking

NC BioGrid

MCNC Enterprise Grid

NC Statewide Grid Initiative

MCNC & The Grid Infrastructure for North Carolina

NCREN North Carolina Research & Education Network

MCNCMicroelectronics Center of North Carolina- Founded in 1980- Independent, private, non-profit organization- Operates NCREN since 1984- Past: Supercomputer Center for universities- Now: Grid Service Provider, offering Video, Network, Grid, and Datacenter Services- 50+ employees

Page 16: NCREN Director on Grid Computing and Networking

MCNC-GCNS Roadmap

Awareness Creation

Easy Access

Grid Service Provider

Page 17: NCREN Director on Grid Computing and Networking

Our Vision :

The Three Waves of Grid Computing

The Research Wave The Industry Wave The Consumer Wave

Technology, Prototypes Grid-Enabled Products Commodity Virtual Organizations Enterprise Solutions IT Utility Standards Interoperability Integration GGF, IETF, OASIS GGF, EGA, IETF, OASIS Legal, Ethical, Political Orgs

GCNS: “Awareness Creation” GCNS: “Easy Access” GCNS: “Grid Service Provider”

Page 18: NCREN Director on Grid Computing and Networking

GCNS Mission Advance education, innovation and economic

development

throughout North Carolina by delivering next generation information technology that enables the academic, research, government

and business communities

to discover, create, share and apply knowledge.

Page 19: NCREN Director on Grid Computing and Networking

Higher Education Grid Constituencies

• Chancellors/Presidents: University Outreach

• CIO’s: Providing and running IT infrastructure

• CRO’s, Deans, Key faculty: Education and Research

• Students

Page 20: NCREN Director on Grid Computing and Networking

Chancellors/Presidents

• Play key important role in NC further education and economic development

• Improve global competitive position of NC universities

• Offer the best infrastructure for research and education

• Become early part and active participant in the next-gen Advanced Internet

Page 21: NCREN Director on Grid Computing and Networking

CIO’s

• Offer service-oriented IT architecture (hide complexity)

• Enable much more efficient resource management

• Balance resource utilization within and across Campuses

Page 22: NCREN Director on Grid Computing and Networking

CRO’s, Deans, Key Faculty

• Offer and use larger and richer set of resources

• Develop, provide, and use more powerful tools for joint research collaborations

• Enable researchers to concentrate on research (and not on the resources)

Page 23: NCREN Director on Grid Computing and Networking

Core Strategy Components

• Briefing key constituencies about Grid Computing• Train key IT and research staff• Develop joint Graduate/PhD program• Build and use NC Grid as foundation (think globally, start

locally)• Grid-enable existing resources and join the NC Grid• Offer training to users to grid-enable applications• Work with users to port applications to the NC Grid• Start QuadA project with partners

Page 24: NCREN Director on Grid Computing and Networking

Grid Technology Evaluation Center The Epicenter for Accelerating Grid Adoption

Influential Networkof Key-Players

Independent Lab:Eval & Testing

Joint BizDev, PR & Marketing

Grid Training& Education

Customer BriefingCenter

From Innovation to Deployment

Socio-EconomicalMission

Page 25: NCREN Director on Grid Computing and Networking

So, are there really any working Grids out there?

•Yes!….In North Carolina even…

+ Wachovia+ NC BioGrid+ SCOOP+ EPA

Page 26: NCREN Director on Grid Computing and Networking

NC BioGrid

One of the nation’s first grid test beds for computing, data storage and networking resources for life sciences research

• Installed in Summer 2002, heterogeneous hardware and OS platforms

• More than 80 organizations

• Dedicated systems for testing grid middleware and developing grid applications for bioinformatics

• Spans multiple administrative domains with systems located at MCNC, NC State, UNC-CH & Duke

• Established a Certificate Authority

Established to research and implement new grid computing technologies that will enable researchers and educators throughout North Carolina to take full advantage of the genomic revolution

.

Page 27: NCREN Director on Grid Computing and Networking

SURA SCOOP South-eastern Coastal Ocean Observing Program

The Challenge: - More than half of the nation's tidal shores home to 80 million people - coastal zone is undergoing environmental and ecological changes - threaten the sustainability of the region's economies and marine resources

The Solution: - develop a Grid of sensors and linked computers - fully integrating several observing systems in the southern region - provide data, in real-time and at high speed, for more reliable, accurate and timely information To help guide effective coastal stewardship, plan for extreme events, facilitate safe maritime operations, and support coastal military security.

Page 28: NCREN Director on Grid Computing and Networking

• And, how about other states?

• Grid Development efforts announced in•Colorado•West Virginia•Louisiana•Texas•Virginia

Plus….the following are also working….

Page 29: NCREN Director on Grid Computing and Networking

Access Gridfor Education and Training

• Builds on our proven and advanced video infrastructure

• Utilizes Multicast Backbone

• Real time, continuous presence audio, video, data

• Purpose-built sites encourage natural, seamless interaction

– Desktop implementation available

• Multi-Platform: including Win X, Linux/Unix– Mac OS X support coming soon

H.323 Voice & Video Over IP (V2oIP), Ubiquitous 352 x 288 @ 128 kbps - 2 Mbps, Multipoint capable

H.320 Multipoint via third-party

MPEG-2 720 x 480 @ 5.5 Mbps NCREN Video Network supports multipoint

Page 30: NCREN Director on Grid Computing and Networking

Earth System Grid

Goal: address technical obstacles to the sharing & analysis of high-volume data from advanced earth system models

Page 31: NCREN Director on Grid Computing and Networking

Field Equipment

Laboratory Equipment

Remote Users

Remote Users: (K-12 Faculty and Students)

Instrumented Structures and Sites

Leading Edge Computation

Curated Data Repository

Laboratory Equipment

Global Connections

(Faculty, Students, Practitioners) Simulation

Tools Repository

Network for Earthquake Eng. Simulation

www.neesgrid.org

Page 32: NCREN Director on Grid Computing and Networking

Butterfly.net: Enterprise Optimization

• A scalable, resilient infrastructure for creating & running massive multiplayer games

• Developers avoid upfront costs• Improved end-user experience• Demonstrated 8x increase in

profitability over centralized model• Dynamic provisioning & on demand

capacity• Uses Globus Toolkit & runs on IBM

Global Services hosting environment

Page 33: NCREN Director on Grid Computing and Networking

• Metropolitan Grid across the Universities

• Stable, well-managed HPC resources

• Supporting multi-disiplinary research

• And local industries (e.g. Rolls Royce)

Slides: Courtesy James Coomer

Page 34: NCREN Director on Grid Computing and Networking

Grid Collaboration MCNC - Universities

1 + 1 = 31 + 1 = 3

CommonCommonInterests Interests

Grid ComputingGrid ComputingHigher EducationHigher Education

Economic DevelopmentEconomic DevelopmentThought LeadershipThought Leadership

UniversitiesUniversities MCNCMCNC Top Research Deployment Top Research Deployment Education Education InfrastructureInfrastructureVertical Applications Horizontal Vertical Applications Horizontal Middleware Middleware

Complementary FocusComplementary Focus

Page 35: NCREN Director on Grid Computing and Networking

The Innovation Engine

Time Machines

[email protected] http://www.mcnc.org

Thank You !

Page 36: NCREN Director on Grid Computing and Networking

32-CPU SGI AltixLinux SMP Server

128-CPU IBM LinuxCluster (64 nodes)

8-TB Storage

LSF Master Job Scheduler

Grid Gatekeeper / Interactive Nodes

Global Grid Resource DB

(GIIS)

Users

Campus Grids

GCNS Enterprise GridGCNS Enterprise Grid

AvakiDataGrid

Data G

rid A

ccess Servers

(8 total, i.e. 1 p

er 8 no

des)

Page 37: NCREN Director on Grid Computing and Networking

Network, Grid and Data Center Services

NCREN

State-wide Grid

Services

Enterprise Grid

Services

Value-add Information

Systems Services

Self-serve Data Center

Services

DATA CENTER

Hosting & Infrastructure Grid Computing

GTEC, NLR, ANR and other Innovation Initiatives

Information Security Services

Data Archival Services

Information Assurance

DEPLOYMENT

Page 38: NCREN Director on Grid Computing and Networking

Co-location Services

• GCNS supplies server cabinets– 19” EIA, 42-RU, 4-post APC-brand enclosures– Perforated, lockable doors– 84” high x 24” wide x 40” deep– Additional APC cabinet accessories available– $1,200 per cabinet one-time setup fee

• Base monthly rate of $500* per cabinet– Includes two 20-Amp, 110-VAC power circuits– One 100-Mbps network uplink– Four public IP addresses– 30-day average network utilization of up to 0.5-Mbps

•Additional network utilization options are available– *20% discount available for educational customers

Page 39: NCREN Director on Grid Computing and Networking

Data Mirroring and Archival

Network ApplianceR200 Nearstore Filer

8-TB initially(scales up to 96-TB)

Virtual Filers

DataGrid

Interface

NetAppSnapVault

NetAppSnapMirror

NetAppSnapMirror

Customer 1Grid

Users

Customer 2Linux

Servers

Customer 3NetAppServers

Customer 4NetAppServers

NCRENNCREN

Page 40: NCREN Director on Grid Computing and Networking

NCREN

Qwest Internet (Oc12 PoS)

Abilene (Internet2 OC48 PoS)

Level3 (GbE)

RaleighRTP

OC48 SRP Ring counter-rotating ring <=50ms reroute Fully active redundancy

Greensboro

Winston-Salem

Charlotte

Level3 (GbE)

Duke (GbE) NCSU (GbE)

UNC-CH (GbE)

UNC-GNCATASU

WFUWSSUNCSA

UNC-C

76097609

Wilmington

Fayetteville

Greenville

ECUECSUCMST

FSUUNCP

UNCW

AshevilleUNCAWCUERC

Greenville

OC12 SRP Ring counter-rotating ring <=50ms reroute Fully active redundancy

OC12 SRP Ring counter-rotating ring <=50ms reroute Fully active redundancy

NCCU

Qwest (OC12 PoS)

Hickory

Page 41: NCREN Director on Grid Computing and Networking

GTEC Projects Driving Grid Adoption

2000 20004 2008

MCNC Grid SP Grid ColLab Kids Grids Gaming Grid School Grids Grid-Info Grid Download-The-Grid AAAA Project Grid Training Courses Startup Grid Grid Appliance Grid Portal GGF & EGA NC Statewide Grid MCNC Enterprise Grid NC BioGridMCNC Supercomputing

Page 42: NCREN Director on Grid Computing and Networking

CY03CY03 CY04CY04 CY0CY055 CY06CY06

MCNC: 3-Year GCNS Grid Roadmap

Easy Access: Training, Web Courses

Access Grid Node

Grid Appliance

QuadA

Grid Service Provider:Network, Computing, Data, Video Partner with SC Sites Build GTEC Service Portfolio Start Grid Consulting Annual GSP Workshop

Awareness Creation:Conferences, Workshops, PRMCNC Enterprise GridNC BioGridNC Statewide Grid