NSF’s Evolving Cyberinfrastructure Program

Guy Almes <galmes@nsf.gov>

Office of Cyberinfrastructure

Cyberinfrastructure2005

Lincoln

16 August 2005

2

Nat

ion

al S

cien

ce F

ou

nd

atio

n Overview

Cyberinfrastructure in Context

Existing Elements

Organizational Changes

Vision and High-performance Computing planning

Closing thoughts

3

Nat

ion

al S

cien

ce F

ou

nd

atio

n

Cyberinfrastructure in Context Due to the research university’s mission:

each university wants a few people from each key research specialty

therefore, research colleagues are scattered across the nation / world

Enabling their collaborative work is key to NSF

4

Nat

ion

al S

cien

ce F

ou

nd

atio

n

Traditionally, there were two approaches to doing science: theoretical / analytical experimental / observational

Now the use of aggressive computational resources has led to third approach in silico simulation / modeling

5

Nat

ion

al S

cien

ce F

ou

nd

atio

n

Historical Elements Supercomputer Center program from 1980s

NCSA, SDSC, and PSC leading centers ever since

NSFnet program of 1985-95 connect users to (and through) those centers 56 kb/s to 1.5 Mb/s to 45 Mb/s within ten years

Sensors: telescopes, radars, environmental Middleware: of growing importance

6

Nat

ion

al S

cien

ce F

ou

nd

atio

n

‘97

Partnerships for Advanced Computational Infrastructure

• Alliance (NCSA-led)• NPACI (SDSC-led)

‘93

HayesReport

BranscombReport

‘95 ‘99

PITACReport

Terascale Computing

Systems

‘00

ITRProjects

ETFManagement & Operations

‘03

AtkinsReport

FY‘05‘08

Core Support

• NCSA• SDSC

Discipline-specificCI Projects

Supercomputer Centers

• PSC• NCSA• SDSC• JvNC• CTC

‘85

7

Nat

ion

al S

cien

ce F

ou

nd

atio

n

Explicit Elements Advanced Computing

Variety of strengths, e.g., data-, compute- Advanced Instruments

Sensor networks, weather radars, telescopes, etc. Advanced Networks

Connecting researchers, instruments, and computers together in real time

Advanced Middleware Enable the potential sharing and collaboration

Note the synergies!

8

Nat

ion

al S

cien

ce F

ou

nd

atio

nCRAFT: A normative example – Sensors + network + HEC

Univ OklahomaNCSA and PSCInternet2UCAR Unidata ProjectNational Weather Service

9

Nat

ion

al S

cien

ce F

ou

nd

atio

n

Current Projects within OCI Office of Cyberinfrastructure

HEC + X Extensible Terascale Facility (ETF) International Research Network Connections NSF Middleware Initiative Integrative Activities: Computational Science Integrative Activities: Education, Outreach & Training Social and Economic Frontiers in Cyberinfrastructure

10

Nat

ion

al S

cien

ce F

ou

nd

atio

n TeraGrid: One Component• A distributed system of

unprecedented scale• 30+ TF, 1+ PB, 40Gb/s net

• Unified user environment across resources

• User software environment User support resources

• Integrated new partners to introduce new capabilities

• Additional computing, visualization capabilities

• New types of resources: data collections, instruments

• Built a strong, extensible Team

• Created an initial community of over 500 users, 80 PIs

• Created User Portal in collaboration with NMI

courtesy Charlie Catlett

11

Nat

ion

al S

cien

ce F

ou

nd

atio

n

Key TeraGrid Resources Computational

very tightly coupled clusters LeMieux and Red Storm systems at PSC

tightly coupled clusters Itanium2 and Xeon clusters at several sites

data-intensive systems DataStar at SDSC

memory-intensive systems Maverick at TACC and Cobalt at NCSA

12

Nat

ion

al S

cien

ce F

ou

nd

atio

n

Online and Archival Storage e.g., more than a PB online at SDSC

Data Collections numerous

Instruments Spallation Neutron Source at Oak Ridge Purdue Terrestrial Observatory

13

Nat

ion

al S

cien

ce F

ou

nd

atio

n TeraGrid DEEP Examples

Lattice-Boltzman SimulationsPeter Coveney, UCLBruce Boghosian, Tufts

Joel Saltz, OSUReservoir Modeling

Animation pointed to by 2003 Nobel chemistry prize announcement.

Klaus Schulten, UIUC

Aquaporin Mechanism

Groundwater/Flood ModelingDavid Maidment, Gordon Wells, UT

Atmospheric ModelingKelvin Droegemeier,

OU

Advanced Support for

TeraGrid Applications:TeraGrid staff are “embedded” with applications to create

- Functionally distributed workflows

- Remote data access, storage and visualization

- Distributed data mining- Ensemble and

parameter sweeprun and data management

courtesy Charlie Catlett

14

Nat

ion

al S

cien

ce F

ou

nd

atio

n

Cyberresources

Key NCSA Systems Distributed Memory Clusters

Dell (3.2 GHz Xeon): 16 Tflops Dell (3.6 GHz EM64T): 7 Tflops IBM (1.3/1.5 GHz Itanium2): 10 Tflops

Shared Memory Clusters IBM p690 (1.3 GHz Power4): 2 Tflops SGI Altix (1.5 GHz Itanium2): 6 Tflops

Archival Storage System SGI/Unitree (3 petabytes)

Visualization System SGI Prism (1.6 GHz Itanium2+

GPUs)

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

courtesy NCSA

15

Nat

ion

al S

cien

ce F

ou

nd

atio

nCyberresources

Recent Scientific Studies at NCSA

Computational Biology

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Weather Forecasting

Molecular Science

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Earth Science

courtesy NCSA

16

Nat

ion

al S

cien

ce F

ou

nd

atio

n

Computing: One Size Doesn’t Fit AllD

ata

capa

bilit

y(I

ncre

asin

g I/

O a

nd s

tora

ge)

Compute capability(increasing FLOPS)

SDSC Data Science Env

Campus, Departmental and

Desktop Computing

Traditional HEC Env

QCD

Protein Folding

CPMD

NVOEOL

CIPRes

SCECVisualization

Data Storage/Preservation Extreme I/O

1. 3D + time simulation

2. Out-of-CoreENZOVisualization

CFD

ClimateSCEC

Simulation ENZOsimulation

Can’t be done on Grid(I/O exceeds WAN)

Distributed I/OCapable

courtesy SDSC

17

Nat

ion

al S

cien

ce F

ou

nd

atio

n

SDSC Resources COMPUTE SYSTEMS DataStar

1,628 Power4+ processors, soon growing to 2,396 processors

IBM p655 and p690 nodes 4 TB total memory Up to 2 GBps I/O to disk

TeraGrid Cluster 512 Itanium2 IA-64

processors 1 TB total memory

Intimidata Only academic IBM Blue

Gene system 2,048 PowerPC processors 128 I/O nodes

http://www.sdsc.edu/user_services/

SCIENCE and TECHNOLOGY STAFF, SOFTWARE, SERVICES

User Services Application/Community Collaborations Education and Training SDSC Synthesis Center Community SW, toolkits, portals, codes http://www.sdsc.edu/

DATA ENVIRONMENT 1 PB Storage-area Network

(SAN) 6 PB StorageTek tape library DB2, Oracle, MySQL Storage Resource Broker HPSS 72-CPU Sun Fire 15K 96-CPU IBM p690s

http://datacentral.sdsc.edu/

Support for community data collections and

databases

Data management,

mining, analysis, and preservation

courtesy SDSC

18

Nat

ion

al S

cien

ce F

ou

nd

atio

n

Overview of IRNC Program 2004 Solicitation (NSF 04-560, see www.nsf.gov) “NSF expects to make a small number of awards to provide

network connections linking U.S. Research networks with peer networks in other parts of the world”

“The availability of limited resources means that preference will be given to solutions which provide the best economy of scale and demonstrate the ability to link the largest communities of interest with the broadest services”

Follow-on to “High-Performance International Internet Services” (HPIIS) 1997

19

Nat

ion

al S

cien

ce F

ou

nd

atio

n

2005 IRNC Awards Awards

TransPAC2 (U.S. – Japan and beyond) GLORIAD, (U.S. – China – Russia – Korea) Translight/PacificWave (U.S. – Australia) TransLight/StarLight, (U.S. – Europe) WHREN (U.S. – Latin America)

Example use – Open Science Grid involving partners in U.S. and Europe, mainly supporting high energy physics research based on LHC

20

Nat

ion

al S

cien

ce F

ou

nd

atio

n

NSF Middleware Initiative (NMI)

Program began in 2001 Purpose - To design, develop, deploy and support a

set of reusable and expandable middleware functions that benefit many science and engineering applications in a networked environment

Program encourages open source development Program funds mainly development, integration,

deployment and support activities

21

Nat

ion

al S

cien

ce F

ou

nd

atio

n

Example NMI-funded Activities

GridShib – integrating Shibboleth campus attribute services with Grid security infrastructure mechanisms

UWisc Build and Test facility – community resource and framework for multi-platform build and test of grid software

Condor – mature distributed computing system installed on 1000’s of CPU “pools” and 10’s of 1000’s of CPUs.

22

Nat

ion

al S

cien

ce F

ou

nd

atio

n

Organizational Changes Office of Cyberinfrastructure

formed on 22 July 2005 had been a division within CISE

Cyberinfrastructure Council chair is NSF Director; members are ADs

Vision Document started HPC Strategy chapter drafted

Advisory Committee for Cyberinfrastructure

23

Nat

ion

al S

cien

ce F

ou

nd

atio

n

Education &Training

DataTools &Services

Collaboration &Communication

Tools &Services

Cyberinfrastructure Components

High PerformanceComputing

Tools & Services

24

Nat

ion

al S

cien

ce F

ou

nd

atio

n

Vision Document Outline Call to Action Strategic Plans for …

High Performance Computing Data Collaboration and Communication Education and Workforce Development

Complete document by 31 March 2006

25

Nat

ion

al S

cien

ce F

ou

nd

atio

n

Strategic Plan for High Performance Computing Covers 2006-2010 period Enable petascale science and

engineering by creating a world-class HPC environment Science-driven HPC Systems Architectures Portable Scalable Applications Software Supporting Software

Inter-agency synergies will be sought

26

Nat

ion

al S

cien

ce F

ou

nd

atio

n

Cyberinfrastructure VisionNSF will lead the development and support of a comprehensive

cyberinfrastructure essential to 21st century advances in science and engineering.

Internet2 Universities206 University Members, May 2005

Internet2 Universities206 University Members, May 2005Science Communities and Outreach

¥ Communities¥ CERNÕs Large Hadron Collider

experiments

¥ Physicists working in HEP andsimilarly data intensive scientificdisciplines

¥ National collaborators and thoseacross the digital divide indisadvantaged countries

¥ Scope¥ Interoperation between LHC

Data Grid Hierarchy and ETF

¥ Create and Deploy ScientificData and Services Grid Portals

¥ Bring the Power of ETF to bearon LHC Physics Analysis: Helpdiscover the Higgs Boson!

¥ Partners¥ Caltech

¥ University of Florida

¥ Open Science Grid and Grid3

¥ Fermilab

¥ DOE PPDG

¥ CERN

¥ NSF GriPhyn and iVDGL

¥ EU LCG and EGEE

¥ Brazil (UERJ,É )

¥ Pakistan (NUST, É )

¥ Korea (KAIST,É )

LHC Data Distribution Model