NEESgrid: Virtual Collaboratory for Advanced Earthquake ... · NEESgrid: Virtual Collaboratory for Advanced Earthquake Engineering ClusterWorld Conference June 26, 2003 Tom Prudhomme

1

NEESgrid: Virtual Collaboratory for Advanced Earthquake Engineering

ClusterWorld ConferenceJune 26, 2003

Tom PrudhommeNSF NEESgrid System Integration Project

Division Director, CybercommunitiesNational Center for Supercomputing Applications

[email protected]

2

Presentation Topics

NEES OverviewNEESgrid OverviewUser Requirements and UsabilityNEES and the World of Scalable Grid Computing

3

What is NEES?

NEES = Network for Earthquake Engineering Simulation– NEES is a distributed array of experimental sites, grid-based data

repositories, tool archives, and computational resourcesNEES has four components– The consortium, operational management of NEES 2004-2014– The consortium development team (CDT) builds the consortium– The experimental sites provide facilities, data and content– The systems integration (SI) effort builds the collaboratory

SI drivers include collaboration, seamless access to resources (telepresence, curated repositories, scalable HPC), experimental-numerical coupling, discoveryThe full title of NEES is the “George E. Brown, Jr. Network for Earthquake Engineering Simulation”

4

NEES: Physical and Computational Simulation

Network for Earthquake Engineering Simulation

Goal: create collaborative network of experimental sites

at fifteen U.S. universities

5

NEES Phase I Equipment Sites

6

UC Davis Soil Centrifuge

7

Oregon State Tsunami Facility

8

Computational Simulation

Example: San Francisco-Oakland Bay Bridge– Multiple non-linear systems– Foundation is saturated (fully-coupled multiphysics)– Complexity from juxtaposition of forms (unstructured)

FearsLab Donald G. Fears Structures LaboratoryUniversity of Oklahoma

9

NEES: Community Building

A Diverse Community– multidisciplinary groups

• engineering, architecture, geology, • urban planning, computing , …

– geographic distribution of people/resourcesEnablers of Change– distributed, unique experimental facilities– multimodal collaboration systems– distributed, large-scale data archives– leading edge computing systems

Opportunities and Challenges– creating 21st century engineering infrastructure– nurturing a sustainable, multidisciplinary community

10www.touchgraph.com

11www.touchgraph.com

NEES

Collaboration

Grids

Collaboration

12

The NEES WinNew engineering capabilities– rapid assembly of virtual teams– access for all to remote facilities and experiments– interfaces to distributed data archives/experiment repositories

National and international cyberinfrastructure leverage– corporate and government commitments– billions of dollars in leverage

• commoditization of infrastructureDistributed facility and collaboration access– NEES experimental sites (ES)– distributed collaborators

Strong security features – secure user-level experiment control and data sharing

Resource discovery and monitoring services – available resource identification and continuous status monitoring

13

Field Equipment

Laboratory Equipment

Remote Users

Remote Users: (K-12 Faculty and Students)

High-Performance Network(s)

Instrumented Structures and Sites

Leading Edge Computation

Curated Data Repository

Laboratory Equipment

Global Connections

(FY 2005 – FY 2014)

Simulation Tools Repository

14

NEESgrid High-Level Architecture

Capabilities (data, simulation, collaboration, visualization, telepresence) validated and improved via demonstration scenarios and delivered via

– APIs and tools for users– services and interfaces at equipment and resource sites– management services for operation

System Resources: Compute, Network, Data Storage, Testing Sites

Grid Resource Management Middleware

APIs Supporting Higher Level Information Services

Collaboration Services

Data & InformationServices

TelepresenceServices

Simulation & Analysis Services

Portal, Web and 3rd Party End User Interfaces

APIs, Tools and Libraries Supporting End User Interfaces

Insight

15

NEESgrid Conception

NEESpop A

Experimental Equipment

Video I/O

Audio I/O

Site A: Experimental Data Producer

Hub A

Telepresence Equipment

Active PI

Data Cache

Site B: Remote Lead Investigator

Hub B

Internet Fabric and Operations

Site C: Passive Collaborator

Teleobservation Equipment

Passive co- PI

Data Cache

Hub C

Data Repository

Operations Center

Experimental Component

Campus Net Component

NEESgrid Component

16

NEESgrid Project Phases

Requirements and Design(7/00 – 2/02)

Component Testing(~8/01 – 6/02)

Early Adopter ProgramUNR, RPI, Oregon State

(3/02 – 1/03)

Operational System Prototype (7/03)

System Transition to Consortium (9/04)

Scoping Study, User Requirements Surveys & Site Visits,Project Planning Documents, System Architecture Specification

Testing Telepresence, Collaboration and Data Systems Interfaces; Grid Components

First Demonstration of Integrated NEESgrid System using UNR Testing Site, and Releaseof NEESpop Software to Equipment Sites

We are here (4/03)

Multiple Site IntegratedSystem Test

17

NEESgrid Deployment Relationships

A small set of adventurous sites

Expanded set of sites eager to engage

Available for all

Alpha 1.0Alpha 1.0Basic Functionality

Spring 2003

Release 2.0Release 2.0

Advanced Functionality

Fall 2003

Release 3.0Release 3.0

Production Hardened

Spring 2004

Expan

ding O

perat

ions

Expan

ding S

ite D

eploy

ments

Expan

ded T

raining

Free NEESpop Distribution

Alpha 1.1June 2003

18

Component Systems

Grid Services and System Software– Argonne National Laboratory– USC/Information Sciences Institute

Telepresence System (ANL)Collaboration Framework (Univ. of Michigan)Simulation Repository and Portal (OU)Data Repository and Tools (NCSA)Experimental Test Management (All)

19

Experimental Testing: 10/02 Demo

Earthquake simulation at UNR early adopter site– biaxial shake table with cameras and instrumentation– 40% scale model of a two span bridge

• concrete slab over steel girders

Bridge model instrumented with sensors– strain gauges, load cells, displacement, acceleration, …

20

Browser-based collaborative frameworkElectronic notebook for data recordingExperiment management tools– data and metadata

Streaming data and video– tele-observation of experiment– data channels from sensors

Data analysis and visualization


SAP2000 model

Tele-observation

MATLAB GUI Interface

21


22

Alpha 1.0 Software ReleaseAvailable February 12, 2003 to NEES early adopter sites– based on software from the November 2002 demonstration

Initial release features– includes hardened and tested software components

• logically packaged as service/system sets• NCSA Grid Packaging Technology (GPT)

– supports enhanced security mechanismsSoftware support– deployment team for help with software installation and

configuration– April workshop for user support and engagement– NEESGrid-discuss mailing list

• archived and searchableAlpha 1.1 update recently released

23gx&&

Multi-Site, On-Line Simulation Test (MOST)July 2003

UIUC Experimental Model

gx&&

f1

m1

NCSA Computational Model

m1

f1f2 f2

Colorado Experimental Model

gx&&

SAC Consortium Benchmark Structure

24

Multi-Site, On-Line Simulation Test (MOST)

ColoradoColoradoExperimental Model

gx&&

f2m1, θ1

F2

F1

e

gx&&=

gx&&

f1, x1

UIUCUIUCExperimental Model

gx&&

m1

f1 f2

NCSANCSAComputational Model

SIMULATIONSIMULATIONCOORDINATORCOORDINATOR

NEESpop NEESpop

NEESpop

UIUC MUST-SIM•Dan Abrams•Amr Elnashai•Dan Kuchma•Bill Spencer• and othersColorado FHT•Benson Shing•and others

25

MOST Column Test Specimens

Illinois Test Specimen

Colorado Test Specimen

26

Collaboration Framework

Team Research Scenario– Middle State University

• PI - Elliot Sullivan• PhD Student – Erica Granger

– State Institute of Technology – Equipment Site• PI – Eduardo Ruiz• Masters Student – Hideo Ishii

– Valley College• Harish Gupta – Electrical Engineer

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN

27

Elliot Sullivan

Develops composite lateral-resistance systemsConducts research using his school’s Reaction WallInterested in dynamic shake table testingHears about a new funding initiativeLearns about NEES at a conference


28

Month 1 – Visiting NEES


29

Month 1 – Joining NEES


30

Month 1 – Selecting Worksites


31

Month 1 – Consortium Worksite


32

Month 1 – Equipment Site Listing


33

Month 1 – Making Contact


34

Month 1 – State Tech Worksite


35

Month 1 –State Tech Equipment


36

Month 2 – Creating a Worksite


37

Month 3 – Discussing Proposal


38

Month 3 – Proposal Versioning


39

Month 4 – Planning Meeting


40

Month 9 – Scheduling Equipment


41



42



43

Month 10 – Publicity for Project


44

Month 10 – Uploading Resources


45

Month 12 – Test Day (Sullivan)


46

Month 12 – Test Day (Gupta)


47

Month 13 – Data Browser


48

Month 14 – Analytic Tools


49

Month 15 – Report Versioning


50

Month 15 – Revising Report


51SCHOOL OF INFORMATION

UNIVERSITY OF MICHIGAN

52

Data and Metadata

Data repository– 256 GB of storage area network (SAN) disk– NCSA archive support (~1.5 PB capability)– 24x7 operation

Repository data ingestion– Northridge finite element bridge model (March)– Selected SAC data (June)– MOST experiment data and metadata (July)– Bring your own data workshop(s) (Summer 2003)

Publicly accessible (Fall 2003)

53

Data Repository

Local storageserver

NEESpop

NMDS

HTTPS

User

CHEFGridFTP

Site

Local/remoteparticipant

54

NEESgrid Telepresence (TPM)

TelePresenceServer

Internet ApplianceTCP/IP Network Fixed Video Source

TeleRobotic Video Source

Sound/Audio Feeds

NEES PublicResources

NEES PrivateResources

DAQ & Control Systems

Data Lines to Expt. Equipment.

NEES Major Equipment

Video Switch

Video Amplifier

NEES POP

ARGONNE CHICAGO

55

Current TPM Installations

Early Adopters (EA)– UNR (tpm.ce.unr.edu) – ORST (neestpm.wave.orst.edu) – RPI (tpm.nees.rpi.edu)

MOST-SIM Demo– Colorado (neestp.colorado.edu) – UIUC (cee-zztt.cee.uiuc.edu)

Others– Electronic notebooks

• all NEES ESs at Argonne• neestpm.mcs.anl.gov

– UC-Davis• private Electronic Notebook

ARGONNE CHICAGO

56

Computational Simulation

Terascale Framework– New client-server

engineering portal for grid computing

– Scalable framework for finite-element HPC

OpenSEES Framework– simulating structural

and geotechnicalsystem performance

– Modeling and computational simulation tools

FearsLab Donald G. Fears Structures LaboratoryUniversity of Oklahoma

57

NEESgrid Training

Assist the NEES Equipment Sites – deploying NEESgrid software at their sites

Assist advanced users – learning to use NEESgrid capabilities

Beginning in April 2003– a series of six training sessions, each 2.5 days long– series hosted at NCSA/University of Illinois

First session targets developers at EA sites– system architecture overview and APIs– hands on “bring your application” interaction

Other session topics to follow– system administration

58

Science and Engineering Grids

Source: Randy Butler

59

NEES ImpactScience and engineering futures

– increasingly complex problems (disciplinary and interdisciplinary)– distributed, collaborative virtual teams– integrated experiment, theory, and simulation

NEES shares these features with other disciplines (for example)– high-energy physics (ATLAS and CMS)

• atlas.web.cern.ch– fundamental physics (LIGO)

• www.ligo.caltech.edu– astronomy/cosmology (NVO)

• www.us-vo.org– quantitative biology (BIRN)

• birn.ncrr.nih.gov– geosciences network (GEON)

• www.geongrid.orgThese and other projects are building

– infrastructure for 10-20 year research horizons– same teams and methods as NEES and NEESgrid

60

Globus Toolkit Growth

0

5000

10000

15000

20000

25000

30000

1997 1998 1999 2000 2001 2002

Dow

nloa

ds p

er M

onth

from

ftp.

glob

us.o

rg

DARPA, NSF, and DOE begin funding Grid work

NASA initiatesInformation Power Grid,DOE increases support

Globus Project winsGlobal Information

InfrastructureAward

MPICH-Greleased

The Grid: Blueprint for a New ComputingInfrastructure published

GT 1.0.0Released

Early ApplicationSuccesses Reported

GT 1.1.1Released

GT 1.1.2Released

GT 1.1.3Released

NSF & European CommissionInitiate Many New Grid Projects

GT 1.1.4 andMPICH-G2 Released

Anatomy of the GridPaper Released

FirstEuroGlobusConference

SignificantCommercial

Interest inGrids

NSF GRIDS CenterInitiated, DOE begins

SciDAC program

GT 2.0 betaReleased

Physiology of the GridPaper Released

GT 2.0Released

GT 2.2Released

Does not include downloads from:NMI, UK eScience, EU Datagrid,IBM, Platform, etc.

Source: Ian Foster

61

NSF TeraGrid

NCSA, SDSC, Argonne, CaltechUnprecedented capability– 13.6 trillion flop/s– 600 terabytes of data– 40 gigabits per second– Accessible to thousands

of scientists working onadvanced research

www.teragrid.org

62

Grid Communities & Technologies

Yesterday– Small, static communities, primarily in science– Focus on sharing of computing resources– Globus Toolkit as technology base

Today– Larger communities in science; early industry– Focused on sharing of data and computing– Open Grid Services Architecture emerging

Tomorrow– Large, dynamic, diverse communities that share a wide variety of

services, resources, data– New issues: Trust, distributed RM, knowledge

ARGONNE CHICAGO

63

“Gridified” Infrastructure

Industrial Perspective on Grids:A Wide Range of Applications

FinancialServices

DerivativesAnalysis

Statistical Analysis

Portfolio Risk

Analysis

DerivativesAnalysis

Statistical Analysis

Portfolio Risk

Analysis

Manufacturing

Mechanical/ Electronic

Design

Process Simulation

FiniteElement Analysis

Failure Analysis

Mechanical/ Electronic

Design

Process Simulation

FiniteElement Analysis

Failure Analysis

LS / Bioinformatics

Cancer Research

Drug Discovery

Protein Folding

Protein Sequencing

Cancer Research

Drug Discovery

Protein Folding

Protein Sequencing

Other

Web Applications

Weather Analysis

Code Breaking/ Simulation

Academic

Web Applications

Weather Analysis

Code Breaking/ Simulation

Academic

Sources: IDC, 2000 and Bear Stearns- Internet 3.0 - 5/01 Analysis by SAI

Grid

Ser

vice

s M

arke

t Opp

ortu

nity

200

5

Unique by Industry with Common Characteristics

Energy

Seismic Analysis

Reservoir Analysis

Seismic Analysis

Reservoir Analysis

Entertainment

Digital Rendering

Digital Rendering

Massive Multi-Player

Games

Massive Multi-Player

Games

Streaming Media

Streaming Media

ARGONNE CHICAGO

64

Global Knowledge Communities

65

Designing for The Long Term: Future NEES Users

http://www.neesgrid.org