Building Cyber-Infrastructure and Supporting eScience

Satoshi SekiguchiDirector, Grid Technology Research Center,

AIST, JAPANs.sekiguchi@aist.go.jp

Grid TechnologyGrid Technology

Upper MiddleUpper Middle

HPCHPC

Portal/ASPPortal/ASP

Operation/Human resoureOperation/Human resoure

High speed network (over 1G – 10G)

Ubiquitous GridUbiquitous Grid

IPv6IPv6 SecuritySecurity

Wireless networkTremendous Sensor

Huge datastream

GPSGPS

CamCam

sensorsensor

MobileMobile

Bloodpressure

Bloodpressure

Earth QEarth Q

Database/ExpressionDatabase/Expression

FTTH/xDSL/Wireless AP/100M-FTTH/xDSL/Wireless AP/100M-

DistributedStorage10PB

DistributedStorage10PB

Community InfraCommunity InfraScience and Engineeringplatform

Science and Engineeringplatform

Mega Computing1PFLOPSPC x 100MSC x 10000

Mega Computing1PFLOPSPC x 100MSC x 10000

Peta Scale GridPeta Scale Grid

Nano-techInformatic

s

Nano-techInformatic

s

Access Grid

Access Grid

ChemInfoma

ChemInfoma

Personal Wether s

ervice

Personal Wether s

ervice

Bio-Informatics

Bio-Informatics

MedicalInformatic

s

MedicalInformatic

s

Lower MIddleLower MIddle

P2P gridP2P grid

WebserviceWebservice

eBiz, service

eBiz, service SolutionSolution

UtilityUtility

CommunityCommunity

ITBL/SuperSINETSETI/United Device

METI program

ＪＳＴ

ＩＰＡ

ＪＳＴ

Server tech

Grid enabled EngineeringGrid enabled Engineering Grid enabled BusinessGrid enabled Business

GRID: Concept and Reality

Key Concept:

Resource VirtualizationComputers, storage, sensors, networks, softwares, people/organization …

WWW – Whenever, Wherever, WhoeverTransparency – Ubiquitous (?)

Similarity – Electric Power Service

Plug-in at consumer

Public utilities – social infrastructure99.9....9% availability, reliability, security

安定・安心・安全

Detector for ALICE experiment

Detector forLHCb experiment

GRID: Consumers scenario (sensors)

VLBI: Kashima 34m telescope

To provide infrastructure and facilities needed for next major stages of collaborative research in:

genomics and bioscience

particle physics

astronomy

climatology

engineering design

social sciences

Medical Engineering

JVO Bio Grid

Correspondences between agencies (US-JP)

JPN

METI

MEXT

AIST

GTRC

Univ

U.Tokyo

Titech

JAERI

JAMSTEC

RIKEN

NII

Council S&T

ES ctr

US

DOE

DOC

NSF

Coordination Office

JST

JSSP

NEDO

IPA

Sci, Tech,& Edu(sports, culture)

Commerce,Industry, Energy

RICS

IMS

Env.,nanotech, Bio, Life, etc

TACC

Matsuoka

Sekiguchi

E-Science & Computation ScienceSOUM

CRL

AIST Grid Projects (Underway)

project participants FundingAgency, Period, Budget

(2002/Total)

Focus Accomplishment

Ninfhttp://ninf.apgrid.org/

AISTTITECH

U. TsukubaKyoto U.

JST, METI, IPA 1994- $2.5M/6.0M

Develop Grid Middleware and application (Ninf-G /Grid RPC)

Ninf-G Release,Preliminary application development

ApGridhttp://www.apgrid.org/

Grid partnership in Asia Pacific Region

MEXT, 2002-2004 $0.3M / $0.9M

partnership for Grid computing in the Asia Pacific region. Resource sharing, Grid development, mutual support for applications development

Engineering guideline availableAnnual workshops and core meeting for engineering

Grid Data Farm http://datafarm.apgrid.org

AIST,TITEC,KEK,U.-Tokyo,NEC

METI, 2002 – 2002 $ 2.3M

Construction of a Peta- to Exascale parallel filesystem exploiting local storages of PCs spread over the world-wide Grid

Involved in the ATLAS/CERN Project and the preliminary version of the GFarm is available. Also, demonstrated at SC02

GRID Drug Design

AISTU. Tsukuba

Tokushima U.JST, 2001-03, $0.4M / 1.2M

Drug design system on grids Prototyped

AIST GridAIST

Osaka U.AIST, 2002-04 $1.5M / 4M

Access Grid, Medical application, tele science

Prototyped

Quantum Chemistry Grid

AIST AIST, $1M Provide user friend portals Open for limited users

Grid Consortium Japan

AIST and Business section

$0.1M Venues for info exchange, GGF franchise 36 companies, 150 memberships

Other Grid related Projects (Underway)

project participants Contacts SupportAgency, Period,

Budget (2002/Total)

Focus Accomplishment

Bio GridOsaka-u, pharmacy

Industry, NEC, AIST,

MEXT 2002-2006 Bio Application deployment

Consortium organized

ITBLJAERI, RIKEN, NAL, http://www.itbl.

jp/MEXT, 2001-2005 Application for

supercomputer ensemble

1st phase done

Super SINET

Universities http://www.nii.ac.jp/

MEXT, 2001-04

unknown

10G backbone 1st phase done

NaReGIUniversityNational Labs

NA MEXT, 2003-07,

FY02 4.5B JPY

Fy03 2B JPY (x5yrs)

National Research Grid Infrastructure

Inaugurated

Business

Grid

IT vendors

AIST

NA METI, 2003-05

Total 2.8B JPY x 3yrs

Business application Approved/prepare for public call

National Research Grid Initiative (NAREGI)

A new R&D project funded by MEXT

FY2003-FY2007

~2B JPY budget in FY2003

1.5B for Grid R&D, 0.5B for Nano-tech apps.

One of Japanese Government’s Grid Computing Projects

Selected National Labs. Universities and Industry are to be involved in the R&D activities

National Research Grid Initiative

To develop a Grid Software System R&D in Grid Middleware and Upper Layerprototyping for future Grid Infrastructure in scientific research

To provide a Testbed to prove that the High-end Grid Computing Environment

100+Tflop/s expected by 2007practically utilized in the Nano-science Simulations over the Super SINET.

To Contribute to Standardization Activities, e.g., GGF

To Participate in International Collaboration Europe, U.S., Asian Pacific

Participating Organizations

National Institute of Informatics (NII/MEXT)

Site for R&D in Grid Software and Networking

Institute for Molecular Science (IMS/MEXT)

Site for R&D in Computational Nano-sciences and Simulation Software Platform

Universities and National Laboratories(Joint R&D)

AIST, Tokyo Inst. Tech, Osaka-u, Kyushu-u, Kyushu Inst. Tech., etc.

Research Collaboration

ITBL Project, National Supercomputing Centers etc.

Participating Vendors

IT and Chemicals/ Materials

Hokkaido University

HITACHI SR8000HP Exemplar V2500HITACHI MP5800/160Sun Ultra Enterprise 4000

Tohoku University

NEC SX-4/128H4(Soon SX-7)NEC TX7/AzusA

University of Tokyo

HITACHI SR8000HITACHI SR8000/MPPOthers (in institutes)

Nagoya University

FUJITSU VPP5000/64FUJITSU GP7000F model 900/64FUJITSU GP7000F model 600/12

Osaka University

NEC SX-5/128M8HP Exemplar V2500/N

Kyoto University

FUJITSU VPP800FUJITSU GP7000F model 900 /32FUJITSU GS8000

Kyushu University

FUJITSU VPP5000/64HP GS320/32FUJITSU GP7000F 900/64

Inter-university Computer Centers(excl. National Labs) circa 2002

Tokyo Inst. Technology (Titech)

NEC SX-5/16, Origin2K/256HP GS320/64

University of Tsukuba

FUJITSU VPP5000CP-PACS 2048 (SR8000 proto)

Proposed Solutions

Diversity of Resources ( 松竹梅 “ Shou-Chiku-Bai”)松 (“shou” pine) – ES – like centers 40-100Teraflops x (a few), 100-300 TeraFlops nationwide竹 (“chiku” bamboo) – Medium-sized machines at SCs, 5-10 TeraFlops x 5, 25-50 TeraFlops aggregate / Center, 250-500 TeraFlops total梅 (“bai” plumb) – small clusters and PCs spread out throughout campus in a campus Grid x 5k-10k, 50 -100 TeraFlops / Center, 500-1 PetaFlop Nationwide

Division of Labor between “Big” centers like ES and Univ. Centers, Large-medium-small resourcesA National Grid Infrastructure to virtualize/federate these Resources ($120 mil approved 2003-2007)

Univ SCs

ES’s

Original slide: courtesy of S.Matsuoka

Everyday Grid – Tier model

高速ネットワーク (over 1G – 10G)

Larger Jobs

SITE-A SITE-B SITE-C

松 PINE

竹 Bamboo

梅 Plum

Center for Grid R&D (Jinbo-cho, Tokyo)

AIST super cluster for e-Science platform

Target performance: 10 ～ 20 TFLOPS for LinpackChallenge for one of the fastest clusters in the world

Development of ‘real-use’ cluster systemDelivered 1Q(’04)

SoftwareSCore System Software

for Cluster managemnet and communication

Compilers and utilitiesC,C++ 、 FORTRAN(77, 90)MPI 、 OpenMPParallel debugger, Performance monitoring tools

GRID supportGlobus, condor, GridRPC, etc

Computing nodes (2000 ～ 1000 Pus)Opteron 2GHz4GB/proc

Interactive nodesNetwork for computation

RDMA support, >3Gbps/link, High bi-section BWMyrinet…

Network for management and data transfer

Gigabit EthernetFile system

Disks of each node are used for scratchShared file system (NAS …)

Components

Overviews

(26) MyrinetE128

(4) Cluster Unit Group

(66) 2GbMyrinet

P32 Cluster

(2) MyrinetE128

(256)2Gb Myrinet

(1) Catalyst 6513

(143)1Gb Ethernet

(66) 2GbMyrinet

(66) 2GbMyrinet

(66) 2GbMyrinet

(1) 10Gb

Ethernet

M64 Cluster

(2) SystemService Node

(2)

1Gb

Eth

erne

t

(2) 2Gb Myrinet

(1) Catalyst 6513

Public Network

(1) 10Gb

Cluster Unit

(2) CentralManagement Node

(2)

1Gb

Eth

erne

t

Cluster Unit Cluster Unit Cluster Unit

(6)

1Gb

Eth

erne

t

(6)

1Gb

Eth

erne

t

(6)

1Gb

Eth

erne

t

(6)

1Gb

Eth

erne

t

Original slide: courtesy of IBM Japan

Summary

E-Science is a great concept for future our lifeScientists, Education, School, HomeCommercial scene

Grid meets e-Science needs perfectlyResource sharing & virtualizationTele-Science, remote-collaboration

Unfortunately Japan government tends to have “e-Japan” rather than “e-Science”, sigh

Program like “e-Science” or “Cyber Infra”Projects launched separately