Large-Scale Distributed Computing in the Netherlands

Dutc hG rid

an overviewDavid Groep, NIKHEF, 2003-01-20

What is DutchGrid?

• collaboration of 12 organisations• established in 2000• functions as a platform,

getting manpower & hardware from partner orgs

• major contributors– NIKHEF– WTCW Virtual Laboratory Project (ICES/KIS-II)– ASCI (5 combined Dutch CS Faculties)– Foundation NCF (NWO)

DutchGrid Sites

KNMI

Nijmegen

Delft

Leiden

Amsterdam

VU

WCW

Utrecht

Telin

NIKHEFSARAUvAAMOLFCWI

DutchGrid site map 20021104David Groep, NIKHEF

ASTRON, JIVE

WCW Amsterdam

Resources

• ASCI DAS-2 (VU, UvA, Leiden, TUDelft, Utrecht)– 200 dual P-III 1GHz CPUs– homogeneous clusters, 5 locations

• NIKHEF DataGrid clusters– 75 dual P-III ~ 1GHz – 1Gb/s IPv4 + 1Gb/s IPv6

• NCF Gridnational computer facilities foundation from NWO

– 66 node dual AMD-K7 Fabric Research Cluster (NIKHEF)– 32 node duals “production quality” cluster (SARA)*– 10Gb/s optical “lambda” test bed†

– …

• BioASP – various smaller O(10 node) clusters

Resources (cont.)

SARA – National HPC Centre• Processing

– SGI 1024 processor MPP

• Mass storage– StorageTek NearLine tape robot– currently: 500 TByte– will grow “indefinitely” (with funds from NCF)

• User expertise centre

In TotalOperations/deployment effort ~ 8 FTE availableResearch effort O (40 FTE) over 12 organisations

Resource Usage

Farm use by Experiment

Atlas19%

BioMed6%

CMS25%

D00%

EarthOb6%

Iteam38%

LHCb2%

Tutorial0%

Alice2%

WP62%

NIKHEF App TB 2002

excl. local users

CR1-1 CR1-2

CR2-1 CR2-2BR2

AR5

BR1AMS

10GPOS

2.5GPOS

BR1CHI

622M PO Prot.Tyco/IEEAF

AMSTERDAM

2x2.5Gb

622Mb L2 VPN

2xGbE622Mb “rest”Global Xing

622Mb to “rest”

NY

CR1-1 CR1-2

CR2-1 CR2-2BR2

AR5

BR1AMS

10GPOS

2.5GPOS

BR1CHI

622M PO Prot.Tyco/IEEAF

AMSTERDAM

622Mb L2 VPN

NY

CR1-1 CR1-2

CR2-1 CR2-2BR2

AR5

BR1AMS

10GPOS

2.5GPOS

622M PO Prot.Tyco/IEEAF

622Mb L2 VPN

NY

Networks

IGRID6509

BR4

SP01

OC48cSR

StarLight6509

ON

S15

454

ON

S

StarLightT640

ABILENE NY

ABILENE NY

ARGONNE IL

ARGONNE IL

SingARENTANet2

…

SingARENTANet2

…

StarLightRTR

STARTAP

GbE

GbE

CERNRTR

ONS

10G POS Tyco

10G POS Level3

2xGbE

10GbE

2xGbE

2.5Gb lambda

OC

48c

lam

bda

7xGbE

GbE

GbE’s

CHICAGO, IL

CERN,CH

18xGbEUvANIKHEF“NCSA”“ANL”

NIKHEF network load

Monthly graph NIKHEF-SURFnet5 IPv4 interconnectOctober/November 2002

New Initiatives inside NL

NCFGridstrengthening Grid Infrastructure in NL

• 4 partners (NIKHEF, ASCI, SARA, ASTRON)• 600 kEUR funding• hardware (3 clusters, 10Gb/s networking)

and people (1.5 FTE over 3 years)

• fabric research + production-quality grids• approved August 2002

First cluster assembled and being tested…

New Initiatives inside NL (2)

Virtual Laboratory for e-ScienceDutch “knowledge infrastructure” project

• Higher-level services to efficiently use global resources• Both “bottom-up” and “top-down” in same project

• 14 partners, 6 application domains• middleware research programme

LSDS SecAAA Transport

IntVisual.

Inf. Mngt

Intg ExpEnvs

UIs

DISc. FoodInf BioInf Dutella EcoGrid …

Scaling&

Validation

in planning stageO (50 MEUR)

What is happening where in NL?

DutchGrid Services

• Authentication Services (Certification Authority)– EDG CACG “proper” CA– Student and educational CA– EDG “Tutorial” CA

• Regular coordination and tutorials• Installation support

– Distributing EDG software to other (HEP) sites

• User support

Principal Apps today

• HEP: DØ, Atlas, LHCb, Alice, BaBarMonte-Carlo and Analysis

• OPERA – ozone monitoring

• WTCW Virtual Laboratoryvisual analysis environment for (small-scale) experimental science– Chemo-physical surface analysis (PIXE,FT-IR,SIMS)– BioASP – correlating “omics” databases– Virtual Surgery (planning by-passes, artery repairs)

• EcoGridcorrelating bio-diversity information databases

• Radio AstronomyLOFAR, AVO, VLBI – semi-online and offline processing

Research Areas in NL

• High-bandwidth networkingSURFnet, UvA, SARA, NIKHEF

• Generic AAA and `security’UvA, NIKHEF

• Cluster, Storage Management & MonitoringNIKHEF, SARA

• Adaptive programming environmentsFree University Amsterdam

• Memory-processor co-allocation TU Delft

• Federated Information & Content ManagementUvA

• Interactive visualisationCWI, UvA

Summary

• DutchGrid Platform started informally in 2000• de-facto focus for Grid computing in NL• Full centre,

operations (CA, web, …) supported by partners

• mix of people from all ‘layers’networks, fabrics, security&AAA, prog. environments, HPC visualisation, semantic DBs, applications

• Collaboration between users and providers leads to great opportunities …