Data Grids for Next Generation Experiments

Harvey B NewmanHarvey B NewmanCalifornia Institute of TechnologyCalifornia Institute of Technology

ACAT2000ACAT2000Fermilab, October 19, 2000Fermilab, October 19, 2000

http://l3www.cern.ch/~newman/grids_acat2k.ppthttp://l3www.cern.ch/~newman/grids_acat2k.ppt

Physics and Technical GoalsPhysics and Technical Goals

The extraction of small or subtle new “discovery” The extraction of small or subtle new “discovery” signals from large and potentially overwhelming signals from large and potentially overwhelming backgrounds; or “precision” analysis of large samplesbackgrounds; or “precision” analysis of large samples

Providing rapid access to event samples and subsets Providing rapid access to event samples and subsets from massive data stores, from ~300 Terabytes in 2001 from massive data stores, from ~300 Terabytes in 2001 Petabytes by ~2003, ~10 Petabytes by 2006, to ~100 Petabytes by ~2003, ~10 Petabytes by 2006, to ~100 Petabytes by ~2010.Petabytes by ~2010.

Providing analyzed results with rapid turnaround, byProviding analyzed results with rapid turnaround, bycoordinating and managing the coordinating and managing the LIMITED LIMITED computing, computing, data handling and network resources effectivelydata handling and network resources effectively

Enabling rapid access to the data and the collaboration, Enabling rapid access to the data and the collaboration, across an ensemble of networks of varying capability, across an ensemble of networks of varying capability, using heterogeneous resources.using heterogeneous resources.

Four LHC Experiments: The Four LHC Experiments: The Petabyte to Exabyte Petabyte to Exabyte

ChallengeChallengeATLAS, CMS, ALICE, LHCBATLAS, CMS, ALICE, LHCB

Higgs + New particles; Quark-Gluon Plasma; CP ViolationHiggs + New particles; Quark-Gluon Plasma; CP Violation

Data written to tapeData written to tape ~25 Petabytes/Year and UP ~25 Petabytes/Year and UP (CPU: 6 MSi95 and UP) (CPU: 6 MSi95 and UP)

0.1 to 1 Exabyte (1 EB = 100.1 to 1 Exabyte (1 EB = 101818 Bytes) Bytes) (~2010) (~2020 ?) Total for the LHC Experiments(~2010) (~2020 ?) Total for the LHC Experiments

LHC Vision: Data Grid HierarchyLHC Vision: Data Grid Hierarchy

Tier 1

Tier2 Center

Online System

Offline Farm,CERN Computer Ctr > 30 TIPS

FranceCenter

FNAL Center

Italy Center

UK Center

InstituteInstitute

InstituteInstitute ~0.25TIP

S

Workstations

~100 MBytes/se

c

~2.5 Gbits/sec

100 - 1000 Mbits/sec

1 Bunch crossing; ~17 interactions per 25 nsecs; 100 triggers per second. Event is ~1 MByte in size

Physicists work on analysis “channels”

Each institute has ~10 physicists working on one or more channels

Physics data cache

~PBytes/sec

~0.6-2.5 Gbits/sec

Tier2 CenterTier2 CenterTier2 Center

~622 Mbits/sec

Tier 0 +1

Tier 3

Tier 4

Tier2 Center Tier 2

Experiment

Why Worldwide Computing?Why Worldwide Computing?Regional Center Concept: AdvantagesRegional Center Concept: Advantages

Managed, fair-shared access for Physicists everywhereManaged, fair-shared access for Physicists everywhere Maximize total funding resources while meeting the Maximize total funding resources while meeting the

total computing and data handling needstotal computing and data handling needs Balance between proximity of datasets to appropriate Balance between proximity of datasets to appropriate

resources, and to the usersresources, and to the users Tier-N ModelTier-N Model

Efficient use of network: higher throughputEfficient use of network: higher throughput Per Flow: Local > regional > national > internationalPer Flow: Local > regional > national > international

Utilizing all intellectual resources, in several time zonesUtilizing all intellectual resources, in several time zones CERN, national labs, universities, remote sitesCERN, national labs, universities, remote sites Involving physicists and students at their home institutionsInvolving physicists and students at their home institutions

Greater flexibility to pursue different physics interests, Greater flexibility to pursue different physics interests, priorities, and resource allocation strategies by regionpriorities, and resource allocation strategies by region

And/or by Common Interests (physics topics, subdetectors,…)And/or by Common Interests (physics topics, subdetectors,…) Manage the System’s ComplexityManage the System’s Complexity

Partitioning facility tasks, to manage and focus resourcesPartitioning facility tasks, to manage and focus resources

SDSS Data Grid (In GriPhyN): SDSS Data Grid (In GriPhyN):

A Shared VisionA Shared VisionThree main functions:Three main functions: Raw data processing on a Grid (FNAL)Raw data processing on a Grid (FNAL) Rapid turnaround with TBs of dataRapid turnaround with TBs of data Accessible storage of all image dataAccessible storage of all image data

Fast science analysis environmentFast science analysis environment(JHU)(JHU)

Combined data access + analysis Combined data access + analysis of calibrated dataof calibrated data

Distributed I/O layer and processing Distributed I/O layer and processing layer; shared by whole collaborationlayer; shared by whole collaboration

Public data accessPublic data access SDSS data browsing for SDSS data browsing for

astronomers, and studentsastronomers, and students Complex query engine for the publicComplex query engine for the public

US-CERN BW RequirementsUS-CERN BW RequirementsProjection Projection (PRELIMINARY)(PRELIMINARY)

2001 2002 2003 2004 2005 2006

Installed LinkBW in MbpsIncl. New SLACThroughput [*]

310

(120)

622

(250)

1600

(400)

2400

(600)

4000

(1000)

6500 [#]

(1600)

[#] Includes ~1.5 Gbps Each for ATLAS and CMS, Plus Babar, Run2 and Other[*] D0 and CDF at Run2: Needs Presumed to Be to be Comparable to BaBar

Daily, Weekly, Monthly and Yearly Statistics on Daily, Weekly, Monthly and Yearly Statistics on the 45 Mbps US-CERN Linkthe 45 Mbps US-CERN Link

RD45, RD45, GIODGIOD Networked Object DatabasesNetworked Object Databases Clipper/GC Clipper/GC High speed access to Objects or File data High speed access to Objects or File data FNAL/SAM FNAL/SAM for processing and analysisfor processing and analysis SLAC/OOFS SLAC/OOFS Distributed File System + Objectivity Distributed File System + Objectivity Interface Interface NILE, Condor:NILE, Condor: Fault Tolerant Distributed ComputingFault Tolerant Distributed Computing

MONARCMONARC LHC Computing Models: LHC Computing Models: Architecture, Simulation, Strategy, PoliticsArchitecture, Simulation, Strategy, Politics

ALDAPALDAP OO Database Structures & Access Methods OO Database Structures & Access Methods for Astrophysics and HENP Datafor Astrophysics and HENP Data

PPDGPPDG First Distributed Data Services and First Distributed Data Services and Data Grid System PrototypeData Grid System Prototype

GriPhyN GriPhyN Production-Scale Data GridsProduction-Scale Data Grids EU Data GridEU Data Grid

Roles of ProjectsRoles of Projectsfor HENP Distributed Analysisfor HENP Distributed Analysis

Grid Services Architecture [*]Grid Services Architecture [*]

GridGridFabricFabric

GridGridServicesServices

ApplnApplnToolkitsToolkits

ApplnsApplns

Data stores, networks, computers, display Data stores, networks, computers, display devices,… ; associated local servicesdevices,… ; associated local services

Protocols, authentication, policy, resource Protocols, authentication, policy, resource management, instrumentation, discovery,etc.management, instrumentation, discovery,etc.

......RemotRemot

eevizviz

toolkittoolkit

RemotRemotee

comp.comp.toolkittoolkit

RemotRemotee

datadatatoolkittoolkit

RemotRemotee

sensorssensorstoolkittoolkit

RemotRemotee

collab.collab.toolkittoolkit

A Rich Set of HEP Data-Analysis A Rich Set of HEP Data-Analysis Related ApplicationsRelated Applications

[*] [*] Adapted from Ian Foster: there are computing grids, Adapted from Ian Foster: there are computing grids, access (collaborative) grids, data grids, ...access (collaborative) grids, data grids, ...

The Particle Physics Data Grid (PPDG)The Particle Physics Data Grid (PPDG)

First Round Goal: First Round Goal: Optimized cached read access to 10-100 Gbytes Optimized cached read access to 10-100 Gbytes drawn from a total data set of 0.1 to ~1 Petabytedrawn from a total data set of 0.1 to ~1 Petabyte

PRIMARY SITEPRIMARY SITEData Acquisition,Data Acquisition,

CPU, Disk, CPU, Disk, Tape RobotTape Robot

SECONDARY SITESECONDARY SITECPU, Disk, CPU, Disk, Tape RobotTape Robot

Site to Site Data Replication Service

100 Mbytes/sec

ANL, BNL, Caltech, FNAL, JLAB, LBNL, ANL, BNL, Caltech, FNAL, JLAB, LBNL, SDSC, SLAC, U.Wisc/CSSDSC, SLAC, U.Wisc/CS

Multi-Site Cached File Access Service

UniversityUniversityCPU, Disk, CPU, Disk,

UsersUsers

PRIMARY SITEPRIMARY SITEDAQ, Tape, DAQ, Tape,

CPU, CPU, Disk, RobotDisk, Robot

Satellite SiteSatellite SiteTape, CPU, Tape, CPU, Disk, RobotDisk, Robot

UniversityUniversityCPU, Disk, CPU, Disk,

UsersUsers

UniversityUniversityCPU, Disk, CPU, Disk,

UsersUsers

UniversityUniversityCPU, Disk, CPU, Disk,

UsersUsers

UniversityUniversityCPU, Disk, CPU, Disk,

UsersUsers

Satellite SiteSatellite SiteTape, CPU, Tape, CPU, Disk, RobotDisk, Robot

Matchmaking, Co-Scheduling: SRB, Condor, Globus services; HRM, NWSMatchmaking, Co-Scheduling: SRB, Condor, Globus services; HRM, NWS

PPDG WG1: Request ManagerPPDG WG1: Request Manager

tape system

HRM

Replicacatalog

NetworkWeatherServicePhysical file

transfer requests GRID

RequestInterpreter

DiskCache

Event-file Index

DRM

DiskCache

RequestExecutor

Logical Set of Files Request

Planner(Matchmaking)DRMDisk

Cache

CLIENT CLIENT

Logical Request

REQUEST MANAGER

LLNL

Earth Grid System Prototype Earth Grid System Prototype Inter-communication DiagramInter-communication Diagram

Disk

Client

Request Manager

ISIGSI-

wuftpd

Disk

SDSCGSI-pftpd

HPSS

LBNLGSI-wuftpd

Disk

ANLGSI-

wuftpd

Disk

NCARGSI-

wuftpd

Disk

LBNL

Diskon

Clipper

HPSS

HRM

ANLReplica Catalog

GIS with NWS

GSI-ncftp

GS

I-ncftpGSI-n

cftp

LDAP Script

LDAP C API or Script

GSI-ncftp

GSI-ncftpGSI-ncftp CORBA

Grid Data Management Grid Data Management Prototype (GDMP)Prototype (GDMP)

Distributed Distributed Job Job ExecutionExecution and and Data Handling:Data Handling:

GoalsGoals TransparencyTransparency PerformancePerformance Security Security Fault ToleranceFault Tolerance AutomationAutomation

Submit job

Replicate data

Replicatedata

Site A Site B

Site C

Jobs are executed locally or

remotely Data is always

written locally Data is replicated

to remote sites

Job writes data locally

GDMP V1.1: Caltech + EU DataGrid WP2 Tests by CALTECH, CERN, FNAL, Pisa for CMS “HLT” Production 10/2000;

Integration with ENSTORE, HPSS, Castor

WorkPackageNumber

Work Package title Leadcontractor

WP1 Grid Workload Management INFN

WP2 Grid Data Management CERN

WP3 Grid Monitoring Services PPARC

WP4 Fabric Management CERN

WP5 Mass Storage Management PPARC

WP6 Integration Testbed CNRS

WP7 Network Services CNRS

WP8 High Energy Physics Applications CERN

WP9 Earth Observation Science Applications ESA

WP10 Biology Science Applications INFN

WP11 Dissemination and Exploitation INFN

WP12 Project Management CERN

EU-Grid ProjectEU-Grid ProjectWork PackagesWork Packages

GriPhyN: PetaScale GriPhyN: PetaScale Virtual Data GridsVirtual Data Grids

Build the Foundation for Petascale Virtual Data GridsBuild the Foundation for Petascale Virtual Data Grids

Virtual Data Tools

Request Planning &

Scheduling ToolsRequest Execution & Management Tools

Transforms

Distributed resources(code, storage,

computers, and network)

Resource Management

Services

Resource Management

Services

Security and Policy

Services

Security and Policy

Services

Other Grid ServicesOther Grid

Services

Interactive User Tools

Production TeamIndividual Investigator

Workgroups

Raw data source

Data Grids: Better Global Resource Data Grids: Better Global Resource Use Use andand Faster Turnaround Faster Turnaround

Build Information and Security InfrastructuresBuild Information and Security Infrastructures Across Several World RegionsAcross Several World Regions Authentication: Prioritization, Resource AllocationAuthentication: Prioritization, Resource Allocation

Coordinated use of computing, data handling and Coordinated use of computing, data handling and network resources through:network resources through: Data caching, query estimation, co-schedulingData caching, query estimation, co-scheduling Network and site “instrumentation”: performance Network and site “instrumentation”: performance

tracking, monitoring, problem trapping and tracking, monitoring, problem trapping and handlinghandling

Robust TransactionsRobust Transactions Agent Based: Autonomous, Adaptive, Agent Based: Autonomous, Adaptive,

Network Efficient, Resilient Network Efficient, Resilient Heuristic, Adaptive Load-Balancing Heuristic, Adaptive Load-Balancing

E.g. Self-Organzing Neural Nets (Legrand) E.g. Self-Organzing Neural Nets (Legrand)

GRIDs In 2000: SummaryGRIDs In 2000: Summary

Grids will change the way we do science Grids will change the way we do science and engineering: computation to large scale dataand engineering: computation to large scale data

Key services and concepts have been Key services and concepts have been identified, and development has startedidentified, and development has started

Major IT challenges remainMajor IT challenges remain AnAn Opportunity & Obligation for HEP/CSOpportunity & Obligation for HEP/CS

CollaborationCollaboration Transition of services and applications to production Transition of services and applications to production

use is starting to occuruse is starting to occur In future more sophisticated integrated services and In future more sophisticated integrated services and

toolsets (Inter- and IntraGrids+) could drive advances toolsets (Inter- and IntraGrids+) could drive advances in many fields of science & engineeringin many fields of science & engineering

HENP, facing the need for Petascale Virtual Data, HENP, facing the need for Petascale Virtual Data, is both an early adopter, and a leading developer of is both an early adopter, and a leading developer of Data Grid technologyData Grid technology