U.S. Grid Projects: Grid3 and Open Science GridPaul AveryUniversity of Floridaavery@phys.ufl.eduInternational ICFA Workshop on HEP, Networking & Digital Divide Issues for Global e-Science Daegu, Korea May 23, 2005

Paul Avery

U.S. Trillium Grid PartnershipTrillium = PPDG + GriPhyN + iVDGLParticle Physics Data Grid:$12M (DOE)(1999 2006)GriPhyN:$12M (NSF)(2000 2005)iVDGL:$14M (NSF)(2001 2006)Basic composition (~150 people)PPDG:4 universities, 6 labsGriPhyN:12 universities, SDSC, 3 labsiVDGL:18 universities, SDSC, 4 labs, foreign partnersExpts:BaBar, D0, STAR, Jlab, CMS, ATLAS, LIGO, SDSS/NVOCoordinated internally to meet broad goalsGriPhyN:CS research, Virtual Data Toolkit (VDT) developmentiVDGL:Grid laboratory deployment using VDT, applicationsPPDG:End to end Grid services, monitoring, analysisCommon use of VDT for underlying Grid middlewareUnified entity when collaborating internationally

Paul Avery

Goal: Peta-scale Data Grids forGlobal ScienceVirtual Data ToolsRequest Planning & Scheduling Tools Request Execution & Management Tools Distributed resources (code, storage, CPUs, networks)Resource Management Services Security and Policy Services Other Grid Services Interactive User ToolsProduction TeamSingle ResearcherWorkgroupsRaw data source PetaOps Petabytes Performance

Paul Avery

Grid Middleware: Virtual Data ToolkitSources(CVS)PatchingGPT srcbundlesNMIBuild & TestCondor pool22+ Op. SystemsBuildTestPackageVDTBuildMany ContributorsBuildPacman cacheRPMsBinariesBinariesBinariesTestA unique laboratory for testing, supporting, deploying, packaging, upgrading, & troubleshooting complex sets of software!

Paul Avery

VDT Growth Over 3 Years# of componentsVDT 1.0Globus 2.0bCondor 6.3.1VDT 1.1.7Switch to Globus 2.2VDT 1.1.11Grid3VDT 1.1.8First real use by LCGwww.griphyn.org/vdt/

Paul Avery

Chart1

43731637316

43734737347

53740837408

53743837438

103753037530

103753037530

103753037530

103762237622

113768137681

133774237742

163777337773

213783437834

213786537865

213795637956

223804738047

223810838108

381692238169

382312238231

382922238292

383222238322

384122238412

383533835326

383843838429

383843838430

384123841231

384433844332

384433844332

VDT 1.1.x

VDT 1.2.x

VDT 1.3.x

Graph

Date2/1/024/1/026/1/027/1/0210/17/0210/28/0210/30/021/1/033/1/035/1/036/1/038/1/039/1/0312/16/033/2/045/7/04

Total components4456101010101112151919192020

Version11.11.1.11.1.21.1.31.1.41.1.51.1.61.1.71.1.81.1.91.1.101.1.111.1.121.1.131.1.14

Graph

43731637316

43734737347

53740837408

53743837438

103753037530

103753037530

103753037530

103762237622

113768137681

133774237742

163777337773

213783437834

213786537865

213795637956

223804738047

223810838108

381692238169

382312238231

382922238292

383222238322

384122238412

383533835326

383843838429

383843838430

384123841231

384433844332

384433844332

VDT 1.1.x

VDT 1.2.x

VDT 1.3.x

Software List

1.01.11.1.11.1.21.1.31.1.41.1.51.1.61.1.71.1.81.1.91.1.101.1.111.1.121.1.131.1.141.2.01.2.11.2.21.2.31.2.41.3.01.3.11.3.21.3.31.3.41.3.5

Mar-2002Apr-2002Jun-2002Jul-2002Oct-2002Oct-2002Oct-2002Jan-2003Mar-2003May-2003Jun-2003Aug-2003Sep-2003Dec-2003Mar-2004May-2004Jul-2004Sep-2004Nov-2004Dec-2004Mar-2005Jan-2005Feb-2005Feb-2005Mar-2005Apr-2005Apr-2005

4455101010101113162121212222

2222222222

262930313232

Globus 2.0bGlobus 2.0bGlobus 2.0Globus 2.0Globus 2.0Globus 2.0Globus 2.0Globus 2.0Globus 2.2.4Globus 2.2.4Globus 2.2.4Globus 2.2.4Globus 2.2.4Globus 2.2.4Globus 2.2.4Globus 2.2.4Globus 2.2.4Globus 2.2.4Globus 2.2.4Globus 2.2.4Globus 2.2.4Globus 3.2.1Globus 3.2.1Globus 3.2.1Globus 3.2.1Globus 3.2.1Globus 3.2.1

Condor 6.3.1Condor 6.3.1Condor 6.3.3Condor 6.4.1Condor 6.4.3Condor 6.4.3Condor 6.4.3Condor 6.4.3Condor 6.4.7Condor 6.5.1Condor 6.5.1Condor 6.5.3Condor 6.5.5Condor 6.6.0Condor 6.6.1Condor 6.6.5Condor 6.6.5Condor 6.6.6Condor 6.6.7Condor 6.6.7Condor 6.6.7Condor 6.7.3Condor 6.7.4Condor 6.7.4Condor 6.7.6Condor 6.7.6Condor 6.7.6

GDMP 2.1GDMP 3.0GDMP 3.0GDMP 3.0.7GDMP 3.0.7GDMP 3.0.7GDMP 3.0.7GDMP 3.0.7RLS 1.1.8RLS 1.1.8RLS 2.0.8RLS 2.0.9RLS 2.0.9RLS 2.0.9RLS 2.1.4RLS 2.1.5RLS 2.1.5RLS 2.1.5RLS 2.1.5RLS 2.1.5RLS 2.1.5RLS 2.1.5RLS 3.0RLS 3.0RLS 3.0RLS 3.0RLS 3.0

ClassAds 0.9.2ClassAds 0.9.2ClassAds 0.9.2ClassAds 0.9.2ClassAds 0.9.3ClassAds 0.9.3ClassAds 0.9.3ClassAds 0.9.4ClassAds 0.9.4ClassAds 0.9.4ClassAds 0.9.4ClassAds 0.9.4ClassAds 0.9.5-betaClassAds 0.9.5-betaClassAds 0.9.5ClassAds 0.9.5ClassAds 0.9.5ClassAds 0.9.5ClassAds 0.9.7ClassAds 0.9.7ClassAds 0.9.7ClassAds 0.9.7ClassAds 0.9.7ClassAds 0.9.7ClassAds 0.9.7ClassAds 0.9.7ClassAds 0.9.7

Replica 2.0Replica 2.0Replica 2.0Replica 2.0Replica 2.0Replica 2.2.4Replica 2.2.4Replica 2.2.4Replica 2.2.4Replica 2.2.4Replica 2.2.4Replica 2.2.4Replica 2.2.4Replica 2.2.4Replica 2.2.4Replica 2.2.4Replica 2.2.4Replica 2.2.4Replica 2.2.4Replica 2.2.4Replica 2.2.4Replica 2.2.4Replica 2.2.4Replica 2.2.4Replica 2.2.4

DOE/EDG CA certsDOE/EDG CA certsDOE/EDG CA certsDOE/EDG CA certsDOE/EDG CA certsDOE/EDG CA certsDOE/EDG CA certsDOE/EDG CA certsDOE/EDG CA certsDOE/EDG CA certsDOE/EDG CA certsDOE/EDG CA certsDOE/EDG CA certsDOE/EDG CA certsDOE/EDG CA certsDOE/EDG CA certsDOE/EDG CA certsDOE/EDG CA certsDOE/EDG CA certsDOE/EDG CA certsDOE/EDG CA certsDOE/EDG CA certsDOE/EDG CA certs

ftsh 0.9.9ftsh 0.9.9ftsh 0.9.9ftsh 1.0ftsh 1.0ftsh 2.0ftsh 2.0ftsh 2.0.2ftsh 2.0.2ftsh 2.0.2ftsh 2.0.2ftsh 2.0.3ftsh 2.0.5ftsh 2.0.5ftsh 2.0.5ftsh 2.0.5ftsh 2.0.5ftsh 2.0.5ftsh 2.0.5ftsh 2.0.5ftsh 2.0.5ftsh 2.0.5ftsh 2.0.5

EDG mkgridmap 1.0.10EDG mkgridmap 1.0.10EDG mkgridmap 1.0.10EDG mkgridmapEDG mkgridmapEDG mkgridmapEDG mkgridmapEDG mkgridmapEDG mkgridmapEDG mkgridmapEDG mkgridmapEDG mkgridmapEDG mkgridmapEDG mkgridmapEDG mkgridmapEDG mkgridmapEDG mkgridmapEDG mkgridmapEDG mkgridmapEDG mkgridmapEDG mkgridmapEDG mkgridmapEDG mkgridmap

EDG CRL Update 1.2.5EDG CRL Update 1.2.5EDG CRL Update 1.2.5EDG CRL UpdateEDG CRL UpdateEDG CRL UpdateEDG CRL UpdateEDG CRL UpdateEDG CRL UpdateEDG CRL UpdateEDG CRL UpdateEDG CRL UpdateEDG CRL UpdateEDG CRL UpdateEDG CRL UpdateEDG CRL UpdateEDG CRL UpdateEDG CRL UpdateEDG CRL UpdateEDG CRL UpdateEDG CRL UpdateEDG CRL UpdateEDG CRL Update

Enhanced job managerEnhanced job managerEnhanced job managerEnhanced job managerGLUE Schema 1.0GLUE Schema 1.0GLUE Schema 1.0GLUE Schema 1.0GLUE Schema 1.0GLUE Schema 1.0GLUE Schema 1.0GLUE Schema 1.0GLUE Schema 1.0GLUE Schema 1.0GLUE Schema 1.0GLUE Schema 1.0GLUE Schema 1.0GLUE Schema 1.0GLUE Schema 1.0GLUE Schema 1.0GLUE Schema 1.0GLUE Schema 1.0GLUE Schema 1.0

VDS 1.0.1VDS 1.0.3VDS 1.0.3VDS 1.1.1VDS 1.1.3VDS 1.2.0VDS 1.2.3VDS 1.2.6VDS 1.2.10VDS 1.2.13VDS 1.2.14VDS 1.2.14VDS 1.2.14VDS 1.3.5VDS 1.3.5bVDS 1.3.5bVDS 1.3.5bVDS 1.3.5bVDS 1.3.5b

JavaJavaJavaJavaJavaJavaJavaJavaJavaJavaJavaJavaJavaJavaJavaJavaJavaJava

Netlogger 2.0.13Netlogger 2.0.13Netlogger 2.0.13Netlogger 2.2Netlogger 2.2Netlogger 2.2Netlogger 2.2Netlogger 2.2Netlogger 2.2Netlogger 2.2Netlogger 2.2Netlogger 2.2Netlogger 2.2Netlogger 2.2Netlogger 2.2Netlogger 2.2Netlogger 3.2.4Netlogger 3.2.4

Gatekeeper-AuthzGatekeeper-AuthzGatekeeper-AuthzGatekeeper-AuthzGatekeeper-AuthzGatekeeper-AuthzGatekeeper-AuthzGatekeeper-AuthzGatekeeper-AuthzGatekeeper-AuthzGatekeeper-AuthzGatekeeper-AuthzGatekeeper-AuthzGatekeeper-AuthzGatekeeper-AuthzGatekeeper-AuthzGatekeeper-Authz

MyProxyMyProxyMyProxyMyProxyMyProxy1.11MyProxy1.11MyProxy1.11MyProxy1.11MyProxy1.11MyProxy1.11MyProxy1.11MyProxy1.11MyProxy1.11MyProxy1.11MyProxy1.11MyProxy1.11MyProxy1.11

KX509KX509KX509KX509KX509KX509KX509KX509KX509KX509KX509KX509KX509KX509KX509KX509KX509

System ProfilerSystem ProfilerSystem ProfilerSystem ProfilerSystem ProfilerSystem ProfilerSystem ProfilerSystem ProfilerSystem ProfilerSystem ProfilerSystem ProfilerSystem ProfilerSystem ProfilerSystem ProfilerSystem ProfilerSystem Profiler

GSI OpenSSHGSI OpenSSHGSI OpenSSHGSI OpenSSH 3.0GSI OpenSSH 3.0GSI OpenSSH 3.4GSI OpenSSH 3.4GSI OpenSSH 3.4GSI OpenSSH 3.4GSI OpenSSH 3.4GSI OpenSSH 3.4GSI OpenSSH 3.4GSI OpenSSH 3.4GSI OpenSSH 3.4GSI OpenSSH 3.4GSI OpenSSH 3.4

Monalisa 0.9.5Monalisa 0.9.5Monalisa 0.9.5Monalisa 0.9.5Monalisa 1.2.8Monalisa 1.2.12Monalisa 1.2.12Monalisa 1.2.12Monalisa 1.2.12Monalisa 1.2.12Monalisa 1.2.20Monalisa 1.2.26Monalisa 1.2.26Monalisa 1.2.26Monalisa 1.2.26Monalisa 1.2.32

PyGlobusPyGlobusPyGlobusPyGlobusPyGlobusPyGlobusPyGlobusPyGlobus 1.0.6PyGlobus 1.0.6PyGlobus 1.0.6PyGlobus 1.0.6PyGlobus 1.0.6PyGlobus 1.0.6PyGlobus 1.0.6PyGlobus 1.0.6PyGlobus 1.0.6

MySQLMySQLMySQLMySQLMySQLMySQLMySQLMySQLMySQLMySQLMySQLMySQLMySQLMySQLMySQLMySQL

UberFTP 1.3UberFTP 1.3UberFTP 1.3UberFTP 1.3UberFTP 1.3UberFTP 1.3UberFTP 1.3UberFTP 1.3UberFTP 1.3UberFTP 1.3UberFTP 1.11UberFTP 1.11UberFTP 1.11

DRM 1.2.2DRM 1.2.4DRM 1.2.4DRM 1.2.6aDRM 1.2.6aDRM 1.2.6a

VOMS 1.2.19VOMS 1.3.2VOMS 1.3.2VOMS 1.4.0VOMS 1.4.0VOMS 1.4.0

VOMS Admin 0.7.5VOMS Admin 0.7.5VOMS Admin 0.7.5VOMS Admin 0.7.5VOMS Admin 0.7.5VOMS Admin 0.7.5

TomcatTomcatTomcatTomcatTomcatTomcat

PRIMA 0.1PRIMA 0.1PRIMA 0.1PRIMA 0.1PRIMA 0.2

Certificate ScriptsCertificate ScriptsCertificate ScriptsCertificate ScriptsCertificate Scripts

ApacheApacheApacheApacheApache

jClarens 0.5.3jClarens 0.5.3jClarens 0.5.3jClarens 0.5.3

New GridFTP ServerNew GridFTP ServerNew GridFTP Server

GUMS 1.0.1GUMS 1.0.1

Trillium Science DriversExperiments at Large Hadron ColliderNew fundamental particles and forces100s of Petabytes2007 - ?High Energy & Nuclear Physics exptsTop quark, nuclear matter at extreme density~1 Petabyte (1000 TB)1997 presentLIGO (gravity wave search)Search for gravitational waves100s of Terabytes2002 presentSloan Digital Sky SurveySystematic survey of astronomical objects10s of Terabytes2001 present

Paul Avery

LHC: Petascale Global ScienceComplexity:Millions of individual detector channelsScale:PetaOps (CPU), 100s of Petabytes (Data)Distribution:Global distribution of people & resourcesCMS Example- 20075000+ Physicists 250+ Institutes 60+ CountriesBaBar/D0 Example - 2004700+ Physicists 100+ Institutes 35+ Countries

Paul Avery

LHC Global Data Grid (2007+)CMS ExperimentOnline SystemCERN Computer Center200 - 1500 MB/s>10 Gb/s10-40 Gb/s2.5-10 Gb/sTier 0Tier 1Tier 3Tier 2Physics cachesPCs5000 physicists, 60 countries10s of Petabytes/yr by 20081000 Petabytes in < 10 yrs?Tier 4

Paul Avery

University LHC Tier2 CentersTier2 facilityEssential university role in extended computing infrastructure20 25% of Tier1 national laboratory, supported by NSFValidated by 3 years of experience (CMS, ATLAS)FunctionsPerform physics analysis, simulationsSupport experiment software, smaller institutionsOfficial role in Grid hierarchy (U.S.)Sanctioned by MOU with parent organization (ATLAS, CMS)Selection by collaboration via careful process

Paul Avery

Grids and Globally Distributed TeamsNon-hierarchical: Chaotic analyses + productionsSuperimpose significant random data flows

Paul Avery

CMS: Grid Enabled Analysis ArchitectureSchedulerCatalogsGrid ServicesWeb ServerExecutionPriorityManagerGrid WideExecutionServiceDataManage- mentFully-ConcretePlannerFully-AbstractPlannerAnalysis ClientVirtual Data ReplicaApplicationsMonitoringPartially-AbstractPlannerMetadata HTTP, SOAP, XML-RPCChimeraSphinxMonALISAROOT (analysis tool)PythonCojac (detector viz)/ IGUANA (cms viz)ClarensMCRunjobBOSSRefDBPOOLORCAROOTFAMOSVDT-ServerMOPDBDiscoveryACL managementCert. based accessClients talk standard protocols to Grid Services Web Server Simple Web service API allows simple or complex analysis clients Typical clients: ROOT, Web Browser, . Clarens portal hides complexity Key features: Global Scheduler, Catalogs, Monitoring, Grid-wide Execution serviceAnalysisClient

Paul Avery

Grid3: A National Grid Infrastructure32 sites, 4000 CPUs: Universities + 4 national labsPart of LHC Grid, Running since October 2003Sites in US, Korea, Brazil, TaiwanApplications in HEP, LIGO, SDSS, Genomics, fMRI, CSBrazilwww.ivdgl.org/grid3

Paul Avery

Grid3 ComponentsComputers & storage at ~30 sites: 4000 CPUsUniform service environment at each siteGlobus 3.2: Authentication, execution management, data movementPacman: Installation of numerous VDT and application servicesGlobal & virtual organization servicesCertification & reg. authorities, VO membership & monitor servicesClient-side tools for data access & analysis Virtual data, execution planning, DAG management, execution management, monitoringIGOC: iVDGL Grid Operations CenterGrid testbed: Grid3devMiddleware development and testing, new VDT versions, etc.

Paul Avery

Grid3 Applicationswww.ivdgl.org/grid3/applications

CMS experimentp-p collision simulations & analysisATLAS experimentp-p collision simulations & analysisBTEV experimentp-p collision simulations & analysisLIGOSearch for gravitational wave sourcesSDSSGalaxy cluster findingBio-molecular analysisShake n Bake (SnB) (Buffalo)Genome analysisGADU/GnarefMRIFunctional MRI (Dartmouth)CS DemonstratorsJob Exerciser, GridFTP, NetLogger

Paul Avery

Grid3 Shared Use Over 6 monthsSep 10, 2004 Usage: CPUs

Paul Avery

Grid3 Production Over 13 Months

Paul Avery

U.S. CMS 2003 Production10M p-p collisions; largest ever2 simulation sample manpowerMulti-VO sharing

Paul Avery

Grid3 Lessons LearnedHow to operate a Grid as a facilityTools, services, error recovery, procedures, docs, organizationDelegation of responsibilities (Project, VO, service, site, )Crucial role of Grid Operations Center (GOC)How to support people people relationsFace-face meetings, phone cons, 1-1 interactions, mail lists, etc.How to test and validate Grid tools and applicationsVital role of testbedsHow to scale algorithms, software, processSome successes, but interesting failure modes still occurHow to apply distributed cyberinfrastructureSuccessful production runs for several applications

Paul Avery

Grid3 Open Science GridIteratively build & extend Grid3Grid3 OSG-0 OSG-1 OSG-2 Shared resources, benefiting broad set of disciplinesGrid middleware based on Virtual Data Toolkit (VDT)Consolidate elements of OSG collaborationComputer and application scientistsFacility, technology and resource providers (labs, universities)Further develop OSGPartnerships with other sciences, universitiesIncorporation of advanced networkingFocus on general services, operations, end-to-end performanceAim for July 2005 deployment

Paul Avery

http://www.opensciencegrid.org

Paul Avery

OSG OrganizationEnterprise Technical GroupsResearchGrid ProjectsVOsResearchersSitesService ProvidersUniversities, LabsAdvisory CommitteeCore OSG Staff(few FTEs, manager)OSG Council(all members abovea certain threshold,Chair, officers)Executive Board(8-15 representativesChair, Officers)

Paul Avery

OSG Technical Groups & ActivitiesTechnical Groups address and coordinate technical areasPropose and carry out activities related to their given areasLiaise & collaborate with other peer projects (U.S. & international)Participate in relevant standards organizations.Chairs participate in Blueprint, Integration and Deployment activitiesActivities are well-defined, scoped tasks contributing to OSGEach Activity has deliverables and a plan is self-organized and operated is overseen & sponsored by one or more Technical GroupsTGs and Activities are where the real work gets done

Paul Avery

OSG Technical Groups

GovernanceCharter, organization, by-laws, agreements, formal processesPolicyVO & site policy, authorization, priorities, privilege & access rightsSecurityCommon security principles, security infrastructureMonitoring and Information ServicesResource monitoring, information services, auditing, troubleshootingStorageStorage services at remote sites, interfaces, interoperabilitySupport CentersInfrastructure and services for user support, helpdesk, trouble ticketEducation / OutreachTraining, interface with various E/O projectsNetworks (new)Including interfacing with various networking projects

Paul Avery

OSG Activities

BlueprintDefining principles and best practices for OSGDeploymentDeployment of resources & servicesProvisioningConnected to deploymentIncidence responsePlans and procedures for responding to security incidentsIntegrationTesting & validating & integrating new services and technologiesData Resource Management (DRM)Deployment of specific Storage Resource Management technologyDocumentationOrganizing the documentation infrastructureAccountingAccounting and auditing use of OSG resourcesInteroperabilityPrimarily interoperability between OperationsOperating Grid-wide services

Paul Avery

The Path to the OSG Operating Grid OSG Deployment ActivityMetrics &CertificationApplicationvalidationVO Application SoftwareInstallationOSG Integration ActivityRelease DescriptionMiddlewareInteroperabilitySoftware &packagingFunctionality & Scalability TestsReadiness plan adoptedService deploymentOSG Operations-Provisioning ActivityReleaseCandidateReadinessplan

Effort

Resourcesfeedback

Paul Avery

OSG Integration Testbed>20 Sites and RisingBrazil

Paul Avery

Status of OSG DeploymentOSG infrastructure release accepted for deploymentUS CMS application flood testing successfulD0 simulation & reprocessing jobs running on selected OSG sitesOthers in various stages of readying applications & infrastructure (ATLAS, CMS, STAR, CDF, BaBar, fMRI)

Deployment process underway: End of July?Open OSG and transition resources from Grid3Applications will use growing ITB & OSG resources during transitionhttp://osg.ivdgl.org/twiki/bin/view/Integration/WebHome

Paul Avery

Connections to LCG and EGEEMany LCG-OSG interactions

Paul Avery

Interoperability & FederationTransparent use of federated Grid infrastructures a goalLCG, EGEETeraGridState-wide GridsCampus Grids (Wisconsin, Florida, etc)Some early activities with LCGSome OSG/Grid3 sites appear in LCG mapD0 bringing reprocessing to LCG sites through adaptor nodeCMS and ATLAS can run their jobs on both LCG and OSGIncreasing interaction with TeraGridCMS and ATLAS sample simulation jobs are running on TeraGridPlans for TeraGrid allocation for jobs running in Grid3 model (group accounts, binary distributions, external data management, etc)

Paul Avery

UltraLight: Advanced Networkingin Applications10 Gb/s+ networkCaltech, UF, FIU, UM, MITSLAC, FNALIntl partnersLevel(3), Cisco, NLRFunded by ITR2004

Paul Avery

UltraLight: New Information SystemA new class of integrated information systemsIncludes networking as a managed resource for the first timeUses Hybrid packet-switched and circuit-switched optical network infrastructureMonitor, manage & optimize network and Grid Systems in realtimeFlagship applications: HEP, eVLBI, burst imagingTerabyte-scale data transactions in minutesExtend Real-Time eVLBI to the 10 100 Gb/s RangePowerful testbedSignificant storage, optical networks for testing new Grid servicesStrong vendor partnershipsCisco, Calient, NLR, CENIC, Internet2/Abilene

Paul Avery

iVDGL, GriPhyN Education/OutreachBasics$200K/yrLed by UT BrownsvilleWorkshops, portals, tutorialsNew partnerships with QuarkNet, CHEPREO, LIGO E/O,

Paul Avery

U.S. Grid Summer SchoolFirst of its kind in the U.S. (June 2004, South Padre Island)36 students, diverse origins and types (M, F, MSIs, etc)Marks new direction for U.S. Grid effortsFirst attempt to systematically train people in Grid technologiesFirst attempt to gather relevant materials in one placeToday:Students in CS and PhysicsNext:Students, postdocs, junior & senior scientistsReaching a wider audiencePut lectures, exercises, video, on the webMore tutorials, perhaps 2-3/yearDedicated resources for remote tutorialsCreate Grid Cookbook, e.g. Georgia TechSecond workshop: July 1115, 2005South Padre Island

Paul Avery

QuarkNet/GriPhyN e-Lab Projecthttp://quarknet.uchicago.edu/elab/cosmic/home.jsp

Paul Avery

Student Muon Lifetime Analysis in GriPhyN/QuarkNet

Paul Avery

CHEPREO: Center for High Energy Physics Research and Educational OutreachFlorida International UniversityPhysics Learning CenterCMS ResearchiVDGL Grid ActivitiesAMPATH network (S. America)Funded September 2003$4M initially (3 years)MPS, CISE, EHR, INT

Paul Avery

Science Grid CommunicationsBroad set of activitiesNews releases, PR, etc.Science Grid This WeekKatie Yurkewicz talk

Paul Avery

Grids and the Digital DivideRio de Janeiro + DaeguBackgroundWorld Summit on Information SocietyHEP Standing Committee on Inter-regional Connectivity (SCIC)ThemesGlobal collaborations, Grids and addressing the Digital DivideFocus on poorly connected regionsBrazil (2004), Korea (2005)

Paul Avery

New Campus Research Grids(e.g., Florida)HEPQTPHCSCISEHPC IHPC IIHPC IIIService ProvidersChemQTPGrid InfrastructureApplicationsFacilitiesAstroHCSBioGeoACISMBIACISMBINanoDWICMS Tier2CNSQTP

Paul Avery

US HEP Data Grid TimelineUltraLight approved, $2MGrid CommunicationsGrid Summer School IGrid Summer School IIStart Grid3 operationsStart Open Science Grid operationsFirst US-LHC Grid TestbedsDigital Divide WorkshopsLIGO GridGLORIAD funded

Paul Avery

SummaryGrids enable 21st century collaborative scienceLinking research communities and resources for scientific discoveryNeeded by global collaborations pursuing petascale scienceGrid3 was an important first step in developing US GridsValue of planning, coordination, testbeds, rapid feedbackValue of learning how to operate a Grid as a facilityValue of building & sustaining community relationshipsGrids drive need for advanced optical networksGrids impact education and outreachProviding technologies & resources for training, education, outreachAddressing the Digital DivideOSG: a scalable computing infrastructure for science?Strategies needed to cope with increasingly large scale

Paul Avery

Grid Project ReferencesOpen Science Gridwww.opensciencegrid.orgGrid3www.ivdgl.org/grid3Virtual Data Toolkitwww.griphyn.org/vdtGriPhyNwww.griphyn.orgiVDGLwww.ivdgl.orgPPDGwww.ppdg.netCHEPREOwww.chepreo.orgUltraLightultralight.cacr.caltech.eduGlobuswww.globus.orgCondorwww.cs.wisc.edu/condorLCGwww.cern.ch/lcgEU DataGridwww.eu-datagrid.orgEGEEwww.eu-egee.org

Paul Avery

Extra Slides

Paul Avery

GriPhyN GoalsConduct CS research to achieve visionVirtual Data as unifying principlePlanning, execution, performance monitoringDisseminate through Virtual Data ToolkitA concrete deliverableIntegrate into GriPhyN science experimentsCommon Grid tools, servicesEducate, involve, train students in IT researchUndergrads, grads, postdocs, Underrepresented groups

Paul Avery

iVDGL GoalsDeploy a Grid laboratorySupport research mission of data intensive experimentsProvide computing and personnel resources at university sitesProvide platform for computer science technology developmentPrototype and deploy a Grid Operations Center (iGOC)Integrate Grid software toolsInto computing infrastructures of the experimentsSupport delivery of Grid technologiesHardening of the Virtual Data Toolkit (VDT) and other middleware technologies developed by GriPhyN and other Grid projectsEducation and OutreachLead and collaborate with Education and Outreach effortsProvide tools and mechanisms for underrepresented groups and remote regions to participate in international science projects

Paul Avery

Virtual Data: Derivation & ProvenanceMost scientific data are not simple measurementsThey are computationally corrected/reconstructedThey can be produced by numerical simulationScience & eng. projects are more CPU and data intensivePrograms are significant community resources (transformations)So are the executions of those programs (derivations)Management of dataset dependencies critical!Derivation: Instantiation of a potential data productProvenance: Complete history of any existing data productPreviously:Manual methodsGriPhyN:Automated, robust tools

Paul Avery

Virtual Data Example: HEP Analysisdecay = WWWW e Pt > 20decay = WWWW e decay = WWWW leptonsmass = 160decay = WWdecay = ZZdecay = bbScientist adds a new derived data branch & continues analysis

Paul Avery

Packaging of Grid Software: PacmanLanguage: define software environmentsInterpreter: create, install, configure, update, verify environmentsVersion 3.0.2 released Jan. 2005LCG/ScramATLAS/CMTCMS DPE/tar/makeLIGO/tar/makeOpenSource/tar/makeGlobus/GPTNPACI/TeraGrid/tar/makeD0/UPS-UPDCommercial/tar/makeCombine and manage software from arbitrary sources.% pacman get iVDGL:Grid31 button install: Reduce burden on administratorsRemote experts define installation/ config/updating for everyone at once

Paul Avery

Virtual Data MotivationsIve detected a muon calibration error and want to know which derived data products need to be recomputed.Ive found some interesting data, but I need to know exactly what corrections were applied before I can trust it.I want to search a database for 3 muon events. If a program that does this analysis exists, I wont have to write one from scratch.I want to apply a forward jet analysis to 100M events. If the results already exist, Ill save weeks of computation.

Paul Avery

Background: Data Grid Projects

U.S. Funded ProjectsGriPhyN (NSF)iVDGL (NSF)Particle Physics Data Grid (DOE)UltraLightTeraGrid (NSF)DOE Science Grid (DOE)NEESgrid (NSF)NSF Middleware Initiative (NSF)

EU, Asia projectsEGEE (EU)LCG (CERN)DataGridEU national ProjectsDataTAG (EU)CrossGrid (EU)GridLab (EU)Japanese, Korea ProjectsMany projects driven/led by HEP + CSMany 10s x $M brought into the fieldLarge impact on other sciences, educationDriven primarily by HEP applications

Paul Avery

Virtual Data: Derivation & ProvenanceMost scientific data are not simple measurementsThey are computationally corrected/reconstructedThey can be produced by numerical simulationScience & eng. projects are more CPU and data intensivePrograms are significant community resources (transformations)So are the executions of those programs (derivations)Management of dataset dependencies critical!Derivation: Instantiation of a potential data productProvenance: Complete history of any existing data productPreviously:Manual methodsGriPhyN:Automated, robust tools

Paul Avery

Muon Lifetime Analysis Workflow

Paul Avery

(Early) Virtual Data Languagepythia_inputpythia.execmsim_inputcmsim.exewriteHitswriteDigisbegin v /usr/local/demo/scripts/cmkin_input.csh file i ntpl_file_path file i template_file file i num_events stdout cmkin_param_file end begin v /usr/local/demo/binaries/kine_make_ntpl_pyt_cms121.exe pre cms_env_var stdin cmkin_param_file stdout cmkin_log file o ntpl_file end begin v /usr/local/demo/scripts/cmsim_input.csh file i ntpl_file file i fz_file_path file i hbook_file_path file i num_trigs stdout cmsim_param_file end begin v /usr/local/demo/binaries/cms121.exe condor copy_to_spool=false condor getenv=true stdin cmsim_param_file stdout cmsim_log file o fz_file file o hbook_file end begin v /usr/local/demo/binaries/writeHits.sh condor getenv=true pre orca_hits file i fz_file file i detinput file i condor_writeHits_log file i oo_fd_boot file i datasetname stdout writeHits_log file o hits_db end begin v /usr/local/demo/binaries/writeDigis.sh pre orca_digis file i hits_db file i oo_fd_boot file i carf_input_dataset_name file i carf_output_dataset_name file i carf_input_owner file i carf_output_owner file i condor_writeDigis_log stdout writeDigis_log file o digis_db endCMS Pipeline

Paul Avery

QuarkNet Portal ArchitectureSimpler interface for non-expertsBuilds on Chiron portal

Paul Avery

Integration of GriPhyN and IVDGLBoth funded by NSF large ITRs, overlapping periodsGriPhyN:CS Research, Virtual Data Toolkit(9/20009/2005)iVDGL:Grid Laboratory, applications(9/20019/2006)Basic compositionGriPhyN:12 universities, SDSC, 4 labs(~80 people)iVDGL:18 institutions, SDSC, 4 labs(~100 people)Expts:CMS, ATLAS, LIGO, SDSS/NVOGriPhyN (Grid research) vs iVDGL (Grid deployment)GriPhyN:2/3 CS + 1/3 physics( 0% H/W)iVDGL:1/3 CS + 2/3 physics(20% H/W)Many common elementsCommon Directors, Advisory Committee, linked managementCommon Virtual Data Toolkit (VDT)Common Grid testbedsCommon Outreach effort

Paul Avery

GriPhyN OverviewScienceReviewProductionManagerResearcherinstrumentApplicationsGrid FabricdataServicesServicesVirtualDataplanningPlanningPlanningExecutionplanningVirtual DataToolkitChimera virtual data system

Pegasus planner

DAGman

Globus ToolkitCondorGanglia, etc.Execution

Paul Avery

Chiron/QuarkNet Architecture

Paul Avery

Cyberinfrastructure A new age has dawned in scientific & engineering research, pushed by continuing progress in computing, information, and communication technology, & pulled by the expanding complexity, scope, and scale of todays challenges. The capacity of this technology has crossed thresholds that now make possible a comprehensive cyberinfrastructure on which to build new types of scientific & engineering knowledge environments & organizations and to pursue research in new ways & with increased efficacy.

[NSF Blue Ribbon Panel report, 2003]

Paul Avery

Fulfilling the Promise ofNext Generation Science

Our multidisciplinary partnership of physicists, computer scientists, engineers, networking specialists and education experts, from universities and laboratories, has achieved tremendous success in creating and maintaining general purpose cyberinfrastructure supporting leading-edge science.

But these achievements have occurred in the context of overlapping short-term projects. How can we ensure the survival of valuable existing cyber-infrastructure while continuing to address new challenges posed by frontier scientific and engineering endeavors?

Paul Avery

Production Simulations on Grid3Used = 1.5 US-CMS resourcesUS-CMS Monte Carlo Simulation USCMSNon-USCMS

Paul Avery

Components of VDT 1.3.5Globus 3.2.1Condor 6.7.6RLS 3.0ClassAds 0.9.7Replica 2.2.4DOE/EDG CA certsftsh 2.0.5EDG mkgridmapEDG CRL UpdateGLUE Schema 1.0VDS 1.3.5bJavaNetlogger 3.2.4Gatekeeper-AuthzMyProxy1.11KX509System ProfilerGSI OpenSSH 3.4Monalisa 1.2.32PyGlobus 1.0.6MySQLUberFTP 1.11DRM 1.2.6aVOMS 1.4.0VOMS Admin 0.7.5Tomcat PRIMA 0.2Certificate ScriptsApachejClarens 0.5.3New GridFTP ServerGUMS 1.0.1

Paul Avery

Collaborative Relationships:A CS + VDT PerspectiveComputerScienceResearchVirtualDataToolkitPartner science projectsPartner networking projectsPartner outreach projectsLargerScienceCommunityGlobus, Condor, NMI, iVDGL, PPDGEU DataGrid, LHC Experiments,QuarkNet, CHEPREO, Dig. DivideProductionDeploymentTechTransferTechniques& softwareRequirementsPrototyping& experimentsOther linkagesWork forceCS researchersIndustryU.S.GridsIntlOutreach

Paul Avery