The story of BaBar: an IT perspective

Roger BarlowDESY 4th September 2002

Roger Barlow 4 Sept 2002 BaBar: an IT perspective Slide 2/18

Example:

OPAL

BaBar (Work in progress)

Roger Barlow 4 Sept 2002 BaBar: an IT perspective Slide 3/18

History

1995 TDR C++

1999 First data 0.5 fb-

1

Objectivity / kanga

2000 First results 23 fb-1 File transfer

2001 Sin 2 0.590.140.05 39 fb-1 Tier A sites

2002 Sin 2 0.7410.0670.029 28 fb-1 Grid

Roger Barlow 4 Sept 2002 BaBar: an IT perspective Slide 4/18

Datataking:“Drinking from a firehose”

L1 Trigger L3 Trigger

OPR farm

Data store

200 kB/evt

30-100 Hz

4.2 ns

1-2kHzPowerPCs in VME crates

200+ CPUs

32 SUN U5

Roger Barlow 4 Sept 2002 BaBar: an IT perspective Slide 5/18

C++

“Unorthodox but very nice introduction to programming”

Amazon.com review

Decision taken: approved languages C++ and F90

(Nobody used F90)HEP escapes at last

from FORTRAN!Other experiments

followSoftware immediately

becomes VERY object-oriented

Roger Barlow 4 Sept 2002 BaBar: an IT perspective Slide 6/18

C++ in practiceDesign

Old New Abstract objects

EmcCluster:AbsRecoCaloAbsRecoCalo: AbsEvtObj

EmcClusterMoments: EmcAbsClusterProperty

EmcClusterDistance: EmcAbsClusterProperty

The software was written on time and it worked.

Roger Barlow 4 Sept 2002 BaBar: an IT perspective Slide 7/18

Some lessons

RogueWaveCommercial class libraryBuilt in to software from

startSuperseded by STL,

CLHEPLicence arrangements

restrictive/expensiveNow (painfully) removed

Calibration System impressive technicallyCan’t do what you want it to

Event DisplayPowerful system (‘Event

display server’) written by experts

Unwieldy and unreliable for non-experts away from SLAC

Little used

Roger Barlow 4 Sept 2002 BaBar: an IT perspective Slide 8/18

Objectivity

‘Persistent data’• Objects still present

when a job is run again- stored on disc

• Need to handle pointers

• Need to match data on disk to objects the program knows about

• Need to do this fast and furiously

• Objectivity does this – no alternative

• Performance issues in OPR: solved (hard work+vendor co-operation)

• Performance issues in analysis: not satisfactory (physicists more random) KANGA

• Long-term viability?

Roger Barlow 4 Sept 2002 BaBar: an IT perspective Slide 9/18

Success: the penalty

PEPII deliversand BaBar can handle it

Design Luminosity 3 1033 achieved

Now 5 1033

Heading for 4 1034

BaBar efficiency ~ 95%

(This is a race. KEK and Belle are close behind!)

Data handlingNeed to provide• Disk (or tape) for data

storage• Machines for OPR• Machines for analysisMoore’s Law is not strong

enough

Need new computing model Central Distributed

Roger Barlow 4 Sept 2002 BaBar: an IT perspective Slide 10/18

File Transfer

Data transferFrom SLAC to outside

sitesLarge (RAL, IN2P3) and

small (university)Use network transfer

rather than shipping tapes

ESNET in US, JANET+ GEANT in Europe

Use RAL, Rome, etc as staging posts

File copy programsbbcp and bbftp

developed, now standard

Copy with multiple streams

Achieve 80-100 Mbit/sData available within

daysChasing improvement

continuous task

Roger Barlow 4 Sept 2002 BaBar: an IT perspective Slide 11/18

Spreading the load

Serious computing power at outside institutes

Each UK BaBar group ~1TB fast disc (1998)Compute farm (80

CPU) (2000)Used for analysisUsed for MC

simulation

Roger Barlow 4 Sept 2002 BaBar: an IT perspective Slide 12/18

RAL Tier A Centre

UK GRID Funds available

Tier 1/A centre at RAL

200+ CPUs 35 TB disk this year (increasing as price drops)

Rebate on common fund contribution

Everybody happySimilar centres at IN2P3, Bologna,

plus Karlsruhe in future

Roger Barlow 4 Sept 2002 BaBar: an IT perspective Slide 13/18

Future Development

Many users running jobs on distributed data with distributed CPUsDon’t want technical detail

GRID provides the answerUser specifies data description and jobGrid tools locate files, find CPUs, submit jobsBaBar today: LHC tomorrow

MC:RB

Roger Barlow 4 Sept 2002 BaBar: an IT perspective Slide 14/18

Authentication

Uses RSA encryption to prove to other

computers (‘gatekeepers’) that you are who you say

you are

Certificate Authorities

Certificates issued by trusted organisations to people they know and trust

(or people they trust know and trust)

Establish mutual trust within BaBar for UKHEP,CNRS, etc

But authentication is not authorisation!

Grid Certificate

Grid Certificate

Roger Barlow 4 Sept 2002 BaBar: an IT perspective Slide 15/18

Authorisation

Users with Grid certificates authorised to use resources at all BaBarGrid sites

Getting on this list is made easy (for BaBar users with a Grid certificate)Dynamic account pool avoids need for 500x50 userids

SLAC MANCHESTER

BaBarGrid sites

Grid Certificat

e

Grid Certificat

e BaBar VO listcron

jobcron job

gridmap files

Roger Barlow 4 Sept 2002 BaBar: an IT perspective Slide 16/18

The BaBar VO

Roger Barlow 4 Sept 2002 BaBar: an IT perspective Slide 17/18

The BaBar Grid Demonstrator

Rudimentary Grid submission and retrieval

Full features to be added later

Roger Barlow 4 Sept 2002 BaBar: an IT perspective Slide 18/18

The Future: Some

Possibilities

SRB as universal file system

Extend VO system for

authorisation

Desk-to-desk Video

conferencing

More Compute and Storage farms

Metadata specification using xml

Join EDG testbed rollout

Framework VI