LHC computing

HEP 101 Lecture #8ayana arce

Outline• Major computing systems for LHC

experiments:– (ATLAS) Data Reduction – (ATLAS) Data Production– (ATLAS) Data Analysis

• End-user tools:– Exercise: plotting and fitting data with

ROOT– homework: writing a toy Monte Carlo

DATA REDUCTIONmanaging the data volume

overview: the data reduction chain

100100110000011011010010011100100011011010001110010001010000100001100010101001101001

100100110000011011010010011100100011011010001110010001010000100001100010101001101001

100100110000011011010010011100100011011010001110010001010000100001100010101001101001

100100110000011011010010011100100011011010001110010001010000100001100010101001101001

100100110000011011010010011100100011011010001110010001010000100001100010101001101001

100100110000011011010010011100100011011010001110010001010000100001100010101001101001

1001001100000110110100100111001000110110100011100101011111001000011111010

101001101101

100100110000011011010010011100100011011010001110010001010000100001100010101001101001

Hardware Trigger (prefilter)

Event Filter (software event selection)

data reconstruction and distribution

40 MHz

75 kHz

~500 MB/sec

100100110000011011010010011100100011011010001110010001010000100001100010101001101001

100100110000011011010010011100100011011010001110010001010000100001100010101001101001

300 Hz

The TDAQ system• Trigger:

– (almost) real-time filtering of collision events– Events read every ~25ns:

• how long does the trigger take to decide?• DAQ:

– Sends event data through the trigger and readout systems

– Merges trigger and detector conditions data with event data

local (event fragments)

~1700 nodes (8/12 core, 16/24 GB)

dedicated L3~10 Gb links

flexible L2/L3 processors10 Gb links

ATLAS

full events

ATLAS trigger system

Example: electron trigger

DATA PRODUCTIONmanaging the data volume

Global data processing and storage• LHC data output estimate: 15 PB/year

(and we prefer multiple copies)– Stored and processed on WLCG:

shared by all CERN experiments– Your “local” Tier-1: BNL– Your local Tier-3: in your

backpack!• Every stored physics event is

modeled by many simulated events– thus most resources are spent in

Monte Carlo simulation

note: ATLAS computing systems alone must handle MILLIONS of production/analysis jobs daily

analyzecreate MC

backup RAWreprocess (re-reconstruct)

store RAWcalibratereconstruct (6k cores)Tier

0

Tier 1

Tier 2

Tier 1

Tier 1

Tier 2Tier 2 Tier 2 Tier 2

Tier 2

Tier 2Tier 2

Tier 2

físicosphysicists

物理学者

38 T2 centers120k cores totalcernVM environment

ATLAS Tier computing: roles

Production: dataATLAS trigger convert

MERGE&

derive

bytestream

RECO

esd

aod

tag

D3PD

aod

RDO(raw)

pattern recognition

sorting

Production: Monte Carlo

MERGE&

deriveRECO

esd

aod

tag

D3PD

aod

MONTE CARLO PRODUCTION CHAINRDO(raw)

pick random x, random yif y2 < 1-x2: increment area

What is Monte Carlo, really?• HEP predictions require a

lot of convolution integrals– one reason: QM!

Monte Carlo calculation of π

pick random x, random yif y2 < 1-x2: increment area

What is Monte Carlo?• HEP predictions require a

lot of convolution integrals– one reason: QM!

• The Monte Carlo Method: – use random numbers

as an integration tool

Monte Carlo calculation of π

this is probably the simplest way to use a computerfor a calculation… but it works!

What is Monte Carlo?

Z picks mass

and decay angles

electronET

• The Monte Carlo Method: – use random numbers as

an integration tool• Very intuitive picture of

convolution integrals:– a series of choices from

probability distributions

What is Monte Carlo?

Z picks mass

and decay angles

electronET

calorimeter (mis)measurement

observed electron ET

• The Monte Carlo Method: – use random numbers as

an integration tool• Very intuitive picture of

convolution integrals:– a series of choices from

probability distributions

Meet your (3-part) Monte Carlo

Slides: Sjöstrand

Meet your MC: PYTHIA, HERWIG, MadGraph, MCFM, MC@NLO, BaurMC, POWHEG, &c.…

Meet your MC: PYTHIA, HERWIG, MadGraph, MCFM, MC@NLO, BaurMC, POWHEG, &c.…

Meet your MC: PYTHIA, HERWIG/JIMMY, Sherpa…

Meet your MC: PYTHIA, HERWIG/JIMMY, Sherpa…

What’s the third part?• Detector simulation:

up to 5 minutes for a high-mass event (lots of particles, each individually tracked through hundreds of detector elements)

why is this essential?

DATA ANALYSISmeasurements and discoveries!

ATLAS computing for usersProgramming languages• Main programming

languages: – FORTRAN (some

generators)– C++ (main

reconstruction algorithms, analysis)

– python (steering, analysis)

Interactive interfaces• Main interface: athena

– reads all data formats– C++ ; steered by

python– this runsall simulation

and reconstruction– can run your analysis

too…but excecutable typically 4GB

• Light interface: ROOT

Data representation• always organized by

event• global quantities:

– metadata – missing energy…

• physics object lists:– muons– jets– tracks– “truth” particles…

• object properties: – hits on tracks– jet constituents

µ tracktracktracktracktracktracktrack

jetjetjet

track hittrack

hit

event

“n-tuple” “tree”

Data representationEvent number

nTracks track pT track eta track phi track layers…

0 3 12.4 0.3 2.1 30

8.1 1.1 1.0 14

5.0 -0.9 4.0 17

1 2 24.5 1.1 0.2 22

20.5 0.9 3.3 17

2 1 2.0 1.9 1.4 5

3 5 40.4 0.1 0.8 21

… … … …

User’s interface to nature: histograms

Pseudocode:histo = makeHisto(nbins=50, firstbin=0*GeV, lastbin=200*GeV)for thisEvent in allEvents:

if HasZBoson( thisEvent ):m = reconstructZBosonMass( thisEvent )

histo.FillWith( m )

``Hello World’’ for HEP computing: making a histogram

TH1F::Fill(value,weight)

TH1F(“name”, “title; x title; y title”, nBins, firstBinValue,

LastBinValue)

EXAMPLE!note: in code examples, your input is given in green

Let’s measure the kaon lifetime (again)!

• open the ROOT file:– you% root Hep101Data_2013.root

• How to see everything in the file:– root [1] new TBrowser();

the file contains one histogram

(taken from your homework)

Some ROOT features:root [0] double x(3.0),y(4.0); sqrt(x*x+y*y)(const double)5.00000000000000000e+00root [1] TLorentzVector pion(1500,0,0,1506.482);root [2] printf("The mass is %3.4g\n", pion.M( ));The mass is 139.6root [3] TMath::C( <TAB>Double_t C() // m s^-1root [4] TMath::C()(Double_t)2.99792458000000000e+08

Mathematical functions in ROOT

• Simple:FitPanel (under Tools)

• Also easy:root [9] KaonDecays->Fit(“expo”)

• More explicit:root [10] TF1 f("f","[0]*exp(-x/(100*[1]*TMath::C()))",0,60);

//free parameters specified in bracketsroot [11] KaonDecays->Fit(f);

• Complete program (from Dave)

Next steps• You can download ROOT:

– root.cern.ch• Homework: write your own Monte Carlo

generator to solve Problem 2 from lecture 5 a neutral pion beam with energy E decays to

two photons. What is the photon energy distribution in the laboratory frame?

• Feel free to contact atarce@phy.duke.edu with solutions, questions, etc!!

homework hint: random numbers

• Use the ROOT class TRandom3 for good performance.

• Example– root [1] TRandom3 r;– root [2] float random1 = r.Gaus(0,35);

//generate a gaussian-distributed random number with mean 0 and width 35;

– root [3] float random2 = r.Flat(0,2*TMath::Pi()); //generate a scalar meson decay angle

Postscript: if you don’t like C++

>>> import ROOT #from ROOT import * also works>>> pion = ROOT.TLorentzVector(1500,0,0,1506.482);>>> print "The mass is", pion.M(), "MeV"The mass is 139.5994854 MeV