1 Glen Cowan Statistics Forum News
Statistics Forum News
Glen Cowan
Eilam Gross
ATLAS Statistics Forum
CERN, 3 December, 2008
2 Glen Cowan Statistics Forum News
Higgs Combination Exercise CSC note HG11 completed
Statistics Book Statistics FAQ
Interactions with physics groups
ATLAS/CMS Joint Statistics Meetings
Recent Statistics Forum activity
3 Glen Cowan Statistics Forum News
Higgs combination complete (p 1473 of CSC book)Describes statistical methods for combination of search channels.
Applies combination to (a subset of) Higgs channels:
WW→e(0-jet and 2-jet), , ,ZZ(*)→4l
Treats systematics by means of profile likelihood method.
Considers fixed-mass hypothesis; look-elsewhere-effect mustbe studied separately.
NB H and WW CSC notes describe floating-mass fits.
Some approximations used for discovery/exclusion significance.
Valid for L > 2 fb. For lower L need toy MC methods.
Does not represent final word on methods -- other developments ongoing (Bayesian, CLs, look-elsewhere-effect,...)
4 Glen Cowan Statistics Forum News
CSC HG11 PeopleEditors: Eilam Gross, Glen Cowan
Contacts for individual channels:
WW Bill Quayle
Yaquan Fang
Kyle Cranmer
4l Andrea Di Simone
Note referee: Fabiola Gianotti
Chapter referees: Fabio Cerutti, Bruno Mansoulie
Chapter editors: Rachid Mazini, Bill Murray
+ e.g. Louis Fayard, Aleandro Nisati, Karl Jakobs,
Dave Charlton, Ofer Vitells...
5 Glen Cowan Statistics Forum News
Reminder of combination method
represents all nuisance parameters, e.g., background rate, shapes =1 is SM.
=0 is background only.
The single-channel likelihood function uses Poisson modelfor events in signal and control histograms:
There is a likelihood Li(,i) for each channel, i = 1, …, N. The full likelihood function is
6 Glen Cowan Statistics Forum News
Profile likelihood ratio
To test hypothesized value of , construct profile likelihood ratio:
Maximized L for given
Maximized L
Equivalently use q = 2 ln ():
data agree well with hypothesized → q small
data disagree with hypothesized → q large
7 Glen Cowan Statistics Forum News
p-value / significance of hypothesized Test hypothesized by giving p-value, probability to seedata with ≤ compatibility with compared to data observed:
Equivalently use significance:
8 Glen Cowan Statistics Forum News
Sampling distribution of q
To compute p-value of hypothesized , need distribution f (q|).
For 5discovery we need a p-value of 2.8 × 10;
To estimate this using MC needs > 108 events.
Wilk’s theorem: in large sample limit f (q|) should approach
with w = ½. (a “half-chi-square” distribution).
Validation exercise shows approximation OK for L ≥ 2 fb.
We use the half-chi-square approximation in the current note;for lower luminosities we will need MC to get f (q|)(feasible for exclusion limits at 95% CL).
9 Glen Cowan Statistics Forum News
Example from validation exercise: ZZ(*) → 4l
5 level
(One minus)cumulativedistributions.Band gives 68%CL limits.
½2
½2
½2
½2
Distributions of q0 for 2, 10 fb from MC compared to ½2
10 Glen Cowan Statistics Forum News
SensitivityDiscovery:
Generate data under s+b (= 1) hypothesis;Test hypothesis = 0 → p-value → Z.
Exclusion:Generate data under background-only (= 0) hypothesis;Test hypothesis .If = 1 has p-value < 0.05 exclude mH at 95% CL.
Estimate median significance by setting data equal to expectationvalues (Asimov data) or by using MC.
For median, can combine significances of individual channels.
For significance of e.g. a real data set, need global fit.
11 Glen Cowan Statistics Forum News
Combined discovery sensitivity
5
12 Glen Cowan Statistics Forum News
Combined discovery sensitivity vs. mH, L
Approximationsnot reliable below2 fb (usuallyconservative)
13 Glen Cowan Statistics Forum News
p-value of SM hypothesis (=1) vs. mH
mH > 115 at 95% CL with 2 fb
14 Glen Cowan Statistics Forum News
Combined exclusion sensitivity vs. mH, L
Approximationsnot reliable below2 fb (usuallyconservative)
15 Glen Cowan Statistics Forum News
Summary of the Higgs Combination
Final draft onhttp://indico.cern.ch/conferenceDisplay.py?confId=25831
higgs_combo_csc_28nov08.pdf
CSC book imminent, we are provisionally pages 1473 to 1516
Many issues/problems identified and worked through useful.
Only subset of methods explored much more work left.
Method now documented, global fit tools under development.
We need to learn how to do this more quickly...
16 Glen Cowan Statistics Forum News
Statistics Forum Website: FAQ
Some general items:PDG Chapters,Pedestrian's guide,Glossary, ...
17 Glen Cowan Statistics Forum News
Statistics Forum FAQ Notes
This is a living document
18 Glen Cowan Statistics Forum News
Statistics Forum FAQ NotesThe “FAQ” consists of a collection of notes on specific questionsuse cases, examples, ...
Bayesian methods for ATLAS Higgs search (GC)Comparison of significance from profile and integrated
likelihoods (GC, EG) Discovery significance with statistical uncertainty in the
background estimate (EG, OV, GC)Error analysis for efficiency (GC)How to measure efficiency (DC)MC statistical errors in ML fits (GC)Covariance matrix for histogram made using seed events (GC)
If you have a note which you think should be included here, orif you are interested to write such a note or comment on a note or request a note on a specific subject please let us know.
19 Glen Cowan Statistics Forum News
Interaction with physics groupsOver past year much interaction with Higgs group
Combination exercise (relevant to other searches as well...)
Continue to extend this to other groups (Exotics, SUSY, B-Physics,...)E.g. SUSY Group (Dan T.) posed several question which we have tried to answer in notes (see FAQ – Profile vs integrated likelihoods,correlations in histograms arising from resampling).
Encourage talks from physics groups in Statistics Forum, e.g.,Exotics Lepton+X (Luis F., 24.9.08) – discussions ongoingSUSY (Sascha C., today)B-Physics (Emmerich K., 28.7.08) – ongoing
Will push for further interaction – written note is preferred outcome.
20 Glen Cowan Statistics Forum News
Joint ATLAS/CMS Statistics Forum
Summer 08: agree to develop RooStats as common framework. Keep eye on ability to carry out independent validation. Key players:
Kyle Cranmer (ATLAS)Gregory Schott (CMS)Wouter Verkerke (RooFit)Lorenzo Moneta (Root)
Ongoing discussions on methodologyIdeal is to use several methods (profile likelihood, Bayesian, Cls,...)for each result.
Meetings:25 September 200829 July 20084 September 200717 July 2007
21 Glen Cowan Statistics Forum News
Summary
Higgs combination exercise completeValuable exercise especially for developing profile likelihoodTime to move on
Encourage interaction with all physics groups
ATLAS/CMS interaction focused on RooStats ( Kyle talk next)