- Home
- Documents
*1 Glen Cowan Statistics Forum News Glen Cowan Eilam Gross ATLAS Statistics Forum CERN, 3 December,...*

Click here to load reader

View

219Download

0

Embed Size (px)

Statistics Book Statistics FAQ

Interactions with physics groups

ATLAS/CMS Joint Statistics Meetings

WW→e(0-jet and 2-jet), , ,ZZ(*)→4l

Treats systematics by means of profile likelihood method.

Considers fixed-mass hypothesis; look-elsewhere-effect must

be studied separately.

Some approximations used for discovery/exclusion significance.

Valid for L > 2 fb. For lower L need toy MC methods.

*

Contacts for individual channels:

+ e.g. Louis Fayard, Aleandro Nisati, Karl Jakobs,

Dave Charlton, Ofer Vitells...

=1 is SM.

The single-channel likelihood function uses Poisson model

for events in signal and control histograms:

There is a likelihood Li(,i) for each channel, i = 1, …, N.

The full likelihood function is

*

Maximized L for given

data agree well with hypothesized → q small

data disagree with hypothesized → q large

*

Equivalently use significance:

For 5discovery we need a p-value of 2.8 × 10;

To estimate this using MC needs > 108 events.

Wilk’s theorem: in large sample limit f (q|) should approach

with w = ½. (a “half-chi-square” distribution).

Validation exercise shows approximation OK for L ≥ 2 fb.

We use the half-chi-square approximation in the current note;

for lower luminosities we will need MC to get f (q|)

(feasible for exclusion limits at 95% CL).

*

5 level

(One minus)

CL limits.

½2

½2

½2

½2

*

Test hypothesis = 0 → p-value → Z.

Exclusion:

Test hypothesis .

If = 1 has p-value < 0.05 exclude mH at 95% CL.

Estimate median significance by setting data equal to expectation

values (Asimov data) or by using MC.

For median, can combine significances of individual channels.

*

Approximations

*

Approximations

Final draft on

CSC book imminent, we are provisionally pages 1473 to 1516

Many issues/problems identified and worked through useful.

Only subset of methods explored much more work left.

Method now documented, global fit tools under development.

*

*

Statistics Forum FAQ Notes

The “FAQ” consists of a collection of notes on specific questions

use cases, examples, ...

likelihoods (GC, EG)

background estimate (EG, OV, GC)

Error analysis for efficiency (GC)

How to measure efficiency (DC)

MC statistical errors in ML fits (GC)

Covariance matrix for histogram made using seed events (GC)

If you have a note which you think should be included here, or

if you are interested to write such a note or comment on a note or

*

Continue to extend this to other groups (Exotics, SUSY, B-Physics,...)

E.g. SUSY Group (Dan T.) posed several question which we have

tried to answer in notes (see FAQ – Profile vs integrated likelihoods,

correlations in histograms arising from resampling).

Encourage talks from physics groups in Statistics Forum, e.g.,

Exotics Lepton+X (Luis F., 24.9.08) – discussions ongoing

SUSY (Sascha C., today)

*

Summer 08: agree to develop RooStats as common framework.

Keep eye on ability to carry out independent validation. Key players:

Kyle Cranmer (ATLAS)

Gregory Schott (CMS)

Wouter Verkerke (RooFit)

Lorenzo Moneta (Root)

Ideal is to use several methods (profile likelihood, Bayesian, Cls,...)

for each result.

Time to move on

ATLAS/CMS interaction focused on RooStats ( Kyle talk next)

Interactions with physics groups

ATLAS/CMS Joint Statistics Meetings

WW→e(0-jet and 2-jet), , ,ZZ(*)→4l

Treats systematics by means of profile likelihood method.

Considers fixed-mass hypothesis; look-elsewhere-effect must

be studied separately.

Some approximations used for discovery/exclusion significance.

Valid for L > 2 fb. For lower L need toy MC methods.

*

Contacts for individual channels:

+ e.g. Louis Fayard, Aleandro Nisati, Karl Jakobs,

Dave Charlton, Ofer Vitells...

=1 is SM.

The single-channel likelihood function uses Poisson model

for events in signal and control histograms:

There is a likelihood Li(,i) for each channel, i = 1, …, N.

The full likelihood function is

*

Maximized L for given

data agree well with hypothesized → q small

data disagree with hypothesized → q large

*

Equivalently use significance:

For 5discovery we need a p-value of 2.8 × 10;

To estimate this using MC needs > 108 events.

Wilk’s theorem: in large sample limit f (q|) should approach

with w = ½. (a “half-chi-square” distribution).

Validation exercise shows approximation OK for L ≥ 2 fb.

We use the half-chi-square approximation in the current note;

for lower luminosities we will need MC to get f (q|)

(feasible for exclusion limits at 95% CL).

*

5 level

(One minus)

CL limits.

½2

½2

½2

½2

*

Test hypothesis = 0 → p-value → Z.

Exclusion:

Test hypothesis .

If = 1 has p-value < 0.05 exclude mH at 95% CL.

Estimate median significance by setting data equal to expectation

values (Asimov data) or by using MC.

For median, can combine significances of individual channels.

*

Approximations

*

Approximations

Final draft on

CSC book imminent, we are provisionally pages 1473 to 1516

Many issues/problems identified and worked through useful.

Only subset of methods explored much more work left.

Method now documented, global fit tools under development.

*

*

Statistics Forum FAQ Notes

The “FAQ” consists of a collection of notes on specific questions

use cases, examples, ...

likelihoods (GC, EG)

background estimate (EG, OV, GC)

Error analysis for efficiency (GC)

How to measure efficiency (DC)

MC statistical errors in ML fits (GC)

Covariance matrix for histogram made using seed events (GC)

If you have a note which you think should be included here, or

if you are interested to write such a note or comment on a note or

*

Continue to extend this to other groups (Exotics, SUSY, B-Physics,...)

E.g. SUSY Group (Dan T.) posed several question which we have

tried to answer in notes (see FAQ – Profile vs integrated likelihoods,

correlations in histograms arising from resampling).

Encourage talks from physics groups in Statistics Forum, e.g.,

Exotics Lepton+X (Luis F., 24.9.08) – discussions ongoing

SUSY (Sascha C., today)

*

Summer 08: agree to develop RooStats as common framework.

Keep eye on ability to carry out independent validation. Key players:

Kyle Cranmer (ATLAS)

Gregory Schott (CMS)

Wouter Verkerke (RooFit)

Lorenzo Moneta (Root)

Ideal is to use several methods (profile likelihood, Bayesian, Cls,...)

for each result.

Time to move on

ATLAS/CMS interaction focused on RooStats ( Kyle talk next)