FD events and fit considerations

• I intend to cover two topics in this talk:– FD beam events

• Selecting events, signal and backgrounds, event characteristics…– Event selection by topological cuts, data/MC comparison (Niki)

– Event scan, selection by topology/timing, data/MC comparison (David)

– Optimisation of fiducial volume for CC analysis, data and cosmic background rates (Andy C.)

– Beam energy decision & oscillation fit at 1e20 p.o.t• General considerations

– Reminder of 1e20 sensitivity & break-point between LE/ME sensitivity

• Fit procedure– my thoughts on data/mc match-up in ND

– ND/FD extrapolation – various approaches:

“Know nothing” extrapolation (Trish/Jenny)

Flux extraction from fit to ND data (Masaki)

D. A. Petyt – CC summary talk 15/10/05

FD event studies - goals

• Look at FD “spill” events (with blinding scheme imposed) to determine– Whether we observe neutrino interactions at roughly

the expected rate (i.e constant number of nu/pot)

– Whether the events look “OK” – both in terms of general appearance and reconstructed quantites

– What backgrounds exist and how to remove them

Analysis #1 – FD event selection using topological cuts - Niki

• Event sample used – “Open” dataset from Jun 1- Aug 31, corresponding to 4.07e19 p.o.t.

This sample uses the same “beam” reconstruction config. as the neutrino events

This sample uses the “cosmic” reconstruction config – the cosmics in this sample won’t properly represent those in the “spill” sample, but can be used as a check

Selection efficiency & background rejection

(from MC)

• Selected events were scanned and classified as signal or background

True neutrinos should be located in a 10us time window

• These are typical “junk” events – radioactive noise in the spill trigger window forming a single low energy shower

Comparison of cuts and visual scan

Analysis #2 – FD scan/selection cuts for events with tracks - David

• Scanned reconstructed FD “spill” files (with blinding applied) from Jun 1 – 19 Sept (LE-10 running, 5.1e19 p.o.t).

• 1348 events scanned and classified into the following categories:

• Comments on the scan:– Beam neutrino events are in general very distinctive and it is very easy to distinguish

them from background– The large number of “junk” events is dominated by radioactive noise in the spill trigger

window. These are also pretty distinctive although it may be difficult to separate these from the very lowest energy NC events

– There are a few LI events in the “spill” files, most (if not all) are accompanied by trigger PMT hits (a cut on this removes 15 or the 17 LI events). In any case, they are rather distinctive and do not look at all like neutrino interactions

– The “problem” events are typically small events that occur close to the edge of the detector

• Small showers that could be NC events or incoming junk• Short tracks where the directionality is not clear.

Nu Rock mu Cosmic Junk LI unknown Nu? Rock mu?

140 33 103 1037 17 6 9 3

Cut #2: timing wrt FD spill prediction: –20us<t<30us

Junk event

Cut #3: event must contain a reconstructed track

Vast majority of “junk” is 0 track + 1 shower…

Cut #4: track direction cosine wrt beam >0.6

Cut #5: track vertex r2<14 m2

Cosmics pile up around detector edge – expect uniform distribution for neutrinos

Cut #6: track vtx z>0.2m from SM ends, z<28m

Result of cut sequence

Cut Nu Rock mu Cosmic Junk LI unknown Nu? Rock mu?

NONE 140 33 103 1037 17 6 9 3

Litime<0 140 33 103 1037 2 6 9 3

-20<tfar<30 s 139 33 39 468 0 6 9 3

Ntrack>0 123 33 34 1 0 2 4 3

Dircosneu>0.6 118 33 7 1 0 2 4 3

Track fid cuts 88 0 0 0 0 0 1 0

• On the assumption that all 139 candidates are real neutrinos, the efficiency of the selection cuts is 88/139=63.3%.

Effect of cuts on MC events

• Efficiency of cuts on MC: 81797/128631=63.6%, which agrees well with data (assuming scan efficiency is high of course…)

Cut Numu CC NC Nue CC

NONE 102361 26635 1516

Raw ph<500000 100712 (98.4%) 26466 1453

Ntrack>0 95315 (94.6%) 15443 1091

Dircosneu>0.6 92851 (97.4%) 14079 1021

Trk vtx r2<14 77610 (83.6%) 11703 868

Track z vtx cuts 70346 (90.6%) 10647 804

Neutrinos vs time

• Ratio of neutrinos/pot seems pretty constant.

FidVol optimisation – Andy C.- Want to optimise the current analysis cuts (PID and

FidVol) and see if there are any differences between optimal cuts for best resolutions in sin22 and m2.

- Proposed new FidVol cuts for CC:

- 20cm distance to edge cut & 40cm inner radial cut

- 3 plane cut at front of SM1&2 and back of SM1

- 13 plane cut at the back of SM2

- If all optimal cuts are used improvement in resolution of 14.8% in sin22 and 10.2% in m2.

Parameter measurement errors vs fiducial cut

• Increase in statistics over-compensates loss in resolution for radial cut – implication is to push this as far as you can before running into problems with background contamination

• Optimum z vtx cut –13 planes. There is a minimum here – as you go further downstream in z, the events become poorly measured and identified and the increase in statistics in offset by worse energy resolution

Radial cut End of SM 2 cut

Potential Energy Bias

- There is some loss in shw energy for these new events, but the difference is only in the order of 3-4%, so as long as simulation of shw size is reasonable, should not be an issue.

- Can assume trk energy reconstruction will suffer no bias due to the use of the TrackFitter.

Event passing existing cuts : New events added by relaxing cuts

Effect on cosmic MC- ~50,000 Full spectrum Cosmics (processed with SR beam config) corresponds to

~36hrs of exposure.

- Detector is ‘open’ to Cosmics for only 100s every spill, approx every 2.5s, so a total exposure time of 5.2s in 36hrs, so this sample corresponds to ~100 yrs spill data.

- Application of a cut on trk direction relative to beam and track dt/ds removes most of cosmic background with little impact on beam events

- Background with relaxed cuts =0.68 cosmic/yr in spill window

- Loss of beam MC acceptance ~4%

All cosmic Old fidvol cuts Relaxed fidvol cuts

FD Beam data-Looked at July, August and half of September R1_18 blinded spill data.

Applied sensible Beam info cuts:•PoT > 1.5E11•-2mm < hpos < 0mm•0mm < vpos < 2mm•-5000V < Magnet Voltage < -4500V•-179.5 kA < Horn Current < -178.5 kA•Target Position > 0

Also require reconstructed track, PID>-0.4 and LI_time < 0

-With existing Fidvol conditions:

- 34 events

- 32 Beam

- 1 Cosmic

-With relaxed Fidvol conditions:

- 45 events

- 43 Beam (34% increase)

- 1 Cosmic (0% increase)- From MC we expect increases of 27% and 349% in Beam and Cosmics

- Due to Beam cuts this represents ~1.5 months data taking, so would expect to see 0.3 cosmics for existing fidvol and 1.4 for relaxed fidvol.

Problems #1: events occurring before FD spill time

• Niki found 6 of 106 neutrino events that occur (just) outside the 0-10 us spill timing window– A visual scan shows that the

events look OK– The time distribution of the

strips in the event relative to the trigger time (bottom plot) also looks reasonable

• The earliest strip in these events occurs >200ns before the spillserver prediction. Is there an understood mechanism for this?– GPS “error” quoted as 200

ns…

Problems #2 – Events with “late” digits• Niki also noticed that the time width of

events (time of latest strip – time of earliest strip) was of the order of 10-50 us for almost ½ of the neutrino candidates, as opposed to the expected ~100ns time width

• This is caused by the reconstruction code picking up random noise hits in the spill trigger window and adding them to the event– Given the short time window, this should

not significantly affect reconstructed quantities

– These hits can (and should) be removed by applying some loose timing criteria in the event formation

Problems #3 – Displaced event vertices

• Andy observed that the event vertex for cosmic events tends to be displaced whenever there is a reconstructed shower (brem, delta ray...) on the track– We think the vertex finder in the beam

config assumes the primary track and shower should originate at the same point, and computes an “average” vertex position if they do not

– The net result of this is that the event vertex of cosmic events can be within the fiducial volume, even though the primary track vertex is not

• Should therefore use track vertex for fiducial cuts if it is available– Josh is aware of this and improvements

will be made to the next software release

Track vertex

Shower vertex

shower

Good CC candidate

Good NC candidate

Good anti-neutrino candidate

Event characteristics and data/MC comparisons

• The next several slides show comparisons between basic distributions for selected data and MC events (with the same cuts applied)

• Because of blinding/oscillations, we don’t expect all distributions to agree– Physics distributions will be distorted, but we should at least be

able to check if there are any glaring pathologies (i.e. unphysical “spikes”)

– Lower level quantities (such as pulse height/plane) should be unaffected

– I have indicated the distributions that will be significantly affected by blinding/oscillations with the label “B”

– All plots are normalised to the same area.– I will show plots separately for Niki’s events (events with 0 and

1+ tracks) and my events (events with 1+ tracks only)

Event vertices and track direction

Track vertex x-y projection

• LHS plot should be uniform in x-y

• RHS plot should be focussed towards origin for numu events and defocussed for numu_bar events

Track Vertices #2

Background could appear in these areas

Background could appear in these areas

Numbers of reconstructed tracks & showers

Track fit parameters

• Track fitting performance seems very similar between data and MC.

• Similar rate of failures <5% (slightly lower in data than MC)

• Track q/p distribution distorted by blinding/oscillations…

B

Track variables

• Track length is (should be!) distorted.

• Track ph/plane variables should be largely invariant– Slight excess of

high ph/plane events in data?

• Track digits/plane slightly lower in data– Could be caused by

lower per plane tracking efficiency in the data

B

Event length and ph variables

• Reasonable match-up given the statistics – track ph/plane looks a bit higher in data…

Event length (planes) Pulse height per plane

Track pulse height per planeShower pulse height

B

B

Shower reconstruction

• Data showers tend to reconstruct more clusters with higher average energy– Could be an artifact

of blinding

• However shower pulse height per digit is also somewhat higher – this should be largely invariant to blinding/osc.

B

“Physics” quantities

• Most of these will be distorted by blinding and oscillations

• However, the general shape of the distributions seems reasonable – there are no glaring “pathologies”…

B B

B

BB

Per-strip quantities

• Track and shower ph/strip seem slightly (~5%) lower in data than MC.

Slight excess of NC/deficit of CC – blinding and/or oscillations!

BB

Likelihood-based PDF

• Optimal cut to separate CC and NC (for R1.18) is PID>-0.2• Higher fraction of NC-like events in the data, which is largely due to the

shorter event length distribution. This could be due to oscillations and/or blinding…

B

FD event selection conclusions• Beam neutrino events are very distinctive and it is quite easy to

isolate them with either timing and/or topology cuts– The backgrounds appear to be small and can probably be reduced further.

This may allow us to use a larger fiducial region for the analysis– So far, the best selection efficiency we have achieved is 87%

• Short events at the edge of the detector tend to be the most problematic

– The rate of neutrino events observed follows the beam intensity well

• So far, the match-up of data and MC shows no large discrepancies:– At least as far as we can tell with these statistics and with

blinding/oscillations– There are some small differences in “invariant” quantities, which should

be investigated further, but no pathological effects have been observed– There are some minor timing/reco issues to be looked at further or fixed

(offset in true-predicted time, event vertex shift etc.)

Some thoughts on 1e20 p.o.t analysis and opening the box…

• Our plan is to open the box once we have accumulated ~1e20 p.o.t. and make our first determination of the value of m2

• This should provide a check as to whether we are running at the right beam energy– Older calculations show the the cross-over

between LE and ME sensitivity is m2~0.005 eV2. Should be re-done for newer beam/reco, but I doubt that the cross-over value will change much

• Given the difference between the signals at m2 = 0.002 and 0.005, it’s clear that we should be able to tell them apart unless we have very major differences between data and MC

Comments on ND data/MC match-up

• In order to write a paper on the 1e20 data, we will need to be able to demonstrate that we understand our near detector data at a level sufficient to predict the FD spectrum without significant bias in the oscillation measurement.

• At the moment, despite significant study, there are still fairly large data/MC differences– These could be detector/reco effects, physics (beam/x-sec) or both…

• How do we do the extrapolation in this case?– Ideally, we want to factorise detector-specific differences (which can be

modelled or fixed) and the remaining differences (which can be attributed to physics) should be within the systematic error envolope from beam/x-sec uncertainties

– However, there may be residual differences that won’t be understood on this timescale

Alternate approach to ND/FD fits – Trish/Jenny

• Take what we measure in the ND, use the Far/Near ratio and selection efficiencies from the MC to predict what the FD spectrum should look like in the case of no oscillations and oscillations with given values of (delta m^2, sin^2(2theta)). Use Minuit to fit for sin^2(2theta) and delta m^2.

• Approach different from David's method—provides an independent check

• Doesn't assume differences between data and MC in the ND spectrum are parameterizable

• Uses bin by bin differences between data and MC in the ND as basis for errors on the FD spectrum

• Minimization technique using Minuit is fast – Doesn't need multiple loops over nuisance parameters – Allows for 1000's of pseudo-experiments

Calculating errors on the FD spectrum prediction

1. Find the relative difference between Data/MC in ND for each bin of reco energy.

2. Determine the error in bins of true energy by histogramming the true energy for each bin of reco energy, weighted by 1+/- the relative error from step 1, then add up these histograms.

3. Determine relative error for each bin of true neutrino energy.4. Relative errors on the true energy applied to the true energy in

the FD. 5. Add Beam syst. errors from F/N ratio (linearly)6. Histogram reco energy in FD for each bin of true energy,

weighted by 1+/- the relative error of step 4. Sum histograms. Half the spread between the 1+error and 1-error histograms is taken as the error in each bin of reconstructed energy in the FD.

Computing the FD error envelope

Far/near

Error bars on FD prediction are +/- 1 sigma systematic

Example results

Systematics dominated here – doesn’t scale as sqrt(N)

Potential Pitfalls at 1e20

0.0022eV2

0.005eV2

Fits are more stable at high dmsq…

Calibration +5% • Difference between

ND and FD calibration can be studied

• Effects worse at low m2

• Extra fit parameter can alleviate this problem

Using ND data to estimate neutrino flux - Masaki

• motivation MdEEEEN )()()(

Fluxcross-section

efficiency

Fiducial mass

Parameters measure by the ND data

• procedure

2

2

22

22 ))1((

MCData

MCjiData NfN

True ET

rue

yE (10%) and Eshw (10%) are also considered in fit

32 CC + 8 NC parameters

Select CC enriched sample (PID by David)

pMELE pHE

Data (March)

Data

MC

Data

MC

Data

MC

NCanti-

CC

R1.18

NC

CC efficiency

NC miss-ID

#events/1017POT before and after PID cut (CC fraction %)

-9.3%

-16.1%

-14.2%

LE data (March) 511.1+/-11.0 384.3+/-9.5LE data 486.7+/-2.4 365.6+/-2.1LE MC 537.1+/-4.4(86.3%) 423.9+/-3.9(95.8)

pME data 911.8+/-13.3 655.3+/-11.3pME MC 1022.7+/-8.6 (85.0) 781.1+/-7.5 (96.4)

pHE data 1395.8+/-9.4 988.7+/-7.9pHE MC 1521.8+/-12.5 (83.9) 1151.8+/-10.9

(96.8)

Pid shifted to lower values

ND distributions before & after fit

Reco. y < 0.1

Reco. y > 0.8

0.1 < y < 0.2

0.5 < y < 0.6

0.4 < y < 0.5

0.3 < y < 0.4

0.6 < y < 0.8

0.2 < y < 0.3

Reconstructed E2 = 919.2/152d.o.f.

Reco. y < 0.1

Reco. y > 0.8

0.1 < y < 0.2

0.5 < y < 0.6

0.4 < y < 0.5

0.3 < y < 0.4

0.6 < y < 0.8

0.2 < y < 0.3

Reconstructed E2 = 136.6/152d.o.f.

Data: black MC: red (normalized by POT) Best fit: blue

Result of the fit

Constrained fit – sigma(f)=20% Unconstrained fit

-0.1

0.18

Best-fit parameters

NC

CC

E=+5%, Eshw=-13%

-0.5

0.3

2 = 107.2/112 d.o.f.

NC

CC

Best-fit parameters

E=+5%, Eshw=-13%

Fit wants more low E, low y events…

Comments & summary

• Matching up data/MC in the ND and understanding all the differences may take some time.– We will probably have to live with some level of disagreement for

the 1e20 measurement

• How good is “good enough”?– What bias is introduced in the FD fit if we assume all the ND

differences are due to detector/reco effects and not physics?• Given the statistics, this probably won’t shift things much…

• We are unlikely to fool ourselves for the purposes of the beam decision…

• Higher threshold for writing a paper of course.

• Jeff will now talk about the procedure and timescale for opening the box for the 1e20 measurement…