View from ROC-West !

DRC report 17 Feb 2016

Nuno Barros

1

FNAL, Feb 17, 2016

• Noise hunting

• Increased prescale

• Running with all components in disk writing mode (later part of the week)

• SSP tests

• PTB testing

• DAQ stress testing

• More stable after “fixing” the prescale

• Run control “hanging”

• Typically can take ~30 min of data until DAQ crashes2

Major activities

• At about 6 pm on Friday the noise suddenly increased

• Shifter noticed and posted in elog, but nobody noticed the post until next day

• Need to define better procedure to react to events during shifts

3

Noise problems

• High frequency modes showed up again

• First at 500 kHz

• Then “moved” to 650 kHz

• Usually got better after applying Alan’s “trick”

• Ramping bias voltage 100 V and then ramping down again

• Seemed to work until yesterday

4

Noise observationsGood

Really badNot as bad

• Noise seems to increase suddenly

• Small increase over time followed by a sudden shot up

• Run where the noise is high usually crashes (still trying to get this noise increase on file)

5

Noise observations (cont)

• When noise is high RCE’s seem to miss more sync pulses

• When noise was lower failure rate was ~10%

• When noise is high several runs in a row fail to start (miss sync pulse)

• Saturday increase trigger prescale to 8 so that we could take data continuously with disk writing

• DAQ became more unstable

• Turns out the trigger divider was broken (prescale 8 didn’t work). Now prescale set to 7.

6

Still on noise

• RCE’s (or the FEMB attached) 3 and 14 still misbehaving.

7

Recurring issues

• New firmware update tested on PTB

• Fix two remaining glitches

• Fix problem in trigger prescale

• Fix missing microslice at begin of run

• Need to update lbne-artdaq at same time firmware is updated

• Holding it until sync update with John Freeman

• PTB high statistics run taken overnight on Monday (shifter stopped the DAQ at 8 am next day)

• SSP tests being run hourly by shifter

• J. Insler running other tests during the night after DAQ crashes

• SSP01 removed from configuration (lbne-artdaq does not start when it is included)8

Other testing

9

SSP noise

• Behaving better after sorting out the prescale issue

• RCE timeouts still the major reason for crashing

• Occasionally still see crashes showing signs of back pressure

• Happened a couple of times yesterday during the evening shift

• Does not make sense given a prescale of 7 (trigger rate ~0.2 Hz)

• Run control hanged a couple of times

• Seems to be more stable now10

DAQ stability

• Need to get the noise low

• Right now data is almost unusable

• Update lbne-artdaq and PTB firmware

• Will provide better control over the trigger prescales

• Probably temporarily replace vertical (BSU) trigger by other horizontal trigger

• Need expert call list

• Currently responsibility of DRC to know who to contact

• Need available experts11

Next steps

• Could implement a shower trigger

• Based on theory that a cosmic shower sends the TPC into the oscillation pattern

• The current BSU trigger is not being used

• Trigger consists in coincidence of a single counter hit in the roof (RM) with a single counter hit in the cryostat (CL)

• Complement of this trigger would be more than one counter in the roof in coincidence with more than one counter in the cryostat

• It exists in the firmware already

• Implementing the current trigger requires implementing this one

• Need to estimate the rate of this trigger to see if it would be of any use12

Next steps (proposal)