The big picture I

Operations Specialists Paranal [calChecker] [HC monitor] September 2013

The big picture I

Why do we take calibrations?

SCIENCE observations contain:• scientific information

• OB: question by astronomer to the Universe• science observation: answer by the Universe

Reinhard Hanuschik, QC Garching


Calibrations

CALIBs contain:• information about instrumental effects (ARC, FLAT, BIAS)• or atmospheric effects (STD flux, telluric)

SCIENCE reduction:• raw science calibrations with pipeline


Calibrations

CALIBs:• clean, specialized measurement of certain instrumental effect• principle: input known see what the instrument does with this

input know about instrument


Assumptions

Instrument is stable between night and daytime

• then: daytime calibrations sufficient

meaning: instrument conditions between day and night identical to acceptable level

if not (due to unexpected conditions): • science cannot be calibrated, lost


Criteria

Two important quality criteria on calibrations

1. Completeness

If CALIB missing, or earlier/later CALIB exists but is inapplicable:

SCIENCE cannot be calibrated

CALIB completeness saving SCIENCE from last night

provided by calChecker


Criteria

2. Quality

good quality valid measurement of instrumental conditions

valid: e.g. good flux and SNR

• overexposed flat: always bad• well-exposed ARC, but grating not in same position as for science:

bad

Also:• HC calibs check instrument status rather than individual science

CALIB quality ensures science of next night

provided by HC monitor


The big picture II

Where in the dataflow is QC and daytime calib checks?

What is done on Paranal and in Garching?


Garching

automatic process for every instrument:• calChecker every 30 minutes (tellTracker 15 min)• HC monitor every hour• running 24/7

output web pages identical for Garching (HQ) and Paranal (PL)• for convenience: mirror site ‘PL’ (replicated within 15 min)


Garching

QC scientists:• each is responsible for about 4 instruments• 5/7 schedule, daytime only • no shift system, no backups! If sick/travel: only automatic services• in charge of configuration and maintenance of these tools

• e.g. calibration plan, scoring system

• review of past day results• analysis of calChecker issues• analysis of red scores on HC monitor and AB monitor• produce master calibrations


Garching

If issues found:• fix them (if pipeline related), or ticket to developer

or• understand them (based on trending, experience etc.)

and• inform Paranal (mail, comment editors)

QC Garching has the first chance!


Paranal

Daytime 7/7:• review (double-check) calChecker and HC monitor, to fill gaps

(weekends, issues in Garching evening/night) • react on Garching signals, coordinate actions, PPRS• pro-active: discover issues (never rely 100% on QC Garching, the

QC scientist might be sick or on holiday)• but no need for duplication:

if all scores are green, you can continue with the next instrument

no need for visual checks (unless HC system incomplete)! much better: check product monitors (P…) for details


Ideal workflow to check calChecker I

Daytime 7/7:

• calScores green all ok, done, next instrument!

• don’t forget cal4cal• don’t forget long-term calibrations• calScores red investigate

MISS, NOK: check if calOb queue wasinterrupted/stopped restart

• if yellow: wait for end of daytime calib, no need to check earlier!

• nighttime CALIB missing check with OB grade, USD, QCG

• provide your analysis text: very useful!


Ideal workflow to check calChecker II

analysis editor:• choose flag (MISS, NOK, OK)• enter comment• your email address• submit

to be taken into account upon next execution (within ≤ 30min)

• contact QC


Ideal workflow to check HC monitor IHC instrument score:• green/OK all ok, done, next instrument• red/NOK find reports (navigation or ‘score

overview’)check scores and reportscheck comments, ‘Report news’

• understand origin, enter ‘Report news’, contact QC


Ideal workflow to check HC monitor IIuseful info for understanding issues:• plots• ‘Report news’• QC comments from the product

pages• earlier HC plots (history)• tutorial (QC docu)

Don’t hesitate to provide your own comments under ‘edit’!


What to do if …

• red flags on calChecker?- analyze issue, take actions- e.g. science graded ‘C’, no issue: enter analysis, issue gets

flagged ‘OK’ (upon next execution)- MISS or NOK calibration take it, issue turns ‘OK’ upon next

execution

• calChecker outdated?- inform QC

• data transfer/NGAS access not working?- inform DHA

• also check out ‘HELP’


What to do if …

• red scores on HC monitor?- analyze issue- if you think the instrumental issue is gone: take

another calibration; score will automatically turn green if true (within an hour or less)

- provide comment ‘Report news’

• often: page still cached press ‘Ctrl+Shift+R’ to refresh page

• HC monitor outdated (older than 24 hours)?- inform QC

• Find more under ‘HELP’


What to do if …

• red flag for ‘parser’ or ‘ftp’ on HC monitor?- page still cached? press ‘Ctrl+Shift+R’ to

refresh page- if real issue: inform QC

• comment editor used but comment not visible?- page still cached? press ‘Ctrl+Shift+R’ - might take up to 15 minutes until replication to

‘PL’

Documents

The big picture I