JWST Time-Series Pipeline

Nikole K. LewisSTScI

Data Pipeline for Transiting Exoplanets

• The foundation for the Spitzer and Hubble data pipelines were put in place well before the discovery of the first transiting exoplanet in 2000. – Most transiting exoplanet observers start with raw or

basic calibrated data and use homegrown code.– Spitzer has dedicated a fair amount of effort in recent

years to supporting time-series observations.

JWST must provide a data pipeline and data products that support the needs of time-series observations!

Time-Series Pipeline• Ground Rules:– Don’t apply any corrections that can’t easily be

undone such that sensitivity of the solution to the correction can be tested.

– Flag, but do not correct, questionable data. – Absolute calibration is not required.– Model-based corrections should be viewed with

caution.– Because of long temporal baseline, much ‘self-

calibration’ can be performed.– Data products should not represent averages of

individual frames/ramps or a sum over the entire exposure.

‘Vanilla’ Time-Series NIR Spectroscopic Pipeline

Raw IntegrationsLevel 1B

Ramps to SlopesLevel 2A

Photometric/Spectral ExtractionLevel 2B

Level 3 ‘Drizzle’ Products

TSO Pipeline: Level 1B

Raw Integrations

Pipeline Steps Data Products

Raw Integrations

Level 2A

TSO Pipeline: Level 2AMask Bad Pixels

Saturation Check

Subtract Super Bias

Reference Pixel Correction

Apply linearity correction

Subtract Dark

Jump Detection/Fit the ramp

Calibrated Ramps (ADU)

DQ 2D array

Calibrated Ints. (ADU/s)

2D Pixel Saturation Limit

2D Super Bias

Mean Ref Pix Val

3D Coefficients

3D Dark Frame

Algorithms

TSO Pipeline: The IPC Correction

• What is the IPC Correction?• The IPC correction violates the following

ground rules for the TSO pipeline:– Don’t apply any corrections that can’t easily be

undone such that sensitivity of the solution to the correction can be tested.

• TSO pipeline requires non-IPC corrected reference files

TSO Pipeline: Saturation

• Ground Rule: Flag, but do not correct, questionable data.

• Many transiting exoplanet observers will push on saturation limits to access brightest host stars and achieve the highest SNR.

• Need to advise on potential corrections beyond nominal half-well limit.

Saturation Check DQ 2D array2D Pixel Saturation Limit

TSO Pipeline: Bias Subtraction

• Subtractive steps are important in setting the appropriate relative flux level.

• Non-IPC corrected bias frames should be used (~1% percent difference, Gaussian noise)

Subtract Super Bias2D Super Bias

• Many transiting exoplanet observations will be taken with small subarrays that do not incorporate many/any reference pixels.

• In small sample case, simple mean of the reference pixels should be removed.– More complicated schemes are difficult to

reverse/track effect of during analysis.

TSO Pipeline: Reference Pixel Correction/ Bias Drift

Reference Pixel CorrectionMean Ref Pix Val

• A necessary evil.• Needed to avoid under/overestimating the

magnitude of relative features in time-series data.

• User will need to consider propagating uncertainties in correction into final results.

TSO Pipeline: Non-Linearity Correction

Apply linearity correction3D Coefficients

TSO Pipeline: Dark Subtraction

• Subtractive steps important to set relative flux level.

• Non-IPC corrected dark frames should be used.– Correlated noise in difference between IPC and non-

IPC darks (~5% effect)• Calibrated ramps are an extremely useful data

product that many will use as analysis starting point (basic calibrated data).

Subtract Dark Calibrated Ramps (ADU)

3D Dark Frame

• Observations of bright host stars will necessitate the use of a small number of groups.– Limited jump detection with nominal methods• Could use deviations in PSF shape or temporal stack

– Small sample for ramp fitting, instead use simple ‘last minus first’ methodology to set ADU/s

TSO Pipeline: Sample up the Ramp

Jump Detection/Fit the ramp Calibrated Ints. (ADU/s)

Algorithms

TSO Pipeline: Level 2BCalibrated Ints.

(ADU/s)

Background/sky subtraction

Wavelength Calibration

Sensitivity/Flat Correction

Spectral Extraction

Sigma Clip + Hist

Algorithms

2/3D Flat ‘Frame’

Algorithms

Sky/Background Value (ADU/s)

2D wavelength map per Int.

Calibrated Ints (ADU/s)

Spectral Orders per Integration

TSO Pipeline: Background Subtraction

• Remember subtractive steps are important.• Serves as a catch-all (sky, remaining bias/dark

offsets, etc.)• Current methodology uses single value, may

need to evolve to deal with ‘striping’.

Background/sky subtractionSigma Clip + Hist Sky/Background Value (ADU/s)

• Optimal wavelength calibration procedures will vary based on instrument.

• Stellar spectrum may provide additional information to the user. – WFC3 observations of transiting exoplanets were

used to refine the wavelength solution (e.g. Wilkins et al. (2011)).

TSO Pipeline: Wavelength Calibration

Wavelength CalibrationAlgorithms 2D wavelength map per Int.

• Possibly an unnecessary step since division steps will not affect relative measurements.

• Potential drifts in stellar centroid/trace will necessitate this correction.

• Wavelength dependence is important!

TSO Pipeline: Sensitivity/Flat Correction

Sensitivity/Flat Correction2/3D Flat ‘Frame’ Calibrated Ints (ADU/s)

• Spectral extraction methodologies/algorithms will be instrument specific.

• Simple extraction methods that provide easy traceability/replication are best for pipeline.

• Uncertainties will need to be flagged.

TSO Pipeline: Spectral Extraction

Spectral ExtractionAlgorithms Spectral Orders per Integration

TSO Pipeline: Quick-look Data Products

• Stacking data into a single product for the entire exposure is of little utility (no drizzle).

• Simply summing across all wavelengths and presenting relative ADU/s as a function of time is far more useful.

White Light Curve (function of time)Spectral Orders per Integration

Time tagging

• What sets limits on accuracy of time tags?– 10 microsecond precision on frame time– Onboard time tags (UTC) are generated every 10.74

s, with 64 ms accuracy– Corrections to onboard clock can be applied on 12hr

contact intervals. Linear corrections applied on 1.024 s intervals ( with 0.5 ms accuracy). Onboard clock cannot drift by more than 1 s from true UTC time in 24 hour period (requirement and likely worse case scenario)

TSO Time Tagging and Fits File Formatting Module

Time tagging

• Bottom line: 64 ms onboard clock sampling rate is limiter to time-tagging accuracy.

• Time-tags will be associated with each ‘group’ in data products and available in several flavors (BJD_TT, HJD).

TSO Time Tagging and Fits File Formatting Module

Fits File Formatting

• Current plan is to deliver one fits file per exposure w/time-tags as an extension (ala Kepler).

• For 2 day long exposure each fits file would be 35 GB and level 1-3 data products would be on the order of 180 GB.

• For a more nominal transit observation each fits file would be on the order of a few GB.

TSO Time Tagging and Fits File Formatting Module

Key Take Away Points

• Efforts are currently underway to construct a ‘Vanilla’ TSO pipeline that will generate data products of the most utility for time-series observations.

• Current plan is for the Pipeline (python code) to be available for download to allow reprocessing of data.

• Additional tools/resources will be available for necessary ‘non-pipeline’ processing.