IRWG/NORS Error Analysis Workshop, NCAR, January 2013

1

IRWG / NORS Error Analysis Workshop

Welcome

Jan/28/2013

IRWG/NORS Error Analysis Workshop, NCAR, January 2013

2

a. invite 6-10 IRWG members, quite flexible here, but targeting an attendee from each group or region so that it is small enough to function as a hands on workshop, invitees would be contributing to the final products,

b. build a core critical group who are very familiar with the standard processing methods and tools thus helping to create the homogeneous product we've promised to deliver,

c. complete and then implement the tools back at their home institute for testing, verification and operational use.

d. the codes and documentation would be available to all IRWG/NORS members at the IRWG web site.

We believe that if we as a group use largely the same codes, development / maintenance effort is reduced. Also if we pool development resources from many groups core expertise will consequently be distributed widely amongst the IRWG, ultimately yielding more homogeneous data.

We've discussed this with several others in the group and we propose a workshop to focus on and finalize the tools to thoroughly perform the task of moving from retrievals through the error analysis to the GEOMS hdf files. Specifically we would:

Why are we here?

Jan/28/2013

IRWG/NORS Error Analysis Workshop, NCAR, January 2013

3Jan/28/2013

a. a core group who are very familiar with a standard processing method, issues and tools,

b. a deeper understanding of error issues and limitations

c. further implementation of a general set of tools and documentation required to produce reasonable error estimates and create the IRWG HDF data that would be available to all IRWG/NORS members at the IRWG web site.

d. agreement on and a path toward ‘completing’ these tools

e. plans to implement these methods back at their home institute for testing, verification and operational use.

at the end of the meeting?

IRWG/NORS Error Analysis Workshop, NCAR, January 2013

4

Meeting Outline

1. Review:• Components of the IRWG standard error budget

• Retrieval to HDF / batch processing / idlhdfcr8.pro

• Error calculation programs

• Sfit4 features for error calculations

2. Implement (in existing state): • sfit4 ver. B3.999 + testcases

• Batch codes including spectra preparation codes

• error analysis codes

3. Define immediate next steps• For sfit4, batching, error codes

Jan/28/2013

IRWG/NORS Error Analysis Workshop, NCAR, January 2013

5

1. Components of the IRWG standard error budget

• Random:– Modulation amplitude– Modulation phase– Measurement noise– Temperature profile– Solar pointing– Interfering species– Fitted parameters (slope, shift…)

• Systematic:– Retrieved gas absorption line(s) intensity– Retrieved gas absorption line(s) air broadened line half-width

Its critical for homogeneity of the IRWG wide archive that there is agreement on the components and how they are calculated.

Jan/28/2013

IRWG/NORS Error Analysis Workshop, NCAR, January 2013

6

During this meeting we will:

• Create a broad list of possible error sources

• Categorize them into:– those that we can reliably calculate

– those that contribute negligibly

– those that yet need to be included

• Define how to calculate those we include so each group does them equivalently.

• Attempt to better define Systematic & Random – is this the best way to describe errors?

• Are some components in both?

1.1 Components of the IRWG standard error budget

Jan/28/2013

IRWG/NORS Error Analysis Workshop, NCAR, January 2013

7

2. Retrieval to HDF / batch processing / idlhdfcr8.pro

• Review existing dataflows– Gather retrieval outputs

– Perform error calculations

– Produce idlhdfcr8.pro inputs or similar

• Define a dataflow any group can use

• Is idlhdfcr8.pro adequate?

• Attempt to implement skeleton batching

While not critical to the IRWG archive in practice it would be efficient for a default standard batch dataflow.

During this meeting we will:

Jan/28/2013

IRWG/NORS Error Analysis Workshop, NCAR, January 2013

8

3. Error calculation programs

• Review existing procedures implemented by groups• Define features of an ‘ideal’ method• Some mix of EEO and ST (see Hase ’05)?• Maintain provision for expanding to new error

calculations given (especially) item 1 above

Groups have implemented different codes to calculate the error component for the subset of defined errors that they do. Given items 2 above & 4 below how will this be

streamlined?

During this meeting we will:

Jan/28/2013

4. Sfit4 features for error calculations

• Discuss built in options for calculation of several Kb

• Discuss new outputs specifically for the hdf input

• Review new general features of sfit4 (inputs, outputs etc.)

• Review error & hdf specific features of sfit4

• Aid attendees in implementing sfit4

During this meeting we will:

Some hooks for perturbations have been employed in sfit2In conjunction with item 3 above what is the efficient use of built in Kb calculations

vs repeated calls.

Jan/28/2013 IRWG/NORS Error Analysis Workshop, NCAR, January 2013 9

IRWG/NORS Error Analysis Workshop, NCAR, January 2013

10

Error in state due to uncertainty in parameter

Jan/28/2013

Dy = (Sa–1 + Kx

TS–1Kx)–1Kx

TS–1

Dy - Contribution function

Kb - Sensitivity of forward model to parameter b

Sb – Uncertainty* in parameter b

Spar – Uncertainty in due to uncertainty in parameter bX̂�

Spar= GyKbSbKbTGy

T

*uncertainty in Sb must be defined for all retrievals

From forward model runs or from retrieval

IRWG/NORS Error Analysis Workshop, NCAR, January 2013

11

Possible Components Already in K

• BckGrdSlp - background slope• BckGrdCur - background curvature• DWNumShft - differential wavenumber shift• SWNumShft - single wavenumber shift• IWNumShft - independent wavenumber shift• EmpApdFcn - empirical apodization function• EmpPhsFnc - empirical phase function• SolLnShft - solar line shift• SPhsErr - simple phase error• ZeroLev - zero level offset• Interfering species - n other fit species

Jan/28/2013

These may be fit so will have derivatives defined in the final K output file. Given that and estimates of their uncertainty (sensitivity tests?) the effect on the

retrieval may be calculated.

IRWG/NORS Error Analysis Workshop, NCAR, January 2013

12Jan/28/2013

Components Requiring new Kb

Sensitivity to these parameters require calculating a new Kb separate from the retrieval. Likely for each retrieval though.

• When not fitted:– Modulation amplitude– Modulation phase– Measurement noise– Temperature profile– Interfering species– Fitted parameters (slope, shift…)

• Surely not fitted:– Retrieved gas absorption line(s) intensity– Retrieved gas absorption line(s) air broadened line half-width– Solar pointing– ?