Principles of Metrology and their applicability to Earth

FIDUCEO has received funding from the

European Union’s Horizon 2020 Programme

for Research and Innovation, under Grant

Agreement no. 638822

Principles of Metrology and their applicability to

Earth Observation

Nigel Fox

Magna Carta - 1215

One of the oldest documents formalising measurement in the UK

“There is to be one measure of wine and ale

and corn within the realm, namely the

London quarter, and one breadth of cloth,

and it is to be the same with weights.” ‘measurements’ of the Earth if they are to be trusted,

meaningful and interoperable should be treated in the

same way traceable to international agreed standards

Documented methods, estimated uncertainties,

supporting evidence

For EO and Climate ECVs needs some translation &

adaptation of standards and methods:

Organisation of World

Metrology

The Convention of the Metre

(Convention du Mètre)

International System of Units (SI)

(Système International d'Unités)

Mutual Recognition Arrangement

(CIPM-MRA)

1875

1960

1999

• Identical worldwide

• Century-long stability

• Absolute accuracy

• How do we make sure a wing

built in one country fits a

fuselage built in another?

• How do we make sure the SI

units are stable over

centuries?

• How do we improve SI over

time without losing

interoperability and stability?

Three principles

Traceability

Uncertainty Analysis

Comparison

SI

Primary standard

Secondary standard

Laboratory calibration

Industrial / field measurement

Traceability

Unit definition

At BIPM and NMIs

Users

Incre

asin

g u

ncert

ain

ty

Traceability:

An unbroken chain

SI

Documented

procedures

Rigorous

uncertainty

analysis

Audits

Transfer

standards

Rigorous Uncertainty

Analysis

The Guide to the expression of Uncertainty in Measurement (GUM)

• The foremost authority and guide to the expression and calculation of uncertainty in measurement science

• Written by the BIPM, ISO, etc.

• Covers a wide number of applications

• Also a set of supplements

http://www.bipm.org/en/publications/guides/gum.html

http://www.bipm.org/en/publications/guides/gum.html

Accredited Calibration

Laboratories

auditing procedures

transfer

standards

calibration

INDUSTRY

EURAMET

Regional

comparisons

CONVENTION OF THE METRE

Key comparison of primary unit

National Metrology Institutes SIM APMP

Mutual Recognition

Arrangement

Lab-to-lab

(results of a scientific

comparison)

Protocol

Measurements

Pre-Draft A

Draft A

Draft B

Final report

• Written by working group

• Approved by participants

• Reviewed then approved by CCPR

• Results sent to pilot only

• Often star-form

• Relative results (intra-lab consistency)

• Review of uncertainty statements

• No results shown

• Discussion on dealing with outliers (blind)

• First time participants see results

• Review by participants

• Review by experts

• Approval by CCPR

• Published

Used to:

• Validate CMCs

• Inform customers

MRA Formal comparison

Luminous Intensity key

comparison

Ongoing research:

Outliers

-2.0%

-1.5%

-1.0%

-0.5%

0.0%

0.5%

1.0%

1.5%

2.0%

0 2 4 6 8 10

Participants

De

via

tio

n f

rom

KC

RV

(all)

KCRV(subset) - KCRV(all) X_i - KCRV(all) X_i(excluded) - KCRV(all)

c2obs 20.5

c20.05(n) 15.507

Error bars: expanded unc.(k = 2)

If you exclude 5 or 7 the others

(including 7 or 5) become consistent

CCPR Guidelines on

comparisons

CMC Database

https://kcdb.bipm.org/

Evidence: Formal peer review or audit of procedures,

participation in a relevant key comparison (within 10

years) with declared uncertainties defended, review

within region and between regions

https://kcdb.bipm.org/

• Identical worldwide

• Century-long stability

• Absolute accuracy

Achieved through:

• Traceability

• Uncertainty Analysis

• Comparison

The traceability

chain is broken

No reference in

space …

No reference in

space … yet

www.npl.co.uk/truths

http://www.npl.co.uk/truths

The Quality Assurance framework for Earth Observation (QA4EO)

Looks to make the GUM accessible to the EO community

QA4EO Principle:

‘All data and derived products shall have associated

with them a fully traceable indicator of their quality’,

documented and quantitatively tied to an

international standard ideally SI

Operational framework: Principles and scope (space example)

All activities which contribute to the

delivery of an end product derived

from an input measurand

Pre-Flight

- Requirement/Design Specification

- Instrument build: characterisation/calibration

- Data processing: algorithms, ref/support data,

Post-Launch

- Instrument performance

- Output data quality characteristics:

- accuracy

- equivalence to others (sensors/in-situ)

- Processing – high level products

- Data distribution/archive …

Collection – Processing – Validation - Delivery

Archive

Reprocessed

+QI

+QI

Fiducial Reference

measurments (FRMs)

23

& have Uc levels fit for the application they are used for

Uc for Validation

measurements MUST

also be evaluated and

compared to assess

consistency with that

derived by sensor

FRM comparisons

24 www.frm4sts.org

FRM4STS lab

comparison

PHASE 1: PREPARATION

Invitation to participate October 2015

Preparation and formal agreement of the protocols Jan - March 2016

PHASE 2: MEASUREMENTS

Comparison of participants’ radiometers and blackbodies June 2016

Field comparison of participants’ radiometers at NPL June/July 2016

Participants send all data and reports to pilot July 2016

PHASE 3: ANALYSIS AND REPORT WRITING

Pre-draft A: Participants send preliminary report describing

their measurement system and uncertainties to the pilot. This

will be circulated to all participants.

April 2016

Receipt of comments from participants May 2016

Draft A (results circulated to participants) July 2016

Final draft report circulated to participants August 2016

Draft B submitted to CEOS WGCV September 2016

Final Report published October 2016

Difference between the mean of the values reported by

participating blackbodies from the values measured by

AMBER (shown in blue) and PTB (shown in red) for a

nominal blackbody temperature of 25 oC.

Traceability

Delivered through common processing chain evaluated for Uc

Site 1

RadCalNet portal

Calibration & QC

& Processing

Raw measurements

Surface reflectance and atmosphere products (RadCalNet specific)

FTP FTP

RadCalNet Processing

& QC

Hyperspectral TOA

reflectance @ 30 mn

interval for nadir view

Site 2

Calibration & QC

& Processing

Raw measurements

Surface reflectance and atmosphere products (RadCalNet specific)

QA site

owner,

NPL

support

on Uc

QA site

owner,

NPL

support

on Uc





Uncertainty Analysis applied to Earth

Observation

Sam Hunt

X1

X2

X3

Y

Error effects Input quantities

Measurement model

Output quantity

Principle of


1 2, , , 0NY f X X X

Measurement function

The Measurement

Equation

Uncertainty analysis





some

uncertainty in

the +0 too!

X1

X2

X3

Y


Locally linearised model

Output quantity

1u X

2,au X

2,bu X

2,cu X

3u X

u Y

Propagation of

Uncertainty 1 2

yn

f f f

x x x

C

1 2, , , 0NY f X X X

GUM: Law of Propagation

of Uncertainties

2 1

2 2c

1 1 1

2 ,

n n n

i i ji i ji i j i

f f fu y u x u x x

x x x

1 2y

n

f f f

x x x

C

21 1 2 1

22 1 2 2

21 2

, ,

, ,

, ,

1 2

1

2

n

E n

n n n

u E u E E u E E

u E E u E u E E

u E E u E E un E

n

U

Algebraic form

Matrix form

For each effect you need

to know

Size of uncertainty

Sensitivity coefficient

Uncertainty probability distribution

Form and scale of error correlation

• Spectrally

• Spatially

• Temporally

Error correlation form --

example 3 thermal channels, common reference blackbody (ICT),

uncertain temperature (same for all channels)

1 1 11 1 11 1 1

Correlation coefficient – channel to channel

Temperature covariance– channel to channel

𝑢 𝑇 0 00 𝑢 𝑇 00 0 𝑢 𝑇

1 1 11 1 11 1 1

𝑢 𝑇 0 00 𝑢 𝑇 00 0 𝑢 𝑇

Error correlation form

3 spectral channels, common reference blackbody (ICT),

uncertain temperature (same for all channels)

Temperature covariance– channel to channel

𝑢 𝑇 0 00 𝑢 𝑇 00 0 𝑢 𝑇

1 1 11 1 11 1 1

𝑢 𝑇 0 00 𝑢 𝑇 00 0 𝑢 𝑇

Earth radiance covariance – channel to channel

E, 1 E, 1

E, 2 E, 2

E, 3 E, 3

0 0 0 0

0 0 1 1 1 0 0

0 0 0 0 1 1 1 0 0 0 0

0 0 1 1 1 0 0

0 0 0 0

T T T T T

L L

T Tu T u T

L Lu T u T

T Tu T u T

L L

T T

C V R V C

Using this…

E, 1 E, 1

E, 2 E, 2

E, 3 E, 3

0 0 0 0

0 0 1 1 1 0 0

0 0 0 0 1 1 1 0 0 0 0

0 0 1 1 1 0 0

0 0 0 0

T T T T T

L L

T Tu T u T

L Lu T u T

T Tu T u T

L L

T T

C V R V C

Error covariance between measured radiance in different

channels due to uncertainty associated with ICT temperature

So, when you have a retrieval combining these three channels,

the error correlation affects the uncertainty via off-diagonals

E, 1 E, 2 E, 1, ,y f L L L

Earth count noise

(independent channel to

channel… )

E, 1 E, 1

E, 2 E, 2

E, 3 E, 3

E, 1 E, 1

E E

E, 2 E, 2

E E

E, 3 E, 3

E E

0 0 0 0

0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0

0 0 0

1 0 0

0 0

0 1

0

1

0

C C

CE CE CE CE CE C C

C C

L L

C Cu u

L Lu u

C Cu u

L L

C C

C V R V C

… everything stays diagonal

Y


Measurement model

Output quantity

Monte Carlo Approach

1 2, , , 0NY f X X X

Both Earth count and

temperature and …

ch

effects,

i i i i i

i

U C V RV C

E, 1 E, 1

E, 2 E, 2

E, 3 E, 3

1 1 1

1 1

0 0 0 0

0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0

0 0 0 0

1

1 1 1

T T T T T

L L

T Tu T u T

L Lu T u T

T Tu T u T

L L

T T

C V R V C

E, 1 E, 1

E, 2 E, 2

E, 3 E, 3

E, 1 E, 1

E E

E, 2 E, 2

E E

E, 3 E, 3

E E

0 0 0 0

0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0

0 0 0

1 0 0

0 0

0 1

0

1

0

C C

CE CE CE CE CE C C

C C

L L

C Cu u

L Lu u

C Cu u

L L

C C

C V R V C

+





FIDUCEO developments for EO radiance

uncertainty

Chris Merchant

EO Radiance


Understand the measurement equation for

radiance

Identify all known sources of error (effects)

Quantify their error correlations and distributions

(uncertainty)

Propagate to get radiance uncertainty

Structured approach centred on measurement

equation

The equation used to calculated “calibrated radiance” in the FCDR

Measurement equation

Gain

TOA Earth Radiance

True signal

GT

Measured Signal

C

E

M = GTRE

T +dCE

Measured gain

GM = GT +dGT

Measured Earth Radiance

Should respect the laws of physics (doesn’t always!) Should reflect the instrument

𝐶𝐸𝑀 = 𝐶𝐸

𝑇 + 𝛿𝐶𝐸

𝐿𝐸𝑇

𝐿𝐸𝑀 =𝐶𝐸𝑀

𝐺𝑀

𝐿 = 𝑓 𝐶𝐸; 𝐶𝐶𝐶𝑇, 𝐶𝑊𝐶𝑇; 𝑎1, 𝑎2, … + 0


𝜕𝐿

𝜕𝐶𝐸


𝜕𝐿

𝜕𝐶𝐸

𝑢(𝐶𝐸) digitisation 𝑢~1

2√3

𝑟~0


𝜕𝐿

𝜕𝐶𝐸

𝑢(𝐶𝐸) digitisation 𝑢~1

2√3

𝑟~0

𝑢(𝐿)~∆𝐿

2√3

𝑟~0


𝜕𝐿

𝜕𝐶𝐸

𝑢(𝐶𝐸) digitisation 𝑢~∆𝐿

2√3 detector &

amplifier noise

NOAA 16 HIRS Ch 5 (2001-2015)

Nois

e /

counts

June 1 June 30

Empirical assessment using

Allan deviation of the counts

timeseries viewing cal. targets

Plots: Gerrit Holl


𝜕𝐿

𝜕𝐶𝐸

𝑢(𝐶𝐸) digitisation 𝑢~∆𝐿

2√3

𝑟~0

detector &

amplifier noise

See poster by Gerrit Holl


𝜕𝐿

𝜕𝐶𝐸

𝑢 𝐶𝐸

𝑟~𝑒𝑚𝑝𝑖𝑟𝑖𝑐𝑎𝑙

digitisation 𝑢~∆𝐿

2√3

𝑟~0

detector &

amplifier noise

See poster by Gerrit Holl

𝜕𝐿

𝜕𝐶𝐶𝐶𝑇


𝜕𝐿

𝜕𝐶𝐸

𝑢 𝐶𝐸


digitisation 𝑢~∆𝐿

2√3

𝑟~0

detector &

amplifier noise

𝜕𝐿


𝐶𝐶𝐶𝑇 = 𝑤𝑠𝐶𝐶𝐶𝑇𝑝,𝑠

𝑠𝑐𝑎𝑛𝑠𝑝𝑖𝑥𝑒𝑙𝑠

𝑢(𝐶𝐶𝐶𝑇𝑝,𝑠)


𝜕𝐶𝐶𝐶𝑇𝑝,𝑠


𝜕𝐿

𝜕𝐶𝐸

𝑢 𝐶𝐸


digitisation detector &

amplifier noise

𝜕𝐿







Form of f.

Interpolation /

extrapolation

assumption.


𝜕𝐿

𝜕𝐶𝐸

𝑢 𝐶𝐸


digitisation detector &

amplifier noise

𝜕𝐿







Form of f.

Interpolation /

extrapolation

assumption.

Real example of measurement

equation analysis diagram

Example of AVHRR

Illustrates

• non-linear measurement

equation

• branching structure

(secondary and tertiary

measurement equations)

• uncertainty in calibration

parameters

(harmonisation

uncertainty)

For each effect you need

to know

Size of uncertainty

Sensitivity coefficient

Uncertainty probability distribution

Form and scale of error correlation

• Spectrally

• Spatially

• Temporally

Challenge of complexity...

Investigating each source

of uncertainty

Error correlation between

measured values (Type B)

Earth view Calibration target view Space view


measured values



measured values



measured values


Error correlation

dimensions

LEO:

Pixel-to-pixel

Scanline-to-scanline

Orbit-to-orbit

Temporal

Spectral

GEO:

Pixel-to-pixel

Scanline-to-scanline

Image-to-image

Temporal

Spectral

Codify the uncertainty analysis

Name of effect Earth Count Noise

Averaged Space Count Noise

Averaged IWCT Count Noise

Affected term in measurement

function

CE CS Ct

Instruments in the series affected All All All

Correlation

type and form

Pixel-to-pixel [pixels] Random* Rectangular Absolute

Rectangular Absolute

from scanline to scanline

[scanlines]

Random* Triangular Triangular

between images

[images]

N/A N/A N/A

Between orbits [orbit] Random Random Random

Over time [time] Random Random Random

Correlation

scale

Pixel-to-pixel [pixels] [0]

from scanline to scanline

[scanlines]

[0] n = 51 n = 51

between images

[images]

N/A N/A N/A

Between orbits [orbit] [0] [0] [0]

Over time [time] [0] [0] [0]

Channels/

bands

List of channels / bands

affected

All All All

Correlation coefficient matrix Identity

matrix (1s

down

diagonal only)*

Identity matrix (1s

down diagonal only)*

Identity matrix (1s

down diagonal only)*

Uncertainty PDF shape

Digitised Gaussian

Digitised Gaussian Digitised Gaussian

units Counts Counts Counts

magnitude Provided per

pixel

Provided per scanline

Provided per scanline

Sensitivity coefficient , Eq 4-1 , Eq 4-2 , Eq 4-3

For each effect (“end twig” of

tree), we formalise the

conclusions of the uncertainty

analysis.

For documentation, the analysis is

documented and summarised in

an effects table.

The set of codified effects table

+ satellite data + harmonisation

results comprise the “full FCDR”.

Error correlation forms

Rectangle Absolute

• Fully systematic or systematic within a calibration

period

Triangle Relative

• Rolling averages

Bell-shaped Relative

• Weighted rolling averages, splines, smoothing,

other

Repeating

• E.g. once per orbit, diurnal or seasonal cycles

Mixed

Table descriptor Comments Example

Name of effect A unique name Internal calibration target count

noise

Affected term in measurement function Name and standard symbol

Instruments in the series affected Identifier All instruments all satellites

Correlation type

and form

Pixel-to-pixel [pixels] One of the types Rectangular absolute from scanline to scanline

[scanlines] Triangular relative

between images

[images] N/A for orbiting satellite

Between orbits [orbit] Random Over time [time] Random

Correlation scale Pixel-to-pixel [pixels] As needed to define type [-∞,∞] (fully correlated across

scan) from scanline to scanline

[scanlines] n = 51 (51 scanlines averaged

in rolling average) between images

[images] N/A for orbiting satellite

Between orbits [orbit] 0 Over time [time] 0

Channels/bands List of channels / bands

affected Channel names All channels

Error correlation coefficient

matrix A matrix Identity matrix (diagonal).

Uncertainty PDF shape

Functional form Gaussian

units Units Counts

magnitude Given once per orbit file

Sensitivity coefficient Value, equation or

parameterisation of sensitivity

of measurand to term

ICTC

E

ICT

L

C

E

ICT

L

C

Why a metrological approach?

Why consider all sources of uncertainty?

See blog article http://www.fiduceo.eu/node/237

If you compare two measurements on different space-time scales the dominant sources of uncertainty in that difference change.

Why a metrological approach?

Why consider all sources of uncertainty?

Example

of SST CDR

Specifying an “accuracy” target here …

... only weakly constrains

the uncertainty here





Applying FIDUCEO thinking to AVHRR

Jon Mittaz

CURRENT STATUS QUO:

Without a FIDUCEO

approach

No pixel level uncertainty estimates with data – just the design specification

AVHRR IR Channels over whole series life (1978-present) have had 4 different operational calibration algorithms

• Linear calibration (even though the 11/12 micron channels are non-linear): TIROS-N to NOAA-8

• Non-linear correction from lookup table: NOAA-9 to NOAA-12

• Non-linear correction from lookup tables including a “negative radiance of space” term: NOAA-14

• Walton et al. (1998) calibration: NOAA-15 to present(may apply to some NOAA-14 as well)

• Coefficients for older AVHRRs are available from Walton et al. for AVHRR/2 and AVHRR/3 instruments (not for AVHRR/1 – TIROS-N, NOAA-06, NOAA-8, NOAA-10)

Difficult to make a consistent time series from the data

Many different source of error are still present in the operational data

If a metrological approach had been taken from the beginning these

problems would have been significantly reduced

AVHRR problems start

with pre-launch

Not done well… Very simple test environment

• Goal to meet design specifications only

• AVHRR/1 and AVHRR/2 IR channels good to 1K only

• AVHRR/3 IR channels good to 0.5K

Calibration Targets -

ECT (180->320K) &

Space Target @ 70K

Run at 5 instrument temperatures

of 10, 15, 20, 25, 30°C (15,20,25°C

for early sensors + NOAA-16)

Not exactly a high tech setup…

AVHRR operational

calibration (Walton et al.)

Derived on the basis of the pre-launch data

For 11&12 µm channels (non-linear) a linear estimate of the radiance id first derived and

the radiance is then correct for the non-linearity

2

210 LinLinEarthS

BBS

SBBSLin NaNaaRadianceCC

CC

NNNN

NS is the ‘Negative Radiance of Space’ and a0, a1 and a2 are coefficients derived from pre-launch

calibration data

It looks like it

works well

(standard

deviation from

pre-launch ~

0.08K 11µm

channel)

BUT Taken from Walton et al. (1998)

Applying the operational calibration to pre-launch data (AVHRR/3,

Walton et al. 1998)

(which it was derived from in the first place…)

You’d hope you’d get the

‘right’ answer

• Actually see biases of

up to 1K…

Problems with pre-launch

data

Non-physical aspects of

Walton et al.

• Just at face value there are a number of problems

with the calibration equations

• Uses a “Negative Radiance of Space” whatever that

is…

• Total equation has variable non-linear term determined

from the way algorithm works rather than being based

on any physical understanding of the detector system

itself

2

2

2

21

2

210

)(

)()21())1((

EarthS

BBS

SBB

EarthS

BBS

SBBSSS

Earth

CCCC

NRa

CCCC

NRNaaNaNaa

R

THE FIDUCEO method of trying to understand the instrument from using a

physics based approach would not had had these issues

AVHRR: Problems with pre-

launch measurements

1. Instrument temperature drifted by ~ 1K but assumed

constant in deriving calibration

2. The test chamber was not temperature controlled

ICT

Calibration model including extra

component

Calibration model including extra

component

0.1K

0.5K

0.3K

Mismatch between measured ICT

temperature (PRT) and inferred ICT

temperature (ECT Calibration) (PRT

coefficient correction proposed in

1991 which would have broken PRT

traceability)

AVHRR

Test

Chamber

Taking a metrological approach from the start would have spotted these

problems

AVHRR Operational

Algorithm: Impacts on in-

orbit calibration

AVHRR pre-launch data had a lot of problems which

were not noted when developing operational

calibrations including the current one

No account was taken of the possible change in

calibration between the pre-launch testing and the in-

orbit environment

• Pre-launch data was run so that the AVHRR was

always thermally stable

• This is not the case when in orbit…

Along with pre-launch a detailed analysis of in-orbit

data should be undertaken beyond checking the

design specifications

Things can (do) change when you launch an instrument into space and

need to check for such changes

• AVHRR comparisons against

IASI show strong trends in

operational calibration

• Refitting calibration gives

significant improvements

• Demonstrates that you must

monitor the instrument in-

orbit for changes from pre-

launch data

• Thermal environment will

be different

AVHRR: Problems with in-

orbit data

This bias is still present in the operational AVHRR Level 1B

AVHRR: FIDUCEO, a

metrological approach

At Level 1 we start with the “Traceability Tree”

• Starts with the measurement equation

• Looks at each term and breaks it down into however many underlying processes are needed to get back to root process

• Links lowest level processes to their impact and associated uncertainty on the observed Earth radiance

Correlated error terms are also considered

Note that the process of obtaining metrologically traceable uncertainties also means removing all systematic error sources as far as possible

• Guide to the expression of Uncertainty in Measurement (GUM) (2008) Section 3.2.4

• It is assumed that the result of a measurement has been corrected for all recognized significant systematic effects and that every effort has been made to identify such effects.

OTfCaCC

CaRaaR InstrEE

T

TTE

)(2

2

2

210

AVHRR: FIDUCEO, a

metrological approach

AVHRR: Noise Effects

Detector noise case has been shown earlier but note

that currently many people are using a constant

NeDT=0.12K

• This value is the design specification and has

very little to do with the true behaviour of the

instrument

NOAA-07

FIDUCEO has forced an investigation of all effects and highlights time

variable effects not highlighted in the operational calibration.

AVHRR: Solar

contamination

Solar contamination of the Internal Calibration Target

(ICT) is a big problem

• Direct solar radiance contaminates the calibration

system

• Impact of solar radiation puts complex gradients

across ICT impacting its accuracy

There is an operational correction available (from

1995 onward) which tries to correct the direct solar

radiance part only

• Simple detection algorithm only, not based on

physical processes on-board

Improved solar

contamination modelling

Operational detection

(can still fail) Improved Modelling Operational modelling No operational detection

pre late 1994

Improvements from FIDUCEO due to using a physics

based model of how the instrument behaves

FIDUCEO

Operational

Orbit Drift effects

As the satellite orbit drifts there is a change in its

thermal state

• Not considered in current operational calibration

Due to changes in stray light components as the

thermal structure of the instrument changes over time

impacting the calibration

• Leads to time dependent biases which will

significantly impact geophysical retrievals such as

SST

An SST example

From Pathfinder SST

(v6) which kept SST

retrieval coefficients

constant. Operational

calibration used.

Note strong bias as

a function of time

Related to instrument

temperature (a proxy

for thermal

environment)

Can be modelled

(and is in FIDUCEO)

FIDUCEO and the AVHRR

By applying FIDUCEO principles to the AVHRR we

have gained

• Justifiable and defensible uncertainties at the

pixel level and beyond including error correlations

• An understanding of the instrument and all

associated effects which include

• Significantly reduced scene temperature biases

• Identified and significantly reduced orbit drift effects

• Significantly reduced impact of solar contamination

• …

Leads to improved AVHRR data as well as

traceable multi-component uncertainties





Building new uncertainty concepts into current

and future missions

Chris Merchant

Uncertainty for CDRs

Provide uncertainty estimates

Follow metrological conventions

Give u per datum if necessary

Uncertain ≠ Bad quality

Explain the uncertainty info

Give advice to users on usage

Validate the uncertainties

Error correlation matters

DOI 10.5194/essd-9-511-2017

Use of radiance

uncertainties

For model-observation comparisons in “observation space”

For data assimilation, helping to build first-principles error

covariance estimates to confront/improve estimates

inferred within DA system

For proper estimation of Climate Data Record

uncertainties across spatio-temporal scales

• FIDUCEO exemplars – coming next year

Raw satellite data (L0)

Calibrated radiances (L1)

Climate data record (L2)

Gridded CDR (L3)

Analysed / processed (L4+)

Climate index / information

• Decision

• Insurance

• Liability

Evolve good practice towards … Good practice Apply to

Level 1 radiance provided with uncertainty

estimates per datum.

Key heritage sensor series.

Planned missions.

Multi-mission series should be harmonised. Key heritage sensor series.

Planned missions.

Propagate radiance uncertainties to inform

level 2 (swath) and 3 (gridded) geophysical

data.

Climate data records (CDRs)

and environmental data

records.

Propagate CDR uncertainty to higher-levels. Climate information derived

(in part) from CDRs

Decision makers and other users access and

trust information on uncertainty.

Presentation of climate

information in climate

services.

Not just for heritage

sensors

We think the principles and techniques we are learning

on the historical sensors have much wider applicability

Particularly, they can be embedded into space agency

practice for adding value by including per-datum

uncertainty in L1

• reprocessing of archive mission data

• specification of instruments and products for

future missions

An uncertainty/traceability focus in

Phase B-D Aspect Compliance focus Metrology focus Estimating the

magnitude of

pixel-level

uncertainty (e.g.,

in radiance)

Worst-case combination of

uncertainty from error sources

to compared against a

(generally) aggregated total

uncertainty requirement.

Deliberately pessimistic to

ensure compliance and

acceptance.

Individual

models/calculations of

uncertainty from error

sources, traceably

documented per error

source. Realistic combination

to inform expected in-flight

characteristics. Characterising

the error-

correlation

structure across

pixels and

channels

Only in response to specific

relevant requirements (e.g.

cross-talk limits). Not

considered for many error

sources.

Integral part of uncertainty

characterisation for all error

sources

An uncertainty/traceability focus in

Phase B-D Aspect Compliance focus Metrology focus Traceably documenting uncertainty information

Documentation focused on acceptance milestones. Results perhaps mixed with commercially sensitive and confidential material, usually not available in a form supporting traceability

Documentation freely available and organised such as to support systematic traceability

Dissemination of understanding of error sources to users

Not actively or systematically attempted -- generic information may be published. Not quantitatively integrated into satellite products

Understanding is embedded in product processing chain in order to include quantitative uncertainty information directly in satellite products at L1

http://www.fiduceo.eu/blogs

Beyond FIDUCEO – link to “Green paper”

Satellite missions: metrological upgrade

Harmonisation and Recalibration

Why worry about all sources of errors?

http://www.fiduceo.eu/blogs

The National Physical Laboratory

is operated by NPL Management

Ltd, a wholly-owned company of

the Department for Business,

Energy and Industrial Strategy

(BEIS).

Thank you

MetEOC and MetEOC-2 were funded by EMRP MetEOC-3 is funded under EMPIR

FIDUCEO has received funding from the European

Union’s Horizon 2020 Programme for Research and

Innovation, under Grant Agreement no. 638822

Discussion 1

Each table please spend 10 mins on each Q. Identify

a rapporteur/notetaker to record main points and feed

back in plenary.

Q1 What degree of need for improved,

transparent uncertainty information is recognised

amongst users/product/service developers?

Q2 What are benefits and challenges to applying

EO-metrology principles to L1 and L2?

Q3 Is the current approach to instrument

uncertainty characterisation and pre-flight cal/val

adequate (from point of view of ultimate users of

L1 and derived data)? If no, what problems are

caused?

Discussion 2

Full set of discussion questions will be: Q1 What degree of need for improved, transparent uncertainty information

is recognised amongst users/product/service developers?

Q2 What are benefits and challenges to applying EO-metrology principles

to L1 and L2?

Q3 Is the current approach to instrument uncertainty characterisation and

pre-flight cal/val adequate (from point of view of ultimate users of L1 and

derived data)? If no, what problems are caused?

Q4 - What should next case studies be for L1?

Q5 - What priority case studies should we address next for L1 to L2+ ?

Q6 What additional information from instrument dev and pre-flight cal

should be made available to users and how?

Q7 How could we build the core principles of providing uncertainty into

the development of phase of new missions?

Q8 Are there additional steps that can be built into in-flight operational

missions to validate and test performance?

Q9 What activities/strategies do we need to consider to validate Uc of L1

and L2 products and ensure their interoperability?

Q10 Is targeted training on Uc analysis needed, and how to develop this?

Discussion 2:

Q4 - What should next case studies be for L1?

Q5 - What priority case studies should we

address next for L1 to L2+ ?

Discussion 3

Q6 What additional information from instrument

dev and pre-flight cal should be made available to

users and how?

Q7 How could we build the core principles of

providing uncertainty into the development of

phase of new missions?

Q8 Are there additional steps that can be built

into in-flight operational missions to validate and

test performance?

Discussion 4

Q9 What activities/strategies do we need to

consider to validate Uc of L1 and L2 products

and ensure their interoperability?

Q10 Is targeted training on Uc analysis needed,

and how to develop this?

Documents

Principles of Metrology and their applicability to Earth