THE GUIDE TO THE EXPRESSION OF UNCERTAINTY IN MEASUREMENT ... · The Guide to the Expression of Uncertainty in Measurement (GUM) IMS 2004 Workshop: Statistical Methods and Analysis

The Guide to the Expression of Uncertainty in Measurement (GUM)IMS 2004 Workshop: Statistical Methods and Analysis for Microwave Measurements

1

THE GUIDE TO THE EXPRESSION OF

UNCERTAINTY IN MEASUREMENT (GUM)

Uwe ArzBernd R.L.Siebert

Physikalisch-Technische BundesanstaltBundesallee 100, D-38116 Braunschweig, Germany

IMS 2004 WorkshopStatistical Methods and Analysis for Microwave Measurements

Fort Worth (TX), June 7th 2004


2

Table of Content

Introduction Decisions based on incomplete knowledgeDefinition of uncertainty - Why should we use the GUM?

Basic ConceptsProbability distribution function (PDF) for a quantity.

Model to link input quantities to output quantity (measurand).

ExampleMeasurement of a DC current

Basic ProceduresPropagation of probability distribution functions.Propagation of uncertainties - Step-wise procedure - Budget

Summary


3

Introduction (1/6) Decisions based on incomplete knowledge

We want to hike for a few days in the mountains

We makea plan

We check for huts or at least shelters

The properunit for distance is hours not kilometres

We check the

weatherforecast

In the end wedecide based on

incompleteknowledge

as good as we canor think we need to.

We behave like true Bayesians

:


4

Introduction (2/6) Definition of Uncertainty (1/3)

VIM 3.9 (taken from GUM): uncertainty of measurementparameter, associated with the result of a measurement,that characterises the dispersion of the values that could reasonably be attributed to the measurand

Note 1The parameter may be, for example, a standard deviation (or multiple of it),or the half-width of an interval having a stated level of confidence

Comments:

A standard deviation characterises a probability distribution(square root of the variance).

The half-width statement provides a probability statement.


5



Note 2Uncertainty of measurement comprise, in general, many components.Some of these components may be evaluated from the statisticaldistribution of the results of a series of measurements and can becharacterised by experimental standard deviations.The other components, which can also be characterised by standarddeviations, are evaluated from assumed probability distributions, based on experience or other information.

Comment:

The use of experimental standard distributions is called TYPE Aand the use of “assumed” distributions TYPE B evaluation.


6



Note 3It is understood that the result of the measurement is thebest estimate of the value of the measurand,and that all components of uncertainty, including those arising fromsystematic effects, such as components associated with correctionsand reference standards, contribute to the dispersion.

Comment:

This note will be important, as it puts requirements on the so-calledmodel for the evaluation of uncertainty.


7

Organisation Internationalede Metrologie Legale

Bureau Internationaldes Poids et Mesures IUPAC

International Union of Pureand Applied Chemistry

The worlds leading organisations support it:

Introduction (5/6) Why should one use the GUM ? (1/2)

international laboratory

accreditation cooperation

GUM: Guide to the expression of uncertainty in Measurement


8

Introduction (6/6) Why should one use the GUM ? (2/2)

GUM: Guide to the expression of uncertainty in Measurement

GUM supports:

• Traceability to SI units(Système International d’unités).

GUM is: based on sound theory and,in principle, easy to understand.

• a fully consistent and transferableevaluation of measurement uncertainty and

GUM features a simple transparent procedure that provides solutions for most problems encountered in practice.


9

Basic concepts (1/11) Probability distribution function (1/6)

Nomenclature:Y output quantity Xi input quantity y = EY (best estimate) xi = EXi (best estimate)u2(y) = VAR(Y) (uncertainty) u2(xi) = VAR(X) (uncertainty)

η = possible value of Y ξi = possible value of Xi

The expectation value x of that PDF isthe best estimate for the value of thequantity:

( ) ⋅⋅== ξξξ dE XgXx

( ) ( ) ( )( ) −== ξξξ dVAR 22 xgXxu X

Knowledge about values that one can reasonably attribute toa quantity X is expressed by a probability distribution function.

The standard deviation of that PDFis the uncertainty associatedwith that value:


10


u(y)u(y)

η1 values of measurand

probability densityfor measurand Y

η2

u(y)u(y)

y

prob. for the value to bebetweenη1 and η2

(Expanded uncertainty)(Expanded uncertainty)

(uncertainty)(uncertainty)

y is best estimate(result of measurement)

u(y)=(VAR(Y))1/2u(y)=(VAR(Y))1/2

U=2u(y)U=2u(y)

U=2u(y)


11


How does one obtain the PDF for an input quantity?

Remember Note 2 (second paragraph)The other components (of uncertainty) , which can also be characterised by standard deviations, are evaluated from assumed probabilitydistributions are based on experience or other information.

TYPE B

The qualifier “assumed” is misleading.

There is a sound theoretical basis for inferring the PDF fora quantity based on the information given.

The Principle of Maximal (Information) Entropy PM(I)E andBayes’ theorem form this basis.


12


TYPE B -continued : theoretical basis (1)-

(((( )))) (((( ))))i

n

iin plnpp,...,p,pH

====−−−−====

121

The principle of maximum information entropy yields:a rectangular (uniform) PDF if one knows:the values ξ of the quantity X are contained in an interval

a Gaussian (normal) PDF if one knowsthe best estimate x and u(x)

formulated by Shannon ensuresthe most probable PDF that takes only account of the information given

The principle of maximum information entropy (here for discrete probabilities):

A pioneer in using the PME was the physicistE. T. Jaynes (1922 - 1998)

Principle of MaximumInformation Entropy


13


TYPE B -continued : theoretical basis (2)-

Thomas Bayes, 1702 - 1761, English reverend, found thistheorem. It was used by Laplace. Later statisticians did not accept this theorem. The theorem was rediscovered by Jeffreys (1938) and is since gaining increasingly acceptance asthe theoretical basis for inference.

(((( )))) (((( )))) (((( )))) ξξϕξξξϕ d d II,DlCI,D XX ====

In modern nomenclature:

The posterior PDF ϕX(ξ |D,I ) taking account of new data D

results from prior PDF ϕX(ξ |I ) taking account of prior information I

as product of a constant C, the Likelihood l (ξ |D,I ) and the prior PDF.

C follows from the normalisation of the posterior PDF

l is the product of the prior PDF at the new data D

Bayes’ Theorem


14


The most often encountered PDFs are the Gaussian (normal) or the rectangular (uniform).TYPE B -continued-

The GUM also describes:a triangulara trapezoidal anda U-shaped PDF

?

TYPE B analysis is sometimes cumbersome, but in the end it leads to a better understandingof the experiment and often helps to optimise it.

Knowledge given PDF Best estimate Uncertainty Mean value µ and variance σ2 Gaussian x = µ u(x) = σ Limits a− and a+ Rectangular x = ½ (a− + a+) u(x) =

12 1 (a+ − a−)


15


Remember Note 2 (first paragraph) : Uncertainty of measurementcomprise, in general, many components. Some of these components maybe evaluated from the statistical distribution of the results of a series ofmeasurements ....

TYPE AHow does one obtain the PDF for an input quantity?

Call the repeatedly measured input quantity Xi quantity Q. With n statistically independent observations (n > 1), theestimate of the value of Q is the arithmetic mean or theaverage of the individual observed n values qj .

====

====n

jjq

nq

1

1

Repeatability conditions!

An estimate of the variance ofthe underlying PDF is theexperimental variance s²(q)

2

1

2

11

)qq(n

)q(sn

jj −−−−

−−−−====

====

The best estimate of the variance of thearithmetic mean, is theexperimental variance of the mean

n

)q(s)q(s

22 ====


16


TYPE A -continued-n

)q(s)q(s

22 ====

experimental varianceof the mean

Clearly, few values cannot represent the “underlying PDF”.

n -1 is termeddegree offreedom ν

( )2

12

, 12

1 +−

+

)2Γ(π

+Γ=

ν

ν νννt

tfT )1()1()(

1)1( ;)21(

−Γ−=Γ=Γ=Γ

xxx

π

This fact is described theoretically by the so-called t-distribution

We aim usually at a coverage probabilityof 0,95; the coverage interval is thengiven by

−+ vv tn

sxt

n

sx ,025,0,025,0 ,

96,1 ........ 78,2 18,3 ,025,04,025,03,025,0 −=−=−= ∞=== vvv tttSome values:


17


TYPE A -continued-n

)q(s)q(s

22 ====

experimental varianceof the mean

What can be learned from repeated experiments?

We aim usually at a coverage probabilityof 0,95; the coverage interval is thengiven by

−+ vv tn

sxt

n

sx ,025,0,025,0 ,

For the computation of uncertaintyit does not play any role whetherthe information is of TYPE A or BBoth methods lead to PDFs!

In the end, one obtains via(Bayesian) probability theory a probability distribution function

Eccentricity.....General:Influencesthat can notbe modelled


18

Basic Concepts (10/11) Model links input quantities to measurand

The MODEL relates the input quantities

to the output quantity Y (measurand)

X1 , X2, X3 , ..., XN

YYSourceSourceCauseCause

Ideal MeasurementIdeal Measurement

X=h(Y)X=h(Y)IndicationIndicationEffectEffect

Y=h -1(X)Y=h -1(X) =f (X) =f (X)(usually simple relation)(usually simple relation)

YYSourceSourceCauseCause

Real MeasurementReal Measurement

∆X1∆X1

δX1δX1X2,...X2,...

X1=h(Y, ∆X1, δX1, X2,...)X1=h(Y, ∆X1, δX1, X2,...)Y is "influenced " prior Y is "influenced " prior to its indicationto its indication

Y=f (X1, ∆X1, δX1, X2, ...)Y=f (X1, ∆X1, δX1, X2, ...)

(to find h-1( Y,...) can be difficult)(to find h-1( Y,...) can be difficult)

INDSt INDXt

Measurand Y: measurement deviation of the calibrated thermometer

tbath ? tINDS and tINDX

Standard To becalibrated


19

ξ1

ξ2

ξ3

η

Y=f ( , , )X1 X2 X3

ΥΥΥΥ

Basic Concepts (11/11) PDF for the Xi and model yield PDF for Y

MODELrelates input quantitiesX1 , X2, X3 , ..., XNto output quantity Y

Probabilitydistributionfunction (PDF)

represents knowledgeabout possible valuesof a quantity

( ) ...d d),...,( ),...,( ...)( 111,...,1 NNYNXXY fggN

ξξξξηδξξη ∞

∞−

∞

∞−

−⋅=


20

Basic procedures (1/5) Propagation of PDFs

∞

∞−

∞

∞−

∞

∞−

=

−=

NNXX

Y

ddy-fg

dygyu

Nξξξξξξ

ηηη

. )),...,()(,...,(...

))(( )(

12

1N1,...,

22

1

The standard deviation of that PDF is the uncertainty associated with thatvalue:

The expectation value x of that PDF is the best estimate for the value of thequantity:

∞

∞−

∞

∞−

∞

∞−

== NNXXY ddfgdgyN

ξξξξξξηηη . ),...,(),...,(... )( 11N1,...,1

( ) ...d d),...,( ),...,( ...)( 111,...,1 NNYNXXY fggN

ξξξξηδξξη ∞

∞−

∞

∞−

−⋅=

The explicit knowledge of gY(ηηηη) is not needed, it is onlyneeded if one needs to state the expanded uncertainty.

*

* Methods for computing gY(η): Monte Carlo


21

Basic procedures (2/5) Propagation of uncertainties (1/2)

If the model is linear(or linearized, e.g. via Taylor expansion)

However, for computing the expanded uncertainty one needs to know the PDF for Y!

Knowing the shape of the PDFs is not needed to compute u(y)!

and if all xi, u(xi) and correlation coefficients are known one obtains the solution:

*

= =

=N

i

N

jjjjiii xucxxrxucyu

1 1

)(),()()(

=

+=N

iiiN xcyf

11 ),..,( ξξ

1:),( ;EE

)COV(),( =

⋅= ii

ii

jiji xxr

XX

XXxxr

*


22

Basic procedures (3/5) Propagation of uncertainties (2/2)

For many cases encountered in practice,the GUM provides a straightforward method to evaluatethe measurement uncertainty u(y)without using the PDF explicitly:

( ) )(),()( 1 1

2jjii

N

i j

N

j i

xuxxrxux

f

x

fyu

= = ∂∂

∂∂=

u (y) : combined standard uncertainty of yui(y) : standard uncertainty of xi

r (xi, xj) : correlation coefficient: sensitivity coefficients

ix

f

∂∂


23

Basic procedures (4/5) Step-wise procedure

(a) State the model (identify in quantities and “link” them)(b) Identify and apply all corrections(c) Obtain the sensitivity coefficients and list all sources of

uncertainty(d) Compute uncertainties for TYPE A contributions(e,f) Determine uncertainties for TYPE B contributions

(g) Compute the value of the measurand and the uncertaintyassociated with it

(h) Calculate the expanded uncertainty and the correspondingcoverage factor

(i) Report the (complete) result of the measurement by statingthe best estimate, the expanded uncertainty, the coveragefactor and the PDF for the measurand.

Summary of the step-wise procedure suggested in EA-04/02** DKD-3


24

Generic uncertainty budget for the measurand Y assuming a linear model

Input quan-tity & unit

Expectation value

Standard uncertainty

PDF ν(*) Sensitivity coefficient

Uncertainty index(+) in %

X1 x1 u(x1) N ∞ c1 60 X2 x2 u(x2) R ∞ c2 20 X3 x3 u(x3) t 5 c3 5 ... ... ... ... ... ... ... XN xN u(xN) t 20 cN remainder Y y u(y) N

y =f(x1,..., xN) u2(y) = [(c1 u(x1)]

2+...+[(cN u(xN)]2 k0,95=2 U=2u

(*) Degree of freedom (+) Uncertainty index: 100 ci2u2(xi) /u

2(y)

Basic procedures (5/5) Budget

A summary in form of a budget table should be provided:

Steps a-f

Step g

Step h

Step i: state the result: Value ± Expanded Uncertainty,coverage factor and probability

and indicate PDF for Y


25

SRC

Example: Measurement of a DC current (1/5)

IX

VINDV RM

IX : DC current we want to measure

VIND : indicated voltage on voltmeter

RM : precision measurement resistor (calibration certificate available)

Ideal measurement: “cause-effect propagation”

INDTRANS

IX

RM

V VIND

Principle: measure voltage across resistor with digital voltmeter


26


Real measurement:

Model equations:

SRC INDTRANS

IX

RM

V VIND

δ ISRC δ VTRANS δ VINDX

-δ VIND0∆ VINDX

TRANS0 INDX INDINDIND

M0M

Mef M

SRCeff M

X

)1(1

VVVVVV

tRRr

RR

IR

VI

f

δδδα

δ

−+−∆−=∆⋅+=

+=

−= (1)

(2)

(3)

(4)


27


Explanation of influential quantities:

VIND – indicated electrical voltage: 12 readings TYPE A evaluationaverage 100.03 mV, experimental standard deviation is 0.099 mV, experimental standard deviation of the mean is 0.028 mV – PDF normal

∆ VIND – systematic deviation of digital voltmeter according to manufacturer specs range is ±0.045 mV – PDF rectang.

δ VINDX , δ VIND0 – deviation due to limited resolution of digital voltmeter, occurs when indicating both actual voltage and zero voltage range is ±0.005 mV – PDF rectang.

δ VTRANS – deviation due to external influences on measurement setup range is estimated to be within ±0.020 mV – PDF rectang.


28


Explanation of influential quantities (continued):

RM0 – resistance of precision reference measurement resistor as stated on calibration certificate: 10.018m Ω ±6.01•10-3 m Ω (k=2) for ambient temperature of 23ºC – PDF normal.

∆ t – deviation from ambient temperature of 23ºC: range is ± 2K – PDF rectang.

α – temperature coefficient of resistor – given by manufacturer as value of 50.0 •10 -3K-1 . Influence of uncertainty in α negligible compared to uncertainty in ∆ t treated as constant with no uncertainty.

r – ratio of measurement resistance RM to input resistance of voltmeter RIN .Manufacturer spec says RIN larger than 1 M Ω range of r is 0...10 •10-9 ,best estimate is 5 •10-9 , half width of interval is 5 •10-9 - PDF rectang.

δ ISRC – deviation due to leakage currents in setup range is -1...0 mA,best estimate is –0.5 mA , half width of interval is 0.5 mA - PDF rectang.


29

Finally - state the result: IX=9.986A ± 10mA (k=2), PDF normal.


Input quantity

Expectation value

Standard uncertainty

PDF ν(*) Sensitivity coefficient

Uncertainty index(+) in %

VIND 100.03 mV 0.0284 mV N 11 0.10 mΩ-1 31.4

∆VIND 0.0 mV 0.026 mV R ∞ -0.10 mΩ-1 26.2

δVIND X 0.0 mV 2.89•10-3 mV R ∞ -0.10 mΩ-1 0.3

δVIND 0 0.0 mV 2.89•10-3 mV R ∞ -0.10 mΩ-1 0.3

δVTRANS 0.0 mV 11.5•10-3 mV R ∞ -0.10 mΩ-1 5.2

RM 0 10.018 mΩ 3.01•10-3 mΩ N 50 -1.0 AmΩ-1 35.0

∆ t 0.0 K 1.15 K R ∞ -0.5•10-3AK-1 1.3

r 5•10-9 2.89•10-9 R ∞ 10 A 0.0

δ I SRC -0.5 mA 0.289 mA R ∞ -1.0 0.3

IX 9.9855 A 5.07 mA N 87

y =f(x1,..., xN) u2(y) = [(c1 u(x1)]

2+...+[(cN u(xN)]2 k0,95=2 U=2u

(*) Degree of freedom (+) Uncertainty index: 100 c i2 u2(xi) / u

2(y)

More examples, incl. microwave applications: http://www.european-accreditation.org


30

Summary and conclusions

The Guide to the expression of uncertainty in Measurement (GUM)• published in 1993 by ISO in name of BIPM IEC IFCC ISO IUPAC

IUPAP and OIML• universally applicable, internally consistent and transparent procedure to

evaluate measurement data traceable to SI units• is world-wide accepted and applied.

The GUM provides basis of standardised uncertainty evaluation.

Uncertainty is a quantitative measure of reliability and quality of values of quantities we rely on in our daily life, in industry and science.

Uncertainty is a, if not the, backbone of quality assurance, inthe accreditation business and in engineering and science.

GUM concordant uncertainty evaluation provides insight.

Updated version of this talk: ftp://ftp.ptb.de/pub/ptb/abt2/arz01/IMS2004.pdf

Documents

THE GUIDE TO THE EXPRESSION OF UNCERTAINTY IN MEASUREMENT ... · The Guide to the Expression of Uncertainty in Measurement (GUM) IMS 2004 Workshop: Statistical Methods and Analysis