Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
1
Measurement Uncertainty
Part I - Fundamentals
Walter Bich
INRIM – Istituto Nazionale di Ricerca Metrologica
Torino (Italia)
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
2
Framework
• 1977-79 BIPM questionnaire on uncertainties
• 1980 Recommendation INC-1
• 1981 Establishment of WG3 on uncertainties under ISO TAG4: BIPM, IEC,
IFCC, ISO, IUPAC, IUPAP, OIML
• 1981 Recommendation CI-1981
• 1986 Recommendation CI-1986
• 1993 Guide to the expression of uncertainty in measurement
• 1995 Reprint with minor corrections
• 1997 Establishment of the Joint Committee for Guides in Metrology JCGM.
ILAC joins in 1998
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
3
• Present Chair: the BIPM Director
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
Joint Committee for Guides in
Metrology
• The JCGM has two working groups (WGs)
•WG 1 has responsibility for the Guide to the expression of uncertainty in
measurement, GUM
•WG 2 has responsibility for the International vocabulary of Basic and General
Terms in metrology, VIM
•See also www.bipm.org
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
4
WG1 published documents
JCGM 101:2008 (Monte Carlo)
JCGM 100:2008 (GUM 1995 with minor corrections)
JCGM 104:2009 (Introduction to uncertainty)
JCGM 102:2011 (Any number of output quantities)
All these are also published by OIML, ISO and IEC. Adopted by
IFCC, ILAC, IUPAC and IUPAP
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
JCGM 106:2012 (conformity assessment). Approved, to be published
soon
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
5
WG1 planned documents
JCGM 103 (Modelling)
JCGM 105 (Fundamental principles)
JCGM 100 (GUM revision)
JCGM 107 (Least-squares applications)
JCGM 108 (Markov Chain Monte Carlo)
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
6
WG2 document
JCGM 200:2012 (VIM3) The third version of this basic document.
Source of definitions of concepts and terms
Need for the two WGs to be as consistent as possible.
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
Not always an easy task!
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
7
Measurement, properties and
quantities quantity
property of a phenomenon, body, or substance, where the property has a
magnitude that can be expressed as a number and a reference (VIM3,1.1)
a reference can be a measurement unit, a measurement procedure, a
reference material, or a combination of such
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
‘property’, ‘magnitude’ and ‘reference’ are explicitly considered as
“primitive”, therefore undefined
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
8
Measurement units
measurement unit
real scalar quantity, defined and adopted by convention, with which any other
quantity of the same kind can be compared to express the ratio of the two
quantities as a number (VIM3 1.9)
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
A measurement unit is thus a particular kind of reference
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
9
Ordinal quantities
ordinal quantity
quantity, defined by a conventional measurement procedure, for which a total
ordering relation can be established, according to magnitude, with other
quantities of the same kind, but for which no algebraic operations among
those quantities exist (VIM3, 1.26)
EXAMPLE 1 Rockwell C hardness.
EXAMPLE 2 Octane number for petroleum fuel.
EXAMPLE 3 Earthquake strength on the Richter scale.
EXAMPLE 4 Subjective level of abdominal pain on a
scale from zero to five.
NOTE 1 Ordinal quantities can enter into empirical
relations only and have neither measurement units nor
quantity dimensions. Differences and ratios of ordinal
quantities have no physical meaning.
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
10
VIM3 definitions
quantity value (value of a quantity)
number and reference together expressing magnitude of a quantity
(VIM3,1.19)
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
Length of a given rod: 5.34 m or 534 cm
numerical quantity value
number in the expression of a quantity value, other than any number serving
as the reference (VIM3,1.20)
Numerical values of the length of a given rod: 5.34 (unit metre) or 534
(unit centimetre)
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
11
VIM definitions
quantity calculus
set of mathematical rules and operations applied to quantities other than
ordinal quantities (VIM3, 1.21)
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
quantity numerical quantity value
unit
quantity value!
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
12
VIM3 definitions
measurement
process of experimentally obtaining one or more quantity values that can
reasonably be attributed to a quantity (VIM3, 2.1)
measurand
quantity intended to be measured (VIM3, 2.3)
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
My loose definition: process aimed at improving the state
of knowledge about a measurand
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
13
VIM definitions
measurement result
set of quantity values being attributed to a measurand together with any other
available relevant information
NOTE 1 A measurement result generally contains “relevant information” about the
set of quantity values, such that some may be more representative of the measurand
than others. This may be expressed in the form of a probability density function
(PDF).
NOTE 2 A measurement result is generally expressed as a single measured quantity
value and a measurement uncertainty. If the measurement uncertainty is considered
to be negligible for some purpose, the measurement result may be expressed as a
single measured quantity value. In many fields, this is the common way of
expressing a measurement result.
NOTE 3 In the traditional literature and in the previous edition of the VIM,
measurement result was defined as a value attributed to a measurand and explained
to mean an indication, or an uncorrected result, or a corrected result, according to
the context.
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
14
VIM definitions
2.10
measured quantity value
quantity value representing a measurement result
2.11
true quantity value
true value of a quantity
true value
quantity value consistent with the definition of a quantity
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
15
VIM definitions
2.27
definitional uncertainty
component of measurement uncertainty resulting from the finite amount of detail in
the definition of a measurand
NOTE 1 Definitional uncertainty is the practical minimum measurement uncertainty
achievable in any measurement of a given measurand.
Examples: Distance Varenna – Paris
Atomic weight of C
Air density in my laboratory
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
16
GUM scheme
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
17 International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
Error (horror?)
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
18 International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
More horror
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
19
Probable error
Now, re-interpret 𝜀 as probable error
Probability comes into play, and all its tools are at hand
First, 𝜀 is taken as a random variable
From here, and for a while, apologies to those who already know!
Most of the material taken from Wikipedia…
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
20
Random variables A random variable (RV) is a variable whose value is subject to variations due to
chance, or can take on a set of possible different values, each with an associated
probability.
They may also conceptually represent either the results of an "objectively" random
process (e.g. rolling a die), or the "subjective" randomness that results from
incomplete knowledge of a quantity.
The meaning of the probabilities assigned to the potential values of a random
variable is not part of probability theory itself, but instead related to philosophical
arguments over the interpretation of probability.
The mathematics works the same regardless of the particular interpretation in use.
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
21
Probability distributions
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
22
Probability density function
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
23
Cumulative distribution function
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
24
RVs and pdfs
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
25
Gaussian pdf
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
From Wikipedia
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
26
Gaussian CDF
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
From Wikipedia
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
27
Moments of RVs
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
28
Moments of RVs
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
29
A relevant pdf
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
30
Student’s t-distribution
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
From Wikipedia
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
31
Random vectors
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
32
Covariance matrix
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
33
Covariance
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
34
Correlation
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
35
Covariance in measurement
The role of covariance in measurement uncertainty is fundamental, and typically
underevaluated
«Neglecting covariances» is not a way to get rid of them.
Rather, it is a very stringent statement that they are equal to zero, a situation rarely
occurring in multivariate measurements
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
36
Multivariate Gaussian pdf
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
37
From the ugly duckling …
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
38
… to the beautiful swan
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
39
A question
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
40
Orthodox solution
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
41
Orthodox solution cont.
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
42
Orthodox solution fails When there is no sample, no statistics can be applied and the frequentist approach
fails.
In a Bayesian approach, a pdf can be associated to any quantity to describe the
state of knowledge on it. Guidance in JCGM 101:2008.
In the GUM this approach is adopted only in Type B evaluations, to associate an
uncertainty to the corresponding estimates. The uncertainty is the standard deviation
of the pdf.
However, to remain in the frequentist framework, degrees of freedom are artificially
attached also to these uncertainties!
The Bayesian framework works both for random and systematic errors
The GUM is being revised in this sense
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
43
A dirty trick
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
44
Multivariate generalization
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
45
Multivariate generalization
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
46
Limitations
We talked so far about pdfs, but we used only their first and second moments.
This procedure has two orders of limitations:
The reliability of the variance of the output quantity depends on the amount of non-
linearity in the neighbourhood of the estimates.
The variance of the output quantity is of little interest to end users, who typically
need a coverage interval, i.e., an interval containing the value of the measurand with
a stated probability, based on the information available.
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
47
Solution Instead of propagating only first and second moments of the input quantities Xi, their pdfs are propagated through the model. The method is more demanding in terms of amount of knowledge. One has to assign a pdf to each input quantity, based on the experimental data or other knowledge.
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
Assignment of input pdfs
•The principle of maximum entropy –you maximize a functional S, the
“information entropy”, under constraints given by the information
•According to the available information, it is based on
•Bayes’ theorem, typically when a series of indications is available
48 International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
Assignment of input pdfs
•FOCUS: if a sample of indications is available, you assign a Student’s t-
distribution. Departure from the GUM!
•Luckily, extensive literature and guidance in Supplement 1 do most of the
job
•A dozen pdfs for the various cases are suggested
49 International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
Problem Given the joint pdf of the N input quantities X, find the pdf of the output
quantity y.
Using the Jacobian method (any textbook on mathematical statistics)
1d...d... NY fgg X
Is the Dirac function
50 International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
Solutions Analytical: A closed-form solution can be found only in the most simple (and
therefore uninteresting) cases. In general, the integrals involved in this solution must
be solved numerically. Not viable.
•Numerical 1: by numerical integration of the formal expression
51 International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
The Supplement 1 approach: MCM
Numerical 2 (adopted in Supplement 1):
• Numerical simulation.
• Method selected: Monte Carlo (MCM).
• Tools: suitable random number generators for the various pdfs, reasonable computing power.
• Outcome: a numerical approximation for the output distribution (in various possible forms).
YG
52 International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
Output of MCM
• From the numerical approximation for the output distribution, the required statistics, such as
• the best estimate for the measurand,
• its standard uncertainty, and
• the endpoints of a prescribed coverage interval can be obtained.
53 International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
The method in a nutshell
From each input pdf draw at random a value xi for the random variable
Xi.
Use the resulting vector xr (r = 1,…M) to evaluate the model, thus obtaining
a corresponding value yr. The latter is a possible value for the measurand
Y.
Iterate M times the preceding two steps, to obtain M values yr for Y .
54 International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
Representations of the probability
distribution for y
Sort the M values yr for Y in non-decreasing order.
•Discrete (G):
Take G as the set Mry r ,...,1 ,
•Sufficient for most applications
55 International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
Representations of the probability
distribution for y
In the form of a piecewise-linear function, suitably obtained from G
(details in the Supplement).
Useful, e.g. for further samplings
•Continuous: YG~
56 International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
Representations of the pdf for y
•Assemble the (M) yr values into a histogram with suitable cell widths
(subjective!) and normalize to one.
Useful for visual inspection of the pdf, or
when the sorting time (M large and simple model) is excessive
57 International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
Coverage interval(s)
•The novelty with respect to the GUM is that, since the pdf is usually
asymmetric, there is more than one coverage interval (for a given coverage
probability p)
Shortest (contains the mode)
Probabilistically symmetric 21 p
•Endpoints quantiles p and
58 International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
Comparison of the two approaches
• The Monte Carlo approach works in a broader class of problems than the GUM approach. In this sense, it is more general, therefore
• It can be used to validate the results provided by the GUM uncertainty framework, however
• It is based on the same principles underlying the GUM
• Descends naturally from the GUM
• It is to be used in conjunction with the GUM
59 International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
Comparison of the two approaches
60 International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
Red dotted: GUM
Solid blue: S1
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
Comparison of the two approaches
(continued)
• The best estimate (as the expectation of the numerical approximation
for the output distribution) does not necessarily coincide with that
provided by the GUM.
• Also the standard uncertainties do not coincide. The GUM value may be
smaller. This is a consequence of the pdf recommended for sampled data
(Student).
• The main output is a coverage interval, not the standard uncertainty
61 International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
• There is no longer any need for degrees of freedom.
• No longer uncertainty of the uncertainty!
Type A and B do not apply to pdfs (luckily…)
• Actually, the approach of Supplement 1 is intrinsically Bayesian.
Comparison of the two approaches
(continued)
62 International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
Internal (in)consistency of the GUM
Poor, if an interval of confidence is required
Acceptable, as far as the issue is standard uncertainty
presence of two different views of probability, and
frequentist choice concerning expanded uncertainty
Reason(s):
63 International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
(In)consistency between the GUM and
Supplement 1 (its first creature…)
Resulting measurand values and associated standard uncertainties are different, but also
standard uncertainty
is uncertain with the GUM,
has no uncertainty with Supplement1
64 International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
Remedy
(Mildly) revise the GUM, by
Adopting a Bayesian evaluation of standard uncertainty also for
sampled data (Type A), for example
GUM1GUM2
3
1ii xu
n
nxu
65 International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
To go back to reality…
If your experiment needs statistics, you ought to
perform a better experiment
(Lord Rutherford)
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012 66
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
Or, if you prefer…
the only statistics you can trust are those you
falsified yourself
(Churchill)
67
ISTITUTO NAZIONALE DI RICERCA METROLOGICA
End of part I
International School of Physics Enrico Fermi "Metrology
and Physical Constants" Varenna 17 - 27 July 2012