Removing the 'Uncertainty' from Measurement Uncertainty Thursday, 28 August 2008

Dr Robert SymonsNational Measurement Institute

2

Presentation Overview

DefinitionsChemical and Biological Metrology Branch

Chemical Reference MaterialsChemical Proficiency TestingBioanalysisChemical Reference Methods

Case StudyAflatoxins in Food ProductsDrugs in Sport

3

Definitions – ISO/IEC Guide 99:2007International Vocabulary of Metrology (VIM)

Metrology– Science of measurement and its application

Measurement– Process of experimentally obtaining one or more quantity values

that can reasonably be attributed to a quantityMeasurement result– Set of quantity values being attributed to a measurand together

with any other available relevant informationMeasurand– quantity intended to be measured

4

Definitions – ISO/IEC Guide 99:2007International Vocabulary of Metrology (VIM)

Measurement Uncertainty (u)– Non-negative parameter characterising the dispersion of the quantity

values being attributed to a measurand, based on the information used.Expanded Measurement Uncertainty (U)– Product of a combined standard measurement uncertainty and a factor

larger than one.Coverage Factor– Number larger than one by which a combined standard measurement

uncertainty is multiplied to obtain an expanded measurement uncertainty. – NOTE: A coverage factor is usually symbolised k and nominally k = 2

representing the 95% confidence interval.– U = u x 2

5

What is Measurement Uncertainty?Associated with the Result

It is the number after the ± !3.2 ± 0.1 mg kg-1

6

What is the NMI doing?Measurement Uncertainty

Chemical Reference Materials Team– pure chemical substances certified in accordance with

metrological principles using established, international best practice protocols ISO Guide 34:2000;

– pure chemical certified reference materials (CRMs) are used to calibrate the measurement process and they all have common characteristics:

– assigned values are accompanied by an uncertainty statement;

– information is given on the methods used to assign values;– the intended use of the CRM is described;– the reported values of CRMs are, by definition, traceable to

SI.– illicit drugs, sports drugs and agrichemicals.

O

NH

HN

OCl

ClCl

Ketotriclabendazole

7

8

What is the NMI doing?Measurement Uncertainty

Chemical Proficiency Testing Team– The principal aims of our program:

– to provide testing laboratories with a tool to improve the accuracy and traceability of their chemical measurements;

– to provide Australian laboratories and NATA with information on the current 'state of the practice' in each area of analysis;

– to evaluate laboratory capability and encourage improvements in laboratory methods and performance from 'state of the practice' to 'fit for purpose‘;

– to enable participating laboratories to assess their performance relative to domestic and international peer laboratories and hence to improve the comparability of results between laboratories and between countries;

– to develop and promote a 'fit for purpose' and affordable metrological approach to proficiency testing;

– to provide practical advice and support to laboratories regarding traceability of chemical measurements and estimation of measurement uncertainties.

– Food, environmental and illicit drugs.

9

What is the NMI doing?Measurement Uncertainty

Bioanalysis Team– Accuracy and precision in quantitative real-time PCR analysis– Criteria for method validation and development of uncertainty

budgets are essential components of any analytical procedure to ensure fitness-for-purpose

– Through the CCQM Working Group on Bioanalysis, NMI is participating in pilot studies aimed at identifying factors thatcontribute to the uncertainty associated with PCR measurements and developing approaches for improving measurement accuracy and precision

– New research with digital PCR that may lead to primary methods development

10

What is the NMI doing?Measurement Uncertainty

Chemical Reference Methods Team– Primary method development;– High accuracy methods for both inorganic and organic analyses;– Traceable to The International System of Units (SI);– Methods based upon Isotope Dilution Mass Spectrometry;– Produce matrix certified reference materials (CRMs) e.g., pesticides in

tomato puree and nitrofurans in prawns;– Provide reference values for NMI’s proficiency testing programs;– Determine the best estimate for the uncertainty of results;– Responsible for demonstrating Australia’s capabilities in CCQM

international intercomparisons.

11

CCQM-K24 Cd in Rice

13.6

13.8

14.0

14.2

14.4

14.6

14.8

15.0

15.2

15.4

KoreaUK JapanSwitzerlandUS AustraliaNetherlandsGermanyEU PR ChinaCanadaSth Africa

Laboratory

Amou

nt c

onte

nt (n

mol

/g)

U = ± 1.7%

Consultative Committee for the Amount of Substance – Metrology in Chemistry

12

U = ± 15%

Consultative Committee for the Amount of Substance – Metrology in Chemistry

CCQM-K24 Cd in Rice

10.0

11.0

12.0

13.0

14.0

15.0

16.0

17.0

18.0

19.0

20.0

KoreaUK JapanSwitzerlandUS AustraliaNetherlandsGermanyEU PR ChinaCanadaSth Africa

Laboratory

Amou

nt c

onte

nt (n

mol

/g)

13

2000 European Study: Lead in winen = 129 laboratories

13.616.319.021.724.427.129.832.535.237.940.6

0 10 20 30 40 50 60 70 80 90

Con

cent

ratio

n in

mic

rogr

am/L

itre

11 (9%) below - 50%

50%28 (22%) above

50%

14

Why do we need matrix Certified Reference Materials?

15

Aflatoxins in Milk Powder(without reference material)

0

0.2

0.4

0.6

0.8

1

1.2

1.4

0 1 2 3 4 5 6 7 8 9 101112 1314151617 1819202122 232425

Lab No.

Mas

s fra

ctio

n of

afla

toxi

n (u

g/kg

)

16

Aflatoxin levels in Milk Powder(with reference material)

0

0.2

0.4

0.6

0.8

1

1.2

1.4

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

Lab No.

Mas

s fr

actio

n of

afla

toxi

n (u

g/kg

)

17http://ec.europa.eu/food/food/chemicalsafety/contaminants/comm_dec_2006_504guidance_en.pdf

18

II.19. Acceptance of a lot or sublot and interpretation of results

For groundnuts, nuts and dried figs intended for direct human consumption:

– acceptance if none of the laboratory samples exceeds the maximum limit, taking into account the measurement uncertainty and the correction for recovery;

– rejection if one or more of the laboratory samples exceeds the maximum limit;

– beyond reasonable doubt taking into account the measurement uncertainty and correction for recovery*,

* The measurement of uncertainty should be subtracted from the analytical result after correction for recovery. This result is the analytical result which should be used when judging compliance of a consignment with EU legislation.

19Action: reject accept accept accept

Interpretation of the measurement of uncertainty when considering compliance with a statutory limit, where the circle is

the analytical result.

maximumlimit

( i )Result lessuncertaintyabove limit

( ii )Result

above limitbut limitwithin

uncertainty

( iii )Result belowlimit but limit

withinuncertainty

( iv )Result plusuncertaintybelow limit

20

21

The mean value of the results of three aliquots for the ‘A’ Sample for Threshold Substances minus the value of measurementuncertainty determined by the Laboratory must exceed the relevant Threshold……………………Adverse Analytical Finding or Atypical Finding decisions shall be based on the mean of the measured concentrations, taking into account the measurement uncertaintywith the coverage factor, k, and a level of confidence of 95%. Reports and documentation shall give the mean concentration withthe associated uncertainty.

22Action: reject accept accept accept

Interpretation of an adverse analytical finding taking into account measurement of uncertainty

by the World Anti-Doping Agency

maximumlimit

( i )Result lessuncertaintyabove limit

( ii )Result

above limitbut limitwithin

uncertainty

( iii )Result belowlimit but limit

withinuncertainty

( iv )Result plusuncertaintybelow limit

23

RESPONSIBILITIES

OF THE LABORATORY• to supply results that are fit-for-purpose• to supply expanded measurement uncertainty values associated

with the result when requested by the client• not to interpret or comment on results• not to comment on sampling

OF THE REGULATOR• to make the decision whether the result is compliant or not• to decide if sampling is representative of the lot

24

Sample arrives at the laboratory

LABORATORY CUSTOMER

Reproducibility between

laboratories SR

Method and Laboratory Bias•Certified reference material•Interlaboratory comparison•Method validation

3.2 ± 0.1

Analytical Report

QC -Reproducibility

within laboratory Rw

MEASUREMENT UNCERTAINTY MODEL

25

What is the NMI doing?Measurement Uncertainty

Training Team– an overview of ISO's Guide to the Expression of Uncertainty in Measurement– modeling a measurement– identifying uncertainty components– basic statistical concepts– uncertainty calculations– practical exercises– useful spreadsheet functions– application in the workplace– sensitivity coefficients– sample calculations

26

Conclusions

NMI can provide pure substance reference materialsNMI can provide matrix certified reference materialsNMI can provide proficiency testing programsNMI can provide training

27

National Measurement Institute1 Suakin StreetPymble NSW 2073Australia

Phone: + 61 2 9449 0111

Email: info@measurement.gov.au