UNCERTAINTY INTERVALS IN CERTIFIED REFERENCE … · LGC 7107 LGC QC1002 SMRD 2000 LGC 7101 LGC...

UNCERTAINTY INTERVALS IN

CERTIFIED REFERENCE MATERIALS

FOR NUTRIENTS IN FOODS

Wayne R. Wolf1, Katherine M. Phillips2

1. Food Composition Methods Development Laboratory, Beltsville Human Nutrition Research Center

United States Department of Agriculture, Beltsville, Maryland, 20705, USA

2. Biochemistry Department , Virginia Polytechnic Institute and State University, Blacksburg, VA, 24061

The Future of Reference Materials – Science and Innovation

IRMM, Geel, Belgium, 23-25 November 2010

• 30 plus years of helping to provide Food Related CRMs ----

• Initial Question was: How many Food CRMs do we need?:

• Answer is: ?

• Present Question: How Good do the Food CRMs need to be?

1987 RMs for biological RMs

exclusively for elements (BERM-2)

11987

Parr, et. al. Fresenius Z. Anal. Chem (1987) 326:601-608.

1992 Organic Components in food RMs

(BERM-5)

• Major components and major elements in five food RMs - BCR

• Determinations of vitamins in food and preparation of RMs for vitamin analysis –BCR

• Trace organics in food and agriculture materials (4 papers)

– Agrochemical residues

1994 System for defining food matrices

(BERM -6)

1997 Food CRMs characterized by

collaborative process (BERM-7)

2007 Summary of Food RMs

(BERM 10)

USDA National Food Nutrient

Analysis Program

• NFNAP was designed to update the USDA nutrient database (USDA Standard Reference, SR) by collecting foods consumed in the US according to a statistical sampling plan for analysis of key foods and nutrients

• Over 3000 samples of more than 875 food items have been collected and analyzed for more than100 nutrients and other dietary components as part of the NFNAP

Use of CRM in NFNAP• NFNAP QC system included CRM and control

samples developed specifically for the program

• Blind samples of CRM were sent to labs for

analysis to

– qualify for a NFNAP contract

– QC analysis with routine unknown samples

• 2600+ values from 9 laboratories for 77 nutrients

in 26 CRM have been obtained over 6.5 years

NFNAP CRM Data Set

• What can this set of data, generated by contract analytical laboratories, on well characterized known samples, tell us about the general state of practice of the measurement system for generating food composition data?

• This is not an interlaboratory evaluation, nor an assessment of individual methodology.

• These data can give insight of the present state of practice of the measurement system for generating food composition data.

Goals

• Overview of Food CRM availability by matrix and nutrient

• Uncertainty intervals for CRM

• Preliminary assessment of results from routine analysis by contract laboratories during NFNAP for nutrients in CRM with certified or reference values

– Nutrient level issues

– Matrix issues

• Begin dialog of how good do CRM need to be (?)

Identification of Food CRM

• Knowledge of existing suppliers

• IAEA Database for Natural Matrix Reference

Materials

• Supplier websites (selected)

– National Institute of Standards and Technology (NIST)

– Institute for Reference Materials and Measurements (IRMM)

– American Association of Cereal Chemists (AACC)

– National Research Centre for Certified Reference

Materials (China)

– National Research Council of Canada (NRC )

– LGC Promochem

Matrix Triangle

1

2

3

4

56

7 89

1

2

3

4

56 7 8

9

NIST 1563

LGC 7104

NIST 2384

NIST 2387

NIST 8415

NIST 1546

LGC 7106

LGC 7000

LGC 7150

LGC 7151

LGC 7152

100% Fat

100% Protein100% Carbohydrate

SRM 3244

NIST 8418

SRM 1974a

NIST 1946

LGC 7176

NIST 1566b

LGC 7001NIST 1570a

NIST 2385

NIST 8435

NIST 1544

NIST 1548a

NIST 1846

LGC 7105

NIST 8432

NIST 8433

NIST 8436

NIST 2383

BCR-382

BCR-381

LGC 7107

LGC QC1002

SMRD 2000

LGC 7101

LGC 7103

Nutrient / CRM Availability1= certified

2= reference

Sector RM M M Pr Ash Fat Suc Fru Glu Lac Mal Gal Fib St5 BCR 121 WholemealFlour5 BCR 189 WholemealFlour5 BCR 381 RyeFlour 1 1 15 BCR 382 WheatFlour 1 1 15 BCR 383 FreezeDriedGreenBeans 1 15 BCR 431 FreezeDriedBrusselsSprouts5 BCR 485 FreezeDriedMixedVeg5 BCR 679 FreezeDriedWhiteCabbage5 ERMBC 515 FreezeDriedCarrots 15 ERMBC 516 FreezeDriedApple 15 ERMBC 518 DriedBranBreakfastCereral 15 GBW 07605 Tea5 GBW 08508 Rice

5 LGC 7003 WhiteBreadFlour 1 15 LGC 7017 SugarConfectionery 1 1 1 15 LGC 7103 SweetDigestiveBiscuit 1 1 1 15 LGC 7107 MadeiraCake 1 1 1 1 15 NIST 1567a WheatFlour5 NIST 1568a RiceFlour5 NIST 2383 Baby Food 2 2 2 2 25 NIST 8432 CornStarch 2 2 2 25 NIST 8433 CornBran 2 2 2 25 NIST 8436 DurumWheatFlour 2 2 2 2 25 VMA 195 Cereal 1 1 1 15 VMA 399 Cereal 1 1 1 1

Proximates CHO

CRM Over All Sectors1= certif ied

2= reference

Sector RM M M Pr AshFat Suc Fru Glu Lac Mal Gal Fib St Ca K Mg Na P Cu Fe Mn Se Zn A B1 B2 Nia B6 B12Fol PA C D E Car K FA

BCR 122 Margarine

BCR 162 SoyaMaizeOil

BCR 163 BeefPorkFat Blend

BCR 164 AnhydrousMilkFat

LGC 7104 SterilizedCream

NIST 1563 FortifiedCoconutOil

2 NIST 2384 BakingChocolate

LGC 7112 PigLiverPaste

NIST 2387 Peanut Butter

GBW 08552 FreezeDriedPorkMuscle

LGC 7000 BeefPorkMeat

LGC 7101 MackerelPaste

LGC 7106 ProcessedCheese

LGC 7152 ProcessedPork

NIST 1546 MeatHomogenate

NIST 1845 WholeEggPowderNIST 8415 WholeEggPowder

BCR 121 WholemealFlourBCR 189 WholemealFlour

BCR 381 RyeFlour

BCR 382 WheatFlour

BCR 383 FreezeDriedGreenBeans

BCR 431 FreezeDriedBrusselsSprouts

BCR 485 FreezeDriedMixedVeg

BCR 679 FreezeDriedWhiteCabbage

ERMBC 515 FreezeDriedCarrots

ERMBC 516 FreezeDriedApple

ERMBC 518 DriedBranBreakfastCereral

GBW 07605 TeaGBW 08508 Rice

LGC 7003 WhiteBreadFlour

LGC 7017 SugarConfectionery

LGC 7103 SweetDigestiveBiscuit

LGC 7107 MadeiraCake

NIST 1567a WheatFlour

NIST 1568a RiceFlour

NIST 2383 Baby Food

NIST 8432 CornStarch

NIST 8433 CornBranNIST 8436 DurumWheatFlour

VMA 195 Cereal

VMA 399 Cereal

BCR 421 MilkPowder

LGC 7105 RicePudding

NIST 1544 Total Diet

NIST 1548a FreezeDriedMixedDiet

NIST 1846 InfantFormulaNIST 8435 WholeMilkPowder

LGC 7001 PorkMeat

NIST 1549 NonFatMilkPowder

NIST 1566b FreezeDriedOysterTissue

NIST 1570a FreezeDriedSpinachLeavesNIST 2385 SlurriedSpinach

LGC 7176 CannedCatFood

NIST 1946 FishTissue

BCR 422 FreezeDriedCodMuscle

BCR 487 FreezeDriedPigsLiver

LGC 7160 CrabPaste

NIST 1577b Freeze-DriedBovineLiver

NIST 8418 WheatGlutenNRCDORM 2 FreezeDriedDogfishMuscle

ERMBC 514 FreezeDriedHaricotBeans

ERMBC 517 DriedFullFatSoya

ERMCE 278 FreezeDriedMuscleTissue

LGCQC 1003 Chocolate NRCDOLT 3 FreezeDriedDogfishLiver

9

5

4

6

7

Vitamins Other

1

3

Proximates CHO Minerals Trace elements

8

Uncertainty Intervals for CRM

• Assigned uncertainty value in the CRM

• Uncertainty intervals as a % of target values

were determined

• UIP = uncertainty / target*100

Un

ce

rtain

ty a

s %

of T

arg

et V

alu

e

(UIP

)

0

10

20

30

40

50

60

70

80

90

1

51

101

151

201

251

301

351

401

451

501

551

601

651

701

751

(Uncertainty / Target value ) *100

Comparison of UIP for Vitamins and

Minerals

0

5

10

15

20

25

30

35

40

45

501

11

21

31

41

51

61

71

81

91

101

111

121

131

141

151

(Un

cert

ait

y/T

arg

et

valu

e)*

100

Vitamins

Minerals

UIP by Nutrient Class and

Concentration

0

10

20

30

40

50

60

70

80

90

1.0E

-04

1.0E

-03

1.0E

-02

1.0E

-01

1.0E

+00

1.0E

+01

1.0E

+02

1.0E

+03

1.0E

+04

1.0E

+05

Nutrient concentration (mg / 100 g)

UIP

Prox Carbo Min Trace VitWaterS VitFatS AA FA Other

UIP Summary

• Most UIP for CRM nutrients were less than 10%, but some had a far greater uncertainty

– 64% of the UIP were <10%

– 25% were 10%-20%

– 11% were >20%

• The UIP for proximates, minerals, and trace elements were most consistently <10% of the assigned value

• On average UIP were significantly higher for vitamins

Use of CRM in NFNAP• NFNAP QC system included CRM (as well as control

samples developed specifically for the program*)

• Blind QC samples, including the CRM were sent to labs for analysis

– To qualify for a NFNAP contract

– As QC samples batched with routine NFNAP foodsamples

• All values are shown, including results where labs did not qualify for specific nutrients

• 2600+ values from 9 laboratories for 77 nutrients in 26 CRM have been obtained over 6.5 years

*Phillips, K.M., Patterson, K.Y., Rasor, A.S., Exler, J., Haytowitz, D., Holden, J.M., and Pehrsson, P. (2006). Quality-control materials in the USDA National Food and Nutrient Analysis Program. Anal. Bioanal. Chem., 384(6): 1341-1355.

Evaluation of Lab Results

• Each reported nutrient value was evaluated for deviation from the target certificate value

– (Result – target value) / uncertainty interval

– (“Z Score”)

– Calculation designated as UI Unit (UIU)

• A value within +/-1 UIU lies within the assigned certificate concentration range

Vitamins

Laboratory

UIU

−20

−10

0

10

20A Thiamin Riboflavin Niacin B6 B12 Folate Panto

AcidC α Carot

A B C D E A B C D E A B C D E A B C D F A B C D F A C D HA B C DA C DA C DA C DA C D

tocoph

UIU by Both Lab and Matrix

• Overall summary of UIUs allows a preliminary

identification of nutrient analyses that may be of

inadequate accuracy and/or precision

• Further, nutrient – matrix combinations that

present analytical challenges can be identified

-18-16-14-12-10

-8-6-4-202468

101214161820

1 5 9

13

17

21

25

29

33

37

41

45

49

53

57

61

65

69

73

(Re

su

tls

-TV

) /

Un

ce

rta

inty

In

terv

al

Niacin By Lab and Matrix

Lab A �

Lab B �

Lab C �

Lab E �

Lab D �

Milk/Infant formula

Meat

Babyfood

Vegetables/Peanut butter

-8

-7

-6

-5

-4

-3

-2

-1

0

1

2

3

4

5

1 5 9

13

17

21

25

29

33

37

41

45

49

53

57

61

65

69

73

(Re

su

lts

-TV

) / U

nc

ert

ain

ty In

terv

al

Thiamin by Lab and Matrix

Lab A �

Lab B �

Lab C �

Lab E �

Lab D �

Flour/Cereal

Milk/Infant formulaMeat

Baby food

Vegetables/Peanut butter

CRM ID Year Nia UIP RiboUIP Thia UIP Pyri UIP

NIST1846 1996 12 5.7 13.8 11.9

BCR 431 1997 7

NIST 2383 1997 12 14.1 16.5 14.6

BCR 121 1998 8.4 24.9

BCR 485 1998 11.1 16.7

BCR 487 1998 5.2 12.8 15

NIST 1546 1999 10.5 29.5 20.4 46.9

NIST 8435 1999 12.2 30.9 28.9 31.4

NIST 2384 2001 16.5 13.2

NIST 2387 2003 4.2 20.2 13.3

NIST 2385 2003 14.4 6.8

NIST 3244 2006 3.3 7.7 17.6 6.5

NIST1849 2009 2.4 5.7 8.2 10.5

NIST 3280 2009 1.6 12.8 11.3 9.4

GBW 10037 2010 5.6

NFNAP Composite Control Samples

• * Composite Control food samples sent as blind QC samples with each batch for contract analysis

• Evaluation of >25,000 values assayed at multiple labs over 6.5 years for over 100 nutrients is planned to further evaluate lab performance and measurement reproducibility

• .

Pyridoxine RSD vs Concentration

Data from KM Phillips, VPI&SU Personal communication

Dietary Supplements Quality

Assurance Program- NIST/ODS

How Good do the Data Need to Be?

Fit for Purpose Factors

• Importance of the nutrient in the diet

• Concentration of the nutrient in the food

• % of nutrient requirement in a serving

• Application of the data, e.g.

– Food composition databases

– Validation of diets for clinical feeding trials

(highest precision and accuracy needed)

– Food labeling

Evaluation of

Sources of Method Bias

• Recovery - Spiked amounts

• Difference from CRM –”True Value”

– Calibration errors

– Actual Method Bias - Matrix effects

(Extraction, Detection)

– Requires Tight Uncertainty Limits on CRM

to determine extent of bias.

Conclusions For Food Related CRMs

• Uncertainty intervals need to be tighter for many food CRMs to validate accuracy of results (e.g. definitive rather than consensus certification)

• Suggested UI of < 5%

• UI for elements in foods are at this level

• UI for vitamins are approaching this level

• CRM should be renewed ca. every ten years to reflect advances in technology

Thank you

for listening

Stakeholder Panel for Infant Formula and Adult Nutritionals

8-12 November 2010, AOAC INTERNATIONAL

• Objective: To develop standard method performance requirements for priority nutrients…

• First group: Vitamins A, D, B12, Folic Acid and inositol.

• Fitness-for-Purpose statements were developed in September

Vitamin A

Codex Upper Limit 125

ug/100g, RTF Typical Label Claim 63

Codex Lower Limit 42

SMP Model Performance Required (no bias, no product uncertainty) CUL to CLL CUL to LC

Max Intermed Precision (RSDI) 8.3 5.5

Max Interlab Reproducibility(RSDR) 14.4 9.6

Max Overall Interlab RSD 16.7 11.1

Evaluation of

Sources of Method Bias

• Recovery - Spiked amounts

• Difference from CRM –”True Value”

– Calibration errors

– Actual Method Bias - Matrix effects

(Extraction, Detection)

– Requires Tight Uncertainty Limits on CRM

to determine extent of bias.

SMPR Vitamin A

SPIFAN –SMPR

• Process is developing SMPR initially for 5 components in IF/AN

• Will develop SMPR for at least 15 more in 12-24 months.

• Should be able to gain estimates of required level of UI for CRMs in IF/AN which can be a basis for these 20 components in a wider range of foods.

NFNAP Composite Control Samples

• * Composite food matrix control materials sent as blind QC samples with each batch for contract analysis

• * 25,000 data points.

How Good do the Data Need to Be?

Current Thinking and Future Plans

• CRM are needed to evaluate analytical accuracy

and to provide interlaboratory reference data

• In-house QC materials for precision:

– Retrospective inspection of NFNAP control sample results planned to evaluate lab precision

• To evaluate new methodology and changes in

methods – material challenges

• Need better definition of “Fit for Purpose” issues

2000 Food Matrix CRM development (BERM-8)

2003 Food CRMs for Vitamins available

(BERM-9)

Food and Nutrition MetrologyGeneration of High Quality Data

Measurement System Requirements

* Appropriate Validated Analytical Methods that are fit for purpose– Appropriate precision

– Adequate selectivity

– Needed sensitivity

* Accuracy assessed with appropriate external matrix standards (CRM’s)

• Proficiency of Analysts –Appropriate QC System

Vitamin D

Codex Upper Limit 1.7

ug/100g, RTF Typical Label Claim 1

Codex Lower Limit 0.7

SMP Model Performance Required (no bias, no product uncertainty) CUL to CLL CUL to LC

Max Intermed Precision (RSDI) 7.1 4.2

Max Interlab Reproducibility(RSDR) 12.4 7.2

Max Overall Interlab RSD 14.3 8.3

SMPR Vitamin A - Initial Draft