The international validation of bio- and chemical-analytical screening methods for dioxins and dioxin-like PCBs: The DIFFERENCE project rounds 1 and 2

Talanta 63 (2004) 1169–1182

The international validation of bio- and chemical-analyticalscreening methods for dioxins and dioxin-like PCBs:

the DIFFERENCE project rounds 1 and 2

J. Van Locoa,∗, S.P.J. Van Leeuwenb, P. Roosa, S. Carbonnellea,J. de Boerb, L. Goeyensa, H. Beernaerta

a Scientific Institute for Public Health (IPH), Juliette Wytsmanstraat 14, B-1050 Brussels, Belgiumb Animal Sciences Group, Netherlands Institute for Fisheries Research, P.O. Box 68, 1970 AB, IJmuiden, The Netherlands

Received 26 February 2004; received in revised form 13 May 2004; accepted 13 May 2004

Available online 4 July 2004

Abstract

The European research project DIFFERENCE is focussed on the development, optimisation and validation of screening methods for dioxinanalysis, including bio-analytical and chemical techniques (CALUX, GC-LRMS/MS, GC×GC-ECD) and on the optimisation and validationof new extraction and clean-up procedures. The performance of these techniques is assessed in an international validation study and theresults are compared with the reference technique GC-HRMS. This study is set up in three rounds and is in accordance with the InternationalHarmonized Protocol for Proficiency Studies and the ISO 5725 standard. The results of the first two rounds are very promising in particular forGC-LRMS/MS. The results obtained with this technique were as accurate as the results reported by the labs using the GC-HRMS. The initialresults reported for GC×GC-ECD overestimate the dioxin concentration in the samples. The results reported by the labs using the CALUXtechnique underestimate the total TEQ concentrations in the samples, compared to the GC-HRMS reference method. The repeatability ofthe CALUX is significantly higher than the other screening techniques. It was shown that accelerated solvent extraction (ASE) is a validalternative extraction and clean-up procedure for fish oil and vegetable oil. The results obtained with CALUX and GC-HRMS after ASE areequivalent to the results obtained with the classical extraction and purification procedures.© 2004 Elsevier B.V. All rights reserved.

Keywords: Screening methods; Method validation; Interlaboratory comparison; Dioxins; Dioxin-like PCBs

1. Introduction

The European research project DIFFERENCE (Dioxinsin Food and Feed Reference Methods and New CertifiedReference Materials)[1] is aimed at the development, op-timisation and validation of screening methods for dioxinanalysis, including bio-analytical and chemical techniques(CALUX, GC-LRMS/MS and GC×GC-ECD) and at theoptimisation and validation of new extraction and clean-upprocedures including pressurised liquid extraction (PLE),accelerated solvent extraction (ASE), microwave-assistedextraction (MAE) and supercritical fluid extraction (SFE).

∗ Corresponding author. Tel.:+32 2 642 53 56; fax:+32 2 642 56 91.E-mail address: [email protected] (J. Van Loco).

Furthermore the project will focus on the feasibility test-ing of the production and certification of five high qual-ity certified reference materials (CRMs) for dioxins, furans,indicator PCBs and dioxin-like PCBs in food and animalfeed.

The purpose of the validation protocol in the DIFFER-ENCE project is to ensure that the bio-analytical and chemi-cal analytical screening methods for dioxins and dioxin-likePCBs (dl-PCBs) respond to the EU criteria. Screeningmethods are used to distinguish between compliant andnon-compliant samples. The requirements for analyticalmethods for the official control of dioxins and dioxin-likePCBs in food and feeding stuffs are laid down in the EUcommission directives 2002/69/EC and 2002/70/EC[2,3].The analytical procedures must have a high sensitivity, alow limit of detection and a high accuracy.

0039-9140/$ – see front matter © 2004 Elsevier B.V. All rights reserved.doi:10.1016/j.talanta.2004.05.047

1170 J. Van Loco et al. / Talanta 63 (2004) 1169–1182

This international validation protocol, which is basedon the International Harmonized Protocol for ProficiencyTesting [4], will provide information about the accuracy(trueness and precision), ruggedness, detection capabilityand selectivity of the bio- and chemical-analytical screeningmethods in three rounds. The first round focussed on thegoodness-of-fit of the calibration curve and on the accuracyof the methods. In round 2 the detection capability and se-lectivity were assessed. The robustness and the accuracy ofthe methods were evaluated in round 3. This paper reportsthe results of the first 2 rounds of the validation study.

2. Material and methods

2.1. Test material

The materials that have been prepared for round 1 and2, including details on the origin of the standards used forspiking, are mentioned inTable 1.

2.1.1. Preparation of materialsAll solvent and oil based materials were ampouled. The

amber coloured ampoules (Nederlandse Ampullen Fabriek,Nijmegen, The Netherlands) were used without prior clean-ing, which has been demonstrated to be a safe approach forPCBs and other POPs for the QUASIMEME interlaboratorystudies[5].

Table 1Materials used in round 1 (1–17) and round 2 (18–26) for the evaluation of the screening techniques

Material number Container Material Volume/weight Solvent

1 Ampoule Blank solvent 1 ml DMSO2 Ampoule Standard 2,3,7,8-TCDD: 0.04 ng TEQ ml−1 1 ml DMSO3 Ampoule Standard 2,3,7,8-TCDD: 0.1 ng TEQ ml−1 1 ml DMSO4 Ampoule Standard 2,3,7,8-TCDD: 0.4 ng TEQ ml−1 1 ml DMSO5 Ampoule Standard 2,3,7,8-TCDD: 1.6 ng TEQ ml−1 1 ml DMSO6 Ampoule Standard 2,3,7,8-TCDD: 6.25 ng TEQ ml−1 1 ml DMSO7 Ampoule Blank solvent 1 ml Nonane8 Ampoule Standard 2,3,7,8-TCDD: 0.1 ng TEQ ml−1 1 ml Nonane9 Ampoule Standard 2,3,7,8-TCDD: 0.5 ng TEQ ml−1 1 ml Nonane

10 Ampoule Standard 2,3,7,8-TCDD: 5 ng TEQ ml−1 1 ml Nonane11 Ampoule Standard 2,3,7,8-TCDD: 50 ng TEQ ml−1 1 ml Nonane12 Ampoule Standard 2,3,7,8-TCDD: 100 ng TEQ ml−1 1 ml Nonane13 Ampoule Standard 2,3,7,8-TCDD: 200 ng TEQ ml−1 1 ml Nonane14 Ampoule Quality control sample (QCS), 3 pg dioxin and 3 pg PCB-TEQ/g oil 5 g Vegetable Oil15 Glass jar Milk sample 250 ml NA16 Ampoule Fish oil (herring, close to 4 pg dioxin-TEQ/g oil) 7 ml NA17 Ampoule Clean fish extract of fatty fish (fat removed), equivalent of 5 g fat intake 5 ml Pentane18 Ampoule Blank vegetable oil 5 g Vegetable Oil19 Ampoule Vegetable oil+ 0.2 pg dioxin- and 0.2 pg PCB-TEQ/g oil 5 g Vegetable Oil20 Ampoule Vegetable oil+ 0.75 pg dioxin- and 0.75 pg PCB-TEQ/g oil 5 g Vegetable oil21 Ampoule Vegetable oil+ 1.5 pg dioxin- and 01.5 pg PCB-TEQ/g oil 5 g Vegetable oil22 Ampoule Vegetable oil+ 3.0 pg dioxin- and 3.0 pg PCB-TEQ/g oil (QC-OIL) 5 g Vegetable oil23 Ampoule Vegetable oil+ 6.0 pg dioxin- and 6.0 pg PCB-TEQ/g oil 5 g Vegetable oil24 Ampoule Vegetable oil (see material 22)+ PCB-spike (200 ng/g oil) 5 g Vegetable oil25 Ampoule Vegetable oil (see material 22)+ PCN-spike (10 ng/g oil) 5 g Vegetable oil26 Ampoule Vegetable oil (see material 22)+ PCDE-spike (20 ng/g oil) 5 g Vegetable oil

Material 1 is pure DMSO solvent (Acros, Geel, Belgium).Materials 2–6 have been produced by diluting gravimetri-cally a standard of 2,3,7,8-TCDD (Cambridge Isotope Lab-oratories, Andover, MA, USA) with DMSO. Material 7 ispure nonane solvent (Merck, Darmstadt, Germany). Mate-rials 8–13 have been produced by diluting gravimetricallya standard of 2,3,7,8-TCDD (Wellington, Guelph, Ontario,Canada) with nonane.

Material 14 is a vegetable oil (corn oil) which has beenpurchased in a local super market in the Netherlands (DekaMarkt). Prior to spiking, the levels of dioxins and dl-PCBshave been determined in the oil by RIKILT—Institute forFood Safety, Wageningen, The Netherlands. The spikingprofile was based on a profile of PCDD/Fs and dl-PCBs inherring. A commercial mixture containing all WHO diox-ins and furans was used and additionally 2,3,4,7,8-PeCDF,1,2,3,4,6,7,8-HpCDF, OCDF, 1,2,3,4,6,7,8-HpCDD andOCDD (all obtained from Wellington Laboratories, Guelph,Ontario, Canada) were spiked to resemble the herring profile.The WHO-non-ortho-PCBs (PCB 77, 81, 126 and 169) wereall individually spiked and the WHO-mono-ortho-PCBswere spiked using a standard solution, obtained from RIK-ILT (containing PCB 105, 114, 118, 123, 156, 157, 169and 189), with additional spiking of PCB 105, 118 and 156(Ultra Scientific, North Kingstown, RI, USA). The spikedmilk sample (material 15) was produced by spiking dioxinand dl-PCB congeners to 20 litres of sterilized whole milkwhich had been purchased from a local supermarket in The

J. Van Loco et al. / Talanta 63 (2004) 1169–1182 1171

Netherlands (Deka Markt). The spiking-profile of the diox-ins and dl-PCBs was obtained from Dutch raw milk moni-toring data (RIKILT). All 17 WHO congeners were spikedat the level of interest using a standard solution containingall congeners (Wellington). Furthermore, the following indi-vidual congeners were added to approach the milk congenerprofile: 2,3,4,7,8-PeCDF, OCDF, 1,2,3,4,6,7,8-HpCDD,OCDD. The WHO-non-ortho-PCBs were spiked from stan-dard solutions of the individual congeners (obtained fromRIKILT). The WHO-mono-ortho-PCBs were spiked us-ing a mixture of these PCBs (RIKILT standard solution).Furthermore, the indicator PCBs (PCB 28, 101, 118, 138,153 and 180 (all obtained from Ultra Scientific) have beenadded to the milk for the homogeneity study. To enablequantification, the indicator PCBs were spiked at higherconcentration levels. Due to this fact, PCB 118 had beenadded twice: once as a mono-ortho in the RIKILT standardsolution and again as an indicator PCB at higher level.Therefore, the second addition resulted in a somewhat un-balanced mono-ortho-PCB TEQ and the total TEQ withPCB 118 being the predominant congener (with a con-centration of 4.7 pg TEQ/g fat for PCB-118 on a total of5.1 pg PCB TEQ/g fat). The crude fish oil sample (mate-rial 16) was obtained as a remainder of a project on theupgrading herring by-products (e.g. heads).[6] The her-ring was caught in May 2000, west of the Shetland Islands(60.50◦N/03.00◦W). The oil was filtered over 0.45�mpaper filter (Schleicher & Schuell, Dassel-Relliehausen,Germany) to remove solid particles and subsequentlyampouled.

The clean fish extract (CFE) (material 17) was producedby extracting a pooled eel sample from Dutch freshwaterlocations. After extraction, portions of 5 g fat were cleanedover acidic silica columns (48 g silica per column). Thesolvent was evaporated and the residue was redissolvedin n-heptane (Promochem, Wesel, Germany, Picogradepurity). Twenty-five ampoules were produced containing5 ml of clean fish extract (CFE) which is equivalent to 4 gof fat.

The blank vegetable oil (material 18) is of the same ori-gin as material 14 but without spiking the dioxins and thedl-PCBs. The spiked vegetable oils (materials 19–23) wereprepared by analogy with material 14. Their spiking levelsare given inTable 1. The materials 24, 25 and 26 havealso been prepared from material 14. An in-house standardsolution of 29 PCBs (including the mono-ortho-PCBs 105,118 and 156) was spiked to the required level of 200 ng/goil (material 24). Material 25 was prepared by additionalspiking of polychlorinated naphthalene’s (PCN) 27, 28, 36,52, 54, 67, 68, 71, 53, 66, 73 and 74 (Wellington Labo-ratories) to a total level of 10 ng/g oil. Material 26 wasprepared by spiking it with a polychlorinated diphenylether (PCDE) standard solution (Cambridge Isotope Lab-oratories). The standard contained native and 13C-labeledmono-decaCDEs at a level of 20 ng/g oil (sum of allPCDEs).

2.1.2. Homogeneity study of the materials used in rounds1 and 2

Homogeneity tests have been carried out with the spikedmilk sample and the fish oil sample in order to deter-mine whether the materials are homogeneous. The standardsolutions, the quality control oil (QC-oil) and the cleanfish extract of round 1 and also the spiked vegetable oilsof round 2 were assumed to be homogeneous in all theampoules.

The homogeneity testing was carried out according toBCR and ISO guidelines[7]. For between homogeneitytesting, the seven indicator PCBs were analysed in 10randomly selected lots out of the complete batch. Thewithin homogeneity testing was carried out by five repli-cate analyses in one sample. Statistics (ANOVA) werecarried out to compare between and within sample vari-ances. Only results of PCB analysis that were above thelimit of quantification were used for determination of thehomogeneity.

The reasoning behind using the indicator PCBs for anal-ysis of the homogeneity is based on the fact that at very lowlevels of PCDD/Fs it is likely that one would determine thewithin laboratory method variance (typically 5–20% usingisotope dilution), reflecting the competence of the laboratoryto analyse at very low levels instead of the (in)homogeneityof the sample. A possible intrinsic heterogeneity will there-fore possibly not be detected[8]. However, at the concen-tration level of the PCBs, the within laboratory methodvariance can be very low (<3%–5%), which improvesthe power of the method to detect heterogeneity in thematerial.

Furthermore, it is expected that the PCDD/Fs will behavephysically similar to PCBs and are therefore similarly dis-tributed, also at lower levels, in the sample.

The samples were analysed with GC-ECD. The instru-mental variance was 0.6–2.7%. For both the milk sampleand the herring oil sample the variances were not signifi-cantly different between the between-homogeneity samplesand the within-homogeneity samples. The RSDs were allbelow 6%, except for the PCBs 101 and 138 in herring oilwhich showed a RSD from 7.2 to 20.2%. The reason forthese elevated RSD values is not known. As the RSDs ofthe indicator PCBs were almost all below 6% and sincethe sample intake for dioxin analysis is typically 20 timeshigher than the sample intake used during the homogene-ity study, it is unlikely that possible inhomogeneity willcontribute to the variance resulting from the interlaboratorystudies. It was concluded that the samples can be consid-ered homogeneous and are suitable for the interlaboratorystudy.

2.2. Validation protocol

The international validation protocol, which is based onthe International Harmonized Protocol for Proficiency Test-ing, will provide information about the trueness, precision,


ruggedness, detection capability and selectivity of the bio-and chemical-analytical screening methods. This validationprotocol is executed in three rounds. The first round primar-ily focussed on the goodness-of-fit of the calibration curveand provided the first data concerning repeatability and re-producibility of the screening methods. The objective ofthe second round was to assess the detection capability andselectivity of the method. Information about the detectioncapability of the methods was obtained with the proceduresdescribed in the ISO 11843-2[9]. Furthermore the accuracyof the results obtained with the methods applied could beinvestigated, because the exact amount added to the samplesis known. Round three will provide more data on the preci-sion and robustness of the methods. The ISO 5725 “Accu-racy (trueness and precision) of measurement methods andresults”[10] is used as guidance to evaluate the accuracy ofthe bio- and chemical-analytical screening methods. In par-ticular the repeatability, within-lab reproducibility and thereproducibility of the methods were assessed. The protocolis detailed inTable 2. The information obtained during thethree rounds will be used to gauge the ruggedness of theanalytical methods. During the whole validation process aquality control solution is used to assure the validity of thedata.

2.3. Statistical evaluation of the results

2.3.1. Proficiency testing scoring techniquesThe results were evaluated according to the international

harmonized protocol for proficiency testing of chemical an-alytical laboratories[4]. It determines that for the quanti-tative results of the laboratories thez-scores are calculatedaccording to the following equation:

z = x − X

σp

wherex: lab result,X: assigned value,σp: target value forthe standard deviation.

The target value for the standard deviation can be deter-mined via the (modified) Horwitz function[11], but pref-

Table 2validation protocol of bioanalytical and chemical analytical screening methods

erence is given to the use of the acceptance criteria in theEuropean directives 2002/69/EC and 2002/70/EC[2,3]. Thestandard deviation is therefore derived from:

σp = CVmax

100× X

With CVmax = 30%.The CVmax is based on the acceptance criteria for screen-

ing methods as laid down in the EU decision 2002/69/EC[2]. This approach is in close agreement with the (modified)Horwitz function as presented by Thompson[11]. In thiscaseσp is defined as:σp = 0.22X.

The assigned value (X) is calculated using the added con-centration (standard solutions and QC-oil) or using the me-dian of the results obtained from the three laboratories usingthe GC-HRMS.

The sum of the squaredz-scores (SSZ) is calculated togive a composite score of the individual results for eachlaboratory.

SSZ=∑

z2

The SSZ is evaluated by comparing it with criticalχ2 valueswith n degrees of freedom (where n is the number of scores)and a probability of 0.95 and 0.997 which corresponds withz-scores of 2 and 3.

2.3.2. Method validation parametersThe repeatability (r), the within-lab reproducibility

(W) and the reproducibility (R) are calculated usinga two-factor nested ANOVA as explained in the ISO5725-3 [10]. The sources of variation are given in theTable 3.

The repeatability, the within-lab reproducibility and thereproducibility variances are as follows for a balanced nesteddesign:

S2r = MSrepl

S2W = S2

r + S21

S2R = S2

r + S21 + S2

0


Table 3ANOVA-table explaining the contribution of the variance of the laboratories, the analytical runs and the replicate measurements to the total variance

Source Sum of squares (SS) Degrees of freedom (d.f.) Mean square (MS) Expected mean square (EMS)

Lab SSL nLab − 1 MSLab σ2 + 2σ21 + 6σ2

0Run SSR nLab × nRun − nLab MSRun σ2 + 2σ2

1Replicate SSE n − nLab × nRun MSrepl σ2

Total SSTOT n − 1 = nLab × nRun × nrepl − 1 MSTOT

nLab: number of participating laboratories;nRun: number of analytical runs (=3); nrepl: number of replicates per run (=2).

with

S20 = 1

6(MSLab − MSRun)

S21 = 1

2(MSRun − MSrepl)

The repeatability and the within-lab reproducibility vari-ances for each lab are analogously derived using asingle-factor design.

The apparent recovery[12] is estimated by dividing themean of the lab results through the reference value and thecoefficient of variation (CV) is obtained by dividing, re-spectively, theSr, SW and SR through the mean of the labresults.

2.3.3. Detection capabilityThe methodology for the determination of the minimum

detectable value (MDV) in the case of a linear regressionmodel (LRM), has been extensively described in the ISO11843-2 [9]. Under the assumption of linearity, normal-ity, independence and homoscedasticity, the MDV (= xd) isgiven by:

MDV = xd = δSy

b

√1

K+ 1

IJ+ x̄2

J∑

(xi − x̄)2

In case of weighted linear regression models (WLRM), theMDV is given by:

MDV = xd

= δ

b

√S2

xd+

[(1

J∑

wi

)+ x̄2

w

J∑

wi(xi − x̄w)2

]S2

y

with b: estimate of the slope;δ: non-centrality parameter;I: number of reference states (= number of replicates perconcentration for the spiked or reference samples),i = 1,2,. . . , I; J: number of preparations for the reference states(= number of concentrations for the spiked or referencesamples);Sy: standard error of the estimate;Sxd: residualstandard deviation atx = xd; wi: applied weights (wi = 1 inthe case of unweighted regression);xi: spiked concentration;x̄: mean of the concentrations and

x̄w =∑

wixi∑wi

The weights are calculated by taking the reciprocal ofthe variance function. The variance function [VARi =

Table 4Overview of the participating laboratories and the used techniques

Lab Method Remarks

A CALUXB ASE + GC-HRMSC GC-HRMSD CALUXE CALUX TCCD calibration curve used for

quantificationE∗ CALUX Results quantified by comparison with the

value of a reference sampleF GC-HRMSG GC-LRMS/MSH ASE+CALUXI GC×GC-ECDI∗ GC×GC-ECD Reprocessed data after the initial

presentation of the results of the validationstudy in Brussels, February 2003

J GC-HRMSK GC×GC-ECD

(c + dxi)2] is estimated by a linear regression of the stan-dard deviations versus the concentration.

2.4. Analytical methods

The following methods[13–15]are evaluated in the vali-dation study: chemical activated luceferase gene expression(CALUX), multi-dimensional GC with electron capture de-tection (GC×GC-ECD), GC with low resolution mass spec-trometry (GC-LRMS/MS), and GC with high resolution MS(GC-HRMS). Accelerated solvent extraction (ASE)[16] isevaluated as a combined extraction and clean-up technique.Details on the methods used are provided at the web-site ofthe DIFFERENCE project[17]. Table 4gives an overviewof the participating laboratories and the techniques used(Table 4).

3. Results and discussions

3.1. Results of round 1

The aim of the first round was to test the goodness-of-fitof the calibration curve by analysis of standards with undis-closed concentrations of TCDD inn-nonane or in DMSO.Furthermore, information on repeatability and reproducibil-ity was obtained from the analysis of fish oil and a spikedmilk sample. A quality control sample was analysed eachround to check the performance of the methods.


3.2. Standard solutions

The aim of the standard solutions with undisclosed con-centrations of TCDD was to test the goodness-of-fit of thecalibration curve. The standards were analysed in duplicateby direct injection in the GC. However, some labs have di-luted the standard solutions. The relative deviations of theGC-method results are presented inFig. 1.

Five different concentrations of TCDD in DMSO and ablank DMSO solution were prepared at RIVO. The standardswere analysed in duplicate by direct addition to the cellmedium. The CALUX bio-assay results are all, except one,positively biased. A graphical representation of the relativedeviation of the results for the different standard solutionscan be found inFig. 2.

Fig. 1. Relative deviation between measured (GC screening methods) and the assigned value for the standard solutions in nonane (A, B, C, E, F and G).

It can be concluded that the GC-method results arebetter than the CALUX bio-assays. However, only oneGC×GC-ECD lab (I) and one GC-LRMS/MS lab (G)have provided results. The other GC labs have used theGC-HRMS reference technique.

3.3. Quality control sample

The quality control sample is a vegetable oil spiked with amixture of approximately 3 pg dioxin TEQ/g oil and approx-imately 3 pg PCB TEQ/g oil. The samples were analysedonce as unknown samples (lab J has analysed the sample induplicate).

The z-scores are visualized inFig. 3. Note that theCALUX labs D and E only have reported total TEQ val-


Fig. 2. Relative deviation between measured (CALUX-methods) and the assigned value for the standard solutions in DMSO (A, C, D, E and F).

ues. These labs are not separating the dioxins from thePCB fraction. The total TEQz-scores are satisfactory forall the labs, except lab K. The major contribution of theerror can be assigned to the dioxin TEQ result. Also theother GC×GC-ECD lab (I) has az-score larger than 2 forthe dioxin TEQ results. After the first presentation of theresults in Brussels, February 2003, this lab has reprocessedits data, resulting in much betterz-scores for the dioxinTEQ results (lab I∗).

3.4. Spiked samples

3.4.1. MilkThe aim of the milk sample was to provide data on the

within-laboratory reproducibility and repeatability of a realmatrix sample.

The milk samples were prepared by spiking them witha mixture of dioxins and dl-PCBs at a concentration of

10.23 pg total TEQ/g fat. The milk samples were analysedby the participants in duplicate on three different analyticalruns with different equipment and different operators when-ever feasible. The data are obtained with CALUX (3 labs),GC-HRMS (3 labs), GC-LRMS/MS (1 lab), GC×GC-ECD(2 labs).

A large variation in the reported results is noticed. Themean results reported by the GC-HRMS labs vary from 8.7to 14.1 pg total TEQ/g lipid. The milk sample is a spikedsample and it appeared that the PCB-118 congener wasspiked in an unusual high concentration of 4.7 pg TEQ/glipid. This resulted in calibration problems for most of theGC methods.

The CALUX labs (A, D and E) have reported the lowesttotal TEQ concentrations for the milk sample. However, incomparison with the GC-methods, the CALUX results arenot corrected for recovery. All the GC methods are usinginternal standards or isotopic dilution to correct for the ex-


Fig. 3. z-scores for the total, dioxin and PCB-TEQ of the quality control oil.

Fig. 4. Total TEQ SSZ-scores for the milk samples.


Table 5Statistical summary of the total TEQ results (upperbound) for the milk sample

Lab

A C D E F G I I∗ J KCALUX GC-HRMS CALUX CALUX GC-HRMS GC-LR MS/MS GC×GC-ECD GC×GC-ECD GC-HRMS GC×GC-ECD

Number 6 6 6 6 6 6 6 6 6 6Average pg

TEQ/g fat)7.61 14.06 3.95 3.93 9.61 10.83 15.32 15.17 8.71 19.89

Sr (pgTEQ/g fat)

1.14 0.44 0.59 0.66 0.61 0.54 1.05 1.28 0.62 1.99

CVr 15.0% 3.1% 14.9% 16.7% 6.4% 5.0% 6.9% 8.4% 7.1% 10.0%SW (pg

TEQ/g fat)1.35 0.44 1.52 1.08 1.03 0.54 1.20 1.28 0.64 2.83

CVW 17.7% 3.1% 38.5% 27.6% 10.7% 5.0% 7.8% 8.4% 7.3% 14.2%

traction yield. The highest results were reported by the labsusing the GC×GC-ECD technique. The SSZ-scores are vi-sualized inFig. 4. (The SSZ-score is a combination score ofthe 6 individualz-scores.) The SSZ-scores for the CALUXlabs D and E and the GC×GC-ECD labs I and K are unsat-isfactory.

A summary of the statistical evaluation of the lab resultsis given inTable 5. In this Table the mean, repeatability andwithin-lab reproducibility standard deviation (Sr and SW)and coefficient of variation are given. Normality of the re-sults for each lab is evaluated withχ2 goodness-of-fit andShapiro–WilksW-tests. Normality was not rejected and out-liers are not detected with Grubbs’ test at the 99% confi-dence level.

The precision of the analytical methods is assessed byevaluating the repeatability and within-lab reproducibilitystandard deviation and coefficients of variation. Note thatthe coefficients of variation for the CALUX methods (labsA, D and E) are significantly higher than the CV’s for theGC screening methods (labs G, I and K). One might expectthat the repeatability CV (CVr) is between 1/2 and 2/3 of theHorwitz CV [18]. Using the modified Horwitz equation[11]the CVr should be between 11 and 14.7%. All the CALUXlabs have reported higher CVr’s. The criterion that CV<

30% for screening methods[2] is violated by lab D.

3.4.2. Fish oilTo assess the within-lab repeatability and reproducibil-

ity the fish oil samples are analysed in duplicate in threedifferent analytical runs. The analyses are performed usingdifferent equipment and different operators whenever fea-sible (lab H has only reported five results and lab K onlythree results). The data are obtained with CALUX (3 labs),GC-HRMS (3 labs), GC-LRMS (1 lab), GC×GC-ECD (2labs). The samples were also analysed by accelerated solventextraction (ASE) followed by a detection and quantificationof the results with GC-HRMS and CALUX.

Box and whisker plots of the total TEQ upperbound resultsfor the samples are presented inFig. 5. The values rangebetween 1.94 and 15.54 pg total TEQ/g fat. It was observedthat the results obtained with CALUX (labs A, D, E, H) Fig. 5. Visual representation of the fish oil (upperbound) data in box-plots.


Fig. 6. Total TEQ SSZ-scores for the fish oil.

are significantly lower than the results reported by the GCscreening labs (G, I, I∗ and K), except for lab A. Lab A hasa much larger variance than the other laboratories.

The labs D, E and H, all using the CALUX method, havez < 2 values for some of the total TEQ results. Thez-scoresfor lab A, which is also applying the CALUX methodol-ogy, are satisfactory for the total TEQ results, but not forthe dioxin or PCB TEQ results. Lab K (GC×GC-ECD) hasreported too high values for the total TEQ and the dioxinTEQ results. The other laboratories have all satisfactory re-sults. The ASE is a valid dioxin and PCB extraction andpurification alternative, because thez-scores of lab B areall satisfactory. An overall score of the labs for this sam-ple is given by the SSZ (Fig. 6). The overall scores for thelabs D, E, H and K are unsatisfactory. The labs D, E and Hare all using the CALUX methodology. As explained ear-lier in the text, the results of the CALUX labs are not cor-rected for recovery, while the GC methods are. Assumingthat these labs have a recovery of 70% a SSZ of 21.2 forlab D and a SSZ= 13.4 for lab E will be obtained. Theseresults are still not satisfactory. Apparently, recovery is notthe only factor that influences the CALUX results. It is alsoknown[19] that CALUX underestimates the PCB TEQ in asample. This phenomenon can be illustrated with the PCBTEQ results of lab A, another CALUX lab. The PCB TEQresults are, compared to the GC-HRMS results, significantlylower.

A summary of the statistical evaluation of the lab resultsis given in Table 6. Normality of the results for each labis evaluated and was not rejected at the 99% confidencelevel. Three labs (A, D and H) have a CVW > 30%. Theseare all CALUX labs. The maximum CVr (=14.7%)[18] isexceeded by the labs D, E and H. A more extensive vari-ance components analysis for the CALUX screening meth-ods (labs A, D and E) is performed according to the ISO5725 standard. The method ASE+ CALUX (lab H) was notincluded in the evaluation. The between lab-reproducibility

and repeatability CV for the CALUX method is 79.0% and17.2%, respectively.

The differences between the participating laboratories arestatistically evaluated by applying a one-way ANOVA onthe total TEQ results followed by the Bonferroni test[20].The ANOVA decomposes the variance of TOT TEQ (pgTEQ/g fat) into two components: a between-lab componentand a within-lab component. In ANOVA the between-laband the within-lab component are compared via anF-test(F = 50.2). Since theP-value of theF-test is less than 0.05there is a statistically significant difference between themean TOT TEQ (pg TEQ/g fat) from one lab to another atthe 95.0% confidence level. The ASE extraction/purificationtechnique can also be evaluated with this analysis. Theresults of lab H (ASE+ CALUX) are not significantly dif-ferent from the results of lab E (CALUX) and the resultsof ASE + GC-HRMS (lab B) are not significantly differentfrom the results of the labs F, G, J and C (GC-HRMS andGC-LRMS/MS). Since the results are situated within thesame homogeneous group (Bonferroni test) it can be con-cluded that ASE is equivalent compared with the classicextraction/purification techniques for fish oil. It should bestressed that in the classical extraction/purification proce-dures fish oil is normally analysed without an extraction step.

3.5. Detection capability and selectivity

The aim of round 2 was to determine the detection ca-pability and selectivity of the methods. During this roundvegetable oil samples spiked with a mixture of dioxins anddl-PCBs at a concentration of 0, 0.4, 1.5, 3, 6 and 12 pgtotal TEQ/g fat are analysed under within-lab reproducibil-ity conditions. This means that they are analysed onceduring four independent analytical runs, by different oper-ators and using different equipment whenever feasible. Theprocedures described in the ISO 11843-2[9] were used togauge the detection capability of the analytical techniques.


Tabl

e6

Sta

tistic

alsu

mm

ary

ofth

eto

tal

TE

Qre

sults

(upp

erbo

und)

for

the

fish

oil

Lab

AB

CD

EF

GH

II∗

JK

CA

LUX

AS

E+

HR

MS

GC

-HR

MS

CA

LUX

CA

LUX

GC

-HR

MS

GC

-LR

MS

/MS

AS

E+C

ALU

XG

C×G

C-E

CD

GC×

GC

-EC

DG

C-H

RM

SG

C×G

C-E

CD

Num

ber

66

66

66

65

66

63

Ave

rage

(pg

TE

Q/g

fat)

10.8

910

.25

11.0

43.

103.

938.

919.

083.

1414

.00

12.1

89.

8419

.03

S(p

gT

EQ

/gfa

t)3.

530.

600.

310.

971.

010.

740.

571.

011.

020.

590.

252.

82C

V32

.4%

5.8%

2.8%

31.3

%25

.8%

8.3%

6.3%

32.0

%7.

3%4.

8%2.

5%14

.8%

S r(p

gT

EQ

/gfa

t)1.

350.

580.

300.

950.

660.

830.

560.

490.

340.

250.

27−

CV

r12

.4%

5.6%

2.7%

30.7

%16

.7%

9.3%

6.1%

15.5

%2.

4%2.

1%2.

8%−

S W(p

gT

EQ

/gfa

t)3.

890.

600.

320.

981.

080.

830.

571.

101.

130.

640.

272.

82C

VW

35.7

%5.

9%2.

9%31

.5%

27.6

%9.

3%6.

3%34

.9%

8.1%

5.3%

2.8%

14.8

% It was shown by Van Loco et al.[21], that heteroscedas-ticity of the data has a major impact on the detectioncapability. Therefore, heteroscedasticity of the variancewas evaluated and corrected for by assuming that the stan-dard deviation is linearly dependent on the concentration.The variance function VARi = (c + dXi)2 is estimated bya linear regression of the standard deviations versus theconcentration.

The detection capability data, here expressed as MDV,are summarized inTable 7. Results below the lowest spikedconcentration are expressed as “<”, because the variancefunction below this concentration is obtained by extrapola-tion. This is the case for labs C and G. Their MDV is lowerthan the 0.367 pg total TEQ/g oil concentration in the lowestspike. One should not conclude that the detection capabil-ity of the GC-LRMS/MS is better than the GC-HRMS sincethe experiments on the GC-LRMS/MS were performed un-der optimal conditions, while the GC-HRMS is used in rou-tine conditions. The lowest MDV of the CALUX methodsis 0.9 pg total TEQ/g oil, which is close to the highest MDVof the GC-HRMS labs.

It was not possible to provide a good estimate for theMDV for the labs D and H. Both labs are using the CALUXtechnique. An explanation can be found in the low cor-relation coefficient for their calibration lines: respectively0.837 and 0.811. The correlation coefficients of the otherlaboratories were all above the required 0.95[2,3].

The apparent recovery of the CALUX methods was evalu-ated. The apparent recovery is defined as the observed valuederived from an analytical procedure by means of a cali-bration graph divided by the reference value[12]. The ap-parent recovery for the CALUX labs D, E and H is verylow (18–44%). The apparent recovery of the CALUX lab Ais function of the concentration, since the calibration graphdoes not pass through zero. At the lower concentrations therecovery is larger than 100%. At a concentration around4.5 pg TEQ/g the recovery is 100% and at higher concentra-tions the recovery is lower than 100%. Hence, the methodbias is positive at lower concentrations and negative at the

Table 7Detection capabilities of the analytical methods for dioxins and dl-PCB’sin vegetable oil

Lab Method MDV (pg TEQ/g oil)

A CALUX 3.83B ASE + GC-HRMS 0.57C GC-HRMS <0.37D CALUX 7.79a

E CALUX 0.90E∗ CALUX 1.04F GC-HRMS 0.50G GC-LRMS <0.37H ASE+CALUX 4.86a

I GC×GC-ECD <1.42J GC-HRMS 0.88

a The correlation coefficient of the dose-response curves is lower than0.85.


Fig. 7. Within-lab reproducibility of the CALUX labs as function of the concentration.

higher concentrations. This was also observed for the otherCALUX. However, this has not been statistically confirmed.The apparent recovery for the GC×GC-ECD lab I is around108%. The apparent recoveries for the GC-LRMS/MS andGC-HRMS are approximately 100%. At very low concen-trations the GC-methods are positively biased. This is prob-ably caused by the presence of traces in the blank vegetableoil, since the vegetable oil is off-the-shelf purchased withoutfurther purification.

The precision of the method as function of the concen-tration is shown in theFigs. 7 and 8. In all cases the rela-tive standard deviation (RSD) decreases by higher concen-trations. The RSD of the CALUX technique is higher thanthe GC screening methods (GC×GC and GC-LRMS/MS).The RSD of the CALUX labs is around 20% at the higherconcentration range. The RSD of the GC-labs is below 10%at the higher concentration range.

The selectivity of the screening methods is evaluatedby spiking PCB, PCN and PCDE to the 6 pg total TEQ/gvegetable oil. The influence of possible interferences isevaluated with ANOVA[20]. No interferences are detected.However due to an error by the preparation of the in-terference samples, the PCB interference spike contained2.7 pg TEQ/g oil extra of mono-ortho-PCBs (PCB 105,118 and 156) in comparison with the reference spike. TheCALUX methods could not detect this additional amountof PCB TEQ in the sample. This confirms that the CALUXtechnique has a weakness in detecting and quantifyingmono-ortho-PCBs. This can be easily explained by the verylow REP (Relative Potency) values of the mono-ortho-PCBs[19]. When the REP-values are taken into account instead

of the TEF-values, the additional amount of PCB TEQ isonly 0.07 pg PCB TEQ/g.

The GC-screening methods all detected the additionalamount of PCBs in the sample. However, the resultsreported by the labs B and I were significantly higher thanthe results of the other GC labs. There was also a slight,but statistically significant, increase found by lab C (usingGC-HRMS) for the total TEQ concentration of the PCDE in-terference sample. No explanation was found for this PCDEinterference.

4. Conclusions

Excellent results were reported with the GC-LRMS/MSmethod. Based on these results, this method would be a goodcandidate screening method for the analysis of dioxins andPCBs. The results obtained with this technique were as accu-

Table 8Overview of the performance of the different screening techniques fordioxins and dl-PCBs

Parameter GC-HRMS CALUX GC-LRMS/MS GC×GC-ECD

Goodness-of-fit ++ + ++ +Repeatability ++ + ++ +Within-lab

reproducibility++ +/− ++ +

Accuracy ++ +/− ++ +/−Detection

capability+ +/− ++ +/−

Selectivity ++ +/− ++ +


Fig. 8. Wthin-lab reproducibility for the GC labs as function of the concentration.

rate as the results reported by the labs using the GC-HRMStechnique. All thez-scores are satisfactory. The repeatabilityand the within-lab reproducibility of the method are below7%. The MDV is below 0.36 pgTEQ/g oil. However, thisconclusion is only based on the results of one lab.Table 8shows an overview of the results of round 1 and 2.

The CALUX results were also promising, taken into con-sideration that the results of the CALUX technique are notcorrected for recovery, while the results obtained by theGC-labs were all corrected for recovery by the use of inter-nal standards or isotopic dilution. Labs D and E have accu-racy problems with the total TEQ determination in the milkand fish oil sample. The repeatability of the CALUX tech-nique is around 15% for the milk and the fish oil samples.The within-lab reproducibility is higher (up to 38%). TheCALUX technique has all the features to become an excel-lent screening technique if they can find a way to correctfor the bias and can limit the variability of the results. Itshould be mentioned that a reduced variation will automati-cally lead to a much lower minimum detectable value. Lab E

has already a MDV (=0.9 pg TEQ/g oil) close to the MDVof one of the GC-HRMS labs. A second issue that shouldbe addressed by the CALUX labs is the sensitivity of themethod for dl-PCBs. The low REP values for the dl-PCBscause an underestimation of the PCB TEQ compared to theGC-HRMS reference method.

The initial results reported by lab I and K (GC×GC-ECD)were not very good. The results submitted by these labstend to overestimate the dioxin concentration in the samples.After the initial presentation of the results in Brussels, lab Ihas reprocessed their data resulting in satisfactoryz-scores.The MDV of lab I is below 1.4 pg TEQ/g oil.

Accelerated solvent extraction (ASE) is a valid alternativeextraction and clean-up procedure for fish oil and vegetableoil. The results obtained with CALUX and GC-HRMS afterASE are equivalent to the results obtained with the classicalextraction and purification procedures.

The next round of the validation study will focus on ro-bustness, trueness, repeatability and reproducibility of themethods, which will be assessed through analysis of differ-


ent food and feed matrices. The time and related costs of thevarious techniques will also be considered in the next phaseof the project.

Acknowledgements

The European Commission is gratefully thanked forsupporting the DIFFERENCE project (contract G6RD-CT-2001-00623). The following partners and their co-workersand subcontractors are gratefully acknowledged for theirhard work and stimulating contribution to the project: D.Fraise (CARSO), R. van Cleuvenbergen and G. Schoeters(VITO), E. Björklund (Lund University), J. Rivera (IIQAB-CSIC), J. Santos (University of Barcelona), R. Hoogenboomand W. Traag (RIKILT), P. Haglund (Umeå University).

References

[1] S.P.J. van Leeuwen, J. Van Loco, J. de Boer, Organohalogen Com-pounds 60 (2003) 267–270.

[2] Commission Directive 2002/69/EC of 26 July 2002 laying downthe sampling methods and the methods of analysis for the officialcontrol of dioxins and the determination of dioxin-like PCBs infoodstuffs.

[3] Commission Directive 2002/70/EC of 26 July 2002 establishingrequirements for the determination of levels of dioxins and dioxin-likePCBs in feedingstuffs.

[4] M. Thompson, R. Wood, J. AOAC Int. 76 (1993) 926–940.[5] J. de Boer, J. van der Meer, L. Reuthergardh, J.A. Calder, J. AOAC

Int. 77 (6) (1994) 1411–1422.

[6] I. Aidos, A. van de Padt, R.M. Boom, J.B. Luten, J. Agric. FoodChem. 48/49 (2001) 3697–3704.

[7] BCR Information—Guidelines for Feasibility Studies on CertifiedReference Materials, European Commission, Directorate-General forResearch EUR 20574 EN, 2002.

[8] J. de Boer, Marine Pollut. Bull. 35 (1997) 84–92.[9] ISO 11843-2, Capability of Detection—Part 2: Methodology in the

Linear Calibration Case, ISO, Geneva, 2000.[10] ISO 5725 1-6, Accuracy (Trueness and Precision) of Measurement

Methods and Results, ISO, Geneva, 1994.[11] M. Thompson, Analyst 125 (2000) 385–386.[12] D.T. Burns, K. Danzer, A. Townshend, Pure Appl. Chem. 74 (11)

(2002) 2205–2211.[13] I. Van Overmeire, G.C. Clark, D.J. Brown, M.D. Chu, W.M. Cooke,

M.S. Denison, W. Baeyens, S. Srebrnik, L. Goeyens, Environ. Sci.Policy 4 (2001) 345–357.

[14] C. Danielsson, K. Wiberg, P. Korytar, J. de Boer, P. Haglund,Organohalogen Compounds 60 (2003) 395–398.

[15] F.J. Santos, M. Ábalos, J. Malavia, E. Abad, J. Rivera, M.T. Galceran,Organohalogen Compounds 60 (2003) 452–455.

[16] S. Sporring, K. Wiberg, E. Björklund, P. Haglund, OrganohalogenCompounds 60 (2003) 1–4.

[17] http://www.dioxins.nl.[18] D.L. Massart, B.G.M. Vandeginste, L.M.C. Buydens, S. De Jong, P.J.

Lewi, J. Smeyers-Verbeke, Handbook of Chemometrics and Quali-metrics: Part A, Elsevier, Amsterdam, 1997.

[19] D.J. Brown, M. Chu, I. Van Overmeire, A. Chu, G.C. Clark,Organohalogen Compounds 53 (2001) 211–214.

[20] J. Neter, M.H. Kutner, C.J. Nachtsheim, W. Wasserman, AppliedLinear Statistical Models, fourth ed., WCB/McGraw-Hill, 1996.

[21] J. Van Loco, V. Hanot, G. Huysmans, M. Elskens, J.M. Degroodt,H. Beernaert, Anal. Chim. Acta 483 (2003) 413–418.

http://www.dioxins.nl

Documents

The international validation of bio- and chemical-analytical screening methods for dioxins and dioxin-like PCBs: The DIFFERENCE project rounds 1 and 2