Upload
others
View
7
Download
0
Embed Size (px)
Citation preview
1
Interlaboratory test performance study (TPS) for the evaluation of molecular methods
to detect Xylella fastidiosa in the vector Philaenus spumarius
Ref.: 17-XFAST-EU
May 2017-February 2018
2
CONTENT:
1 GENERAL INFORMATION ................................................................................................................................ 3
1.1 Objectives ................................................................................................................................................ 3
1.2 Organizers ................................................................................................................................................ 5
1.3 Participating Laboratories ....................................................................................................................... 5
1.4 Documents and instructions .................................................................................................................... 7
1.5 Timeline of the test performance study .................................................................................................. 7
2. PANEL OF EXPERIMENTAL SAMPLES ............................................................................................................. 7
3. DIAGNOSTIC PROCEDURES EVALUATED ........................................................................................................ 9
4. ANALYSIS OF THE RESULTS .......................................................................................................................... 15
5. RESULTS ....................................................................................................................................................... 17
5.1 Homogeneity and stability..................................................................................................................... 17
5.1.1 Homogeneity .................................................................................................................................. 17
5.1.2 Stability ........................................................................................................................................... 21
5.2 RESULTS OF THE TPS .............................................................................................................................. 23
5.2.1 On spiked insects ............................................................................................................................ 23
5.2.2 On natural insects ........................................................................................................................... 38
6. SUMMARY OF THE OVERALL VALUES RECOVERED FOR EACH PERFORMANCE CRITERIA .......................... 41
7 DISCUSSION AND CONCLUSION ................................................................................................................... 42
8. REFERENCES ................................................................................................................................................. 44
8. ACKNOWLEDGMENTS.................................................................................................................................. 44
Annex I ............................................................................................................................................................. 45
Annex II ............................................................................................................................................................ 46
Annex III ........................................................................................................................................................... 51
Annex IV ........................................................................................................................................................... 55
This report has been prepared based on the analyses of the results received by all participant
laboratories.
Authors:
Bruno Legendre, Anses
Dimitri Molusson, Anses
Valerie Olivier, Anses
Aude Chabirand, Anses
Françoise Poliakoff, Anses
3 1 GENERAL INFORMATION
1.1 Objectives The test performance study (TPS) is a way in which the performances of one or more methods are
assessed.
For this test, standardized samples were prepared with known status regarding the presence of the target
pathogen. In addition naturally infected samples were also used.
These were sent out to participating laboratories that analyze them using the provided protocols. The
organizer analyzed the results and provided a report detailing all participants’ results in confidential
manner together with actual sample status.
This test performance study aims to assess the efficiency and accurateness of different molecular methods
used for the detection of the bacterium Xylella fastidiosa into the insect vector Philaenus spumarius. The
study is part of the research activities aiming at implementing and validating reliable and sensitive
detection procedures for X. fastidiosa in the framework of the following ongoing European projects:
- EUPHRESCO project (2015-F-146) “Harmonized protocol for monitoring and detection of Xylella
fastidiosa in its host plants and its vectors”
- H2020 “POnTE – Pest Organisms Threatening Europe (635646)” – Work package 4: “Implementation
and validation of diagnostic kits for early and rapid detection of target pathogens in host plants and
vectors”
This test performance study has been organized in accordance with the EPPO 7/122 guideline and the
following performance criteria have been considered in the data analyses:
Diagnostic sensitivity: proportion of positive samples giving a positive result.
Diagnostic specificity: proportion of negative samples giving a negative result.
Accuracy: the accuracy is the closeness of agreement between a test result and the accepted
reference value.
Repeatability (or accordance): level of agreement between replicates of a sample tested under the
same conditions.
Reproducibility: ability of a test to provide consistent results when applied to aliquots of the same
sample tested under different conditions (time, persons, equipment, location, etc.).
4
Limit of detection (analytical sensitivity): Smallest amount of target that can be detected reliably in matrix.
The diagnostic methods evaluated during this TPS included several molecular approaches described
in the EPPO diagnostic protocol 7/24 (2). The detailed laboratory procedures tested in the different
laboratories are reported in the technical sheet in Annex I. The samples were processed either
manually or, according to the kit or protocol, using automatized platforms.
IMPORTANT NOTES: In order to provide to all laboratories the same materials and to avoid any risk
related to the movement of infective materials, samples were prepared by spiking inactivated
bacterial suspensions of Xylella fastidiosa subsp. fastidiosa (ATCC35879) into crushed insects
prepared according to each extraction method. This condition may have affected the yield of
bacterial target DNA using the different extraction methods. Thus, the results obtained may not
reflect the performances of these methods achieved when using fresh infected samples.
5
1.2 Organizers This study has been conceived and promoted by the French Agency for food, environmental and
occupational health and safety (Anses), Plant health laboratory, Bacteriology Virology and GMO unit,
Angers (France), and implemented in collaboration with Anses - Plant health laboratory, RAPT, Saint Pierre
de La Réunion, France.
1.3 Participating Laboratories Participant laboratories are listed below. During the test they have been identified with an anonymous
identification code, ensuring results confidentiality.
Laboratories Country
AGES, Department for Molecular Diagnostics of Plant Diseases Austria
Biology and Ecology of Soil Microbiota, Institute for Sustainable
Agriculture ( CSIC) Spain
Central laboratory for plant quarantine Bulgaria
Centro di difesa e certificazione, CREA Italy
CIHEAM-IAMB Italy
Danish Veterinary and Food Agency, Ringsted Laboratory Denmark
DEFRA (FERA Science LTD) United Kingdom
Departement for Agriculture and Forestry Science (DAFNE),
University of Tuscia Italy
French Agency for food, environmental and occupational health
and safety (Anses), LSV, UBVO France
ILVO Plant Health Belgium
Instituto por la protezione sostenibile, CNR
UNIBA, Dipartimento di Scienze del Suolo, della Planta e delli
Alimenti
Italy
Institute Valenciano de investigaciones agrarias (IVIA),
Bacteriology Spain
Julius Kuehn-Institute - (JKI), Institute for national and
international plant health, Braunschweig Germany
6
Laboratories Country
Julius Kühn-Institut (JKI), Institute AG, Kleinmachnow Germany
Laboratorio Oficial de Sanitad Vegetal de las Islas Baleares (OSVIB) Spain
National institute of agrarian and veterinarian research (INIAV) Portugal
Plant Health Laboratories, Department of agriculture, food and the
marine Ireland
Science and Advice for Scottish Agriculture (SASA) United Kingdom
USDA-ARS USA- California
WUR, Wageningen Plant Research The Netherlands
7
1.4 Documents and instructions Participants received the following documents containing the information for the contract and the
instructions describing the protocols and the panel of samples to be used for each diagnostic method.
Technical sheet (Annex I)
Participant contract (Annex II)
Excel files with spreadsheet to register and report the results (Annex III)
1.5 Timeline of the test performance study The panels of samples were prepared from September 13th to October 03rd 2017; this also included the
panel of samples to be used for the homogeneity and stability tests.
The homogeneity tests for all diagnostic methods considered in this TPS were performed on the October
04th and 12th 2017, immediately after preparing the different batches of spiked samples.
The stability tests were performed December 11th, 12th and 13th, 2017.
Aliquoted samples were then kept at -20°C prior to be shipped. Shipment was organized during the week of
23 - 27 October, 2017.
Participants were requested to perform the selected diagnostic tests and send the results to the organizers
by the week 48 (November 27th – December 1st), 2017.
All the raw data (qualitative results: positive, negative) received from the different participating
laboratories were collected in separate excel files.
This final report was delivered in February, 2018.
2. PANEL OF EXPERIMENTAL SAMPLES Spiked samples
The samples consisted of crushed heads of insects (Philaenus spumarius collected from Xylella-free areas)
macerated in the appropriate extraction buffers according to the different protocols under evaluation:
- one crushed head in 200 µL demineralized sterile water for the QuickPick™ extraction
- one crushed head in 500 µL CTAB buffer for the CTAB extraction.
Samples have been spiked with a calibrated heat inactivated bacterial suspension in water of Xylella
fastidiosa subsp. fastidiosa (ATCC35879). The panel included 13 samples with three different bacterial
concentrations (5.103 bact./head, 5.104 bact./head and 5.105 bact./head), Xylella free samples and one lure.
8
The panel of samples included:
Type of samples Number of replicates
5.105 bact./head 2
5.104 bact./head 2
5.103 bact./head 4
Healthy sample 4
Plus one tube of Positive Amplification Control (PAC)
Samples were aliquoted and shipped in microcentrifuge safe lock tubes (2 mL) and sent frozen with ice
pack. Any laboratories reported any sample degradation or apparent alteration.
Naturally infected and no-infected samples
Panel of 20 natural samples consisted in:
- 15 insects (Philaenus spumarius) produced in containment facilities of CNR and sent to organizer
by Maria Saponari, CNR, Italy and
- 5 insects collected in Xylella free areas (sent to organizer by Maria Saponari, CNR, Italy; Michael
Maixner, Julius Kühn-Institut, Germany; Gudrun Strauss, AGES, Austria; FREDON Nord-Pas-de-Calais,
France).
Insects were stored in 96% (v/v) ethanol at -20°C.
Insects were shipped in microcentrifuge safe lock tubes (2 mL) and sent frozen in 96% (v/v) ethanol.
Panels were provided for the 3 following extraction procedures: Yaseen et al, 2015 extraction, CTAB
extraction and Bio-Nobile QuickPick™ extraction.
Maria Saponari (CNR, Italy) who provided insects from contaminated area into 2 batches, estimated the
contamination rate of her specimens of about 15% for the first batch and > at 40% for the second batch.
The part of 15 specimens of the panels were composed of 12 specimens from the first batch (estimated
contamination rate = 15%) and of 3 specimens of the second batch (contamination rate estimated > 40%).
So the theoretical contamination rate of the panels is estimated > 19%, so an average > 2.85 positive
insects per panel. In addition, the contamination rate was also appreciated trough the homogeneity and
stability tests results (section 5.1).
9
3. DIAGNOSTIC PROCEDURES EVALUATED Molecular tests were consisted of:
DNA extraction:
QuickPick™ Plant DNA kit (Bio-Nobile)
CTAB DNA extraction protocol (EPPO PM7/24)
Yaseen et al., 2015 DNA extraction (Tris HCl; Triton x-100; EDTA extraction buffer) (for natural
insects only)
Amplification:
real time PCR Harper et al., 2010 (erratum 2013)
real time PCR Harper et al., 2010 (erratum 2013) in duplex with primers Ioos et al., 2009 (internal
control)
real time PCR Francis et al., 2006
PCR Francis et al., 2006 using SYBR dying
LAMP Harper et al., 2010 modified by Yaseen et al., 2015
10
Table 1. Laboratory following procedures for conducting analysis for DNA extraction and real time PCR and/or LAMP on spiked crushed insects
Laboratory
Real-Time PCR SYBR LAMP
Harper et al, 2010 Duplex
Harper et al, 2010 Ioos et al., 2009
Francis et al., 2006 Francis et al., 2006 Harper et al, 2010 modified
by Yaseen et al., 2015
QuickPickTM
CTAB QuickPickTM
CTAB QuickPickTM
CTAB QuickPickTM
CTAB QuickPickTM
CTAB
L01 X (StratageneMX
3000)
L02 X (AB Step One Plus)
X (AB Step One Plus)
X (AB Step One Plus)
L03 X (AB 7500 Fast) X (AB 7500 Fast) X (AB 7500 Fast)
X (Genie® II (Optigène))
L04 X (AB Step One Plus)
X (AB Step One Plus)
X (AB Step One Plus)
L05 X (AB 7900HT) X (AB 7900HT) X (AB 7900HT) X (AB 7900HT) X (AB 7900HT) X (AB 7900HT) X (Genie® II
(Optigène)) X (Genie® II (Optigène))
L06 X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96) X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
L07 X (AB 7900) X (AB 7900) X (BioRad
CFX96) X (BioRad CFX96) X (AB 7900) X (AB 7900) X (BioRad
CFX96) X (BioRad
CFX96)
L08 X (BioRad
C1000 + CFX96) X (BioRad
C1000 + CFX96) X (BioRad
C1000 + CFX96)
X (Genie® II (Optigène))
L09 X (Bioneer
Exicycler96) X (Bioneer
Exicycler96) X (Bioneer
Exicycler96)
L10 X (Roche Lightcycler 480
I)
X (Roche Lightcycler 480
I)
X (Roche Lightcycler 480
I)
X (Roche Lightcycler 480 I)
X (Roche Lightcycler 480
I)
X (Roche Lightcycler 480
I)
X (Roche Lightcycler 480
I)
X (Roche Lightcycler 480
I)
L11 X (Eppendorf Realplex 4
Mastercycler S)
X (Eppendorf Realplex 4
Mastercycler S)
X (Eppendorf Realplex 4
Mastercycler S)
X (Eppendorf
Realplex 4 Mastercycler S)
L12 X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96) X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
L13
X (ECO-ILLUMINA)
X (ECO-ILLUMINA) X (ECO-ILLUMINA)
X (ECO-ILLUMINA) + (ICGENE mini (Embiotech)
11
L14 X (AB QanStudio 5)
X (AB QanStudio 5)
X (AB QanStudio 5)
X (AB QanStudio 5)
X (AB QanStudio 5)
X (AB QanStudio 5)
L15 X (AB 7500) X (AB 7500) X (AB 7500) X (Genie® II
(Optigène))
L16 X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
L17 X (Qiagen
RotorgeneQ) X (Qiagen
RotorgeneQ) X (Qiagen
RotorgeneQ)
L18 X (BioRad CFX) X (BioRad CFX) X (BioRad CFX) X (BioRad CFX)
L19 X (BioRad
CFX96) X (BioRad CFX96) X (BioRad
CFX96)
X (ICGENE mini
(Embiotech)
L20 X (AB Step One
Plus) X (AB Step One
Plus) X (AB Step One
Plus)
X (AB Step One
Plus)
12
Table 2. Laboratory following procedures for conducting analysis for DNA extraction and real time PCR and/or LAMP on natural insects (DNA extraction QuickPick™ Plant DNA kit and/or CTAB)
Laboratory
Real-Time PCR SYBR LAMP
Harper et al, 2010 Duplex
Harper et al, 2010 Ioos et al., 2009
Francis et al., 2006 Francis et al., 2006 Harper et al, 2010 modified by Yaseen et al., 2015
QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB
L01
L02
L03
L04 X (AB Step One Plus)
X (AB Step One Plus)
X (AB Step One Plus)
X (AB Step One Plus)
L05 X (AB 7900HT) X (AB 7900HT)
X (AB 7900HT) X (AB 7900HT)
X (AB 7900HT) X (AB 7900HT)
X Genie® II (Optigène)
X Genie® II (Optigène)
L06 X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96) X (BioRad CFX96)
L07 X (AB 7900) X (AB 7900) X (BioRad CFX96)
X (BioRad CFX96)
X (AB 7900) X (AB 7900) X (BioRad CFX96) X (BioRad CFX96)
L08
L09
L10 X (Roche Lightcycler 480
I)
X (Roche Lightcycler
480 I)
X (Roche Lightcycler 480
I)
X (Roche Lightcycler
480 I)
X (Roche Lightcycler 480
I)
X (Roche Lightcycler
480 I)
X (Roche Lightcycler 480 I)
X (Roche Lightcycler 480 I)
L11 X (Eppendorf Realplex 4
Mastercycler S)
X (Eppendorf Realplex 4
Mastercycler S)
X (Eppendorf Realplex 4
Mastercycler S)
X (Eppendorf Realplex 4
Mastercycler S)
L12 X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96) X (BioRad CFX96)
L13 X (ECO ILUMINA)
X (ECO ILUMINA)
X (ECO ILUMINA)
X (ECO ILUMINA) + (ICGENE mini
(Embiotech)
L14
L15
L16 X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
X (BioRad CFX96)
L17
L18
L19
L20
13
Table 2 bis. Laboratory following procedures for conducting analysis for DNA extraction and real time PCR
and/or LAMP on natural insects (DNA extraction protocol Yaseen et al., 2015)
Laboratory
LAMP
DNA extraction protocol Yaseen et al., 2015 Harper et al, 2010 modified by Yaseen et al., 2015
Thermocycleur Portable device
Genie® II (Optigene)
portable device ICGENE mini (Embiotech)
L01
L02
L03
L04 X X
L05 X
L06
L07 X
L08
L09
L10 X X
L11
L12 X X
L13 X X
L14
L15
L16 X
L17
L18 X X
L19
L20
14
Table 3. DNA extraction with QuickPick™ Plant DNA kit with robot or by hand
Laboratory
QuickPick™ extraction with robot or by hand
L01 Robot
L02 By hand (with magnet pipette)
L03 Not applicable
L04 By hand (with magnet pipette)
L05 Robot KingFisher™ mL
L06 By hand (with magnetic rack)
L07 Robot BioSprint 15
L08 Not applicable
L09 Not applicable
L10 By hand (with magnetic rack)
L11 By hand (with magnet pipette)
L12 Robot KingFisher™ mL
L13 Not applicable
L14 By hand (with magnetic rack)
L15 Robot
L16 By hand (with magnetic rack)
L17 Not applicable
L18 By hand (with magnetic rack / magnet pipette)
L19 Not applicable
L20 Not applicable
15 4. ANALYSIS OF THE RESULTS
Diagnostic results were primarily based on qualitative data, i.e. each sample was assigned to one of the
following categories: negative or positive, based on the interpretation criteria for each diagnostic method.
Upon receiving all results, for each sample the following values were assigned after decrypting the sample
code:
PA= positive agreement; PD= positive deviation; ND = negative deviation; NA = negative agreement
Table 4. Definition of the parameters adapted from ISO 16140
Laboratory results Assigned value
Positive Negative
Positive PA= positive agreement PD= positive deviation
Negative ND= negative deviation NA= negative agreement
These assigned values were then used to retrieve the performance criteria evaluated as a function of the
qualitative results and expressed as percentage values (please see Chabirand et al., 2014 for more detailed
information).
Table 5. Details on the performance criteria
Performance criteria Definition Calculation
Sensitivity (SE) Closeness of agreement between the laboratory result and the assigned value for samples for which the assigned value is positive
SE= NPA/N+
Specificity (SP) Closeness of agreement between the laboratory result and the assigned value for samples for which the assigned value is negative
SP=NNA/N-
Accuracy (AC) Closeness of agreement between the laboratory result and the assigned value
AC= (NPA+NNA)/N
Repeatability (DA) Closeness of agreement between independent test results obtained under conditions of repeatability, i.e. conditions under which independent test results are obtained by the same method, on identical test samples in
DA denotes the percentage chance of obtaining the same result (positive, negative or indeterminate) from two identical samples analyzed in the same laboratory
16
the same laboratory, by the same operator, using the same equipment, within a short period of time
Reproducibility The reproducibility is the probability of finding the same result for two identical samples analyzed in two different laboratories
The % of reproducibility is calculated based on the number of interlaboratory pairs of same results/total number of interlaboratory pairs
17 5. RESULTS
5.1 Homogeneity and stability The homogeneity and the stability were assessed for the artificially, naturally contaminated and healthy
samples.
For studying the homogeneity and the stability of the spiked samples, the real time PCR Harper et al., 2010 have
been used after DNA extraction using the QuickPick™ Plant DNA kit and the CTAB protocol.
For studying the homogeneity and the stability of the natural samples, the real time PCR Harper et al., 2010
have been used after DNA extraction using the QuickPick™ Plant DNA kit.
Homogeneity and stability detailed calculation are presented in Annex IV.
5.1.1 Homogeneity
For the spiked samples, the homogeneity study covers 10 samples at the concentration of 5.103 bact./head, 10
healthy samples, 5 samples at the concentration of 5.104 bact./head and 5 samples at the concentration of 5.105
bact./head.
For evaluating the average rate of contamination of the natural insects, 3 panels of 20 insects have been
analyzed.
18
Table 6 Homogeneity results for spiked samples using the QuickPick ™Plant DNA kit and real
time PCR Harper et al., 2010
Panel QuickPick™ Ct Value Average Standard deviation
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
5.103 b/head 31,32 Positive
5.103 b/head 31,32 Positive
5.103 b/head 30,63 Positive
5.103 b/head 30,23 Positive
5.103 b/head 30,82 Positive
5.103 b/head 30,8 Positive
5.103 b/head 30,14 Positive
5.103 b/head 30,05 Positive
5.103 b/head 31,05 Positive
5.103 b/head 30,62 Positive
5.103 b/head 31,34 Positive
5.103 b/head 31,29 Positive
5.103 b/head 30,45 Positive
5.103 b/head 30,6 Positive
5.103 b/head 30,96 Positive
5.103 b/head 30,77 Positive
5.103 b/head 30,16 Positive
5.103 b/head 30,16 Positive
5.103 b/head 31,42 Positive
5.103 b/head 31,22 Positive
30,77 0,45
N/A N/A
Panel QuickPick™ Ct Value Average Standard deviation
5.104 b/head 26,98 Positive
5.104 b/head 26,88 Positive
5.104 b/head 27,43 Positive
5.104 b/head 27,41 Positive
5.104 b/head 27,13 Positive
5.104 b/head 27,21 Positive
5.104 b/head 27,4 Positive
5.104 b/head 27,21 Positive
5.104 b/head 27,21 Positive
5.104 b/head 27,1 Positive
5.105 b/head 23,61 Positive
5.105 b/head 23,7 Positive
5.105 b/head 23,7 Positive
5.105 b/head 23,73 Positive
5.105 b/head 23,51 Positive
5.105 b/head 23,55 Positive
5.105 b/head 23,3 Positive
5.105 b/head 23,48 Positive
5.105 b/head 23,74 Positive
5.105 b/head 23,83 Positive
0,1727,20
23,62 0,15
19
Table 6 bis Homogeneity results for spiked samples using CTAB protocol and real time PCR
Harper et al., 2010
According to the ISO 13528 standard, all the quantitative and qualitative results obtained with the spiked
samples are homogeneous, either with the QuickPick™ extraction or either with the CTAB extraction, regardless
of the different target concentrations. Detailed calculations are presented in Annex IV.
Panel CTAB Ct Value Average Standard deviation
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
Healthy sample N/A Negative
5.103 b/head 32,49 Positive
5.103 b/head 32,82 Positive
5.103 b/head 32,54 Positive
5.103 b/head 32,85 Positive
5.103 b/head 32,61 Positive
5.103 b/head 32,98 Positive
5.103 b/head 32,58 Positive
5.103 b/head 32,39 Positive
5.103 b/head 32,68 Positive
5.103 b/head 32,73 Positive
5.103 b/head 32,8 Positive
5.103 b/head 32,51 Positive
5.103 b/head 32,73 Positive
5.103 b/head 32,16 Positive
5.103 b/head 32,33 Positive
5.103 b/head 32,29 Positive
5.103 b/head 32,17 Positive
5.103 b/head 32,31 Positive
5.103 b/head 32,98 Positive
5.103 b/head 32,31 Positive
32,58 0,24
N/A N/A
Panel CTAB Ct Value Average Standard deviation
5.104 b/head 29,08 Positive
5.104 b/head 29,71 Positive
5.104 b/head 28,77 Positive
5.104 b/head 28,77 Positive
5.104 b/head 29,01 Positive
5.104 b/head 29,09 Positive
5.104 b/head 28,68 Positive
5.104 b/head 28,99 Positive
5.104 b/head 28,84 Positive
5.104 b/head 28,87 Positive
5.105 b/head 25,54 Positive
5.105 b/head 25,61 Positive
5.105 b/head 25,51 Positive
5.105 b/head 25,52 Positive
5.105 b/head 25,25 Positive
5.105 b/head 25,33 Positive
5.105 b/head 25,75 Positive
5.105 b/head 25,72 Positive
5.105 b/head 25,33 Positive
5.105 b/head 25,27 Positive
28,90 0,14
25,48 0,17
20
Table 7 Homogeneity results for natural panels using the QuickPick ™Plant DNA kit and real
time PCR Harper et al., 2010
Homogeneity results for natural samples gave 12 positive insects out of a total of 45 potentially positive insects,
so a contamination rate of 26.67%. The assigned value being not known for the naturally infected samples, the
homogeneity of the panel is not verifiable according to ISO 13528 standard.
Panel 2 QuickPick Ct 1 Ct 2 Result
Healthy sample N/ A N/ A Negative
Healthy sample N/ A N/ A Negative
Healthy sample N/ A N/ A Negative
Healthy sample N/ A N/ A Negative
Healthy sample N/ A N/ A Negative
Proba + about 40% N/ A N/ A Negative
Proba + about 40% N/ A N/ A Negative
Proba + about 40% 38,32 N/ A Negative
Proba + about 15% 37,90 N/ A Negative
Proba + about 15% 30,75 30,65 Positive
Proba + about 15% 36,72 35,24 Positive
Proba + about 15% N/ A N/ A Negative
Proba + about 15% 32,03 32,59 Positive
Proba + about 15% N/ A N/ A Negative
Proba + about 15% N/ A N/ A Negative
Proba + about 15% N/ A N/ A Negative
Proba + about 15% N/ A N/ A Negative
Proba + about 15% 38,67 N/ A Negative
Proba + about 15% N/ A N/ A Negative
Proba + about 15% N/ A N/ A Negative
Panel 3 QuickPick Ct 1 Ct 2 Result
Healthy sample N/ A N/ A Negative
Healthy sample N/ A N/ A Negative
Healthy sample N/ A N/ A Negative
Healthy sample N/ A N/ A Negative
Healthy sample N/ A N/ A Negative
Proba + about 40% 37,23 38,39 Positive
Proba + about 40% N/ A N/ A Negative
Proba + about 40% 32,09 32,61 Positive
Proba + about 15% N/ A N/ A Negative
Proba + about 15% N/ A N/ A Negative
Proba + about 15% 26,83 26,83 Positive
Proba + about 15% N/ A N/ A Negative
Proba + about 15% N/ A N/ A Negative
Proba + about 15% N/ A N/ A Negative
Proba + about 15% 28,89 29,12 Positive
Proba + about 15% 36,52 38,08 Positive
Proba + about 15% N/ A N/ A Negative
Proba + about 15% N/ A N/ A Negative
Proba + about 15% N/ A N/ A Negative
Proba + about 15% N/ A N/ A Negative
Panel 1 QuickPick Ct 1 Ct 2 Result
Healthy sample N/ A N/ A Negative
Healthy sample N/ A N/ A Negative
Healthy sample N/ A N/ A Negative
Healthy sample N/ A N/ A Negative
Healthy sample N/ A N/ A Negative
Proba + about 40% N/ A N/ A Negative
Proba + about 40% N/ A N/ A Negative
Proba + about 40% 30,41 30,82 Positive
Proba + about 15% N/ A N/ A Negative
Proba + about 15% N/ A N/ A Negative
Proba + about 15% N/ A N/ A Negative
Proba + about 15% 29,15 30,08 Positive
Proba + about 15% 29,97 30,41 Positive
Proba + about 15% N/ A N/ A Negative
Proba + about 15% N/ A N/ A Negative
Proba + about 15% N/ A N/ A Negative
Proba + about 15% N/ A N/ A Negative
Proba + about 15% 39,02 N/ A Negative
Proba + about 15% 32,63 32,60 Positive
Proba + about 15% N/ A N/ A Negative
21
5.1.2 Stability
Stability tests were conducted after the deadline indicates for the results delivery by each participant
laboratory.
For the spiked samples, the stability covers 3 samples at the concentration of 5.103 bact./head, 2 samples at the
concentration of 5.104 bact./head, 2 samples at the concentration of 5.105 bact./head and 3 healthy samples.
For the stability study on natural insects, 2 panels of 20 insects have been analyzed.
Table 8 Stability results for spiked samples using the QuickPick™ Plant DNA kit and real time PCR Harper et al.,
2010
Table 8 bis Stability results for spiked samples using the CTAB protocol and real time PCR Harper et al., 2010
According to the ISO 13528 standard, all the quantitative and qualitative results obtained with the spiked
samples are stable, either with the QuickPick™ extraction or either with the CTAB extraction, regardless of the
Panel QuickPick Ct 1 Ct 2 Result Average
Standard
deviation
Healthy sample N/A N/A Negative N/A N/A
Healthy sample N/A N/A Negative N/A N/A
Healthy sample N/A N/A Negative N/A N/A
5.103 b/head 30,53 30,49 Positive
5.103 b/head 32,17 32,14 Positive
5.103 b/head 31,34 31,17 Positive
5.104 b/head 27,65 27,54 Positive
5.104 b/head 27,18 27,2 Positive
5.105 b/head 23,89 23,89 Positive
5.105 b/head 23,32 23,22 Positive
31,31 0,67
27,39 0,21
23,58 0,31
Panel CTAB Ct 1 Ct 2 Result Average
Standard
deviation
Healthy sample N/A N/A Negative N/A N/A
Healthy sample N/A N/A Negative N/A N/A
Healthy sample N/A N/A Negative N/A N/A
5.103 b/head 32,13 32,1 Positive
5.103 b/head 34,12 34,37 Positive
5.103 b/head 32,11 32,27 Positive
5.104 b/head 28,1 28,18 Positive
5.104 b/head 27,94 27,95 Positive
5.105 b/head 24,63 24,73 Positive
5.105 b/head 24,55 24,44 Positive
32,85 0,99
28,04 0,10
24,59 0,11
22
different target concentrations. Detailed calculations are presented in Annex IV.
Table 9 Stability results for natural samples using the QuickPick™ Plant DNA kit and real time PCR Harper et al.,
2010
Stability results for naturally infected samples gave 7 positive insects on a total of 28 potentially positive insects,
so a contamination rate of 25%. The assigned value being not known for the naturally infected samples, the
stability of the panel is not verifiable according to ISO 13528 standard. However no significant differences are
observed between the contamination rate calculated for the homogeneity (26.5%) study and the rate calculated
for the stability study (25.0%). However this rate of contamination is higher than the theoretical expected one
with a rate of about 19% (section 2).
Panel 1 QuickPick Ct 1 Ct 2 Result
Healthy sample N/A N/A Negative
Healthy sample N/A N/A Negative
Healthy sample N/A N/A Negative
Healthy sample N/A N/A Negative
Healthy sample N/A N/A Negative
Proba + about 40% N/A N/A Negative
Proba + about 40% 28,64 28,49 Positive
Proba + about 15% N/A N/A Negative
Proba + about 15% N/A N/A Negative
Proba + about 15% 39,02 N/A Negative
Proba + about 15% N/A N/A Negative
Proba + about 15% N/A N/A Negative
Proba + about 15% N/A N/A Negative
Proba + about 15% N/A N/A Negative
Proba + about 15% 31,68 31,78 Positive
Proba + about 15% N/A N/A Negative
Proba + about 15% N/A N/A Negative
Proba + about 15% 27,28 27,46 Positive
Proba + about 15% N/A N/A Negative
Panel 2 QuickPick Ct 1 Ct 2 Result
Healthy sample N/A N/A Negative
Healthy sample N/A N/A Negative
Healthy sample N/A N/A Negative
Healthy sample N/A N/A Negative
Healthy sample N/A N/A Negative
Proba + about 40% N/A N/A Negative
Proba + about 40% N/A N/A Negative
Proba + about 15% N/A N/A Negative
Proba + about 15% N/A N/A Negative
Proba + about 15% N/A N/A Negative
Proba + about 15% 35,5 35,02 Positive
Proba + about 15% 32,53 33,06 Positive
Proba + about 15% 38,17 37,68 Positive
Proba + about 15% N/A N/A Negative
Proba + about 15% 37,2 N/A Negative
Proba + about 15% 39,94 39,17 Negative
Proba + about 15% 36,08 37,14 Positive
Proba + about 15% N/A N/A Negative
Proba + about 15% N/A N/A Negative
23
5.2 RESULTS OF THE TPS It is important to note that the following rules have been applied to determine the positivity threshold of
the samples for the different methods:
Method Cut-off rule Other rule Real time PCR Harper et al. 2010 Sample positive if Ct value ≤ 38 -
Duplex Real time PCR Harper et al. 2010, Ioos et al., 2009
Sample positive if Ct value ≤ 38 -
Real time PCR Francis et al. 2006 Sample Positive if Ct value ≤ 38 -
PCR Francis et al. 2006 using SYBR Green dye Sample positive if Ct value ≤ 35 (as recommended in PM7/24)
Melting peak between 83°C-85°C
Lamp Harper et al. 2010 modified by Yaseen et al., 2015
No cut-off Sample positive if presence of an exponential amplification curve
The following decision rules for determining the positivity or negativity of the samples are also applied:
2 amplification replicates per sample
Result for replicate 1 Result for replicate 2 Final decision
Positive Positive Positive
Positive Undetermined (e.g. Ct value between 38 and 40 for qPCR)
Positive
Positive Negative Negative
Negative Negative Negative
Note: Following the previous rules, results (positive/negative) sent by the participants have needed in some
cases to be harmonized.
5.2.1 On spiked insects
5.2.1. 1 Qualitative results
Diagnostic sensitivity
24
Table 10. Diagnostic sensitivity calculated using the results obtained in real time PCR, PCR (SYBR Green) and
LAMP tests using the DNA extracts prepared following 2 different protocols.
Figure 1. Diagnostic sensitivity calculated using the results obtained in real time PCR tests using the DNA
extracts prepared following 2 different protocols (restricted series: L18 results excluded due to systematic
false negative results).
In the figure 1, the unrepresentative results of Lab18 are excluded of results series because of systematic
false negative results.
Diagnostic sensitivity (%)
QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB
L01 100 N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A
L02 100 N/A 100 N/A 87,5 N/A N/A N/A N/A N/A N/A N/A
L03 N/A 100 N/A 100 N/A 100 N/A N/A N/A N/A N/A 100
L04 100 N/A N/A N/A 100 N/A N/A N/A 100 N/A N/A N/A
L05 100 100 100 100 100 100 N/A N/A N/A N/A 100 100
L06 100 100 100 100 50 100 N/A N/A 87,5 87,5 N/A N/A
L07 100 100 100 100 100 87,5 N/A N/A 100 87,5 N/A N/A
L08 N/A 100 N/A N/A N/A 100 N/A 100 N/A N/A N/A 100
L09 N/A 100 N/A 100 N/A 100 N/A N/A N/A N/A N/A N/A
L10 50 100 50 100 62,5 100 N/A N/A 87,5 100 N/A N/A
L11 100 N/A 100 N/A 0 N/A N/A N/A 100 N/A N/A N/A
L12 100 87,5 100 100 0 62,5 25** 37,5 100 62,5 N/A N/A
L13 N/A 100 N/A 100 N/A 100 N/A N/A N/A 100 N/A 87,5
L14* 100 100 100 100 50 87,5 N/A N/A N/A N/A N/A N/A
L15 100 N/A 100 N/A 100 N/A N/A N/A N/A N/A 100 N/A
L16 100 100 N/A N/A 75 100 87,5 100 N/A N/A N/A N/A
L17 N/A 100 N/A 100 N/A 100 N/A N/A N/A N/A N/A N/A
L18 0 N/A 0 N/A 0 N/A N/A N/A 0 N/A N/A N/A
L19 N/A 100 N/A 100 N/A 100 N/A N/A N/A N/A N/A 100
L20 N/A 100 N/A 100 N/A 100 N/A N/A N/A 100 N/A N/A
All laboratories 88,46 99,11 84,81 100,00 60,42 95,54 56,25 79,17 82,14 89,58 100,00 97,50
Restricted series*** 95,83 99,11 94,37 100,00 65,91 95,54 56,25 79,17 95,83 89,58 100,00 97,50
N 13 14 10 12 12 14 2 3 7 6 2 5
* Only one replicate for L14
** L12 with cut-off value of 40: amplification performed on DNA dilution because of lack of DNA extract - Melting peak between 81,5 and 82°C
*** L18 results excluded (systematic false negatives)
Duplex Harper et al , 2010 Ioos et
al. , 2009Cut-off = 38
Real-Time PCR SYBR Green
LaboratoryFrancis et al ., 2006
Cut-off = 38
Francis et al ., 2006Cut-off = 35 / Melting peak 83-85°C
Harper et al , 2010Cut-off = 38
LAMP
Thermocycleur Portable device
Harper et al , 2010 modified by Yaseen et a l ., 2015
No cut-off
25
Data show at first look a better sensitivity with CTAB protocol for the majority of the different amplification
methods. Further analysis shows that for the 3 different real time PCR the degradation of the performance
is entirely due to laboratories performing QuickPick™ extraction using a by hand protocol. In that case, the
5 laboratories performing the automated protocol aim 100% of sensitivity (Figure 1 bis).
For the real time PCR using SYBR Green dye, the melting peak has to be in the range 83°C - 85°C (according
to EPPO PM7/24). However the laboratory Lab12 got a melting peak in the range 81.5°C – 82°C, even with
the PAC control. So for this laboratory, the results are validated in these conditions.
Note: It is important to keep in mind that the differences in cut off values for the different methods are
sources to disadvantage (e.g. Real time PCR Francis using SYBR Green dye with a Ct value of 35) or
advantage the sensitivity of the methods (LAMP method without cut off value). The cut off value are those
indicated into EPPO PM7/24 if mentioned. For the real time PCR Francis using SYBR Green dye, when
applying a cut-off of 38, the sensitivity rate rises from 56% to 75% for spiked samples using QuickPick™
extraction and rises from 79% to 83% for spiked samples using CTAB protocol.
In this TPS, the real time PCR Francis et al., 2006 using SYBR Green dye presents a lesser sensitivity than the
real time PCR Harper et al., 2010 (in simplex or duplex) but the results are not really significant due to the
low number of participants (QuickPick™ N = 2 / CTAB N = 3). So, no conclusion must be taken without
caution.
26
Figure 1 bis. Diagnostic sensitivity calculated using the results obtained in real time PCR tests using the DNA
extracts prepared following 2 different protocols (restricted series: L18 results excluded due to systematic
false negative results) – For QuickPick™ extraction: series restricted to the 5 laboratories which have used a
robot.
Real time PCR methods and LAMP method present comparable performance, except real time PCR Francis
et al., 2006 for which the sensibility is lower, especially after QuickPickTM extraction (see. note on cut-off
effect).
27
Tableau 11 Ct value comparison for the real time PCR Harper et al., 2010 - DNA extraction with the
QuickPick™ method versus CTAB method
Quantitative results
Real time PCR Harper et al ., 2010
Laboratory Sample
Ct value average
extract. QuickPick
Standard
deviation Ct
extract. QuickPick
Ct value average
extract. CTAB
Standard deviation
Ct extract. CTAB
∆ Ct [QuickPick-
CTAB]
5.105 bact/head 24,82 0,66 29,27 0,34 -4,45
5.104 bact/head 27,96 0,16 30,82 1,52 -2,86
5.103 bact/head 31,52 0,37 33,73 1,59 -2,21
5.105 bact/head 23,62 0,35 23,37 0,20 0,25
5.104 bact/head 27,45 0,17 26,76 0,10 0,69
5.103 bact/head 31,19 0,55 32,78 2,32 -1,59
5.105 bact/head 25,66 1,05 22,34 0,16 3,32
5.104 bact/head 31,25 0,13 26,21 0,10 5,04
5.103 bact/head 33,24 0,19 30,00 0,19 3,24
5.105 bact/head 23,88 0,21 27,97 0,64 -4,09
5.104 bact/head 27,24 0,15 29,87 2,48 -2,63
5.103 bact/head 30,57 0,50 36,72 1,38 -6,15
5.105 bact/head 26,87 2,69 24,86 0,14 2,01
5.104 bact/head 27,46 0,52 28,28 0,20 -0,82
5.103 bact/head 36,45 2,11 31,33 0,70 5,12
5.105 bact/head 27,77 2,64 22,73 0,28 5,04
5.104 bact/head 28,79 0,85 26,21 0,12 2,58
5.103 bact/head 33,27 0,86 29,54 0,33 3,73
Lab06 results excluded
Lab14
(QuickPick by
hand)
Lab05
(Quickick with
robot)
Lab07
(Quickick with
robot)
Lab10
(QuickPick by
hand)
Lab12
(Quickick with
robot)
Lab16
(QuickPick by
hand)
28
Figure 2 Ct value comparison for the real time PCR Harper et al., 2010 - DNA extraction with the QuickPick™
method versus CTAB method
Tableau 11 bis Ct value comparison for the real time PCR Harper et al., 2010 - DNA extraction with the
QuickPick™ method versus CTAB method – Averages and standard deviations
In the figures 2 are compared the Ct values of the laboratories which performed both QuickPick™ (with
robot N=3; by hand method N=3) and CTAB extractions (N=6). No significant differences are observed with
the real time PCR Harper et al., 2010 regardless of the extraction method performed and the target
concentrations (about minus 0,3 Ct in favor of the CTAB method).
Sample
(Ct value average
extract.
QuickPick)
average
Standard
deviation
(Ct value extract.
QuickPick)
(Ct value
average extract.
CTAB)average
Standard
deviation
(Ct value extract.
CTAB)
∆ Ct [QuickPick-
CTAB] average
5.105 bact/head 25,44 1,51 25,09 2,64 0,35
5.104 bact/head 28,36 1,39 28,03 1,80 0,33
5.103 bact/head 32,71 1,95 32,35 2,44 0,36
29
Diagnostic specificity
Table 12. Diagnostic specificity calculated using the results obtained in real time PCR, PCR (SYBR Green) and
LAMP tests using the DNA extracts prepared following 2 different protocols.
Figure 3. Diagnostic specificity calculated using the results obtained in real time PCR using the DNA extracts
prepared following 2 different protocols.
Diagnostic specificity (%)
QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB
L01 100 N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A
L02 100 N/A 100 N/A 100 N/A N/A N/A N/A N/A N/A N/A
L03 N/A 100 N/A 100 N/A 100 N/A N/A N/A N/A N/A 100
L04 100 N/A N/A N/A 100 N/A N/A N/A 100 N/A N/A N/A
L05 100 100 100 100 100 100 N/A N/A N/A N/A 100 100
L06 75 0 0 0 100 100 N/A N/A 100 75 N/A N/A
L07 100 100 100 100 100 100 N/A N/A 100 100 N/A N/A
L08 N/A 100 N/A N/A N/A 100 N/A 100 100 N/A N/A 100
L09 N/A 100 N/A 100 N/A 100 N/A N/A N/A N/A N/A N/A
L10 100 100 100 100 100 100 N/A N/A 100 100 N/A N/A
L11 100 N/A 100 N/A 100 N/A N/A N/A 100 N/A N/A N/A
L12 100 100 100 100 100 100 100 100 100 100 N/A N/A
L13 N/A 100 N/A 100 N/A 100 N/A N/A N/A 100 N/A 100
L14* 100 100 100 100 100 100 N/A N/A N/A N/A N/A N/A
L15 100 N/A 100 N/A 100 N/A N/A N/A N/A N/A 100 N/A
L16 75 100 N/A N/A 100 100 75 100 N/A N/A N/A N/A
L17 N/A 100 N/A 100 N/A 100 N/A N/A N/A N/A N/A N/A
L18 100 N/A 100 N/A 100 N/A N/A N/A N/A N/A N/A N/A
L19 N/A 100 N/A 100 N/A 100 N/A N/A N/A N/A N/A 100
L20 N/A 100 N/A 100 N/A 100 N/A N/A N/A 100 N/A N/A
All laboratories 96,15 92,86 90,00 91,67 100,00 100,00 87,50 100,00 100,00 95,83 100,00 100,00
Restricted series** 97,92 100,00 100,00 100,00 100,00 100,00 87,50 100,00 100,00 100,00 100,00 100,00
N 13 14 10 12 12 14 2 3 7 6 2 5
* Only one amplification replicate for L14
** L06 results excluded (high number of false positives / about 40% on healthy insects)
Portable device
Laboratory
Real-Time PCR SYBR Green LAMP
Duplex Harper et al , 2010 Ioos et
al. , 2009Cut-off = 38
Francis et al ., 2006Cut-off = 38
Francis et al ., 2006Cut-off = 35 / Melting peak 83-85°C
Harper et al , 2010 modified by Yaseen et a l ., 2015
No cut-off
Thermocycleur
Harper et al , 2010Cut-off = 38
30
Figure 3 bis. Diagnostic specificity calculated using the results obtained in real time PCR using the DNA
extracts prepared following 2 different protocols (restricted series: L06 results excluded due to high
number of false positive results).
If regarding the figure 3 bis, excluding laboratory L06 which obtained 40% of false positive on healthy
samples, no significant difference of performance are observed between QuickPick™ and CTAB protocols,
except for Francis et al., 2006 SYBR Green method (QuickPick™: N= 2; CTAB: N=3).
31
Accuracy
Table 13. Accuracy calculated using the results obtained in real time PCR, PCR (SYBR Green) and LAMP tests
using the DNA extracts prepared following 2 different protocols.
Figure 4. Accuracy calculated using the results obtained in real time PCR, PCR (SYBR Green) and LAMP tests
using the DNA extracts prepared following 2 different protocols.
Accuracy (%)
QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB
L01 100 N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A
L02 100 N/A 100 N/A 91,67 N/A N/A N/A N/A N/A N/A N/A
L03 N/A 100 N/A 100 N/A 100 N/A N/A N/A N/A N/A 100
L04 100 N/A N/A N/A 100 N/A N/A N/A 100 N/A N/A N/A
L05 100 100 100 100 100 100 N/A N/A N/A N/A 100 100
L06 91,67 66,67 72,73 66,67 66,67 100 N/A N/A 91,67 83,33 N/A N/A
L07 100 100 100 100 100 100 N/A N/A 100 91,67 N/A N/A
L08 N/A 100 N/A N/A N/A 100 N/A 100 N/A N/A N/A 100
L09 N/A 100 N/A 100 N/A 100 N/A N/A N/A N/A N/A N/A
L10 66,67 100 66,67 100 75 100 N/A N/A 91,67 100 N/A N/A
L11 100 N/A 100 N/A 33,33 N/A N/A N/A 100 N/A N/A N/A
L12 100 91,67 100 91,67 33,33 75 50 58,33 100 75 N/A N/A
L13 N/A 100 N/A 100 N/A 100 N/A N/A N/A 100 N/A 91,67
L14* 100 100 100 100 66,67 91,67 N/A N/A N/A N/A N/A N/A
L15 100 N/A 100 N/A 100 N/A N/A N/A N/A N/A 100 N/A
L16 91,67 100 N/A N/A 83,33 100 83,33 100 N/A N/A N/A N/A
L17 N/A 100 N/A 100 N/A 100 N/A N/A N/A N/A N/A N/A
L18 33,33 N/A 33,33 N/A 33,33 N/A N/A N/A 33,33 N/A N/A N/A
L19 N/A 100 N/A 100 N/A 100 N/A N/A N/A N/A N/A 100
L20 N/A 100 N/A 100 N/A 100 N/A N/A N/A 100 N/A N/A
All laboratories 91,03 97,02 87,27 96,53 73,61 97,62 66,67 86,11 88,10 91,67 100,00 98,33
Restricted series** 96,21 99,36 95,83 99,24 78,33 97,44 66,67 86,11 98,33 93,33 100,00 98,33
N 13 14 10 12 12 14 2 3 7 6 2 5
* Only one amplification replicate for L14
** L06 results excluded (high number of false positives / about 40% on healthy insects) and L18 results excluded (systematic false negatives)
Laboratory
Real-Time PCR SYBR Green LAMP
Duplex Harper et al , 2010 Ioos et
al. , 2009Cut-off = 38
Francis et al ., 2006Cut-off = 38
Francis et al ., 2006Cut-off = 35 / Melting peak 83-85°C
Harper et al , 2010 modified by Yaseen et a l ., 2015
No cut-off
Thermocycleur Portable device
Harper et al , 2010Cut-off = 38
32
Figure 4 bis. Accuracy calculated using the results obtained in real time PCR, PCR (SYBR Green) and LAMP
tests using the DNA extracts prepared following 2 different protocols (restricted series: L06 results excluded
(high number of false positives) and L18 results excluded (systematic false negatives))
Figure 4 ter Accuracy calculated using the results obtained in real time PCR, PCR (SYBR Green) and LAMP
tests using the DNA extracts prepared following 2 different protocols (restricted series: L06 results excluded
(high number of false positives)and L18 results excluded (systematic false negatives)) – For QuickPick™
extraction: series restricted to the 5 laboratories which have used a robot
33
The accuracy data show similar results with Harper et al.2010 with real-time PCR in simplex and duplex and
with LAMP, whatever the extraction methods CTAB or QuickPick™ Plant DNA kit. In contrast false results
are got by Francis et al. 2006 PCR with Taqman® or SYBR Green mainly after QuickPick™ kit extraction.
These results (figures 4, 4bis, 4ter) show a lab effect on the result reliability: loose of target during the
process, cross contamination. The use of QuickPick™ kit by hand increases a risk of false negative results. So
these points must be considered as very critical in a procedure.
Repeatability
Table 14. Repeatability calculated using the results obtained in real time PCR, PCR (SYBR Green) and LAMP
tests using the DNA extracts prepared following 2 different protocols.
Repeatability (%)
QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB
L01 100 N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A
L02 100 N/A 100 N/A 100 N/A N/A N/A N/A N/A N/A N/A
L03 N/A 95,83 N/A 95,83 N/A 100 N/A N/A N/A N/A N/A 100
L04 100 N/A N/A N/A 100 N/A N/A N/A 100 N/A N/A N/A
L05 100 100 100 100 100 100 N/A N/A N/A N/A 100 100
L06 95,83 100 93,75 100 87,50 95,83 N/A N/A 100 100 N/A N/A
L07 100 100 100 95,83 100 95,83 N/A N/A N/A 91,67 N/A N/A
L08 N/A 100 N/A N/A N/A 100 N/A 100 N/A N/A N/A 100
L09 N/A 100 N/A 100 N/A 100 N/A N/A N/A N/A N/A N/A
L10 100 95,83 100 87,5 100 100 N/A N/A 95,83 100 N/A N/A
L11 100 N/A 100 N/A 100 N/A N/A N/A 95,83 N/A N/A N/A
L12 100 95,83 100 91,67 87,50 100 100 100 100 91,67 N/A N/A
L13 N/A 100 N/A 100 N/A 100 N/A N/A N/A 91,67 N/A 100
L14* N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A
L15 100 N/A 100 N/A 100 N/A N/A N/A N/A N/A 100 N/A
L16 100 100 N/A N/A 100 100 100 100 N/A N/A N/A N/A
L17 N/A 100 N/A 100 N/A 100 N/A N/A N/A N/A N/A N/A
L18 100 N/A 100 N/A 100 N/A N/A N/A 100 N/A N/A N/A
L19 N/A 95,83 N/A 95,83 N/A 100 N/A N/A N/A N/A N/A N/A**
L20 N/A 100 N/A 100 N/A 100 N/A N/A N/A 100 N/A N/A
All laboratories* 99,65 98,72 99,31 96,97 97,73 99,36 100,00 100,00 98,61 95,83 100,00 100,00
N 12 13 9 11 11 13 2 3 6 6 2 4
* L14 results excluded (only one amplification replicate)
** L19 results excluded (no details on replications)
Laboratory
Real-Time PCR SYBR Green LAMP
Duplex Harper et al , 2010 Ioos et
al. , 2009Cut-off = 38
Francis et al ., 2006Cut-off = 38
Francis et al ., 2006Cut-off = 35 / Melting peak 83-85°C
Harper et al , 2010 modified by Yaseen et a l ., 2015
No cut-off
Thermocycleur Portable device
Harper et al , 2010Cut-off = 38
34
Figure 5. Repeatability calculated using the results obtained in real time PCR, PCR (SYBR Green) and LAMP
tests using the DNA extracts prepared following 2 different protocols.
Results are very repeatable inside laboratories. No significant differences between the different methods
are observed in term of repeatability.
Reproducibility
Table 15. Reproducibility calculated using the results obtained in real time PCR, PCR (SYBR Green) and
LAMP tests using the DNA extracts prepared following 2 different protocols.
Reproducibility (%)
QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB
All laboratories 83,67 95,81 77,57 95,16 64,87 96,96 72,66 81,94 78,13 87,24 100,00 95,50
Restricted series* 93,54 99,06 91,60 98,89 68,41 94,60 72,66 81,94 97,63 92,00 100,00 95,50
N 13 14 10 12 12 14 2 3 7 6 2 5
*L6 (high number of false positives / about 40% on healthy insects) and L18 (systematic false negative) results excluded
Harper et al , 2010Cut-off = 38
Real-Time PCR SYBR Green LAMP
Duplex Harper et al , 2010 Ioos et
al. , 2009Cut-off = 38
Francis et al ., 2006Cut-off = 38
Francis et al ., 2006Cut-off = 35 / Melting peak 83-85°C
Harper et al , 2010 modified by Yaseen et a l ., 2015
No cut-off
Thermocycleur Portable device
35
Figure 6. Reproducibility calculated using the results obtained in real time PCR, PCR (SYBR Green) and LAMP
tests using the DNA extracts prepared following 2 different protocols.
Figure 6 bis. Reproducibility calculated using the results obtained in real time PCR, PCR (SYBR Green) and
LAMP tests using the DNA extracts prepared following 2 different protocols (restricted series: L06 results
excluded (high number of false positives) and L18 results excluded (systematic false negatives))
36
The reproducibility values are better for the real time PCR methods after a CTAB extraction for real time
PCR. Here again, we can suspect the by hand QuickPick™ method to damage the performance of the
methods. In fact, for example, for the real time PCR Harper et al., 2010, if we hold only the results of the
laboratories which have used a robot (N=5), the reproducibility aim 100%. On the contrary, if we hold only
the results of the laboratories which have used a by hand protocol (N= 8), the reproducibility value is of
72.77% (figure 6 ter).
The real time PCR Francis et al., 2006 using TaqMan® technology or SYBR Green dye presents a lesser
reproducibility than the real time PCR Harper et al., 2010 (in simplex or duplex). But for the PCR Francis et
al., 2006 using SYBR Green the results are not really significant due to the low number of participants
(QuickPick™ N = 2 / CTAB N = 3).
Figure 6 ter. Reproducibility calculated using the results obtained in real time PCR, PCR (SYBR Green) and
LAMP tests using the DNA extracts prepared following 2 different protocols (restricted series: L06 results
excluded (high number of false positives) and L18 results excluded (systematic false negatives)) – For
QuickPick™ extraction: series restricted to the 5 laboratories which have used a robot
37
Analytical sensitivity (limit of detection)
Table 16. Analytical sensitivity (limit of detection) calculated using the results obtained in real time PCR,
PCR (SYBR Green) and LAMP tests using the DNA extracts prepared following 2 different protocols.
For all the methods and for the great majority of the laboratories, low target concentration samples (5.103
bact./insect head) are detected (see the Table 16 for the details).
Analytical sensitivity = Limit of detection (probability of detection 100%) Sample concentration range: 5.103, 5.104 and 5.105 bact./head
QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB
L01 5.103 N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A
L02 5.103 N/A 5.103 N/A 5.104 N/A N/A N/A N/A N/A N/A N/A
L03 N/A 5.103 N/A 5.103 N/A 5.103 N/A N/A N/A 5.103 N/A 5.103
L04 5.103 N/A N/A N/A 5.103 N/A N/A N/A 5.103 N/A N/A N/A
L05 5.103 5.103 5.103 5.103 5.103 5.103 N/A N/A N/A N/A 5.103 5.103
L06 5.103 5.103 5.103 5.103 5.105 5.103 N/A N/A 5.103 5.103 N/A N/A
L07 5.103 5.103 5.103 5.103 5.103 5.103 N/A N/A 5.103 5.104 N/A N/A
L08 N/A 5.103 N/A N/A N/A 5.103 N/A 5.103 N/A N/A N/A 5.103
L09 N/A 5.103 N/A 5.103 N/A 5.103 N/A N/A N/A N/A N/A N/A
L10 5.105 5.103 5.105 5.103 5.104 5.103 N/A N/A 5.104 5.103 N/A N/A
L11 5.103 N/A 5.103 N/A No positive N/A N/A N/A 5.103 N/A N/A N/A
L12 5.103 5.104 5.103 5.103 - 5.104 5.105 5.105 5.103 5.104 N/A N/A
L13 N/A 5.103 N/A 5.103 N/A 5.103 N/A N/A N/A 5.103 N/A 5.103
L14* 5.103 5.103 5.104 5.103 5.105 5.103 N/A N/A N/A N/A N/A N/A
L15 5.103 N/A 5.103 N/A 5.103 N/A N/A N/A N/A N/A 5.103 N/A
L16 5.103 5.103 N/A N/A 5.104 N/A 5.103 5.103 N/A N/A N/A N/A
L17 N/A 5.103 N/A 5.103 N/A 5.103 N/A N/A N/A N/A N/A N/A
L18 No positive N/A No positive N/A No positive N/A N/A N/A No positive N/A N/A N/A
L19 N/A 5.103 N/A 5.103 N/A 5.103 N/A N/A N/A N/A N/A 5.103
L20 N/A 5.103 N/A 5.103 N/A 5.103 N/A N/A N/A 5.103 N/A N/A
All laboratories
Lowest LOD 5.103 5.103 5.103 5.103 5.103 5.103 5.103 5.103 5.103 5.103 5.103 5.103
Harper et al , 2010Cut-off = 38Laboratory
Real-Time PCR SYBR Green LAMP
Duplex Harper et al , 2010 Ioos et
al. , 2009Cut-off = 38
Francis et al ., 2006Cut-off = 38
Francis et al ., 2006Cut-off = 35 / Melting peak 83-85°C
Harper et al , 2010 modified by Yaseen et a l ., 2015
No cut-off
Thermocycleur Portable device
38
5.2.2 On natural insects
5.2.2.1 On natural insects (DNA extraction QuickPick™ Plant DNA and/or CTAB)
Table 17 Results on natural insects (DNA extraction QuickPick™ Plant DNA and/or CTAB)
Homogeneity and stability studies allow estimating the contamination rate of potentially contaminated
insects at about 25% (DNA extraction using QuickPick only), so 3.75 positive insects per 15 potentially
contaminated insects. A panel contains 5 healthy insects and 15 potentially contaminated insects. The
percentage of positive insects per laboratory ranges from 0% to 60% with average ranging from 3.33% to
28.57% according to the methods and laboratories (L06 results excluded due to high number of false
positives also on the spiked samples).
On the basis of a binomial distribution, with a probability of success of 25% and a number of tests of 15
(number of potentially contaminated insects), with an alpha risk of 5% (confidence level of 95%), observed
values should range into the interval of [1; 7]. On the table 16, values under the lower bound of probability
are highlighted in blue and values above the highest bound of probability are highlighted in orange. Values
within the interval are highlighted in green.
With the QuickPick™ kit extraction, all the data got by laboratories except L06 are in the interval of [1; 7]
with PCR Harper et al. 2010 by real-time PCR in simplex/duplex and by LAMP. The number of positive
QuickPickTM Plant DNA kit and CTAB protocol
Number % Number % Number % Number % Number % Number % Number % Number % Number % Number % Number % Number %
L01 - - - - - - - - - - - - - - - - - - - - - - - -
L02 - - - - - - - - - - - - - - - - - - - - - - - -
L03 - - - - - - - - - - - - - - - - - - - - - - - -
L04 2 13,3 8 53,3 - - - - 2 13,3 6 40,0 - - - - 2 13,3 3 20,0 - - - -
L05 3 20,0 3 20,0 3 20,0 3 20,0 3 20,0 3 20,0 - - - - - - - - 3 20,00 2 13,33
L06 9 60,0 18 120,0 14 93,3 17 113,3 0 0,0 3 20,0 - - - - 1 6,7 0 0,0 - - - -
L07 2 13,3 4 26,7 2 13,3 4 26,7 1 6,7 4 26,7 - - - - 1 6,7 2 13,3 - - - -
L08 - - - - - - - - - - - - - - - - - - - - - - - -
L09 - - - - - - - - - - - - - - - - - - - - - - - -
L10 5 33,3 1 6,7 3 20,0 1 6,7 4 26,7 1 6,7 - - - - 3 20,0 1 6,7 - - - -
L11 2 13,3 - - 2 13,3 - - 0 0,0 - - - - - - 0 - - - - - - -
L12 1 6,7 9 60,0 1 6,7 8 53,3 0 0,0 5 33,3 0 0,0 1 6,7 1 6,7 3 20,0 - - - -
L13 - - 3 20,0 - - 3 20,0 - - 1 6,7 - - - - - - 1 6,7 - - 3 20,00
L14 - - - - - - - - - - - - - - - - - - - - - - - -
L15 - - - - - - - - - - - - - - - - - - - - - - - -
L16 4 26,7 2 13,3 - - - - 2 13,3 2 - 1 6,7 1 6,7 - - - - - - - -
L17 - - - - - - - - - - - - - - - - - - - - - - - -
L18 - - - - - - - - - - - - - - - - - - - - - - - -
L19 - - - - - - - - - - - - - - - - - - - - - - - -
L20 - - - - - - - - - - - - - - - - - - - - - - -
MIN 1 6,67 1 6,67 1 6,67 1 6,67 0 0,00 1 6,67 0 0,00 1 6,67 0 6,67 0 0,00 3 20,00 2 13,33
MAX 9 60,00 18 120,00 14 93,33 17 113,33 4 26,67 6 40,00 1 6,67 1 6,67 3 20,00 3 20,00 3 20,00 3 20,00
Average 3,50 23,33 6,00 40,00 4,17 27,78 6,00 40,00 1,50 10,00 3,13 21,90 0,50 3,33 1,00 6,67 1,33 10,67 1,67 11,11 3,00 20,00 2,50 16,67
MIN* 1 6,67 1 6,67 1 6,67 1 6,67 0 0,00 1 6,67 0 0,00 1 6,67 0 6,67 1 6,67 3 20,00 2 13,33
MAX* 5 33,33 9 60,00 3 20,00 8 53,33 4 26,67 6 40,00 1 6,67 1 6,67 3 20,00 3 20,00 3 20,00 3 20,00
Average* 2,71 18,10 4,29 28,57 2,20 14,67 3,80 25,33 1,71 11,43 3,14 22,22 0,50 3,33 1,00 6,67 1,40 11,67 2,00 13,33 3,00 20,00 2,50 16,67
N = 8 8 6 6 8 8 2 2 6 6 1 2
Number of potentially contaminated insects tested: 15
* L06 results excluded due to high number of false positives
LAMP
Positive samples
Real-Time PCR SYBR Green
Thermocycleur
Harper et al , 2010Cut-off = 38
QuickPickTM CTAB QuickPickTM CTAB
Duplex Harper et al , 2010 Ioos et al. ,
2009Cut-off = 38
QuickPickTM CTAB QuickPickTM QuickPickTMCTAB CTAB
Francis et al ., 2006Cut-off = 35
Francis et al ., 2006Cut-off = 35 / Melting peak 83-85°C
QuickPickTM CTAB
Portable device
Harper et al , 2010 modified by Yaseen et a l ., 2015
No Cut-off
Laboratory
39
insects is a little lower with LAMP. By testing with Francis et al., 2006 PCR, false negative results are
obtained in 3 laboratories. These results are in line with those obtained on spiked sample results in the first
part of this report.
With CTAB extraction, 2 laboratories others than L06 obtained results over 7 positive insects per panel. In
this case, we do not have the results of homogeneity /stability data on natural insects because the number
of natural insects being low and as the results of Ct obtained in homogeneity results between CTAB and
QuickPick™ kit were very close, we considered do not test panel of natural insects with CTAB extraction for
homogeneity and stability tests. In this case, it is impossible to conclude if the number of positive insects is
due to cross contamination or a better sensitivity after CTAB extraction on natural insects.
5.2.2.2 On natural insects (DNA extraction protocol Yaseen et al., 2015)
Table 18 Results on natural insects (DNA extraction protocol Yaseen et al., 2015)
Yaseen et al , 2015 DNA extraction
% Number % Number %
L01 - - - - -
L02 - - - - -
L03 - - - - -
L04 0,0 0 0,0 - -
L05 - 0 - -
L06 - - - - -
L07 0,0 - - - -
L08 - - - - -
L09 - - - - -
L10 0,0 - - 0 0,0
L11 - - - - -
L12 0,0 0 0,0 - -
L13 0,0 - - 0 0,0
L14* - - - - -
L15 - - - - -
L16 - - - 2 13,3
L17 - - - - -
L18 53,3 - - 3 20,0
L19 - - - - -
L20 - - - - -
Number of potentially contaminated insects tested: 15
Laboratory
-
-
0
-
-
0
-
-
-
-
ThermocycleurGenie II
Portable device
Harper et al , 2010 modified by Yaseen et a l ., 2015
No Cut off
Positive samples
LAMP
ICGENE
Number
0
-
0
0
-
-
-
-
8
-
40
Surprisingly, only 2 laboratories, using the ICGene device, got positive results with this method on a total of
8 laboratories. These results are maybe due to a problem with experience of laboratories working for the
first time on this protocol and with transferability of the method. No others explanations have been found.
41
6. SUMMARY OF THE OVERALL VALUES RECOVERED FOR EACH PERFORMANCE CRITERIA In table 18 are summarized the values of the performance criteria obtained for all tested diagnostic methods tested in this proficiency test.
Table 19. Values of the performance criteria obtained for all tested diagnostic methods on spiked samples.
Values of the performance criteria obtained for all tested diagnostic methods (on artificially spiked insects)
QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB QuickPickTM CTAB
% 88,46 99,11 84,81 100,00 60,42 95,54 56,25 79,17 82,14 89,58 100,00 97,50
% (restricted series)* 95,83 99,11 94,37 100,00 65,91 95,54 56,25 79,17 95,83 89,58 100,00 97,50
Nbr of lab with false negative 2 1 3 1 8 3 2 1 3 3 0 1
% 96,15 92,86 90,00 91,67 100,00 100,00 87,50 100,00 100,00 95,83 100,00 100,00
% (restricted series)** 97,92 100,00 100,00 100,00 100,00 100,00 87,50 100,00 100,00 100,00 100,00 100,00
Nbr of lab with false positive 2 1 1 1 0 0 1 0 0 0 1 0
% 91,03 97,02 87,27 96,53 73,61 97,62 66,67 86,11 88,10 91,67 100,00 98,33
% (restricted series)*** 96,21 99,36 95,83 99,24 78,33 97,44 66,67 86,11 98,33 93,33 100,00 98,33
Repeatability % 99,65 98,72 99,31 96,97 97,73 99,36 100,00 100,00 98,61 95,83 100,00 100,00
% 82,49 95,51 76,37 94,76 64,77 96,45 72,66 81,94 78,13 87,24 100,00 95,50
% (restricted series)**** 92,94 98,98 91,90 98,78 69,14 96,18 72,66 81,94 97,63 92,00 100,00 95,50
13 14 10 12 12 14 2 3 7 6 2 5
* L18 results excluded (systematic false negatives)
** L06 results excluded (high number of false positives / about 40% on healthy insects)
*** L06 results excluded (high number of false positives / about 40% on healthy insects) and L18 results excluded (systematic false negatives)
**** L6 (high number of false positives / about 40% on healthy insects), L14 (one repetition only for amplification) and L18 (systematic false negative) results excluded
N =
Reproducibility
performance criteria
Portable device
Diagnostic sensitivity
Diagnostic specificity
Accuracy
Real-Time PCR SYBR Green LAMP
Harper et al , 2010Cut-off = 38
Duplex Harper et al , 2010
Ioos et al. , 2009Cut-off = 38
Francis et al ., 2006Cut-off = 38
Francis et al ., 2006Cut-off = 35 / Melting peak 83-
85°C
Harper et al , 2010 modified by Yaseen et a l ., 2015
No cut-off
Thermocycleur
42 7 DISCUSSION AND CONCLUSION
On spiked insects
The real time PCR Francis et al., 2006 using SYBR Green dye presents a lesser sensitivity than the real time
PCR Harper et al., 2010 (in simplex or duplex) but the results are not really significant due to the low
number of participants for this method (QuickPick™ N = 2 / CTAB N = 3). Moreover, the cut-off value for
this method is 35 (value indicated in EPPO standard PM7/24) against 38 for the real time PCR of this TPS
using TaqMan® technique, and so the method is also “disadvantaged”. No difference appears between the
results got by PCR Harper et al., 2010 in duplex and simplex, so it could be recommended to use the duplex
integrating an internal control.
For this TPS, insect spiked samples are composed of head only and eyes were not removed. The results
obtained in term of sensitivity and Ct values are very similar to results obtained by the organizer in a
previous characterization and validation study. So, the presence of eyes, elsewhere known to contain PCR
inhibitors, seems for this species (Philaenus spumarius), in order to not damage the performance of the
methods.
No significant differences between the different methods are observed in term of repeatability.
The real time PCR Francis et al., 2006 using TaqMan® technology or SYBR Green dye presents a lesser
reproducibility than the real time PCR Harper et al., 2010 (in simplex or duplex). But for the PCR Francis et
al., 2006 using SYBR Green the results are not really significant due to the low number of participants
(QuickPick™ N = 2 / CTAB N = 3).
For the all the methods and for the great majority of the laboratories, low target concentration samples
(5.103 bact./insect head) are detected.
It is important to note that for the QuickPick™ DNA extraction method, this TPS shows some significant
differences between the results obtained by the laboratories using a robot and laboratories using a by hand
protocol, especially in term of sensitivity and reproducibility. So it is recommend the use of the QuickPick™
kit preferably in association with the use of a robot.
Results delivered by LAMP method are very similar to thus obtain with real time PCR for the different
performance criteria. But it should be noted that no cut-off value are applied for LAMP because of the
absence of cycle during the amplification phase, and so this method is advantaged.
43
On potentially naturally contaminated insects
With these samples, the major difficulty for the analysis of the results is that the real status of each
specimen is not known but only a probable contamination rate of the panel. This explains the difficulty to
extricate a tendency from the analysis of the results on natural insects.
For the Yaseen et al., 2015 simplified DNA extraction method followed by the LAMP Harper et al.,2010
modified by Yaseen et al., 2015, only 2 participants got positives results on the 15 insect panel on a total of
8 laboratories. These results are maybe due to a problem with experience of laboratories working for the
first time on this protocol and with transferability of the method. No others explanations have been found.
In conclusion, this TPS shows the real-time method PCR and LAMP Harper et al. 2010 allow to detect X.
fastidiosa on heads of Philaenus spumarius with eyes with reliability in terms of:
- good sensitivity (detection limit on spiked samples in this TPS is 5.103 b/mL),
- good specificity,
- good repeatability and reproducibility.
For the PCR Francis et al., 2006(Taqman and SYBR-Green), the results show a lesser sensitivity. The
differences between cut-off calculation can be questionnable with the method using SYBR Green.
This TPS put in light some critical points and recomendations for the laboratories : be vigilent on the risk of
false negative results during the extraction step, mainly by using the QuickPick™ Plant DNA kit by hand and
not the robot. The use of the robot increases the sensitivity and the reproducibility. This device can be
recomended.
CTAB is the most used extraction method by laboratories of this TPS. Nevertheless, differences in
experience with this method can impact on the results.
It is also necessary to avoid DNA cross contamination during the procedures.
44
8. REFERENCES - Chabirand A, Anthoine G, Pierson O, Hostachy B, 2014. The organization of proficiency testing in plant pathology (qualitative methods of analysis) according to the ISO/IEC 17043: example of the French national reference laboratory. Accred Qual Assur (2014) 19: 111–125 DOI 10.1007/s00769-014-1034-y. - Francis M., Hong Lin, Cabrera-La Rosa J., Harshavardhan Doddapaneni and Cirevelo E. L. (2006). Genome-based PCR Primers for Specific and Sensitive Detection and Quantification of Xylella fastidiosa, European Journal of Plant Pathology. Volume115, Number 2, 203 - Harper SJ, Ward LI, Clover GRG, 2010. Development of LAMP and real-time PCR methods for the rapid detection of Xylella fastidiosa for quarantine and field applications. Phytopathology 100, 1282–1288. - Ioos R., Fourrier C., Iancu G., Gordon T.R. 2009. Sensitive Detection of Fusarium circinatum in Pine Seed by Combining an Enrichment Procedure with a Real-Time Polymerase Chain Reaction Using Dual-Labeled Probe Chemistry. Phytopathology 99-5-0582 - PM 7/24 (2) (2016), Xylella fastidiosa. EPPO Bull, 46: 463–500. doi:10.1111/epp.12327. - Yaseen T., Drago S., Valentini F., Elbeaino T., Stampone G., Digiaro M.and D’onghioa A.M., On-site
detection of Xylella fastidiosa in host plants and in “spy insects” using the real-time loop-mediated
isothermal amplification method, Phytopathologia Mediterranea (2015) 54, 3, 488−496
8. ACKNOWLEDGMENTS Maria Saponari, Instituto por la protezione sostenibile, CNR, Bari, Italy for providing insect specimens and
for its advice and informations,
Michael Maixner, Julius Kühn-Institut, Siebeldingen, Germany for providing insect specimens,
Gundrun Straus, AGES, Vienna, Austria for providing insect specimens,
Philippe Reynaud and his staff from the Anses Plant health laboratory Entomology unit, Montferrier-sur-Lez, France, for the identification of insect specimens
45 Annex I
Technical sheet: detailed protocols
Please see file attached.
46
Annex II Participant contract
47
48
49
50
51
Annex III Result sheets form
TPS 17-XFAST-EU
Test performance study for the evaluation of
molecular methods to detect Xylella fastidiosa in
Philaenus spumarius
Instructions to complete the results form
Organism Xylella fastidiosa
Participant Laboratory
1/ Five sheets (from 2 to 6) can be used to fill out this result file. First, delete the ones you
don't need (according to the panels you have choosen).
2/ Please, in participant laboratory, fill out your name, your code, the number of panels you
have received.
3/ For each sheet, please enter the panel code
4/ In each sheet, you will find a square for one PCR method (# 1 for Harper et al., 2010 for
example). As PCR Harper (simplexed and duplexed), Francis (Taqman version) and LAMP
Yaseen are compulsory, these squares must be filled out. Squares #4 and #6 must be filled
out if the PCR are performed.
5/ Blue cells must be filled out (may be not for the optional methods), and white cells can be
used to give more information, or provide observations.
6/ Please, return this result file by e-mail in Excel form, and in PDF at these three electronic
addresses :
[email protected] ; [email protected] ; [email protected]
Number of naturally panel(s) received
Number of artificially contaminated panel(s) received
Laboratory code (L__)
Laboratory's name
52
# 1 Real-time PCR Harper
Sample ID Cq 1 Cq 2 Mean Result (positive or negative) potential comments
1
2
3
4
5
6
7
8
9
10
11
12
13
PAC
NAC
# 3 LAMP Real-time PCR Harper modified by Yaseen
Sample ID "Cq 1" "Cq 2" Mean Result (positive or negative) potential comments
1
2
3
4
5
6
7
8
9
10
11
12
13
PAC
NAC
# 5 Real-time PCR Francis
Sample ID Cq 1 Cq 2 Mean Result (positive or negative) potential comments
1
2
3
4
5
6
7
8
9
10
11
12
13
PAC
NAC
Panel code
on 40 cycles, positive if Cq mean < 38. Real-time curve
Observations :
Thermocycler :
on 60 cycles of 30 secondes, positive if exponential curve. Real-time curve
Observations :
Thermocycler :
on 45 cycles, positive if Cq mean < 38. Real-time curve
Observations :
Thermocycler :
Date of validation
Additional
comments
Validation by the test correspondent
Name
Date of analysis
Used robot or magnet
Signature
53
# 2 Real-time PCR Harper duplexed Ioos
Sample ID Cq 1 Cq 2 Mean Result (positive or negative) potential comments
1
2
3
4
5
6
7
8
9
10
11
12
13
PAC
NAC
# 4 LAMP Real-time PCR with Portable Device
Sample ID "Cq 1" "Cq 2" Mean Result (positive or negative) potential comments
1
2
3
4
5
6
7
8
9
10
11
12
13
PAC
NAC
# 6 Real-time SYBRgreen PCR Francis (optional)
Sample ID Cq 1 Cq 2 Mean Result (positive or negative) potential comments
1
2
3
4
5
6
7
8
9
10
11
12
13
PAC
NAC
Sample ID Tm 1 Tm 2 Mean
1
2
3
4
5
6
7
8
9
10
11
12
13
PAC
NAC
Real-time curveCq from HARPER (FAM) on 40 cycles, positive if Cq mean < 38.
Observations :
Thermocycler :
Used protocol (Enbiotech or Optigene for example) : Real-time curve
Observations :
Portable device (brand)
on 40 cycles, positive if Cq mean < 35 and 83°C<Tm<85°C. Real-time curve
Observations :
Thermocycler :
Melting peaks
Observations :
54
# 3 LAMP Real-time PCR Harper modified by Yaseen
Sample ID "Cq 1" "Cq 2" Mean Result (positive or negative) potential comments
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
PAC
NAC
# 4 LAMP Real-time PCR with Portable Device
Sample ID "Cq 1" "Cq 2" Mean Result (positive or negative) potential comments
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
PAC
NAC
on 60 cycles of 30 secondes, positive if exponential curve. Real-time curve
Used protocol (Enbiotech or Optigene for example) : Real-time curve
Observations :
Observations :
Thermocycler :
Portable device (brand)
Name
Signature
Date of validation
Date of analysis
Additional
comments
Validation by the test correspondent
55
Annex IV Homogeneity and stability calculation
Please see file attached (in French)