JCM Accepts, published online ahead of print on 30 April 2014 J

Xpert MTB/RIF EQA panel evaluation

1

A multi-centre feasibility study to assess External Quality Assessment panels for Xpert® MTB/RIF 1 assay in South Africa 2

Running title: Xpert MTB/RIF EQA feasibility study 3

Authors: 4

Lesley Scott1#, Heidi Albert3, Chris Gilpin4, Heather Alexander5, Kyle DeGruy5, Wendy Stevens1,2 5

Author affiliations: 6

1Department of Molecular Medicine and Haematology, School of Pathology, Faculty of Health 7 Sciences, University of the Witwatersrand, Johannesburg, South Africa 8

2National Health Laboratory Service, Johannesburg, South Africa 9

3Foundation for Innovative New Diagnostics (FIND), Geneva, Switzerland 10

4 Global TB Programme, World Health Organization, WHO, Geneva, Switzerland 11

5Division of Global HIV/AIDS, U.S. Centers for Disease Control and Prevention, Atlanta, USA 12

Institution where work was performed: Department of Molecular Medicine and Haematology, 13 School of Pathology, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, 14 South Africa 15

Corresponding author#: 16

Lesley Scott 17

7 York Road Parktown 18

Department of Molecular Medicine and Haematology, School of Pathology, Faculty of Health 19 Sciences, University of the Witwatersrand, WITS Medical School, 20

Parktown, 2193 21

Johannesburg, South Africa 22

[email protected] 23

24

JCM Accepts, published online ahead of print on 30 April 2014J. Clin. Microbiol. doi:10.1128/JCM.03533-13Copyright © 2014, American Society for Microbiology. All Rights Reserved.

on April 6, 2018 by guest

http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/


2

Abstract 25

External quality assessment (EQA) for Xpert MTB/RIF is part of the quality system required for 26

clinical and laboratory practise. Five newly developed EQA panels using the following different 27

matrices: lyophilised sample (Vircell, Granada, Spain), dried tube specimen (CDC), liquid (Maine 28

Molecular Quality Control, Inc,Scarborough, Maine), artificial sputum (GLI) and dried culture spot 29

(NHLS) were evaluated at eleven GeneXpert testing sites in South Africa. 30

Panels comprised Mycobacterium tuberculosis complex (MTBC) negative, MTBC positive (including 31

rifampicin [RIF] susceptible and resistant) and non-tuberculosis mycobacteria material inactivated 32

and safe for transportation. Twelve qualitative and quantitative variables were scored as acceptable 33

(1) or unacceptable (0); overall panel performance score for Vircell, CDC, GLI and NHLS was 9 of 12 34

and while MMQCI- scored 6 of 12 (owing to the need for cold chain maintenance). All panels 35

showed good compatibility with Xpert MTB/RIF testing and none showed PCR inhibition. Liquid or 36

dry matrix did not appear to be a distinguishing criterion as both had reduced scores on insufficient 37

volumes, need for extra consumables and ability to transfer to the Xpert MTB/RIF cartridge. 38

EQA is an important component of the quality system required for diagnostic testing programmes, 39

but must be complemented by routine monitoring of performance indicators and instrument 40

verification. The study aims to introduce EQA concepts for Xpert MTB/RIF testing and evaluate five 41

potential EQA panels. 42

Words: 196 43

44

Key words: Xpert MTB/RIF, External Quality Assessment, five panel evaluation 45

46


http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/


3

Introduction: 47

The endorsement of the Xpert MTB/RIF (Xpert) assay by the World Health Organization (WHO) in 48

December 2010 (1) led to an unprecedented commitment by tuberculosis (TB) programs and donors 49

worldwide to implement this technology in efforts to improve the diagnosis of and initiate a 50

continuum of care to reduce the burden of disease. South Africa leads the implementation with 51

more than 2,400 000 sputum specimens tested using Xpert MTB/RIF (November 2013) on 284 52

instruments in 207 smear microscopy testing laboratories country wide, with a further 6 Gx 80 53

instrument placements in the pipelineThe South African experience to date has highlighted several 54

areas for program strengthening realised during their national phased implementation: the need for 55

improved technical and clinical training; the need for laboratory information systems for data 56

management and the need for quality system components such as an external quality assessment 57

(EQA) program, to name a few (2, 3)). The quality system needs to encompass the pre-analytical, 58

analytical and post analytical process to ensure on-going test quality, and an EQA program is one of 59

the internationally recognised tools in this armamentarium (4, 5). In this context for Xpert MTB/RIF, 60

a panel must require intact MTB that can be processed outside a BSL3 (or even BSL2) laboratory 61

infrastructure. 62

Currently no EQA program exists for the Xpert MTB/RIF test which represents a paradigm shift in 63

molecular testing for TB. A methodology using dried culture spots (DCS) has been developed in 64

South Africa for verifying GeneXpert(6) instruments on installation to ensure instrumentation is fit 65

for purpose, and has been used as an EQA for a clinical trial with the AIDS Clinical Trial Group 66

(ACTG) in sites in Africa, Brazil, Peru and the United States. This study therefore investigated the 67

performance of 5 EQA panels being developed in response to this need. The five EQA panels 68

consisting of varying matrices developed specifically for Xpert EQA were donated from both 69

commercial and non-commercial manufacturers: dried culture spots (DCS) developed in South 70

Africa for the National Health Laboratory Services (NHLS), National Priority Program (NPP)(6); dried 71

tube specimens (DTS) developed by the Centers for Disease Control and Prevention (CDC)(7); 72


http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/


4

artificial sputum developed by the Global Laboratory Initiative (GLI)/technical expert advisory group 73

to WHO and Stop TB Partnership; lyophilized samples developed by Vircell (Granada, Spain) in 74

conjunction with the Foundation for Innovative New Diagnostics (FIND) and liquid matrix specimens 75

developed by Maine Molecular Quality Controls Inc. (MMQCI, Scarborough, Maine)(8) as shown in 76

Figure 1. The performance evaluation of all five panels was conducted in South Africa at 11 Xpert 77

testing sites. One reference laboratory tested each of the panels and the results served as the 78

reference standard against which the quantitative results from the 11 Xpert testing sites were 79

compared. 80

81

Materials and methods: 82

Panel preparation and distribution 83

Examples of the five panels are illustrated in Figure 1. The NHLS DCS panel consisting of a 84

characterised MTBC RIF susceptible isolate (H37Rv) grown in bulk single cell culture which had been 85

inactivated using Xpert MTB/RIF sample reagent (SR) buffer (a kit component), followed by 86

quantification by flow cytometry and spotted with blue dye onto Whatman 903 filter cards (Merck) 87

(6). For the present study the same technique was used to create an EQA panel consisting of 4 88

samples: 1 MTBC RIF susceptible, 1 MTBC RIF resistant, 1 non-tuberculosis mycobacterium (NTM) 89

and 1 negative (water), each spotted onto a separate Whatman filter paper card. The CDC panels 90

were created by dilutions from known cultures grown in Mycobacteria Growth Indicator Tube 91

(MGIT) (Becton Dickenson, Sparks, Maryland)media followed by chemical inactivation with Xpert 92

MTB/RIF Sample Reagent (SR), glass bead disruption and drying in a Class II biosafety cabinet(7). This 93

CDC panel consisted of 4 samples: 1 MTBC RIF susceptible, 2 MTBC RIF resistant, and 1 negative. The 94

GLI panels were composed of artificial sputum containing heat killed bacilli and consisted of 4 95

samples: 1 MTBC RIF susceptible, 2 MTBC RIF resistant, and 1 negative. A lyophilised panel from 96


http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/


5

Vircell was developed in conjunction with the Foundation for Innovative New Technologies (FIND) 97

and consisted of 4 samples: 1 MTBC RIF susceptible, 1 MTBC RIF resistant, 1 non-tuberculosis 98

mycobacterium (NTM) and 1 negative. The MMQCI-liquid panel was composed of transfected MTBC 99

DNA in Escherichia coli chemically fixed and killed from Maine Molecular Quality Inc. (MMQCI). The 100

MMQCI panel consisted of 3 samples: 1 MTBC RIF susceptible, 1 MTBC RIF resistant, and 1 negative. 101

All panels were centrally received and re-packaged according to WHO safety protocols (9) using gas 102

impermeable and sealed transport bags and boxes and couriered together to a convenience sample 103

of 11 randomly selected NHLS Xpert laboratories. Testing sites represented a variety of testing 104

volumes and levels of the TB laboratory network. All panels where transported at room 105

temperature except the MMQCI panel which required cold-chain maintenance. The Xpert sites 106

varied both in geography and service level and were visited prior to the commencement of the study 107

to establish feasibility and willingness of participants. The reference laboratory was the Research 108

Diagnostic Laboratory in the Department of Molecular Medicine and Haematology, University of the 109

Witwatersrand, Medical School in Johannesburg. To blind study participants to the panel 110

compositions, barcodes were generated, encoding the panel matrix as well as the contents of each 111

sample. Each panel box also contained panel specific testing instructions produced by the developer. 112

Extra consumables were required for the NHLS panel including a universal container/Nunc 113

tube/sputum jar into which the perforated spot is placed and a pipette tip. An extra pipette was also 114

required for the NHLS, GLI and MMQCI panels to dispense the SR buffer into the containers. These 115

were not supplied by the EQA program as they are standard laboratory consumables. 116

Technologists were provided bench aids and received training on how to upload the results onto 117

www.tbgxmonitor.com. TBGx Monitor® is a web based application developed by the Department of 118

Molecular Medicine and Haematology of the NHLS and the University of the Witwatersrand team to 119

manage their DCS verification program of all GeneXperts implemented in the field(6). An additional 120

tab was included to allow sites to upload the CSV (comma separated value) file format generated by 121


http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/


6

the GeneXpert software after each EQA sample was analysed on the GeneXpert instruments. The 122

sites were also provided with a testing schedule, describing the order in which the tests were to be 123

performed, to minimise any confounding variables. A qualitative questionnaire accompanied the 5 124

panels sent to each site and contained questions relating to the condition of the material, label 125

clarity, ease of operation, sufficient material and ease of protocol use. A general comments section 126

allowed for narrative capture. The processing of the panels took place over two weeks, with three 127

panels being performed in the first week and the remaining two panels, in the second. This was done 128

in order to minimise interruption of daily work-flow at these routine testing sites. The GeneXpert 129

modules used for testing each sample are randomly assigned by the GeneXpert software and 130

therefore not determined by the operators. 131

Panel evaluation and scoring 132

A measure of overall panel performance was determined using a scoring system across the 133

qualitative and quantitative variables. A score of 1 was awarded for acceptable performance and a 134

score of 0 was awarded where issues were documented. Performance was calculated based on 10 135

qualitative and 2 quantitative evaluation criteria, and summarised as the most frequent score 136

reported across 11 testing sites. The qualitative criteria (also termed discrete variable) were: (1) 137

correct results reported for each panel compared to the result of the sample performed at the 138

reference centre; (2) instrument errors related to sample volume (too little volume/too viscous/fluid 139

transfer failed) testing such as 5006 and 5007; (3) the need for cold chain maintenance during 140

transportation of a panel and therefore the need for additional packaging; (4) the need for extra 141

consumables to perform a panel test; (5) questionnaire observations related to condition of goods 142

received, (6) clarity of the standard operating procedures (SOPs), (7) ease of opening samples, (8) 143

ease of handling samples, (9) ease of rehydration of sample, and (10) ease of transferring sample to 144

Xpert testing cartridge. For example: (a) small packaging and therefore lower courier cost received 145

a score of 1 while large packaging and higher courier cost received a 0; (b) the need for cold chain 146


http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/


7

maintenance was scored as 0 while panels with no cold chain requirement received a score of 1; (c) 147

in the case of the quantitative variables, a cycle threshold (Ct) SD similar to the majority group was 148

awarded 1 and an outlier (higher than the majority group) was awarded 0 for any probe (A-E); an 149

invalid result was scored 0. These criteria may need to be refined with time and experience In 150

addition to the qualitative reported result (MTB detected, MTB not detected) generated by the 151

Xpert MTB/RIF assay, a semi-quantitative range (very low, low, medium, high) based on the value of 152

the Ct is also reported (10). The Ct is the threshold at which fluorescence from the hybridization 153

probes increases as the target region is amplified. This is further explained by the assay’s molecular 154

characteristics: Xpert is an automated molecular technology that amplifies the rpoB gene of 155

Mycobacterium tuberculosis and detects this target using molecular beacons (11, 12). Resistance to 156

Rifampicin (RIF) is determined by delayed or drop out hybridization of the molecular beacon probes 157

(5 probes span the rpoB target and 1 probe hybridizes to the internal control). Delayed hybridization 158

occurs when there is >4 cycle threshold (Ct) difference (for assay version G4) between any probes 159

and drop out is reported when the Ct is zero (13). The mean Ct value and the standard deviation 160

(SD) of the Ct for each probe (A-E) of each panel were calculated. The amount of variability in the Ct 161

values of each probe (A-E) across the panels was used as a quantitative evaluation criterion as a 162

potential measure of bacterial consistency in each EQA matrix. For example a qualitative Xpert 163

result of RIF resistance could be generated by a delayed or a dropout probe(s). Their final outcome 164

result of RIF resistance is the same, but the SD for values ranging from 0 (drop out) to 40 (highest Ct 165

value that could indicate delayed hybridization) would be high. 166

A sample from each panel was also processed at the reference laboratory and referred to as the 167

reference result. Bar charts were used to visualize the SD of each probe set across all panels. PCR 168

inhibition of the internal control (SPC) was also reported if any sample yielded an invalid result. 169

Quantitative criteria for scoring included (1) probe Ct SD and (2) PCR inhibition of any sample, 170


http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/


8

Several other observations were also noted during the study that would be considered important in 171

managing a national EQA program but were not used to compare panel performance in this study. 172

These were number and proportion of returned results (categorised by results uploaded onto TBGx 173

Monitor, results returned by email, results received as paper returns, and results not returned) and 174

issues such as unreadable samples (bar scans errors) or incorrect labels. 175

Statistical Analysis 176

Summary descriptives and performance indicators were tabulated for each EQA panel. The mean 177

and standard deviation (SD) were calculated for the quantitative variable of cycle threshold (Ct) for 178

each probe (A-E, and the internal control [SPC]). The consensus (across all panels) SD of the Ct was 179

determined through visualization on a bar chart and used to separate the panel Ct values into the 180

consensus and outlier groups. MS Excel was used for all calculations. Performance scores are 181

provided in binary format. 182

Results: 183

Panel compositions and general observations 184

MTBC positive RIF resistant, MTBC positive RIF susceptible, and negative samples were included in 185

all panels. NHLS and Vircell were the only panels to also include an NTM sample. Two of the five 186

panels (MMQCI and GLI) used a liquid matrix and the remaining three (Vircell, CDC and NHLS) used 187

dried matrix formats. MMQCI was the only panel with 3 samples (only three were purchased for this 188

study); while all the other panels consisted of 4 samples (Table 1). Results were returned for 96% 189

(201/209) of samples. All (100%) results were returned for the CDC, MMQCI and NHLS panels, but 190

91% of GLI panel results and 82% of Vircell panel results were returned. The “no returns” came from 191

two sites which could not read the 2D bar-code labels affixed to the samples. All participants were 192

encouraged to make use of the TBGx Monitor website for submission of their CSV files, however only 193

33% of the MMQCI CSV files and 40-43% of the CSV files from all other panels were submitted on 194


http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/


9

line, suggesting some difficulty in use of the TBGx Monitor. This was similarly noted in the narrative 195

comments of the questionnaire. The Vircell panel had 4 unreadable 2D bar scans. Two sites would 196

have yielded non-conformances in an EQA program: one site did not return the questionnaire and 197

one site mixed all the labels across the panels and would have generated incorrect results. The latter 198

samples were correctly identified through their CSV files, and therefore were included in the data 199

analysis. 200

Qualitative panel performance 201

Expected results were observed for the CDC, GLI and NHLS panels. Using the Vircell panels, one 202

negative sample yielded a false-positive result and with MMQCI, one positive (with RIF resistance) 203

was missed. Using the CDC panels no GeneXpert run errors were observed. Vircell and GLI panels 204

each had one sample where the instrument generated a 2008 error which relates to syringe pressure 205

and therefore not related to the tested material. MMQCI panels showed two insufficient volume 206

errors (5007, 5006) in spite of being a liquid matrix, and GLI and NHLS each had 1 insufficient volume 207

error (Table 1). 208

Of the qualitative questionnaire responses: GLI had no issues; NHLS had 1 issue (“not easy to open”); 209

CDC had 2 issues (“not easy to open” and “not easy to handle”). Vircell and MMQCI had three issues 210

each namely “not easy to transfer”, “material insufficient to test”, and “protocol not easy to follow”. 211

The most expensive panel to transport was the MMQCI panel as this required cold chain 212

maintenance. This was also the heaviest and largest package, due to the addition of cold packs. 213

Courier costs were dependant on weight, distance and whether road and/or air transportation was 214

needed. The laboratory, to which delivery was most expensive, was the most remote, requiring both 215

air and road transportation. 216

Quantitative performance of panels 217


http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/


10

Table 2 presents the average and standard deviation (SD) of each probe’s Ct and SPC Ct across all 218

panels. The sample size across panels per probe analysed, ranged from 7 to 16. The SD values for 219

each probe across all panels are represented in two bar charts in Figure 2 for MTB detected RIF 220

resistant samples and RIF susceptiblesamples. The Vircell followed by the MMQCI panels had the 221

lowest SD on all probes compared to the other panels. The panel with the most variability (SD 222

greater than the majority consensus SD of all panels [SD>2.3) for the RIF susceptible samples was the 223

CDC panel (probes A-E SD>2.3) followed by GLI (probes A and B SD >2.3). The SD on the probes of 224

the RIF resistant sample was highest across 4 probes for the GLI (probes A-D SD>2.3), and 1 probe 225

for CDC (probe C SD>2.3). Although the CDC, GLI and NHLS panels showed more variability on their 226

SPC, no invalids (SPC inhibition) were reported for any sample across all panels indicating the panel 227

materials did not cause any PCR inhibition. 228

Overall panel score 229

The overall performance of the five panels is summarised in Table 3 using the scoring system for the 230

main variables examined. Overall the Vircell, CDC, GLI and NHLS panels all generated the same score 231

of 9 out of a possible score of 12. The MMQCI was the only panel with a lower score of 6. In more 232

detail: Vircell yielded an incorrect result, standard operating procedure (SOP) was unclear and 233

material was not easy to handle; CDC was not easy to open or handle and there was greater 234

variability on amplification and probe hybridization; the GLI panel had insufficient volume, required 235

extra consumables and had greater variability on amplification and probe binding; NHLS had 236

insufficient volume, required extra consumables and was not easy to open; MMQCI yielded an 237

incorrect result, had insufficient volume, required cold chain and extra consumables, SOP was 238

unclear and material was not easy to transfer to the Xpert cartridge. 239

240

Discussion: 241


http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/


11

The implementation of a new assay in settings which are often not experienced with laboratory 242

testing creates many challenges in the “quality system”(14), even though the test in question is 243

designed for users with minimal expertise. The “quality system”, as described by CLSI (Clinical 244

Laboratory Standard Institute) is intended to ensure quality of the overall testing process (specimen 245

collection to result reporting for patient management); to detect and reduce errors; to improve 246

consistency within and between laboratories; contain costs and ensure customer satisfaction. This 247

encompasses twelve elements such as organization, personnel, equipment, quality control, 248

assessment, facility and safety (15). 249

Many quality system components have often been lacking in resource poor settings in general (16, 250

17) and quality assessment programmes for Xpert MTB/RIF have been no exception. In particular 251

there are three components that appear critical to ensure the quality and accuracy of Xpert MTB/RIF 252

testing: (1) verification of each GeneXpert module (on installation or repair) to ensure the 253

instrument is “fit for purpose” before testing and reporting on clinical specimens; (2) external quality 254

assessment to ensure the entire testing process (pre-analytical, analytical and post-analytical) is 255

quality managed and (3) continuous performance monitoring (error rate, potential for 256

contamination, usage, etc.). Verification should be performed on each module and although to date, 257

published evidence on the use of EQA panels for instrument verification is available only for the 258

NHLS panel (6), the other panels evaluated in the current study may also prove suitable for 259

verification. 260

This feasibility study on five EQA panels addresses the gap in EQA for Xpert MTB/RIF and showed 261

little overall differences in scores between the Vircell, CDC, GLI or NHLS panels indicating that any of 262

these panels are likely to be suitable for use in an EQA programme. The MMQCI-liquid panel 263

received a lower score as it was the only panel that required cold chain maintenance and the 264

material was not easy to transfer into the Xpert cartridge. This cold chain requirement is likely to be 265

a challenge in many low resource settings. All panels were received in good condition and therefore 266


http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/


12

good for shipping across distances, and all were easy to rehydrate and dissolve fully in the SR buffer, 267

showing good compatibility with the Xpert testing process. In addition none of the panel materials 268

caused any PCR inhibition. The matrix requirement (liquid or dry) for EQA material did not appear to 269

be a distinguishing criterion as both liquid and dried formats received reduced scores due to 270

insufficient volumes, the need for extra consumables, and ease of transfer to the Xpert cartridge. 271

The increased variation (increased SD) in the Ct (due to some Xpert MTB/RIF tests reporting delayed 272

probe hybridization and some reporting drop out probes with no hybridisation) of the CDC and GLI 273

panels may be attributed to the use of MGIT bulk stock which may have increased clumping of 274

bacterial cells compared to the bulk stock manufactured in single cell format used in the NHLS- 275

product, or other factors inherent to their specific preparation. Minimal variation may, however, be 276

more attractive to an EQA program monitoring RIF resistance rates using differences in probe drop 277

out or probe delayed hybridisation as well as differences in Ct values across the reported semi-278

quantitative categories, since studies are already describing the potential use of Xpert for patient 279

monitoring using Ct values(13, 18). 280

In keeping with international recommendations for EQA that a preferred matrix should bear 281

resemblance to the clinical sample and one that is relatively cost effective to produce, easy, stable 282

and safe to transport, as well as being accurate and precise (15, 19, 20), a liquid specimen format for 283

Xpert MTB/RIF EQA may prove more suitable. Xpert programs would also need guidance on EQA 284

testing schedules and perhaps could draw from other existing molecular EQA schemes: the NHLS 285

National Priority Program in South Africa, for example, subscribes to international schemes that 286

provide three to four panels per year for molecular testing such as HIV viral load. 287

Factors such as SOP clarity, label bar-code scanning, and use of the web based program for result 288

entry highlight the need for any EQA program to be accompanied by training and on-going 289

improvements. Although the web based program, TBGx Monitor was under development during the 290

study and raised concerns of difficulty in use by some South African Xpert users, TBGx Monitor was 291


http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/


13

also used in a concurrent pilot study conducted in collaboration with ACTG experienced clinical trial 292

sites. The ACTG trial included 6 sites in USA, South Africa, Brazil and Peru. The NHLS panels were 293

tested and despite English not being the first language in some sites, none of the sites reported 294

difficulty in uploading their results. Efficient use of both the GeneXpert instrument and the TBGx 295

Monitor software require basic computer skills. Thus, additional computer training may be required 296

for technicians with limited previous experience with computers. 297

The third component in the quality system identified as important to Xpert testing is continuous 298

performance monitoring, since EQA only provides a snap shot. Connectivity standards(21) have 299

made possible several commercial products for hospital, clinic and laboratory information systems 300

as well as instrument information systems, and these should be applied to all testing platforms. In 301

South Africa, for example, seventeen national HIV viral load molecular testing laboratories 302

comprising two platforms (CAP/CTM from Roche Molecular Systems, Branchburg, NJ and 303

m2000sp/m2000rt Abbott Molecular Inc., Des Plaines, Illinois, USA) are centrally monitored 304

in real time for instrument down time, utility, error rate, contamination, etc., which results in rapid 305

response to problems. A similar concept should apply to the GeneXpert as part of managing the 306

entire quality system. Such a remote monitoring system is under development by the GeneXpert 307

manufacturer, Cepheid (Sunnyvale, CA, USA). 308

309

Acknowledgement 310

Disclaimer: The opinions expressed herein are those of the authors and do not necessarily represent 311

the official position of the U.S. Centers for Disease Control and Prevention. Financial support: This 312

project has been funded in part by the President’s Emergency Plan for AIDS Relief (PEPFAR) through 313

the U.S. Centers for Disease Control and Prevention, under the terms of (3U2GPS0001328-04). The 314

Grand Challenges Canada, grant # 0007-02-01-01-01 (Wendy Stevens) also provided support. The 315


http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/


14

authors wish to thank the South African National Health Laboratory Service GeneXpert participating 316

sites. Lesley Scott also wishes to disclose she is the inventor of the Dried Culture Spot product which 317

is undergoing PCT application (PCT/IB2011/052016) through the University of the Witwatersrand. 318

319


http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/


15

320

Table 1: Description of the panels and summary of qualitative data 321

Table 2: Qualitative results describing the mean values and SD per probe compared to the reference. 322

The probes that identified the result as MTBC detected RIF resistant are highlighted in grey 323

Table 3: Overall performance score of the five EQA pilot panels. 324

Figure 1: Representation of the five EQA panels designed for the Xpert MTB/RIF assay. 325

326

Figure 2: Bar charts of SD for each probe and internal control (SPC) across all panels for two groups 327 of samples: (a) MTB detected RIF susceptible and (b) MTB detected RIF resistant. The dotted line 328 separates the SD values into two groups >2.3 and <2.3 329

330

331


http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/


16

332

References 333

1. http://www.who.int/tb/laboratory/mtbrifrollout/en/index.html. 2012. WHO monitoring 334 of Xpert MTB/RIF roll-out. World Health Organisation. 335

2. Stevens W. 2012. Analysis of needed interface between molecular diagnostic tests and 336 conventional mycobacteriology: Country experience in South Africa. In 43rd Union World 337 Conference on Lung Health SoAmdwasatipc (ed.), Kuala Lumpur, Malaysia, November. 338

3. Boehme C. April 2012. Xpert MTB/RIF: Technology update, Workshop on TB diagnostics and 339 laboratory services: Actions for Care delivery and sustainability Global Laboratory Initiative, 340 WHO, Annecy, France. 341

4. Parsons LM, Somoskovi A, Gutierrez C, Lee E, Paramasivan CN, Abimiku A, Spector S, 342 Roscigno G, Nkengasong J. 2011. Laboratory diagnosis of tuberculosis in resource-poor 343 countries: Challenges and opportunities. Clin Microbiol Rev 24:314-350. 344

5. Palamountain KM, Baker J, Cowan EP, Essajee S, Mazzola LT, Metzler M, Schito M, Stevens 345 WS, Young GJ, Domingo GJ. 2012. Perspectives on introduction and implementation of new 346 point-of-care diagnostic tests. J Infect Dis 205 Suppl 2:S181-190. 347

6. Scott LE, Gous N, Cunningham BE, Kana BD, Perovic O, Erasmus L, Coetzee GJ, Koornhof H, 348 Stevens W. 2011. Dried Culture Spots for Xpert MTB/RIF External Quality Assessment: 349 Results of a Phase 1 Pilot Study in South Africa. J Clin Microbiol 49:4356-4360. 350

7. DeGruy K, Zilma R, Alexander H. 2012. Dried Tube Specimens (DTS) for Xpert MTB/RIF 351 Proficiency Panels., ASLM (African Society for Laboratory Medicine) conference, Cape Town, 352 South Africa, December. 353

8. http://www.mmqci.com/qc-m110.php. INTROL TB panel M110. MMQCI, accessed April 354 2014. 355

9. http://www.who.int/ihr/infectious_substances/en/. Accessed May 2013. A Guide for 356 Shippers of Infectious Substances, 2013, International Health Regulations (IHR), World 357 Health Organisation. 358

10. Helb D, Jones M, Story E, Boehme C, Wallace E, Ho K, Kop J, Owens MR, Rodgers R, Banada 359 P, Safi H, Blakemore R, Lan NT, Jones-Lopez EC, Levi M, Burday M, Ayakaka I, Mugerwa RD, 360 McMillan B, Winn-Deen E, Christel L, Dailey P, Perkins MD, Persing DH, Alland D. 2009. 361 Rapid detection of Mycobacterium tuberculosis and rifampin resistance by use of on-362 demand, near-patient technology. J Clin Microbiol 48:229-237. 363

11. Boehme CC, Nabeta P, Hillemann D, Nicol MP, Shenai S, Krapp F, Allen J, Tahirli R, 364 Blakemore R, Rustomjee R, Milovic A, Jones M, O'Brien SM, Persing DH, Ruesch-Gerdes S, 365 Gotuzzo E, Rodrigues C, Alland D, Perkins MD. 2010. Rapid molecular detection of 366 tuberculosis and rifampin resistance. N Engl J Med 363:1005-1015. 367

12. Helb D, Jones M, Story E, Boehme C, Wallace E, Ho K, Kop J, Owens MR, Rodgers R, Banada 368 P, Safi H, Blakemore R, Lan NT, Jones-Lopez EC, Levi M, Burday M, Ayakaka I, Mugerwa RD, 369 McMillan B, Winn-Deen E, Christel L, Dailey P, Perkins MD, Persing DH, Alland D. 2010. 370 Rapid detection of Mycobacterium tuberculosis and rifampin resistance by use of on-371 demand, near-patient technology. J Clin Microbiol 48:229-237. 372

13. Blakemore R, Nabeta P, Davidow AL, Vadwai V, Tahirli R, Munsamy V, Nicol M, Jones M, 373 Persing DH, Hillemann D, Ruesch-Gerdes S, Leisegang F, Zamudio C, Rodrigues C, Boehme 374 CC, Perkins MD, Alland D. 2011. A multisite assessment of the quantitative capabilities of 375 the Xpert MTB/RIF assay. Am J Respir Crit Care Med 184:1076-1084. 376

14. Parekh BS, Kalou MB, Alemnji G, Ou CY, Gershy-Damet GM, Nkengasong JN. 2010. Scaling 377 up HIV rapid testing in developing countries: comprehensive approach for implementing 378 quality assurance. Am J Clin Pathol 134:573-584. 379

15. CLSI. Accessed May 2013. http://www.clsi.org/wp-380 content/.../LabGuidelinesStandardsGP26Jan2012.pdf , Quality management system: 381


http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/


17

Development and managment of laboratory documents; approved guidelines, 6th Edition. 382 The CLSI (Clinical Laboratory Standards Institute) Quality System, 2011, An Overview of the 383 Essential Elements of a PT/EQA Program. 384

16. Nkengasong JN, Nsubuga P, Nwanyanwu O, Gershy-Damet GM, Roscigno G, Bulterys M, 385 Schoub B, DeCock KM, Birx D. 2010. Laboratory systems and services are critical in global 386 health: time to end the neglect? Am J Clin Pathol 134:368-373. 387

17. Petti CA, Polage, C. R., Quinn, T. C., Ronald, A. R. & Sande, M. A. 2006. Laboratory medicine 388 in Africa: a barrier to effective health care. Clin Infect Dis 42:377-382. 389

18. Theron G, Pinto L, Peter J, Mishra HK, Mishra HK, van Zyl-Smit R, Sharma SK, Dheda K. The 390 use of an automated quantitative polymerase chain reaction (Xpert MTB/RIF) to predict the 391 sputum smear status of tuberculosis patients. Clin Infect Dis 54:384-388. 392

19. Noble MA. 2007. Does external evaluation of laboratories improve patient safety? Clin Chem 393 Lab Med 45:753-755. 394

20. MA. N. Accessed May 2013. http://wwwn.cdc.gov/mlp/pdf/GAP/Noble.ppt, An Overview of 395 the Essential Elements of a PT/EQA Program. 396

21. CLSI. 2008. Point of care connectivity: Approved Standard - Second edition: POCT1-A2 397

Clinical and Laboratory Standards Institute 26. 398

399


http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/

Vircell CDC MMQCI GLI DCS

matrix Lyophilised Dried Tube

specimen Liquid (E.coli) Artificial sputum Dried Culture Spot

negative 1 1 1 1 1

MTB negative, NTM 1 0 0 0 1

MTB positive, RIF sensitive 1 1 1 1 1

MTB positive, RIF resistant 1 2 1 2 1

number of testing sites 11 smear microscopy sites, now performing Xpert MTB/RIF, 1 Reference site

number of samples sent to sites, n 44 44 33 44 44

Return of results

Total returned results (%) 36 (81.8) 44 (100) 33 (100) 40 (90.9) 44 (100)

CSV files uploaded onto tbgxmonitor (%) 18 (40.9) 19 (43) 11 (33) 18 (40.9) 18 (40.9)

CSV returned by email that could be uploaded (%) 16 (36.4) 16 (36.4) 15 (45) 16 (36.4) 20 (45)

Paper returns 2 (4.5) 9 (20) 7 (21) 6 (13.6) 6 (13.6)

no returns 8 (18) 0 0 4 (9) 0

Qualitative results

True negative (n/total) (17/18) (11/11) (10/10) (10/10) (10/10)

False negative (n/total) (1/18)* (0/11) (0/10) (0/10) (0/10)

(* MTB positive, RIF

sensitive)

True MTB detected, RIF sensitive (n/total) (8/8) (11/11) (10/10) (10/10) (11/11)

False MTB detected, RIF sensitive (n/total) (0/8) (0/11) (0/10) (0/10) (0/11)

True MTB detected, RIF resistant (n/total) (9/9) (22/22) (10/11) (18/18) (11/11)

False MTB detected, RIF resistant (n/total) (0/9) (0/22) (1/11)# (0/18) (0/11)

(# MTB not detected)

True negative NTM (n/total) na na na na (11/11)


http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/

False negative, NTM (n/total) (0/11)

Errors 1 0 2 2 1

2008 (instrument) 5006 (insufficient

volume) 2008

(instrument) 5007 (insufficient volume)

5006 (insufficient

volume) 5006 (insufficient

volume)


http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/

Vircell CDC MMQCI GLI DCS Reference mean

MTB detected, RIF sensitive mean SD mean SD mean SD mean SD mean SD Vircell CDC MMQCI GLI DCS

n 7 10 8 8 10 1 1 1 1 1

Probe A, mean (SD) 19.6 1.7 22.0 2.9 24.1 2.1 15.7 2.3 19.4 2.1 20.4 22.2 20.9 12.2 21.9

Probe B, mean (SD) 21.4 1.6 23.6 3.0 25.5 2.1 17.6 2.5 21.0 2.2 21.8 23.5 22.5 13.5 22.9

Probe C, mean (SD) 20.0 1.8 22.3 3.0 24.5 2.1 16.0 2.2 19.8 2.2 20.7 22.6 21.2 12.4 22.0

Probe D, mean (SD) 21.2 2.0 23.5 2.8 25.5 1.9 17.4 2.3 20.8 2.2 21.6 23.5 22.3 13.3 23.0

Probe E, mean (SD) 20.9 1.6 23.3 2.8 25.2 2.3 17.0 2.2 20.8 1.9 22.3 23.9 22.6 13.9 23.7

SPC, mean (SD) 24.8 2.1 26.8 3.2 25.7 2.1 27.8 2.1 27.6 3.1 26.7 24.3 26.6 28.7 30.5

MTB detected, RIF resistant mean SD mean SD mean SD mean SD mean SD

n 8 8 8 16 10 1 1 1 2 1

Probe A, mean (SD) 19.3 2.4 20.9 1.8 0.0 0.0 16.4 4.9 20.9 2.6 18.4 33.9 0.0 12.0 11.7 17.4

Probe B, mean (SD) 20.9 2.5 21.9 1.8 0.0 0.0 18.1 4.9 22.7 2.6 19.7 33.7 0.0 12.2 13.2 19.0

Probe C, mean (SD) 19.9 2.5 19.9 21.3 22.7 2.3 16.9 5.0 21.2 2.7 18.9 0.0 20.4 12.5 12.3 17.7

Probe D, mean (SD) 21.1 2.5 22.5 1.5 0.0 0.0 18.1 4.7 0.0 0.0 20.2 33.8 0.0 13.4 13.1 0.0

Probe E, mean (SD) 0.0 0.0 22.3 1.9 23.2 2.2 0.0 0.0 22.1 2.6 0.0 35.6 21.6 0.0 0.0 18.9

SPC, mean (SD) 26.4 3.5 26.0 2.3 25.0 2.0 29.4 3.3 27.4 3.4 23.6 31.7 28.3 26.9 28.4 27.0

MTB detected, RIF resistant mean SD mean SD mean SD mean SD mean SD

n na 9 na na na 1

Probe A, mean (SD) 26.4 2.6 27.6

Probe B, mean (SD) 27.8 2.4 27.9

Probe C, mean (SD) 26.8 2.7 27.0

Probe D, mean (SD) 0.0 0.0 0.0

Probe E, mean (SD) 27.6 2.8 28.5

SPC, mean (SD) 27.3 1.9 25.6

MTB not detected, NTM mean SD mean SD mean SD mean SD mean SD

n all values negative na na na 9 1

Probe A, mean (SD) 0.0 0.0 0.0

Probe B, mean (SD) 0.0 0.0 0.0

Probe C, mean (SD) 24.4 18.4 0.0

Probe D, mean (SD) 0.0 0.0 0.0

Probe E, mean (SD) 0.0 0.0 0.0

SPC, mean (SD) 27.1 1.7 28.5 *two sample sin the GLI panel were the same and analysed together since they had the same Rif resistant

profile


http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/


http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/


http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/


http://jcm.asm

.org/D

ownloaded from

http://jcm.asm.org/

Documents

JCM Accepts, published online ahead of print on 30 April 2014 J