Module OVERVIEW OF EXTERNAL QUALITY ASSESSMENT

1

ModuleModule

OVERVIEW OF EXTERNAL OVERVIEW OF EXTERNAL QUALITY ASSESSMENTQUALITY ASSESSMENT

2

ContentContent OverviewOverview

• What is EQA?

• EQA for AFB Smear Microscopy• Panel testing• Blinded rechecking• On-site evaluation

• Advantages and disadvantages of EQA methods

• Identification of the most feasible EQA method for a country

3

External Quality Assessment (EQA)External Quality Assessment (EQA)

• A system for objectively checking the laboratory’s performance using an external agency or facility

• Ensures customers (physicians, patients and health authorities) that the laboratory can produce reliable results

• An indispensable part of a laboratory quality management system

4

The Quality SystemThe Quality System

EXTERNAL QUALITY

ASSESSMENT

EXTERNAL QUALITY

ASSESSMENT

Quality System Essentials (QSE)

Set of coordinated activities that function as building blocks for quality management

Quality System Essentials (QSE)

Set of coordinated activities that function as building blocks for quality management

5

ISO 15189:2007 ISO 15189:2007 Requirements Regarding EQARequirements Regarding EQA

• The laboratory should participate in interlaboratory comparisons such as those organized by EQA schemes

• The lab management shall monitor the results of EQA and participate in the implementation of corrective actions

6

EQA BenefitsEQA Benefits

• Allows inter-lab comparison of performance

• Serves as an early warning system

• Identifies systematic problems

• Provides objective evidence of laboratory quality

• Serves as an indicator for focusing improvement efforts

• Identifies training needs

• Source of continuing education

• Source of material for practice

7

EQA for AFB Smear Microscopy EQA for AFB Smear Microscopy

Consensus document: External Quality Assessment for AFB Smear Microscopy (2002)

8

EQA Methods for AFB Smear Microscopy

External Quality Assessment (EQA) –

a process to assess laboratory performance; allows to assess labs’ capabilities and performances by comparing their results with those obtained in other labs in the network

On-site supervision

Panel testing

Blinded rechecking

9

Panel Testing ProcessPanel Testing Process

• NRL sends out sets of stained and/or unstained sputum smears for testing

• Laboratory technicians analyze smears and return results to NRL

• Results are evaluated, scores are sent to participants

• Appropriate corrective actions are undertaken (and documented), if needed

10

National reference laboratory

Intermediate laboratory

Peripheral laboratories

A POSSIBLE SCHEME OF A A POSSIBLE SCHEME OF A PANEL TESTING ROUNDPANEL TESTING ROUND


11

Blinded RecheckingBlinded Rechecking

• Random sampling of routine slides from a peripheral laboratory for rechecking by a higher-level laboratory

• The widely used system for rechecking of “10% of negative and 100% of positive smear” is no more recommended

• The proposed blinded rechecking method is based on the Lot Quality Assurance System (LQAS)

12

National reference laboratory


Intermediate laboratory

SECOND CONTROLLER

Routine slides: random sampling

Discordant slides

FIRST CONTROLLER



A Proposed Scheme of Organizing Blinded A Proposed Scheme of Organizing Blinded Rechecking for AFB Smear MicroscopyRechecking for AFB Smear Microscopy

13

On-Site SupervisionOn-Site Supervision• Periodic visits to the laboratory to assess laboratory

practices to:• Obtain a realistic picture of laboratory practices• Provide assistance with problem areas, including

training

• Laboratory vs. TB supervisors

• When planning site visits, to consider:

• Frequency

• Use of checklists

• Follow-up visits• Monitoring corrective actions• Training

14

EQA methods: Which one is better?EQA methods: Which one is better?

• Advantages and drawbacks for all types• Influencing factors:

• NTP goals set up for laboratory services development

• TB prevalence in the country• Centralization / decentralization of health

services • Resources available and projected

• Cost, time• Logistical questions

15

Panel TestingPanel Testing

ADVANTAGESADVANTAGES DISADVANTAGES:DISADVANTAGES:

•Low workload for a peripheral center

•Improves laboratory credibility

•Rapid response countrywide possible

•Use of stained and unstained smears can help to identify the source of a problem

•May lead to identification of faulty equipment

•Does not measure routine performance

•High workload for NRL

•May not be motivating to improve daily performance

16

• Minimal first step for EQA with limited resources

• Rapid assessment of gross deficiencies

• Evaluates proficiency of laboratory technicians prior to and following training

• A tool during problem-oriented supervision

Panel Testing: Indications for UsePanel Testing: Indications for Use

17

Blinded RecheckingBlinded Rechecking


•Low workload for a peripheral laboratory

•Motivates to improve daily performance

•Reflects reality of routine performance

•Higher workload for a higher level center

•Needs close adherence to elaborated procedures

•Can not be used with very low positivity rates

Indications for use:

• The best method for evaluating lab performance

• Countrywide

• Ongoing and permanent

18

On-Site SupervisionOn-Site Supervision


•Direct personal contact

•Motivating to staff

•Observation of actual work

•Identifies causes of errors

•Permits verification of equipment quality and function

•Usually poor coverage

•Labor intensive

•Costly

•Needs very good supervisors

19

On-Site Supervision: Indications for UseOn-Site Supervision: Indications for Use

• Complementary to rechecking and panel testing for constructive feedback and problem solving

• Implementation and monitoring of quality improvement measures

• Data collection and flow of information among laboratory levels

20

Blinded rechecking or panel testing should be complemented by very targeted problem-oriented supervision conducted by a trained staff

EQA: Important IssueEQA: Important Issue

21

Situational Analysis:Situational Analysis:How to Identify the Most Feasible EQA How to Identify the Most Feasible EQA

method(s) For a Country? (I)method(s) For a Country? (I)

COUNTRY A:

• Populous; vast territory

• High burden of TB

• Several hundreds of geographically dispersed peripheral laboratories

• Intermediate laboratories’ infrastructure is not well developed; many intermediate labs experience lack of staff

• NRL is well equipped and staffed

22


method(s) For a Country? (II)method(s) For a Country? (II)

COUNTRY B:

• Less populated (compared to Country A); compact territory

• High burden of TB

• Smear microscopy laboratory network: one microscopy center for 100 000 population

• High to medium positivity rates

• Well developed structure of intermediate laboratories, supervised by NRL

• Lack / rotation of staff is a typical problem for many laboratories

23


method(s) For a Country? (III)method(s) For a Country? (III)

COUNTRY C:• Populous country

• Low burden of TB

• Only few laboratories dealing with sputum smear microscopy

• Low volume sputum smear examinations; very low positivity rates

• AFB smear-positive specimens are rechecked at NRL

24

Key Messages (I):Key Messages (I):• EQA can be defined as a system for objectively

checking the laboratory’s performance using an external agency or facility

• EQA for AFB smear microscopy allows participating laboratories to access their capabilities and performances by comparing their results with those in other laboratories in the network

• EQA for AFB sputum smear microscopy include three methods:• Panel testing• Blinded rechecking• On-site supervision

25

Key Messages (II):Key Messages (II):

• Effective use of each of EQA methods will highly depend on resources, NTP goals, TB prevalence, health services’ structure and the stage of laboratory quality assurance activities development in the country

• Blinded rechecking or panel testing should be complemented by very targeted problem-oriented supervision conducted by a trained staff

Documents

Module OVERVIEW OF EXTERNAL QUALITY ASSESSMENT