Upload
nirmala-last
View
660
Download
8
Embed Size (px)
Citation preview
Assuring the Quality of Laboratory Testing in Countries
Fighting the HIV/AIDS Epidemic
CDC
November 29-30, 2000
Test Verification&
Test Validation
Niel T. Constantine, Ph. D.Professor of Pathology
Director Clinical Immunology
University of MarylandSchool of Medicine
AndInstitute of Human Virology
Test Verification &Test Validation
I. Considerations when determining the utility of testsA. Protocols for Evaluation of TestsB. Reference TestsC. AlgorithmsD. Choice and Number of SamplesE. Testing ConditionsF. Resolution of Discordant ResultsG. Indicators of test performance
Considerations When Determining the Utility of Tests
Protocols for Evaluations of Tests
Protocols for Evaluation of Tests
• Essential to set guidelines.
• Must be followed exactly.
• Must outline all characteristics of samples and procedures.
• Must describe detailed algorithm to follow for discordant results.
• Must include QA/QC section.
Considerations When Determining the Utility of Tests
Reference Tests
Reference Tests
• Needed to fully characterize samples.
• Choice depends on purpose of testing.Concordance – against reference screening test.
Accuracy – against confirmatory test.
• Must be careful about “pre-selected samples” to evaluate false positives.
• Should be tests that are recognized by the scientific community.
Considerations When Determining the Utility of Tests
Algorithms
UNAIDS and WHORecommended Alternative Algorithms
• To maximize accuracy while minimizing cost
• Depends on objectives of the test and the prevalence of infection
Table 2 UNAIDs and SHO reccommendations for HIV testing strategies Tableau 2 Recommandations de I’onusida et de I’OMS aux strategies
according to test objective and prevalence of infection in the de depistage du VIH, en fonction de l’obectif du test et de la
sample population prevalence de l’infection dans la population
Objective of testing Prevalences of infections Testing strategy
Objectif du dépistage Prévalences de l’infection Stratégie de dépistage
Transfustion/transplant safety All Prevalences
Sécurité des transfusions/transplantations Toutes prévalences
Surveillance >10%
10%
Clinical signs/symptoms of >30% HIV Infection- SignnesCliniques/symptôms de 30% l’infection à VIHa
Asymptomatic >10% Asymptomatique
10%
aaWorld Health Organizaion, Intenm proposal for a WHO staging system for HIV infection and desease (WER World Health Organizaion, Intenm proposal for a WHO staging system for HIV infection and desease (WER no.29, 1990, pp 221-228)- Organisation mondiale de la sante. Echelle provisoire OMS proposee pour la no.29, 1990, pp 221-228)- Organisation mondiale de la sante. Echelle provisoire OMS proposee pour la determinationdes strades de l’infecrtiono VIH et de la malodie (REN no 29, 1990. P.221-228)determinationdes strades de l’infecrtiono VIH et de la malodie (REN no 29, 1990. P.221-228)
Considerations When Determining the Utility of Tests
Choice and Number of Samples
Choice and Number of Samples
Samples:
• Should represent population where test will be performed.
• Same matrix of sample (e.g. plasma).
• Must meet guidelines stated by manufacturer (e.g. not
lipemic).
• Avoid multiple freeze/thaw, etc.
• Use “clean”samples.
• Multiple aliquots if possible.
• Must be well categorized.
Choice and Number of SamplesSamples:• Should represent population where test will be performed.• Same matrix of sample (e.g. plasma).• Must meet guidelines stated by manufacturer (e.g. not lipemic).• Avoid multiple freeze/thaw, etc.• Use “clean”samples.• Multiple aliquots if possible.
Numbers of Samples:
• The more the better (min. 30 positives, 200 negatives).
• Depends on purpose of testing (e.g. blood donors).
• Include appropriate percent of variants.
• Perform precision and reproducibility studies (lg. Volumes).
HIV Classification
HIV
HIV-1 HIV-2
O M
A, B, C, D, E, F, G, H, I, J
Types
Clades
ANT 70, MVP5180,
VAU
ROD NIH2
Groups
Guidelines for Classification
Types: HIV-1 and HIV-2 50% homology
Subtypes/Groups: HIV-1 group M, N and O 60-70% homology
Clades: HIV-1 Clades A-J >70% homology
N
Considerations When Determining the Utility of Tests
Testing Conditions
Testing Conditions
• Must test under identical conditions.(e.g. same lab, equipment, day, tech).
• Use non-expired kits that have been properly stored.
• Follow manufacturer’s recommendations.
• Sample integrity.
• Test in a blinded fashion.
Considerations When Determining the Utility of Tests
Resolution of Discordant Results
Resolution of Discordant Results
• Check sample integrity, labeling, paperwork, and procedures.
• Repeat by same technologist.
• Repeat blindly by another technologist.
• Repeat reference test blindly.
• Repeat at different laboratory.
• Determine true status by other means.
• What parameters would these investigate?
Resolution of Discordant ResultsPossible Variants
• Synthetic peptide tests
• Specific Western blots
• Specific IFAs
• Combination tests– Dot blots
– Immunoconcentration tests
– Augmented blots and LIA
• PCR - specific
Rapid Assay Evaluation AlgorithmRapid Assay +
ELISA -Rapid Assay -
ELISA +Discordant Results
Repeat Rapid & ELISA
Western Blot Assay (FDA Licensed)
Negative Indeterminate Positive
ResolvedIFA (FDA Licensed)
Negative Indeterminate Positive
ResolvedSample Volume
> 1 mLSample Volume
(<1 mL & >0.2 mL)
RT-PCR Assay
Negative Positive
Inconclusive Resolved
P24 Ag Assay (FDA Licensed)
Negative Positive
Inconclusive Ag Neutralization
Positive Negative
Resolved
Resolved
Considerations When Determining the Utility of Tests
Indicators of Test Performance
Indicators of the Value of a Diagnostic Assay
• Sensitivity
• Specificity
• Test efficiency
• Delta values
• Predictive values
Sensitivity of Tests
• Sensitivity (epidemiologic)
• Sensitivity (analytical)– Low titer
– Seroconversion
– Dilutions
Indicators of the Value of a Diagnostic Assay
Sensitivity = True Positives True Positives + False Negatives
Specificity = True Negatives True Negatives + False
Positives
X 100%
X 100%
Indicators of the Value of a Diagnostic Assay
Positive Predictive = True Positives
Value True Positives + False PositivesX 100%
X 100%Negative Predictive = True Negatives
Value True Negatives +False Negatives
Predictive ValuesAssume: Test Sensitivity = 100% / Specificity = 99.5%Assume: Test Sensitivity = 100% / Specificity = 99.5%
Population #1Population #1, where the prevalence of infection is high (5%)
• Population: 1000 sera tested50 sera from infected individuals950 sera from non-infected individuals
• Test Results: 50 positives: 45 from the infected group5 false pos from the non-infected group
• Therefore, the positive predictive value is:
PPV = 45 = 90%
45+5
• 9 out of 10 positive results will be from infected persons
Predictive ValuesAssume: Test Sensitivity = 100% / Specificity = 99.5%Assume: Test Sensitivity = 100% / Specificity = 99.5%
Population #2Population #2, where the prevalence of infection is low (0.7%)
• Population: 1000 sera tested7 sera from infected individuals993 sera from non-infected individuals
• Test Results: 7 positives: 2 from the infected group5 false pos from the non-
infected group
• Therefore, the positive predictive value is:
PPV = 2 = 28.6%
2+5
Predictive Values
• Therefore, the same test that yields the same number of false-positives produces a different positive predictive value when testing two different populations
Predictive Values
• Therefore, the same test that yields the same number of false-positives produces a different positive predictive value when testing two different populations.
• The chance of a positive result being from a truly infected individual in the low prevalence population is only 28.6% (2 true positive detected by the test and 5 false-positives).
Predictive Values
• Therefore, the same test that yields the same number of false-positives produces a different positive predictive value when testing two different populations.
• The chance of a positive result being from a truly infected individual in the low prevalence population is only 28.6% (2 true positive detected by the test and 5 false-positives).
• This indicates that a positive result by the test will be from an infectd individual in only one of four cases (a guess could yield better chance!).
Test Verification &Test Validation
II. Quality Assurance and Errors
A. Common ErrorsB. Quality Assurance Needs
1. Fundamentals of QA2. Quality Control3. Quality Assessment4. Equipment Issues5. 10 Key Issues for QA
Most Common Errors
• Transcription
• Carelessness
– Procedures
– Specimens
• Environmental conditions
• Pipettes and pipetting
Clerical Errors
• Logging specimens
• Aliquoting
• Worksheets
• Result printouts
• Translating results
• Computer entering
• Reports
• Supervisory Review
Specimen Problems
• Insufficient volume for repeating
• Hemolysis, lipemia, and bacterial contamination
• Insufficient and inadequate labeling
• Misidentified specimens
• Frozen / Thawed (multiple)
Other Types of Errors
• Kit Dependent Problems.
• Technologist – dependent errors.
• Inter-lot variations and Intra-lot variations.
• Environmental problems.
• Non repeatable results.
• Inter-laboratory and Intra-laboratory variations.
• Equipment problems.
Quality AssuranceFundamental for Quality Test Results
• Record keeping• Monitoring laboratory staff• Vigilance in the laboratory• Verification of true positive and true negatives• Parallel testing of resubmitted samples• Reporting of results• Confidentiality • Interaction with physicians• Storage of specimens for follow-up testing• Laboratory efficiency• Total quality management
Components of Quality ControlRecord Keeping
• Kit lot numbers (expiration and open dates).• Clearly label reagents with date opened or
prepared (include open and expiration date) on each label.
• Daily temperature monitoring and recording i.e. Incubators water baths, ambient.
• Performance of controls and action taken when out-of-range.
• Photograph or clear photocopies of Western blots.
• Ratios of in-house controls to cut-off values.
Components of Quality ControlControls
• Kit controls: Use as directed by the manufacturer.• In-house controls: preferably three levels to monitor
variability between runs and lot numbers of kits.– Low positive – absorbance enough above cut-off that
it should not be misclassified because of expected run-to-run variability.
– High positive – well above the cut-off.– Negative – well below cut-off.
• Storage of in-house control sera:– Dispense in aliquots sufficient for one week of use.– Freeze at -20°C in a non-self-defrosting freezer.– Thaw each aliquot once, store at 4 °C when not in
use, do not refreeze and discard after 1 week.
Trend
Monitoring by External Controls
Shift
Quality Assessment
Internal Quality Assessment
– Known Reactors
– Unknown Reactors
– Blind Testing
External Quality Assessment
– Proficiency Panels
– Blind Proficiency Panels
Equipment Issues
Pipette Calibrations
ESSENTIAL FOR ACCURACYESSENTIAL FOR ACCURACY
• Frequency
– At least every 6 months
• Reasoning
– 1l inaccuracy = 10% error (total volume of 10 l)
– Controls – o.k., borderline specimens – loss of sensitivity
Quality Assurance: What Must Be Done?10 Key Issues
• Detailed SOP with total compliance.• Supervising review of all paperwork.• Develop checklists for monitoring all activities.• Dev. Organizational schemes for processing,
documentation, and assessment.• Monitor staff – blind proficiencies.• Neat and complete documentation of all results.• No deviation from procedures.• Maintain confidentiality.• Endorse safety measures.• Vigilance.
Test Verification &Test Validation
III. Introduction of a New Test
A. Selection
B. Characteristics
C. Approved versus Non-Approved tests
D. Continual Monitoring
Selection
• Availability
• Appropriateness
• Cost and bulk purchases
• Shelf life and robustness
• Storage
• Publications and WHO evaluations
• Regulations
Characteristics
• Laboratory capabilities
• Testing Purpose
• Simplicity
• Cost Concerns
• Sample type
• Test limitations
• Test principles and antigens
• Test indices
Approved VersusNon-approved Tests
• Which can be used?
• When approved tests are unavailable.
• Validation of non-approved tests.
• Documentation necessities and qualifications.
Continual Monitoring
• Necessity to monitor new tests.
• How long to monitor.
• Methods of monitoring.
• Looking for trends.
• Changing tests – Parallel testing.
• Documentation.
Test Verification &Test Validation
IV. Special Considerations for Developing Countries
A. Selection of Tests and algorithms
B. Testing under non-optimal conditions.
C. Best fit Strategies
D. When Systems Fail
Special Considerations for Developing Countries
Selection of Tests and Algorithms
Selection of Tests
• Infrastructure
• Supportability
• Expertise
• Accessibility
• Cost Concerns
• Algorithms
Algorithms
• What’s effective?• What can be used?• Established and recommended algorithms.• Use of additional strategies.• Differences due to geographical origins of samples.• Cost effectiveness.• Sample pooling.• Blood donations vs. diagnostic testing.• Different algorithms within the same country.• Epidemiological testing.
Simple, Rapid Test Alternative Algorithm
Rapid Test #1
Positive Negative
P/N OR P/P
Rapid Test #2*
Repeat in Duplicate
REPORT N/N
REPORT
Positive Negative REPORT indeterminate
Resolve with other testsReport as Positive
*Different configuration or antigens
Special Considerations for Developing Countries
Testing Under Non-optimal Conditions
Testing Under Non-optimal Conditions
• Use of expired kits.• Unsatisfactory environmental conditions.• Limited number of test kits.• Limited equipment (e.g. thermometers).• Non-calibrated pipettes.• Old equipment.• Poor integrity of samples.• Questionably labeled specimens.
Special Considerations for Developing Countries
Best Fit Strategies
Best-fit Strategies(to Test or Not to Test?)
• Consequences and necessities.• Cost effective strategies.• Pooling of samples.• Saving reagents.• Parallel testing.• Sequential testing.• Mixing reagents.• Alternate testing areas.• Testing when temperatures and conditions fail.
Pooling of Samples
• In what situations can pooling be used?
• How many samples can be pooled?
• Accuracy.
• Final sample dilution of pools.
• Proper sample size for evaluation.
• Effects of the presence of HIV Antigens.
Non-approved Testing Strategies
• Re-use of rapid tests.
• Modification of test kits:
Cutting WB strips.
Halving reagents.
• Pooling of samples.
Special Considerations for Developing Countries
When Test Systems Fail
When Test Systems Fail
• Trouble shooting.
• Repeat testing.
• Alternative testing.
• Other personnel, other laboratories.
– Getting help.
• Documentation.
• Reporting.
• Consequences.
Reasons for the Need for Improved Assays
• Early diagnosis.
• Resolution of indeterminate results.
• Differentiation of retroviral infections.
• Less expensive tests.
• Simple and foolproof tests.
• Detection of viral types and variants.
• Multiple combination tests.
• Detection of infection in the newborn.