Upload
gabriel-migliarino
View
221
Download
5
Tags:
Embed Size (px)
DESCRIPTION
Effective Control of the Examination Process.Group 2 Report “Reunion de Expertos Expertos” 5 CICLO International De Conferencias De La Calidad
Citation preview
Effective Control of theEffective Control of the Examination ProcessExamination Process
Group 2 ReportGroup 2 Report““Reunion de Reunion de ExpertosExpertos””
5 CICLO International De 5 CICLO International De ConferenciasConferencias De La De La CalidadCalidad
11
Group 2 Participants
Laura Mercapide, Argentina
Amadeo Saez, Brasil
Aida Porras, Colombia
Oscar Martinez, Colombia
Arturo Terres, Mexico
Eduardo Brambila, Mexico
Enrique Amaya, Peru
Margarita Iturriza, Venezuela
Erik Mendoza, Mexico
James Westgard, United States
Our Focus is on the Examination Process
Pre-analytic and post-analytic errors are also of concern, but our charge is to focus on the analytical part of the examination process
Evidence in scientific literature indicates analytical errors are still major source of problems leading to mistreatment and harm to patients
What errors have been observed in What errors have been observed in the Total Testing Process?the Total Testing Process?
Pre-analytic Analytic Post-analytic
•Patient preparation•Specimen acquisition•Specimen processing•Sample transport•Physician test order
•Sample aliquot•Analyzer setup•Test calibration•Quality control•Reportable test
•Test report•Transmittal of report•Receipt of report•Review of test results•Action on test results
60% 15% 25%Plebani & Carraro. Clin Chem 2007:53:1338-42
55
Lab Errors and Patient CareLab Errors and Patient Care
PlebaniPlebani--CarraroCarraro study study
ClinClin ChemChem 2007:53:13382007:53:1338--4242
51,746 tests 51,746 tests
393 questionable results393 questionable results
160 confirmed as laboratory errors160 confirmed as laboratory errors
46 caused inappropriate patient care46 caused inappropriate patient care
24 of those were analytical errors24 of those were analytical errors
Analytical errors are still major cause (over Analytical errors are still major cause (over 50% cases) of inappropriate patient care50% cases) of inappropriate patient care
66
Need for Guidance on “Effective Control of Examination Process” for
Latin American Countries
Important to be relevant and practical for local laboratories
Must consider national and governmental interests and requirements
Must consider professional assessment of needs and practice guidelines
General guidance on General guidance on ““effective effective control of examination processcontrol of examination process””??
ISO 15189: Medical Laboratories ISO 15189: Medical Laboratories –– Particular Particular requirements for quality and competencerequirements for quality and competence
ISO 15198: Validation of user quality control ISO 15198: Validation of user quality control procedures by the manufacturerprocedures by the manufacturer
CLSI C24CLSI C24--A3: Statistical Quality Control for A3: Statistical Quality Control for Quantitative Measuring Processes Quantitative Measuring Processes –– Principles Principles and Definitionsand Definitions
CLSI EP23CLSI EP23--P: Laboratory Quality Control based P: Laboratory Quality Control based on Risk Managementon Risk Management
88
ISO 15189 Guidance for ISO 15189 Guidance for ““Assuring Assuring QualityQuality”” of Examination Processof Examination Process
5.6.1. 5.6.1. ““The laboratory shall design The laboratory shall design internal quality control systems that verify internal quality control systems that verify the attainment of the the attainment of the intended quality intended quality of resultsof results…”…”
Medical relevanceMedical relevance of laboratory tests is an of laboratory tests is an important consideration!important consideration!
Comparability of test results is important for Comparability of test results is important for medical relevance!medical relevance!
How define medical relevance?How define medical relevance?99
ISO 15189 “Assuring quality”
5.6.2 ..determine uncertainty of results, where relevant and possible
5.6.3 …ensure that results are traceable
5.6.4 …participate in interlaboratory comparisons
5.6.5 …if EQA not available, develop a mechanism for determining acceptability
5.6.6 For examinations performed using different procedures or at different sites, define a mechanism for verifying comparability of results
Particular issues assigned to this work group
What frequency of QC is sufficient?
How important are recommendations from manufacturers for QC? Should laboratory modify recommendations?
How often for EQC or PT?
How assure quality?
1212
(3) Validate MethodPerformance (CV,bias)
(1) Define Goals for Intended Use
(TEa, Dint)
(3a) Manufacturer’sClaims
(1a) Regulatory &AccreditationRequirements
(1b) Clinical and Medical Applications
(2) Select AnalyticMeasurement
Procedure
(2a) Traceability& Calibration
(2b) Manufacturer’sReference Methods
& Materials
(4) Design SQC(rules, N, F)
(5) FormulateAQC Strategy
(11) Improve AQCEffectiveness
(5b) Lab Evaluationof Residual Risk
(5a) Manufacturer’sRisk Analysis
(6) Develop AQC Plan
(10) Monitor AQCEffectiveness (f), EQA
(7) Implement AQC System
(6a) QC Toolbox
(8) Verify Attainmentof Intended Quality
of Test Results
(9) Measure Quality& Uncertainty
What frequency of QC?What frequency of QC?
CLSI guidance on defining run lengthCLSI guidance on defining run length
““An analytical run is an interval (i.e., a period An analytical run is an interval (i.e., a period of time or series of measurements) within of time or series of measurements) within which the accuracy and precision of the which the accuracy and precision of the measuring system is expected to be stable; measuring system is expected to be stable; between which between which eventsevents may occur causing the may occur causing the measurement process to be more susceptible measurement process to be more susceptible (i.e., greater risk) to errors that are important (i.e., greater risk) to errors that are important to detect.to detect.””
1313
Factors Affecting Run Length (1)Factors Affecting Run Length (1) Events and nonEvents and non--eventsevents
Event driven QCEvent driven QC
Known, scheduled and expected changesKnown, scheduled and expected changes
NonNon--event driven QCevent driven QC
Other things that happenOther things that happen
ParvinParvin’’ss conceptsconcepts
Ref: Ref: ParvinParvin, , GronowskiGronowski. Effect of analytical . Effect of analytical run length on QC performance and the QC run length on QC performance and the QC planning process. planning process. ClinClin ChemChem 1997;11:21491997;11:2149-- 2154.2154.
1414
Factors Affecting Run Length (2)Factors Affecting Run Length (2) Mode of OperationMode of Operation
Batch modeBatch mode
All patient specimens and control samples are All patient specimens and control samples are analyzed togetheranalyzed together
Patient results not reported until control Patient results not reported until control results are validatedresults are validated
Continuous modeContinuous mode
Patient results are being reported as they are Patient results are being reported as they are determineddetermined
Control samples are analyzed periodicallyControl samples are analyzed periodically
1515
Factors affecting Run Length (3) Cost-Effectiveness
Number of controls relative to number of patient specimens
Costs of repeat analyses for out-of-control runs
Cost for number of levels of controls
Factors Affecting Run Length (4)Factors Affecting Run Length (4) StrategiesStrategies
ManufacturerManufacturer’’s instructions may provide s instructions may provide minimum strategy minimum strategy –– e.g., 2 levels per daye.g., 2 levels per day
+ Event QC to assess significance of + Event QC to assess significance of changes in the testing processchanges in the testing process
+ non+ non--event QC to monitor process during event QC to monitor process during routine operationroutine operation
MultiMulti--stage QC for stage QC for ““startup,startup,”” ““monitoring,monitoring,”” and and ““patient data QCpatient data QC””
1717
Factors Affecting Run Length (5)Factors Affecting Run Length (5) StrategiesStrategies
Sigma QC Sigma QC –– relative amount of QC can be relative amount of QC can be related to methodrelated to method’’s sigmas sigma--performanceperformance
Risk analysis and residual risks Risk analysis and residual risks –– guidance for guidance for susceptibility testingsusceptibility testing
Consensus of experts Consensus of experts –– professional practice professional practice standardsstandards
Experienced judgment Experienced judgment –– knowledgeable analyst knowledgeable analyst has expertise about stability and susceptibility of has expertise about stability and susceptibility of testing processtesting process
1818
Run length and frequency of controls
Models
Measures Statistical
Assumptions
Empirical
Economic
Residual risks
Error frequency, f Q/P
Strategies
Sigma
Risk analysis
Consensus of experts
Event+Monitor
Manuf. recommendations
Batch processing
Experienced judgment
Residual risksEvent only
Cost of repeat analysis
Operations
Batch mode
System stability
Continuous mode
Reporting interval
Unexpectedevents
LIS
Susceptibility
Batch size
Components
Residual risks
Maintenance
Calibration, reagents
Analyte stability
Events
Operators
PartsLab conditions
Stats
QC performance goals
1919
What frequency of QC is sufficient? Recommendations (1)
Strategy
Define length of run
In terms of time, numbers of patient samples, mode of operation
Importance of “events” or changes that occur with the process that require verification by controls
Medically important concentrations for controls
General practice to use two levels of controls
Sometimes advisable to have three levels
Many factors to consider to optimize run length or frequency of QC
What frequency of QC? Recommendations (2)
For small runs, utilize “batch” strategy
QC at beginning
QC at end
Release results after inspect all controls and reviewing patient results when necessary
What frequency of QC? Recommendations (3)
For large runs, highly automated systems with continuous reporting of results
Controls at beginning of run
Right QC design to detect medically important errors
+ Controls for events
e.g., Change of reagent lots
+ Controls to monitor performance during run
Or, possibly use mean or median of patient data to monitor stability during run
+ Controls at end of run
How important are the manufacturer’s QC directions?
Recommendations (4)
Provide minimum requirements that the laboratory must satisfy
E.g., calibration, preventive maintenance, etc.
Laboratory is still responsible for design of IQC system
Intended clinical use
Observed method performance
Necessary QC rules and Numbers of measurements
How important manufacturers QC directions? Recommendations (5)
Need for “independent” control
“Third party control
Traceability is an important responsibility of manufacturer
Calibration materials and process
Verification/validation of method performance is an important responsibility of the laboratory
EQA/PT important responsibility in monitoring/measuring accuracy over time
“Commutability” important characteristic of materials
How often EQA/PT? Recommendations (6)
At least monthly
With fast turnaround of results to be useful for identifying bias and making improvements in the laboratory
Approved EQA program preferred
ILAC G13:08/2007
ISO 17043
Most essential information – bias observed vs “assigned value”
Other issues of interest
Medical relevance
“Intended use,” “intended quality of test results”
Traceability
Comparability of test results
Validation of method performance
Design of IQC
Available planning approach, tools
How assure quality?
2727
(3) Validate MethodPerformance (CV,bias)
(1) Define Goals for Intended Use
(TEa, Dint)
(3a) Manufacturer’sClaims
(1a) Regulatory &AccreditationRequirements
(1b) Clinical and Medical Applications
(2) Select AnalyticMeasurement
Procedure
(2a) Traceability& Calibration
(2b) Manufacturer’sReference Methods
& Materials
(4) Design SQC(rules, N, F)
(5) FormulateAQC Strategy
(11) Improve AQCEffectiveness
(5b) Lab Evaluationof Residual Risk
(5a) Manufacturer’sRisk Analysis
(6) Develop AQC Plan
(10) Monitor AQCEffectiveness (f), EQA
(7) Implement AQC System
(6a) QC Toolbox
(8) Verify Attainmentof Intended Quality
of Test Results
(9) Measure Quality& Uncertainty
ISO 15198 ISO 15198 Validation of QC ProceduresValidation of QC Procedures
QC procedures shall be validated to assure QC procedures shall be validated to assure that failures are not a hazard to patients that failures are not a hazard to patients
Recommends use of risk analysisRecommends use of risk analysis
Conventional statistical quality control Conventional statistical quality control procedures (e.g., as described in CLSI procedures (e.g., as described in CLSI C24) are considered adequateC24) are considered adequate
Validation may be based on simulated effects Validation may be based on simulated effects of errors on performance dataof errors on performance data
2828
CLSI C24 QC Planning Process
2929
Define quality specifications for test
Select appropriate control materials
Determine method performance
Identify quality control strategies
Predict QC performance
Specify goals for QC performance
Select QC to satisfy goals
CalculateSigma
%TEa-%Bias%CV
UtilizeSigma-metricQC Selection
Tool
SigmaSigma--metrics QC Selection Toolmetrics QC Selection Tool 2 Levels Control2 Levels Control
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0.0 1.0 2.0 3.0 4.0
1.65 2.65 3.65 4.65 5.65
13s/2of32s/R4s/31s/6x 0.07 ----- 6 113s/22s/R4s/41s/8x 0.03 ----- 4 213s/22s/R4s/41s 0.03 ----- 4 112.5s 0.04 ----- 4 112.5s 0.03 ----- 2 113s/22s/R4s 0.01 ----- 2 113s 0.00 ----- 2 113.5s 0.00 ----- 2 1
Pfr Ped N R
Pro
babi
lity
for
Rej
ecti
on (
P)
Systematic Error (SE, multiples of s)
Sigma Scale
DesirableError
DetectionDesirable
FalseRejection
3 4 5
3030
EP22, EP23 on Risk Analysis
Original purpose of CLSI project was to develop scientific approach for defining frequency of QC
Adopted “risk analysis” approach
Failure-modes and risk should provide guidance on need for control mechanisms and frequency of QC
Analytical QC Plan should be the outcome of the risk analysis process
Important Considerations in Important Considerations in Future QC SystemsFuture QC Systems
DesignDesign
Is there a scientific basis and approach for selecting Is there a scientific basis and approach for selecting parameters and setting limits on basis of parameters and setting limits on basis of intended intended quality of resultsquality of results??
ValidationValidation
Is there an objective approach for assessing the Is there an objective approach for assessing the reliability of technical and medical decisionsreliability of technical and medical decisions on on control status?control status?
ControlControl
Is there a quantitative process for monitoring and Is there a quantitative process for monitoring and verifying the verifying the attainment of intended quality of attainment of intended quality of test resultstest results??
3232