Risk-Based QC for Clinical Laboratories

Learning Objectives

• Describe the concept of risk management in laboratory medicine

• Summarize the basic process for developing a risk-based QC plan

• List resources that can be utilized as an aid to the process

• State the five components evaluated in a risk assessment

What is risk?

What is Risk?

• Combination of the probability of occurrence of harm and the severity of that harm (ISO 15190; ISO/IEC Guide 51)

• Harm - physical injury or damage to the health of people, or damage to property or the environment [ISO/IEC Guide 51:1999, definition 3.3]

• Severity - measure of the possible consequences of a hazard [ISO 14971- 2007 definition 2.25]

• Hazard - potential source of harm [ISO/IEC Guide 51:1999, definition 3.5]

Why now? Risk assessment for laboratories?

• Consolidation and decreasing budgets

• One size doesn’t fit all

• What is appropriate QC frequency?

Other Factors Favoring Risk Approach

• Protect the patient’s well being

• Guard the laboratory’s reputation

• ISO 15189 requires risk to be addressed

4.14.6 Risk Management

“The laboratory shall evaluate the impact of work processes and potential failures on examination results as they affect patient safety, and shall modify processes to reduce or eliminate the identified risks and document decisions and actions taken.”

ISO 15189: 2012

7

5.6.2.2 Quality Control Materials

“Quality control materials shall be periodically examined with a frequency that is based on the stability of the procedure and the risk of harm to the patient from an erroneous result.”

ISO 15189: 2012

8

Creating a Risk-Based Plan

VERIFY EFFECTIVENESS

IMPLEMENT THE PLAN

DEVELOP THE QC

PLAN

RISK ANALYSIS

PLANNING

ENLIGHTENMENT

Enlightenment • ISO 14971: 2012

Medical devices – Application of risk management to medical devices

• ISO/TS 22367: 2009 Medical laboratories – Reduction of error through risk management and continual improvement

• CLSI EP23-A Laboratory QC Based on Risk Management

• Westgard: Six Sigma Risk Analysis • Testing Documentation…

Gathering information • Internal audits • Laboratory complaint logs/ user surveys • Instructions for use (IFU) • Historical laboratory QC, calibration and PT records • Instrument manual • Manufacturer blogs or webpages for users • FDA Medical Device report database search

– https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfMDR/Search.cfm

• PT summary reports • Inter-laboratory reports provided by control

manufacturers

Specific Information from Manufacturers

• Reliability scores • Mean time between failure (MTBF) • Black box processes when necessary • Information about on-board (embedded) controls • Information about electronic controls

Planning

13

Before you start

Put together a small quality team – 5-7 members – Stakeholders – Desirable characteristics

14

Team Members: Key Characteristics • Inquisitive • Team player • Knowledgeable • Open minded • Creative thinker • Good communicator • Pragmatic • Able to finish a task

Team Dynamics

• Have a facilitator to keep team on track • Have objectives for each team meeting • Brainstorm – open, non-judgmental

discussion • All members have equal standing

Risk Analysis

17

Some Sources of Risk

• Communication processes • Environmental • Knowledge of test operators • Management commitment • Outside influences • Resources • Technical components • Process/Procedure • Financial – decisions based on cost not quality

18

Process Map: Example

19

5 Components Evaluated

– Specimen

– Environment

– Reagent

– Test system

– Personnel

20

Test System Pre-Analytical Calibration

Calibration Verification

Maintenance daily, w / m / semi-annual Electrical Monitoring, Surge Protection Dedicated Circuit Water Supply (if required) Water quality Water integrity (air) Humidity (mfr requirement) Temp. (mfr requirement)

PT Performance

Calibration of ancillary equip

Analytical Consumables (quality)

Reagent dispense

Sample dispense

Reaction Chamber Temperature Measurement Filter wheel Light source integrity Clot detection Interfering substances Quality Determination QC approach used QC materials used QC frequency QC Rules Patient risk (# patients between QCs)

Post-Analytical Results: review / approve

Result Transmission

Retrospective Review

Trend Analysis

Sigma Metrics

Frequency of recalibration

Freq. of Device Failures

Verification of Test Results

Information Gathering / Analysis Test System for POCT

Pre-Analytical Calibration

Calibration Verification

Maintenance daily, w / m / semi-annual Electrical Monitoring, Surge Prot Dedicated Circuit, Battery Water Supply (if required) Water quality Water integrity (air) Humidity (mfr requirement) Temp. (mfr requirement)

PT Performance

Calibration of ancillary equip

Analytical Consumables (quality)

Reagent dispense

Sample dispense

Reaction Chamber Temperature, Black Box Measurement Filter wheel Light source integrity Clot detection Interfering substances Quality Determination QC approach used QC materials used QC frequency QC Rules Patient risk (# patients between QCs)

Post-Analytical Results: review / approve

Result Transmission

Test Report

Retrospective Review

Trend Analysis

Sigma Metrics

Frequency of recalibration

Freq. of Device Failures

Verification of Test Results

Analysis Brainstorming a POCT Test

PROCESS MAPPING

OR FISHBONE DIAGRAM

Analytical Reaction Chamber Black Box Measurement Interfering Substances Quality Determination QC approach used QC materials used QC frequency Patient risk (# patients between QCs)

Possible Hazard Points

What Can Go Wrong?

Analysis Brainstorming a POCT Test

PROCESS MAPPING

OR FISHBONE DIAGRAM

Analytical Reaction Chamber Black Box Measurement Interfering Substances Quality Determination QC approach used QC materials used QC frequency Patient risk (# patients between QCs)

Possible Hazard Points

• Does the the device manual or the product insert describe in detail the analytical sequence?

• Does the product insert, the device manual or the manufacturer describe in sufficient detail how the function checks work and are themselves

• Has the manufacturer provided a product reliability score or the mean time between failure?

Analysis Brainstorming a POCT Test

PROCESS MAPPING

OR FISHBONE DIAGRAM

Analytical Reaction Chamber Black Box Measurement Interfering Substances Quality Determination QC approach used QC materials used QC frequency Patient risk (# patients between QCs)

Possible Hazard Points

• Does the product insert….

• How is the patient cleared for interfering substances?

Analysis Brainstorming a POCT Test

PROCESS MAPPING

OR FISHBONE DIAGRAM

Analytical Reaction Chamber Black Box Measurement Interfering Substances Quality Determination QC approach used QC materials used QC frequency Patient risk (# patients between QCs)

Possible Hazard Points

• Does the product insert….

• How is the patient cleared for … • What QC modality is used?

Function checks only? Electronic QC only? Traditional QC?

• First party, second party or third party controls used?

• Embedded control used? • Liquid or solid phase QC? Shortcomings? • How frequently is QC frun? • Can errors/mistakes/failures/hazards be

detected immediately? • How are QC limits established? • What QC rules are used? • How many patient test results are

reported between QC testing events?

Analysis Brainstorming a POCT Test

PROCESS MAPPING

OR FISHBONE DIAGRAM

Analytical Reaction Chamber Black Box Measurement Interfering Substances Quality Determination QC approach used QC materials used QC frequency Patient risk (# patients between QCs)

Possible Hazard Points

• Does the product insert….

• How is the patient cleared for … • What QC modality is used?

Function checks only? Electronic QC only? Traditional QC?

• First party, second party or third party controls used?

• Embedded control used? • Liquid or solid phase QC? Shortcomings? • How frequently is QC frun? • Can errors/mistakes/failures/hazards be

detected immediately? • How are QC limits established? • What QC rules are used? • How many patient test results are

reported between QC testing events?

Risk Exposure: # Patients Between QC Events • Consider cost and feasibility of retesting versus

cost of increased control testing

• How many patient samples are affected by an error or malfunction and is undetected until the next QC event? Assume a lab testing 500 samples/day.

• 1 / month: exposure = 15,000 patients • 1 / week: exposure = 3500 patients • 1 / day: exposure = 500 patients • 1 / 8-hour shift: exposure = 63 patients • 2 / 8-hour shift: exposure = 31 patients • 1 / patient sample: exposure = 1patient

– Bracketed QC

• 1 / 20 patient samples exposure = 20

Grading and Ranking Risk

• Can use FMEA approach – Grade (score 1-5 or 1-10) for occurrence,

severity, detection – Multiply scores to get Risk Priority Number

(RPN) – Rank for importance by RPN and Acceptance

criteria • How much risk is acceptable? Set by team

Alternative grading system

Adapted from ISO 14971

Negligible Minor Serious Critical Catastrophic

Frequent not ok not ok not ok not ok not ok

Probable ok not ok not ok not ok not ok

Occasional ok ok ok not ok not ok

Remote ok ok ok ok not ok

Inconceivable ok ok ok ok ok

Severity of harm

Prob

abili

ty

Develop and Implement Risk Mitigation Plan

31

Response to Risk Findings

• Get more information • Accept the risk – do nothing • Reallocate resources • Eliminate the risk entirely • Transfer the risk • Develop and implement a Risk Mitigation

Plan

32

Risk Mitigation Plan Must Be Based On: • Sources of potential device failures • Potential for errors/mistakes and device

failures • Impact of failures and errors/mistakes • Ability to detect failures and

errors/mistakes • Residual risk post mitigation • Unique conditions 33

Unique Risk Conditions Affecting QC Frequency in the Plan

• Immediate application of test result • Result leads to significant medical response • Result affects diagnosis, prognosis or treatment • Low volume, infrequently performed test • Technique sensitive tests • Low sigma (<3.0) tests • Analyte stability • Use of first or second party control materials as primary

controls • Use of electronic or imbedded controls as primary

controls

Consider Increasing QC Frequency

When: • Risk of reporting an erroneous result is

moderate to high • Risk that an erroneous result can harm a

patient • Actionable residual risk is present

Verify Effectiveness

36

Quality Assessment • On-going review for effectiveness

– No periodicity required • Resource documents (a few)

– Historical QC data – PT records – Patient result review – Specimen rejection logs – Preventive action/corrective action records – Competency assessment records

37

The Risk-Based Plan

VERIFY EFFECTIVENESS

IMPLEMENT THE PLAN

DEVELOP THE QC

PLAN

RISK ANALYSIS

PLANNING

ENLIGHTENMENT

Evaluation of residual risk

• Re-estimate risk potential after mitigation to evaluate residual risk – Is it acceptable?

39

Records

No…not these

Records This might be a bit much…..

4.13 Control of Records

• “The laboratory shall have a documented procedure for

identification, collection, indexing, access, storage, maintenance, amendment and safe disposal of quality and technical records…

Records shall include the following… n) Risk management records”

ISO 15189: 2012

42

As with all ISO quality systems –Document, document, document

To summarize: • Form a team • Set a timeline and objectives • Set the scope • Gather relevant information • Perform the analysis

– Identify hazards and prioritize the importance • Evaluate the risk, decide on and implement

mitigations • Verify effectiveness

To summarize

• Keep it simple

• Stay on point

• Do not rely solely on manufacturer

templates

Most important

When doing the risk assessment

and making decisions about risk

and what is acceptable quality,

ALWAYS be guided by what is best

for your patients.

References

• ISO 14971:2012 Medical devices – Application of risk management to medical devices

• ISO 15189:2012 Medical laboratories – Requirements for quality and competence

• ISO/TS 22367 Medical laboratories –Reduction of error through risk management and continual improvement