Risk Assessment Tools for Identifying Hazards and

Risk Assessment Tools for Identifying Hazards and

Evaluating Risks Associated with

IVD AssaysRobert C. Menson, PhD

AACC Annual MeetingPhiladelphia, PA

22 July 2003

© Menson & Associates, Inc AACC 2003 Edutrak 2

What Risks Must Be Managed?

Risk to safety of patients, users, handlers

Business Regulatory

Product liability


Elements of the Risk Management Process

Risk Analysis

Risk Evaluation

Risk Control

Post-Production Information

Ass

essm

ent

Man

agem

ent


Risk Assessment Tools

Risk MatrixFTA=Fault Tree AnalysisPHA= Preliminary Hazard AnalysisFME(C)A=Failure Mode Effects (Criticality) AnalysisHAZOP=Hazard Operability AnalysisHACCP=Hazard Analysis and Critical Control Point


Applications of Risk Analysis

DesignProductionPremarket NotificationsComplaintsMDRChange ControlFailure AnalysisEtc.


Risk ManagementLife Cycle

Concept & Feasibility Development Scale-Up & Transfer Production

System Level Assessment

Design Assessment

Process Assessment

Customer FeedbackAssessment


Risk Management& Design Control

DesignControl

RiskAssessment

Requirements Plan SpecificationsTest

Methods &Results

ProductionsMethods

Change Records

RiskManagement

Plan

PreliminaryHazard

Analysis

Detailed Analysis(FMEA, FTA, HACCP, HAZOP)

RiskManagement

Report

Risk Reviews

DevelopmentPlanningConcept& Feasibility Scale-Up & Transfer Production


Key Concepts of Risk

The frequency of the potential harm;How often the loss may occur;

The consequences of that loss;How large the loss might be;

The perception of the loss;How seriously the stakeholders view the risk that might affect them.


Risk Management Terms

Intended Use/PurposeHarmHazardRiskResidual RiskTolerable Risk


Intended Use/Purpose

Use of a Product, Process or Service in accordance with the specifications, instructions and information provided by the manufacturer

ANSI/AAMI/ISO 14971:2000, definition 2.5


Harm

Physical injury or damage to health of people, or damage to property or the environment.

ISO/IEC Guide 51:1999, definition 3.3“Guidelines for inclusion of safety aspects in

standards.”


Hazard

Potential source of Harm

ISO/IEC Guide 51:1999, definition 3.5


Risk

Combination of the probability of occurrence of harm and the severity of harm.



Residual Risk

Risk remaining after protective measures have been taken.



Tolerable Risk

Risk which is accepted in a given context based on the current values of society.



Hazard Identification

HazardIdentifcation

BrainstormingPHAFTA

FMEA

LawsCodes

Standards

EventsMDRs

AccidentsEtc

Listof

Hazards


Risk Matrix

HIITOccasional

IILTRemote

HHILProbable

HHILFrequent

SevereMajorMinorNegligible

SeverityFrequency


ALARP CurveIn

crea

sing

Pro

babi

lity

of O

ccur

renc

e

Intolerable Region

ALARP

BroadlyAcceptable

Region

MaximumTolerable

Risk

Increasing Severity of Harm


FMEA vs FTAFMEA FTA

Assumes failure of the functionality of a productIdentifies part/module failure as cause of functional failure

Assumes component or part failureIdentifies functional failure as a result of part failure


FMEA vs. FTADone for entire design

Systematic way to predict new problems

A bottoms-up analysis

People expect the same results from FTA which is not true

Too difficult to do for entire designSystematic way to predict causes for usually know problems

A top down analysis

People do not expect the same results from and FMEAOften a fault tree is used for a problem or an accident.

FMEA FTA


FTAAssumes fault and analyzes possible causesConnection tool for PHA* to subsystems or modulesTop downDeductiveEvaluate system (or subsystem) failuresConsidered more structured than FMEAGraphical presentation--visual picture*Preliminary Hazard Analysis


FailuresPrimary--Due to internal causes that include poor design or use of inappropriate materialsSecondary--Due to failures in the operation that include equipment failureControl--Due to failures in the systems that are in place to protect the quality and safety

e.g. raw material outside specificationfailure of safety switchfailure of test method


FTA Basic SymbolsBasic Flow

FAULT

AND

OR

Fault in a box indicates that it is a result of subsequent faults

Connects a preceding fault with a subsequent fault that could cause a failure

Connects two or more faults that must occur simultaneously to cause the preceding fault


FTA During Design P O O R F ITIN H A N D

D O E S N O TR E S T R IC TS P IL L A G E

O R

O R

A S P E C T R A T IOT O O H IG H

M A J O RD IA M E T E R

T O O L A R G E

A S P E C T R A T IOT O S M A L L

B A S EF L A T N E S S

B A S E S IZ ET O O S M A L L

R IM T O F IL LIN S U F F C IE N T

CompetingDesign Criteria


FTA Lab Failure

Patient Death OR

Other

Lab Error OR

Outliers

Systematic

Random

OR

CalibrationError

Interference

Other


FTA During ReliabilityHAZARD

SYSTEMFAILURE

DRIFT>LIMIT

CMPT A FAILS CMPT B FAILS CMPT C DRIFTS REFERENCEDRIFTS

AN

D

OR

OR 4. x 10-9

4. x 10-91. x 10-16

3. x 10-9 1. x 10 -92. X 10 -85. x 10 -9


What is FMEA?What is FMEA?

FMEAFailure Mode and Effects Analysis

FMECAFailure Mode Effects and Criticality Analysis

FMEAFailure Mode and Effects Analysis

FMECAFailure Mode Effects and Criticality Analysis


What is FMEA?

Powerful prioritization toolInductiveHigh effective tool for identifying critical quality attributesHigh structuredMethodicalBreaks large complex designs into manageable steps


FMEA

Bottom up approachEvaluates specific failuresDetailed analysis tool

Use in conjunction with PHA and FTA Complements FTA

May lead to different failure results


Advantages of FMEA

Less analyst dependent than FTAAllows direct criticality assessment of componentsValuable troubleshooting aidIdentifies areas of weak designIdentifies areas of high riskPrevention planningIdentifies change requirements


Disadvantages of FMEADoes not consider operator errorTediousMay not apply to all systems--especially softwareMay require extensive testing to gain informationMay miss some failure modes

Time pressuresInformation missing


DefinitionsCriticality -- Weighting of hazard severity with the probability of failureSeverity -- Seriousness of effect through its impact of the system functionOccurrence -- Likelihood a specific failure will be caused by a specific cause under current controlsVerification -- Ability of the current evaluation technique to detect potential failure during designDetection -- Ability of the current manufacturing controls to detect potential failure before shipping


DefinitionsDefinitions

Risk Priority Number (RPN)

= (S) x (O) x (D) or (V)Severity (S)Likelihood of occurrence (O)Likelihood of detection (D)Likelihood of verification (V)

Risk Priority Number (RPN)

= (S) x (O) x (D) or (V)Severity (S)Likelihood of occurrence (O)Likelihood of detection (D)Likelihood of verification (V)


FMEA ExampleP O T E N T IA L F A IL U R E M O D E A N D E F F E C T S A N A L Y S IS

Im u n n o a s s a y D e v ic e F M E A N u m b e r S u b s y s te m P a g e o f X C o m p o n e n t _ S u p p o rt_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ R e s p o n s ib ility P re p a re d B y P ro c e s s F M E A D a te (O rig .) (re v .) C o re T e a m : _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

D e vic e / P o te n tia l P o te n tia l S P o te n tia l O C u rre n t D R R e c o m m e n d e d R e s p o n s ib ilit

y A c tio n R e s u lts

F u n c tio n F a ilu re E ffe c t(s ) C a u s e (s ) C o n tro ls P A c tio n (s ) a n d T a rg e t A c tio n s R M o d e o f F a ilu re o f F a ilu re N C o m p le te D a te T a k e n S O D P

N S u p p o rt S y s te m

H o ld s A n tib o d y

-

S e p a ra te s A n a ly te

F a ils to S e p a ra te a n a ly te

N o A n s w e r E rro n e o u s R e s u lt F a ls e H i F a ls e L o

4

79

W ro n g M a te ria l

3

33

5

55

6 0

1 0 5 1 3 5

D e v e lo p s C o lo r


Process FMEA Process FMEA Identifies potential product-related process failure modes Assesses the potential customer effects of the failuresIdentifies the potential internal and external manufacturing or assembly process causesIdentifies process variables on which to focus controls for

reducing occurrence or increasing detection of the failure conditions

Identifies potential product-related process failure modes Assesses the potential customer effects of the failuresIdentifies the potential internal and external manufacturing or assembly process causesIdentifies process variables on which to focus controls for

reducing occurrence or increasing detection of the failure conditions


Sources of Process Defects?

Sources of Process Defects?

Omitted processingProcessing errorsErrors setting up work piecesMissing partsWrong parts

Omitted processingProcessing errorsErrors setting up work piecesMissing partsWrong parts

Adjustment errorAdjustment errorProcessing wrong Processing wrong work piecework pieceMisMis--operationoperationEquipment not set Equipment not set up properlyup properlyTools and fixtures Tools and fixtures improperly preparedimproperly prepared


FMEA Summary

Powerful tool for summarizing:Important modes of failureFactors causing these failuresEffects of these failuresRisk prioritizationIdentifying plan to control and monitorCataloging risk reduction activities


HAZOP

Haz ard and Op erability StudyBottom up analysisDeviations from design intentionsSystematic brainstorming based on guide words


HAZOPGuide Words

No/NotMoreLessAs well asOther than


Hazop ModelDesign Statement

MaterialActivity Destination

Transfer Powder Hopper


HAZOP

LiquidWrong powder

Other than

Larger tankInaccurate gage

Pump fastMore

Valve closedHopper full

Tank emptyValve closedLine blockedPump broken

NoHopperPowderTransfer


HAZOP Plan

Interlock

Operator trainingPM

Low

Med

Med

Valve closedLine blockedPump broken

Powder flow

NO

WhoActionRiskCausesDeviationGuide


HACCP

Risk Management SystemBiological HazardsChemical HazardsPhysical Hazards

RequiresPrerequisite Quality System Program Traditionally GMPs


HACCP Steps1. Conduct hazard analysis and identify

preventive measures2. Identify Critical Control Points.3. Establish critical limits4. Monitor each critical control point5. Establish corrective action to be taken when

deviation occurs6. Establish verification procedures7. Establish record-keeping system


HACCP Decision TreeP re v e n tiv eM e a s u re s

E x is t

S te p E lim in a teo r R e d u c e

C ritic a lC o n tro l P o in t

Is c o n tro lN e c e ss a ry fo r

S a fe ty

S to pN o t a C C P

M o d ify S te p

N o

N o

Y e sY e s

Y e s

E x ce s sC o n ta m in a tio n

P o s s ib le

N o

N o

S u b s eq u en tS te p s e lim in a te

o r R e d u c eY e s

Y e s

N o


HACCP Worksheet

Biological

Chemical

Physical

Is this step a critical control point? (Y/N)

What preventative measures can be applied to prevent

the significant hazards?

Justify your

decisions for column

3.

Are any potential safety

hazards significant?

(Y/N)

Identify potential hazards

introduced, controlled or

enhanced at this step(1)

Material/processin

g step

654321

Product Description:

Method of Storage and Distribution

Intended Use and Consumer

Firm Name:

Firm Address:


HACCP Plan

(7) Who

(6)Frequency

(5)How

(4)What

(10)Verificat-

ion

(9)Record

s

(8)Correct-

iveActions

Monitoring

(3)Critical

Limits for each

Action

(2)Significant Hazards

(1)Critical Control Point

Product Description:

Method of Storage and Distribution:

Intended use and Consumer:

Firm Name:

Firm Address


Contact Information

Robert C. Menson, Phd

Menson & Associates, IncQuality Systems and Compliance

For FDA Regulated IndustriesPO Box 480

Newport, RI 02840Phone: 1.401.847.6320

Email: [email protected]


IVD Hazards, Patient Harmand Risk Controls

Donald M. Powers, PhD

AACC Annual MeetingPhiladelphia, PA

22 July 2003

Documents

Risk Assessment Tools for Identifying Hazards and