Embed Size (px)
DESCRIPTION
This presentation from IVT's 4th Annual reviews what to do when you have an exception, critical vs. non-critical exceptions, and learning how to prevent exceptions.
Citation preview
Validation Exceptions – Managing life’s little (and big) unexpected events( g) p
Brian K. Nunnally, Ph.D.
What is an exception?p
An error or a failure which occurs duringAn error or a failure which occurs during Verification or Validation
William Shakespeare: "It is neither good nor bad, but thinking makes it so."
Types of Exceptionsyp p
Three levels of exceptions:Three levels of exceptions:CriticalN iti lNon-criticalSimple
Simple Exceptionsp p
Simple exceptions are documentation orSimple exceptions are documentation or protocol errors
Th f d i t tiThese errors are found prior to execution or are “obvious” errors or typos
These e ceptions ha e no impact on theThese exceptions have no impact on the validation (e.g. no risk)
Simple Exceptions examplesp p p
Wrong test selected for productWrong test selected for productIncorrect specification for productLoss of power to the equipment being tested.
Other examples from the participants?Other examples from the participants?
Non-critical Exceptionsp
Non critical exceptions are errors in theNon-critical exceptions are errors in the protocol or execution of the validation which have no impact on the validationhave no impact on the validation
These errors are found during or after executionA t i d d t d t i i tAssessment is needed to determine no impactNeed to document the risk (or lack thereof)
Non-critical Exceptions lexamples
Operator not trained to perform the operationOperator not trained to perform the operationAn interruption in the computer system or th i t’ fthe equipment’s performance
Other examples from the participants?
Critical Exceptionsp
Critical exceptions are errors which have anCritical exceptions are errors which have an impact on the validation
Th f d d i ft tiThese errors are found during or after executionAssessment is used to determine impactN d t d t th i kNeed to document the risk
Critical Exceptions examplesp p
Acceptance criterion failureAcceptance criterion failureAn interruption in the computer system or th i t’ fthe equipment’s performance
Other examples from the participants?
Basic Exception Process Flowp
SME involvement throughout isSME involvement throughout is criticalYou cannot document enough, even if all of it does not make iteven if all of it does not make it into the final event write-upCAPA may not be needed for simple events, but consider ifsimple events, but consider if there are many of them or the same ones repeatedly
General Process Considerations
Commence exception resolution process as soonCommence exception resolution process as soon as event is observedInvolve SMEs/QA throughout process asInvolve SMEs/QA throughout process as appropriate
Get agreement on next steps and/or resultsg pPrior to execution, or after the completionIncluding all elements of the process
Immediately notify the relevant SME/QA if there is any potential Product/Material impactConduct a risk assessment to determine impact and actions
What is the risk and impact of the exception?the exception?
Event severity x Testing Stage = ImpactEvent severity x Testing Stage = ImpactEvent Severity:
What kind of event is it? How was the event caused?
Testing Stage: Where are we in the validation/verification process? Installation? Performance Qualification?
Event Typesyp
Typographical errorsTypographical errorsThese errors are found prior to execution or are “obvious” errors or typos
External IssuesExternal IssuesFailures caused by factors external to the process or system under test
Execution ErrorsExecution ErrorsFailures to follow instructions within the Test Document or referenced procedures.
Document Generation ErrorsDocument Generation ErrorsIncorrect detail in procedures, acceptance criteria or referenced documentation
Acceptance Criteria FailuresAcceptance Criteria FailuresPre-defined acceptance criteria have not been achieved.
Testing Stageg g
New system installation and the exceptionNew system installation and the exception does not impact the validation intentN t i t ll ti d th tiNew system installation and the exception does impact the validation intentNew or Existing system qualificationProcess ValidationRe-qualification
Should testing stop?g p
Depends on the impact scoreDepends on the impact scoreDepends on consultation with SMEs and QAAre you willing to live with a failure? Will it open more questions than it solves?p q
Note: It does not depend on the timeline!Note: It does not depend on the timeline!
Exception example #1p p
An acceptance criterion was failedAn acceptance criterion was failed.However, the acceptance criterion was poorly set (limited knowledge was available to set the acceptance ( g pcriterion).
What level of exception is this?What do you do?
Exception example #2p p
The acceptance criterion was passed but theThe acceptance criterion was passed, but the result was right at the limit.
Assay variability is high for the test.Assay variability is high for the test. Is this an exception? Why or why not?What do you do?What do you do?
Exception example #3p p
During validation of the cleaning process theDuring validation of the cleaning process, the Operator executing the protocol failed to sample the correct locations. Is this an exception? Why or why not?What do you do?What do you do?
Preventing Exceptions
The key to preventing tiexceptions
Understanding and reducing variationUnderstanding and reducing variation.Systems thinking (managing the flow through the entire process as an entity)entire process as an entity).
Types of Variation (Including Sampling Analytical and Production)Sampling, Analytical and Production)
Variation in Quality Characteristic (specific activityVariation in Quality Characteristic (specific activity, potency, pH, temperature…etc.Process variation Variation in the way ourProcess variation. Variation in the way our processes behave and are managed or controlled.Variation in volume flowing through a process.Variation in volume flowing through a process.
Why focus on variability?y y
Reduced variation in Quality CharacteristicsReduced variation in Quality Characteristics reduces rejects, rework and deviations. It provides high First Pass Yields.Reduced variation in Volume (smoother flow) increases output capacity and reduces capital requirementsrequirements. Reduced variation in Volume (smoother flow) decreases cycle time and Work-In-Progress.y g
Which one will be more likely to fail an acceptance criterion?fail an acceptance criterion?
2600
Day shift Following night shift
1800
2000
2200
2400
2600
t Ave
rage
1000
1200
1400
1600
0 0 0 0 0 0 0 0 0 0 0 0 0
1-Po
in
0 0 0 0 0 0 0 0 0 050 70 90 110
130
150
170
190
210
230
250
270
290
310
330
350
370
390
410
430
450
470
490
More examples…pPotencyy
pH
Correlation?
Before ProjectR2 = 0.2594
23
25
27
Before Project
15
17
19
21
3000 4000 5000 6000 7000 8000
R2 = 0.9107
27
29After Project
19
21
23
25
15
17
3000 4000 5000 6000 7000 8000
What should you do?y
CRITICAL ERRORS PER WORKER PER WEEK
WEEK1 2 3 4 5 6 7 8 9 10 11 12 TOTAL
Mary 0 0 0 0 0 0 0 0 0 0 0 0 0
Joe 0 0 0 0 0 0 0 0 0 0 0 0 0
Eva 1 0 0 2 0 0 3 0 0 1 0 0 7Eva 1 0 0 2 0 0 3 0 0 1 0 0 7
Fred 0 0 0 1 0 0 2 0 0 0 0 0 3
Jim 0 0 0 0 0 0 0 0 0 0 0 0 0
Ed 0 0 0 0 0 2 0 0 0 0 0 0 2Ed 0 0 0 0 0 2 0 0 0 0 0 0 2
Kate 0 0 0 0 0 0 0 0 0 0 0 0 0
Carl 0 0 0 1 0 0 0 0 0 0 0 0 1
TOTAL 13TOTAL 13
The lesson from Tribus
Bring a team together to focus on ensuring the aimBring a team together to focus on ensuring the aim is clear to all participantsFocus on systems to reduce and remove possibilityFocus on systems to reduce and remove possibility for errorsImprove systems as a way to prevent human error.Improve systems as a way to prevent human error.Be very careful when assigning blame to human error or to a specific personp p
Expectations on Exceptions
Three keys to exceptionsy p
DocumentDocumentDocumentDocument!
Keep it simple!p p
Document the standard questions:Document the standard questions:WhoWh tWhatWhen Why WhereHow
Other requirementsq
Root Cause DeterminationRoot Cause DeterminationIf root cause is unknown, a study should be conductedco duc ed
Impact DeterminationHow severe is the exception?How severe is the exception?What is the impact to intent?
Corrective Action DeterminationCorrective Action DeterminationHow will this be prevented in routine use?
Re-Testing / Re-executionRe Testing / Re executionIs it necessary? Why or why not?
More exceptions examples…
Exception example #4p p
During the annual re qualification / re verification ofDuring the annual re-qualification / re-verification of the performance of an autoclave which has been used for manufacturing of product, the g p ,thermocouples did not achieve the required temperature. Is this an exception? Why or why not?What do you do?
Exception example #5p p
During the verification of the installation of VesselDuring the verification of the installation of Vessel which has not been used to manufacture product, power was lost during testing of the agitator. p g g gIs this an exception? Why or why not?What do you do?What do you do?
Exception example #6p p
During the re validation / re verification of CleaningDuring the re-validation / re-verification of Cleaning validation of a process which has NOT been used to manufacture product, some samples were p , paccidentally not taken. Is this an exception? Why or why not?p y yWhat do you do?
Exception example #7p p
During the execution of an alarm checkDuring the execution of an alarm check (operational verification) for a system not used to manufacture product, the test document had the p ,incorrect sequence of events. Is this an exception? Why or why not?p y yWhat do you do?
