It's What You Don't Know That Can Hurt You...

Ray Garrison

Who am I?Ray Garrison

Director of Quality AssuranceStryker Orthopaedics – Alameda CA

-Over 24 years in Manufacturing, Quality d R l C liand Regulatory Compliance

-MBA –University of Florida-Six Sigma Black Belt-Held leadership positions at Biomet and

J h & J hJohnson & Johnson

The views expressed in this presentation are my own and do not reflect the views

Ray Garrison

p p yof my employer or other organizations with which I am or have been affiliated.All cited trademarks are the property of their respective owners.

Framing the discussionIn the time we have together today, we will explore:will explore:- Probabilities and Rationalizations

K d U k I t- Known and Unknown Incompetence- Identifying more of the Unknown- How to quickly assess and respond to events

Ray Garrison

Framing the discussionUndesirable, unknown,

l d f tunplanned for events can and do happen.

What you don’t know is farWhat you don’t know is far more relevant than what

d kyou do know.

Think about this for a moment….Ray Garrison

Undesirable, unknown, unplanned for events can and do

Brooksley Born the Cassandra of the Derivatives Crisis

unplanned for events can and do happen.

Brooksley Born, the Cassandra of the Derivatives CrisisTuesday, May 26, 2009 A little more than a decade ago, Born foresaw a financial cataclysm, accurately predicting that exotic investments known as over-the-counter derivatives could play a crucial role in a crisis much like the one nowderivatives could play a crucial role in a crisis much like the one now convulsing America. Her efforts to stop that from happening ran afoul of some of the most influential men in Washington, men with names like Greenspan and Levitt and Rubin and Summers -- the same Larry Summers who is now a key economic adviser to President Obama. She was the head of a tiny government agency who wanted to regulate the derivatives They were thegovernment agency who wanted to regulate the derivatives. They were the men who stopped her. The same class of derivatives that preoccupied Born -- including the now-infamous "credit-default swaps" -- have been blamed for accelerating last fall's financial implosion. But from 1996 to 1999, when Born was the chairman p ,of the Commodity Futures Trading Commission, the U.S. economy was roaring and she was getting nowhere with predictions of doom.*

Ray Garrison *Washington Post

Undesirable, unknown, unplanned for events can and do

hFukushima tsunami safety plan: a single page memo raises questions,

sparks outrage over Japanese nuke site's disaster readiness

happen.

TOKYO — Japanese nuclear regulators trusted that the reactors at Fukushima Dai-ichi were safe from the worst waves an earthquake could muster based on a single-page memo from the plant operator nearly a decade ago.

In the Dec. 19, 2001 document — one double-sized page obtained by The Associated Press under Japan's public records law — Tokyo Electric Power Co. rules out the possibility of a tsunami large enough to knock the plant offline and gives scant details to justify this conclusion, which proved to be wildly optimistic*

The document wasn’t updated once in nine years, even as earthquake sciencenine years, even as earthquake science advanced – and last year, when TEPCO took another look at safety preparation, it was the same conclusion: The plant would

Ray Garrison *MSNBC.com**AP

stay dry no matter what.**

Undesirable, unknown, unplanned for events can and dounplanned for events can and do

happen.Tue Jan 24, 2012 4:21pm EST

Johnson & Johnson (JNJ.N) took ( )quarterly charges of more than $3 billion largely related to the recall of its defective artificial hips and gave a 2012 earnings forecast below analysts'

ti t *estimates.*

* Reuters

Ray Garrison

Undesirable, unknown, unplanned for events can and dounplanned for events can and do

happen.

The pharmaceutical giant Pfizer agreed to pay $2 3 billion to settle civil andto pay $2.3 billion to settle civil and criminal allegations that it had illegally marketed its painkiller Bextra, which has p ,been withdrawn. It was the largest health care fraud settlement and the largest criminal fine of any kind ever.*

* New York Times

Ray Garrison

Undesirable, unknown, unplanned for events can and dounplanned for events can and do

happen.4/11/12 – Johnson & Johnson/ / Jo so & Jo so

A state judge in Arkansas ordered Johnson & Johnson to pay a $1.1 billion fine after a jury found the company had minimized the risks of its p yantipsychotic drug Risperdal.The Arkansas jury's decision and the judge's penalty are just the latest legal setbacks for

$p y j gRisperdal. A South Carolina judge upheld a $327 million penalty against J&J and its Janssen unit late last year.E li thi th d t $158Earlier this year, the company agreed to pay $158 million to settle charges it improperly marketed Risperdal in Texas and caused the state's Medicaid program to spend too much on theMedicaid program to spend too much on the medicine.*

Ray Garrison *Bloomberg News

Undesirable, unknown, unplanned for events can and do

4/18/12 – St. Jude Medical

unplanned for events can and do happen.

/ 8/ S JEarlier this month, St Jude halted sales of its QuickSite and QuickFlex leads, wires that carry electricity from defibrillators to the heart, due toelectricity from defibrillators to the heart, due to concerns the insulation could wear away and expose the wires. Around the same time, HeartRhythm, a prominent medical journal, y p jpublished an article by Dr. Robert Hauser of the Minneapolis Heart Institute that took a critical look at St Jude's Riata lead line that was

ll d i 2011 Th i l d d th t threcalled in 2011. The piece concluded that the St Jude product failed at a higher rate than Medtronic Inc's (MDT.N) lead, prompting St Jude to ask for a retraction of the articleJude to ask for a retraction of the article, drawing even more attention to the issue.

Ray Garrison

Undesirable, unknown, l d f tunplanned for events can

and do happen.

What you don’t know is farWhat you don’t know is far more relevant than what

d kyou do know.

Ray Garrison

How do Unknown/Improbable Events Occur?

Undesirable, unknown, unplanned for events

Confluence of known and/or unknown inputsand/or unknown inputs

Known and unknown indicatorsindicators

Probabilities, planning, monitoring metrics

ProbabilitiesProbabilitiesProbability: “a strong likelihood or chance of

thi ” F St ti ti “th l ti fsomething” For Statistics: “the relative frequency with which an event occurs or is likely to occur”*Before you go out tonight, Las Vegas:

“Each game you play at a casino has a statistical probability against you winning. Every single time.”“ f“The slot machine odds are some of the worst, ranging from a one in 5,000 to a one in about 34 million chance of winning the top prize when using the maximum coin play ”**using the maximum coin play.

*Dictionary.com**Investopedia.com – Casino Stats, Why Gamblers Rarely Win.

Ray Garrison

ProbabilitiesOdds of being killed by lightening - 1 in 2 million.

Odds of being killed in a car crash - 1 in 5,000.

Odds of being killed in a tornado - 1 in 2 million.

Odds of being killed in a plane crash -1 in 25 million.

The thing about probabilities is that no one thinks g pit will happen to them…..

Ray Garrison

The Difficulty of PredictionyYou are using the past to predict the future – Not always a goodthe future Not always a good indictor.What do the mutual fund commercials say? Past performance is not a predictor of future performancefuture performance.We are not fortune tellersTechnology interconnectionsTechnology, interconnections, interdependencies, complexities exist that make predictions diffi ldifficult

Ray Garrison

RandomnessPhysicist Leonard Mlodinow of the California Institute of Technology – “our brains never evolved a probability network and thus ourevolved a probability network, and thus our folk intuitions are ill equipped to deal with many aspects of the modern world. Our intuitions are misleading when it comes tointuitions are misleading when it comes to probabilistic problems.”*

Extraordinary events do not always requireExtraordinary events do not always require extraordinary causes. Given enough time, they can happen by chance. Knowing this, we can learn to judge decisions by the spectrum j g y pof potential outcomes they might have produced rather than by the particular result that actually occurred.*

Ray Garrison

* Scientific American: How Randomness Rules Our World and Why We Cannot See It

Examples of Items that Hamper Evaluation/AssessmentEvaluation/Assessment

Incomplete and overlapping information from multiple units can cause

i / l d iexecutives/regulators to underestimate risk and leverage.Interconnected dependencies that we are aware of or unaware ofInsufficient grasp of systemic risk

Ray Garrison

We Always Try To Rationalize: ?Why?

We try to make sense of theWe try to make sense of the world around use.We try to simplify to understandWe try to simplify to understand.We extrapolate and make conclusions using insufficient orconclusions using insufficient or incorrect data.We are uncomfortable with theWe are uncomfortable with the unknown, uncertainty.

Ray Garrison

Undesirable, unknown, l d f tunplanned for events can

and do happen.

What you don’t know is farWhat you don’t know is far more relevant than what

d kyou do know.

Ray Garrison

Conscious versus Unconscious Incompetence

Ray Garrison

Cognitivedesignsolutions.com

Conscious IncompetenceConscious IncompetenceYou’re out of your leaguey gYou’re in way over your headEgo Says – I can handle thisg yWorried about how you look, what you will sayHow are knowledge/skill gaps communicated, evaluated, mitigated?Ignoring gaps hoping you will have what’sIgnoring gaps hoping you will have what s needed before you have to deal with the gap.g p

Ray Garrison

Unconscious IncompetenceYou don’t know what you don’t knowDon’t have a clueCompletely taken by surpriseAssumptions of capability, control, p p y, ,and detection are incorrectDenying relevance / usefulness of indicators or dataThe dim light at the end of the tunnel is a Train coming right at you and You Don’t See It

fPerception is out of line with realityGAPS HAMPER RESPONSE!

Ray Garrison

Sources of Indicator Data to Help Identify the UnknownHelp Identify the Unknown

Traditional indicators: Complaints,Traditional indicators: Complaints, NCR’s, CAPA’sYour People are your greatest resource. p y gAsk what they see, hear, experience.Debriefing / Learning from Near MissesFault Seed TestingMonitoring Regulatory Agencies Literature ReviewsProfessional Organizations/Groupsg p

Ray Garrison

Sources of Indicator Data to f

Allow Time to think about the following:

Help Identify the UnknownAllow Time to think about the following:

Focused, regular discussions on what could happen and how the organization pp gwould react.Emergency / Contingency Planningg y g y gData Analysis:

Creating new ways of looking at current dataCreating new ways of looking at current dataCreating/Capturing new dataTransposing data matricesp g

Ray Garrison

Sources of Indicator Data to f

Internal Audits:

Help Identify the UnknownInternal Audits:

How robust are your internal audit programs?programs?Are you challenging the audit teams to dig deep?dig deep?Audit Team TrainingAudit Team / Organization MindsetAudit Team / Organization MindsetTrends and Trending

Ray Garrison

Sources of Indicator Data to f

How Can You Identify the Unpredictable?Help Identify the Unknown

y pActive Thought ActivitiesProcess MappingBrainstorming internal/external unpredictable eventsFraming the event contextFraming the event context○ What○ Where○ When○ To What Extent

Ray Garrison

Preparing for the Unknown in Our Personal LivesPersonal Lives

We prepare for unknown (potentially p p (p ydamaging) events in our personal lives

Emergency FundsInsurance: Life/Home/Car/DisabilityInsurance: Life/Home/Car/DisabilityDocuments: Wills/Trusts/POA’s

Ray Garrison

Can You Prepare for the ?Unpredictable?

An Event (any event) OccursAn Event (any event) Occurs. What do you do?What process is there?What process is there?Who is involved?Who today in your organization manages the gray, the big problems, the

i ?emergencies? How do they do it?

Ray Garrison

How Can You Mitigate?How Can You Mitigate?A Response PlanA Response Plan

Knowing your processes intimatelyKnowing where the control points areKnowing where the control points are

A Rapid Response Team – The key here is “Rapid”! Analysis Paralysis will killis Rapid ! Analysis Paralysis will kill you!!!Always focused on the customer inAlways focused on the customer in decision making (For Med Device / Pharma - Patient / Doctor)Pharma Patient / Doctor)

Ray Garrison

Risk Management Standards & ToolsTools

ISO 14971: Medical Devices – Application of Risk M t t M di l D iManagement to Medical DevicesISO 16085: Systems and Software Engineering –Life Cycle Processes – Risk ManagementISO 31000 Ri k M t P i i l dISO 31000 Risk Management – Principles and Guidelines

F il M d Eff t A l i ISO 60812Failure Mode Effects Analysis: ISO 60812Preliminary Hazard Analysis (PHA)Fault Tree Analysis (FTA): IEC 61025Event Tree Analysis (ETA): IEC 62502Assurance Case Analysis: ISO 15026-2

Ray Garrison

Additional Topics of Study that fcan be beneficial

Complex Systems ThinkingComplex Systems ThinkingBecause complex systems are inherently non-linear, small events can suddenly produce large and unexpected effects This is a key point in systemsunexpected effects. This is a key point in systems theory.

StatisticsStatisticsTruly understanding what data is used, how data is used, what analysis was completed, what the results are indicating. g○ Incorrect/inappropriate data may be used.○ Incorrect/inappropriate Statistic Tools may be used.

Ray Garrison

Undesirable, unknown, unplanned , pfor events can and do happen.

What you don’t knowWhat you don t know is far more relevant than what you do yknow.

Ray Garrison

So What Are YOU Going To ?Do?

Engage and Embrace the UnknownEngage and Embrace the UnknownAcknowledge missing and undefined variables in our neat little equationsvariables in our neat little equationsAllow Yourself and Your Teams time to critically think about these topicscritically think about these topics

Ray Garrison

QUESTIONS?

Ray Garrison

THANK YOUTHANK YOUFor Your Time!

Ray Garrison