54
‘Obvious’ Errors: Anything but Simple Gareth Lock Cognitas Incident Research and Management

"Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Embed Size (px)

Citation preview

Page 1: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

‘Obvious’ Errors: Anything but Simple

Gareth Lock!!Cognitas Incident Research and Management

Page 2: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Background

Page 3: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Some humour…

Page 4: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Some humour…

Page 5: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

OC Rec Case Study - Report"Four divers were diving on a wreck to a maximum depth of approximately  22m.   One of the divers entered the wreck.  Another diver then went after him and found him unconscious with his regulator in his mouth.  He brought him out  and recovered him to the surface.  He was lifted into the boat and the emergency services were alerted.  The diver was airlifted to hospital where he later died.  The diver had a 15l main cylinder and a 3l pony cylinder.  After the event the main cylinder was found to be full.  A  press report of  the Coroner's inquest suggests that the casualty had 'mixed up his air supply tanks'.  The cause of death was drowning."

Page 6: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

OC Rec Case Study Introduction❖ Club dive on wreck in 24m!❖ Diver A 19st, 6’2”!❖ BCD, 12L & pony, !❖ 1st dive of the season, not dived for

10 months, 4 dives previous year!❖ Buddy Checks carried out!❖ OOG after 10mins!❖ Separation!❖ ‘Body’ found, CBL, sent up !❖ Problems in recovering onto RHIB

Page 7: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Risk Perception/Acceptance vs Safety❖ Risk is:!

❖ Relative!

❖ Personal (Voluntary)!

❖ Experience/Knowledge!

❖ Environment influenced!

❖ Goal influenced

Page 8: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Risk Perception/Acceptance vs Safety❖ Risk is:!

❖ Relative!

❖ Personal (Voluntary)!

❖ Experience/Knowledge!

❖ Environment influenced!

❖ Goal influenced

Page 9: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Risk Perception/Acceptance vs Safety❖ Risk is:!

❖ Relative!

❖ Personal (Voluntary)!

❖ Experience/Knowledge!

❖ Environment influenced!

❖ Goal influenced

Page 10: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Risk Perception/Acceptance vs Safety❖ Risk is:!

❖ Relative!

❖ Personal (Voluntary)!

❖ Experience/Knowledge!

❖ Environment influenced!

❖ Goal influenced

❖ Is safety absence of risk?!

❖ Can we measure a negative?

Page 11: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Risk Perception/Acceptance vs Safety❖ Risk is:!

❖ Relative!

❖ Personal (Voluntary)!

❖ Experience/Knowledge!

❖ Environment influenced!

❖ Goal influenced

❖ Is safety absence of risk?!

❖ Can we measure a negative?

❖ “Safety” is ability to sustain required operations under both expected and unexpected conditions.

Page 12: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Causality Models❖ Cause and Effect Trees!

❖ Denoble's Triggers!

❖ Systems Approach !

❖ ETTO!

❖ HFACS-D

Page 13: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Small World Problem

How likely is it that any two persons, selected arbitrarily from a large population, can be linked via common acquaintances and how long will the links be on average?

Page 14: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Small World Problem

In risk assessment, the problem is how likely it is that two events are indirectly coupled and how many steps in between are required on average…The small world phenomenon demonstrates the importance of this, namely that things (actions) that seemingly have no relation to each other still may affect each other. - ETTO, Hollnagel, 2009

Page 15: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Cause and Effect Trees❖ Simple, easy to understand!

❖ Bimodal - this happened, caused that!

❖ Biases!

!

!

!

!

Page 16: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Cause and Effect Trees

Page 17: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Cause and Effect Trees

Page 18: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Cause and Effect Trees

Page 19: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Cause and Effect Trees

Page 20: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Cause and Effect Trees

Page 21: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Cause and Effect Trees

Page 22: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Hindsight bias!

Confirmation bias!

Cause and Effect Trees

Page 23: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Cause and Effect Trees

Action

1. Action based on event history and current environment/situation

Page 24: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Cause and Effect Trees

Action

1. Action based on event history and current environment/situation

!!

Difference between the expected !and unexpected outcome is not

clear cut!

3. Unexpected outcome - ‘Error’

2. Expected outcome - ‘Good’

Page 25: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Cause and Effect Trees

Action

Outcome of Previous Action

4. Hindsight provides "feedback of ‘correct’ action

1. Action based on event history and current environment/situation

!!

Difference between the expected !and unexpected outcome is not

clear cut!

3. Unexpected outcome - ‘Error’

2. Expected outcome - ‘Good’

Page 26: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Cause and Effect Trees

Action

Outcome of Previous Action

4. Hindsight provides "feedback of ‘correct’ action

1. Action based on event history and current environment/situation

!!

Difference between the expected !and unexpected outcome is not

clear cut!

3. Unexpected outcome - ‘Error’

2. Expected outcome - ‘Good’

4. Hindsight provides "incorrect feedback of action "if they thought outcome was ‘Good’

or

Page 27: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Denoble's Triggers❖ 947 fatalities analysed

Page 28: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Denoble's Triggers❖ 947 fatalities analysed

❖ Triggers!

❖ OOA (41%)!

❖ Entrapment (20%)!

❖ Equipment Problems (15%)

Denoble, P.J., Caruso, J.L., Dear, G.d.e. .L., Pieper, C.F. & Vann, R.D., 2008, Common causes of open-circuit recreational diving fatalities, Undersea & hyperbaric medicine : journal of the Undersea and Hyperbaric Medical Society, Inc, 35(6), pp. 393-406

Page 29: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Denoble's Triggers❖ 947 fatalities analysed

❖ Triggers!

❖ OOA (41%)!

❖ Entrapment (20%)!

❖ Equipment Problems (15%)

❖ What, not why

Denoble, P.J., Caruso, J.L., Dear, G.d.e. .L., Pieper, C.F. & Vann, R.D., 2008, Common causes of open-circuit recreational diving fatalities, Undersea & hyperbaric medicine : journal of the Undersea and Hyperbaric Medical Society, Inc, 35(6), pp. 393-406

Page 30: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Systems ApproachTechnological Failure Human Failure Organisational Failure

1900 1950 2000

Simple Linear Models!Independent Causes

Complex Linear Models!Active and Latent

Non-Linear Models!Tractable/Intractable Dynamic Complexity

Adapted from Hollnagel, E., 2009, The ETTO Principle:

Page 31: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Efficiency Thoroughness Trade Off (ETTO)

❖ ETTO!

❖ Heuristics!

❖ Shortcuts

Page 32: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Efficiency Thoroughness Trade Off (ETTO)

❖ ETTO!

❖ Heuristics!

❖ Shortcuts

❖ Performance Variability!

❖ Approximations!

❖ Normalisation of Deviance

Page 33: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Efficiency Thoroughness Trade Off (ETTO)Examples of Heuristics• Looks fine"

• Not really important"

• Normally OK, no need to check"

• I’ve done it millions of times before"

• Will be checked by someone else"

• Has been checked by someone else"

• This way is much quicker"

• No time (or resource) to do it now"

• Can’t remember how to do it"

• We always do it this way"

• Looks like X, so must be X

Page 34: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

HFACS-Diving

Page 35: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

HFACS-Diving❖ Based on HFACS Model

Page 36: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

HFACS-Diving❖ Based on HFACS Model❖ Constructed after reviewing 18

pieces of literature - 232 factors!❖ Validated by 13 SMEs

Poor  decision  to  con,nue  dive

Incorrect  assembly  of  CCR/scrubber  packing

Incorrect  maintenance  of  pO2  (eCCR)

Page 37: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

HFACS-Diving❖ Based on HFACS Model❖ Constructed after reviewing 18

pieces of literature - 232 factors!❖ Validated by 13 SMEs

❖ 96 factors!❖ Rules/Violations!❖ 5 equipment based!❖ Fiscal drivers*!❖ Latent Medical*"❖ Direct Contravention of Trg*

Poor  decision  to  con,nue  dive

Incorrect  assembly  of  CCR/scrubber  packing

Incorrect  maintenance  of  pO2  (eCCR)

Page 38: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

OC Rec Review Incident Case Study❖ General Fitness!

❖ Dive and Rescue!

❖ ‘Dived Up’!

❖ Checks and Config!

❖ OOA response!

❖ Separated!

❖ Body recovered

Page 39: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

OC Rec Review Incident Case StudySkill Based Errors (Incorrect In Water Skills)!

Failure to Monitor Gauge and end dive when reached mininum gas pressure e.g. surface with 50bar/500psi Incorrect response for Out of Air (OOA) Equipment problems (use of, not failure of)

Knowledge Based Failure to Understand Equipment usage!Failure to Complete Pre-Dive Buddy/Self Checks

Condition of Operator (Physiological State) Drug or Alcohol Intoxication

Personal Readiness (Training/Skills)

Insufficient Training (Lack of currency of skills to ensure that they are upto date, 'Dive Fit')

Page 40: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

HFACS-D - Data as at 11/2/14

Page 41: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

HFACS-D - Data as at 11/2/14❖ Incidents (n=341)!

❖ 31% physical OOG/below agency minimums!

❖ 9% uncontrolled buoyant ascents!

❖ 23% unplanned separation

Page 42: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

HFACS-D - Data as at 11/2/14❖ Incidents (n=341)!

❖ 31% physical OOG/below agency minimums!

❖ 9% uncontrolled buoyant ascents!

❖ 23% unplanned separation

❖ Dive plan deviation - 18%

Page 43: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

HFACS-D - Data as at 11/2/14❖ Incidents (n=341)!

❖ 31% physical OOG/below agency minimums!

❖ 9% uncontrolled buoyant ascents!

❖ 23% unplanned separation

❖ Dive plan deviation - 18%

❖ Decision to continue when should have ended - 23%

Page 44: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

HFACS-D - Data as at 11/2/14

Page 45: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

❖ ETTO!❖ Looks fine to me 10%!❖ Normally worked/Worked before 24%

HFACS-D - Data as at 11/2/14

Page 46: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

❖ ETTO!❖ Looks fine to me 10%!❖ Normally worked/Worked before 24%

❖ Attitude!❖ Complacency 41%, Lack of Situational Awareness 32%!❖ Risk Perception/Acceptance 50%!❖ Behaviour 20% not changed

HFACS-D - Data as at 11/2/14

Page 47: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

❖ ETTO!❖ Looks fine to me 10%!❖ Normally worked/Worked before 24%

❖ Attitude!❖ Complacency 41%, Lack of Situational Awareness 32%!❖ Risk Perception/Acceptance 50%!❖ Behaviour 20% not changed

❖ Social!❖ 8-12% direct, indirect or fiscal pressures

HFACS-D - Data as at 11/2/14

Page 48: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

How to Improve Things❖ Attitudinal Changes Essential!

❖ Stop Throwing Rocks at Those Who Make Mistakes

Page 49: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

How to Improve Things❖ Attitudinal Changes Essential!

❖ Stop Throwing Rocks at Those Who Make Mistakes

❖ Promote Personal Responsibility!

❖ We ALL make mistakes, most of them our own doing!

❖ Group Polarisation/Risky Shift

Page 50: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

How to Improve Things❖ 'Just Culture'!

❖ 'Who Decides'!

❖ Recreational vs Voluntary Instruction vs Professional Instruction

Page 51: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

How to Improve Things❖ 'Just Culture'!

❖ 'Who Decides'!

❖ Recreational vs Voluntary Instruction vs Professional Instruction

Page 52: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

How to Improve Things❖ Work Together!

❖ Improved Reporting !

❖ Sharing of Data!

❖ Common Models

All the Information Possible to be Known

Diver’s Knowledge

Friends’s Knowledge

Researcher’s Knowledge

http://www.divingincidents.org.uk

@DISRC #DISMS

Page 53: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

Summary❖ Risk and Safety are compromises!

❖ Too many factors to use simple models!

❖ No one ‘root cause’!

❖ Need detailed narratives!

❖ We can improve things…

Page 54: "Obvious Errors: Anything but simple" - Don't Judge when you don't know the whole story

http://www.cognitas.org.uk

http://www.divingincidents.org.uk

@DISRC #DISMS

“What good is hindsight if we supposedly learn something but don’t apply it to our own/community’s foresight?”