Simple Structured Risk AssessmentSimple Structured Risk Assessment(Learning from experience – the Learning from experience – the

importance of Near Miss and importance of Near Miss and Incident Reporting/ InvestigationIncident Reporting/ Investigation))

““BOW TIES and BARRIERS”BOW TIES and BARRIERS”David Slater - Cardiff University

October 19th 2011

19/10/2011 1Bow Ties and Incidents

Risk Assessment is a Simple, Natural Process

19/10/2011 2Bow Ties and Incidents

incomplete risk picture

How to Manage my Risk?

I’ve done this often before

Will my boss/shareholders support me if ..…?

I want to be safe Is it different from usual?

save time save ££££

I like to do a good job be safe

I want the business to succeed I am judged on….

Does doing this feel right?

19/10/2011 3Bow Ties and Incidents

and if I get it wrong……

19/10/2011 4Bow Ties and Incidents

or……

$

19/10/2011 5Bow Ties and Incidents

and quite possibly……

19/10/2011 6Bow Ties and Incidents

What are the hazards - how bad could it be?

$

19/10/2011 7Bow Ties and Incidents

Failure thro’ Imperfection – “Human” Failure thro’ Imperfection – “Human” Error?Error?

• In the Swiss Cheese model, individual weaknesses are modelled as holes in slices of Swiss cheese, such as this Emmental. They represent the imperfections in individual safeguards or defences, which in the real world rarely approach the ideal of being completely proof against failure.

19/10/2011 Bow Ties and Incidents 8

Bow Ties – an overviewBow Ties – an overview• Bow ties evolved out of Reason’s “Swiss cheese” mould. They added a crucial insight• There is a point between “Cause” and “Consequences” where you lose control. This is the

“Knot”.• Up to the “knot” any "barriers" are there to stop you losing control - they are a measure

of the VulnerabilityVulnerability of the system.• After the knot the outcome is often pure chance (slipping on ice, falling off a ladder!). Any

barriers here are to avoid/reduce the consequences (seat belts, air bags!). Their effectiveness is then a measure of the ResilienceResilience of the system.

• The big advantage of the method is that it is an overview of the incident structure(and underlines (justifies?) the importance of recording near misses near misses - the one's that don't get past the knot!)

• It gives especially non technical people a feel for where their performance or otherwise affects a particular barrier and the purpose of resilience barriers which are not necessarily redundant (BP gulf of Mexico).

• The advantages of adopting this way of analyzing accidents, is that you quickly find that the majority fit a reasonably small set of Bow tie templatesBow tie templates.

• By recording incident and near miss data on to these templates you start to build up a real life indication of barrier effectiveness (Swiss Cheese permeability) or not?

• Finally you can use this recorded data to calculate a value of system integrity system integrity using LOPA. 19/10/2011 Bow Ties and Incidents 9

What prevents the hazards being realised?

cause

cause

cause

cause

cause

“barriers” or“safeguards”cause

Loss ofcontrol

19/10/2011 10Bow Ties and Incidents

What mitigates the consequences?

$consequence

consequence

consequence

consequence

consequence

“barriers” or“safeguards”

Loss ofcontrol

19/10/2011 11Bow Ties and Incidents

A shared picture of how the hazard is managed

$cause

cause

cause

cause

cause

consequence

consequence

consequence

consequence

consequence

“barriers” or“safeguards”

cause

Loss ofcontrol

19/10/2011 12Bow Ties and Incidents

Fault-tree

Accident

Caus

es

Event-tree

Dam

age

The “Bowtie” MethodologyThe “Bowtie” MethodologyEverything leading up to the

accidentEverything following the

Accident

19/10/2011 13Bow Ties and Incidents

After the Knot! (La Deluge?)After the Knot! (La Deluge?)

19/10/2011 14Bow Ties and Incidents

FOR A “SIMPLE” MODEL THERE ARE SOME FOR A “SIMPLE” MODEL THERE ARE SOME FUNDAMENTAL INSIGHTS FROM THE “BOW TIE” FUNDAMENTAL INSIGHTS FROM THE “BOW TIE”

PARADIGMPARADIGM

• The Knot is highly significant, it is the point where we lose The Knot is highly significant, it is the point where we lose controlcontrol

• A logical (and useful )definition of A logical (and useful )definition of “Vulnerability” “Vulnerability” then follows then follows as - as - “The Propensity to loss of control”“The Propensity to loss of control”

i.e. i.e. The Left Hand (LHS)The Left Hand (LHS)

• And similarly And similarly “Resilience” “Resilience” is - is - “ “The Effectiveness and depth of Defences, once control is lost”The Effectiveness and depth of Defences, once control is lost” i.e. i.e. The Right Hand Side (RHS)The Right Hand Side (RHS)• Aren’t these more rational and rigorous definitions Aren’t these more rational and rigorous definitions ?

19/10/2011 15Bow Ties and Incidents

MANAGEMENT IMPLICATIONS OF THE MANAGEMENT IMPLICATIONS OF THE ““BOW TIE” PARADIGMBOW TIE” PARADIGM

• LHS – Reduce LHS – Reduce VULNERABILITY VULNERABILITY (avoid the accident!)(avoid the accident!)Design out branches, ideally ensure inherent safety, limits

and boundaries ;Design in checks and balances. (ABS) “RISK” RISK” is then what you can’t control or guarantee to

stop!• RHS – (Its going to happen), ensure RHS – (Its going to happen), ensure RESILIENCE!RESILIENCE!Barrier effectiveness/ performance checks, availability/

(maintainability), permeability, and degradation (complacency, relevance/ credibility/(short) Cuts)

- Panic Button, Fail to Safety, ESD, Dump and Recover, Dead man’s handle, Response, Redundancy. (Airbags)

• If the consequences are really serious – Plan for Worst If the consequences are really serious – Plan for Worst case survivability (or if in doubt, don’t do it!)case survivability (or if in doubt, don’t do it!)

19/10/2011 16Bow Ties and Incidents

Fault-tree

Accident

Caus

es

Event-tree

Dam

age

LHS -Take away the causes, LHS -Take away the causes, Reduce VulnerabilityReduce Vulnerability

19/10/2011 17Bow Ties and Incidents

Fault-tree

Accident

Caus

es

Event-tree

Dam

age

• But how effective are the Barriers really?But how effective are the Barriers really?

LHS - can’t remove cause or-LHS - can’t remove cause or- for all RHS pathways – Put up barriersfor all RHS pathways – Put up barriers

19/10/2011 18Bow Ties and Incidents

Barrier effectivenessBarrier effectiveness• What are the “barriers”?

• How do I know what makes a barrier effective?

• How do I know when it won’t work?

• With multiple barriers could one failure go undetected?

• How might my actions impair the effectiveness of a barrier?

• How can I improve the effectiveness of barriers?

19/10/2011 19Bow Ties and Incidents

Barriers are the same as in the Barriers are the same as in the Swiss Cheese modelSwiss Cheese model

Lines of defence

Defects

19/10/2011 20Bow Ties and Incidents

- and fail as in the Swiss Cheese - and fail as in the Swiss Cheese model!model!

Lines of Defence

19/10/2011 21Bow Ties and Incidents

This means we can still see a path This means we can still see a path thro’ - An Accident Sequence ……thro’ - An Accident Sequence ……

Fault-tree

Accident

Caus

es

Event-tree

Dam

age

19/10/2011 22Bow Ties and Incidents

Bow Ties and BarriersBow Ties and Barriers

Used extensively now in Hazardous IndustriesEqually useful in all Operational Phases• Before – Design, • During – “Tool Box Talks”, and • After – Incident investigation

This is a case where an offshore crane operator dropped a load of drill pipe in a lay down area which was supposed to be “off limits” and controlled by CCTV and “Safety Rules”!.

Banksman controls crane operations

CCTV in driver'scabin

Lack of visibility dueto poor design

No more than twopeople in laydown

area (PTW)Injuries and fatalities

C Craneoperations

C.01c Accident inlaydown area

Accident

19/10/2011 23Bow Ties and Incidents

BARRIERS– Defence in Depth or Leaky BARRIERS– Defence in Depth or Leaky False Comforters?False Comforters?

Safeguard 3

Safeguard 1

Safeguard 5Safeguard

2Safeguard

4

Loss of Control

Accident

19/10/2011 24Bow Ties and Incidents

19/10/2011 25Bow Ties and Incidents

A Common FormatA Common Format• .

This is consistent with the standard Bow tie analysis output, HAZOP output and Incident/ occupational databases (such as Story builder) and accident investigation (cause and effect) Templates.

• F1 – Cause/ Threat frequency – From a choice of sources (incident databases, Delphi sessions, Markow/Montecarlo distribution functions (from e.g. Modelrisk, @Risk, Goldsim, etc))

• P1,P2, P3,- The Probabilities of Failure on Demand of the Protective Barriers• P4, P5, P6 - The Probabilities of Failure on Demand of the Mitigation Barriers• N11,N2, N3 – The consequences of the unmitigated and mitigated outcomesSo What? – (Stage 4)From the sequence and Data above, the analyst has now the ability to print out

and record a range of essential outputs (displayed in real time if he wishes).• F2 – the Expected Frequency of the Loss of Control (Top event) – the

VULNERABILITY to that threat. (F1xP1xP2xP3--• F3 – the Expected Frequency of the Consequences identified – the

RESIDUAL RISK (F2 x P4xP5 xP6 ----• The system RESILIENCE is then F2 / F3 etc and PIG outputs are logF and logN

Threat /CauseThreat /Cause

Loss of ControlLoss of Control

ConsequencesConsequencesProtection BarriersProtection Barriers

Mitigation BarriersMitigation Barriers

.

19/10/2011 26Bow Ties and Incidents

A Virus AttackA Virus Attack• How would this work for a Virus attacking our Computer System?.

• This is still consistent with the standard Bow tie analysis output, • F1 – Cause/ Threat frequency – How likely is it my system will be

attacked? Probably - Very likely = at least once per year and increasing? (from records?)

• P1,P2, P3,- The Probabilities of Failure on Demand of the Protective Barriers which are?-Standard Firewalls, Training/standards compliance, access restriction?

• P4, P5, P6 - The Probabilities of Failure on Demand of the Mitigation Barriers – Virus removal patch- unless its a new virus, isolation, quarantine, Hard Disk firewall

• For each of these we can use incident records, or intelligence estimates, plus the option of cloud sources, real time monitoring, Dependency analysis and/or a combination of all of the above..

Threat /CauseThreat /Cause

Loss of ControlLoss of Control

ConsequencesConsequencesProtection BarriersProtection Barriers

Mitigation BarriersMitigation Barriers

.

19/10/2011 27Bow Ties and Incidents

Barriers can be Hard and SoftBarriers can be Hard and Soft

• And helpful visuallyAnd helpful visually

19/10/2011 28Bow Ties and Incidents

PFD’s from Actual PFD’s from Actual Incident RecordsIncident Records

• We can utilise actual incident databases –

• Company ones are best(such as TRACTION in BP)

• This StorybuilderStorybuilder

database is available in the Netherlands

19/10/2011 29Bow Ties and Incidents

Lets work thro’ an ExampleLets work thro’ an Example

The Macondo Well Incident - The Macondo Well Incident - Outcomes:Outcomes:

• - Safety: 11 fatalities / 115 rescued / Rig - Safety: 11 fatalities / 115 rescued / Rig sinkssinks

• - Environment: Largest oil spill in US - Environment: Largest oil spill in US historyhistory

• - Multiple inquiries- Multiple inquiries• - Regulatory agency reorganizations- Regulatory agency reorganizations• - Many new technical and permitting - Many new technical and permitting

requirementsrequirements19/10/2011 30Bow Ties and Incidents

What Went What Went Wrong?Wrong?

1. “Wrong” kind of cement in well casing. (Hard, Extant, Unrevealed)

2. Drill pipe NRV failed. (Hard, Design)3. Staff “misread” key pressure reading

(Soft, Human, Procedures, Training) 4. Rig crew did not recognise the (oil &

gas) influx (Soft, Human, Training)5. At the surface – flow diverter failed to

dump oil and gas overboard(Hard, Design, Management of Change)

6. Oil and gas vented directly on to the rig (drilling floor)( Hard, Design)

7. Fire Detection/Prevention system failed – “allowing flammable gas into the engine rooms” (Hard, Design)

8. The “failsafe” blowout preventer (BOP Stack) failed. Fire prevented remote shutdown, but the BOP had flat batteries and a faulty solenoid anyway. (Jackpot!)19/10/2011 Bow Ties and Incidents 31

(The following is taken from the BP report on the Macondo Well incident and is used as an illustrative application, treating the information as a given and not necessarily accurate!)

Lets take BP’s Barriers!Lets take BP’s Barriers!

19/10/2011 32Bow Ties and Incidents

BP project had LHS Barriers, (to reduce BP project had LHS Barriers, (to reduce their vulnerability, of Three kinds)their vulnerability, of Three kinds)

1. It was designed to ensure well containment and “shut-inability”!(Barriers 1&2)

2. There was a range of instrumentation, procedures, training and designated management responsibilities to monitor, check and assure “normal” behaviour(Barriers 3&4)

3. There was a dedicated “Blow out Preventer” function, cabin, full time operator, instrumentation and emergency valves to protect against “Loss of Control” (Barriers 5&6)

But ---Design, construction, systems and procedures all failed --- it was (de facto?) very VULNERABLE?VULNERABLE?

19/10/2011 33Bow Ties and Incidents

The project had RHS – (Resilience) Barriers The project had RHS – (Resilience) Barriers

again three kinds again three kinds 1. Fire and Gas Detection, and Ignition

prevention/suppression to avoid Fire, (Barrier 7 shown the wrong side of knot in BP’s diagram?)

2. Emergency Procedures/drills to Isolate/ disperse potential casualties ( note it worked for support vessels) (Barrier ? unclaimed)

3. Sub-sea wellhead BOP valves to seal in the well.Again all failed , so they were highly Vulnerable and as it

turned out also had no effective defences(B8) = Zero Resilience?= Zero Resilience? How can that be? What mouse was eating their CHEESE?19/10/2011 34Bow Ties and Incidents

Lets put some numbers in?Lets put some numbers in?

• The knot (Top Event) in the Bow Tie should be the uncontrolled release of oil and gas, the fire and environmental effects are the consequences.

• F1 in this case is the expected frequency of occurrence of hitting a high pressure gas pocket.

• (If they had used the HAZOP input spreadsheet this would have been a cause of the deviation MORE PRESSURE)

• This is a function of specific geology, but generic data say from OREDA suggest it is to be expected at least once per hole? (say 10 per year conservatively)19/10/2011 35Bow Ties and Incidents

Probabilities of Failure on Demand – Probabilities of Failure on Demand – PFD’sPFD’s

• First Protective Barrier – Cemented casing – probability of failure on demand can be estimated from direct experience with this contractor in his track record and an estimate of the quality of the crew.(10 -1?)

• Second Protective Barrier – Non Return Valve – If we took this from engineering plant commissioning and operational data, some companies would have assessed the probability of its failure on demand as 50%!(0.5)

• Third Protective Barrier – Crew/operator training and procedures. Most people would look at the latest audit data for this region/ crew? Are they in compliance (also check the auditor –PFD – 10 -1?)

• We would also be able to interface to the BSI/Infogov online compliance and audit checking package – Proteus; which could return (via XML) an indication of the status of compliance with procedures and ISO Standards. Which status %, is a measure of (the inverse of) the probability of failure number we require.

19/10/2011 36Bow Ties and Incidents

Real Time Status?Real Time Status?• Fourth (and final) Protective Barrier –The Blow out Preventer –• This is such a crucial piece of equipment that, although we could

take historical data for its failure probability, the more useful way would be to monitor the real time status of the equipment. If there is a control cabin with all the relevant feeds available, our NIMBi module could relay its status data in real time. In the Gulf of Mexico incident there were queries as to its availability due to flat batteries, this would have given a probability of failure on demand of 1?

• Once you’re past the BOP, you’re out of control (past the knot on the Bow Tie!). Hence the other four BP barriers come after, as mitigation, not protection.

• (The expected frequency of a blow out was then their expected state of vulnerability– from 1 in 10 to 1 in a 100 per year?)

19/10/2011 37Bow Ties and Incidents

Display it in Real Time?Display it in Real Time?• Wouldn't that have been very useful?

19/10/2011 38Bow Ties and Incidents

Expected Mitigation Effectiveness?Expected Mitigation Effectiveness?

• Now let’s deal with the barriers which were designed to mitigate the consequences of an uncontrolled release of oil and gas. (It is planned to include access to a consequence modelling package (perhaps based on PHAST or similar), to give quantitative estimates of impacts- probably in categories)

• We can then assess the RESILIENCE of the system.

19/10/2011 39Bow Ties and Incidents

Barrier PFD’sBarrier PFD’s• First Mitigation Barrier – Flow Diverter – This was no

longer used as a diverter and so is a missing barrier – (P = 1)• Second Mitigation Barrier – Fire and Gas Detection

system/Alarms – Again seems not to have been adequate for the scale of incident? (P = 1?)

• Third Mitigation Barrier – Fire /Explosion suppression – inadequate for scale of release? (P = 1)

• Fourth Barrier – Evacuation survival procedures – Support Vessel Response was prompt, as required, but in total not effective in containing consequences (P=1?)

• Hence the residual risk was still 1 in 10 to 1 in 100 – not the kind of (lack of) resilience that the operators could have ignored, had they been aware.

19/10/2011 40Bow Ties and Incidents

So What is the Relevance to the BP So What is the Relevance to the BP Incident?Incident?

• The company needs to look again at the effect of “influencing factors” affecting human performance in that environment.

• Economic and (lack of?) Regulatory pressures need also to be identified and their mode of influence recognised.

• It was almost certainly not the result of the sudden, simultaneous and statistically improbable failure of 8 completely independent Barriers!(extant fail and unrevealed)

• But we do need to recognise their inherent complexity as more than simple “Reason-able?”“cheese” Barriers!

• We need to use “state of the art” tools to manage “state of the art” projects’ and move on from nice pictures and analogies.

• We need to learn from incidents – yes, but we would prefer to predict and avoid them ---if all else fails---?

19/10/2011 41Bow Ties and Incidents

Monitoring BarriersMonitoring Barriers• Knowledge of the status of Barriers is key:• Formal focused in-depth reviews –

excellent,• but infrequent• - TTS (e.g. Statoil) − 5 yearly• - Audits − 3 yearly• - Planned Inspections − 1 year• Lessons learned from Incident

investigations −• excellent AND high frequency• - BSCAT approach − every incident / near

miss• means some barriers failed / degraded• - For many facilities this is 100+ events /

year• - Only current answer - collect statistics

and root causes• Can we afford to wait?

Barrier Status – a to fBarrier Status – a to f

19/10/2011 42Bow Ties and Incidents

Top-Ten Missed Opportunities from Top-Ten Missed Opportunities from Accident Investigation Accident Investigation (Kletz, 2003)(Kletz, 2003)

• Accident investigations often find only a single cause• Accident investigations are often superficial• Accident investigations list human error as a cause• Accident reports look for people to blame• Accident reports list causes that are difficult to remove• We change procedures rather than designs• We may go too far!• We do not let others learn from our experience• We read or receive only overviews• We forget the lessons learned,

and allow the accident to happen again19/10/2011 43Bow Ties and Incidents

Esso cost cutting

Physical accident sequence

Incorrect operation of manual bypass valve

Expl-osion

Organisational Absence of engineers

Inadequate procedures & training

Focus on LTIs

Poor auditing

Govt/ regulatory system Inadeq regulatory system

Societal THE ESSO LONGFORD EXPLOSIONTHE ESSO LONGFORD EXPLOSION

Corporate Exxon control failure

Loss of supply

Govt failure to providealternative supply

2 wk site closure

Plant inter-connections

Warm oil restart

Embrittle-ment of heat exchanger

Warm oil pump trip

Cond-ensate overflow

Failure of incident reporting system

Failure to HAZOP GP1

Market forces

Operating in alarm mode

Poor shift handover

Poor maintenance priorities

Maintenance backlog

Poor engineer-ing design

Poor super-vision

Failure to ID interconnection hazard

Poor change mgt

19/10/2011 44Bow Ties and Incidents

So What?So What?• We need to accept reality and the lessons of recent history.• Modern Infrastructure Systems have become (“stiff”); too

complicated for simplistic risk management approaches!• Complex Systems can have “Normal Accidents” (Perrow).• Management requires a more "Holistic” overview of how

incidents occur (Hopkins).• We need to adopt a much more thoughtful and structured

approach to Risk Management and Incident Investigation• And we need to ensure we have a system for recording,

analysing and monitoring/ warning us of our actual incident and near miss records to really “learn the lessons!”

• “Bow Ties” is a “Cheese" development which fits the bill!• We can now focus on designing in “Adequate RESILIENCE”!-• rather than “Acceptable Risk”! • If it can --, It will ! – 19/10/2011 Bow Ties and Incidents 45

Risk Management really is a matter of life or Death!

19/10/2011 46Bow Ties and Incidents