Jesse Baker - RISSB - Industry incident reporting, what’s the BCR?

Major Rail Occurrence Forum - Derailments

29/30 April 2014

Industry Incident Reporting, what’s the BCR? Jesse Baker – Manager Safety & Systems RISSB

Who are RISSB? •  Not for profit company

•  Wholly owned by the ARA (Industry)

•  Half funded by Government, half by industry

•  Primary business is standards, rules, codes of practice and guidelines

•  But really the name of the game is harmonisation

….. but this speech is about Industry incident reporting and we have ON-S1, OC-G1 and the CFF

Thankyou Questions?

Why do we report on occurrences? – Why do we measure things at all?

•  “Measurement is the first step that leads to control and eventually to improvement. If you can’t measure something, you can’t understand it. If you can’t understand it, you can’t control it. If you can’t control it, you can’t improve it.”

― H. James Harrington

•  When something goes wrong (an occurrence) we get a specific type of opportunity to understand (learn).

•  But why report?

The evolution of our reporting regime •  1992 – Commonwealth Govt forms National Rail Corporation

•  National Rail lobbies the State Regulators Panel

•  Late ‘90s ON-S1 was published

•  c5 years later OC-G1 was published

•  2012 ON-S1 & OC-G1

•  2008 NTC National Strategy for Rail Safety Data

•  CFF first published in 2009 – non mandatory

•  The future …. SISAR !!!

ON-S1 •  Terminology and descriptors

•  Reporting requirements

•  Occurrence categories and definitions

o  Derailment: An incident where one or more rolling stock wheels leave the rail or track during railway operations.

o  General description o  Number of wagons o  Operational status o  Chain of events

•  Terminology and descriptors

•  Guiding principles for classification

•  Occurrence categories and definitions

o  Running Line Derailment includes / excludes o  Yard Derailment includes / excludes

OC-G1

The problem with ON-S1 and OC-G1

NTC National Strategy for Rail Safety Data

•  Better focused national data

•  Better data quality

•  Better data consistency and comparability

Contributing Factors Framework

•  Starting to become more intelligent •  But take-up isn’t great!

Contributing Factors Framework

Decision making more generally •  None of this data collection is useful unless it can lead to decision

making

•  But there is another way

A thought experiment •  What would you rather?

$1,000 Right now

Roll a dice up to 100 times 1 & 5 you win $700 2 nothing 3, 4 & 6 you lose $300

(0.333 x 700) + (0.167 x 0) – (0.5 x 300) = $83.10 x 100 = $8,310

The human condition

•  We’re vulnerable to:

o  Several types of bias:

Ø  Decision making, belief, and behavioural

Ø  Social

Ø  Memory

o  Logical fallacies

Decision making, belief, & behavioural biases Ambiguity effect Dura0on neglect Irra0onal escala0on Reac0ve devalua0on Anchoring or focalism Empathy gap Just-‐world hypothesis Recency illusion A@en0onal bias Endowment effect Less-‐is-‐be@er effect Restraint bias Availability heuris0c Essen0alism Loss aversion Rhyme as reason effect Availability cascade Exaggerated expecta0on Mere exposure effect Risk compensa0on / Peltzman effect Backfire effect Experimenter's or expecta0on bias Money illusion Selec0ve percep0on Bandwagon effect Func0onal fixedness Moral creden0al effect Semmelweis reflex Base rate fallacy or base rate neglect Focusing effect Nega0vity effect Social comparison bias Belief bias Forer effect or Barnum effect Nega0vity bias Social desirability bias Bias blind spot Framing effect Neglect of probability Status quo bias Cheerleader effect Frequency illusion Normalcy bias Stereotyping Choice-‐suppor0ve bias Gambler's fallacy Observa0on selec0on bias Subaddi0vity effect Clustering illusion Hard-‐easy effect Observer-‐expectancy effect Subjec0ve valida0on Comfort zone effect Hindsight bias Omission bias Survivorship bias Confirma0on bias Hos0le media effect Op0mism bias Time-‐saving bias Congruence bias Hot-‐hand fallacy Ostrich effect Unit bias Conjunc0on fallacy Hyperbolic discoun0ng Outcome bias Well travelled road effect Conserva0sm or regressive bias Iden0fiable vic0m effect Overconfidence effect Whole only effect Conserva0sm (Bayesian) IKEA effect Pareidolia Zero-‐risk bias Contrast effect Illusion of control Pessimism bias Zero-‐sum heuris0c Curse of knowledge Illusion of validity Planning fallacy Decoy effect Illusory correla0on Post-‐purchase ra0onaliza0on Deja vu effect Impact bias Pro-‐innova0on bias Denomina0on effect Informa0on bias Pseudocertainty effect Dis0nc0on bias Insensi0vity to sample size Reactance

Social biases Actor-‐observer bias Illusion of asymmetric insight Projec0on bias

Defensive a@ribu0on hypothesis Illusion of external agency Self-‐serving bias

Dunning–Kruger effect Illusion of transparency Shared informa0on bias

Egocentric bias Illusory superiority System jus0fica0on

Extrinsic incen0ves bias Ingroup bias Trait ascrip0on bias

False consensus effect Just-‐world phenomenon Ul0mate a@ribu0on error

Forer effect (aka Barnum effect) Moral luck Worse-‐than-‐average effect

Fundamental a@ribu0on error Naive cynicism

Group a@ribu0on error Naïve realism

Halo effect Outgroup homogeneity bias

Memory biases Bizarreness effect Illusion of truth effect Processing difficulty effect Choice-‐suppor0ve bias Illusory correla0on Reminiscence bump Change bias Lag effect Rosy retrospec0on Childhood amnesia Leveling and Sharpening Self-‐relevance effect Conserva0sm or Regressive Bias Levels-‐of-‐processing effect Source confusion Consistency bias List-‐length effect Spacing effect Context effect Misinforma0on effect Spotlight effect

Cross-‐race effect Modality effect Stereotypical bias Cryptomnesia Mood-‐congruent memory bias Suffix effect Egocentric bias Next-‐in-‐line effect Sugges0bility Fading affect bias Part-‐list cueing effect Telescoping effect False memory Peak-‐end rule Tes0ng effect Genera0on effect (Self-‐genera0on effect) Persistence Tip of the tongue phenomenon Google effect Picture superiority effect Verba0m effect Hindsight bias Posi0vity effect Von Restorff effect

Humor effect Primacy effect, Recency effect & Serial posi0on effect Zeigarnik effect

Particularly relevant biases •  Availability heuristic •  Confirmation bias •  Focussing effect •  Hindsight bias •  Illusion of control •  Normalcy bias •  Misinformation effect

•  But you might think you’re immune: Naïve realism The belief that we see reality as it really is – objectively and without bias; that the facts are plain for all to see; that rational people will agree with us; and that those who don't are either uninformed, lazy, irrational, or biased.

Logical fallacies •  Formal fallacies

o  Propositional fallacies

o  Quantification fallacies

o  Formal syllogistic fallacies

•  Informal fallacies

o  Faulty generalisations

o  Red herring fallacies

•  Conditional or questionable fallacies

Logical fallacies (shibing the) Burden of proof Argument to modera0on Fallacy of the undistributed middle Naturalis0c fallacy fallacy[38] (an0-‐naturalis0c fallacy[39])

Abusive fallacy Argumentum ad baculum False analogy Nega0ve conclusion from affirma0ve premises (illicit affirma0ve)

Accident Argumentum ad hominem False a@ribu0on Nirvana fallacy (perfect solu0on fallacy)

Ad hominem Argumentum ad populu False authority (single authority) No true Scotsman

Affirma0ve conclusion from a nega0ve premise (illicit nega0ve) Argumentum verbosium False dilemma Onus probandi

Affirming a disjunct Associa0on fallacy (guilt by associa0on) Faulty generaliza0ons Overwhelming excep0on

Affirming the consequent Base rate fallacy Gambler's fallacy Pathe0c fallacy

Ambiguous middle term Begging the ques0on (pe00o principii) Gene0c fallacy Pe00o principii

Appeal to accomplishment Broken window fallacy Hasty generaliza0on Poisoning the well

Appeal to authority (argumentum ab auctoritate) Bulverism (Psychogene0c Fallacy) Hedging Post hoc ergo propter hoc La0n for "aber this, therefore because of this"

Appeal to consequences (argumentum ad consequen0am) Cherry picking (suppressed evidence, incomplete evidence) Historian's fallacy Proof by verbosity (argumentum verbosium, proof by in0mida0on)

Appeal to emo0on Chronological snobbery Homunculus fallacy Prosecutor's fallacy

Appeal to equality Circular cause and consequence If-‐by-‐whiskey Psychologist's fallacy

Appeal to fear Circular reasoning (circulus in demonstrando) Ignora0o elenchi (irrelevant conclusion, missing the point) Red herring

Appeal to fla@ery Conjunc0on fallacy Illicit major Reduc0o ad Hitlerum (playing the Nazi card)

Appeal to mo0ve Con0nuum fallacy Illicit minor Regression fallacy

Appeal to nature Correla0on proves causa0on (cum hoc ergo propter hoc) Incomplete comparison Reifica0on (hyposta0za0on)

Appeal to novelty (argumentum novita0s/an0quita0s) Definist fallacy Inconsistent comparison Retrospec0ve determinism

Appeal to pity (argumentum ad misericordiam) Denying the antecedent Induc0ve fallacy Shotgun argumenta0on

Appeal to poverty (argumentum ad Lazarum) Ecological fallacy Infla0on of conflict Slippery slope (thin edge of the wedge, camel's nose)

Appeal to probability Equivoca0on Informal fallacies Special pleading

Appeal to ridicule Etymological fallacy Judgmental language Straw man

Appeal to spite Existen0al fallacy Ke@le logic Suppressed correla0ve

Appeal to tradi0on (argumentum ad an0quitam) Fallacy of composi0on Ludic fallacy Syllogis0c fallacies

Appeal to wealth (argumentum ad crumenam) Fallacy of division Masked man fallacy (illicit subs0tu0on of iden0cals) Texas sharpshooter fallacy

Argument from (personal) incredulity Fallacy of exclusive premises Mind projec0on fallacy Thought-‐termina0ng cliché

Argument from fallacy Fallacy of four terms (quaternio terminorum) Misleading vividness Tu quoque ("you too", appeal to hypocrisy, I'm rubber and you're glue)

Argument from ignorance Fallacy of many ques0ons Moral high ground fallacy Two wrongs make a right

Argument from repe00on (argumentum ad nauseam) Fallacy of quo0ng out of context (contextomy) Moralis0c fallacy Wishful thinking

Argument from silence (argumentum e silen0o) Fallacy of rela0ve priva0on Moving the goalposts (raising the bar) Wrong direc0on

Argument from silence (argumentum ex silen0o) Fallacy of the single cause (causal oversimplifica0on[29]) Naturalis0c fallacy

So how on earth do we ever get anything done!!!

•  Debate about the irrationality of biases

•  Consensus decision making can be strong

•  There is such a thing as ‘the fallacy fallacy’

Accounting for our deficiencies •  The information lifecycle

•  Good data/information, & decision tools/frameworks hugely important o  RISSB Taking Safe Decisions o  Safety Performance Indicators

The information age •  Information (knowledge) is power

•  There is more information in one edition of the New York Times than the average person in 17th - century England would have come across in a lifetime

•  Watson – IBM’s supercomputer can read 1M books per second

•  Neural networks

•  Turing test

….. but what’s the cost of being smarter ???

But we’re doing ok now aren’t we? •  Many organisations are very good at information gathering / decision

making

•  There is always room for improvement

•  Especially at a national level (remember ON-S1 and OC-G1 ?)

•  Don’t be a victim to your own bias!

o  Overconfidence effect o  Or worse, the Dunning-Kruger

effect!

So what is SISAR? •  Safety Information System for Australasian Rail •  Based on UKs SMIS and SRM

Why would SRM be good for us?



So what’s the BCR?

•  Impossible to put a $ value on it

•  Old school attitude = get into diminishing returns

•  New attitude = safety (& reliability) pays

5 x 9s organisations (99.999 ) $,

Thankyou (for real this time) Questions?

Business

Jesse Baker - RISSB - Industry incident reporting, what’s the BCR?