BIASES IN REASONING IN AVIATION NAVIGATION

Stephen Hill1 and Andrew Gilbey2

1. School of Psychology2. School of Aviation

Confirmation Bias

“A tendency to seek out or interpret information in a way that confirms one’s preconceptions and avoid

information that contradicts them”

Confirmation Bias and the Erebus Disaster

True Course

PerceivedCourse

DC 10, ZK NZP

Confirmation Bias and the Erebus Disaster

“Those on the flight deck interpreted features of the topography ahead in such a way as to confirm their mindset, not challenge it. The flat slopes of Ross Island and Beaufort Island were misinterpreted as features associated with McMurdo Sound instead of Lewis Bay at the foot of Erebus.”

(McGregor, 2006, p. 21).

Confirmation Bias (CB)

• A many splendoured thing– ‘Motivated’ CB versus ‘Cognitive’ CB

• Term used ‘rather loosely’ in psychology and related disciplines

• Most famous and popular CB tasks– Peter Wason’s 4 card task– Wason’s Rule Discovery Task

The Rule Discovery Task

• Wason (1960)– Ps required to discover experimenter’s rule (RT) (e.g.,

ascending numbers) by asking ‘yes/no’ questions– Given a ‘seed’ triple 2-4-6– Ps can propose as many triples as they like in order to work

out RT

• Findings– Ps exhibit a high degree of confidence in incorrect

hypotheses (~20% success rate, 75% eventually get it right)

– Ps typically guess (their RH) the rule is ‘3 ascending even numbers’

– Ps adopt a positive testing strategy (PTS) = trying out triples for which the answer is expected to be ‘yes’ given RH

‘Confirmation Bias’ and the Positive Testing Strategy (PTS)

• Problems with Confirmation Bias– Wason conflated use of a PTS with an attempt to confirm

one’s favoured hypothesis (e.g., Wetherick, 1962)– Also unclear whether it makes sense to insist that people try

and ‘falsify their best guess’ (Poltiek, 1996)– Not a ‘principle of reasoning’ (e.g., Evans, 1989, 2006)– Not nearly as common as once thought (e.g., McKenzie,

2006)

• CB is best seen as a suboptimal outcome rather than a suboptimal behavioural strategy– Focus has moved to the choice of test strategies used by Ps

• PTS is a useful strategy in many contexts (Klayman & Ha, 1987, 1989)

Possible Relationships between H and T

H T H

H

HT

T

T

Disjoint Overlapping

Embedded Surrounding

U U

U U

PTS is only really problematic when H is embedded in T

Getting Lost

• Unlike Erebus and similar events, most lost pilots successfully get ‘un-lost’

– Suggests pilots might be pretty good at testing location hypotheses

• Pilots are instructed to follow ‘General Aviation Lost Procedures’*

1. Orbit a landmark2. Draw a circle of uncertainty (COU) on map3. Look for a distinctive landmark (e.g., farm name on

roof) on the ground and try and find it in COU

*NB: only if a pilot realises they are lost of course.

Confirmation Bias and Navigation

• Aim: Construct a navigation equivalent of the RDT based on a ‘toy version’ of general aviation lost procedures

• Explore whether …1. Pilots are better than the average person at

‘resisting confirmation bias’ in a navigation situation.

2. Find out what kinds of strategies are used to reason about location.

The Location Discovery Task

• Mapping the RDT on to the LDT …– RT → where you really are

– RH → where you think you are (COU)

– Triples → landmarks

• All relationships between RT and RH can be explored but we were most interested in the (difficult) embedded relationship

The Location Discovery Task

• 5 studies– Level of expertise varied (students, pilots, orienteers)– Information given about CB (control, intervention)

• Ps asked to imagine being lost

• Given a ‘circle of uncertainty’ (→ RH for most people)

• Asked to select 1 of 3 visible landmarks that is best for working out one’s location– * We also asked people to provide reasons for their choices

1. There is a picnic area approx 20m from where you are standing and a stream crosses under the road

2. In the distance to the east there are high bush covered ranges

3. There is a junction and to your left there is an unsealed road (light brown) leading off to the west

+

+

-

Motorcycle Accident Scenario

0

0.5

1

1.5

2

2.5

3

Students Pilots Orienteers

Control

Intervention

Chance level

* *

**

*

* Significantly worse than chance

** Significantly better than chance

Mea

n D

isco

nfi

rmat

ory

Res

po

nse

s (o

ut

of

3)

Peformance within Scenarios

• Performance in scenarios varied widely• Sea Scenario 8.0% correct on average• Motorcycle Scenario 46.5%• Flying Scenario 24.1%

• Despite this performance rankings were very similar across scenarios

1. Orienteers2. Pilots (intervention)3. Psychology students (intervention)4. Pilots (non-intervention)* - except for Sea where Pilots were 5th

5. Psychology students (non-intervention)** - except for Sea where Psychology students were 4th

Performance Across Scenarios

0

0.2

0.4

0.6

0.8

1

Sea Flying Motorcycle

Orienteers

Others

Scenario (in order of difficulty)

Pro

port

ion

Co

rrec

t

Peformance Across Scenarios

• Why did performance vary so dramatically across scenarios?

– It appears many Ps might not be using the map at all!

• Landmarks may have been selected based on how uncommon (diagnostic) Ps believed them to be

Scenario Correct

Correct landmark

Most common choice

Sea 8% High bush clad peak behind town

Small aircraft landing

behind town

Motorcycle 46.5% Picnic area with stream running under road

Picnic area with stream running under

road Flying 24.1% Large pine

plantation either side of road and railway

Railway line with branch line leading off

east

Implications

• Why are orienteers better than pilots?• Why were pilots no better than non-experts?

– Evidence mixed on power of expertise to improve performance on RDT and related tasks

– Theorists argue that people find it very difficult to naturally consider multiple hypotheses

• But dual goal studies show it helps if you can

– Are orienteers inclined to consider dual goals?• Are they used to being put in situations where they are ‘tricked’

to think they are located where they are not?• Could this drive a natural inclination to dual goal use?

Implications

• What is the best solution for training pilots?

– Encouraging negative testing is difficult• Klayman & Ha (1987) suggest we have a positivity bias because

PTS works well in so many situations

– Difficult to consider more than one hypothesis (Principle of singularity)

– Many lost situations are perfectly amenable to a single hypothesis, positive testing strategy

• Therefore not a lot of demand for dealing with ‘embedded situations’

Implications

• What is the best solution for training pilots?– Perhaps change the situation to one where a single

hypothesis combined with PTS is most likely to work?• = turn situation (relationship between T and H) into a disjoint, or at

worst, overlapping one

– How can we do this?1. Use highly diagnostic landmarks wherever possible2. Use multiple landmarks to increase chance of use of a unique

landmark3. Individuate landmarks as much as possible to increase diagnosticity

– BUT these strategies won’t guarantee avoiding CB so also …4. Teach them about confirmation bias?5. Perhaps encourage them to do some orienteering …?