The influence of hierarchy The influence of hierarchy on probability judgmenton probability judgment

David A. LagnadoDavid A. Lagnado

David R. ShanksDavid R. Shanks

University College LondonUniversity College London

Level of hierarchy can Level of hierarchy can modulate judgmentmodulate judgment

Consider two statements about the next World Consider two statements about the next World CupCup It is most likely that Brazil will winIt is most likely that Brazil will win It is most likely that a European team will It is most likely that a European team will

winwin

These appear to support opposing predictions, These appear to support opposing predictions, but both may be truebut both may be true

Shows the importance of the level at which Shows the importance of the level at which probabilistic information is representedprobabilistic information is represented

Hierarchical structureHierarchical structure

Pervasive feature of how we represent Pervasive feature of how we represent the worldthe world Reflects pre-existing physical and social Reflects pre-existing physical and social

hierarchieshierarchies Readily generated through conceptual Readily generated through conceptual

combinationcombination

Category hierarchies serve both to Category hierarchies serve both to organize our knowledge, and to organize our knowledge, and to structure our inferences structure our inferences

Inference using a hierarchyInference using a hierarchy

One powerful feature of a category hierarchy One powerful feature of a category hierarchy is that given information about categories at is that given information about categories at one level, you can make inferences about one level, you can make inferences about categories at another level. categories at another level.

This allows you to exclude alternatives, or This allows you to exclude alternatives, or reduce the number you need to consider reduce the number you need to consider

Tabloid Broadsheet

TimesGuardianMirrorSun

Probabilistic Inference using a Probabilistic Inference using a hierarchyhierarchy

In many real-world situations we must base In many real-world situations we must base our initial category judgments on imperfect our initial category judgments on imperfect cues, degraded stimuli, or statistical data. cues, degraded stimuli, or statistical data.

What effect do such What effect do such probabilistic probabilistic contexts contexts have on the hierarchical inferences that we have on the hierarchical inferences that we are licensed to make? are licensed to make?

Tabloid Broadsheet

TimesGuardianMirrorSun

Commitment heuristicCommitment heuristic

Commitment heuristicCommitment heuristic - When people select - When people select the most probable category at the the most probable category at the superordinate level, they assume that it superordinate level, they assume that it contains the most probable subordinate contains the most probable subordinate category. category.

This leads to the neglect of subordinates from This leads to the neglect of subordinates from the less probable superordinate.the less probable superordinate.

Tabloid Broadsheet

TimesGuardianMirrorSun

How adaptive is this How adaptive is this heuristic?heuristic?

The efficacy of such a heuristic depends The efficacy of such a heuristic depends on the precise structure of the on the precise structure of the environment. environment.

In certain environments it confers In certain environments it confers considerable advantages considerable advantages increases inferential power by focus increases inferential power by focus

on appropriate subcategories on appropriate subcategories reduces computational demands by reduces computational demands by

avoiding complex Bayesian avoiding complex Bayesian calculations. calculations.

But in some environments it can lead to But in some environments it can lead to anomalous judgments and inferences. anomalous judgments and inferences.

Non-aligned hierarchyNon-aligned hierarchy

In the above sample the most frequently read In the above sample the most frequently read typetype of paper is a Tabloid, but the most of paper is a Tabloid, but the most frequently read paper is a Broadsheet (the frequently read paper is a Broadsheet (the Guardian). Guardian).

Non-aligned hierarchy: Non-aligned hierarchy: the most probable the most probable superordinate category does not contain the superordinate category does not contain the most probable subordinate category. most probable subordinate category.

Tabloid 60 Broadsheet 40

Times 5 Guardian 35Mirror 30Sun 30

Real world examplesReal world examples

Word frequencies: the superordinate BE- is Word frequencies: the superordinate BE- is more frequent than BU-, but the more frequent than BU-, but the subordinate BUT is more frequent than any subordinate BUT is more frequent than any of the other subordinates (BET, BED…etc.)of the other subordinates (BET, BED…etc.)

NHS statistics on survival rate for NHS statistics on survival rate for operations for different areas & sub-areasoperations for different areas & sub-areas You are more likely to survive a hip operation in You are more likely to survive a hip operation in

Surrey rather than Essex, but the best sub-area Surrey rather than Essex, but the best sub-area for survival is Colchester (in Essex).for survival is Colchester (in Essex).

Experiments 1 and 2Experiments 1 and 2

Learning phaseLearning phase - participants exposed to a - participants exposed to a non-aligned hierarchical environment in non-aligned hierarchical environment in which they learn to predict voting which they learn to predict voting behavior from newspaper readership. behavior from newspaper readership.

100 trials ‘reading/voting profiles’100 trials ‘reading/voting profiles’

Screen during learning Screen during learning phasephase

Broadsheet

Chronicle

Tabloid

Herald Reporter Globe

○ Liberal

○ Progressive

Screen during learning Screen during learning phasephase

Broadsheet

Chronicle

Tabloid

Herald Reporter Globe

○ Liberal

○ Progressive

Reading profile for J. K.

Screen during learning Screen during learning phasephase

Broadsheet

Chronicle

Tabloid

Herald Reporter Globe

○ Liberal

○ Progressive

Reading profile for J. K.

Outcome feedback

Structure of environmentStructure of environment

Tabloid 60 Broadsheet 40

Times 5Guardian 35Mirror 30Sun 30

Party A Party B

50 50

Judgment phaseJudgment phase

Which paper is X most likely to read?

X is selected at random

What is the probability that X votes for one party rather than the other?

Which type of paper is X most likely to read?

Baseline

Type

Paper

Results of Experiment 1Results of Experiment 1

0

10

20

30

40

50

60

70

80

90

100

aligned non-aligned

Mean p

robabili

ty r

ating f

or

Part

y B

baselinetypepaper

Probability ratings for Party B rather than Probability ratings for Party B rather than Party A with judgments divided into those Party A with judgments divided into those based on aligned and non-aligned choicesbased on aligned and non-aligned choices

Experiment 2Experiment 2

Replication of Experiment 1, with Replication of Experiment 1, with frequency as well as probability frequency as well as probability response formatsresponse formats

Frequentist hypothesis that probability Frequentist hypothesis that probability biases reduced with frequency format biases reduced with frequency format

Results of Experiment 2Results of Experiment 2

Mean ratings for Party B rather than Party A Mean ratings for Party B rather than Party A collapsed across probability and frequency ratingscollapsed across probability and frequency ratings

0

10

20

30

40

50

60

70

80

90

100

aligned non-aligned

Mean r

ating f

or

Part

y Bbaselinetypepaper

Summary of ResultsSummary of Results

Participants allow their initial probability Participants allow their initial probability judgment about category membership judgment about category membership (newspaper readership) to shift their (newspaper readership) to shift their rating of the probability of a related rating of the probability of a related outcome (voting preference), even outcome (voting preference), even though all judgments are made on the though all judgments are made on the basis of the same statistical data. basis of the same statistical data.

When their prior choices were non-When their prior choices were non-aligned this led to a switch in predictions aligned this led to a switch in predictions about the outcome categoryabout the outcome category

Conclusions Conclusions

These biases are explicable by the These biases are explicable by the Commitment heuristicCommitment heuristic:: The priming question commits people The priming question commits people

to just one inferential path, leading to just one inferential path, leading them to compute an erroneous them to compute an erroneous estimate for the final probability.estimate for the final probability.

This is understandable given the This is understandable given the complexity of the normative complexity of the normative Bayesian computation.Bayesian computation.

Comparison of Bayesian and Comparison of Bayesian and commitment heuristic computations commitment heuristic computations

(just type level inference)(just type level inference)

0.40.6

0.770.23 0.1

0.9

P(A) = (0.6 . 0.77) + (0.4 . 0.1)= 0.46 + 0.04= 0.5

P(A) = 0.77

Bayesian computation Simplified heuristic computation

Type of paper? Type of paper?

Tabloid Broadsheet Tabloid

Party A Party B Party A

0.77

0.6

ConclusionsConclusions

Simplifying heuristic that assumes Simplifying heuristic that assumes that environment is alignedthat environment is aligned

Empowers inference when Empowers inference when hierarchical structure is aligned, hierarchical structure is aligned, otherwise can lead to errorotherwise can lead to error

Suggests tendency to reason Suggests tendency to reason as ifas if a a probable conclusion is trueprobable conclusion is true

Process level accountsProcess level accounts Associative model Associative model

People learn predictive relations between People learn predictive relations between category options (at both levels of hierarchy) category options (at both levels of hierarchy) and outcome. At test responses to category and outcome. At test responses to category questions prime the appropriate associations questions prime the appropriate associations and lead to a biased rating of the outcome. and lead to a biased rating of the outcome.

Frequency-based model Frequency-based model People encode event frequencies in the People encode event frequencies in the

learning phase. At test responses to the learning phase. At test responses to the category question serves as the reference category question serves as the reference class for subsequent class for subsequent conditional conditional probability probability judgments about voting preferences. judgments about voting preferences.

ImplicationsImplications Importance of the level at which probabilistic Importance of the level at which probabilistic

data is represented to (or by) a decision maker data is represented to (or by) a decision maker E.g., using NHS statistics to decide on E.g., using NHS statistics to decide on

hospitalhospital

How do people search through hierarchical How do people search through hierarchical statistical data?statistical data?

People’s judgments can be manipulated by the People’s judgments can be manipulated by the level at which statistical information is level at which statistical information is representedrepresented

More generally, in multi-step inferences people More generally, in multi-step inferences people are susceptible to biased probability judgmentsare susceptible to biased probability judgments