Cascaded Inference

David Lagnado

Evidence Project

Plan of seminar

Some ideas about cascaded inferencePrevious work on hierarchical inference

Cascaded inference

A chain of probabilistic inferences ‘Inference upon inference’

Pervasive in many domains Legal Medical Criminal Advertising Everyday

Cascaded inference

Horse race example

WEATHER

(Sun, rain, drizzle, frost…)

TRACK CONDITION

(Heavy, soft, good, firm…)

WINNER

(‘Waltzing Along’, ‘Persian Weaver’, ‘Ride the Storm’…)

Cascaded inference

Legal example

TESTIMONY

Witness testifies that accused was at crime scene

EVENT

Accused was at crime scene

CLAIM

Accused is guilty of crime

Cascaded inference

Used car example

Salesman’s report

Maintenance history

Car reliability

Cascaded inference

How do people do it? Very little recent research

Most done in 1971-73 Using abstract problems No comprehensive normative model

Current psychological models also seem inadequate

Normative model for cascaded inference?

Modified Bayes theorem (Dodson), Jeffrey’s rule, Chain rule

Weighted sum across all possible paths

A B C

Given evidence A:

P(C|A) = P(C|B).P(B|A) + P(C|~B).P(~B|A)

A

~B

C

B

~C

E.g., A = heavy rain; B = muddy track; C = ‘Ride the Storm’ wins

P(C|B)

P(C|~B)P(B|A)

P(~B|A)

P(~C|~B)

P(~C|B)

Simplifying heuristics

Various studies claim that people use simplifying strategies (as-if, best guess) Gettys et al., 1973; Schum et al., 1973; Steiger &

Gettys, 1972

Treat inference from first stage as-if it is true Focus on most probable path, ignore alternative

less probable paths Such strategies lead to overestimation of final

probability judgments

As-if or Best guess model

A

~B

C

B

~C

.7

.3

.7

.3

.1

.9

Select the most probable outcome at first stage of inference (from A to B)

E.g., A = heavy rain; B = muddy track; C = ‘Ride the Storm’ wins

As-if or Best guess model Select the most probable outcome at first stage of

inference (from A to B) Assume this is true for the second stage inference

(from B to C) Judged probability of C given A = .7 This overestimates the correct value = .52

A

~B

C

B

~C

.7

E.g., A = heavy rain; B = muddy track; C = ‘Ride the Storm’ wins

Current study

Looking at both abstract and applied settings Some studies claim that overestimation only occurs in

abstract settings

Vary presentation of probability information to see if this improves inferences Frequency tables Pie charts Network diagrams?

Shortcomings with previous research

Neglects structural assumptions Causal models Conditional independencies etc

These determine appropriate normative model, and likely to influence people’s reasoning

Current psychological models of probabilistic reasoning do not take these factors into account Belief activation (neural networks) Belief updating (rule-based) Heuristic models

Influence of causal model

How are cascaded inferences affected by assumptions about causal structure?

Different causal models can imply different conditional independencies

A B C

A and C are (unconditionally) dependent

BUT A and C are conditionally independent given B

A B C

A and C are (unconditionally) independent

BUT A and C are conditionally dependent given B

Causal models and cascaded inference

Measles Spots Itchy

Evidence = Measles

Infer that spots are probable

Infer that itchiness is probable

Measles Spots Chicken pox

Evidence = Measles

Infer that spots are probable

Do not infer that Chicken pox is probable

Evidence does not alter P(Chicken pox)

Causal structure influences permissible inferences

Causal models and cascaded inference

Schum – ‘conditional non-independence’

A B C

E.g., A = Weather, B = Track condition, C = Winner

Some horses may run better/worse in the rain, independent of the track condition

Moral

Normative models for cascaded inference depend on causal and structural assumptions

Future research

Construct cognitive model of cascaded inference that allows for causal and structural assumptions

How well does this conform to normative models? Various studies suggest that people are poor at probability

estimation/computation, but are quite good at qualitative causal reasoning

Decision aids To support inference when there are many variables,

complex computations etc. To correct for any systematic biases Better presentation of probability information E.g., graphical representations …

Interdisciplinary aid

What are the appropriate normative models (statistics)?

What are the plausible computational algorithms?

Are there economical heuristic procedures? What are the cognitive mechanisms that people

use (psychology, neuroscience)? Are there naturally occurring inference problems

of this kind (epidemiology, forensic, legal, history)?

Some earlier studies on hierarchical inference

Cascaded inferenceHierarchically structured informationLearning and judgment

Learning and judgment with category hierarchies

Hierarchical structure Pervasive feature of how we represent the world Organizes knowledge and structures inference

FLU

Type BType A Type C

Inference using a hierarchy

One powerful feature of a category hierarchy is that given information about categories at one level, you can make inferences about categories at another level.

This allows you to exclude alternatives, or reduce the

number you need to consider

Tabloid Broadsheet

TimesGuardianMirrorSun

Probabilistic inference using a hierarchy

In many real-world situations we must base our initial category judgments on imperfect cues, degraded stimuli, or statistical data.

What effect do such probabilistic contexts have on the hierarchical inferences that we are licensed to make?

Tabloid Broadsheet

TimesGuardianMirrorSun

Commitment heuristic

Reason ‘as-if’ probable info is true (but reduce overall confidence)

Commitment heuristic - When people select the most probable category at the superordinate level, they assume that it contains the most probable subordinate category (and vice-versa)

This leads to the neglect of subordinates from the less probable superordinate

Tabloid Broadsheet

TimesGuardianMirrorSun

How effective is such a heuristic?

Depends on the structure of the environment In certain environments it is advantageous

increases inferential power by focus on appropriate subcategories

reduces computational demands by avoiding complex Bayesian calculations

But in some environments it leads to anomalous judgments and choices

Non-aligned hierarchy

The most frequently read type of paper is a Tabloid, but the most frequently read paper is the Guardian (a Broadsheet).

Non-aligned hierarchy: the most probable superordinate category does not contain the most probable subordinate

category.

Tabloid 60 Broadsheet 40

Times 5 Guardian 35Mirror 30Sun 30

Cascaded inference

Inference from hierarchy to a related category Different levels can support opposing predictions

Tabloid 60 Broadsheet 40

Times 5 Guardian 35Mirror 30Sun 30

Party A Party B

Experiments 1 & 2

What is effect of manipulating level of representation on subsequent probability judgment?

Learning phase - participants exposed to a non-aligned hierarchical environment in which they learn to predict voting behavior from newspaper readership.

100 trials ‘reading/voting profiles’

Screen during learning phase

Broadsheet

Sun

Tabloid

Mirror Guardian Times

○ Party A

○ Party B

Screen during learning phase

Broadsheet

Sun

Tabloid

Mirror Guardian Times

○ Party A

○ Party B

Reading profile for J. K.

Screen during learning phase

Broadsheet

Sun

Tabloid

Mirror Guardian Times

○ Party A

○ Party B

Reading profile for J. K.

Outcome feedback

Structure of environment

Tabloid 60 Broadsheet 40

Times

5

Guardian

35

Mirror

30

Sun

30

Party A Party B

50 50

NB Overall each party equally frequent

Judgment phase

Which paper is X most likely to read?

X is selected at random

What is the probability that X votes for Party B rather than A?

Which kind of paper is X most likely to read?

Baseline

General

Specific

Predictions

Which paper is X most likely to read?

X is selected at random

What is the probability that X votes for Party B rather than A?

Which kind of paper is X most likely to read?

Baseline

General

Specific

General choice Tabloid -> Party A

Specific choice Guardian -> Party B

Based on commitment heuristic

Results

Which paper is X most likely to read?

X is selected at random

What is the probability that X votes for Party B rather than A?

Which kind of paper is X most likely to read?

Baseline

General

Specific

General choice Tabloid -> Party A

Specific choice Guardian -> Party B

47%

28%

75%

Mean probability rating

Summary

People allow their initial categorization to shift their inferences, even though all judgments are based on the same statistical data.

Simplifying heuristic that assumes that environment is aligned

Empowers inference when hierarchical structure is aligned, otherwise can lead to error

Suggests tendency to reason as if a probable conclusion is true

Applied to Medical choices

In medical settings treatment options (and survival rates) often grouped to facilitate understanding and communication

Can this lead to errors?

SURGERY DRUGS

Type2

Type1

Type2

Type1

Medical choices

Suppose success rates are non-aligned Will grouping affect people’s treatment choices?

SURGERY 60%

DRUGS 40%

Type2

10%

Type1

70%

Type2

60%

Type1

60%

Medical choices experiment

Participants learn about success rates trial-by-trial

SURGERY 60%

DRUGS 40%

Type2

10%

Type1

70%

Type2

60%

Type1

60%

Medical choices experiment

Ungrouped control – 75% chose most effective treatment

Drug4

10%

Drug3

70%

Drug2

60%

Drug1

60%

Medical choices experiment

Grouped condition – 25% chose most effective treatment (first asked about best superordinate treatment)

SURGERY 60%

DRUGS 40%

Type2

10%

Type1

70%

Type2

60%

Type1

60%

Summary

Grouping with a non-aligned hierarchy can lead to poorer choices

How generalizable is this? Does it depend on memory processes? Will it apply to summary presentations?

Consider situations where people must use large databases with various levels of hierarchy (e.g., NHS statistics)

Useful biases?

Systematic biases reveal something about judgment and learning processes

Need not be indictment of human reasoning Heuristic strategies might be well adapted to the

inferential tasks that we commonly confront

Challenge for future

Do people use simplifying heuristics in cascaded and hierarchical inference?

If so, how can we work with these heuristics to improve human judgment?