Thinking about Evidence

David LagnadoUniversity College London

Leonard Vole accused of murdering a rich elderly lady Miss French

Vole had befriended French and visited her regularly including night of murder

Vole needed money

French changed her will to include him; shortly after he enquired about luxury cruises

Maid testified Vole was with French at time of death

Blood on Vole’s jacket same type as French

Romaine, Vole’s wife, was to testify that he was with her at time of murder

But instead Romaine appears as witness for prosecution

Testifies that Vole was not with her, returned later with blood on his jacket, and said “I’ve killed her”

Letters written by Romaine to lover – reveals her plan to lie and incriminate Vole

Vole is acquitted!

Evidential reasoning

• How do people reason with uncertain evidence?

• How do they assess and combine different items of evidence?– What representations do they use?– What inference processes?

• How do these compare with normative theories?

Reasoning with legal evidence

• Legal domain– E.g. juror, judge, investigator, media

• Complex bodies of interrelated evidence– Forensic evidence; witness testimony; alibis;

confessions etc

• Need to integrate wide variety of evidence to reach singular conclusion (e.g. guilt of suspect)

Descriptive models of juror reasoning

• Belief adjustment model (Hogarth & Einhorn, 1992)

– Sequential weighted additive model – Over-weights later items– Ignores relations between items of evidence

• Story model (Pennington & Hastie, 1992)

– Evidence evaluated through story construction – Holistic judgments based on causal models– No formal, computational or process model

Descriptive models of juror reasoning

• Coherence-based models (Simon & Holyoak, 2002)

– Mind strives for coherent representations– Evidential elements cohere or compete– Judgments emerge through interactive process that

maximizes coherence– Bidirectional reasoning (evidence can be re-evaluated

to fit emerging conclusions)

How should people do it?• Bayesian networks?• Nodes represent evidence statements or hypotheses• Directed links between nodes represent causal or

evidential relations• Permits inference from evidence to hypotheses (and

vice-versa)

Guilt

Maid

Vole is guilty

Maid testifies that Vole was with Miss French

Blood

Blood on Vole’s cuffs

CutVole cut wrist slicing ham

Partial BN of ‘Witness for Prosecution’

Partial Bayesian net for Sacco and Vanzetti trial

Applicable to human reasoning?

• Vast number of variables

• Numerous probability estimates required

• Complex computations

Applicable to human reasoning?

• Fully-fledged BNs unsuitable as model of limited-capacity human reasoning

• BUT –

a key aspect is the qualitative relations between variables (what depends on what)

• Judgments of relevance & causal dependency critical in legal analyses

• And people seem quite good at this!– Blood match raises probability of guilt – Alibi lowers it (not much!) Guilt

Blood Alibi

+ -

Qualitative causal networks (under construction!)

• People reason using small-scale qualitative networks

• Require comparative rather than precise probabilities

• Guided by causal knowledge

• More formalized & testable version of story model?

Empirical studies

• Discrediting Evidence

• Alibi Evidence

Discredited evidence• How do people revise their beliefs once an item

of evidence is discredited? – When testimony of one witness is shown to be

fabricated, how does this affect beliefs about testimony of other witnesses, or even other forensic evidence?

– E.g., Romaine’s discredited testimony

• Involves a distinctive pattern of inference

Explaining away

Vole cut himself

P(G|B&C) < P(G|B)

Finding out C too lowers the probability of G

Despite its simplicity and ubiquity, this pattern of inference is hard to capture on most psychological models of inference (e.g., associative models, mental models, mental logic)

P(G|B) > P(B)

Finding out B raises probability of G

Blood on Vole’s cuffs

Vole is guilty of murder

Guilt

Blood

Cut

Discrediting vs. direct evidence

Guilt

Blood

Cut

Bayesian network model

Causal model

CUT only becomes relevant to guilt given BLOOD

Important to distinguish ‘explaining away’ from simply adding (negative) evidence

Weighted additive model

Standard regression model

Guilt

Blood Cut

Experimental questions• Do people use causal models to reason with

evidence in online tasks?• Do they model discrediting evidence by

‘explaining away’?• How does the discredit of one item of evidence

affect other items?

YES when same source

EVIDENCE 1

Neighbour says that suspect has stolen previously

NO when different source

EVIDENCE 1

Footprints outside house match suspect’s

HYPOTHESIS: Local man did it

Scenario: House burglary, local man arrested

EVIDENCE 2

Neighbour says he saw suspect outside house on night of crime

Neighbour is lying because he dislikes suspect

?Does the discredit of item 2 affect item 1?

Extension of discredit• When do people extend the discredit of one item to other

items?• SAME

– E.g. two statements from same neighbour• SIMILAR

– E.g. two statements from two different neighbours• DIFFERENT

– E.g., one statement and one blood test

• Causal model approach would expect people to distinguish SAME from DIFFERENT cases

BN models

GUILT

Witness A

Witness B

Discredit

Same/Similar

GUILT

Blood test

Witness

Discredit

Different

Experiment 1• Mock jurors given simplified criminal cases• Four probability judgments (of guilt)

– Baseline– Stage 1 (Evidence 1) Footprint match– Stage 2 (Evidence 2) Neighbour sees suspect– Final (Discredit 2) Neighbour is lying

Compare judgments at Final stage and Stage 1 Does discredit return judgments to Stage 1?

Vary relations between items of evidence– SAME, SIMILAR, DIFFERENT source

Witness1 Witness2 Discredit2 Both items undermined

Forensic1 Witness2 Discredit2

When discredit presented LAST, it is extended regardless of relations between items

Results Final judgments significantly

lower than at Stage 1 for all conditions

Discredit does not simply remove item 2; also affects belief in item 1

Summary• Discrediting information extended regardless of

relation to other evidence• This pattern is consistent with Belief Adjustment

model– Recency effect leads to over-weighting of discrediting

information– Neglect relations between items

• Further test of BAM: manipulate order of evidence presentation

Experiment 2• Vary order of presentation of evidence

– LATE……E1 E2 D2– EARLY….E2 D2 E1

– Both orders ‘ought’ to lead to same conclusions

• Relatedness – SAME, DIFFERENT

Witness1 Witness2 Discredit2 Both items undermined

Forensic1 Witness2 Discredit2

When discredit presented LAST, it is extended regardless of relations between items

Results: Late condition Final judgments lower

than at Stage 1 for both conditions

Discredit does not simply remove item 2

Replicates EXP 1

When discredit presented EARLY, only extended to related items

Results: Early condition Pattern of judgments differ

for SAME and DIFF SAME

Final = Stage 2 DIFF

Final > Stage 2 Appropriate sensitivity to

relation between items

Witness1 Discredit1 Both items underminedWitness2

Witness1 Forensic1Discredit1 Only 1st item undermined

Problematic for current models

• Why are people ‘rational’ in early but not late condition?

• Belief Adjustment model – Cannot explain early condition because does not

consider relations between evidence

• Story model – Cannot explain bias in late condition (and needs to be

adapted to online processing)

Coherence-based/grouping account

• Mind strives for most coherent representation• Evidence grouped as +ve or -ve relative to guilt• +ve and -ve groups compete, but within-group

items mutually cohere (irrespective of exact causal relations)

• When an item of one group is discredited, this affects other items that cohere with it

LATE condition• Incriminating evidence grouped together

(regardless of source)• Discredit affects the group (not just individual

item)

GUILT

A

+

B

+

D

+ +

EARLY condition• First item of evidence discredited• Second item only discredited if from related

source• No grouping effect

GUILT

B

+

D

A

+++

Study 3• Grouping hypothesis predicts that coherent

groupings only emerge with elements that share the same direction (cf. Heider, 1946)

• Therefore discredit extended when evidence items both +ve or both -ve, but not with mixed items

Design• Four evidence conditions

1. A+, B+, discredit B+

2. A-, B-, discredit B-

3. A+, B-, discredit B-

4. A-, B+, discredit B+

• Two levels of relatedness: similar and different• Predictions

– 1&2 non-mixed -> discredit affects both items– 3&4 mixed -> discredit affects only second item

Examples: Condition 2 - - different

Neighbour says she was with suspect at time of crime

Neighbour lying because in love with suspect

Lab tests reveal no footprint match

Evidence 1 Evidence 2 Discredit

Examples: Condition 3 + - different

Lab tests reveal footprint match

Evidence 1 Evidence 2 Discredit

Neighbour says she was with suspect at time of crime

Neighbour lying because in love with suspect

Results

Summary• Grouping hypothesis

supported• Discredit extended when

items share common direction, not when mixed

• Mutually coherent elements stand or fall together (even when no clear causal relation between them)

• Romaine & Agatha Christie knew this!

Alibi evidence

• Often crucial evidence (if true, absolves suspect)• Treated with suspicion• Hard to generate (even if innocent)• Very little formal or empirical work• Ongoing psychological studies – what makes a good

alibi? (e.g., how much detail is best)• Also interesting from normative viewpoint

Witness vs. Alibi models

H

E

E*

Suspect committed crime

Witness report of suspect at crime scene

Suspect at crime scene

H

E

A Suspect claims he was not at crime scene

DSuspect motivated to lie

In alibi case – if suspect says he wasn’t there, but he was, this raises likelihood of guilt (beyond that if you just find out he was there)

To understand alibi evidence – need to represent potential deception

With impartial witness – knowing that suspect was at crime scene ‘screens off’ witness report from guilt judgment

+

++

+

+-

P(H|E&E*)=P(H|E) P(H|E&A)>P(H|E)Even though P(H|A)<P(H)

Pilot study

• Compare discredit of witness vs. alibi evidence • Manipulate reason for discredit

– Deception (X was lying in his statement)– Error (X was mistaken in his statement)

• Mock jurors given crime scenarios• 3 judgments of guilt

– Baseline– After statement (alibi/witness)– After discredit of statement

0

10

20

30

40

50

60

70

80

90

100

1 2 3

Judgment stage

Pro

bab

ility

of g

uilt

Witness/Error

Witness/Deception

Alibi/Error

Alibi/Deception

Alibi – discredit returns belief to baseline in error condition, but greatly enhances guilt in deception condition

Fits with causal network predictions

Witness – discredit returns belief to baseline (j1 = j3) irrespective of reason

Results

General alibi model

H

E

A Suspect claims he was not at crime scene

DSuspect motivated to lie

Case 1: Suspect provides alibi

Higher motivation to lie if guilty than if innocent

(hence link from H to D)

Given alibi, discovery of E incriminates via two routes

E raises likelihood of H directly

E raises likelihood of H indirectly

(via its effect on D)

++

+-

No screening-off ie P(H|E&A) > P(H|E)

General alibi model

H

E

A Friend claims suspect was not at crime scene

DFriend motivated to lie

Case 2: Close relative/friend provides alibi

AND they know whether or not suspect is guilty

Higher motivation to lie if guilty than if innocent

(hence link from H to D)

Given alibi, discovery of E incriminates via two routes

++

+-

No screening-off ie P(H|E&A) > P(H|E)

General alibi model

H

E

A Friend claims suspect was not at crime scene

DFriend motivated to lie

Case 3: Close relative/friend provides alibi

BUT they do NOT know whether suspect is guilty

Motivation to lie irrespective of actual guilt or innocence of suspect

(effectively no link from H to D)

Given alibi, discovery of E incriminates only via direct route

+

+-

Screening-off ie P(H|E&A) = P(H|E)

General alibi model

H

E

A Stranger claims that suspect was not at crime scene

DStranger motivated to lie

Case 4: Impartial stranger provides alibi

AND they do NOT know whether suspect is guilty

Low Motivation to lie AND this is unrelated to actual guilt or innocence of suspect

(effectively no link from H to D)

Given alibi, discovery of E incriminates only via direct route

+

+-

Screening-off ie P(H|E&A) = P(H|E)

Experimental study

• Do people conform to these models?• Background info:

– eg Victim is attacked on her way home … suspect is arrested

• Alibi: ‘suspect was elsewhere at time of crime’• Manipulate who provides the alibi• Discredit Alibi

– e.g., suspect seen on CCTV near crime scene at time of crime

Alibi provider

Motivated to lie?

Knows H? Prediction

Suspect YES YES P(H|E&A) >

P(H|E)

Close friend YES YES P(H|E&A) >

P(H|E)

Work colleague

MAYBE NO P(H|E&A) =

P(H|E)

Stranger NO NO P(H|E&A) =

P(H|E)

Results so far

>

=

=

=

• Scenarios don’t clarify that close friend knows H (as shown by subjects’ judgments about this)

• Strong order effects ---ALIBI, CCTV >> CCTV, ALIBI

Conclusions so far

• People construct and use causal models• ‘Explaining-away’ inferences• Grouping of variables can lead to biases• Sensitive to Alibi model• Puzzling order effect with Alibis

• Judgment involves both causality and coherence?

Thank you!• Leverhulme/ESRC Evidence project

– Nigel Harvey– Phil Dawid– Amanda Hepler– Gianluca Baio

• Students– Miral Patel– Nusrat Uddin– Charlotte Forrest