Base Rate Neglect (continued) Psychology 466: Judgment & Decision Making Instructor: John Miyamoto 10/29/2015: Lecture 05-2 Note: This Powerpoint presentation

Base Rate Neglect (continued)

Psychology 466: Judgment & Decision Making

Instructor: John Miyamoto 10/29/2015: Lecture 05-2

Note: This Powerpoint presentation may contain macros that I wrote to help me create the slides. The macros aren’t needed to view the slides. You can disable or delete the macros without any change to the presentation.

Outline

• Experimental manipulation of base rate information.Another demonstration of base rate neglect.

• The inside view and the outside view.

• Attribute substitution in probability judgment.

Psych 466, Miyamoto, Aut '15 2

Bayes Rule – What Is It? Why Is It Important?


• Reverend Thomas Bayes, 1702 – 1761

British clergyman & mathematician

• Bayes Rule is fundamentally important to:o Bayesian statisticso Bayesian decision theoryo Bayesian models in psychology

P Data|Hypothesis P(Hypothesis)P(Hypothesis|Data) =

P(Data)

1

n

i ii

P(Data) P Data | Hypothesis P Hypothesis

Explanation of Bayes Rule

H = a hypothesis, e.g., H = hypothesis that the patient has cancer

= the negation of the hypothesis,

e.g., ~H = hypothesis that the patient does not have cancer

D = the data, e.g., D = + test result for a cancer test

Bayes Rule (Odds Form)


P H| D

P(~ H | D)

~ H

Memorable Form of the Bayes Rule (Odds Version)

P D | H P(H)

P D | ~ H P(~ H)

Posterior OddsLikelihood Ratio(diagnosticity)

Prior Odds

H = a hypothesis, e.g., H = hypothesis that the patient has cancer

= the negation of the hypothesis,

e.g., ~H = hypothesis that the patient does not have cancer

D = the data, e.g., D = + test result for a cancer test

Bayes Rule (Odds Form)


P H| D

P(~ H | D)

~ H

Interpretation of Medical Test Result

P D | H P(H)

P D | ~ H P(~ H)

Posterior Odds = (Likelihood Ratio) x (Prior Odds)

6Psych 466, Miyamoto, Aut '15

Three Types of Empirical Violations of Bayes Rule

• Given statistical information about prior probabilities and likelihoods,

people's judgments of posterior probability

seriously violate Bayes Rule.

o Example: Physicians judgments of P( +Cancer | +Test Result) are much too high

• Judging probability in the context of widely-known base rates. o Tom W: Judgments ignore well-known base rates.

• Judging probability when base rates are experimentally

manipulated between subjects.

o Preview of finding: Judgments are approximately the same in conditions where the base rate is low or where it is high.

Lawyer/Engineer Problem

Previous Topic .

Next Topic .

Previous Topic .

.


Lawyer/Engineer Problem (K&T, 1973)

DESCRIPTION OF JACK: Jack is a 45-year-old man. He is married and has four children. He is generally conservative, careful, and ambitious. He shows no interest in political and social issues and spends most of his free time on his many hobbies which include home carpentry, sailing, and mathematical puzzles. (Some subjects saw other versions of this description.)

o Question with High Base Rate for Engineer (High Prior Odds): If Jack's description were drawn at random from a set of 30 descriptions of lawyers and 70 descriptions of engineers, what would be the probability that Jack is one of the engineers?

o Question with Low Base Rate for Engineer (Low Prior Odds): If Jack's description were drawn at random from a set of 70 descriptions of lawyers and 30 descriptions of engineers, what would be the probability that Jack is one of the engineers?

Bayes Rule Analysis of Lawyer/Engineer Problem

8

Bayesian Analysis of Lawyer/Engineer Problem

Psych 466, Miyamoto, Aut '15 Results for Lawyer/Engineer Problem

P(engineer | description)

P(lawyer | description)

P(description | engineer) P(engineer)

P(description | lawyer) P(lawyer)

High Base Rate Condition

30 Lawyers, 70 Engineers

Prior Odds of Engineer = 70/30

This is large

Low Base Rate Condition

70 Lawyers, 30 Engineers

Prior Odds of Engineer = 30/70

This is small


Bayesian Analysis of Lawyer/Engineer Problem

• Notice that is the same in

high base rate condition and low base rate condition.

• Therefore the difference in the posterior

probabilities in the two conditions

should be influenced only by the prior odds:

P(description | engineer)

P(description | lawyer)

P(engineer)

P(lawyer)

Results for Lawyer/Engineer Problem

P(engineer | description)

P(lawyer | description)

P(description | engineer) P(engineer)

P(description | lawyer) P(lawyer)


Lawyer/Engineer Problem - Results

X-axis: Judged P(eng | desc)

in Low Base Rate Condition

Y-axis: Judged P(eng | desc)

in High Base Rate Condition

• Each dot is the result for a different

description – some are strongly

suggestive of engineer; others are

strongly suggestive of lawyer.

Dotted Curve is the Predicted Response for a Bayesian Reasoner

P H | D

P(H | D)

P D | H P(H)

P D | H P(H)

0 20 40 60 80 100

02

04

06

08

01

00

Probability of Engineer (Low Prior)

Pro

ba

bili

ty o

f E

ng

ine

er

(Hig

h P

rio

r)




• Curve is the correct response for

Bayes Rule.

• Diagonal is the predicted response

if base rate is ignored.

• Data look more like the

base rate neglect hypothesis

than the correct (normative)

response.

Discuss 2 odd points on this graph

0 20 40 60 80 100

02

04

06

08

01

00


Pro

ba

bili

ty o

f E

ng

ine

er

(Hig

h P

rio

r)


P H | D

P(H | D)

P D | H P(H)

P D | H P(H)



Subjects were given no description about Jack. They were only told that random sampling of names from a 30:70 or 70:30 pile had occurred.

Subjects were given an uninformative description about Jack, e.g., Jack is 53 years old and is married.

Conclusions re Lawyer/Engineer Problem

Psychologically:

'No Information'

'Worthless, Irrelevant Information'0 20 40 60 80 100

02

04

06

08

01

00


Pro

ba

bili

ty o

f E

ng

ine

er

(Hig

h P

rio

r)



Conclusions re Lawyer/Engineer Problem

• Subjects were insensitive to base-rate information

(30:70 versus 70:30).

• Subjects showed awareness of base-rate when no description of Jack was

given. (So base-rate is not treated as always irrelevant.)

• Subjects ignored base-rate when irrelevant information

about Jack was given. o Psychologically: 'No Information' 'Worthless, Irrelevant Information'

Summary of Evidence for the Similarity Thesis


Representativeness Heuristic & the Similarity Hypothesis

Similarity Hypothesis: When asked to judge a probability, people often

substitute a judgment of similarity for the judgment of probability.

Lawyer/Engineer Problem: People judge P(eng | desc) based

on similarity of Jack’s description to a typical engineer.

Subjects fail to use base rate information because it is not

relevant to similarity.

Bayes Rule Says that We Should Combine Similarity Information with Distributional Information


Why Do People Fail to Use Base Rate Information?

• Bayes Rule says that we should combine base-rates with case-specific,

descriptive information (diagnosticity of the data).

• Psychologically, we don't have a reasoning strategy for combining these two

types of information, so we use one or the other.

P H | D

P(~ H | D)

P D | H P(H)

P D | ~ H P(~ H)

Objections to K&T claims – End of Base Rate Neglect Topic


Objections to the Claim – People Neglect Base Rates

• Distinguish between two claims:o People often/always neglect base rates in real world decisions.o People base probability judgments on similarity and this causes them

to neglect base rates in particular situations.

• Subjects are unmotivated – nothing is at stake in their judgmentso Replications with large monetary payoffs for being accurateo Base rate neglect in market investment decisions

• Claim: Correct use of base rates is found if uncertainty is expressed as frequencies rather than probabilities. (Gigerenzer & Hoffrage, 1995; Sloman, Over,

Slovak, & Stibel, 2003) o Evidence supports the view that expressing uncertainty as frequencies increases

people’s tendency to use base rates, but usually only to a small degree.

Inside View and Outside View

The Inside View and the Outside View

• Inside View: Look closely at the particular case. What do you see that would guide you towards a prediction?

• Outside View: Place the particular case in the context of many other similar cases. What usually happens in the population of similar cases?

Psych 466, Miyamoto, Aut '15 17Frequency Format for the Lawyer/Engineer Problem

Example: Outside View of the Lawyer/Engineer Problem


High Base Rate Condition Low Base Rate Condition

30 Lawyers 80 Engineers

Sounds like Jack

Sounds like Jack

70 Lawyers 30 Engineers

Sounds like Jack

Sounds like Jack

Outside View (Frequency Format) of the Lawyer/Engineer Problem


Lawyer/Engineer Problem – Frequency Format (Outside View)

DESCRIPTION OF JACK (SAME AS BEFORE):

o Question with High Base Rate for Engineer (High Prior Odds): Imagine a mixed pile of 1,000 personality descriptions; 700 are descriptions of engineers and 300 are descriptions of lawyers. How many engineers sound like Jack? How many lawyers sound like Jack? What is the probability that Jack is one of the engineers.

o Question with Low Base Rate for Engineer (Low Prior Odds): Imagine a mixed pile of 1,000 personality descriptions; 300 are descriptions of engineers and 700 are descriptions of lawyers. How many engineers sound like Jack? How many lawyers sound like Jack? What is the probability that Jack is one of the engineers.

The Outside View focuses on the entire distribution of possibilities. How

often will a particular outcome (he’s an engineer) occur in this distribution?

Inside View of the Lawyer/Engineer Problem


Lawyer/Engineer Problem (K&T, 1973)

DESCRIPTION OF JACK: Jack is a 45-year-old man. He is married and has four children. He is generally conservative, careful, and ambitious. He shows no interest in political and social issues and spends most of his free time on his many hobbies which include home carpentry, sailing, and mathematical puzzles. (Some subjects saw other versions of this description.)

o Question with High Base Rate for Engineer (High Prior Odds): If Jack's description were drawn at random from a set of 30 descriptions of lawyers and 70 descriptions of engineers, what would be the probability that Jack is one of the engineers?

o Question with Low Base Rate for Engineer (Low Prior Odds): If Jack's description were drawn at random from a set of 70 descriptions of lawyers and 30 descriptions of engineers, what would be the probability that Jack is one of the engineers?

Inside View vs Outside View in Planning Judgments

Inside View vs Outside View in Planning Judgments

• Inside View: ♦ Think about your plans. What do you need to get this job done?

♦ How long will each component take? Extrapolate to the entire project.

• Outside View:♦ Think about your past experience with projects that are similar in

complexity. Do people complete these projects on time?

♦ What has caused delays in the past?

♦ Is the current project likely to be different from past experiences with similar projects?

Psych 466, Miyamoto, Aut '15 21Attribute Substitution


Attribute Substitution in Probability Judgment

• Target attribute: Probability of an event,

o E.g., Will the U.S. have a large military presence in Afghanistan in 2020?

• Heuristic attribute: Easy-to-judge attribute that is related to the probability of the event.

o E.g., similarity to other political situations or availability of analogous political situations.

o A.k.a. the “proxy” attribute.

• Hypothesis that motivated Kahneman & Tversky’s research:

People substitute similarity or availability for probability. o Similarity & availability – System 1o Probability theory – System 2

When Does Attribute Substitution Occur?


When is Attribute Substitution Likely to Occur? (K&F)

K&F: Attribute substitution is more likely when:

1. the target attribute is relatively inaccessible, i.e., hard to evaluate or

unfamiliar;

2. a semantically and associatively related attribute is highly accessible

(heuristic attribute);

3. the substitution of the heuristic attribute in the judgment is not rejected by

critical operations of System 2.

Repeat Slide with Examples of Attribute Substitution


When is Attribute Substitution Likely to Occur? (K&F)

K&F: Attribute substitution is more likely when:

1. the target attribute is relatively inaccessible, i.e., hard to evaluate or

unfamiliar;

2. a semantically and associatively related attribute is highly accessible

(heuristic attribute);

3. the substitution of the heuristic attribute in the judgment is not rejected by

critical operations of System 2.

Examples of common attribute substitutions:o Habitual substitution of similarity for probability (representativeness)o Habitual substitution of availability for probability (availability)o Habitual substitution of fluency for recollective memory processeso Affective response for more cognitive evaluations like worth or virtue.

Recipe for Demonstrating Attribute Substitution: Method I


How to Demonstrate Occurrence of Attribute Substitution

Method I

• Collect judgment data for the heuristic attribute and the target attribute.

• Show that judgments of the heuristic attribute are highly correlated with

judgments of the target attribute.

• The demonstration is especially strong when there are other objective

considerations that should reduce the correlation between the heuristic

attribute and the target attribute.

Tom W problems exemplifies this strategy


Tom W Problem - Results

Correlation between judged base rate and probability rank = -.65.

Correlation between similarity rank and probability rank = +.97.

Tom W and Bayes Rule


Graph of Tom W Results

• Heuristic attribute (similarity) and target attribute (probability)

are highly correlated (+.97).

• Rule-based System 2 requires attention to base rate.

Subjects appear to disregard base rate (correlation = -.65).

How to Demonstrate Occurrence of Attribute Substitution: Method II

Similarity Rank Rank of Judged Base Rate

Ran

k o

f Ju

dge

d P

rob

abil

ity

Ran

k o

f Ju

dge

d P

rob

abil

ity

Similarity versus Judged Probability Base Rate versus Judged Probability


How to Demonstrate Occurrence of Attribute Substitution

Method II

• Experimentally manipulate the strength of the heuristic attribute.

• Show that judgments of the target attribute are influenced in the predicted

direction by variation of the heuristic attribute.

• Again, the demonstration is especially strong when there are other objective

considerations that should reduce the correlation between the heuristic

attribute and the target attribute.

Ease of Recall & Availability Exemplifies this Method

Example: Ease of Recall Affects Judgment

• Making it easy to remember examples where subject has been assertive makes subject think she is more assertive.

♦ Easier to recall 6 examples than 12 examples.

♦ Easier to recall examples when smiling than when frowning.

• Target attribute: Evaluation of how assertive you are.

• Heuristic attribute: Evaluation of how easy it is to recall the examples of being assertive.

• In this example, the experimenter manipulates the heuristic attribute (makes recall easier or harder for different subjects),and shows that the attribution (self-rating of assertiveness) increases in the predicted direction.

Psych 466, Miyamoto, Aut '15 29Summary of Attribute Substitution


Summary

• A common form of heuristic reasoning is attribute substitution:

I.e., judging a target attribute in terms of a more available heuristic attribute.

• There can be more or less direct conflicts between heuristic reasoning and

rule-governed or theory-governed reasoning.

• Heuristic reasoning is often context sensitiveo Different contexts may suggest different heuristics.o Different contexts may produce different emphases on heuristic versus rule-

governed reasoning.

• Heuristic reasoning is sometimes good and sometimes bad

(for the person doing the reasoning).

Class discussion of attribute substitution


Class Discussion

• Can we generate examples of attribute substitution

in everyday life?

• Example: o Target Attribute: Will my relationship with XXX last?o Heuristic Attribute: How well do I get along with XXX?

• Example: o Target Attribute: Will a career doing YYY be rewarding for me

over the long run?o Heuristic Attribute: How much fun do people have when doing YYY?

Exercise for the Class - END



Outline

• Representativeness heuristic had two aspects:

Similarity Thesis

Irregularity Thesis

Irregularity Thesis Misconceptions of Chance

• Examplies of misconceptions of chance

Independent, Stationary Model for Random Events


Independent, Stationary Model for Random Events

• Independent stationary model:o Independent – what happens in the past has no influence over what happens next.o Stationary – the probability of the event doesn't change over time.

Examples:

• Chance of rolling a "2" with a die is independent and stationary.

• Chance of “heads” when flipping a coin is independent and stationary.

• Chance of rain on successive days is not independent (rain or dry on day 1 is

correlated with rain or dry on day 2) and not stationary (chance of rain

varies depending on the season).

NOTE: There really are “dry streaks” and “wet streaks”.

Subjective Perception of Event Clusters


Subjective Perception of Event Clusters

• People sometimes think that airplane crashes happen in clusters (several in

close temporal proximity).

• Vaught & Dawes tested whether airplane crashes happen in clusters.

Best fitting model is independent and stationary.

• V&D's result does not deny the existence of clusters –

they deny that clusters are caused by karma, fate or other unseen forces.

• Consider example of people attempting to flip a coin mentally –

mental coin flips are too patternless.

• Gambler's fallacy (http://news.bbc.co.uk/2/hi/europe/4256595.stm,;

http://www.guardian.co.uk/italy/story/0,12576,1410701,00.html).

Fallacy: A random pattern should break up a sequential pattern.

Gambler's Fallacy in an Italian Lottery

http://news.bbc.co.uk/2/hi/europe/4256595.stm




Gambler's Fallacy in an Italian Lottery

• http://news.bbc.co.uk/2/hi/europe/4256595.stm,

http://www.guardian.co.uk/italy/story/0,12576,1410701,00.html

Italian National Lotto (weekly): o You can bet on any digit from 1 to 100. o 5 numbers are drawn at random in 10 different Italian cities.

• Between May 2003 to February 2005, the number 53 was NOT drawn in the

Venice lottery (152 consecutive lotteries).

• Pr(no "53" in 100 draws) = .0059 (about 1 in 170).

Pr(no "53" in 150 draws) = .00046 (about 5 in 10,000)

• Frenzy of betting on "53" in the Venice lotto.o Woman drowned herself; she left a note stating that she had lost

her family's saving on "53.“ A man in Signa shot his wife and son before killing himself. At least 4 deaths, and other physical violence attributed to "53" madness. Also, personal bankrupties.

Gambler's Fallacy in an Italian Lottery - Consequences


http://www.guardian.co.uk/italy/story/0,12576,1410701,00.html


Gambler's Fallacy in an Italian Lottery (cont.)

• Total of 3.5 billion Euros (about 4.2 billion dollars) was bet on "53".

• About $27.5 million was spent per month before

the February appearance of "53."

• $806 million was spent in the month of January before the February

appearance of "53." (Almost 30 times the average monthly betting on "53.")o Does this indicate a rise in "53" betting towards the end?

• The eventual winner who bet on "53" got about $768 million.

Hot Hand in Basketball


Hot Hand in Basketball (Gilovich, Vallone & Tversky)

• If players do get a "hot hand," then a player has a higher chance of making a

basket when he/she is hot than when he/she is cold. o The "hot hand" hypothesis implies that shooting baskets is not independent and

stationary

• Independent & stationary (I&S) model for a player shooting baskets: Player

has a constant p-chance of making a shot.

This chance is the same for every shot in a sequence of shots.

• The “hot hand” hypothesis and the I&S model contradict each other.

• 91% of basketball fans (Stanford & Cornell games) believed that a player

has a better chance of making a shot after making 2 or 3 than after missing 2

or 3.

Conditional Probability Results for Philadelphia 76ers


Conditional Probabilities of Making a Shot

1980-81 Philadelphia 76ers.

H = Hit = Made Shot M = Miss

P(H | 3 M) P(H | 2 M) P(H | 1 M) P(H) P(H | 1 H) P(H | 2 H) P(H | 3 H)

Clint Richardson

.50 .47 .56 .50 .49 .50 .48

Julius Erving

.52 .51 .51 .52 .53 .52 .48

Lionel Hollins

.50 .49 .46 .46 .46 .46 .32

Andrew Toney

.52 .53 .51 .46 .43 .40 .34

Runs Test for Streaks


Analysis of Runs

• A run is a succession of H or a succession of misses.o H H M H M M M H. 5 runs

• The Wald-Wolfowitz runs test is a test for stationarity (low p-value on the

test means that stationarity is violated).

• Rejection of null hypothesis (p < .01) was found for 1 of 9 players (Daryl

Dawkins).

• Analysis of shots grouped into four successive shots. Stationarity predicts

the number of groups with 0 shots made, 1 shot made, ..., 4 shots made.

Stationarity predictions are supported for all players.

• Also found no evidence for game by game variation in the the chance of

making a shot.o Remarkable: All teams have equally good defense?

Conclusions re Hot Hand


Conclusion re the Hot Hand in Basketball

• Human intuition assumes that randomness looks even more random

than real randomness.

• Therefore real randomness looks like there are non-random patterns

in the data.

• Therefore we are prone to infer causal forces where none exists.

• In the case of the "hot hand", the causal force is physical or psychological

process that changes the probability of success on the next shot.

• Website devoted to research into sports streaks (Alan Reifman):

http://thehothand.blogspot.com/

Illusion of Control



Illusion of Control

• Ellen Langer demonstrated that people perceive themselves to have control

over outcomes even when they have no control.

• Experiment varied contingency between subjects' responses and the

probability of "winning."o High contengency – subjects' strategy had a big effect on chance of "winning"o Low contingency – subjects' strategy had a small effect on chance of "winning"o No contingency - subjects' strategy had NO effect on chance of "winning"o Subjects in ALL conditions reported that their strategies affected

their chances of reward, i.e., they could do something to givethemselves a better chance of reward.

• Illusion of control – subjects in NO contingency condition still thought they

had some efficacy (some way to improve their chances).

Illusion of Control in a Simulated Computer Investments


Illusion of Control in Simulated Investments

Fenton-O'Creevy, M., Nicholson, N., Sloane, E., & Willman, P. (2003). Trading on illusions: Unrealistic

perceptions of control and trading performance. Journal of Occupational and Organizational Psychology,

76, 53-68.

• Traders from four British investment banks played a computer game.

• Traders make fictional investments which may or may not influence

the value of an investment index (measure of a stock's value).

• Computer program was rigged so that the value of the investment

index was independent from the traders' actions.

• The traders reported that they were able to influence the investment index to

varying degrees. (Reality was that they had no influence.)

• Traders with the greater illusion of control earned less on the average than

other traders in the investment game. Their managers rated as lower on rsk

management and analytical ability.

Conclusions re Misperception of Randomness


Conclusions re the Perception of Randomness

• "Hot hand" is an illusion. o Website for "hot hand" research (Alan Reifman):


• Human intuition assumes that randomness looks even more random

than real randomness.

• Therefore real randomness looks like there are non-random patterns

in the data.

• Therefore we are prone to infer causal forces where none exists.

Outline of Attribute Substitution Topic



Set Up for Instructor• Classroom Support Services (CSS),

35 Kane Hall, 206-543-9900

• CSS: Try setting your resolution to 1024 by 768

• Run Powerpoint. For most reliable start up: o Start laptop & projector before connecting them togethero If necessary, reboot the laptop

Documents

Base Rate Neglect (continued) Psychology 466: Judgment & Decision Making Instructor: John Miyamoto 10/29/2015: Lecture 05-2 Note: This Powerpoint presentation