9 – Judgment

Most people do not understand chance and risk, often with dire consequences.

These errors reflect misconceptions about probability.

These misconceptions are revealed by people’s response to the following questions:

The Banker Question

Linda is 31, single, outspoken, and highly educated.

Which one of the following statements is more likely to be true?

She is a banker.

She is a banker and a democrat.

Correct Answer Banker. chance of just one chance of both

conjunction fallacy chance of both chance of just one

(Tversky & Kahneman, 1983)

Hospital Question

At Big City Hospital, about 50 babies are born daily.

At Small Town Hospital, about 10 babies are born daily.

At each hospital, someone counts the number of girls and boys born each day.

If at least 60% of babies born that day are girls, a pink sticker is placed on the calendar.

In a typical year, which hospital will have more pink stickers?

● Big City Hospital

● Small Town Hospital

● About the same

 

Modal Answer

About the same

Correct Answer

Small Town Hospital

Explanation

smaller sample less accurate survey

Example

# of births Probability of at least 60% girls

10 38%

50 10%

(Kahneman & Tversky, 1972)

Coin Question

A fair coin turns up Heads 5 times in a row.

Which is more likely to happen on the next toss?

● Heads

● Tails

● Neither – chance is 50-50

 

Correct Answer Neither. A coin has no memory!

 

Common Error Tails

gambler’s fallacy event is less likely if it just occurred

More gambler’s fallacy

When I was a kid, my best friend was 1 of 3 kids in the family.

All 3 kids were boys.

Then their mom got pregnant.

The mom said, “Since I have 3 boys, I’ll probably have a girl. ” (gambler’s fallacy)

Truth = 50%

(Though she had a girl.)

blank

Demo: Imagine rolling a fair die 25 times. Write the outcomes below (1, 2, 3, 4, 5, or 6).

___ ___

___ ___

___ ___

___ ___

___ ___

___ ___

___ ___

___ ___

___ ___

___ ___

___

___

___

___

___

Step 2 Write check by each outcome that matches previous outcome

4

4 6

3

3 3 2

On average, there should be 4 repetitions

Typical subject < 4 repeitions

Why? Gambler’s Fallacy

More Gambler’s Fallacy

When teachers write multiple-choice tests, repeats are too rare.

More gambler’s fallacy

May 2012

My insurance company phone rep tells me that I need more hurricane coverage.

I point out that hurricane hasn’t hit Florida since 2005.

She says that’s why I need more coverage – the chances are getting higher!

By her logic, Iowa is really in danger.

Note: Cost of insurance coverage has climbed every year since 2005.

Weather Facts

No hurricane has hit Tampa since they started keeping records in 1851

en.wikipedia.org/wiki/List_of_Florida_hurricanes

Data show Florida hurricanes come in clusters.

Example. 4 in 2004 and 3 in 2005 

That is, chance of hurricane is LESS if it’s been a long time since last one.

Hot Hand

Fact: In basketball, players often have streaks.

Example. A 50% shooter might hit 4 in a row or she might miss 4 in a row.

Widespread Belief: Chance of hit after a hit > Chance of hit after a miss

False! Basketball statistics Chance of hit after a hit = Chance of hit after a

miss

hot hand fallacy = success is more likely after a success than after a failure

This fallacy occurs commonly in games of skill and sometimes in games of

luck.

(Gilovich, Vallone, & Tversky, 1985)

Data from 9 players from Philadelphia 76ers

Data show that chance of hit is NOT greater after a hit

(Gilovich, Vallone & Tversky, 1985)

blank

Poetry Lover Question

Tom is a randomly selected U.S. citizen.

He is asked if he likes poetry. He answers “yes.”

Which is more likely to be his profession?

● Trucker

● Latin Professor

 

Correct Answer Trucker

 Explanation

Even if only 0.3% of truckers like poetry, they exceed 1000.

Yet fewer than 1000 people are Latin Professors.

Thus, answer depends partly on base rate (percent of population with trait X)

Wrong answer is example of base rate neglect

Disease Question

On average, 1 in 191 people has Disease X.

There is a blood test for X.

Of people who have X, 100% test positive. (True Positive rate = 100%)

Of people who don’t have X, 10% test positive. (False Positive rate = 10%)

During a routine physical, Homer tests positive for X.

Intuitively, what is the chance that Homer has X?

● less than 10%

● about 50%

● higher than 80%

Correct Answer 5%

Explanation

base rate TP

FP Chance that Homer has X

Disease X 1 in 191 100% 10%

5%

Disease Y 1 in 11 100% 10%

50%

Disease X 191 1 has X 1 TP

190 do not 19 FPs

1 / 20 = 5%

Disease Y 11 1 has X

1 TP

10 do not

1 FP

1 / 2 = 50%

Examples

blood test

base rate TP

FP

Disease Q 1 in 91 100% 10%

Disease R 21 in 91 100% 10%

Disease S 1 in 11 100% 0%

1. Quinn tests positive for Q, and Rick tests positive for R.

Which event is less likely: Quinn has Q or Rich has R

(smaller base rate)

2. Reece has R. Find the chance that he will test positive for R.

100%. True positive rate = chance that person with R will test

positive.

3. Stan tests positive for S. What is chance that Stan has S?

100%. The test is perfect.

Study

Harvard Med students and faculty given Disease Question (with slightly different data).

Results

Answer Percent of Subjects

2% (correct) 18%

95% 45%

Troubling Implication

Patient tests positive for X

Doctor tells patient his chance of having X is 95% but truth is 2%

Patient chooses risky surgery and dies.

(Casscells, Schoenberger, & Graboys, 1978, NEJM)

Base rate neglect in academia …

What is the chance that a manuscript will be accepted?

Academics rely heavily on their judgment of manuscript’s quality.

They tend to ignore the journal’s acceptance rate (i.e., base rate).

Journal acceptance rates vary dramatically.

For APA journals, acceptance rate is between 9% and 69% (2009 data)

For all psych journals, range is even greater.

blank

Conditional Probabilities

Disease question requires understanding of conditional probabilities.

conditional probability = Chance that A is true if B is true

In Disease X question,

Chance that Homer has disease X if he tested positive) = 5%

Chance that Homer will test positive if he has disease X) = 100%

Conditional probabilities are hard to understand.

(e.g., Cosmides & Tooby, 1996; Gigerenzer, 2002)

Example

In 1995, celebrity O. J. Simpson was put on trial for murdering his ex-wife.

Simpson had previously battered her.

His attorney: “Only 0.04% of male batterers later murder the battered woman.”

Translation: If Mr. X once battered Ms. Y, the chance that X will murder Y = 0.04%

Relevant data:

If Mr. X once battered Ms. Y, the chance that X is murderer if Y was murdered = 93%

Neither statistic was presented to jury.

(I. J. Good, Nature, 1995, 1996; Strogatz, New York Times, 2010)

People better understand risk if they are presented with frequency data.

Study

Doctors given two versions of the same question

Conditional Probability VersionThe probability that a woman has breast cancer is 1%.If she has breast cancer, the probability of a positive mammogram is 80%. If a woman does not have breast cancer, the probability of a positive result is 10%. Take, for example, a woman who has a positive result. What is the probability that she actually has breast cancer?

Natural Frequency Version10 out of every 1000 women have breast cancer. Of the 10 women with breast cancer, 8 will have a positive result on mammography. Of the 990 women who do not have breast cancer, 99 will still have a positive result. Take, for example, a sample of women who have positive mammograms. How many of these women actually have breast cancer?

Results

(Hoffrage & Gigerenzer, 1998)

Hungry Sailors

10 starving men in a lifeboat agree to sacrifice one man so that the others will live.

They put 9 green marbles and 1 red marble in a bag.

One at a time, each man chooses a marble.

After each marble is chosen, the marble is NOT placed back in the bag.

The first player to choose red is the loser.

Is it better to choose first or tenth, or does it not matter?

Answer: It does not matter

Hungry Sailors Version 2

Same as above, except each man replaces his marble before next man chooses.

Is it better to choose first or tenth, or does it not matter?

  Answer: Tenth

The whale ship Essex was sunk by a whale on November 20, 1820. Twenty men

escaped in three small craft and wandered the Pacific. After being unable to catch

fish, they ultimately resorted to cannibalism by common consent. Three months

later there were two boats and five men left. Rescuers found bug-eyed stick figures

hunkered over a pile of human ribs, with finger bones stashed in their pockets.

 

(Mark Shone, 2000)

A similar scenario took place in Edgar Allen Poe novel

The Narrative of Arthur Gordon Pym of Nantucket

blank

Two Girls Question

A woman moves into the neighborhood.

When you meet her, you find out that her name is Beth and that she has two kids.

Later, you see one of Beth’s kids playing in her yard, and the kid is a girl.

What is the chance that Beth’s other child is also a girl?

 

Answer 1/3

Explanation Random sample of 400 women with exactly 2 kids.

first born second born ~ N_

B B 100

B G 100

G B 100 1/3 of these have 2 girls

G G 100

The Birthday Question

A group of 41 people is randomly selected from the general population.

What is the chance that at least two of them will have the same birthday?

10% 30% 50% 70% 90%

Answer 90%

Explanation

The question is: does any person share a birthday with any other person.

41 people 820 different pairs.

(e.g., Voracek et al., 2008)

# of people # of possible matches Probability of a match 23 253 50% 32 496 75% 41 820 90% 50 1225 97%366 66795 100%

n n(n-1)/2- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - # of people P(match) = 1 – P(no match) 

2 1 – (364/365) = .003 person 2

3 1 – (364/365)(363/365) = .008 person 2 person 3

Calculation assumes 1) all birthdates are equally common (not quite true)

2) no Feb 29

Correcting for these assumptions has no discernible effect on answer.

 

 

 

 

Monty Hall Dilemma

A game show contestant is shown 3 curtains.

One curtain has a prize behind it. (Stagehands randomly choose prize location.)

The emcee knows which curtain has the prize behind it.

Player chooses a curtain.

Emcee then opens a non-chosen losing curtain.

Player is then allowed to stay with original choice or switch.

Should player switch or stay, or does it not matter?

 

Modal Response It doesn’t matter.

 

Correct Answer Switch 2/3 chance of winning

 

 

 10-Curtain Version of Monty Hall Dilemma

1 of 10 curtains has a prize behind it. (Stagehands randomly choose prize location.)

The emcee knows which curtain has the prize behind it.

Player chooses curtain.

Emcee then opens 8 non-chosen losing curtains.

Player is then allowed to stay or switch.

Should player switch or stay, or does it not matter?

Chance of winning if player switches = 9/10

The End

 

Bankers

Banker-Democrats

Correct Answer Banker. chance of just one chance of both

conjunction fallacy chance of both chance of just one

(Tversky & Kahneman, 1983)

Disease Question

On average, 1 in 91 people has Disease X.

There is a blood test for X.

Of people who have X, 100% test positive. (TP = True Positive rate = 100%)

Of people who don’t have X, 10% test positive. (FP = False Positive rate = 10%)

During a routine physical, Homer tests positive for X.

What is the chance that Homer has X?

10% 30% 50% 70% 90%

Correct Answer 10%

Modal Answer 90% base rate neglect

(continued)

Explanation

base rate TP

FP Chance that Homer has X

Disease X 1 in 91 100% 10% 10%

Disease Y 21 in 91 100% 10% 75%

Disease X 91 1 has X 1 TP

90 do not 9 FPs

1 / 10 = 10%

Disease Y 91 21 have X

21 TPs

70 do not

7 FPs

21 / 28 = 75%

Real Medical Study

Success rate for two kidney stone treatments

How can this be?

Patients were NOT randomly assigned.

Instead, doctors usually chose A for large stones, which are harder to treat.

Simpson’s Paradox trend reverses when samples combined

(Simpson, 1951)

Treatment A Treatment B

All Patients 78% (273/350) 83% (289/350)

Small Stones 93% (81/87) 87% (234/270)

Large Stones 73% (192/263) 69% (55/80)

More Simpson’s Paradox - UC Berkeley gender bias

UC Berkeley sued by because of these 1973 admission data

Then data were partitioned by department.

Applicants  % admittedMen 8442 44%Women 4321 35%

Department Men WomenApplicants  % admitted Applicants  % admitted

A 825 62% 108 82%B 560 63% 25 68%C 325 37% 593 34%

D 417 33% 375 35%E 191 28% 393 24%

F 272 6% 341 7%

Question

Scientists compute p in order to test the null hypothesis (H0).

What does p equal?

P(data given H0 is true) OR P(H0 is true given data)

correct common belief

(if true, p would be useful)

The Disease X question is formally answered by using:

Solution to Disease X question:

Let A = disease X is present and let B = positive test

P(A) = 1/91, which means that P(not A) = 90/91

P(B | A) = P(positive test given presence of X) = TP = 100% = 1

P(B | not A) = P (positive test given absence of X ) = FP = 10% = 0.10

P(X | positive test) = P(B | A) = (1)(1/91) / [(1)(1/91) + (0.1)(90/91)] = 0.1

Bayes’ Theorem

where P(A | B) = Probability of A given that B has happened

and P(B | A) = Probability of B given that A has happened

(Bayes, 1763)

Based on studies described thus far today, few people understand chance.

However…

others argue that these studies are misleading because:

The questions require algorithms, but people use “fast-and-frugal” heuristics.

The questions were contrived to exploit fallibility of heuristics.

The heuristics work well in everyday life.

(e.g., Gigerenzer, 2002)

(e.g., Gigerenzer, 2002)

Two versions of mammogram data for women over 50

Relative Risk version

If woman has mammogram, her chance of cancer death in next 10 years drops 25%

Frequency version

If 1000 women skip mammogram, 4 will die of breast cancer in next decade.

If all 1000 have mammogram, the number who die will decrease from 4 to 3.

Frequency Data are easier to understand and more informative.

(e.g., Gigerenzer & Edwards, 2003)

Hungry Sailors

Six men in a lifeboat are starving, so they agree that one must be sacrificed.

They put 5 green marbles and 1 red marble in a bag.

One at a time, each man chooses a marble.

The first player to choose red is the loser.

After each marble is chosen, the marble is NOT placed back in the bag.

Is it better to choose first or sixth, or does it not matter?

Going first has obvious disadvantage: You’re the only player who must choose!

But going first has an advantage: Only 1 in 6 is red. For others, 1 in 5, 1 in 4, …

Answer: It does not matter

Optional

P(Man A gets red) = 1/6

P(Man B gets red) = P (A doesn’t choose red AND B does) = (5/6)(1/5) = 1/6

Hungry Sailors Version 2

Six men in a lifeboat are starving, so they agree that one must be sacrificed.

They put 5 green marbles and 1 red marble in a bag.

One at a time, each man chooses a marble.

The first player to choose red is the loser.

After each marble is chosen, the marble is placed back in the bag.

Is it better to choose first or sixth, or does it not matter?

 

Answer: Sixth

Optional

P(Man A gets red) = 1/6 = 6/36

P(Man B gets red) = P(A gets green AND B gets red)= (5/6)(1/6) = 5/36