Behavioral Economics – Decision Support

Teaching Bayesian Reasoning

Birte Gröger

Agenda

1. Bayesian Method/Inference2. Information Formats3. Teaching Methods4. Training Effectiveness5. Studies and Experiments6. Results and Conclusion

Teaching Bayesian Method in Less Than Two Hours

Bayesian Method/Inference

• Named after Thomas Bayes, published 1763

• Describing conditional probabilities (A|B) given another event (B)

• Update beliefs in light of new evidence• Transfer prior probability P(A) into

posterior probability

Bayes Rule in Theory

Bayesian Method/Inference

• Studies show: Bayesian inference is alien to human inference– Neglect or overweighing of base rates

(conservatism)– Cognitive illusions = systematic deviations

• Studies attempting to teach Bayesian reasoning with no success

The Problems

Information Formats

• Cognitive algorithms work on information information needs representation format

• Mathematical probability and percentage = recent developments

• Input format for human minds: natural frequencies

Probability vs. Natural Frequencies

Information Formats

1. Bayesian computations = simpler, when information represented in natural frequencies

2. Natural frequencies = corresponding to the information format encountered throughout most of our evolutionary development

Crucial Theoretical Results

Ten of every 1,000 women who undergo a mammography have breast cancer.Eight of every 10 women with breast cancer who undergo a mammography will test positive. Ninety-nine of every 990 women without breast cancer who undergo a mammography will test positive.

The probability that a woman who undergoes a mammography will have breast cancer is 1%.If a woman undergoing a mammography has breast cancer, the probability that she will test positive is 80%.If a woman undergoing a mammography does not have cancer, the probability that she will test positive is 10%.

Information Formats

Example Comparison – Mammography Problem

Teaching Methods

• Teaching: showing people how to construct frequency representations

• Mechanism: tutorial, practices, feedback

Overview

Rule Training Frequency Grid Frequency Tree

Teaching Methods

• Explanation how to extract numerical information by computer system

• Translation of base-rate information in components of Bayes’ formula

• Insert probabilities• Calculation of result

Rule Training

Teaching Methods

Rule Training

Teaching Methods

• Representation cases by squares• Indicate squares according to base rates– Shaded percentage of population– Circled pluses (+) for hit rate on shaded

squares– Circled pluses for false alarm rate on non-

shaded squares• Calculate ratio: pluses in shaded squares

divided by all circled pluses

Frequency Grid

Teaching Methods

Frequency Grid

Teaching Methods

• Constructing reference class and breaking-down into four subclasses

• System: explanation how to obtain frequencies

• Inserting into corresponding nodes• Calculation by dividing number of

true positive by sum of all positives

Frequency Tree

Teaching Methods

Frequency Tree

Training Effectiveness

• Explanation of program and instructions• Answer format/solution as a formula• Systematically varied order of problems• Scoring criteria

Evaluation

strict

• Match exact value• Obscure fact that

participants created sound but inexact response

liberal

• Match value +/- 5%• Increased possibility

including non-Bayesian algorithms

At baseline (w/o training

– Test 1)

Immediately after training

(Test 2)

About a week after training

(Test 3)

1 to 3 months after training

(Test 4)

Training Effectiveness

Measures

• Comparing solution rates

• Traditional: steep decay curve• Expectation now: decay not as quick with

frequency training

Studies and Experiments

Study 1a

• 62 University of Chicago students

• 4 groups in 3 training methods and one w/o training as control

• All 4 tests with 10 problems each

• Old and new problems

• High attrition rates (increasing # of participants)

Study 1b

• 56 Free University of Berlin students

• Prevent high attrition rates with later payments and bonus based on results

• 2 groups with the different frequency trainings

• Reduced number of problems

• No attrition

Study 2

• 72 University of Munich students

• Issue of used graphical aids in frequency conditions

• Longer period of time between Test 3 and 4

• Use also graphical aid for rule training probability tree

Structure

Studies and Experiments

Results – Study 1a

• Substantial improvement in Bayesian reasoning

• High level of transfers: average performance in new problems almost as god as in old problems

• Increase in median number of inferences in the frequency grid condition

Studies and Experiments

Results – Studies 1b and 2

Study 1b Study 2

Conclusion

• Prove that Bayesian computations are simpler using natural frequencies

• Environmental change illusions• Idea: teach people to represent information

according to cognitive algorithms• Translation in representation format =

major tool for helping to attain insight• High immediate effects, better transfer to

other problems and long-term stability

Teaching Bayesian Reasoning is possible