Bayesian Analysis and Social Sciences (Paladini)

BAYESIAN ANALYSIS AND SOCIAL SCIENCES. A TOOL FOR ALL TRADES

DR. STEFANIA PALADINI

Coventry University

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

* In a very simple way, all B.A (=Bayesian Analysis) reduces to a single, but crucial, question:

Do we change opinion when facts change?

And if so....

how do we modify our beliefs in the light of additional information?


Reverend Bayes was a Presbyterian minister and

an amateur mathematician,

living in England in the first part of XVIII century...

...and to be perfectly true, he is famous due to his friend Richard Price....

....who found, edited and published Bayes’ An Essay towards solving a Problem in the Doctrine of Chances in 1763


WHAT SO SPECIAL ABOUT BAYES (OR THE BAYES-PRICE FORMULA, BY OUR MODERN

ACADEMIC STANDARDS)?

.... We need to remember that in 1700s probability was not as we know it today... it what essentially related to GAMBLING (if you remember its origin, that makes sense!)


Probability was until then related to answering questions such as:

What are my chances to deal four aces in three consequent runs of poker hands?

This is something we call now “the

classical approach

to probability”


Bayes’ geniality was to pose, in a way, the OPPOSITE question:

if somebody playing with me deals three consecutive 4 aces in a row, what is the

probability is cheating (or the probability he/ she is just damn lucky)?

THIS IS CALLED THE INVERSE PROBABILITY PROBLEM AND IT CHANGED THE DISCIPLINE

FOREVER


PROBABILIT(IES) – HOW MANY OF

THEM?

(Gonick L. & W. Smith, 1993)


Now what??


In the classical/frequentist approach, to calculate probability of a complex event we use something

called conditional probability:

with


In the Bayesian approach, we use conditional probability in a slight different way:

H: is the hypothesis; E is the Evidence (your data)P(E given H) : it is the likelihood function

P(H) is the priorP(H given E) is the posterior probability, ie, the

resulting probability. Also called REVISED PROBABILIY


If you look at the formula, you will see that the posterior probability is proportional to two components:

-The likelihood the distribution of the unobserved variable given data / ie. a probability model on the data observed - the Prior: our evaluation / subjective assessment

You see where the problem lies here.....


Let’s do an example...

An initial suggestion though for people not used to probability calculations

– always better to visualise them, especially when dealing with conditional and inverse probability.

That’s why we use:Decision Trees

(I will show you both ways!)


“The Ailing Plant Problem” Before going on vacation for a week, you ask your

spacey friend to water your ailing plant. Without water, the plant has a 90 percent chance of

dying. Even with proper watering, it has a 20 percent chance

of dying. And the probability that your friend will forget to water

it is 30 percent.

(a) What’s the chance that your plant will survive the week?

(b) If your friend forgot to water it, what’s the chance it’ll be dead when you return?

(c) If it’s dead when you return, what’s the chance that your friend forgot to water it?


0.3

0.7

0.1= live

0.2=die

0.9 = die

0.8= live

0.9* 0.3 = 0.27

0.9* 0.1 = 0.03

0.7* 0.8 = 0.56

0.7* 0.2 = 0.14

0.27/ (0.27+0.14) = 0.6585

No water

water

=0.59


(a) What’s the chance that your plant will survive the week? (b) If your friend forgot to water it, what’s the chance it’ll be dead when you return? (c) If it’s dead when you return, what’s the chance that your friend forgot to water it?


This means that if you plant is dead, your friend forgot about it at the 65.8%. In real life, you should find somebody more reliable!

However, this example shows that the initial assessment, ie, the chances YOU JUDGE your friend will forget to water it (30%) is decisive in obtaining a final result

IT IS THE (IN)FAMOUS PRIOR (again)


(Titterington, 1982)

EVERYTHING IS IN THE PRIOR -CHOOSE IT WISELY AND RECURSIVELY -ADJUST IT TO THE NEW FACTS!

Posterior becomes the new Prior in

successive runs!


BAYESIAN ANALYSIS – A TOOL FOR ALL TRADES.

Fields in which Bayesian analysis is particularly welcome and widely used:

* Decision making* Quantitative Finance

* Risk assessment / evaluation * Environmental analysis

* Oil & Gas E&P .... and let’s not forget health & natural science


BAYESIAN ANALYSIS AND SECURITY

The use of Bayesian techniques (no matter the way they have been defined) in security

is long established.

A few examples:- Turing’s Machine and the Enigma code- The US Navy and the Soviet Submarine- The Rand Corporation and the assessment of

the likelihood of a nuclear war


BAYESIAN ANALYSIS AND INTELLIGENCE

CIA’s interest as well is not so recent Zlotnick’s article “Bayesian Theorem for

Intelligence analysis” is from the 1970s.... He identifies three features which distinguish

B.A from conventional intelligence analysis: * Quantify probabilistic judgements * In BA = using a set of all (alternative)

hypotheses no cognitive bias * BA : focusing on single pieces of evidence in

a systematic way instead than on a whole body this also is known to reduce bias


Which are the intelligence areas where BA looks more promising?

a few have been identified in time.

* Strategic warning : weighting odds of two competitive H. (binomial) attack/ no attack from another state

* Terrorism : the US Center for Risk and Economic Analysis of

Terrorism Events (CREATE) routinely uses BA in its evaluations

this is understandable, given the enhanced randomness of terrorism VS war


BAYESIAN ANALYSIS AND INTELLIGENCE

• Recently (in the last 10 years) a series of tools, compounding BA with other quantitative methods, have been developed – such as:

Multi-Entity Bayesian networks (MEBNs) Hidden Markov Models (HMMs) Bayesian networks (BNs)

Generally for evaluating asymmetric threats

( Kardes & Hall, 2005)


STILL....

A lot of suspect and mistrust for the intelligence application of Bayesian techniques.

Why?

For a start: Bayesian reasoning is very

counterintuitive “Bayesian math points to a fairly slow learning

curve” (Blair, 2004)


Intelligence Analysis and Biases – why do we fail?

“In making rough probability judgments, people commonly depend upon one of several simplified rules of thumb that greatly ease the burden of decision.

Using the "availability" rule, people judge the probability of an event by the ease with which they can imagine relevant instances of similar events or the number of such events that they can easily remember.

With the "anchoring" strategy, people pick some natural starting point for a first approximation and then adjust this figure based on the results of additional information or analysis. Typically, they do not adjust the initial judgment enough.

Expressions of probability, such as possible and probable, are a common source of ambiguity that make it easier for a reader to interpret a report as consistent with the reader's own preconceptions. The probability of a scenario is often miscalculated.

Data on "prior probabilities" are commonly ignored unless they illuminate causal relationships.”

(Heier, 1999)


Some studies (Gigerenze & Hoffrage, 1995) have found out that human brain has difficulties figuring probabilities:

- 1 out of 100 seems better than 1% (frequency instead of probability)

- Even better - NATURAL FREQUENCES (ie, one in which the information about the prior

probability is included in presenting the conditional probabilities; 80 items out of 100 has the character in exam)

This has been already discovered in intelligence studies applying Bayesian analysis


... On another hand – ie, from the quantitative side of the spectrum - there are the century-old suspicions about the PRIOR!

How can we really trust a hunch??

In a way, it seems not to different than gambling on a hypothesis (again)

This is the typical criticism by the frequentist approach


And finally....there are some common problems which affect ALL kind of intelligence analysis, and not just B.A. such as....

* misjudging evidence * confounding cause and effect * nonstationarity (short life span of the

evidence collected) * non independences of the H evaluated

BA is not a catch-all remedy – but it is a powerful tool is used in the appropriate way!


(1) General References

(D.M.Titterington, 1982, Irreverent Bayes, BIAS 9 (1)-16-18) Gonick Ll. & W. Smith, 1993, The Cartoon Guide to Statistics, New

York, NY : Collins A gentle introduction to Bayes: http://yudkowsky.net/rational/bayes Gigerenzer G.& U. Hoffrage 1995 How to Improve Bayesian

Reasoning Without Instruction: Frequency Formats, Psychological Review, 102(4)684–704

A nice (and useful) Bayesian applet: http://psych.fullerton.edu/mbirnbaum/bayes/BayesCalc.htm

Iversen, Gudmund R. Bayesian Statistical Inference. Beverly Hills: Sage Publications, Inc., 1984.

Steven Strogatz, “Chances are” http://opinionator.blogs.nytimes.com/2010/04/25/chances-are/

http://psych.fullerton.edu/mbirnbaum/bayes/BayesCalc.htm


(2) References : Bayes & Intelligence

Zlotnick. J.(1970) Bayes' Theorem for Intelligence Analysis, available at https://www.cia.gov/library/center-for-the-study-of-intelligence/kent-csi/vol16no2/html/v16i2a03p_0001.htm

Elisabeth Paté-Cornell (2001), Project: "Local risk assessment in a crisis risk management context: a tactical application of the Bayesian approach to intelligence analysis in a dynamic situation“ available at http://create.usc.edu/2011/03/bayesian_approach_to_intellige.html

Pate-Cornell, M. E.(2002): "Fusion of Intelligence Information: A Bayesian Approach", Risk Analysis, Vol. 22, No. 3, 2002. Pp. 445-454.

Hong Y. and G. Apostolakis (1992): "Conditional Influence Diagrams in Risk Management", Risk Analysis, Vol. 13, No.6, Pp. 625-636.

Kardes, E., and Hall, R (2005): "Survey of Literature on Strategic Decision Making in the Presence of Adversaries", Report, National Center for Risk and Economic Analysis of Terrorism Events, University of Southern California, Los Angeles, CA.

Hausken, K. (2002). “Probabilistic risk analysis and game theory”, Risk Analysis, Vol.22.

Heuer, R. J. (1999) Psychology of Intelligence Analysis, CIA Blair, Bruce. “The Logic of Intelligence Failure.” Center for Defense Information,

March 9, 2004, http://www.cdi.org/blair/logic.cfm Fisk, Charles E. “The Sino-Soviet Border Dispute: A Comparison of the Conventional

and Bayesian Methods for Intelligence Warning.” Studies in Intelligence 16, no. 2 (1972): 52-62

https://www.cia.gov/library/center-for-the-study-of-intelligence/kent-csi/vol16no2/html/v16i2a03p_0001.htm

https://www.cia.gov/library/center-for-the-study-of-intelligence/kent-csi/vol16no2/html/v16i2a03p_0001.htm

http://create.usc.edu/2011/03/bayesian_approach_to_intellige.html

http://www.cdi.org/blair/logic.cfm


Sharon Bertsch McGrayne

THE THEORY THAT WOULD NOT DIEHow Bayes’ Rule Cracked the Enigma Code, Hunted Down Russian Submarines and Emerged Triumphant From Two Centuries of Controversy

2011, Yale University Press.

Documents

Bayesian Analysis and Social Sciences (Paladini)