Putting Decision Science Into the Art of Intelligence Analysis (Dhami)

7/31/2019 Putting Decision Science Into the Art of Intelligence Analysis (Dhami)

1/16

Mandeep K. Dhami, PhDUniversity of Surrey


2/16

Probability, risk, choice, decision

What is the levelof threat posedby Anonymous?

Which al-Qaedaoperatives should beprioritized for furtherintelligence gathering?

What is theprobability that Iranwill have nuclearweapons in 2012?


3/16


4/16

Psychology of Intelligence Analysis

Heuer (1999) focused on

(1) basic cognitive processes (perception, memory);

(2) analytic judgment, incomplete information, critical and creativethinking, structuring analytic problems, competing hypotheses;

(3) biases in evaluating evidence, judging causality, hindsight.

Some useful insights, BUT outdated and limited.

Following Heuer, others have suggested ways to improve analyticthinking and writing (e.g., Cooper, 2005; Sinclair, 2010).


5/16

UNFORTUNATELY, there is little empirical work on intelligenceanalysis. Past work:

does not examine all phases of the intelligence cycle (i.e.,direction, collection, processing and dissemination).

does not seriously consider the analyst mind-task interaction.

has not sufficiently distinguished between normative,descriptive, and prescriptive questions.

has not explored the relative value of correspondence v.

coherence approaches to intelligence analysis.

Potential for behavioural and social sciences in understanding andimproving intelligence analysis (e.g., Campbell & Mandel, 2010;Fischhoff & Chauvin, 2011a, 2011b; Pool, 2010).


6/16

Decision Science

Science of judgment anddecision making (JDM) hasexisted for over 60 years.

Inter-disciplinary.

JDM relies on basiccognitive processes. E.g., attention,

perception, memory,knowledge representation,language, informationprocessing


7/16

JDM is a dynamic skill that

has neural bases,

has evolved,

develops through the

life-span, and can be improved (see

Dhami et al., 2011).


8/16

Overall, JDM research focuses on:

1. Normative Qs: How shouldpeople make decisions? Internally coherent v. externally correspondent.

2. Descriptive Qs: How do people actuallymake decisions?

Information, weighting, integration, effectiveness(accuracy, errors), consistency, agreement, confidence,insight.

3. Prescriptive Qs: How can we improvepeoples decisionmaking?

Training, guidance, feedback, decision aid, improve taskstructure/reduce constraints.


9/16

Two dominant approaches (see Hammond, 1990; Hastie, 2001):

Coherence

Competence as

internal consistency;Performance assessedagainst normativebenchmarks e.g., SEU,probability theory

Heuristics & Biases

Analytic thinking

Decision analysis andtools

Correspondence

Competence asempirical accuracy;

Performance assessedagainst externalcriterion

Heuristics &Achievement

Intuitive and quasi-rational thinking

Cognitive feedbackand decision aids

Kahneman &Tversky

Hammond,

Klein,Gigerenzer

experienced/professional

samples,real/

simulatedtasks

studentsamples,artificialtasks in lab


10/16

Main research findings

Use simple, fast & frugal heuristics that ignore the vast majority ofavailable and relevant information.

But, sometimes this leads to good performance!

Internallyinconsistent, and disagree with those of others.

Affected byunwanted influences e.g., information framing, emotion.

Lack insight into their cognitive strategies.

Over confident in their JDM abilities, given their level ofperformance.


11/16

General explanations are that JDM affected by:

Limitations ofhuman mind, e.g., limited memory,attention, processing capacity, etc.

Taskconstraints e.g., time pressure, limited information orinformation overload, complex information

representations, etc.


12/16

Realisation that cognitive processes are adapted to task (seeDhami, Hertwig, & Hoffrage, 2004).

Task properties induce intuitive and analytic/deliberativethinking (Hammond, 1996, 2001).

People mostly use quasi-rationality(mix of both).

Analytic/deliberative thinking does not always outperformintuition.


13/16

THUS, research needs to understand task and human (Dhami &Thompson, 2012).

Performance can be improved by

increasing match between task properties and mode ofcognition;

and byovercoming human limitations and constraints oftask.


14/16

Evidence-based policy and

practice for intelligence analysis

How should thisanalytic toolrepresentinformation?

Does collaboration

improve intelligenceanalysis?

How do I bestcommunicate myuncertainty?

Does structuringanalytic work f lowsimprove intelligence

analysis?


15/16


16/16

Any Questions?

Documents

Putting Decision Science Into the Art of Intelligence Analysis (Dhami)