A View from the Bottom

Peter DayanGatsby Computational Neuroscience Unit

Neural Decision Making

• bewilderingly vast topic • models playing a central role

– so beware of self-confirmation + battles

3

• Ethology/Economics(?)– optimality– logic of the approach

• Psychology– economic choices – instrumental/Pavlovian conditioning

• Computation

• Algorithm

• Implementation/Neurobiologyneuromodulators; amygdala; prefrontal cortex

nucleus accumbens; dorsal striatum

prediction: of important eventscontrol: in the light of those predictions

Neural Decision Making

Imprecision & Noise

• computation– Bayesian sensory inference– Kalman filtering and optimal learning– metacognition

– exploration/exploitation

– game theory

Imprecision & Noise

• algorithm– multiple methods of choice

• instrumental: model-based; model-free– (note influence on RTs)

• Pavlovian: evolutionary programming

– uncertainty-modulated inference and learning

– DFT/drift diffusion decision-making

– MCMC methods for inference

Imprecision & Noise

• implementation– (where does the noise come from?)

– evidence accumulation– Q-learning and dopamine– metacognition and the PFC– acetylcholine/norepinephrine and uncertainty-

sensitive inference and learning

Diffusion to Bound

Britten et al, 1992

Diffusion to Bound• expected reward, priors affect

starting point• some evidence for urgency

signal• works for discrete evidence

(WPT)• less data on >2 options• micro-stimulation works as

expected• decision via striatum/superior

colliculus/etc?• choice probability for single

neuronsGold & Shadlen, 2007

9

dopamine and prediction error

no prediction prediction, reward prediction, no reward

TD error

Vt

R

RL

tttt VVr 1

)(t

Probability and Magnitude

Tobler et al, 2005

Fior

illo

et a

l, 20

03

Risk Processing

< 1 sec

0.5 sec

You won40 cents

5 secISI

19 subjects (dropped 3 non learners, N=16)3T scanner, TR=2sec, interleaved234 trials: 130 choice, 104 single stimulusrandomly ordered and counterbalanced

2-5secITI

5 stimuli:

40¢20¢

0/40¢0¢0¢

Neural results: Prediction errors

what would a prediction error look like (in BOLD)?

Neural results I: Prediction errors in NAC

unbiased anatomical ROI in nucleus accumbens (marked per

subject*)

* thanks to Laura deSouza

raw BOLD(avg over all

subjects)

Value Independent of Choice CauuvrECQ tttt ,1|)(),1( 1

**

),1(),2(max),1(),1( CQaQrCQCQ at

Roesch et al, 2007

Metacognition

• Fleming et al, 2010

• contrast staircase for performance; type II ROC for confidence

Structural Correlate

• also associated white matter (connections)

Discussion

• what can economics do for us?– theoretical, experimental ideas– experimental methods– like behaviorism…

• what can we do for economics?– large range of constraints– objects of experimental inquiry precisely aligned

with economic notions– grounding/excuse for complexity…