Embed Size (px)
Citation preview
Neurophysiology and Neuropharmacology of
Decision Making
by
Arwen Long
Department of NeurobiologyDuke University
Date:
Approved:
Michael Platt, Advisor
David Fitzpatrick, Chair
Scott Huettel
William Wetsel
Dissertation submitted in partial fulfillment of the requirements for the degree ofDoctor of Philosophy in the Department of Neurobiology
in the Graduate School of Duke University2009
Abstract(Neuroscience (0317))
Neurophysiology and Neuropharmacology of Decision Making
by
Arwen Long
Department of NeurobiologyDuke University
Date:
Approved:
Michael Platt, Advisor
David Fitzpatrick, Chair
Scott Huettel
William Wetsel
An abstract of a dissertation submitted in partial fulfillment of the requirements forthe degree of Doctor of Philosophy in the Department of Neurobiology
in the Graduate School of Duke University2009
Copyright c© 2009 by Arwen LongAll rights reserved except the rights granted by the
Creative Commons Attribution-Noncommercial Licence
Abstract
Negotiating the complex decisions that we encounter daily requires coordinated neu-
ronal activity. The enormous variety of decisions we make, the intrinsic complexity
of the situations we encounter, and the extraordinary flexibility of our behaviors
suggest the existence of intricate neural mechanisms for negotiating contexts and
making choices. Further evidence for this prediction comes from the behavioral al-
terations observed in illness and after injury. Both clinical and scientific evidence
suggest that decision signals are carried by electrical neuronal activity and influenced
by neuromodulatory chemicals. This dissertation addresses the function of two puta-
tive contributors to decision-making: neuronal activity in posterior cingulate cortex
and modulatory effects of serotonin. I found that posterior cingulate neurons respond
phasically to salient events (informative cues; intentional saccades; and reward deliv-
ery) across multiple contexts. In addition, these neurons signal heuristically guided
choices across contexts in a gambling task. These observations suggest that posterior
cingulate neurons contribute to the detection and integration of salient information
necessary to transform event detection to expressed decisions. I also found that
lowering levels of the neuromodulator serotonin increased the probability of making
risky decisions in both monkeys and mice, suggesting that this neurotransmitter con-
tributes to preference formation across species. These results suggest that posterior
cingulate cortex and serotonin each contribute to decision formation. In addition, the
unique serotonergic projections to posterior cingulate cortex, as well as the frequent
iv
implication of altered serotonergic and posterior cingulate function in psychiatric dis-
orders, suggest that the confluence of cingulate and serotonergic activity may offer
key insights into normal and pathological mechanisms of decision making.
v
To
Aunt Ona
Aunt Clarice
Auntie Anna
Aunt Helen
and
to my grandmothers:
Dorothy, Genevieve and Kitty
vi
Contents
Abstract iv
List of Tables xii
List of Figures xiii
List of Abbreviations and Symbols xv
Acknowledgements xviii
1 Introduction 1
1.1 Decisions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.1 Theories of rational choice . . . . . . . . . . . . . . . . . . . . 3
1.1.2 Heuristic approaches . . . . . . . . . . . . . . . . . . . . . . . 7
1.1.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.2 Salience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.2.1 Defining salience . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.2.2 Integrating features: defining salience maps . . . . . . . . . . 13
1.2.3 Salience map example: frontal eye fields . . . . . . . . . . . . 15
1.2.4 Summarizing salience and salience maps . . . . . . . . . . . . 17
1.3 Neural contributions to decisions . . . . . . . . . . . . . . . . . . . . 18
1.3.1 The process of decision: From perception to outcome . . . . . 18
1.3.2 Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.3.3 Action evaluation and selection . . . . . . . . . . . . . . . . . 21
vii
1.3.4 Outcome monitoring (and prediction) . . . . . . . . . . . . . . 24
1.3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
1.4 Posterior cingulate cortex . . . . . . . . . . . . . . . . . . . . . . . . 30
1.4.1 CGp dysfunction in disease . . . . . . . . . . . . . . . . . . . 30
1.4.2 CGp anatomy . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
1.4.3 Functional characteristics of CGp activity . . . . . . . . . . . 32
1.4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
1.5 Serotonin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
1.5.1 Serotonin is implicated in psychiatric disorders . . . . . . . . . 37
1.5.2 Serotonin is implicated in risky decision making . . . . . . . . 39
1.5.3 Summary and experimental direction . . . . . . . . . . . . . . 40
1.6 Experimental rationale and specific aims . . . . . . . . . . . . . . . . 40
2 Neurons in posterior cingulate cortex respond phasically to infor-mative events 44
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
2.4 Materials and methods . . . . . . . . . . . . . . . . . . . . . . . . . . 53
2.4.1 Surgical and behavioral procedures . . . . . . . . . . . . . . . 53
2.4.2 Behavioral paradigms . . . . . . . . . . . . . . . . . . . . . . . 54
2.4.3 Task contexts . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
2.5 Figures and Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3 Decision heuristic signals in posterior cingulate cortex 67
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.2 Materials and methods . . . . . . . . . . . . . . . . . . . . . . . . . . 70
viii
3.2.1 Surgical and behavioral procedures. . . . . . . . . . . . . . . . 70
3.2.2 Behavioral tasks . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.2.3 Microelectrode recording techniques. . . . . . . . . . . . . . . 71
3.2.4 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
3.3.1 Monkeys use a threshold-comparative decision heuristic . . . . 72
3.3.2 Neuronal activity reflects contextual choice bias . . . . . . . . 74
3.3.3 CGp activity reflects heuristically defined outcomes . . . . . . 76
3.3.4 CGp activity distinguishes outcome/decision combinations . . 77
3.3.5 CGp activity predicts the probability of repeating a choice . . 78
3.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
3.5 Figures and Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4 Serotonin shapes risky decision making in monkeys 94
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
4.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
4.2.1 Surgical and training procedures . . . . . . . . . . . . . . . . . 98
4.2.2 Behavioral paradigms . . . . . . . . . . . . . . . . . . . . . . . 98
4.2.3 Rapid Tryptophan Depletion . . . . . . . . . . . . . . . . . . . 99
4.2.4 Experimental schedule . . . . . . . . . . . . . . . . . . . . . . 100
4.2.5 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
4.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
4.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
4.5 Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
5 Low serotonin increases risky decisions in mice 121
5.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
ix
5.2 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 122
5.3 Experimental procedures . . . . . . . . . . . . . . . . . . . . . . . . . 125
5.3.1 Behavioral procedures . . . . . . . . . . . . . . . . . . . . . . 125
5.3.2 Serotonin depletion . . . . . . . . . . . . . . . . . . . . . . . . 126
5.3.3 Analysis of mouse data . . . . . . . . . . . . . . . . . . . . . . 127
5.4 Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
6 Conclusions 134
6.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
6.2 CGp, decisions and salience . . . . . . . . . . . . . . . . . . . . . . . 135
6.2.1 Summary of results . . . . . . . . . . . . . . . . . . . . . . . . 135
6.2.2 The breadth of relevant information: from novelty to memory 136
6.2.3 CGp as a putative salience map . . . . . . . . . . . . . . . . . 138
6.2.4 On the biphasic directionality of CGp responses . . . . . . . . 141
6.3 Serotonin, decisions and risk . . . . . . . . . . . . . . . . . . . . . . . 142
6.3.1 Summary of results . . . . . . . . . . . . . . . . . . . . . . . . 142
6.3.2 Caveats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
6.4 Future directions: Serotonergic contributions to CGp function . . . . 146
6.4.1 5HT1A receptors in CGp . . . . . . . . . . . . . . . . . . . . . 146
6.4.2 5HT2A receptors in CGp . . . . . . . . . . . . . . . . . . . . . 148
6.5 Future directions: other decision regions with serotonergic input . . . 149
6.5.1 Prefrontal cortex . . . . . . . . . . . . . . . . . . . . . . . . . 150
6.5.2 Anterior cingulate cortex . . . . . . . . . . . . . . . . . . . . . 150
6.5.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
A Supplementary Information for Rapid Tryptophan Depletion 152
A.1 RTD mix composition . . . . . . . . . . . . . . . . . . . . . . . . . . 152
x
A.2 Plasma tyrosine measurements . . . . . . . . . . . . . . . . . . . . . . 153
A.3 Mix administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
A.4 Safety premiums . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
A.5 Effect of reward and choice histories . . . . . . . . . . . . . . . . . . . 155
A.6 Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
B Supplemental Information: Low serotonin increases risky decisionsin mice 161
Bibliography 168
Biography 196
xi
List of Tables
2.1 Neuronal responses to events across contexts. . . . . . . . . . . . . . . 66
3.1 Monkeys’ decisions in the gambling task are motivated by integrationof short reward and choice histories. . . . . . . . . . . . . . . . . . . . 90
3.2 CGp neuronal activity is modulated by contextual biases. . . . . . . 91
3.3 CGp neurons track previous wins and losses across contexts. . . . . . 92
3.4 CGp neuronal activity predicts the probability of repeating a choiceacross contexts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
xii
List of Figures
2.1 Choice task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
2.2 Operant (saccade) task . . . . . . . . . . . . . . . . . . . . . . . . . 59
2.3 Pavlovian (fixation) task . . . . . . . . . . . . . . . . . . . . . . . . . 60
2.4 Monkeys distinguish large from small rewards. . . . . . . . . . . . . 61
2.5 CGp neurons respond to informationally relevant events in the choicetask. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
2.6 Neurons respond to relevant events in the operant task . . . . . . . . 63
2.7 CGp neurons respond to visual stimuli and reward in the Pavloviantask . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
2.8 CGp neurons report uncued rewards. . . . . . . . . . . . . . . . . . . 65
3.1 Monkeys use a win-stay/lose-shift heuristic to make decisions acrosscontexts in a simple gambling task. . . . . . . . . . . . . . . . . . . . 83
3.2 CGp neurons signal choice within context. . . . . . . . . . . . . . . . 86
3.3 CGp neurons signal heuristically-assessed outcomes across contexts. 87
3.4 CGp neurons signal outcome/decision combinations in the decisionepochs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
3.5 CGp neurons signal the probability of repeating choices across contexts. 89
4.1 Gambling task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
4.2 RTD lowers plasma tryptophan in three monkeys . . . . . . . . . . . 115
4.3 Serotonin depletion systematically decreases preference for the safeoption in monkeys. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
4.3 ...continued . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
xiii
4.3 ...continued . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
4.4 Serotonin depletion increases the subjective value of the risky optionin three macaques. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
4.5 Tryptophan depletion increases risky choices following sub-maximalrewards. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
5.1 Mouse gambling task . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
5.2 PCPA lowers brain serotonin . . . . . . . . . . . . . . . . . . . . . . . 129
5.3 Lowering brain serotonin increases the probability of a risky choice. . 130
5.4 Lowering brain serotonin increases the probability of a risky choice ina subset of mice. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
5.5 Example vehicle mouse learning curve. . . . . . . . . . . . . . . . . . 132
5.6 Example PCPA mouse learning curve. . . . . . . . . . . . . . . . . . 133
6.1 Neural components of evaluative action selection. . . . . . . . . . . . 137
6.2 A simple salience map. . . . . . . . . . . . . . . . . . . . . . . . . . . 139
A.1 Plasma tryptophan levels diminish following RTD but not after a bal-anced amino acid mix. . . . . . . . . . . . . . . . . . . . . . . . . . . 157
A.2 RTD, unlike a balanced amino acid mix, shifts choice frequencies rel-ative to baseline. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
A.3 The plasma tryptophan/tyrosine ratio, like plasma tryptophan, con-firms successful tryptophan depletion. . . . . . . . . . . . . . . . . . . 159
A.4 Monkeys’ ability to distinguish reward sizes is not affected by RTD. . 160
B.1 Fractional dopamine is unaffected by PCPA treatment. . . . . . . . . 162
B.2 Fractional dopamine levels do not depend on changes in brain serotonin.163
B.3 Vehicle and PCPA mice have indistinguishable initial preferences. . . 164
B.4 Vehicle and PCPA risk preferences differ despite similar early experi-ences. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
B.5 Shifted risk preferences reflect global changes in choice probability. . . 166
B.6 Para-chlorophenylalanine-treated mice nosepoke more frequently thansham controls. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
xiv
List of Abbreviations and Symbols
Abbreviations
5-HT serotonin
5HTT serotonin transporter
ACC anterior cingulate cortex
AD Alzheimer’s disease
ADHD attention-deficit and hyperactivity disorder
AIC Aikake’s Information Criterion
ANOVA Analysis of Variance
APOE apolipoproteinE
BLA basolateral amygdala
BR the monkey Broome
C57BL/6J C57BL/6J (wild-type)
CGp posterior cingulate cortex
CNS central nervous system
CV coefficient of variance
xv
DA dopamine
DLPFC dorsolateral prefrontal cortex
DRN dorsal raphe nuclei
EV expected value
FEF frontal eye fields
GM General Mills
HPLC high performance liquid chromatography
IT inferior temporal region
LED light-emitting diode
LIP lateral intraparietal cortex
LNAA large neutral amino acid
MT area MT
NI the monkey Niko
OCD obsessive-compulsive disorder
OFC orbitofrontal cortex
PCPA para-chlorophenylalanine
PFC prefrontal cortex
PSE point of subjective equivalence
PTSD post-traumatic stress disorder
xvi
PSTH peri-stimulus time histogram
RPE reward prediction error
RTD rapid tryptophan depletion
SH the monkey Sherry
SSRI selective serotonin reuptake inhibitors
TPH tryptophan hydroxylase
VTA ventral tegmental area
WSLS Win-Stay-Lose-Shift
WT wild-type
xvii
Acknowledgements
It has been a privilege to work with my advisor, Michael Platt. His expansively
creative research ideals and incomparable ability to spin a story continue to amaze
and inspire me. My committee — David Fitzpatrick, Scott Huettel, Bill Wetsel and
(formerly) Jill Stowe — have put up with some utterly dreadful presentations and
have encouraged and challenged me. I thank them for their assistance on my voyage
into the unknown. My studies of serotonergic function have been highly collabora-
tive ventures, and I thank Cynthia Kuhn, Ramona Rodriguiz and Bill Wetsel for
their assistance, encouragement, and contributions to my intellectual and profes-
sional development. The HPLC measurements described in these experiments were
performed by Maria Bartolome (macaque plasma tryptophan), Masato Fukui and
Dipendra Aryal (mouse brain serotonin). Ramona Rodriguiz, O’Dhaniel Mullette-
Gillman and John Pearson have mentored and advised me, and I have appreciated
their insight, encouragement and example. Melissa Furlong, Nandish Shah, James
Zhang, Betty Jiang, and Sarah Boltuck helped me develop and implement the mouse
experiments described in this thesis. It was a pleasure to work with them and I wish
them well. Several colleagues volunteered helpful comments on portions of this the-
sis — Rebecca Ebitz, Jeff Klein, Victoria Long, O’Dhaniel Mullette-Gillman, John
Pearson and Stephen Shepherd— and Bill Wetsel and Stephen Shepherd offered
strategic advice that helped me formulate and write this work. Justin Long helped
me make the figures used in the Discussion. As a Duke MSTP student, I owe partic-
xviii
ular homage to Sal Pizzo, Pat Burks, and Marjorie Miller for their commitment to
education, to excellence, and to us as students. My medical deans, Mark Sebastian
and Phil Goodman, have advised, encouraged and supported me. Bob Drucker has
often hinted that, if all else fails, I could definitely get into a pediatrics residency.
I really don’t know how to thank him, but it’s been good to know that there is a
child-sized parachute nearby. Finally, I would like to thank my MSTP colleagues for
their camaraderie and commiseration, my mother for teaching me how to write, the
many friends who fed and encouraged me during the writing of this document, the
wee free Nac MacFeegles, and my family (including Squiggles, Peanut and Wasabi)
for their love and general awesomeness.
xix
1
Introduction
Every day, humans encounter myriads of seemingly trivial decisions. Surprisingly,
even though our behavior depends on highly adapted neurochemical processes, we
often find simple decisions difficult. Choosing the morning’s breakfast food and
selecting the day’s professional attire may challenge us within the first hour after
awakening. The more complex decisions that we consider — how to invest money,
where to live, what career to pursue — reveal even more competing factors that
influence our choices.
Even those decisions that seem to require only discrimination between better
and worse options, or between reward and punishment, depend on multi-factorial
evaluative processes. Valuation depends not only on quantifiable properties like size,
weight, nutritional content, and dollar value, but also on the decision-maker’s current
physiological and emotional state, past experience, socially acquired information,
and idiosyncratic biases. Thus, although today’s calculated breakfast preference
should depend rationally on the relative prices of the two options, experientially on
yesterday’s choice, and energetically on the nutritional content of the two items, our
decision may ultimately depend on the marketing strategies deployed by Kellog’s
1
and General Mills (GM). Similarly, investment choices that ought to follow rational
calculations based on value and probability often reflect seemingly irrational and
highly inconsistent biases toward (or against) perceived risk.
Decisions become even more challenging if, as in many psychiatric disorders, the
neural systems that support decision-making are abnormal. Where we debate Eg-
gos versus cereal, a depressed patient may face anhedonic disinterest in breakfast;
a schizophrenic patient may choose to avoid food for fear of poison; and an acutely
manic patient may be so overwhelmed with grandiose plans that breakfast is irrele-
vant. Thus, counter-adaptive changes to the neuromodulatory systems that mediate
decision making result in severe functional impairments.
Similarly, other neurological insults such as stroke and trauma cause distinctive
behavioral alternations. The associations between pathological lesions and particular
behavioral abnormalities are important clues to understanding the anatomical and
chemical contributions to decision processes. Even with the help of such hints as
the astonishing transformation of Phineas Gage and the forgetful geniality of H.M.,
however, comprehensively mapping neural decision processes has been challenging.
Evaluative decisions appear to depend on recursive processing in the regions known
as “association cortex”, which link sensory inputs to motor outputs. Unlike primary
and secondary sensorimotor cortices, which were functionally characterized based on
the observed movements and described sensations resulting from local stimulation,
association cortices are not easily linked to distinct inputs and outputs. Thus, assign-
ing particular decision-related functions to these brain structures has required careful
experimental design. In this Introduction, I will discuss our current understanding of
decisions from both behavioral and biological perspectives, with a particular focus on
probabilistic components of decisions, before considering two particularly intriguing
contributors to decision-making, posterior cingulate cortex and the neuromodula-
tor serotonin. These ruminations will conclude in a summary of the experimental
2
rationale behind the specific aims of this thesis.
1.1 Decisions
1.1.1 Theories of rational choice
Over the last several centuries, economists and decision researchers have debated
whether decisions depend upon rational calculation or heuristic estimations. Ideal-
istic economic conceptions of human rationality are often traced back to Bernoulli’s
famous solution for the St. Petersburg paradox (Bernoulli, 1738/1954). When asked
to explain why people would only pay small fees to play a simple gamble with infi-
nite expected return, Bernoulli suggested that such decisions depend on a subjective
analysis of the gamble’s expected value and the participant’s current state. This
rational decision framework, one of the earliest expected utility theories, admits that
decisions are subjective and context-dependent but suggests that they depend on
extensive (specifically, logarithmic) internal calculations of value.
Enlightenment economists and philosophers continued to develop the idea that
decisions depended on personal, subjective utility through the 19th century. Adam
Smith argued repeatedly, and across domains, that economic interactions depended
on self-interested valuation of available options: whether choosing an occupation
or acting cooperatively, “It is his [the individual’s] own advantage... which he has
in view” (Smith, 1776/1976)1. Notably, although Bernoulli’s aim was to develop
“rules... whereby anyone could estimate his prospects from any risky undertaking in
light of one’s specific financial circumstances” (Bernstein, 1996) and Smith’s inter-
ests were primarily economic, neither entirely excluded the possibility that decision-
making agents might use other factors in calculating subjective utilities. The English
philosopher Jeremy Bentham defined utility based on personal pleasure (Bernstein,
1996), but the focus of individual utility shifted completely toward monetary value
1 See, for example, I.ii and IV.ii
3
when John Stuart Mill introduced an imaginary rational agent whose primary con-
cern was the ”[desire] to possess wealth” (Mill, 1844/2000). Although Mill’s “Eco-
nomic Man” was admittedly “speculative”, the conceptual simplicity and relative
tractability of analyzing an exclusively economic decision has made this fictitious
figure useful to behavioral scientists and economic thinkers.
The mathematician John von Neumann and the economist Oskar Morgenstern
rigorously axiomatized the definition of a rational agent, now dubbed “homo eco-
nomicus” (Neumann & Morgenstern, 1944; Neumann, 1928/1959). Like Bernoulli,
Smith and Mill, von Neumann and Morgenstern assumed that individuals attempt to
maximize an evaluative, subjectively determined utility. Their expected utility the-
ory strongly resembled Bernoulli’s — in both theories, for example, an agent choosing
between two equal-valued options should be indifferent — and although the implica-
tions of the two theories are not identical, both definitions have influenced modern
research. Together, these descriptions of utility form the basis of current economic
utility theories.
Risk in rational choice theories
Both Bernoulli’s expected utility and the von Neumann/Morgenstern formulation
estimate utility based on potential outcomes; importantly, both also depend on
known outcome probabilities, known colloquially as uncertainty or expressed eco-
nomically as risk. Notably, risk specifically implies known probabilities; uncertainty
due to unknown probabilities is defined as ambiguity (Ellsberg, 1961; Knight, 1921).
Thus, any outcome with a known probability attached to it is economically risky,
while outcomes with unknown probabilities are ambiguous. Risk can be defined as
the simple probability of reward (Bernoulli, 1738/1954), reformulated as a decision
weight (Kahneman & Tversky, 1979), or calculated mathematically as the coefficient
of variance (CV) of reward (E. Weber, Shafir, & Blais, 2004).
4
Empirical challenges to expected utility theories
Risk and ambiguity induce strong behavioral biases in people and animals that are
difficult to explain on the basis of rational probability estimations or utility calcu-
lations. Committed gamblers will insist on buying car insurance beyond the legally
required minimum (Kahneman & Tversky, 1979; Lee, McGreevy, & Barraclough,
2005). Thus, even though people are risk-seeking in one context (e.g., gambling),
they avoid risk in other contexts (e.g., insuring a car), suggesting that they may
not have consistent, rational reactions toward risk. The observation that betting at
horse races shifts away from the most favored (and most likely to win) horses near
the end of the day, even though the probability of a win increases toward the last
race, provides additional evidence that responses to probabilistic bets do not depend
on rationally calculated expected utilities (McGlothlin, 1956).
Several famous examples of irrational responses to uncertainty, such as the Al-
lais and Ellsberg paradoxes (Allais, 1953; Ellsberg, 1961), have challenged behav-
ioral economists to develop more comprehensive models to describe decision making
(Lopes, 1995). In both paradoxes, people choosing between probabilistic outcomes
with positive or zero magnitudes demonstrate unexpected preference reversals that
are inexplicable under an expected utility framework. In addition, the observation
that humans are more averse to ambiguity than to risk — even when the difference
does not change rational expected value calculations (Ellsberg, 1961) — challenges
the assumptions of expected utility frameworks.
Prospect Theory
Similarly, although expected utility theories often assume that attitudes toward risk
should be consistent across contexts, choices made between positive, risky rewards
and positive, guaranteed alternatives differ strongly from choices between risky and
guaranteed options when reward values are negative: people are more likely to choose
5
a guaranteed option than a risky gamble when outcomes are positive, but switch
to choosing the gamble when outcomes are negative (Kahneman & Tversky, 1979).
This striking behavioral pattern led Daniel Kahneman and Amos Tversky to propose
that we evaluate risk in a non-linear fashion, and that we differentially value gains
and losses. Using these assumptions, they developed a modified form of expected
utility theory, known as Prospect Theory, that better accommodates many of the
empirical behaviors that violate expected utility theories (Camerer, 2000; Kahneman
& Tversky, 1979; Tversky & Kahneman, 1992).
The utility of expected utility
While expected utility theory and its derivatives have provided useful tools for an-
alyzing decisions across a variety of contexts, the tendency to analyze utility solely
in terms of monetary value — or even in terms of estimated value — continues
to challenge behavioral and biological decision researchers. Humans and animals
consistently distinguish simple, non-probabilistic magnitudes, and animals’ behav-
ior under simple matching law tasks indicates that they also discriminate reward
probabilities (R. J. Herrnstein Richard J., 1961; Hinson & Staddon, 1983; Lau &
Glimcher, 2005; A. N. McCoy, Crowley, Haghighian, Dean, & Platt, 2003; Wolford,
Miller, & Gazzaniga, 2000). Thus, we might suppose that organisms could achieve
economic rationality by integrating reward magnitude and probability to maximize
utility. On the other hand, we suspect intuitively and know empirically that humans,
like most other species, frequently make economically irrational decisions that violate
the predictions of expected utility theories (Allais, 1953; Ellsberg, 1961; Kahneman
& Tversky, 1979) .
Empirical evidence confirms that our choices are influenced by emotion, culture,
and choice complexity (Dijksterhuis, Bos, Nordgren, & Baaren, 2006; Hsee & Rot-
tenstreich, 2004; McClure et al., 2004; Rottenstreich & Hsee, 2001; E. U. Weber
6
& Hsee, 1998). Even the matching behavior that confirms sensitivity to probabilis-
tic reinforcement results in submaximal choice patterns (R. Herrnstein & Heyman,
1979). Just as paradoxical responses to risk prompted the development of mathe-
matical models such as expected utility, these observations of behavior that cannot
be explained by utility theory have led both to the refinement of utility theory
(e.g., Prospect Theory) and to the development of alternative behavioral explana-
tions (Kahneman & Tversky, 1979; Simon, 1955; Gigerenzer, Czeslinski, & Mar-
tignon, 1999/2002).
1.1.2 Heuristic approaches
To better explain the limitations and irrationalities found by empirical studies of
behavior, Herbert Simon challenged homo economicus and the framework of utility
theory, and proposed that models of rational choice should consider the limitations of
the organism as well as its environment. Simon agreed with Bernoulli’s assumption
that decisions depended on both the agent and the environment, but argued that
rationality was bounded by man’s limited knowledge and computational ability (Si-
mon, 1955). In light of these observations, Simon suggested that human decision
processes might simplify the problem of maximizing over a large set of alternatives.
Rather than waiting to choose the optimal solution, we might choose the first satis-
factory option. This alternative to utility theory’s optimization, which is known as
satisficing, balances our satisfaction (or utility) against computational overload.
Satisficing and aspiration levels
Satisficing can also be explained as a simple rule-of-thumb, or heuristic, in which
available options are compared to a threshold. When the value, worth, or utility of
an option under consideration surpasses the desired threshold, or aspiration level,
then it is deemed satisfactory. This decision rule replaces absolute maximization
7
with a much easier decision rule: instead of assigning and remembering utilities for
all possible options (incidentally, a process which may not be environmentally or
energetically feasible), a decision maker needs only to compare each option to the
aspiration level.
Evidence for this strategy can be found in multiple situations and across species.2
In Chapter 2, I show that monkeys choosing between two visual cues are much more
likely to choose an option if its associated juice reward exceeds 153 ul; although there
is some variability in the monkeys’ choices (an observation that may result from their
need to explore alternatives in a frequently changing environment), this observation
suggests that the monkeys may be categorizing reward values as larger or smaller
than a 153 ul threshold. Similarly, in Chapter 3, I show that the same monkeys again
appear to apportion choices around a threshold (203 ul). While the threshold varies
with context, the behavioral pattern appears similar in both studies; furthermore,
previous analyses indicate that an aspiration level can — and probably should —
vary with context (Selten, 1998).
Win-Stay-Lose-Shift
Several game theoretic analyses have suggested that a heuristic strategy built on
comparisons to an aspiration level, Win-Stay-Lose-Shift (WSLS) (Thorndike, 1911),
is the optimal solution for repeated prisoner’s dilemma games (Imhof, Fudenberg,
& Nowak, 2007; Nowak & Sigmund, 1993). In a repeated interaction context where
multiple choices are available simultaneously, this decision rule leads agents to repeat
decisions that led to supra-threshold rewards (if win, then stay) but to switch away
from an option after receiving a sub-threshold payout (if lose, then switch) (Bar-
2 For a related example, in which stickleback fish cooperation and defection can be described as atit-for-tat strategy, see (Milinski, 1987). If cooperation parallels a win, and defection matches a loss,then much of the literature on repeated-interaction cooperative behavior and tit-for-tat strategiescould be reframed in WSLS terms (cf (Wedekind & Milinski, 1996)).
8
raclough, Conroy, & Lee, 2004). In a repeated prisoner’s dilemma context, where
there are two possible choices (cooperate or defect) and two possible outcomes (win
or lose), using an WSLS strategy allows an agent to both exploit available options
and to quickly change decisions in response to experienced consequences.
Intriguingly, this strategy may also be successful for more complicated social
interactions. In a language immersion setting, where students learn by interacting
with other students, the decision to use (and thus learn) a particular language can
depend on the number of other people using that language (Matsen & Nowak, 2004).
This number represents an aspiration level; if enough students use the language, then
the aspiration level is surpassed, and a new student will join the conversation. On the
other hand, if the number of students using the language is below the aspiration level,
then a learner will switch to an alternative language. The proposed aspiration level
— two to three students — is comparable to the number of concordant individuals
needed to begin an informational cascade (Banerjee, 1992; Bikhchandani, Hirshleifer,
& Welch, 1998), suggesting that switch/stay decisions based on a simple comparative
decision rule may contribute to a wide variety of social phenomena.
The utility of heuristics
Although the WSLS heuristic supports successful, approximately optimal decisions
across multiple contexts, using only a single heuristic for all decision environments
eventually leads to counterproductive behavior. In changing environments, heuristic
decision makers adapt by modifying favorite heuristic strategies (e.g., changing an as-
piration level) or by choosing alternative heuristic decision rules (Gigerenzer & Todd,
1999; Tversky & Kahneman, 1974). Such flexible decision making allows agents to
simultaneously approximate optimization while maximizing speed and minimizing
computational effort (Payne, Bettman, & Johnson, 1988). These observations, when
combined with experimental evidence for Prospect Theory, probability matching, and
9
local maximization, suggest that humans and other organisms use a wide variety of
strategies to negotiate probabilistic environments.
1.1.3 Summary
Although utility theories provide useful mathematical summaries of the interplay
between objective valuation and self-interested consideration, their rigorous axioms
limit their explanatory power. Increasingly nuanced versions of utility theories have
been adapted to empirical observations (Kahneman & Tversky, 1979; Tversky &
Kahneman, 1992), but experimental evidence continues to challenge such models.
Alternative heuristic, or rule-of-thumb, summaries of decisions have evolved to sum-
marize and rationalize our variegated choice strategies.
Importantly, the observation that decision making depends on prioritizing envi-
ronmental information for internal processing is explicit in the heuristic and bounded
rationality perspectives of decision making.Since humans have only limited process-
ing capacity and time, simplification and prioritization maximize personal satisfac-
tion despite the excessive computational demands of the environment (Gigerenzer
et al., 1999/2002; Gigerenzer, 2000; Simon, 1955). Similarly, neuronal decision pro-
cesses need to filter, process and prioritize relevant information — in other words, to
manipulate our internal representation of the external environment into a tractable
and useful form. Such manipulations can be quantified in terms of information the-
ory and entropy, or as energy-efficient processing (Montague, 2007; Shannon, 1948).
Alternatively, they may qualitatively link vivid environmental features to internal
representations of salience (Hall, 1994; Posner, Snyder, & Davidson, 1980). In the
next section, I will summarize the salience view of informational and attentional
prioritization before describing a current framework for understanding internal rep-
resentations of salience.
10
1.2 Salience
Even before we approach a decision, when we open our eyes and survey our environ-
ment, we need to make sense of our surroundings. The visual sensory information
that reaches our brain contains relevant and irrelevant information. Whether our
gaze falls on a fruiting orchard, a lion-filled savannah or a modern art gallery, our
ability to negotiate the risks and rewards — be they food, death, or art critics — de-
pends critically on our ability to extract relevant information from our environment.3
The visual system, in particular visual orienting, is a particularly useful background
system for testing stimulus detection and perception. Since extensive research has
thoroughly characterized the basic anatomy and function of visual sensation and eye
movement, many perceptual and decision-related studies depend on visual stimuli
and visuosaccadic responses to investigate internal decision processes. In this sec-
tion, I will first define salience before describing criteria for characterizing a region as
a salience map representing an abstract visual salience. Next, I will use the example
of the frontal eye fields to demonstrate that neural mapping of salience is a realistic
proposal.
1.2.1 Defining salience
The word salience is often used to describe the ephemeral quality that makes envi-
ronmental features stand out. Salience is defined literally by this attentional promi-
nence (Simpson & Weiner, 1989), and in colloquial use, we generally assume that
anything that attracts our attention must be salient. The latter observation reveals
an important distinction in our understanding of salience and attentional modu-
lation: while some stimuli stand out because of intrinsic physical features, others
3 Some of the most entrancing examples of information come from social information processing innon-human primates. Although baboons pay more attention to perineal swellings and hippopotamithan to art critics, their complex social interactions offer a particularly captivating illustration ofthe power and relevance of contextual information (Cheney & Seyfarth, 2007; Sapolsky, 2005).
11
become prominent through experience.
Physical features such as visual contrast, bright color, and sudden appearance
contribute to the attention-grabbing power of salience (James, 1892). Such intrinsic
features might evoke rapid, strong responses from our visual system as a result of
evolutionary selection processes that encouraged sensitivity to environmental infor-
mation relevant for survival. In attentional terms, these features can be described
as contributing to “bottom-up” regulation of attention, that is, activating neuronal
responses at early stages of neuronal processing, without extensive post-sensory mod-
ulation.
Alternatively, if we have acquired associations with the stimulus that make it
important, it may catch our attention because of those associations. The underlying
acquired characteristics, which are not intrinsic but reflect experience, valuation or
conditioning, can endow a stimulus with informational, emotional or motivational
relevance. Recognition of such non-physical features involves modulation of sensory
regions by higher-level cortices, a contribution known as “top-down” regulation.
Both intrinsic and acquired characteristics contribute to our understanding of
salience. Notably, the attention-grabbing qualities resulting from either type of fea-
ture do not necessarily indicate valence (e.g., reward versus punishment). Thus, the
color of a bright red object sighted from afar could alert us to the possibility that
important information is present, even though the object might not be immediately
recognizable as a sweet, nutritious fruit or as a slimy, poisonous frog. If we are driv-
ing, however, we may quickly react to a red light by slowing down or we may barely
notice the red litter along the median. In this thesis, I assume that both intrinsic
and acquired characteristics contribute to the attention-grabbing qualities that we
summarize as salient.
12
1.2.2 Integrating features: defining salience maps
Neurons in regions such as area MT (MT), inferior temporal region (IT) and amyg-
dala respond to specific features of the external environment (motion, objects and
emotion, respectively). While these representations highlight external locations with
particularly strong feature-specific salience, they maintain separate detailed repre-
sentations that can be further developed by identifying locations with multiple co-
existing salient features. This possibility raises a new question. Is there a brain
region that integrates the feature-specific information encoded by areas such as MT,
IT and the amygdala to form a cohesive representation of external salience? If such a
salience map exists — that is, if some region of the brain encodes attention-grabbing
power across space and across multiple features — where is it? Before addressing this
question, I will describe the characteristics that motivate classification of a region as
a salience map. Then, since frontal eye fields (FEF) has been well-characterized as
a probable salience map, I will describe evidence that the frontal eye fields function
as a salience map for detecting relevant visual stimuli (K. G. Thompson & Bichot,
2005) and I will propose that posterior cingulate cortex functions as a salience map
for mapping reward and motivational characteristics.
The core characteristic of a salience map is that it provides a spatial representa-
tion of attentional relevance based on integrated feature-specific maps. Thus, instead
of responding to a particular feature, and binding spatial location to a specific fea-
ture, neurons that contribute to a salience map should respond more generally to
noteworthy stimuli. I suggest that five characteristics of neuronal activity support
the description of a salience map.4
1. Neurons respond to stimuli across a variety of contexts. The anatomic and
functional convergence of local feature maps causes sensitivity to the integrated
4 For complementary descriptions, see (Fecteau & Munoz, 2006) and (K. G. Thompson & Bichot,2005).
13
strength of attention-grabbing or motivationally relevant features. This criterion
also suggests that neuronal activity may reflect both intrinsic visual features (or
“bottom-up” features) and learned relevance (or “top-down” features such as those
resulting from conditioned associations between a cue and reward information).
2. Neuronal responses are not feature-selective. Neurons may have poor sen-
sitivity to unitary attention-grabbing visual features such as color, contrast, orienta-
tion or motion, although their responses may be correlated with features when they
are task-relevant.
3. Neuronal activity reflects sensorimotor integration. Neuronal activity mod-
ifies perceptual processing and motor preparation in similar ways.
4. Neurons have spatial receptive fields. They respond more strongly to stim-
uli presented at a defined spatial locus than at other loci. Importanly, this is a com-
parative definition that does not preclude stimulus-sensitivity outside of the neuronal
receptive field.
5. Stimulus responses are not exclusively spatial. Neurons respond to salient
stimuli independent of eye movements, and may even respond to stimuli outside of
the neuronal receptive field. One implication of this response pattern is that neurons
may respond to stimuli that attract covert attention, but are not foveated.
Taken together, these criteria describe responses that permit prioritization of spa-
tial location by a somewhat abstract measure of salience that depends on integration
of feature-selective responses. Although there may be a universal salience map in the
brain, previous studies suggest that multiple salience maps may exist which integrate
over different (but potentially overlapping) feature sets and perform different (but
potentially overlapping) functional roles. For example, neurons in the frontal eye
fields appear to integrate both distinctive visual features and task-relevant visual
14
characteristics to localize task-relevant cues within visual space. In contrast, the
amygdala — despite its strong association with fear responses (Amaral, 2003) —
may eventually be described as a salience map due to its responses to task-relevant
non-social stimuli (Ousdal et al., 2008). Since there is extensive experimental ev-
idence supporting the characterization of FEF as a salience map, I will use it as
an example to illustrate the functional characteristics of a salience map. In later
sections, I will also provide evidence for classifying posterior cingulate cortex as a
salience map.
1.2.3 Salience map example: frontal eye fields
Neurons in the macaque frontal eye fields demonstrate all five of the features that
comprise a salience map; in fact, an extensive body of literature documents FEF
responses to salient information across contexts, across visual features, and across
space. This section will summarize the evidence that FEF functions as a salience
map, using the criteria described above (p. 13).
FEF neurons are sensitive to stimulus relevance. FEF neuronal activity discriminates
between informationally relevant and irrelevant stimuli presented in the neuronal
receptive field. When monkeys perform an oddball task, in which saccades to one
unique stimulus (the oddball) within a set of stimuli results in reward, FEF neurons
respond more strongly if an oddball is presented in the neuronal receptive field than
to an identically situated, motivationally irrelevant stimulus (Schall, Hanes, Thomp-
son, & King, 1995; Schall & Thompson, 1999). In addition, task-relevant learning
contributes to this effect (Bichot & Schall, 1999), indicating that both contextual
visual distinctiveness and learned contextual salience are integrated in FEF.
Finally, in experiments that parallel the random-dot motion experiments de-
scribed above, FEF neuronal activity approximates behavioral discrimination (K. G.
15
Thompson, Bichot, & Sato, 2005). When monkeys could distinguish between targets
and distractors based on color differences, neuronal responses varied according to
the relevance of the stimulus placed in the neuronal receptive field. However, FEF
neurons did not distinguish between targets that were behaviorally indistinguishable
due to similar coloring.
FEF neurons are not specific for feature detection. Importantly, FEF neuronal activity
discriminates targets from distractors based on features other than color similarity.
When all targets presented include randomly moving dots and the oddball is iden-
tified by unique motion coherence (i.e., in the opposite direction to all the other
targets), FEF neurons differentiate the presence of oddball versus distractor in the
neuronal receptive field (Sato, Murthy, Thompson, & Schall, 2001). Thus, FEF per-
ceptual discrimination is maintained whether salient, distinguishing information is
conveyed by color or by motion coherence. Further evidence for this assertion comes
from the observation that FEF neurons respond to informationally salient, task-
relevant oddballs as well as to task-irrelevant, visually-distinctive distractors (Bichot,
Chenchal Rao, & Schall, 2001).
FEF neurons contribute to visuomotor activity. Stimulating FEF neurons, even in the
absence of task demands, elicits directional saccades (Bruce, Goldberg, Bushnell, &
Stanton, 1985). Thus, activity in frontal eye fields can be presumed to contribute to
visuomotor indications of internal decision processes.
FEF neurons have spatial receptive fields. Early mapping studies demonstrated that
FEF neurons respond to the appearance of small spots of light and that these
responses are spatially selective, with large quadrant- or hemifield-sized receptive
fields (Mohler, Goldberg, & Wurtz, 1973; Bruce & Goldberg, 1985).
16
FEF neurons respond to covertly attended stimuli, independent of saccade. In a “no-go”
version of the oddball tasks described above — in which monkeys did not saccade to
the oddball — Thompson and colleagues found that FEF neurons discriminate odd-
balls from other cues (K. G. Thompson, Bichot, & Schall, 1997). Notably, the timing
and responsiveness of neuronal activity did not indicate the presence or absence of
saccade. Furthermore, in experiments that dissociated the oddball’s location from
the direction of the saccade, FEF neuronal activity reflects the location of the oddball
rather than the direction of the saccade (Murthy, Thompson, & Schall, 2001). Thus,
while FEF neurons appear to contribute to motor planning and may bind salient
information to spatial locations, their activity appears to encode the locations of
salient stimuli.
These studies confirm that FEF neurons are likely to integrate salient information
and represent the probability of motivationally relevant information or stimuli across
space. Thus, FEF activity promotes directing attention to potentially informative or
motivating stimuli, and thereby supports stimulus evaluation and action selection.
1.2.4 Summarizing salience and salience maps
The example of FEF neurons demonstrates that at least one part of the brain re-
sponds to a generalized, or integrated, abstraction of attention-grabbing features in
visual space. As described above, microstimulation experiments suggest that FEF’s
representation of visual space contributes to visual orienting and attentional alloca-
tion. Importantly, however, the observation that FEF neurons do not discriminate
large from small rewards (Ding & Hikosaka, 2006; Leon & Shadlen, 1999) suggests
that these neurons provide primarily spatial information. This possibility raises
yet another question: is there a brain region that integrates reward/motivational
information with spatial information, or that can serve as a salience map for as-
sessing reward? Before providing evidence that a strategically located region of the
17
brain, the posterior cingulate cortex, may fulfill this function, I will suggest a simple
characterization of decision processes in which CGp links perceived, prioritized and
remembered motivational information to outcome evaluation and action selection.
1.3 Neural contributions to decisions
As behavioral researchers and economists try to describe and understand decision-
making strategies, neuroscientists have focused on the neural mechanisms that con-
tribute to decisions. The difficulty of describing human decisions (e.g., as econom-
ically rational, heuristic, or adaptive across contexts) hints at the challenges that
hinder description of neural decision processes. This section will present a simple
framework for understanding the neural processes that contribute to decision making.
1.3.1 The process of decision: From perception to outcome
The process of decision making requires detection and characterization of relevant
stimuli within the external sensory environment. Evaluative processes characterize
relevant stimuli and predict the value of possible responses. During this process,
working memory is activated to recall previously experienced action-outcome evalu-
ations. The outcomes of actions that have been selected and completed are evaluated
and stored in long-term memory. At each of these stages, efficient, effective decisions
require that relevant information be prioritized. The process of decision making is
not, of course, as simple as this description might suggest (but cf (Rangel, Camerer,
& Montague, 2008)). The regions that subserve these stages, even when they are
anatomically distinct, modulate each other and interact. Furthermore, the recursive
nature of memory storage and outcome recall introduces additional layers of feedback
loops.
To make consideration of these non-linear processes tractable, I will first describe
three components of decisions — detection, evaluative action selection and outcome
18
evaluation — with examples of brain regions that contribute to each component.
Although memory is critical for many relevant learning processes, this thesis does
not focus on the contributions of memory to decisions, and I will not review these
contributions in any detail. Then, after proposing that the regions that subserve
action selection and outcome evaluation are linked by posterior cingulate cortex and
modulated by serotonergic projections, I will focus on two specific contributors to
decision making, posterior cingulate cortex and serotonin.
1.3.2 Detection
Simple stimulus detection is generally tested by asking a subject to respond as soon a
relevant stimulus becomes evident. For example, to determine the minimal signal the
retina can detect, a subject sitting in a darkened room is asked whether he (or she)
observed a flash of light after opening a shutter (Hecht, Shlaer, & Pirenne, 1942).
This simple procedure identifies the signal strength necessary to cross the subject’s
perceptual threshold (Posner et al., 1980). These experiments comprise the simplest
perceptual judgments, in which we report whether or not a stimulus is present.
The ability to discriminate the directionality of moving dots is representative
of discriminatory, perceptual evaluations that range in complexity from detecting a
flash of light or tactile vibration to culturally-influenced assessments of the riskiness
of a financial purchase (Hecht et al., 1942; Hernandez, Zainos, & Romo, 2000; E. U.
Weber & Hsee, 1998). Results obtained by recording from MT neurons in behaving
monkeys illustrate the probability that neuronal activity subserves discrimination of
the external characteristics that shape our behavioral responses (Romo, Hernandez,
Zainos, & Salinas, 1998; Romo, Hernandez, Zainos, Brody, & Lemus, 2000; Salz-
man, Murasugi, Britten, & Newsome, 1992); and with the supposition that regions
whose activity discriminates features are likely to contribute to contingent behavioral
responses.
19
Stimulus detection example: discriminating random dot coherence
In these experiments, complex perceptual decisions were tested by requiring monkeys
to report the direction of moving dots. In an experiment designed by Newsome
and colleagues (Newsome, Britten, & Movshon, 1989), monkeys shown an array
of randomly moving dots make a saccade to indicate whether the majority of the
dots are moving to the right or to the left. If a monkey correctly discriminates the
direction of movement, then it is rewarded with a squirt of fruit juice. Behaviorally,
these experiments establish that about 6% of the dots need to be moving in the same
direction for the monkeys to report a directional signal with reasonable accuracy (i.e.
82% correct).
In addition to testing behavioral sensitivity, Newsome et. al. recorded from
neurons in MT (Newsome et al., 1989). They found that neuronal activity, like
behavior, was sensitive to directionality: when the dots were moving almost entirely
randomly, MT neuronal activity was similar whether the dots were slightly more
likely to move toward the left or the right. Even before the dot coherence approached
and surpassed the coherence threshold for behavioral discrimination, MT neuronal
activity began to differentiate dot directionality. When a discriminable percentage
of the dots moved in a neuron’s preferred direction while within its receptive field,
that neuron fired more strongly than if the dots moved in the opposite direction.
To confirm that MT neurons not only reflect perceptual stimulus differences, but
contribute to the translation from external stimuli through perception to behavioral
responses, the authors performed another experiment. Instead of recording from MT
neurons, they applied small electrical currents to microcolumns of MT neurons with
congruent direction-selectivity (Salzman et al., 1992). This microstimulation biased
the monkeys’ responses in the preferred direction of the stimulated microcolumn,
suggesting that MT activity contributes to perceptual evaluation or action selection
20
involved in indicating the perceived direction of motion.
Other evidence for neuronal feature discrimination
While MT neurons respond selectively to motion, other regions of the brain respond
to other qualities. For example, IT neurons respond selectively to particular shapes,
such as faces or hands (Gross, 2008). New data suggests that the amygdala, which
was once thought to detect only threatening or fear-filled stimulus (Amaral, 2003),
detects a broader range of emotional features (Pessoa & Ungerleider, 2004) and
discriminates social signals such as the emotional valence of relevant faces (Pessoa,
Kastner, & Ungerleider, 2002).
The contributions of both IT and amygdalar activity to behavior are demon-
strated by the functional impairments resulting from lesions in these two brain re-
gions. Lesioning the entire temporal lobe, which eliminates both the amygdala and
the inferior temporal region region, causes a unique constellation of signs epony-
mously named Kluver-Bucy syndrome: affected animals fail to recognize objects,
learn poorly and show decreased emotional reactivity, diminished fear responses and
increased sexual drive. The emotional hyporeactivity and loss of fearful behavior can
be replicated with focal amygdala lesions (Aggleton & Passingham, 1981). Consis-
tent with IT responses to visual objects and its proposed role in object recognition,
the impairments in visual function and learning result from IT lesions (Gross, 1994).
1.3.3 Action evaluation and selection
In perceptual discrimination experiments, actions acquire attentional priority via
association with reward. If a monkey correctly discriminates the direction of maximal
coherence of field of randomly moving dots, he receives a reward. If the monkey
is instead offered a choice between two simultaneously available options, he has to
evaluate the relative desirability, or in economic terms, calculate the expected utility,
21
of the two options in order to choose to his best advantage. Since monkeys generally
choose the largest, sweetest or highest-probability option present, it is reasonable to
assume that these choices reveal the motivational value of options.
Thus, just as monkeys’ responses to flashes of light or to randomly moving dots
disclose perceptual thresholds, monkeys’ choices between rewarding options reveal
their preferences, or the relative motivational value placed on each option. This be-
havioral metric allows comparison of neuronal responses to decision variables such as
reward magnitude and probability. Importantly, action evaluation is a fundamentally
memory-dependent processes; thus, although I am not detailing memory’s contribu-
tions to decisions, I note that the monkey must remember the previous outcomes
to evaluate the worth of potential actions. Three brain regions, lateral intraparietal
cortex (LIP), prefrontal cortex (PFC) and anterior cingulate cortex (ACC), have
been implicated in evaluation and action selection; they are particularly worth not-
ing as examples of neural contributions to action selection. Each of these regions
functions slightly differently in action selection: LIP and PFC, respectively, appear
to prioritize value-based and historical or rule-based contributions to decisions, while
ACC selects and maintains decision strategies. In this section, I will summarize the
evidence for each of these contributions before discussing outcome evaluation regions.
Lateral intraparietal cortex
Neurons in LIP, a visuomotor region that appears to link reward evaluation to vi-
suosaccadic responses, signal both the magnitude and probability of available re-
wards (M. Platt & Glimcher, 1999). To demonstrate LIP’s sensitivity to these
decision-theoretic variables, Platt and Glimcher showed monkeys two conditioned
visual cues (one red and one green) before indicating the rewarded saccade with a
color-matched central cue. To compare responses to reward magnitude and proba-
bility, Platt and Glimcher varied the juice payout and reward probability separately.
22
Unsurprisingly, LIP neurons encoded only movement directionality once the rewarded
saccade was signaled; but before the required saccade direction was revealed, activity
scaled with the magnitude and probability associated with the cue in the neuronal
receptive field. When monkeys were allowed to choose freely between the same two
visual cues, both behavioral preferences and neuronal activity reflected the economic
expected value, or motivational salience, of available options. Thus, LIP activity re-
flected the magnitude and probability of reward, and predicted behavioral responses .
A pioneering study that used choices between juice rewards and socially relevant im-
ages to determine the motivational value of visual social information demonstrated
that LIP activity reflected a more abstract measure of desirability (Klein, Deaner, &
Platt, 2008).
Prefrontal cortex
A dramatic clinical correlation between PFC damage and erratic behaviors (Damasio,
Grabowski, Frank, Galaburda, & Damasio, 1994; Ratiu, Talos, Haker, Lieberman,
& Everett, 2004) has encouraged scientists to think of PFC as a region involved in
cognitive control. More recent studies have suggested a role for PFC in learning
choice-outcome associations and implementing resulting behavioral “rules” (Balleine
& Dickinson, 1998; Lee, Rushworth, Walton, Watanabe, & Sakagami, 2007; Miller &
Cohen, 2001). Evidence for this function comes from neurophysiological recordings
in macaques that demonstrate reward predictive activity, or expectation signals,
before reward delivery and outcome reporting after reward (Hikosaka & Watanabe,
2000; M. Watanabe, 1996). Recent studies confirm that in a matching pennies game,
PFC neurons encode past choices and rewards as well as the historically influenced
decision value that contributes to future decisions (Barraclough et al., 2004).
23
Anterior cingulate cortex
Although anterior cingulate cortex (ACC) has often been described as a neural con-
flict manager, recent studies have revealed a probable role for ACC in learning and
updating strategic decisions. Monkeys with focal ACC lesions fail to learn from
submaximal responses: they fail to update behavior by reversing options and main-
taining newly optimal choice patterns following submaximal or erroneous experi-
ences (Kennerley, Walton, Behrens, Buckley, & Rushworth, 2006; Rudebeck et al.,
2008). Where the regions described above as reward-sensitive or evaluative often
encode values and preferences that are primarily determined by reward magnitude
and probability, ACC neurons appear to represent the motivational value of an ac-
tion (Matsumoto, Matsumoto, Abe, & Tanaka, 2007; Rushworth, Buckley, Behrens,
Walton, & Bannerman, 2007). Contributions to learning from failures represent im-
portant components of both utilitarian and heuristic decision strategies (cf (B. Y.
Hayden, Pearson, & Platt, 2009)).
1.3.4 Outcome monitoring (and prediction)
Regions such as ACC, PFC and LIP that evaluate options and select actions depend
on memory for the maintenance of reward information across time, and on memory
retrieval to predict that information. Not surprisingly, then, regions that respond
differentially to outcomes also often also predict outcomes or respond differentially
to the relative expected values of options. These retrospective responses to remem-
bered value contribute prospectively to decision-making. Examples of these processes
have been found in neurons in the orbitofrontal cortex, dorsal raphe nuclei, ventral
tegmental area and posterior cingulate cortex, which are associated with evaluative
and predictive reward responses that contribute to action selection and decision.
24
Orbitofrontal cortex
When monkeys choose between options with punishment and reward outcomes, or-
bitofrontal cortex (OFC) neuronal firing rates reflect the relative values of available
rewards (Roesch & Olson, 2004). If, as this observation suggests, OFC signals re-
ward outcomes, then neurons in this region should reflect the relative magnitudes of
available rewards. In two more recent studies, Padoa-Schioppa and Assad quantified
monkeys’ preferences for juice rewards to develop a “menu” of options with relative
values indicated by the monkeys’ choices (Padoa-Schioppa & Assad, 2006, 2008).
The interpretive framework that these authors propose is somewhat complicated,
but the both studies suggest that, consistent with previous studies showing correla-
tions between reward preferences and OFC activity (Hikosaka & Watanabe, 2004),
neurons in OFC can encode relevant decision variables such as the juice chosen, the
strength of preference, and the relative value of the reward chosen. Macaque lesion
studies that demonstrate failure to update decision values with experienced out-
comes following OFC lesions provide further evidence for the hypothesis that OFC
contributes to outcome evaluation (Rudebeck et al., 2008; Schoenbaum, Chiba, &
Gallagher, 2000; Wallis, 2006).
Dorsal raphe nucleus
Neurons that produce the neurotransmitter serotonin may also contribute to re-
ward sensitivity. Serotonergic dysfunction is associated with anhedonic and risk-
seeking maladaptations (Deakin, 2003; Dolan, Anderson, & Deakin, 2001), and sero-
tonin is thought to convey information about negative consequences (Zhang, Lu,
& Bargmann, 2005), but little is known about the outcome sensitivity of the dor-
sal raphe nuclei (DRN) cells that produce serotonin. A recent study showed that
DRN neuronal activity predicted future reward magnitudes and responded differen-
tially to rewards delivered (Nakamura, Matsumoto, & Hikosaka, 2008), suggesting
25
that these neurons (and by extension, serotonin, if the neurons recorded are in fact
serotonergic) may contribute to evaluative processes. While there are few studies
that investigate the the contributions of serotonergic DRN neurons to decision, there
is extensive circumstantial and clinical evidence that serotonin modulates affective
and evaluative components of decision. These observations, especially in light of
the extensive anatomical spread of neuromodulatory efferents, raise important ques-
tions about the nature of neuromodulatory contributions to decisions. Serotonergic
contributions are particularly intriguing given the clinical evidence for serotonergic
involvement in risky decision making. In Section 1.5, I will discuss this evidence and
outline my rationale for investigating serotonergic contributions to decisions.
Dopaminergic neurons
The observation that dopaminergic neurons in the ventral tegmental area of the mid-
brain respond to reward probability and to the absence of predicted reward has led
to the hypothesis that these neurons encode a learning signal, often called reward
prediction error (RPE). As monkeys learn a cue-reward association, dopaminergic
responses to reward transfer to the cue. These neurons then respond with phasic
increases to unexpected rewards and their activity is suppressed after omission of
expected rewards (Fiorillo, Tobler, & Schultz, 2003; Schultz, Tremblay, & Holler-
man, 1998); thus, their activity appears to indicate unexpected positive and neg-
ative changes in predicted reward. Signaling the difference between expected and
received reward is important for learning and for updating information about expe-
rienced rewards. In addition to the evidence provided by the neurophysiology studies
just referenced, neuroimaging studies have implicated ventral tegmental area (VTA)
neurons in reward monitoring and updating, and functional studies confirm that
dopamine modulates responses to reward and contributes to addiction (Blum et al.,
2000; Dreber et al., 2009; Fiorillo et al., 2003; Kuhnen & Knutson, 2005; Wise, 2002).
26
Posterior cingulate cortex
Tonic CGp neuronal activity discriminates large from small rewards and tracks the
risk level associated with a risky gamble (A. N. McCoy et al., 2003; A. McCoy
& Platt, 2005b). CGp is thus sensitive to both reward magnitude and risk when
contextually relevant. These results may reflect discrimination that contributes to
representation of the decision value, i.e. the motivational salience of an option,
or more generally responsiveness to contextually relevant features. Notably, CGp
neurons also respond when expected reward is omitted, and maintain a representation
of received reward between trials (A. N. McCoy et al., 2003; B. Y. Hayden, Nair,
McCoy, & Platt, 2008). CGp microstimulation biases choice away from repetition of
a preferred risky gamble (B. Y. Hayden et al., 2008), suggesting that CGp contributes
to the evaluation of and response to economic or heuristic calculations of utilities.
Accumulating evidence suggests that, consistent with its anatomical connections to
regions that bias action selection and strategic decisions, posterior cingulate cortex
links outcome evaluation to strategic action selection. Intriguingly, CGp’s tendency
to respond to informationally relevant cues across contexts raises the possibility that
CGp may function as a salience map. I will discuss the possible functions of posterior
cingulate cortex in more detail in Section 1.4.
1.3.5 Summary
Although the neuronal contributions to decision making are complex and continue to
challenge characterization, accumulated evidence implicates neurons in both detec-
tion and differentiation of visual stimuli, and in reward differentiation and outcome
evaluation. Regions such as MT and IT contribute to detecting relevant stimulus
features, such as motion and shapes, that are necessary for simple discriminatory
decision making and in more complex, naturalistic decision contexts. Neuronal ac-
tivity in LIP, PFC and ACC participate in action evaluation and selection, while
27
neurons in OFC, DRN, VTA and CGp reflect reward outcomes (Fiorillo et al., 2003;
Kennerley et al., 2006; Klein et al., 2008; M. Platt & Glimcher, 1999; Wallis, Ander-
son, & Miller, 2001; Wallis & Miller, 2003a; Walton, Croxson, Behrens, Kennerley,
& Rushworth, 2007; Schoenbaum et al., 2000; Schultz, 2001).
In previous sections of this thesis, I have hinted that CGp might tie together the
regions that contribute to outcome evaluation and action selection. It is difficult
to isolate entirely distinct functions for each of the regions describe above, partly
since neurons in each of the regions described represents some version of outcome or
decision value. However, the differences in their functional contributions to decisions
suggest complementary contributions to a multi-factorial decision process. Thus, LIP
contributions to action selection depend on valuative signals that are often correlated
with utility theories’ expected value calculations (M. Platt & Glimcher, 1999); but
ACC updates outcome evaluation for strategy selection (Kennerley et al., 2006).
Thus, these two regions might represent alternative modes of evaluation that con-
tribute complementary (though potentially competitive and interactive) calculations
to action selection.
Both these brain regions — like most of the regions described above — are
anatomically linked to CGp. ACC and LIP send and receive projections to and
from CGp(Parvizi, Van Hoesen, Buckwalter, & Damasio, 2006). Thus, like PFC and
OFC, which both receive projections from CGp (Pandya, Van Hoesen, & Mesulam,
1981; Vogt & Pandya, 1987), neuronal activity in these regions is likely influenced
by CGp’s putative outcome-evaluative representations. In addition, the projections
from ACC and LIP to CGp are likely to refine CGp reward estimations and update
the value of actions. Summarily, current evidence suggests that the action selection
processes in LIP, ACC and PFC — which prioritize outcome valuation, strategy
application, and history-based rule application, respectively (Kennerley et al., 2006;
Klein et al., 2008; M. Platt & Glimcher, 1999; Wallis et al., 2001; Wallis & Miller,
28
2003a; Walton et al., 2007)— utilize outcome-evaluative signals from CGp, which are
maintained over time and interact with memory. In return, CGp receives signals from
LIP and ACC that may update spatial and action-predictive prioritization. The VTA
neurons described above respond strongly to environmental changes, particularly in
reward contingencies, and are thus likely to signal unexpected events that might con-
tribute to strategic or heuristic shifts via connections with ACC and PFC (Fiorillo et
al., 2003; Schultz, 2001). Finally, CGp’s projections to the OFC may contribute to
the formation of stimulus-outcome associations, whether by responses to unexpected
events or by maintaining representations of reward outcomes (Morecraft, Geula, &
Mesulam, 1992 Sep 15, 1993; Pandya et al., 1981; Rolls, 2000; Schoenbaum et al.,
2000). Of note, the DRN neurons referenced above project extensively throughout
frontal and decision-related regions. These putatively serotonergic neurons project
to PFC, ACC, OFC and CGp and are likely to modulate activity in each of these
regions (Baumgarten & Gothert, 1997).
Although these proposed functions are necessarily speculative, they are consistent
with evidence implicating each of these regions in decision making; and suggest that
decisions are, consistent with the complexity and variety of behavioral explanations
for decisions (Sections 1.1.1, 1.1.2), dependent on multiple modes of evaluation, pro-
cessing, and strategic action selection. In particular, this summary model sites CGp
at a decision-making nexus where it appears to contribute uniquely to recognition
and processing of salient motivational and informational events. I will discuss CGp
contributions to decisions in more detail in the next section. Finally, I will emphasize
the importance of neuromodulatory processes for decision making by discussing the
evidence that implicates serotonin in decisions.
29
1.4 Posterior cingulate cortex
Although it is difficult to simplify neural decision processes into independent stages,
most of the regions described above can be identified as contributing to action se-
lection or to outcome evaluation. Posterior cingulate cortex appears to link these
two processes, and may function as a salience map for prioritizing attentional al-
location based on reward-based or decision-relevant features. In this section, I will
describe the evidence for CGp contributions to decisions, from the clinical, anatom-
ical, and functional points of view. I will first describe clinical observations that
implicate CGp dysfunction in abnormal decision tendencies. I will then summarize
the anatomical clues to CGp function before discussing the functional evidence that
links CGp activity to outcome evaluation, attention and spatial awareness. Finally,
I will comment on the possibility that CGp may act as a salience map, prioritizing
visuospatial attention based on integrated information about such acquired features
as reward and decision value.
1.4.1 CGp dysfunction in disease
Hints that posterior cingulate cortex contributes to decision making and atten-
tional processes have come from the disrupted functionality observed in disease
or after cortical damage. The abnormal decision patterns observed in schizophre-
nia and borderline personality disorder have been associated with CGp dysfunc-
tion (Ha et al., 2004; Hazlett et al., 2005; Haznedar et al., 2004). Memory failures
in Alzheimer’s disease (AD) have been linked with the timing and magnitude of
decreases in CGp energy metabolism (Minoshima et al., 1997; Valla, Berndt, &
Gonzalez-Lima, 2001; Valla et al., 2002); and the normal functional connectivity
between CGp and hippocampus disappears in AD (Sorg et al., 2007). Importantly,
the apolipoproteinE (APOE) genotype that predicts worse prognosis is correlated
30
with great CGp abnormalities (Drzezga et al., 2005). Post-mortem histopathologi-
cal studies also confirm striking posterior cingulate cortex atrophy in AD (B. Jones
et al., 2006). Consistent with these observations, which link CGp dysfunction to
suboptimal memory and spatial attention, removing cingulate cortex causes neglect
and amnesia (Valla et al., 2001; Watson, Heilman, Cauthen, & King, 1973). In
addtiion, CGp hypoactivity is associated with both the diagnosis of schizophrenia
and with decreased ability to focus attention on a relevant clue to the exclusion of
distractors (Laurens, Kiehl, Ngan, & Liddle, 2005; Laurens, Kiehl, & Liddle, 2005).
Although the physiological bases of these diseases are complex and multifactorial
(and generally poorly understood), the observed symptoms suggest associations be-
tween CGp hypofunction and altered attention and memory — both associations
that are consistent with the proposal that CGp contributes to directing attention,
outcome evaluation and spatial orienting.
1.4.2 CGp anatomy
The observation that posterior cingulate cortex’s anatomical connections include
both evaluative regions and pre-motor cortices (A. McCoy & Platt, 2005a) also sup-
ports the hypothesis that CGp activity might mediate the transition from outcome
evaluation to action selection. CGp has extensive reciprocal connections with the
visuomotor regions LIP and FEF (Beckmann, Johansen-Berg, & Rushworth, 2009;
Klein et al., 2008; Morecraft et al., 1993; Pandya et al., 1981; M. Platt & Glimcher,
1999; Parvizi et al., 2006). CGp is also closely interconnected with ACC, a region
thought to contribute to selecting decision strategies and learning from errors (Ken-
nerley et al., 2006; Parvizi et al., 2006). Posterior cingulate neurons also project
to PFC, which appears to update and apply decision rules (Pandya et al., 1981;
Vogt & Pandya, 1987; Wallis et al., 2001; Wallis & Miller, 2003a); and to OFC,
which is thought to encode and update evaluative decision variables (Barraclough et
31
al., 2004; Pandya et al., 1981). Finally, CGp also interfaces with hippocampal and
parahippocampal regions that subserve memory, an important link for recalling past
rewards in order to predict outcomes and evaluate potential actions (Bachevalier &
Nemanic, 2008; Beckmann et al., 2009; Pandya et al., 1981; Parvizi et al., 2006; Vogt
& Pandya, 1987; Vogt, Finch, & Olson, 1992).
1.4.3 Functional characteristics of CGp activity
Studies of CGp activity during visual and decision tasks further support the sus-
picion, introduced by anatomical and clinical observations, that CGp contributes
to decisions by linking visuomotor, evaluative and attentional information. In the
following sections, I will summarize evidence that CGp does fulfill this role. First,
CGp neurons respond following visual cues and visual saccades, and these responses
are often spatially selective. In addition, although attention to tasks and differenti-
ation of task-relevant information appear to operate on different time scales, CGp
neurons monitor and differentiate task-relevant reward information. Finally, CGp
evaluative responses reflect more abstract judgments than juice reward magnitude,
as they discriminate between auditory cues as well as degrees of morality.
CGp reports visual cues and saccades
Although initial explanations of CGp function labeled the region as limbic or paral-
imbic, electrophysiological studies quickly established that the region’s functionality
involved sensorimotor processes and spatial memory. Single-neuron recordings in
cats revealed phasic responses after visual saccades and tonic modulation by orbital
position (C. Olson & Musil, 1992), and analogous experiments in macaques confirmed
these findings (C. Olson, Musil, & Goldberg, 1996). Although neurons in these early
studies responded only infrequently to abrupt-onset visual cues, they did respond
both to low-frequency flashes of light and to large visual targets; CGp neurons also
32
responded to contextual changes such as dimming or brightening ambient light (C.
Olson & Musil, 1992). More recent studies have confirmed and extended these find-
ings, both by demonstrating that these neurons respond phasically to small visual
cues and by clarifying the spatial encoding schemes used in CGp (Dean, Crowley, &
Platt, 2004; Dean & Platt, 2006; B. Hayden, Smith, & Platt, 2009).
CGp neurons are spatially selective
The studies that initially characterized CGp visual responses confirmed that CGp
neurons are often spatially selective (Vogt et al., 1992): in cats performing visual
fixation and visual saccadic tasks, CGp neuronal activity varied with the direction
of gaze and, after saccade, of saccade (C. Olson & Musil, 1992). Although spatial
selectivity across this population of neurons was not clearly linked to contraversive or
ipsiversive saccades, later studies using macaques indicated that CGp neurons tended
to have broad contralateral tuning: neurons responded more strongly following con-
traversive than ipsiversive saccades, and responded more strongly to contralateral
target onset than to ipsilateral targets (Dean et al., 2004; C. Olson et al., 1996).
Further experiments revealed that while posterior cingulate neurons can encode in
either retinocentric (eye-centered) or non-retinocentric (head-centered) coordinate
systems, they were more likely to use non-retinocentric coding. Similarly, while CGp
neurons can encode spatial location as either egocentric or allocentric (relative to sur-
roundings), allocentric encoding is more frequent (Dean & Platt, 2006). Thus, while
CGp’s spatial sensitivity involves multiple coordinate systems, it is biased toward a
contextual mapping consistent with a role in attentional allocation.
CGp responses operate on phasic and tonic time-scales
The observation that CGp neuronal activity has both phasic and tonic compo-
nents (C. Olson & Musil, 1992) suggests that CGp responses contribute to both
33
the immediate recognition and the memory of relevant events. A recent analysis of
CGp activity in monkeys performing an evaluative decision task demonstrated that
neuronal responses to reward were maintained through a 1-second interval following
reward, and into the next trial (B. Y. Hayden et al., 2008).
While this observation suggests that tonic CGp activity contributes to the mem-
ory necessary for evaluating the value of potential actions, other studies link tonic
modulations to attention rather than to decision. For example, both neuroimaging
and neurophysiological experiments show that CGp activity decreases when atten-
tion is allocated to task performance or to monitoring visual cues, and that phasic
responses to visual stimuli are independent of tonic modulations (Fox et al., 2005;
B. Hayden et al., 2009). It is difficult to clearly link these tonic modulations with
CGp contributions to decisions, but worth noting that these observations support a
role for CGp in attention and, potentially, memory.
CGp neurons monitor task-relevant reward information
In a single-target saccade task for liquid reward, CGp neuronal activity reflects re-
ward sizes across blocks, suggesting that CGp neurons may encode saccade value
based on experienced reward information (i.e. an acquired feature of visual cues) (A. N.
McCoy et al., 2003). Furthermore, CGp neurons differentiate levels of risk in a gam-
bling task where monkeys choose between guaranteed reward and a variable reward
gamble (A. McCoy & Platt, 2005b). Consistent with the supposition that CGp con-
tributes to reward evaluation and action selection, CGp activity measured in each of
these studies reflected task-relevant information that predicted behavior: in the first
study, reward magnitude, and in the second study, risk.
In addition, the temporal development of CGp responses in these tasks — i.e.,
responses consistently follow events rather than preceding them — suggests that
CGp monitoring of task-relevant information could contribute to memory-guided de-
34
cisions. As previously noted, CGp neurons maintain a representation of experienced
reward between trials; and microstimulation of CGp following reward delivery biases
choices away from the generally preferred risky option (B. Y. Hayden et al., 2008).
CGp responds ubiquitously across contexts
Imaging studies provide extensive evidence that CGp activity contributes to at-
tentional, evaluative or visuospatial processes across a variety of contexts. These
studies measure activity over longer time windows than most related neurophysio-
logical studies and are thus likely to describe tonic CGp function (B. Y. Hayden et
al., 2008). Several studies confirm that CGp differentiates rewards from non-rewards
and losses in financial decisions (Knutson, Fong, Bennett, Adams, & Hommer, 2003;
Kuhnen & Knutson, 2005). In addition, CGp activity correlates with moral judg-
ments and predict punishment magnitude in socially oriented studies (Buckholtz et
al., 2008; Robertson et al., 2007), and differentiates emotional, threatening and neu-
tral words (Maddock, Garrett, & Buonocore, 2003; Maddock & Buonocore, 1997).
Although such social information seems distant from economic reward evaluation,
these studies may indicate a broader, multi-contextual role for evaluative function-
ality in posterior cingulate cortex.
1.4.4 Summary
The studies summarized above support the hypothesis that CGp links evaluative
processes that depend on experienced reward to visuomotor decision output. CGp
neuronal responses following both reward and saccade reflect subjective valuation of
experienced rewards, and thus could contribute to outcome monitoring and evalu-
ation. Furthermore, CGp maintains such evaluative representations over time, and
CGp microstimulation affects future decisions. The combination of observed value-
representation, visuospatial sensitivity and attention observed in CGp in consistent
35
with a role for CGp in decision making.
Does CGp function as a salience map?
It should also be evident that CGp fulfills many of the criteria for definition as a
salience map (Section 1.2.2, p. 13). CGp neurons respond to visual stimuli across a
variety of contexts and, in addition, respond to auditory stimuli, changes in the ex-
ternal environment and internally motivated sensory signals such as pain (Morrison,
Peelen, & Downing, 2006). Furthermore, these neurons respond differentially based
on decision-related information such as reward and risk; and the observation that
CGp neurons respond differentially to equal-valued risky and safe options confirms
that neuronal activity is not tied to reward magnitude independent of other contribu-
tions to task-relevant decision variables (A. McCoy & Platt, 2005b). CGp neuronal
activity reflects both perceptual processing, specifically decision variables, and motor
preparation. CGp neurons have broad, often contralateral spatial receptive fields.
Remaining questions
However, several questions remain unanswered. While CGp neurons respond to
visual stimuli across multiple contexts, it is not clear whether these responses are
exclusively spatial, i.e. whether evaluative information in decision or saccade tasks is
inextricably linked to spatial and saccadic maps. In addition, while CGp responses to
risk may be task-relevant, the correlation between risk levels and monkeys’ subjective
expected utility (as revealed by their choice allocation) introduces the possibility that
CGp neurons respond to risk as a unitary feature.
This thesis describes two sets of experiments designed to address these questions.
First, in Chapter 2, I will describe results that demonstrate that CGp responses
generalize across and beyond visual contexts. Specifically, I show that CGp neu-
rons respond phasically to visual cues, after saccade, and following reward delivery
36
across multiple contexts; and, furthermore, that CGp neurons also respond to reward
delivery independent of visual cues. Second, to confirm that CGp neurons track task-
relevant decision variables in the presence of risk (Chapter 3), I will describe evidence
that as monkeys’ preferences shift across contexts in a gambling task, CGp neurons
track the revealed preferences rather than risk, and thus reflect an integrated measure
of reward and motivational information.
1.5 Serotonin
Several of the brain regions described above that contribute to decision making (Sec-
tion 1.3) receive extensive innervation from serotonergic neurons in the DRN. Re-
gions that contribute to action selection, such as the PFC and ACC , and regions
that differentiate reward outcomes, such as the OFC and CGp, all receive significant
input from serotonergic neurons (Baumgarten & Gothert, 1997; Varnas, Halldin, &
Hall, 2004). This observation suggests that serotonin is likely to modulate decision
making. Serotonin, like CGp, has been implicated in decision making by evidence
from psychiatric disorders with profound behavioral deficits. Although the evidence
provided by the clinical observations described below is associative and does not es-
tablish causal mechanisms, it suggests that serotonin contributes to decisions and,
more specifically, to decisions involving risk. In this section, I will summarize the
evidence for a link between serotonergic function and decision making that led me
to suppose that lowering serotonin might cause an increased tendency to make eco-
nomically risky choices.
1.5.1 Serotonin is implicated in psychiatric disorders
One of the disorders most strongly linked with serotonergic dysfunction is depres-
sion, a disorder in which negative emotional states are associated with altered per-
ceptions of the world (Deakin, 2003; Jans, Riedel, Markus, & Blokland, 2007). De-
37
pression is associated with increased impulsive behavior as well as increased risk of
suicide (Apter et al., 1990; d’Acremont & Linden, 2007). Genetic studies have shown
that alternative forms of the serotonin transporter gene modulate risk for depression
following stressful events, and are linked to differential probabilities of suicidal be-
havior (Caspi et al., 2003; Nielsen et al., 1998); in addition, serotonin transporter
polymorphisms modulate the risk of post-traumatic stress disorder (PTSD) (Koenen
et al., 2009). The increased risk of suicide associated with depression may depend on
physiological changes in serotonergic function (Brown & Linnoila, 1990; Nielsen et al.,
1994, 1998; Nordstrom et al., 1994). Serotonergic deficits have also been associated
with addictive behaviors, attention-deficit and hyperactivity disorder (ADHD) and
impulsive personality disorders as well as impulsive/compulsive syndromes such as
Tourette’s and obsessive-compulsive disorder (OCD) (Deakin, 2003; Fletcher, 1995;
Grados, 2009; Harrison, Everitt, & Robbins, 1999; Higley et al., 1996; Mossner,
Muller-Vahl, Doring, & Stuhrmann, 2007; Wong et al., 2008).
Since serotonin contributes to both neurogenesis and formation of brain structure,
developmental deficits of serotonergic function may contribute to the pathology of
psychiatric disorders (Sodhi & Sanders-Bush, 2004). Both autism and Down’s syn-
drome are thought to reflect disrupted serotonergic development (Whitaker-Azmitia,
2001; Chugani et al., 1999; DeLong, 1999; Pardo & Eberhart, 2007).
Drugs that modulate serotonergic activity ameliorate symptoms of many of the
conditions just mentioned. For example, selective serotonin reuptake inhibitors
(SSRI) have been used to treat autism as well as depression, and some of the anti-
psychotics used to treat schizophrenia act at serotonergic receptors (e.g. risperidone,
clozapine) (Roth, Hanizavareh, & Blum, 2004; Veenstra-VanderWeele, Anderson, &
Cook, 2000). This observation further supports the supposition that serotonergic
dysfunction contributes to altered behavioral patterns observed in psychiatric condi-
tions. Thus, both observational clinical evidence and successful treatment strategies
38
implicate serotonergic mechanisms in disorders associated with increased impulsivity
or abnormal responses to risk, such as depression, schizophrenia, impulsive person-
ality disorders, PTSD and autism (Akhondzadeh, 2001; Cook, 1990; Corchs, Nutt,
Hood, & Bernik, n.d.; Deakin, 2003; DeLong, 1999; Pardo & Eberhart, 2007; Sven-
ningsson et al., 2006; Marek, Carpenter, McDougle, & Price, 2003).
1.5.2 Serotonin is implicated in risky decision making
Notably, genetic studies also suggest that serotonin may contribute to impulsive
and risk-seeking behaviors (Chau, Roth, & Green, 2004; Gainetdinov et al., 1999;
Higley et al., 1996; Kreek, Nielsen, Butelman, & LaForge, 2005; Nordstrom et al.,
1994). The observation that regions with extensive serotonergic innervation, includ-
ing frontal cortical regions such as ACC (Kennerley et al., 2006), OFC (Padoa-
Schioppa & Assad, 2008), CGp, and prefrontal cortex (Barraclough et al., 2004),
contribute to strategic decision making supports the general supposition that sero-
tonin contributes to decisions. In fact, low serotonin is associated with pathological
gambling as well as excessive impulsivity (Dolan et al., 2001; Nordin & Sjodin,
2006); and serotonin receptor variants predict willingness to take risk in financial
decisions (Kuhnen & Chiao, 2009).
Behavioral research also indicates that serotonin contributes to impulsivity, or
unwillingness to wait for reward (Winstanley, Dalley, Theobald, & Robbins, 2004).
Subjects are more likely to choose a smaller reward delivered sooner than a larger,
delayed reward following dietary manipulations that lower brain serotonin levels
than under normoserotonergic conditions (Schweighofer et al., 2008). Although im-
pulsivity may not represent the same psychological construct as economic risk (J.
Evenden, 1999a), behavioral studies suggest that temporal impatience and economic
risk-attitudes are closely related (B. Y. Hayden & Platt, 2007; M. L. Platt & Huettel,
2008).
39
1.5.3 Summary and experimental direction
Thus, disorders associated with serotonergic dysfunction present with evidence of im-
paired decision-making, particularly with respect to risk. This observation suggests
that serotonin contributes to outcome evaluation and action selection, though it does
not specify a particular mechanistic contribution. The trend for dysfunctional sero-
tonin to be associated with increased risk-taking behaviors, especially given recent
evidence for a correlation between low serotonin and pathological gambling (Nordin
& Sjodin, 2006), supports the specific hypothesis that low serotonin levels may cause
increased risk-taking behaviors.
I tested this hypothesis in two species of animals. In Chapter 4, I demonstrate
that lowering brain serotonin in monkeys choosing between guaranteed and risky
reward options increases the likelihood that the monkeys will choose the risky option.
In Chapter 5, I describe an analogous experiment in mice, with complementary
results.
1.6 Experimental rationale and specific aims
The overarching goal of my dissertation was to investigate the neural processes that
contribute to decision making. Since risk is an essential component of decision mak-
ing, the correlation between posterior cingulate activity and risk preference (A. Mc-
Coy & Platt, 2005b), and between dysfunctional serotonergic systems and patho-
logically excessive risk-taking (Deakin, 2003; Nordin & Sjodin, 2006; Akhondzadeh,
2001) suggested that both CGp and serotonin may be particularly relevant compo-
nents of the decision-making process. The first two aims of this dissertation focus
on the role of CGp in decision-making. Since I suspected that CGp functions as a
salience map, I first asked whether CGp neurons detect contextually relevant events
across tasks, before determining that CGp activity reflects heuristically-guided pref-
40
erences across risky lotteries. The third and fourth aims ask whether serotonin
modulates revealed preferences for a risky option in two species, mice and monkeys.
Aim 1: To determine whether CGp neuronal activity reports salient events across contexts,
both inclusive and independent of visuospatial and visuomotor relevance
The hypothesis that posterior cingulate cortex monitors salient information — or,
as posited above, functions as a salience map — predicts that neuronal activity
in CGp should phasically report the occurence of informationally relevant stimuli;
that while CGp neurons might encode the spatial directionality of target-directed
saccades motivated by reward, they should report the appearance of relevant stimuli
independent of spatial location and independent of saccade; and, furthermore, that
CGp neurons should report the occurrence of uncued, non-visual events such as
unexpected reward. To test these hypotheses, I recorded from CGp neurons in awake
monkeys performing four tasks:
1. To confirm that CGp neurons respond to informative stimuli, including rewards,
in a decision task, I recorded from individual CGp neurons while monkeys performed
a two-target visual choice task in which reward size was contingent on target color
but not on location.
2. To confirm that CGp neurons respond to informative operant stimuli when no
decision is available, I recorded from CGp neurons in monkeys performing a single-
target operant task in which reward size was contingent on target color.
3. To confirm that CGp responses to visual stimuli are independent of visuomotor
planning, I recorded from CGp neurons in monkeys performing a single-target Pavlo-
vian task in which reward size was cued by target color.
4. To confirm that CGp neurons respond to stimuli independent of task relevance, I
recorded from CGp neurons in monkeys receiving uncued rewards.
41
I found that that CGp neurons responded following motivated saccades, and that
this post-saccadic activity was modulated by spatial location; that CGp neurons
responded to informationally relevant visual cues across all three tasks, largely inde-
pendent of spatial information; and that CGp neurons responded to reward delivery
whether cued or not. These results are consistent with the hypothesis that phasic
CGp activity reflects the occurrence of relevant events, and that CGp might function
as a salience map.
Aim 2: To determine whether CGp neuronal responses to risk reflect encoding of contex-
tually relevant information, i.e. contextual preferences and choice probability.
If posterior cingulate cortex activity reflects salient information across contexts, and
contributes to the development of preferences, then tonic CGp activity should reflect
and predict preferences across contexts. Thus, CGp neurons that are sensitive to
risk as a distinctive feature of frequently chosen options (A. McCoy & Platt, 2005b)
should reflect preference for a more rewarding safe option even in the presence of a
risky option. I tested this hypothesis by recording from CGp neurons in two contexts
differentiated by the reward sizes available to monkeys performing a gambling task.
In one context, when mean reward values for a safe option and a risky option were
equal, the monkeys preferred the risky option; but when the safe option offered a
higher average reward than the risky option, monkeys preferred the safe option. CGp
neurons reflected the choice preference within each context, and predicted reward-
history-dependent preferences across both contexts. These results confirm that CGp
monitors contextually integrated reward information and clarify the function of CGp
in decisions made under risk.
Aim 3: To assess the contribution of brain serotonin to economically risky decisions in
monkeys.
42
Extensive clinical and circumstantial evidence associates low serotonin (5-HT) with
increased risk-taking behaviors. If serotonin plays a causal role in the development
or expression of risk preferences, then lowering brain 5-HT should increase the prob-
ability of risky choices. I tested this hypothesis in three monkeys choosing between
a risky option and a safe option (gambling task, as described in Aim 2). When
the monkeys underwent rapid tryptophan depletion (RTD), a dietary manipulation
that decreased brain serotonin levels, they became more likely to choose the risky
option. Comparing the monkeys’ behavior across a range of lotteries confirmed that
the monkeys valued the risky option more under low-serotonin conditions: they were
more willing to give up juice available from the safe option to choose the risky gam-
ble. These results confirm a causal relationship between low serotonin and increased
tendencies to choose economically risky options.
Aim 4: To assess the contribution of brain serotonin to economically risky decisions in
mice.
Since serotonergic mechanisms are broadly conserved across species, lowering brain
serotonin should have the same effect in mice as in monkeys. I first established basic
mouse behavior in a task analogous to the gambling task used in monkeys. Mice,
like many other species, strongly prefer a safe option to a double-or-nothing gamble.
I then used para-chlorophenylalanine (PCPA), an irreversible inhibitor of the rate-
limiting enzyme in serotonin synthesis, tryptophan hydroxylase (TPH), to lower
brain serotonin levels in a subset of mice and compared their behaviors to that of
sham-treated mice. PCPA-treated mice, like monkeys, are more likely to choose the
risky gamble than normo-serotonergic mice. This observation confirms that serotonin
plays an important role in modulating behavioral responses to environmental risk.
43
2
Neurons in posterior cingulate cortex respondphasically to informative events
Neurons in posterior cingulate cortex (CGp) signal reward expectations, reward un-
certainty and level of task engagement via tonic modulations in activity but respond
phasically to the onset of visual targets, after saccades to those targets, and following
rewards delivered for gaze shifts. The presence of both tonic and phasic domains of
response suggests the hypothesis that CGp neurons dynamically signal the instanta-
neous allocation of neural resources to the current information stream. In this view,
salient events that provide new information should trigger CGp neurons to respond
phasically while state-dependent vigilance levels should modulate tonic activity. To
test the impact of event salience on phasic responses, we recorded from 91 CGp neu-
rons while two monkeys (Macaca mulatta) performed three tasks for fluid rewards as
well as while they received random uncued fluid rewards. Individual CGp neurons
responded to the onset of visual targets associated with rewards, whether or not
they triggered overt orienting; operantly-conditioned saccades rewarded with fluid;
and reward delivery. Notably, CGp neurons were as likely to respond phasically to
44
unpredicted, surprising rewards outside of any task as to fully predicted rewards
delivered for task performance. These results support the hypothesis that CGp dy-
namically signals changes in the ongoing stream of information linking the external
environment to internal state.
2.1 Introduction
The term salience is often used to describe stimuli that stand out from the ongoing
stream of sensory inputs, engage attention, and trigger enhanced processing in the
brain (James, 1892; Muller & Rabbitt, 1989; Posner et al., 1980). Although typi-
cally associated with sensory processing, the concept of salience can in principle be
extended to other types of events, such as motor behavior and reward outcomes,
that violate expectations or alter the relationship between the internal and exter-
nal environment. Salience broadly writ not only contributes to enhanced sensory
processing (Treisman & Gelade, 1980; Posner et al., 1980), but also learning (Mack-
intosh, 1975; Pearce & Hall, 1980) and motor adaptation (Abrams & Jonides, 1988;
Rosenbaum, 1980). Although the neural mechanisms underlying the attentional
modulation of sensory processing have been the subject of intense scrutiny, the pro-
cesses that mediate the extraction of information from other types of salient events,
such as motor behavior and reward, remain poorly understood.
Based on evidence from anatomy, brain imaging, and neurophysiology, we hy-
pothesized that posterior cingulate cortex (CGp) contributes to the processing of
informational salience, independent of its sensory provenance. CGp is extensively
interconnected with brain areas involved in attention, such as parietal cortex, as
well as brain areas involved in motivation and emotion, such as anterior cingulate
cortex (ACC) and orbitofrontal cortex (OFC) (A. McCoy & Platt, 2005a; Pandya
et al., 1981; Parvizi et al., 2006). Furthermore, brain imaging studies demonstrate
activation in CGp in response to salient visual events (B. Hayden et al., 2009; C.
45
Olson et al., 1996), overt orienting (Berman et al., 1999), emotional events (Greene,
Nystrom, Engell, Darley, & Cohen, 2004; Maddock, 1999; Morrison et al., 2006),
recalling information from autobiographical memory (Cabeza et al., 2003; Maddock,
Garrett, & Buonocore, 2001), and reward itself (Kable & Glimcher, 2007; Knutson
et al., 2003; Kuhnen & Knutson, 2005; Nieuwenhuis et al., 2005; Small, Zatorre,
Dagher, Evans, & Jones-Gotman, 2001; Small et al., 2003).
Moreover, neurophysiological studies have shown that CGp neurons respond pha-
sically to infrequent or unpredictable visual stimuli and overt orienting (Dean et al.,
2004; C. Olson & Musil, 1992) as well as task-relevant reward delivery and omis-
sion (A. N. McCoy et al., 2003). Furthermore, CGp neurons respond with tonic mod-
ulations in activity to signal reward expectations (A. N. McCoy et al., 2003; Kuhnen
& Knutson, 2005; B. Y. Hayden et al., 2008), memory for reward outcomes (B. Y.
Hayden et al., 2008), choice (A. McCoy & Platt, 2005b; B. Y. Hayden et al., 2008),
and reward uncertainty (A. McCoy & Platt, 2005b). Together, these observations
are consistent with the hypothesis that posterior cingulate cortex contributes to the
detection and transmission of salient information independent of sensory modality
(e.g., visual vs. gustatory), overt orienting, or active decision making.
To test this hypothesis, we studied the activity of 91 CGp neurons in four task
contexts in two male macaque monkeys. In three contexts, monkeys received juice
reward for either choosing a target for overt orienting based on its color-reward as-
sociations, shifting gaze to a single target associated with reward, or fixating while
a peripheral stimulus was illuminated followed by reward. In the fourth context,
monkeys received uncued reward. In each context, we measured phasic responses
by comparing activity following events such as visual cue onset, saccade initiation,
and reward delivery to the immediately preceding pre-event baseline. Neurons re-
sponded to informative visual stimulus onset in all three contexts, whether or not
overt orienting to target was required; following saccade initiation whether a choice
46
was or was not required; and to both cued and surprising reward. Furthermore, CGp
neurons responded as often to uncued, surprising reward as to cued reward. These
results suggest that CGp dynamically signals salient information that contributes
to the on-line assessment of the sensory environment, monitoring of movement, and
evaluation of reward.
2.2 Results
To confirm that monkeys learned the reward associated with red and green cues, we
examined choices between large and small rewards in a color-based target choice task
(Figure 2.1). In this task, a large reward was associated with one color and a small
reward with the other in a block of trials. Notably, the location of the colored cues
was not predictable (Lau & Glimcher, 2005, 2008; Sugrue, Corrado, & Newsome,
2004). Monkeys chose the cue associated with the large reward more often than the
small reward across the range of potential reward options (Figure 2.4a). Monkeys
behavior was well fit with a logistic function with a steep slope indicating excellent
reward-color association (beta=0.04, p<0.01).
Sensitivity to color-cued reward information was also evident in single-target
operant trials in which monkeys shifted gaze to a single target location that was
differentially rewarded based on target color (Figure 2.2). Consistent with a pre-
vious report (A. N. McCoy et al., 2003), the slope of the line relating peak eye
velocity to saccade amplitude (the main sequence) decreased as reward magnitude
increased (Figure 2.4b: linear regression R = 0.045, p<0.01). In addition, reac-
tion time increased as reward magnitude increased (Figure 2.4c: linear regression
R=0.062, p<0.01). On such delayed saccade trials, increased response latency and
decreased velocity may reflect a speed-accuracy trade-off motivated by the desire for
larger rewards (Dean & Platt, 2006).
Monkeys also performed Pavlovian fixation trials in which they received reward
47
after maintaining central fixation while a red or green peripheral cue was illuminated;
again, color predicted reward size within a block of trials (Figure 2.3). Reward mag-
nitude did not influence the probability of error in any of the tasks (choice, error
probability in large reward vs small reward trials, n.s.; operant, n.s.; Pavlovian, n.s.);
since the monkeys average error rates on these tasks are low (choice: BR 6.4% error,
NI 16.5%; operant: BR 8.6%, NI 14.9%; Pavlovian: BR 5.9%, NI 16.3%), this obser-
vation suggests that the monkeys were strongly motivated to collect as much reward
as possible within an experimental session. Next, we assessed whether neurons in
CGp responded to the occurrence of salient events while monkeys performed each of
these tasks as well as following uncued rewards. We recorded from a total of 91 CGp
neurons across all tasks. A subset of these neurons were studied in each task: colored
target choice, 38 neurons; single target operant task, 30 neurons; Pavlovian fixation
task, 48 neurons; uncued reward delivery, 68 neurons. Each task had a different set
of salient events. The choice task included illumination of two targets, choice via
saccade, and reward delivery. The operant orienting task included illumination of a
single target in a predictable location, gaze shift, and reward delivery. The Pavlovian
fixation task included fixation onset, peripheral target onset, and reward delivery.
Finally, the uncued reward task included only the unpredictable delivery of reward.
To probe neuronal responses to these events, we compared neuronal activity in the
two 200ms epochs following each event to a 200ms epoch (baseline) immediately
preceding the relevant event.
Figure 2.5 shows peri-stimulus time histograms (PSTHs) for two example neurons
recorded during the color choice task; one neuron reported target onset (a), and the
other responded after saccades (b) and reward delivery (c). Across the subpopulation
of 38 neurons, CGp neurons responded to all three salient events with either increased
or decreased firing rates, as described previously (McCoy et al., 2003, (A. N. McCoy
et al., 2003)). Across epochs, more cells responded by significantly increasing activity
48
after saccade (Table 2.1), while more cells tended to decrease activity after reward
(Table 2.1). Since a sizeable fraction of CGp neurons shows some degree of spatial
selectivity (Dean et al., 2004; C. R. Olson, Musil, & Goldberg, 1993; C. Olson et al.,
1996; C. Olson & Musil, 1992), we also assessed whether phasic responses were mod-
ulated by chosen target location. To test this possibility, we used a factorial ANOVA
to identify significant interactions between neuronal responses to events and chosen
target location. (ANOVA, firing rate vs epoch and chosen target location). Neurons
were most likely to distinguish spatial location following saccade onset (Table 2.1);
only a small fraction of the population signaled target location following cue onset (2
neurons) or reward delivery (4 neurons), consistent with previous results (A. N. Mc-
Coy et al., 2003). To examine the aggregate response, we sorted the cells by whether
responses to each event were positive or negative (cue, saccade, reward; cells with
significant or nonsignificant event responses) and calculated the mean firing rate per
epoch for each group (Figure 2.5c,d). Both populations responded significantly after
all events (though activity in the increasing population did not differ from baseline
in the second cue epoch).
These observations confirmed that CGp neurons report salient events, such as
visual cue onset and reward delivery, that contribute to reward associations and the
expression of choice. We next asked whether neuronal activity was modified by events
in an operant context where cues were informationally relevant but choice was not
required and monkeys earned reward by shifting gaze to a single target (Figure 2.2).
We found neurons that reported cue, saccade and reward on these trials. The activity
of the first example neuron shown in Figure 2.6a,b increased following target onset
and saccade onset, while that of the second neuron (c,d) decreased following saccade
onset and reward. Many neurons responded to events with positive and negative
changes in firing frequency (Table 2.1). Again, neurons responded more strongly
following saccade than following cue or reward, both individually (Table 2.1) and
49
across the population (Figure 2.6c,d).
Having confirmed that CGp neurons report salient events associated with overt
orienting for rewards, we next asked whether CGp neurons also respond phasically to
visual stimuli and rewards in a Pavlovian context, where conditioned color cues pre-
dict reward in the absence of gaze shifts. As in the choice and operant tasks, neurons
responded to both cue onset and reward delivery (Figure 2.7a,b; example neurons).
In the subpopulation of 48 neurons, more neurons responded to reward delivery than
to cue onset (Table 2.1). Figure 2.7c,d shows the neuronal population activity sorted
by positive or negative responses. These data make plain that CGp neurons respond
phasically to informationally relevant information independent of active choice or
overt orienting for reward (contra (C. Olson et al., 1996) but consistent with (C. R.
Olson et al., 1993)).
The presence of phasic responses to task events across all three contexts sug-
gest the possibility that CGp neurons respond to any surprising or motivationally
informative events, including uncued reward delivery in the absence of visual stimu-
lation. To test this idea, we studied the neuronal activity of 68 CGp neurons while
monkeys received uncued rewards while sitting quietly in the experimental cham-
ber. Consistent with data from the three active tasks, CGp neurons responded to
reward delivery with increasing (Figure 2.8a) or decreasing (Figure 2.8b) phasic re-
sponses. Across the subpopulation, more neurons responded with decreasing firing
rate than increasing firing rate (Table 2.1). To summarize population activity, we
sorted the neurons into groups that responded to reward with increased (Figure 2.8c)
or decreased (Figure 2.8d) phasic response and plotted the mean activity during the
three epochs analyzed surrounding reward. We found that both groups of neurons
responded significantly to reward in both epochs.
Since neurons responded both to rewards associated with task performance and
after delivery of uncued rewards, we asked whether the probability or magnitude
50
of responses to reward varied across contexts. We speculated that phasic neuronal
responses to reward might vary with the informational demands of each task context.
To test this idea, we first compared the proportions of neurons that responded to
reward in either post-reward epoch (Choice task, 44.5% responded; Operant, 53.3%;
Pavlovian, 39.6%; Reward, 32.3%) across the four contexts using pairwise t-tests to
look for significant differences between each pair of contexts (i.e. Choice vs Operant,
Choice vs Pavlovian, etc; one-way ANOVA including all four groups n.s., F=1.4, all
post-hoc t-tests n.s. corrected for multiple comparisons, Tukey HSD). None of these
comparisons revealed significant differences across these four contexts.
2.3 Discussion
Our results demonstrate that CGp neurons respond phasically to informative events
in a variety of contexts, independent of sensory stimulation, active choice, or overt
orienting. Consistent with previous reports (Dean et al., 2004; C. Olson et al.,
1996), the spatial sensitivity of these phasic responses was strongest immediately
following saccade initiation. Most notably, CGp neurons responded to uncued re-
wards outside of task context as well as to rewards delivered to motivate behavior
(contra (C. Olson et al., 1996) but see (C. R. Olson et al., 1993)). We cautiously
interpret this observation as suggesting that phasic CGp responses may reflect the
occurrence of informationally-significant events independent of their sensory or mo-
tor provenance. If so, CGp activity may signal the dynamic allocation of neural
processing to information in the ongoing processing stream that links the internal
and external environments.
Some learning theories have emphasized the importance of the vividness, or
salience, of informative events for drawing attention and inducing learning (Hall,
1994). Strongly predictive conditioned stimuli provide information relevant to be-
havior: in a decision context, known reward associations contribute to choice forma-
51
tion; in an operant context, conditioned stimuli prompt behavioral responses; and in
Pavlovian contexts, conditioned stimuli predict future events. Previously published
data showed that responses of CGp neurons to conditioned stimuli approximately
parallel behavioral indices of learning (Gabriel & Orona, 1982; A. McCoy & Platt,
2005b), suggesting that these neurons may bind motivational information to external
events (Freeman, Cuppernell, Flannery, & Gabriel, 1996). Our observation that CGp
neurons respond to informative events in a variety of contexts (decision, operant and
Pavlovian), support the hypothesis that these neurons monitor the environment for
behaviorally important information.
In classic learning models, stimulus-response relationships that are fully predicted
induce no new learning (Kamin, 1969; Rescorla & Wagner, 1972). Thus, unpre-
dictable stimuli contribute more to new learning than do predictable stimuli (Dick-
inson, 1980; Mackintosh, 1975; Pearce & Hall, 1980). This fundamental observation
is summarized as associative strength in classical conditioning or attentional theories
of learning, belief uncertainty in Bayesian updating models, and as a reward predic-
tion error in temporal difference learning algorithms (Courville, Daw, & Touretzky,
2006; Pearce & Hall, 1980; Rescorla & Wagner, 1972; Sutton & Barto, 1981; Sutton,
1988). Rare and unexpected events, such as the surprising rewards used in this study,
modulate both the allocation of attention and learning rate (Shannon, 1948; Pearce
& Hall, 1980; Posner et al., 1980; Einhauser, Mundhenk, Baldi, Koch, & Itti, 2007;
Itti & Baldi, 2008).
Such novel or changing stimuli activate neurons in the basolateral amygdala
(BLA), which may bind new task-relevant information to appropriate stimuli (Hol-
land & Gallagher, 2004; Pickens et al., 2003). Unlike CGp neurons, BLA neurons do
not maintain representations of stimulus-association encoding after learning (Got-
tfried, O’Doherty, & Dolan, 2002) and do not obviously encode reward valuation.
They do, however, reflect stimulus intensity and novelty (Gottfried et al., 2002;
52
J. P. O’Doherty, Deichmann, Critchley, & Dolan, 2002), consistent with a role in
the recognition and encoding of new information. Similarly, the phasic responses of
CGp neurons to unexpected reward and reward omission (A. N. McCoy et al., 2003)
suggest sensitivity to the informational content of events. In this view, CGp may
serve as a general-purpose learning device (Gabriel, Sparenborg, & Kubota, 1989)
that serves to allocate neural resources to the ongoing stream of information about
external stimulus events, rewards and punishers, decisions, and motor behavior. Fu-
ture functional studies using lesion or microstimulation techniques will be needed to
fully evaluate this hypothesis.
2.4 Materials and methods
2.4.1 Surgical and behavioral procedures
All procedures were approved by the Duke University Institutional Animal Care and
Use Committee and were designed and conducted in compliance with the Public
Health Services Guide for the Care and Use of Animals. Using standard surgical
techniques, a head restraint prosthesis, scleral search coil and small stainless steel
recording chamber were surgically implanted. The chamber was located over poste-
rior cingulate cortex, at the intersection of the midsagittal and interaural planes, and
was kept sterile with regular antibiotic washes and sealed with sterile caps (cf. (Dean
et al., 2004)). The animals received analgesics and antibiotics after all surgeries.
Monkeys indicated their choices and performed tasks with gaze shifts. Eye po-
sition was sampled via a scleral search coil at 500 Hz (Riverbend Instruments) and
recorded by computer (Gramalkn Experiment Control System, Ryklin Software).
Visual stimuli were LEDs presented on a large stimulus display panel (LEDtronics);
the LEDs were positioned one degree apart and fill 50 degrees of the visual field on
the horizontal axis and 40 degrees on the vertical axis. A computer-driven solenoid
was used to deliver liquid rewards whose magnitude was linearly correlated with the
53
open time of the solenoid.
2.4.2 Behavioral paradigms
We recorded from 91 single neurons in the posterior cingulate cortex of two male
rhesus macaques while they either performed one of three tasks (saccade color choice
trials, saccade single target trials and continuous fixation trials) or received reward
delivered at random, unpredictable times (reward only trials). In each task, red and
green peripheral target LEDs on an LED board (A. N. McCoy et al., 2003) were
used to indicate high or low reward delivery, and the color-reward associations were
reversed every 25-50 trials. Fluid reward delivery on successful trials was controlled
by solenoid open time as described previously (A. N. McCoy et al., 2003; A. McCoy &
Platt, 2005b; B. Y. Hayden & Platt, 2007), but within each task, failure to complete
a trial through peripheral target fixation was not rewarded (error trial).
2.4.3 Task contexts
Choice trials. Two target LEDs were placed diametrically around a central fixation
point. In each trial, one LED was red and the other green, but the color-location
association was varied randomly. One color was associated with a large reward on
every trial and the other with a small reward. The color-reward associations varied
at the end of every block of 25-50 trials. To start a trial, the monkey was required to
foveate a centrally located LED. Then the two peripheral targets were illuminated
and after a brief delay, the central fixation LED was turned off, cuing the monkey
to fixate on either of the two peripheral targets. After fixation, juice reward was
delivered via a computer-driven solenoid.
Operant trials. Each trial again began with foveation of a central illuminated LED.
A randomly selected red or green LED was illuminated in only a single target location
following central fixation. When the central LED turned off, the monkey fixated on
54
the target LED to receive a fluid reward. The LED colors again indicated high or
low reward size within a block of 25-50 trials, and color-reward associations were
varied at the end of every block; target location did not vary within a session.
Pavlovian (conditioning) trials. As in the saccade single target trials, monkeys fixated
on a central illuminated LED to begin the trial and trigger illumination of a single
red or green peripheral LED in the neuronal receptive field. However, the central
and target LEDs were turned off simultaneously, and the monkey was not required
to saccade to the target LED. Target LED color cued high or low reward within each
block and this association changed at the end of every block, but target location
remained constant within each session.
Uncued reward trials. The monkeys received unpredictable, irregular juice rewards
while sitting in a dimly lit room (backlighting at 12V for monkey BR and 5V for mon-
key NI) in the absence of any task or pertinent visual stimuli. All rewards were the
same size (solenoid open time 150 ms). Microelectrode recording techniques. Single
electrodes (Frederick Haer) were lowered by microdrive (Kopf) until the waveform of
a single neuron was isolated. Individual action potentials were identified in hardware
by time and amplitude criteria (BAK electronics). Neurons were selected on the basis
of the quality of isolation and visual or saccade-related modulation. We used a hand-
held digital ultrasound device (Sonosite 180) placed against the recording chamber
to confirm that recordings were made in areas 23 and 31 in the cingulate gyrus and
ventral bank of the cingulate sulcus, anterior to the intersection of the marginal and
horizontal rami. Analysis Custom Matlab functions were used to process and ana-
lyze neuronal activity and eye movements. Neuronal spike rates (hz) were calculated
during three 200ms epochs surrounding each event (cue illumination, saccade initi-
ation, reward delivery). The baseline firing rate for each event was defined as the
55
mean activity in the 200ms epoch preceding each event, and two 200ms post-event
epochs were defined immediately following each event (0-200ms, 200-400ms). Thus,
we analyzed neuronal activity in nine 200ms epochs for the choice and operant task
(cue, saccade, reward), six 200ms epochs for the Pavlovian task (cue, reward), and
three for the uncued reward task (reward). We used one-way ANOVAs and t-tests
to compare the firing rate in each post-event epoch to the baseline measured before
the relevant event (significance at p<0.05).
56
2.5 Figures and Tables
57
T2T1
T2T1
T2T1
T2T1
T2
Fixation onset
Target onset
Delay
Fix offset
Choice
Reward
FixTarget 1Target 2
EyeReward
T1T2
25 trials
Time
Example Reward Schedule25050 225
75 200100250
50
Time
Target Side:R,L,R,R,L,R,LL,R,L,L,R,L,R
Figure 2.1: Choice task: monkeys chose between two distinctly colored visualtargets. Within each block of 25-50 trials, each color (red, green) cued a single rewardsize but appeared randomly at the two symmetrically placed target locations. Toppanel, trial events; bottom panel, sample reward schedule.
58
T1
T1
T1
T1
Fixation onset
Target onset
Delay
Fix offset
Choice
Reward
FixTarget 1Target 2
EyeReward
25 trials
Time
Example Reward Schedule (T1):25050 225
75 200100250
50
Time
T1 Color:
T1
Figure 2.2: Operant (saccade) task: monkeys saccade to a single target locationto receive reward. Monkeys fixated the central cue to begin each trial. After asingle peripheral colored target was illuminated (one target location per session), themonkeys maintained fixation until the central cue dimmed. They received a reward(cued by color) after looking at the peripheral target.
59
T1
T1
T1
Fixation onset
Target onset
Delay
Target off
Reward
Trial end
FixTarget 1Target 2
EyeReward
25 trials
Time
Example Reward Schedule (T1):25050 225
75 200100250
50
Time
T1 Color:
Figure 2.3: Pavlovian (fixation) task: monkeys receive reward after continuouslyfixating a central cue during peripheral target presentation. Monkeys fixated a cen-tral cue to begin each trial, and maintained fixation while a peripheral colored targetwas illuminated. Once the peripheral target dimmed, monkeys received a reward,with magnitude cued by color, and the trial ended.
60
00.10.20.30.40.50.60.70.80.9
1
Prob
abilit
y of
cho
osin
g op
tion
40 60 80 100 120 140 160 180 200 220 240 260Reward size (ms solenoid open time)
Indifference
100100
175175
200200
Res
pons
e la
tenc
y (m
s)R
espo
nse
late
ncy
(ms)
125125
150150
Large reward trial
**
Small reward trial0
5
20
35
40
45
50
Mai
n se
quen
ce (
peak
Hz/
sacc
ade
leng
th)
10
15
25
30
ANOVA: F=13.3, p<0.01
Large reward trial
**
Small reward trial
ANOVA: F=19.0, p<0.01
A
B C
Figure 2.4: Monkeys distinguish large from small rewards. A. Choice behavior.Monkeys choose the large reward more often across all pairs of reward options, con-firming that they distinguished large and small rewards. B, C. Monkeys responsesdistinguish large from small rewards in the operant task. The slope of the line relatingpeak eye velocity to saccade amplitude (the main sequence) was inversely correlatedto reward magnitude (B). Additionally, the monkeys response latency increased asreward magnitude increased (C), confirming that monkeys were sensitive to rewardsize on single-target trials.
61
)zh( etar gniriF)zh( etar gniriF
0
5
10
15
20
TimeBR050309
Saccadeinitiation Reward
deliveryTargetonset
200ms
* *
0
1
2
3
4
5
Time
Saccadeinitiation
BR050310b
200ms
0
1
2
3
4
5
Time
Rewarddelivery
BR050310b
200ms
***Population: decreasing cells
0
1
2
3
4
56
Cue Saccade Reward
)zh( etar gnirif naeM
0
1
2
3
4
56
Cue Saccade Reward
Population: increasing cells
)zh( etar gn ir if n aeM
B C
A
E
D
Figure 2.5: CGp neurons respond to informationally relevant events in the choicetask. An example neuron that responds to target onset by sharply increasing firingrate is shown in A. The peri-stimulus time histogram (PSTH) is aligned to targetonset; arrows indicate target onset, saccade initiation and reward delivery. The lightgrey bar highlights the 200ms baseline epoch preceding target onset, and the two darkgrey bars show the two 200ms epochs analyzed following target onset. Similarly, Band C show a neuron that responded after saccades (B, aligned to saccade initiation)and rewards (C, aligned to reward delivery); light grey bars indicate baseline epochsbefore relevant events, and dark grey bars enclose post-movement and post-rewardepochs. Post-event epochs with firing that differs significantly from the relevantpre-event epoch are indicated with an asterisk (p<0.05). D. Population activity ofcells with increasing activity during either of the two epochs following each event; E,cells with decreasing activity. Light bars indicate baseline activity during the 200msepoch preceding each event; dark bars show activity during the two 200ms epochsfollowing each event. Asterisks indicate a significant difference between the relevantepoch and the relevant baseline (p<0.05). Error bars are standard error of the mean.
62
)zh( etar gnirif naeM
Cue Saccade Reward0
1
2
3
4
56
Population: increasing cells
Population: decreasing cells
Cue Saccade Reward01
2
3
4
67
5
Saccadeinitiation Reward
delivery
NI050623
200ms
*
0
5
10
15
Time
Rewarddelivery
Saccadeinitiation
NI050630
200ms
*
0
5
10
15
20
Time
Targetonset
NI050623
200ms
*)zh( etar gniriF
0
5
10
15
Time
Reward
Saccadeinitiation
NI050630
200ms
**
)zh( etar gniriF
0
5
10
15
20
Time
)zh( etar gn ir if n aeM
A
C D
B
F
E
Figure 2.6: Neurons respond to relevant events in the operant task. A, B. PSTHshowing the activity of a single neuron whose activity increased immediately fol-lowing target onset (A, aligned to stimulus onset) and eye movement (B, alignedto saccade); panels C and D illustrate the firing of a single neuron whose activitydecreased following both movement (C, aligned to saccade) and reward (D, alignedto reward). Arrows indicate salient events; light grey bars include the baseline epochpreceding an event, while dark grey bars indicate the two epochs analyzed followingan event; asterisks indicate that activity in a post-event epoch differed significantlyfrom the baseline activity preceding the relevant event (p<0.05). Neuronal activitysorted by response direction (E, increasing from baseline; F, decreasing). Light barsindicate baseline activity during the 200ms epoch preceding each event; dark barsshow activity during the two 200ms epochs following each event. Asterisks indicate asignificant difference between the relevant epoch and the relevant baseline (p<0.05).Error bars, S.E.M.
63
Time
)zh( etar gniri f naeM
Cue Saccade Reward0
1
2
3
4
56
Population: increasing cells
Population: decreasing cells
Cue Saccade Reward01
2
3
4
67
5
)zh( etar gniriF
0
2
4
6
8
Targetonset
Rewarddelivery
BR050601
200ms
*
)zh( etar gniriF
Rewarddelivery
Targetonset
BR050525a
200ms
**
0
10
5
Time
)zh( etar gn ir if n aeM
A
B D
C
Figure 2.7: CGp neurons respond to visual stimuli and reward in the Pavloviantask. The single neuron shown in A increases firing following target onset and reward(PSTH aligned to target onset), while the neuron shown in B decreases activity fol-lowing reward (aligned to reward). Light grey bar, 200ms baseline epoch; dark bars,post-event epochs. Asterisks indicate a significant difference between firing rate inan epoch and that of the immediately preceding baseline epoch. CGp population ac-tivity calculated after sorting individual neurons by response direction (C, increasingfrom baseline; D, decreasing). Light bars indicate baseline activity during the 200msepoch preceding each event; dark bars show activity during the two 200ms epochsfollowing each event. Asterisks indicate a significant difference between the relevantepoch and the relevant baseline (p<0.05). Error bars, S.E.M.
64
A
B D
)zh( etar gniriF
Time
Population: decreasing cells
Cue Saccade Reward0
1
2
3
4
5
)zh( etar gn ir if n aeM
**Rewarddelivery
BR050608b
200ms
)zh( etar gniriF
0
10
5
*
Rewarddelivery
BR050426
200ms
0
10
5
15
)zh( etar gnirif naeM
Cue Saccade Reward0
1
2
3
4
56
Population: increasing cells
C
Figure 2.8: CGp neurons report uncued rewards. Individual posterior cingulateneurons respond to reward delivery. A. PSTH for a single CGp neuron whose ac-tivity increased significantly following reward delivery. B. PSTH for a neuron whoseactivity decreased after reward. Light grey bar, 200ms baseline epoch; dark bars,post-event epochs. Asterisks indicate significantly different neuronal activity relativeto the baseline epoch. C, D: Posterior cingulate population responses to reward cal-culated from groups of neurons sorted by increasing (C) or decreasing (D) activityfollowing reward. Light grey bars indicate mean baseline frequency; dark grey, activ-ity during the early (0-200ms) and late (200-400ms) epochs after reward. Asterisksindicate a significant change relative to baseline. Error bars are S.E.M.
65
Tab
le2.
1:N
euro
nal
resp
onse
sto
even
tsac
ross
conte
xts
.W
eco
mpar
edneu
ronal
acti
vit
yduri
ng
two
200-
ms
epoch
s(e
arly
:0-
200m
s;la
te:
200-
400m
s)af
ter
each
even
t(c
ue,
sacc
ade,
rew
ard)
toa
200-
ms
bas
elin
eim
med
iate
lypre
cedin
gth
ere
leva
nt
even
t(i
.e.
-200
-0m
s).
Inea
chof
the
four
task
conte
xts
,neu
rons
resp
onded
toev
ents
by
bot
hin
crea
sing
and
dec
reas
ing
acti
vit
yre
lati
veto
bas
elin
e.N
um
ber
sin
par
enth
eses
indic
ate
tota
lnum
ber
ofneu
rons
inea
chta
sk;
box
esle
ftbla
nk
reflec
tin
applica
ble
anal
yse
s,e.
g.th
eab
sence
ofsa
ccad
ein
the
Pav
lovia
nta
sk.
Choi
ce(3
8)O
per
ant
(30)
Pav
lovia
n(4
8)R
ewar
d(6
8)In
crea
seD
ecre
ase
Incr
ease
Dec
reas
eIn
crea
seD
ecre
ase
Incr
ease
Dec
reas
eE
arly
cue
42
21
21
Lat
ecu
e4
63
65
5E
arly
sacc
ade
86
47
Lat
esa
ccad
e10
59
5E
arly
rew
ard
23
05
75
715
Lat
ere
war
d6
104
1111
57
15
66
3
Decision heuristic signals in posterior cingulatecortex
In economics, decision making is often modeled as a policy of maximizing subjective
expected utility. By contrast, applying a heuristic, or rule of thumb, can mini-
mize the need for explicit utility calculations, thus reducing cognitive load. One
such heuristic, the WSLS strategy, renders decisions based on a simple comparison
between rewards received and a threshold. Previous studies have implicated dor-
solateral prefrontal cortex and anterior cingulate cortex, brain regions involved in
behavioral control, in the application of strategic heuristics to action selection. How-
ever, the role of heuristic coding in the neural transition from outcome valuation
to action selection remains obscure. To examine these influences, we recorded from
neurons in CGp, a brain area implicated in reward evaluation and risky decision
making. Monkeys repeatedly chose between a probabilistic reward and a guaranteed
option in two reward contexts. Although preferences varied with context, monkeys’
local choice patterns followed a relative-outcome-based decision rule analogous to the
classic WSLS strategy. Consistent with contextual effects on preferences in this task,
neuronal activity in CGp reflected the history of both rewards and choices; impor-
67
tantly, neuronal representation of choice bias also depended on context. Consistent
with heuristic encoding of rewards, neuronal activity differentiated above-threshold
“wins” from below-threshold “losses”. Finally, these signals predicted the likelihood
of staying with the same choice or exploring the alternative on the next trial, con-
cordant with the use of a WSLS strategy. These results support the hypothesis that
CGp contributes to a heuristic-based evaluation of rewards and strategic decision
making.
3.1 Introduction
Normative decision making models often assume that choosers compute the subjec-
tive expected utility of available options in a Bayesian fashion using complete knowl-
edge of the environment (Bernoulli, 1738/1954; Neumann & Morgenstern, 1944).
In this framework, decisions between alternatives are rendered by comparing their
relative subjective utilities, which are monotonic functions of their objective val-
ues. Alternatively, the cognitive demands of such decisions can be simplified by
the use of strategic rules of thumb, or heuristics (Simon, 1955; Gigerenzer, 2008).
One such strategy, Win-Stay-Lose-Shift (WSLS), compares experienced rewards to
a subjective threshold, or aspiration level, to choose between exploitation, or repeat-
ing choices, and exploration, or sampling alternatives. The uncertainty inherent in
adaptive decision making encourages strategies that select exploration or exploitation
based on integration of previous experience (Daw, O’Doherty, Dayan, Seymour, &
Dolan, 2006) or critical re-evaluation of options’ estimated subjective values (Bayer
& Glimcher, 2005; J. O’Doherty, Critchley, Deichmann, & Dolan, 2003; P. N. To-
bler, O’doherty, Dolan, & Schultz, 2006). Alternatively, when choice patterns depend
more strongly on recent history than on long-term integration, such strategies use
comparisons between recent reward and a normative reward threshold (Lee, Con-
roy, McGreevy, & Barraclough, 2004; Soltani, Lee, & Wang, 2006) as the relevant
68
decision variables. Although recent research has implicated dorsolateral prefrontal
cortex (DLPFC), ACC, and LIP in experience-based decision making (M. Platt &
Glimcher, 1999; Barraclough et al., 2004; Kennerley et al., 2006), the role of strategic
heuristics in the transformation of outcome valuation to action selection, both within
and across reward contexts, remains unclear.
One brain area that may contribute to heuristically-guided reward outcome evalu-
ation and decision formation is the posterior cingulate cortex (CGp). Anatomically,
CGp is connected with brain areas involved in cognitive control, including ACC;
action selection, such as the supplementary eye fields and LIP; and reward and mo-
tivation, such as the amygdala and caudate (Pandya et al., 1981; Morecraft et al.,
1993; A. McCoy & Platt, 2005a). Moreover, neuronal activity in CGp reflects re-
ward magnitude (B. Y. Hayden et al., 2008) and omission (A. N. McCoy et al., 2003),
tracks reward uncertainty (A. McCoy & Platt, 2005b), and predicts future choices in
a simple two-alternative task (B. Y. Hayden et al., 2008). Finally, microstimulation
in CGp promotes switching away from a risky gamble toward a safe alternative (B. Y.
Hayden et al., 2008).
We tested whether CGp activity reflected the use of a heuristic model of outcome
evaluation by recording the activity of neurons in this area while monkeys chose be-
tween a safe option and a risky gamble in two contexts. Across contexts, we found
that choices reliably followed a simple heuristic analogous to WSLS. Moreover, CGp
neurons encoded both choices and outcomes in a simplified scheme compatible with
this heuristic; their activity reflected choice bias within contexts but was rescaled
across contexts ACCording to the WSLS heuristic. Furthermore, across contexts
CGp neurons predicted future choices. Our results demonstrate that CGp activ-
ity reflects reward categorization and predicts decision probabilities in a manner
compatible with a simple heuristic encoding, thus implicating this area in outcome
evaluation and decision making guided by such strategies.
69
3.2 Materials and methods
3.2.1 Surgical and behavioral procedures.
All procedures were approved by the Duke University Institutional Animal Care and
Use Committee and were designed and conducted in compliance with the Public
Health Service’s Guide for the Care and Use of Animals. Using standard surgical
techniques, a head restraint prosthesis, scleral search coil and small stainless steel
recording chamber were surgically implanted. The chamber was located over poste-
rior cingulate cortex, at the intersection of the midsagittal and interaural planes, and
was kept sterile with regular antibiotic washes and sealed with sterile caps (cf. (Dean
et al., 2004). The animals received analgesics and antibiotics after all surgeries.
3.2.2 Behavioral tasks
Monkeys indicated their choices with gaze shifts. Eye position was sampled via a
scleral search coil at 500 Hz (Riverbend Instruments) or an infrared camera at 60
Hz (Arrington) and recorded by computer (Gramalkn Experiment Control System,
Ryklin Software). Visual stimuli were yellow circles (1◦; ±10◦fixation accuracy re-
quired) presented on a computer monitor positioned 50cm in front of the monkey. A
computer-driven solenoid was used to deliver liquid rewards.
During gambling trials, two targets were illuminated diametrically around a cen-
tral fixation circle (Figure 3.1a). One target was associated with a “safe” reward
outcome of a constant amount of juice on every trial, while choosing the other,
“risky,” target resulted in either a larger or smaller juice reward, each with proba-
bility 0.5. The locations of the safe and risky targets were varied every 25-50 trials.
The monkeys were free to select either target. In an initial set of behavioral ex-
periments, the sizes of the large and small rewards offered for the risky option, as
well as the size of the reward offered for the safe option, were varied independently.
70
For electrophysiological recordings, we then selected one reward context in which
monkeys consistently preferred the risky option (“Equal Values” Condition: safe,
143 uL juice; risky, 43 or 243 uL), and one context in which monkeys preferred the
safe option (“Biased Values” Condition: safe, 213 uL; risky, 43 or 243 uL). In each
recording session, we randomly varied both the reward context and the locations of
the safe and risky targets every 25-50 trials. Only recording sessions with a minimum
of four blocks — fully counterbalanced, with two for each reward context and two
for each location assignment — were included in analysis.
3.2.3 Microelectrode recording techniques.
Single electrodes (Frederick Haer) were lowered by microdrive (Kopf) until the wave-
form of a single neuron was isolated. Individual action potentials were identified in
hardware by time and amplitude criteria (BAK electronics). Neurons were selected
on the basis of the quality of isolation and visual or saccade-related modulation.
We used a hand-held digital ultrasound device (Sonosite 180) placed against the
recording chamber to confirm that recordings were made in areas 23 and 31 in the
cingulate gyrus and ventral bank of the cingulate sulcus, anterior to the intersection
of the marginal and horizontal rami.
3.2.4 Analysis
Data were analyzed off-line using custom Matlab (The MathWorks, Inc.) scripts to
compute saccade direction, amplitude, latency, and peak velocity. Custom Matlab
and R scripts were used to complete additional data processing. Statistics were
computed using Matlab or Statistica. For behavioral data, we calculated the mean
frequency of risky choice per day for each set of available rewards and used multiple
regression and ANOVAs to quantify the relationship between available reward and
the probability of choosing the risky target. To further explore the local influences
71
on decisions, we fit each monkey’s behavior using a model (Lau & Glimcher, 2005)
that incorporates experienced reinforcement, previous choices, and contextual bias:
log(pR/pS) =∑j=1:n
(αj(rjR − rj
S)) + γ
where pR= probability of a risky choice and (pS probability of a safe choice. The
right-hand side of the equation comprises three pieces: a weighted sum over past
trials of reward outcomes ( rjR = reward received from a risky choice on trial j,
rjS = reward received from a safe choice on trial j); a weighted sum over previous
choices (cjR = 1 for a risky choice, 0 otherwise; cjS = 1 for a safe choice, 0 otherwise);
and an overall bias toward or away from the risky option (γ). We used Akaike’s
Information Criterion (AIC) to compare the explanatory power of models including
a moving window of one to ten trial histories prior to the current trial, and retained
the window size that produced the best fit (lowest AIC value).
We measured firing rates for each trial during three 400-ms intervals, each aligned
to a specific event during the trial: (i) 0-400ms following illumination of the eccen-
tric targets (“post-target”); (ii) 0-400ms following saccade onset (“post-choice”); (iii)
50-450ms following reward onset (“post-reward”; excluding a 50ms window imme-
diately following reward onset to remove the solenoid artifact). We used ANOVAS
to determine the relationships between neuronal firing rates and contextual choice
allocation as well as between neuronal activity and outcomes with respect to context.
Logistic regression was used to quantify the relationship between neuronal firing rate
and probability of repeating a choice, independent of reward.
3.3 Results
3.3.1 Monkeys use a threshold-comparative decision heuristic
Previous studies from our lab showed that monkeys are risk-seeking when offered a
choice between constant rewards and risky gambles with equivalent mean expected
72
values (A. McCoy & Platt, 2005b) but adapt to delayed reward availability in the
same task by more frequently choosing the safe option (B. Y. Hayden & Platt, 2007).
Thus, monkeys’ choices in this task are context-dependent. We therefore predicted
that increasing the value of the safe reward relative to the expected value of the risky
option would bias monkeys’ choices toward the safe option. To test this hypothesis,
we modified the payoffs of the gambling task (Figure 3.1a) by varying both the
risk (reward coefficient of variance) of the gamble and the magnitude of the safe
reward (Figure 3.1b). For both monkeys, there was a significant inverse correlation
between the ratio of the expected values of the two options (EVsafe/EVrisky) and the
likelihood of choosing the risky option (Linear regression, mean frequency of risky
choice per session and EV ratio vs EV ratio: monkey NI, R=0.59, p<0.01; monkey
BR, R=0.68, p<0.01).
Given this behavior, we reasoned that a model that reflected both choice his-
tory and reward experiences would accurately describe the monkeys’ decisions. To
quantify the effects of past reinforcement, preceding choices, and contextual bias on
decisions, we used a model in which choice probabilities depend on each of these
factors (Lau & Glimcher, 2005). Consistent with previous studies (A. McCoy &
Platt, 2005b; B. Y. Hayden et al., 2008), we found that monkeys’ choices were best
explained by their experiences on the most recent two trials (AIC1,2 <AIC3-10).
Although choices were influenced by both previous choice and reward history, recent
rewards influenced decisions more strongly than did prior choices (Table 3.1).
We therefore explored the relationship between reward outcomes and ensuing
decisions. The probability of repeating a choice was correlated with reward size
(Figure 3.1c); monkeys were most likely to repeat choices after receiving large re-
wards but sampled more frequently following small rewards, a pattern consistent with
the application of the “win-stay/lose-switch” heuristic. In this strategy, rewards are
classified as “wins” or “losses” relative to an internal threshold, or aspiration level (Si-
73
mon, 1955). To test this idea formally, we estimated a normative reward threshold
based on the reward size at which monkeys were equally likely to switch or stay with
the preceding choice (Figure 3.1d; 203 uL juice reward). We then categorized re-
wards greater than the 203 uL threshold as “wins” and designated rewards less than
this threshold as “losses”. Consistent with a WSLS heuristic, the monkeys repeated
choices significantly more often following wins (82.8 ± 2.8% repeats) than after losses
(39.1 ± 3.8% repeats; ANOVA, mean daily frequency of repeating a choice after a
win or loss vs previous outcome (win/loss), F=81.9, p<0.01).
Based on these findings, we identified two reward contexts for electrophysiological
recordings: in the first, the mean expected values of the two options were equal (Equal
value condition), but in the second, the safe option offered a larger reward than the
expected value of the risky option (Biased value condition). To confirm that the
monkeys’ behavior reflected the use of a WSLS strategy in these two contexts, we
determined the probability of repeating a choice following above-threshold rewards,
or wins, and below-threshold rewards, or losses (Figure 3.1e). As predicted by this
strategy, the monkeys almost always repeated choices after wins (EV: 90.0% ±1.5
(SEM) repeats, BV: 91.1% ±1.6) but were more likely to sample the alternative
option after losses (EV: 57.8%±4.2 repeats, BV: 51.5%±2.8; main effects ANOVA,
mean probability of repeating per session and outcome category vs reward context
and win/loss outcomes: reward context n.s., win/loss F=168.46, p<0.01).
3.3.2 Neuronal activity reflects contextual choice bias
Previous studies have shown that the activity of CGp neurons varies with decision
variables such as the subjective value of a target (A. McCoy & Platt, 2005b) or the
likelihood of choosing a particular option in the future (B. Y. Hayden & Platt, 2007).
Based on these observations, we hypothesized that neuronal activity in CGp would
also reflect context-dependent decisions arising from simple heuristics like WSLS. To
74
test this idea, we proposed that neuronal activity reflected the monkeys’ empirical
choice biases in each context. In addition, we hypothesized that CGp activity would
reflect a heuristically-defined categorization of reward outcomes as wins and losses.
We then reasoned that if CGp mediated heuristic decision making, neuronal activ-
ity would encode both reward outcomes and switch/stay decisions. Thus, neuronal
activity should differentiate the possible outcome/decision combinations. Further-
more, we predicted that if CGp contributed to the development of heuristic decisions,
neuronal activity would predict ongoing and future stay or switch decisions.
To test these predictions, we first assessed the relationship between neuronal
activity and the frequency of choosing risky and safe options across contexts. We
identified neurons whose activity reflected contextdependent choice frequencies by
a factorial ANOVA comparing firing rates across choice (risky vs safe) and context
(equal value vs biased value). Within the population of 38 neurons, 26 cells (68%)
had activity that was significantly modulated by choice allocation during at least one
epoch (post-target, post-choice, post-reward; Table 3.2). Figure 3.2 (A,B) shows the
activity of one such neuron following choice. Firing rate increased following choices of
the risky option in the Equal value context and following choices of the safe option
in the Biased value context (ANOVA: choice n.s., context n.s., interaction term
F=10.97, p<0.01), confirming that the activity of these neurons reflects choice bias.
Across the population (Figure 3.2 C, D, E), neuronal activity was also significantly
modulated by choice bias during the 400 ms following target onset (ANOVA, choice
n.s. (p=0.051), context n.s., interaction F=49.07, p<0.01), following choice (choice
F=4.67, p=0.03, context n.s., interaction F=32.34, p<0.01) and following reward
delivery (choice n.s., context n.s., interaction F=15.93, p<0.01).
If, as this result suggests, population activity in CGp mediates observed decision
behavior, firing rates should predict the reward ratio at which monkeys were indif-
ferent to the two options (Padoa-Schioppa & Assad, 2008). Thus, if neuronal firing
75
rates in the post-choice epoch are directly proportional to the frequency of select-
ing a given option, it should be possible to estimate behavioral indifference points
from firing rates (cf Figure 3.11b: Subject NI, EVsafe/EVrisky = 1.2; Subject BR,
EVsafe/EVrisky = 1.3). Consistent with our hypothesis, we found that post-decision
firing rates did predict the monkeys’ indifference point (EVsafe/EVrisky = 1.2).
Notably, although neuronal firing reflected relative choice bias an indicator of
both subjective expected utility and heuristic preference – within each reward con-
text, the neuronal representation of the risky option rescaled across contexts. In all
epochs, population activity was higher when the monkeys chose the risky option in
the equal values context than when they chose the risky option in the biased values
context. Thus, the representation of choice bias in CGp is relative and heuristic
rather than absolute and invariant (but see (Padoa-Schioppa & Assad, 2008)).
3.3.3 CGp activity reflects heuristically defined outcomes
Next, since the monkeys’ behavior was well described by a WSLS strategy, we pre-
dicted that neuronal activity would reflect the recent history of rewards categorized
as wins or losses by this heuristic. To test this idea, we analyzed the relationship
between neuronal firing and wins and losses across contexts. In this analysis, the
activity of the example neuron shown previously was significantly modulated by the
previous reward outcome across contexts (Figure 3.3 3a,b; ANOVA, firing rate vs
previous win/loss and context: win/loss F(1,133)=4.78, p=0.03, context n.s., inter-
action term n.s.). The activity of 26 neurons (68%) was significantly influenced by
prior wins and losses as defined by the heuristic threshold (Table 3.3). Population
activity (Figure 3.3c,d,e) also reflected previous wins and losses before (win/loss on
the previous trial F=21.00, p<0.01, context n.s., interaction n.s. (p=0.07)) and
after choice (win/loss on the previous trial F=12.49, p<0.01, context n.s., interac-
tion n.s.), and after reward delivery (win/loss on the current trial F=50.39, p<0.01,
76
context F=8.06, p<0.01, interaction n.s.). Crucially, this pattern of response to
wins and losses was independent of context, suggesting that CGp activity reflects
strategically-relevant variables derived from the underlying task parameters.
3.3.4 CGp activity distinguishes outcome/decision combinations
We next reasoned that if CGp neurons mediate the translation of outcomes into
heuristically-guided decisions, neuronal activity should distinguish the four possible
outcome/choice situations (win-stay, loss-stay, win-switch, loss-switch). We there-
fore sorted trials into four categories based on preceding outcome and ensuing choice
and used a one-way ANOVA to identify cells whose activity differed across these
four categories. This analysis showed that the activity of 17 (50%) of the neurons
studied, including the example neuron shown previously (Figure 3.4a; ANOVA, firing
rate vs categories, F=8.0, p<0.01), varied significantly across these four situations
(post-target epoch, 8 cells significant; post-choice epoch, 8 cells, post-reward epoch,
8 cells). Population activity also differentiated the outcome/choice combinations
(Figure 3.4b,c,d, ANOVA, firing rate vs categories: post-target, F=13.2, p<0.01;
post-saccade, F=5.3, p<0.01; post-reward, F=14.6, p<0.01). Notably, the response
patterns of the population changed between the early and post-reward epochs. Neu-
ronal activity following target onset (Figure 3.4b) and saccade (Figure 3.4c) differ-
entiated between relative wins and losses as well as between stay/switch decisions
(main effects ANOVA, firing rate vs previous trial’s win/loss outcome and current
trial’s switch/stay decisions: post-target effect of outcome, F=5.3, p<0.05, effect
of decision, F=19.4, p<0.01; post-saccade effect of outcome, F=5.0, p<0.05, effect
of decision, F=4.0, p<0.05), implying an integration of outcome and decision infor-
mation in these decision-related epochs. In contrast, neuronal activity immediately
following reward delivery (Figure 3.4d) differentiated outcomes but not future deci-
sions (main effects ANOVA, reward epoch firing rate vs the current trial’s outcome
77
and the next trial’s decision: effect of outcome, F=33.7, p<0.01; effect of decision
n.s.), suggesting that CGp neurons encode outcomes but not decisions immediately
following reward. Thus, CGp activity appears to shift from outcome encoding in
the reward epoch to integrated outcome/decision encoding in the decision epochs.
This temporal progression suggests that CGp neurons assess outcomes immediately
following reward and translate these outcomes into decision biases, raising the pos-
sibility that neuronal activity during the decision epochs though not during the
reward epoch – should predict decisions independent of reward outcomes.
3.3.5 CGp activity predicts the probability of repeating a choice
We therefore asked whether neuronal activity predicted the probability of repeating
choices across contexts and across epochs. The post-sACCadic firing rate of the ex-
ample neuron shown previously predicted the likelihood of repeating a choice (same
example neuron, Figure 3.5a; data fit with cumulative normal function, switch/stay
choice on the current trial vs firing rate: Wald stat 8.1, p<0.01). Since both neu-
ronal activity and future choices depended on reward outcome, we repeated this
analysis with previous reward included as an independent regression term to deter-
mine whether firing rate predicted future choices independent of reward outcome.
Even after controlling for the effects of reward outcome on choice (Wald stat 7.4,
p<0.01), the firing rate of the example neuron continued to significantly predict
decisions (Wald stat 5.3, p=0.02). Likewise, 22 of the neurons studied (58%) signif-
icantly predicted decisions during at least one of the epochs analyzed (Table 3.4);
of these, 15 (68%) predicted the probability of repeating a choice during the deci-
sion epochs even when previous outcome was included as a factor in the analysis.
Only 4 neurons (18%) predicted choice on the next trial independent of outcome in
the reward epochs, suggesting that neuronal encoding shifts from reflecting outcome
during the reward epoch to predicting decision during the decision epochs.
78
Similarly, we used logistic regression to assess whether the probability of repeating
a choice depended on population firing rate following target onset, choice and reward
delivery; again, since rewards influence later choices, we also included the most recent
outcome as a factor in this regression. CGp activity predicted the probability of
repeating a choice during the two decision epochs (Figure 3.5b,c,d; ANCOVA, switch
or stay on this trial vs firing rate and previous win/loss: post-target firing rate Wald
stat 20.3, p<0.01 and previous outcome Wald stat 1052.0, p<0.01; post-sACCadic
firing rate Wald stat 4.2, p=0.03 and previous reward Wald stat 1058.9, p<0.01)
but not after reward (ANCOVA, switch or stay on the next trial vs firing rate and
current trial’s win/loss: effect of post-reward firing rate Wald stat 0.96, n.s., and
reward Wald stat 771.8, p<0.01). Thus, CGp neuronal activity shifts from reflecting
outcome information immediately after reward to predicting decisions prior to choice.
These data support the hypothesis that CGp neurons contribute to the expression
of heuristically-guided behaviors by linking reward outcomes to decisions.
3.4 Discussion
Neuronal activity in CGp, as well as other brain areas ( (Leon & Shadlen, 1999;
M. Platt & Glimcher, 1999; Tremblay & Schultz, 1999; A. N. McCoy et al., 2003;
Wallis & Miller, 2003b; Cromwell, Hassani, & Schultz, 2005; Matsumoto & Hikosaka,
2008; Padoa-Schioppa & Assad, 2008), varies with rewards received and subjective
biases for a particular option (A. McCoy & Platt, 2005b; Kable & Glimcher, 2007;
B. Y. Hayden et al., 2008) in a particular context. Such observations are consistent
with the hypothesis, motivated by economic theory, that CGp encodes the subjective
utility of options under consideration or already chosen (A. McCoy & Platt, 2005b;
Kable & Glimcher, 2007). By contrast, applying a heuristic, or rule of thumb, can
minimize the need for explicit utility calculations, thus reducing cognitive load. We
found that the choices of monkeys across two contexts were consistent with a simple
79
win-stay-lose-shift heuristic, which renders decisions based on a simple comparison
between rewards received and a threshold (Simon, 1955). Monkeys more frequently
repeated a choice following receipt of supra-threshold rewards (wins) but tended to
explore the alternative option after subthreshold rewards (losses).
Categorizing reward magnitude relative to a threshold aspiration level ameliorates
the difficulty of explicit analytic decisions required for classical utility maximization.
In addition, this heuristic simplification allows ACCommodation across a large range
of rewards by rescaling neuronal value representations within a finite range of firing
rates. Macaques, like humans, may benefit from simplified computations that reduce
cognitive load during decision making (Simon, 1955). Heuristic use explains behav-
iors such as loss aversion in humans (Camerer, 2000) and capuchins (Chen, Lakshmi-
narayanan, & Santos, 2006) that would be considered irrational under classical utility
assumptions. In such cases, decision-making agents simplify reward assessment by
comparing gains and losses to a contextually determined reference point, or status
quo (Kahneman & Tversky, 1979). Similarly, in iterative Prisoner’s Dilemma games,
most humans use the aptly named WSLS heuristic to choose between cooperative
behavior and defections (Wedekind & Milinski, 1996). Although similar threshold-
comparative rules may apply across contexts, heuristic selection is sensitive to small
environmental changes. Monkeys playing an inspection game against a computer
component whose algorithm depended only on the monkeys’ previous choices used a
WSLS strategy (Barraclough et al., 2004). However, when the computer’s strategy
incorporated both reward and choice information, effectively neutralizing the benefit
of a WSLS strategy, the monkeys optimized their decision strategy by abandoning
WSLS in favor of random, unpredictable choice allocation. The fact that monkeys
relied on simple biases except when confronted with a savvy opponent suggests that
cognitive load reduction may be a fundamental goal of decision making–at least in
monkeys. Such heuristics may also prove important for understanding the neuronal
80
circuits mediating decisions. Previous studies indicated that neuronal activity in
posterior cingulate cortex varied with contextual biases (A. McCoy & Platt, 2005b)
and reward outcomes (A. McCoy & Platt, 2005b; B. Y. Hayden et al., 2008), and
predicted future decisions (B. Y. Hayden et al., 2008). Importantly, our observation
that posterior cingulate’s representation of the risky option rescales across contexts
indicates that CGp represents relative, heuristic biases rather than monotonically
encoding absolute subjective values. Our results also demonstrate that CGp signals
heuristically defined win/loss outcomes. In addition, neuronal activity distinguished
WSLS outcome/decision combinations and predicted future WSLS decisions. These
results suggest that CGp activity not only reflects variables important for updating
strategic decision calculations but also contributes to the outputs of internal heuristic
analyses.
The temporal evolution of CGp activity further supports the hypothesis that CGp
contributes to heuristic decision making by translating outcomes into decisions. The
transition from outcome-only to integrated outcome/decision encoding suggests that
CGp contributes to future strategic decisions. This hypothesis is supported by the
observation that microstimulation in CGp concurrent with reward delivery increases
monkeys’ tendency to explore a less frequency chosen option within a single context
(Hayden et al., 2008, (B. Y. Hayden et al., 2008)). Our observations suggest that
this effect is due to posterior cingulate’s contribution to action selection via heuristic
encoding of reward values: by categorizing rewards in a simple, context-dependent
menu, CGp condenses rich contextual information and transforms outcome valuation
for action selection via a computationally parsimonious decision rule.
Several other cortical areas have been implicated in outcome assessment, decision
formation or decision expression. Several of these regions, including DLPFC, LIP
and ACC, are reciprocally connected with CGp (Pandya et al., 1981; Vogt & Pandya,
1987; Morecraft et al., 1993). Neurons in DLPFC, like those in CGp, maintain out-
81
come information across trials and signal changes in behavior (Barraclough et al.,
2004). The application of prior reward information to repeated decisions appears
to depend on ACC activity: although monkeys with ACC lesions initially respond
to changes in reward contingencies, they fail to sustain these preferences (Kenner-
ley et al., 2006). This observation, in combination with the observation that ACC
activity reflects the value of actions rather than objects (Matsumoto et al., 2007),
suggests that ACC supports heuristic use across repeated decisions. These observa-
tions strongly suggest that ACC and CGp work together to evaluate reward outcomes
and form decision biases using simple heuristics such as WSLS.
82
3.5 Figures and Tables
Figure 3.1: Monkeys use a win-stay/lose-shift heuristic to make decisions acrosscontexts in a simple gambling task.
83
A. On each trial, a monkey initially fixated (1-2 deg) a central yellow LED (200-
800 ms). Two peripheral, diametrically opposed, centrally equidistant yellow LEDs
were then illuminated (200-800 ms). The fixation LED was then extinguished, cuing
a free choice via gaze shift to either target (1-2 deg) within 350 ms. Correct trials
were rewarded with juice and a 300 ms noise. Example Reward Schedule: the risk
and mean reward sizes for each target were varied orthogonally. In experiments
used to assess reward sensitivity, the range of reward differences for the risky target
was 0-303uL across blocks of trials, and the expected value difference between the
two options varied from 0 to 153 uL across blocks. B. The likelihood of a risky
choice decreases inversely to the relative value of the safe reward. We varied the
magnitude of the safe reward and variance of the risky rewards independently to
establish preferences across a range of options. Based on these observations, we
selected a subset of options with equal expected values at which the monkeys more
often choice the risky option (Equal value condition; blue circle) and a subset biased
toward the safe choice at which the monkeys more frequently chose the safe option
(Biased value condition; red circle). Lines shown are the best-fit lines for each monkey
(NI grey dashed; BR continuous black lines). Points represent the mean probability
of the risky choice across sessions, and points and lines are staggered for visibility;
whiskers represent 1 S.E.M. C. Choice allocation at a subset of target values results
in two reward contexts with opposite choice biases. In the Equal value condition,
the average expected values of the two options are equal and monkeys choose the
risky option more frequently. In the Biased value condition, the safe reward size is
larger than the mean risky reward and monkeys more often select the safe option.
D. The probability of repeating a previous choice depends linearly on the previous
reward received; monkeys repeat choices more frequently following large rewards
but switch to the alternative after small rewards. The monkeys become indifferent
between switch and stay decisions at 203 uL juice reward E. Monkeys’ choices reflect
84
a win-stay-lose-shift strategy. Following supra-threshold rewards, or wins, monkeys
repeated approximately 90% of choices. In contrast, monkeys were roughly indifferent
between switching and staying following sub-threshold rewards, or losses.
85
Figure 3.2: A,B: Peri-stimulus time histograms for a single CGp neuron aligned tosaccade onset. Firing rates were largest in each context when monkeys selected theirpreferred option (A: Equal values; B: Biased values). The grey block indicates the400 ms window following saccade initiation. C-E: Population responses also signalcontextual choice bias. When the two options have equal expected values and therisky choice is chosen more often, posterior cingulate activity is higher when monkeysselect a risky option (Equal value context; C, 0-400ms after target onset, post-hocFisher’s LSD, p<0.05; D, 0-400 ms after saccade initiation, p<0.01; E, 50-450 msafter reward onset, p=0.06). In contrast, posterior cingulate activity is relativelyincreased following a safe choice when available average rewards are unequal (Biasedvalue context) and the safe choice is preferred (after target, p<0.01; after saccade,p<0.01; after reward, p<0.01).
86
Figure 3.3: CGp neurons signal heuristically-assessed outcomes across contexts.A, B: Peri-stimulus time histograms for a single CGp neuron aligned to saccadeonset demonstrate that the firing rates of the neuron were largest in each contexton trials following receipt of wins (A: Equal values; B: Biased values). The greyblock indicates the 400 ms window analyzed following saccade initiation. C, D, E:Population firing rates following wins exceed those following losses in both the equaland biased value contexts (C, 0-400ms after target onset, ANOVA firing rate vsmost recent previous reward p<0.01; D, 0-400 ms after saccade initiation, p<0.01;E, 50-450 ms after reward delivery, p<0.01).
87
Figure 3.4: CGp neurons signal outcome/decision combinations in the decisionepochs. In the simple WSLS behavior described in this paper, the two possible out-comes and two possible decisions produce four possible outcome/decision situations(lose/switch, win/switch, lose/stay, win/stay). A. The example neuron shown previ-ously differentiated these four outcome/decision situations after choice and respondedmost strongly to wins and on trials on which the monkey chose to return to the pre-viously chosen option. B, C, D: During the two epochs surrounding choice indication(B, C) and the post-reward epoch (D), CGp population activity distinguished thefour outcome/decision combinations. Notably, CGp activity distinguished both winsfrom losses and switch decisions from stay decisions during the decision epochs, butonly differentiated wins from losses in the reward epoch.
88
Figure 3.5: CGp neurons signal the probability of repeating choices across contexts.A. The activity of the example neuron shown in this plot (the same neuron shownin Fig 2, 0-400ms following saccade initiation) significantly predicted the probabilitythat, on the current trial, the monkey repeated the preceding choice; increased firingrate predicted higher probabilities of staying with an option rather than switchingto the alternative. B, C, D: The CGp population response predicts the probabilityof repeating a choice during the decision epochs. After both target onset (B) andsaccade (C), CGp firing rate predicts the probability of repeating a choice on thecurrent trial (binomial regression, decision to repeat the previous choice vs firingrate). Similarly, CGp activity following reward delivery (C) predicts the probabilityof repeating a choice on the next trial (binomial regression, decision to repeat thecurrent trial’s choice on the next trial vs firing rate).
89
Tab
le3.
1:M
onke
ys’
dec
isio
ns
inth
ega
mbling
task
are
mot
ivat
edby
inte
grat
ion
ofsh
ort
rew
ard
and
choi
cehis
tori
es.
We
fit
each
mon
keys’
choi
ces
wit
ha
model
that
incl
uded
pre
vio
us
rew
ards,
pre
vio
us
choi
ces,
and
conte
xtu
albia
s,an
duse
dA
kaik
e’s
Info
rmat
ion
Cri
teri
ato
iden
tify
the
his
tory
lengt
h(k
;num
ber
ofpre
vio
us
tria
ls)
wit
hth
egr
eate
stex
pla
nat
ory
pow
er.
To
adju
stfo
rth
eeff
ect
ofre
war
dm
agnit
ude
onch
oice
,w
em
ult
iplied
each
rew
ard
coeffi
cien
tby
the
mea
nex
pec
ted
valu
eof
the
risk
ych
oice
and
rep
ort
the
adju
sted
valu
esher
e.
Equal
Valu
eC
ondit
ion
Bia
sed
Valu
eC
on
dit
ion
(BR
,N
I,k=
2)(B
R,
k=
2;N
I,k=
1)M
onkey
BR
:P
revio
us
tria
lP
enult
imate
tria
lP
revio
us
tria
lP
enult
imate
tria
lR
ewar
dco
effici
ent
0.70
760.
4508
1.07
680.
1592
Choi
ceco
effici
ent
-0.1
964
0.27
900.
0068
0.83
96B
ias
1.07
07-0
.164
6M
onkey
NI:
Pre
vio
us
tria
lP
enult
imate
tria
lP
revio
us
tria
lP
enult
imate
tria
lR
ewar
dco
effici
ent
1.32
860.
7551
1.19
20C
hoi
ceco
effici
ent
-1.2
108
0.06
430.
0601
Bia
s0.
9193
-0.4
034
90
Tab
le3.
2:C
Gp
neu
ronal
acti
vit
yis
modula
ted
by
conte
xtu
albia
ses.
We
use
da
fact
oria
lA
NO
VA
toco
mpar
eth
eeff
ect
ofch
oice
and
conte
xt
onneu
ronal
acti
vit
yduri
ng
the
400m
sep
och
sfo
llow
ing
targ
etap
pea
rance
,sa
ccad
ein
itia
tion
and
rew
ard
del
iver
y.A
sign
ifica
nt
inte
ract
ion
term
(p<
0.05
)in
dic
ated
that
ace
ll’s
acti
vit
yw
assi
gnifi
cantl
yco
rrel
ated
wit
hco
nte
xtu
albia
sac
ross
conte
xts
.C
onsi
sten
tw
ith
pre
vio
us
studie
s,w
efo
und
that
neu
ronal
acti
vit
yw
asei
ther
pos
itiv
ely
orneg
ativ
ely
corr
elat
edw
ith
choi
cebia
s.
Post
-targ
et
Post
-sacc
ade
Post
-rew
ard
Num
ber
of
posi
tively
corr
ela
ted
cell
s
Ris
ky
choic
e2
65
Conte
xt
810
11In
tera
ctio
n(w
ith
bia
s)4
136
Num
ber
of
negati
vely
corr
ela
ted
cell
s
Ris
ky
choic
e4
410
Conte
xt
57
6In
tera
ctio
n(w
ith
bia
s)4
56
91
Tab
le3.
3:C
Gp
neu
rons
trac
kpre
vio
us
win
san
dlo
sses
acro
ssco
nte
xts
.T
oas
sess
the
contr
ibuti
onof
heu
rist
ical
lyca
tego
rize
dou
tcom
esto
pos
teri
orci
ngu
late
acti
vit
y,w
eas
sess
edth
ere
lati
onsh
ipb
etw
een
neu
ronal
firi
ng
and
pre
vio
us
outc
omes
(AN
OV
A,
firi
ng
rate
vs
win
/los
sou
tcom
e).
Neu
rons
whos
eac
tivit
ydep
ended
sign
ifica
ntl
yon
pre
vio
us
outc
omes
(p<
0.05
)w
ere
cate
gori
zed
asp
osit
ivel
y(i
ncr
easi
ng
acti
vit
yfo
llow
ing
win
sre
lati
veto
loss
es)
orneg
ativ
ely
(hig
her
firi
ng
rate
for
loss
esth
anw
ins)
corr
elat
edw
ith
outc
omes
.
Post
-targ
et
Post
-sacc
ade
Post
-rew
ard
Num
ber
of
posi
tively
corr
ela
ted
cell
s5
311
Num
ber
of
negati
vely
corr
ela
ted
cell
s4
310
92
Tab
le3.
4:C
Gp
neu
ronal
acti
vit
ypre
dic
tsth
epro
bab
ilit
yof
rep
eati
ng
ach
oice
acro
ssco
nte
xts
.T
oan
alyze
the
contr
ibuti
onof
neu
ronal
acti
vit
yto
swit
ch/s
tay
dec
isio
ns,
we
com
par
edth
epro
bab
ilit
yof
rep
eati
ng
ach
oice
toneu
ronal
firi
ng
rate
for
each
neu
ron
(bin
omia
lre
gres
sion
,pro
bab
ilit
yof
rep
eat
vs
firi
ng
rate
).N
euro
ns
whos
eac
tivit
ysi
gnifi
cantl
ypre
dic
ted
the
pro
bab
ilit
yof
rep
eati
ng
adec
isio
n(p<
0.05
)w
ere
furt
her
cate
gori
zed
by
the
rela
tive
stre
ngt
hof
thei
rre
spon
ses
onsw
itch
orst
aytr
ials
.
Post
-targ
et
Post
-sacc
ade
Post
-rew
ard
Sta
y>
swit
ch5
311
Sta
y<
swit
ch4
310
93
4
Serotonin shapes risky decision making in monkeys
Some people love taking risks, while others avoid gambles at all costs. The neural
mechanisms underlying individual variation in preference for risky or certain out-
comes, however, remain poorly understood. Although behavioral pathologies associ-
ated with compulsive gambling, addiction, and other psychiatric disorders implicate
deficient serotonin signalling in pathological decision making, there is little experi-
mental evidence demonstrating a link between serotonin and risky decision making,
in part due to the lack of a good animal model. We used dietary RTD to acutely
lower brain serotonin in three macaques performing a simple gambling task for fluid
rewards. To confirm the efficacy of RTD experiments, we measured total plasma
tryptophan using high performance liquid chromatography (HPLC) with electro-
chemical detection. Reducing brain serotonin synthesis decreased preference for the
safe option in a gambling task. Moreover, lowering brain serotonin function signifi-
cantly decreased the premium required for monkeys to switch their preference to the
risky option, suggesting that diminished serotonin signalling enhances the relative
subjective value of the risky option. These results implicate serotonin in risk-sensitive
decision making and, further, suggest pharmacological therapies for treating patho-
94
logical risk preferences in disorders such as problem gambling and addiction.
4.1 Introduction
Making adaptive decisions requires evaluation of the costs and benefits of available
options as well as their uncertainty or risk. In economics, risk is typically defined
as known probabilistic variation in the distribution of outcomes (Sharpe, 1964;
E. Weber et al., 2004). Economists and psychologists have long known that risk
so defined strongly influences decisions made by humans and nonhuman animals
(Bernoulli, 1738/1954; Neumann & Morgenstern, 1944; Kahneman & Tversky, 1979).
Typically, individuals prefer safe bets to risky gambles. However, these preferences
can reverse depending on financial status (Guiso & Paiella, 2007), physiological
state (Caraco, 1981), and contextual features of the task such as the delay between
rewards (Rachlin, 2000; B. Y. Hayden & Platt, 2007), reward options available
(Rottenstreich & Hsee, 2001; Bateson, 2002; Dickhaut et al., 2003), and whether
the choice is between probabilistic losses or gains (Kahneman & Tversky, 1979).
Importantly, the same individual can be risk-seeking in one context and risk-averse
in another, like the compulsive gambler who insures his car (Friedman & Savage,
1948).
The neural correlates of probabilistic reinforcement and risk-sensitive decision
making have been probed in a number of recent neurophysiological and neuroimaging
studies. For example, activation of prefrontal and parietal cortex is correlated with
risk (Knutson & Cooper, 2005; Huettel, Stowe, Gordon, Warner, & Platt, 2006),
and neurons in posterior cingulate cortex have been shown to signal the subjective
utility of risky options (A. McCoy & Platt, 2005b). Building on these correlative
findings, brief disruption of neural processing within the right dorsolateral prefrontal
cortex by transcranial magnetic stimulation increases risk-taking behavior in a simple
gambling task (Knoch et al., 2006), thus directly implicating this structure in the
95
control processes mediating risk-sensitive decision making.
Individual differences in risk preference may result from neuromodulatory influ-
ences on these brain areas. Indeed, the impaired decision making found in multiple
psychiatric disorders, including addiction, attention-deficit hyperactivity disorder,
pathological gambling, schizophrenia, depression, and personality disorders, is some-
times characterized as increased preference for risk (Kapur & Remington, 1996; J.
Evenden, 1999a; B. J. Jones & Blackburn, 2002; Lyne, Kelly, & O’Connor, 2004;
Sodhi & Sanders-Bush, 2004; Nordin & Sjodin, 2006), and neuromodulators such
as dopamine and serotonin have been implicated in these disorders. In particular,
impaired function of the serotonergic neuromodulatory system is found in several
risk-seeking behavioral pathologies (J. Evenden, 1999b; B. J. Jones & Blackburn,
2002; Marek et al., 2003; Chau et al., 2004; Clarke et al., 2005; Svenningsson et al.,
2006; Jans et al., 2007) and genetic studies have implicated serotonergic dysfunction
in impulsive and risk-taking behavior (Nordstrom et al., 1994; Higley et al., 1996;
Gainetdinov et al., 1999; Chau et al., 2004; Kreek et al., 2005).
These observations invite the hypothesis that serotonin signaling contributes to
decision making by systematically altering perceptions of risk and reward or the
translation of those perceptions into decision and action. Previous studies have
tested the functional relationship between altered serotonin function and impulsiv-
ity (J. Evenden, 1999a; Winstanley, Dalley, et al., 2004), temporal discounting
(Schweighofer et al., 2008)), and reward cue processing (R. Rogers et al., 2003).
While these studies reflect multiple facets of risk (J. Evenden, 1999a; M. L. Platt
& Huettel, 2008), the functional relationship between serotonin and preferences for
economically defined risk, or uncertainty, has not yet been assessed. We therefore
designed our experiment to measure behavioral responses to risk by parameteriz-
ing economic uncertainty and value. Specifically, we probed the role of serotonin in
the processes mediating choices between risky and certain options by systematically
96
altering serotonin levels in three adult male rhesus macaques (Macaca mulatta) per-
forming a gambling task (Figure 1a). We used rapid tryptophan depletion (RTD),
a well-characterized dietary manipulation that rapidly, reliably, and reversibly low-
ers brain serotonin (Moja, Cipolla, Castoldi, & Tofanetti, 1989; Reilly, McTavish,
& Young, 1997; Klaassen, Riedel, Deutz, Someren, & Praag, 1999), to bypass the
complex, often poorly understood, interactions between serotonergic antagonists and
neurophysiological systems and induce a global deficit in serotonin synthesis. Since
RTD significantly lowers brain serotonin by dramatically reducing plasma, and there-
fore brain, tryptophan (Young, Ervin, Pihl, & Finn, 1989; Reilly et al., 1997), plasma
tryptophan serves as an index for brain serotonin levels after RTD (Tagliamonte,
Biggio, Vargiu, & Gessa, 1973; Young et al., 1989; Reilly et al., 1997). We modi-
fied this paradigm for use in awake, behaving monkeys and confirmed tryptophan
depletion within each subject by measuring total plasma tryptophan (Figure 1b).
The task we used was designed specifically to probe decision making in the pres-
ence of economic risk (A. McCoy & Platt, 2005b). In this task, modeled on a classic
foraging task (Kacelnik & Bateson, 1996), animals were offered a choice between
two options. The safe option offered a guaranteed juice reward while the risky op-
tion offered either a larger or smaller volume, selected randomly. This task, which
we dubbed a “gambling task”, allowed us to quantify both risk preference, defined
by preferences when the two options had equal expected value, and the shift in the
amount the monkeys will pay for the risky option, defined in economics as the “safety
premium” (Asch & Quandt, 1990). Overall, we found that lowering brain serotonin
decreased the monkeys’ likelihood of choosing the safe option and increased their val-
uation of the risky option. Our results demonstrate for the first time that serotonin
signaling functionally contributes to decision making under economic risk.
97
4.2 Methods
4.2.1 Surgical and training procedures
All procedures were approved by the Duke University Institutional Animal Care and
Use Committee and were designed and conducted in compliance with the Public
Health Service’s Guide for the Care and Use of Animals. Surgical and training
procedures were performed as described in detail elsewhere (A. N. McCoy et al.,
2003). The monkeys’ eye position was monitored with either a scleral search coil
(A. N. McCoy et al., 2003) or using a small infrared camera (Arrington Eyetracker)
placed in the monkey’s peripheral visual field and focused on the pupil. This camera
sampled horizontal and vertical eye position at 60Hz and transmitted this information
to the Gramalkn Experiment Control System (ryklinsoftware.com). Since the 60Hz
Eyetracker transmitted data at a slower rate than the 500Hz scleral search coil system
used by the other two subjects (monkeys BRand NI), the temporal parameters within
each task were modified to allow sufficient time for the system to recognize saccade
initiation and completion. However, we held the inter-trial interval constant at 1
second.
4.2.2 Behavioral paradigms
Visual stimulus presentation and eye position measurement were performed as de-
scribed previously (A. McCoy & Platt, 2005b). On each visual gambling trial, a
monkey initially fixated (± 1-2◦) a central yellow light-emitting diode (LED) (200-
800 ms). Two peripheral yellow LEDs were then illuminated diametrically opposite
each other, equidistant from the fixation LED (200-800 ms). The fixation LED was
extinguished, cuing the monkey to initiate a gaze shift to either target (± 1-2◦) within
350 ms. Correct trials were rewarded with juice and a 300 ms noise; no reward was
given if the monkeys failed to complete the trial. Juice rewards were controlLED by
98
a computer-driven solenoid that, when open, permitted juice flow through a tube to
the monkey’s mouth. Within the range of reward values used, solenoid open time
linearly predicts the volume of juice delivered (A. McCoy & Platt, 2005b). Within
each block of 25-50 trials, one target was associated with a a “safe” reward outcome
of a constant amount of juice on every trial, while the other “risky” target was ran-
domly rewarded with less or more than the certain amount of juice with probability
0.5. The locations of the safe and risky targets were varied every block. The sizes
of the large and small reward offered for the risky option, as well as the size of the
reward offered for the safe option, were varied orthogonally (Figure 4.1). The range
of reward differences for the risky target was 20-250ms (23-253uL) across blocks of
trials, and the expected value difference between the two options varied from 0 to
80 ms (0-83uL) across blocks. When there was no payoff difference between the
two options, risk (the coefficient of variation in the size of the large and small re-
ward offered by the risky option) was varied as previously described (A. McCoy &
Platt, 2005b). Similarly, for two monkeys, when risk was held constant, the payoff
difference between the average reward sizes of the safe and risky options was varied.
4.2.3 Rapid Tryptophan Depletion
For depletion (RTD) and control experiments, the monkey was fed a low-protein
diet (80% fruit and vegetables, 20% normal monkey biscuits) for 24 hours. The
monkey was then fed a mix (Supplementary Info., Appendix A) prepared no more
than 48 hours previously. Since previous research had already demonstrated that
maximal tryptophan depletion is reached within 3-6 hours following administration
of the depletion mix (Carpenter et al., 1998), we performed behavioral testing 3-6
hours after administration of the RTD mix. A subset of depletions were performed
by feeding the monkey a portion of the depletion mix on the evening preceding the
experiment, as a replacement for the monkey’s normal low-protein meal, and the
99
remainder of the mix 3-6 hours before behavioral testing was performed, to mitigate
satiation. Within one hour after either baseline or RTD behavioral testing, a 1-3 mL
venous blood sample was drawn under approved anesthetic criteria for analysis of
total plasma tryptophan levels. Immediately following the blood draw, the monkey
was given his normal daily supply of biscuits and fruit to quickly re-establish normal
tryptophan levels (Carpenter et al., 1998).
Plasma total tryptophan was measured by HPLC followed by electrochemical
detection using a modification of literature methods (Krstulovic et al., 1984). Sam-
ples were injected directly onto a C18 reverse-phase ODC column and eluted with a
mobile phase comprised of 0.5 M citric acid, 0.05 M Na2HPO4, 0.1 mM EDTA and
8% acetonitrile. Samples were detected with a BAS LC-4B amperometric detector
with a dual 3 mM carbon electrode at a potential of .85V vs. an Ag/AgCl reference
electrode. Samples were quantitated in comparison to external standards.
Total tryptophan has been shown to relate predictably to free plasma tryptophan:
if anything, this measure underestimates the degree of depletion for free plasma
tryptophan that is available for transport into the central nervous system (CNS)
(Moja et al., 1989). The degree of depletion observed in the monkeys was equivalent
to that reported in humans and nonhuman primates following this procedure (Moja
et al., 1988; Young et al., 1989; Klaassen et al., 1999). We additionally confirmed
successful depletion in a subset of experiments by measuring plasma tyrosine and
comparing the ratio of plasma tryptophan to plasma tyrosine (Supplementary Info.,
Appendix A).
4.2.4 Experimental schedule
Each monkey performed the gambling task under both depletion (RTD) and control
conditions. The monkeys were well-trained before data collection commenced. To
minimize the effects of order, experience or time, the order of experiments was pseu-
100
dorandomized, with no more than four consecutive repetitions of the same condition
permitted (e.g. Figure 4.1).
4.2.5 Analysis
Behavioral data were analyzed off-line; custom Matlab and R scripts were used for
data processing, including sorting reward and risk variables (Matlab script, T. Han-
son, Duke University) and computing saccade direction, amplitude, latency, and
peak velocity. Monkeys performed from 102–443 trials per day (mean=256 trials:
monkey BR, mean=278 trials; monkey SH, mean=195 trials; monkey NI, mean=432
trials; blocks in which the monkey failed to sample both targets were excluded from
analysis.). Thus, although performing studies in primates precludes study of a large
number of subjects, each subject performed a large number of trials across repeated
experiments (Total trials: Monkey BR, 3573; Monkey NI, 2466; Monkey SH, 1964).
Such repeated experiments in a small number of subjects can provide a reliable pop-
ulation estimate of behavior.
To analyze the behavioral changes induced by RTD, we calculated the mean
frequency of choosing the safe choice for each monkey, experimental session, and
experimental context (e.g. reward, depletion) and determined the effect of RTD on
the probability of choosing the safe target using Analysis of Variance (ANOVA); we
also calculated the temporal window over which monkeys integrated the history of
rewards (Supplementary Methods, Appendix A). In addition, response latency and
peak saccade velocity provide metrics useful for testing the possibility that lower-
ing serotonin levels affects motor control or attention (Baumgarten & Grozdanovic,
1995; Gobbi, Murphy, Lesch, & Blier, 2001; K. Watanabe, Lauwereyns, & Hikosaka,
2003 Oct). Notably, since saccade velocity increases linearly with saccade amplitude,
eye velocity is commonly normalized with saccade amplitude, a measure known as
the “main sequence” (Bahill, Clark, & Stark, 1975). The sensitivity of these markers
101
necessitates their use on a trial-by-trial basis rather than as daily means. Thus, we
used all choices to analyze these aspects of behavior and, to confirm that behavioral
changes reflected serotonin depletion rather than changes in attention or motor per-
formance, additionally included response latency and peak eye velocity relative to
saccade magnitude (the “main sequence”) as co-factors in logistic regression analyses
(A. McCoy & Platt, 2005b). Statistics were computed using Statistica (StatSoft).
4.3 Results
We first asked whether low serotonin levels influenced monkeys’ preferences as re-
vealed by a simple risky decision making task in which monkeys prefer the risky
option when choices with equal expected value are presented rapidly (every 1-3 sec)
(A. McCoy & Platt, 2005b; B. Y. Hayden & Platt, 2007). Based on prior results,
we held the inter-trial interval constant at 1 second, a range in which monkeys were
previously risk-seeking, and assessed preferences in monkeys in two different contexts
designed to elicit different choice likelihoods: one in which the two options had the
same value and another in which the safe option had a higher value than the risky
option. In the first condition, monkeys chose between safe and risky options with
equal expected value and were thus able to choose either option without forfeiting
long-term average reward intake. In the second context, the monkeys forfeited re-
ward by choosing the risky option. Thus, orthogonalizing risk and payoff differences
permitted quantification of the effects of RTD on risk preferences and reward valu-
ation in two different contexts. At baseline, the combination of these two contexts
resulted in an apparent overall preference for the safe option, though monkeys clearly
chose the risky option more frequently when both options were matched for expected
value.
We confirmed successful serotonin depletion in monkeys by measuring levels of
plasma tryptophan, a reliable marker for brain serotonin (Tagliamonte et al., 1973;
102
Young et al., 1989; Reilly et al., 1997), in each macaque immediately following each
session. RTD effectively depleted plasma tryptophan in each of the three monkeys
and across all 3 monkeys considered together (Figure 4.2). In contrast, plasma
tryptophan levels did not decrease when monkeys were fed a balanced amino acid
mix that also contained tryptophan (Supplementary Figure A.1).
Overall, monkeys chose the safe option significantly less often following RTD
(Figure 4.3a). Since these experiments involved multiple monkeys and reward con-
texts, we used multiple linear regression to investigate the influence of subject iden-
tity and relative reward value on revealed preferences (Multiple regression, mean
daily frequency of safe choice vs depletion condition, subject, reward context, or-
der of experiments, consumption time, mix volume: Depletion condition regression
coefficient=0.19, p=0.03; Subject n.s.; Reward context regression coefficient=0.78,
p<0.01; order, coefficient=-0.22, p<0.01; consumption time n.s.; mix volume n.s.).
Importantly, the effect of RTD on choice frequency was significant independent of the
order of experiments, timing of mix consumption (morning vs evening, as described
in Methods), or liquid volume consumed with the mix (n.s.). Although baseline
preference for the safe option varied across individuals, each monkey chose the safe
target less frequently following RTD (Figure 4.3b). Furthermore, the monkeys chose
the safe option less frequently following RTD (Figure 4.3c) whether they initially
preferred the risky option (Equal expected value (EV) context) or the safe option
(Unequal EV context). Moreover, following consumption of the balanced amino acid
mix, monkeys’ preferences were similar to baseline preferences (Supplementary Fig-
ure A.2), indicating that the behavioral changes observed following RTD were due
to the physiological effects of tryptophan depletion rather than consumption of the
amino acid drink.
Since monkeys’ preferences were significantly modulated by decreasing serotonin,
we asked whether preferences varied continuously with plasma tryptophan, and by
103
extension, serotonin levels in the CNS. We used the mean probability of choos-
ing the safe option in each session to calculate the overall frequency of safe choices
and found a weak but significant linear correlation between the frequency of safe
choices and measured plasma tryptophan (y =0.39+0.0015x; y is the probability
of the safe choice, x is plasma tryptophan; r=0.40, p=0.03). The effect of plasma
tryptophan levels on preference for the safe option survived inclusion of individ-
ual and reward context in the model (tryptophan level regression coefficient=0.20,
p=0.01; subject regression coefficient=0.22, p<0.01, all subjects in the same direc-
tion; reward EVcontext regression coefficient=0.77, p<0.00001, both contexts in the
same direction). Although this observation suggests graded modulation of neural
systems mediating reward processing and decision making, the strong influence of
inter-individual variation and decision context prompts us to tender this conclusion
with caution.
One interpretation of these results is that that the subjective value of the risky
option increased (or the subjective value of the safe option decreased) following
serotonin depletion. Analysis of the monkeys’ safety premium (Asch & Quandt,
1990), or point of subjective equivalence (PSE) between the risky and safe options,
confirmed this supposition. To calculate the safety premium for each experimental
condition, we used regression to find the optimal fit describing the relationship be-
tween the probability of choosing the safe option and the difference between the two
options’ expected value (Figure 4.4). We found strong linear correlations between
choice likelihood (y) and the difference in EV (x = (EVsafe−EVrisky)) between safe
and risky options:
Baseline, y = 0.32 + 0.006x (r = 0.93, p < 0.001)
RTD,y = 0.26 + 0.005x (r = 0.91, p = 0.01)
Notably, both relationships have similar slope and differ primarily in their intercept.
Based on these analyses, we calculated that the solenoid open time at which monkeys
104
were indifferent to the two options (choice likelihood=0.5) increased by 60% when
tryptophan depleted (baseline, 30ms solenoid open time; RTD, 48 ms), a payoff
difference about double that which monkeys reliably discriminate when rewards are
delivered in the absence of risk (A. N. McCoy et al., 2003). We repeated this analysis
for each monkey (Supplementary Info, Appendix A) and found that each subject’s
safety premium increased following tryptophan depletion (Monkey SH, 60% in-
crease: baseline PSE=20ms, RTD PSE=32ms; Monkey NI, 39% increase: baseline
PSE=38ms, RTD PSE=53ms; Monkey BR, 100% increase: baseline PSE=26ms,
RTD PSE=52ms).
In addition to confirming that the monkeys value the safe option less (or the risky
option more) when tryptophan-depleted, this analysis suggests that RTD does not
negatively affect monkeys’ ability to discriminate relative reward sizes. To confirm
that low serotonin did not impair monkeys’ reward discrimination ability, we exam-
ined behavior in a simple reward discrimination task (A. N. McCoy et al., 2003).
We found no significant differences in performance in the baseline and depleted con-
ditions (Supplementary Figure A.4), suggesting that RTD does not affect reward
discrimination.
Serotonin might contribute to valuation of risky and safe options via opponency
with dopamine, a hypothesis that posits that serotonergic activity conveys a nega-
tive reward signal by inhibiting dopamine neurons (Daw, Kakade, & Dayan, 2002).
Although our experiment was not designed to specifically test this hypothesis with
negative rewards or aversive outcomes, we suspected that the rewards associated
with the safe or risky option might represent relative gains and losses compared to
the reference point of the mean expected reward. Previous observations show that,
consistent with this idea, monkeys were more likely to sample the safe option after
receiving a smaller than average reward (loss) than after a larger reward (gain) (A.
McCoy & Platt, 2005b). If serotonergic responses do report undesirable outcomes,
105
the decreased serotonin function resulting from RTD should preferentially diminish
the effects of losses, thus inducing the monkeys to choose the risky option more
frequently after receiving a small reward. To test this prediction, we analyzed the
effects of RTD on the likelihood of a safe choice on trials immediately following a
loss, a win, or a safe reward outcome (Figure 4.5). After RTD, the monkeys were
significantly less likely to choose the safe option on trials following such “losses” while
their preferences did not change significantly following “wins.” Although these results
do not definitively confirm the serotonin-dopamine hypothesis, reframing wins and
losses relative to the large reward suggests that RTD more strongly affects choices
following relative losses (safe reward and small risky reward; Baseline 61.5%±3.5%;
RTD 50.3%±3.1%; ANOVA; logistic regression, choice vs depletion, Wald stat 54.7,
p<0.000001) than following relative gains (large risky reward).
Alternatively, since serotonin also contributes to memory and impulsive behavior
(Riedel, Sobczak, & Schmitt, 2003; Winstanley, Dalley, et al., 2004), these effects
might reflect a shift in the temporal window over which the monkeys integrated the
history of prior rewards. To test this possibility, we determined the dependence of
monkeys’ choices on rewards received over the preceding ten trials (Supplementary
Methods, Appedix A). The monkeys’ choices were influenced by the preceding two
to three trials in both baseline (2.5±0.4 trials) and RTD (3.3±0.4) conditions (t-test
n.s.). Thus, at least in this well-learned task, RTD does not appear to strongly influ-
ence the monkeys’ memory or attention to the history of reward. Since low serotonin
levels can also influence motor control and attention (Baumgarten & Grozdanovic,
1995; Gobbi et al., 2001), we investigated whether RTD affected behavioral indices
of these processes. Although response latency appeared to decrease following RTD
(2963 baseline trials, mean latency 209.1±0.7 ms; 5070 RTD trials, mean latency
207.2±0.5 ms; one-way ANOVA F=5.7, p=0.17), these effects were very small and
largely driven by one monkey (Subject BR 1068 baseline trials, mean 198.3±0.9 ms,
106
2478 RTD trials, mean 198.5±0.9 ms, ANOVA F=0.02, n.s.; Subject NI 858 base-
line trials, mean 223.5±1.2 ms, 1608 RTD trials, mean 217.4±1.0 ms, ANOVA F
=14.23, p<0.01; Subject SH 1037 baseline trials, mean 208.4±1.1 ms, 984 RTD tri-
als, mean 212.3±1.2 ms, ANOVA F=5.5, p=0.019). However, peak saccade velocity
as a function of amplitude (the “main sequence”) increased significantly following
RTD (2935 baseline trials with mean 42.0±0.3, main sequence slope=42.1; 5054 RTD
trials with mean 48.5±0.3, main sequence slope=21.3, one-way ANOVA F=164.1,
p<0.00001). This oculomotor effect included a significant subject effect (ANOVA;
effect of RTD, F=66.2, p<0.00001; effect of subject, F=257.0, p<0.00001) although
all three monkeys showed an increase in velocity as a function of amplitude (subject
BR, 1067 baseline trials with mean 46.0±0.2, 2478 RTD trials with mean 52.6±0.5;
subject NI, 858 baseline trials with mean 45.2±0.3, 1608 RTD trials with mean
49.2±0.2; subject SH, 1010 baseline trials with mean 35.0±0.87, 968 RTD trials
with mean 36.7±0.90). Thus, movement velocity, a potential index of motivation,
increased following RTD—perhaps reflecting greater valuation of the potential large
rewards.
There was no overall effect of tryptophan depletion on the frequency of errors
(incomplete trials; 3087 baseline trials, mean error frequency 4.0%±0.3%; 5288 RTD
trials, mean error frequency 4.1%±0.2%; ANOVA F=6.9, p<0.01, subject effect
F=87.2, p<0.00001), though one monkey failed to complete slightly, but signifi-
cantly, more trials following serotonin depletion (subject NI, 879 baseline trials
with 2.4%±0.5% errors, 1696 RTD trials±0.5%, ANOVA F=11.2, p<0.001; subject
BR, 1076 baseline trials with 0.8%±0.2% errors, 2517 trials with 1.5%±0.2% errors,
ANOVA F=2.9, p=0.09; subject SH, 1132 baseline trials with mean 8.4%±0.8% er-
rors, 1075 RTD trials with 8.5%±0.8% errors, ANOVA F=0.004, n.s.). Furthermore,
RTD did not affect the frequency with which monkeys switched targets, a measure of
exploratory sampling behavior (ANOVA, daily mean sampling frequency vs depletion
107
condition, F=1.0, n.s., mean baseline sampling frequency±s.e.m. =25.5%±2.4%,
mean RTD sampling frequency 29.1%±2.5%).
Thus, the only consistent effect of RTD on metrics of motor performance and
attention was an increase in saccade velocity. Nonetheless, we further assessed the
contribution of tryptophan depletion and choice context on preferences by including
these metrics as factors in our analysis. To accurately reflect trial-by-trial variation in
saccade metrics, we coded each choice as a binomial variable (safe, risky) and assessed
factor effects across all correct trials using logistic regression. This analysis confirmed
the main result that tryptophan depletion significantly decreased preference for the
safe choice (Wald stat 38.7, p<0.00001), even in the presence of significant effects
of subject (Wald stat 34.1, p<0.00001), relative reward context (Wald stat 689.6,
p<0.00001) and peak velocity as a function of amplitude (Wald stat 4.7, p=0.03).
Response latency was not correlated with preference (Wald stat 2.9, p=0.09).
4.4 Discussion
Our results demonstrate for the first time that reducing brain serotonin synthesis
decreases preference for a safe reward option in monkeys. Furthermore, the mon-
keys’ safety premium (the amount monkeys would “pay” to choose the risky option)
increased with diminishing serotonin levels, confirming that decreased preference for
the safe option reflects a decrease in its subjective value relative to the risky op-
tion. This shift in preferences appears to reflect choices following relative losses, an
observation that supports serotonin’s putative role as indicator of negative rewards.
Lowering brain serotonin function also resulted in modest changes in attention
and motor control. RTD increased saccade velocity as a function of amplitude (the
“main sequence”) but did not influence error frequency or response latency. These
results suggest that low serotonin is associated with faster behavioral responses,
consistent with increased motivation, attention or temporal impulsivity (Scholes et
108
al., 2007). Importantly, the effects of reduced serotonergic activity on preferences
were independent of changes in behavioral metrics of attention or motor control,
confirming the main effect that lowering serotonin function systematically decreases
preference for the safe option.
While this paper demonstrates behavioral modulation following successful tryp-
tophan depletion in awake, behaving monkeys, several caveats are warranted. To
ensure the monkeys’ comfort while maintaining consistent amino acid intake, we
varied the amount of water used in the RTD mix. Consistent with previous exper-
iments (A. McCoy & Platt, 2005b), we found that monkeys’ preferences did not
depend on the volume of fluid intake. Similarly, the effectiveness of RTD did not
depend on the slightly variable wait time between mix consumption and behavioral
testing or on the order of experiments (although behavior did shift over time). The
consistent behavioral effects of RTD despite variability in these experimental param-
eters suggest that changes in risk preferences revealed here were the result of changes
in serotonergic function. The similarity between monkeys’ behavior at baseline and
after consuming a balanced amino acid mix further supports this supposition, as it
indicates that observed behavioral changes were due to low tryptophan rather than
to consumption of an amino acid bolus in fluid. Furthermore, neither plasma trypto-
phan nor the plasma tryptophan/tyrosine ratio (an imperfect proxy for the plasma
tryptophan/LNAA ratio but cf (Carpenter et al., 1998)) differed between baseline
and balanced mix conditions.
Our data further supports the observation that context modulates the likelihood
of choosing a safe option in the presence of uncertainty (Kahneman & Tversky,
1979; A. McCoy & Platt, 2005b; B. Y. Hayden & Platt, 2007). We confirmed that
low serotonin diminishes the likelihood of safe choices independent of the initial safe
choice likelihood, indicating that serotonin exerts a global influence on reward valua-
tion and decision making independent of context. Since monkeys’ preferences shifted
109
consistently toward the risky choice across contexts, rather than toward indifference,
the use of additional reward contexts also provided an additional confirmation that,
in this task, serotonin depletion did not diminish the ability to learn to discriminate
rewards.
It is worth noting that many of the tasks used to probe the influence of the
serotonergic system on behavior address temporal impulsivity. Since calculations
of both time and probability-weighted value contribute to time-sensitive decision
making (Kacelnik & Bateson, 1996; J. Evenden, 1999a; B. Y. Hayden & Platt,
2007), as does the context within which choices are made (Kahneman & Tversky,
1979; Small et al., 2001; Huettel, Song, & McCarthy, 2005), impulsive behavior
might reflect either unwillingness to wait or preference for risky options. By holding
delays constant, we were able to focus on the contribution of serotonin signalling
to the valuation of probabilistic rewards. Although the methods of this study do
not directly address temporal impulsivity, the increased saccade velocity observed
in our study is consistent with heightened temporal impulsivity following serotonin
depletion as described in previous studies (Harrison, Everitt, & Robbins, 1997;
Winstanley, Dalley, Theobald, & Robbins, 2003; Scholes et al., 2007).
The distinction between risk sensitivity and temporal impulsivity reflects the
complexity of both the serotonergic system and the behavioral pathologies it influ-
ences. In fact, prior studies of the influence of serotonin on decision making have
often yielded contradictory results (Jacobs & Azmitia, 1992; Hoyer, Hannon, &
Martin, 2002; Varnas et al., 2004). serotonin signaling relies on over fifteen distinct,
differentially localized receptor subtypes whose functions often conflict. Because in-
dividual components of the serotonin system may contribute conflicting signals in
the decision process, this system is not truly the sum of its parts. Thus, results
obtained using nonspecific agents (such as selective serotonin reuptake inhibitors, or
SSRIs) must be interpreted with caution. For example, although SSRIs suppress
110
temporally impulsive behavior in pigeons and rats (Bizot, Le Bihan, Puech, Ha-
mon, & Thiebot, 1999; J. L. Evenden, 1999; J. L. Evenden & Ryan, 1999; Wolff
& Leander, 2002) and serotonin depletion increases temporal impulsivity (Mobini,
Chiang, Ho, Bradshaw, & Szabadi, 2000), specific 5-HT2 receptor agonists slightly
increase impulsivity (J. L. Evenden, 1998, 1999) while 5-HT1A receptor agonists
shift preferences toward indifference (J. L. Evenden & Ryan, 1999).
Given the effects of global serotonin levels on risk sensitivity observed in this
study, it is likely that multiple components of the serotonin system, rather than any
single element, contribute to these behaviors. Thus, a single perturbation of the
system, whether via environmentally induced serotonin depletion or genetic modi-
fications of components of the serotonin system, modulates behavior via a cascade
of signaling events (Canli & Lesch, 2007). These complicated polygenic and epige-
netic interactions, which obscure the detailed mechanics of serotonergic contributions
to behavior, contribute to the combination of genetic and contextual influences on
psychiatric diseases (Bennett et al., 2002; Jans et al., 2007; Kreek et al., 2005;
Ren-Patterson et al., 2006). Nonetheless, our results demonstrate that serotonin
contributes to the neural processes that translate perceived rewards and risk into
action.
Although humans, unlike the monkeys studied here, tend to be risk-averse for
gains, monkeys and humans both show contextual modulation of risk preferences
(Kahneman & Tversky, 1979; Rachlin, 2000; Bateson, 2002; A. McCoy & Platt,
2005b; B. Y. Hayden & Platt, 2007). Notably, humans become risk-seeking when
gambling for small financial rewards, which may be comparable to the small juice
rewards our monkeys receive (Markowitz, 1952). These behavioral observations
suggest that our results may generalize to humans. Furthermore, the serotonergic
system is strongly conserved across primate species (Bennett et al., 2002), suggesting
that the modulatory mechanisms influencing risky decision-making may be common
111
to monkeys and humans.
Our results suggest the possibility that low serotonin levels may underlie the will-
ingness of problem gamblers to continue betting despite excessive losses (Rachlin,
2000). Based on these considerations, it seems surprising that low serotonin lev-
els should persist at such high frequencies in the human population (Caspi et al.,
2003). From an evolutionary point of view, however, low serotonin may not always
be pathological and in some circumstances may in fact be beneficial. For example,
developmentally diminished serotonin found in adolescent vervet monkeys may pro-
mote impulsive and risk-taking behaviors that improve social status in adulthood
even as they introduce immediate risk or danger (Higley et al., 1996; Fairbanks
et al., 2004). Similarly, high CNS serotonin turnover during the mating season may
support increased aggression and mating behavior in rhesus macaques, increasing the
probability of reproduction (Mehlman et al., 1997). The persistence of low serotonin
function at high frequencies within the population may thus reflect evolutionary
pressures favoring risky, but potentially advantageous behavior. Thus, although low
serotonin levels are often associated with behavioral pathologies, willingness to take
risks may be beneficial in some contexts. Nonetheless, our observation that low
serotonin decreases valuation of safe rewards relative to risky options emphasizes the
continued importance of investigating potential serotonergic therapies for behavioral
pathologies like problem gambling, addiction and schizophrenia.
112
4.5 Figures
113
T2T1
T2T1
T2T1
T2T1
T2
Fixation onset
Target onset
Delay
Fix offset
Choice
Reward
FixTarget 1Target 2
EyeReward
Safe targetRisky target
25 trials Time
Example Reward Schedule
150125,175
150100,200
14040,240 40,240
20050,250
150
Time
EV ∆
40,240210
0 0 0 0 60 70Reward CV 0.167 0.333 0.667 0.714 0.714 0.714
Time
Example Treatment OrderB B RTDRTD B RTDRTD B
Figure 4.1: Top panel, task: after monkeys fixated a central stimulus, two periph-eral targets were illuminated. The central stimulus was then dimmed to indicate theopportunity to saccade to either target. Middle panel, example reward schedule: ineach block, one target was associated with a guaranteed reward while the other targetoffered a variable reward. The risk and mean reward sizes for each target were var-ied orthogonally. Bottom panel, example treatment order (B, Baseline; RTD, RapidTryptophan Depletion): the treatment for each session was pseudorandomized tominimize the effects of treatment order.
114
*
0
4
6
8
2
10P
lasm
a tr
ypto
phan
(ug
/ml)
**
NIRTD
SHBaseline
BR
**
NISH BR
Figure 4.2: Ingestion of tryptophan-free amino acid mixture significantly lowersplasma tryptophan in three monkeys. Baseline levels (blue bars) were measuredfrom macaques in their normal physiological state (monkey SH, mean ± s.e.m. =8.53±0.78 ug/ml (n=6 measurements), monkey BR9.21±0.35 ug/ml (n=4), mon-key NIbaseline mean±s.e.m. 9.8±0.60 ug/ml (n=2)) and post-RTD levels (red bars)were measured following a 24-hour low-protein diet and administration of the RTDmix (monkey SH 3.36±1.00 ug/ml (n=5), monkey BR3.20±0.73 ug/ml (n=9), mon-key NI2.29±2.45 ug/ml (n=4)). RTD significantly decreased plasma tryptophan lev-els in each monkey (monkey SH, ANOVA, p<0.01, F=17.2; monkey BR, p<0.001,F=27.5; monkey NI, ANOVA, p=0.016, F=16.0) and across the population (baselinemean±s.e.m. =8.97±0.42 ug/ml, RTD mean±s.e.m. =3.04±0.51 ug/mL, ANOVA,F=253.1, p<0.00001; subject effect n.s.). * indicates p<0.05, ** indicates p<0.01.
115
Low Tryptophan
*
Normal Tryptophan0
0.1
0.2
0.3
0.4
0.5
0.6
Pro
babi
lity
of s
afe
choi
ce
(a) Monkeys less frequently chose the safe option following serotonin depletionthan in baseline conditions (ANOVA of mean probability of safe choice per sessionvs tryptophan depletion condition, F=5.38, p=0.028; mean baseline preferencefor the safe choice±s.e.m. =53.4%±3.2% across 12 sessions with 2963 trials;mean RTD preference for the safe choice 42.9%±3.0% across 18 sessions with5070 trials)
Figure 4.3: Serotonin depletion systematically decreases preference for the safeoption in monkeys.
116
0.3
0.4
0.5
0.6P
roba
bilit
y of
saf
e ch
oice
Monkey SH Monkey BRMonkey NI
Indifference
BaselineRTD
(b) Each monkey chose the safe option less frequently following RTD, despite dif-ferences in baseline preferences for each subject (Monkey BR, 4 baseline sessionswith mean probability of safe choice±s.e.m. 48.4%±7.0% and 9 RTD sessionswith mean probability of safe choice 36.0%±4.7%; Monkey NI, 2 baseline ses-sions with 47.4%±2.8% and 4 RTD sessions with mean probability of safe choice44.2%±0.5%; Monkey SH, 6 baseline sessions with mean probability of safechoice 58.7%±3.8% and 5 RTD sessions with mean probability of safe choice54.3%±0.8%).
Figure 4.3: Serotonin depletion systematically decreases preference for the safeoption in monkeys.
117
0.2
0.7
0.3
0.4
0.5
0.6
Equal
Pro
babi
lity
of s
afe
choi
ce
0.8
Unequal
Indifference
BaselineRTD
(c) RTD reduced preference for the safe option whether both options had thesame expected value (“Equal”; 12 baseline sessions with mean probability of thesafe choice±s.e.m. 35.5%±3.8% and 18 RTD sessions with mean probability ofchoosing the safe option 28.6%±2.8%) or unequal expected value (“Unequal”; 12baseline sessions with mean probability of safe choice±s.e.m. 73.6%±4.1% and17 RTD sessions with mean probability of safe choice 61.3±4.0%). Importantly,preference for the safe option always diminished following RTD, thus ruling outimpaired reward discrimination. Error bars: 1 S.E.M. * indicates p<0.05, ** p<0.01.
Figure 4.3: Serotonin depletion systematically decreases preference for the safeoption in monkeys.
118
0.7
0.3
0.4
0.5
0.6
Pro
babi
lity
of s
afe
choi
ce
8070503010Safe EV - risky EV (ms juice)
0.9
0.8
0.2
1
0 20 40 60
RTD r=0.91, p=0.01
Indifference
Baseline r=0.93, p<0.01
Figure 4.4: Serotonin depletion increases the subjective value of the risky optionin three macaques. To test whether serotonin depletion influences valuation of riskyrewards, we calculated the amount of juice monkeys were willing to pay for therisky option. The payment difference between the two options when the monkeyswere indifferent indicates the monkeys’ willingness to pay for risk, or safety premium(arrows: fringed arrow = baseline, filled arrow = RTD). In all cases, the subjectivevalue of the safe option decreased following RTD. Lines shown were fitted by linearregression to the probability of choosing the safe choice at each safety premiumin each session; each point represents the mean of these probabilities. Blue line,baseline; red, RTD.
119
0.2
0.7
0.3
0.4
0.5
0.6
0.8
0.1
0.9P
roba
bilit
y of
saf
e ch
oice
Loss WinSafe
Indifference
BaselineRTD
Previous reward
0
1.0
**
n.s.
*
Figure 4.5: Tryptophan depletion increases risky choices following sub-maximalrewards. Tryptophan depleted monkeys are more likely to select the risky choice fol-lowing receipt of safe rewards (Baseline 69.8%±4.7%; RTD 57.4%±4.9%; ANOVA;logistic regression, choice vs depletion, Wald stat 35.9, p<0.000001) or small riskyrewards (Baseline preference for the safe choice 53.3% ±4.8%; RTD 43.3±3.6%; lo-gistic regression, choice vs depletion, Wald stat 5.2, p<0.05), both of which representlosses relative to the large risky reward. In contrast, RTD did not affect preferencesfollowing receipt of the large risky reward (Baseline 11.2%±2.7%; RTD 11.5%±1.9%;logistic regression, choice vs depletion, Wald stat 1.0, n.s.). Bar height represents themean safe choice likelihood across sessions following receipt of the indicated choice.Error bars: 1 S.E.M. * indicates p<0.05, ** p <0.01.
120
5
Low serotonin increases risky decisions in mice
5.1 Summary
Extensive clinical evidence implicates serotonergic dysfunction in psychiatric dis-
orders associated with abnormal responses to risk (Deakin, 2003; Fletcher, 1995;
Grados, 2009; Harrison et al., 1999; Higley et al., 1996; Mossner et al., 2007; Wong
et al., 2008). Genetic variations in serotonergic effectors modulate probabilities of
psychiatric illness (Caspi et al., 2003; Koenen et al., 2009; Nielsen et al., 1998), and
serotonergic hypofunction is tied to pathological gambling, impulsivity and agres-
sion (Apter et al., 1990; Dolan et al., 2001; Higley et al., 1996; Nordin & Sjodin,
2006). Previous work from our lab confirmed a causal role for low serotonin in in-
creasing gambling behavior in monkeys offered a choice between a variable, or risky,
reward and a guaranteed reward (Long et. al., under review). To develop an assay
to map the detailed serotonergic contributions to decisions made in the presence
of economic risk, we trained 20 sham-injected C57BL/6J (wild-type) (C57BL/6J)
mice to choose between a risky reward and a guaranteed reward. We then compared
the behavior of 20 C57BL/6J mice treated with PCPA, an irreversible inhibitor of
121
the rate-limiting enzyme in serotonin synthesis (Koe & Weissman, 1968), to that
of the wild-type (WT) controls. High performance liquid chromatography (HPLC)
confirmed that PCPA treatment consistently lowered brain serotonin levels. WT
mice were strongly averse to the risky option, but a significant subset of the PCPA-
treated mice demonstrated risk-taking behavior. This result confirms that lowering
brain serotonin increases individuals’ likelihood of risky choices relative to guaranteed
alternatives.
5.2 Results and Discussion
To assay murine risk preferences, we trained mice to choose between a guaranteed,
or safe, condensed milk reward (1uL) and a variable, or risky, option with 50%
probability of reward (2uL) (Fig 5.1). Since the expected values of the two options
are equal, measuring the frequency of risky choices in this context allows estimation
of attitudes toward risk, or risk-preferences. In economic terms, choosing the risky
option more frequently than chance (i.e. than indifference) is described as risk-
seeking behavior; choosing the risky option less frequently than chance indicates risk-
aversion. Such choices arouse strong biases across species (Bateson, 2002; Kacelnik
& Bateson, 1996; Kahneman & Tversky, 1979), and evidence from social observation
and behavioral pathologies implicates low serotonin in risk-seeking behaviors (Higley
et al., 1996; Kuhnen & Chiao, 2009; Nordin & Sjodin, 2006; Nordstrom et al., 1994).
To test whether brain serotonin levels modulate risk preference in this simple
decision task, we randomly sorted the population of 68 mice into two groups: the
sham control group (33 mice) received vehicle injections while the experimental group
(35 mice) received PCPA injections to block serotonin synthesis and lower brain
serotonin. We confirmed the success of this manipulation by measuring frontal cortex
serotonin levels in a subpopulation of mice (21 sham and 25 PCPA), and found
that PCPA treatment significantly lowers frontal cortex serotonin concentrations
122
relative to sham controls (Fig 5.2, one-way ANOVA brain serotonin per mouse vs
group, F=26.0, p<0.01). Since PCPA may also decrease CNS dopamine (Dailly,
Chenu, Petit-Demouliere, & Bourin, 2006), we also measured brain dopamine and
dopaminergic metabolites. Since measures of brain dopamine were unaffected by
PCPA treatment (Fig. B.1, one-way ANOVA, F=0.38, n.s.), and not predicted by
brain 5-HT concentration (Fig. B.2, linear regression, dopamine fraction vs 5-HT
R2 = 0.08, n.s.), we concluded that potential behavioral differences between control
and experimental mice would reflect serotonergic modulation.
Like most animal species (Bateson, 2002), mice strongly prefer a safe option over a
risky option with equal expected value (Fig 5.3). However, PCPA-treated mice were
significantly more likely than controls to choose the risky option overall (Fig 5.3, one-
way ANOVA, risky choice frequency per mouse per day vs group, F=4.5, p<0.05),
an effect that resulted from strong risk-seeking behavior in a significant subset of the
PCPA-treated mice (Fig 5.4; one-sided t-test, number of risk-seeking or risk-neutral
mice vs the predicted risk-averse population: sham mice n.s., PCPA group p<0.05).
Importantly, the two groups of mice had similar initial biases and, across the
population, did not demonstrate initial biases distinct from indifference (Fig B.3,
one-way ANOVA, risky choice frequency on the first day vs treatment group, F=0.08,
n.s.). Notably, while vehicle-treated controls became risk-averse despite their initial
risk-preferences, PCPA mice were more likely to continue to choose the risky option
if their initial bias favored the risky option (Fig B.4), suggesting that low serotonin
might enhance primacy effects.
These observations led us to question whether low 5-HT altered learning in this
context, particularly given previous evidence that low 5-HT impairs learning and
compromises memory consolidation (Riedel et al., 2003; R. D. Rogers et al., 1999).
We found that even when mice differed in their overall risk attitudes, the behavior
of both control (Fig 5.5) and PCPA-treated (Fig 5.6) mice was well fit by simple
123
exponential models. We therefore asked whether, as predicted by models that posit
a role for serotonin in communicating information about negative rewards (Daw et al.,
2002; Deakin, 2003), the observed shift in mouse preferences might be dependent on
decreased sensitivity to relative losses. Our data did not support this interpretation,
as risk-seeking PCPA-treated mice were more likely to choose the risky option after
any of three possible reward outcomes (Fig B.5).
Finally, since lowering brain serotonin speeds behavioral responses in rats, con-
sistent with increased motivation, attention or temporal impulsivity (Harrison et
al., 1997; Scholes et al., 2007), we asked whether PCPA treatment induced simi-
lar behavioral changes in this task. We found that PCPA-treated mice nosepoked
significantly more often for reward (Fig B.6; ANOVA, number of nosepokes for re-
ward vs treatment group and choice: interaction term, F=22.8, p<0.01; post-hoc
LSD t-test, groups significantly different (p<0.01) both on safe choice trials and on
risky choice trials ), consistent with the hypothesis that decreased serotonin might
increase impulsive or perseverative behavior. Notably, this effect was stronger on
trials where the mouse chose the safe option, indicating that this observation is not
simply reflective of risky choices. However, the reaction time between trial initia-
tion and choice indication did not differ significantly between groups even though
mice were significantly slower to nosepoke for the risky option (ANOVA, reaction
time vs group and choice: effect of group, F=3.4, p=0.06; choice, F=15.9, p<0.01;
interaction, F=2.2, n.s.). Including these factors in our analysis (logistic regression,
binomial safe/risky choice vs treatment, reward nosepokes and reaction time) con-
firmed the main effect that lowering brain serotonin increases risk preferences (Wald
stat 42.5, p<0.01) even in the presence of a significant effect of reaction time (Wald
stat 19.7, p<0.01); nosepoke frequency did not predict choices (Wald stat 1.0, n.s.).
124
5.3 Experimental procedures
5.3.1 Behavioral procedures
All procedures were approved by the Duke University Institutional Animal Care and
Use Committee and were designed and conducted in compliance with the Public
Health Services Guide for the Care and Use of Animals. Four to six week old wild-
type C57BL/6J mice were habituated to the experimental room for a minimum of
1 week before commencing training. After reaching a minimum weight of 20g, the
mice were food restricted to 90-95% of their age-adjusted baseline weights.
Each operant chamber (MED Associates) contained three nosepoke holes: an
initiating nosepoke hole in the back wall of the chamber and two reward nosepokes
on the opposite (front) wall of the operant chamber, which were positioned at the
same height and symmetric around the center of the wall. A LED within each
nosepoke was illuminated to cue availability for poking. Nosepokes were detected
with infrared beams situated within each nosepoke. Each trial began with a tone and
illumination of both a house light positioned above the chamber and the initiating
nosepoke LED (Fig 2a). To initiate the trial, the mouse was required to nosepoke
into the initiating nosepoke. Two reward nosepokes were immediately illuminated.
Once the mouse poked into a reward nosepoke, a liquid dipper containing condensed
milk automatically rose to an inferior opening in the nosepoke, and the mouse was
able to consume the milk. After a 5 second interval for complete consumption, the
dipper was lowered and the next trial began. Since inter-trial intervals affect risk-
preferences in an analogous macaque task (B. Y. Hayden & Platt, 2007), there was
no inter-trial interval. All trials were controlled and monitored by custom-written
software (Med PC) that recorded each choice as well as the number of nosepokes per
trial, latency to initiation and latency to collect reward after initiation.
Mice were initially habituated to the operant chambers and trained to perform
125
a simple, risk-less choice task. Side preferences were extinguished before mice were
exposed to the risky choice task. After this initial behavioral training, mice were di-
vided randomly into two groups: sham (initial injection 10ml/kg vehicle, thereafter
3.3ml/kg vehicle injection; n=33 mice) and PCPA (initial injection 300mg/kgPCPA,
thereafter 100mg/kg PCPA dissolved to allow a 3.3ml/kg bolus; n=35 mice). Injec-
tions were performed once daily following each experimental session. 24 hours after
the first injections, the reward values in each chamber were lateralized randomly
and counterbalanced across animals, so that one reward nosepoke offered 0.02 mL of
condensed milk (safe) and the other reward nosepoke provided either 0 or 0.04 mL of
condensed milk with equal probability (risky). Mice performed 30 risky choice trials
per session following completion of 4 forced choice trials (2 right, 2 left) included
to ensure daily sampling of each reward option. The mice were then tested for 14
sessions before they were euthanized painlessly.
5.3.2 Serotonin depletion
We depleted brain serotonin using the tryptophan hydroxylase inhibitor PCPA (Koe
& Weissman, 1968; Koe, 1971; Koe & Corkey, 1976), which effectively depletes brain
serotonin within 24 hours and maintains low serotonin levels for weeks when admin-
istered daily (Cahir, Ardis, Reynolds, & Cooper, 2007). Successful brain serotonin
depletion was confirmed by post-mortem analyses in a subpopulation of mice. Sham
injected and PCPA-treated mice were maintained on daily injections until 18-24 hours
preceding sacrifice. Immediately following euthanasia, the brains of each mouse were
removed and the frontal cortex, ventral striatum and dorsal striatum were dissected
and flash-frozen, with the exception of two lost dorsal striatal samples from PCPA-
treated mice. These regions were stored at -40C and 5-HT and dopamine (DA) levels
were analyzed using HPLC.
126
5.3.3 Analysis of mouse data
Behavioral data were translated using MedPC2XL and analyzed using custom R
scripts and Statistica. We calculated the mean frequency of choosing the risky choice
for each mouse and each experimental session, and used ANOVAs to assess the effect
of PCPA treatment on the probability of choosing the safe target. In addition, we
quantified the relationship between serotonin depletion and physiological metrics on
a per-trial basis, using ANOVAs to compare the number of headpokes and latency
to nosepoke for reward following trial initiation in the sham and PCPA-treated con-
ditions. Finally, we coded choice as a binary variable (safe or risky) and included
these behavioral indicators of motor activity and attention as factors in logistic re-
gression to assess their potential contributions to behavioral preferences. Statistics
were computed using Statistica (StatSoft).
127
5.4 Figures
Start signal
Central NPTarget onset
Choice
Reward
Time
NP
Figure 5.1: Mouse gambling task. To begin each trial, a mouse nosepoked intoan illuminated (LED) nosepoke centered at a low height on the back wall of anoperant chamber. The associated LED was immediately extinguished and LEDswithin two symmetrically positioned peripheral reward nosepokes on the opposite(front) wall of the chamber were illuminated. All choices of the safe reward nosepokewere rewarded with condensed milk; the risky nosepoke was rewarded stochasticallywith 50% probability of a reward twice the size of the safe reward.
128
0
500
700
900
100
1200B
rain
ser
oton
in (
pg/u
l)
PCPAVehicle
**
300200
400
600
800
1100
1300
1000
Figure 5.2: PCPA lowers brain serotonin relative to vehicle-injected controls. Toconfirm the neural effects of PCPA treatment, we used HPLC to measure serotoninlevels in the frontal cortex of vehicle-injected (blue) and PCPA-treated (red) mice.Error bars, S.E.M., double asterisk, p<0.01 relative to baseline.
129
PCPA
*
Vehicle0
0.1
0.2
0.3
0.4
0.5
1.0P
roba
bilit
y of
ris
ky c
hoic
e0.9
0.8
0.7
0.6
Figure 5.3: Lowering brain serotonin increases the probability of a risky choice.Mice treated with PCPA (red) were significantly more likely to choose the riskyoption than sham controls (blue). Bar indicates the mean of the daily risky choicefrequencies, error bars represent 1 S.E.M. Asterisk, p<0.05 relative to baseline.
130
PCPA
*
Vehicle0
0.1
0.2
0.3
0.4
0.5
1.0P
roba
bilit
y of
ris
ky c
hoic
e0.9
0.8
0.7
0.6
Figure 5.4: Lowering brain serotonin increases the probability of a risky choicein a subset of mice. While the vehicle-treated mice overwhelmingly preferred thesafe option, a significant number of PCPA-treated mice chose the risky option at orabove chance. Points, mean of daily risky choice frequencies per mouse; error bars,1 S.E.M. one-sided t-test, p<0.05
131
0.2
0.7
0.3
0.4
0.5
0.6
0.8
0.1
0.9P
roba
bilit
y of
ris
ky c
hoic
e
0
1.0
Session number (days)2 4 6 8 10 12 140
Mouse 73 (vehicle)
R=0.98
Figure 5.5: Example vehicle mouse learning curve. Control mouse preferences shiftto the safe option over several days of experience.
132
0.2
0.7
0.3
0.4
0.5
0.6
0.8
0.1
0.9P
roba
bilit
y of
ris
ky c
hoic
e
0
1.0
Session number (days)2 4 6 8 10 12 140
Mouse 1055 (PCPA)
R=0.78
Figure 5.6: Example PCPA mouse learning curve. This example PCPA mouse hada normal (though inverted) learning curve and, like the control mouse, its preferencesstrengthened over several days of experience.
133
6
Conclusions
6.1 Overview
Although this thesis is focally directed toward understanding the neural mechanisms
of decision making, it describes two very different sets of experiments that use very
distinct investigative techniques. Furthermore, the exact demarcation of the bound-
aries of these experimental groups may be questioned: are these experiments dif-
ferentiated by the neural substrate (CGp vs serotonin) or by task (cross-contextual
investigations vs decisions under risk)? These nested distinctions hamper attempts
to summarize this data briefly and concisely; as with Russian Matryoshka dolls, it is
difficult to decide which groupings work best. As I have previously suggested a model
in which anatomically defined brain regions interact via neuronal projections and are
modulated by contributions from neurochemicals such as serotonin (Section 1.3.5), I
will use this distinction to organize this discussion. I will first summarize my studies
of posterior cingulate cortex function before addressing serotonergic contributions to
preferences expressed under risk and turning to the question of how serotonin might
modulate CGp activity.
134
6.2 CGp, decisions and salience
6.2.1 Summary of results
In the Introduction of this thesis, I suggested that CGp linked outcome evaluation
to action selection, and that its contributions to decisions included maintaining an
integrated representation of stimulus value and future action value. The implications
of these proposals — namely, that CGp neurons should respond to informationally
relevant events whether task-relevant or surprising, and that CGp activity should
track integrated measures of stimulus evaluation that predict future choices — led
to the experiments described in Chapters 2 and 3. The results described in these
chapters thus add supporting evidence to the hypothesis that CGp reports and tracks
motivationally salient stimuli, and thus contributes to learning, memory and decision
making.
Thus, the evidence described in this thesis, in combination with findings that
CGp differentiates outcomes, maintains evaluative representations across time, and
encodes stimulus-outcome associations, supports a model in which CGp links out-
come evaluation and action selection. Figure 6.1 depicts such a model based on the
anatomical connections between regions described in Sections 1.3.3 and 1.3.4. In
this model, CGp transmits both immediate and remembered (or maintained) infor-
mation about integrated outcome evaluation to regions such as OFC (and possibly
ACC) that encode stimulus-outcome associations. CGp also contributes to rule-
based decisions via projections to PFC. Recursive connections between ACC and
CGp, and between LIP and CGp, contribute to refining evaluation-based (LIP) or
strategic (ACC) action selection. In addition, recursive connections between CGp
and the hippocampal formation contribute to updating and retrieving memory, thus
ensuring that outcome evaluation will be stored in memory and later retrieved for
prospective action selection. Finally, this model includes regions not directly con-
135
nected to CGp whose activity contributes to outcome evaluation (VTA, DRN) or
encoding new information (BLA).
6.2.2 The breadth of relevant information: from novelty to memory
Comparing CGp responses to those in the BLA, a region that, like CGp, sends ef-
ferents to both ACC and OFC, reveals the importance of memory and encoding
motivational salience for CGp function. BLA responses to novel and changing envi-
ronmental conditions have led to the suggestion that the BLA, like CGp, contributes
to ACC’s role in strategy selection (as well as to recognition of error or “conflict”
signals) and to the formation of stimulus-outcome associations in OFC (Holland &
Gallagher, 2004). Unlike CGp, whose responses maintain representations of reward
even after stimulus novelty fades, BLA responses fade as stimulus-outcome associa-
tions are learned and encoded in OFC.
Thus, in information theoretic or learning theory terms, BLA responses are spe-
cific to surprising stimuli that strongly motivate learning (Courville et al., 2006;
Rescorla & Wagner, 1972; Shannon, 1948). While CGp responses reflect the infor-
mational salience of surprise, they also encode predictive information that is as likely
to maintain as to change stimulus-outcome associations. CGp contributions to de-
cisions — whether through connections to outcome-evaluative regions such as OFC
or to action-selection regions such as LIP, ACC and PFC — reflect both learned
and surprising reward information. These experiments thus provide evidence for
the possibility that CGp might appropriately be characterized as a salience map for
motivationally relevant information such as reward; both the observation that CGp
neurons report task-relevant and surprising events and the confirmed prediction that
CGp neurons track the motivational information of available options are consistent
with this description.
136
DRNNN
OHOH NHNH33
++
Brainstem Areas
CGp
LIP
Medial
Lateral
VTA
OFC
BLA
PFC
Action selection
Outcome evaluation
HPC/PHG
VisuomotorOutput
Memory
ACC
Figure 6.1: Neural components of evaluative action selection. This figure highlightsthe anatomical connections between CGp and regions that evaluate outcomes (blue)or select actions (green). In addition, it highlights the reciprocal connections betweenCGp and the hippocampal formation (magenta), which is implicated in memory. No-tably, CGp appears to be at a crossroads between memory, outcome evaluation andvisuomotor regions that contribute to action selection. The orange arrows from thedorsal raphe nuclei (DRN) (small orange molecule: serotonin) depict the seroton-ergic projections from the DRN to other decision-related structures shown. (ACC,anterior cingulate gyrus; BLA, basolateral amygdala; CGp, posterior cingulate gyrus;DRN, dorsal raphe nucleus; HPC/PHG, hippocampus, parahippocampal gyrus; LIP,lateral intraparietal cortex; OFC, orbitofrontal cortex; PFC, prefrontal cortex; VTA,ventral tegmental area)
137
6.2.3 CGp as a putative salience map
These observations support the argument, initially proposed in the Introduction,
that CGp’s function, like its anatomical location, resembles a crossroads in decision
making and that CGp’s contributions to outcome evaluation and action selection
can be described using a salience map framework (Figure 6.2). In Section 1.4, I
briefly summarized evidence from a variety of studies that supports my hypothesis
that CGp functions as a salience map. In this section, I will demonstrate that the
data described in this paper provides evidence that CGp fulfills all five of the criteria
proposed for a salience map (Section 1.2).
1. CGp neurons respond to stimuli across a variety of contexts. As described
in Chapter 2, CGp neurons respond phasically to visual stimuli across three task con-
texts (decision, operant, Pavlovian). In addition, these neurons respond following
saccades and rewards in each of these contexts and also respond to uncued rewards in
a context without reward-relevant visual cues. Importantly, each of these events —
cue, saccade and reward — is experientially linked to past, current or future rewards.
The observed responses to these reward-related events across contexts are consistent
with my suggestion that the putative CGp salience map deals with motivational
information.
2. CGp neuronal responses are not exclusively feature-selective. Previous ob-
servations that CGp neuronal activity tracks reward magnitude in a reward discrim-
ination task (A. N. McCoy et al., 2003) and risk coefficient of variance in a gambling
task (A. McCoy & Platt, 2005b) allow the possibility that CGp responses are selective
for unique features of reward within separate contexts. However, the data described
in Chapter 3 indicates that these responses reflect an integrative measure of con-
textually relevant motivational information rather than unitary feature-selectivity:
138
0
5
10
15
Firin
g ra
te (H
z)
0
5
10
15
Firin
g ra
te (H
z)
0
5
10
15
Firin
g ra
te (H
z)
Figure 6.2: A simple salience map. In this diagram, the rhesus monkey (left)receives cherry juice rewards while viewing stimuli presented on a computer monitor(right). Atop the monitor, a small videocamera monitors the monkey’s eye position.Each stimulus (red, yellow, green) is associated with differently sized rewards (large,medium, small). In this paradigm, motivational salience depends exclusively onreward magnitude. As shown in the greyscale salience map (center screen, scaledfrom black (low motivational salience) to white (high motivational salience)), regionsof space associated with high reward have high salience (white) relative to bothsmaller rewards (greys) and the minimally salient background (black). The threerepresentative PSTHs (left, thought bubble) show a possible neuronal encoding ofthe salience strength of each visual cue, i.e., a direct proportionality between the sizeof phasic responses and the motivational salience bound to the corresponding regionof space.
139
neuronal activity reflects preferences within each of two decision contexts that dis-
sociate risk from reward.
3. CGp neuronal activity reflects sensorimotor integration. As mentioned in
the previous point, CGp neuronal activity reflects contextually determined pref-
erence, an integrative measure of reward probability and magnitude (Chapter 3).
Furthermore, CGp neurons reflect a heuristically categorized measure of reward de-
sirability (win/loss) and neuronal activity predicts heuristically categorized motor
output (stay/shift) (Chapter 3). This pattern of activity suggests that CGp neu-
ronal activity encodes both sensory input and motor output.
4. CGp neurons have spatial receptive fields. Phasic CGp responses to sac-
cade depend on the spatial location of the chosen target in a decision task (Chap-
ter 2). This observation is consistent with previous studies that demonstrated that
the tonic activity of CGp neurons is modulated by target location and saccadic
endpoints and that CGp neurons have broad visuospatial receptive fields following
saccade (Dean & Platt, 2006; Dean et al., 2004; C. R. Olson et al., 1993; C. Olson
& Musil, 1992; C. Olson et al., 1996).
5. CGp responses to stimuli are not exclusively spatial. CGp neurons respond
phasically following uncued reward delivery, that is, in the absence of visuospatial
information (Chapter 2). In addition, neuronal responses to visual cues and to re-
wards within the decision task context are not significantly modulated by spatial cue
location (Chapter 2).
Thus, the data described in this paper supports my hypothesis that the posterior
cingulate cortex may function as a salience map to direct attention toward desirable
actions based on experienced and remembered outcomes. However, it is important
to note that while the experiments described in this thesis involve stimuli that are
140
associated with reward, they do not conclusively demonstrate that CGp neurons
respond only to reward-indicative cues. Further experiments will be required to
determine whether CGp neurons respond exclusively to cues and events associated
with gustatory rewards, or whether these neurons also encode the motivational value
of visual social or auditory stimuli (Klein et al., 2008). Evidence that posterior
cingulate cortex neuronal activity does reflect the motivational value of such non-
gustatory stimuli, as suggested by functional neuroimaging (Maddock & Buonocore,
1997; Maddock et al., 2003; Pierce, Haist, Sedaghat, & Courchesne, 2004; Shah et al.,
2001), would strongly support arguments that CGp integrates broadly defined mo-
tivational information across contexts, and that CGp neuronal responses are indeed
integrative rather than feature-selective.
6.2.4 On the biphasic directionality of CGp responses
The studies described in this thesis reveal that posterior cingulate neurons respond
phasically to events with either increases or decreases in firing rate. This observation,
like the observation that CGp activity often increases (or decreases) after both large
rewards and unexpected rewards (cf. (A. N. McCoy et al., 2003)), introduces diffi-
culties in the interpretation of CGp responses. Unlike VTA neurons, which increase
firing rate when rewards are unexpectedly large and decrease activity when rewards
are unexpectedly omitted (Fiorillo et al., 2003; P. Tobler, Fiorillo, & Schultz, 2005),
CGp neurons are unlikely to linearly code increases and decreases in reward size con-
sistently across the population. However, other reward-evaluative regions of brain,
including the OFC and striatum (Cromwell & Schultz, 2003; Hassani, Cromwell, &
Schultz, 2001; Tremblay & Schultz, 1999), contain neurons that are both positively
and negatively correlated with reward size (or that either increase or decrease activ-
ity following reward omission). Thus, it is not uncommon to find subpopulations of
neurons with either increasing or decreasing correlations to reward outcomes.
141
Although explanations for this phenomenon are necessarily speculative, several
reasonable options may be posited. First, neurons that increase or decreasing firing
rate in response to relevant events may represent subpopulations of neurons with
distinct patterns of anatomic connectivity: they may integrate information from dif-
ferent regions of the brain, and they may send projections to distinct anatomical
endpoints. For example, CGp neurons with projections to the anterior cingulate
cortex might respond to stimuli by decreasing firing while CGp neurons projecting
to the LIP might respond by increasing activity. Second, these neurons may repre-
sent different subpopulations based on molecular expression patterns. For example,
variability in the serotonergic receptors expressed across CGp may modulate va-
lenced responses to reward: cells with 5-HT1A receptors might respond to external
stimuli by decreasing activity (since these receptors are inhibitory) while cells with-
out these receptors might increase activity. While this particular pattern of activity
is highly speculative, the importance of genetic expression patterns for defining ei-
ther anatomical regions and subregions or subpopulations of cells should not be
underestimated (C. L. Thompson et al., 2008). A recent study demonstrated that
within the hippocampus, genetic expression patterns define both classical anatom-
ical regions and smaller subregions (C. L. Thompson et al., 2008). Notably, many
of the molecules that defined the map described in this paper were cellular adhesion
molecules that guide and maintain the connections that underlie neuronal circuitry:
the molecular and circuit explanations for distinct subpopulations with differentially
valenced responses may be closely linked.
6.3 Serotonin, decisions and risk
6.3.1 Summary of results
The experiments described in monkeys and mice confirm that low serotonin shifts
choice tendencies toward risky options, suggesting that the low serotonin observed
142
in multiple behavioral disorders may in fact be causal. Importantly, these results
are consistent across two species, consistent with observations that the serotonergic
system is widely conserved (K. P. Lesch et al., 1997; Murphy, Lerner, Rudnick,
& Lesch, 2004). Furthermore, the demonstration that mice learn in this simplified,
economically-interpreted foraging task encourages me to think that future work could
expand on this task to probe the behavioral effects of neuromodulatory systems in
more detail. While rodents are commonly trained to perform ratio- and interval-
based psychological tasks, few studies of economic and behavioral decisions have
been performed in mice. In fact, although the literature on rat decisions is extensive,
my initial literature reviews identified only one paper that demonstrated that mice
could discriminate liquid reward magnitudes in a similar operant task (Isles, Humby,
& Wilkinson, 2003). Similar tasks are under development elsewhere (Bari, Theobald,
Mari, & Robbins, 2008; Watabe-Uchida & Uchida, 2008), but to my knowledge, the
work I report in this paper is the first demonstration of economic risk-sensitivity in
mice using an operant assay.
6.3.2 Caveats
The experiments that I have described using serotonergic manipulations in monkeys
and mice depend on techniques that globally lower serotonergic activity. While these
techniques have been well characterized and are widely used, it is worth noting three
caveats that apply to these experiments.
First, the use of rapid tryptophan depletion (RTD) requires subjects that are will-
ing to tolerate the dietary depletion mix. When this technique is used in human
studies, most subjects only consume this protein-heavy, unpleasant-tasting mixture
once or twice. In addition, the reimbursements offered for experimental partici-
pation effectively serve as bribes for tolerating moderately unpleasant experiences.
The situation is somewhat different with macaques, as the available subject pool is
143
smaller and the number of experiments performed with each subject larger (these
are, it should be noted, strategic experimental decisions that are consistent with re-
sponsible use of animals in research). In addition, it is difficult to bribe a macaque
to consume an unpleasant mix now in order to receive extra treats later. More
importantly, since macaques do not verbalize discomfort using human language, eth-
ical considerations necessitate careful observation for even small signs of discomfort.
While we found that we were able to modify experimental constraints to minimize
potential discomfort without sacrificing experimental rigor (see Appendix A), we also
found that monkeys lost interest in consuming the mix after repeated experiences.
While we were not able to determine whether this disinterest was due to a discomfort
resulting from satiety or due to anhedonic or unpleasant aftereffects of consuming
the mix (Moore et al., 2000; Young & Leyton, 2002), these possibilities are worth
noting when considering experimental designs that involve repeated dietary depletion
experiments in individual animals. One simple alternative would be to perform only
small numbers of experiments in a much larger set of subjects. In this regard, I noted
that several monkeys seemed to find the depletion mix interesting and novel when
they first experienced it, so a larger subject pool might allow sufficient data collection
while avoiding habituation. In addition, pharmacological treatments like the para-
chlorophenylalanine (PCPA) used in mice offer an alternative to dietary depletion
methods. I initially chose to avoid using PCPA in monkeys due to the potential for
kidney damage (Ikonomov, Stoynev, Goranova-Stoyneva, Popov, & Minkova, 1990).
However, after extensive observations of mice treated with PCPA, I suspect that
while chronic PCPA treatment would make monkeys irritable, it would be safe to
treat monkeys with PCPA acutely with careful observation and veterinary monitor-
ing. However, these concerns should be addressed carefully given the importance of
repeated recording sessions for electrophysiology.
Second, RTD and PCPA modulate global rather than focal serotonin levels. Using
144
methods that globally manipulate serotonin levels provided an important demon-
stration of the causal impact of low serotonin on economically risky decisions. In
addition, it allowed me to bypass the serotonergic system’s extraordinary complex-
ity (Baumgarten & Gothert, 1997). However, this global manipulation may be anal-
ogous to hitting a circuit board with a hammer – it will probably break the system
(and this is in fact an important discovery), but it won’t tell you which components
do what. Further experiments using genetic modifications, receptor-specific phar-
macological agents, or locally applied agonists and antagonists will help clarify the
functions of specific receptors. Additionally, in combination with electrophysiology,
such techniques should clarify the effect of serotonergic signaling on neuronal activ-
ity. Notably, given the availability of genetic techniques for rodent models and the
relative ease of increasing the numbers of individuals studied, these experiments will
likely be most profitable if performed using rodent rather than primate models.
Finally, one notable difference between mouse and monkey behavior under hypo-
serotonergic conditions is that while the preferences of all three individual monkeys
tested shifted away from the safe option, toward the risky gamble, only a subset
of mice within the population switched their preferences toward the gamble. This
observation may reflect the strength of mouse aversion to the risky option, since null
reward sizes induce much stronger aversion than small, non-zero outcomes. Alterna-
tively, it may result from behavioral inflexibility associated with a limited menu of
reward options or from early overtraining on safe options. Indeed, we were unable
to perform within-subject testing in the mice since initial behavioral testing revealed
that inbred C57BL/6J mice were unable to perform simple reversal learning in this
context. Although the between-subject experimental design used in mice allowed
us to test and compare learning rates across control and low-serotonin conditions,
the strong perseverative tendencies observed in this task prompt consideration of
alternative experimental designs, particularly of designs that allow presentation of
145
variably sized rewards, variable reward schedules, gambles with non-zero rewards,
and flexible location-outcome contingencies.
Of note, the differences in training regimes associated with the two species may
have contributed to the inconsistent evidences for impulsivity. Since the monkeys
were extensively trained, perhaps even overtrained, the minimal changes in visuosac-
cadic parameters revealed little evidence for increased impulsivity; it is likely that
the monkeys’ saccades were so thoroughly stereotyped that even the exquisite sensi-
tivity of the visual monitoring techniques used could detect no changes. In contrast,
the mice had relatively little exposure to the task. In fact, since each mouse only
performed 30 trials per day, the mice had less experience at the end of the experi-
ment than the monkeys had after a single session. This naivety allowed detection of
behavioral changes that revealed increased impulsivity.
6.4 Future directions: Serotonergic contributions to CGp function
Since the experiments described in this thesis support roles for both CGp and sero-
tonin in risky decision making, they raise an obvious question: do serotonin and CGp
interact, and if so, how? Anatomical evidence suggests that such an interaction is
probable: DRN projections target CGp and, although there are far fewer serotonin
receptors in CGp than in ACC (Varnas et al., 2004), the unusually high concen-
tration of 5-HT1A receptors in posterior cingulate cortex suggests the potential for
unique serotonergic modulation of CGp neurons (Barnes & Sharp, 1999; Baumgarten
& Gothert, 1997; Roth et al., 2004).
6.4.1 5HT1A receptors in CGp
5-HT1A receptors are inhibitory receptors that are generally found on the pre-
synaptic terminals of serotonergic synapses. Thus, as inhibitory autoreceptors, they
stereotypically cause feedback inhibition of serotonergic release (Baumgarten & Gothert,
146
1997; Stahl, 2008). Their presence in CGp as post-synaptic receptors is therefore
particularly intriguing, although it should be noted that these receptors are also
present in high concentrations in “limbic” structures such as the amygdala and the
hippocampus (Varnas et al., 2004). Presumably, increasing serotonergic activity in
CGp would result in inhibition of CGp neurons via 5-HT1A responses, dampening
CGp activity and responsiveness. In contrast, diminishing serotonin release in CGp
(for example, by RTD or PCPA treatment), might release baseline inhibition and
increase responsiveness to reward information and external stimuli.
It is difficult to predict the exact relationship between serotonergic contributions
to CGp and behavior, particularly given the minute effects of CGp microstimula-
tion on behavior (B. Y. Hayden et al., 2008). The complexity of local interactions,
upstream effectors and downstream contributions make such speculation even more
challenging. It is possible, however, that increased CGp responsiveness would affect
both phasic and tonic activity. Increased phasic activity might strengthen the signal
of salient events, thus heightening attention to external events. If the tonic activity
that distinguishes between large and small rewards is similarly upregulated, and we
assume that the resulting gain modulation resembles a linear multiplier, then the
difference between neuronal activity reflecting large and small rewards will increase.
At first glance, this change should strengthen the distinction between large and small
rewards, sharpening preferences for large rewards (notably, if we assume that the gain
modulation resembles a logarithmic process, then this effect becomes even stronger).
However, the observation that CGp microstimulation biases monkeys away from a
preferred option after large rewards (B. Y. Hayden et al., 2008) suggests that in-
creased CGp activity may instead blunt preferences by increasing the probability
of choosing otherwise low probability options. This latter speculation would better
explain the blunted preferences observed in depression, a disorder which is associated
with increased CGp activity (Mayberg et al., 1997; Milak et al., 2005).
147
Notably, functional imaging studies have only infrequently indicated that RTD
significantly changes activity in posterior cingulate cortex. More frequently, studies
identify RTD-induced changes in PFC, ACC, the basal ganglia and the inferior pari-
etal region (Fusar-Poli et al., 2006; Lamar et al., 2007). Studies of both attention and
executive function have demonstrated changes in these regions but not in CGp follow-
ing RTD (Bremner et al., 1997; Evers, Veen, Deursen, et al., 2006; Evers, Veen, Jolles,
Deutz, & Schmitt, 2006; Horacek et al., 2005). However, RTD does modulate poste-
rior cingulate cortical activity in depressed patients relative to controls (Neumeister
et al., 2004) and in normal subjects performing memory tasks (Allen et al., 2006).
Thus, it is possible that the putative CGp contributions to the effects of RTD on
risk-sensitive behavior are modulated by memory-related functions — a hypothesis
not inconsistent with the supposition that increased interest in the risky option may
reflect decreased sensitivity to (or memory of) less desirable outcomes (see Chapters 4
and 5).
6.4.2 5HT2A receptors in CGp
The observation that RTD only infrequently modulates CGp activity may be due to
conflicting interactions of serotonergic receptors found in CGp. While the inhibitory
5-HT1A receptors comprise the majority of serotonergic receptors in CGp, two ex-
citatory receptors have been found in this region: 5-HT1F and 5-HT2A (Barnes &
Sharp, 1999; Baumgarten & Gothert, 1997; Jakab & Goldman-Rakic, 1998; Morilak,
Garlow, & Ciaranello, 1993). Since the 5-HT1F receptors are poorly understood and
have been studied primarily as potential targets for migraine treatments (Ramadan,
Skljarevski, Phebus, & Johnson, 2003; Agosti, 2007), I will not discuss them further.
The 5-HT2A receptors are well-known since anti-psychotic drugs, in particular the
atypical anti-psychotics that ameliorate the negative symptoms of schizophrenia, an-
tagonize these receptors (Jakab & Goldman-Rakic, 1998; B. J. Jones & Blackburn,
148
2002; Marek et al., 2003).
5-HT2A receptor function has been implicated in impulsivity and aggression as
well as anxiety (J. L. Evenden, 1998; Johnson, Gallagher, & Holland, 2009; Nomura
& Nomura, 2006; Weisstaub et al., 2006; Winstanley, Theobald, Dalley, Glennon,
& Robbins, 2004). Increasing 5-HT2A activity decreases aggression (Johnson et al.,
2009) and selectively antagonizing 5-HT2A receptors decreases impulsive premature
responses (Winstanley, Theobald, et al., 2004). Since the behavioral and neuronal
mechanisms that underlie aggression, impulsivity and economic risk-seeking may not
be identical (J. Evenden, 1999a), speculation about behavior on a simple gambling
task on the basis of these results must necessarily be limited. However, measures
of aggression are correlated with the probability of exceedingly risky behaviors such
as suicide (Malkesman et al., 2009). If aggression and economic risk-preferences
are related, one might speculate that up-regulating 5-HT2A receptors in posterior
cingulate would decrease preference for a risky option. A more obvious prediction
is that locally antagonizing 5-HT2A receptors in CGp would increase willingness
to wait for larger rewards in a temporal delay task (for potential tasks, see (B. Y.
Hayden & Platt, 2007; Long & Platt, 2005)).
6.5 Future directions: other decision regions with serotonergic input
The suggestion that serotonergic manipulations should affect CGp activity is a spe-
cific instance of a larger hypothesis: that serotonergic manipulations should modulate
activity in brain regions that receive extensive DRN projections. Thus, neuronal ac-
tivity in regions such as the prefrontal cortex and anterior cingulate cortex should
also be modulated by serotonergic manipulations.
149
6.5.1 Prefrontal cortex
The prefrontal cortex is richly supplied with excitatory 5-HT2A receptors (Roth
et al., 2004). Focal serotonin depletion in PFC inhibits global ability to reverse
learned associations (Clarke, Dalley, Crofts, Robbins, & Roberts, 2004). However,
this treatment appears to affect specific types of behaviors. Prefrontal serotonin
depletion does not impair shifting attention between different dimensional qualities
of compound visual stimuli (Clarke et al., 2005). While this example confirms that
serotonin does contribute to normal PFC function, it also suggests that the inter-
action between neuromodulator and neuro-anatomical substrate may contribute to
specialized regional functionality.
Correlational studies of the relationship between neuronal activity and serotonin
transporter (5HTT ) genotype have provided additional evidence for the importance
of serotonergic contributions to neuronal function. The 5HTT gene is found in two
common forms (polymorphisms), the short (S) and long (L) alleles, that influence
the probability of depression (Canli & Lesch, 2007; Caspi et al., 2003; Hariri et al.,
2002; Heinz et al., 2007; K. P. Lesch & Gutknecht, 2005; K.-P. Lesch, 2007; Lenze et
al., 2005). Subjects with the S-allele, which predicts higher likelihood of depression,
have stronger functional connectivity between the PFC and the amygdala (Heinz
et al., 2005). This observation suggests that the form of the serotonin transporter
contributes to the strength of coupling between brain regions. In addition, it impli-
cates PFC-amygdalar connections in the pathology of depression, although it does
not specify whether this increased functional connectivity is causal or symptomatic.
6.5.2 Anterior cingulate cortex
The ACC contains particularly high concentrations of the 5HTT (Roth et al., 2004).
We might therefore speculate that modifying 5HTT activity could modulate both ob-
served behavior and neuronal activity in ACC. In fact, ACC activity in subjects with
150
S-allele 5HTT polymorphisms exceeds that observed in subjects with L-allele poly-
morphisms (Canli et al., 2005). Additionally, since 5HTT polymorphisms are linked
to the strength of functional coupling between the PFC and the amygdala (Heinz et
al., 2005), 5HTT polymorphisms might modulate coupling between ACC and other
decision-related regions. If connectivity is generally increased in people with the
5HTT S-allele, then the functional correlation between ACC activity and activity in
regions such as CGp and OFC should be stronger in the presence of the S-allele.
6.5.3 Summary
These examples demonstrate that anatomically segregated serotonergic projections
impact both behavior and localized neuronal activity. They also hint at the variety of
techniques available to test serotonergic contributions to activity in a brain region.
Both local and global serotonin depletion can be used to test the effects of low
serotonin levels; alternatively, receptor-specific pharmacological agents can be used
to inhibit or excite specific serotonin receptors (again, either locally or globally); and
global or site-directed mutagenesis can, similarly, be used to probe the contributions
of individual serotonin receptors. Teasing apart the mechanisms and interactions of
the serotonergic system and its interactions with anatomical substrates, including
the posterior cingulate cortex, will require a melange of techniques and experimental
approaches.
151
Appendix A
Supplementary Information for Rapid TryptophanDepletion
A.1 RTD mix composition
Rapid Tryptophan Depletion (RTD) mixes were composed of 0.83g alanine, 0.48g
glycine, 0.48g histidine, 1.2g isoleucine, 2.03g leucine, 1.65g lysine, 0.85g phenylala-
nine, 1.8g proline, 1.04g serine, 1.04g threonine, 1.04g tyrosine, 1.35g valine, 18.7mg
methionine, 17mg cysteine, 30.6mg arginine. To confirm that tryptophan levels were
not affected by the flavor included in the mix, in a subset of experiments the RTD
mix was replaced by a volume-matched mix of flavor and water. Since plasma tryp-
tophan levels did not differ significantly between these control experiments and the
normal baseline measurements (ANOVA n.s.), we combined the behavioral data from
these experiments in our analysis. Based on the distribution of plasma tryptophan
measurements (Figure 4.3a), we divided the data into two categories for analysis, a
normal tryptophan category with plasma tryptophan <6.5 ug/mL and a low tryp-
tophan category with plasma tryptophan ≤6.5 ug/mL. To control for the possibility
that consuming the amino acid mixture might cause behavioral changes indepen-
152
dent of lowered serotonin levels, we included 0.35g tryptophan in the depletion mix
for a separate subset of experiments (balanced mix). Unlike the RTD mix, this
balanced mixture did not significantly lower plasma tryptophan levels (Supplemen-
tary Fig A.1; ANOVA, plasma tryptophan vs treatment (baseline, balanced, RTD):
F=36.3, p<0.01; baseline vs balanced, post-hoc Bonferroni test, n.s.; baseline vs
RTD, Bonferroni p<0.01; balanced vs RTD, Bonferroni p<0.01); furthermore, the
monkeys’ behavior following administration of the balanced mix could not be dis-
tinguished from baseline (Supplementary Figure A.2; ANOVA, choice vs treatment
(baseline, balanced, RTD) and reward context: effect of treatment F=4.98, p<0.01,
effect of context, F=56.3, p<0.01, interaction term n.s. with both contexts in same
direction; post-hoc Bonferroni t-test, choices after RTD vs baseline, p<0.01; choices
after RTD vs balanced mix, p<0.05; post-hoc Bonferroni t-test, choices after bal-
anced mix vs baseline, n.s. (p>0.5)).
A.2 Plasma tyrosine measurements
Since tryptophan competes with large neutral amino acids (LNAAs), like tyrosine,
for access to transport across the blood-brain barrier, the decreased plasma tryp-
tophan/LNAA ratio that results from dietary tryptophan depletion contributes to
RTD’s effect on brain serotonin. Since we were able to acquire only limited volumes
of serum for analysis, we only measured plasma tyrosine levels as a proxy for LNAA
concentrations. Tyrosine was measured by HPLC followed by electrochemical detec-
tion using a modification of literature methods (Le Masurier, Oldenzeil, Lehman,
Cowen, & Sharp, 2006). Samples were injected directly onto a C18 reverse-phase
ODC column and eluted with a mobile phase comprised of 0.1 M Na2HPO4, 0.1
mM EDTA, 2.7 mM octane sulfonic acid and 12% methanol, pH 3.8. Samples were
detected with a BAS LC-4B amperometric detector with a dual 3 mM glassy carbon
electrode at a potential of .8V vs. an Ag/AgCl reference electrode. Samples were
153
quantitated in comparison to external standards.
We then calculated the plasma tryptophan/tyrosine ratio. As shown in Supple-
mentary Figure A.3, this ratio decreased significantly in RTD experiments relative
to baseline and balanced amino acid sessions (ANOVA, tryptophan/tyrosine ratio vs
experiment, F=14.2, p<0.01; Fischer/Bonferroni post-hoc t-test, baseline vs Tryp+,
n.s. but RTD vs baseline or Tryp+, p<0.01).
A.3 Mix administration
Since the amino acid mixture used may be unpleasant, we carefully monitored the
monkeys’ behavior for signs of discomfort. We found over the course of experiments
that the monkeys drank the RTD mix more slowly as they approached completion
of large (e.g. 200mL) mixes, suggesting satiety. In order to minimize potential dis-
comfort to the monkeys while maintaining consistent amino acid intake, we lowered
the volume of liquid used in the RTD mix and found that the monkeys consumed
the mix more quickly. Our analysis indicated no effect of fluid volume, independent
of treatment condition, on preferences, consistent with previous observations using
this task (A. McCoy & Platt, 2005b).
In addition, to ensure that the monkeys consumed the full amino acid mix, we
considered giving them access to the mix earlier. Since previously published exper-
iments demonstrated that RTD decreases plasma tryptophan for at least 18 hours
(Carpenter et al., 1998), we reasoned that allowing the monkey to consume part of
the mix the night before testing would minimize potential discomfort while ensuring
consistent tryptophan depletion. We carefully monitored amino acid and water in-
take and monitored both behavioral and biochemical results to determine whether
this procedural variation affected either biochemical depletion or behavior. Since
our analysis demonstrated that this procedural accommodation did not affect either
behavioral or biochemical results (Results, Chapter 4), we concluded that this data
154
was acceptable to include in our analysis.
We further note that, while we based the composition of our amino acid mix
on that found in the primate (monkey and human) literature, experiments in both
vervet monkeys and rats have used a minimal RTD mix composed of only 7 amino
acids (Moja et al., 1989; Young et al., 1989). The persistence and specificity of
serotonergic modulation found across these experiments despite the differences in
methodology suggests that dietary tryptophan depletion is robust across a variety
of procedural variations.
A.4 Safety premiums
To calculate each monkey’s safety premium, we used regression to identify the best-fit
line describing the relationship between the choice likelihood (y) and the EV differ-
ence between the safe and risky options (x):
Monkey SH
Baseline, y = 0.34 + 0.008x(r = 0.91, p < 0.00001)
RTD, y = 0.31 + 0.007 ∗ x(r = 0.98, p < 0.0000001)
Monkey NI
Baseline, y = 0.31 + 0.005x(r = 0.73, p = 0.016)
RTD, y = 0.34 + 0.003x(r = 0.65, p = 0.002)
Monkey BR
Baseline, y = 0.37 + 0.005x(r = 0.65, p = 0.012)
RTD, y = 0.24 + 0.005x(r = 0.61, p < 0.001)
A.5 Effect of reward and choice histories
We assessed the dependence of monkeys’ choices on reward history using a model
in which the relative probability of risky (pR) and safe (pS) choices depends on the
155
weighted sum over n past trials of reward outcomes (rjR = reward received from a
risky choice on trial j, rjS = reward received from a safe choice on trial j) and a
constant term (γ):
log(pR/pS) =∑j=1:n
(αj(rjR − rj
S)) + γ
To determine the number of past rewards (n) that best predicted choices, we
used Aikake’s Information Criterion (AIC) to determine the explanatory power of
models including reward history from the first to tenth trial preceding the current
trial in every session. We then compared the kernels formed by weighting the reward
coefficients (alpha) with the Akaike weights; these distributions were not significantly
different (t-test n.s.).
A.6 Figures
156
0
4
6
8
2
10
Pla
sma
tryp
toph
an (
ug/m
l)
RTDBaseline RTD+Tryp
12
Figure A.1: Plasma tryptophan levels diminish following RTD but not after abalanced amino acid mix. Plasma tryptophan levels measured after administration ofa balanced amino acid mixture (RTD mix + tryptophan; n=5 measurements) are notstatistically different from baseline measurements. After RTD, plasma tryptophandecreased significantly relative to either baseline or balanced conditions.
157
0.3
0.4
0.5
0.6P
roba
bilit
y of
saf
e ch
oice
Baseline RTDTryp+ control
Indifference
Figure A.2: RTD, unlike a balanced amino acid mix, shifts choice frequencies rel-ative to baseline. Monkeys choose the safe option less frequently following RTDthan at baseline or after administration of a balanced (RTD + tryptophan) mix-ture. However, choice frequencies following consumption of a balanced mixture wereindistinguishable from baseline frequencies.
158
0
0.6
1.0
0.2
2.0
Pla
sma
tryp
toph
an/ty
rosi
ne
0.4
0.8
1.2
1.4
1.6
1.8
RTDBaseline RTD+Tryp
**
Figure A.3: The plasma tryptophan/tyrosine ratio, like plasma tryptophan, con-firms successful tryptophan depletion. After RTD (n=7 measurements), the ra-tio of plasma tryptophan/tyrosine dropped significantly relative to both baseline(n=4 measurements; Bonferroni post-hoc t-test, p<0.01) and balanced (n=5 mea-surements; Bonferroni post-hoc t-test, p<0.01) conditions. However, the trypto-phan/tyrosine ratio did not change significantly in the balanced condition relative tobaseline (Bonferroni post-hoc t-test, n.s.).
159
0.5
0.6
0.7
0.8
Low Tryptophan
Pro
babi
lity
of c
orre
ct c
hoic
e0.9
Normal Tryptophan
n.s.
Figure A.4: Monkeys’ ability to distinguish reward sizes is not affected by RTD.To determine whether RTD dimishes monkeys’ ability to distinguish reward sizes,we measured their preferences with a simple discrimination task. As in the gamblingtask, monkeys initially foveated (±1-2◦) a central yellow LED (200-800ms) to begineach trial, after which two peripheral LEDs (one red, one green) were illuminateddiametrically opposite each other. Extinguishing the fixation LED cued a gaze shiftto either target (± 1-2◦) within 350 ms. Successful trials were rewarded with juiceand a 300ms noise. The color and location of the peripheral LEDs were randomlymatched on each trial so that each color appeared in each location on approximatelyhalf of the trials in a given block, but each color indicated a consistent, guaranteedreward within a block of 25-30 trials. We assigned one color to a large reward sizeand one to a small reward size within each block and tested discriminatory abilityacross a range of reward differences. We describe the choice of the larger optionas the correct choice. The monkeys’ performance did not differ between baselineand depleted conditions (ANOVA, correct frequency per block vs condition, F=1.4,p=0.23; baseline 68.8±3.7% correct choices across 38 blocks; RTD 75.8±4.7% correctchoices across 33 blocks).
160
Appendix B
Supplemental Information: Low serotonin increasesrisky decisions in mice
161
PCPAVehicle0
0.1
0.2
0.3
0.4
0.5
Fra
ctio
nal b
rain
dop
amin
e0.7
0.6
Figure B.1: Fractional dopamine is unaffected by PCPA treatment. For eachmouse, we calculated the ratio of brain dopamine to dopamine and its metabolite
DOPAC: [DA][DA]+[DOPAC]
. Comparing the resulting fractional dopamine metric between
conditions did not reveal a significant difference between treatment groups.
162
Fra
ctio
nal d
opam
ine
15107.52.5Frontal serotonin (pg/ug)
0 5 12.5
0.2
0.7
0.3
0.4
0.5
0.6
0.8
0.1
0.9
0
1.0
Figure B.2: Fractional dopamine levels do not depend on changes in brain sero-tonin. In case comparing fractional dopamine across treatment conditions failed toreveal a subtle relationship between dopamine and serotonin levels, we regressedthe fractional dopamine vs brain serotonin. Frontal cortex serotonin levels did notsignificantly predict fractional dopamine (linear regression, R2 = 0.08, n.s.).
163
PCPAVehicle0
0.1
0.2
0.3
0.4
0.5
Pro
babi
lity
of r
isky
cho
ice
(day
1) 0.7
0.6
Figure B.3: Vehicle and PCPA mice have indistinguishable initial preferences.Across the population, vehicle and control mice are approximately indifferent on thefirst day of testing (i.e. when naive to risky and safe options; one-way ANOVA, riskychoice frequency per mouse vs treatment group, F=0.08, n.s.).
164
Pro
babi
lity
of r
isky
cho
ice
(all
days
)
0.2
0.7
0.3
0.4
0.5
0.6
0.8
0.1
0.9
0
1.0
0.2 0.70.3 0.4 0.5 0.6 0.80.1 0.90 1.0Probability of risky choice (first day)
Figure B.4: Vehicle and PCPA risk preferences differ despite similar early ex-periences. Mice that initially choose the risky target less than chance (lower leftquadrant) are likely to become risk averse. Similarly, control mice that choose therisky target more often than chance are likely to become risk averse (lower rightquadrant). In contrast, a significant subset of the PCPA mice — mostly mice thatinitially choose the risky option more than chance — become risk seeking (upperright quadrant). Vertical dashed line demarcates indifference between safe and riskyoptions on the first day of testing; horizontal dashed line, indifference across days.
165
Pro
babi
lity
of r
isky
cho
ice
0.2
0.7
0.3
0.4
0.5
0.6
0.8
0.1
0.9
0
1.0
No reward Large riskySafe
Previous rewardFigure B.5: Shifted risk preferences reflect global changes in choice probability.We sorted mice by both treatment group and overall risk attitude. Blue, controlmice; red, PCPA treated mice; solid lines, mice with overall safe preferences; dashedlines, mice more likely to choose the risky option overall.
166
1.0
2.2
1.2
1.4
1.8
2.0
Risky choice trials
Nos
epok
es fo
r re
war
d (n
)2.4
Safe choice trials
1.6 **
**
Figure B.6: PCPA-treated mice nosepoke more frequently than sham controls, adifference that is enhanced on trials when mice choose the safe option. Data aresimilar when mice are sorted by overall risk attitudes (data not shown). Asterisk,p<0.01, post-hoc Fisher LSD (or Bonferroni, for that matter) t-test.
167
Bibliography
Abrams, R. A., & Jonides, J. (1988, Aug). Programming saccadic eye movements.J Exp Psychol Hum Percept Perform, 14 (3), 428–443.
Aggleton, J. P., & Passingham, R. E. (1981, Dec). Syndrome produced by lesions ofthe amygdala in monkeys (macaca mulatta). J Comp Physiol Psychol, 95 (6),961–977.
Agosti, R. M. (2007, 08). 5ht1f- and 5ht7-receptor agonists for the treatment ofmigraines. CNS & Neurological Disorders - Drug Targets, 6 (4), 235–237.
Akhondzadeh, S. (2001). The 5-HT hypothesis of schizophrenia. IDrugs, 4 (1369-7056), 295-300.
Allais, M. (1953). The foundations of a positive theory of choice involving risk anda criticism of the postulates and axioms of the American School. In M. Allais& O. Hagen (Eds.), Expected utility hypotheses and the allais paradox (Vol. 21,p. 503-546). Dordrecht, Holland: J. Reidel.
Allen, P. P., Cleare, A. J., Lee, F., Fusar-Poli, P., Tunstall, N., Fu, C. H. Y., et al.(2006, Sep). Effect of acute tryptophan depletion on pre-frontal engagement.Psychopharmacology (Berl), 187 (4), 486–497.
Amaral, D. G. (2003, Dec). The amygdala, social behavior, and danger detection.Ann N Y Acad Sci, 1000, 337–347.
Apter, A., Praag, H. M. van, Plutchik, R., Sevy, S., Korn, M., & Brown, S. L. (1990,May). Interrelationships among anxiety, aggression, impulsivity, and mood: aserotonergically linked cluster? Psychiatry Res, 32 (2), 191–199.
Asch, P., & Quandt, R. E. (1990). Risk love. The Journal of Economic Education,21 (4), 422-426.
Bachevalier, J., & Nemanic, S. (2008). Memory for spatial location and object-placeassociations are differently processed by the hippocampal formation, parahip-pocampal areas th/tf and perirhinal cortex. Hippocampus, 18 (1), 64-80.
168
Bahill, A. T., Clark, M. R., & Stark, L. (1975). The main sequence, a tool forstudying human eye movements. Mathematical Biosciences, 24 (3-4), 191–204.
Balleine, B. W., & Dickinson, A. (1998). Goal-directed instrumental action: contin-gency and incentive learning and their cortical substrates. Neuropharmacology,37 (4-5), 407–419.
Banerjee, A. V. (1992). A simple model of herd behavior. The Quarterly Journal ofEconomics, 107 (3), 797–817.
Bari, A., Theobald, D. E., Mari, A. C., & Robbins, T. W. (2008). Serotonin mod-ulates sensitivity to reward and negative feedback in the probabilistic reversallearning task. Society for Neuroscience.
Barnes, N. M., & Sharp, T. (1999). A review of central 5-HT receptors and theirfunction. (No. 0028-3908).
Barraclough, D. J., Conroy, M. L., & Lee, D. (2004). Prefrontal cortex and decisionmaking in a mixed-strategy game. Nat Neurosci, 7 (4), 404–410.
Bateson, M. (2002). Recent advances in our understanding of risk-sensitive foragingpreferences. Proc Nutr Soc, 61 (0029-6651), 509-16.
Baumgarten, H. G., & Gothert, M. (1997). Serotoninergic neurons and 5-HT recep-tors in the cns [Book].
Baumgarten, H. G., & Grozdanovic, Z. (1995). Psychopharmacology of central sero-tonergic systems. (No. 0176-3679).
Bayer, H., & Glimcher, P. (2005, Jul). Midbrain dopamine neurons encode a quan-titative reward prediction error signal. Neuron, 47 (1), 129-41.
Beckmann, M., Johansen-Berg, H., & Rushworth, M. F. S. (2009, 1). Connectivity-based parcellation of human cingulate cortex and its relation to functionalspecialization. Journal of Neuroscience, 29 (4), 1175–1190.
Bennett, A. J., Lesch, K. P., Heils, A., Long, J. C., Lorenz, J. G., Shoaf, S. E., etal. (2002). Early experience and serotonin transporter gene variation interactto influence primate cns function. Mol Psychiatry, 7 (1), 118–122.
Berman, R. A., Colby, C. L., Genovese, C. R., Voyvodic, J. T., Luna, B., Thulborn,K. R., et al. (1999). Cortical networks subserving pursuit and saccadic eyemovements in humans: an MRI study. Hum Brain Mapp, 8 (1065-9471), 209-25.
169
Bernoulli, D. (1954). Specimen theoriae novae de mensura sortis [Exposition of anew theory on the measurement of risk] (L. Sommer, Trans.). Econometrica,22, 23-36. (Original work published 1738)
Bernstein, P. L. (1996). Against the gods: The remarkable story of risk. New York,NY: John Wiley & Sons, Inc.
Bichot, N. P., Chenchal Rao, S., & Schall, J. D. (2001). Continuous processing inmacaque frontal cortex during visual search. Neuropsychologia, 39 (9), 972–982.
Bichot, N. P., & Schall, J. D. (1999, Jun). Effects of similarity and history on neuralmechanisms of visual selection. Nat Neurosci, 2 (6), 549–554.
Bikhchandani, S., Hirshleifer, D., & Welch, I. (1998, Summer). Learning from thebehavior of others: Conformity, fads, and informational cascades. Journal ofEconomic Perspectives, 12 (3), 151–170.
Bizot, J., Le Bihan, C., Puech, A. J., Hamon, M., & Thiebot, M. (1999). Serotoninand tolerance to delay of reward in rats. Psychopharmacology (Berl), 146 (4),400-12.
Blum, K., Braverman, E. R., Holder, J. M., Lubar, J. F., Monastra, V. J., Miller,D., et al. (2000). Reward deficiency syndrome: a biogenetic model for thediagnosis and treatment of impulsive, addictive, and compulsive behaviors. JPsychoactive Drugs, 32 Suppl(0279-1072), i-iv, 1-112.
Bremner, J. D., Innis, R. B., Salomon, R. M., Staib, L. H., Ng, C. K., Miller, H. L.,et al. (1997, Apr). Positron emission tomography measurement of cerebralmetabolic correlates of tryptophan depletion-induced depressive relapse. ArchGen Psychiatry, 54 (4), 364–374.
Brown, G. L., & Linnoila, M. I. (1990, Apr). Csf serotonin metabolite (5-hiaa)studies in depression, impulsivity, and violence. J Clin Psychiatry, 51 Suppl,31–41.
Bruce, C. J., & Goldberg, M. E. (1985, Mar). Primate frontal eye fields. i. singleneurons discharging before saccades. J Neurophysiol, 53 (3), 603–635.
Bruce, C. J., Goldberg, M. E., Bushnell, M. C., & Stanton, G. B. (1985, Sep). Pri-mate frontal eye fields. ii. physiological and anatomical correlates of electricallyevoked eye movements. J Neurophysiol, 54 (3), 714–734.
Buckholtz, J. W., Asplund, C. L., Dux, P. E., Zald, D. H., Gore, J. C., Jones, O. D.,et al. (2008, Dec). The neural correlates of third-party punishment. Neuron,60 (5), 930–940.
170
Cabeza, R., Dolcos, F., Prince, S. E., Rice, H. J., Weissman, D. H., & Nyberg, L.(2003). Attention-related activity during episodic memory retrieval: a cross-function fmri study. Neuropsychologia, 41 (3), 390–399.
Cahir, M., Ardis, T., Reynolds, G., & Cooper, S. (2007). Acute and chronic tryp-tophan depletion differentially regulate central 5-ht(1a) and 5-ht (2a) receptorbinding in the rat. Psychopharmacology (Berl), 190 (4), 497-506.
Camerer, C. F. (2000). Prospect theory in the wild: evidence from the field. InD. Kahneman & A. Tversky (Eds.), Choices, values and frames (p. 288ff).Cambridge University Press.
Canli, T., & Lesch, K. (2007). Long story short: the serotonin transporter in emotionregulation and social cognition. Nat Neurosci, 10 (9), 1103–1109.
Canli, T., Omura, K., Haas, B. W., Fallgatter, A., Constable, R. T., & Lesch,K. P. (2005, 08). Beyond affect: A role for genetic variation of the serotonintransporter in neural activation during a cognitive attention task. Proceedingsof the National Academy of Sciences of the United States of America, 102 (34),12224–12229.
Caraco, T. (1981). Energy budgets, risk and foraging preferences in dark-eyed juncos(junco hyemalis). Behavioral Ecology and Sociobiology, 8 (3), 213–217.
Carpenter, L. L., Anderson, G. M., Pelton, G. H., Gudin, J. A., Kirwin, P. D., Price,L. H., et al. (1998). Tryptophan depletion during continuous csf sampling inhealthy human subjects. Neuropsychopharmacology, 19 (1), 26-35.
Caspi, A., Sugden, K., Moffitt, T. E., Taylor, A., Craig, I. W., Harrington, H., et al.(2003). Influence of life stress on depression: moderation by a polymorphismin the 5-htt gene. Science, 301 (5631), 386–389.
Chau, D. T., Roth, R. M., & Green, A. I. (2004). The neural circuitry of rewardand its relevance to psychiatric disorders. Curr Psychiatry Rep, 6 (1523-3812),391-9.
Chen, M. K., Lakshminarayanan, V., & Santos, L. R. (2006). How basic are behav-ioral biases? evidence from capuchin monkey trading behavior. The Journalof Political Economy, 114 (3), 517-537.
Cheney, D. L., & Seyfarth, R. M. (2007). Baboon metaphysics: the evolution of asocial mind. Chicago: University of Chicago Press.
Chugani, D. C., Muzik, O., Behen, M., Rothermel, R., Janisse, J. J., Lee, J., et al.(1999). Developmental changes in brain serotonin synthesis capacity in autistic
171
and nonautistic children. Ann Neurol, 45 (0364-5134), 287-95.
Clarke, H. F., Dalley, J. W., Crofts, H. S., Robbins, T. W., & Roberts, A. C. (2004).Cognitive inflexibility after prefrontal serotonin depletion. Science, 304 (1095-9203 (Electronic)), 878-80.
Clarke, H. F., Walker, S. C., Crofts, H. S., Dalley, J. W., Robbins, T. W., & Roberts,A. C. (2005). Prefrontal serotonin depletion affects reversal learning but notattentional set shifting. J Neurosci, 25 (1529-2401 (Electronic)), 532-8.
Cook, E. H. (1990). Autism: review of neurochemical investigation. (No. 0887-4476).
Corchs, F., Nutt, D. J., Hood, S., & Bernik, M. (n.d.). Serotonin and sensitivityto trauma-related exposure in selective serotonin reuptake inhibitors-recoveredposttraumatic stress disorder. Biological Psychiatry, In Press, Corrected Proof,–.
Courville, A. C., Daw, N. D., & Touretzky, D. S. (2006, Jul). Bayesian theories ofconditioning in a changing world. Trends Cogn Sci, 10 (7), 294–300.
Cromwell, H. C., Hassani, O. K., & Schultz, W. (2005). Relative reward processingin primate striatum. Exp Brain Res, 162 (4), 520–525.
Cromwell, H. C., & Schultz, W. (2003, May). Effects of expectations for differentreward magnitudes on neuronal activity in primate striatum. J Neurophysiol,89 (5), 2823–2838.
d’Acremont, M., & Linden, M. Van der. (2007). How is impulsivity related todepression in adolescence? evidence from a french validation of the cognitiveemotion regulation questionnaire. Journal of Adolescence, 30 (2), 271–282.
Dailly, E., Chenu, F., Petit-Demouliere, B., & Bourin, M. (2006). Specificity andefficacy of noradrenaline, serotonin depletion in discrete brain areas of swissmice by neurotoxins. J Neurosci Methods, 150 (1), 111–115.
Damasio, H., Grabowski, T., Frank, R., Galaburda, A. M., & Damasio, A. R. (1994,May). The return of phineas gage: clues about the brain from the skull of afamous patient. Science, 264 (5162), 1102–1105.
Daw, N. D., Kakade, S., & Dayan, P. (2002). Opponent interactions between sero-tonin and dopamine. Neural Netw, 15 (4-6), 603–616.
Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006).Cortical substrates for exploratory decisions in humans. Nature, 441 (7095),876-9.
172
Deakin, J. F. W. (2003). Depression and antisocial personality disorder: two con-trasting disorders of 5HT function. J Neural Transm Suppl(0303-6995), 79-93.
Dean, H. L., Crowley, J. C., & Platt, M. L. (2004). Visual and saccade-relatedactivity in macaque posterior cingulate cortex. J Neurophysiol, 92 (5), 3056-68.
Dean, H. L., & Platt, M. L. (2006). Allocentric spatial referencing of neuronalactivity in macaque posterior cingulate cortex. J Neurosci, 26 (4), 1117-27.
DeLong, G. R. (1999, Mar). Autism: new data suggest a new hypothesis. Neurology,52 (5), 911–916.
Dickhaut, J., McCabe, K., Nagode, J., Rustichini, A., Smith, K., & Pardo, J. (2003,Mar). The impact of the certainty context on the process of choice. Proc NatlAcad Sci U S A, 100 (6), 3536-41.
Dickinson, A. (1980). Contemporary animal learning theory. Cambridge: CambridgeUniversity Press.
Dijksterhuis, A., Bos, M. W., Nordgren, L. F., & Baaren, R. B. van. (2006). Onmaking the right choice: the deliberation-without-attention effect. Science,311 (5763), 1005-7.
Ding, L., & Hikosaka, O. (2006, Jun). Comparison of reward modulation in thefrontal eye field and caudate of the macaque. J Neurosci, 26 (25), 6695–6703.
Dolan, M., Anderson, I. M., & Deakin, J. F. (2001, Apr). Relationship between 5-ht function and impulsivity and aggression in male offenders with personalitydisorders. Br J Psychiatry, 178, 352–359.
Dreber, A., Apicella, C. L., Eisenberg, D. T. A., Garcia, J. R., Zamore, R. S., Lum,J. K., et al. (2009). The 7r polymorphism in the dopamine receptor d4 gene(drd4) is associated with financial risk taking in men. Evolution and HumanBehavior, 30 (2), 85–92.
Drzezga, A., Riemenschneider, M., Strassner, B., Grimmer, T., Peller, M., Knoll, A.,et al. (2005). Cerebral glucose metajellyfishmamasm in patients with AD anddifferent APOE genotypes. (No. 1526-632X (Electronic)).
Einhauser, W., Mundhenk, T. N., Baldi, P., Koch, C., & Itti, L. (2007). A bottom–up model of spatial attention predicts human error patterns in rapid scenerecognition. Journal of Vision, 7 (10), 1-13.
Ellsberg, D. (1961). Risk, ambiguity, and the savage axioms. Quarterly Journal ofEconomics, 75, 643-669.
173
Evenden, J. (1999a). Impulsivity: a discussion of clinical and experimental findings.J Psychopharmacol, 13 (0269-8811), 180-92.
Evenden, J. (1999b). The pharmacology of impulsive behaviour in rats v: the effectsof drugs on responding under a discrimination task using unreliable visualstimuli. Psychopharmacology (Berl), 143 (2), 111-22.
Evenden, J. L. (1998). The pharmacology of impulsive behaviour in rats iv: theeffects of selective serotonergic agents on a paced fixed consecutive numberschedule. Psychopharmacology (Berl), 140 (3), 319-30.
Evenden, J. L. (1999). The pharmacology of impulsive behaviour in rats vii: theeffects of serotonergic agonists and antagonists on responding under a discrimi-nation task using unreliable visual stimuli. Psychopharmacology (Berl), 146 (4),422-31.
Evenden, J. L., & Ryan, C. N. (1999). The pharmacology of impulsive behaviour inrats vi: the effects of ethanol and selective serotonergic drugs on response choicewith varying delays of reinforcement. Psychopharmacology (Berl), 146 (4), 413-21.
Evers, E. A. T., Veen, F. M. van der, Deursen, J. A. van, Schmitt, J. A. J., Deutz,N. E. P., & Jolles, J. (2006, Aug). The effect of acute tryptophan depletionon the bold response during performance monitoring and response inhibitionin healthy male volunteers. Psychopharmacology (Berl), 187 (2), 200–208.
Evers, E. A. T., Veen, F. M. van der, Jolles, J., Deutz, N. E. P., & Schmitt, J. A. J.(2006, Aug). Acute tryptophan depletion improves performance and modulatesthe bold response during a stroop task in healthy females. Neuroimage, 32 (1),248–255.
Fairbanks, L. A., Jorgensen, M. J., Huff, A., Blau, K., Hung, Y.-Y., & Mann, J. J.(2004). Adolescent impulsivity predicts adult dominance attainment in malevervet monkeys. Am J Primatol, 64 (1), 1-17.
Fecteau, J. H., & Munoz, D. P. (2006, Aug). Salience, relevance, and firing: apriority map for target selection. Trends Cogn Sci, 10 (8), 382–390.
Fiorillo, C., Tobler, P., & Schultz, W. (2003, Mar). Discrete coding of rewardprobability and uncertainty by dopamine neurons. Science, 299 (5614), 1898-902.
Fletcher, P. J. (1995). Effects of combined or separate 5,7-dihydroxytryptaminelesions of the dorsal and median raphe nuclei on responding maintained by aDRL 20s schedule of food reinforcement. Brain Res, 675 (0006-8993), 45-54.
174
Fox, M. D., Snyder, A. Z., Vincent, J. L., Corbetta, M., Van Essen, D. C., & Raichle,M. E. (2005). The human brain is intrinsically organized into dynamic, anti-correlated functional networks. Proc Natl Acad Sci U S A, 102 (27), 9673-8.
Freeman, J. H. J., Cuppernell, C., Flannery, K., & Gabriel, M. (1996, Oct). Limbicthalamic, cingulate cortical and hippocampal neuronal correlates of discrimi-native approach learning in rabbits. Behav Brain Res, 80 (1-2), 123–136.
Friedman, M., & Savage, L. J. (1948). The utility analysis of choices involving risk.The Journal of Political Economy, 56 (4), 279-304.
Fusar-Poli, P., Allen, P., McGuire, P., Placentino, A., Cortesi, M., & Perez, J. (2006).Neuroimaging and electrophysiological studies of the effects of acute trypto-phan depletion: a systematic review of the literature. Psychopharmacology(Berl), 188 (2), 131-43.
Gabriel, M., & Orona, E. (1982, Jun). Parallel and serial processes of the prefrontaland cingulate cortical systems during behavioral learning. Brain Res Bull, 8 (6),781–785.
Gabriel, M., Sparenborg, S., & Kubota, Y. (1989). Anterior and medial thala-mic lesions, discriminative avoidance learning, and cingulate cortical neuronalactivity in rabbits. Exp Brain Res, 76 (2), 441–457.
Gainetdinov, R. R., Wetsel, W. C., Jones, S. R., Levin, E. D., Jaber, M., & Caron,M. G. (1999). Role of serotonin in the paradoxical calming effect of psychos-timulants on hyperactivity. Science, 283 (5400), 397-401.
Gigerenzer, G. (2000). Adaptive thinking: rationality in the real world. New York:Oxford University Press.
Gigerenzer, G. (2008). Rationality for mortals: how people cope with uncertainty.Oxford: Oxford University Press.
Gigerenzer, G., Czeslinski, J., & Martignon, L. (2002). How good are fast and frugalheuristics? Decision Science and Technology. (Original work published 1999)
Gigerenzer, G., & Todd, P. M. (1999). Simple heuristics that make us smart. NewYork: Oxford University Press.
Gobbi, G., Murphy, D. L., Lesch, K., & Blier, P. (2001). Modifications of theserotonergic system in mice lacking serotonin transporters: an in vivo electro-physiological study. J Pharmacol Exp Ther, 296 (3), 987–995.
Gottfried, J. A., O’Doherty, J., & Dolan, R. J. (2002, Dec). Appetitive and aversive
175
olfactory learning in humans studied using event-related functional magneticresonance imaging. J Neurosci, 22 (24), 10829–10837.
Grados, M. A. (2009, Apr). The genetics of obsessive-compulsive disorder andtourette’s syndrome: what are the common factors? Curr Psychiatry Rep,11 (2), 162–166.
Greene, J. D., Nystrom, L. E., Engell, A. D., Darley, J. M., & Cohen, J. D. (2004,Oct). The neural bases of cognitive conflict and control in moral judgment.Neuron, 44 (2), 389–400.
Gross, C. G. (1994, 9). How inferior temporal cortex became a visual area. CerebralCortex, 4 (5), 455–469.
Gross, C. G. (2008). Single neuron studies of inferior temporal cortex. Neuropsy-chologia, 46 (3), 841–852.
Guiso, L., & Paiella, M. (2007). Risk aversion, wealth, and background risk (Eco-nomics Working Papers No. ECO2007/47). European University Institute.(available at http://ideas.repec.org/p/eui/euiwps/eco2007-47.html)
Ha, T. H., Youn, T., Ha, K. S., Rho, K. S., Lee, J. M., Kim, I. Y., et al. (2004). Graymatter abnormalities in paranoid schizophrenia and their clinical correlations.Psychiatry Res, 132 (0165-1781), 251-60.
Hall, G. (1994). Pavlovian conditioning: laws of association. In N. J. Mackintosh(Ed.), Animal learning and cognition (pp. 15–44). 525 B St, Ste 1900, SanDiego, CA 92101-4495: Academic Press, Inc.
Hariri, A. R., Mattay, V. S., Tessitore, A., Kolachana, B., Fera, F., Goldman, D., etal. (2002, 7). Serotonin transporter genetic variation and the response of thehuman amygdala. Science, 297 (5580), 400–403.
Harrison, A. A., Everitt, B. J., & Robbins, T. W. (1997). Central 5-HT depletionenhances impulsive responding without affecting the accuracy of attentionalperformance: interactions with dopaminergic mechanisms. Psychopharmacol-ogy (Berl), 133 (0033-3158), 329-42.
Harrison, A. A., Everitt, B. J., & Robbins, T. W. (1999). Central serotonin depletionimpairs both the acquisition and performance of a symmetrically reinforcedgo/no-go conditional visual discrimination. Behav Brain Res, 100 (0166-4328),99-112.
Hassani, O. K., Cromwell, H. C., & Schultz, W. (2001). Influence of expectationof different rewards on behavior-related neuronal activity in the striatum. J
176
Neurophysiol, 85 (6), 2477–2489.
Hayden, B., Smith, D., & Platt, M. (2009, Mar). Electrophysiological correlates ofdefault-mode processing in macaque posterior cingulate cortex. Proc Natl AcadSci U S A.
Hayden, B. Y., Nair, A. C., McCoy, A. N., & Platt, M. L. (2008, Oct). Posteriorcingulate cortex mediates outcome-contingent allocation of behavior. Neuron,60 (1), 19–25.
Hayden, B. Y., Pearson, J. M., & Platt, M. L. (2009). Fictive reward signals inanterior cingulate cortex. Science, in press.
Hayden, B. Y., & Platt, M. L. (2007). Temporal discounting predicts risk sensitivityin rhesus macaques. Curr Biol, 17 (1), 49-53.
Hazlett, E. A., New, A. S., Newmark, R., Haznedar, M. M., Lo, J. N., Speiser,L. J., et al. (2005). Reduced anterior and posterior cingulate gray matter inborderline personality disorder. Biol Psychiatry, 58 (0006-3223), 614-23.
Haznedar, M. M., Buchsbaum, M. S., Hazlett, E. A., Shihabuddin, L., New, A., &Siever, L. J. (2004). Cingulate gyrus volume and metajellyfishmamasm in theschizophrenia spectrum. Schizophr Res, 71 (0920-9964), 249-62.
Hecht, S., Shlaer, S., & Pirenne, M. H. (1942, 7). Energy, quanta, and vision. TheJournal of General Physiology, 25 (6), 819–840.
Heinz, A., Braus, D. F., Smolka, M. N., Wrase, J., Puls, I., Hermann, D., et al.(2005, Jan). Amygdala-prefrontal coupling depends on a genetic variation ofthe serotonin transporter. Nat Neurosci, 8 (1), 20–21.
Heinz, A., Smolka, M. N., Braus, D. F., Wrase, J., Beck, A., Flor, H., et al. (2007).Serotonin transporter genotype (5-httlpr): effects of neutral and undefinedconditions on amygdala activation. Biol Psychiatry, 61 (8), 1011–1014.
Hernandez, A., Zainos, A., & Romo, R. (2000). Neuronal correlates of sensory dis-crimination in the somatosensory cortex. Proceedings of the National Academyof Sciences of the United States of America, 97 (11), 6191–6196.
Herrnstein, R., & Heyman, G. (1979, Mar). Is matching compatible with reinforce-ment maximization on concurrent variable interval variable ratio? J Exp AnalBehav, 31 (2), 209–223.
Herrnstein, R. J., Richard J. (1961, Jul). Relative and absolute strength of responseas a function of frequency of reinforcement. J Exp Anal Behav, 4, 267–272.
177
Higley, J. D., Mehlman, P. T., Higley, S. B., Fernald, B., Vickers, J., Lindell, S. G., etal. (1996). Excessive mortality in young free-ranging male nonhuman primateswith low cerebrospinal fluid 5-hydroxyindoleacetic acid concentrations. ArchGen Psychiatry, 53 (0003-990X), 537-43.
Hikosaka, K., & Watanabe, M. (2000). Delay activity of orbital and lateral prefrontalneurons of the monkey varying with different rewards. Cereb Cortex, 10 (3),263–271.
Hikosaka, K., & Watanabe, M. (2004). Long- and short-range reward expectancy inthe primate orbitofrontal cortex. Eur J Neurosci, 19 (4), 1046–1054.
Hinson, J., & Staddon, J. (1983, Nov). Matching, maximizing, and hill-climbing. JExp Anal Behav, 40 (3), 321–331.
Holland, P. C., & Gallagher, M. (2004). Amygdala-frontal interactions and rewardexpectancy. Curr Opin Neurobiol, 14 (0959-4388), 148-55.
Horacek, J., Zavesicka, L., Tintera, J., Dockery, C., Platilova, V., Kopecek, M., etal. (2005). The effect of tryptophan depletion on brain activation measuredby functional magnetic resonance imaging during the stroop test in healthysubjects. Physiol Res, 54 (2), 235–244.
Hoyer, D., Hannon, J. P., & Martin, G. R. (2002). Molecular, pharmacological andfunctional diversity of 5-HT receptors. Pharmacol Biochem Behav, 71 (0091-3057), 533-54.
Hsee, C. K., & Rottenstreich, Y. (2004). Music, pandas, and muggers: on theaffective psychology of value. (No. 0096-3445).
Huettel, S. A., Song, A. W., & McCarthy, G. (2005, March). Decisions under un-certainty: probabilistic context influences activation of prefrontal and parietalcortices. J Neurosci, 25 (13), 3304–3311.
Huettel, S. A., Stowe, C. J., Gordon, E. M., Warner, B. T., & Platt, M. L. (2006).Neural signatures of economic preferences for risk and ambiguity. Neuron,49 (5), 765-75.
Ikonomov, O. C., Stoynev, A. G., Goranova-Stoyneva, I. I., Popov, A., & Minkova,V. A. (1990). Nephrotoxic effect of the specific brain serotonin depletor para-chlorophenylalanine. Acta Physiol Hung, 76 (0231-424X), 191-9.
Imhof, L. A., Fudenberg, D., & Nowak, M. A. (2007, Aug 7). Tit-for-tat or win-stay,lose-shift? J Theor Biol, 247 (3), 574–580.
178
Isles, A. R., Humby, T., & Wilkinson, L. S. (2003). Measuring impulsivity inmice using a novel operant delayed reinforcement task: effects of behaviouralmanipulations and d-amphetamine. Psychopharmacology (Berl), 170 (4), 376-82.
Itti, L., & Baldi, P. (2008). Bayesian surprise attracts human attention. VisionResearch, In press, corrected proof, –.
Jacobs, B. L., & Azmitia, E. C. (1992). Structure and function of the brain serotoninsystem. Physiol Rev, 72 (0031-9333), 165-229.
Jakab, R. L., & Goldman-Rakic, P. S. (1998, 01). 5-hydroxytryptamine2a serotoninreceptors in the primate cerebral cortex: Possible site of action of hallucinogenicand antipsychotic drugs in pyramidal cell apical dendrites. Proceedings of theNational Academy of Sciences of the United States of America, 95 (2), 735–740.
James, W. (1892). Psychology. New York: H. Holt and company.
Jans, L. A. W., Riedel, W. J., Markus, C. R., & Blokland, A. (2007). Seroton-ergic vulnerability and depression: assumptions, experimental evidence andimplications. Mol Psychiatry, 12 (6), 522–543.
Johnson, A. W., Gallagher, M., & Holland, P. C. (2009, Jan). The basolateralamygdala is critical to the expression of pavlovian and instrumental outcome-specific reinforcer devaluation effects. J Neurosci, 29 (3), 696–704.
Jones, B., Barnes, J., Uylings, H., Fox, N., Frost, C., Witter, M., et al. (2006, Jan).Differential regional atrophy of the cingulate gyrus in Alzheimer disease: Avolumetric MRI study. Cereb Cortex.
Jones, B. J., & Blackburn, T. P. (2002). The medical benefit of 5-HT research.Pharmacol Biochem Behav, 71 (0091-3057), 555-68.
Kable, J., & Glimcher, P. (2007). The neural correlates of subjective value duringintertemporal choice. Nat Neurosci.
Kacelnik, A., & Bateson, M. (1996). Risky Theories–The Effects of Variance onForaging Decisions. Amer. Zool., 36 (4), 402-434.
Kahneman, D., & Tversky, A. (1979). Prospect theory: an analysis of decision underrisk. Econometrica, 5, 297–323.
Kamin, L. J. (1969). Predictability, surprise, attention, and conditioning. InB. Campbell & R. M. Church (Eds.), Punishment and aversive behavior (pp.279—296). New York: Appleton-Century-Crofts.
179
Kapur, S., & Remington, G. (1996). Serotonin-dopamine interaction and its relevanceto schizophrenia. Am J Psychiatry, 153 (0002-953X), 466-76.
Kennerley, S. W., Walton, M. E., Behrens, T. E. J., Buckley, M. J., & Rushworth,M. F. S. (2006). Optimal decision making and the anterior cingulate cortex.Nat Neurosci, 9 (7), 940–947.
Klaassen, T., Riedel, W., Deutz, N., Someren, A. van, & Praag, H. van. (1999, Jan).Specificity of the tryptophan depletion method. Psychopharmacology (Berl),141 (3), 279-86.
Klein, J. T., Deaner, R. O., & Platt, M. L. (2008, Mar). Neural correlates of socialtarget value in macaque parietal cortex. Curr Biol, 18 (6), 419–424.
Knight, F. H. (1921). Risk, uncertainty and profit. Hart, Schaffner and Marx;Houghton Mifflin Co.
Knoch, D., Gianotti, L. R. R., Pascual-Leone, A., Treyer, V., Regard, M., Hohmann,M., et al. (2006). Disruption of right prefrontal cortex by low-frequency repeti-tive transcranial magnetic stimulation induces risk-taking behavior. J Neurosci,26 (24), 6469-72.
Knutson, B., & Cooper, J. (2005, Aug). Functional magnetic resonance imaging ofreward prediction. Curr Opin Neurol, 18 (4), 411-7.
Knutson, B., Fong, G. W., Bennett, S. M., Adams, C. M., & Hommer, D. (2003).A region of mesial prefrontal cortex tracks monetarily rewarding outcomes:characterization with rapid event-related fmri. Neuroimage, 18 (2), 263-72.
Koe, B. K. (1971). Tryptophan hydroxylase inhibitors. Fed Proc, 30 (3), 886–896.
Koe, B. K., & Corkey, R. F. J. (1976). Inhibition of rat brain tryptophan hydroxy-lation with p-chloroamphetamine. Biochem Pharmacol, 25 (1), 31–35.
Koe, B. K., & Weissman, A. (1968). The pharmacology of para-chlorophenylalanine,a selective depletor of serotonin stores. Adv Pharmacol, 6 (Pt B), Suppl 1:29-47.
Koenen, K. C., Aiello, A. E., Bakshis, E., Amstadter, A. B., Ruggiero, K. J., Acierno,R., et al. (2009, 3). Modification of the association between serotonin trans-porter genotype and risk of posttraumatic stress disorder in adults by county-level social environment. American Journal of Epidemiology, 169 (6), 704–711.
Kreek, M. J., Nielsen, D. A., Butelman, E. R., & LaForge, K. S. (2005). Geneticinfluences on impulsivity, risk taking, stress responsivity and vulnerability todrug abuse and addiction. Nat Neurosci, 8 (1097-6256), 1450-7.
180
Krstulovic, A. M., Friedman, M. J., Colin, H., Guiochon, G., Gaspar, M., & Pajer,K. A. (1984). Analytical methodology for assays of serum tryptophan metabo-lites in control subjects and newly abstinent alcoholics: preliminary investiga-tion by liquid chromatography with amperometric detection. J Chromatogr,297, 271–281.
Kuhnen, C. M., & Chiao, J. Y. (2009). Genetic determinants of financial risk taking.PLoS ONE, 4 (2), e4362.
Kuhnen, C. M., & Knutson, B. (2005). The neural basis of financial risk taking.Neuron, 47 (5), 763-70.
Lamar, M., Cutter, W. J., Rubia, K., Brammer, M., Daly, E. M., Craig, M. C., etal. (2007). 5-ht, prefrontal function and aging: fmri of inhibition and acutetryptophan depletion. Neurobiology of Aging, In Press, Corrected Proof, –.
Lau, B., & Glimcher, P. W. (2005). Dynamic response-by-response models of match-ing behavior in rhesus monkeys. J Exp Anal Behav, 84 (3), 555–579.
Lau, B., & Glimcher, P. W. (2008, May). Value representations in the primatestriatum during matching behavior. Neuron, 58 (3), 451–463.
Laurens, K., Kiehl, K., & Liddle, P. (2005, Jan). A supramodal limbic-paralimbic-neocortical network supports goal-directed stimulus processing. Hum BrainMapp, 24 (1), 35-49.
Laurens, K., Kiehl, K., Ngan, E., & Liddle, P. (2005, Jun). Attention orientingdysfunction during salient novel stimulus processing in schizophrenia. SchizophrRes, 75 (2-3), 159-71.
Lee, D., Conroy, M. L., McGreevy, B. P., & Barraclough, D. J. (2004). Reinforcementlearning and decision making in monkeys during a competitive game. BrainRes Cogn Brain Res, 22 (1), 45–58.
Lee, D., McGreevy, B. P., & Barraclough, D. J. (2005). Learning and decisionmaking in monkeys during a rock-paper-scissors game. Brain Res Cogn BrainRes, 25 (2), 416–430.
Lee, D., Rushworth, M. F. S., Walton, M. E., Watanabe, M., & Sakagami, M.(2007, 8). Functional specialization of the primate frontal cortex during decisionmaking. Journal of Neuroscience, 27 (31), 8170–8173.
Le Masurier, M., Oldenzeil, W., Lehman, C., Cowen, P., & Sharp, T. (2006). Ef-fect of acute tyrosine depletion using a branched chain amino acid mixture ondopamine neurotransmission in the rat brain. Neuropsychopharmacology, 31,
181
310-317.
Lenze, E. J., Munin, M. C., Ferrell, R. E., Pollock, B. G., Skidmore, E., Lotrich, F.,et al. (2005). Association of the serotonin transporter gene-linked polymorphicregion (5-httlpr) genotype with depression in elderly persons after hip fracture.Am J Geriatr Psychiatry, 13 (5), 428-32.
Leon, M. I., & Shadlen, M. N. (1999). Effect of expected reward magnitude onthe response of neurons in the dorsolateral prefrontal cortex of the macaque.Neuron, 24 (2), 415–425.
Lesch, K.-P. (2007). Linking emotion to the social brain. the role of the serotonintransporter in human social behaviour. EMBO Rep, 8 Spec No, S24-9.
Lesch, K. P., & Gutknecht, L. (2005). Pharmacogenetics of the serotonin transporter.Prog Neuropsychopharmacol Biol Psychiatry, 29 (6), 1062–1073.
Lesch, K. P., Meyer, J., Glatz, K., Flugge, G., Hinney, A., Hebebrand, J., et al.(1997). The 5-ht transporter gene-linked polymorphic region (5-httlpr) in evo-lutionary perspective: alternative biallelic variation in rhesus monkeys. rapidcommunication. J Neural Transm, 104 (11-12), 1259–1266.
Long, A., & Platt, M. (2005, Nov). Decision making: the virtue of patience inprimates. Curr Biol, 15 (21), R874-6.
Lopes, L. L. (1995). Algebra and process in the modeling of risky choice. In J. R.Busemeyer, R. Hastie, & D. Medin (Eds.), Decision making from the perspectiveof cognitive psychology. Academic Press.
Lyne, J., Kelly, B. D., & O’Connor, W. T. (2004). Schizophrenia: a review ofneuropharmacology. Ir J Med Sci, 173 (0021-1265), 155-9.
Mackintosh, N. J. (1975, Jul). A theory of attention: Variations in the associabilityof stimuli with reinforcement. Psychological Review, 82 (4), 276-298.
Maddock, R. J. (1999). The retrosplenial cortex and emotion: new insights fromfunctional neuroimaging of the human brain. Trends Neurosci, 22 (7), 310–316.
Maddock, R. J., & Buonocore, M. H. (1997). Activation of left posterior cingulategyrus by the auditory presentation of threat-related words: an fmri study.Psychiatry Res, 75 (1), 1–14.
Maddock, R. J., Garrett, A. S., & Buonocore, M. H. (2001). Remembering familiarpeople: the posterior cingulate cortex and autobiographical memory retrieval.Neuroscience, 104 (3), 667–676.
182
Maddock, R. J., Garrett, A. S., & Buonocore, M. H. (2003). Posterior cingulatecortex activation by emotional words: fmri evidence from a valence decisiontask. Hum Brain Mapp, 18 (1), 30–41.
Malkesman, O., Pine, D. S., Tragon, T., Austin, D. R., Henter, I. D., Chen, G.,et al. (2009). Animal models of suicide-trait-related behaviors. Trends inPharmacological Sciences, 30 (4), 165–173.
Marek, G. J., Carpenter, L. L., McDougle, C. J., & Price, L. H. (2003). Synergisticaction of 5-HT2a antagonists and selective serotonin reuptake inhibitors inneuropsychiatric disorders. Neuropsychopharmacology, 28 (0893-133X), 402-12.
Markowitz, H. (1952). The utility of wealth. The Journal of Political Economy,60 (2), 151-158.
Matsen, F. A., & Nowak, M. A. (2004, Dec 28). Win-stay, lose-shift in languagelearning from peers. Proc Natl Acad Sci U S A, 101 (52), 18053–18057.
Matsumoto, M., & Hikosaka, O. (2008). Negative motivational control of saccadiceye movement by the lateral habenula. Prog Brain Res, 171, 399–402.
Matsumoto, M., Matsumoto, K., Abe, H., & Tanaka, K. (2007, May). Medial pre-frontal cell activity signaling prediction errors of action values. Nat Neurosci,10 (5), 647–656.
Mayberg, H. S., Brannan, S. K., Mahurin, R. K., Jerabek, P. A., Brickman, J. S.,Tekell, J. L., et al. (1997, Mar). Cingulate function in depression: a potentialpredictor of treatment response. Neuroreport, 8 (4), 1057–1061.
McClure, S. M., Li, J., Tomlin, D., Cypert, K. S., Montague, L. M., & Montague,P. R. (2004). Neural correlates of behavioral preference for culturally familiardrinks. Neuron, 44 (2), 379–387.
McCoy, A., & Platt, M. (2005a, Mar). Expectations and outcomes: decision-makingin the primate brain. J Comp Physiol A Neuroethol Sens Neural Behav Physiol,191 (3), 201-11.
McCoy, A., & Platt, M. (2005b, Sep). Risk-sensitive neurons in macaque posteriorcingulate cortex. Nat Neurosci, 8 (9), 1220-7.
McCoy, A. N., Crowley, J. C., Haghighian, G., Dean, H. L., & Platt, M. L. (2003).Saccade reward signals in posterior cingulate cortex. Neuron, 40 (0896-6273),1031-40.
McGlothlin, W. H. (1956). Stability of choices among uncertain alternatives. The
183
American Journal of Psychology, 69 (4), 604–615.
Mehlman, P. T., Higley, J. D., Fernald, B. J., Sallee, F. R., Suomi, S. J., & Linnoila,M. (1997, Sep). Csf 5-hiaa, testosterone, and sociosexual behaviors in free-ranging male rhesus macaques in the mating season. Psychiatry Res, 72 (2),89–102.
Milak, M. S., Parsey, R. V., Keilp, J., Oquendo, M. A., Malone, K. M., & Mann,J. J. (2005, Apr). Neuroanatomic correlates of psychopathologic componentsof major depressive disorder. Arch Gen Psychiatry, 62 (4), 397–408.
Milinski, M. (1987, Jan). Tit for tat in sticklebacks and the evolution of cooperation.Nature, 325 (6103), 433–435.
Mill, J. S. (2000). Essays on some unsettled questions of political economy. BatocheBooks. (Original work published 1844)
Miller, E. K., & Cohen, J. D. (2001). An integrative theory of prefrontal cortexfunction. Annu Rev Neurosci, 24, 167–202.
Minoshima, S., Giordani, B., Berent, S., Frey, K. A., Foster, N. L., & Kuhl, D. E.(1997). Reduction in the posterior cingulate cortex in very early Alzheimer’sdisease. Ann Neurol, 42 (0364-5134), 85-94.
Mobini, S., Chiang, T. J., Ho, M. Y., Bradshaw, C. M., & Szabadi, E. (2000).Effects of central 5-hydroxytryptamine depletion on sensitivity to delayed andprobabilistic reinforcement. Psychopharmacology (Berl), 152 (4), 390-7.
Mohler, C. W., Goldberg, M. E., & Wurtz, R. H. (1973, Oct). Visual receptive fieldsof frontal eye field neurons. Brain Res, 61, 385–389.
Moja, E., Cipolla, P., Castoldi, D., & Tofanetti, O. (1989). Dose-response decreasein plasma tryptophan and in brain tryptophan and serotonin after tryptophan-free amino acid mixtures in rats. Life Sci, 44 (14), 971-6.
Moja, E., Stoff, D., Gessa, G., Castoldi, D., Assereto, R., & Tofanetti, O. (1988).Decrease in plasma tryptophan after tryptophan-free amino acid mixtures inman. Life Sci, 42 (16), 1551-6.
Montague, P. R. (2007, Oct-Dec). Neuroeconomics: a view from neuroscience. FunctNeurol, 22 (4), 219–234.
Moore, P., Landolt, H., Seifritz, E., Clark, C., Bhatti, T., Kelsoe, J., et al. (2000,Dec). Clinical and physiological consequences of rapid tryptophan depletion.Neuropsychopharmacology, 23 (6), 601-22.
184
Morecraft, R. J., Geula, C., & Mesulam, M. M. (1992 Sep 15). Cytoarchitectureand neural afferents of orbitofrontal cortex in the brain of the monkey. J CompNeurol, 323 (3), 341–358.
Morecraft, R. J., Geula, C., & Mesulam, M. M. (1993). Architecture of connectivitywithin a cingulo-fronto-parietal neurocognitive network for directed attention.Arch Neurol, 50 (0003-9942), 279-84.
Morilak, D. A., Garlow, S. J., & Ciaranello, R. D. (1993). Immunocytochemicallocalization and description of neurons expressing serotonin2 receptors in therat brain. Neuroscience, 54 (3), 701–717.
Morrison, I., Peelen, M., & Downing, P. (2006). The sight of others’ pain modulatesmotor processing in human cingulate cortex. Cereb Cortex (1047-3211 (Print)).
Mossner, R., Muller-Vahl, K. R., Doring, N., & Stuhrmann, M. (2007, Jul). Role ofthe novel tryptophan hydroxylase-2 gene in tourette syndrome. Mol Psychiatry,12 (7), 617–619.
Muller, H. J., & Rabbitt, P. M. (1989, May). Reflexive and voluntary orienting ofvisual attention: time course of activation and resistance to interruption. JExp Psychol Hum Percept Perform, 15 (2), 315–330.
Murphy, D. L., Lerner, A., Rudnick, G., & Lesch, K.-P. (2004). Serotonin trans-porter: gene, genetic disorders, and pharmacogenetics. Mol Interv, 4 (2), 109–123.
Murthy, A., Thompson, K. G., & Schall, J. D. (2001, Nov). Dynamic dissociation ofvisual selection from saccade programming in frontal eye field. J Neurophysiol,86 (5), 2634–2637.
Nakamura, K., Matsumoto, M., & Hikosaka, O. (2008, May 14). Reward-dependentmodulation of neuronal activity in the primate dorsal raphe nucleus. J Neu-rosci, 28 (20), 5331–5343.
Neumann, J. von. (1959). [On the theory of games of strategy]. In A. W. Tucker &R. D. Luce (Eds.), Contributions to the theory of games (Vol. IV). (Reprintedfrom Mathematische Annalen, 1928, 100, 295-320)
Neumann, J. von, & Morgenstern, O. (1944). Theory of games and economic behavior(Sixtieth-Anniversary Edition, 2004 ed.). Princeton, NJ: Princeton UniversityPress.
Neumeister, A., Nugent, A. C., Waldeck, T., Geraci, M., Schwarz, M., Bonne, O.,et al. (2004, Aug). Neural and behavioral responses to tryptophan depletion
185
in unmedicated patients with remitted major depressive disorder and controls.Arch Gen Psychiatry, 61 (8), 765–773.
Newsome, W. T., Britten, K. H., & Movshon, J. A. (1989, Sep). Neuronal correlatesof a perceptual decision. Nature, 341 (6237), 52–54.
Nielsen, D. A., Goldman, D., Virkkunen, M., Tokola, R., Rawlings, R., & Linnoila,M. (1994). Suicidality and 5-hydroxyindoleacetic acid concentration associatedwith a tryptophan hydroxylase polymorphism. Arch Gen Psychiatry, 51 (1),34–38.
Nielsen, D. A., Virkkunen, M., Lappalainen, J., Eggert, M., Brown, G. L., Long,J. C., et al. (1998). A tryptophan hydroxylase gene marker for suicidality andalcoholism. Arch Gen Psychiatry, 55 (7), 593–602.
Nieuwenhuis, S., Heslenfeld, D. J., Geusau, N. J. A. von, Mars, R. B., Holroyd, C. B.,& Yeung, N. (2005, May). Activity in human reward-sensitive brain areas isstrongly context dependent. Neuroimage, 25 (4), 1302–1309.
Nomura, M., & Nomura, Y. (2006). Psychological, neuroimaging, and biochemicalstudies on functional association between impulsive behavior and the 5-ht2areceptor gene polymorphism in humans. Ann N Y Acad Sci, 1086 (0077-8923(Print)), 134-43.
Nordin, C., & Sjodin, I. (2006). Csf monoamine patterns in pathological gamblersand healthy controls. J Psychiatr Res, 40 (5), 454-9.
Nordstrom, P., Samuelsson, M., Asberg, M., Traskman-Bendz, L., Aberg-Wistedt,A., Nordin, C., et al. (1994). Csf 5-hiaa predicts suicide risk after attemptedsuicide. Suicide Life Threat Behav, 24 (1), 1-9.
Nowak, M., & Sigmund, K. (1993, Jul 1). A strategy of win-stay, lose-shift thatoutperforms tit-for-tat in the prisoner’s dilemma game. Nature, 364 (6432),56–58.
O’Doherty, J., Critchley, H., Deichmann, R., & Dolan, R. J. (2003). Dissociat-ing valence of outcome from behavioral control in human orbital and ventralprefrontal cortices. J Neurosci, 23 (21), 7931–7939.
O’Doherty, J. P., Deichmann, R., Critchley, H. D., & Dolan, R. J. (2002). Neuralresponses during anticipation of a primary taste reward. Neuron, 33 (5), 815–826.
Olson, C., & Musil, S. (1992). Posterior cingulate cortex: sensory and oculomotorproperties of single neurons in behaving cat. Cereb Cortex, 2 (6), 485-502.
186
Olson, C., Musil, S., & Goldberg, M. (1996, Nov). Single neurons in posteriorcingulate cortex of behaving macaque: eye movement signals. J Neurophysiol,76 (5), 3285-300.
Olson, C. R., Musil, S. Y., & Goldberg, M. E. (1993). Posterior cingulate cortex andvisuospatial cognition: properties of single neurons in the behaving monkey. InB. A. Vogt & M. Gabriel (Eds.), Neurobiology of cingulate cortex and limbicthalamus. Birkhauser.
Ousdal, O. T., Jensen, J., Server, A., Hariri, A. R., Nakstad, P. H., & Andreassen,O. A. (2008, 10). The human amygdala is involved in general behavioral rele-vance detection: Evidence from an event-related functional magnetic resonanceimaging go-nogo task. Neuroscience, 156 (3), 450–455.
Padoa-Schioppa, C., & Assad, J. A. (2006, May). Neurons in the orbitofrontal cortexencode economic value. Nature, 441 (7090), 223–226.
Padoa-Schioppa, C., & Assad, J. A. (2008). The representation of economic value inthe orbitofrontal cortex is invariant for changes of menu. Nat Neurosci, 11 (1),95–102.
Pandya, D. N., Van Hoesen, G. W., & Mesulam, M. M. (1981). Efferent connectionsof the cingulate gyrus in the rhesus monkey. Exp Brain Res, 42 (0014-4819),319-30.
Pardo, C. A., & Eberhart, C. G. (2007, Oct). The neurobiology of autism. BrainPathol, 17 (4), 434–447.
Parvizi, J., Van Hoesen, G. W., Buckwalter, J., & Damasio, A. (2006). Neuralconnections of the posteromedial cortex in the macaque. Proc Natl Acad SciU S A, 103 (5), 1563–1568.
Payne, J. W., Bettman, J. R., & Johnson, E. J. (1988, 07). Adaptive strategyselection in decision making. Journal of Experimental Psychology: Learning,Memory, and Cognition, 14 (3), 534–552.
Pearce, J. M., & Hall, G. (1980). A model for pavlovian learning: variations inthe effectiveness of conditioned but not of unconditioned stimuli. Psychol Rev,87 (6), 532–552.
Pessoa, L., Kastner, S., & Ungerleider, L. G. (2002, Dec). Attentional control of theprocessing of neural and emotional stimuli. Brain Res Cogn Brain Res, 15 (1),31–45.
Pessoa, L., & Ungerleider, L. G. (2004). Neuroimaging studies of attention and the
187
processing of emotion-laden stimuli. Prog Brain Res, 144 (0079-6123), 171-82.
Pickens, C. L., Saddoris, M. P., Setlow, B., Gallagher, M., Holland, P. C., & Schoen-baum, G. (2003, Dec). Different roles for orbitofrontal cortex and basolateralamygdala in a reinforcer devaluation task. J Neurosci, 23 (35), 11078–11084.
Pierce, K., Haist, F., Sedaghat, F., & Courchesne, E. (2004). The brain response topersonally familiar faces in autism: findings of fusiform activity and beyond.Brain, 127 (1460-2156 (Electronic)), 2703-16.
Platt, M., & Glimcher, P. (1999, Jul). Neural correlates of decision variables inparietal cortex. Nature, 400 (6741), 233-8.
Platt, M. L., & Huettel, S. A. (2008). Risky business: the neuroeconomics of decisionmaking under uncertainty. Nat Neurosci, 11 (4), 398–403.
Posner, M. I., Snyder, C. R., & Davidson, B. J. (1980, Jun). Attention and thedetection of signals. J Exp Psychol, 109 (2), 160–174.
Rachlin, H. (2000). The science of self-control [Book].
Ramadan, N. M., Skljarevski, V., Phebus, L. A., & Johnson, K. W. (2003, 10). 5-ht1f receptor agonists in acute migraine treatment: a hypothesis.. Cephalalgia,23 (8).
Rangel, A., Camerer, C., & Montague, P. R. (2008, Jul). A framework for studyingthe neurobiology of value-based decision making. Nat Rev Neurosci, 9 (7), 545–556.
Ratiu, P., Talos, I.-F., Haker, S., Lieberman, D., & Everett, P. (2004, May). Thetale of phineas gage, digitally remastered. J Neurotrauma, 21 (5), 637–643.
Reilly, J. G., McTavish, S. F., & Young, A. H. (1997). Rapid depletion of plasmatryptophan: a review of studies and experimental methodology. J Psychophar-macol, 11 (0269-8811), 381-92.
Ren-Patterson, R. F., Cochran, L. W., Holmes, A., Lesch, K.-P., Lu, B., & Murphy,D. L. (2006). Gender-dependent modulation of brain monoamines and anxiety-like behaviors in mice with genetic serotonin transporter and bdnf deficiencies.Cell Mol Neurobiol, 26 (4-6), 755–780.
Rescorla, R. A., & Wagner, A. R. (1972). A theory of pavlovian conditioning:Variations in the effectiveness of reinforcement and nonreinforcement. In A. H.Black & W. F. Prokasy (Eds.), Classical conditioning ii. Appleton-Century-Crofts.
188
Riedel, W. J., Sobczak, S., & Schmitt, J. A. J. (2003). Tryptophan modulation andcognition. Adv Exp Med Biol, 527 (0065-2598 (Print)), 207-13.
Robertson, D., Snarey, J., Ousley, O., Harenski, K., Bowman, F. D., Gilkey, R., etal. (2007). The neural processing of moral sensitivity to issues of justice andcare. Neuropsychologia, 45 (4), 755-66.
Roesch, M. R., & Olson, C. R. (2004, Apr). Neuronal activity related to rewardvalue and motivation in primate frontal cortex. Science, 304 (5668), 307–310.
Rogers, R., Tunbridge, E., Bhagwagar, Z., Drevets, W., Sahakian, B., & Carter, C.(2003, Jan). Tryptophan depletion alters the decision-making of healthy vol-unteers through altered processing of reward cues. Neuropsychopharmacology,28 (1), 153-62.
Rogers, R. D., Blackshaw, A. J., Middleton, H. C., Matthews, K., Hawtin, K., Crow-ley, C., et al. (1999). Tryptophan depletion impairs stimulus-reward learningwhile methylphenidate disrupts attentional control in healthy young adults:implications for the monoaminergic basis of impulsive behaviour. Psychophar-macology (Berl), 146 (4), 482-91.
Rolls, E. T. (2000). The orbitofrontal cortex and reward. Cereb Cortex, 10 (1047-3211), 284-94.
Romo, R., Hernandez, A., Zainos, A., Brody, C. D., & Lemus, L. (2000, Apr).Sensing without touching: psychophysical performance based on cortical mi-crostimulation. Neuron, 26 (1), 273–278.
Romo, R., Hernandez, A., Zainos, A., & Salinas, E. (1998, Mar). Somatosensorydiscrimination based on cortical microstimulation. Nature, 392 (6674), 387–390.
Rosenbaum, D. A. (1980, Dec). Human movement initiation: specification of arm,direction, and extent. J Exp Psychol Gen, 109 (4), 444–474.
Roth, B. L., Hanizavareh, S. M., & Blum, A. E. (2004). Serotonin receptors representhighly favorable molecular targets for cognitive enhancement in schizophreniaand other disorders. (No. 0033-3158).
Rottenstreich, Y., & Hsee, C. K. (2001). Money, kisses, and electric shocks: on theaffective psychology of risk. Psychol Sci, 12 (0956-7976), 185-90.
Rudebeck, P. H., Behrens, T. E., Kennerley, S. W., Baxter, M. G., Buckley, M. J.,Walton, M. E., et al. (2008, Dec). Frontal cortex subregions play distinct rolesin choices between actions and stimuli. J Neurosci, 28 (51), 13775–13785.
189
Rushworth, M. F. S., Buckley, M. J., Behrens, T. E. J., Walton, M. E., & Bannerman,D. M. (2007). Functional organization of the medial frontal cortex. Curr OpinNeurobiol, 17 (2), 220–227.
Salzman, C. D., Murasugi, C. M., Britten, K. H., & Newsome, W. T. (1992, Jun).Microstimulation in visual area mt: effects on direction discrimination perfor-mance. J Neurosci, 12 (6), 2331–2355.
Sapolsky, R. M. (2005). Monkeyluv: and other essays on our lives as animals. NewYork: Scribner.
Sato, T., Murthy, A., Thompson, K. G., & Schall, J. D. (2001, May). Searchefficiency but not response interference affects visual selection in frontal eyefield. Neuron, 30 (2), 583–591.
Schall, J. D., Hanes, D. P., Thompson, K. G., & King, D. J. (1995, Oct). Saccadetarget selection in frontal eye field of macaque. i. visual and premovementactivation. J Neurosci, 15 (10), 6905–6918.
Schall, J. D., & Thompson, K. G. (1999). Neural selection and control of visuallyguided eye movements. Annu Rev Neurosci, 22, 241–259.
Schoenbaum, G., Chiba, A. A., & Gallagher, M. (2000). Changes in functionalconnectivity in orbitofrontal cortex and basolateral amygdala during learningand reversal training. J Neurosci, 20 (13), 5179–5189.
Scholes, K. E., Harrison, B. J., O’Neill, B. V., Leung, S., Croft, R. J., Pipingas, A.,et al. (2007). Acute serotonin and dopamine depletion improves attentionalcontrol: findings from the stroop task. Neuropsychopharmacology, 32 (7), 1600–1610.
Schultz, W. (2001). Reward signaling by dopamine neurons. Neuroscientist, 7 (1073-8584), 293-302.
Schultz, W., Tremblay, L., & Hollerman, J. R. (1998). Reward prediction in primatebasal ganglia and frontal cortex. Neuropharmacology, 37 (4-5), 421–429.
Schweighofer, N., Bertin, M., Shishida, K., Okamoto, Y., Tanaka, S. C., Yamawaki,S., et al. (2008). Low-serotonin levels increase delayed reward discounting inhumans. J Neurosci, 28 (17), 4528–4532.
Selten, R. (1998). Aspiration adaptation theory. Journal of Mathematical Psychology,42 (2-3), 191–214.
Shah, N. J., Marshall, J. C., Zafiris, O., Schwab, A., Zilles, K., Markowitsch, H. J., et
190
al. (2001, 4). The neural correlates of person familiarity: A functional magneticresonance imaging study with clinical implications. Brain, 124 (4), 804–815.
Shannon, C. E. (1948). A mathematical theory of communication. Bell SystemTechnical Journal, 27, 379-423, 623-656.
Sharpe, W. F. (1964, Sep). Capital asset prices: A theory of market equilibriumunder conditions of risk. The Journal of Finance, 19 (3), 425-442.
Simon, H. A. (1955). A behavioral model of rational choice. Quarterly Journal ofEconomics, 69, 7–19.
Simpson, J. A., & Weiner, S. C. (1989). The Oxford English Dictionary. London,England: Oxford University Press.
Small, D. M., Gitelman, D. R., Gregory, M. D., Nobre, A. C., Parrish, T. B., &Mesulam, M.-M. (2003). The posterior cingulate and medial prefrontal cortexmediate the anticipatory allocation of spatial attention. Neuroimage, 18 (1053-8119), 633-41.
Small, D. M., Zatorre, R. J., Dagher, A., Evans, A. C., & Jones-Gotman, M. (2001).Changes in brain activity related to eating chocolate: from pleasure to aversion.Brain, 124 (0006-8950), 1720-33.
Smith, A. (1976). An inquiry into the nature and causes of the wealth of nations.In R. H. Campbell & A. Skinner (Eds.), The glasgow edition of the worksand correspondence of adam smith. Oxford University Press. (Original workpublished 1776)
Sodhi, M. S. K., & Sanders-Bush, E. (2004). Serotonin and brain development. IntRev Neurobiol, 59 (0074-7742), 111-74.
Soltani, A., Lee, D., & Wang, X.-J. (2006, Oct). Neural mechanism for stochasticbehaviour during a competitive game. Neural Netw, 19 (8), 1075–1090.
Sorg, C., Riedl, V., Muhlau, M., Calhoun, V. D., Eichele, T., Laer, L., et al. (2007).Selective changes of resting-state networks in individuals at risk for alzheimer’sdisease. Proc Natl Acad Sci U S A, 104 (47), 18760–18765.
Stahl, S. M. (2008). Stahl’s essential psychopharmacology: neuroscientific basisand practical applications (3rd ed, Fully rev. and expanded ed.). Cambridge:Cambridge University Press.
Sugrue, L. P., Corrado, G. S., & Newsome, W. T. (2004). Matching behavior and therepresentation of value in the parietal cortex. Science, 304 (5678), 1782–1787.
191
Sutton, R. S. (1988, 08 01). Learning to predict by the methods of temporal differ-ences. Machine Learning, 3 (1), 9–44.
Sutton, R. S., & Barto, A. G. (1981, Mar). Toward a modern theory of adaptivenetworks: expectation and prediction. Psychol Rev, 88 (2), 135–170.
Svenningsson, P., Chergui, K., Rachleff, I., Flajolet, M., Zhang, X., El Yacoubi, M.,et al. (2006). Alterations in 5-HT1b receptor function by p11 in depression-likestates. Science, 311 (1095-9203 (Electronic)), 77-80.
Tagliamonte, A., Biggio, G., Vargiu, L., & Gessa, G. L. (1973). Free tryptophanin serum controls brain tryptophan level and serotonin synthesis. Life Sci II,12 (6), 277-87.
Thompson, C. L., Pathak, S. D., Jeromin, A., Ng, L. L., MacPherson, C. R., Mortrud,M. T., et al. (2008, Dec). Genomic anatomy of the hippocampus. Neuron,60 (6), 1010–1021.
Thompson, K. G., & Bichot, N. P. (2005). A visual salience map in the primatefrontal eye field. Prog Brain Res, 147, 251–262.
Thompson, K. G., Bichot, N. P., & Sato, T. R. (2005, Jan). Frontal eye field activitybefore visual search errors reveals the integration of bottom-up and top-downsalience. J Neurophysiol, 93 (1), 337–351.
Thompson, K. G., Bichot, N. P., & Schall, J. D. (1997, Feb). Dissociation ofvisual discrimination from saccade programming in macaque frontal eye field.J Neurophysiol, 77 (2), 1046–1050.
Thorndike, E. L. (1911). Animal intelligence. Online Resource (Christopher D.Green).
Tobler, P., Fiorillo, C., & Schultz, W. (2005, Mar). Adaptive coding of reward valueby dopamine neurons. Science, 307 (5715), 1642-5.
Tobler, P. N., O’doherty, J. P., Dolan, R. J., & Schultz, W. (2006). Human neurallearning depends on reward prediction errors in the blocking paradigm. JNeurophysiol, 95 (1), 301–310.
Treisman, A. M., & Gelade, G. (1980, Jan). A feature-integration theory of attention.Cogn Psychol, 12 (1), 97–136.
Tremblay, L., & Schultz, W. (1999). Relative reward preference in primate or-bitofrontal cortex. Nature, 398 (6729), 704–708.
192
Tversky, A., & Kahneman, D. (1974). Judgement under uncertainty: Heuristics andbiases. Science, 185, 1124–1131.
Tversky, A., & Kahneman, D. (1992). Advances in prospect theory: Cumulativerepresentation of uncertainty. Journal of Risk and Uncertainty, 5, 297–323.
Valla, J., Berndt, J., & Gonzalez-Lima, F. (2001, Jul). Energy hypometajellyfishma-masm in posterior cingulate cortex of Alzheimer’s patients: superficial laminarcytochrome oxidase associated with disease duration. J Neurosci, 21 (13), 4923-30.
Valla, J., Chen, K., Berndt, J. D., Gonzalez-Lima, F., Cherry, S. R., Games, D.,et al. (2002). Effects of image resolution on autoradiographic measurementsof posterior cingulate activity in pdapp mice: implications for functional brainimaging studies of transgenic mouse models of alzheimer’s disease. Neuroimage,16 (1), 1-6.
Varnas, K., Halldin, C., & Hall, H. (2004). Autoradiographic distribution of sero-tonin transporters and receptor subtypes in human brain. Hum Brain Mapp,22 (1065-9471), 246-60.
Veenstra-VanderWeele, J., Anderson, G. M., & Cook, E. H. (2000, 12). Pharmacoge-netics and the serotonin system: initial studies and future directions. EuropeanJournal of Pharmacology, 410 (2-3), 165–181.
Vogt, B. A., Finch, D. M., & Olson, C. R. (1992). Functional heterogeneity incingulate cortex: the anterior executive and posterior evaluative regions. CerebCortex, 2 (6), 435–443.
Vogt, B. A., & Pandya, D. N. (1987, Aug 8). Cingulate cortex of the rhesus monkey:Ii. cortical afferents. J Comp Neurol, 262 (2), 271–289.
Wallis, J. D. (2006, May). Evaluating apples and oranges. Nat Neurosci, 9 (5),596–598.
Wallis, J. D., Anderson, K. C., & Miller, E. K. (2001). Single neurons in prefrontalcortex encode abstract rules. Nature, 411 (6840), 953–956.
Wallis, J. D., & Miller, E. K. (2003a). From rule to response: neuronal processes inthe premotor and prefrontal cortex. J Neurophysiol, 90 (3), 1790–1806.
Wallis, J. D., & Miller, E. K. (2003b). Neuronal activity in primate dorsolateral andorbital prefrontal cortex during performance of a reward preference task. EurJ Neurosci, 18 (7), 2069–2081.
193
Walton, M. E., Croxson, P. L., Behrens, T. E. J., Kennerley, S. W., & Rushworth,M. F. S. (2007). Adaptive decision making and value in the anterior cingulatecortex. Neuroimage, 36 Suppl 2, T142-54.
Watabe-Uchida, M., & Uchida, N. (2008). Sensory- and reward-based decision mak-ing paradigm in mice. Society for Neuroscience.
Watanabe, K., Lauwereyns, J., & Hikosaka, O. (2003 Oct). Effects of motivationalconflicts on visually elicited saccades in monkeys. Exp Brain Res, 152 (3),361–367.
Watanabe, M. (1996, Aug). Reward expectancy in primate prefrontal neurons.Nature, 382 (6592), 629–632.
Watson, R. T., Heilman, K. M., Cauthen, J. C., & King, F. A. (1973). Neglect aftercingulectomy. Neurology, 23 (0028-3878), 1003-7.
Weber, E., Shafir, S., & Blais, A. (2004, Apr). Predicting risk sensitivity in humansand lower animals: risk as variance or coefficient of variation. Psychol Rev,111 (2), 430-45.
Weber, E. U., & Hsee, C. (1998). Cross-cultural differences in risk perception,but cross-cultural similarities in attitudes towards perceived risk. ManagementScience, 44 (9), 1205–1217.
Wedekind, C., & Milinski, M. (1996, Apr 2). Human cooperation in the simultaneousand the alternating prisoner’s dilemma: Pavlov versus generous tit-for-tat. ProcNatl Acad Sci U S A, 93 (7), 2686–2689.
Weisstaub, N. V., Zhou, M., Lira, A., Lambe, E., Gonzalez-Maeso, J., Hornung,J.-P., et al. (2006). Cortical 5-ht2a receptor signaling modulates anxiety-likebehaviors in mice. Science, 313 (5786), 536-40.
Whitaker-Azmitia, P. M. (2001). Serotonin and brain development: role in humandevelopmental diseases. Brain Res Bull, 56 (0361-9230), 479-85.
Winstanley, C. A., Dalley, J. W., Theobald, D. E. H., & Robbins, T. W. (2003).Global 5-HT depletion attenuates the ability of amphetamine to decrease im-pulsive choice on a delay-discounting task in rats. Psychopharmacology (Berl),170 (0033-3158), 320-31.
Winstanley, C. A., Dalley, J. W., Theobald, D. E. H., & Robbins, T. W. (2004). Frac-tionating impulsivity: contrasting effects of central 5-HT depletion on differentmeasures of impulsive behavior. Neuropsychopharmacology, 29 (0893-133X),1331-43.
194
Winstanley, C. A., Theobald, D. E. H., Dalley, J. W., Glennon, J. C., & Robbins,T. W. (2004). 5-HT2a and 5-HT2c receptor antagonists have opposing ef-fects on a measure of impulsivity: interactions with global 5-HT depletion.Psychopharmacology (Berl), 176 (0033-3158), 376-85.
Wise, R. A. (2002). Brain reward circuitry: insights from unsensed incentives.Neuron, 36 (0896-6273), 229-40.
Wolff, M. C., & Leander, J. D. (2002). Selective serotonin reuptake inhibitorsdecrease impulsive behavior as measured by an adjusting delay procedure inthe pigeon. Neuropsychopharmacology, 27 (0893-133X), 421-9.
Wolford, G., Miller, M. B., & Gazzaniga, M. (2000, Mar). The left hemisphere’s rolein hypothesis formation. J Neurosci, 20 (6), RC64.
Wong, D. F., Brasic, J. R., Singer, H. S., Schretlen, D. J., Kuwabara, H., Zhou, Y.,et al. (2008, May). Mechanisms of dopaminergic and serotonergic neurotrans-mission in tourette syndrome: clues from an in vivo neurochemistry study withpet. Neuropsychopharmacology, 33 (6), 1239–1251.
Young, S. N., Ervin, F. R., Pihl, R. O., & Finn, P. (1989). Biochemical aspects oftryptophan depletion in primates. Psychopharmacology (Berl), 98 (0033-3158),508-11.
Young, S. N., & Leyton, M. (2002). The role of serotonin in human mood and socialinteraction. insight from altered tryptophan levels. (No. 0091-3057).
Zhang, Y., Lu, H., & Bargmann, C. I. (2005). Pathogenic bacteria induce aversiveolfactory learning in caenorhabditis elegans. Nature, 438 (1476-4687 (Elec-tronic)), 179-84.
195
Biography
Arwen Long was born in Evanston, IL on May 6, 1981. She wrote this document
faster than you might think, slept less than you might imagine, and defeated a few
pink robots along the way. Someday Arwen will become a retired hippie and move to
Vermont, where she will run an organic food-n-flower farm when she is not joy-riding
past the fields of fragrant bovine beasts.
Degrees earned
M. Phil., Medical Sciences (2002)
Churchill College, University of Cambridge, Cambridge, England
Thesis title: Mechanical and Biochemical Aspects of Traumatic Brain Injury
Advisor: Mark O’Connell, Department of Neurosurgery
M.S. Biochemistry/B.S., Biochemistry, Biology, Music (2001; summa cum laude)
Brandeis University, Waltham, MA
Thesis title: RadA, a Protein Implicated in Escherichia coli DNA Repair
Advisor: Lizbeth Hedstrom, Biochemistry
Honors and Awards
2009 Gertrude B. Elion Mentored Medical Student Research Award
2006 National Institutes of Health Kirchstein pre-doctoral fellowship
2001 Winston Churchill Scholarship
2000 Barry M. Goldwater Award
196
1999 Justice Brandeis Scholarship (through 2001)
1999 Phi Beta Kappa
1997 National Merit Scholarship
197
