Upload
vukhue
View
220
Download
1
Embed Size (px)
Citation preview
Heather M. Wied (Previous Name: Heather M. Lueck)
E-mail: [email protected]
Education
M.D./Ph.D. Program, August 2008 – present
Ph.D., August 2014
M.D., expected May 2016
University of Maryland School of Medicine Medical Scientist Training Program
Post-Baccalaureate Premedical Program, July 2005 – June 2006
The Johns Hopkins University
B.A., June 2003
University of California, Los Angeles
Major in Psychology with a Minor in Neuroscience (summa cum laude)
Honors & Awards
Distinctions
F30 NIH NRSA Ruth L. Kirschstein Predoctoral MD/PhD Fellowship, March 2012 -
present
T32 NIH Predoctoral Training Grant, Program in Neuroscience, September 2010 -
August 2011
The Candidates’ Speaker (student chosen to address graduates at commencement),
UCLA College of Letters and Science Commencement, June 2003
Charles E. and Sue K. Young Undergraduate Student Award for 2002-2003, UCLA
Distinguished Bruin Award, UCLA Alumni Association, 2003
Departmental Honors, UCLA Psychology Department, 2002
Associations
American Association for the Advancement of Science (AAAS), April 2013 - present
Society for Neuroscience, October 2008 – present
Combined Accelerated Program in Psychiatry, University of Maryland School of
Medicine, October 2008 – June 2010
Alumni Scholar, UCLA, November 2002 – present
Phi Beta Kappa, June 2003 – present
Neuroscience Undergraduate Society, UCLA, September 2002 – June 2003
Golden Key International Honours Society, May 2002 – present
Psi Chi, National Honors Society in Psychology, September 2001 – present
Research, Employment and Teaching Experience
Predoctoral Research, University of Maryland School of Medicine, Program in
Neuroscience, 2010-2014. Supervised by Geoffrey Schoenbaum, M.D., Ph.D.
Faculty/Student Research, University of Maryland School of Medicine, Program in
Neuroscience, Summer 2009. Supervised by Elizabeth M. Powell, Ph.D.
Clinical and Computational Auditory Neuroscience Internship, Johns Hopkins School of
Medicine, Department of Neurology, Summer 2009. Supervised by Dana Boatman-
Reich, Ph.D., CCC-A. Funded by NIDCD.
Senior Research Assistant, Johns Hopkins School of Medicine, Department of
Neurology, 2006-2008. Supervised by Dana Boatman-Reich, Ph.D., CCC-A
Clinical Research Tutorial, Johns Hopkins School of Medicine, Department of
Neurology, 2006. Supervised by Dana Boatman-Reich, Ph.D., CCC-A
Teaching Assistant, Fieldwork in Behavioral Modification, UCLA Department of
Psychology, Fall 2004. Supervised by O. Ivar Lovaas, Ph.D.
Faculty/Student Research, UCLA, Department of Psychology, 2001-2004. Supervised by
O. Ivar Lovaas, Ph.D.
Faculty/Student Research, UCLA, Department of Psychology, 2002-2003. Supervised by
Michael S. Fanselow, Ph.D.
Teaching Assistant, Behavioral Modification, UCLA Department of Psychology, Spring
2002. Supervised by O. Ivar Lovaas, Ph.D.
Senior Discrete Trial Instructor, Lovaas Institute for Early Intervention, 2004.
One-to-One Behavioral Instructor, Lovaas Institute for Early Intervention, 2002-2004.
Research Publications
Published Manuscripts
Wied HM*, Sadacca BF*, Johnson W, Saini S, Jones J, Berg BA, Schoenbaum G.
Orbitofrontal cortex signals inferred value. (in preparation). (* shared first
authorship)
Cooch NK, Stalnaker TA, Wied H, Chaudhary S, McDannald MA, Liu TL, Schoenbaum
G. Orbitofrontal lesions eliminate value-selective activity in cue-responsive ventral
striatal neurons. (Submitted).
McDannald MA, Wegener M, Wied HM, Liu TL, Stalnaker TA, Jones JL, Esber GR,
Trageser J, Schoenbaum G. Orbitofrontal neurons acquire responses to ‘valueless’
Pavlovian cues during unblocking. (Submitted).
Stalnaker TA, Cooch NK, McDannald MA, Liu TL, Wied H, Schoenbaum G. (2014)
Orbitofrontal neurons infer the value and identity of predicted outcomes. Nature
Communications, 5:3926 doi: 10.1038/ncomms4926.
Wied HM, Jones JL, Cooch NK, Berg BA, Schoenbaum G. (2013). Disruption of model-
based behavior and learning by cocaine self-administration in rats.
Psychopharmacology, 229 (3): 493-501.
Quinn JJ, Wied HM, Liu D, Fanselow MS. (2009). Post-training excitotoxic lesions of
the dorsal hippocampus attenuate generalization in auditory delay fear conditioning.
European Journal of Neuroscience, 29(8): 1692–1700.
Sinai A, Crone NE, Wied HM, Franaszczuk PJ, Miglioretti D, Boatman-Reich D. (2009).
Intracranial mapping of auditory perception: event-related recordings and
electrocortical stimulation. Clinical Neurophysiology, 120(1): 140-149.
Quinn JJ, Wied HM, Ma QD, Tinsley MR, Fanselow MS. (2008). Dorsal hippocampus
involvement in delay fear conditioning depends upon the strength of the tone-
footshock association. Hippocampus, 18(7): 640-654.
Boatman DF, Trescher WH, Smith C, Ewen J, Los J, Wied HM, Gordon B, Kossoff EH,
Gao Q, Vining EP. (2008). Cortical auditory dysfunction in benign rolandic epilepsy.
Epilepsia, 49(6): 1018-1026.
Wied HM, Morrison PF, Gordon B, Zimmerman AW, Vining EP, Boatman DF. (2007).
Cortical auditory dysfunction in childhood epilepsy: electrophysiologic evidence.
Current Pediatric Reviews, 3(4): 317-327.
Published Abstracts and Presentations
(Please note - previous name Heather M. Lueck)
Wied HM, Cooch N, Jones JL, Berg BA, Schoenbaum G. (2013). Cocaine use disrupts
behavior and learning that depends on inferred value. The Catecholamines Gordon
Research Conference. West Dover, VT.
Wied HM, Cooch N, Jones JL, Berg BA, Schoenbaum G. (2013). Cocaine use disrupts
behavior and learning that depends on inferred value. National MD/PhD Student
Conference. Keystone, CO.
Wied HM, Cooch N, Jones JL, Esber G, Lucantonio F, Johnson W, Schoenbaum G.
(2013). Assessing the effects of cocaine use on different forms of value based
behavior and learning. Winter Conference on Brain Research. Breckenridge, CO.
McDannald MA, Wegener M, Wied HM, Liu TL, Stalnaker TA, Jones JL, Esber GR,
Trageser J, Schoenbaum G. (2012). Signaling reward prediction for value and identity
in rodent orbitofrontal cortex during Pavlovian unblocking. 2012 Society for
Neuroscience Annual Meeting. New Orleans, LA.
Wied HM, Ribeiro A, Leach JB, Powell EM. (2009). Using biomaterials to direct neural
cell fate response to ligands. World Stem Cell Summit.
Wied HM, Hinchey T, Sinai A, Crone N, Los J, Franaszczuk P, Boatman-Reich D.
(2009). Intracranial mapping of human auditory association cortex using simple and
complex sounds. The Association for Research in Otolaryngology.
Wied HM, Sinai A, Crone N, Franaszczuk P, Miglioretti D, Boatman-Reich D. (2008).
Intracranial mapping of auditory perception: event-related recordings and
electrocortical stimulation. University of Maryland School of Medicine Medical
Student Research Day.
Fanselow MS, Quinn JJ, Tinsley MR, Lueck HM, Liu D, Martikyan A. (2003). The
hippocampus, episodic retrieval and associative fear conditioning. Society for
Neuroscience.
Tinsley MR, Quinn JJ, Lueck H, Fanselow MS. (2002). Anisomycin attenuates
consolidation of predator fear. Society for Neuroscience.
Research Support
Ongoing Research Support
F30 DA033100, 03/08/2012 - present
The Role of Dopaminergic Error Signaling in Outcome-Specific Learning
Role: PI
Completed Research Support
T32 NS063391, 09/01/2010-08/31/2011
University of Maryland School of Medicine, Training Program in Neuroscience
PI: Margaret M. McCarthy, Ph.D.
Role: Trainee
Abstract
Title of Dissertation: The Orbitofrontal Cortex and Inferred Value: Neural Correlates and
the Effects of Cocaine
Heather M. Wied, Doctor of Philosophy, 2014
Dissertation Directed by:
Geoffrey Schoenbaum, M.D., Ph.D.
National Institute on Drug Abuse
Branch Chief and Senior Investigator, Tenured
Cellular Neurobiology Research Branch
Behavioral Neurophysiology Research Section
The orbitofrontal cortex (OFC) is important for guiding behavior and making
decisions based on predicted outcomes. Recently, the Schoenbaum lab, using a sensory
preconditioning task, demonstrated that the OFC is critical for the ability to make
decisions in novel situations based on predicted outcomes, and that it is not essential for
decisions based off of previous experiences. This study also demonstrated that the OFC
was important for learning when expectations about predicted outcomes were not met.
This study laid the foundation for this investigation which seeks to understand the
neurophysiological mechanism of how the OFC might signal this inferred value, and if
cocaine exposure disrupts OFC's ability to use inferred value to adequately guide
behavior and new learning.
To test the hypothesis that the OFC is signaling inferred value in the sensory
preconditioning task, electrodes were implanted into the OFC in rats. During the
preconditioning phase the mean firing rate of neurons to the stimulus-stimulus cues were
strongly correlated. This correlation was still significant at the time of testing, suggesting
the conditioned cues firing rate predicted the preconditioned cues firing rate.
Cocaine addiction is characterized by impaired decision-making. Evidence
suggests that cocaine-induced changes in the OFC may lead to difficulty with flexible
and adaptive model-based behavior. The second part of this study tests the hypothesis
that exposure to cocaine through self-administration would disrupt the ability of animals
to use inferred value to adequately guide behavior and new learning. Rats self-
administered cocaine for 14 days, and after a four-week withdrawal period, were tested
using sensory preconditioning and blocking. The previous exposure to cocaine disrupted
both the expression of behavior and learning that is contingent upon inferred values, but
it did not affect behavior or learning when normal behavior could be supported by cached
values. These results are similar to the effects observed when the OFC is inactivated
during the critical sensory preconditioning probe test and during blocking. Moreover,
they are likewise consistent with the idea that dysfunction in decision making that arises
in drug addiction is potentially mediated through neural deficits within the OFC.
The Orbitofrontal Cortex and Inferred Value:
Neural Correlates and the Effects of Cocaine
by
Heather M. Wied
Dissertation submitted to the Faculty of the Graduate School of the
University of Maryland, Baltimore in partial fulfillment
of the requirements for the degree of
Doctor of Philosophy
2014
©Copyright 2014 by Heather M. Wied
All rights reserved
iii
Acknowledgements
I offer my sincerest gratitude to Dr. Geoffrey Schoenbaum for being my advisor
and offering his support throughout my graduate school training. He has been an
outstanding mentor, teacher and advocate. He routinely offered insights for professional
growth and achievement, including constructive feedback on the presentation and
communication of scientific concepts. Dr. Schoenbaum has been incredibly supportive of
my goals in becoming a clinician-scientist, understanding the difficult timelines of the
MD/PhD program and helping to ensure I return to clinic rotations on time. I am also
grateful for the members of my thesis committee: Drs. Matthew Roesch, Gregory Elmer,
Asaf Keller and Patricio O’Donnell.
I want to thank the entire Schoenbaum lab for the many discussions about science
and the orbitofrontal cortex. Their dedication was inspiring and made it a joy to be in the
lab. In particular, I cannot thank Dr. Brian Sadacca enough for his patience and guidance
as I learned about electrophysiology and how to analyze the data.
I also want to acknowledge the University of Maryland School of Medicine and
the MD/PhD Program and Program in Neuroscience for providing me with this
outstanding opportunity. Additionally, I want to thank my fellow MD/PhD students for
their support through these years.
Lastly, I simply could not have persisted without the constant love and support of
my family and friends, especially my husband, Aaron, my parents John and Pamela
Lueck, and Douglas and Christine Wied. I am most indebted to Aaron and his unfailing
love and support of my dreams of becoming a clinician-scientist. And, of course, to
iv
Christopher: Thank you for understanding no matter what time of day or night, that
mommy was going to the “lab” and that I always missed you and your daddy!
v
Table of Contents
I. Introduction ................................................................................................................. 1
1. Orbitofrontal Cortex: Anatomy................................................................................ 2
2. Orbitofrontal Cortex: Behavior ................................................................................ 7
A. Reversal Learning ................................................................................................ 8
B. Outcome Devaluation ......................................................................................... 11
C. Unblocking ......................................................................................................... 14
D. Pavlovian-to-Instrumental Transfer ................................................................... 16
E. Pavlovian Overexpectation ................................................................................ 17
F. Sensory Preconditioning .................................................................................... 20
3. Orbitofrontal Cortex: Neurophysiology ................................................................. 26
4. Cocaine: Effects on the Orbitofrontal Cortex ........................................................ 29
A. Cocaine: Effects on Orbitofrontal Cortex Dependent Behaviors ....................... 32
B. Cocaine: Effects on Orbitofrontal Cortex Neurophysiology.............................. 37
II. Investigating the role of the orbitofrontal cortex in inferred value ........................ 42
1. Introduction ............................................................................................................ 42
2. Materials and Methods ........................................................................................... 43
A. Subjects .............................................................................................................. 43
B. Apparatus ........................................................................................................... 44
C. Surgical Procedures ............................................................................................ 45
D. Histology ............................................................................................................ 45
E. Behavioral Task.................................................................................................. 45
F. Single-Unit Recording........................................................................................ 48
3. Results .................................................................................................................... 49
A. Sensory Preconditioning Behavioral Results: .................................................... 49
B. Sensory Preconditioning in Probe Tests that demonstrated A>C Behavior....... 51
C. Neural Probe Test Results: ................................................................................. 55
4. Discussion .............................................................................................................. 60
III. Disruption of model-based behavior and learning by cocaine self-administration in
rats… ............................................................................................................................... 63
1. Abstract .................................................................................................................. 63
2. Introduction ............................................................................................................ 64
3. Materials and Methods ........................................................................................... 67
A. Subjects: ............................................................................................................. 67
vi
B. Apparatus: .......................................................................................................... 67
C. Surgical procedures: ........................................................................................... 68
D. Self-Administration: ........................................................................................... 69
E. Sensory Preconditioning: ................................................................................... 69
F. Inferred Value Blocking: .................................................................................... 71
G. Response measures: ........................................................................................... 72
H. Statistical Analyses: ........................................................................................... 73
4. Results .................................................................................................................... 73
A. Self-Administration: ........................................................................................... 73
B. Sensory Preconditioning: ................................................................................... 74
C. Inferred Value Blocking: .................................................................................... 78
5. Discussion .............................................................................................................. 80
IV. Conclusions ............................................................................................................ 85
References ........................................................................................................................ ix
vii
List of Figures
Figure I.1 Comparative anatomy of the human, monkey and rat frontal cortex. ............... 4
Figure I.2 Anatomical relationships of the OFC in rat and monkey. .................................. 6
Figure I.3 Effect of OFC lesions on reversal learning. ..................................................... 10
Figure I.4 Effect of OFC lesions on outcome devaluation. ............................................. 13
Figure I.5 Effect of OFC lesions on blocking. ................................................................. 15
Figure I.6 Effect of OFC lesions on Pavlovian-to-instrumental transfer. ........................ 17
Figure I.7 Effects of OFC inactivation on changes in behavior after overexpectation.... 19
Figure I.8 Effect of OFC inactivation on sensory preconditioning. ................................ 24
Figure I.9 Effect of OFC inactivation on learning based on inferred value. .................... 25
Figure I.10 Conditioned responding and cue-evoked activity summates at the start of
compound training. ............................................................................................... 28
Figure I.11 Decreased gray matter concentration in the OFC. ......................................... 30
Figure I.12 Decreased glucose metabolism in the OFC. .................................................. 31
Figure I.13 Changes in spine density. ............................................................................... 32
Figure I.14 Effect of cocaine on reversal learning. ........................................................... 34
Figure I.15 Effect of cocaine on outcome devaluation. .................................................... 35
Figure I.16 Effect of cocaine on pavlovian overexpectation. .......................................... 37
Figure I.17 The effect of cocaine on OFC cue-selectivity neurons in reversal learning. . 39
Figure I.18 The effect of cocaine on OFC cue-selectivity neurons in Pavlovian
overexpectation. .................................................................................................... 41
Figure II.1 Sensory preconditioning. ................................................................................ 46
viii
Figure II.2 Sensory preconditioning: Number of nosepokes during preconditioning and
probe test. .............................................................................................................. 51
Figure II.3 Distribution of sessions that demonstrated greater responding to cue A than to
cue C. .................................................................................................................... 52
Figure II.4 Sensory preconditioning behavior results from the sessions that demonstrated
A greater than C responding. ................................................................................ 53
Figure II.5 Sensory preconditioning: Number of nosepokes during preconditioning and
probe test from sessions that demonstrated A greater than C responding. ........... 55
Figure II.6 Proportion of neurons with significant cue-selective activity during the
preconditioning. .................................................................................................... 56
Figure II.7 Proportion of neurons with significant cue-selective activity during the 10
second auditory cues during probe test. ................................................................ 58
Figure II.8 Cue-evoked activity during probe test. ........................................................... 59
Figure III.1 Self-administration. ....................................................................................... 74
Figure III.2 Cocaine self-administration disrupts the expression of behavior that depends
upon inferred values. ............................................................................................. 77
Figure III.3 Cocaine self-administration disrupts learning that depends upon inferred
values. ................................................................................................................... 79
1
I. Introduction
The interest and modern understanding of the frontal lobe was sparked by a young
railroad worker in Cavendish, Vermont named Phineas Gage. He survived a remarkable
explosion on September 13, 1848 that propelled an iron tamping bar into his head and
through his frontal lobes, including the orbitofrontal cortex. After his unlikely recovery
from the physical wounds, the attending physician, Harlow (1868), described a change in
Gage's behavior as follows:
"He is fitful, irreverent, indulging at times in the grossest
profanity (which was not previously his custom),
manifesting but little deference for his fellows, impatient of
restraint or advice when it conflicts with his desires, at times
pertinaciously obstinate, yet capricious and vacillating,
devising many plans of future operations, which are no
sooner arranged than they are abandoned in turn for others
appearing more feasible. A child in his intellectual capacity
and manifestations, he has the animal passions of a strong
man. Previous to his injury, although untrained in the
schools, he possessed a well-balanced mind, and was looked
upon by those who knew him as a shrewd, smart
businessman, very energetic and persistent in executing all
his plans of operation. In this regard his mind was radically
changed, so decidedly that his friends and acquaintances
said he was 'no longer Gage.'"
Similar to the reported changes in Gage's behavior, Damasio (1994) described patients
with orbitofrontal damage as being impulsive, disinhibited and perseverative. Oftentimes,
patients with prefrontal lesions have difficulty with insight, inference and decision-
making. Orbitofrontal cortex damage appears to leave patients inflexible and their
2
behavior is more stimulus-driven. Many times patients are observed as acting for instant
gratification instead of thinking about long term consequences. These behavior changes
are often very evident to family and friends; but too often patients with frontal lobe
lesions are not recognized as readily in the medical field, as they pass most neurological
exams.
The orbitofrontal cortex continues to be of great interest and many recent
behavioral and neurophysiological studies have begun to better elucidate its role. The
anatomy, behavior and neurophysiology of such experiments will be reviewed. In
addition, recent evidence suggests that the psychostimulant cocaine affects the
orbitofrontal cortex as well, leading to behaviors oftentimes seen in patients with
orbitofrontal cortex damage. This appears to include impulsive decisions and stimulus-
based responding. The evidence of how cocaine may affect the orbitofrontal cortex is also
reviewed.
1. Orbitofrontal Cortex: Anatomy
The orbitofrontal cortex (OFC) is located in the ventral part of the frontal lobe.
The OFC has remarkably similar connections across species, offering important clues as
to the critical function of this brain region, despite anatomical differences between
humans, primates and rodents. Although the OFC is loosely defined in terms of borders
throughout the different species, and the cytoarchitecture is different, especially in
comparison of humans and primates to rodents (Figure I.1), the connections are very
similar. Rose and Woolsey (1948) defined the prefrontal cortex in terms of the
projections of the mediodorsal nucleus of the thalamus (MD). Though the regions in
3
rodents lack the dense granular layer characteristic of much of the primate prefrontal
cortex, defining it according to the MD projections allows homologies to be drawn
between prefrontal regions in non-primate and primate species. This includes the rodent
orbitofrontal cortex. Specifically, the projections of the medial and central mediodorsal
thalamus to orbital and agranular ventral and dorsal insular areas of the rat prefrontal
cortex is homologous to the primate orbitofrontal region (Leonard, 1969, Krettek and
Price, 1977, Groenewegen, 1988, Ray and Price, 1992, Schoenbaum and Setlow, 2001)
as seen in Figure I.2.
The non-primate mediodorsal nucleus of the thalamus receives afferents that are
similar to the primate MD afferents including from the amygdala, medial temporal lobe,
and ventral pallidum/ventral tegmental area, and olfactory input from the piriform cortex
(Krettek and Price, 1977, Groenewegen, 1988, Ray and Price, 1992). In rodents, the OFC
that is similar to the primate orbitofrontal region includes the dorsal and ventral agranular
cortex as well as lateral and ventrolateral orbital regions; however, the defined area of the
orbitofrontal cortex that is similar across species does not include the structures that
surround the medial wall of the hemispheres, including the medial or ventromedial wall
in non-primates (Roesch and Schoenbaum, 2006).
4
Figure I.1 Comparative anatomy of the human, monkey and rat frontal cortex.
(a,b) Architectonic maps of the medial (top) and orbital (bottom) surfaces of the
frontal lobe in humans (Ongur et al., 2003) (a) and monkeys (Carmichael and Price,
1994) (b). (c) Medial (top) and lateral (bottom) frontal cortex in rats (Palamero-
Gallagher and Zilles, 2004). Agranular cortex lacks layer IV. Dysgranular cortex
contains a rudimentary layer IV. Granular cortex has a well-developed layer IV. Layer
IV neurons are described as granular because their cell bodies are small and round,
and changes in this layer are clearly visible as one transitions from agranular to
granular cortex.
Abbreviations: AC, anterior cingulate area; AON, anterior olfactory nucleus; c,
caudal; cc, corpus callosum; Fr2, second frontal area; I, insula; i, inferior; Ia, agranular
infralimbic cortex; IL, infralimbic cortex; l, lateral; LO, lateral orbital area; m, medial;
M1, primary motor area; MO, medial orbital area; o, orbital; p, posterior; Par, parietal
cortex; Pir, Piriform cortex; PL, prelimbic cortex; r, rostral; s, sulcal; v, ventral; VO,
ventral orbital area. Numbers indicate cortical fields, except that after certain areas,
such as Fr2 and AC1, they indicate subdivisions of cortical fields. *Source of image (Wallis, 2012)
5
Afferent input to OFC is also similar across species. In both primates and rodents,
the OFC receives information from all sensory modalities (Carmichael and Price, 1995).
Additionally, the connections between the OFC and limbic system, including the
amygdala, are extensive (Carmichael and Price, 1995). These reciprocal connections of
the basolateral amygdala to the agranular insular cortex are thought to be involved in
affective and motivational aspects of learning (Brown and Schafer, 1888, Kluver and
Bucy, 1939, Weiskrantz, 1956, Everitt and Robbins, 1992, LeDoux, 1996, Holland and
Gallagher, 1999, Davis, 2000, Gallagher, 2000, Baxter and Murray, 2002, Roesch and
Schoenbaum, 2006). The OFC also has strong efferent projections to the ventral striatum
overlapping with innervations from limbic structures such as the amygdala and
hippocampus (Groenewegen et al., 1987, McDonald, 1991, Haber et al., 1995, Haber and
Knutson, 2010). As seen in Figure I.2, the similarities across species in the connections of
the OFC with limbic structures and the ventral striatum are compelling. The similarities
in these connections suggest similarities in functional interactions with the major
components of the forebrain across humans, primates and non-primates species including
rodents. Anatomically, the OFC is an ideal candidate for the integration of sensory and
associative information. This region is ideally suited to evaluating outcomes, as it
receives input from all sensory modalities.
6
Figure I.2 Anatomical relationships of the OFC in rat and monkey.
Based on the connections with mediodorsal thalamus, amygdala and striatum, the orbital and
agranular insular areas in rat prefrontal cortex are homologous to primate orbitofrontal cortex.
In both species, orbitofrontal cortex receives conspicuous input from sensory cortices,
associative information from amygdala and outputs to the motor system through striatum.
Abbreviations: AId, dorsal agranular insula; AIv, ventral agranular insula; c, central; m,
medial; ABL, basolateral amygdala; rABL, rostral basolateral amygdala; CD, caudate; NAc,
nucleus accumbens core; VP, ventral pallidum; LO, lateral orbital; VO, ventral orbital,
including ventrolateral and ventromedial orbital regions.
* Source of image (Roesch and Schoenbaum, 2006)
OFC
OFC
Rat Monkey
MDMD
Amygdala Striatum Amygdala
AId 1211/13
14
CD
NAc
VP
ABL
mcmcAIv LO VO
rABL
NAc
VP
Striatum
Sensory Information
Motor Output
7
2. Orbitofrontal Cortex: Behavior
The OFC has been thought to be important for guiding behavior and making
decisions based on predicted outcomes. Corticolimbic circuits in the brain have been
implicated in signaling information about expected outcomes and their general value.
Encoding the general or cached value of items facilitates rapid and transitive decision
making — comparing, say, apples and oranges — and discrepancies in this predicted
value also drive learning. Recent work suggests that brain circuits also maintain
information specific to particular outcomes and that decisions - and learning - can be
driven by changes in outcome information even when value remains unchanged.
When our actions are made simply on the basis of previously experienced
rewards, we use these cached or so called “model-free” value representations. In these
situations, the value of an action or predictive cue is represented in a common currency
without any knowledge about the specific form and features of the reward it predicts. In
fact, the sequential structure of the cue-reward association is made without thinking about
the specifics of the reward. Although model-free representations allow for fast actions
without thinking, there is often much more involved in some decisions that we make. For
these more intricate and computationally demanding decisions that require flexibility,
model-based representations are essential; decisions and actions are made based on
expected future outcomes according to a more complex model of the environment. This
allows decisions to be made even if a person has never experienced the direct reward
from taking this action since it can be predicted in the moment from the previously
acquired model of the task or environment.
8
The OFC is thought to be important for associative learning. Over the years, it has
become clear that it has a specific and critical role in the ability to integrate and signal
information about specific outcomes (Burke et al., 2008, McDannald et al., 2011,
McDannald et al., 2005, Gallagher et al., 1999, Izquierdo et al., 2004, Ostlund and
Balleine, 2007). In this role, it appears to be most important in novel, unfamiliar or
unprecedented situations, integrating associative information from other areas about past
experiences and providing an in-the-moment prediction about what may happen based off
of the history of previous experiences. Although initially the OFC was shown to be
critical for reversal learning and thus important for flexible and outcome-guided
behavior, its role as a point of integration in this network to in-the-moment guided
behaviors has become increasingly more clear from a series of elegant behavioral
experiments. This includes outcome devaluation, blocking, Pavlovian-to-instrumental
transfer, Pavlovian overexpectation and, most recently, sensory preconditioning. These
experiments help to more clearly define the role of the orbitofrontal cortex in model-
based behaviors.
A. Reversal Learning
In reversal learning, animals are taught that one stimulus leads to a reward, while
another stimulus leads to no reward or to a punishment. Once these cue-outcome
associations are well established, they are then reversed. The ability to understand that
the cue-outcome associations have switched requires the animal to be flexible and to
adapt to its environment. An animal must be able to change its behavior in the event of an
9
unexpected outcome. It has been found that the OFC is required for this flexibility, as
animals with OFC lesions across species are unable to adapt easily to these changes in the
cue-outcome associations. Importantly, animals are able to learn the initial discrimination
as fast as control animals (Schoenbaum et al., 2003). While the animals with OFC lesions
can eventually acquire the new cue-outcome associations after a much longer period of
time and many more trials, they have a markedly difficult time reversing their behaviors
when these cue-outcome associations are then reversed back to the original associations.
This difficulty in reversal learning tasks has been repeatedly shown throughout many
species and experiments, as seen in Figure I.3 (Jones and Mishkin, 1972, Rolls et al.,
1994, Bechara et al., 1997, Meunier et al., 1997, Chudasama and Robbins, 2003, Fellows
and Farah, 2003, McAlonan and Brown, 2003, Hornak et al., 2004, Izquierdo et al., 2004,
Pais-Vieira et al., 2007, Bissonette et al., 2008, Reekie et al., 2008, Teitelbaum, 1964,
Butter, 1969, Schoenbaum et al., 2003). However, reversal learning is not specific, and it
has also been suggested that the OFC is important for response inhibition to a previously
rewarded cue (Jones and Mishkin, 1972, Eagle et al., 2008, Ferrier, 1876, Izquierdo and
Jentsch, 2012, Jentsch and Taylor, 1999, Man et al., 2009), or that it is important for
updating specific outcome expectancies to guide decisions (Schoenbaum et al., 2009,
Holland and Gallagher, 2004, Wallis, 2007).
10
Figure I.3 Effect of OFC lesions on reversal learning.
(a) Rats were trained to sample odors at a central port and then respond at a nearby
fluid well. In each odor problem, one odor predicted sucrose and a second quinine.
Rats had to learn to respond for sucrose and to inhibit responding to avoid quinine,
and then the odor-outcome associations were reversed. Shown is the number of trials
required by controls and OFC-lesioned rats to meet a 90% performance criterion. OFC
lesions have intact retention of a previously learned discrimination, but impaired
reversal learning. Adapted from (Schoenbaum et al., 2003) (b) Group mean errors to
criterion for initial learning and nine serial reversals in object reversal learning in
monkeys with bilateral OFC lesions. OFC-lesioned monkeys are specifically impaired
in reversal learning. Adapted from (Izquierdo et al., 2004) (c) Initial stimulus-
reinforcer association learning and reversal learning performance in subjects with
damage to the ventromedial prefrontal region and in controls. Initial learning and
reversal performance is expressed as trials before the learning criterion is met.
Subjects with lesions make more than twice as many errors as healthy controls in the
reversal phase of the task. *P < 0.05; **P < 0.01. Adapted from (Fellows and Farah,
2003)
*Source of image (Lucantonio et al., 2012)
11
B. Outcome Devaluation
Outcome devaluation experiments in rodents and monkeys have led to a much
better understanding of the function of the OFC (Izquierdo and Murray, 2010, Holland
and Rescorla, 1975, Holland and Straub, 1979, Gallagher et al., 1999, Pickens et al.,
2005, Pickens et al., 2003, Izquierdo and Murray, 2004, Izquierdo et al., 2004, Machado
and Bachevalier, 2007, West et al., 2011, Murray et al., 2007). From these experiments, it
is apparent that the OFC is critical for integrating information about reward predictive
cues and the new value of those rewards. In outcome devaluation experiments, a neutral
cue is paired with the delivery of a reward such as an appetitive food outcome. Once this
association is well established, the outcome is devalued by pairing the reward with illness
or satiation, thereby decreasing the conditioned response to the reward-predictive cue.
This decrease in responding is examined by looking at the behavior when the conditioned
cue is presented by itself in extinction. This decrease in responding to the cue occurs
immediately, even though the cue was never explicitly paired with the devalued reward.
The decline in conditioned responding after outcome devaluation is not new learning
since the cue was never paired directly with the devalued reward. Instead, the animal
must presumably use the previously acquired association between the cue and the non-
devalued food, combined with an updated representation of the outcome and its new
value, to guide or modify responding. Notably, animals with OFC lesions do not show
this ability (Figure I.4b and c) (Izquierdo and Murray, 2010, Gallagher et al., 1999,
Pickens et al., 2005, Pickens et al., 2003, Izquierdo and Murray, 2004, Izquierdo et al.,
2004, Machado and Bachevalier, 2007, West et al., 2011, Murray et al., 2007). The
12
animals with OFC lesions do not show an impairment in their ability to learn the initial
cue-outcome association or in their ability to devalue the food, only in their ability to
apply this knowledge to guide their behavior in the final test. The OFC is thought to be
important for helping to imagine the outcome that is coming when the cue is presented,
such that, although the animals have never directly experienced the associated cue
leading to the devalued reward, they are able to imagine that the cue would lead to the
devalued reward decreasing their response to the cue. Accordingly, human fMRI studies
have also demonstrated that there are blood-oxygen-level dependent (BOLD) signal
changes in the OFC when the value of expected outcomes is changed, providing
additional support to the idea that the OFC signals changes in value of outcomes
(O'Doherty et al., 2000a, Gottfried et al., 2003) (Figure I.4a).
13
Figure I.4 Effect of OFC lesions on outcome devaluation.
(a) Changes in BOLD signal in human orbitofrontal cortex after reinforcer
devaluation. Subjects were scanned during presentation of odors of different foods.
Subsequently, one food was devalued by overfeeding and then subjects were
rescanned. Subjective appetitive ratings of the odor (top) and BOLD response to the
odor-predicting cue in orbitofrontal cortex (bottom) decline for satiated but not
nonsatiated foods. (b) Changes in Pavlovian conditioned responding in sham and
OFC-lesioned rats after reinforcer devaluation. Rats were trained to associate a light
cue with food. Subsequently, the food was devalued by pairing it with LiCl-induced
illness and response to the cue was assessed in a final probe session. Rats with OFC
lesions fail to show any effect of devaluation on conditioned responding (percentage
of time in food cup), despite normal conditioning and devaluation of the food reward.
(c) Changes in discriminative responding in sham and orbitofrontal-lesioned monkeys
after reinforcer devaluation. Monkeys were trained to associate different objects with
different food rewards. Subsequently, one food was devalued by overfeeding and then
discrimination performance was assessed in a probe test. The figure illustrates a
difference score comparing post- and pre-satiation bias; OFC-lesioned monkeys fail to
bias their choices away from objects associated with the satiated food. *P < 0.05; **P
< 0.01. Adapted from (Gottfried et al., 2003, Gallagher et al., 1999, Izquierdo et al.,
2004).
*Source of image (Lucantonio et al., 2012)
14
C. Unblocking
McDannald et al. (2011) provided additional evidence of the OFC’s role in
outcome-related learning. In order to determine the differences between outcome-specific
and value-based learning in the OFC, they used a Pavlovian unblocking procedure
(Kamin, 1969). In this procedure, a rat is trained that a cue is associated with a particular
food outcome. After this association is learned, a second cue is presented in compound
with the first cue, but leading to the same food outcome. In this situation, learning to the
second cue is blocked because the reward is completely predicted by the first cue as the
food outcome is exactly the same. In unblocking, the second cue is presented in
compound with the first cue, but is followed by a different food outcome, either in
quantity of food (value) (Holland, 1984) or in type of food (identity) (Rescorla, 1999).
This new outcome goes against the animals’ original expectation. Therefore, in
unblocking, learning occurs to the second cue. In McDannald et al.’s (2011) study, they
were able to determine the role of OFC and support the specific-outcome theory by
demonstrating that OFC lesions in rats made prior to training procedures leave
unblocking based on value fully intact but impair unblocking based on specific reward
features such as identity (Figure I.5). These rats had no problems in conditioning and
could fully differentiate between two different reward flavors. However, this study
specifically demonstrated that the OFC is critical for learning about changes in reward
identity.
15
Figure I.5 Effect of OFC lesions on blocking.
Contributions of the orbitofrontal cortex to blocking. The effects of pre-training OFC
lesions on unblocking driven by changes in reward flavor or increases in reward value
are summarized (McDannald et al., 2011). (A) Rats were trained to associate three
visual cues with different flavors and quantities of reward. In an unblocking phase
these visual cues were compounded with novel auditory and aspects of the reward
selectively changed. A probe test was administered in which the novel auditory cues
were presented in isolation and extinction. (B) Representative control (left) and
neurotoxic lesion (right) are shown. (C) Food cup responding to cues X and Y are
shown for control (left) and OFC-lesioned rats (right). Only control rats showed
evidence of learning to cue Y, the cue signaling a change in reward flavor. (D) Food
cup responding to cues X and Z are shown for control (left) and OFC-lesioned rats
(right). Both control and OFC-lesioned rats showed evidence of learning to cue Z, the
cue signaling an increase in reward number.
*Source of image (McDannald et al. 2014)
16
D. Pavlovian-to-Instrumental Transfer
Preceding McDannald et al.'s (2011) results, Balleine's lab had demonstrated
results that support the idea the OFC lesions do not impair instrumental learning or
Pavlovian conditioning, but impair specific transfer of information. In Pavlovian-to-
instrumental transfer, rats are taught that one response leads to one preferred outcome,
while another response leads to another equally preferred outcome. During this training
period, the animals are also taught that one context leads to the same outcome as the first
response, while a second context leads to the second equally preferred outcome. Once
these associations are well established, a transfer test is given in which the context is
presented in combination with the instrumental responding while no rewards are given.
When rats are presented with the context cues, the animal then performs the action that
leads to the same reward more than the action that leads to the other reward. This
phenomenon is called specific Pavlovian-to-instrumental transfer and depends upon the
cues and actions evoking a representation of the same specific rewards. When the OFC is
lesioned, rats fail to change their behaviors to match the context cues thus failing to
demonstrate this specific form of Pavlovian-to-instrumental transfer (Ostlund and
Balleine, 2007, Scarlet et al., 2012) (Figure I.6).
17
E. Pavlovian Overexpectation
As previously mentioned, the specific role of the OFC in reversal learning is hard
to identify: is it due to an inability to understand that changes have occurred to the
outcome, is it due to a deficit in response inhibition, or is it due to changes in some other
function such as impaired learning mechanisms? Support for the latter hypothesis comes
from work using Pavlovian overexpectation (Takahashi et al., 2009). In Pavlovian
overexpectation, animals are taught that one cue predicts a certain reward and that a
second cue also predicts the same reward. When these cues are then presented in
Figure I.6 Effect of OFC lesions on Pavlovian-to-instrumental transfer.
Effect of OFC lesions on Pavlovian-to-instrumental transfer. Mean lever presses per
minute (+ SEM) during the pre-CS period (Pre-CS), the CS that predicted the same
outcome as the response (Same), and the CS that predicted a different outcome (Diff).
*Source of image (Ostlund and Balleine, 2007)
18
compound together, there is an increase in responding in normal animals. This is
interpreted as the animals having integrated the predictions of the two cues, in effect
predicting or imagining that they will be getting double the reward. In fact, the animals
receive the same amount of reward during the compound cue as they were previously
given with the individual cues. In a probe test, in which the cues are presented alone
again without reward, the response to the cues is decreased. This decrease in responding
is thought to be due to the fact that the summed predictions for the cues were not met.
Thus, this discrepancy between the predicted and received outcomes drives inhibitory
learning which leads to decreased responding when the cues are presented alone again.
Takahashi et al. (2009) demonstrated that inactivation of the OFC with baclofen and
muscimol in rats on compound days eliminates this increase in responding to the
compound cues seen in controls (Figure I.7). This suggests that the inactivation of the
OFC did not allow the rats to summate their expectations to the compound of the cues;
rather, they were unable to integrate the two associations together and, thus, did not
expect to receive double the reward. Furthermore, when the OFC-inactivated animals
were then presented the cues alone, without inactivation, they did not decrease their
responding, thereby demonstrating they did not undergo the inhibitory learning that the
control animals showed. Takahashi et al. (2009) provides support to the idea that the OFC
may be important for inhibiting or reversing previously acquired responses due to its role
in signaling expected outcomes.
19
Figure I.7 Effects of OFC inactivation on changes in behavior after
overexpectation.
Top and bottom rows of plots indicate control and OFC-inactivated groups,
respectively. (A) Percentage of responding to food cup during cue presentation across
10 days of conditioning. (B) On left, percentage of responding to food cup during cue
presentation across 4 days of compound training. On right, red and blue bars indicate
average normalized percentage responding to A1/V1 and A2, respectively. (C)
Percentage of responding to food cup during cue presentation in the probe test. Line
graph shows responding across the eight trials and the bar graph shows average
responding in these eight trials. Red, blue, and white colors indicate responding to A1
or A1/V1, A2, and A3 cues, respectively (*p < 0.05; **p < 0.01). Error bars = SEM.
Adapted from (Takahashi et al., 2009). *Source of image (Takahashi et al., 2009)
20
F. Sensory Preconditioning
Sensory preconditioning offers the unique ability to directly assess if behavior is
driven by cached value or inferred value (Jones et al., 2012). As in model-free
reinforcement learning, cached value representations are values that are based off of prior
reinforcement history. Conversely, inferred value, specifically a prediction of the
outcome, must be calculated in-the-moment based upon the relationship between the cues
and the environment. This requires a knowledge of the causal structure of the
environment, or a model-based representation.
Most importantly, sensory preconditioning was able to help sort out the two
dominant theories of the function of the OFC: the neuroeconomic view and the
associative view. Neuroeconomics is the study of the neural control of value-based or
economic decision making. The associative view believes the OFC anticipates outcomes
and engages in model-based reinforcement learning in which there is a map of the
environment that helps to make predictions about outcomes and rewards. Therefore, the
associative view does not require that the OFC be used only in value-guided decisions,
but that a new value would be estimated in a new experience or situation based off prior
experiences.
While the economic value is typically measured through preferences, it appears to
include both cached and inferred value, with the cached value based upon previous
experiences. This is a simple model that is fast and efficient, as long as experiences never
change. But because not all situations are the same or similar, the fact that this cached
value is inflexible makes this model problematic in explaining complex decision making.
21
The cached value is not associated with the outcome itself. This makes it difficult to
update cached values, as they do not take into account the changes in the value of the
expected reward.
The model-based system is a flexible system and allows for new experiences to
have predicted outcomes. It also allows for learning to occur when these predictions do
not match the expected outcome. In this flexible system, there is an associative model of
the world that helps us make decisions with the most up-to-date information. This
associative view of the world helps us predict outcomes across many different scenarios
and different environments and also allows us to constantly learn as we update our
predictions through each and every experience, whether it agrees with what we predicted
or not. The associative view predicts the OFC is providing the flexibility to adapt with
each new experience.
The alternative hypothesis is that the OFC performs the same function in all
settings and that it specifically contributes to value-guided behavior and learning when
value must be inferred or derived from model-based representations. Under this
interpretation, this would hold true even for economic decision making. Similar ideas
were tested and supported in an experiment using sensory preconditioning (Jones et al.,
2012).
In a sensory preconditioning experiment run in the Schoenbaum lab (Jones et al.,
2012), rats were exposed to a pairing of two auditory cues (A→B and C→D) (Figure I.8).
After the rats were exposed to these pairings, one of these cues (B) was paired with a
reward, while the other cue (D) predicted no reward. Once B and D were successfully
conditioned, and thus predictive of reward (B) or no reward (D), the rat was presented
22
with the preconditioned cues (A and C) that were originally paired with the cue that
predicts the reward (B) or no reward (D). The rats demonstrated a strong response to the
preconditioned cue (A) that was paired with the cue that predicted reward (B). But they
did not respond to the cue (C) that had been paired with the cue that did not predict
reward (D). The role of the OFC became evident when the OFC was inactivated by
baclofen and muscimol on the critical probe test with the presentation of A and C alone.
Rats still responded to B but not A, C or D. Because rats still responded to B, it is clear
that the OFC is not critical for cached value. This is contrary to what would be predicted
if neuroeconomics and value-based theories of the OFC include cached value. Instead,
OFC inactivation affected the ability of the rats to infer value to A, which is the cue they
had never directly experienced before with reward. In control animals, the OFC was
available to help rats predict that if B predicts reward, then, as in a classic syllogism, A
may also predict a reward. But because the OFC was offline at the time of the probe test,
the rats did not have access to this cognitive map of the environment, or model-based
representation, that would have helped them predict that A leads to B that leads to
reward. This demonstrates that control animals had access to these predictions about
outcomes and inferred value to the preconditioned cue (A) due to the ability to access this
associative map of their environment. It appears then in sensory preconditioning
differential responding to A versus C (but not B and D) reflects the ability of the animals
to access a model-based representation. In this experiment, animals were able to integrate
the preconditioned stimulus-stimulus associations that did not have any value with the
subsequent associations between the stimulus and reward / non-reward.
23
Jones’ et al. (2012) experimental findings clearly support the outcome-specific
theories of the OFC, as OFC inactivation had no effect on B and D responding (the
cached value cues) but significantly impaired the differential responding to A and C (the
inferred value cues) (Figure I.8). Neuroeconomics and value theories would have
predicted that OFC inactivation would have led to impaired responding to both cached
value cues (B and D) and inferred value cues (A and C), but this was not the case.
Therefore, consistent with the specific-outcome theory, the OFC is necessary for
responding to inferred value or model-based representations of the outcome, but not
when responding can be based on a cached value.
24
Figure I.8 Effect of OFC inactivation on sensory preconditioning.
The OFC is necessary when behavior is based on inferred value. Figures show the
percentage of time spent in the food cup during presentation of the cues during each of
the three phases of training: preconditioning (A and B), conditioning (C and D), and
the probe test (E and F). OFC was inactivated only during the probe test. Cannulae
positions are shown below; vehicle (black circles), OFCi (gray circles). *P < 0.05,
**P < 0.01 or better. Error bars show SEM.
*Source of image (Jones et al., 2012)
25
Jones et al. (2012) went on further to demonstrate that not only was the OFC
required for inferred value, but that it was also critical for learning in a blocking task
based on inferred value. Control animals were impressively able to use the inferred value
cue to block learning, while rats whose OFC was inactivated during training were unable
to use inferred value to block learning (Figure I.9).
Figure I.9 Effect of OFC inactivation on learning based on inferred value.
The OFC is necessary when learning is based on inferred value. Figures show the
percentage of time spent in the food cup during presentation of the cues during
blocking (A and B) and the subsequent probe test (C and D). OFC was inactivated
during blocking. *P < 0.05, **P < 0.01 or better. Error bars show SEM.
*Source of image (Jones et al., 2012)
26
This sensory preconditioning experiment confirmed much of what we know about
the OFC, but provided a novel setting that was able to combine findings into one setting:
its role in predicting specific outcomes and its role in learning. Sensory preconditioning
demonstrated the role of the OFC in using model-based representations for behavior and
learning independent of reversal or outcome devaluation. This study led me question if
the OFC is signaling this inferred value in sensory preconditioning.
3. Orbitofrontal Cortex: Neurophysiology
Numerous studies have demonstrated that neural activity in the orbitofrontal
cortex (OFC) anticipates expected outcomes (Schoenbaum et al., 1998, Levy and
Glimcher, 2011, Padoa-Schioppa and Assad, 2006, O'Doherty et al., 2002, Gottfried et
al., 2003, Thorpe et al., 1983, Tremblay and Schultz, 1999, Kennerley et al., 2011, Sul et
al., 2010), and signal features of these expected outcomes (Schoenbaum et al., 1998,
Delamater, 2007, Ostlund and Balleine, 2007, Steiner and Redish, 2012, Luk and Wallis,
2013). However, it has also been proposed that the OFC signals a value that is
independent of those features (Padoa-Schioppa, 2011, Levy and Glimcher, 2012). The
literature that supports the role of the OFC as a value encoder is supported by single unit
activity and blood-oxygen-level dependent (BOLD) responses in the OFC that track
value (Padoa-Schioppa and Assad, 2006, Plassmann et al., 2007, Levy and Glimcher,
2011) (O'Doherty et al., 2000b, Gottfried et al., 2003). However, it cannot be ruled out
that the value in these studies is a feature of the outcome. Specific features which may
include a value feature may still be the underlying basis of the neural signal.
27
Recently, the Schoenbaum lab demonstrated OFC's neural involvement in a
Pavlovian overexpectation task (Takahashi et al., 2013) demonstrating that the OFC
signals novel estimates regarding the expected outcomes in Pavlovian overexpectation, as
described above. The OFC was demonstrated to be necessary for learning in this
Pavlovian overexpectation task (Takahashi et al., 2009). This study showed that OFC
neural activity demonstrated in-the-moment integration of cue-evoked expectations for
the expected outcome. Takahashi et al. (2013) demonstrated that OFC neural activity
increased to reward-predictive cues across training. Subsequently, these OFC neurons
increased their firing immediately when two cues were presented in compound; when the
animals did not get what they expected, this was clearly reflected in the firing rates of the
neurons as their activity decreased across compound training sessions (Figure I.10).
Additionally, when these cues were then presented again by themselves after compound
training, OFC neural activity was also decreased. These data show that the OFC reflects
the prediction of the expected outcome, and that this signaling becomes important in
updating associative representations in other areas of the brain as expectations are not
met.
28
The results from the Pavlovian overexpectation experiment strongly support the
role of the OFC in estimating new outcomes due to spontaneous integration of outcome
expectations. Specifically, these results do not support proposals that OFC represents a
value in common neural currency as neuroeconomics may suggest (Levy and Glimcher,
2011, Montague and Berns, 2002, Padoa-Schioppa and Assad, 2006, Padoa-Schioppa,
2011, Padoa-Schioppa and Assad, 2008, Plassmann et al., 2007). This is because the
neural summation was greater than the sum of the parts. Indeed, this summation
represents a novel expectation of something the rats have never experienced.
Figure I.10 Conditioned responding and cue-evoked activity summates at the
start of compound training.
Line plot indicates the ratio between normalized firing to A1/V and A2 during each
compound training session (CP1–CP4). N indicates number of cue-responsive neurons
in each session. A1/A2 ratio increased significantly in the compound phase of the
probe, and then gradually decreased (ANOVA, ∗∗p < 0.01, ∗p < 0.05). The line plot in
inset indicates normalized firing to A1/V and A2 across six trials in the second half of
the compound probe session, with red diamonds for A1 and blue squares for A2. Error
bars = SEM. *Source of image (Takahashi et al., 2013)
29
As demonstrated from both behavioral and neurophysiological experiments cited
above, there is increased evidence that OFC is signaling information about specific-
outcomes, even in new situations where the animal may imagine the value of the
upcoming outcome.
As described, the OFC is important for guiding behavior and making decisions
based off predicted outcomes and the specific-features of those outcomes. In particular,
the OFC is important to drive learning when expected outcomes or predictions are not
met.
4. Cocaine: Effects on the Orbitofrontal Cortex
Recent evidence strongly suggests that cocaine can cause structural, molecular
and functional changes in the orbitofrontal cortex (OFC) (Lucantonio et al., 2012).
Although human studies are able to demonstrate there are differences in the OFC of
people who abuse cocaine and those who do not, these studies are generally unable to
determine if the cocaine caused the differences or if the differences were there prior to
the cocaine use. In contrast, animal studies have helped to determine that cocaine has
specific effects on the OFC.
OFC structural abnormalities have been reported, including decreased gray
matter concentration as seen in Figure I.11 and decreases in frontal white matter integrity
in psychostimulant users (Franklin et al., 2002, O'Neill et al., 2001b, Ersche et al., 2011)
(Volkow et al., 1988, O'Neill et al., 2001a, Lyoo et al., 2004). These anatomical
differences have also been observed in functional neuroimaging, where a decrease in
30
glucose metabolism has been seen throughout the brain, and are particularly decreased
when actively using cocaine (Figure I.12) (London et al., 1990, Volkow et al., 1990).
When cocaine users are actively craving cocaine during periods of abstinence, the OFC is
hypermetabolic (Volkow et al., 1991). However, with prolonged withdrawal, the OFC in
cocaine users appears to be hypoactive (Volkow et al., 1993, Adinoff et al., 2001,
Volkow et al., 2001).
Figure I.11 Decreased gray matter concentration in the OFC.
On the left, whole brain renderings showing regions of decreased gray matter density (p < .01)
in the cocaine-dependent group relative to the control group. From left to right: left and right
sagittal, posterior, and anterior, right and left side, and ventral and dorsal views of regions
showing decreased gray matter density. On the right, sagital slices through the midline show
continuous deficits in gray matter extending from the ventromedial orbitofrontal (VMOC)
cortex through the anterior cingulate (AC) region.
*Source of image (Franklin et al. 2002)
31
In animal studies, rodents that self-administered amphetamines produced an
altered dendritic morphology in different areas of the brain after prolonged withdrawal
(Figure I.13). Interestingly, the medial prefrontal cortex pyramidal neurons and nucleus
accumbens medium spiny neurons demonstrated an increase in dendritic length and spine
density, while the OFC demonstrated a decrease in spine density (Kolb et al., 2004,
Crombag et al., 2005).
Figure I.12 Decreased glucose metabolism in the OFC.
Images of the brain in a healthy control and in an individual addicted to cocaine. The
images were obtained with positron emission tomography and [18F]fluoro-2-
deoxyglucose to measure glucose metabolism, which is a sensitive indicator of
damage to the tissue in the brain. Note the decreased glucose metabolism in the OFC.
*Source of image (Volkow and Li, 2004)
32
A. Cocaine: Effects on Orbitofrontal Cortex Dependent Behaviors
The above data shows that there are changes structurally and molecularly in the
OFC that occur due to cocaine and psychostimulant abuse. However, the evidence that
cocaine affects the OFC functionally comes from experiments that have demonstrated
Figure I.13 Changes in spine density.
On the right, mean (± SEM) spine density (the number of spines/10 μm) on apical and
basilar dendrites of pyramidal neurons in the medial and orbitofrontal cortices in the
amphetamine self-administration (red bars), sucrose-reward training (orange bars) or
in untreated control groups (white bars). (* or Ɨ
, significant difference at p<0.05). On
the left, brain regions (shaded areas) and neuronal types analyzed for alterations in
spine density as a function of past experience with amphetamine or sucrose taking.
Abbreviations: CA1, CA1 field of the hippocampus; DG, dentate gyrus; Nacc, nucleus
accumbens (shell); MPC, medial prefrontal cortex (Cg3); OFC, orbital frontal cortex
(AID); a, apical dendrites; b, basilar dendrites. Adapted from Paxinos and Watson
(1997).
*Source of image (Crombag et al., 2005)
33
deficits in OFC-dependent behaviors, including reversal learning, outcome devaluation,
and Pavlovian overexpectation.
Jentsch et al. (2002) demonstrated that cocaine resulted in reversal learning
deficits. In this study, monkeys were given intraperitoneal injections of cocaine (2 mg/kg
or 4 mg/kg) for 14 days. After this cocaine exposure, the monkeys were tested after 9 or
30 days of withdrawal from cocaine. As would be expected in an OFC-dependent
behavior, the monkeys who were exposed to cocaine were still able to perform the
original discrimination, but had difficulty and made many more errors when these cue-
outcome associations were reversed (Figure I.14). Since these findings, these same
deficits in reversal learning have been demonstrated across species, including in humans
(Fillmore and Rush, 2006, Ersche et al., 2008) and rats (Schoenbaum et al., 2004, Calu et
al., 2007) (Figure I.14).
34
As previously noted, there are limitations to the specificity of reversal learning
and to the explanation of what the deficits actually mean. It is important, therefore, to
consider what other OFC-dependent behaviors cocaine affects. These long-lasting effects
were demonstrated in an outcome devaluation experiment where the cocaine-exposed rats
performed similarly to OFC-lesioned rats (Figure I.15) (Schoenbaum and Setlow, 2005).
Rats were exposed to cocaine injections (30 mg/kg) for 14 days and underwent
withdrawal for 3 weeks. As seen in the OFC-lesioned experiment, rats demonstrated
Figure I.14 Effect of cocaine on reversal learning.
(a) Incorrect responses in monkeys exposed to non-contingent cocaine (4 mg/kg, once
daily for 14 d) or saline in a reversal task. Compared with controls, cocaine-treated
monkeys showed similar acquisition but impaired discrimination-reversal learning. (b)
Rats were trained to self-administer cocaine (0.75 mg/kg per infusion, 4 hours per day
for 14 days) and then tested on the an odor discrimination reversal task, after
approximately 3 months of withdrawal from the drug. Cocaine self-administration has
no effect on retention but impairs reversal learning. (c) Consecutive incorrect
responses immediately after the change in reward contingencies in chronic cocaine
users. Cocaine users show response perseveration to the previously rewarded stimulus.
*P < 0.05; **P < 0.01. (Jentsch et al., 2002, Ersche et al., 2008, Schoenbaum and
Shaham, 2008).
*Source of image (Lucantonio et al., 2012)
35
normal conditioning and normal extinction of conditioned responding, but importantly,
they were unable to change their conditioned responding in response to devaluation of the
food. They continued to respond to the cue that predicted the devalued cue. The animals
that were exposed to cocaine, it was observed, were not able to update their expectations
about the outcome to the conditioned cue. The cocaine-exposed rats were unable to use
the representation of the outcome value to guide behavior, as seen in animals with OFC
lesions.
Most recently, the effect of cocaine exposure was demonstrated to affect
Pavlovian overexpectation. The results were once again comparable to the deficits seen
Figure I.15 Effect of cocaine on outcome devaluation.
Effect of cocaine on changes in conditioned responding as a result of reinforcer
devaluation in rats. Shown is the mean percentage of time spent in the food cup during
presentation of the food-predicting cue after devaluation in the extinction probe test.
Red bars, devalued groups; blue bars, non-devalued groups. During the devaluation
test, cocaine-exposed rats fail to change conditioned responding as a result of previous
devaluation of the food reward. *p < 0.05. Adapted from Schoenbaum and Setlow,
2005.
*Source of image (Lucantonio et al., 2012)
36
with OFC inactivation during the compound cues (Lucantonio et al., 2014). In this
experiment, rats self-administered cocaine (0.75 mg/kg) for 14 days and then went
through withdrawal for three weeks. After withdrawal, rats failed to show summation
during compound training in addition to not showing spontaneous reduction in
responding to the compounded cue in the later probe test (Figure I.16).
As evidenced by previous OFC lesion and inactivation studies, cocaine use
appears to have very similar behavioral results as OFC lesions and inactivation,
demonstrating that cocaine has the potential for functionally disrupting the OFC.
Specifically, cocaine use does not appear to impair basic learning, but causes a specific
deficit in the ability to adjust behaviors in response to unexpected outcomes. These
impairments do not appear to be short term. In animal models, these deficits appear to
last for weeks to months after the last drug exposure. Thus, it seems very likely that the
OFC functional deficits that cocaine causes may contribute to the long term difficulties
seen with drug relapse and addiction.
37
B. Cocaine: Effects on Orbitofrontal Cortex Neurophysiology
The evidence from behavioral experiments described above suggests that cocaine-
exposed animals that undergo withdrawal have deficits in OFC-dependent behaviors. The
Figure I.16 Effect of cocaine on Pavlovian overexpectation.
Top and bottom rows of plots indicate sucrose and cocaine groups, respectively. (B)
Percentage of responding to food cup during cue presentation across 4 days of
compound training. Red and blue bars on the right indicate average normalized
percentage responding to A1+V1 and A2, respectively. (C) Percentage of responding
to food cup during cue presentation in the probe test. Red, blue, and white colors
indicate responding to A1 or A1+V1, A2, and A3 cues, respectively (*p < 0.05; **p <
0.01). Error bars = SEM.
*Source of image (Lucantonio et al., 2014, in review)
38
OFC acts within a circuit, and there are other areas of the brain that are also affected by
cocaine. However, further implications of systemic cocaine on the OFC are demonstrated
in neurophysiology experiments.
Similar to behavioral experiments, OFC neurons seem to be directly affected by
previous cocaine exposure even after months of withdrawal. The Schoenbaum lab has
demonstrated that cocaine-exposed rats do not develop neural correlates as controls do in
reversal learning and overexpectation (Lucantonio et al., 2014, Stalnaker et al., 2006).
Stalnaker et al. (2006) demonstrated that cocaine-exposed rats did not differ in OFC
neuronal baseline firing rates, or percentage of cue-selective or outcome-selective
neurons during initial learning. However, cocaine-exposed animals did not demonstrate
the same signaling of predicted outcomes as control animals, as their outcome-selective
neurons did not become activated during cue-sampling. In fact, the cue-selective activity
in cocaine-exposed animals was just as likely to signal an unpredicted outcome as a
predicted outcome (Figure I.17). Therefore, cocaine exposure resulted in OFC neurons
failing to signal the expected outcome. Additionally, unlike controls, rats that were
exposed to cocaine and performed poorly in the reversal learning task did not show
evidence of reversal encoding in the OFC. These results suggest that the rats who were
exposed to cocaine do not have the same ability to learn from unexpected outcomes.
39
Furthermore, in a recent study by the Schoenbaum lab (Lucantonio et al., 2014)
similar results were seen in Pavlovian overexpectation. Rats exposed to cocaine failed to
learn in response to changes in expected outcomes. However, the neural signature also
demonstrated that the cocaine-exposed rats’ OFC neurons did increase firing to reward-
predictive cues in training. But unlike the control animals, these neurons did not increase
Figure I.17 The effect of cocaine on OFC cue-selectivity neurons in reversal
learning.
Cue-selectivity indices for neurons that developed outcome-expectant firing during
the pre-criterion block, firing differentially after the rat's response, in anticipation of
either sucrose or quinine delivery. On the top row are shown the populations that
developed quinine-expectant firing, and on the bottom row are shown the populations
that developed sucrose-expectant firing. Red or blue bars represent neurons that were
significantly selective for one or the other of the two odors. In both quinine-expectant
and sucrose-expectant populations, neurons in control rats were more likely to develop
cue-selectivity to the cue that predicted their preferred outcome. Thus the distribution
for quinine-expectant neurons is skewed to the left, and that in sucrose-expectant
neurons in skewed to the right. In contrast, in both populations in cocaine-treated rats,
neurons were equally likely to develop cue-selectivity to either cue. Thus, the
distributions are symmetrically distributed around zero. *Source of image (Stalnaker et al., 2009)
40
in firing when the cues were played in compound and did not show spontaneous decline
in the activity of OFC neurons during extinction (Figure I.18).
The neural data offer further support and understanding of the behavioral results
seen in animals exposed to cocaine. The failure of OFC to signal expected outcomes
leads to a lack of feedback and therefore they are unable to learn when they experience
unexpected outcomes. Cocaine-exposed rats’ OFC neurons do not seem to signal
integration of the cue-evoked expected outcomes. This could possibly explain the delays
seen in reversal learning and the lack of overexpectation behavior. This could also
explain why human cocaine abusers have such difficulty in changing their drug-seeking
behaviors, as they are not able to update their expectations as easily when they
experience the negative consequences of drug use.
41
Figure I.18 The effect of cocaine on OFC cue-selectivity neurons in Pavlovian
overexpectation.
Conditioned responding and cue-evoked activity did not summate at the start of
compound training in the cocaine group. A. Distribution of summation of index scores
for firing to A1 and A2 in the compound probe for control sucrose animals on the top
row and cocaine-exposed animals on the bottom row. Black bars represent neurons in
which the difference in firing was statistically significant (t-test, p < 0.05). B. Line
plot indicates the ratio between normalized firing to A1+V and A2 during each
compound training session (CP-CP4). N's indicate the number of cue-responsive
neurons in each session. A1/A2 ratio increased significantly in the compound phase of
the probe for controls and then gradually decreased; while the A1/A2 ratio did not
increase significantly in the compound phase of the probe, or in subsequent sessions,
for cocaine-exposed animals.
*Source of image (Lucantonio et al., 2014, in review)
42
II. Investigating the role of the orbitofrontal cortex in inferred
value
1. Introduction
The orbitofrontal cortex (OFC) is thought to be an important part of a network for
signaling information about expected outcomes. One critical role the OFC plays within
this network appears to be that of the integrator, incorporating information about events
and predicting outcomes in novel situations or environments not specifically experienced
in the past.
Neural activity of the OFC has been shown to anticipate expected outcomes
(Schoenbaum et al., 1998, Levy and Glimcher, 2011, Padoa-Schioppa and Assad, 2006,
O'Doherty et al., 2002, Gottfried et al., 2003, Thorpe et al., 1983, Tremblay and Schultz,
1999, Kennerley et al., 2011, Sul et al., 2010, Takahashi et al., 2013). As described
previously, the Schoenbaum lab recently demonstrated that the OFC signals the novel
estimates regarding the expected outcomes in a Pavlovian overexpectation task
(Takahashi et al., 2013). In this study, it was demonstrated that neural activity in the OFC
at the time of summation increases suddenly on the very first trial of the compound cue,
then declines as the expectations of the compound cue are not met. These data clearly
demonstrate that OFC reflects the ability to predict expected outcomes in the moment.
Additional evidence supporting these conclusions is seen in that rats exhibited impaired
learning as their OFC neural signaling was absent (Takahashi et al., 2013).
43
As discussed previously, Jones et el. (2012) demonstrated that the OFC was
necessary for responding based on inferred but not cached value in a sensory
preconditioning task, supporting the view of the OFC as critical to integration.
Inactivation of the OFC during the probe test with baclofen and muscimol revealed that
rats were unable to infer value to the preconditioned cue that had previously been paired
with the reinforced cue, while the rats still responded to the cue that had been paired
directly with the reward. This experiment confirmed much of what we know about the
OFC, although sensory preconditioning was a novel setting that was able to combine
findings into one setting: its role in predicting specific outcomes and its role in learning.
Sensory preconditioning allows us to demonstrate the role of the OFC in using model-
based representations for behavior and learning independent of reversal or outcome
devaluation. The Jones et al. (2012) sensory preconditioning study demonstrated that the
OFC is involved in this ability to infer value, but in order to understand what the OFC is
actually doing neurophysiologically, we can look more specifically at what the OFC is
signaling during performance in the task. Based on these results, and the previous
findings that OFC neural activity correlated with predicting outcomes, it is hypothesized
that the OFC is signaling the inferred value in sensory preconditioning, even though the
preconditioned cue has never been paired directly with the reward.
2. Materials and Methods
A. Subjects
Twenty-nine male Long-Evans rats were obtained at 250-300g from Charles
River Labs, Wilmington, MA. Upon arrival, they were housed individually and were
44
given ad libitum access to food and water, except during behavioral training and testing.
During sensory preconditioning, rats were given 10 minutes of water each day after
behavior sessions. Rats were maintained on a 12 hour light/dark cycle and were trained
and tested during the light cycle. All behavior and recording were conducted at National
Institute on Drug Abuse, Intramural Research Program, in accordance with University of
Maryland School of Medicine and National Institutes of Health (NIH) guidelines.
B. Apparatus
Behavioral training and testing were conducted in four aluminum boxes
measuring 18" x 18" with sloping walls on the front and back of the box that narrowed to
12" x 12" at the bottom. The left and right walls were straight. In the center of either the
left or right wall a recessed dipper was placed approximately 2 cm above the floor grid.
The sucrose dipper delivered 0.04 mL of a 10% sucrose solution. Auditory cues were
used during the behavioral training and testing. The tone speaker was located on the wall
above the dipper and calibrated to ~70 dB. During Run 1, the stimuli included a tone,
siren, clicker and white noise. The rats were exposed to the tone cues produced by an
audio generator that played a pure tone frequency and a siren tone that alternated between
two frequencies every 100 ms. The clicker (2 Hz) was mounted on the wall above the
dipper, and a white noise speaker calibrated to ~65 dB was placed external to the
recording box. During Run 2, four distinct customized auditory cues were played for the
additional round of sensory preconditioning using an Arduino Uno USB Board
(Smartprojects, Italy).
45
C. Surgical Procedures
Using aseptic techniques, drivable bundles of sixteen 25-µm diameter Formvar-
insulated nichrome wires (80% nickel/20% chromium) (A-M Systems, Sequim, WA),
were manufactured and implanted as in prior recording experiments (Roesch et al.,
2007a). Electrodes were implanted in the left (n=16), right (n=8) or bilateral (n=5) OFC:
3.0 mm anterior to bregma, 3.2 mm laterally, and 4.0 mm ventral to the surface of the
brain in each rat.
D. Histology
At the end of the study, the rats were euthanized with an overdose of isoflurane.
The final electrode position was marked by the passage of current through each
microwire. The brains were then perfused with 0.9% saline and removed from the skulls,
fixed with 10% formalin and transferred to a 30% sucrose 0.1PBS at 4ºC. Brains were
then cut 50 µm, counterstained and electrode placements confirmed.
E. Behavioral Task
i) Sensory Preconditioning:
The sensory preconditioning procedure consisted of three phases (Figure II.1) , using
procedures similar to those in our prior studies (Jones et al., 2012):
46
(a) Preconditioning:
Rats were shaped to retrieve 10% liquid sucrose from the dipper (n = 28 total
subjects). After shaping, all rats received two days of sensory preconditioning involving
auditory cues. Each day, rats were exposed to two blocks of six trials. A block of trials
consisted of presentations of paired auditory cues (A→B and C→D). Specifically, a 10 s
presentation of cue A would be followed immediately by a 10 s presentation of cue B, in
a six-trial block. The second block would consist of six 10 s presentations of cue C, each
followed immediately by a 10 s presentation of cue D. The inter-trial intervals varied
from 3-6 minutes. In Run 1, auditory cues A and C were either white noise or clicker
Figure II.1 Sensory preconditioning.
Figures show the percentage of time spent in the food during presentation of the cues
during each of the three phases: preconditioning (A), conditioning (B), and the probe
test (C). The experimental timeline is schematized at the bottom of the figure.
*p<0.05. Error bars = SEM.
47
counterbalanced, and auditory cues B and D were either siren or tone counterbalanced. In
Run 2, all four customary auditory cues were counterbalanced for cues A, B, C and D.
The order of cue blocks was counterbalanced across each of the two sensory
preconditioning days.
(b) Conditioning:
After two days of preconditioning, rats underwent Pavlovian conditioning for six
consecutive days. Each day rats received a single training session in which a total of 12
trials were presented: six trials of cue B paired with a 10% liquid sucrose reward and six
trials of cue D paired with no sucrose reward. Auditory cue B was paired with three rises
of the sucrose dipper during the 10 s presentation (at 2.9, 6.4 and 9.4 s with each dipper
completing its lift in 0.6 s). Alternatively, auditory cue D was presented alone for 10 s
with no sucrose reward delivered. Trials were counterbalanced including three trials of
alternating blocks for a total of six reinforced auditory cue B presentations and six
unreinforced auditory cue D presentations. The inter-trial intervals varied between 3-6
minutes.
(c) Probe Test:
After conditioning, a single probe test day included reminder training of three
trials of B paired with sucrose and three trials of D unpaired. After the presentations of B
and D, rats were exposed to counterbalanced auditory cues A and C presented in
extinction (six presentations each). The inter-trial intervals varied between 3-6 minutes.
48
(d) Response measures:
During sensory preconditioning, the percentage of time the rats spent with their
head in the sucrose well during cue presentation was measured by an infrared photo beam
positioned at the front of the dipper. The analysis for auditory cues presented was for the
10 s of cue presentation. Baseline response measures in the corresponding pre-cue
periods were subtracted from the cue-evoked response to account for individual variation
in baseline activity. For probe tests, only the first two trials of the preconditioned cues
were analyzed for behavior.
(e) Statistical Analyses:
Data were acquired using Plexon Multichannel Acquisition Processor systems
(Dallas, TX). Raw data were processed in Matlab (Natick, MA) to extract number of
nosepokes and percentage time spent in the sucrose well for sensory preconditioning. For
all statistical tests, an alpha level of 0.05 was used. All behavioral data were analyzed in
Statistica, using repeated measures analysis of variance with Bonferroni post-hoc testing
when appropriate (p < 0.05).
F. Single-Unit Recording
Neural activity was recorded using four identical Plexon Multichannel
Acquisition Processor Systems (Dallas, TX) attached to the behavior chambers described
above. Signals from the electrode wires were amplified and filtered as described in
Roesch et al. (2007a). Waveforms with greater than a 2.5:1 signal to noise ratio were
extracted from active channels with event timestamps and were not inverted before data
analysis.
49
Wires were screened for activity daily; if no activity was detected, the rat was run
in the behavioral task and after the session the electrode assembly was advanced 40 to 80
µm.
i) Statistical Analyses
Units were sorted using Offline Sorter software from Plexon Inc. (Dallas, TX).
Sorted files were then processed and analyzed in Matlab to extract unit timestamps and
relevant event markers.
3. Results
A. Sensory Preconditioning Behavioral Results:
We recorded a total of 43 rounds of training and probe testing in the rats. Twenty-
eight rounds of the training and probe tests were from Run 1 with the auditory stimuli
that was used in previous experiments (Wied et al., 2013, Jones et al., 2012).
Additionally, fifteen rats that still had good neural data were trained and tested again with
a new set of auditory cues in Run 2.
i) Preconditioning:
In preconditioning, all rats were taught to associate two pairs of neutral auditory
cues (A→B and C→D). Food cup responding and number of nosepokes during the
presentation of each cue was taken as an index of conditioning. Rats responded at or near
baseline levels to all cues (Figure II.1A and Figure II.2A). A one-way ANOVA
comparing the percent of time the rats spent in the food cup to each cue demonstrated no
significant effects (F(3,126) = 1.02; p = 0.39). Similarly, a one-way ANOVA comparing the
50
number of nosepokes into the food cup demonstrated to each cue revealed no significant
effects (F(3,126) = 1.38; p = 0.25).
ii) Conditioning:
Over the six days of conditioning, all rats learned to discriminate between the
rewarded auditory cue B and the non-rewarded auditory cue D (Figure II.1B). Rats
increased responding to B, but not to D, during these six conditioning sessions. A 2-
factor ANOVA (cue x session) revealed significant main effects of cue (F(1,280) = 192.12,
p < 0.0001) and session (F(5,280) = 15.54, p = 0.0001), and a significant interaction
between cue and session (F(5,280) = 17.13, p < 0.0001). As seen in the control animals
previously (Wied et al., 2013) the rats acquired a discrimination between B and D
responding in session 1. Bonferroni post-hoc testing demonstrated that rats responded
significantly more to B than D in all sessions (ps < 0.0001) and responded significantly
more to B in sessions 3-6 than in session 1 (ps < 0.05). This increase in responding and
discrimination appears to be due to the prior experience with reward retrieval, as the
recording animals had a longer history of shaping than in Jones et al. (2012).
iii) Probe Test:
In the probe test, all rats responded more to B, the cue paired with sucrose, than to
any other auditory cue including the non-reinforced cue (D) and the preconditioned cues
(A, C). (Figure II.1C).
A two-factor ANOVA (reward predictive x primary versus preconditioned cues)
analyzing responding indicated a significant main effect of reward predictive cues (F(1,84)
= 190.65, p < 0.0001), a significant main effect of primary versus preconditioned cues
51
(F(1,84) = 215.19, p < 0.0001), and a significant interaction between reward predictive cues
and primary versus preconditioned cues (F(1,84) = 185.67, p < 0.0001). Bonferroni post-
hoc testing showed that rats responded significantly more to B than A, C, and D (p<
0.0001). Additionally, there was not a significant difference in the number of nosepokes
to A and C; t(42) = 0.86, p = 0.40 (Figure II.2B).
B. Sensory Preconditioning in Probe Tests that demonstrated A>C Behavior
As clearly shown in Figure II.1 and Figure II.2, there was no A greater than C
behavioral effect overall in the probe test as seen in previous studies. As stated from the
hypothesis, the central question is what neurons signal when sensory preconditioning is
successful. In order to represent an analysis of successful data, we examined only the
animals and sessions that demonstrated greater responding to A than to C in the probe
test (Figure II.3). Of the 43 rounds of training and probe testing we only saw 26 probe
tests that demonstrated greater responding to cue A than to cue C which was significantly
Figure II.2 Sensory preconditioning: Number of nosepokes during
preconditioning and probe test.
Figures show the number of nosepokes during presentation of the cues during
preconditioning (A) and the probe test (B). Error bars = SEM.
52
less than expected (χ2
1,N=43 = 4.85, p = 0.03) (Figure II.3). These sessions did not differ in
their behavior on a two-factor ANOVA (cue x group) in preconditioning in percent
responding (F1,201 = 0.15, p = 0.70) or nosepokes (F1,201 = 0.15, p = 0.70). These
sessions also did not differ in a three-factor ANOVA (cue x group x session) for
conditioning days (F1,430 = 0.02, p = 0.88 ) when compared to all 43 probe tests. The only
difference was their behavior on probe test (Figure II.4). The following data were from
22 rats, 15 sessions from Run 1 and 11 sessions from Run 2.
i) Preconditioning:
In preconditioning, rats responded at or near baseline levels to all cues (Figure
II.4A and Figure II.5B). A one-way ANOVA demonstrated no significant effects when
Figure II.3 Distribution of sessions that demonstrated greater responding to cue
A than to cue C.
Twenty-six out of 43 sessions demonstrated rats nosepokes to cue A, the cue paired
with the reinforced cue (B), more than to cue C, the cue paired with the non-reinforced
cue (D), on the first two trials of the probe test.
53
comparing the percent of time the rats spent in the dipper (F(3,75) = 1.83; p = 0.15) or
when comparing the number of nosepokes into the dipper (F(3,75) = 1.27; p = 0.29) to each
cue.
ii) Conditioning:
Over the six days of conditioning, rats learned to discriminate between the
rewarded auditory cue B and the non-rewarded auditory cue D (Figure II.1). Rats
increased responding to B, but not to D during these six conditioning sessions. A 2-factor
ANOVA (cue x session) revealed a significant main effect of cue (F(1,150) = 187.16, p <
Figure II.4 Sensory preconditioning behavior results from the sessions that
demonstrated A greater than C responding.
Figures show the percentage of time spent in the food cup during presentation of the
cues during each of the three phases from the 26 sessions that demonstrated A greater
than C behavior during probe test: preconditioning (A), conditioning (B), and probe
test (C). The experimental timeline is schematized at the bottom of the figure.
*p<0.05. Error bars = SEM.
54
0.0001) and a significant interaction between cue and session (F(5,150) = 2.60, p = 0.028).
Bonferroni post-hoc testing demonstrated that rats responded significantly more to B than
D in all sessions (ps < 0.0001), and responded significantly more to B in session 6 than in
session 1 (p < 0.05).
iii) Probe Test:
In the probe test, rats responded more to B, the cue paired with sucrose, than to D,
the non-reinforced cue (Figure II.4C and Figure II.5B). Rats exhibited significantly
greater responding within the average of the first two trials to the preconditioned cue (A),
that had been previously paired with the rewarded cue (B), than to the preconditioned
control cue (C), that had been previously paired with the non-reinforced cue (D).
A two-factor ANOVA (reward predictive x primary versus preconditioned cues)
analyzing responding indicated a significant main effect of reward predictive cues (F(1,50)
= 190.92, p < 0.0001), a significant main effect of primary versus preconditioned cues
(F(1,50) = 64.82, p < 0.0001), and a significant interaction between reward predictive cues
and primary versus preconditioned cues (F(1,50) = 94.17, p < 0.0001). Bonferroni post-hoc
testing showed that rats responded significantly more to B than D (p< 0.0001) and
significantly more to A than to either C or D (p’s < 0.05). Similarly, there was a
significant difference in the number of nosepokes to A and C; t(25) = 4.27, p = 0.0002.
These results demonstrate greater response to A, the cue paired previously with the
reinforced cue (B), than to C, the cue previously paired with the non-reinforced cue (D).
55
C. Neural Probe Test Results:
The results to be presented below came from the 26 sessions in which we
observed evidence of rats responding to A more than C during probe test. The neural data
were analyzed comparing the 10 s auditory cues.
i) Preconditioning:
On the first day of preconditioning, we recorded 193 neurons. Figure II.7A and
Figure II.7C show that 46 of these neurons exhibited either a significant excitatory phasic
response (n = 32) and/or inhibitory phasic response (n = 15) over baseline to at least one
of the auditory cues. Additionally, there was a strong correlation demonstrating that A-C
firing significantly correlated to B-D firing (r = 0.88, p < 0.0001). Similar firing patterns
are seen in paired auditory cues A and B as well as to C and D (Figure II.6D).
Figure II.5 Sensory preconditioning: Number of nosepokes during
preconditioning and probe test from sessions that demonstrated A greater than C
responding.
Figures show the number of nosepokes during presentation of the cues during
preconditioning (A) and the probe test (B) from the 26 sessions that demonstrated A
greater than C behavior during probe test. Error bars = SEM.
56
ii) Probe Test:
We recorded 225 neurons in the probe test in animals that demonstrated
numerically greater responding to the inferred value cue A than to the control cue C.
Figure II.7A shows that 64 of these neurons exhibited either a significant excitatory
Figure II.6 Proportion of neurons with significant cue-selective activity during
the preconditioning.
A: Proportion of units that were significantly responsive to one of the four cues in
preconditioning by either increasing or decreasing firing rate compared to baseline. B:
Index summary of all excitatory (●) and inhibitory (◌ ) units recorded during
preconditioning. A significant correlation demonstrated A-C firing predicted B-D
firing ( r = 0.88, p < 0.0001). C: Proportion of cue-responsive units that increased
firing rate compared to baseline during preconditioning to auditory cues A,B,C and/or
D. D. Population responses of all 32 cue-responsive neurons to each set of 10 s cues:
A→B cues (in black, with blue and black bars indicating the period of each cue
presentation), and C→D (in gray, with red and gray bars indicating the period of each
cue presentation). Gray shading represents SEM.
57
phasic response (n = 44) and/or inhibitory phasic response (n = 22) over baseline to at
least one of the cues. As seen in Figure II.7C, the majority of the cue-evoked neurons
fired to B paired with reward (n = 27), while cue D that had been paired with no reward
had the least number of cue-responsive neurons (n = 6). Preconditioned cues A and C had
equal numbers of neurons represented (n's = 8). As seen in Figure II.7B, a significant
correlation demonstrated that B-D firing predicted A-C firing ( r = 0.25, p = 0.044).
Although the proportion of cue-responsive units that fired to the various auditory cues
changed over conditioning, we see the same correlation as we did on the first day of
preconditioning. As B became predictive of reward and D of non-reward, the relationship
between cues A and B, as well as C and D did not fade from the original significant
correlation seen in Figure II.6B.
58
The distributions of index scores for mean firing to A-C and B-D in the probe test
are seen in Figure II.8A and Figure II.8C. The neurons that fired significantly different
from baseline, shown in black, demonstrated a significant shift from zero for B-D (p <
0.05), but not for cues A-C. These differences were further demonstrated when
population responses to all 44 cue-responsive neurons were plotted (Figure II.8B and
Figure II.8D), and a paired t-test comparing cues B and D demonstrated a significant
Figure II.7 Proportion of neurons with significant cue-selective activity during
the 10 second auditory cues during probe test.
A: Proportion of units that were significantly responsive to one of the four cues by
either increasing or decreasing firing rate compared to baseline during probe test
during the 10 s auditory cues. B: Index summary of all excitatory (●) and inhibitory
(◌ ) units recorded during probe test. A significant correlation demonstrated B-D
firing predicted A-C firing ( r = 0.25, p < 0.05). C: Proportion of cue-responsive units
that increased firing rate compared to baseline to auditory cues A,B,C and/or D during
probe test.
59
difference in excitatory mean firing rates (t31 = 4.02, p < 0.001), but no significant
difference between A and C (t13 = 0.95, p = 0.36).
Figure II.8 Cue-evoked activity during probe test.
A,C: The distribution of index scores for mean firing to A-C and B-D during the
probe test. The black bars represent excitatory neurons in which the difference in
firing to cues was significantly different from baseline. The distribution of index
scores did not shift from zero for A-C (p> 0.05) (fig. A), but did significantly shift
above zero for B-D (p < 0.05) (fig. C) . B,D: Population responses of all 44 cue-
responsive neurons to A and C (fig. B), and B and D (fig. D). There were no
significant differences in mean firing to A versus C, but there were significant
differences in mean firing rates to B versus D (t31 = 4.02, p < 0.001). Light shadowing
in blue, red, gray and black represent SEM.
60
4. Discussion
In this study, we were able to analyze sessions that responded more to the inferred
value cue A that had been preconditioned with the rewarded cue B than to cue C that had
been preconditioned with the non-rewarded cue D. Throughout training and testing there
was an increase in the number and fraction of neurons responding to B from the first day
of preconditioning to the probe test. On probe test, there were significantly more neurons
with greater mean firing rates to the reinforced cue B than to the non-reinforced cue D.
Additionally, we saw a significant correlation that demonstrated B firing correlated with
A firing, and D firing correlated with C firing on probe test. This was the same
correlation that was observed on the first day of preconditioning. Although it is clear
from these scatter plots and correlations that A-B and C-D firing rates separate with
learning, it appears these relationships do not go away completely. In fact, the significant
correlation on probe test suggests there is still a strong enough relationship between these
cues to make an inference.
However, it cannot be ruled out that due to the lack of overall A greater than C
behavior, and the lack of a significant difference seen between A and C mean firing, that
the OFC does not signal this inferred value. From previous experiments, we know that
OFC is somehow involved in inferred value and, therefore, instead of signaling this
inferred value, perhaps the OFC is critical because it signals the meaning or value of B.
We also cannot rule out that the apparent sessions that we analyzed showing A greater
than C behavior were essentially random, as this lack of responding was statistically
different from what was expected from previous experiments. The behavioral results in
this study are different from our previous studies in which we saw very few failed
61
sessions. In the few sessions that did fail in previous studies, typically the animals’
behavior was at baseline, and not the C greater than A behavior seen in many of our
failed sessions in this experiment. Therefore, even in the sessions we analyzed, we are
not able to conclusively state that we are analyzing actual conditioned responding versus
just random behavior. If we are observing just random behavior, then failure to see
encoding in OFC is exactly what we would have predicted.
There are a number of differences in this recording experiment in comparison to
our previous experiments in which the majority of rats demonstrated the ability to infer
value. This was the first time this behavior was run in recording rigs rather than behavior
rigs. The recording rigs are a different size and require the animals to behave with a cable
attached to their heads for all conditions of the training and testing. Additionally, due to
the cable attached to their head, typically the rats needed more shaping in order to
retrieve the reward. Moreover, the reinforcer was a liquid sucrose reward delivered by a
dipper. This differed from previous successful sensory preconditioning studies that used a
food reward (Wied et al., 2013, Jones et al., 2012). The change was made for recording in
order to eliminate the artifact caused by the chewing of the sucrose pellets. Furthermore,
this was the first time that we ran a second round of behavior with new auditory cues in
order to record more neurons.
Interestingly, there is promising data that demonstrate that B-D firing correlates
with A-C firing even though we do not see a significant difference in the mean firing
rates between the inferred value cue and the cue that predicted the non-reinforced cue.
This may be due to the limited number of neurons that responded to these cues. To more
clearly determine if the OFC is signaling these inferred values, or if the OFC is signaling
62
the meaning or value of B, it will be necessary to run more rats in a similar task as above,
perhaps using sucrose reward pellets instead of a liquid sucrose reward to eliminate one
variable that was changed from the original successful behavioral studies. Although
chewing artifact may be seen, the most important aspect is that the rats must demonstrate
the behavior robustly in order to make any definitive conclusions about what the OFC is
signaling. These experiments are currently being conducted in the Schoenbaum lab and
we hope to clarify the role of the OFC neurophysiologically in its ability to infer value.
63
III. Disruption of model-based behavior and learning by cocaine
self-administration in rats1
1. Abstract
Rationale: Addiction is characterized by maladaptive decision-making, in which
individuals seem unable to use adverse outcomes to modify their behavior. Adverse
outcomes are often infrequent, delayed, and even rare events, especially when compared
to the reliable rewarding drug-associated outcomes. As a result, recognizing and using
information about their occurrence puts a premium on the operation of so-called model
based systems of behavioral control, which allow one to mentally simulate outcomes of
different courses of action based on a knowledge of the underlying associative structure
of the environment. This suggests that addiction may reflect, in part, drug-induced
dysfunction in these systems. Here we tested this hypothesis.
Objectives: To test whether cocaine causes deficits in model-based behavior and
learning independent of requirements for response inhibition or perception of costs or
punishment.
Methods: We trained rats to self-administer sucrose or cocaine for 2 weeks. Four
weeks later, the rats began training on a sensory preconditioning and inferred value
blocking task. Like devaluation, normal performance on this task requires representations
of the underlying task structure; however, unlike devaluation, it does not require either
response inhibition or adapting behavior to reflect aversive outcomes.
Results: Rats trained to self-administer cocaine failed to show conditioned
responding or blocking to the preconditioned cue. These deficits were not observed in
1 Heather M. Wied, Joshua L. Jones, Nisha K. Cooch, Benjamin A. Berg, and Geoffrey Schoenbaum.
As published in Psychopharmacology, August 16, 2013.
64
sucrose-trained rats nor did they reflect any changes in responding to cues paired directly
with reward.
Conclusions: These results show that cocaine disrupts the operation of neural
circuits that mediate model-based behavioral control.
Keywords: addiction, cocaine, orbitofrontal, sensory preconditioning, blocking
2. Introduction
Addiction is characterized by a pattern of maladaptive decision-making in which
subjects continue to seek drugs despite serious adverse consequences (Gawin, 1991,
Mendelson and Mello, 1996). This pattern of behavior is captured in the diagnostic
criteria for substance dependence in the DSM-IV-TR (APA, 2000), which require drug-
seeking to be persistent in the face of attempts to stop or cut back in order to avoid
various negative outcomes or costs. While there are a number of ways to describe such
apparent deficits – inability to inhibit established behaviors (Jentsch and Taylor, 1999,
Winstanley et al., 2010), increased desire or wanting of the drug (Robinson and Berridge,
2008, Robinson and Berridge, 2000), loss of sensitivity to punishment or costs
(Vanderschuren and Everitt, 2004, Deroche-Gamonet et al., 2004) or disruption of the
normal homeostatic set point of reward and anti-reward systems (Koob and Le Moal,
2008) – none of these sufficiently captures the complex array of the behavioral deficits.
Addicted individuals clearly can act affirmatively to obtain desired outcomes and can
even learn relatively complex strategies or policies in order to obtain drugs. They can
also redirect behavior when confronted with immediate consequences or costs. Yet,
65
despite these apparently intact abilities, they are unable to modify drug seeking in
response to real world events that most people would respond to appropriately.
One distinguishing feature of these real world events – one which is typically not
captured by experimental manipulations – is that the certain consequences of drug use
tend to be highly variable and unpredictable, delayed, and even rare events. Often, these
consequences are adverse, such as financial and family problems, physical injury, or
incarceration. All of these are all highly likely over the long-term; however, they tend to
be unlikely in the very short term, at least as compared to the immediate goal and certain
experience associated with the drug-associated cues and drug high, or alleviation of
withdrawal symptoms. Further, these long-term events are unlikely to happen over and
over again, so it is difficult to learn about them in the same way that one can learn about
the repetitive, immediate rewarding aspects of drug use. As a result, the successful
application of information about these consequences likely requires so-called model-
based systems of behavioral control, which allow one to utilize information about the
causal structure of the environment, which may be only anecdotally or incidentally
related to the behavior at hand, in order to mentally simulate the possible outcomes of a
behavior (Daw et al., 2005, Niv et al., 2006). Failure of these systems would result in
behavior – and learning - that is heavily dependent on cached or pre-computed policies
developed through prior direct experience. To that end, learning and behavior after
cocaine exposure appear normal or even enhanced under conditions in which model-free
systems are sufficient (Wyvell and Berridge, 2001, Taylor and Horger, 1999, Taylor and
Jentsch, 2001, Roesch et al., 2007b, Simon et al., 2007, Mendez et al., 2010).
66
Thus, the core behavioral features of addiction may reflect, in part, either a pre-
existing or drug-induced dysfunction in the model-based control of behavior (Lucantonio
et al., 2012). Consistent with this idea, studies using reinforcer devaluation have shown
that exposure to psychostimulants, either passively or by active self-administration,
causes long-lasting deficits in the ability of rats to adjust learned behaviors after changes
in the value of the associated reward (Nelson and Killcross, 2006, Schoenbaum and
Setlow, 2005). These deficits are observed spontaneously after devaluation (i.e. without
new learning) and appear in the face of apparently normal acquisition of both the
behavior (CS→US or R→US) and of the new reward value (US→illness).
While these studies are consistent with the idea that some drugs affect model-
based processing, they are similar and thus provide only a single data point. Further, they
require the animal to appreciate an adverse outcome in order to redirect or withhold
responding. This leaves open, at least somewhat, questions over whether the deficits
reflect problems with specific functions, such as sensitivity to punishment or the ability to
inhibit responding, rather than a more general impairment in the integrity of model-based
processing, which would be orthogonal to these idiosyncratic task requirements. The case
that drugs can disrupt model-based processing would be much strengthened by the
provision of convergent evidence from a task that shares the requirement for inferential
reasoning that defines reinforcer devaluation, but does not depend on either of these more
specific capabilities. Here we provide such evidence, using a sensory preconditioning and
inferred value blocking task in which we have previously demonstrated the role of the
orbitofrontal cortex in model based behavior and learning (Jones et al., 2012).
67
3. Materials and Methods
A. Subjects:
Forty-five adult male Long-Evans rats (Charles River Laboratories, Wilmington,
MA) (n = 30 controls including 13 naïve, non-surgical controls and 17 sucrose-trained
surgical controls; n = 15 cocaine-trained) weighing 275-325 g on arrival were
individually housed and given ad libitum access to food and water, except during
behavioral training and testing. During these procedures, rats were food deprived to 85-
90% of their baseline body weight. Rats were maintained on a 12-hour light/dark cycle
and trained and tested during the light cycle. Experiments were performed at the
University of Maryland School of Medicine and National Institute on Drug Abuse,
Intramural Research Program, in accordance with University of Maryland and NIH
guidelines.
B. Apparatus:
Self-administration and behavioral training and testing were conducted in
standard behavior boxes measuring 30.48 cm x 25.40 cm x 30.48 cm (Coulbourn
Instruments) enclosed within sound-attenuating boxes. During self-administration, each
chamber was equipped with two 4-cm wide levers on opposite walls located 8 cm above
the grid floor. The lever located on the right side of the chamber was active and
retractable and led to activation of the infusion pump, while the lever on the left side of
the box was inactive and stationary. For cocaine self-administration, each rat had silastic
tubing that was protected by a metal spring that extended from the intravenous catheter to
a liquid swivel (Instech Laboratories, Plymouth Meeting, PA) mounted to an arm fixed
68
outside the behavior chamber. Tygon tubing extended from the liquid swivel to an
infusion pump located outside the chamber. For sucrose self-administration, 0.04 mL of a
10% sucrose solution was placed in a recessed dipper in the center of the right wall
approximately 2 cm above the floor grid.
Sensory preconditioning and blocking were conducted in a different room, but
using the same type of standard apparatus in sound attenuating boxes. A recessed food
cup was placed in the center of the right wall approximately 2 cm above the floor. The
feeder was mounted outside the behavior chamber and delivered 45 mg sucrose pellets
into the food cup. Auditory cues were used during the behavioral training and testing.
The tone speaker was located on the wall above the food cup and calibrated to ~70 db.
The tone cues were produced by an audio generator that produced three tones, one as a
pure tone frequency, and a siren tone that alternated between two frequencies every 100
ms. In addition, a clicker (2 Hz) was mounted on the wall above the food cup, and a
white noise speaker calibrated to ~65 db was placed external to the behavior box behind
the right wall. Visual cues that were used during inferred value blocking included a house
light and cue light that were placed on the wall to the left or the right of the food cup
approximately 10 cm above the floor.
C. Surgical procedures:
Thirty-two rats underwent surgery for implantation of chronic intravenous jugular
catheters. Rats were anaesthetized with ketamine (100 mg/ kg, i.p., Sigma) and xylazine
(10 mg/kg, i.p., Sigma). A silastic catheter was inserted into the right jugular vein and
passed subcutaneously to either the top of the skull where it was attached to a modified
22-gauge cannula (Plastics One, Roanoke, VA) and mounted to the rat’s skull with dental
69
cement and skull screws or fixed to a cannula on nylon mesh which was sutured
subcutaneously between the scapulae. Carprofen (5 mg/kg, s.c., Pfizer) was given during
surgery and for two days post-surgery to relieve post-operative pain. After surgery, and
throughout self-administration training, catheters were flushed daily with
gentamicin/saline solution (0.16% gentamicin, BioSource International) to prevent
infection.
D. Self-Administration:
Rats recovered for 14 days following surgery before starting self-administration
training. These rats were trained to lever press for cocaine-HCl (0.75 mg/kg/infusion; n =
15) or sucrose (10% w/v; n = 17) for 14 days (Figure III.1). Rats were trained to press the
active lever to self-administer cocaine under a fixed ratio (FR1) schedule of
reinforcement with each press delivering a 4 s infusion of cocaine. Each infusion was
followed by a 40 s timeout period. All self-administration sessions lasted 3 h, with 15-
min timeout periods after each hour. To prevent overdose, the number of cocaine
infusions was limited to 20 per hour such that when the rat had received 20 cocaine
infusions the active lever was retracted for the remainder of the hour. The same
procedures were followed for animals that self-administered sucrose, except the rats were
shaped to press the active lever to receive 0.04 mL of a 10% sucrose solution delivered in
a recessed dipper.
E. Sensory Preconditioning:
Catheterized rats (n=32) began sensory preconditioning four weeks after the end
of self-administration. At the same time, we also trained naïve rats (n=13) who had
received neither surgery nor any prior training. The sensory preconditioning procedure
70
consisted of three phases (Figure III.2), using procedures identical to those in our prior
study (Jones et al., 2012):
i) Preconditioning:
Rats were shaped to retrieve sucrose pellets from the food cup in a single session
that included 10 deliveries of two 45 mg sucrose pellets (n = 45 total subjects; n = 30
controls; n = 15 cocaine). After shaping, all rats received two days of sensory
preconditioning involving auditory cues. Each day, rats were exposed to two blocks of
six trials. A block of trials consisted of presentations of paired auditory cues (A-B and C-
D). Specifically, a 10 s presentation of cue A would be followed immediately by a 10 s
presentation of cue B, in a six-trial block. The second block would consist of six
presentations of the pairing a 10 s presentation of cue C followed immediately by a 10 s
presentation of cue D. The inter-trial intervals varied from 3-6 minutes. Auditory cues A
and C were either white noise or clicker counterbalanced, and auditory cues B and D
were either siren or tone counterbalanced. The order of cue blocks was counter-balanced
across each of the two sensory preconditioning days.
ii) Conditioning:
After two days of preconditioning, rats underwent Pavlovian conditioning for six
consecutive days. Each day rats received a single training session in which a total of 12
trials were presented: six trials of cue B paired with a food reward and six trials of D
paired with no food reward. Auditory cue B was paired with three sucrose pellets that
were delivered into the food cup during the 10 s presentation (one pellet was delivered
after 3, 3 and 4 s time elapsed). Alternatively, auditory cue D was presented alone for 10
s with no sucrose delivered. Trials were counterbalanced including three trials of
71
alternating blocks for a total of six reinforced auditory cue B presentations and six
unreinforced auditory cue D presentations. The inter-trial intervals varied between 3-6
minutes.
iii) Probe Test:
After conditioning, a single probe test day included reminder training of three
trials of B paired with sucrose and three trials of D unpaired. After the presentations of B
and D, rats were exposed to two blocks of counterbalanced auditory cues A and C
presented in extinction (six presentations each). The inter-trial intervals varied between 3-
6 minutes.
F. Inferred Value Blocking:
This procedure consisted of two training phases (Figure III.3) using procedures
identical to those in our prior study (Jones et al., 2012):
i) Blocking:
A subset of rats from the prior experiment (n = 35; n = 22 controls; n = 13
cocaine) underwent additional compound conditioning to test whether cocaine would also
disrupt the normal ability of the inferred value to block new learning. These rats were
trained for two days with preconditioned auditory cues A and C paired with novel light
cues X and Y (houselight; flashing cue light; counterbalanced), with all presentations
reinforced with a food reward. Auditory cues A and C were played together in compound
with visual cues X and Y (i.e. AX and CY) for 10 s each. Each compound cue was
presented in two blocks of three trials, alternating cue blocks, for a total of 12 trials each
day (6 AX and 6 CY, counterbalanced). Each 10 s compound cue was reinforced by the
72
delivery of a sucrose pellet at 3, 6, and 10 s after cue onset. Inter-trial intervals were
between 3-6 minutes.
ii) Probe Test:
After compound conditioning, rats were given a single probe test consisting of
reminder training of three trials each of AX and CY paired with normal sucrose delivery,
followed by two blocks of visual cues X and Y presented alone (four presentations each)
for 10 s each without sucrose delivery. The inter-trial intervals varied between 3-6
minutes. The first three trial presentations of X and Y were analyzed.
G. Response measures:
During self-administration, active and inactive lever pressing was recorded.
During sensory preconditioning, the percentage of time rats spent with their head in the
food cup during cue presentation was measured by an infrared photo beam positioned at
the front of the food cup. The analysis for auditory and visual cues presented alone was
restricted to the last 5 s of cue presentation. Baseline response measures in the
corresponding pre-cue periods were subtracted from the cue-evoked response to account
for individual variation in baseline activity. Rearing behavior to visual stimuli during
inferred value blocking in both the compound conditioning sessions and blocking probe
test was scored from video recordings with a handheld timer. In order to correct for time
spent rearing (Takahashi et al. 2009; Jones et al. 2012), the percentage of time responding
during the visual stimuli, and corresponding baseline periods, was calculated as follows:
% of responding = 100* ([% of time in food cup]/[100 - % of time rearing]).
73
H. Statistical Analyses:
Data were acquired using Coulbourn Graphic State 2 software (Coulbourn
Instruments). Raw data were processed in Matlab to extract lever presses in self-
administration and percentage time spent in the food cup for sensory preconditioning and
inferred value blocking. For all statistical tests, an alpha level of 0.05 was used. The
naïve and surgical control groups did not differ at the end of conditioning or during probe
testing, thus their data were collapsed to form a single control group. All data were
analyzed in Statistica, using repeated measures analysis of variance with Bonferroni post-
hoc testing when appropriate (p < 0.05).
4. Results
A. Self-Administration:
Chronic intravenous jugular catheters were implanted in rats (17 sucrose controls
and 15 cocaine). After recovering from surgery, these rats self-administered either
cocaine or sucrose for 14 days (Figure III.1). Rats acquired cocaine and sucrose self-
administration as demonstrated by a significant interaction between lever and day in the
analysis of active and inactive lever presses for cocaine (F(13,338) = 2.01, p < 0.05) and
sucrose (F(13,403) = 5.99, p < 0.01). In addition, there was a significant effect of training
day for the number of infusions of cocaine (F(13,182) = 15.96, p < 0.01) and number of
sucrose deliveries (F(13,208) = 5.24 , p < 0.01).
74
B. Sensory Preconditioning:
i) Preconditioning:
In preconditioning, rats were taught to associate two pairs of neutral auditory cues
(A→ B and C→ D). Food cup responding during the presentation of each cue was taken
as an index of conditioning. Rats in both groups responded at or near baseline levels to all
cues (Figure III.2. A and B). A 2-factor ANOVA (cue x treatment) comparing the percent
of time the rats spent in the food cup demonstrated no significant main effects or any
interactions (F's < 2.08; p's > 0.16) (Figure III.2A and B).
Figure III.1 Self-administration.
Rats were trained to press the active lever to self-administer a 10% liquid sucrose solution or
cocaine-HCl (0.75 mg/kg/infusion). Figures show the mean number of lever presses on the
active and passive levers for sucrose controls (A) and cocaine rats (B) over 14 days. The
mean number of sucrose or cocaine reward deliveries during each session is also shown. Error
bars = SEM.
* Source of image (Wied et al., 2013)
75
ii) Conditioning:
During the six days of conditioning, the rats learned to discriminate between the
rewarded auditory cue B and the non-rewarded auditory cue D (Figure III.2C and D).
Rats in both groups increased responding to B, but not to D, during these six conditioning
sessions. A 3-factor ANOVA (cue x treatment x session) revealed significant main
effects of cue (F(1,86) = 221.99, p < 0.01) and session (F(5,430) = 27.62, p < 0.01) and
significant interactions between treatment and cue (F(1,86) = 9.88, p < 0.01), treatment
and session (F(5,430) = 2.34, p < 0.05), cue and session (F(5,430) = 13.01, p < 0.01), and cue,
treatment and session (F(5,430) = 2.38, p < 0.05). Effects involving treatment were
primarily due to differences between the two groups in the initial rate of conditioning;
controls acquired the discrimination between B and D faster than cocaine animals (p <
0.05) (session 1 versus session 3). Thus, while control and cocaine-experienced rats
responded similarly to cue D in all sessions (Bonferroni post-hoc correction; p’s > 0.05),
responding to cue B differed in the first session (p < 0.05) but not thereafter (p’s > 0.05).
This likely reflected the prior experience with reward retrieval for controls. Accordingly,
there were no significant differences between groups by the end of conditioning (p’s >
0.05).
iii) Probe Test:
In the probe test, both control and cocaine-experienced rats responded more to B,
the cue previously paired with sucrose, than to D, the non-reinforced cue (Figure III.2E
and F). Controls also exhibited significantly greater responding to the preconditioned cue,
A, which had been previously paired with the rewarded cue, B, than to the preconditioned
control cue, C (Figure III.2E). Cocaine-experienced rats showed no difference in
76
responding to the preconditioned cues A and C (Figure III.2F). A 2-factor ANOVA (cue
x treatment) indicated a significant main effect of cue (F(3,129) = 85.76, p < 0.01) and a
significant interaction between cue and treatment (F(3,129) = 2.90, p < 0.05). Bonferroni
post-hoc testing showed that both groups responded significantly more to B than D (p’s <
0.05), and controls responded significantly more to A than to either C or D (p’s < 0.05),
while cocaine-experienced rats responded similarly to A, C, and D (p’s > 0.05).
77
Figure III.2 Cocaine self-administration disrupts the expression of behavior that
depends upon inferred values.
The experimental timeline is schematized at the bottom of the figure. Figures show the
percentage of time spent in food cup during presentation of the cues during each of the three
phases of training: preconditioning (A,B), conditioning (C,D), and the probe test (E,F).
Control rats are presented in the top panels (A,C,E) and the cocaine-experienced rats are
shown in the bottom panels (B,D,F). *p < 0.05, **p < 0.01 or better. Error bars = SEM.
* Source of image (Wied et al., 2013)
78
C. Inferred Value Blocking:
i) Blocking:
During the two days of compound conditioning, the pre-conditioned cues, A and
C, were presented in compound with new visual cues, X and Y. AX and CY were both
reinforced with the same sucrose reward previously paired with B. A 3-factor ANOVA
(treatment x cue x session) demonstrated significant main effects of session (F(1,66) =
22.69, p < 0.01) and treatment (F(1,66) = 9.62, p < 0.01) and a significant interaction
between cue and session (F(1,66) = 6.35, p < 0.05). While cocaine-experienced rats
exhibited slightly lower responding overall, there were no significant interactions with
treatment and other factors (F's < 0.91, p's > 0.34) (a).
ii) Probe Test:
In the probe test, rats were presented with reinforced AX and CY reminder trials,
followed by probe trials, in which X and Y were presented alone. A 2-factor ANOVA
(treatment x cue) on conditioned responding during the probe trials revealed a significant
interaction between cue and treatment (F(1,33) = 8.83, p < 0.05). Bonferroni post-hoc
testing showed that controls responded significantly more to Y than to X (p < 0.05)
(Figure III.3C), whereas rats exposed to cocaine responded similarly to these two cues (p
> 0.05) (Figure III.3D). Subsequent comparisons of cue-evoked responding to responding
in the baseline period showed that while controls exhibited both a significant main effect
of period (F(1,42) = 19.7, p < 0.01) and a significant period by cue interaction (F(1,42) =
79
7.00, p < 0.05), cocaine-experience rats exhibited only a main effect of period (F(1,24) =
7.68, p < 0.05; other p's > 0.66).
Figure III.3 Cocaine self-administration disrupts learning that depends upon inferred
values.
The experimental timeline is schematized at the bottom of the figure. Figures show the
percentage of time spent in food cup during presentation of the cues during blocking
conditioning (A,B) and the subsequent probe test (C,D). Control rats are presented in the top
panels (A,C) and the cocaine-experienced rats are shown in the bottom panels (B, D). **p <
0.01 or better. Error bars = SEM.
* Source of image (Wied et al., 2013)
80
5. Discussion
Here, we have shown that cocaine self-administration disrupts behavior and
learning that depends upon inferred value. In a sensory preconditioning task, cocaine-
experienced rats were not able to use the relationship between the preconditioned cue and
the subsequently conditioned cue to infer value in order to guide behavior, nor were they
able to mobilize this information in order to modulate learning during the blocking task.
These effects were enduring, appearing during probe tests conducted 5-6 weeks after the
last contact with cocaine. In addition, they were specific inasmuch as the cocaine-
experienced rats responded like controls to the cues that were directly paired with reward.
These data add to prior reports that psychostimulants affect reinforcer devaluation
(Nelson and Killcross, 2006, Schoenbaum and Setlow, 2005). As in the sensory
preconditioning task used here, normal changes in behavior after reinforcer devaluation
requires access to a model-based representation of the causal structure of the task, so that
never before experienced consequences can be mentally simulated. As noted earlier, rats
that have been exposed to psychostimulants show long-lasting deficits in their ability to
inhibit learned Pavlovian and instrumental behaviors after reinforcer devaluation. Our
current results show a similar deficit, but in a setting that requires neither response
inhibition nor the ability to perceive and respond appropriately to aversive information.
In addition, the current data extend these findings and show that these operations are
impacted both for the purpose of guiding behavior appropriately as well as for new
learning derived from this information, since cocaine-experienced rats were also unable
to use the inferred value of the preconditioned cue to block learning. Overall, these
81
results substantially strengthen the idea that exposure to some addictive drugs disrupts the
neural circuits that mediate model-based processing.
Currently, candidates for mediating this model-based processing include
prefrontal regions, such as the medial and orbital prefrontal cortices. Changes in
instrumental responding after reinforcer devaluation as well as extinction learning,
particularly recall, depend critically on input from medial prefrontal cortex to dorsal
striatal regions (Milad and Quirk, 2002, Quirk et al., 2000, Rhodes and Killcross, 2004,
Rhodes and Killcross, 2007, Killcross and Coutureau, 2003, Corbit and Balleine, 2003,
Zapata et al., 2010). Similarly, flexible responding to Pavlovian cues after devaluation
and in other settings requires orbitofrontal output that impacts areas in the dorsal and
ventral striatum (Pickens et al., 2003, Izquierdo et al., 2004, Machado and Bachevalier,
2007, West et al., 2011). Altered prefrontal processing would therefore be expected to
lead to behavioral deficits in these areas. Indeed, orbitofrontal inactivation causes
precisely the deficits reported here after cocaine use in the sensory preconditioning and
inferred value blocking tasks (Jones et al., 2012).
Despite the similarity of our cocaine-induced deficits to those seen in OFC-
inactivated rats (Jones et al., 2012), the current data cannot eliminate the possibility that
the deficit occurs because cocaine-experienced rats are unable to acquire the stimulus-
stimulus association. That is, because cocaine-exposure occurred prior to any
conditioning, it is possible that the probe test deficits are a result of an inability to learn
the stimulus-stimulus contingency in the first phase of the experiment. Such an effect
might reflect an impact of cocaine on hippocampal regions known to be important for
model-based behaviors and stimulus-stimulus learning (Wimmer and Shohamy, 2011,
82
Wimmer and Shohamy, 2012). However we are not aware of evidence that stimulus-
stimulus learning is impaired by cocaine exposure and, given the strong evidence
implicating the prefrontal regions in addiction and in the type of processing isolated in
later phases of the task, we would view this as a more parsimonious explanation of the
current results.
It is also worth noting that although deficits in some model-based behaviors have
been conceived of as reflecting strengthened processing of the countervailing “habits” in
striatal regions (Robbins and Everitt, 1999, Belin and Everitt, 2008), the current results
cannot be easily explained by such a mechanism. This is because stronger encoding of
the cached value or policy associated with the rewarded cue, in concert with an intact
model-based system, should have led to stronger, not weaker, responding to the
preconditioned cue, and better, not worse, blocking. Thus, the only reasonable
explanation that accounts for the current results is alterations in model-based processing.
While this does not preclude a contribution of stronger habits to other aspects of drug-
seeking, it is, in fact, consistent with what we and others have reported, which is that
psychostimulants cause rather modest changes in information processing in dorsal
striatum (Takahashi et al., 2007) while substantially altering that in the orbitofrontal
cortex (Homayoun and Moghaddam, 2006, Stalnaker et al., 2006).
So, what are the implications of these findings for understanding and, more
importantly, addressing behavioral issues in addiction? One implication is that
individuals with a history of psychostimulant use will be less responsive than other
patients to so-called talking therapies, which require insight and mental simulation to
have their effect. This would be consistent with the high rates of relapse after even
83
apparently successful psychotherapeutic interventions (Dutra et al., 2008). A second
implication is that psychostimulant users are likely to be less adept than other patients at
learning when outcomes are not as rewarding as expected, especially to the extent that
these differences are defined by observational and inference based evidence. As noted in
our introduction, we believe this type of deficit is apparent in the diagnostic criteria for
addiction.
Identifying a functional deficit and associated circuit can allow for better insight
into these behaviors and perhaps targeting of affected individuals with pharmacologic
agents known to enhance prefrontal function. For example, imaging research has long
identified frontostriatal, and particularly orbitofrontal, circuits as abnormally active (or
inactive) in psychostimulant users (Goldstein and Volkow, 2011). Abnormal activity
patterns have been associated with reactivity to drug-associated cues, craving, and also
deficits in reward perception. One particularly interesting result is that BOLD signal in
psychostimulant users fails to reflect monetary reward gradients normally (Goldstein et
al., 2007b). That is, while BOLD signal increases generally to a monetary reward in
cocaine users, it does not scale with the amount or magnitude. This result has been
replicated several times and is associated with altered P300 and striatal signaling,
chronicity of drug use, impairments in self-control and even lack of insight (Parvaz et al.,
2012, Moeller et al., 2012a, Konova et al., 2012, Goldstein et al., 2007a). The latter
speculation is particularly intriguing in light of the current results, since insight arguably
depends upon the same process of mental simulation that characterizes model-based
processing. Accordingly, we would suggest that the failure of BOLD signal in frontal
areas to scale with reward likely reflects dysfunctional processing and failure of operation
84
of these model-based systems. This is because the primary difference between a $10 bill
and a $100 bill lies not in its sensory qualities, but rather in the quantity and quality of
goods that one can obtain with each. Fully appreciating this difference requires inference
and mental simulation. Lacking this ability, patients with orbitofrontal BOLD signal fail
to identify any difference between these two green pieces of paper – both are similarly
significant.
If this is correct, then these imaging findings provide a marker for altered
prefrontal function in specific individuals. Such a diagnostic tool might then be combined
with pharmacologic agents known to enhance prefrontal dependent processing. This
effort has been the focus of recent endeavors with methylphenidate (Moeller et al.,
2012b, Goldstein et al., 2010), but additional candidates include nicotinic and alpha-2
adrenergic receptor agents. These have recently been shown to have highly specific
effects on the maintenance of information in working or scratchpad memory in prefrontal
cortex (Wang et al., 2011, Castner et al., 2011). Similarly, serotonergic tone has been
shown to be necessary for reversal learning and devaluation (Clarke et al., 2004,
Schilman et al., 2010, Nonkes et al., 2010). Our results suggest that approaches that
combine these agents with typical psychotherapeutic interventions that appropriately
target individuals who show abnormal processing in orbital or other prefrontal areas
might prove especially beneficial.
85
IV. Conclusions
We make decisions each and every day as a natural consequence of interacting
with our environment. Some of these decisions are easy. Others are more complex or
difficult, particularly those that require us to evaluate actions in novel situations. How,
then, do we make these decisions given these unfamiliar circumstances? And if we make
a "wrong" decision, that is to say where the outcome is adverse or runs contrary to our
predicted outcome, how do we learn to adjust our behavior the next time to produce a
more favorable result? Or more colloquially, how do we learn from our mistakes? The
orbitofrontal cortex (OFC) is critical for making these more complex, in-the-moment
decisions, and for learning to update our expectations when experiences differ from our
assumptions. Making decisions in these unfamiliar situations requires us to use a model-
based representation of our situation and for us to have an idea of the predicted outcome
of our decision. Understanding that the OFC is important for these decisions led me to
wonder: 1) How are inferred values and their predicted outcomes signaled in the OFC?
2) Is this ability to make decisions in more complex situations and the ability to learn
from our mistakes completely seized when people abuse psychostimulant drugs,
specifically cocaine?
The sensory preconditioning task recently used in the Schoenbaum lab
demonstrated further evidence that the OFC is critical for the ability to make decisions in-
the-moment in novel situations or environments based on predicted outcomes. But the
research also suggests that the OFC is not essential for habit-like decisions based off of
repeated and previous experiences. Additionally, this study demonstrated that the OFC
was important for learning when expectations about predicted outcomes were not met.
86
This ability to recognize and compare predictions and outcomes is critical for learning.
This study laid the foundation for my dissertation, as I set out to understand the
neurophysiological mechanism of how the OFC might signal this inferred value. Previous
reports have demonstrated that cocaine disrupts the OFC. In addition, the prevalence of
long term health issues, failed family relationships and repeated incarcerations provide
evidence that addicts face challenges in making good decisions based on long term
outcomes. I hypothesized that rats exposed to cocaine would have difficulties similar to
those seen when the OFC was inactivated. Ultimately, my research aimed to provide
evidence that cocaine affects the ability to infer value, and thus cocaine-exposed rats
subsequently fail to learn when expected outcomes are not met.
In my first study, I hypothesized that the OFC is signaling the inferred value in
the sensory preconditioning task. During the sensory preconditioning phase, the mean
firing rate of the stimulus-stimulus cues were strongly correlated. The strength of this
original relationship decreased throughout conditioning as the cues were paired with
either a reward or no reward. However, there was still a significant correlation at the time
of testing, strongly suggesting that the conditioned cues firing rate predicted the
preconditioned cues firing rate. This significant correlation was maintained and was
enough to make an inference on the probe test when the preconditioned auditory cues
were played. These results are intriguing, and although ultimately incomplete, suggest
that with more neural data we may have a clearer understanding of what the OFC is
signaling. These data were not able to distinguish whether or not the inferred value cue
was predicted by the ability of the OFC to signal this inferred value cue, or if the OFC
may be signaling the meaning or value of the conditioned cue. As more sessions are
87
recorded, with better behavioral outcomes showing stronger inferred value in their
behavior, it will be interesting to explore in greater detail what the OFC neurons are
specifically encoding. Recent studies indicate that the ventral striatum plays a wider role
in learning than can be explained by the signaling of cached values, one that is more
consistent with integration of outcome and value information (McDannald et al., 2011).
Important future studies include recording in downstream projections of the OFC in this
corticolimbic circuit, such as the ventral striatum and ventral tegmental area. It would be
interesting to observe if this integration continues downstream to the midbrain dopamine
neurons as well.
While my findings do not fully articulate the specifics of what the neurons in the
OFC are encoding during the sensory preconditioning task, the results from my study on
the effects of cocaine on the OFC are much clearer. The long-term effects of cocaine
exposure paint a depressing picture as results from the rats who self-administrated
cocaine demonstrated disrupted behavior and learning that depends on inferred value but
not on cached value. These data substantially strengthen the viewpoint that cocaine
disrupts the neural circuits that mediate model-based processing. In the sensory
preconditioning task, 14 days of self-administration of cocaine and 5-6 weeks of
withdrawal led to the inability of rats to use the relationship between the cue and the
subsequently conditioned cue to infer value in order to guide behavior. They were also
unable to mobilize this information in order to modulate learning during a blocking task.
These results, though not altogether unexpected, are extremely discouraging in
terms of some of the common treatments that are emphasized for patients with abuse. In
many therapy situations, patients are asked to discuss their treatments and think about the
88
consequences of their actions, which requires insight to have an effect. But these results
suggest that this type of therapy would not be beneficial for patients as they may be
unable to have this insight about their drug abuse. The potential that addicts may be
unable to learn from the detrimental consequences of their behavior, and that their
behaviors are more stimulus-based due to the inability to compare the predicted outcomes
with reality, make it more difficult for them to stop their behavior of using cocaine. In
order to help addicts, we must think of other therapies. In particular, it would be of great
interest to see if we could reverse these deficits somehow using techniques and strategies
with the potential to enhance prefrontal function. These might include pharmacological
agents or repetitive transcranial magnetic stimulation (rTMS). In addition to experiments
to better understand what the OFC is signaling, it could be compelling to record the
neural signals in the cocaine-exposed animals to see if there are differences compared to
normal controls. Furthermore, it may be of interest to examine if cocaine-exposed rats
show the same strong correlation between stimulus-stimulus signaling during
preconditioning, or if this relationship of the conditioned cue predicting the firing of the
preconditioned cue is completely destroyed by the time of probe test day.
Since Phineas Gage survived the railroad accident back in 1848, which led to an
observed radical transformation of his character, the orbitofrontal cortex (OFC) and its
role in behavior and decision making has been of great interest to the field of
neuroscience. It is through patients like Gage, as well as behavioral and
neurophysiological experiments, that we have a greater understanding of the OFC's role
in behavior, decision making and learning. My research aspires to be a first step in
looking at what the OFC signals during the sensory preconditioning task, and it will be
89
interesting to see how everything unfolds and how, and if, the neurons signal this inferred
value. Additionally, it will be important to see if these signals are demonstrated in
downstream targets of the OFC, such as the ventral striatum and ventral tegmental areas,
including dopamine neurons. The results of the cocaine study further demonstrate that
long term effects of the drug not only affect the ability to predict outcomes, but also
affect further learning. This research demonstrates a great need to figure out therapeutic
targets to reverse these damaging effects of cocaine on the OFC so that addicts have the
potential to overcome these deficits in order to stop abusing drugs and realize the long
term consequences of their actions.
ix
References
ADINOFF, B., DEVOUS, M. D., SR., BEST, S. M., GEORGE, M. S., ALEXANDER,
D. & PAYNE, K. 2001. Limbic responsiveness to procaine in cocaine-addicted
subjects. Am J Psychiatry, 158, 390-8.
APA 2000. Diagnostic and Statistical Manual of Mental Disorders (4th ed., text revision).
Washington DC.
BAXTER, M. G. & MURRAY, E. A. 2002. The amygdala and reward. Nature Reviews
Neuroscience, 3, 563-573.
BECHARA, A., DAMASIO, H., TRANEL, D. & DAMASIO, A. R. 1997. Deciding
advantageously before knowing the advantageous strategy. Science, 275, 1293-
1294.
BELIN, D. & EVERITT, B. J. 2008. Cocaine seeking habits depend upon dopamine-
dependent serial connectivity linking the ventral with the dorsal striatum. Neuron,
57, 432-441.
BISSONETTE, G. B., MARTINS, G. J., FRANZ, T. M., HARPER, E. S.,
SCHOENBAUM, G. & POWELL, E. M. 2008. Double dissociation of the effects
of medial and orbital prefrontal cortical lesions on attentional and affective shifts
in mice. Journal of Neuroscience, 28, 11124-11130.
BROWN, S. & SCHAFER, E. A. 1888. An investigation into the functions of the
occipital and temporal lobes of the monkey's brain. Philosophical Transactions of
the Royal Society of London B, 179, 303-327.
BURKE, K. A., FRANZ, T. M., MILLER, D. N. & SCHOENBAUM, G. 2008. The role
of orbitofrontal cortex in the pursuit of happiness and more specific rewards.
Nature, 454, 340-344.
BUTTER, C. M. 1969. Perseveration and extinction in discrimination reversal tasks
following selective frontal ablations in Macaca mulatta. Physiology and Behavior,
4, 163-171.
CALU, D. J., STALNAKER, T. A., FRANZ, T. M., SINGH, T., SHAHAM, Y. &
SCHOENBAUM, G. 2007. Withdrawal from cocaine self-administration
produces long-lasting deficits in orbitofrontal-dependent reversal learning in rats.
Learn Mem, 14, 325-8.
CARMICHAEL, S. T. & PRICE, J. L. 1994. Architectonic subdivision of the orbital and
medial prefrontal cortex in the macaque monkey. J Comp Neurol, 346, 366-402.
x
CARMICHAEL, S. T. & PRICE, J. L. 1995. Limbic connections of the orbital and
medial prefrontal cortex in macaque monkeys. Journal of Comparative
Neurology, 363, 615-641.
CASTNER, S. A., SMAGIN, G. N., PISER, T. M., WANG, Y., SMITH, J. S.,
CHRISTIAN, E. P., MRZLJAK, L. & WILLIAMS, G. V. 2011. Immediate and
sustained improvements in working memory after selective stimulation of alpha7
nicotinic acetylcholine receptors. Biol Psychiatry, 69, 12-18.
CHUDASAMA, Y. & ROBBINS, T. W. 2003. Dissociable contributions of the
orbitofrontal and infralimbic cortex to pavlovian autoshaping and discrimination
reversal learning: further evidence for the functional heterogeneity of the rodent
frontal cortex. Journal of Neuroscience, 23, 8771-8780.
CLARKE, H. F., DALLEY, J. W., CROFTS, H. S., ROBBINS, T. W. & ROBERTS, A.
C. 2004. Cognitive inflexibility after prefrontal serotonin depletion. Science, 304,
878-880.
CORBIT, L. H. & BALLEINE, B. W. 2003. The role of prelimbic cortex in instrumental
conditioning. Behavioral Brain Research, 146, 145-157.
CROMBAG, H. S., GORNY, G., LI, Y., KOLB, B. & ROBINSON, T. E. 2005. Opposite
effects of amphetamine self-administration experience on dendritic spines in the
medial and orbital prefrontal cortex. Cereb Cortex, 15, 341-8.
DAMASIO, A. R. 1994. Decartes Error, New York, Putnam.
DAVIS, M. 2000. The role of the amygdala in conditioned and unconditioned fear and
anxiety. In: AGGLETON, J. P. (ed.) The Amygdala: A Functional Analysis.
Oxford: Oxford University Press.
DAW, N. D., NIV, Y. & DAYAN, P. 2005. Uncertainty-based competition between
prefrontal and dorsolateral striatal systems for behavioral control. Nature
Neuroscience, 8, 1704-1711.
DELAMATER, A. R. 2007. The role of the orbitofrontal cortex in sensory-specific
encoding of associations in pavlovian and instrumental conditioning. Annals of
the New York Academy of Science, 1121, 152-173.
DEROCHE-GAMONET, V., BELIN, D. & PIAZZA, P. V. 2004. Evidence for addiction-
like behavior in the rat. Science, 305, 951-953.
DUTRA, L., STATHOPOULOU, G., BASDEN, S. L., LEYRO, T. M., POWERS, M. B.
& OTTO, M. W. 2008. A meta-analytic review of psychosocial interventions for
substance use disorders. Am J Psychiatry, 165, 179-187.
xi
EAGLE, D. M., BAUNEZ, C., HUTCHESON, D. M., LEHMANN, O., SHAH, A. P. &
ROBBINS, T. W. 2008. Stop-signal reaction-time task performance: role of
prefrontal cortex and subthalamic nucleus. Cerebral Cortex, 18, 178-188.
ERSCHE, K. D., BARNES, A., JONES, P. S., MOREIN-ZAMIR, S., ROBBINS, T. W.
& BULLMORE, E. T. 2011. Abnormal structure of frontostriatal brain systems is
associated with aspects of impulsivity and compulsivity in cocaine dependence.
Brain, 134, 2013-24.
ERSCHE, K. D., ROISER, J. P., ROBBINS, T. W. & SAHAKIAN, B. J. 2008. Chronic
cocaine but not chronic amphetamine use is associated with perseverative
responding in humans. Psychopharmacology (Berl), 197, 421-31.
EVERITT, B. J. & ROBBINS, T. W. 1992. Amygdala-ventral striatal interactions and
reward-related processes. In: AGGLETON, J. P. (ed.) The Amygdala:
Neurological Aspects of Emotion, Memory, and Mental Dysfunction. Oxford:
John Wiley and Sons.
FELLOWS, L. K. & FARAH, M. J. 2003. Ventromedial frontal cortex mediates affective
shifting in humans: evidence from a reversal learning paradigm. Brain, 126, 1830-
1837.
FERRIER, D. 1876. The Functions of the Brain, New York, GP Putnam's Sons.
FILLMORE, M. T. & RUSH, C. R. 2006. Polydrug abusers display impaired
discrimination-reversal learning in a model of behavioural control. J
Psychopharmacol, 20, 24-32.
FRANKLIN, T. R., ACTON, P. D., MALDJIAN, J. A., GRAY, J. D., CROFT, J. R.,
DACKIS, C. A., O'BRIEN, C. P. & CHILDRESS, A. R. 2002. Decreased gray
matter concentration in the insular, orbitofrontal, cingulate, and temporal cortices
of cocaine patients. Biol Psychiatry, 51, 134-42.
GALLAGHER, M. 2000. The amygdala and associative learning. In: AGGLETON, J. P.
(ed.) The Amygdala: A Functional Analysis. Oxford: Oxford University Press.
GALLAGHER, M., MCMAHAN, R. W. & SCHOENBAUM, G. 1999. Orbitofrontal
cortex and representation of incentive value in associative learning. Journal of
Neuroscience, 19, 6610-6614.
GAWIN, F. H. 1991. Cocaine addiction: psychology and neurophysiology. Science, 251,
1580-6.
GOLDSTEIN, R. Z., ALIA-KLEIN, N., TOMASI, D., ZHANG, L., COTTONE, L. A.,
MALONEY, T., TELANG, F., CAPARELLI, E. C., CHANG, L., ERNST, T.,
SAMARAS, D., SQUIRES, N. K. & VOLKOW, N. D. 2007a. Is decreased
xii
prefrontal cortical sensitivity to monetary reward associated with impaired
motivation and self-control in cocaine addiction? American Journal of Psychiatry,
164, 43-51.
GOLDSTEIN, R. Z., TOMASI, D., ALIA-KLEIN, N., COTTONE, L. A., ZHANG, L.,
TELANG, F. & VOLKOW, N. D. 2007b. Subjective sensitivity to monetary
gradients is associated with frontolimbic activation to reward in cocaine abusers.
Drug and Alcohol Dependence, 87, 233-240.
GOLDSTEIN, R. Z. & VOLKOW, N. D. 2011. Dysfunction of the prefrontal cortex in
addiction: neuroimaging findings and clinical implications. Nature Reviews
Neuroscience, 12, 652-669.
GOLDSTEIN, R. Z., WOICIK, P. A., MALONEY, T., TOMASI, D., ALIA-KLEIN, N.,
SHAN, J., HONORIO, J., SAMARAS, D., WANG, R., TELANG, F., WANG, G.
J. & VOLKOW, N. D. 2010. Oral methylphenidate normalizes cingulate activity
in cocaine addiction during a salient cognitive task. Proc Natl Acad Sci U S A,
107, 16667-72.
GOTTFRIED, J. A., O'DOHERTY, J. & DOLAN, R. J. 2003. Encoding predictive
reward value in human amygdala and orbitofrontal cortex. Science, 301, 1104-
1107.
GROENEWEGEN, H. J. 1988. Organization of the afferent connections of the
mediodorsal thalamic nucleus in the rat, related to the mediodorsal-prefrontal
topography. Neuroscience, 24, 379-431.
GROENEWEGEN, H. J., VERMEULEN-VAN DER ZEE, E., TE KORTSCHOT, A. &
WITTER, M. P. 1987. Organization of the projections from the subiculum to the
ventral striatum in the rat. A study using anterograde transport of Phaseolus
vulgaris leucoagglutinin. Neuroscience, 23, 103-120.
HABER, S. N. & KNUTSON, B. 2010. The reward circuit: linking primate anatomy and
human imaging. Neuropsychopharmacology, 35, 4-26.
HABER, S. N., KUNISHIO, K., MIZOBUCHI, M. & LYND-BALTA, E. 1995. The
orbital and medial prefrontal circuit through the primate basal ganglia. Journal of
Neuroscience, 15, 4851-4867.
HARLOW, J. M. 1868. Recovery after passage of an iron bar through the head.
Publications of the Massachusetts Medical Society, 2, 329-346.
HOLLAND, P. C. 1984. Unblocking in Pavlovian appetitive conditioning. J Exp Psychol
Anim Behav Process, 10, 476-97.
xiii
HOLLAND, P. C. & GALLAGHER, M. 1999. Amygdala circuitry in attentional and
representational processes. Trends in Cognitive Sciences, 3, 65-73.
HOLLAND, P. C. & GALLAGHER, M. 2004. Amygdala-frontal interactions and reward
expectancy. Current Opinion in Neurobiology, 14, 148-155.
HOLLAND, P. C. & RESCORLA, R. A. 1975. The effects of two ways of devaluing the
unconditioned stimulus after first and second-order appetitive conditioning.
Journal of Experimental Psychology: Animal Behavior Processes, 1, 355-363.
HOLLAND, P. C. & STRAUB, J. J. 1979. Differential effects of two ways of devaluing
the unconditioned stimulus after Pavlovian appetitive conditioning. Journal of
Experimental Psychology: Animal Behavior Processes, 5, 65-78.
HOMAYOUN, H. & MOGHADDAM, B. 2006. Progression of cellular adaptations in
medial prefrontal and orbitofrontal cortex in response to repeated amphetamine.
Journal of Neuroscience, 26, 8025-8039.
HORNAK, J., O'DOHERTY, J., BRAMHAM, J., ROLLS, E. T., MORRIS, R. G.,
BULLOCK, P. R. & POLKEY, C. E. 2004. Reward-related reversal learning after
surgical excisions in orbito-frontal or dorsolateral prefrontal cortex in humans.
Journal of Cognitive Neuroscience, 16, 463-478.
IZQUIERDO, A. & JENTSCH, J. D. 2012. Reversal learning as a measure of impulsive
and compulsive behavior in addictions. Psychopharmacology (Berl), 219, 607-20.
IZQUIERDO, A. & MURRAY, E. A. 2010. Functional interaction of medial mediodorsal
thalamic nucleus but not nucleus accumbens with amygdala and orbital prefrontal
cortex is essential for adaptive response selection after reinforcer devaluation.
Journal of Neuroscience, 30, 661-669.
IZQUIERDO, A. D. & MURRAY, E. A. 2004. Combined unilateral lesions of the
amygdala and orbital prefrontal cortex impair affective processing in rhesus
monkeys. Journal of Neurophysiology, 91, 2033-2039.
IZQUIERDO, A. D., SUDA, R. K. & MURRAY, E. A. 2004. Bilateral orbital prefrontal
cortex lesions in rhesus monkeys disrupt choices guided by both reward value and
reward contingency. Journal of Neuroscience, 24, 7540-7548.
JENTSCH, J. D., OLAUSSON, P., DE LA GARZA, R., 2ND & TAYLOR, J. R. 2002.
Impairments of reversal learning and response perseveration after repeated,
intermittent cocaine administrations to monkeys. Neuropsychopharmacology, 26,
183-90.
xiv
JENTSCH, J. D. & TAYLOR, J. R. 1999. Impulsivity resulting from frontostriatal
dysfunction in drug abuse: implications for the control of behavior by reward-
related stimuli. Psychopharmacology, 146, 373-390.
JONES, B. & MISHKIN, M. 1972. Limbic lesions and the problem of stimulus-
reinforcement associations. Experimental Neurology, 36, 362-377.
JONES, J. L., ESBER, G. R., MCDANNALD, M. A., GRUBER, A. J., HERNANDEZ,
A., MIRENZI, A. & SCHOENBAUM, G. 2012. Orbitofrontal cortex supports
behavior and learning using inferred but not cached values. Science, 338, 953-6.
KAMIN, L. J. 1969. Predictability, suprise, attention, and conditioning. In: CAMPBELL,
B. A. & CHURCH, R. M. (eds.) Punishment and Aversive Behavior. New York:
Appleton-Century-Crofts.
KENNERLEY, S. W., BEHRENS, T. E. & WALLIS, J. D. 2011. Double dissociation of
value computations in orbitofrontal and anterior cingulate neurons. Nature
Neuroscience, AOP.
KILLCROSS, S. & COUTUREAU, E. 2003. Coordination of actions and habits in the
medial prefrontal cortex of rats. Cerebral Cortex, 13, 400-408.
KLUVER, H. & BUCY, P. C. 1939. Preliminary analysis of the temporal lobes in
monkeys. Archives of Neurology and Psychiatry, 42, 979-1000.
KOLB, B., PELLIS, S. & ROBINSON, T. E. 2004. Plasticity and functions of the orbital
frontal cortex. Brain Cogn, 55, 104-15.
KONOVA, A. B., MOELLER, S. J., TOMASI, D., PARVAZ, M. A., ALIA-KLEIN, N.,
VOLKOW, N. D. & GOLDSTEIN, R. Z. 2012. Structural and behavioral
correlates of abnormal encoding of money value in the sensorimotor striatum in
cocaine addiction. European Journal of Neuroscience, 36, 2979-2988.
KOOB, G. F. & LE MOAL, M. 2008. Review. Neurobiological mechanisms for
opponent motivational processes in addiction. Philosophical Transactions of the
Royal Society of London B, 363, 3113-3123.
KRETTEK, J. E. & PRICE, J. L. 1977. The cortical projections of the mediodorsal
nucleus and adjacent thalamic nuclei in the rat. Journal of Comparative
Neurology, 171, 157-192.
LEDOUX, J. E. 1996. The emotional brain, New York, Simon and Schuster.
LEONARD, C. M. 1969. The prefrontal cortex of the rat. I. Cortical projections of the
mediodorsal nucleus. II. Efferent connections. Brain Research, 12, 321-343.
xv
LEVY, D. J. & GLIMCHER, P. W. 2011. Comparing apples and oranges: Using reward-
specific and reward-general subjective value representation in the brain. Journal
of Neuroscience, 31, 14693-14707.
LEVY, D. J. & GLIMCHER, P. W. 2012. The root of all value: a neural common
currency for choice. Current Opinion in Neurobiology, 22, 1027-1038.
LONDON, E. D., CASCELLA, N. G., WONG, D. F., PHILLIPS, R. L., DANNALS, R.
F., LINKS, J. M., HERNING, R., GRAYSON, R., JAFFE, J. H. & WAGNER, H.
N., JR. 1990. Cocaine-induced reduction of glucose utilization in human brain. A
study using positron emission tomography and [fluorine 18]-fluorodeoxyglucose.
Arch Gen Psychiatry, 47, 567-74.
LUCANTONIO, F., STALNAKER, T. A., SHAHAM, Y., NIV, Y. & SCHOENBAUM,
G. 2012. The impact of orbitofrontal dysfunction on cocaine addiction. Nat
Neurosci, 15, 358-66.
LUCANTONIO, F., TAKAHASHI, Y., HOFFMAN, A., CHANG, C. Y.,
CHAUDHARY, S., SHAHAM, Y., LUPICA, C. R. & SCHOENBAUM, G. 2014.
Orbitofrontal activation restores insight lost after cocaine use. in review.
LUK, C.-H. & WALLIS, J. D. 2013. Choice coding in frontal cortex during stimulus-
guided or action-guided decision-making. Journal of Neuroscience, 33, 1864-
1871.
LYOO, I. K., STREETER, C. C., AHN, K. H., LEE, H. K., POLLACK, M. H.,
SILVERI, M. M., NASSAR, L., LEVIN, J. M., SARID-SEGAL, O., CIRAULO,
D. A., RENSHAW, P. F. & KAUFMAN, M. J. 2004. White matter
hyperintensities in subjects with cocaine and opiate dependence and healthy
comparison subjects. Psychiatry Res, 131, 135-45.
MACHADO, C. J. & BACHEVALIER, J. 2007. The effects of selective amygdala,
orbital frontal cortex or hippocampal formation lesions on reward assessment in
nonhuman primates. European Journal of Neuroscience, 25, 2885-2904.
MAN, M. S., CLARKE, H. F. & ROBERTS, A. C. 2009. The role of the orbitofrontal
cortex and medial striatum in the regulation of prepotent responses to food
rewards. Cerebral Cortex, 19, 899-906.
MCALONAN, K. & BROWN, V. J. 2003. Orbital prefrontal cortex mediates reversal
learning and not attentional set shifting in the rat. Behavioral Brain Research,
146, 97-130.
MCDANNALD, M. A., LUCANTONIO, F., BURKE, K. A., NIV, Y. &
SCHOENBAUM, G. 2011. Ventral striatum and not orbitofrontal cortex are both
xvi
required for model-based, but not model-free, reinforcement learning. Journal of
Neuroscience, in press.
MCDANNALD, M. A., SADDORIS, M. P., GALLAGHER, M. & HOLLAND, P. C.
2005. Lesions of orbitofrontal cortex impair rats' differential outcome expectancy
learning but not conditioned stimulus-potentiated feeding. Journal of
Neuroscience, 25, 4626-4632.
MCDONALD, A. J. 1991. Organization of the amygdaloid projections to the prefrontal
cortex and associated striatum in the rat. Neuroscience, 44, 1-14.
MENDELSON, J. H. & MELLO, N. K. 1996. Management of cocaine abuse and
dependence. N Engl J Med, 334, 965-72.
MENDEZ, I. A., SIMON, N. W., HART, N., MITCHELL, M. R., NATION, J. R.,
WELLMAN, P. J. & SETLOW, B. 2010. Self-administered cocaine causes long-
lasting increases in impulsive choice in a delay discounting task. Behavioral
Neuroscience, 124, 470-477.
MEUNIER, M., BACHEVALIER, J. & MISHKIN, M. 1997. Effects of orbital frontal
and anterior cingulate lesions on object and spatial memory in rhesus monkeys.
Neuropsychologia, 35, 999-1015.
MILAD, M. R. & QUIRK, G. J. 2002. Neurons in medial prefrontal cortex signal
memory for fear extinction. Nature, 420, 70-74.
MOELLER, S. J., HAJCAK, G., PARVAZ, M. A., DUNNING, J. P., VOLKOW, N. D.
& GOLDSTEIN, R. Z. 2012a. Psychophysiological prediction of choice:
relevance to insight and drug addiction. Brain, 135, 3481-3494.
MOELLER, S. J., HONORIO, J., TOMASI, D., PARVAZ, M. A., WOICIK, P. A.,
VOLKOW, N. D. & GOLDSTEIN, R. Z. 2012b. Methylphenidate enhances
executive function and optimizes prefrontal function in both health and cocaine
addiction. Cereb Cortex.
MONTAGUE, P. R. & BERNS, G. S. 2002. Neural economics and the biological
substrates of valuation. Neuron, 36, 265-284.
MURRAY, E. A., O'DOHERTY, J. & SCHOENBAUM, G. 2007. What we know and do
not know about the functions of the orbitofrontal cortex after 20 years of cross-
species studies. Journal of Neuroscience, 27, 8166-8169.
NELSON, A. & KILLCROSS, S. 2006. Amphetamine exposure enhances habit
formation. Journal of Neuroscience, 26, 3805-3812.
xvii
NIV, Y., JOEL, D. & DAYAN, P. 2006. A normative perspective on motivation. Trends
Cogn Sci, 10, 375-81.
NONKES, L. J., TOMSON, K., MAERTIN, A., DEDEREN, J., MAES, J. H. &
HOMBERG, J. 2010. Orbitofrontal cortex and amygdalar over-activity is
associated with an inability to use the value of expected outcomes to guide
behavior in serotonin transporter knockout rats. Neurobiology of Learning and
Memory, 94, 65-72.
O'DOHERTY, J., DEICHMANN, R., CRITCHLEY, H. D. & DOLAN, R. J. 2002.
Neural responses during anticipation of a primary taste reward. Neuron, 33, 815-
826.
O'DOHERTY, J., ROLLS, E. T., FRANCIS, S., BOWTELL, R., MCGLONE, F.,
KOBAL, G., RENNER, B. & AHNE, G. 2000a. Sensory-specific satiety-related
olfactory activation of the human orbitofrontal cortex. Neuroreport, 11, 893-897.
O'DOHERTY, J., ROLLS, E. T., FRANCIS, S., BOWTELL, R., MCGLONE, F.,
KOBAL, G., RENNER, B. & AHNE, G. 2000b. Sensory-specific satiety-related
olfactory activation of the human orbitofrontal cortex. Neuroreport, 11, 893-7.
O'NEILL, J., CARDENAS, V. A. & MEYERHOFF, D. J. 2001a. Effects of abstinence
on the brain: quantitative magnetic resonance imaging and magnetic resonance
spectroscopic imaging in chronic alcohol abuse. Alcohol Clin Exp Res, 25, 1673-
82.
O'NEILL, J., CARDENAS, V. A. & MEYERHOFF, D. J. 2001b. Separate and
interactive effects of cocaine and alcohol dependence on brain structures and
metabolites: quantitative MRI and proton MR spectroscopic imaging. Addict Biol,
6, 347-361.
ONGUR, D., FERRY, A. T. & PRICE, J. L. 2003. Architectonic subdivision of the
human orbital and medial prefrontal cortex. J Comp Neurol, 460, 425-49.
OSTLUND, S. B. & BALLEINE, B. W. 2007. Orbitofrontal cortex mediates outcome
encoding in Pavlovian but not instrumental learning. Journal of Neuroscience, 27,
4819-4825.
PADOA-SCHIOPPA, C. 2011. Neurobiology of economic choice: a goods-based model.
Annual Review of Neuroscience, 34, 333-359.
PADOA-SCHIOPPA, C. & ASSAD, J. A. 2006. Neurons in orbitofrontal cortex encode
economic value. Nature, 441, 223-226.
xviii
PADOA-SCHIOPPA, C. & ASSAD, J. A. 2008. The representation of economic value in
the orbitofrontal cortex is invariant for changes in menu. Nature Neuroscience,
11, 95-102.
PAIS-VIEIRA, M., LIMA, D. & GALHARDO, V. 2007. Orbitofrontal cortex lesions
disrupt risk assessment in a novel serial decision-making task for rats.
Neuroscience, 145, 225-231.
PARVAZ, M. A., MALONEY, T., MOELLER, S. J., WOICIK, P. A., ALIA-KLEIN, N.,
TELANG, F., WANG, G.-J., SQUIRES, N. K., VOLKOW, N. D. &
GOLDSTEIN, R. Z. 2012. Sensitivity to monetary reward is most severely
compromised in recently abstaining cocaine addicted individuals: A cross-
sectional ERP study. Psychiatry Research: Neuroimaging, 203, 75-82.
PICKENS, C. L., SADDORIS, M. P., GALLAGHER, M. & HOLLAND, P. C. 2005.
Orbitofrontal lesions impair use of cue-outcome associations in a devaluation
task. Behavioral Neuroscience, 119, 317-322.
PICKENS, C. L., SETLOW, B., SADDORIS, M. P., GALLAGHER, M., HOLLAND, P.
C. & SCHOENBAUM, G. 2003. Different roles for orbitofrontal cortex and
basolateral amygdala in a reinforcer devaluation task. Journal of Neuroscience,
23, 11078-11084.
PLASSMANN, H., O'DOHERTY, J. & RANGEL, A. 2007. Orbitofrontal cortex encodes
willingness to pay in everyday economic transactions. Journal of Neuroscience,
27, 9984-9988.
QUIRK, G. J., RUSSO, G. K., BARRON, J. L. & LEBRON, K. 2000. The role of
ventromedial prefrontal cortex in the recovery of extinguished fear. Journal of
Neuroscience, 20, 6225-6231.
RAY, J. P. & PRICE, J. L. 1992. The organization of the thalamocortical connections of
the mediodorsal thalamic nucleus in the rat, related to the ventral forebrain -
prefrontal cortex topography. Journal of Comparative Neurology, 323, 167-197.
REEKIE, Y. L., BRAESICKE, K., MAN, M. S. & ROBERTS, A. C. 2008. Uncoupling
of behavioral and autonomic responses after lesions of the primate orbitofrontal
cortex. Proceedings of the National Academy of Sciences, 105, 9787-9792.
RESCORLA, R. A. 1999. Learning about qualitatively different outcomes during a
blocking procedure. Animal Learning and Behavior, 27, 140-151.
RHODES, S. E. V. & KILLCROSS, S. A. 2004. Lesions of rat infralimbic cortex
enhance recovery and reinstatement of an appetitive Pavlovian response. Learning
and Memory, 11, 611-616.
xix
RHODES, S. E. V. & KILLCROSS, S. A. 2007. Lesions of rat infralimbic cortex
enhance renewal of extinguished Pavlovian responding. European Journal of
Neuroscience, 25, 2498-2503.
ROBBINS, T. W. & EVERITT, B. J. 1999. Drug addiction: bad habits add up. Nature,
398, 567-570.
ROBINSON, T. E. & BERRIDGE, K. C. 2000. The psychology and neurobiology of
addiction: an incentive-sensitization view. Addiction, 95, S91-S117.
ROBINSON, T. E. & BERRIDGE, K. C. 2008. Review. The incentive sensitization
theory of addiction: some current issues. Philosophical Transactions of the Royal
Society of London B, 363, 3137-3146.
ROESCH, M. R. & SCHOENBAUM, G. 2006. From associations to expectancies:
orbitofrontal cortex as gateway between the limbic system and representational
memory. In: ZALD, D. H. & RAUSCH, S. L. (eds.) The Orbitofrontal Cortex.
London: Oxford University Press.
ROESCH, M. R., STALNAKER, T. A. & SCHOENBAUM, G. 2007a. Associative
encoding in anterior piriform cortex versus orbitofrontal cortex during odor
discrimination and reversal learning. Cereb Cortex, 17, 643-52.
ROESCH, M. R., TAKAHASHI, Y., GUGSA, N., BISSONETTE, G. B. &
SCHOENBAUM, G. 2007b. Previous cocaine exposure makes rats hypersensitive
to both delay and reward magnitude. Journal of Neuroscience, 27, 245-250.
ROLLS, E. T., HORNAK, J., WADE, D. & MCGRATH, J. 1994. Emotion-related
learning in patients with social and emotional changes associated with frontal lobe
damage. Journal of Neurology, Neurosurgery, and Psychiatry, 57, 1518-1524.
ROSE, J. E. & WOOLSEY, C. N. 1948. The orbitofrontal cortex and its connections with
the mediodorsal nucleus in rabbit, sheep, and cat. Res Pub Ass Nerv Ment Dis, 27,
210-232.
SCARLET, J., DELAMATER, A. R., CAMPESE, V., FEIN, M. & WHEELER, D. S.
2012. Differential involvement of the basolateral amygdala and orbitofrontal
cortex in the formation of sensory-specific associations in conditioned flavor
preference and magazine approach paradigms. Eur J Neurosci, 35, 1799-809.
SCHILMAN, E. A., KLAVIR, O., WINTER, C., SOHR, R. & JOEL, D. 2010. The role
of the striatum in compulsive behavior in intact and orbitofrontal-cortex-lesioned
rats: possible involvement of the serotonergic system.
Neuropsychopharmacology, 35, 1026-1039.
xx
SCHOENBAUM, G., CHIBA, A. A. & GALLAGHER, M. 1998. Orbitofrontal cortex
and basolateral amygdala encode expected outcomes during learning. Nature
Neuroscience, 1, 155-159.
SCHOENBAUM, G., ROESCH, M. R., STALNAKER, T. A. & TAKAHASHI, Y. K.
2009. A new perspective on the role of the orbitofrontal cortex in adaptive
behaviour. Nature Reviews Neuroscience, 10, 885-892.
SCHOENBAUM, G., SADDORIS, M. P., RAMUS, S. J., SHAHAM, Y. & SETLOW, B.
2004. Cocaine-experienced rats exhibit learning deficits in a task sensitive to
orbitofrontal cortex lesions. Eur J Neurosci, 19, 1997-2002.
SCHOENBAUM, G. & SETLOW, B. 2001. Integrating orbitofrontal cortex into
prefrontal theory: common processing themes across species and subdivision.
Learning and Memory, 8, 134-147.
SCHOENBAUM, G. & SETLOW, B. 2005. Cocaine makes actions insensitive to
outcomes but not extinction: implications for altered orbitofrontal-amygdalar
function. Cerebral Cortex, 15, 1162-1169.
SCHOENBAUM, G., SETLOW, B., NUGENT, S. L., SADDORIS, M. P. &
GALLAGHER, M. 2003. Lesions of orbitofrontal cortex and basolateral
amygdala complex disrupt acquisition of odor-guided discriminations and
reversals. Learn Mem, 10, 129-40.
SCHOENBAUM, G. & SHAHAM, Y. 2008. The role of orbitofrontal cortex in drug
addiction: a review of preclinical studies. Biol Psychiatry, 63, 256-62.
SIMON, N. W., MENDEZ, I. A. & SETLOW, B. 2007. Cocaine exposure causes long-
term increases in impulsive choice. Behavioral Neuroscience, 121, 543-549.
STALNAKER, T. A., ROESCH, M. R., FRANZ, T. M., BURKE, K. A. &
SCHOENBAUM, G. 2006. Abnormal associative encoding in orbitofrontal
neurons in cocaine-experienced rats during decision-making. European Journal of
Neuroscience, 24, 2643-2653.
STEINER, A. P. & REDISH, A. D. 2012. The road not taken: neural correlates of
decision making in orbitofrontal cortex. Frontiers in Neuroscience, 6, 131.
SUL, J. H., KIM, H., HUH, N., LEE, D. & JUNG, M. W. 2010. Distinct roles of rodent
orbitofrontal and medial prefrontal cortex in decision making. Neuron, 66, 449-
460.
TAKAHASHI, Y., ROESCH, M. R., STALNAKER, T. A., HANEY, R. Z., CALU, D. J.,
TAYLOR, A. R., BURKE, K. A. & SCHOENBAUM, G. 2009. The orbitofrontal
xxi
cortex and ventral tegmental area are necessary for learning from unexpected
outcomes. Neuron, 62, 269-280.
TAKAHASHI, Y., ROESCH, M. R., STALNAKER, T. A. & SCHOENBAUM, G. 2007.
Cocaine exposure shifts the balance of associative encoding from ventral to
dorsolateral striatum. Frontiers in Integrative Neuroscience, 1:11. doi:
10.3389/neuro.07/011.2007.
TAKAHASHI, Y. K., CHANG, C. Y., LUCANTONIO, F., HANEY, R. Z., BERG, B.
A., YAU, H. J., BONCI, A. & SCHOENBAUM, G. 2013. Neural estimates of
imagined outcomes in the orbitofrontal cortex drive behavior and learning.
Neuron, 80, 507-18.
TAYLOR, J. R. & HORGER, B. A. 1999. Enhanced responding for conditioned reward
produced by intra-accumbens amphetamine is potentiated after cocaine
sensitization. Psychopharmacology, 142, 31-40.
TAYLOR, J. R. & JENTSCH, J. D. 2001. Repeated intermittent administration of
psychomotor stimulant drugs alters the acquisition of Pavlovian approach
behavior in rats: differential effects of cocaine, d-amphetamine and 3,4-
methylenedioxymethamphetamine ("ecstasy"). Biological Psychiatry, 50, 137-
143.
TEITELBAUM, H. 1964. A comparison of effects of orbitofrontal and hippocampal
lesions upon discrimination learning and reversal in the cat. Experimental
Neurology, 9, 452-462.
THORPE, S. J., ROLLS, E. T. & MADDISON, S. 1983. The orbitofrontal cortex:
neuronal activity in the behaving monkey. Experimental Brain Research, 49, 93-
115.
TREMBLAY, L. & SCHULTZ, W. 1999. Relative reward preference in primate
orbitofrontal cortex. Nature, 398, 704-708.
VANDERSCHUREN, L. J. M. J. & EVERITT, B. J. 2004. Drug seeking becomes
compulsive after prolonged cocaine self-administration. Science, 305, 1017-1019.
VOLKOW, N. D., CHANG, L., WANG, G. J., FOWLER, J. S., DING, Y. S., SEDLER,
M., LOGAN, J., FRANCESCHI, D., GATLEY, J., HITZEMANN, R.,
GIFFORD, A., WONG, C. & PAPPAS, N. 2001. Low level of brain dopamine
D2 receptors in methamphetamine abusers: association with metabolism in the
orbitofrontal cortex. Am J Psychiatry, 158, 2015-21.
VOLKOW, N. D., FOWLER, J. S., WANG, G. J., HITZEMANN, R., LOGAN, J.,
SCHLYER, D. J., DEWEY, S. L. & WOLF, A. P. 1993. Decreased dopamine D2
xxii
receptor availability is associated with reduced frontal metabolism in cocaine
abusers. Synapse, 14, 169-77.
VOLKOW, N. D., FOWLER, J. S., WOLF, A. P. & GILLESPI, H. 1990. Metabolic
studies of drugs of abuse. NIDA Res Monogr, 105, 47-53.
VOLKOW, N. D., FOWLER, J. S., WOLF, A. P., HITZEMANN, R., DEWEY, S.,
BENDRIEM, B., ALPERT, R. & HOFF, A. 1991. Changes in brain glucose
metabolism in cocaine dependence and withdrawal. Am J Psychiatry, 148, 621-6.
VOLKOW, N. D., VALENTINE, A. & KULKARNI, M. 1988. Radiological and
neurological changes in the drug abuse patient: a study with MRI. J Neuroradiol,
15, 288-93.
WALLIS, J. D. 2007. Orbitofrontal cortex and its contribution to decision-making.
Annual Review of Neuroscience, 30, 31-56.
WALLIS, J. D. 2012. Cross-species studies of orbitofrontal cortex and value-based
decision-making. Nat Neurosci, 15, 13-9.
WANG, M., GAMO, N. J., YANG, Y., JIN, L. E., WANG, X. J., LAUBACH, M.,
MAZER, J. A., LEE, D. & ARNSTEN, A. F. 2011. Neuronal basis of age-related
working memory decline. Nature, 476, 210-213.
WEISKRANTZ, L. 1956. Behavioral changes associated with ablations of the
amygdaloid complex in monkeys. Journal of Comparative Physiology and
Psychology, 9, 381-391.
WEST, E. A., DESJARDIN, J. T., GALE, K. & MALKOVA, L. 2011. Transient
inactivation of orbitofrontal cortex blocks reinforcer devaluation in macaques. J
Neurosci, 31, 15128-35.
WIED, H. M., JONES, J. L., COOCH, N. K., BERG, B. A. & SCHOENBAUM, G. 2013.
Disruption of model-based behavior and learning by cocaine self-administration
in rats. Psychopharmacology (Berl), 229, 493-501.
WIMMER, G. E. & SHOHAMY, D. 2011. The striatum and beyond: hippocampal
contributions to decision making. In: DELGADO, M., PHELPS, E. A. &
ROBBINS, T. W. (eds.) Attention and Performance XXII. Oxford, UK: Oxford
University Press.
WIMMER, G. E. & SHOHAMY, D. 2012. Preference by association: How memory
mechanisms in the hippocampus bias decisions. Science, 338, 270-273.
WINSTANLEY, C. A., OLAUSSON, P., TAYLOR, J. R. & JENTSCH, J. D. 2010.
Insight into the relationship between impulsivity and substance abuse from studies
xxiii
using animal models. Alcoholism, Clinical and Experimental Research, 34, 1306-
1318.
WYVELL, C. L. & BERRIDGE, K. C. 2001. Incentive sensitization by previous
amphetamine exposure: increased cue-triggered "wanting" for sucrose reward.
Journal of Neuroscience, 21, 7831-7840.
ZAPATA, A., MINNEY, V. L. & SHIPPENBERG, T. S. 2010. Shift from goal-directed
to habitual cocaine seeking after prolonged experience in rats. Journal of
Neuroscience, 30, 15457-15463.