CNTRICS Final Task Selection: Long-Term Memory · 2012-04-06 · Table 1. Long-Term Memory in Schizophrenia Relational encoding and retrieval: The processes involved in memory for

CNTRICS Final Task Selection: Long-Term Memory

JohnD.Ragland1,2,RoshanCools3,Michael Frank4,DiegoA. Pizzagalli5,6, Alison Preston7, Charan Ranganath8, andAnthony D. Wagner9

2Department of Psychiatry and Behavioral Sciences, UC DavisImagingResearchCenter,University ofCalifornia atDavis, 4701XStreet, Sacramento, CA 95817; 3Neuroscience Community,University of Cambridge; 4Department of Psychology, Universityof Arizona; 5Department of Psychology, Harvard University;6Stanford University; 7Department of Psychology and Center forLearning andMemory,University ofTexas atAustin; 8Departmentof Psychology, University of California at Davis; 9Department ofPsychology and Neurosciences Program, Stanford University

Long-term memory (LTM) is a multifactorial construct,composed of different stages of information processingand different cognitive operations that are mediated by dis-tinct neural systems, some of which may be more respon-sible for the marked memory problems that limit the dailyfunction of individuals with schizophrenia. From the outsetof the CNTRICS initiative, this multidimensionality wasappreciated, and an effort was made to identify the specificmemory constructs and task paradigms that hold the mostpromise for immediate translational development. Duringthe second CNTRICS meeting, the LTM group identifieditem encoding and retrieval and relational encoding and re-trieval as key constructs. This article describes the processthat the LTM group went through in the third and finalCNTRICS meeting to select nominated tasks within the2 LTM constructs and within a reinforcement learning con-struct that were judged most promising for immediate de-velopment. This discussion is followed by each nominatingauthors’ description of their selected task paradigm, endingwith some thoughts about future directions.

Key words: episodic memory/schizophrenia/relationalmemory/item memory

Introduction

From the outset, the CNTRICS initiative appreciatedthat long-term memory (LTM) is a broad and multidi-mensional construct, encompassing multiple stages of in-

formation processing and engaging distinct neuralsystems, some of which are likely to be more central tothememory problems that limit the daily function of indi-viduals with schizophrenia. A great deal of this under-standing grew from clinical research employingtraditional neuropsychological memory tasks such asthe California Verbal Learning Test1 andWechslerMem-ory Scale.2 However, unlike the previous MATRICS ini-tiative3 that focused on these neuropsychologicalmeasures, the goal of the CNTRICS initiative was toidentify tasks from the cognitive neuroscience worldthat hold promise for translational development fordrug discovery.4 Accordingly, during the second of 3CNTRICS meetings, 2 LTM domains5 were identifiedas the most promising constructs for immediate transla-tional development: (1) relational encoding and retrieval,defined as ‘‘the processes involved in memory for stimuli/elements and how they were associated with coincidentcontext, stimuli or events’’ and (2) item encoding andretrieval, defined as ‘‘the processes involved in memoryfor individual stimuli or elements irrespective of contem-poraneously presented context or elements.’’ The LTMgroup was also assigned the construct of reinforcementlearning, defined as ‘‘acquired behavior as a functionof both positive and negative reinforcers including theability to (a) associate previously neutral stimuli withvalue, as in Pavlovian conditioning; (b) rapidly modifybehavior as a function of changing reinforcement contin-gencies; and (c) slowly integrate over multiple reinforce-ment experiences to determine probabilistically optimalbehaviors in the long run.’’ At the end of the secondmeet-ing, a call went out to the scientific community to engagein an online submission process to nominate tasks thatassess these 3 constructs to be considered for ongoingdevelopment.As part of the nomination process, scientists were

asked to provide evidence for each task’s construct val-idity, link to neural circuits, clarity of cognitive mecha-nisms, availability of an animal model, link to neuralsystems through neuropsychopharmacology, amenabil-ity for use in neuroimaging, evidence of impairment inschizophrenia, and psychometric characteristics. In thethird CNTRICS meeting, the LTM breakout groupwas asked to select the 2 most promising tasks withineach construct based on these same criteria, with an un-derstanding that although all tasks may not meet all

1To whom correspondence should be addressed; tel: 916-734-5802, fax: 916-734-8750, e-mail: [email protected].

Schizophrenia Bulletin vol. 35 no. 1 pp. 197–212, 2009doi:10.1093/schbul/sbn134Advance Access publication on October 16, 2008

� The Author 2008. Published by Oxford University Press on behalf of the Maryland Psychiatric Research Center. All rights reserved.For permissions, please email: [email protected].

197

requirements (eg, animal model, psychometric character-istics), tasks without clear evidence of construct validityand link to a neural circuit should not be given furtherconsideration. The purpose of this article is to reporton the outcome of this deliberation process and providethe reader with the nominating authors’ description ofthe (1) item encoding and retrieval, (2) relational encod-ing and retrieval, and (3) reinforcement learning tasksthat were judged to be ready for immediate translationaldevelopment.

For the construct of relational encoding and retrieval,2 tasks—associative inference paradigm (AIP) and rela-tional and item encoding and retrieval (RIER)—werejudged ready for further development and will be de-scribed below. The third nominated task was a transitiveinference paradigm (TIP). Group members wereimpressed with TIP’s link to neural circuits, availabilityof an animal model, link to neuropsychopharmacology,and evidence of impairment in schizophrenia. However,the greater complexity of the TIP vs AIP resulted ina somewhat lower score for construct validity and ledto a decision to select the AIP over the TIP for immediatedevelopment. Two tasks were considered for the itemencoding and retrieval construct—RIER and inhibitionof current irrelevant memories task. Of these, only theRIER (which assesses both item and relational memory)was chosen. The nominating author’s acknowledgementthat the inhibition of current irrelevant memories taskhas ‘‘unknown’’ construct validity precluded it from fur-ther consideration. Within the reinforcement learningconstruct, the complementary nature of several of thenominated tasks lead the working group to recommend3 tasks for further development—the probabilistic re-ward task, the probabilistic selection task, and the prob-abilistic reversal learning task. The weather predictiontask was the fourth task nominated. Because the nomi-nating author described 3 different possible learningstrategies, questions arose about the task’s construct val-idity and it did not receive further consideration. The rec-ommended tasks are shown in table 1. Below are thenominating authors’ descriptions of the selected taskswithin each of the 3 LTM constructs.

Relational Encoding and Retrieval

Associative Inference Paradigm

Description. Relational representations bind distinctelements of an event into a memory representationthat captures the relationships between the elements.6,7

Relational representations are thought to underlie mne-monic flexibility that allows for the generative use ofstored knowledge about elements of experience to ad-dress new questions posed by the environment. TheAIP provides a means to examine mnemonic flexibilityand the nature of the relational representations that sup-port the use of memory in novel situations.

In the AIP, participants receive explicit training on 2sets of paired associates (eg, AB and BC) and are thentested on whether they can infer from these associationsthe relationship between A and C. Specifically, partici-pants learn an initial set of AB associations, whereeachAmight consist of a unique face and each B a uniquehouse (figure 1a). Then, participants learn an overlappingset of associations consisting of the same B stimuli (eg,the same houses) paired with a new set of C stimuli(eg, another unique set of faces). Thus, during learning,each B stimulus (a house) is associated with 2 differentstimuli, A and C (2 unique faces), though the A and Cstimuli are not directly experienced together.During a subsequent memory test, participants make

2-alternative forced-choice judgments that depend onmemory for the learned associations (AB and BC) andthe inferential relationship between A and C. For alltest trials, the incorrect choice item (foil) is a stimulusthat had been studied in another pairing, ensuring thatthe 2-choice stimuli are equally familiar. This aspect ofthe design provides construct validity because perfor-mance requires memory for the relations between stimulirather the memory for individual items. That is, memoryjudgments for trained pairs (AB, BC) cannot be deter-mined from stimulus familiarity and must be made basedon learned associations. Similarly, for inferential pairs(AC) whose relationship was not studied, participantsmust retrieve the relation between the A and C stimulithat emerges from the overlapping associations withthe same B stimulus.

ConstructValidity. The logic of the AIP rests on the the-oretical argument that relational representations sepa-rately code elements of an event, maintaining the

Table 1. Long-Term Memory in Schizophrenia

Relational encoding and retrieval: The processes involved inmemory for stimuli/elements and how they were associated withcoincident context, stimuli, or events.

Recommended for immediate development:(1) Associative inference paradigm(2) Relational and item encoding and retrieval (RIER) task

Item encoding and retrieval: The processes involved in memory forindividual stimuli or elements irrespective of contemporaneouslypresented context or elements.

Recommended for immediate development:(1) RIER task

Reinforcement learning: Acquired behavior as a function of bothpositive and negative reinforcers including the ability to (a)associate previously neutral stimuli with value, as in Pavlovianconditioning; (b) rapidly modify behavior as a function ofchanging reinforcement contingencies; and (c) slowly integrateover multiple reinforcement experiences to determineprobabilistically optimal behaviors in the long run.

Recommended for immediate development:(1) Probabilistic reward task(2) Probabilistic selection task(3) Probabilistic reversal learning task

J. D. Ragland et al.

198

compositionality of the elemental representations and or-ganizing them in terms of their relations to one an-other.6,7 The compositional nature of these relationsallows for reactivation of representations from partial in-put (pattern completion),8,9 a process thought to underlieevent recollection. Further, maintenance of the composi-tionality of elements in relational representations allowsflexible use of learned information at retrieval, making itpossible to infer relations across stimuli without explicittraining. This flexibility is important for associative infer-ence decisions that putatively require inference to solvenovel (untrained) stimulus configurations at test (AC re-trieval decisions). However, relational representationsmay also contribute to inferential decisions throughpattern completion at encoding (D. Shohamy andA. D. Wagner, unpublished data).9,10 When learningan overlapping set of pairs, AB and BC, pattern comple-tion to AB during presentation of BC would enable anAC relation to be encoded. At test, inferential logic isnot required to make an AC decision because a storedAC relation exists and can be directly retrieved.

Neural Systems. Neuroanatomical and computationalmodels have proposed that the medial temporal region,and the hippocampus, in particular, is essential for pat-tern completion processes that support performance inthe AIP.9,11,12 Recent neuroimaging work with humanshas revealed activation in anterior hippocampal regionsat retrieval during inferential judgments relative to explic-itly learned associations (figure 1b).13

Pharmacological and BehavioralManipulation. Data onpharmacological manipulation effects on the AIP are notyet available. Behavioral variation in performance hasbeen observed in the AIP, with awareness of the overlap-ping relationships between stimuli during encoding per-haps being essential to inferential performance at test.For example, participants who are explicitly informedabout the overlapping relationships between stimuli priorto learning or who spontaneously acquire such awareness

during learning perform more accurately on AC judg-ments than uninformed or unaware subjects.14

Animal Models. Complementary animal work hasdocumented impaired associative inference judgmentsfollowing hippocampal lesion. In rats, lesions of hippo-campus proper impair AC judgments without disruptingthe ability to encode or retrieve the explicitly trained asso-ciations (AB and BC).15

Performance in Schizophrenia. How schizophreniaaffects performance in the AIP is currently unknown.However, in the hierarchical TIP, schizophrenia impairsrelational judgments (BD) with performance on nonrela-tional judgments (AE) remaining intact.16 Functionalneuroimaging has further demonstrated that impairedmemory performance in schizophrenia during consciousrecollection17 and the hierarchical inference task18 is as-sociated with decreased activation in hippocampalregions.

PsychometricData. These data are not yet available forthe AIP.

Future Directions. The AIP has a clearly defined con-struct and good theoretical work done on neural systems.Future studies should build psychometric evidence oftest-retest reliability, extend examination of the AIP toschizophrenia populations, and increase examinationof pharmacological affects on performance. In particular,medial temporal lobe regions that are thought to supportrelational memory performance in the AIP receive inputfrom and provide feedback to dopamine (DA) releasingneurons in the midbrain that are associated with rewardand motivation.19 Alteration in medial temporal lobe-midbrain interactions may exist in schizophrenia giventhe abnormal transmission of DA observed in the disease,and these alterations may have important implicationsfor relational memory function. Medications used totreat schizophrenia may also influence interactionsbetween medial temporal lobe structures and midbrainDA regions thus impacting memory. Determining howmedication affects medial temporal lobe function andperformance in the AIP may yield new insights into dis-ease treatment.

Item Encoding and Retrieval and Relational Encoding andRetrieval

Relational and Item Encoding and Retrieval Task

Description. The RIER task combines 2 paradigms pre-viously used to study item-specific and relational episodicmemory in schizophrenia. The first is a levels-of-processing(LOP) task designed to control for group differences initem-specific encoding strategies and, thereby, generateequivalent recognition and source memory performance

Fig. 1. Associative Inference Paradigm. a) Participants encodeoverlapping face-house pairs (AB, BC) and are tested on theinferential relationship between pairs (AC). b) Anteriorhippocampal activation associated with inferential retrieval of ACpairs.13 Reprinted with Permission from Wiley 8/27/08.

199

CNTRICS Final Task Selection

between schizophrenia patients and controls.20,21 The sec-ond is a relational encoding task22 that produces a signifi-cant recognition deficit in patients with schizophrenia.During the encoding phase of RIER, participants are pre-sented with 2 trial types in separate blocks. During ‘‘item-specific’’ encoding blocks, participants are presented witha series of trials in which they are shown a single object andasked to rate whether it is pleasant or unpleasant. During‘‘relational’’ encoding blocks, participants are presentedwith a series of trials in which 3 objects are shown andtheymust judgewhether the objects are in the correct orderin terms of weight (from lightest to heaviest). In each studyblock,participants encode12objects, anda totalof3blocksarecompletedforeachencodingcondition.Thesequenceofencoding blocks is counterbalanced to minimize ordereffects. During the retrieval phase of the task, participantsfirst complete a yes/no item recognition test consisting ofa random sequence of 72 previously studied objects (36fromitemspecificand36 fromrelational)and72previouslyunseen foil objects. Next, participants are given an associa-tive recognition test consisting of objects that were previ-ously studied on relational trials. The test includes 18‘‘intact’’ pairs consisting of objects that were originallystudied on the same trial and 18 ‘‘recombined’’ pairs con-sisting of objects that were originally studied on differenttrials. Subjects are asked to indicate if the pairs are intactor rearranged.

Construct Validity. Behavioral research has distin-guished between item-specific and relational encodingstrategies.22–24 Common item-specific encoding strategiesinvolve making a semantic decision about an item (eg,‘‘pleasant’’/‘‘unpleasant,’’ ‘‘abstract’’/‘‘concrete’’),whereasrelational encoding strategies include imagining 2 or moreitems interacting or linking 2 or more words in the contextof a sentence or story. It is thought that relational encodingpromotes memory for associations among items, whereasitem-specific encoding enhances the distinctiveness of spe-cific item.23–26 In the episodicmemory literature, relationalencoding has been linked to the function of the hippocam-pus, which is thought to support the binding of novel rep-resentations.27–29 The distinction between relational vsitem-specific encoding has also been supported by neuroi-maging studies of working memory (WM) that haverevealed dissociations between brain regions involved initem-specificWMmaintenanceandregions involved inma-nipulation of relationships fbetween items while they arebeing maintained. Research has shown that dorsolateralprefrontal cortex (DLPFC) is selectively activated on trialsin which relationships among items are processed.30More-over, engagement of the DLPFC during relational WMprocessingpredicts successfulLTMretrieval.22,31,32Severalstudies have shown that although both relational and item-specific encoding tasks are effective, they tend to have dif-ferent effects on memory performance.23–26 For example,relational encoding is optimal when memory for associa-

tionsbetween itemswillbe tested(eg,pairedassociate learn-ing), whereas item-specific encoding is optimal whenmemory for item details is tested (R. S. Blumenfeld andC. Ranganath, unpublished data). The available evidencetherefore indicates that the construct of relational encodingand retrieval has validity at both the cognitive and neurallevel of analysis and that it is supported by both hippocam-pal and DLPFC-mediated mechanisms.In prior work,22 it has been shown that the associative

portion of the RIER task promotes LTM by buildingassociations between triplets of words, whereas this evi-dence was not seen for a control task that involved pas-sive rehearsal of words. Preliminary results (see below)suggest that performance is impaired in patients relativeto controls. In contrast, item-specific semantic encodinghas been shown to improve memory by facilitatingencoding of distinctive item-specific information becauseit does not encourage building of relationships amongitems.24 As described below, item-specific semanticencoding tasks promote robust levels of memory perfor-mance in patients with schizophrenia.20,33,34

Neural Systems. Several recent studies have demon-strated that DLPFC activation during relational encod-ing reliably predicts successful LTM.22 However,DLPFC activity is generally not correlated with success-ful item-specific encoding (see Blumenfeld and Ranga-nath 31 for review). For example, in a recent studyfrom Dr Ranganath’s labortory22 using a variant ofthe relational encoding task used in the RIER paradigm,participants were scanned while performing the 2 WMtasks (figure 2a). On ‘‘rehearse’’ trials, subjects were re-quired to rehearse a set of 3 words across a 12-second de-lay period, whereas on ‘‘reorder’’ trials, participants wererequired to rearrange a set of 3 words based on the weightof the object that each word referred to over the delay.Although both conditions required maintenance acrossthe delay, reorder trials also required participants to eval-uate relationships between items in the memory set alonga single dimension (weight). Analyses of subsequentLTMperformance showed significantly more reorder trials inwhich all 3 items were recollected than would be expectedbased on overall item hit rates alone (figure 2b), but thesame was not true for rehearse trials. This result suggeststhat, on reorder trials, participants successfully encodedrelationships between the items in each memory set.Consistent with the idea that the DLPFC is involved inrelational processing in WM, DLPFC activation was in-creasedduring reorder trials comparedwith rehearse trials(figure 2c). Furthermore, DLPFC activation during reor-der, but not rehearse trials, was positively correlated withsubsequent LTM performance. No such relationship wasevidentduring rehearse trials. In contrast, activation in theleft ventrolateral prefrontal cortex (VLPFC) (Broadmannarea 44/6) and in the hippocampus was correlated withsubsequent memory performance on both rehearse and

200


reorder trials. Results from this study andothers32 suggestthat theDLPFCmay be specifically recruited during rela-tional encoding, adding support to the validity of thisneural construct.

Pharmacological and BehavioralManipulation. Data onpharmacological manipulation effects on the RIER arenot yet available. As noted above, when patients are pro-vided with an item-specific encoding strategy, there areno longer group differences in item recognition andsource retrieval performance20,21 or in functional mag-netic resonance imaging (fMRI) activation in theVLPFC.33,34

Animal Models. The kinds of relational encoding pro-cesses that are manipulated in the RIER paradigm havenot been extensively investigated in animal models, inpart because it is difficult to directly manipulate encod-ing strategies in nonhuman animals. Some relevant ev-

idence, however, comes from studies of WM tasks inmonkeys. For example, a single-unit recording study35

showed that neurons in the monkey dorsal prefrontalcortex encoded information about temporal order rela-tionships between a series of items presented in a WMtask. In contrast, ventral prefrontal neurons tended toencode the physical features of objects to be maintained.Another study demonstrated that lesions to mid-DLPFC impaired memory for sequences of actions.36

This prior work demonstrates the feasibility of investi-gating associative memory in animal models althoughdirect translation of the RIER to rodent models maynot be feasible, in which case nonhuman primate modelsmay be more appropriate.

Performance in Schizophrenia. Research by Dr Raglandand others has revealed consistent evidence of episodicmemory deficits in schizophrenia linked to impairedorganizational processes. During initial experiments,

Fig. 2. Results From Blumenfeld and Ranganath.22 a) Example stimuli and task timing for working memory trials. b) Difference betweenobserved and expected numbers of recollected triplets from eachmemory set. Themean difference between the observed number of trials forwhichall 3wordswere successfully judgedas rememberedand the expectednumberof such trials given theoverall hit rate is separatelyplottedfor reorder and rehearse trials. A positive difference indicated that subsequent memory performance was benefited by enhanced inter-itemassociations. Error bars depict the SEM across subjects, and the asterisk denotes that the observed expected difference was statisticallysignificant forreorder trials. c)Timecourseofactivation inprefrontal regionsof interest (ROIs).Theactivity in the reorderandrehearse task isplotted separately for the left dorsolateral prefrontal cortex (DLPFC), andanterior ventrolateral prefrontal cortex (aVLPFC)was correlatedwith subsequent LTM performance specifically during reorder trials. In contrast, delay period activation in the posterior ventrolateralprefrontal cortex (pVLPFC)was predictive of subsequent LTMonboth rehearse and reorder trials. The error bars in the time courses reflectthe SEM at each time point for the reorder and rehearse tasks for each ROI. Reprinted with permission from Society for Neuroscience.

201


subjects were studied with explicit word list encodingtasks where no strategy was provided.37,38 Duringdebriefing, controls were more likely to engage in item-specific semantic processing, and we suspected thatpatients were employing a less effective strategy. A lev-els-of-processing paradigm39 tested whether providinga semantic, item-specific encoding strategy could im-prove patients’ performance and prefrontal function.As predicted, patients showed the same benefit as healthycontrols from item-specific semantic processing on bothrecognition20,34 and source memory tasks21 and showedrobust activation in the ventrolateral portions of the pre-frontal cortex.33,34 These results suggest that item-specificencoding processes may be relatively spared comparedwith relational encoding and motivate inclusion of a se-mantic item-specific encoding condition in the RIERparadigm to address issues of generalized deficit.

In contrast, pilot data from Dr Ragland and DrRanganath using the relational encoding conditionfrom the RIER task suggest that when individualswith schizophrenia are provided with a relational encod-ing strategy, they may perform more poorly than con-trols (figure 3). Specifically, preliminary data froma sample of 15 patients with schizophrenia (8 females)and 16 demographically matched healthy volunteersshowed main effects of group [F(1,29) = 7.04], p <.05,task [F(1,29) =13.7, p < .001], and a task by group in-teraction [F(1,29) =5.06, p < .05] on recognition accu-racy. As can be seen in Figure 3, this interaction wasdue to patients having worse recognition accuracy rela-tive to healthy volunteers on the relational memory re-order condition [F(1,29) =10.0, p < .005], but not on theitem-specific rehearse condition [F(1,29) =3.84, p < .06].

PsychometricData. These data are not yet available forthe RIER.

Future Directions. The RIER task has well-establishedconstruct validity, identification of specific prefrontalcognitive control systems underpinning task perfor-mance, and preliminary evidence for a relative deficitin relational vs item-specific encoding and retrieval per-formance in schizophrenia. Improved understanding ofthe specific role that the hippocampus and MTL playin task performance, development of animal models,and building psychometric evidence of test-retest reliabil-ity were identified as important future directions. Al-though relational memory can be examined in animalmodels, direct translation of the RIER to animal modelswill require directly manipulating encoding strategies.Meeting this goal may be more feasible in nonhuman pri-mates then in rodent models. As with all LTM tasks,establishing adequate test-retest reliability is also a chal-lenge given the likelihood of substantial practice effectson task performance. This may necessitate developmentof parallel forms of the RIER.

Reinforcement Learning

Probabilistic Reward Task

Description. This task is based on a differential rein-forcement schedule that provides an objective assessmentof participants’ propensity to modulate behavior asa function of reward history.40 The task, which was mod-ified from an earlier paradigm described by Tripp andAlsop,41 is rooted within the behavioral model of signaldetection42 and the generalized matching law.43,44

Figure 4 provides an illustration of the probabilistic re-ward task (adapted from Pizzagalli et al40). The taskincludes 300 trials, divided into 3 blocks of 100 trials,which are separated by a 30-second break. A trial startswith the presentation of an asterisk for 1400milliseconds,immediately followed by a schematic mouthless cartoonface presented for 500 milliseconds. Next, either a short(11.5 mm) or long (13 mm) mouth is briefly presented onthe screen for 100 milliseconds. The mouthless faceremains visible until the participant makes a response.For each trial, participants are asked to determine whichmouth stimulus was presented by pressing either the ‘‘z’’key or the ‘‘/’’ key on a PC keyboard (counterbalancedacross subjects). For each block, the 2 mouth stimuliare presented equally often using a pseudorandomized se-quence allowing up to 3 consecutive presentations of thesame stimulus. Within each block, only 40 correct trialsare followed by reward feedback (eg, ‘‘Correct!! You won5 cents’’), presented for 1500milliseconds immediately af-ter a correct response. If a reward feedback is presented,an additional blank screen is presented for 250 millisec-onds. For nonrewarded trials, a blank screen is presentedfor 1750 milliseconds.

Fig. 3. Recognition accuracy of controls (blue circles) and patients(red triangles) for rehearse and reorder tasks. Error bars depict theSEM across subjects, and the asterisk denotes a significant groupdifference for the reorder but not rehearse task.

202


Critically, an asymmetric reinforcer schedule is used toinduce a response bias.45 Thus, correct identification ofone mouth (‘‘rich stimulus’’) is rewarded 3 times morefrequently than correct identification of the other mouth(‘‘lean stimulus’’). Only 40 correct trials are rewarded ineach block (30 rich, 10 lean) to ensure that each partic-ipant is exposed to the same (or a very similar) number ofrewards. To achieve this goal, a controlled reinforcerprocedure is used: if a participant makes an incorrectresponse on a trial scheduled to be rewarded, the feed-back is delayed until the next correct response of thesame stimulus.Before the task, participants are instructed that the

goal of the task is to win as much money as possibleand that not all correct responses will receive a rewardfeedback. Importantly, participants are not informedthat one of the stimuli will be rewarded more frequently.Note that due to the probabilistic nature of the task, par-ticipants cannot infer which stimulus is more advanta-geous based on the outcome of a single trial; instead,in order to optimize their choices, participants need to‘‘integrate’’ reinforcement history over time. Dependingon the monetary reward used for each trial, participantsearn approximately $6 (5 cent per trial)40 or approxi-mately $24 (20 cent per trial).46

The main variable of interest is response bias, whichcan be computed41,42 as

log b=1

2log

�Richcorrect3Leanincorrect

Richincorrect3Leancorrect

�:

As evident from the formula, a high-response biasemerges when participants tend to correctly identifythe stimulus associated with more frequent rewards(rich hits) and to misclassify the lean stimulus (leanmisses). To examine general task performance, secondaryanalyses consider hit rates [(number of hits)/(number ofhits þ number of misses)], reaction time, and discrimina-bility. Discriminability, which assesses the subjects’ abil-

ity to perceptually distinguish between the stimuli andcan thus be used as an indication of task difficulty, iscomputed as

log d =1

2log

�Richcorrect3Leancorrect

Richincorrect3Leanincorrect

�:

In addition to these variables, the probability of spe-cific responses as a function of the immediately preced-ing trial can be computed to evaluate the strength ofa response bias as a function of (a) which stimulushad been rewarded in the preceding trial and (b) prox-imity of reward delivery. For example, the probabilityof selecting ‘‘rich’’ or ‘‘lean’’ in trials immediatelyfollowing a correctly identified, rewarded rich trial vsa correctly identified, nonrewarded rich trial may becomputed.47 Finally, in several studies, we have foundthat reward learning, which can be measured bysubtracting response bias in block 1 from responsebias in block 3, showed strong construct and predictivevalidity.40,47

Construct Validity. Initial construct validity comesfrom studies evaluating samples hypothesized to be char-acterized by dysfunctional reinforcement learning.48,49

Subjects with elevated depressive symptoms,40 unmedi-cated patients with major depressive disorder(MDD),50 and medicated euthymic patients with bipolardisorders47 showed reduced response bias toward themore frequently rewarded stimulus (figure 5a). More-over, trial-by-trial probability analyses revealed thatMDD subjects were impaired at expressing a responsebias toward the more frequently rewarded cue in the ab-sence of immediate reward (manifested as increased missrates), whereas they were responsive to delivery of singlerewards. Increased miss rates for the more frequentlyrewarded stimulus correlated with anhedonic symptoms(r = 0.52, P < .05), even after considering anxiety symp-toms and general distress.50

Neural Systems. Compared with learners, nonlearnersshowed significantly lower activation in response to re-ward feedback in dorsal anterior cingulate cortex(dACC) regions that have been previously implicatedin integrating reinforcement history over time.51,52

This result is consistent with the hypothesis that non-learners have blunted reinforcement sensitivity. More-over, the ability to develop a response bias towardthe more frequently rewarded stimulus correlatedwith dACC activation (r = 0.40, P < .030). A final fea-ture of this study was that some of the participants per-formed a monetary incentive delay (MID) task53,54

during fMRI. Relative to nonlearners, learners showedlarger basal ganglia responses to reward feedback (mon-etary gains) in theMID task. These findings suggest thatparticipants developing a response bias toward the more

Fig.4.SummaryofTaskDesign.Ateachtrial,participantsareaskedto select via bottom press whether a short or long mouth had beenpresented. Figure modified with permission from Pizzagalli et al.40

Reprinted with Permission from Elsevier.

203


frequently rewarded stimulus had stronger dACC andbasal ganglia responses to reward outcomes.

Pharmacological and Behavioral Manipulation. Twostudies have shown that response bias in the probabilisticreward task is modulated by pharmacological manipula-tions affecting DA either directly46 or indirectly.55 In thefirst study, a single 0.5 mg dose of the D2/3 agonist pra-mipexole or placebo was administered to healthy volun-teers 2 hours before performing the task.46 Consistentwith predictions based on animal evidence,56–58 prami-pexole impaired reinforcement learning (see Frank andO’Reilly59 for a prior human study postulating similareffects). Compared with placebo, subjects receiving pra-mipexole showed lower response bias toward the more

frequently rewarded stimulus (figure 5b).46 Control anal-yses confirmed that reduced response bias was not due totransient adverse effects.The aim of a second pharmacological study was to test

the hypothesis that nicotine might increase responsive-ness to reward-related cues,55 based on prior findingsthat nicotine increases appetitive responding through ac-tivation of presynaptic nicotinic receptors on mesocorti-colimbic DA neurons in animals,60,61 and increases theincentive value of monetary reward in humans.62 Usinga randomized, double-blind, placebo-controlled cross-over design, Barr et al55 administered a single dose oftransdermal nicotine (7–14 mg) to 30 psychiatricallyhealthy adult nonsmokers. Nicotine increased responsebias toward the more frequently rewarded stimulus,and this effect persisted over time, as demonstrated bya greater response bias during the placebo session in par-ticipants who received nicotine in the first compared withthe second session (1 week later).

AnimalModels. In collaborationwithAthinaMarkou atUniversity of California at San Diego, Dr Pizzagalli isdeveloping a task analogous to the human probabilisticreward task for use in rodent studies.

Performance inSchizophrenia. In a recent study, Heereyet al63 used the probabilistic reward task in a sample of 40clinically stable and medicated outpatients with schizo-phrenia. The authors found that, compared with healthycontrols, patients with schizophrenia had a reduced abil-ity to discriminate between the stimuli but a similar re-sponse bias. The authors concluded that schizophreniais characterized by intact sensitivity to reward and abilityto modify responses based on reinforcements. Althoughintriguing, the interpretation of these findings is some-what difficult because all patients were medicated atthe time of testing. Moreover, no information was pro-vided about the smoking status of participants, in partic-ular whether patients and controls were matched for thisvariable. In light of the recent finding that nicotineenhances response bias in the probabilistic rewardtask,55 and given high rates of smoking in schizophre-nia,64,65 it is unclear whether the null findings reportedin63 might be partially due to group differences in smok-ing status and/or history.

Psychometric Data. The test-retest reliability of theprobabilistic reward task over approximately 38 dayswas r = 0.57, P < .004. Satisfactory test-retest reliabilityof reward learning (r = 0.56 over an averaged period of 39days period) also emerged in an independent sample.66 Ina recent study evaluating monozygotic (n = 20) and dizy-gotic (n = 15) twin pairs, the heritability of reward respon-siveness was estimated to be 48%.67 Due to the limitedsample size of this twin study, these heritability estimatesshould be considered preliminary.

Fig. 5. Selected Findings Derived From the Probabilistic RewardTask. Response bias toward themore frequently rewarded stimulusis reduced in (a) unmedicated major depressive disorder subjects;50

(b) healthy controls receiving a single dose of a D2/3 agonistassumed to activate dopamine (DA) autoreceptors and thus reducephasic DA bursts to unpredictable reward;47 and (c) healthycontrols exposed to an acute stressor.67 A and C reprinted withpermission from Elsevier. B reprinted with permission fromSpringer Science and Business Media.

204


Future Directions. As noted above, the probablistic re-ward task has a growing body of evidence for its con-struct validity and neural bases. However, there areseveral important avenues for future development ofthis task. First, additional work is needed to determinethe degree to which performance on this task is sensitiveto reward learning deficits in schizophrenia. The pub-lished study on schizophrenia performance on this taskdid not reveal deficits in reward sensitivity. If additionalwork confirms the integrity of reward processing in thistask in schizophrenia, this may not be a particularly use-ful task for schizophrenia research (though it may still bevery valuable for work on mood disorders or other dis-orders with impaired reward sensitivity). Second, the cur-rent version of the task is quite long (300 trials) thatreduces the feasibility of using it in clinical trials settings.Thus, one important future direction for the developmentof this task is to determine whether a shorter version (eg,only 1–2 blocks) would have sufficient sensitivity and re-liability. Third, the test-retest reliability, while good foran experimental task, is overall less than would be opti-mal for a task to be used in clinical trials and would ben-efit from efforts to increase reliability.

Probabilistic Selection Task

Description. The probabilistic selection task68,69 meas-ures participants’ ability to learn from positive andnegative feedback by integrating reinforcement probabil-ities over many trials. Three different stimulus pairs (AB,CD, EF) are presented in random order, and participantshave to learn to choose 1 of the 2 stimuli. Feedback fol-lows the choice to indicate whether it was correct or in-correct, but this feedback is probabilistic: In AB trials,a choice of stimulus A leads to positive feedback in80% of trials, whereas a B choice leads to negative feed-back in these trials. CD and EF pairs are less reliable:stimulus C is correct in 70% of trials, while E is correctin 60% of trials. Over the course of training, participantslearn to choose stimuli A, C, and Emore often than B, D,or F. Note that learning to choose A over B could be ac-complished either by learning that choosing A leads topositive feedback or that choosing B leads to negativefeedback (or both). To evaluate whether participantslearn more about positive or negative outcomes of theirdecisions, performance is subsequently probed in a test/transfer phase in which all novel combinations of stimuliare presented and no feedback is provided. Positive feed-back learning is assessed by reliable choice of the mostpositive stimulus A in this test phase, when presentedwith other stimuli (AC, AD, AE, and AF). Negative feed-back learning is assessed by reliable avoidance of themost negative stimulus B when presented with thesame stimuli (BC, BD, BE, and BF). The extent to whichparticipants perform better in choose-A or avoid-B pairshas been associated with a ‘‘Go’’ or ‘‘NoGo’’ learning

bias and is very sensitive to dopaminergic state, manip-ulation, and genetics.59,69–73

In addition to the probabilistic reinforcement learningbiases, the task can also probe other aspects of reinforce-ment-based decision making. For example, the tendencyto rapidly learn from a single instance of reinforcement inthe initial trials of the task is thought to rely on distinctprocess from that involved in integrating feedback prob-abilities over trials.71 Similarly, when faced with noveltest pairs, participants adaptively modulate their re-sponse times in proportion to the degree of reinforcementconflict. High conflict choices involving stimuli with sim-ilar reinforcement probabilities are associated with lon-ger response times than those associated with divergentreinforcement probabilities, a process thought to dependon interactions between dorsomedial frontal cortex andthe subthalamic nucleus.72

Construct Validity. Performance on the probabilistic se-lection task is defined by the ability to choose the probabi-listically most optimal stimulus. Of course, many factorscan contribute to better or worse performance asidefrom reinforcement learning, including attention, motiva-tion, fatigue, WM, etc. However, the main measure ofinterest in the task is within subject (ie, the ability to choosethe most positive stimulus is contrasted with that of avoid-ing the most negative stimulus), thereby controlling foroverall performance levels and specifically assessing thecontribution of reinforcement. This relative positive tonegative feedback learning measure is reliably altered bydopaminergic manipulation in a range of populations,and moreover, similar effects in positive vs negative learn-ing have been observed in other tasks meant to measuresimilar constructs but using different stimuli, motorresponses, and task rules (ie, probabilisticGo/NoGo learn-ing task59 and probabilistic reversal learning task74).

NeuralSystems. Within the task paradigm, positive andnegative feedback learning in this task are thought to relyon striatal D1 and D2 receptors, respectively. Asdescribed above, probabilistic positive and negative feed-back learning are sensitive to dopaminergic manipula-tion. Increases in dopaminergic stimulation, likely inthe striatum, lead to better positive learning but causeimpairments in negative feedback learning.59,71,72 Theseeffects are thought to arise because DA bursts that occurduring positive outcomes75 support ‘‘Go learning’’ viaD1 receptors,68,76 whereas DA dips that occur duringnegative outcomes75 support ‘‘NoGo learning’’ via D2 re-ceptor disinhibition.68,76 Thus, an increase in striatal DA(eg, due to pharmacology) would continually stimulateD2 receptors and effectively block the effects of DAdips needed for NoGo learning.68,76 Conversely, DA de-pletion is associated with relatively better negative feed-back learning but worse positive feedback learning. Atthe individual difference level, genes that control D1

205


and D2 DA function in the striatum are predictive ofprobabilistic positive and negative learning, whereasgenes that control DA function in prefrontal cortexare predictive of rapid trial-to-trial learning from nega-tive feedback.70 Negative feedback learning is also asso-ciated with enhanced error-related negativity (brainpotentials originating from anterior cingulate cortex)76,77

and activation of this same region in functional neuroi-maging.73 Lesions to medial prefrontal cortex are associ-ated with both deficits in the acquisition of reinforcementcontingencies (patients took longer to learn) and im-paired negative feedback learning in the test phase.78

Finally, the subthalamic nucleus (a component withinthe basal ganglia network) is believed to delay responsesduring high conflict decisions. Supporting this claim,deep brain stimulation of the subthalamic nucleus causespremature responding in these high conflict choices.71

Pharmacological and Behavioral Manipulation. As pre-viously noted, the probabilistic selection task is sensitiveto pharmacological manipulation. DA agonists, includ-ing levodopa and D2 agonists, impair negative feedbacklearning in Parkinson patients while sometimes im-proving positive feedback learning.72 In attentiondeficit/hyperactivity disorder, stimulant medications(methylphenidate and amphetamine), which elevate stria-tal DA, improved positive but not negative feedbacklearning.77 In healthy participants, low doses of D2 ago-nists and antagonists, which may act presynaptically tomodulate DA release, predictably alter positive and neg-ative feedback learning.59

Animal Models. There is currently no available animalmodelof theprobabilistic selection task.However,prelim-inary unpublished results from Claudio DaCunha’s labo-ratory in Brazil suggest that rats can learn a reduced formof the taskusingodordiscriminationand2pairsof stimuli.In a related project, Rui Costa and colleagues have devel-opeda forced-choice task requiringmice to learn to chooseand avoid behaviors associatedwith positive and negativetastants.79 Mice with elevated striatal DA levels showedenhanced bias to approach rewarding tastants togetherwith a reduced bias to avoid aversive tastants, similar tothe data reported in humans. In monkeys, striatal D1 re-ceptor blockade abolishes the normal response speedingobserved when a large reward is available (a measure ofGo learning), whereas D2 receptor blockade leads togreater response slowing when smaller than averagerewards are available (a measure of NoGo learning).80

Performance in Schizophrenia. In a preliminary study,patients with schizophrenia showed large deficits in learn-ing the standard version of the probabilistic selectiontask, which uses Japanese Hiragana characters as stim-uli.81 Subsequent testing using verbalizable stimuli (pic-tures of every day objects such as bicycles) showed that

patients can learn the task but show selective deficits inearly acquisition (thought to rely on prefrontal struc-tures), which correlated with their negative symptoms.81

In the test phase, patients showed intact ‘‘NoGo’’ learn-ing but selectively impaired ‘‘Go’’ learning. Further, allthe genetic polymorphisms predictive of learning inthis task are candidate genes for schizophrenia.

PsychometricData. Practice effects for the probabilisticselection task have been assessed in Frank andO’Reilly.59

Different stimuli are used across sessions. On average,participants are faster to learn the task after multiple ses-sions, but this practice does not affect relative positive vsnegative feedback learning.

FutureDirections. Aswith the probabilistic reward task,the probabilistic selection task has good construct validityandgood theoretical and empiricalworkon its neural sub-strates. One issue in regard to construct validity is the de-gree to which the speed of initial learning influences theability to learn from positive vs negative reinforcement.In other words, if fast learners experience relatively littlenegative feedback associated with the ‘‘B’’ stimulus, theymay show reduced ‘‘negative feedback’’ learning. How-ever, this may not reflect a true impairment in learningfrom negative feedback but rather an absence or a reduc-tion in the actual experience of negative feedback duringthe task. Like many paradigms from cognitive neurosci-ence, the current version of the probabilistic rewardtask does not have an immediate animal homologueand is likely too long to be used in standard clinical trials,though it may be amenable to use in early-phase studies.However, work examining the degree to which the taskcould be shortened or simplified, while still retaining con-struct validity, would facilitate its use in clinical trials. Inaddition,more information is neededonall psychometricsaspects of the task, including test-retest reliability, floorand ceiling effects, and practice effects.

Probabilistic Reversal Learning Task

Description. This task was developed by TrevorRobbins and Robert Rogers and first published in Law-rence et al82 and Swainson et al.83 On each trial, subjectsare presented 2 visual patterns (rectangles of coloredstripes; figure 6). These patterns appear in 2 randomlychosen boxes out of 4 possible boxes. The task consistsof 2 stages, starting with a simple probabilistic visualdiscrimination, in which subjects have to make a2-alternative forced-choice between 2 colors. The ‘‘cor-rect’’ stimulus (which is always the first stimulus touched)receives an 80:20 ratio of positive:negative feedback, andthe opposite ratio of reinforcement is given for the ‘‘in-correct’’ stimulus. In the second, reversal stage, the con-tingencies are reversed, without warning, so that thepreviously ‘‘incorrect’’ color is now correct and vice versafor the previously ‘‘correct’’ color.

206


Although all subjects receive a total of 80 trials, a learn-ing criterion of 8 consecutive correct trials is generally im-posed for the purposes of analysis. Main performancemeasures are failure or success at each stage, mean errorsto criterion, and mean latencies. Failure/success rates areanalyzed using the likelihood-ratio method for contin-gency tables.84 Subjects failing stage 1 are excludedfrom subsequent analyses of error rates and latenciesat stage 2. They are included when error rates and laten-cies at stage 1 are analyzed (to obtain a measure of acqui-sition). Measures of perseveration and maintenance canalso be obtained. Consecutive-perseverative responses re-fer to consecutive responses made, directly following re-versal of the reinforcement contingencies, to the colorthat was correct during stage 1 but is now incorrect instage 2. Stage 1-perseverative errors (in the terminologyof Jones and Mishkin85) refer to errors in blocks of 8 tri-als (of the reversal stage), in which performance withinthese blocks falls significantly below chance (�1 correctresponse). Both these perseveration scores tend to corre-late significantly with total errors made in stage 2 of thetask. Therefore, each should be converted to a score in-dicating the proportion of total errors in stage 2. Main-tenance errors refer to the errors made subsequent tocriterion being reached (ie, the number of responses toincorrect stimulus/total trials remaining), subject to therebeing at least 10 trials after criterion is reached.

Construct Validity. The probabilistic reversal learningtask gauges adaptive behavior, which requires anticipa-tion of biologically relevant events, ie, rewards and pun-ishments, by learning signals of their occurrence. Theability to predict events interacts with the ability tostrengthen and weaken actions when these actions areclosely followed by rewards and punishments. These pro-

cesses are often collectively referred to as reinforcementlearning. Models of reinforcement learning use a tempo-ral difference prediction error signal, representing the dif-ference between expected and obtained events, to updatetheir predictions based on states of the environment.86

Probabilistic learning tasks are commonly used to assessreinforcement learning and neural activity associatedwith prediction errors.87–89 For example, O’Dohertyet al88,89 used probabilistic learning tasks to establishthat activity in the (ventral) striatum, and orbitofrontalcortex was positively correlated with the prediction errorsignal.More generally, it should be noted that demands for

reinforcement learning are particularly high in probabi-listic reversal learning paradigms. However, reversallearning constitutes a special case of reinforcement learn-ing, and adequate performance also depends on otherprocesses including prepotent response inhibition andstimulus switching. On a related note, it should be recog-nized that simple reinforcement learning models do notencompass all aspects of probabilistic reversal learning,as implemented in our paradigm. For example, Hamptonet al90 have shown that behavioral performance on neuralactivity (in the ventromedial prefrontal cortex) duringprobabilistic reversal learning was fit better by a modelthat also simulated knowledge of the abstract task struc-ture than by a simple reinforcement learning model thatdid not incorporate such a higher order knowledge aboutinterdependencies between actions.

NeuralSystems. Anumberofneuropsychological studieshave revealed that (probabilistic anddeterministic) reversallearning is disrupted by frontal lobe lesions.91–94 The defi-cits appear restricted to patients with ventromedial frontallesions, do not to extend to patients with dorsolateral pre-frontal lesions, and cannot be attributed to problems withinitial acquisition of stimulus-reinforcement contingencies.Recently, Hornak et al93 observed deficits on the probabi-listic reversal learning task in a small number of patientswith DLPFC lesions, but posttest debriefing revealedthat these patients had failed to pay attention to the crucialfeedbackprovidedon the screen. Ina further study, a groupof patients with frontal variant frontotemporal dementiashowed impairments on the probabilistic reversal learningtask, while showing intact performance on executive func-tions associated with the DLPFC.95

The first fMRI study of this task96 revealed suprathres-hold activity in the VLPFC, anterior cingulate cortex,and posterior parietal cortex during the final reversalerrors relative to the baseline correct responses. A priorihypotheses allowed more focused region of interest anal-yses, which revealed that the activity in the VLPFC wassignificantly larger on the final reversal errors than on theother (eg, probabilistic) errors that did not lead to switch-ing. It should be noted that at a lower statistical threshold(uncorrected for multiple comparisons), the DLPFC was

Fig. 6. Screen Display of the Probabilistic Reversal Learning Task.

207


also active during the final reversal errors. Such addi-tional (albeit relatively less) activity in the DLPFC wasconfirmed in later fMRI studies with this task97–101

and with a slightly adapted version of the task.100 Finally,more recent studies that employed more optimal acqui-sition sequences have also revealed reversal-related activ-ity in the more anterior orbitofrontal cortex.97,100 In ourinitial study, reversal-related activity was also observed inthe ventral striatum, centered on the nucleus accumbens.Reversal-related activity was also observed in the nucleusaccumbens in patients with Parkinson disease (who hadabstained from their medication97). However, the exactcomputation carried by the nucleus accumbens duringprobabilistic reversal learning is still under investigation.

Pharmacological and Behavioral Manipulation. Proba-bilistic reversal learning is sensitive to dopaminergicand serotoninergic manipulations but not to noradren-ergic manipulations in humans. Withdrawal of dopami-nergic medication, such as levodopa and DA receptoragonists, in patients with mild Parkinson disease (PD)improves performance.101 These results are consistentwith Mehta et al,102 who showed that administrationof the DA receptor agonist bromocriptine impaired per-formance on this task in young healthy volunteers whileimproving spatial memory. Additional support for theoverdose hypothesis came from a recent pharmacolog-ical fMRI study, which revealed that dopaminergicmedication in mild PD patients abolished reversal-re-lated activity in the ventral striatum (particularly inthe nucleus accumbens).97 The effect of medicationwas particularly large on final reversal errors that ledto behavioral switching. Interestingly, another pharma-cological fMRI study in healthy volunteers revealed thatadministration of methylphenidate (60 mg oral) alsoabolished reversal-related activity in the ventral stria-tum (particularly in the ventral putamen), an effectthat was again particularly prominent on the final rever-sal errors.98 Discrepancies in the precise localization ofthis effect within the ventral striatum indicate the needfor further work.

Serotoninergic manipulation also affects performanceon the task. However, the nature of the effect is quali-tatively different from that observed after dopaminergicmanipulation. Chamberlain et al103 observed that acuteadministration of citalopram, a selective serotonin reup-take inhibitor, but not atomoxetine, a selective nor-adrenaline reuptake inhibitor, increased the numberof switches after probabilistic errors (ie, misleading pun-ishment). This effect was attributed to a presynapticmechanism of action, paradoxically reducing serotoninlevels. Consistent with this hypothesis, Evers et al104

found that the dietary acute tryptophan depletion pro-cedure, which lowers serotonin synthesis, enhanced neu-ral activity in the dorsomedial prefrontal cortex duringpunishment on switch and nonswitch trials. Thus, in

contrast to dopaminergic medication, serotoninergicmanipulation affected the processing of punishmentirrespective of switching. Interestingly, patients withmajor depression, which has been associated with sero-toninergic abnormality, were found to suffer a deficit onprobabilistic reversal learning that did not reflect per-severative responding but rather reflected an inabilityto maintain responding to the usually correct stimulusin the face of misleading, probabilistic errors.99,105

Finally, Taylor-Tavares et al99 observed an inverse cor-relation between suppressed activity in the amygdaladuring probabilistic, misleading errors and the tendencyto switch after these misleading errors. Critically, thisrelationship was abolished in depressed patients. Addi-tional pharmaco-fMRI studies examined differentversions of the current task paradigm.106–111

Animal Models. There is a long history of animal workon the probabilistic reversal learning task.85,112,113 Whenassessing convergence between animal and human stud-ies, it is important to consider subtle differences in taskdesign. First, studies with experimental animals haveused deterministic rather than probabilistic contingen-cies, so that animals never obtain ‘‘misleading’’ punish-ment or reward. Second, the nature of reward andpunishment is qualitatively different, with reward consti-tuting prolonged periods of access to juice or food in ani-mals but (often) bonus points of positive feedback inhumans. On the other hand, punishment may consistof periods of darkness or reward omission in animalsbut bonus point loss or negative feedback in humans.Nevertheless, there is remarkable convergence. Consis-

tent with neuroimaging studies in humans, animal workhas implicated the orbitofrontal cortex in reversal learn-ing.85,113,114 Single-cell recordings have also revealed thatthe firing of orbitofrontal neurons changes with altera-tions in reward contingencies.115,116 More recent sin-gle-cell recording studies suggest that the activity oforbitofrontal neurons reverses more slowly than thatof neurons in the amygdale,117,118 which may suggestthat the orbitofrontal cortex may indirectly facilitate flex-ibility in downstream regions (such as the amygdala) bysignaling the expected value of outcomes rather than bydirectly inhibiting previously relevant responses.In addition to orbitofrontal and amygdala findings,85

animal studies have implicated the ventral striatum in re-versal learning. Specifically, lesions of the ventrolateralpart of the head of the caudate nucleus induced a persev-erative response tendency during (object) reversal learn-ing in monkeys.112 Subsequent work with rodents andnonhuman primates indicates that lesions of nucleusaccumbens also disrupts performance on (object and/or spatial) reversal learning tasks.119,120 However, the im-pairment following nucleus accumbens lesions is gener-ally not restricted to the reversal stages of the task, butextends to initial acquisition stages, suggesting a more

208


general role in the learning of stimulus-reinforcementcontingencies rather than in reversal specifically.119

Psychopharmacological work with marmosets sup-ports human evidence that reversal learning is sensitiveto serotoninergic manipulations. Clarke et al121–123

revealed that depletion of serotonin in the orbitofrontalcortex with the neurotoxin 5,7-dihydroxytriptamine(DHT) impaired reversal learning and induced persev-erative responding to the previously rewarded stimulus.It might be noted that the perseverative nature of thedeterministic reversal deficit after 5,7-DHT lesions inmarmosets is qualitatively different from the inappro-priate switching seen after tryptophan depletion inhumans. This discrepancy may reflect differences inthe task used in marmosets and humans (probabilisticvs deterministic; emphasis on punishment) or, morelikely, differences in the effect of the manipulation onthe degree of serotonin depletion in the brain.124,125

Performance in Schizophrenia. Waltz and Gold126 re-cently employed a modified version of the probabilisticreversal learning task in 34 patients with schizophreniaand 26 controls. Although patients and controls per-formed similarly on the initial acquisition of probabilisticcontingencies, patients showed substantial learningimpairments when reinforcement contingencies werereversed, achieving significantly fewer reversals.

PsychometricData. These data are not yet available forthe probabilistic reversal learning task.

Future Directions. The probabilistic reversal learningtask has clearly established validity for both the rein-forcement learning and the reversal learning constructsof the task paradigm. As in the other reinforcement learn-ing paradigms, there is also solid work identifying neuralsubstrates, developing animal models, and investigatingeffects of pharmacological manipulation. In future work,it will be important to more fully investigate the impactof schizophrenia on task performance. Establishing thatpatient impairments are specific to the reversal conditionwill address alternate generalized deficit explanations. Aswith most tasks being translated from the cognitive neu-roscience domain, substantial work will also be needed toestablish and optimize psychometric characteristics.

Acknowledgments

Appreciation goes to Dr Cameron Carter andDrDeannaBarch, who spearheaded the CNTRICS initiative.Appreciation also goes to Jody Conrad, Carol Cox,and Deb Tussing, who were central to the successfulorganization of the CNTRICS meetings. Finally, wewould also like to thank the additional members of thebreakout group including, Deanna Barch, Robert

Buchanan, Nelson Cowan, Melissa Fisher, Michael Green,Ann Kring, Gina Kuperberg, Steve Marder, DanielMathalon, Michael Minzenberg, Trevor Robbins, MartinSarter, Larry Seidman, Carol Tamminga, Steve Taylor,Andy Yonelinas, Jong Yoon and Jared Young.

References

1. Delis DC, Kramer JH, Kaplan E, Ober BA. CaliforniaVerbal Learning Test (CVLT) Manual. New York, NY:The Psychological Corporation; 1987.

2. Wechsler DA. Wechsler Adult Intelligence Scale Revised(WAIS-R), Manual. Cleveland, OH: The PsychologicalCorporation; 1981.

3. NuechterleinKH,GreenMF,KernRS, et al. TheMATRICSConsensus Cognitive Battery, part 1: test selection, reliability,and validity. Am J Psychiatry. 2008;165:203–213.

4. Carter CS, Barch DM. Cognitive neuroscience-basedapproaches to measuring and improving treatment effectson cognition in schizophrenia: the CNTRICS initiative.Schizophr Bull. 2007;33:1131–1137.

5. Ranganath C, Minzenberg M, Ragland JD. The cognitiveneuroscience of memory function and dysfunction in schizo-phrenia. Biol Psychiatry. 2008;19:417–427.

6. Eichenbaum H. A cortical-hippocampal system for declara-tive memory. Nat Rev Neurosci. 2000;1:41–50.

7. Eichenbaum H, Cohen NJ. From Conditioning to ConsciousRecollection: Memory Systems of the Brain. New York, NY:Oxford University Press; 2001.

8. Marr D. Simple memory: a theory for archicortex. PhilosTrans R Soc Lond B Biol Sci. 1971;262:23–81.

9. O’Reilly RC, Rudy JW. Conjunctive representations inlearning and memory: principles of cortical and hippocam-pal function. Psychol Rev. 2001;108:311–345.

10. Frank MJ, Rudy JW, O’Reilly RC. Transitivity, flexibility,conjunctive representations, and the hippocampus. II. Acomputational analysis. Hippocampu. 2003;13:341–354.

11. Nakazawa K, Quirk MC, Chitwood RA, et al. Requirementfor hippocampal CA3 NMDA receptors in associative mem-ory recall. Science. 2002;297:211–218.

12. Nakazawa K, Su LD, Quirk MC, Rondi-Reig L, WilsonMA, Tonegawa S. Hippocampal CA3 NMDA receptorsare crucial for memory acquisition of one-time experience.Neuron. 2003;38:305–315.

13. Preston AR, Shrager Y, Dudukovic NM, Gabrieli JD. Hip-pocampal contribution to the novel use of relational informa-tion in declarative memory. Hippocampus. 2004;14:148–152.

14. Lee SS, Jensen AR. The effect of awareness on three-stage me-diatedassociation.JVerbLearnVerbBehav. 1968;7:1005–1009.

15. BunseyM,EichenbaumH.Conservationofhippocampalmem-ory function in rats and humans. Nature. 1996;379:255–257.

16. Titone D, Ditman T, Holzman PS, Eichenbaum H. Levy DL.Transitive inference in schizophrenia: impairments in rela-tional memory organization. Schizophr Res. 2004;68:235–247.

17. Heckers S, Rauch SL, Goff D, et al. Impaired recruitment ofthe hippocampus during conscious recollection in schizo-phrenia. Nat Neurosci. 1998;1:318–323.

18. Ongur D, Cullen TJ, Wolf DH, et al. The neural basis ofrelational memory deficits in schizophrenia. Arch GenPsychiatry. 2006;63:356–365.

209


19. Lisman JE, Grace AA. The hippocampal-VTA loop: con-trolling the entry of information into long-term memory.Neuron. 2005;46:703–713.

20. Ragland JD, Moelter ST, McGrath C, et al. Levels-of-processing effect on word recognition in schizophrenia.Biol Psychiatry. 2003;54:1154–1161.

21. Ragland JD, Valdez JN, Loughead J, Gur RC, Gur RE.Functional magnetic resonance imaging of internal sourcemonitoring in schizophrenia: recognition with and withoutrecollection. Schizophr Res. 2006;87:160–171.

22. Blumenfeld RS, Ranganath C. Dorsolateral prefrontal cortexpromotes long-term memory formation through its role inworking memory organization. J Neurosci. 2006;26:916–925.

23. Bower GH. Imagery as a relational organizer in associativelearing. J Verb Learn Verb Behav. 1970;9:529–533.

24. Hunt RR, Einsten GO. Relational and item-specificinformation in memory. J Verb Learn Verb Behav.1981;920:497–514.

25. Bower GH. Organizational factors in memory. Cogn Psy-chol. 1970;1:18–46.

26. Hunt RR, McDaniel MA. The enigma of organization anddistinctiveness. J Mem Lang. 1993;32:421–445.

27. Eichenbaum H. Remembering: functional organization ofthe declarative memory system. Curr Biol. 2006;16:R643–R645.

28. Eichenbaum H, Yonelinas AR, Ranganath C. The medialtemporal lobe and recognition memory. Annu Rev Neurosci.2007;30:123–152.

29. Squire LR. Memory systems of the brain: a brief history andcurrent perspective. Neurobiol Learn Mem. 2004;82:171–177.

30. Postle BR, Berger JS, D’Esposito M. Functional neuroana-tomical double dissociation of mnemonic and executivecontrol processes contributing to working memory perfor-mance. Proc Natl Acad Sci USA. 1999;96:12959–12964.

31. Blumenfeld RS, Ranganath C. Prefrontal cortex and long-term memory encoding: an integrative review of findingsfrom neuropsychology and neuroimaging. Neuroscientist.2007;13:280–291.

32. Murray LJ, Ranganath C. The dorsolateral prefrontal cor-tex contributes to successful relational memory encoding.J Neurosci. 2007;27:5515–5522.

33. Ragland JD, Gur RC, Valdez JN, et al. Levels-of-processingeffect on frontotemporal function in schizophrenia duringword encoding and recognition. Am J Psychiatry.2005;162:1840–1848.

34. Bonner-Jackson A, Haut K, Csernansky JG, Barch DM.The influence of encoding strategy on episodic memoryand cortical activity in schizophrenia. Biol Psychiatry.2005;58:47–55.

35. Ninokura Y, Mushiake H, Tanji J. Integration of temporalorder and object information in the monkey lateral prefron-tal cortex. J Neurophysiol. 2004;91:555–560.

36. Petrides M. Impairments on nonspatial self-ordered and ex-ternally ordered working memory tasks after lesions of themid-dorsal lateral part of the lateral frontal cortex of mon-key. J Neurosci. 1995;15:359–375.

37. Ragland JD, Gur RC, Raz J, et al. Effect of schizophreniaon frontotemporal activity during word encoding and recog-nition: a PET cerebral blood flow study. Am J Psychiatry.2001;158:1114–1125.

38. Ragland JD, Gur RC, Valdez J, et al. Event-related fMRI offrontotemporal activity during word encoding and recogni-tion in schizophrenia. Am J Psychiatry. 2004;161:1004–1015.

39. Craik FIM, Lockhart RS. Levels of processing: a frameworkfor memory research. J Verb Learn Verb Behav. 1972;12:599–607.

40. Pizzagalli DA, Jahn AL, O’Shea JP. Toward an objectivecharacterization of an anhedonic phenotype: a signal-detectionapproach. Biol Psychiatry. 2005;57:319–327.

41. Tripp G, Alsop B. Sensitivity to reward frequency in boyswith attention deficit hyperactivity disorder. J Clin ChildPsychol. 1999;28:366–375.

42. Davison MC, Tustin RD. The relation between the general-ized matching law and signal-detection theory. J Exp AnalBehav. 1978;29:331–336.

43. Baum WM. On two types of deviation from the matchinglaw: bias and undermatching. J Exp Anal Behav. 1974;22:231–242.

44. Herrnstein RJ. On the law of effect. J Exp Anal Behav.1970;13:243–266.

45. McCarthy D, Davison M. Signal probability, reinforcement,and signal detection. J Exp Anal Behav. 1979;32:373–382.

46. Pizzagalli DA, Evins AE, Schetter EC, et al. Single dose ofa dopamine agonist impairs reinforcement learning inhumans: behavioral evidence from a laboratory-based mea-sure of reward responsiveness. Psychopharmacology (Berl).2008;196:221–232.

47. Pizzagalli DA, Goetz E, Ostacher M, Iosifescu D, Perlis RH.Euthymic patients with bipolar disorder show decreased re-ward learning in a probabilistic reward task. Biol Psychiatry.2008;64:162–168.

48. Hasler G, Drevets WC, Manji HK, Charney DS. Discover-ing endophenotypes for major depression. Neuropsychophar-macology. 2004;29:1765–1781.

49. Leibenluft E, Charney DS, Pine DS. Researching the patho-physiology of pediatric bipolar disorder. Biol Psychiatry.2003;53:1009–1020.

50. Pizzagalli DA, Iosifescu D, Hallett LA, Ratner KG, FavaM. Reduced hedonic capacity in major depressive disorder:evidence from a probabilistic reward task. J Psychiatr Res.In press.

51. Kennerley SW, Walton ME, Behrens TEJ, Buckley MJ,Rushworth MFS. Optimal decision making and the anteriorcingulate cortex. Nat Neurosci. 2006;9:940–947.

52. Rushworth MF, Buckley MJ, Behrens TE, Walton ME,Bannerman DM. Functional organization of the medialfrontal cortex. Curr Opin Neurobiol. 2007;17:220–227.

53. Dillon DG, Holmes AJ, Jahn AL, Bogdan R, Wald LL,Pizzagalli DA. Dissociation of neural regions associatedwith anticipatory versus consummatory phases of incentiveprocessing. Psychophysiology. 2008;45:36–49.

54. Knutson B, FongGW,Bennett SM,AdamsCM,HommerD.A regionofmesial prefrontal cortex tracksmonetarily reward-ing outcomes: characterization with rapid event-relatedfMRI. NeuroImage. 2003;18:263–272.

55. Barr RS, Pizzagalli DA, Culhane MA, Goff DC, Evins AE.A single dose of nicotine enhances reward responsiveness innon-smokers: Implications for development of dependence.Biol Psychiatry. 2008;63:1061–1065.

56. Fuller RW, Clemens JA, Hynes MD, III. Degree of selectiv-ity of pergolide as an agonist at presynaptic versus postsyn-aptic dopamine receptors: implications for prevention ortreatment of tardive dyskinesia. J Clin Psychopharmacol.1982;2:371–375.

57. Tissari AH, Rossetti ZL, Meloni M, Frau MI, Gessa GL.Autoreceptors mediate the inhibition of dopamine synthesis

210


by bromocriptine and lisuride in rats. Eur J Pharmacol.1983;91:463–468.

58. PierceyMF, HoffmannWE, SmithMW, Hyslok DK. Inhibi-tion of dopamine neuron firing by pramipexole, a dopamineD3 receptor-preferring agonist: comparison to other dopa-mine receptor agonists. Eur J Pharmacol. 1996;312:35–44.

59. Frank MJ, O’Reilly RC. A mechanistic account of striataldopamine function in human cognition: psychopharmaco-logical studies with cabergoline and haloperidol. BehavNeurosci. 2006;120:497–517.

60. Dani JA, Harris RA. Nicotine addiction and comorbiditywith alcohol abuse and mental illness. Nat Neurosci.2005;8:1465–1470.

61. Kenny PJ, Markou A. Nicotine self-administration acutelyactivates brain reward systems and induces a long-lasting in-crease in reward sensitivity. Neuropsychopharmacology.2006;31:1203–1211.

62. Dawkins L, Powell JH, West R, Powell J, Pickering A. Adouble-blind placebo controlled experimental study of nico-tine: I. Effects on incentive motivation. Psychopharmacology(Berl). 2006;189:355–367.

63. Heerey EA, Bell-Warren KR, Gold JM. Decision-makingimpairments in the context of intact reward sensitivity inschizophrenia. Biol Psychiatry. 2008;64:62–69.

64. de Leon J, Tracy J, McCann E,McGrory A, Diaz FJ. Schizo-phrenia and tobacco smoking: a replication study in anotherUS psychiatric hospital. Schizophr Res. 2002;56:55–65.

65. Kumari V, Postma P. Nicotine use in schizophrenia: the selfmedication hypotheses. Neurosci Biobehav Rev. 2005;29:1021–1034.

66. Santesso DL, Dillon DG, Birk JL. Individual differences inreinforcement learning: behavioral, electrophysiological,and neuroimaging correlates. NeuroImage.42:807–816.

67. Bogdan R, Pizzagalli DA. The heritability of hedonic capac-ity and perceived stress: a twin study evaluation of candidatedepressive phenotypes. Psychol Med. 2008;28:1–8.

68. Frank MJ, Seeberger LC, O’reilly RC. By carrot or by stick:cognitive reinforcement learning in Parkinsonism. Science.2004;306:1940–1943.

69. Frank MJ, Woroch BS, Curran T. Error-related negativitypredicts reinforcement learning and conflict biases. Neuron.2005;47495–501.

70. Frank MJ, Santamaria A, O’Reilly RC, Willcutt E. Testingcomputational models of dopamine and noradrenalinedysfunction in attention deficit/hyperactivity disorder.Neuropsychopharmacology. 2007;32:1583–1599.

71. Frank MJ, Samanta J, Moustafa AA, Sherman SJ. Holdyour horses: impulsivity, deep brain stimulation, and medi-cation in parkinsonism. Science. 2007;318:1309–1312.

72. Frank MJ, Moustafa AA, Haughey HM, Curran T,Hutchison KE. Genetic triple dissociation reveals multipleroles for dopamine in reinforcement learning. Proc NatlAcad Sci USA. 2007;104:16311–16316.

73. Klein TA,Neumann J,ReuterM,Hennig J, vonCramonDY,Ullsperger M. Genetically determined differences in learningfrom errors. Science. 2007;318:1642–1645.

74. Cools R, Altamirano L, D’Esposito M. Reversal learning inParkinson’s disease depends on medication status and out-come valence. Neuropsychologia. 2006;44:1663–1673.

75. Schultz W, Dayan P, Montague PR. A neural substrate ofprediction and reward. Science. 1997;275:1593–1599.

76. Frank MJ. Dynamic dopamine modulation in the basalganglia: a neurocomputational account of cognitive deficits

in medicated and nonmedicated Parkinsonism. J CognNeurosci. 2005;17:51–72.

77. Frank MJ, D’Lauro C, Curran T. Cross-task individual dif-ferences in error processing: neural, electrophysiological,and genetic components. Cogn Affect Behav Neurosci.2007;7:297–308.

78. Wheeler EZ, Fellows LK. The human ventromedial frontallobe is critical for learning from negative feedback. Brain.2008;131(pt 5):1323–1331.

79. Costa RM, Gutierrez R, de Araujo IE, et al. Dopamine lev-els modulate the updating of tastant values. Genes BrainBehav. 2007;6:314–320.

80. Nakamura K, Hikosaka O. Role of dopamine in the primatecaudate nucleus in reward modulation of saccades. J Neuro-sci. 2006;26:5360–5369.

81. Waltz JA, Frank MJ, Robinson BM, Gold JM. Selective re-inforcement learning deficits in schizophrenia support pre-dictions from computational models of striatal-corticaldysfunction. Biol Psychiatry. 2007;62:756–764.

82. Lawrence AD, Sahakian BJ, Rogers RD, Hodges JR,Robbins TW. Discrimination, reversal, and shift learningin Huntington’s disease: mechanisms of impaired responseselection. Neuropsychologia. 1999;37:1359–1374.

83. Swainson R, Rogers RD, Sahakian BJ, Summers BA,Polkey CE, Robbins TW. Probabilistic learning and reversaldeficits in patients with Parkinson’s disease or frontal ortemporal lobe lesions: possible adverse effects of dopaminer-gic medication. Neuropsychologia. 2000;38:596–612.

84. Robbins TW. Methods for measuring spontaneous motor?activity. In: Iversen II, Iversen SD, Snyder SH, eds. Hand-book of Psychopharmacology.New York, NY: Plenum Press;1977.

85. Jones B, Mishkin M. Limbic lesions and the problem ofstimulus-reinforcement associations. Exp Neurology. 1972;36:362–377.

86. Sutton R, Barto A. Reinforcement Learning. Cambridge,Mass: MIT Press; 1998.

87. Cohen MX. Neurocomputational mechanisms of reinforce-ment-guided learning in humans: a review. Cogn AffectBehav Neurosci. 2008;8:113–125.

88. O’Doherty J, Dayan P, Friston K, Critchley H, Dolan R.Temporal difference models and reward-related learning inthe human brain. Neuron. 2003;38:329–337.

89. O’Doherty J, Dayan P, Schultz J, Deichmann R, Friston K,Dolan R. Dissociable role of ventral and dorsal striatum ininstrumental conditioning. Science. 2004;304:452–454.

90. Hampton AN, Bossaerts P, O’Doherty JP. The role of theventromedial prefrontal cortex in abstract state-based infer-ence during decision making in humans. J Neurosci.2006;26:8360–8367.

91. Daum I, Schugens MM, Channon S, Polkey CE, Gray JA.T-maze discrimination and reversal learning after unilateraltemporal or frontal lobe lesions in man. Cortex.1991;27:613–622.

92. Fellows L, Farah M. Ventromedial frontal cortex mediatesaffective shifting in humans: evidence from a reversal learn-ing paradigm. Brain. 2003;126:1830–1837.

93. Hornak J, O’Doherty J, Bramham J, et al. Reward-relatedreversal learning after surgical excisions in orbito-frontalor dorsolateral prefrontal cortex in humans. J Cogn Neuro-sci. 2004;16:463–478.

94. RollsE,HornakJ,WadeD,McGrath J.Emotion-related learn-ing in patients with social and emotional changes associated

211


with frontal lobe damage. J Neurol Neurosurg Psychiatr.1994;57:1518–1524.

95. Rahman S, Sahakian B, Hodges J, Rogers R, Robbins T.Specific cognitive deficits in mild frontal variant frontotem-poral dementia. Brain. 1999;122:670–673.

96. Cools R, Clark L, Owen AM, Robbins TW. Defining theneural mechanisms of probabilistic reversal learning usingevent-related functional magnetic resonance imaging. J Neu-rosci. 2002;22:4563–4567.

97. Cools R, Lewis S, Clark L, Barker R, Robbins TW.L-DOPA disrupts activity in the nucleus accumbens duringreversal learning in Parkinson’s disease. Neuropsychophar-macology. 2007;32:180–189.

98. Dodds CM, Muller U, Clark L, van Loon A, Cools R,Robbins TW. Methylphenidate has differential effects onblood oxygenation level-dependent signal related to cogni-tive subprocesses of reversal learning. J Neurosci. 2008;28:5976–5982.

99. Taylor Tavares JV, Clark L, Furey ML, Williams GB,Sahakian BJ, Drevets WC. Neural basis of abnormal re-sponse to negative feedback in unmedicated mood disorders.NeuroImage. 2008;42:1118–1126.

100. Remijnse PL, Nielen MM, Uylings HB, Veltman DJ. Neuralcorrelates of a reversal learning task with an affectively neu-tral baseline: an event-related fMRI study. NeuroImage.2005;26:609–618.

101. Cools R, Barker RA, Sahakian BJ, Robbins TW. Enhancedor impaired cognitive function in Parkinson’s disease asa function of dopaminergic medication and task demands.Cereb Cortex. 2001;11:1136–1143.

102. Mehta MA, Swainson R, Ogilvie AD, Sahakian BJ, RobbinsTW. Improved short-term spatial memory but impairedreversal learning following the dopamine D2 agonist bromo-criptine in human volunteers. Psychopharmacology.2001;159:10–20.

103. Chamberlain SR, Muller U, Blackwell AD, Clark L,Robbins TW, Sahakian BJ. Neurochemical modulation ofresponse inhibition and probabilistic learning in humans.Science. 2006;311:861–863.

104. Evers EA, Cools R, Clark L, et al. Serotonergic modulationof prefrontal cortex during negative feedback in probabilis-tic reversal learning. Neuropsychopharmacology. 2005;30:1138–1147.

105. Murphy F, Michael A, Robbins TW, Sahakian BJ. Neuro-psychological impairment in patients with major depressivedisorder: the effects of feedback on task performance.Psychol Med. 2003;33:455–467.

106. CohenMX, Elger CE,Weber B. Amygdala tractography pre-dicts functional connectivity and learning during feedback-guided decision-making. NeuroImage. 2008;39:1396–1407.

107. Cohen MX, Krohn-Grimberghe A, Elger CE, Weber B.Dopamine gene predicts the brain’s response to dopaminer-gic drug. Eur J Neurosci. 2007;26:3652–3660.

108. Hampton AN, Adolphs R, Tyszka MJ, O’Doherty JP. Con-tributions of the amygdala to reward expectancy and choicesignals in human prefrontal cortex.Neuron. 2007;55:545–555.

109. Hampton AN, O’Doherty JP. Decoding the neural sub-strates of reward-related decision making with functionalMRI. Proc Natl Acad Sci USA. 2007;104:1377–1382.

110. O’Doherty J, Critchley H, Deichmann R, Dolan RJ. Disso-ciating valence of outcome from behavioral control in hu-man orbital and ventral prefrontal cortices. J Neurosci.2003;23:7931–7939.

111. O’Doherty J, KringelbachM, Rolls E, Hornak J, Andrews C.Abstract reward and punishment representations in thehuman orbitofrontal cortex. Nat Neurosci. 2001;4:95–102.

112. Divac I, Rosvold HE, Szwarcbart MK. Behavioral effects ofselective ablation of the caudate nucleus. J Comp PhysiolPsychol. 1967;63:184–190.

113. Iversen S, Mishkin M. Perseverative interference in monkeysfollowing selective lesions of the inferior prefrontal convex-ity. Exp Brain Res Experimentelle Hirnforschung.1970;11:376–386.

114. Dias R, Robbins TW, Roberts AC. Dissociation in prefron-tal cortex of affective and attentional shifts. Nature.1996;380:69–72.

115. Rolls ET, Critchley HD, Mason R, Wakeman EA. Orbito-frontal cortex neurons: role in olfactory and visual associa-tion learning. J Neurophysiol. 1996;75:1970–1981.

116. Thorpe SJ, Rolls ET, Maddison S. The orbitofrontal cortex:neuronal activity in the behaving monkey. Exp Brain ResExperimentelle Hirnforschung. 1983;49:93–115.

117. Schoenbaum G, Saddoris MP, Stalnaker TA. Reconcilingthe roles of orbitofrontal cortex in reversal learning andthe encoding of outcome expectancies. Ann N Y Acad Sci.2007;1121:320–335.

118. Stalnaker TA, Roesch MR, Franz TM, Calu DJ, Singh T,Schoenbaum G. Cocaine-induced decision-making deficitsare mediated by miscoding in basolateral amygdala. NatNeurosci. 2007;10:949–951.

119. Annett L, McGregor A, Robbins T. The effects of ibutenicacid lesions of the nucleus accumbens on spatial learningand extinction in the rat. Behav Brain Res. 1989;31:231–242.

120. Stern C, Passingham R. The nucleus accumbens in monkeys(macaca fascicularis). Exp Brain Res Experimentelle Hirnfor-schung. 1995;106:239–247.

121. Clarke H, Dalley J, Crofts H, Robbins T, Roberts A. Cog-nitive inflexibility after prefrontal serotonin depletion. Sci-ence. 2004;304:878–880.

122. Clarke HF, Walker SC, Crofts HS, Dalley JW, Robbins TW,Roberts AC. Prefrontal serotonin depletion affects reversallearning but not attentional set shifting. J Neurosci. 2005;25:532–538.

123. ClarkeHF,WalkerSC,Dalley JW,RobbinsTW,RobertsAC.Cognitive inflexibility after prefrontal serotonin depletion isbehaviorally and neurochemically specific. Cereb Cortex.2007;17:18–27.

124. Cools R, Roberts AC, Robbins TW. Serotoninergic regula-tion of emotional and behavioural control processes. TrendsCogn Sci. 2008;12:31–40.

125. van der Plasse G, Meerkerk DT, Lieben CK, Blokland A,Feenstra MG. Lack of evidence for reduced prefrontal cor-tical serotonin and dopamine efflux after acute tryptophandepletion. Psychopharmacology. 2007;195:377–385.

126. Waltz JA, Gold JM, Waltz JA, Gold JM. Probabilistic re-versal learning impairments in schizophrenia: further evi-dence of orbitofrontal dysfunction. Schizophr Res. 2007;93:296–303.

212


Documents

CNTRICS Final Task Selection: Long-Term Memory · 2012-04-06 · Table 1. Long-Term Memory in Schizophrenia Relational encoding and retrieval: The processes involved in memory for