7
Research Report Human posterior parietal cortex maintains color, shape and motion in visual short-term memory Masahiro Kawasaki a,b, , Masataka Watanabe b , Jiro Okuda c , Masamichi Sakagami c , Kazuyuki Aihara a a Institute of Industrial Science, University of Tokyo and ERATO Aihara Complexity Modeling Project, JST, Japan b Department of Quantum Engineering and System Science, Graduate School of Engineering, University of Tokyo, Japan c Brain Science Research Center, Tamagawa University Research Institute, Tokyo, Japan ARTICLE INFO ABSTRACT Article history: Accepted 11 March 2008 Available online 1 April 2008 We used functional magnetic resonance imaging (fMRI) to investigate the neural substrate of visual short-term memory for objects defined by features processed in the dorsal and the ventral visual streams. Here we adopted the conventional delayed recognition task, whereas in addition to the more commonly used visual features of color and shape, motion direction was applied to define an item. Our behavioral results indicated that the capacity limit of visual short-term memory of motion direction was approximately two, which was significantly lower than those of color and shape, about three or four. We also found that storage capacity was significantly reduced when subjects were required to retain all three features superimposed in space. Meanwhile, fMRI results revealed that activity in the posterior part of the superior parietal lobe was memoryload dependent for all three features indicating that it collects and stores visual information from both the two visual processing streams, whereas the anterior part was load dependent only for motion. © 2008 Elsevier B.V. All rights reserved. Keywords: Visual short-term memory Parietal cortex Feature integration Functional magnetic resonance imaging Color Shape Motion 1. Introduction We can perceive and retain an object defined by multiple features as a single unified experience despite that the visual features are processed by two functionally specialized systems in our brain, mainly ventral processing stream for color and shape, and dorsal processing stream for motion (DeYoe and Van Essen, 1988; Mishkin et al., 1983). Of particular interest is that there is no single cortical area which receives heavy projections from high-level modules of the two visual processing streams, enough to collect the massive information of the whole visual scene for online perception, a classic problem known as the binding problem (Treisman and Gelade, 1980). The question arises then, whether binding of visual features from the two streams is also a problem during mental maintenance of the visual scene, i.e. visual short-term memory (VSTM). To address the issue, we examined VSTM load-dependent activity which was correlated with the number of items in VSTM for color, shape, or motion, during a delayed recognition task using items defined by moving colored random dots within shaped frames (Fig. 1.A, B). Although previous psycho- logical (Alvarez and Cavanagh, 2004; Cowan, 2001; Luck and Vogel, 1997; Olson and Jiang, 2002; Vogel et al., 2001; Wheeler and Treisman, 2002; Xu, 2002) and functional imaging (Jha and BRAIN RESEARCH 1213 (2008) 91 97 Corresponding author. Institute of Industrial Science, University of Tokyo and ERATO Aihara Complexity Modeling Project, JST, 4-6-1, Komaba, Meguro-ku, Tokyo, 153-8505, Japan. Fax: +81 3 5452 5723. E-mail address: [email protected] (M. Kawasaki). 0006-8993/$ see front matter © 2008 Elsevier B.V. All rights reserved. doi:10.1016/j.brainres.2008.03.037 available at www.sciencedirect.com www.elsevier.com/locate/brainres

Human posterior parietal cortex maintains color, shape and motion in visual short-term memory

Embed Size (px)

Citation preview

Page 1: Human posterior parietal cortex maintains color, shape and motion in visual short-term memory

B R A I N R E S E A R C H 1 2 1 3 ( 2 0 0 8 ) 9 1 – 9 7

ava i l ab l e a t www.sc i enced i rec t . com

www.e l sev i e r. com/ l oca te /b ra in res

Research Report

Human posterior parietal cortex maintains color, shape andmotion in visual short-term memory

Masahiro Kawasakia,b,⁎, Masataka Watanabeb, Jiro Okudac,Masamichi Sakagamic, Kazuyuki Aiharaa

aInstitute of Industrial Science, University of Tokyo and ERATO Aihara Complexity Modeling Project, JST, JapanbDepartment of Quantum Engineering and System Science, Graduate School of Engineering, University of Tokyo, JapancBrain Science Research Center, Tamagawa University Research Institute, Tokyo, Japan

A R T I C L E I N F O

⁎ Corresponding author. Institute of IndustriaKomaba, Meguro-ku, Tokyo, 153-8505, Japan

E-mail address: [email protected]

0006-8993/$ – see front matter © 2008 Elsevidoi:10.1016/j.brainres.2008.03.037

A B S T R A C T

Article history:Accepted 11 March 2008Available online 1 April 2008

We used functional magnetic resonance imaging (fMRI) to investigate the neural substrate ofvisual short-term memory for objects defined by features processed in the dorsal and theventral visual streams.Hereweadopted the conventional delayed recognition task,whereas inaddition to themore commonly used visual features of color and shape,motion direction wasapplied to define an item. Our behavioral results indicated that the capacity limit of visualshort-termmemoryofmotiondirectionwas approximately two,whichwas significantly lowerthan those of color and shape, about three or four. We also found that storage capacity wassignificantly reducedwhen subjectswere required to retain all three features superimposed inspace. Meanwhile, fMRI results revealed that activity in the posterior part of the superiorparietal lobe was memory–load dependent for all three features indicating that it collects andstores visual information from both the two visual processing streams, whereas the anteriorpart was load dependent only for motion.

© 2008 Elsevier B.V. All rights reserved.

Keywords:Visual short-term memoryParietal cortexFeature integrationFunctional magnetic resonanceimagingColorShapeMotion

1. Introduction

We can perceive and retain an object defined by multiplefeatures as a single unified experience despite that the visualfeatures are processed by two functionally specialized systemsin our brain, mainly ventral processing stream for color andshape, anddorsal processing streamformotion (DeYoeandVanEssen, 1988; Mishkin et al., 1983). Of particular interest is thatthere is no single cortical areawhich receives heavy projectionsfrom high-level modules of the two visual processing streams,enough to collect the massive information of the whole visualscene for online perception, a classic problem known as the

l Science, University of T. Fax: +81 3 5452 5723.(M. Kawasaki).

er B.V. All rights reserved

binding problem (Treisman and Gelade, 1980). The questionarises then, whether binding of visual features from the twostreams is also a problem during mental maintenance of thevisual scene, i.e. visual short-termmemory (VSTM).

To address the issue, we examined VSTM load-dependentactivity which was correlated with the number of items inVSTM for color, shape, ormotion, during a delayed recognitiontask using items defined by moving colored random dotswithin shaped frames (Fig. 1.A, B). Although previous psycho-logical (Alvarez and Cavanagh, 2004; Cowan, 2001; Luck andVogel, 1997; Olson and Jiang, 2002; Vogel et al., 2001; Wheelerand Treisman, 2002; Xu, 2002) and functional imaging (Jha and

okyo and ERATO Aihara Complexity Modeling Project, JST, 4-6-1,

.

Page 2: Human posterior parietal cortex maintains color, shape and motion in visual short-term memory

Fig. 1 – A: Schematic illustrations of one trial sequence for color trials (top). At the beginning of each trial, a number to which thesubjects had to rehearse throughout the trial was auditorily presented for 800 ms. A sample display that contained one to eightitems (orone to four items in the fMRIexperiment)waspresented.After a retention interval of1200ms,a testdisplay that containedoneprobe itemwaspresented, and the subjectsdeterminedwhether therewasa changeornot in theattended featurebetween theprobe and sample items located in the same position. After that, a probe number was presented visually, and the subjects wererequired to indicatewhether thenumber is the sameornotwith a rehearsednumber. Small arrows in the sample and test displaysindicate the direction of themoving dots. Thearrowsand contours of the shapeswerenot present in the actual display. B: Exampleof sample displays; color, shape, motion direction, and conjunction trials. C: The estimated VSTM capacity in the behavioralexperiment: color (red), shape (green),motion direction (blue), and conjunction (black). The horizontal axis indicates the number ofitems in the sample display.

92 B R A I N R E S E A R C H 1 2 1 3 ( 2 0 0 8 ) 9 1 – 9 7

McCarthy, 2001; Linden et al., 2003; Shafritz et al., 2002; Songand Jiang, 2006; Todd and Marois, 2004, 2005; Xu and Chun,2006a,b) studies have investigated VSTM using featuresassociated with the ventral stream (i.e., color, shape andtheir conjunction), there has been little researchwith regard toVSTM of motion.

In addition to the above single feature conditions, we havefurther attempted to investigate the VSTM load-dependentactivity for conjunctive items. Note, however that shape, colorandmotionmightnot be regardedas features of a single object,that is to say, a natural percept of the stimulus is a field ofdynamic coloreddots behinda shapedwindow. If notmore,wemay say thatmultiple features are brought together in a singleretinal position. An alternative way to implement motion asa visual feature would be to move the items themselves.However, we did not take this approach since it would make itpossible for the subjects to use position cues (e.g. finalpositions) to detect changes in motion direction. This is amajor issue due to the larger storage capacity of position thanany other visual features (Simons, 1996).

2. Results

2.1. Behavioral results

Tomeasure theVSTMcapacity limit of color, shape, andmotiondirection, we first carried out a behavioral study using thedelayed recognition taskswherewemanipulated thenumber ofsample items from 1 to 8 under the condition of color (C), shape(S), motion (M) and color–shape–motion (CSM) (Fig. 1.A, B). The

VSTM capacities were estimated by change-detection accuracyand Cowan's K formula. A two-factor repeated-measure analy-sis of variance (ANOVA) revealedmain effects of the numbers ofsample items (F5, 90=32.01, Pb0.001) and of feature type (F2, 90=14.61, Pb0.001). The capacity for color and shape did notsignificantly differ (t(70)=0.61, PN0.54). For both, K values in-creased with the numbers of sample items and saturatedaround 4 to 6 items (C, 4 vs 6 items, t(5)=0.94, PN0.37; S, 4 vs 6items, t(5)=1.41, PN0.19). Thus, these results suggest a VSTMlimit of approximately three or four items for color and shape.This is consistent with the results of previous studies (Cowan,2001; Luck and Vogel, 1997; Vogel et al., 2001). In contrast, theestimated VSTM limit for motion direction was approximatelytwo items (M, 2 vs 3 items, t(5)=0.91, PN0.38), which was sig-nificantly less than that of either color or shape (M vs C, t(5)=5.47, Pb0.01; M vs S, t(5)=3.56, Pb0.02). Furthermore, the es-timatedVSTMcapacity of theCSM conjunctionwas slightly lessthan 2 items, which was significantly lower than that of singlefeatures (CSM vs C, t(5)=7.91, Pb0.01; CSM vs S, t(5)=6.45,Pb0.01; CSM vsM, t(5)=3.83, Pb0.01) (Fig. 1C).

There is concern that the apparent difference in K valuesfor motion vs shape and color reflects a difference in encodingrather than retention. To evaluate this possibility, a supple-mental experiment was conducted in which 4 items werepresented for either 250ms as in the original experiment or for500 ms. Performance was comparable across the two condi-tions (n=6, t(5)=0.41, PN0.69; K=2.29 and K=2.19, respectively;data not shown). That is, performance on themotion directiontrials did not benefit by a manipulation intended to improveencoding, suggesting that VSTM capacity rather than encod-ing difficulty accounted for the observed differences.

Page 3: Human posterior parietal cortex maintains color, shape and motion in visual short-term memory

Fig. 2 – A: The estimated VSTM capacity in the fMRI experiment. B: The brain regions which showed significant load-dependentactivity foreach featureofcolor (red), shape (green), andmotiondirection (blue).Overlappingareaswerecodedaccordinglyasshownbythe topcircles (yellowfor color andshape;magenta forcolorandmotiondirection; cyan for shapeandmotiondirection; andwhitefor color, shape, andmotion direction). (uncorrected; Pb0.001). C: The brain regionsmanifesting significant load-dependent activityduring conjunction condition where subjects were required to maintain all the three features, color, shape and motion directionsimultaneously. (uncorrected; Pb0.001). D:Mean signal changes during the retention interval for group data and the statisticalparametricmap of significant load-dependent activity for each feature on structural scans of the standard brain. The Talairachcoordinates (x, y, z) of peak voxels are 24,−63, 54 for the posterior parietal area and 30,−51, 66 for anterior parietal area. Redboxes outlining the graphs indicate significant load-dependent activity. The letters C, S,M, and CSM in the graphsmean color,shape, motion, and conjunction task, respectively.

93B R A I N R E S E A R C H 1 2 1 3 ( 2 0 0 8 ) 9 1 – 9 7

2.2. fMRI results

To investigate the neural correlates of VSTM under the fourconditions, we performed the fMRI experiments whose designwas identical to that of the behavioral experiment outside of thescanner and expect that numbers of items were manipulatedfrom one to four. The subjects' performance in the scanner wasequivalent to that of the just-discussed behavioral experiment(Fig. 2A). In this experiment, we identified brain regions whereblood oxygen level-dependent (BOLD) activity was sensitive toVSTM load (i.e., regions where activity was significantly cor-related with Cowan's K value). Note that the regions whoseactivity correlated with the number of items in an iconic mem-ory taskwhere subjectswerenot required tomaintain the visualobjects but to detect a red square below the fixation spot wereexcluded from the analysis since theywere considered not to berelated to maintenance but perceptual processing.

Consistent with previous studies (Jha and McCarthy, 2000;Linden et al., 2003; Song and Jiang, 2006; Todd andMarois, 2004,2005; Xu and Chun, 2006a), we found VSTM load-dependentactivity for the single features of color and shape in the bilateralsuperior parietal lobule (Brodmann area 7) in the vicinity of the

intraparietal sulcus (Fig. 2B, D). Activity in the superior parietallobule posterior to the transverse parietal sulcuswas dependenton VSTM load for all features (Fig. 2B; red, green, and blue areas,respectively). Here the load-dependent areas largely overlappedacross the three features, peaking at Talairach coordinates ofx, y, z=24, −66, 50 (right hemisphere) and −19, −63, 57 (lefthemisphere). These overlaps were confirmed by the results ofthe individual analyses (Fig. 3). The Talairach coordinatesclosely match those reported in previous studies using coloreddisks (Todd and Marois, 2004, 2005), shapes (Xu and Chun,2006a), and their conjunction (Song and Jiang, 2006). Interest-ingly, activity in the superior parietal lobule region anteriorto the transverse parietal sulcus, which has previously beenshown to be insensitive to VSTM load for items defined by colorand shape, was load dependent for VSTM of motion direction.

The result of the CSM conjunction condition was consistentwith that of the single feature conditions. Thus, activity in thesame posterior parietal area (peak coordinates: right 24, −69, 51;left −27, −69, 51) correlated with K values (Fig. 2C), as did theactivity in the anterior parietal area,whichwas expected as thiscondition involved retention of motion direction information.More importantly, therewerenoadditional brain areas showing

Page 4: Human posterior parietal cortex maintains color, shape and motion in visual short-term memory

Fig. 3 – The results of individual subject analysis. Scatter plot of K values and BOLD signal change of the right posterior parietalarea (Talairach coordinates 24,−63, 54) which elicited load-dependent activity for all three features in the group analysis (redcircles for color, green triangles for shape, and blue squares for motion direction).

94 B R A I N R E S E A R C H 1 2 1 3 ( 2 0 0 8 ) 9 1 – 9 7

load-dependent activity when features from the two visualstreams were maintained together.

3. Discussion

The current study revealed VSTM differences between featuresprocessedby the dorsal and the ventral visual streams. First, ourbehavioral data showed that VSTM capacity for motion direc-tion was significantly lower than that of color and shape. Sincethe behavioral scoreswere identical for lowVSTM loads (1 and 2items), and increasing the time duration of the sample displayhad no effect on behavioral performance, we may assume thatthe visibility of the three features was the same and differencesin behavioral scores stemmed from differences in storage ca-pacity. Interestingly, in the CSM conjunction condition, therewas a significant decrease in VSTM capacity compared to singlefeature conditions. Althoughweneed to limit our interpretationon this result since the three features are only bound in space,not on a single visual object, it does extend our knowledge onfeature binding. The established view is that simple featuresfrom different dimensions should show little or no competitionand can be stored inparallel (i.e., Olson and Jiang, 2002;Wheelerand Treisman, 2002; Xu, 2002). As one interpretation, we mayconclude that there is an extra cost when features from thetwodistinct processing streams are spatially superimposed andhave to be retained simultaneously. On the other hand, recent

studieshaveproposed that theVSTMcapacitywas varied as thecomplexity (Alvarez and Cavanagh, 2004; Kristjánsson, 2006; Xuand Chun, 2006a,b) or categorization (Olsson and Poom, 2005) ofthe stimulus although there are some controversies (Awh et al.,2007; Luck and Vogel, 1997). As the other explanation, thedecrement of capacity for conjunction would be due to thedifficulty or the perception of three-feature-bound items as theseparated two objects (i.e., a moving field behind a window),similar to previous study which has investigated the effects ofdifferences in apparent depth differences between the parts ofeach object on VSTM performance (Kristjánsson, 2006).

Second, our fMRI data revealed a distinct parietal area lo-cated anterior to the transverse parietal sulcus that plays aspecial role in the retention of motion direction information.Previous studies using the region-of-interest approach haveindicated that areas outside the parietal cortex, such as thelateral occipital complex (Xu and Chun, 2006a) and area V4 (Xuand Chun, 2006b) also show VSTM load-dependent activity (i.e.,for shape and color, respectively). Since these areas are con-sidered to be high-level visual processing modules, the distinctactivity in the parietal cortex, which is exclusive for the motiondirection feature, cannot be dealt with on the same basis. Therelation between this additional parietal activity and thesignificantly decreased VSTM capacity for the motion directionfeature should be clarified in future studies. On the other hand,we could not eliminate the possibility that the anterior parietalactivity would be affected by small eyemovements or difficulty

Page 5: Human posterior parietal cortex maintains color, shape and motion in visual short-term memory

95B R A I N R E S E A R C H 1 2 1 3 ( 2 0 0 8 ) 9 1 – 9 7

of retaining motion, since our experiment did not record eyemovements concomitantly with fMRI measurement.

In contrast, the posterior parietal cortex was sensitive toVSTM load for all visual features, color, shape and motiondirection. The posterior parietal subdivision is believed to codefor the actual features of the maintained objects, as was sug-gested, for example, by Xu and Chun, (2006a). In their study,BOLD activity in this same region varied as a function of objectcomplexity. Togetherwith thepresent results,wemayconcludethat this area maintains visual information from both visualstreams.

It is important to note that there are some papers whichclaim the VSTM systems for object identity and spatial in-formation are partially overlapped (Haxby et al., 2000; Postleet al., 2000), although these two visual processing streams arepartially distinct. Moreover, the VSTM for the spatial informa-tion is assumed to be stored in the parietal regions (Mohr et al.,2006; Munk et al., 2002; Sala and Courtney, 2007). The presentexperiments always required to maintain the location as wellas color, shape, or motion. Thus, although we have focused onthe load effects whichwere different between features and theactivity was not sensitive to the number of location regardlessof feature types, it remains possible that the posterior parietalactivity was affected by the effect of retaining spatial location.

Finally, we refer to the binding problem of online visual per-ception. For decades, neuroscientists have wondered how thevarious features that compose a visual scene are correctly as-signed to specific objects within that scene — a puzzle some-times referred toas “thebindingproblem.”Asonesolution to thisproblem, Treisman and Gelade (1980) proposed a feature in-tegration hypothesis which posits that attentional mechanismsselect objects of interest based on location, and temporarily storethe features observed at that location as entries in “object files”(see also Kahneman et al., 1992; Treisman, 1996, 1998). Accordingto our results on visual maintenance, it may be possible that theposterior parietal region is the neural repository of object files, asingle brain area that gathers and integrates all of the visualfeatures of attentionally selected objects.With respect to featureintegration theory, results from Shafritz et al. (2002) are alsorelevant in that they too have indicated a role for the parietalcortex in featurebindingusing thedelayed recognition task.Theyfound that another subdivision of the posterior parietal cortexshows increased activity during the retention period of delayedrecognition task for conjunctive color–shape features as com-pared with single features, when two objects were presentedsimultaneously at different locations, but not when presentedsequentially at the same position. Interestingly, there was noVSTM load dependence in the reported area. These findings,including our own, are consistent with human lesion studiesshowing that patients with posterior parietal damage failed tocombine object features during online visual perception (Fried-man-Hill et al., 1995; Robertson et al., 1997).

4. Experimental procedures

4.1. Subjects

Six healthy right-handed volunteers (6 males; mean age=25.4 years, range 22–31 years) with normal color vision and

normal or corrected-to-normal visual acuity took part in theinitial behavioral experiment that assessed VSTM capacity forcolor, shape and motion, while twelve healthy right-handedvolunteers (10 males and 2 females; mean age=25.6 years,range, 22–35 years) with normal vision took part in the fMRIexperiment. All subjects gave written informed consent beforethe experiments were performed. The data obtained from threesubjects in the fMRI experiment were excluded from the sta-tistical analysis because they could not discriminate the di-rection of the moving dots in the delayed recognition task.

4.2. Behavioral procedures

In both experiments (i.e., the initial behavioral experiment andthe subsequent fMRI/behavioral experiment), subjects faced acomputer screen upon which dots, contained within a con-toured shape were presented at a particular location. The dotsvaried along the following 3 dimensions: color (all dots withinthe contour were either red, blue, green, or yellow), shape of thecontour within which they were contained (circle, rectangle,triangle, or cross), and motion direction (all dots within thecontour were moving up, down, right, or left — the contourwithin which they were contained did not move).

At the beginning of each trial, a word designating a feature(e.g., color, shape, etc.) to which the subjects should attend waspresented for 2000 ms. After an 800-ms interval, between oneand eight (in the behavioral experiment) or one and four (in thebehavioral/fMRI experiment) dot patterns were presented for250 ms (sample phase). Items were presented at randomlocations in a 3×3 cell matrix of 10.0×10.0° on a black back-ground (0.41 cd/m2). This was followed by a 1200-ms retentioninterval, followed by a 250-ms test display duringwhich a singleprobe itemwaspresented (Fig. 1A). Theprobe itemwasplacedatone of the 8 possible locations previously used on that trial forthe sample item(s). In half of the trials, the probe itemmatchedthesample item. In the remaining trials, one featureof theprobeitem was replaced with that of another item in the sampledisplay, or with a new feature in a case of one sample item. Thesubjects were asked to indicate via button press whether theydetected a change in the probe stimulus (i.e., from the samplephase) or not. Only visual features of interest were varied fromitem to item. Features that were not of interest were keptconstant. The default color, shape, and motion features werewhite, square, and down, respectively. To minimize the use ofcovert verbalization strategies, the subjects were also requiredtomentally rehearse a number (1–99) that was presented over aspeaker just after the instruction cue, and indicate match/non-match between the rehearsed number and a probe number thatwas presented visually just after the test display. The inter-trialinterval (i.e., offset-to-onset) was 1000ms. For the single featureconditions (C, S, M), each subject completed three sessionrepetitions duringwhich the three feature typeswere randomlyintermixed. Each session consisted of 120 trials (3 features [e.g.,shape]×4 ways in which each feature might vary×2 changetypes [changed/not changed]×5 repetitions). In the conjunctioncondition (CSM), all three features were varied between itemsand therefore required subjects tomaintain all three features. Inthis condition, each subject completed one session consisting of144 trials [4 ways in which each feature might vary×2 changetypes×18 repetitions]. VSTM capacity for the 3 single feature

Page 6: Human posterior parietal cortex maintains color, shape and motion in visual short-term memory

96 B R A I N R E S E A R C H 1 2 1 3 ( 2 0 0 8 ) 9 1 – 9 7

conditions and the conjunctive condition were estimated bychange-detection accuracy and Cowan's K formula: K=(num-bers of items in the sample display)×(hit rate+correct rejectionrate−1), where K denotes estimated number of items stored inVSTM (Cowan, 2001).

Furthermore, to examine the effect of iconic representation,40 trials of an iconicmemory taskwere randomly inserted intoeach session, similar to the study of Todd and Marois (2004).The task procedure was identical to that of the change-detection task, except that the subjects were not required tomemorize the sample items. Instead, the subjects were askedtodetect thepresenceor absenceof a small red rectangle abovethe central fixation point in the sample display.

The stimuli were created using Matlab 5.2.0 (Mathworks,Inc. Natick, MA) and projected onto a screen placed at the endof the magnet bore. The subjects viewed the screen via amirror system that was attached to the head coil.

4.3. Image acquisition and analysis

The fMRI experiment was performed using a 1.5-T MRI scanner(Magnetom Sonata, Siemens, Erlangen, Germany) located at theBrain Science Research Center of Tamagawa University. Bloodoxygenation level-dependent (BOLD) contrasts were acquiredwith a T2-weighted gradient-echo echo planar imaging (EPI)sequence (number of axial slices=25, TR=2100 ms, TE=45 ms,flip angle=90°, field of view=192×192 mm2, matrix=64×64,slice thickness=4.0 mm, and slice gap=2.0 mm).

Image preprocessing and statistical analyses were per-formed using statistical parametric mapping software(SPM2; http://www.fil.ion.ucl.ac.uk/spm/spm2.html) on theMatlab platform. All functional volumes were realigned tothe first volume of each subject, corrected for differences inslice timing, spatially normalized to a standard EPI templatein SPM2, and smoothed using a Gaussian kernel (full-widthhalf-maximum, 8 mm). Statistical parametric maps for con-dition-specific effects were calculated for individual subjectsusing a general linear model (GLM). In the GLM, the retentionperiod for each trial condition was modeled as a 1200-msepoch from its onset-to-offset time. A baseline epoch of1000-ms inter-trial interval was also modeled. These epochswere convolved with a canonical hemodynamic responsefunction to create regressors for the GLM. A high-pass filterwith a cut-off period of 128 s was used to eliminate low-frequency noise, and an AR(1) model was used to considerserial correlations in the functional time-series data. Con-trast images representing effects of the fMRI signal changeduring the retention interval relative to the baseline periodwere calculated for each trial condition for each subject.These contrast images were then incorporated into asecond-level, random effects model analysis of analysis ofvariance (ANOVA). A voxel-based multiple regression ana-lysis was performed to isolate the regions in which theactivity increased with the averaged K values. The VSTMload-dependent activity for each feature was identified bythe voxel-wise threshold of Pb0.001, uncorrected for multi-ple comparisons. Here the regions where activity correlatedwith the numbers of items in the iconic memory task wereexcluded from the analysis since they were considered notto be related to maintenance but perceptual processing.

Acknowledgments

The research was supported by Grant-in-Aids for ScientificResearch on Priority Areas – Higher Order Brain Functions –from the Ministry of Education, Culture, Sports, Science andTechnology (17022012 and 17022015) and the 21st centurycenter of excellence (COE) program of the Japan Society forPromotion of Science for Tamagawa University.

R E F E R E N C E S

Alvarez, G.A., Cavanagh, P., 2004. The capacity of visual short-termmemory is set both by visual information load and by numberof objects. Psychol. Sci. 15, 106–111.

Awh, E., Barton, B., Vogel, E.K., 2007. Visual working memoryrepresents a fixed number of items regardless of complexity.Psychol. Sci. 18, 622–628.

Cowan, N., 2001. The magical number 4 in short-term memory: aconsideration of mental storage capacity. Behav. Brain Sci. 24,87–114.

DeYoe, E.A., Van Essen, D.C., 1988. Concurrent processingstreams in monkey visual cortex. Trends. Neurosci. 11,219–226.

Friedman-Hill, S.R., Robertson, L.C., Treisman, A., 1995. Parietalcontributions to visual feature binding evidence from a patientwith bilateral lesions. Science 269, 853–855.

Haxby, J.V., Petit, L., Ungerleider, L.G., Courtney, S.M., 2000.Distinguishing the functional roles of multiple regions indistributed neural systems for visual working memory.NeuroImage 11, 145–156.

Jha, A.P., McCarthy, G., 2000. The influence of memory loadupon delay-interval activity in a working memory task: anevent-related functional MRI study. J. Cogn. Neurosci. 12(Suppl 2), 90–105.

Kahneman, D., Treisman, A., Gibbs, B.J., 1992. The reviewing ofobject files: object-specific integration of information. Cogn.Psychol. 24, 175–219.

Kristjánsson, A., 2006. Surface assignment modulatesobject-formation for visual short-term memory. Perception 35,865–881.

Linden, D.E.J., Bittner, R.A., Muckli, L., Waltz, J.A., Kriegeskorte, N.,Goebel, R., Singer, W., Munk, M.H.J., 2003. Cortical capacityconstraints of visual working memory: dissociation of fMRIload effects in a front-parietal network. NeuroImage 20,1518–1530.

Luck, S.J., Vogel, E.K., 1997. The capacity of visual workingmemoryfor features and conjunctions. Nature 390, 279–281.

Mishkin, M., Ungerleider, L.G., Macko, K.A., 1983. Object vision andspatial vision — 2 cortical pathways. Trends Neurosci. 6,414–417.

Mohr, H.M., Goebel, R., Linden, D.E.J., 2006. Content- andtask-specific dissociations of frontal activity duringmaintenance and manipulation in visual working memory.J. Neurosci. 26, 4465–4471.

Munk, M.H., Linden, D.E., Muckli, L., Lanfermann, H., Zanella, F.E.,Singer, W., Goebel, R., 2002. Distributed cortical systems invisual short-term memory revealed by event-relatedfunctional magnetic resonance imaging. Cerebral Cortex 12,866–876.

Olson, I.R., Jiang, Y., 2002. Is visual short-term memory objectbased? Rejection of the “strong-object” hypothesis. Percept.Psychophys. 64, 1055–1067.

Olsson, H., Poom, L., 2005. Visual memory needs categories. Proc.Natl. Acad. Sci. U. S. A. 102, 8776–8780.

Page 7: Human posterior parietal cortex maintains color, shape and motion in visual short-term memory

97B R A I N R E S E A R C H 1 2 1 3 ( 2 0 0 8 ) 9 1 – 9 7

Postle, B.R., Stern, C.E., Rosen, B.R., Corkin, S., 2000. An fMRIinvestigation of cortical contributions to spatial andnonspatial visual working memory. NeuroImage 11,409–423.

Robertson, L.C., Treisman, A., Friedman-Hill, S.R., Grabowecky, M.,1997. The interaction of spatial and object pathways: evidencefrom Balint's syndrome. J. Cogn. Neurosci. 9, 295–317.

Sala, J.B., Courtney, S.M., 2007. Binding of what and where duringworking memory maintenance. Cortex 43, 5–21.

Shafritz, K.M., Gore, J.C., Marois, R., 2002. The role of the parietalcortex in visual feature binding. Proc. Natl. Acad. Sci. U. S. A. 99,10917–10922.

Simons, D.J., 1996. In sight, out of mind: when objectrepresentations fail. Psychol. Sci. 7, 301–305.

Song, J.H., Jiang, Y., 2006. Visual working memory for simplefeature and complex features: an fMRI study. NeuroImage 30,963–972.

Todd, J.J., Marois, R., 2004. Capacity limit of visual short-termmemory in human posterior parietal cortex. Nature 428,751–754.

Todd, J.J., Marois, R., 2005. Posterior parietal cortex activity predictsindividual differences in visual short-term memory capacity.Cogn. Affect. Behav. Neurosci. 5, 144–155.

Treisman, A., Gelade, G., 1980. A feature integration theory ofattention. Cogn. Psychol. 12, 97–136.

Treisman, A., 1996. The binding problem. Curr. Opin. Neurobiol. 6,171–178.

Treisman, A., 1998. Feature binding, attention and object percep-tion. Philos. Trans. R. Soc. London B 353, 1295–1306.

Vogel, E.K., Woodman, G.F., Luck, S.J., 2001. Storage of features,conjunctions, and objects in visual working memory. J. Exp.Psychol. Human 27, 92–114.

Wheeler, M.E., Treisman, A.M., 2002. Binding in short-term visualmemory. J. Exp. Psychol. Gen. 131, 65–72.

Xu, Y., 2002. Limitation of object-based feature encoding in visualshort-term memory. J. Exp. Psychol. Human 28, 458–468.

Xu,Y., Chun,M.M., 2006a.Dissociableneuralmechanismssupportingvisual short-term memory for objects. Nature 440, 91–95.

Xu, Y., Chun, M.M., 2006b. Brain mechanisms supporting visualshort-termmemory formulti-feature objects. J. Vision 6, 1092a.