9
Research Report Judgments of learning do not reduce to memory encoding operations: Event-related potential evidence for distinct metacognitive processes Ida-Maria Skavhaug a, , Edward L. Wilding b , David I. Donaldson a a The Psychological Imaging Laboratory, Department of Psychology, University of Stirling, Stirling FK9 4LA, UK b Cardiff University, Cardiff University Brain Research Imaging Centre (CUBRIC), School of Psychology, Cardiff, Wales, UK ARTICLE INFO ABSTRACT Article history: Accepted 19 November 2009 Available online 5 December 2009 To examine how judgments of learning (JOLs) are made, we used event-related potentials (ERPs) to compare neural correlates of JOLs and successful memory encoding. Participants saw word pairs, and for each made a JOL indicating how confident they were that they would remember the pairing on a later cued recall task. ERPs were recorded while JOLs were made and were separated according to whether items were: (i) remembered or forgotten on the subsequent test, and (ii) rated likely or unlikely to be remembered. An early positive-going ERP effect was associated with both of these comparisons, whereas a later negative-going effect was present only in the separation based upon JOL ratings. ERP data therefore indicate that JOLs do not reduce to encoding processes that predict the accuracy of memory judgments. © 2009 Elsevier B.V. All rights reserved. Keywords: Memory Encoding Cued Recall Metacognition Judgments of Learning Event-Related Potentials 1. Introduction Judgments of learning (JOLs) are metacognitive assessments of memorability. They are operationalized as estimates of the likelihood that studied material will be subsequently remem- bered (cf. Nelson and Narens, 1990). In most JOL studies, behavioural manipulations have been employed to determine their functional basis. In this experiment, we used brain imaging (event-related potentials ERPs) to investigate the similarities and differences between the cognitive processes that support JOLs, and those that support successful memory encoding. Both neuropsychological (Vilkki et al., 1999; Kennedy and Yorkston, 2000) and psychopharmacological data (Dunlosky et al., 1998; Izaute and Bacon, 2005; Merritt et al., 2005) have indicated dissociations between JOLs and memory. Together, these findings suggest that some processes supporting JOLs operate independently of memory encoding. Consistent with this view, Kao, Davis and Gabrieli (2005) reported functional Magnetic Resonance Imaging (fMRI) results that suggest JOLs are based on a combination of shared and independent neural circuitry. In keeping with previous memory findings (e.g., Wagner et al., 1998; Qin et al., 2007; for reviews see Fernandez and Tendolkar, 2006; Diana et al., 2007), study items that were subsequently remembered rather than forgotten were associ- ated with increased activity in the medial temporal lobes (a subsequent memory effect; Paller et al., 1987). More importantly, whilst some brain regions (including left lateral prefrontal cortex) were equally active for successful encoding and JOLs, other regions (including left ventro-medial and dorso-medial prefrontal cortex) were more active for JOLs than for success- ful memory encoding. BRAIN RESEARCH 1318 (2010) 87 95 Corresponding author. Fax: + 44 1786 467641. E-mail address: [email protected] (I. Skavhaug). 0006-8993/$ see front matter © 2009 Elsevier B.V. All rights reserved. doi:10.1016/j.brainres.2009.11.047 available at www.sciencedirect.com www.elsevier.com/locate/brainres

Skavhaug Et Al 2010. ERPs and Metacognition

  • Upload
    su-aja

  • View
    215

  • Download
    2

Embed Size (px)

DESCRIPTION

erp

Citation preview

  • Research Report

    Judgments of learning do not reduce to memory encodingoperations: Event-related potential evidence for distinctmetacognitive processes

    Ida-Maria Skavhauga,, Edward L. Wildingb, David I. DonaldsonaaThe Psychological Imaging Laboratory, Department of Psychology, University of Stirling, Stirling FK9 4LA, UKbCardiff University, Cardiff University Brain Research Imaging Centre (CUBRIC), School of Psychology, Cardiff, Wales, UK

    A R T I C L E I N F O A B S T R A C T

    Article history:Accepted 19 November 2009Available online 5 December 2009

    To examine how judgments of learning (JOLs) are made, we used event-related potentials(ERPs) to compare neural correlates of JOLs and successful memory encoding. Participantssawword pairs, and for eachmade a JOL indicating how confident theywere that theywouldremember the pairing on a later cued recall task. ERPs were recorded while JOLs were madeand were separated according to whether items were: (i) remembered or forgotten on thesubsequent test, and (ii) rated likely or unlikely to be remembered. An early positive-goingERP effect was associated with both of these comparisons, whereas a later negative-goingeffect was present only in the separation based upon JOL ratings. ERP data therefore indicatethat JOLs do not reduce to encoding processes that predict the accuracy of memoryjudgments.

    2009 Elsevier B.V. All rights reserved.

    Keywords:Memory EncodingCued RecallMetacognitionJudgments of LearningEvent-Related Potentials

    1. Introduction

    Judgments of learning (JOLs) are metacognitive assessmentsof memorability. They are operationalized as estimates of thelikelihood that studied material will be subsequently remem-bered (cf. Nelson and Narens, 1990). In most JOL studies,behavioural manipulations have been employed to determinetheir functional basis. In this experiment, we used brainimaging (event-related potentials ERPs) to investigate thesimilarities and differences between the cognitive processesthat support JOLs, and those that support successful memoryencoding.

    Both neuropsychological (Vilkki et al., 1999; Kennedy andYorkston, 2000) and psychopharmacological data (Dunloskyet al., 1998; Izaute and Bacon, 2005; Merritt et al., 2005) haveindicated dissociations between JOLs and memory. Together,

    these findings suggest that some processes supporting JOLsoperate independently of memory encoding. Consistent withthis view, Kao, Davis and Gabrieli (2005) reported functionalMagnetic Resonance Imaging (fMRI) results that suggest JOLsare based on a combination of shared and independent neuralcircuitry. In keeping with previous memory findings (e.g.,Wagner et al., 1998; Qin et al., 2007; for reviews see Fernandezand Tendolkar, 2006; Diana et al., 2007), study items that weresubsequently remembered rather than forgotten were associ-ated with increased activity in the medial temporal lobes (asubsequent memory effect; Paller et al., 1987). More importantly,whilst some brain regions (including left lateral prefrontalcortex) were equally active for successful encoding and JOLs,other regions (including left ventro-medial and dorso-medialprefrontal cortex) were more active for JOLs than for success-ful memory encoding.

    B R A I N R E S E A R C H 1 3 1 8 ( 2 0 1 0 ) 8 7 9 5

    Corresponding author. Fax: + 44 1786 467641.E-mail address: [email protected] (I. Skavhaug).

    0006-8993/$ see front matter 2009 Elsevier B.V. All rights reserved.doi:10.1016/j.brainres.2009.11.047

    ava i l ab l e a t www.sc i enced i r ec t . com

    www.e l sev i e r . com/ loca te /b ra i n res

    Su Miao Ye Chen

    mailto:[email protected]://dx.doi.org/10.1016/j.brainres.2009.11.047

  • From a theoretical perspective, one interpretation of thefMRI data is that JOLs engage additional metacognitiveassessments that operate downstream of successful encodingoperations. Given the poor temporal resolution of the data,current fMRI results are unable to speak to this possibility.This possibility can, however, be assessed by employing areal-time measure of neural activity, the prediction being thatneural activity differentiating JOLs from successful memoryencoding will occur after (or extend beyond) activity that isshared by these two kinds of information processing opera-tion. Real-time measures of neural activity thus offer theopportunity to contribute to theoretical accounts of JOLs bydelineating the temporal relationships between the differentclasses of cognitive processes that may be engaged.

    In this study, ERPs were acquired whilst JOLs were made.Participants studied a series of word pairs andwere asked howlikely they would be to remember the second word, ifpresented with the first word, during a later test. ERPs wereacquired during the study phase and separated according towhether; (i) the second word was or was not recalledsubsequently, and (ii) the study pair elicited a high or lowJOL. These contrasts permit assessment of the temporal andfunctional correspondences between the neural signatures ofsuccessful memory encoding and JOLs.

    2. Results

    2.1. Behavioral data

    2.1.1. StudyParticipants had a preference for assigning intermediate JOLs(Fig. 1a). ANOVA on response rates revealed a main effect ofJOL [F(4,72)=7.0, p

  • low JOL), hemisphere (left, right) and electrode site (superior,mid, inferior). The outcomes of these analyses for early andlate time windows are shown in Table 1.

    2.2.1. Subsequent memory effects: Table 1aIn the 550- to 1000-ms time window the ANOVAs revealedsignificant interactions between condition and site for all sixrows, reflecting the fact that the subsequent memory effect isa broadly distributed greater relative positivity for remem-bered than for forgotten items that is largest at sites closest tothemidline. From 1300 to 1900ms ANOVA revealed only maineffects of condition from fronto-central to parietal electroderows. As can be seen in Fig. 2, these effects reflect primarily a(weakened) continuation of the effect in the preceding epoch.Given the apparent posterior focus of the early subsequentmemory effect (see Fig. 4), additional analyseswere carried outto investigate whether it was significantly larger at posterior

    electrodes. No statistical support for this impression wasobtained; ANOVA with factors of condition (recall, miss), andelectrode site (FZ, FCZ, CZ, CPZ, PZ, POZ), revealed only a maineffect of condition [F(1.0,19.0)=12.34, plow JOL) that is largest at sites closest tothe midline. Importantly, the similarities between statisticaloutcomes for the two effects do not continue in the later timewindow. From 1300 to 1900 ms ANOVA of the JOL effectsrevealed significant interactions between condition and hemi-sphere from frontal to centro-parietal electrode rows. As Figs. 3and 4 illustrate, ERPs elicited by items attracting high and low

    Fig. 2 Grand average ERPs for subsequently missed items (bold lines) and subsequently recalled items (thin lines). ERPwaveforms are shown for midline frontal (FZ), midline parietal (PZ) and left and right central electrodes (C3 and C4).

    89B R A I N R E S E A R C H 1 3 1 8 ( 2 0 1 0 ) 8 7 9 5

  • JOLsdiffer primarily at left hemisphere sites,where thehigh JOLERPs are markedly more negative-going.

    As for the subsequent memory contrast, additional analy-ses were carried out to investigate whether the apparentposterior maximum of the JOL effect was reliable during theearly time window, using ANOVA with factors of condition(high JOL, low JOL) and electrode site (FZ, FCZ, CZ, CPZ, PZ,POZ). A main effect of condition [F(1.0,19.0)=10.79, p

  • and late JOL effects are generated by at least partially non-overlapping sets of neural generators, and therefore indexdistinct classes of cognitive operations.

    3. Discussion

    We investigated the relationship between judgments oflearning (JOLs) and successful memory encoding usingbehavioral and brain imaging measures. The behavioralresults showed a clear relationship between memory encod-ing and JOLs and the brain imaging results provided novelinsights into this relationship not available via the behavioralone. These insights follow from two critical contrastsbetween ERPs acquired during the experiment study phase;

    ERPs elicited by studied items attracting correct or incorrectjudgments on the subsequent memory test and ERPs elicitedby items attracting high or low JOLs at study. The ERP datawere analyzed for two time windows: early (5501000 ms) andlate (13001900ms). Findings for eachwindow are discussed inturn.

    In both contrasts, reliable and markedly similar effectswere evident from 550 to 1000 ms. If this early positive goingERP effect indexes successful memory encoding (Paller et al.,1987), then the presence of this effect in the high/low JOLcontrast could be taken to suggest that JOLs can be based uponoperations that support successful encoding. The fact thatoverlapping subsets of trials contributed to the subsequentmemory and JOL effects means, however, that the effect couldreflect an encoding process that is only found in the JOLcontrast because of trial overlap, or vice versa. If this was thecase, because trial overlap is not exact, the effect would beconsiderably larger in the condition that was driving it. Thus,the fact that the early effects are of equivalent magnitude forencoding and JOLs therefore provides some evidence for theengagement of commonprocesses during successful encodingand JOL decisions. By this view the presence of the effect inboth contrasts is not an artifact of trial overlap, but a realreflection of shared processing operations.

    Critically, from 1300 to 1900 ms, the subsequent memoryand JOL effects diverged markedly. During the later timewindow a negative going ERP effect was evident for JOLs,but not memory encoding. The later effect differs from theearlier effect in terms of timing, location, polarity andeliciting conditions, strongly suggesting that the two effectsare functionally distinct. In addition, the analyses of scalpdistributions revealed that the neural activity predictingJOLs differed reliably across time windows, indicatingdistinct neural origins (hence functional significances) forthe early shared and late JOL-specific processes. Ourfindings therefore converge with fMRI, neuropsychologicaland behavioural findings in neurologically intact indivi-duals, all of which indicate that the processes supportingJOLs do not reduce to the processes that support successfulencoding.

    The new insight that is not available from precedingstudies is that the JOL-specific neural activity followed activityshared by JOLs and successful encoding operations. Thisobservation is consistent with the view that JOLs are, at leastpartly, a consequence of additional cognitive operations thatoccur downstream of those supporting successful encoding.The time course of the JOL-specific effects revealed here alsoexplains why no comparable neural correlates were found in aprevious ERP study of JOLs. Using faces as the stimuli forwhich JOLs were required, Sommer et al. (1995) observedcomparable neural signatures for JOL and subsequentmemoryeffects. They only examined the ERPs up to 1000 ms post-stimulus, however, thereby precluding identification of late-onsetting JOL effects comparable to the one revealed in thisstudy.

    What is the likely functional significance of the JOL-specificeffect identifiedhere?An important aspect of this finding is thatthe JOL effect emerged froma contrast between items attractingeither high or low JOLs. One consequence of this finding is thatthe effect cannot reflect a general assessment process that is

    Fig. 4 Topographic maps illustrating the scalp distributionsof the subsequent memory effects (subsequent recall minussubsequent miss) and JOL effects (high JOL minus low JOL)over the 5501000 ms and 13001900 ms time windows. Thefront of the head is at the top of each map and the scale barrepresents the size of the effect in V.

    Fig. 5 Map (front of the head at the top) illustrating thechoice of electrodes.

    91B R A I N R E S E A R C H 1 3 1 8 ( 2 0 1 0 ) 8 7 9 5

    Su Miao Ye Chen

    Su Miao Ye Chen

  • Table 1 Outcomes of the analysis of the ERP effects (F=Frontal; FC=Fronto-Central; C=Central; CP=Centro-Parietal; P=Parietal; PO=Parieto-Occipital). (a) Results of ANOVAexamining the subsequent memory effects for the 5501000 ms and 13001900 ms time windows. (b) Results of ANOVA examining the JOL effects for the 5501000 ms and13001900 ms time windows.

    F FC C CP P PO

    (a) Recall/Miss5501000 msCondition F(1.0,19.0)=9.2; p

  • engaged equally whenever a JOL is requiredbecause a JOLwasrequired in all cases any processing associated with therequirement to make a JOL would be invisible in this compar-ison. The effect more likely reflects some aspect of theassessment process that is sensitive to the amount or kind ofinformation that is accessed. For example, low JOL ratingmaybeproducedwhenno information is accessed, andhigh JOL ratingswhen some information is accessed. By this view the electricalrecord indexes a general assessment process that is differen-tially engaged by certain contents, giving rise to the differencesbetween activities across conditions.

    Strong claims about the functional significance of the lateeffect are constrained by the fact that the only electrophys-iological information comes from the result of a binary(high vs. low JOL) contrast. As a result, whether the effect isdriven primarily by high or low JOLs, or reflects a process thatvaries parametrically with JOLs, remains to be seen. Thepresence of the effect, however, indicates that participantshave an imperfect understanding of some of the factors thatinfluence memorability. That is, at least on some trials,participants assigned importance to factors that do not infact contribute substantively to effective encoding. Onepossibility is that the JOL-specific effect reflects the earlierrecovery of a certain kind of content for word pairs that weresubsequently given either high or low JOLs. This account canexplain the current data if the fluency with which informationbecomes available influences JOLs but is a poor predictor ofthe subsequent memorability of the critical stimuli.

    Irrespective of the accuracy of the functional considerations,however, thebehavioral andERP findings in this study (i) supportclaims that the processes differentiating high and low JOLs donot reduce to those that support successful memory encoding,and (ii) demonstrate for the first time that JOL-specific processesoperate downstream of those that are shared between encodingoperations and judgments about the subsequent memorabilityof studiedmaterial. In short, using the high-temporal resolutionof real-time imaging, we provide direct evidence of a distinctneural correlate that may reflect metacognitive processingwhich is engaged when judgments of learning are made.

    4. Experimental procedures

    4.1. Participants

    These were 24 students at the University of Stirling, all ofwhom were right-handed native English speakers with noknown neurological impairments. Participants provided in-formed consent, as approved by the Department of PsychologyEthics Committee at the University of Stirling. They weregiven financial compensation and course credits whereapplicable. Three participants were excluded due to equip-ment failure or excessive EEG artifacts, and one due to poorperformance. The remaining 20 participants (12 female) had amean age of 22 (range: 1730).

    4.2. Stimuli

    Four hundred and twenty word pairs were used as stimuliduring ERP recording (see Appendix 1 for a sample). The pairs

    had a mean forward associative strength of 0.42 (sd=0.16) anda mean backward associative strength of 0.02 (sd=0.02)(according to the norms of Nelson et al., 1998). Twelveadditional word pairs were used for practice.

    4.3. Procedure

    The experiment was conducted on a PC, using Eprime(Psychology Software Tools; www.pstnet.com) and a com-patible response box (Psychology Software Tools). There wasone study session during which JOLs were made, followed bya memory test session (thereby avoiding confounds associ-ated with JOLs made during multiple study-test blocks; seeKoriat et al., 2002; Kelemen et al., 2007). The study phasecomprised 280 trials, each involving a word pair selectedrandomly from the initial 420 pairs. The first word of each ofthe 280 pairs was re-presented at test, along with 140 newwords. Word presentation order was determined randomlyfor each participant. Breaks were at 70 trial intervals, and aninitial practice session familiarized participants with theprocedures.

    Each study trial began with a white fixation crosspresented in the centre of a blue screen for 1000 ms. Aword pair was then presented, one word above and onebelow the central fixation point. Words were displayed inwhite against a blue background, in uppercase 18-pointCourier New font. After 3000 ms a blue screen appeared,replaced after 500 ms by the prompt PROBABILITY TORECALL. This was the instruction for participants toindicate via button press how likely they would be to recallthe second word successfully if presented with the first wordon a subsequent test. Participants were asked to respond ona 5-point scale: 1 (definitely forget), 2 (probably forget), 3(unsure), 4 (probably remember), 5 (definitely remember).The need to make use of the full scale throughout theexperiment was emphasized. Participants were asked to tryto remember the word pairs, but no specific memorizationinstructions were given. After each JOL was made, a bluescreen was presented for 1000 ms before the next trialstarted.

    Each test trial also began with presentation of a whitefixation cross in the centre of a blue screen for 1000 ms. Asingle word was then presented centrally on the blue screen,displayed in white uppercase 18-point Courier New font. Theword remained on the screen for 1500 ms and was followedby a blue screen for 2500 ms, providing a 4 s responsewindow. Participants were instructed to press buttons 1 or 5depending on whether the word was old (presented at study)or new (not presented). If a new response was made (or noresponse occurred within the 4 s window) the trialterminated. Following an old response the prompt CANRECALL? was presented. Participants were asked to pressbuttons 1 or 5 to indicate whether they could or could notremember the word's partner at study. The promptremained visible until a response was made. If theparticipant responded no the current trial was terminated.Following a yes response the prompt RECALL WORDappeared, and participants were instructed to verballycomplete the word pair. After recording the response, theexperimenter initiated the next trial.

    93B R A I N R E S E A R C H 1 3 1 8 ( 2 0 1 0 ) 8 7 9 5

    http://www.pstnet.com

  • 4.4. EEG recording

    EEG was recorded from 62 silver/silver chloride electrodeslocated in an elastic cap (Quick-Cap, Neuromedical Supplies)in accordance with an extended version of Jasper's (1958)international 10/20 system (FP1, FPZ, FP2, AF3, AF4, F7, F5, F3,F1, FZ, F2, F4, F6, F8, FT7, FC5, FC3, FC1, FCZ, FC2, FC4, FC6, FT8,T7, C5, C3, C1, CZ, C2, C4, C6, T8, TP7, CP5, CP3, CP1, CPZ, CP2,CP4, CP6, TP8, P7, P5, P3, P1, PZ, P2, P4, P6, P8, PO7, PO5, PO3,POZ, PO4, PO6, PO8, CB1, O1, OZ, O2, CB2). Additionalelectrodes were placed on the mastoids, above and belowthe left eye (Vertical EOG), and on the outer canthi (HorizontalEOG) to monitor eye movements. Electrodes were referencedto one positioned between CZ and CPZ during recording, thenre-referenced off-line to create an averagedmastoid reference.Electrode impedances were below 5 k at the start of therecording session. Recordings were made using a Synamps2

    amplifier and Neuroscan 4.3 Acquire software (Quick-Cap,Neuromedical Supplies). Signals were amplified with a gain of2010, bandpass filtered (0.140 Hz) and digitized at 250 Hz.

    4.5. EEG data processing

    EEG data were analyzed off-line using Neuroscan 4.3 Editsoftware (Quick-Cap, Neuromedical Supplies: www.neuro.com). Segments of data were rejected if they were excessivelynoisy or clearly saturated (based on visual inspection of wave-forms) to prevent the introduction of artifacts by algorithmsused in later processing stages. Eye blink artifacts wereattenuated using the Neuroscan Ocular Artifact Reductionprocedure (minimum of 32 blinks per participant). EEG studydata were epoched time-locked to stimulus presentation, usinga 104 ms to +2000 ms time window. Epochs were excludedwhere drift (defined as the difference in amplitude between thebeginning and end of each recording epoch) greater than50 Vwas present, or where signal change exceeded100 V. Datawere smoothed over 5 successive points and baseline correctedwith respect to the 104 ms pre-stimulus period.

    Study phase ERPs were formed for four response catego-ries: recalled (items subsequently recognized as old for whichthe study partner was recalled), missed (items judgedincorrectly as being new), high JOL (study pairs assigned aJOL of 4 or 5) and low JOL (JOL of 1 or 2). Study items attractingan unsure JOL (3 responses) were discarded to separate thehigh and low JOL categories. Individual epochswere combinedto produce averaged waveforms for each condition andparticipant. Mean numbers of trials (with a criterion ofminimum of 16 per participant per condition) were 123, 49,87 and 73 for the recalled, missed, high JOL and low JOLcategories.

    The ERP data were analyzed using repeated measuresANOVA, applying the GreenhouseGeisser correction (Green-house and Geisser, 1959) where appropriate. The analyseswere restricted to two post-stimulus time-windows: 5501000and 13001900 ms. These time windows, as well as theelectrode locations submitted to analysis, were selected asthey capture the principal divergences between the criticalERP conditions. The earlier time window is one in whichsubsequent memory effects are typically observed (Palleret al., 1987), and the later time window is designed to best

    identify the novel JOL effects seen in the current data. In theresults section only effects involving condition are reported,as the sites where ERP modulations are largest are interestingonly when they vary with condition.

    Acknowledgments

    Thanks to Mrs. Catriona Bruce for assistance with datacollection.

    DID isamember of the SINAPSECollaboration (www.sinapse.ac.uk), a Pooling Initiative funded by the Scottish FundingCouncil and the Chief Scientist Office of the Scottish Executive.

    Appendix 1

    Typical word pairs included in the experiment.

    WORD1 WORD2 Forwardassociation

    Backwardassociation

    ACRE LAND 0.68 0.02PERFORM ACT 0.29 0.02PRINCIPAL SCHOOL 0.31 0.00LUMBER WOOD 0.59 0.00MOP FLOOR 0.24 0.04

    R E F E R E N C E S

    Diana, R.A., Yonelinas, A.P., Ranganath, C., 2007. Imagingrecollection and familiarity in the medial temporal lobe: athree-component model. Trends Cogn. Sci. 11,379386.

    Dunlosky, J., Domoto, P.K., Wang, M.L., Ishikawa, T., Roberson, I.,Nelson, T.O., Ramsay, D.S., 1998. Inhalation of 30% nitrousoxide impairs people's learning without impairing people'sjudgments of what will be remembered. Exp. Clin.Psychopharmacol. 6, 7786.

    Fernandez, G., Tendolkar, I., 2006. The rhinal cortex: gatekeeperof the declarative memory system. Trends Cogn. Sci. 10,358362.

    Greenhouse, S.W., Geisser, S., 1959. On methods in the analysis ofprofile data. Psychometrika 24, 95112.

    Izaute, M., Bacon, E., 2005. Specific effects of an amnesic drug:effect of Lorazepam on study time allocation and on judgmentof learning. Neuropsychopharmacology 30, 196204.

    Jasper, H.A., 1958. The ten-twenty system of the internationalfederation. Electroencephologr. Clin. Neuropsychol. 10, 371375.

    Kao, Y.-C., Davis, E.S., Gabrieli, J.D.E., 2005. Neural correlates ofactual and predicted memory formation. Nat. Neurosci. 8,17761783.

    Kelemen, W.L., Winningham, R.G., Weaver, C.A., 2007. Repeatedtesting sessions and scholastic aptitude in college students'metacognitive accuracy. Eur. J. Cogn. Psychol. 19, 689717.

    Kennedy, M.R., Yorkston, K.M., 2000. Accuracy of metamemoryafter traumatic brain injury: predictions during verbal learning.J. Speech Lang. Hear. Res. 43, 10721086.

    Koriat, A., Sheffer, L., Ma'ayan, H., 2002. Comparing objective andsubjective learning curves: judgments of learning exhibitincreased underconfidence with practice. J. Exp. Psychol.:General 131, 147162.

    94 B R A I N R E S E A R C H 1 3 1 8 ( 2 0 1 0 ) 8 7 9 5

    http://www.neuro.comhttp://www.neuro.comhttp://www.sinapse.ac.ukhttp://www.sinapse.ac.uk

  • MacCarthy, G., Wood, C.C., 1985. Scalp distribution of event-related potentials: an ambiguity associated with analysis ofvariance models. Electroencephalogr. Clin. Neurophysiol.,Evoked Potentials 62, 203208.

    Merritt, P., Hirshman, E., Hsu, J., Berrigan, M., 2005. Metamemorywithout the memory: are people aware of midazolam-inducedamnesia? Psychopharmacology 177, 336343.

    Nelson, D.L., McEvoy, C.L., Schreiber, T.A., 1998. The University ofSouth Florida word association, rhyme and word fragmentnorms. http://www.usf.edu/FreeAssociation/.

    Nelson, T.O., 1984. A comparison of current measures of theaccuracy of feeling-of-knowing predictions. Psychol. Bull. 95,109133.

    Nelson, T.O., Narens, L., 1990. Metamemory: a theoreticalframework and new findings. In: Metcalfe, J., Shimamura, A.P.(Eds.), 1994. Metacognition: Knowing about knowing. MITPress, Cambridge, MA.

    Paller, K.A., Kutas, M., Mayes, A.R., 1987. Neural correlates ofencoding in an incidental learning paradigm.Electroencephalogr. Clin. Neurophysiol. 67, 360371.

    Qin, S., Piekeman, C., Petersson, K.M., Han, B., Luo, J., Fernandez,G., 2007. Probing the transformation of discontinuousassociations into episodic memory: an event-related fMRIstudy. NeuroImage 38, 212222.

    Sommer, W., Heinz, A., Leuthold, H., Matt, J., Schweinberger, S.R.,1995. Metamemory, distinctiveness, and event-relatedpotentials in recognitionmemory for faces. Mem. Cogn. 23, 111.

    Vilkki, J., Surma-aho, O., Servo, A., 1999. Inaccurate prediction ofretrieval in a face matrix learning task after right frontal lobelesions. Neuropsychology 13, 298305.

    Wagner, A.D., Schacter, D.L., Rotte, M., Koutstaal, W., Maril, A.,Dale, A.M., Rosen, B.R., Buckner, R.L., 1998. Building memories:remembering and forgetting of verbal experiences as predictedby brain activity. Science 281, 11881191.

    95B R A I N R E S E A R C H 1 3 1 8 ( 2 0 1 0 ) 8 7 9 5

    http://www.usf.edu/FreeAssociation/

    Judgments of learning do not reduce to memory encoding operations: Event-related potential evid.....IntroductionResultsBehavioral dataStudyTest

    Brain imaging dataSubsequent memory effects: Table 1aJOL effects: Table 1bAnalyses of scalp distributions

    DiscussionExperimental proceduresParticipantsStimuliProcedureEEG recordingEEG data processing

    AcknowledgmentsAppendix 1References