Recollection and confidence in two-alternative forced choice episodic recognition

Journal of Memory and Language 62 (2010) 183–203

Contents lists available at ScienceDirect

Journal of Memory and Language

journal homepage: www.elsevier .com/locate / jml

Recollection and confidence in two-alternative forced choiceepisodic recognition

Andrew Heathcote *, Beatrice Bora, Emily FreemanSchool of Psychology, The University of Newcastle, Australia

a r t i c l e i n f o

Article history:Received 9 July 2009revision received 30 October 2009Available online 8 December 2009

Keywords:Recognition memoryMemory modelingFace recognition memoryRecognition memory confidenceRemember–know

0749-596X/$ - see front matter � 2009 Elsevier Incdoi:10.1016/j.jml.2009.11.003

* Corresponding author. School of Psychology, PsyUniversity of Newcastle, University Avenue, CallagFax: +61 2 49216906.

E-mail address: [email protected]

a b s t r a c t

We used quantitative modeling of remember–know responses (Tulving, 1985) to investi-gate the processes underlying recognition memory for faces in the choice-similarity para-digm (Tulving, 1981). Similarity between recognition test choices produces opposite effectson confidence and accuracy in this paradigm. We extended existing models of this doubledissociation to account for remember–know responses, by adding a variable recollectioncriterion to Clark’s (1997) single-process model and by adding graded recollection strengthto Dobbins, Kroll, and Liu’s (1998) dual-process model. Both models provided an accurateand comprehensive account of objective and subjective judgments in an experiment weconducted on memory for faces, and of data from Dobbins et al. on memory for naturalscenes. Model selection techniques were used to refine these accounts, providing insightinto the psychological processes proposed by each approach and into their implicationsfor the relationship between recollection and confidence in two-alternative forced choicerecognition.

� 2009 Elsevier Inc. All rights reserved.

Introduction (Yonelinas, 2002). Mandler (1980) illustrated dual-process

In this paper we investigate the relationship betweenconfidence and recollection in episodic recognitionmemory. Episodic recognition requires a decision aboutwhether a test item appeared in a previous study episode.Recollection is a process that retrieves from memory de-tails that were associated with a test item during the studyepisode. For example, on first encountering a person youmay be struck by their hooked nose, which reminds youof a classical Roman statue. When you later meet this per-son again you recollect thinking about their Roman noseon your first encounter. That recollection serves to confirmthat you have previously met the person, allowing you toconfidently greet them on your second meeting.

The idea that episodic recognition is in part based onrecollection is commonly called dual-process theory

. All rights reserved.

chology Building, Thehan 2308, Australia.

u.au (A. Heathcote).

theory with a story about encountering a man on a bus.The man seemed highly familiar, but it was only after aconscious search process, during which memory wasprobed with various potential contexts, that the man wasrecognized as the butcher from the supermarket. Mandlergoes on to propose that, although the context-free famil-iarity occurring in the ‘‘butcher-on-the-bus” example isunusual, it illustrates the two processes that generallyunderpin episodic recognition, and that: ‘‘In the normalcourse of events, the two separate processes occur con-jointly; recognition involves the additive effects of famil-iarity and retrieval.” (p. 253). Rather than a literallyadditive relationship, Mandler proposed an either/or pro-cess for using these two sources of mnemonic information.If recollection occurs, the recognition decision is basedonly on recollection. When recollection fails, the decisionis based on familiarity, as familiarity is always available.This proposal has been adopted by all subsequent dual-process theories, with the exception of Wixted (2007).

Participants are often asked to rate their confidence inrecognition decisions, and are usually more confident in

http://dx.doi.org/10.1016/j.jml.2009.11.003

mailto:[email protected]

http://www.sciencedirect.com/science/journal/0749596X

http://www.elsevier.com/locate/jml

1 We avoided the more commonly used term target-lure similaritybecause that could also apply to the similarity between choices in a two-alternative forced-choice test (i.e., what we call choice-similarity). The termmemory-similarity emphasizes the relationship between a memory traceand a test lure, which is what our memory-similarity manipulation affects,while controlling for choice-similarity.

184 A. Heathcote et al. / Journal of Memory and Language 62 (2010) 183–203

conditions where they are also more accurate. In extendingdual-process theory to explain recognition confidence,Yonelinas (1994) proposed that recollection results inhighly confident recognition. In contrast to Mandler(1980), he assumed familiarity is continuous and normallydistributed, with a higher mean for studied (old) thanunstudied (new) test items, but with equal variance inboth cases. Confidence based on familiarity is determinedin the manner of signal detection theory (Macmillan &Creelman, 2005), by placing a set of criteria on the familiar-ity dimension. Values of familiarity above the largest crite-rion result in a highly confident old decision even in theabsence of recollection. Less confident decisions are givenwhen familiarity falls between intermediate criteria. Yone-linas showed that this model provided an accurate accountof Receiver Operating Characteristic (ROC) data obtainedby having participants rate their confidence in single testitem recognition decisions.

We investigate the extension of the dual-process ap-proach to confidence in two-alternative forced choice(2AFC) episodic recognition decisions. We focus on 2AFCresponding because, in contrast to the usual finding in sin-gle item recognition, confidence and accuracy can becomedissociated (i.e., less accurate responses are given withgreater confidence) when the two choice alternatives aresufficiently similar. In particular, Tulving (1981) found thatepisodic recognition of natural scene stimuli was moreaccurate but less confident when the choice alternativeswere highly similar compared to when they were lesssimilar. His finding is surprising, not only because theconfidence–accuracy dissociation differs from the usualfinding of a positive relationship between confidence andaccuracy, but also because similarity is usually found tobe detrimental rather than helpful for recognition accuracy(e.g., Wickelgren, 1977).

Dobbins, Kroll, and Liu (1998) used Tulving’s (1985)‘‘remember–know” (RK) methodology to investigate thecause of Tulving’s (1981) surprising findings. The RK meth-odology relies on participant’s ability to report whetherepisodic recognition decisions are based on recollectionor familiarity. In an experiment using natural scene stimuliDobbins et al. found that recognition decisions were lessoften classified as based on recollection when choice-sim-ilarity was high than when it was low. They suggested thathigh choice-similarity tends to result in recollection ofboth the target (i.e., the previously studied test alternative)and the distracter (i.e., false recollection). Participants re-spond to the conflicting recollections by basing their recog-nition decision on familiarity. As familiarity, on average,produces lower confidence decisions, confidence tends tobe less when choice-similarity is high than when it islow. However, accuracy is improved because errors causedby false recollection are less common when choice-similar-ity is high. That is, because false recollection is correlatedwith correct recollection when choice-similarity is higher,participants are less often misled by false recollection thanwhen choice-similarity is lower.

In summary, Dobbins et al. (1998) proposed a dual-pro-cess explanation of both Tulving’s (1981) confidence–accu-racy dissociation and of the effect of choice-similarity onRK responses. Recently, Heathcote, Freeman, Etherington,

Tonkin, and Bora (2009) replicated the confidence–accu-racy dissociation with face stimuli. Here we use the RKmethodology to investigate the cause of the confidence–accuracy inversion with face stimuli. To do so we fit anextension of Dobbin’s et al.’s dual-process model to ouraccuracy, confidence and RK data. We also do the samewith an extension of an alternative model of the confi-dence–accuracy dissociation proposed by Clark (1997).Clark’s model is based on MINERVA 2, Hintzman’s (1988)‘‘single-process” theory, which assumes that episodic rec-ognition decisions are based on a single type of memoryrepresentation and matching process. We then comparethe accounts provided by the two alternative models. Be-fore describing the extended models we first describe ourexperimental paradigm and discuss the differing ways inwhich RK data are interpreted by single and dual-processtheories.

The face choice-similarity paradigm

In our paradigm participants briefly studied each of asequence of pictures of male and female faces and werethen tested by 2AFC decisions about pairs of new and oldfaces. The test pairs were either higher (same gender) orlower (different gender) in choice-similarity. In some casesthe distracter or new faces were specifically chosen to besimilar to a studied face. These new faces might appearwith the studied face to which they are similar, in a highchoice-similarity test pair. Alternately, they might be sim-ilar to an untested study face and appear in a low choice-similarity pair. On each test trial participants chose eitherthe left or right member of the pair and rated theirconfidence in the choice. They then gave an RK response,indicating whether their decision was based on the recol-lection of details (remember) or familiarity (know).

This paradigm is based on the experiment reported byDobbins et al. (1998), which in turn was based on severalexperiments reported by Tulving (1981). All of theseexperiments examined memory for scenic pictures ratherthan faces. Choice-similarity was manipulated by compar-ing performance for choices between studied and unstud-ied halves of the same scene versus different scenes.Participants were more confident in their decisions in thelow choice-similarity (different scene) condition, butchoice-similarity had the opposite effect on accuracy:accuracy in the high choice-similarity (same scene)condition was greater than in the low choice-similaritycondition.

Note that the choice-similarity effect occurs when theeffect of memory-similarity (i.e., the similarity between thenew test item and memory traces)1 is controlled. That is,the low choice-similarity pairs used in the comparison hada new item that was similar to an untested study item tothe same degree that the new test item in the high

A. Heathcote et al. / Journal of Memory and Language 62 (2010) 183–203 185

choice-similarity condition was similar to the old test item.Tulving (1981) found that the dissociation between confi-dence and accuracy only occurred when memory-similaritywas sufficiently high. He manipulated memory-similaritybased on the results of a calibration study in which his scenestimuli were rated on the similarity of their halves. The rat-ings were used to sort scenes into higher and lower mem-ory-similarity sets, and each set was used to create higherand lower choice-similarity test pairs.

Heathcote et al.’s (2009) replication of the confidence–accuracy dissociation with face stimuli also used a ratingmethod to create higher and lower similarity sets of facepairs in order to investigate the effect of memory-similar-ity. In contrast to Tulving (1981), they found that the disso-ciation occurred with both lower and higher memory-similarity sets. Likely this occurred because all of their facepairs were chosen to be similar to some degree, and be-cause faces tend to be intrinsically more similar to eachother than parts of scenes. Hence, even Heathcote et al.’slower similarity faces pairs were sufficiently similar tosupport the confidence–accuracy dissociation.

Heathcote et al. (2009) used state-trace analysis (Bam-ber, 1979) to investigate the cause of the confidence–accu-racy dissociation. In contrast to other methods, state-traceanalysis can rigorously determine whether more than asingle psychological dimension (i.e., a single latent vari-able, module, or process) is required to explain a dissocia-tion (Dunn & Kirsner, 1988; Loftus, Oberg, & Dillon, 2004).They found that more than one dimension was required toexplain the confidence–accuracy dissociation theyobserved.

Dobbins et al. (1998) took a different approach to inves-tigating the cause of the confidence–accuracy dissociation,by having participants give an RK response following asimultaneous choice and confidence rating. As well as find-ing that remember responses were less common for higherchoice-similarity pairs than lower choice-similarity pairs,they also found that the choice-similarity effect on accu-racy occurred only in remember responses. In contrast,they found that the confidence effect occurred only inknow responses. These results provide a potential psycho-logical-process explanation for Heathcote et al.’s (2009)state-trace findings. That is, the confidence–accuracy dis-sociation is caused by different effects of choice-similarityon underlying recollection and familiarity processes. How-ever, as we now discuss, other interpretations of RK datahave been suggested.

The remember–know paradigm

The interpretation of RK findings is controversial. It hasbeen suggested that confidence and RK judgments do notreflect separate recollection and familiarity processes.Rather, both depend on a single ‘‘memory strength”dimension, and both types of judgment are made using adecision process that compares memory strength to a setof criteria in the manner of signal detection theory (Don-aldson, 1996). In this view, remember judgments are likehigh confidence judgments, in that they occur whenmemory strength is greater than a high criterion, withparticipants adopting different confidence and RK criteria

depending on experimental demands. Wixted and Stretch(2004) elaborated this view, suggesting that RK criteriaare more variable than confidence criteria because partici-pants are highly practiced at making confidence judg-ments, whereas they only encounter the idea of a RKjudgment just prior to the experiment.

This alternative interpretation is sometimes called thesingle-process or signal-detection RK model. For our pur-poses here the key difference between the two interpreta-tions of RK data is that the dual-process approach assumesconfidence and RK decisions are based on one type of evi-dence (i.e., recollection), or another (i.e., familiarity), butnot both at once. In contrast, the single-process approachassumes that both types of decisions are based the samememory strength dimension, which combines all sourcesof available mnemonic evidence about the test item.

Determining whether the single- or dual-process RKinterpretation is correct has not proved amenable to whatis often thought to be the ideal test, strong inference (Platt,1964). A strong inference test compares models by con-trasting different predictions about the order of perfor-mance across experimental conditions. Such tests areonly valid if the predictions are parameter free, that is, ifthe predictions hold for all reasonable values of the mod-el’s parameters. Dunn (2004) evaluated five tests thathad been proposed by proponents of dual-process ap-proach, and showed that none were valid. He also showedthat quantitative fits of the single-process model werequite consistent with a large database of RK data.

Similarly, Gardiner, Ramponi, and Richardson-Klavehn’s(2002) evidence favoring the dual-process model of RKdata was found to be flawed by Macmillan, Rotello, andVerde (2005), due to Gardiner et al.’s use of the A0 measureof accuracy (see also Benjamin, 2005). Macmillan et al.concluded that quantitative models are necessary to inter-pret RK data (see Dougal & Rotello, 2007; Kapucu, Rotello,Ready, & Seidel, 2008; Rotello, Macmillan, Hicks, & Hautus,2006; Starns & Ratcliff, 2008, for applications of this ap-proach to single item recognition).

However, the dual-process approach has been criticizedby Wixted and Stretch (2004) because it has not been elab-orated to provide a comprehensive quantitative account ofRK data. In particular, they pointed out that most existingdual-process models do not account for false remember re-sponses (i.e., incorrect recognition responses that are clas-sified as being based on recollection). They suggested thata comprehensive dual-process account requires a false rec-ollection process to account for remember errors, and thatboth correct and false recollection processes must providegraded evidence, as is assumed by some dual-process mod-els (e.g., Cary & Reder, 2003). Rotello, Macmillan, Reeder,and Wong (2005) reported empirical results supportingthe graded nature of the remember response in single itemrecognition.

Dunn (2008) made a potentially even more telling crit-icism of the dual-process account. He applied state-traceanalysis to a database of 37 RK studies. His results pro-vided little or no support for the claim that two processesare necessary to explain RK judgments. It is possible thatone dimension was sufficient because the experimentalmanipulations in these studies caused strongly correlated


changes in recollection and familiarity, despite the factthat was proposed not to be the case in the original inter-pretations of these results. If such a correlation is suffi-ciently high, state-trace analysis will indicate that onlyone dimension is necessary to explain the data.

However, given Heathcote et al.’s (2009) finding thatmore than one dimension is required to explain the confi-dence–accuracy dissociation they observe, the choice-sim-ilarity paradigm appears not to be subject to this problem.We note effects due to more than one dimension in thisparadigm do not uniquely support the dual-process ac-count. They could also be explained by a single-processmodel in which choice-similarity affects different parame-ters controlling a single memory strength dimension (e.g.,its mean and variance). Hence, we examined both singleand dual-process accounts of our data, rather than assum-ing one or the other to be correct.

Our examination of the alternative models is quantita-tive and comprehensive, in the sense that we fit each typeof model to the full array of RK, confidence and 2AFC data(i.e., both correct and false response proportions brokendown by both confidence and RK classifications). In orderto do so we elaborated two existing accounts of thechoice-similarity effect, Clark’s (1997) single-process mod-el and Dobbins et al.’s (1998) dual-process model. The elab-orations were along the lines suggested by Wixted andStretch (2004), adding a signal-detection account of RK re-sponses to Clark’s model and graded false and correct recol-lection to Dobbins et al.’s model. We first report the resultsof our experiment in terms of the confidence, accuracy andremember-response-proportion measures used in the anal-ysis of previous choice-similarity experiments. We thenprovide details of the base models, and their elaborations,which we denote the single-process remember–know(SPRK) and dual-process remember–know (DPRK) models.

Table 1Characteristics of the 300 critical experimental face pairs.

Gender Race Similarity Mean rating (%) Number

Female Asian Lower 36 10Higher 60 10

Black Lower 37 20Higher 66 20

White Lower 37 45Higher 60 45

Male Asian Lower 31 10Higher 71 10

Black Lower 35 20Higher 60 20

White Lower 37 45Higher 69 45

Experiment

Face stimuli (105 � 120 pixel black and white bitmaps)were taken from the FERET database (Phillips, Wechsler,Huang, & Rauss, 1998), which provides faces classified bygender and race (Black, Asian and White). Face imageswere sorted into 377 pairs and rated on the similarity be-tween pair members by ten first year psychology studentsusing a 5 point scale (1 = very low to 5 = very high). The facepairs were then rank ordered within gender and race cate-gories using average similarity ratings. Higher and lowersimilarity sets, consisting of 150 pairs each, were createdby selecting lowest and highest ranked pairs. The two setswere used to manipulate memory-similarity.

We crossed memory-similarity with a three levelchoice-similarity factor of the same type used by Tulving(1981), Experiment 1) and Dobbins et al. (1998). Followingtheir nomenclature, studied test items are denoted A,unstudied (new) test items whose pair mates were studiedare denoted by a prime (e.g., B0 if B was studied), and newtest items whose pair mates were never studied are de-noted X0. Hence, the higher choice-similarity condition isdenoted A–A0. The other two conditions, which have lower(and equal) choice-similarity, are denoted A–B0 and A–X0.

A–B0 has the same average match to memory as A–A0, andA–X0 has a lower average match to memory compared toboth A–A0 and A–B0, as the pair mate of X0 was not studied.

In previous experiments with scenes (Dobbins et al.,1998; Tulving, 1981), random assignment was used to cre-ate lower choice-similarity test pairs. In contrast, our lowerchoice-similarity pairs were always made up of faces withdifferent genders (higher choice-similarity pairs made upof faces of the same gender and study lists consisted offaces from only one race with equal numbers of each gen-der). This was done, in light of the structural similarity be-tween randomly chosen pairs of faces, to insure clearlylower perceptual similarity between test pairs in the A–B0

and A–X0 conditions than in the A–A0 condition.Note that Heathcote et al. (2009) performed experi-

ments with a subset of the face stimuli used here and asimilar design, except that they paired different race facesrather than different gender faces to create the lowerchoice-similarity condition, and they omitted the A–X0 con-dition (as in Tulving, 1981, Experiment 2). Analyses ofaccuracy and confidence produced largely the samepattern of results as here, indicating that gender andrace related choice-similarity manipulations had similareffects.

Method

Participants

Participants were 64 introductory psychology studentsat the University of Newcastle, Australia, who receivedcourse credit for participation.

Apparatus and procedure

Table 1 provides the rating results and details of thegender and race classification for the 300 critical pairs.One member of each critical pair was randomly selectedand used to create 15 study lists (2 Asian, 4 Black and 9White), made up of 10 lower and 10 higher memory-sim-ilarity items, half male and half female. Assignment tostudy lists, and study order was random. One face ap-peared before, and one after, the critical faces as primacyand recency buffers which were not tested. The buffer


faces, and faces used for an initial practice study-test cycle,were drawn from the remaining 77 rated face pairs. Testpairs were presented side-by-side. Test lists had equalnumbers of male and female faces. They consisted of 12pairs, two from each of the six memory- and choice-simi-larity conditions. Note that the same critical pair was neverused twice in a study list. For example, if the pair of itemsA–B0 was tested, a test pair containing B (e.g., B–C0) wasnever tested. Study faces were randomly allocated tochoice-similarity conditions, and test pairs were presentedin a random order.

An eight button response box, consisting of left andright hand clusters of three keys and a central pair of keys,was used to simultaneously record confidence and accu-racy, with the left and right clusters labeled 3, 2, 1 and 1,2, 3 from left to right. Larger numbers indicated greaterconfidence, and participants were instructed to make useof all confidence levels. Pressing a button in the left clusterindicated a left test choice, and pressing a button in theright cluster indicated a right choice. The central pair ofkeys was used to make RK judgments. Participants wereinstructed to press the left button, labeled remember, ifthey remembered seeing the face, or particular elementsof the face, or the right button, labeled familiar, if the facewas familiar but they did not remember the face or anyparticular elements of the face (these instructions weremodeled on Dobbins et al., 1998). During study, buttonsmarked 1–3 were used to make face typicality ratings(ranging from 1 = very typical to 3 = very unusual). Studyratings were used to ensure attention to the faces and werenot further analyzed.

Each participant was tested on a PC with a 1168 � 856resolution monitor. The session began with participantsreading instructions on the screen at their own pace. Dur-ing study, faces were displayed one at a time in the middleof the screen for 2 s. After each face appeared, participantswere prompted to make a typicality rating. The test phasebegan immediately after study. In the test phase, face pairsappeared one pair at a time. If no response was made after6 s, the next face pair was displayed. The experiment lastedless than 1 h.

Results

Three participants were excluded from analysis, one be-cause of a computer failure, one because they rarely usedthe lowest confidence rating and one because they rarelyused the highest confidence rating. Linear mixed effectsmodels (Bates. D. M., 2005) with random subject interceptswere used to obtain population mean estimates and stan-dard errors, and to perform inference. We report the re-sults of these analyses in terms of single degree offreedom tests, and test results where p < .05 are describedas reliable.

Note that, as in Tulving’s (1981) experiment, the cross-ing of choice and memory-similarity factors is not strictlyorthogonal. For high choice-similarity (A–A0) pairs, forexample, increasing memory-similarity also increaseschoice-similarity. In the same vein, A–B0 and A–X0 pairsare equated for choice-similarity but differ in memory-similarity. Hence, the single degree of freedom contrasts

that we report are more appropriate than standard ANOVAmain effect and interaction tests.

Fig. 1 presents population estimates of response pro-portion results; the percentages of correct and falseremember responses and the percentages of accurateremember and know responses. We assumed a binomial-probit error model (McCullagh & Nelder, 1989) and usedmaximum-likelihood estimation to analyze this data.These analyses produced standard normal (Z) test statisticsderived from the mixed model parameter estimate vari-ance–covariance matrix. To avoid redundancy we reportonly the corresponding null-hypothesis probability values.

We also applied the same analysis to overall accuracy(i.e., without dividing responses into remember and know).As overall accuracy is not easily obtained from Fig. 1, weaccompany reports of reliable effects in overall accuracywith the corresponding average percentages of correctresponses.

Remember vs. know response proportions

As was found by Dobbins et al. (1998), remember re-sponses were less common when choice-similarity washigher (A–A0) than when it was lower (A–B0 and A–X0).These differences were reliable for both correct (p = .005and p = .012 respectively) and false (p < .001 and p = .014respectively) remember responses. In contrast, the twolower choice-similarity conditions did not differ reliably,although false remember responses were marginally lesscommon in the A–X0 than the A–B0 condition for lowermemory-similarity pairs (p = .051). Memory-similarityhad little effect on the frequency of remember responses,with the exception of a marginally higher overall percent-age of correct remember responses for lower compared tohigher memory-similarity pairs (p = .078).

Accuracy

Consistent with Tulving’s (1981) results, overall accu-racy in the A–A0 condition was reliably greater than in theA–B0 condition when memory-similarity was high (86%vs. 80%, p < .001), but not when it was low. In the A–B0 con-dition, overall accuracy was greater when memory-similar-ity was high than when it was low (86% vs. 80%, p < .001).No other effects of memory-similarity or choice-similarityon overall accuracy approached significance.

Results for our high memory-similarity condition repli-cated Dobbins et al. (1998) in that the accuracy of remem-ber responses was less in the A–B0 condition than in the A–A0 and the A–X0 conditions (p < .001 and p = .009 respec-tively). These effects were not reliable when memory-sim-ilarity was lower (p = .097 and p = .179 respectively),although equivalent trends were evident. In contrast toDobbins et al. (1998), the accuracy of know responseswas reliably affected by choice-similarity, at least whenmemory-similarity was higher (A–B0 < A–A0, p = .003, andA–B0 < A–X0, p = .012). There was essentially no effect ofchoice-similarity on the accuracy of know responses whenmemory-similarity was lower (p > .5 in both cases).

The accuracy of remember responses was also reliablygreater for lower than higher memory-similarity pairs in

5055

6065

Correct

Choice Similarity

Rem

embe

r (%

)

A-A' A-B' A-X'

1520

2530

False

Choice Similarity

Rem

embe

r (%

)

A-A' A-B' A-X'

8590

9510

0

Remember

Choice Similarity

Accu

racy

(%)

A-A' A-B' A-X'

Memory SimilarityLowHigh

6570

7580

Know

Choice Similarity

Accu

racy

(%)

A-A' A-B' A-X'

Fig. 1. Population mean estimates of the proportion of remember responses and accuracy, with 95% confidence intervals derived from the mixed modelvariance–covariance matrix.


the A–B0 (p < .001) and A–X0 (p = .013) conditions, but notin the A–A0 condition. For know responses only the differ-ence in the A–B0 condition was reliable (p = .031).

Confidence

Fig. 2 shows confidence results broken down by accu-racy and RK response. For ease of interpretation, confi-dence was transformed to 0–100 scale, which we refer toas a percentage, using 100 � (r � 1)/2, where r is the 1–3integer confidence rating. We assumed a Gaussian errormodel and used restricted maximum-likelihood estimationin the analysis of confidence. Frequentist inferential resultsfor the Gaussian error model (using t-statistics) werechecked using Bayesian methods, as recommended by Baa-yen, Davidson, and Bates (2008). Both methods producedthe same conclusions and we report null-hypothesis prob-ability values obtained using the latter method.

We also applied the same analysis to overall confidence(i.e., without dividing responses into remember and know).As overall confidence is not easily obtained from Fig. 2, weaccompany reports of reliable effects in overall confidencefor correct and error responses with the correspondingaverage percentages of correct responses.

Choice-similarity affected overall confidence in error re-sponses but had no reliable effect on overall confidence incorrect responses. For error responses to low memory-sim-ilarity pairs, confidence was higher in the A–B0 condition(35%) than in the A–A0 condition (27%, p = .002) and theA–X0 condition (30%, p = .009). When memory-similaritywas high, error response confidence was also higher inA–B0 condition (34%) than in the A–A0 condition (25%,p = .001) but not the A–X0 condition (33%). Confidence incorrect responses was, however, reliably greater whenmemory-similarity was low than when it was high in boththe A–B0 (55% vs. 51%, p = .011) and A–A0 (54% vs. 52%,p = .04) conditions, but not in the A–X0 conditions. Mem-ory-similarity did not have any reliable effects on error re-sponse confidence.

Replicating Dobbins et al. (1998), no choice-similarityeffects were reliable for remember confidence. In contrast,there were reliable effects on know confidence, mainly forerrors. Confidence effects for correct responses were re-stricted to lower memory-similarity pairs, with A–B0 confi-dence reliably lower than A–A0 (p = .005) and A–X0 (p = .03)confidence. For error responses, confidence in the A–B0

condition was reliably greater than confidence in theA–A0 condition. This was true for both lower and higher

6570

7580

Correct Remember

Choice Similarity

Con

fiden

ce (%

)

A-A' A-B' A-X'

5055

6065

70

False Remember

Choice Similarity

Con

fiden

ce (%

)

A-A' A-B' A-X'

2025

3035

Correct Know

Choice Similarity

Con

fiden

ce (%

)

A-A' A-B' A-X'

Memory SimilarityLowHigh

1520

2530

False Know

Choice Similarity

Con

fiden

ce (%

)

A-A' A-B' A-X'

Fig. 2. Population mean estimates of confidence broken down by accuracy and remember–know, with 95% confidence intervals derived from the mixedmodel variance–covariance matrix.


memory-similarity pairs (p = .013 and p = .001, respec-tively). A–B0 confidence was reliably greater than A–X0 con-fidence for lower (p = .03) but not higher (p = .73) memory-similarity pairs. Memory-similarity had only one reliableeffect, greater confidence for remember responses to lowthan high memory-similarity pairs in the A–B0 condition(p = .02).

Discussion

We largely replicated effects related to the proportionof remember responses reported by Dobbins et al. (1998).Remember responses were less common when choice-sim-ilarity was higher (A–A0) than when it was lower (A–B0 andA–X0). We also found that memory-similarity did not havea reliable effect on the proportion of remember responses(note that Dobbins et al. did not manipulate memory-sim-ilarity). Our accuracy results differed from those of Dob-bins et al. in that the A–B0 condition had lower accuracythan the A–A0 and A–X0 conditions for both rememberand know responses, at least when memory-similaritywas higher. In contrast, they found a reliable effect onaccuracy only for remember responses. They also foundthat accuracy was greater in the A–X0 condition than the

other two conditions, as did Tulving (1981), whereas accu-racy was approximately equivalent in the A–A0 and A–X0

conditions in our experiment.Our confidence results replicated some aspects of

Tulving’s (1981) and Dobbins et al.’s (1998) results, butnot others. Like Dobbins et al. we found that choice-simi-larity effects on confidence were only evident for know re-sponses. In our case the confidence effect was most evidentfor false know responses (i.e., incorrect recognition re-sponses that are classified as being based on familiarity),with lower confidence when choice-similarity was higher(A–A0) than when it was lower (A–B0 and A–X0). For correctknow responses our confidence effects were much weaker,and unexpectedly there was a small but reliable reversal ofthe usual finding for the low memory-similarity condition,with lower confidence in the A–B0 condition than the A–A0

and A–X0 conditions. Like Dobbins et al. we also foundweaker overall effects of choice-similarity on error thancorrect responses, although they did report significantlyless confidence in the A–A0 condition than the A–B0 condi-tion for correct responses, whereas we did not find this dif-ference to be reliable. Similarly, our confidence effectswere weaker than those reported by Tulving (1981).

In summary, we replicated with faces many of theprevious findings with scene stimuli (Dobbins et al., 1998;

0.2

0.3

0.4

Den

sity

A-A'

'X-A'B-A


Tulving, 1981), but there was also some divergence ofresults. This divergence brings into question the generalityof the conclusions which we draw from the model analysesof our data reported in the next section. Fortunately,Dobbins et al. provided their results aggregated overparticipants (Table 3, p. 1311) in sufficient detail for us tofit our models. In order to address the question of generality,following the model analysis of our data we report theresults of fitting versions of these models to their aggregatedata.

-2 0 2 4 6

0.0

0.1

Evidence

Fig. 3. Evidence distributions (old test item memory strength minus newtest item memory strength) for Clark’s (1997) model. Criteria for low/medium/high confidence responses are indicated by vertical dotted lines.The criteria are symmetric around zero. Evidence values less than zeroresult in an incorrect response and evidence values greater than zeroresult in a correct response.

Modeling

Clark (1997) showed his single-process model producescorrect predictions for Tulving’s (1981) accuracy and confi-dence data in an ordinal sense. Dobbins et al.’s (1998) dual-process model was in the main specified verbally. Neithertheory has been tested, nor compared, in a comprehensiveand quantitative manner. We first describe these models,and then describe elaborations enabling them to be ap-plied to our paradigm. We then illustrate the elaborationsby reporting the results of fitting a version of each type ofmodel to our data. Next, we describe how we selectedamongst different versions of each model type, and howwe compared between types. Finally, we show that themodels favored in our data also apply to Dobbins et al.’sdata on memory for scenes. We conclude that the selectedmodels provide a coherent overall account that requiresonly changes in the values of the same set of parametersto accommodate differences in stimulus type and experi-mental methodology.

Clark’s (1997) model

In Clark’s (1997) model, 2AFC decisions are based onevidence obtained by subtracting the old test item’s matchto memory from the new test item’s match. Choices aredetermined by the sign of the difference, and confidenceby how far the difference is from zero, relative to a set ofsymmetric confidence criteria arrayed around zero. TheMINERVA 2 memory model (Hintzman, 1988) and argu-ably many other quantitative memory models (see Clark& Gronlund, 1996) predict that higher choice-similarityincreases the correlation between the two matches. Thevariance of the difference between two random variables,Var(X � Y), decreases as their correlation, and hencetheir covariance, Cov(X, Y), increases: Var(X � Y) = Var(X) +Var(Y) � 2 � Cov(X, Y). Therefore, higher choice-similaritydecreases the variance of the evidence (match difference)distribution. The decrease in variance increases accuracy,because it is less likely that the evidence falls on the wrongside of zero, but decreases confidence, because it is lesslikely that the evidence will be extreme enough to exceedthe higher confidence criteria. Fig. 3 provides an exampleof the evidence distributions predicted by Clark’s model.

Clark’s (1997) model predicts a number of equalitiesand inequalities among the standard deviations (s) andmeans (d0) of the evidence distributions in Fig. 3. Given X0

and B0 items should be equally uncorrelated with A items,A–B0 and A–X0 conditions will have equal standard devia-

tions, which are greater than the standard deviation inthe A–A0 condition.

sðA—A0Þ < sðA—B0Þ ¼ sðA—X0Þ ð1Þ

The mean match to memory depends on the similaritybetween the test item and memory traces. All old testitems have the same mean match, which is greater thanthe mean match for any new item, because of a strongmatch to the memory trace established when the old itemwas studied. Amongst new items, A0 and B0 items have anequal mean match, due to their similarity to the memorytrace of their pair mate (A and B respectively), whereas X0

items have a lower match, because they are not specificallysimilar to the memory traces of any particular test item.

d0ðA—A0Þ ¼ d0ðA—B0Þ < d0ðA—X0Þ ð2Þ

Dobbins et al.’s (1998) model

Dobbins et al. (1998) proposed an explanation of theirRK findings based on the dual-process theories of Jacoby(1991) and Yonelinas (1994). Know responses are assumedto be based on a continuous familiarity dimension.Remember responses are the result of a discrete state ofawareness, although Dobbins et al. suggest, following Don-aldson (1996), that this discrete state might be the result ofa threshold placed on an underlying continuous recollec-tion process. Jacoby’s emphasis on the importance of recol-lection in strategic responding motivated the idea thatparticipants make a remember response if only one testitem causes recollection (i.e., when the result of recollec-tion is unambiguous). A know response and confidence rat-ing based on familiarity are made if neither or both testitems cause recollection. That is, recollection is disregarded


altogether if it is conflicting (i.e., if both test items producea recollection when the participant knows only one wasstudied).

Dobbins et al. (1998) suggested that choice-similaritycauses an increase in the probability of recollection, bothfor target (old) test items (pT) and unstudied (new) testitems (pL), which in turn results in a decrease of the prob-ability of a remember response, due to an increased inci-dence of conflicting recollections. They also suggestedthat false recollection does not occur in the A–X0 conditionbecause X0 items are not specifically similar to any studieditem. To account for the small but non-zero incidence offalse recollection in the A–X0 condition, they introduced aguessing parameter into their model. Guessing was as-sumed to occur equally often in each experimental condi-tion and to result in responses that are equally likely tobe correct or incorrect and equally likely to have any levelof confidence.

Dobbins et al. (1998) considered the idea that the prob-ability of a decision based on either correct recollection(pC), or false recollection (pF), can be derived from the sta-tistically independent probabilities of recollection for eachtest item (i.e., pT and pL). However, they rejected this ideabecause they did not think independence between targetand new test items was plausible when choice-similarityis high. In light of this conclusion, we chose to directly esti-mate the probabilities that recognition decisions werebased on either correct recollection (pC) or false recollec-tion (pF), rather than make any assumptions about howthese probabilities are related to individual target andnew item recollection probabilities. The cost of this strat-egy is a proliferation in the number of estimated parame-ters, raising the possibility that the DPRK model might‘‘over-fit” the data. Over-fitting occurs when a model pro-vides a good fit by accommodating not only the systematicstructure in the data but also the effects of measurementerror. Over-fitting has the important practical consequenceof poor prediction of new data, as measurement error willbe different in new data.

One method of addressing over-fitting is to incorporateparameter constraints based on a model’s psychologicalinterpretation. Although Dobbins et al.’s (1998) modellacks the precise constraints discussed previously forClark’s (1997) model, several possibilities are arguable.First, as already discussed, decisions based on false recol-lection may be unlikely in the A–X0 condition, and so pF

can be fixed at zero in this condition. Second, if highchoice-similarity causes false recollection for the new testitem to always be accompanied by a correct recollectionfor the old test item, no recognition decisions will be basedon false recollection in the A–A0 condition (pF = 0). To-gether, these constraints suggest that only the A–B0 condi-tion should have non-zero estimates of pF:

0 ¼ pFðA—X 0Þ ¼ pFðA—A0Þ < pFðA—B0Þ ð3Þ

The second set of constraint relates to the distributionof difference in familiarity between old and new items.The familiarity difference distribution is assumed to bethe basis for know decisions, in much the same way thatmatch difference distributions are the basis of responding

in Clark’s (1997) model. Dobbins et al. (1998) pointed outthat their finding of equal accuracy for know judgmentsmay be explained if ‘‘targets and distracters are assumedto have equal strength” (p. 1312). If the familiarity differ-ence distributions for the A–A0 and A–B0 conditions differin variance, as predicted by Clark’s (1997) model, thisexplanation would not hold. However, it does hold for anequal variance model, which is also consistent with theassumption made about familiarity in Yonelinas’s (1994)dual-process model. Hence, familiarity differences in theA–A0 and A–B0 conditions might be modeled by Gaussiandistributions with the same mean and variance.

Given these assumptions it is also plausible that thefamiliarity difference distribution in the A–X0 conditionhas the same variance as the other conditions. However,it will have a higher mean, as the new (X0) items in thiscondition should be less familiar than the new items inthe other conditions because they are not specifically sim-ilar to any studied item. Stated formally in terms of themean (d0) and standard deviation (s) of the familiarity dif-ference distribution, these constraints are:

d0ðA—A0Þ ¼ d0ðA—B0Þ < d0ðA—X 0Þ ð4ÞsðA—A0Þ ¼ sðA—B0Þ ¼ sðA—X0Þ ð5Þ

In order to fix the scale for familiarity the usual conven-tion (without loss of generality) is to fix s = 1 in one condi-tion. Given (5) this implies s = 1 in all conditions.

The DPRK model

Fig. 4 displays the participant averages of the 72 re-sponse proportions (i.e., 12 response types in each of 6experimental conditions) to which models were fit. If weadopt Yonelinas’s (1994) assumption that recollection-based responding always results in the highest confidencerating, lower confidence remember responses will occuronly due to guessing. The result is a constant remember re-sponse probability for lower confidence responses. As isevident in Fig. 4, this model’s fit will be very poor as thefrequency of correct medium (i.e., rating 2) confidence rec-ollection responses is quite high.

To avoid these poor fits, we elaborated Dobbins et al.’s(1998) model by assuming that, when it occurs, recollec-tion has a continuously varying strength. Note that, in thisview (originally suggested as a possibility by Yonelinas,1994, p. 1352), recollection is still a threshold process inthat it can sometimes completely fail to provide any infor-mation (see Parks & Yonelinas, 2009, for recent evidenceon this issue). Hence, as pointed out by Dobbins et al. (theirFootnote 2), even if the strength of successful recollectionis continuous, the success or failure of recollection still re-sults in distinguishable cognitive states, which can be useddetermine what information contributes to a recognitiondecision. Hence, our elaboration remains consistent withDobbin’s et al.’s model of the choice-similarity effect, anddiffers from other proposals that recollection is not athreshold process but rather always provides continuouslyvarying mnemonic information (e.g., Wixted, 2007).

In particular, the DPRK model assumes recollection,when it occurs, is characterized by separate, but equal

0.00

0.05

0.10

0.15

0.20

0.25

0.30

A-A' High Memory Similarity

Error CorrectCONFIDENCE RATING

Res

pons

e Pr

opor

tion

3 2 1 1 2 3

KnowRememberKnowRemember

0.00

0.05

0.10

0.15

0.20

0.25

0.30

A-A' Low Memory Similarity


Res

pons

e Pr

opor

tion

3 2 1 1 2 3

KnowRemember

0.00

0.05

0.10

0.15

0.20

0.25

0.30

A-B' High Memory Similarity


Res

pons

e Pr

opor

tion

3 2 1 1 2 3

KnowRemember

0.00

0.05

0.10

0.15

0.20

0.25

0.30

A-B' Low Memory Similarity


Res

pons

e Pr

opor

tion

3 2 1 1 2 3

KnowRemember

0.00

0.05

0.10

0.15

0.20

0.25

0.30

A-X' High Memory Similarity


Res

pons

e Pr

opor

tion

3 2 1 1 2 3

KnowRemember

0.00

0.05

0.10

0.15

0.20

0.25

0.30

A-X' Low Memory Similarity


Res

pons

e Pr

opor

tion

3 2 1 1 2 3

KnowRemember

Fig. 4. Average proportions of remember and know responses (triangle symbols) with 95% confidence intervals and average fits (lines with solid points) ofthe 17 parameter DPRK model (see Table 2). Each panel gives results for one of the six experimental conditions. The x-axes specify rating of confidence,ranging from high (3) to low (1) for correct and error responses, with know results displaced slightly to the left and remember to the right in order to avoidoverlap. Confidence intervals were calculated using 100,000 bootstrap replications (Efron & Tibshirani, 1993). Each replication was the average overparticipants of samples from binomial distributions for each participant, B(p, n), where p and n are the observed proportion and number of trials for eachparticipant.


variance, Gaussian distributions of true and false recollec-tion strength. Recollection strength determines recollec-tion confidence in the manner of signal detection theory,using the same criteria as for familiarity based confidencejudgments. We note that, formally, Yonelinas’s (1994)model is a special case of the DPRK model, occurring whenthe recollection means are large, in which case the entirerecollection distribution falls above the upper confidencecriterion, and so all recollection based decisions are alwaysmade with high confidence.

Fig. 5 provides a schematic illustration of the DPRKmodel. On each trial, participants attempt recollection forboth test items. Recollection is either unsuccessful or suc-cessful, and if successful it results in recollection strength

values with a unit variance Gaussian distribution withmean mC for correct recollection (Fig. 5a) and mean mF

for false recollection (Fig. 5b). Fig. 5 illustrates the case,which we found to apply to our data, where correct recol-lection strength is on average greater than false recollec-tion strength (i.e., mC > mF). If both or neither test itemcauses recollection the recognition decision and confi-dence are determined by the difference in their familiari-ties, and a know response is made. The DPRK modelassumes that the familiarity difference distribution has aunit variance Gaussian distribution with mean d0F(Fig. 5c). As in Clark’s (1997) model, positive values indi-cate a correct response and confidence criteria are sym-metric around zero.

-2 0 2 4 6

0.0

0.1

0.2

0.3

0.4

Correct Recollection Strength

Den

sity

(a) CorrectRememberResponse

-2 0 2 4 6

0.0

0.1

0.2

0.3

0.4

False Recollection Strength

Den

sity

(b) IncorrectRememberResponse

-2 0 2 4 6

0.0

0.1

0.2

0.3

0.4

Familiarity (Correct-Error)

Den

sity

(c) KnowResponse

Confidence

LowMediumHigh

Fig. 5. The three strength distributions constituting the DPRK model, for (a) correct and (b) false recollection and (c) correct and false know responses. Thedistribution in (a) has mean mC and the distribution in (b) has mean mF. Proportions of each type of confidence response are shown by the shaded areasunder each distribution.


Fig. 5 shows the proportions of confidence ratings ofeach type by shaded areas under the distribution functions,with heavier shading indicating greater confidence. Theareas of the six shaded regions in Fig. 5c determine theproportions of correct and false know responses at each le-vel of confidence. The same positive criteria used to deter-mine familiarity based confidence are also used todetermine recollection confidence. The areas of the threeshaded regions in Fig. 5a determine the proportions of dif-ferent confidence responses for correct recollections, andthe areas of the three shaded regions in Fig. 5b determinethe proportions of different confidence responses for falserecollections.

We fit the DPRK model, subject to the equality con-straints in (3)–(5), to each participant’s data separately.We used the degree to which the resulting model’s param-eters respected the inequality constraints in (3)–(5) as atest of the plausibility of the model. The fit of this 17parameter model, given the data have 66 degrees of free-dom, was excellent, and it captures all of the systematictrends evident in Figs. 1 and 2. Table 2 gives the medianparameter estimates over participants, and also the param-eter estimates for a fit of this model to the data aggregatedover participants. The model has a guessing parameter andsix correct remember probability parameters (pC), one foreach experimental condition. It has two false recollectionprobability parameters (pF), for A–B0 choices in the low

and high similarity conditions. As predicted by the inequal-ity in (3), these estimates, although small, are greater thanzero.

Four parameters specify mean familiarity differences,one for the A–A0 and A–B0 conditions, and one for the A–X0 condition, with separate estimates for the low and highmemory-similarity conditions. As predicted by theinequality in (4), the median estimates in Table 2 weregreater in the A–X0 condition than in the other conditions,although this was true for only a small majority (55%) ofindividual participants. A further two parameters specifymean false and correct recollection strengths, and theremaining two parameters are the criteria dividing lowand medium confidence responses and medium and highconfidence responses. The lower mean for the false recol-lection distribution allows the model to accommodatethe slightly elevated rates of medium (i.e., rating 2) confi-dence false recollection responses in the A–B0 conditionevident in Fig. 4.

Note that the means of the recollection strength distri-butions, although much larger than the means of the famil-iarity distributions, are inconsistent with Yonelinas’s(1994) model in which recollection always results in a highconfidence response. In particular, the estimates in Table 2indicate that high confidence responses are given for onlyaround 25% of false recollections and 65% of correctrecollections.

Table 2Median parameter estimates from fits of the 17 parameter DPRK to participant data, with parameter estimates from fits to aggregated data in brackets. Notethat the zero false remember probability values in the table are fixed and not estimated.

Higher memory-similarity Lower memory-similarity

A–A0 A–B0 A–X0 A–A0 A–B0 A–X0

Correct remember probability .46 (.47) .47 (.46) .51 (.50) .50 (.50) .53 (.55) .54 (.54)False remember probability 0 .031 (.039) 0 0 .015 (.016) 0Mean familiarity difference .64 (.63) .71 (.67) .75 (.69) .83 (.71)

Mean recollection strength False = 2.1(1.9) correct = 3.3(2.9) Guessing probability = .04 (.04)Confidence criteria Low/medium = 1.02 (.97) Medium/high = 2.97 (2.44)


The SPRK model

We extended Clark’s (1997) theory to address our RKdata by adopting both Donaldson’s (1996) idea that RKjudgments are based on a criterion placed on the samedimension used to make confidence judgments (i.e., thematch difference distribution in Clark’s model), and Wix-ted and Stretch’s (2004) idea that this RK criterion variesfrom trial to trail. The resulting SPRK model assumes thatRK criterion has a normal distribution, N(c, r2), with meanc and standard deviation r. Hence, the SPRK model differsfrom Clark’s model depicted in Fig. 3 through the additionof a single noisy RK criterion. We describe the technical de-tails of how response probability predictions were derivedfor the SPRK model in Appendix to this paper.

We fit a version of the SPRK model incorporating theequality constraints in (1) and (2) to each participant’s dataseparately. The fit of this 12 parameter model was excel-lent, as shown in Fig. 6. Table 3 gives the median parame-ter estimates over participants, and also the parameterestimates for a fit of this model to the data aggregated overparticipants. Four parameters determined mean memorystrength (d0), one for both the A–A0 and A–B0 conditionsand one for the A–X0 condition for each level of memory-similarity. As predicted by the inequality in (2), the medianestimates of d0 in Table 3 were greater in the A–X0 conditionthan the A–A0 and A–B0 conditions, and this was also truefor 63% of individual participants.

Four parameters determine the standard deviation ofmemory strength (s): one for both the A–B0 and A–X0 condi-tions and one for the A–A0 condition for each level of mem-ory-similarity. Note that only three of these parameters areestimated to fit the data as (without loss of generality) thehigh memory-similarity A–A0 standard deviation was set toone to fix the scale on which memory strength is mea-sured. As predicted by the inequality in (1), the medianestimates of s in Table 3 were less in the A–A0 conditionthan in the A–B0 and A–X0 conditions, and this was also truefor 62% of individual participants.

Two parameters determine confidence criteria, and onedetermines the RK criterion, with a further parameter forthe standard deviation (SD) of the RK criterion. The latterparameter, in Table 3, indicates that the level of RK crite-rion variability is only 20% (i.e., .452) of the level of memorystrength variability. Consequently the correlation betweenmemory strength and the RK decision variable is quite high(r � .9). Finally, the SPRK model incorporates the same typeof guessing parameter as the DPRK model, with bothmodels estimating a similar low level of guessing.

Model selection

We tested the equality assumptions (1)–(5) underlyingthe versions of the DPRK and SPRK models just discussedby comparing these models with alternative DPRK andSPRK models that made different assumptions. In this sec-tion we describe the alternative models and the methodsby which we selected the best model. Our model selectionmethodology is largely taken from Busemeyer and Wang(2000) and Wasserman (2000). The reader is referred tothese papers for details, but to make our treatment self-contained we provide a summery here.

The first step in model selection is to determine the de-gree to which a model misfits the data. The deviance statis-tic is the appropriate misfit measure for our data, whichconsists of the number of responses (n) in each of thei = 1. . .6 experimental conditions (A–A0, A–B0 and A–X0 forboth low and high memory-similarity) and j = 1. . .12 re-sponse types (correct/error � remember/know � low/medium/high confidence):

D ¼ �2X6

i¼1

X12

j¼1

log Prij nijjh� ��

ð6Þ

Prij() is the probability of a response in condition i oftype j given by a model with parameter vector h of lengthp, where h is chosen to minimize the deviance (i.e., maxi-mum-likelihood estimation). Deviance has an asymptoti-cally (i.e., in the large sample limit) v2 distribution, withlarger values indicating greater misfit.

All models were fit by searching for parameter valuesthat minimize the model’s deviance. However, simplypicking the model with the smallest deviance is mislead-ing. For example, when the models being compared arenested (i.e., one model is a simpler version of the other be-cause of restrictions on its parameter values) the morecomplex model always has a lesser or equal deviance com-pared to the simpler model. This occurs even when thesimpler model is the true (i.e., data generating) model.One approach to these problems is to select a more com-plex model with p estimated parameters over a simplernested model with q estimated parameters (q < p) if the de-crease in deviance is significant relative to a v2(p � q) dis-tribution. This approach is limited to the comparison ofnested models, and, in any case, is known to pick overlysimple models when sample size, and hence power, islow. Conversely, when sample size is large it picks overlycomplex models.

An alternative approach, which is not limited to nestedmodels, is to pick the model which minimizes an


information criterion. The most commonly used types arethe Akaike Information Criterion (AIC) and the BayesianInformation Criterion (BIC). Both add to the deviance apenalty for complexity that is proportional to the numberof estimated parameters (i.e., models with a larger numberof parameters suffer a larger penalty). We focus here onBIC, as our conclusions were the same when we usedAIC.2 Given a model fit to a total of N responses (N = 72 forour data), BIC imposes the following penalty BIC:

P � lnðNÞ ð7Þ

BIC is designed to select the most probable model amongsta set of models. For a set of models i = 1. . .K with BIC valuesBICi, the probability that a given model is the true model is:

pi ¼e�BICi=2

PKi¼1e�BIC=2

ð8Þ

We describe this probability as pBIC and use it to test ob-served differences in BIC.

DPRK models

Fig. 7 compares three versions of the DPRK model withthe ordinate giving their deviance and BIC. Different mod-els are indicated by labels on the abscissa corresponding totheir number of estimated parameters. Each bar representsaverage participant results for one version of a model. Thedark gray portion of the bar indicates deviance and thelight gray portion of the bar represents the complexitypenalty imposed by BIC. When comparing models the bestfitting model has the lowest dark gray section and the bestmodel according to BIC has the lowest bar overall.

Fig. 7 provides results for three versions of the DPRKmodel. The most complex version has 21 parameters, asit imposes only the two most basic restrictions on param-eters as a function of choice-similarity: the equal varianceassumption (5) and a zero false recollection probability inthe A–X0 condition, as suggested by Dobbins et al. (1998).We examined even more complex models relaxing theseconstraints, but none reduced misfit much and all led tolarge increases in AIC and BIC. The 21 parameter modelhas 4 extra parameters relative to those shown in Table2, two extra false recollection parameters for the highand low memory-similarity A–A0 conditions and two extrad0 parameters allowing the A–A0 and A–B0 conditions tohave unequal mean familiarity differences for the lowand high memory-similarity conditions.

The second version of the DPRK model, with resultsdepicted in Fig. 7, has 19 parameters because it enforcesthe equality on d0 between A–A0 and A–B0 conditions spec-ified in (4). Finally, by adding the equality in (3), which

2 AIC generally prefers more complex models than BIC. It might bethought that our conclusions, which generally preferred simpler models(e.g., SPRK) were due to this property of BIC, so that fact that results basedon AIC (or more correctly, the small-sample adjusted version of AIC, seeWagenmakers & Farrell, 2004) were consistent with those based on BIC isreassuring. We also obtained consistent results when we made differentassumptions about the value of N in the equation for BIC (e.g., that itequalled the total number of binary responses in a data set rather than thenumber of response proportions, which aggregates over binary responses ineach condition).

sets false recollection in the A–A0 condition to zero, we ar-rive at the 17 parameter model considered in detail ear-lier. Fig. 7 shows that the extra constraints increasemisfit but to a much lesser degree than the accompanyingreductions in complexity penalties, so BIC chose the 17parameter model. In particular, BIC for the 19 parametermodel relative to the 21 parameter model improved foralmost every participant (58/61). The overall probabilityof the 19 parameter model relative to the 21 parametermodel was pBIC > .99.

These results support the equality constraint specifiedin (4), and hence the assumption that the difference be-tween new and old test item familiarity is the same inthe A–A0 and A–B0 conditions. BIC for the 17 parametermodel relative to the 19 parameter model also improvesfor almost every participant (60/61). The overall probabil-ity of the 17 parameter model relative to the 19 parametermodel was pBIC > .99. These results support the equality in(3), and hence the assumption that decisions in the A–A0

condition are never based on false recollection.

SPRK models

Fig. 7 also compares three versions of the SPRK model.Relative to the 12 parameter SPRK model discussed previ-ously, the 17 parameter version relaxes the equality con-straints (1) and (2) predicted by Clark’s (1997) model,allowing different mean evidence parameters in the A–A0

and A–B0 conditions and different evidence variance inthe A–B0 and A–X0 conditions. It also relaxes the assumptionthat only the RK criterion is variable by allowing a param-eter3 for variability in the criteria used to make the recogni-tion and confidence decisions (see Benjamin, Diaz, & Wee,2009, for further discussion of variability in criteria in itemrecognition tasks). The 13 parameter version imposes theequalities in (1) and (2) and the 12 parameter version alsoimposes the assumption that only the RK criterion is vari-able (i.e., it is the model with parameter estimates shownin Table 3).

Fig. 7 provides strong support for the equality assump-tions predicted by Clark’s (1997) model. BIC for the 13parameter model, relative to the 17 parameter model, de-creased for every one of the 61 participants, with an overallprobability of pBIC > .99. Fig. 7 also provides support for theassumption that only the RK criterion is variable. BIC forthe 12 parameter model relative to the 13 parameter mod-el, decreased for most participants (57/61), with an overallprobability of pBIC > .99.

Alternative models

Although we chose the simplest of the DPRK models, itstill has five more parameters than the best SPRK model. Isthis extra complexity in the dual-process model required?We tried reducing the complexity of the DPRK model byequating parameters which are similar in Table 2. The best

3 Only a single parameter is required on the assumption that allconfidence criteria are perturbed equally by the sample of criterion noise(see Mueller and Weidemann (2008), for discussion of models in whichcriterion variability differs for different confidence criteria).

0.00

0.05

0.10

0.15

0.20

0.25

0.30

A-A' High Memory Similarity


Res

pons

e Pr

opor

tion

3 2 1 1 2 3

KnowRememberKnowRemember

0.00

0.05

0.10

0.15

0.20

0.25

0.30

A-A' Low Memory Similarity


Res

pons

e Pr

opor

tion

3 2 1 1 2 3

KnowRemember

0.00

0.05

0.10

0.15

0.20

0.25

0.30

A-B' High Memory Similarity


Res

pons

e Pr

opor

tion

3 2 1 1 2 3

KnowRemember

0.00

0.05

0.10

0.15

0.20

0.25

0.30

A-B' Low Memory Similarity


Res

pons

e Pr

opor

tion

3 2 1 1 2 3

KnowRemember

0.00

0.05

0.10

0.15

0.20

0.25

0.30

A-X' High Memory Similarity


Res

pons

e Pr

opor

tion

3 2 1 1 2 3

KnowRemember

0.00

0.05

0.10

0.15

0.20

0.25

0.30

A-X' Low Memory Similarity


Res

pons

e Pr

opor

tion

3 2 1 1 2 3

KnowRemember

Fig. 6. Average proportions of remember and know responses (triangle symbols) with 95% confidence intervals and average fits (lines with solid points) ofthe 12 parameter SPRK model (see Table 3). Each panel gives results for one of the six experimental conditions (MS = memory-similarity). The legends applyto all panels. The x-axes specify rating of confidence, ranging from high (3) to low (1) for correct and error responses, with know results displaced slightly tothe left and remember to the right in order to avoid overlap. Confidence intervals were calculated using 100,000 bootstrap replications (Efron & Tibshirani,1993). Each replication was the average over participants of samples from binomial distributions for each participant, B(p, n), where p and n are theobserved proportion and number of trials for each participant.

Table 3Median parameter estimates from fits of SPRK to participant data, with parameter estimates from fits to aggregated data in brackets. s(A–A0) = 1 in the highermemory-similarity condition by assumption.

d0 s

A–A0 and A–B0 A–X0 A–A0 A–B0 and A–X0

Higher memory-similarity 1.05 (1.03) 1.10 (1.11) 1 1.09 (1.13)Lower memory-similarity 1.18 (1.18) 1.26 (1.22) .99 (1.03) 1.09 (1.08)

Confidence criteria Low/medium = .73 (.72) Medium/high = 1.79 (1.70)RK criterion Mean = 1.17 (1.16) SD = .45 (.62) Guessing probability = .03 (.09)


resulting model, which had the same number of parame-ters as the SPRK model (12), assumed: (a) the same meanfamiliarity for all conditions, (b) the same mean for correct

and false recollection strength, and (c) a zero rememberprobability for lower memory-similarity pairs in the A–B0

condition. As might be expected by the opportunistic

21 19 17 17 13 12

Dual Process Models Single Process Models

Subj

ect A

vera

ge (P

enal

ized

) Dev

ianc

e

620

640

660

680

700

720

740 Deviance

BIC Penalty

MinimumBIC

MinimumDeviance

Fig. 7. Model selection results. Each bar represents average participantresults for one version of a model with the type of the model (DPRK orSPRK) and its number of parameters indicated on the x-axis label. Thedark gray portion of each bar indicates deviance and the light gray portionof the bar represents the complexity penalty imposed by BIC. Horizontaldotted lines indicate the minimum values of each statistic over allmodels.


way in which these reductions were made, they were sup-ported by BIC (pBIC > .99, 61/61).

Even with this simplified model the 12 parameter SPRKmodel was still preferred (pBIC > .99, 40/61), indicating thatit fit better than the 12 parameter DPRK model. In order tomake a fairer comparison, we also applied the sameapproach to the SPRK model. As shown in Table 3,memory-similarity had virtually no effect on the s param-eter, suggesting a simplification from 12 to 10 parameters.The resulting model was preferred according to BIC rela-tive to both the 12 parameter SPRK model (pBIC > .99, 59/61) and the 12 parameter DPRK model (pBIC > .99, 61/61).

We also made a more wide ranging search for betterDPRK models, including simplifications suggested byreviewers (e.g., equating correct remember probabilityacross choice-similarity conditions), and models that al-lowed unequal familiarity and/or recollection strength var-iance while simplifying other aspects of the model, butwere unsuccessful in finding any alternative model that re-duced BIC. Although we do not claim that our search wasexhaustive, we were convinced after many unsuccessfulattempts that the results presented here are at least closeto the best alternative within our DPRK framework. Wealso agree with reviewers of this paper who rightly cau-tioned us against the validity of this ad hoc approach tomodel selection, which is likely to capitalize on chance var-iation in our results. Hence, in the following comparison ofDPRK and SPRK models we focus on the principled alterna-tives described in the last two sections.

DPRK vs. SPRK

Fig. 7 also enables a comparison of the DPRK and SPRKmodels. The 21 parameter DPRK model provides the best

fit (i.e., has minimum deviance) overall, although it is onlyvery marginally better than the 17 parameter SPRK model.In contrast, the 12 parameter SPRK model provides the bestaccount according to BIC. The preference for the SPRK mod-els is very clear; every SPRK model, even the most complex,is preferred to any DPRK model, even the least complex.

The preference for the SPRK model indicated by Fig. 7relies on the appropriateness of the penalties applied forextra model parameters. However BIC does not take ac-count of functional form complexity, that is, model flexibilitydifferences that arise because of differences in the way thatmodel equations combine parameters and data (the sameis true for AIC). Pitt and Myung (2002) discuss minimumdescription length and Bayesian methods of addressingthis limitation. As we found these methods difficult to ap-ply to the models considered here, due to their relativelylarge number of parameters, we used a different approachto this issue proposed by Busemeyer and Wang (2000).This approach selects a model based on an intuitively plau-sible criterion, the generalizability of a model’s data-fittingabilities.

Busemeyer and Wang’s (2000) generalization criterioninvolves obtaining parameter estimates by fitting a modelto one set of data (the calibration sample) and then mea-suring the misfit of predictions based on those parametersfor data from a new experimental design (the validationsample). This approach differs from cross-validation,which as mentioned previously is asymptotically equiva-lent to selection by AIC, where the validation sample isdrawn from the same experimental design. Busemeyerand Wang’s validation sample came from an extrapolationdesign, which measures data for quantitative independentvariable values outside the range examined in the valida-tion sample (see also Wagenmakers, Grunwald, & Steyvers,2006).

Here we use a validation sample drawn from an inter-polated design, which used all but the A–X0 condition ofthe present experiment and a subset of the same faceimages (Heathcote et al., 2009). Apart from the omissionof the A–X0 condition and measuring a different group ofparticipants, this design was new in the sense that it usedstudy lists with mixed race (but same gender) faces, andcreated a low choice-similarity condition by testing facesfrom different races rather than different genders. In par-ticular, the validation data comes from the 38 of the 99participants tested by Heathcote et al. who made RK deci-sions as well as making 2AFC recognition decisions withconfidence ratings.

For each participant’s data in the validation sample wecalculated the average of the deviances obtained using theparameters estimated for each of the 61 participants in thepresent experiment, omitting parameters specific to the A–X0 condition. Fig. 8 presents results averaged over the 38participants in the validation sample for each of the mod-el’s examined in Fig. 7. Deviance results for the validationsample are in the opposite order to deviance for the cali-bration sample. In contrast, they are in agreement withthe order of the BIC results, demonstrating the utility ofthis criterion in selecting models.

In all but one case, the differences between models invalidation deviance are much larger than differences in

21 19 17 17 13 12

Generalization Criterion

Dual Process Models Single Process Models

Subj

ect A

vera

ge D

evia

nce

1200

1210

1220

1230

1240

1250

1260

Fig. 8. Generalization-criterion results based on misfit to data fromHeathcote et al.’s (2009) participants who gave RK responses forpredictions made by the models and associated parameter estimates fitto the present data.


calibration deviance and similar to BIC differences. The oneexception concerns the comparison between the two sim-plest SPRK models, where the 12 parameter model doesnot differ much from the 13 parameter model, althoughits generalization deviance is numerically smaller (1216.9vs. 1217.6). Hence, the generalization-criterion results con-firm the superiority of the SPRK models to DPRK models asa class, the constraints for the DPRK model, and the con-straint derived from Clark (1997) for the SPRK model, butare equivocal on whether only the RK criterion is variable(i.e., the 12 parameter model) or whether confidence crite-ria are also variable (i.e., the 13 parameter model).

Up to now we have not addressed the model parameterdifferences that explain the memory-similarity effect. Inpart this was because the models are not bound to makestrong predictions, as memory-similarity is a between itemmanipulation (i.e., different items make up high and lowmemory-similarity sets). As a result, outcomes may be af-fected by item related factors other than memory-similar-ity (e.g., items in the low similarity set may be moredistinctive and so more likely to support correct recollec-tion). However, one aspect deserves mention because, forboth models, it converges with Heathcote et al.’s (2009)state-trace findings. Choice-similarity effects in the SPRKmodel are mediated by a parameter that is little affectedby memory-similarity, that is, evidence variance (see Ta-ble 3), consistent with a dissociation between the underly-ing causes of these effects. For the DPRK model falserecollection could fulfill a similar role to evidence variance,although the dissociation is less clear cut.

Memory for scenes

In order to investigate the generality of our findingswith faces, we applied our model based analysis to

Dobbins et al.’s (1998) data on memory for scenic stimuli.The focus of this analysis was to determine whether thebest DPRK and SPRK models could provide an accurate ac-count of the scene data, and whether the correspondingparameter estimates were consistent with our findingsfor faces. Neither could be taken for granted in light ofsome marked differences in our findings, such as strongereffects of choice-similarity on know confidence for thescene stimuli. The design of Dobbins et al.’s experimentdiffered from ours in that memory-similarity was notmanipulated, and participants gave a four-level confidencejudgment. Data aggregated over participants were takenfrom their Table 3 (p. 1311) and are plotted in Fig. 9.

Fig. 9 illustrates that, although a similar pattern to ourdata is evident, with less accurate know responses accom-panied by low and medium confidence ratings and moreaccurate remember responses accompanied by higher con-fidence ratings, there are also clear differences. The propor-tion of remember responses at the highest confidencerating was greater, and a substantial proportion of falseremember responses at the highest confidence are evidentin the A–B0 condition. In the DPRK model the latter findingis consistent with false recollection unchecked by correctrecollection, but it is also consistent with a less variableRK criterion in the SPRK model when combined with great-er evidence variability and low accuracy in the A–B0

condition.In the left-hand column, Fig. 9 shows the fit of the ver-

sion of the SPRK model that had 12 parameters for our de-sign. For Dobbins et al.’s (1998) design, this model has 9parameters, with estimates shown in Table 4. Low accu-racy in the A–B0 condition and higher accuracy in the A–X0 condition were attributed to larger differences betweenthe d0 and s parameter estimates than in our data. Guessingwas at approximately the same level, and, as expected, theRK criterion was less variable than in our data. The extraconfidence rating was modeled by a very low criterion.Confidence criteria were generally lower relative to ourdata, with the second criterion approximately equal toour first, and the RK and highest criteria less than the esti-mates for our data.

The right-hand column of Fig. 9 shows the fit of the ver-sion of the DPRK model that had 17 parameters for our de-sign. For Dobbins et al.’s (1998) design this version has 12parameters with estimates shown in Table 5. As would beexpected, the mean estimate was greater for correct thanfalse recollection, recollection probability was greatest inthe A–X0 condition, and least in the A–A0 condition. Bothguessing and the probability of false remember responseswere much higher than for our face data, as was the advan-tage for the A–X0 condition in the mean familiaritydifference.

As might be expected, given its larger number of param-eters, misfit for the DPRK model was less than for the SPRKmodel. However, the difference was only very slight, with adeviance per subject of 426.9 vs. 427.3 (i.e., a .1% differ-ence). Because the data in this case are aggregated overparticipants, none of the model selection methods, whichwe used with our data to account for model complexitydifferences (i.e., BIC and the generalization criterion), werestraightforwardly applicable. Hence, the main conclusion

0.0

0.1

0.2

0.3

0.4

A-A' SPRK Fit


Res

pons

e Pr

opor

tion

4 3 2 1 1 2 3 4

KnowRemember

0.0

0.1

0.2

0.3

0.4

A-B' SPRK Fit


Res

pons

e Pr

opor

tion

4 3 2 1 1 2 3 4

KnowRemember

0.0

0.1

0.2

0.3

0.4

A-X' SPRK Fit


Res

pons

e Pr

opor

tion

4 3 2 1 1 2 3 4

KnowRemember

0.0

0.1

0.2

0.3

0.4

A-A' DPRK Fit


Res

pons

e Pr

opor

tion

4 3 2 1 1 2 3 4

KnowRemember

0.0

0.1

0.2

0.3

0.4

A-B' DPRK Fit


Res

pons

e Pr

opor

tion

4 3 2 1 1 2 3 4

KnowRemember

0.0

0.1

0.2

0.3

0.4

A-X' DPRK Fit


Res

pons

e Pr

opor

tion

4 3 2 1 1 2 3 4

KnowRemember

Fig. 9. Average proportions of remember and know responses from Dobbins et al. (1998) and fits of the 9 parameter SPRK model and 12 parameter DPRKmodel (see Tables 4 and 5). The x-axes specify rating of confidence, ranging from high (4) to low (1) for correct and error responses.

Table 4Parameter estimates for fits of SPRK to Dobbins et al.’s (1998) aggregated data. Note that s(A–A0) = 1 by assumption.

d0 s

A–A0 and A–B0 A–X0 A–A0 A–B0 and A–X0

Evidence .58 1.03 1 1.28Confidence criteria Low/medium = .41 Medium/high = .79 High/highest = 1.26RK criterion Mean = 1.08 SD = .34 Guessing probability = .05

Table 5Parameter estimates for fits of DPRK to Dobbins et al.’s (1998) aggregated data. Note that false remember probability is fixed at zero in the A–A0 and A–X0

conditions.

A–A0 A–B0 A–X0

Correct remember probability .33 .41 .50False remember probability 0 .13 0Mean familiarity difference .24 .50Mean recollection strength Correct = 3.28 False = 3.06 Guessing probability = .13Confidence criteria Low/medium = .60 Medium/high = 1.24 High/highest = 2.37



to be drawn from these analyses is that the SPRK and DPRKmodels both provide accurate and parametrically coherentaccounts of Dobbins et al.’s (1998) data. Both models alsoprovide a coherent and unified account of memory forfaces and memory for scenes in the choice-similarity para-digm. Differences between these cases are accounted forby variations in the models’ parameter values that are con-sistent with the procedural differences between theexperiments.

General discussion

The results of our experiments answer empirical ques-tions about recognition memory for faces and our modelbased analyses answer questions about the underlyingpsychological causes of these findings. On the empiricalfront we found that accuracy was better when a recogni-tion choice was between similar (i.e., same gender) facesthan when it was between dissimilar (i.e., different gender)faces, where in the latter case the incorrect choice was spe-cifically similar to a studied face which was not tested. Thiseffect was not as large as the corresponding effect that hasbeen found with natural scenes (e.g., Tulving, 1981), but itwas large in a relative sense. That is, accuracy for samegender choices was equal to accuracy for a different genderchoice when the unstudied test face was not specificallysimilar to any studied face. In contrast, with natural scenesthe latter type of choice was the most accurate.

Dual-process theory

Dual-process theory can account for these findings interms of false recollection, which occurs when an unstud-ied face is specifically similar to a studied face, and a strat-egy to reduce the effects of false recollection. The strategyis applied when both choices cause recollection. Whensuch conflicting recollections occur, participants can stra-tegically use their knowledge that only one face was stud-ied to disregard recollection altogether and base theirchoice on familiarity. This strategy is particularly effectivewith similar recognition choices, as for these choices falserecollection is accompanied by correct recollection. Hence,accuracy is increased when choices are similar, because thestrategy eliminates wrong responses based on false recol-lection. The same mechanism also explains a decrease inthe proportion of responses classified as based on recollec-tion when choices are similar, as measured by Tulving’s(1985) remember–know (RK) procedure.

The dual-process model was able to account for confi-dence results, but only when it was assumed that thestrength of recollection is graded. When recollectionstrength is graded, recollection does not always result ina highly confident recognition decision. Instead, confidenceis less when the recollected details are vague. We foundthat for faces the average strength of false recollectionwas substantially less than that of correct recollection.Hence, responses based on correct recollection resultedin a greater proportion of high confidence ratings, whereasresponses based on a false recollection resulted in a greaterproportion of medium confidence ratings.

The same type of dual-process model that provided anaccurate account of the confidence–accuracy dissociationwith face stimuli also provided an accurate account ofthe same phenomena with scenic stimuli (Dobbinset al., 1998). Differences in performance for the two typesof stimuli were accounted for by differences in modelparameter estimates. One difference was that responsesbased on false recollection were more common for scenes.Another difference was that for scenes mean false recol-lection strength was not much less than mean correct rec-ollection strength. As a result, high confidence responsesbased on false recollection were more common for scenesthan faces. These results illustrate the utility of a modelbased analysis; it is able to show that empirically diver-gent results can be explained by a common underlyingmechanism.

Although the way in which we added graded recollec-tion to Dobbins et al.’s (1998) dual-process model providesan accurate, coherent and general account of theconfidence–accuracy dissociation, we do not claim thatalternative extensions might not also do so. A more parsi-monious model might be obtained, for example, if thesame graded strength that determines recollection confi-dence also determines whether recollection occurs (e.g.,when strength exceeds a threshold). However, if this wasthe case, when conflicting recollections occur it wouldmake sense for participants to use the difference in recol-lection strengths to inform their recognition decisionrather than discounting recollection entirely, suggesting amore radical departure from Dobbins et al.’s account. Weleave investigation of this and other possibilities, such asWixted’s (2007) suggestion that recollection always occursto some degree, to future research.

Single-process theory

Single-process theory also provides an accurate accountof the confidence–accuracy dissociation caused by choice-similarity. Findings related to recognition accuracy andconfidence are explained by Clark’s (1997) original model,in which decisions are based on the difference between thematch of each test choice to memory. Findings related toRK responses are explained by comparing the match differ-ence to a RK criterion. We found that the RK criterion typ-ically falls between medium and high confidence criteria.Remember responses are less common for choices betweenfaces of the same gender or parts of the same scene for thesame reason that confidence is reduced in these cases; thematch difference tends to be less extreme in these condi-tions, and so is less likely to exceed the RK criterion. A highremember criterion also explains why remember confi-dence does not vary with choice-similarity; most remem-ber responses are made with the highest confidence, andso there is less chance for large confidence differences tooccur. Conversely, know confidence is more able to vary,and so differences in know confidence can more easily oc-cur across conditions.

These points all apply to Donaldson’s (1996) originalexplanation of RK performance assuming a constant RK cri-terion. However, trial to trail variability in the RK criterion,as suggested by Wixted and Stretch (2004), was required


to provide a good quantitative account of the joint fre-quency distribution of RK and confidence responses. Inparticular, a variable criterion allows some lower confi-dence responses to be classified as ‘‘remember” and somehigher confidence responses as ‘‘know”. Our results werealso consistent with some variability in the recognition cri-teria (see Benjamin et al., 2009; Mueller & Weidemann,2008, for evidence supporting this type of variability).Although we could not clearly adjudicate between thesetwo possibilities with the methodology used here,4 thekey point is that, in both cases, recognition and rememberdecisions were highly, but not perfectly, correlated becausethey are both based on a decision variable that shares a largecommon component with the recognition decision variable,that is, the memory match difference.

Dobbins et al. (1998, p. 1306) claimed that Clark’s(1997) model can not ‘‘mimic” remember performance.Clearly our results make it unlikely that the success ofour elaborated version of Clark’s model is due to meremimicry by an overly flexible model. If anything, that pos-sibility must be considered as applying to the dual-processmodel, because it has many more parameters than the sin-gle-process model, and is less able to predict new data.However, the idea that the single-process model succeedsthrough mimicry might have appeal given the commonsubjective experience that confidence is boosted by recol-lection in everyday recognition. Correspondingly, in thelaboratory, why would participants disregard instructionsand base their RK response on test item matches?

An alternative interpretation of our single-process mod-el might provide an explanation. Suppose retrieval ofdetails from memory is occurring, or at least beingattempted, as proposed by Gillund and Shiffrin’s (1984)SAM model of recognition and recall. In SAM, study createsassociations among attended aspects of the internal andexternal study context, including self-strength, anassociation of an item to itself. Recognition and recallperformance depend on the strength of this web of associ-ations, which increases with study attention. For example,the probability of recognizing a test item depends on itsassociation strength (match), and that of the test context,to memory. The probability of recollecting a memory tracedepends in part on the same match, but this match is di-vided by the sum of matches for all memory traces, and re-call only occurs when this ratio exceeds a threshold.

Because recognition and recollection both depend onthe level of attention during study, they tend to be corre-lated; a test item that is easily recognized also tends tobe a good cue for recollection. However, the correlation isnot perfect, because recollection depends on factors thatdo not affect recognition, such as the strength of memorytraces that compete for recollection. For example, in Man-dler’s (1980) ‘‘butcher-on-the-bus” scenario the butcher isrecognized because many previous encounters promoteself-strength, but recollection does not immediately occurbecause the supermarket detail is out-competed by otherdetails that are associated with the bus context. In our

4 A reviewer suggested that the relative variability of RK and confidencecriteria may be investigated by comparing the variability in reaction timesfor making RK and high vs. low confidence decisions.

experimental scenario, in contrast, there is no such contextshift so recognition and recollection tend to be quite highlycorrelated. As a consequence, our model of remember re-sponses, as being dependent on a decision variable that ishighly correlated with the recognition decision variable,provides quite an accurate characterization of performancewhen participants are basing remember responses on theoccurrence of recollection.

In summary, our findings for 2AFC recognition join agrowing body of evidence from single item recognitionthat both the objective and subjective aspects of recogni-tion confidence, and the RK paradigm, are equally well ex-plained, if not better explained, by single-process modelsthan by dual-process models (Dunn, 2004, 2008; Rotello& Macmillan, 2006; Rotello et al., 2006). Given the largebody of evidence taken as supporting single- and dual-pro-cess theories (e.g., Parks & Yonelinas, 2007; Wixted, 2007)our findings cannot decisively adjudicate between thesevery different alternatives. However, we have at least pro-vided a clearly specified extension of these theories to thedomain of 2AFC recognition, which might form the basis offuture development and testing. We also hope that wehave illustrated the potential and importance of a compre-hensive quantitative modeling approach to these issues,both for providing objective evidence about the psycholog-ical processes proposed by each theory, and for providingan objective basis for comparing theories.

Acknowledgments

Thanks to Mark Steyvers for providing the initial set offace pairs taken from the FERET database and sorted intosimilar pairs by members of his laboratory and to AndyYonelinas for clarifications about the proposed thresholdnature of recollection. Thanks also to Neil Macmillan,Shane Mueller, Neil Dobbins and anonymous reviewersfor their guidance. This research was supported by ARCDiscovery Project DP0558407.

A. Appendix

In this appendix, we describe the method by which weobtained predicted response probabilities for SPRK models.Given a RK criterion with a normal distribution, N(c, r2),the probability of a remember response is Pr[N(d0, s2) >N(c, r2)], given memory strength with a mean d0 andstandard deviation s. This remember probability can beexpressed in terms of the probability that a random vari-able incorporating both memory strength and criterionvariability is greater than a fixed criterion: Pr[N(d0, s2 +r2) > c]. The latter mathematical form assumes indepen-dence of memory strength and the remember criterion,but would not be changed in any important way for ourpurposes here if there was some dependence.

We use these results to obtain the SPRK model’s predic-tions about the recognition decision, confidence, and RKdecision using a formally equivalent model where bothtypes of decisions are based on fixed criteria placed on sep-arate normal distributions. The two normal distributionsare correlated due to a memory strength component com-


mon to both. The correlation equals the total proportion ofthe RK decision variable’s variance (s2 + r2) that comesfrom memory strength (s2): r2 = s2/(s2 + r2). Predictionsfor the equivalent model are obtained by integrating abivariate normal distribution over the appropriate areasas depicted in Fig. 10.

In Fig. 10 the y axis represents memory strength and thex axis represents the random variable on which RK deci-sions are based, relative to a fixed RK criterion. Conse-quently the bivariate normal distribution, depicted byequal probability contours in Fig. 10, has a mean of d0 foreach axis, variances of s2 + r2 and s2 for the x and y axesrespectively, and a covariance of s2. The correlation de-picted in Fig. 10 is r � .9, indicating much greater variabil-ity in memory strength than the RK criterion. Note that,although the values in Fig. 10 were taken from fits to thehigh memory-similarity A–A0 condition, the figure couldrepresent any condition with appropriate changes in d0

and s.Fig. 10 is divided by thick solid lines into four rectangu-

lar regions corresponding to know and remember re-sponses that are either correct or false. Within eachrectangle, the areas corresponding to each level of confi-dence are indicated by shading. As in Clark’s (1997) model,the SPRK model assumes that recognition and confidenceresponses are determined relative to symmetric criteriaaround zero. Memory strength greater than zero (i.e., caseswhere the old test item has a greater match than the newtest item) results in a correct recognition decision, whereasmemory strength less than zero (i.e., cases where the newtest item has a greater match than the old test item) resultsin a false recognition decision. The SPRK model alsoassumes that RK decisions are based on criteria that are

Remember-Know

Mem

ory

Stre

ngth

0.01

0.05

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

-2 0 2 4

-20

24 0.01

0.05

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

FalseRemember

CorrectKnow

CorrectRemember

FalseKnow

ConfidenceLowMediumHigh

Fig. 10. The bivarite normal distribution that predicts response proba-bilities for the SPRK model. The distribution is represented by ellipticalequal probability contours, with the probability of a sample fallingoutside the ellipse indicated by numbers on each contour. Rectanglesdemarcated by the thick solid lines correspond to correct and falseremember and know decisions, and shaded areas within each rectanglecorrespond to different levels of confidence.

symmetric around zero. Importantly, the RK criteria arethe same for all choice and memory-similarity conditions,so differences between conditions can only be explainedby Clark’s (1997) underlying model of memory strength.

The criterion that is used to make the RK decision de-pends on the preceding recognition decision. When therecognition decision is correct it is based on a positivememory strength value, and values of the RK decision var-iable greater than the positive RK criterion (indicated bythe thick vertical line to the upper right in Fig. 10) produceremember responses. Otherwise a know response is given.Conversely, when the recognition decision is incorrect it isbased on a negative memory strength value, and values ofthe RK decision variable less than the negative RK criterion(indicated by the thick vertical line to the lower left ofFig. 10) produces remember responses. Otherwise a knowresponse is given. Note that the side on which the remem-ber and know regions occurs swaps for correct and incor-rect responses because a remember classification is madeif the value on the RK dimension is extreme in the samedirection as the recognition decision (i.e., positive for a cor-rect recognition decision and negative for an incorrect rec-ognition decision).

It is important to note that Fig. 10 does not depict atwo-dimensional signal detection model with differenttypes of mnemonic information contributing to confidenceand RK decisions (e.g., Rotello, Macmillan, & Reeder, 2004).Instead, both decisions are based on a common sample ofmnemonic information (memory strength) that is usedfor both the confidence decision and the subsequent RKdecision. The common mnemonic information causes a po-sitive correlation between these decisions. However, thecorrelation is not perfect due to variability in the RK crite-rion. The bivariate distribution in Fig. 10 formally capturesthis relationship between the decisions, as well as the var-iation that affects each individual decision. Hence, integra-tion of the bivariate normal distribution over each of the12 shaded areas depicted in Fig. 10 produces the SPRKmodel’s predictions for each of the 12 types of possibleresponse.

References

Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effectsmodeling with crossed random effects for subjects and items. Journalof Memory and Language, 59, 390–412.

Bamber, D. (1979). State-trace analysis: A method of testing simpletheories of causation. Journal of Mathematical Psychology, 19, 137–181.

Bates, D. M. (2005). Fitting linear mixed models in R. R News, 5, 27–30.Benjamin, A. (2005). Recognition memory and introspective remember/

know judgments: Evidence for the influence of distractor plausibilityon ‘‘remembering” and a caution about purportedly nonparametricmeasures. Memory & Cognition, 33, 261–269.

Benjamin, A. S., Diaz, M. L., & Wee, S. (2009). Signal detection withcriterion noise: Applications to recognition memory. PsychologicalReview, 116, 84–115.

Busemeyer, J. R., & Wang, Y.-M. (2000). Model comparisons and modelselection based on generalization criterion methodology. Journal ofMathematical Psychology, 44, 171–189.

Cary, M., & Reder, L. M. (2003). A dual-process account of the list lengthand strength-based mirror effects in recognition. Journal of Memoryand Language, 49, 231–248.

Clark, S. E. (1997). A familiarity-based account of confidence–accuracyinversions in recognition memory. Journal of Experimental Psychology:Learning, Memory, and Cognition, 23, 232–238.


Clark, S. E., & Gronlund, S. D. (1996). Global matching models ofrecognition memory: How the models match the data. PsychonomicBulletin & Review, 3, 37–60.

Dobbins, I. G., Kroll, N. E. A., & Liu, Q. (1998). Confidence–accuracyinversions in scene recognition: A remember–know analysis. Journalof Experimental Psychology: Learning, Memory, and Cognition, 24,1306–1315.

Donaldson, W. (1996). The role of decision processes in remembering andknowing. Memory & Cognition, 24, 523–533.

Dougal, S., & Rotello, C. M. (2007). ‘‘Remembering” emotional words isbased on response bias, not recollection. Psychonomic Bulletin &Review, 14, 423–429.

Dunn, J. C. (2004). Remember–know: A matter of confidence. PsychologicalReview, 111, 524–542.

Dunn, J. C. (2008). The dimensionality of the remember–know task: Astate-trace analysis. Psychological Review, 115, 426–446.

Dunn, J. C., & Kirsner, K. (1988). Discovering functionally independentmental processes: The principle of reversed association. PsychologicalReview, 95, 91–101.

Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. NewYork: Chapman & Hall.

Gardiner, J. M., Ramponi, C., & Richardson-Klavehn, A. (2002). Recognitionmemory and decision processes: A meta-analysis of remember, know,and guess responses. Memory, 10, 83–98.

Gillund, G., & Shiffrin, R. M. (1984). A retrieval model for both recognitionand recall. Psychological Review, 91, 1–67.

Heathcote, A., Freeman, E., Etherington, J., Tonkin, J., & Bora, B. (2009). Adissociation between similarity effects in episodic face recognition.Psychonomic Bulletin & Review, 16, 824–831.

Hintzman, D. L. (1988). Judgments of frequency and recognition memoryin a multiple-trace memory model. Psychological Review, 95, 528–551.

Jacoby, L. L. (1991). A process dissociation framework: Separatingautomatic from intentional uses of memory. Journal of Memory &Language, 30, 513–541.

Kapucu, A., Rotello, C. M., Ready, R. E., & Seidel, K. N. (2008). Response biasin ‘‘remembering” emotional stimuli: A new perspective on agedifferences. Journal of Experimental Psychology: Learning, Memory andCognition, 34, 703–711.

Loftus, G. R., Oberg, M. A., & Dillon, A. M. (2004). Linear theory,dimensional theory, and the face-inversion effect. PsychologicalReview, 111, 835–865.

Macmillan, N. A., & Creelman, C. D. (2005). Detection theory: A users guide(2nd ed.). New York: Cambridge University Press.

Macmillan, N. A., Rotello, C. M., & Verde, M. F. (2005). On the importanceof models in interpreting remember–know experiments. Memory, 13,607–621.

Mandler, G. (1980). Recognizing: The judgment of previous occurrence.Psychological Review, 87, 252–271.

McCullagh, P., & Nelder, J. A. (1989). Generalized linear models. London:Chapman & Hall.

Mueller, S. T., & Weidemann, C. T. (2008). Decision noise: An explanationfor observed violations of signal detection theory. PsychonomicBulletin & Review, 15, 465–494.

Parks, C. M., & Yonelinas, A. P. (2007). Moving beyond pure signaldetection models: Comment on Wixted (2007). Psychological Review,114, 188–202.

Parks, C. M., & Yonelinas, A. P. (2009). Evidence for a memory threshold insecond-choice recognition memory responses. Proceedings of theNational Academy of Science USA, 106, 11515–11519.

Phillips, P. J., Wechsler, H., Huang, J., & Rauss, P. (1998). The FERETdatabase and evaluation procedure for face recognition algorithms.Image and Vision Computing Journal, 16, 295–306.

Pitt, M. A., & Myung, I. J. (2002). When a good fit can be bad. Trends inCognitive Sciences, 6, 421–425.

Platt, J. R. (1964). Strong inference. Science, 4, 79–95.Rotello, C. M., & Macmillan, N. A. (2006). Remember–know models as

decision strategies in two experimental paradigms. Journal of Memoryand Language, 55, 479–494.

Rotello, C. M., Macmillan, N. A., Hicks, J. L., & Hautus, M. J. (2006).Interpreting the effects of response bias on remember–knowjudgments using signal detection and threshold models. Memory &Cognition, 34, 1598–1614.

Rotello, C. M., Macmillan, N. A., & Reeder, J. A. (2004). Sum–differencetheory of remembering and knowing: A two-dimensional signaldetection model. Psychological Review, 111, 588–616.

Rotello, C. M., Macmillan, N. A., Reeder, J. A., & Wong, M. (2005). Theremember response: Subject to bias, graded, and not a process-pureindicator of recollection. Psychonomic Bulletin & Review, 12, 865–873.

Starns, J. J., & Ratcliff, R. (2008). Two dimensions are not better than one:STREAK and the univariate signal detection model of remember/knowperformance. Journal of Memory and Language, 59, 169–182.

Tulving, E. (1981). Similarity relations in recognition. Journal of VerbalLearning and Verbal Behaviour, 20, 479–496.

Tulving, E. (1985). Memory and consciousness. Canadian Psychology, 26,1–12.

Wagenmakers, E-J., & Farrell, S. (2004). AIC model selection using Akaikeweights. Psychonomic Bulletin & Review, 11, 192–196.

Wagenmakers, E.-J., Grunwald, P., & Steyvers, M. (2006). Accumulativeprediction error and the selection of time series models. Journal ofMathematical Psychology, 50, 149–166.

Wasserman, L. (2000). Bayesian model selection and model averaging.Journal of Mathematical Psychology, 44, 92–107.

Wickelgren, W. A. (1977). Learning and memory. Englewood Cliffs, NJ:Prentice Hall.

Wixted, J. T. (2007). Dual-process theory and signal-detection theory ofrecognition memory. Psychological Review, 114, 152–176.

Wixted, J. T., & Stretch, V. (2004). In defense of the signal detectioninterpretation of remember/know judgments. Psychonomic Bulletin &Review, 11, 616–641.

Yonelinas, A. P. (1994). Receiver-operating characteristics in recognitionmemory: Evidence for a dual-process model. Journal of ExperimentalPsychology: Learning, Memory, and Cognition, 20, 1341–1354.

Yonelinas, A. P. (2002). The nature of recollection and familiarity: Areview of 30 years of research. Journal of Memory and Language, 46,441–517.

Documents

Recollection and confidence in two-alternative forced choice episodic recognition