8/3/2019 Donaldson Measuring Recognition Memo

1/3

Journal of Experimental Psychology: General1992, Vol. 121, No . 3,275-277

Copyright 1992 by the American Psychological Association, Inc.0096-3445/92/S3.00

Measuring Recognition MemoryWayne Donaldson

University of New Brunswick, Fredericton, New Brunswick, Canada

Recent years have seen an expandedinterest in recognition memory tasks. This resurgenceofinterest has also renewed concerns with measurement problems. Comparing 4 models ofrecognition memory, Snodgrass and Corwin (1988) found that measures of bias from thedistribution-free(nonparametric) model were inadequate. However, their analysis wasbased onbias measures that can be shown a priori to be nonindependentof discrimination. This articletraces the history of the nonparametric model and develops a better measure of bias. Theconsequenceof developingthis better measure is that the nonparametricmodel deservesseriousconsideration.

"One recurring analytical problem is the comparison ofperformance measures when both successes (hits) and failures(false alarms) must be considered." Norman (1964, p. 243)began an article with that sentence more than a quarter of acentury ago, and it is still appropriate. How does one makecomparisons when performance involves both the ability toremember (discrimination) and the willingness to say so(bias)? Snodgrass and Corwin (1988) examined four theoret-ical models of yes-no recognition memory and their associ-ated measures of discrimination and bias. When the inde-pendence of various measures of discrimination and bias wastested both theoretically and against data, some models andmeasures fared better than others. The distribution-free (non-parametric) model did particularly badly. However, the non-parametric bias measures developed in the literature can beshown to be unsatisfactory at a conceptual level. This articletraces the historical development of the nonparametric model,details how and why the inadequate bias formulas continueto be used, presents a better measure of response bias, andargues that the nonparametric model remains viable and maybe the best of the four theoretical models.

The Nonparametric Model

The nonparametric model of yes-no recognition memorybegins with a consideration of the unit square that defines allpossible performances when hit rate (H, probability of re-sponding yes to an old item) is plotted against false alarm rate(FA, probability of responding yes to a new item). As shownin Figure 1, when one has only one data point (FA, H), thesquare is divided into four regions by drawing two straight

lines through that data point. One line passes through thepoint where a participant says no to everything (0,0), and the

A preliminary version of the ideas in this article was presented atthe 1991 AnnualMeeting of the Canadian Psychological Association.Special thanksto Joan Gay Snodgrass, particularly for pointing outthe concept in Formula 5, and to an anonymousreviewer, both ofwhose comments made this a much better article. Thanks also toRoman Mureika for preliminarymathematical consultation.

Correspondence concerning this article should be addressed toWayne Donaldson, Psychology Department, University of NewBrunswick,Fredericton, New Brunswick,Canada E3B 6E4.

other passes through the point (1, 1). Any second set of datathat falls in the portion of the square labeled / representsdiscrimination performance inferior to that of the originalpoint. Any point in Area S represents performance superiorto that of the original point. The triangular areas marked Arepresent ambiguous performance when compared with theoriginal data point. In past discussions, these areas haveusually been labeled Al and A2, but I label them AL torepresent performance that is more liberal than the originalpoint and A c to represent more conservative behavior. Per-formance that falls in either of the triangular areas is ambig-uous in comparison with the original data point with regardto discrimination only, and not with regard to bias.

The historical analysis of this unit square can be tracedfrom Norman (1964). Pollack and Norman (1964) suggestedthat in the absence of the full operating characteristic curve,one could estimate the area under that curve as the sum ofthe I region plus one half of the A regions:

A' = I + l/2(Ac + AL) (1)

Work by Green and Moses (1966) suggested that the areaunder the operating characteristic is a bias-free measure ofmemory. Hodos (1970) extended the analysis of the unitsquare to develop measures of response bias. The measuresare related to the relative areas of the two ambiguous regions.If AC= A L , then performance is unbiased. If AC > AL, thenthere is a liberal bias. Rather than use the actual areas of ALand AC to develop his measures, however, Hodos chose toconsider the "two partly overlapping right triangles" (p. 352)that can be identified in Figure 1 as AL + S and Ac + S. Thesubsequent analysis was graphical only, and no computational

formulas were presented. Computational formulas were de-veloped by Grier (1971),and herein lies the source of the laterconfusion. Grier began by using Pollack and Norman's (1964)proposal of the area (A') under the operating curve shown inFormula 1 as a measure of memory. The computationalformula is

A' = 1/2 + [(H - FAX1 + H - FA)]/[4H(1 - FA)]. (2)

Developing Hodos's (1970) suggestions for measures of bias,Grier also moved away from the areas AC and AL and turnedto the "two triangles sharing a common right angle in theupper left corner" (p. 425). He continued to use the same

275

8/3/2019 Donaldson Measuring Recognition Memo

2/3

276 WAYNE DONALDSON

H

FAFigure 1. The unit square thatdefines all possible performanceswhen hit (H) rate (probability of responding yes to an old item) isplotted againstfalse alarm (FA) rate (probability of responding yes toa new item). (A ny point in A rea S represents performance superiorto the original data point. Any point in Area I represents discrimi-nation performanceinferior to that of the original data point. Theareas marked A represent ambiguous performance when comparedwith the original data point. AL represents performance thatis moreliberal than the original point and AC represents performance that ismore conservative thanthe original point.)

labels for these triangles, however. Thatis, when consideringdiscrimination (A'),Grier used the labelA l to identify AL in

Figure 1. The same A l label was then used toidentify thedifferent area AL + S when calculating measures of bias. Twopossible bias measures were suggested:

B' H = [(A L + S) - (Ac + S)]/(A L + S) if A L > AC, (3)

or

B" = [(AL + S) - (Ac + S)]/[(AL + S) + (A c + S)]. (4)

Formula 4 can besimplified as Formula 5:

B" = (A L - A C )/(A L + A c + 2S). (5)

In this form, it is clear that as discrimination decreases thearea S becomes larger and increasingly limits the possiblerange of B". B 'H is similarly affected, although less severely,because S occurs only once in the denominator. Grier pre-sented compu tational form ulas for both m easures. These werethen adopted by Snodgrass and Corwin (1988) under theclearly expressed impression thatthe formulas referredto theareas AL and A c only. The formulas were used to generatevarious functions and quickly led to the rejection of thenonparam etric model because(a) neither measure of bias isindependentof A' and (b) theisobias functions did not fit thedata. Macmillan and Creelman (1990) evaluatedthe samemeasures and rejected them for a variety of reasons, includingthe finding that the rangeof the measureswas notindependentof sensitivity.

The Formulas: Discrimination and Response Bias

The formula for A', as presented by Grier (1971), is appro-priate because it is based on the areas A c and AL. It isreproduced as Form ula 2(shown earlier).

Hodos (1970) and Grier (1971) considered two possible

measures of bias. One, B 'H, proposed taking the differencebetween the two ambiguous areas(AL A c) as a proportionof the larger of the two areas. This necessitatestwo differentformulas, depending on where in the unit square the datapoint falls, and I do not consider this further. Their secondmeasure, B", proposed takingthe difference between the twoambiguous areas(AL A c) as a proportion of their combinedarea (AL + A c) but, as indicated earlier, did not do so becauseof the inclusion of Area S. Formula 6 reflects this and,at thesuggestion of both reviewersof this article, shouldbe consid-ered a new measure of bias rather thana "correction" of theexisting B". To indicate its status I call it B"D:

B"D = (AL - AC)/(AL + Ac). (6)

Computational Formula 7 serves all data points:

B" D = [(1 - H)(l - FA) - HFA]/

[(1 - H)(l - FA) + HFA]. (7)

No bias is indicated byB"D = 0. Negative numbers representliberal bias, positive numbers represent conservative bias,andthe maximum in either direction is1.0.

The isobias function for B"D is

H = (1 - B"D)(1 - FA)/[1 - B"D(1 - 2FA)]. (8)

Relationship Between A' and B" D

Snodgrass and Corwin(1988) examined the relationshipbetween measures of discrimination and bias in the variousmodels. ConfirmingHodos's (1970) findings, Snodgrass andCorwin demonstrated (p. 40, Figure 4, panel D) nonparam et-ric isobias contours using B" that converge toward(0, 0) and(1, 1) at chance performance. This means that bias is hard tomeasure at low levels of discrimination. This was demon-strated a second way by plotting the m axim um possible valuesof B" as discrimination movedfrom chance (A ' = 0.5) toperfect (A' = 1.0). Figure 5, panel D, in Snodgrass and Corw in(1988) shows marked dependence betweenB" limits and A ',as expected from an examination of Formula 5. The use ofB"D and Formula 8 generates a totallydifferent picture. Figure2 showsisobias functionsfor B"D. The isobias co ntou rs clearlymaintain separation at low levels of discrimination. Theseisobias contours are not embarrassed by Snodgrass and Cor-win's empirical findings, either. The maximum possible B"Dvalues as a function of A' have not been plotted. The theo-retical limits are 1.0 for maximum conservative biasand -1.0for maximum liberal bias.At A' = 0.5 (chance),and acceptingFA = 0.01, the actual limit on B"D is 0.9998. At A' = 0.9the limit is still 0.9909, and when A' = 0.95, the limitdrops to only 0.9152. If the experiment is sensitive enoughto detect false alarm rates of less than 0.01, then the actuallimits on B"D are even closer to 1.0.

8/3/2019 Donaldson Measuring Recognition Memo

3/3

MEASURING RECOGNITION MEMORY 277

H

FA

Figure 2. Isobias functions for the measure of bias B" D . (H = hitrate, or the probability of responding yes to an old item; FA = falsealarm rate, or the probability of responding yes to a new item.)

Thus, using formulas actually based on the areas Ac andAL in the unit square produces a measure of bias that isconsistent with the data presented by Snodgrass and Corwin(1988) and is less limited by the level of discriminationperformance than any of the other four models. In addition,B" D should be given a positive rating on all but two ofMacmillan and Creelman's (1990) eight criteria. The measureis monotonic, well behaved both at and below chance, and

independent of sensitivity. It also rates a plus where B" doeson association with a desirable index of sensitivity and use-fulness in theory. Finally, averaging across observers producesonly slightly conservative estimates. Using the simplest caseof two observers with the same bias (e.g., B" D = 0.5) anddifferent sensitivities (A' = .78 and .53), the pooled estimate

of bias is .486. The nonparametric model remains a valid wayof handling a recurring analytical problem.

References

Green, D. M., & Moses, F. L. (1966). On the equivalence of tworecognition measures of short-term memory. Psychological Bulle-tin, 66, 228-234.

Grier, J. B. (1971). Nonparametric indexes for sensitivity and bias:Computing formulas. PsychologicalBulletin, 75, 424-429.

Hodos, W. (1970). Nonparametric index of response bias for use indetection and recognition experiments. PsychologicalBulletin, 74,351-354.

Macmillan, N. A., & Creelman, C. D. (1990). Response bias: Char-acteristics of detection theory, threshold theory, and ''nonparamet-ric" indexes. Psychological Bulletin, 107, 401-413.

Norman, D. A. (1964). A comparison of data obtained with differentfalse-alarm rates. PsychologicalReview, 71, 243-246.

Pollack, I., & Norman, D. A. (1964). A non-parametric analysis ofrecognition experiments. PsychonomicScience, 1, 125-126.

Snodgrass, J. C., & Corwin, J. (1988). Pragmatics of measuringrecognition memory: Applications to dementia and amnesia. Jour-na l of Experimental Psychology: General, 117, 34-50.

Received June 19, 1991Revision received November 4, 1991

Accepted December 2, 1991