Upload
harold-lee
View
213
Download
1
Embed Size (px)
DESCRIPTION
test
Citation preview
TEST REVIEW
The WCST-64: A Standardized Short-Form of theWisconsin Card Sorting Test�
Kevin W. GreveDepartment of Psychology, University of New Orleans, New Orleans, LA, USA, and
Jefferson Neurobehavioral Group, Metairie, LA, USA
ABSTRACT
The Wisconsin Cart Sorting Test (WCST) is a well-established measure of executive function. Practical and®nancial constraints have increased the need for abbreviated neuropsychological procedures. A number ofabbreviated versions of the WCST have been introduced and cogent arguments can be made for one overanother in certain situations. However, the single deck, 64-card WCST (WCST-64) is the most logical andpractical short form. Psychological Assessment Resources (PAR) has recently published a new manual withcomprehensive norms for the WCST-64. This paper reviews the new product, discusses the comparability ofthe WCST-64 and the standard version, and suggests directions for future research.
The Wisconsin Cart Sorting Test (WCST; Grant
& Berg, 1948; Heaton, 1981; Heaton et al., 1993)
is a well-established measure of executive func-
tion. The ever-increasing number of studies incor-
porating the WCST illustrates its value and
popularity. The ®rst 40 years of the WCST's
existence (1948±1988) saw its use in less than
100 published journal articles with over half of
those appearing in the 1980s alone. In contrast,
the last 12 years have witnessed the publication of
over 500 articles using the WCST. The early days
of the test were characterized by both systematic
and nonsystematic variation of almost all aspects
of the WCST (see Heaton, 1981, or Heaton et al.,
1993, for an outline of many of these variations)
with variations in administration and scoring con-
tinuing to appear even now (Stanford, Greve, &
Gerstle, 1997; Stuss et al., 2000). The WCST
methodology with which most neuropsycholo-
gists are familiar was formalized by Heaton with
the publication of the ®rst WCST manual by
Psychological Assessment Resources (PAR) in
1981. With the second edition of the manual
(Heaton et al., 1993), users were provided with
more comprehensive norms and scoring instruc-
tions which took much of the mystery out of
identifying perseverative responses. The use of
these new norms, of course, requires that the test
be administered in `̀ standard fashion''. Standard
fashion frequently means completing all 128
trials, an often painful and time-consuming
process. In a profession where the motivation of
the patient is critical to the validity of our clinical
tasks and the amount of time allotted to an
evaluation is limited by either practical or ®nan-
cial constraints, these are not idle concerns.
�The author served as a compensated beta-tester of the WCST-64 for Psychological Assessment Resources prior tobeing asked to write this review.Address correspondence to: Kevin W. Greve, Ph.D., Department of Psychology, University of New Orleans-Lakefront, New Orleans, LA 70148, USA. Tel.: +1-504/280-6185. Fax: +1-504/280-6048. E-mail: [email protected] for publication: August 29, 2000.
The Clinical Neuropsychologist 1385-4046/01/1502-228$16.002001, Vol. 15, No. 2, pp. 228±234 # Swets & Zeitlinger
Dow
nloa
ded
by [
Seou
l Nat
iona
l Uni
vers
ity]
at 1
8:44
01
Febr
uary
201
6
Attempts to address these issues in the WCST
have led to the development of several abbre-
viated versions.
Nelson (1974) reported the use of a modi®ca-
tion (MCST) which involved the removal of all
ambiguous response cards (i.e., those matching
a key card on more than one dimension) from
the set of 128 cards, thus leaving a deck of 48
unambiguous cards. The smaller response deck
necessitated shorter criterion runs (6 rather than
10 consecutive correct responses) but greatly
simpli®ed the scoring of perseverative responses.
Unfortunately, Nelson also warned subjects of the
impending change of correct dimension thereby
eliminating an essential element of the WCST.
Also, rather than having a ®xed sequence of
`correct' dimensions, the ®rst dimension to which
the subject sorted was considered correct; after
completing that criterion run and being warned of
the changing of the rule, the next new dimension
to which the subject sorted became correct. The
advantage of this version is that it is easier, thus
potentially reducing frustration and ¯oor effects
(Lezak, 1995). Of course it also obviates the need
to discover the correct sorting dimension and to
recognize when the sorting dimension has chan-
ged. The use of this version in a number of
populations has been reported (many of these
studies are reviewed in Zubicaray & Ashton,
1996). In a study directly comparing the MCST
and the standard WCST, van Gorp et al. (1997)
found the two tests generally comparable but
questioned Lezak's (1995) conclusion about the
dif®culty of the task. In reviewing the literature on
the sensitivity of the MCST to frontal lobe patho-
logy, Zubicaray and Ashton (1996) concluded, `̀ it
is likely that the MCST is an altogether different
test from the standard version'' (p. 245).
Variations of Nelson's approach which pre-
serve the essential character of the WCST
administration procedure while using only the
unambiguous cards have also been reported
(Bondi, Kazniak, Bayles, & Vance, 1993;
Greve & Smith, 1991; Hart, Kwentus, Wade, &
Taylor, 1988; Jenkins & Parsons, 1978; Ramage,
Bayles, Helm-Estabrooks, & Cruz, 1999). Greve,
Bianchini, Hartley, and Adams (1999) argued that
this type of modi®cation might be more useful
with older or more severely impaired patients in
rehabilitation settings than standard versions of
the WCST because it might be less prone to ¯oor
effects. While this modi®cation compared favor-
ably with the standard WCST in a small sample of
neurologically intact older adults (Greve & Smith,
1991), the question remains whether it also is `an
altogether different test'.
A more intuitive approach to WCST short-
form development is the use of one deck of
response cards instead of two (the WCST-64),
thus immediately cutting the test in half. This
procedure retains all the features of the standard
WCST except length. Unlike the MCST, the use
of the WCST-64 is relatively recent, ®rst reported
by Haaland, Vranes, Goodwin, and Garry (1987)
in a study of normal aging. In direct comparisons
Heaton and Thompson (1992), Axelrod, Henry,
and Woodard (1992), Sillanpaa et al. (1993), and
Smith-Seemiller, Franzen, and Bowers (1997)
found the WCST-64 to be generally comparable
to the standard version. Axelrod, Jiron, and Henry
(1993) and Paolo, Axelrod, TroÈster, Blackwell,
and Koller (1996) found the WCST-64 sensitive
to the effects of normal aging and pathological
aging (Alzheimer's and Parkinson's disease),
respectively. While there appears to be justi®ca-
tion for believing that the two versions are
comparable and have similar sensitivity to the
effects of neuropathology, Axelrod, Abraham,
and Paolo (1996) cautioned against simply
converting WCST-64 scores to percentages and
then using the standard norms.
Unlike the MCST and the WCST-64, which
have been used repeatedly in published research,
two other variations of the WCST, the WCST-3
(Robinson, Kester, Saykin, Kaplan, & Gur, 1991)
and the Milwaukee Card Sorting Test (Osmon &
Suchy, 1996) have been reported once each. The
WCST-3 is identical to the standard WCST but is
terminated when three, rather than six, catego-
ries, have been successfully achieved. While this
version may be shorter in theory, in practice it
likely will not be since many patients often have
dif®culty achieving even a single category.
Further, it was found inferior to the WCST-64 in
terms of classi®cation accuracy and in under-
estimating performance on the standard WCST
(Robinson et al., 1991). The Milwaukee Card
Sorting Test is a 64-card version administered
REVIEW OF A STANDARDIZED SHORT-FORM OF THE WCST-64 229
Dow
nloa
ded
by [
Seou
l Nat
iona
l Uni
vers
ity]
at 1
8:44
01
Febr
uary
201
6
with the standard instructions designed to quan-
tify the `̀ separate, elemental executive abilities
needed for card sorting performance'' (p. 541;
Osmon & Suchy, 1996). This version requires the
subjects to verbalize how they are sorting each
card. A factor analysis resulted in three factors
similar in interpretation to those usually reported
for the WCST but included a number of idi-
osyncratic scores which make direct comparison
dif®cult.
The prevalence of WCST short-forms in the
research literature illustrates the demand for an
abbreviated version. The need in clinical settings
may be even greater given the multitude of
practical factors affecting clinical test selection.
The WCST-64 is the most logical downward
extension of the WCST, the one for which quality
norms are most readily available, and the one to
which the existing WCST literature is most likely
to generalize. Thus, PAR has now introduced a
comprehensive WCST-64 package.
PAR'S WCST-64 PACKAGE
The WCST-64 package has ®ve elements which
can be purchased together, in various combina-
tions, or individually. These include: (1) a single
deck of 64 response cards and four stimulus cards;
(2) record sheets; (3) scoring software; (4) admin-
istration software; and (5) a professional manual
with comprehensive norms. Needless to say, the
cards themselves differ in no way from the cards
already in use. The record sheets are modeled
after the revised forms used with the standard
version. Speci®cally, the ®rst of three pages has
spaces for general identifying information, refer-
ral information, current medications, behavioral
observation information, and a description of the
testing situation. Devoting space on the record for
most of this information is unnecessary since
most of those data will be collected elsewhere and
need not be listed again. Realistically, it is an
inef®cient use of time to do so and there is ample
room on the response record form (page 2) to
record necessary identifying information and
behavioral observations. Page 3 is the familiar
`scoring area' modi®ed for 64 trials. There is
nothing but a copyright notice on the back. In
short, the WCST-64 score sheet is unnecessarily
long and wasteful; what is really needed would
easily ®t on the front and back of a single letter-
size page.
The scoring / administration software (WCST-
64: CV; Heaton & PAR Staff, 2000) is identical to
the program recently released for the standard
version (WCST: CV3; Heaton & PAR Staff,
1999) except that it accepts/administers only 64
trials and contains the norms for the WCST-64. It
also comes with its own set of paper scoring forms
which contain the same information page and
response area on which one marks the stimulus
card matched rather than the matching dimen-
sions (the WCST-64: CV can accept either type of
response). As with the original version, this
software is very easy to use, allows scoring and
storing of multiple administrations per individual,
and has some useful ¯exibility in terms of the
style of presentation. Unfortunately, this version
is completely independent of the WCST: CV3
software, so if you intend to score / administer and
norm both standard and 64-card protocols with
the computer, you must have both sets of software.
This is an expensive problem that apparently will
not be remedied in the near future. On the plus
side, one can buy just the scoring software which
at US$ 225 is about half the price of the total
software package. A technical problem relevant to
all the WCST software is that one must be able to
simultaneously run a CD ROM drive and ¯oppy
disk drive to load the application. This makes it
nearly impossible to load on most laptop
computers.
Norms for the WCST-64 are easily produced
since no new data need be collected. One need
only rescore for the ®rst 64 cards the protocols for
the original normative groups and this is precisely
what has been done. The WCST-64 generates 10
familiar scores: Total Number Correct (TC), Total
Number of Errors (TE), Perseverative Responses
(PR), Perseverative Errors (PE), Nonperseverative
Errors (NPE), Conceptual Level Responses
(CLR), Number of Categories Completed (CC),
Trials to Complete First Category (T1C), Failure
to Maintain Set (FMS), and Learning to Learn
(LL). As with the standard version, no norms are
provided for TC. For TE, PR, PE, NPE, and CLR
age- and education-corrected standard scores,
230 KEVIN W. GREVE
Dow
nloa
ded
by [
Seou
l Nat
iona
l Uni
vers
ity]
at 1
8:44
01
Febr
uary
201
6
t-scores, and percentile rankings are provided for
adults. Similar scores are provided based upon the
United States Census age-matched adult sample.
For children and adolescents under 20, only age-
corrected norms are provided. Percentile ranks
only, based on the same samples, are provided for
CC, T1C, and LL. Note that there are no percent
scores since all subjects always complete the
same number of trials. That all subjects receive
the same number of trials is a signi®cant strength
of the WCST-64; having two different termination
criteria for the standard version is a signi®cant
methodological ¯aw that cannot really be over-
come with percent scores and which is unlikely to
be corrected for practical reasons. Many research-
ers are using all 128 cards (Stanford, Greve, &
Gerstle, 1997; Stuss et al., 2000) and there is
evidence that the factor structure in that version is
more consistent than with the standard version
(Greve et al., 1999). No norms are provided for
FMS because FMS was `̀ rare in all samples and
did not discriminate between the normal and
clinical groups'' (p. 24, Kongs, Thompson,
Iverson, & Heaton, 2000). One must hope that
FMS, despite its presence in almost all factor
analytic solutions including those for the WCST-
64 (see below), is not a clinically relevant
variable; that question remains open.
The new manual is an excellent addition to the
WCST literature. Structured almost identically to
the 1993 manual, it contains seven chapters and
®ve appendices and runs to 242 pages (the 1993
manual is 230 pages long). The Introduction
(Chapter 1) is a brief overview of the WCST
including a discussion of short-forms. The reader
should not expect anything approaching a com-
prehensive comparative review of this topic; in
fact, the review is disappointingly cursory.
Chapters 2 (Test Materials and Use) and 3
(Administration and Scoring) are little different
in the two manuals with modi®cation necessary
for the WCST-64 and some slight changes in
scoring instructions that seem to re¯ect improve-
ments suggested by experience over the past 7
years of use. Chapter 4 presents the methods used
to norm the WCST-64 while Chapter 5 discusses
interpretive issues. One addition in this latter
chapter is a discussion of the interpretation of
multiple scores. The gist of this section is to
remind users of the high inter-correlations among
many WCST scores and caution them against
necessarily considering those scores independent
sources of information. This section also notes
that a ®nding of at least one score in the impaired
range is a fairly common occurrence even in the
normal population. Three sample cases are
provided. Chapter 6 discusses reliability and
validity. Much of the reliability data are new,
speci®c to the WCST-64. The data on scoring
accuracy and reliability are based on published
research with the standard WCST but are directly
applicable to the WCST-64. This chapter also
contains an updated discussion of validity studies.
Many reported validity studies naturally involve
the standard WCST because there have been few
direct studies of the WCST-64. The authors
cautiously assume that if the two versions of the
WCST are comparable then research done with
the standard version will be generalizable to the
WCST-64. There is reason to believe that the two
are quite comparable and these data are also
presented in the manual (Chapter 7). Both
manuals have ®ve appendices including the two
sets of normative tables (for the United States
Census age-matched adult sample and for the
demographically corrected normative data) and
base-rate data for the normative and clinical
samples. Dropped from the current manual are the
two appendices presenting administration and
scoring variations of the WCST. The ®nal
appendix, a completely new addition, provides
cumulative percentile ranks for the normative
sample by age for selected scores (TE, PR, PE,
NPE, and CLR). Despite their importance, the
base-rate tables are a little hard to follow and their
use in interpretation is not described in the
detailed case illustrations of Chapter 5.
DISCUSSION
As with the standard WCST, the PAR group have
done an excellent job in their technical presenta-
tion of the WCST-64. Of course, an important
consideration in determining the ultimate value of
the WCST-64 is whether the short version is
comparable to the standard version. Comparabil-
ity is not an absolute necessity as the normative
REVIEW OF A STANDARDIZED SHORT-FORM OF THE WCST-64 231
Dow
nloa
ded
by [
Seou
l Nat
iona
l Uni
vers
ity]
at 1
8:44
01
Febr
uary
201
6
and related data for the WCST-64 allow it to stand
alone as a neuropsychological procedure. How-
ever, if the two tests are comparable, then past
research on the standard WCST could more safely
be applied to the WCST-64. In this regard, there
are convincing data to support the comparability
assumption in an overall sense. Some of the
earlier comparison studies have been discussed
above; these tend to support the comparability of
the two versions. Comparisons of the two versions
undertaken speci®cally for the new manual, using
the normative and patient data sets, also support
comparability. Some statistically signi®cant dif-
ferences between PR standard scores for the two
versions were reported but examination of the
sample means indicates that these differences are
generally statistically trivial given the large
sample sizes.
The factor structure and construct validity of
the WCST have been the subject of considerable
research throughout the 1990s (Goldman et al.,
1996; Greve, Love, et al., 1999; Greve, Bianchini,
et al., 1999; Greve, Brooks, Crouch, Williams, &
Rice, 1997; Greve, Ingram, & Bianchini, 1998;
Koren et al., 1998; Paolo, TroÈster, Axelrod, &
Koller, 1995; Sullivan et al., 1993; Wiegner &
Donders, 1999). These studies have reported a
relatively consistent three-factor structure across
a number of different populations though the
exact number and organization of the factors is
dependent on variable and sample selection (see
Greve et al., 1998, for a review of many of the
recent studies). The scores which load most highly
on Factor I re¯ect aspects of executive function,
particularly response in¯exibility (PE, PR, TE)
and, secondarily, disrupted problem solving (per-
cent CLR, CC, TC). The composition of Factor II
(high loading for NPE; moderate loadings for
percent CLR, CC, TC) seems to re¯ect an inef-
fective hypothesis-testing strategy in the absence
of perseveration. Factor II is often absent in high
functioning persons (see for example, Greve et al.,
1997, and Wiegner & Donders, 1999). Factor III
is comprised of scores which seem to measure the
ability to maintain correct responding once the
correct dimension is discovered (high FMS and
commensurately fewer CC).
The manual reports factor analyses of the
WCST-64 in ®ve different subsamples (adult
normative, adult clinical, child normative, child
lesion, child diagnostic) which revealed the same
three-factor structure. These authors interpreted
their factors as re¯ecting a perseveration compo-
nent, a concept-formation component, and a
Failure to Maintain Set component. The analysis
of the Adult Clinical group resulted in a fourth
marginal factor represented only by CLR. This
factor had an Eigenvalue of only .71, which is
well below the usual 1.0 minimum, but accounted
for over 10% of the observed variance. Overall
this solution accounted for 99% of the variance.
The similarity of the factor structure of the
WCST-64 to those solutions for the standard
WCST reported for various populations supports
the notion that the abbreviated version is com-
parably sensitive to the cognitive processes
underlying performance in the standard test.
However, factor analyses of the standard WCST
in the normative and clinical samples have not
been reported. A direct comparison of the factor
structures for the two WCST versions in the same
subjects would have provided stronger evidence
of their comparability.
It is hard to argue that there is not a need for a
briefer version of the WCST in many clinical
contexts. A number of abbreviated versions have
been introduced and cogent arguments can be
made for one over another in certain situations.
Nevertheless, if we as clinicians wish to have a
short-form with quality norms that can also take
advantage of over 50 years of research on the
WCST then the most pragmatic approach is to use
the WCST-64. The PAR group have done an
admirable job in developing the norms for this
version and making a solid case for the compar-
ability of the two versions. The weaknesses of
PAR's WCST-64 are not directly related to the
psychometric features of the test but may be more
an issue with software development and market-
ing; these weaknesses have little or no direct
impact on the clinical utility of the WCST-64. In
terms of direct use of the WCST-64, one should
be cautious about using the WCST-64 to the
exclusion of the standard WCST in research.
Additional research is needed to further establish
the nature and strength of the relationships
between the two versions in a variety of patient
populations and to identify cases in which com-
232 KEVIN W. GREVE
Dow
nloa
ded
by [
Seou
l Nat
iona
l Uni
vers
ity]
at 1
8:44
01
Febr
uary
201
6
parability might be weak and one version might
be superior to the other. Expansion of the WCST-
64 research base can be done by piggy-backing
analysis of the WCST-64 onto studies utilizing the
standard WCST. This approach may not be feasi-
ble for all WCST studies but every effort should
be made to include WCST-64 data when possible,
especially in any research directly examining the
psychometric properties of the WCST.
REFERENCES
Axelrod, B.N., Abraham, E., & Paolo, A.M. (1996).Comparability of standard WCST norms with the64-card version. The Clinical Neuropsychologist,10, 334.
Axelrod, B.N., Henry, R.R., & Woodard, J.L. (1992).Analysis of an abbreviated form of the WisconsinCard Sorting Test. The Clinical Neuropsychologist,6, 27±31.
Axelrod, B.N., Jiron, C.C., & Henry, R.R. (1993).Performance of adults ages 20 to 90 on theabbreviated Wisconsin Card Sorting Test. TheClinical Neuropsychologist, 7, 205±209.
Bondi, M.W., Kaszniak, A.W., Bayles, K.A., & Vance,K.T. (1993). Contributions of frontal systemdysfunction to memory and perceptual abilities inParkinson's Disease. Neuropsychology, 7, 89±102.
Goldman, R.S., Axelrod, B.N., Heaton, R.K., Chelune,G.J., Curtiss, G., Kay, G.G., & Thompson, L.L.(1996). Latent structure of the WCST with thestandardization samples. Assessment, 3, 73±78.
Grant, E.A., & Berg, D.A. (1948). A behavioralanalysis of degree of reinforcement and ease ofshifting to new responses in a Weigl-type cardsorting problem. Journal of Experimental Psychol-ogy, 38, 404±411.
Greve, K.W., & Smith, M.C. (1991). A comparison ofthe Wisconsin Card Sorting Test with the Modi®edCard Sorting Test for use with older adults.Gerontology & Geriatrics Education, 11, 57±65.
Greve, K.W., Bianchini, K.J., Hartley, S.M., & Adams,D. (1999). The Wisconsin Card Sorting Test instroke rehabilitation: Factor structure and relation-ship to outcome. Archives of Clinical Neuropsy-chology, 14, 497±509.
Greve, K.W., Brooks, J., Crouch, J., Williams, M., &Rice, W. (1997). Factorial structure of the Wiscon-sin Card Sorting Test. British Journal of ClinicalPsychology, 36, 283±285.
Greve, K.W., Ingram, F., & Bianchini, K.J. (1998).Latent structure of the Wisconsin Card Sorting Testin a clinical sample. Archives of Clinical Neurop-sychology, 13, 597±609.
Greve, K.W., Love, J.M., Sherwin, E., Mathias, C.W.,Ramzinski, P., Levy, J., Chenault, J.E., & Quinta-nilla, V.A. (1999). Wisconsin Card Sorting Test inchronic traumatic brain injury. Archives of ClinicalNeuropsychology, 14, 731.
Haaland, K.Y., Vranes, L.F., Goodwin, J.S., & Garry,P.J. (1987). Wisconsin Card Sorting Test perfor-mance in a healthy elderly population. Journal ofGerontology, 42, 345±346.
Hart, R.P., Kwentus, J.A., Wade, J.B., & Taylor, J.R.(1988). Modi®ed Wisconsin Sorting Test in elderlynormal, depressed and demented patients. TheClinical Neuropsychologist, 2, 49±56.
Heaton, R.K. (1981). A manual for the Wisconsin CardSorting Test. Odessa, FL: Psychological AssessmentResources.
Heaton, R.K., Chelune, G.J., Talley, J.L., Kay, G.G., &Curtiss, G. (1993). Wisconsin Card Sorting Testmanual: Revised and expanded. Odessa, FL:Psychological Assessment Resources.
Heaton, R.K., & PAR Staff. (1999). Wisconsin CardSorting Test: Computer Version 3 for Windows:Research edition. Odessa, FL: PsychologicalAssessment Resources.
Heaton, R.K., & PAR Staff. (2000). WCST-64: Com-puter Version 3 for Windows-Research Edition.Odessa, FL: Psychological Assessment Resources.
Heaton, R.K., & Thompson, L.L. (1992). WisconsinCard Sorting Test: Is one deck as good as two?Journal of Clinical and Experimental Neuropsycho-logy, 14, 63.
Jenkins, R.L., & Parsons, O.A. (1978). Cognitivede®cits in male alcoholics as measured by amodi®ed Wisconsin Card Sorting Test. AlcoholTechnical Reports, 7, 76±83.
Kongs, S.K., Thompson, L.L., Iverson, G.L., & Heaton,R.K. (2000). WCST-64: Wisconsin Card SortingTest-64 Card Version Professional Manual. Odessa,FL: Psychological Assessment Resources.
Koren, D., Seidman, L.J., Harrison, R.H., Lyons, M.J.,Kremen, W.S., Caplan, B., Goldstein, J.M., Faraone,S.V., & Tsuang, M.T. (1998). Factor structure of theWisconsin Card Sorting Test: Dimensions of de®citin schizophrenia. Neuropsychology, 2, 289±302.
Lezak, M.D. (1995). Neuropsychological assessment(3rd ed.). New York: Oxford University Press.
Nelson, H.E. (1974). A modi®ed card sorting test sen-sitive to frontal lobe de®cits. Cortex, 12, 313±324.
Osmon, D.C., & Suchy, Y. (1996). Fractionating frontallobe functions: Factors of the Milwaukee CardSorting Test. Archives of Clinical Neuropsychology,11, 541±552.
Paolo, A.M., Axelrod, B.N., TroÈster, A.I., Blackwell,K.T., & Koller, W.C. (1996). Utility of a WisconsinCard Sorting Test short form in persons withAlzheimer's and Parkinson's disease. Journal ofClinical and Experimental Neuropsychology, 18,892±897.
REVIEW OF A STANDARDIZED SHORT-FORM OF THE WCST-64 233
Dow
nloa
ded
by [
Seou
l Nat
iona
l Uni
vers
ity]
at 1
8:44
01
Febr
uary
201
6
Paolo, A.M., TroÈster, A.I., Axelrod, B.N., & Koller,W.C. (1995). Construct validity of the WCST innormal elderly and persons with Parkinson's Disease.Archives of Clinical Neuropsychology, 10, 463±473.
Ramage, A., Bayles, K., Helm-Estabrooks, N., & Cruz,R. (1999). Frequency of perseveration in normalsubjects. Brain and Language, 66, 329±340.
Robinson, L.J., Kester, D.B., Saykin, A.J., Kaplan, E.F.,& Gur, R.C. (1991). Comparison of two short formsof the Wisconsin Card Sorting Test. Archives ofClinical Neuropsychology, 6, 27±33.
Sillanpaa, M.C., Vellet, N.S., Anderson, P., Butkus, M.,Voelker, S., & DeLuca, J.W. (1993). The WisconsinCard Sorting Test with children and adolescents:The long and short of it. Journal of Clinical andExperimental Neuropsychology, 15, 68.
Smith-Seemiller, L., Franzen, M.D., & Bowers, D.(1997). Use of Wisconsin Card Sorting Test shortforms in clinical samples. The Clinical Neuropsy-chologist, 11, 421±427.
Stanford, M.S., Greve, K.W., & Gerstle, J.E. (1997).Neuropsychological correlates of self-reportedimpulsive aggression in a college sample. Person-ality and Individual Differences, 23, 961±965.
Stuss, D.T., Levine, B., Alexander, M.P., Hong, J.,Palumbo, C., Hamer, L., Murphy, K.J., & Izukawa,D. (2000). Wisconsin Card Sorting Test perfor-mance in patients with focal frontal and posteriorbrain damage: Effects of lesion location and teststructure on separable cognitive processes. Neuro-psychologia, 38, 388±402.
Sullivan, E.V., Mathalon, D.H., Zipursky, R.B.,Kersteen-Tucker, Z., Knight, R.T., & Pfefferbaum,A. (1993). Factors of the Wisconsin Card SortingTest as measures of frontal-lobe function inschizophrenia and in chronic alcoholism. PsychiatryResearch, 46, 175±199.
van Gorp, W.G., Kaleschstein, A.D., Moore, L.H.,Hinkin, C.H., Mahler, M.E., Foti, D., & Mendez, M.(1997). A clinical comparison of two forms of theCard Sorting Test. The Clinical Neuropsychologist,11, 155±160.
Wiegner, S., & Donders, J. (1999). Performance on theWisconsin Card Sorting Test after traumatic braininjury. Assessment, 6, 179±187.
Zubicaray, G.I., & Ashton, R. (1996). Nelson's (1976)Modi®ed Card Sorting Test: A review. The ClinicalNeuropsychologist, 10, 245±254.
234 KEVIN W. GREVE
Dow
nloa
ded
by [
Seou
l Nat
iona
l Uni
vers
ity]
at 1
8:44
01
Febr
uary
201
6