Upload
vicente-caceres
View
218
Download
1
Tags:
Embed Size (px)
DESCRIPTION
V
Citation preview
PLEASE SCROLL DOWN FOR ARTICLE
This article was downloaded by:On: 17 November 2010Access details: Access Details: Free AccessPublisher Psychology PressInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
The Clinical NeuropsychologistPublication details, including instructions for authors and subscription information:http://www.informaworld.com/smpp/title~content=t713721659
Practice effects in repeated neuropsychological assessmentsRobert J. McCaffreya; Albert Ortegaa; Susan M. Orsilloa; Wendy B. Nellesa; Richard F. Haasea
a The University at Albany, State University of New York,
To cite this Article McCaffrey, Robert J. , Ortega, Albert , Orsillo, Susan M. , Nelles, Wendy B. and Haase, Richard F.(1992)'Practice effects in repeated neuropsychological assessments', The Clinical Neuropsychologist, 6: 1, 32 — 42To link to this Article: DOI: 10.1080/13854049208404115URL: http://dx.doi.org/10.1080/13854049208404115
Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf
This article may be used for research, teaching and private study purposes. Any substantial orsystematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply ordistribution in any form to anyone is expressly forbidden.
The publisher does not give any warranty express or implied or make any representation that the contentswill be complete or accurate or up to date. The accuracy of any instructions, formulae and drug dosesshould be independently verified with primary sources. The publisher shall not be liable for any loss,actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directlyor indirectly in connection with or arising out of the use of this material.
The Clinical Neuropsychologist 1992, Vol. 6, No. 1, pp. 32-42
092@-1637/92/0601-0032$3.00 0 Swets & Zeitlinger
CLINICAL ISSUES
Practice Effects in Repeated Neuropsychological Assessments
Robert J. McCaffrey, Albert Ortega, Susan M. Orsillo, Wendy B. Nelles, and Richard F. Haase The University at Albany, State University of New York
ABSTRACT
The magnitude of the practice effects associated with repeated administration of the same neurops ychological assessment instruments was examined. In two separate research protocols, subjects were administered their respective battery of neuropsychological instruments twice within 7 to 10 days prior to the initiation of any experimental manipulations. Factors of interest included the test-retest reliability correlation coef- ficients, the magnitude of practice effects, and the intercorrelation matrices among the instruments. In general, the test-retest reliabilities of the neuropsychological instruments were consistent with those found in the general psychological assessment literature. The magnitude of practice effects was greatest on the Logical and Figural Memory Subtests of the Wechsler Memory Scale in both protocols. The intercorrelation matrices may be useful in planning sample sizes for future studies since estimates of statistical power will require the consideration of the intercorrelations among groups of dependent variables.
Neuropsychologists working in either a clinical or research capacity may be called upon to perform serial neuropsychological assessments in order to moni- tor the progression of a disease process, to evaluate the efficacy of a therapeutic agent, or to assess the impact of a rehabilitation program. An important factor in any type of serial assessment is the variance attributable to practice effects. Repeated administration of the same assessment instrument may lead to incre- ments in a patient’s performance attributable to the effects of practice alone. In general, instruments that have a speeded component, require an infrequently
Preparation of the article was supported in part by National Institutes of Health Grants HL-35112 and NS-25006 to Robert J. McCaffrey. Reprint requests to: Robert J. McCaffrey, Department of Psychology, The University at Albany, State University of New York, 1400 Washington Avenue, Albany, NY 12222, USA.
Accepted for publication: April 2, 1991.
Downloaded At: 13:33 17 November 2010
PRACTICE EFFECTS 33
practiced response, or that have a single, easily conceptualized solution are likely to result in significant practice effects (Dodrill & Troupin , 1975). Among brain- injured patients, significant practice effects are reported to occur frequently, while large test to retest changes are reported not to be common among neurologically intact subjects (Lezak, 1982). The majority of the neuropsychological literature on practice effects in normals has focused on the subtests of the Wechsler Adult Intelligence Scale (Matarazzo, 1972; Matarazzo, Carmody, & Jacobs, 1980; Shatz, 1981). There is little normative information on practice effects associated with most neuropsychological instruments for the general population (Gill, Reddon, Stefanyk, & Hans, 1988; Maxwell & Niemann, 1984; Maxwell, Wise, Pepping, & Townes, 1984; Wilson, Wilson, Iacoviello, & Risucci, 1982) and very little for actual patient populations (desRosiers & Kavanagh, 1987).
The psychological assessment literature contains data on the test-retest reliability of virtually all of the instruments used to assess cognitive/behavioral performance, almost without exception, however, only reliability coefficients are reported (e.g., Brown, Rourke, & Cicchetti, 1989; Matarazzo, Wiens, Matarazzo, & Manaugh, 1973; Su & Yerxa, 1984). Psychometrically, this is useful information; however, i t is meaningless in terms of evaluating practice effects. For example, a reliabil- ity coefficient of .98 could be obtained if the subjects systematically made a mild, moderate, or substantial increase (or decrease) in performance at the retest compared to the initial test but maintained their same relative rank order on the two administrations of the instrument. Statistically, Shatz (1981) has suggested that the standard error of measurement be used to set up confidence intervals around an individual patient’s score in order to partial out practice effects from other factors related to improvement in patient’s performance across assessments (e.g., recovery of function following a CVA). Another technique involves the use of equated alternate forms of the same instruments. For many neuropsychological assessments, however, these are not available. In addition, Anastasi (1988) points out the gains may also occur at retesting using parallel forms (i.e., the test sophistication effect). In general, these gains are reported to be smaller than those obtained using the same form of an instrument.
Another alternative is to administer the entire neuropsychological battery twice, in order to assess for the degree of practice effects and to obtain a baseline level of performance. The second administration of the neuropsychological bat- tery is used as a baseline for comparison against subsequent assessments. The initial administration of the neuropsychological battery then serves as a meth- odological procedure to reduce the influence of practice effects.
The present study reports on the general issue of practice effects based on two separate ongoing research projects. The selection of the instruments contained in each of the neuropsychological batteries was based on specific research questions. Despite differences between the two subject populations and the neuropsychological batteries, important information regarding test-retest reliability, the degree of practice effects and the intercorrelations among the neuropsychological instruments is presented.
Downloaded At: 13:33 17 November 2010
34 ROBERT J. MCCAFFREY ET AL.
METHOD
Subjects The first project involved an examination of the neuropsychological and physical side- effects of the beta-adrenergic blocker, metoprolol, using a double blind placebo control- led crossover experimental design (McCaffrey, McCoy, Haase, Ortega, & Orsillo, 1990). Twenty-five newly diagnosed, untreated, mild, essential hypertensives (DBP = 90-105 mm Hg) volunteered as subjects. There were 17 males and 8 females with an average age of 50.1 years (SD = 14.0) and 14.1 (SD = 2.9) years of education. The second project was designed to evaluate the neuropsychological sequelae of prophylactic cranial irradiation therapy in patients with small cell lung cancer who presented with no evidence of CNS metastases (McCaffrey et al., 1990). As a control for the influence of pulmonary disease in the cancer patients, the control group was comprised of chronic cigarette smokers. Thirty-three chronic cigarette smokers with a mean consumption of 1.3 packs per day (SD = 0.5) and a mean smoking history of 38.6 years (SD = 12.2) were recruited from the community and paid $50 for their participation. In the smoker group, there were 15 males and 18 females with a mean age and educational level of 59.1 (SD = 9.3) and 14.9 (SD = 3.4) years, respectively.
Apparatus and Procedure For both groups, the subjects were administered their respective neuropsychological bat- tery twice, within 7 to 10 days. The same assessor was used to evaluate the same subject at both the test and retest assessments. This procedure was employed in order to reduce the confounding influence of practice effects.
The rationale for the selection of the neuropsychological instruments was determined by the requirements of the separate research questions. The instruments in each of the neuropsychological batteries are presented in Tables I and 2 and are described in Lezak (1983), except for the following tests. The Span of Attention Test (Kay, 1982) deals with the ability to sustain focused alertness on a task which makes minimal demands on higher cognitive processes. The test consists of a single 8 1/2 x 11 inch page filled with 500 “X”s arranged in 25 rows of 20 Xs, each separated by a dash (e.g., X-X-X ...). The instructions are to circle as many Xs as possible in 400 seconds. The Math and Reading tests were obtained from the University of the State of New York, Regents High School examination, Competency Tests. The versions used were from prior forms of the Regents Competency Test. The Math test involved 20 untimed arithmetic problems. The Reading test involves a passage that is read aloud by the teacher to the students followed by 10 multiple choice questions. The test was modified by having the subjects read the passage themselves and then answer the questions without the aid of the passage. The Static Motor Steadiness Test manufactured by Lafayette Instruments Company (#30211) consisted of having the subject hold a metal stylus in each of four holes of decreasing diameter (0.3 12,O. 187, and 0.125 in.) for separate 15-second trials. The score was the total number of times the stylus came in contact with the diameter of each hole. Simple Auditory Reaction Time was obtained using a Lafayette Instruments Company Choice Reaction Time Apparatus (#63035) and a ClocWCounter (#54035) set at .001 seconds. The reaction time was the average of 30 trials following 5 practice trials.
The neuropsychological instruments were administered and scored using their stand- ardized instructions, except for the following. In order to decrease the amount of uncon- trolled variance attributable to assessor differences in the administration and the scoring of the Logical Memory portions of the Wechsler Memory Scale (Wechsler, 1945). the paragraphs were tape recorded and the instructions and recorded paragraphs played to each subject. For research purposes, subjects were informed that there would be a delayed recall (Russell, 1975) of the Logical and Figural Memory Subtests of the Wechsler Memory Scale at each of the two assessments. The subject’s recall of the Logical Memory, imme-
Downloaded At: 13:33 17 November 2010
PRACTICE EFFECTS 35
diate and delayed, was tape recorded for subsequent verbatim transcription and scoring using Prigatano’s (1978) criteria. The Digit Span Subtest of the WAIS and the Paired- Associate Learning Subtest of the Wechsler Memory Scale were also presented via tape recordings. Since the responses to these two subtests are not as complex as those to the Logical Memory component, the responses were recorded on prepared assessment forms.
Table 1 .Test-retest reliability and practice effects associated with select neuropsychological instruments in essential hypertensives at the practice effects (PE) and baseline (BL) assessments.
Wechsler Memory Scale Logical Memory
Immediate Recall 25 .61 10.23 (3.14) Delayed Recall 25 .74 8.55 (2.65)
Figural Memory Immediate Recall 24 .63 1 1.17 (3.14) Delayed Recall 24 .74 10.08 (3.51)
Paired Associates 25 .53 10.88 (2.73)
Trail Making Test Part A 25 .80 35.75 (12.81) Part B 24 .92 91.60 (49.83)
Grooved Pegboard Test Preferred Hand 25 .85 83.32 (21.31) Nonpreferred Hand 24 .82 90.00 (20.57)
Finger Oscillation Test Preferred Hand 24 5 8 54.67 (11.55) Nonpreferred Hand 24 .82 50.35 (10.95)
Span of Attention 24 .78 320.15 (74.29)
WAIS-R Digit Span Forward 24 .78 8.44 (2.10) Backward 24 .85 7.44 (2.58)
Math Test 24 3 9 14.08 (4.31)
Reading Test 25 .87 5.12 (2.91)
12.36 (2.75) 4 . 0 3 (C.005) 11.34 (3.39) -6.12 (<.0005)
12.17 (2.18) -2.00 (c.05) 11.70 (2.37) -3.3 (<.005)
12.21 (2.39) -2.7 (C.01)
33.86 (14.98) 1.05 (c.152) 87.08 (53.03) 1.31 (<.lo)
78.91 (26.92) 1.56 (c.06) 81.59 (15.79) 3.47 (<.005)
55.94 (10.50) 4 . 6 4 (<.263) 49.78 (10.22) 0.44 (<.332)
314.53 (70.12) 0.57 (<.286)
8.70 (2.69) -0.85 (<.202) 8.13 (3.06) -1.80 (<.042)
15.17 (4.05) -2.10 (<.025)
5.56 (3.11) -1.41 (<.085)
Downloaded At: 13:33 17 November 2010
36 ROBERT I. MCCAFFREY ET AL.
RESULTS
The test-retest reliability ( rJ and the statistical magnitude of the practice effects are presented in Tables 1 and 2 for the qssential hypertensive and chronic smoker groups, respectively. The test-retest reliability ( rJ of each of the neuropsychological instruments was evaluated by calculating Pearson-Product Moment correlations. The correlations ranged from 0.53 to 0.92 among the sample of essential hypertensives and 0.47 to 0.89 for the chronic smokers.
The magnitude of practice effects were examined by computing one-tailed, dependent group, t tests for each of the instruments. The use of the t tests was to document the magnitude of the practice effects. The p values presented in Table 1 and 2 indicate the magnitude of the obtained practice effects.
Table 2.Test-retest reliability and practice effects associated with select neuropsychological instruments in chronic smokers at the practice effects (PE) and baseline (BL) assessments.
Chronic Smokers
Wechsler Memory Scale Logical Memory Immediate Recall 33 .47 9.68 (3.38) 11.12 (3.26)-2.42 (C.025) Delayed Recall 32 .68 8.09 (2.93) 10.55 (2.98)-5.93 (<.0005)
Figural Memory Immediate Recall 32 .53 9.70 (2.94) 11.19 (2.87)-2.98 (<.001) Delayed Recall 32 .69 8.94 (3.50) 11.03 (2.95)-4.43 (<.0005)
Trail Making Test Part B 32 .49 88.95 (24.52) 74.77 (23.21) 3.27 (c.005)
Grooved Pegboard Test Preferred Hand 32 3 0 82.03 (14.87) 79.66 (14.70) 1.58 (<.06) Nonpreferred Hand 32 .69 88.00 (24.07) 83.44 (15.99) 1.68 (<.05)
Symbol Digit Modulities Test 33 .70 46.88 (9.40) 47.88 (10.96) -.72 (<.24)
Motor Steadiness Test Preferred Hand 33 .59 8.55 (6.63) 7.94 (6.90) .59 (<.28) Nonpreferred Hand 33 3 9 10.71 (9.31) 10.30 (9.68) .54 (<.30)
Speech Sounds Perception Test 33 .83 54.76 (3.03) 55.06 (2.84) -1.01 (c.16)
Seashore Rhythm Test 33 .64 26.42 (3.14) 26.24 (3.25) .39 (<.35)
Simple Reaction Time 33 .62 .37 (.08) .35 (.08) 2.23 (<.025)
Downloaded At: 13:33 17 November 2010
Tabl
e 3.
Inte
rcor
rela
tiona
l m
atrix
of t
he n
euro
psyc
holo
gica
l in
stru
men
ts u
sed
in th
e es
sent
ial h
yper
tens
ive
popu
latio
n at
the
prac
tice
effe
cts/
ba
selin
e as
sess
men
ts.
Inst
rum
ents
~ ~
~ ~
~
LM
I L
MD
FM
I FM
D
PA
TMTA
TM
TB
GPP
G
PN
FOT
P FO
TK
SOA
D
SF
DSB
M
T R
T
Wec
hsle
r M
emor
y Sc
ale
Log
ical
Mem
ory
Imm
edia
te R
ecal
l (L
MI)
-
Del
ayed
Rec
all
(LM
D)
,821
.81
-
Imm
edia
te R
ecal
l (F
MI)
.2
7/ .I
3 .1
8/ 2
5
- D
elay
ed R
ecal
l (F
MD
) .3
5/-.0
6 .1
7/ .I
1 ,7
51.8
4 -
Figu
ral
Mem
ory
Paire
d A
ssac
iatc
r (P
A)
.39/
.11
.24/
.17
.27/
.32
.l8/
.24
-
Tra
il M
akin
g Te
st
Part
A
(TM
TA
) -.1
1/-.0
3 .0
9/ .0
7 -.4
8/-.2
4 -3
-.2
3
-.09/
-.03
- Pa
rt B
(T
MT
B)
-.31/
-.15
-.09/
-.I2
-.47/
-23
-.74/
-.24
-.06/
-.21
.77/
30
-
Cro
ovcd
Peg
booi
d T
est
Pref
erre
d H
and
(GPP
) -.0
6/-.0
9 .1
5/-.I
7 -.0
3/-.I
6 -.3
2/-.0
3 -.2
6/-.O
O
.33/
.47
.55/
51
N
onpr
efm
ed Hand
(GPN
) -.G
9/-.2
5 .0
8/-.2
0 -.2
8/-.3
5 -.5
0/-.3
1 -.2
2/-.I
9 ,4
81.6
4 ,6
51 .6
3
Fing
er O
scill
aiio
n Te
st
- 91
/ .57
-
Pref
errc
dHan
d (F
OT
P)
.24/
32
.0
5/ .
I6 -.OW S
O
,041
.40
.21/
.22
-.20/
-.26
-.14/
-.21
-.35/
-.I6
-.37/
-36
-
Non
prcf
ared
Han
d (F
OT
K)
.24/
.34
.O I/
.I2
,041
.44
.16/
.27
.25/
.27
-.28/
-.32
-. 15
/-.2
1 -.3
4/-.2
3 -.3
4/-.3
5 ,7
61.8
4 -
(TM
TA)
$ 2
(TM
TB)
(FO
TP)
(FO
TN)
Span
ofA
tlenr
ion
(SO
A)
,071
.04
,121
.I7
-331
-.28-
.52/
-.13
.17/
.13
.52/
.41
.52/
34
.4
0/ .2
5 .5
6/ .4
9 -.4
3/-.4
1 -.2
51-3
6 -
(SO
N
WA
IS-R
Dig
it Sp
an
Forw
ard
(DSF
) .2
2/ .I
6 .0
5/ .3
2 .3
7/ .2
8 39
1.3
1 .0
8/ .
19 -.
SO/-.
59
-.55/
-.59
-. 4 w
.29
-.56/
-.45
.28/
30
.1
5/ 2
8 -
.38/
-.22
-
Bac
kwar
d (D
SB)
.26/
.16
.10/
.25
.24/
39
,2
91 3
0
,271
.21
-3-.
53
-.5
3/-.5
7 -.4
1/-.3
4 -.4
8/-3
6 ,4
71.2
4 ,5
01.4
5 -.2
1/-.2
4 ,6
91.6
9 -
Mat
h T
csl
(MT
) .2
0/ .LO
-.08/
2
3
.62/
.53
.49/
.52
.42/
.a -.6
1/-3
6 -.5
6/-.5
5 -.3
7/-2
8 -.4
7/-.4
2 ,1
71 3
4
,381
.29
-.20/
-.05
.55/
.54
,571
.51
- (M
T)
Ren
ding
Tcs
r (R
T)
,041
.04
.07/
.22
.181
.05
.33/
.12 -.09/.04 -.
Lo/
-.M
-.41/
-.44
-.29/
-.35
-.36
/-si
-.2
11-.1
7 -.
ia/-
.u
-.w/-.
o4
.i7/
.07
-.06/
-.07
.121
.18
- (R
T)
Not
e: T
he fi
rst c
orrc
latio
n is
the
prac
tice
effe
cts
corr
elat
ion
for borh
inst
rum
ents
and
lhe
second
Cor
rela
tion
IS rh
e co
rrel
atio
n fo
r bo
th in
stru
men
ts a
t bas
elin
e.
w
4
Downloaded At: 13:33 17 November 2010
Tabl
e 4.
Int
erco
rrel
atio
n m
atrix
of t
he n
euro
psyc
holo
gica
l ins
trum
ents
use
d in
the
chro
nic s
mok
er p
opul
atio
n at
the
prac
tice
effe
ctsh
asel
ine
asse
ssm
ents
.
Inst
rum
ents
~
LMI
LMD
FM
I FM
D
TM
TB
G
PP
GPN
SD
MT
MSP
M
SN
SSPT
SR
T R
T
Wec
hler
Mem
ory
Scal
e Lo
gica
l Mem
ory
Imm
edia
te R
ecal
l &
MI)
D
elay
ed R
ecal
l (L
MD
) .9
4/.6
5 -
-
Figu
ral M
emor
y Im
med
iate
Rec
all
(FM
I)
,551
.24
.58/
.31
-
Del
ayed
Rec
all
(FM
D)
,441
.I7
.52/
.22
.84/
.74
-
Trai
l Mak
ing
Test
Pa
rt B
(TM
TB)
-.33/
.06
-.4
5/-.2
9 -.3
2/-.4
1 -.4
1/-.4
3 -
Gro
oved
Peg
boar
d Te
st
Pref
erre
d H
and
(GPP
) -.3
6/-.2
3 -.4
4/-.3
2 -.3
6/-.3
2 -.4
0/-.4
0 ,3
41.3
2 -
Non
pref
emed
Han
d (G
PN)
-.33/
-.20
-.38/
-.34
-SO
/-.39
-.4
51-.4
3 .2
7/ .3
6 .7
3/ .6
9 -
Sym
bol D
igit
Mod
alif
ies T
est
(SD
MT)
-.34
/-.03
,3
31.1
3 .3
2/ .09
,341
.32
-.31/
-33
-.44/
-.41
-.36/
-.56
-
Mot
or S
tead
ines
s Te
st
Ref
erre
d H
and
(MSP
) -.
09/.2
9 -.0
4/.2
9 -.3
1/.0
9 -.3
41.1
0 -.o
O/-.
oO
,171
-.I4
,171
.14
-.27/
-.33
- N
onpr
efem
ed H
and
(MSN
) .0
7/ .2
6 .1
2/ .2
9 -.
lo/
.02
-.04/
.02
-.2
0/-.0
6 .0
4/ .0
2 -.0
2/ .
I5 -
.18/
-.25
.71/
.79
-
Spee
ch S
ound
s Pe
rcep
tion
Test
(S
SPT)
,2
61.2
5 ,3
71.4
8 .1
9/-.0
2 .2
9/ .1
8 -.4
4/-.2
9 -.3
6/-.3
3 -.1
3/-.4
0 -.2
4/ .
53 -.MI .01
.14/
.03
-
..
!a
(TM
TB)
Seas
hore
Rhy
thm
Tes
t (S
RT)
.3
3/ .1
7 ,3
51.4
5 ,5
21.1
3 .5
8/ .2
5 -.2
1/-.I
8
-.38/
-.12
-.23/
-.37
.46/
.32
-.40/
-.03
-.13/
.06
.2
3/ .3
8 -
(SR
T)
Sim
ple
Rea
ctio
n Ti
me
(RT)
.0
5/-.I
8 ,1
21.0
3 .2
8/-.0
9 -.2
8/-.2
4 -.2
8/-.1
8 -.O
W
.34
.09/ .0
5 -.0
2/-.1
9 -.2
1/ .
03 -.
091
.29 -.09/ .
04
-.30/
.09
- (
RT)
Not
e: T
he fi
rst c
omel
atio
n is
the p
ract
ice
effe
cts c
orre
latio
n fo
r bot
h in
stru
men
ts an
d th
e se
cond
corr
elat
ion
is th
e co
rrel
atio
n fo
r bot
h in
stru
men
ts a
t bas
elin
e.
Downloaded At: 13:33 17 November 2010
PRACTICE EFFECTS 39
The intercorrelation matrices for each of the neuropsychological batteries are presented in Tables 3 and 4 rounded to two decimal places.
In addition to providing useful information about the intercorrelations among the neuropsychological variables, these intercorrelations may prove useful to investigators who anticipate their use in future research as a multivariate set of
Table 5. Test-retest reliability of select neuropsychological instruments from the clinical literature.
Instrument Subjects r,, Test-retest Interval Source
Wechsler Memory Scale Logical Memory
Immediate Recall
Delayed Recall
Figural Memory Immediate Recall
Delayed Recall
Paired Associates
Halstead-Reitan Battery
Trail Making Test Part A
Part R
Finger Oscillation Test
5 1 elderly males .93 26 elderly females .77 30 normals and .a8 75 brain-damaged subjects
51 elderly males .90 26 elderly females .67 30 normals and .51 75 brain-damaged
subjects
51 elderly males .90 26 elderly females .93
29 normal, young males .46 16 elderly patients with .78
diffuse cerebrovascular disease
29 normal. young males .44 16 elderly patients with .67
diffuse cerebrovascular disease
2 4
Speech Sounds Perception Test 29 normal, young males .49
16 elderly patients with .67 diffuse cerebrovascular disease
Seashore Rhythm Test 29 normal, young males .37 16 elderly patienu with .58
diffuse cerebrovascular disease
W N S - R Digif Span 40-80 normals .83
Symbol Digit Modalities Test 80 adults
1 day 1 day reliability determined via Russell (1975) internal consistency method
Meer & Raker (1967)
1 day 1 day reliability determined via Russell (1975) internal consistency method
Mccr & Raker (1975)
1 day 1 day
20 weeks 12.4 weeks
20 weeks 12.4 weeks
20 weeks
20 weeks 12.4 weeks
20 weeks 12.4 weeks
Meer & Baker (1967)
Matarazm et al. (1974)
Matarazzn et al. (1974)
Matarazm et al. (1974)
Matarazm et al. (1974)
Matarazm et al. (1974)
1 to 7 weeks Wechsler (198 1)
4 weeks Smith (1982)
Downloaded At: 13:33 17 November 2010
40 ROBERT 1. MCCAFFREY ET AL.
dependent variables, and who wish to conduct prospective, multivariate power analysis (Cohen, 1988). An estimate of the matrix of intercorrelations among the dependent variables is required for these power computations and the data pro- vided here provide a useful empirical estimate.
Other investigators have examined the test-retest reliability coefficients of several of the neuropsychological assessment instruments included in the present report. These data are summarized in Table 5. As shown in Table 5, the charac- teristics of the samples and the test-retest interval varied considerably among the studies.
DISCUSSION
The values of the test-retest reliability correlation coefficients for the neuro- psychological instrument obtained with the essential hypertensive and chronic smoker samples are consistent with those reported in the general psychological assessment literature (Anastasi, 1986). In both samples, the magnitude of practice effects, as indexed by one-tailed t tests, were greatest for the Logical and Figural Memory subtests of the Wechsler Memory Scale. Among the essential hypertensives considerable practice effects were also obtained on the Paired-Associate Subtest of the Wechsler Memory Scale, the Grooved Pegboard Test-nonpreferred hand, the WAIS-R Digit Span-Backwards and the Math Test. Practice effects for the chronic smokers were observed on the Trail Making Test - Part B, the Grooved Pegboard Test-nonpreferred hand, and the Simple Auditory Reaction Time Test. In both samples, the magnitude of practice effects among the remaining instruments was minimal.
The magnitude of the test-retest correlation coefficients among the chronic smokers were generally lower than those obtained on the same instruments in the sample of essential hypertensives. There are at least three factors which may have contributed to these discrepant findings despite comparable methodologies. First, the chronic smokers were paid volunteers whose primary motivation was financial while the essential hypertensives were unpaid subjects whose motivation focused on knowing the potential impact of their medication on cognitive func- tioning. The second factor is that the chronic smokers’ pulmonary functioning may have been compromised relative to the essential hypertensives. Third, the chronic smokers were significantly older ( t (60) = 9.27, p < .Ol) than the essential hypertensives. Thus, the differences obtained between the two samples may have been due to any one or a combination of these factors.
The test-retest reliability measures of both research samples and those re- ported from various normative data sources in the neuropsychological literature vary considerably reflecting different populations, sampling procedures and test- retest intervals. As such, clinical neuropsychologists must take these factors into consideration when attempting to interpret the data from individual patients.
The primary goal of this report was to acknowledge the differences in test-
Downloaded At: 13:33 17 November 2010
PRACTICE EFFECTS 41
retest reliability across different populations, evaluate the extent of practice ef- fects, and report on the intercorrelations matr ices of a select g roup o f neuropsychological instruments. The latter goal was t o provide data that would b e useful t o investigators planning multivariate studies. Clearly, clinical neuropsychologists need t o expand to the existing data base on the factors noted above. In the interim, both the scientist and practitioner should exercise caution when attempting t o interpret the significance o f practice effects and test-retest reliability coefficients i n repeated neuropsychological assessments.
REFERENCES
Anastasi, A. (1988). Psychological testing (6th ed.). New York: Macmillan. Brown, S.J., Rourke, B.P., & Cicchetti, D.V. (1989). Reliabilities of tests and measures
used in the neuropsychological assessment of children. The Clinical Neuro- psychologist, 3, 35 3 -3 68.
desRosiers, G., & Kavanagh, D. (1987). Cognitive assessment in closed head injury: Stability, validity and parallel forms for two neuropsychological measures of recovery. The International Journal of Clinical Neuropsychology, 9, 162-1 73.
Dodrill, C.B., & Troupin, A.S. (1 975). Effects of repeated administration of a comprehensive neuropsychological battery among chronic epileptics. Journal of Nervous and Mental Disease, 161, 185-190.
Gill, D.M., Reddon, J.R., Stefanyk, W.O., & Hans, S.H. (1 986). Finger tapping: Effects of trials and sessions. Perceptual and Motor Skills, 62,675-678.
Kay, S.R. (1 982). The cognitive diagnostic battery. Odessa, FL: Psychological Assessment Resources.
Lezak, M.D. (1982, June). The test-retest stability and retiability of some resrs commonly used in neuropsychological assessment. Paper presented at the fifth European conference of the International Neuropsychological Society, Deauville, France.
Lezak, M.D. (1983). Neuropsychological assessment (2nd ed.). New York: Oxford. Matarazzo, J.D. (1972). Wechsler’s measurement and appraisal of adult intelligence. New
York: Oxford University Press. Matarazzo, J.D., Wiens, A.N., Matarazzo, R.G., & Goldstein, S.G. (1974). Psychometric
and clinical test-retest reliability of the Halstead Impairment Index in a sample of healthy, young, normal men. Journal of Nervous andMental Disease, 158.37-49.
Matarazzo, J.D., Carmody, T.P., & Jacobs, L.D. (1980). Test-retest reliability and stabil- ity of the WAIS: A literature review of implications for clinical practice. Journal of Clinical Neuropsychology, 2,89-105.
Matarazzo, R.G., Wiens, A.N., Matarazzo, J.D., & Manaugh, T.S. (1973). Test-retest reliability of the WAIS in a normal population. Journal of Clinical Psychology, 29, 194-197.
Maxwell, J.K., & Niemann, H. (1984). The Finger-Tip Numberwriting Test: Practice effects versus lateral asymmetry. Perceptual and Motor Skills, 59, 343-351.
Maxwell, J.K., Wise, F., Pepping, M., & Townes, B.D. (1984). Fingertip number-writing errors by psychiatric patients. Perceptual and Motor Skills, 59, 933-934.
McCaffrey. R.J., McCoy, G.C., Haase, R.F., Ortega, A., & Orsillo, S.M. (1990, Novem- ber). Neuropsychological side effects of metoprolol. Presented at the Annual Meeting of the National Academy of Neuropsychology, Reno, NV.
McCaffrey, R.J., Orsillo, S.M., Lefkowicz, D.P., Ortega, A., Haase, R.F., Wagner, H., & Ruckdeschel, J.C. (1990, November). Neuropsychological sequelae of chemo- therapy and prophylactic cranial irradiation: An extension of earlier findings.
Downloaded At: 13:33 17 November 2010
42 ROBERT J. MCCAFFREY ET AL.
Presented at the Annual Meeting of the National Academy of Neuropsychology, Reno, Nevada.
Meer, B., & Baker, J.A. (1967). Reliability of measurements of intellectual functioning of geriatric patients. Journal of Gerontology, 20, 410-41 4.
Prigatano, G.P. (1978). Wechsler memory scale: A selective review of the literature. Journal of Clinical Psychology, 34 , 816-832.
Russell, E.W. (1975). A multiple scoring method for the assessment of complex memory functions. Journal of Consulting and Clinical Psychology, 43,800-809.
Shatz, M.W. (1981). WAIS practice effects in clinical neuropsychology. Journal of Clinical Neuropsychology, 3, 171-179.
Smith, A. (1982). Symbol Digit Modalities Test: Manual. Los Angeles: Western Psycho- logical Services.
Su, R., & Yerxa, E.J. (1984). Comparison of the motor test of the SCSIT and the L- NNBC. The Occupational Therapy Journal of Research, 4,96108.
Wechsler, D. (1945). A standardized memory scale for clinical use. Journal of Psychol- ogy, 19, 87-95.
Wechsler, D. (1981). WAIS-R Manual. New York: Psychological Corporation. Wilson, B.C., Wilson, J.J., Iacovielli, J.M., Risucci, D. (1982). Purdue Pegboard performance
of normal preschool children. Journul of Clinical Neuropsycholugy, 4, 19-26.
Downloaded At: 13:33 17 November 2010