Choosing Assessment Instruments Counseling and Development

Measurement and Evaluation inCounseling and Development 1 –12© The Author(s) 2015 Reprints and permissions: sagepub.com/journalsPermissions.navDOI: 10.1177/0748175615578755mec.sagepub.com

Article

Anxiety disorders are among the most fre-quently reported psychiatric disorders in chil-dren and adolescents (American Psychiatric Association, 2013). Of the subtypes delin-eated in the DSM-5, generalized anxiety dis-order (GAD), separation anxiety disorder (SAD), social phobia (SOP), and specific phobia (SP) were particularly common among school-aged youth (American Psychiatric Association, 2013; Beidel & Turner, 2005; Silverman & Kurtines, 2001). If not treated, anxiety disorders may persist throughout childhood and adolescence resulting in emo-tional, social, and academic impairment well into adulthood (Ramirez, Feeney-Kettler, Flores-Torres, Kratochwill, & Morris, 2006).

For research and clinical purposes, a multi-method, multi-informant approach to assess-ment is recommended (Erford, 2013). Integrating reports from various informants (e.g., students, parents, teachers) may be par-ticularly important when assessing anxiety among children and adolescents, as studies revealed low correlations between parent and youth reports of anxiety symptoms (Southam-Gerow, Flannery-Schroeder, & Kendall, 2003). Lower rater agreement means cross-informant validation may reveal important symptoms

missed when relying on the report of a single respondent. This practice is particularly important for the assessment of anxiety because, in general, lower parent–youth agreement was reported for internalizing behaviors than for more readily observed externalizing behaviors (Achenbach & Rescorla, 2001). Discrepancies in reporting, coupled with discordance regarding which source is more accurate, support the practice of using multiple informants across various settings, rather than relying exclusively on one informant or a single instrument.

A variety of instruments are available for assessing anxiety in children and adolescents both for clinical purposes (e.g., screening, diagnosis, treatment planning) and to deter-mine treatment efficacy (e.g., results of a treat-ment plan, outcome evaluation). These instruments take the form of clinician-initiated interviews, self-report, and other-report (e.g.,

578755 MECXXX10.1177/0748175615578755Measurement and Evaluation inCounseling and DevelopmentErford and Lutzresearch-article2015

1Loyola University Maryland, Timonium, MD, USA

Corresponding Author:Bradley T. Erford, Loyola University Maryland, Timonium Graduate Center, 2034 Greenspring Drive, Timonium, MD 21093, USA. Email: [email protected]

Choosing Assessment Instruments for Anxiety Practice and Outcome Research With School-Aged Youth

Bradley T. Erford1 and Julie A. Lutz1

AbstractUsing effect size results from our meta-analysis for the treatment of anxiety in school-aged youth, the practical and technical aspects of five commonly used anxiety instruments were analyzed, and effect size estimates compared to indicate the best choices for use in anxiety outcome research.

Keywordsanxiety, children, adolescents, counseling, outcome research

at PENNSYLVANIA STATE UNIV on September 15, 2016mec.sagepub.comDownloaded from

mailto:[email protected]

http://mec.sagepub.com/

2 Measurement and Evaluation in Counseling and Development

parent, teacher) instruments. Some omnibus instruments provide a combination of these response formats. For example, the Achenbach System of Empirically Based Assessment (ASEBA; Achenbach & Rescorla, 2001) pro-vides self-, parent-, and teacher-report ver-sions and the State-Trait Anxiety Inventory for Children (STAIC; Spielberger, Gorsuch, Lush-ene, Vagg, & Jacobs, 1973; Strauss, 1987) pro-vides self- and parent-report versions.

A recent meta-analysis (Erford, Kress, Gugiere, Cieri, & Erford, 2015) exploring the effectiveness of counseling as a treatment for anxiety in school-aged youth revealed 14 instruments were used more than once in the 54 selected clinical trials to measure anxiety (see Table 1). Of these 14 anxiety measures, we examined the utility and psychometric characteristics of the five most frequently used instruments to determine usefulness and best choices for future outcome research: (a) the Revised Children’s Manifest Anxiety Scale, Second edition (RCMAS-2; Reynolds & Richmond, 2008), a self-report inventory used as an outcome measure in 24 of the 54 clinical trials (44%); (b) the Child Behavior Checklist (CBCL; Achenbach & Rescorla, 2001), a parent-report instrument used in 21 studies (39%); (c) the STAIC, a self-report (State Trait Anxiety Inventory for Children–Child report [STAIC-C]; Spielberger et al., 1973), and a parent-report (State Trait Anxiety Inventory for Children–Parent form [STAIC-P]; Strauss, 1987) instrument used in 13 stud-ies (24%); (d) the Fear Survey Schedule for Children–Second edition (FSSC-II; Burnham & Gullone, 1997; Gullone & King, 1992), a self-report instrument used in 13 (24%) stud-ies; and (e) the Anxiety Disorders Interview Schedule for DSM-IV–Child version (ADIS-C; Silverman & Albano, 1996), a clinician administered interview used as an outcome measure in 12 (22%) of the studies. We also compared the effect size estimates generated by the Erford et al.’s (2015) meta-analysis to display what researchers and counselors can reasonably expect when they choose certain instruments to measure outcomes in clinical trials and practice. While these effect size comparisons are embedded in the text and discussed within the appropriate context,

Table 1. Outcome Measures Reported in the Erford et al. (2015) Anxiety Meta-Analysis.

Instrument Name and Author(s)

# (% of 54) Studies

Used

aRCMAS-2—Revised Children’s Manifest Anxiety Scale (Second edition; Reynolds & Richmond, 2008)

24 (44)

aCBCL—Child Behavior Checklist (Achenbach & Rescorla, 2001)

21 (39)

aSTAIC—State Trait Anxiety Inventory for Children (Spielberger et al., 1973) or Parents (STAIC-P; Strauss, 1987)

13 (24)

aFSSC-II—Fear Survey Schedule for Children (Second edition; Gullone & King, 1992)

13 (24)

aADIS-IV—Anxiety Disorders Interview Schedule for DSM-IV (Silverman & Albano, 1996)

12 (22)

SPAI-C—Social Phobia Anxiety Inventory for Children (Beidel, Turner, & Morris, 1995)

7 (13)

SAS-C-R—Social Anxiety Scale for Children–Revised (or Adolescence or Parent; LaGreca & Stone, 1993)

6 (11)

SCAS—Spence Children’s Anxiety Scale (Spence, 1998)

5 (9)

MASC—Multidimensional Anxiety Scale for Children (March, Parker, Sullivan, Stallings, & Conners, 1997)

4 (7)

SCARED-R—Screen for Child Anxiety Related Emotional Disorders–Revised (Birmaher et al., 1997)

4 (7)

K-SADS-E—Schedule for Affective Disorder and Schizophrenia for School-Age Children–Epidemiologic version (Orvaschel, 1994)

2 (4)

BAI—Beck Anxiety Inventory (Beck, 1993)

2 (4)

HAM-A—Hamilton Anxiety Rating Scale (Hamilton, 1959)

2 (4)

CY-BOCS—Children’s Yale-Brown Obsessive-Compulsive Scale (Goodman et al., 1989)

2 (4)

Note. Percentage is out of the 54 studies selected in the Erford et al. (2015) meta-analysis. Some studies used more than one instrument as an outcome measure.a. Indicates an instrument reviewed in this article.



Erford and Lutz 3

Table 2 provides a summary of all effect size comparisons in the approximate order pre-sented. We then used this aggregated informa-tion to offer suggestions for best choices of different respondent and age range selection of anxiety assessment instruments to use with school-aged youth in outcome research.

The Revised Children’s Manifest Anxiety Scale–Second Edition

The RCMAS-2 (Reynolds & Richmond, 2008) contains 49 items with a dichotomous

response format requiring a response of yes or no to indicate whether the statement describes the individual’s actions or feelings. The RCMAS-2, a Level B instrument, was designed to measure anxiety in children and adolescents aged 6 to 19 years and yields a score for Total Anxiety and subscale scores for the Physiological Anxiety, Worry, Social Anxi-ety, and Defensiveness subscales. The Defen-siveness scale was designed to identify social desirability and response defensiveness. Raw scores were obtained by summing affirmative responses and were converted to T-scores and percentiles. The RCMAS-2 includes a 10-item

Table 2. Effect Size Posttest Comparisons Among Instruments Used to Assess Anxiety Outcomes in the Erford et al. (2015) Meta-Analysis.

K Measure 1 d Measure 2 d N Statistical Result

21 RCMAS 0.53 CBCL-M-I 0.76 750 RCMAS < CBCL-M-I1 RCMAS 0.59 CBCL-M-A 0.60 94 RCMAS = CBCL-M-A11 RCMAS 0.61 CBCL-F-I 0.60 383 RCMAS = CBCL-F-I12 RCMAS 0.42 STAIC-C 0.47 316 RCMAS = STAIC-C6 RCMAS 0.44 ADIS-IV-C 1.11 136 RCMAS < ADIS-IV-C14 RCMAS 0.47 FSSC 0.39 573 RCMAS = FSSR-C2 RCMAS 1.06 SAS-C/A 0.97 34 RCMAS = SAS-C/A9 CBCL-M-I 0.67 STAIC-C 0.36 327 CBCL-M-I > STAIC-C1 CBCL-M-A 0.60 STAIC-C 0.57 94 CBCL-M-A = STAIC-C3 CBCL-F-I 0.66 STAIC-C 0.55 143 CBCL-F-I = STAIC-C4 CBCL-M-I 0.85 STAIC-M 0.60 190 CBCL-M-I > STAIC-M3 CBCL-M-I 0.72 STAIC-F 0.94 143 CBCL-M-I = STAIC-F3 CBCL-M-I 0.66 STAIC-M 0.66 143 CBCL-F-I = STAIC-M3 CBCL-F-I 0.66 STAIC-F 0.94 143 CBCL-F-I > STAIC-F4 CBCL-M-I 0.76 ADIS-IV-P 1.42 171 CBCL-M-I < ADIS2 CBCL-M-I 0.66 SPAI-C 0.72 99 CBCL-M-I = SPAI-C2 CBCL-M-A 0.03 MASC-P 0.12 157 CBCL-M-A = MASC-P12 CBCL-M-I 0.78 CBCL-F-I 0.56 396 CBCL-M-I > CBCL-I-F2 STAIC-C 0.39 ADIS-IV-C 1.28 89 STAIC-C < ADIS-IV-C1 STAIC-C 0.00 FSSC-P 0.23 12 STAIC-C = FSSC-P6 STAIC-C 0.40 FSSC 0.34 349 STAIC-C = FSSC3 STAIC-M 0.45 FSSC 0.45 209 STAIC-M = FSSC1 STAIC-F 0.50 FSSC 0.45 106 STAIC-F = FSSC-C2 STAIC-C 1.40 SAS-C 0.97 34 STAIC-C = SAS-C7 ADIS-IV-C 1.32 SPAI-C 0.90 190 ADIS-C > SPAI-C4 ADIS-IV-C 1.07 SAS-C 0.74 56 ADIS-C > SAS-C5 SPAI-C 0.74 SAS-C 0.48 112 SPAI-C > SAS-C2 SAS-C 0.14 MASC 0.11 78 SAS-C = MASC

Note. RCMAS = Revised Children’s Manifest Anxiety Scale (and second edition); CBCL = Child Behavior Checklist (-M = Mother, -F = Father, -I = Internalizing Scale, -A = Anxious/Depression subscale); STAIC = State Trait Anxiety Inventory for Children (-C = Child, -M = Mother, -F = Father); ADIS-IV = Anxiety Disorders Interview Schedule for DSM-IV (-C = Child version, -P = Parent version); FSSC = Fear Survey Schedule for Children (-R = Revised, -II = Second edition); SPAI-C = Social Phobia Anxiety Inventory for Children; SAS = Social Anxiety Scale (-C = Children, -A = Adolescence); MASC = Multidimensional Anxiety Scale for Children (-P = Parent version).




Short Form, a cluster of items designed to measure performance anxiety, and an index for Inconsistent Responding. Norms were updated and based on an ethnically diverse standardization sample, which was stratified into three age groups: 6 to 8 years, 9 to 14 years, and 15 to 19 years. The manual pro-vides adjusted T-scores for sex and ethnicity (i.e., Asian/Pacific Islander, Black/African American, and Hispanic/Latino). The RCMAS-2 costs about $1.80 per administra-tion and the examiner kit costs about $100.00.

An internal consistency estimate of α = .92 (n = 3,086) was reported for the Total Anxiety score of the RCMAS-2. Reliability coeffi-cients were consistent across ethnicity, sex, and age. For the RCMAS-2 subscales, reli-ability estimates were α = .75 for Physiologi-cal Anxiety, .86 for Worry, .80 for Social Anxiety, and .79 for Defensiveness. A reliabil-ity estimate of α = .82 was reported for the Short Form Total Anxiety score. Test–retest reliability estimates for the Total Anxiety and Short Form Total Anxiety scores across 1 week were r

tt = .76 (n = 100) and .54 (n =

100), respectively. As a result, researchers should use the more reliable Total Anxiety score in outcome studies, rather than subscale scores or the short form.

Because of the recent publication of the RCMAS-2 (Reynolds & Richmond, 2008), most outcome studies to date have used the previous version, the RCMAS (Reynolds & Richmond, 1985; see Table 2). However, because the intercorrelation between the Total Anxiety scores from the RCMAS and RCMAS-2 exceeded r = .90 (Reynolds & Richmond, 2008), the results from the RCMAS analysis probably have strong gen-eralizability to use of the RCMAS-2. Scale comparison analyses from effect sizes gen-erated in Erford et al. (2015) revealed the mean effect size of the RCMAS was not sig-nificantly different than the mean effect sizes of the STAIC-C (dRCMAS = 0.42, dSTAIC-

C = 0.47, k = 12, n = 316), FSSC (dRCMAS = 0.47, dFSSC

= 0.39, k = 14, n = 573), CBCL Anxious/Depression subscale–Mother report (CBCL-M-A; dRCMAS = 0.59, dCBCL-M-

A = 0.60, k = 1, n = 94), CBCL Internalizing

scale–Father report (CBCL-F-I; dRCMAS

= 0.61, dCBCL-F-I = 0.60, k = 11, n = 383), or the Social Anxiety Scale for both children and adolescents (d

RCMAS = 1.06, d

SAS = 0.97, k =

2, n = 34). The mean effect size of the RCMAS was significantly lower than the ADIS-IV (d

RCMAS = 0.44, d

ADIS-IV = 1.11, k =

6, n = 136) and CBCL Internalizing scale–Mother report (CBCL-M-I; dRCMAS = 0.53, dCBCL-M-I = 0.77, k = 21, n = 750). Because of its high score reliability and validity, and moderate effect size estimate, we highly rec-ommend use of the total score of the RCMAS-2 as a useful self-report measure in both outcome research and clinical practice with school-aged youth.

Achenbach System of Empirically Based Assessment

The ASEBA (Achenbach & Rescorla, 2001) is a Level B, multirespondent clinical inventory, which assesses the report of parents CBCL, youth (Youth Self-Report, YSR), and teachers (Teacher Report Form, TRF). Cost is approxi-mately $1.00 per administration for each com-ponent (CBCL, YSR, TRF) and $400.00 for the examiner’s kit. The CBCL (parent report) is the most commonly used component of the ASEBA in anxiety outcome research and is designed to assess an array of behavioral and emotional problems in children and adoles-cents aged 4 to 18 years. It contains 113 items, each rated as 0 (not true), 1 (somewhat or sometimes true), or 2 (very true or often true) based on observed behavior within the pre-ceding 6 months. The CBCL consists of two broadband scales (Internalizing and External-izing) and eight narrowband, empirically derived subscales. The Internalizing scale is the scale most commonly used in anxiety out-come studies with school-aged youth. It is composed of items from the subscales of Withdrawn, Somatic Complaints, and Anx-ious/Depressed, thus is an imprecise measure of anxiety, specifically. The 2001 version of the CBCL also contains DSM-oriented sub-scales, including a new Anxiety Disorders subscale. Raw scores for each subscale are obtained by summing item ratings and then



Erford and Lutz 5

converting these to T-scores and percentiles. Norms for sex and age groups 4 to 11 years and 12 to 18 years were based on clinical and nonclinical samples. It was noted that the clinic-referred sample was predominantly White (89%), while the nonclinical sample was nationally representative with regard to ethnicity, socioeconomic status, geographical region, and urban-suburban-rural residence.

Coefficient alpha reliability estimates for the Internalizing broadband scale score ranged between α = .89 and .92 in various samples (Achenbach & Rescorla, 2001). One-week test–retest reliability coefficients were r

tt = .86

and .89 for the Anxious/Depressed and the Internalizing scores, respectively. Interrater reliability coefficients for the Anxious/Depressed and Internalizing scores were .77 and .66, respectively. Discriminant analysis revealed both the Anxious/Depressed sub-scale and Internalizing scale were successful in classifying individuals with and without anxiety disorders (Aschenbrand, Angelosante, & Kendall, 2005). The CBCL correlated sig-nificantly with the Conners Parent Rating Scale (r = .56–.86) and the Quay and Peterson Revised Problem Behavior Checklist (r = .52–.88).

Scale comparison analyses from effect sizes generated in the Erford et al. (2015) meta-analysis revealed mixed results for the CBCL (see Table 2). The mean effect size of the CBCL-M-I (Internalizing scale, mother report) was significantly higher than the STAIC-C (d

CBCL-M-I = 0.67, d

STAIC-C = 0.36, k =

9, n = 327) and STAIC-M (dCBCL-M-I

= 0.85, d

STAIC-M = 0.60, k = 4, n = 190), significantly

lower than the ADIS-C (dCBCL-M-I

= 0.76, d

ADIS-IV = 1.42, k = 4, n = 171), and compara-

ble with the Social Phobia Anxiety Inventory for Children (SPAI-C; d

CBCL-M-I = 0.66, d

SPAI-C

= 0.72, k = 2, n = 99) and STAIC-F (dCBCL-M-I

= 0.72, d

STAIC-F = 0.94, k = 3, n = 143). The

mean effect size of the CBCL-F-I (Internaliz-ing scale, father report) was significantly lower than the mean effect size of the STAIC-F (d

CBCL-F-I = 0.66, d

STAIC-F = 0.94, k = 3, n =

143) and not significantly different from the STAIC-C (d

CBCL-F-I = 0.66, d

STAIC-C = 0.55, k =

3, n = 143) and STAIC-M (dCBCL-F-I

= 0.66,

dSTAIC-M

= 0.66, k = 3, n = 143). The CBCL-M-A (Anxious/Depression subscale, mother report) mean effect size was not significantly different than the STAIC-C (d

CBCL-M-A = 0.60,

dSTAIC-C

= 0.57, k = 1, n = 94) and Multidimen-sional Anxiety Scale for Children (MASC) parent version (d

CBCL-M-A = 0.03, d

MASC-P =

0.12, k = 2, n = 157) mean effect sizes. Finally, when mean effect sizes were compared between two parents responding to the CBCL to rate the same child, the CBCL-M-I was found to yield significantly higher effect sizes than the CBCL-F-I (d

CBCL-M-I = 0.78, d

CBCL-F-I

= 0.56, k = 12, n = 396). This result is surpris-ing given that the mothers and fathers were rating the same set of children and may indi-cate that mother report, rather than the Inter-nalizing broadband scale itself, may inflate effect size estimates; that is, mothers may inflate either the pretest severity of the child or the effectiveness of the anxiety treatment. Alternatively, fathers may underestimate the pretreatment severity of the child or the effec-tiveness of treatment.

Unfortunately, the Internalizing broadband scale incorporates social withdrawal, somatic complaints, and depressive symptomology, each of which may obscure the effects of treatment on the core symptoms of anxiety by rendering imprecise estimates of anxiety. When the CBCL or TRF are used in outcome studies to assess anxiety, we recommend researchers use the DSM-based Anxiety sub-scale. The Anxiety subscale aligns most closely with the relevant clinical display under examination and, although the subscale offers slightly less reliable scores than the Internal-izing broadband scale, it yields scores of equivalent or better criterion-related validity when measuring anxiety as a dependent vari-able (Achenbach & Rescorla, 2001; Erford, 2013).

State-Trait Anxiety Inventory for Children

The STAIC (Spielberger et al., 1973), desig-nated now as STAIC-C to distinguish the child version from the parent version below, con-sists of two self-report scales designed to




measure transitory and situational anxiety (state anxiety; S-Anxiety) and anxiety prone-ness (trait anxiety; T-Anxiety) in children aged 9 to 12 years. Each scale contains 20 items; for each item, one of three response options is selected. Within the S-Anxiety scale, words or phrases are selected to indi-cate the individual’s feelings at a particular moment in time. For each statement presented within the T-Anxiety scale, a response of hardly ever, sometimes, or often is selected to indicate how the individual generally feels. Responses are weighted so that higher scores are indicative of greater anxiety or anxiety proneness. Raw scores for the S-Anxiety and T-Anxiety scales are obtained by summing response values and then converting these to T-scores and percentiles. Norms, which were based on a sample including Black and White children, were prepared for grade level (4–6) and sex. The STAIC-C may be administered individually or in a group setting and takes approximately 20 minutes to complete. Each administration of the STAIC costs between $1.00 and $2.00 depending on the quantity purchased and sample kits are available for $40.00.

The STAIC-C, a Level B instrument, has adequate internal consistency for screening level and research purposes, as coefficient alpha reliability estimates ranged between α = .82–.87 for the S-Anxiety scale and α = .78–.81 for the T-Anxiety scale (n = 246). Six-week test–retest reliability estimates for the S-Anxiety scale ranged between r

tt = .31 and

.47 (n = 246); these extremely low stability coefficients were attributed to the transitory nature of anxiety states (Spielberger et al., 1973). Six-week test–retest reliability esti-mates for the T-Anxiety scale were signifi-cantly higher and ranged between r

tt = .65 and

.71 (n = 246). Research on the validity of the STAIC-C is limited (Endler, 1982). Prelimi-nary evidence for the construct validity of the STAIC-C S-Anxiety scale indicated that scores for each item were higher in test condi-tions than in norm conditions. Concurrent validity of the STAIC-C T-Anxiety scale was supported by correlations between both the Children’s Manifest Anxiety Scale (r = .75;

n = 75) and the General Anxiety Scale for Children (r = .63; n = 75).

Modified from only the trait version of the STAIC-C, the STAIC-P (Strauss, 1987) is a Level B, parent-report scale that measures anxiety proneness in children. Similar to the STAIC-C T-Anxiety scale items, each state-ment presented within the STAIC-P requires a response of hardly ever, sometimes, or often to indicate the child’s experience; items are worded from the parent’s point of view. The parent form also includes six items that quan-tify several anxiety-related physiological symptoms.

There is a dearth of evidence regarding the reliability and validity of scores on the STAIC-P. A study conducted by Southam-Gerow et al. (2003) using a clinic-referred sample pro-vided some preliminary support for the psy-chometric properties of the STAIC-P. The authors reported a high internal consistency, with coefficient alpha reliability estimates ranging between α = .84 (n = 233) and .91 (n = 162). Test–retest reliability estimates over an interval of 8 weeks were satisfactory and ranged between r

tt = .71 (n = 72) and .75 (n =

43); over an interval of 1 year, estimates main-tained stability and ranged between .68 (n = 62) and .76 (n = 113). The STAIC-P correlated r = .46 (n = 135) and r = .50 (n = 233) with the CBCL Anxious/Depression subscale–Mother report. The CBCL Anxious/Depression sub-scale–Father report correlated r = .45 (n = 136) and r = .65 (n = 131) with the STAIC-P. The STAIC-P displayed poor discriminant validity. Correlations within the parental dyad (i.e., mother vs. father) were moderately high (r = .60; n = 135), while correlations between STAIC-P and STAIC-C were generally low (r = .13–.25).

Scale comparison analyses from effect sizes generated in the Erford et al. (2015) meta-analytic study (see Table 2) revealed the STAIC-C T-anxiety mean effect size was comparable with both the Revised FSSR–Child report (FSSR-C; d

STAIC-C = 0.40, d

FSSC =

0.34, k = 6, n = 349) and Social Anxiety Scale for Children (SAS-C; d

STAIC-C = 1.40, d

SAS-C =

0.97, k = 2, n = 34). When compared with the FSSR-C, the STAIC-M (d

STAIC-M = 0.45, d

FSSC



Erford and Lutz 7

= 0.45, k = 3, n = 209) and STAIC-F (dSTAIC-F

= 0.50, d

FSSC = 0.45, k = 1, n = 106) mean

effect size was not significantly different. No difference was noted between the d

STAIC-C =

0.00 and dFSSC-P

= 0.23 (k = 1, n = 12). The mean effect size of the STAIC-C was signifi-cantly lower than the ADIS-IV (d

STAIC-C =

0.39, dADIS-IV = 1.28, k = 2, n = 89). In general, the STAIC versions yield moderate effect size estimates with little propensity to inflate results.

Overall, the STAIC is an adequate choice for outcomes research. A huge advantage of using the STAIC is the availability of child and parent versions, although the low to near zero correlations between these rater groups is of great concern. Another concern is that the STAIC-C was designed for use with children aged 9 to 12 years (Grades 4-6) and is com-monly used with older and younger children. If outcome researchers use this instrument with children younger than age 9 years or ado-lescents older than age 12 years, interpretive caution is warranted. Finally, although a good deal of evidence supports the reliability and validity of STAIC scores, the original STAIC was published in 1973 and the parent version in 1987. Thus, the STAIC is in need of item updating to conform to modern conceptual-izations of childhood anxiety, and a modern normative sample is needed to ensure accu-rate interpretations of child and parent scores.

The Fear Survey Schedule for Children–Second Edition

The FSSC-II was created by Gullone and King (1992) and further studied by Burnham and Gullone (1997). The FSSC-II was adapted from the FSSC (Sherer & Nakamura, 1968) and the Revised FSSC (Ollendick, 1983). The FSSC-II is a Level B, self-report assessment designed to measure fear in children and ado-lescents. No cost or publisher information could be located. While this instrument was initially developed for use with children aged 8 to 11 years, subsequent research extended its use to children and adolescents aged 7 to 18 years and the factor structure was con-firmed on large Australian and U.S. samples.

The FSSC-II contains 75 items, each rated as 1 (none), 2 (some), or 3 (a lot) to indicate the level of fear associated with specific stimuli. The FSSC-II yields a score for Total Fear, as well as scores for five subscales: Fear of the Unknown, Fear of Failure/Criticism, Animal Fears, Fear of Death and Danger, and School/Medical Fears. Total scores for each subscale are obtained by summing item ratings; fre-quency scores are obtained by summing the number of items endorsed with a rating of 3. Raw scores may be converted to T-scores and percentiles. Norms are presented separately for sex and age groups 7 to 10 years, 11 to 13 years, and 14 to 18 years. The FSSC-II may be administered individually or in a group setting and can be completed in approximately 15 minutes. Several translations of the FSSC-II are available for cross-cultural use.

Test–retest reliability estimates for the Total Fear score of the FSSC-R were .82 across one week, .85 across 2 weeks, and .62 across 3 months. Over a 3-month interval, subscale test–retest reliability estimates ranged between .70 and .87. Coefficient alpha reliability estimates were consistently above .90 for the Total Fear score and ranged between .57 and .89 for the subscales. The FSSC-R correlated substantially with anxiety measures, including the RCMAS and the trait version of the STAIC. Discriminant analysis revealed FSSC-R scores differentiated among types of SPs as well as among phobia diag-nostic groups (Weems, Silverman, Saavedra, Pina, & Lumpkin, 1999). Perrin and Last (1992) reported that the FSSC-R subscale scores did not discriminate between boys with anxiety disorders, normal controls, and boys with attention-deficit hyperactivity disorder.

Scale comparison analyses from effect sizes generated in the Erford et al. (2015) meta-analytic study (see Table 2) indicate the mean effect size of the FSSC was comparable with those of the RCMAS (d

FSSC = 0.39,

dRCMAS

= 0.47, k = 14, n = 573), STAIC-C (d

FSSC = 0.34, d

STAIC-C = 0.40, k = 6, n = 349),

STAIC-M (dFSSC

= 0.45, dSTAIC-M

= 0.45, k = 3, n = 209), and STAIC-F (dFSSC = 0.45, dRCMAS = 0.50, k = 1, n = 106). Similarly, the FSSC-P mean effect size was comparable with that of




the STAIC-C (dFSSC-P

= 0.23, dSTAIC-C

= 0.00, k = 1, n = 12). Overall, the score reliability and validity of the FSSC-R and FSSC-II were high for a screening-level test and we recom-mend these instruments as dependent vari-ables in clinical trials when the criterion is the measurement of fear characteristics, rather than global or trait anxiety.

Anxiety Disorders Interview Schedule for DSM-IV

The ADIS-IV (Silverman & Albano, 1996) is a Level B, semistructured clinical interview designed to assess anxiety, mood, and exter-nalizing symptoms in children and adoles-cents aged 6 to 16 years. Clinician administered interview protocols are available for use with both children and parents. A kit including the book and five child and parent interview pro-tocols costs about $60. Additional protocols cost about $2.00 each. Compatible with DSM-IV criteria, the ADIS-IV allows for differential diagnosis of anxiety and anxiety-related dis-orders (Silverman & Albano, 2004). Interview questions are organized by diagnostic cate-gory. Each interview question requires a response of yes or no to indicate whether a symptom is present or absent, respectively; affirmative responses often require elabora-tion. Combined symptoms for each diagnosis are rated on a 9-point scale (0–8) to indicate the degree of impairment experienced by the individual. Impairment ratings and total symptom scale scores, derived by summing affirmative responses, inform diagnostic deci-sion making. The companion parent interview may be used with the ADIS-IV child inter-view to obtain combined ratings for cross-validating diagnoses. The ADIS-IV takes approximately 1 hour to administer to either youth or parents.

Concordance of child symptom ratings over an interval of 7 to 14 days yielded kappa coefficients ranging from .63 to .80 (n = 62) and intraclass correlation coefficients ranged from .78 to .95 (n = 62) for SAD, SP, SOP, and GAD. Test–retest reliability estimates for the impairment ratings of SAD, SP, SOP, and GAD were .60, .84, .10, and .72, respectively (Silverman, Saavedra, & Pina, 2001). Interra-

ter reliability for principal diagnosis and anxi-ety disorders was good to excellent (κ = .80–1.00; n = 153), while interrater reliability for attention-deficit hyperactivity disorder was poor (κ = .40; n = 153; Lyneham, Abbott, & Rapee, 2007). The ADIS-IV diagnoses of SOP and SAD correlated significantly with their corresponding factors on the MASC; however, minimal support for the concurrent validity of ADIS-IV GAD was reported (Wood, Piacentini, Bergman, McCracken, & Barrios, 2002).

Scale comparison analyses from effect sizes generated in the Erford et al. (2015) meta-analytic study (see Table 2) revealed the mean effect size of the ADIS-IV was signifi-cantly higher than the mean effect size of all comparison instruments: RCMAS (d

ADIS-IV-C

= 1.11, dRCMAS

= 0.44, k = 6, n = 136), CBCL-M-I (dADIS-IV-P = 1.42, d

CBCL-M-I = 0.76, k = 4, n

= 171), STAIC-C (dADIS-IV-C = 1.28, dSTAIC-C

= 0.39, k = 2, n = 89), SPAI-C (dADIS-IV-C = 1.32, dSPAI-C = 0.90, k = 7, n = 190) and SAS-C (dADIS-IV-C = 1.07, dSAS-C = 0.74, k = 4, n = 56). Thus, it is possible that the ADIS-IV yields inflated effect size estimates for use as an out-come measure and should be used with cau-tion for this purpose. That said, the ADIS-IV is an excellent tool for diagnosis and categori-zation of children being considered for inclu-sion in anxiety treatment studies, prior to randomization to treatment conditions.

Limitations and Implications for Counseling Research and Practice

A significant limitation of this analysis was the small number of comparisons available for most dependent variable combinations. Only 54 clinical trials published between 1990 and 2010 were located and selected into the Erford et al. (2015) meta-analysis and these studies used various designs, including single group designs as well as waitlist, pla-cebo, or treatment as usual comparison condi-tions. On the positive side, 41 of the 54 studies used two or more anxiety measurements and 14 anxiety instruments were used in two or more studies. The five most commonly used scales each appeared in at least 12 of the 54



Erford and Lutz 9

studies, resulting in a k of 1 to 21 study com-parisons on which to base effect size estimate comparisons. Cornwell (1993) and Cornwell and Ladd (1993) indicated that about 20 study comparisons were needed for sufficient power, and as one can surmise from Table 2, only one comparison (dRCMAS vs. dCBCL-M-I) met the 20 comparison threshold. This does not mean that the effect size similarities and differences reported above and in Table 2 are not accurate, just that these findings need to be interpreted with caution due to the small numbers of studies (ks of 1–21) and combined sample sizes (ns of 12–750 participants). In the future, as the number of clinical trials accumulate for studying the efficacy of coun-seling for treating anxiety in school-aged youth, the power of analyses such as this one will improve and allow greater confidence in effect size comparisons and interpretation of results.

A second limitation was that some of the instruments reviewed above lacked sufficient psychometric information (e.g., score reliabil-ity, score validity) to insure confident use, even though each (or an earlier version) has been in existence for at least 15 years. Future research publications involving use of these instruments as outcome measures should pro-vide this important information so that these anxiety instruments can be used with confi-dence to measure anxiety on the given popula-tions of interest. At the very least, authors of clinical trials using more than one anxiety out-come measure should report correlations between these measures and internal consis-tencies for each measure on the sample under study. Likewise, if a waitlist condition is used, reporting the correlation between pretest and posttest scores for the waitlist participants provides evidence of test–retest reliability.

Future research involving clinical trials on the treatment of anxiety in school-aged youth should also use multiple outcome instruments measuring anxiety, and preferably those instruments from among the list provided in Table 1. Use of multiple measures of anxiety not only allows the primary construct (i.e., anxiety) to be measured from multiple respon-dent perspectives (e.g., mother, father, teacher, self-report) but also allows comparisons of

effect sizes between or among the multiple anxiety measures used. Triangulation of treat-ment efficacy from multiple respondents using instruments of demonstrated high psy-chometric quality is critical to understanding whether desired treatment changes actually occurred. Comparisons of effect sizes gener-ated by multiple respondents across multiple measures allow exploration of the robustness of each instrument and respondent group to accurately assess anxiety treatment outcomes. Thus, if one instrument or respondent is less sensitive in documenting therapeutic change, other instruments or respondents may serve to more accurately document these effects. These comparisons of instrument sensitivity can only be accurately discerned through mul-tiple comparisons across numerous studies. Adopting the use of multiple instruments in future clinical trials would not only allow a more sensitive analysis of treatment effective-ness but would also boost the number of stud-ies using multiple instrument comparisons, thus increasing the number of effect size com-parisons (and power) of commonly used instruments in the extant literature.

Future studies should also use anxiety out-come instruments of demonstrated rigor (e.g., high score reliability and validity) to offer more precise measurements of the dependent variables under study. There are numerous anxiety instruments in existence but most have been developed and standardized for use with adult populations. Thus, the selection process for counselors who use anxiety instru-ments with school-aged youth is somewhat simplified because fewer instruments exist for use with this population. The process is sim-plified a bit further when considering that only a few instruments exist, which measure childhood anxiety from other informed respondent perspectives (e.g., parents, teach-ers), options which are generally not available for the assessment of anxiety in adults.

While different anxiety instruments have been developed to identify different aspects of childhood anxiety among varying popula-tions, the findings from this instrument review and effect size comparison study indicate that some youth anxiety assessment instruments are potentially better choices than others for




Table 3. Best Choices for Anxiety Outcome Research With School-Aged Children.

7–8 Years 9–12 Years13–17 Years

Self report RCMAS-2 RCMAS-2 RCMAS-2Parent

reportCBCL-

AnxietyCBCL-

AnxietyCBCL-

AnxietyTeacher

reportTRF-

AnxietyTRF-

AnxietyTRF-

AnxietyClinician

report

Note. RCMAS-2 = Revised Children’s Manifest Anxiety Scale–Second edition; CBCL = Child Behavior Checklist; TRF = Teacher Report Form.

documentation of clinical outcomes and use in clinical screening. Practically speaking, better choices for some researchers and clini-cians may reflect the economic realities of cost; ease of administration, scoring, and interpretation; availability of self- and other-report (e.g., parent, teacher) versions; speci-ficity of the construct under study; or the quality of instrument technical characteristics. Each of the five instruments reviewed above have advantages and disadvantages in these regards. The RCMAS-2 is a moderately priced, self-report instrument for individuals aged 6 to 19 years with good psychometrics. The ASEBA is a 113 item instrument with parent (CBCL), teacher (TRF), and adoles-cent self-report (YSR) versions, is moderately priced, and takes about 15 minutes to com-plete. It also has numerous scales, including Internalizing, Anxious/Depressed, and DSM-Anxiety, with the latter being the preferred and purest form for anxiety screening and out-comes research. The STAIC has brief self-report and parent versions yielding state and trait scales but was only designed and normed for children aged 9 to 12 years. Estimates of score reliability are adequate but substantive evidence of score validity is lacking. The FSSC-II was expanded for use with individu-als aged 7 to 18 years but is relegated to assessment of fear symptoms, rather than gen-eral anxiety, thus limiting its usefulness as an outcome measure. The FSSC-II has adequate to good psychometric properties. Finally, the ADIS-IV was the only clinician-administered instrument reviewed and has good psycho-metric qualities as a moderately priced screen-ing or diagnostic instrument. However, the ADIS-IV takes about 1 hour to administer.

At the core of this study, accuracy of effect size estimates must be considered when evi-dence emerges from use of these instruments to determine the outcomes of clinical inter-ventions (see Erford et al., 2015). In aggregate the RCMAS-2, ASEBA (CBCL, TRF, YSR), STAIC (child, parent), and FSSC-II yielded approximately equivalent effect size esti-mates, with slight variations depending on which respondents were assessed (see Table 2). For example, in some instances, mothers yielded slightly higher effect sizes than other

respondent categories, although this could have reflected sample dependent results. The ADIS-IV produced inflated effect sizes in all comparisons, and its use as a clinical outcome measure should be undertaken with caution as it is likely to reflect significant client progress when other instruments may not. However, it can be a valuable diagnostic tool. This infor-mation on effect size comparisons is particu-larly important in counseling research and practice because practitioners and researchers deserve reasonable and accurate expectations of client outcomes when assessments are used to document treatment effects in schools, clin-ics, and inpatient and outpatient facilities.

Each of these five instruments has advan-tages and disadvantages, are used with differ-ent respondents across different age ranges, and differentially assess varying facets of anxiety. Still, some are more advantageous for documenting clinical outcomes than others. Table 3 presents what we believe are the best choices of anxiety instruments for use with school-aged youth across different respondent categories (i.e., self-report, parent, teacher, and clinician) and age ranges (i.e., 7–8 years, 9–12 years, and 13–17 years) when all factors are considered. Note that no suggestion was made in the clinician report row, primarily because the ADIS-IV probably inflated effect size estimates, sometimes by as much as 100%. The RCMAS-2 has psychometric and construct-based advantages over the YSR, STAIC-C, and FSSC-II, even though all of these yield similar effect size estimates.



Erford and Lutz 11

Finally, the ASEBA’s CBCL-Anxiety (DSM) subscale enjoys numerous advantages over the STAIC-P among the two parent-report instruments reviewed. The TRF-Anxiety (DSM) was the only anxiety-based teacher-report scale in the mix and, like its companion CBCL, is a well-designed, albeit lengthy (about 15 minutes) teacher-report instrument.

In conclusion, numerous anxiety instru-ments exist for use with school-aged youth to determine the clinical efficacy of counseling treatments. These instruments vary in expense, time to administer and score, respondent forms, reliability and validity, practical fea-tures, and average effect sizes generated. Pro-fessional counselors are encouraged to measure the effectiveness of their counseling interventions to reduce anxiety levels in school-aged youth by administering more than one instrument to multiple respondents repeatedly from baseline through follow-up. Likewise, if researchers publishing clinical trials of anxiety treatments would adhere to this practice of using multiple anxiety instru-ments, rapid advancement in our understand-ing and identification of superior instruments will occur, thus better informing future practi-tioners and researchers of optimal practices in the assessment of anxiety treatment outcomes among youth in our society.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Achenbach, T. M., & Rescorla, L. A. (2001). Manual for the Achenbach system of empirically based assessment. Burlington, VT: ASEBA.

American Psychiatric Association. (2013). Diagnostic and statistical manual of men-tal disorders (5th ed.). Washington, DC: American Psychiatric Publishing.

Aschenbrand, S. G., Angelosante, A. G., & Kendall, P. C. (2005). Discriminant valid-ity and clinical utility of the CBCL with

anxiety-disordered youth. Journal of Clinical Child & Adolescent Psychology, 34, 735–746.

Beck, A. T. (1993). Manual for the Beck Anxiety Inventory. San Antonio, TX: Psychological Corporation.

Beidel, D. C., & Turner, S. M. (2005). Childhood anxiety disorders: A guide to research and treatment. New York, NY: Routledge.

Beidel, D. C., Turner, S. M., & Morris, T. L. (1995). A new inventory to assess childhood social anxiety and phobia: The Social Phobia and Anxiety Inventory for Children. Psychological Assessment, 7, 73–79.

Birmaher, B., Khetarpal, S., Brent, D., Cully, M., Balach, L., Kaufman, J., & McKenzie, S. (1997). The Screen for Child Anxiety Related Emotional Disorders (SCARED): Scale con-struction and psychometric characteristics. Journal of the American Academy of Child & Adolescent Psychiatry, 36, 545–553.

Burnham, J. J., & Gullone, E. (1997). The Fear Survey Schedule for Children—II: A psy-chometric investigation with American data. Behaviour Research and Therapy, 35, 165–173.

Cornwell, J. M. (1993). Monte-Carlo comparisons of three tests for homogeneity of independent correlations. Educational and Psychological Measurement, 53, 605–618.

Cornwell, J. M., & Ladd, R. T. (1993). Power and accuracy of the Schmidt and Hunter meta-analytic procedures. Educational and Psychological Measurement, 53, 877–895.

Endler, N. S. (1982). Review of the State-Trait Anxiety Inventory for Children. In O. K. Buros (Ed.), The eighth mental measurements yearbook (pp. 1094-1096). Highland Park, NJ: Gryphon Press.

Erford, B. T. (2013). Assessment for coun-selors (2nd ed.). Belmont, CA: Cengage Wadsworth.

Erford, B. T., Kress, V. E., Giguere, M., Cieri, D., & Erford, B. M. (2015). Meta-analysis: Counseling outcomes for youth with anxi-ety disorders. Journal of Mental Health Counseling, 37(1), 63-94.

Goodman, W. K., Price, L. H., Rasmussen, S. A., Mazure, C., Fleischmann, R. L., Hill, C. L., . . . Charne, D. S. (1989). The Yale-Brown Obsessive-Compulsive Scale: II. Validity. Archives of General Psychiatry, 46, 1006–1011.

Gullone, E., & King, N. J. (1992). Psychometric evaluation of a Revised Fear Survey Schedule for Children and Adolescents. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 33, 987–998.




Hamilton, M. (1959). The assessment of anxiety states by rating. British Journal of Medical Psychology, 32, 50–55.

LaGreca, A. M., & Stone, W. L. (1993). Social Anxiety Scale for Children–Revised: Factor validity and concurrent validity. Journal of Clinical Child Psychology, 22, 17–27.

Lyneham, H. J., Abbott, M. J., & Rapee, R. M. (2007). Interrater reliability of the Anxiety Disorders Interview Schedule for DSM-IV: Child and parent version. Journal of the American Academy of Child & Adolescent Psychiatry, 46, 731–736.

March, J. S., Parker, J. D. A., Sullivan, K., Stallings, P., & Conners, C. K. (1997). The Multidimensional Anxiety Scale for Children (MASC): Factor structure, reliability, and validity. Journal of the American Academy of Child & Adolescent Psychiatry, 36, 554–565.

Ollendick, T. H. (1983). Reliability and valid-ity of the Revised Fear Survey Schedule for Children (FSSC-R). Behaviour Research and Therapy, 21, 685–692.

Orvaschel, H. (1994). Schedule for Affective Disorder and Schizophrenia for School-Age Children Epidemiologic Version. Ft. Lauderdale, FL: Nova Southeastern University, Center for Psychological Studies.

Perrin, S., & Last, C. G. (1992). Do childhood anxiety measures measure anxiety? Journal of Abnormal Child Psychology, 20, 567–578.

Ramirez, S. Z., Feeney-Kettler, K. A., Flores-Torres, L., Kratochwill, T. R., & Morris, R. J. (2006). Fears and anxiety disorders. In G. G. Bear & K. M. Minke (Eds.), Children’s needs III: Development, prevention, and interven-tion (pp. 267–279). Bethesda, MD: National Association of School Psychologists.

Reynolds, C. R., & Richmond, B. O. (1985). Manual for the Revised Children’s Manifest Anxiety Scale. Los Angeles, CA: Western Psychological Services.

Reynolds, C. R., & Richmond, B. O. (2008). Manual for the Revised Children’s Manifest Anxiety Scale: Second edition. Los Angeles, CA: Western Psychological Services.

Sherer, M. W., & Nakamura, C. Y. (1968). A Fear Survey for Children (FSS-FC): A factor analytic comparison with manifest anxiety (CMAS). Behaviour Research and Therapy, 6, 173–182.

Silverman, W. K., & Albano, A. M. (1996). The Anxiety Disorders Interview Schedule for DSM-IV: Child and parent versions. San Antonio, TX: Psychological Corporation.

Silverman, W. K., & Albano, A. M. (2004). Anxiety Disorders Interview Schedule: ADIS-IV child/parent clinician manual. San Antonio, TX: Psychological Corporation.

Silverman, W. K., & Kurtines, W. M. (2001). Anxiety disorders. In J. N. Hughes, A. M. La Greca, & J. C. Conoley (Eds.), Handbook of psychological services for children and adolescents (pp. 225–244). New York, NY: Oxford University Press.

Silverman, W. K., Saavedra, L. M., & Pina, A. A. (2001). Test-retest reliability of anxiety symptoms and diagnoses with the Anxiety Disorders Interview Schedule for DSM-IV: Child and parent versions. Journal of the American Academy of Child & Adolescent Psychiatry, 40, 937–944.

Southam-Gerow, M. A., Flannery-Schroeder, E. C., & Kendall, P. C. (2003). A psychometric evaluation of the parent report form of the State-Trait Anxiety Inventory for Children–Trait Version. Journal of Anxiety Disorders, 17, 427–446.

Spence, S. H. (1998). A measure of anxiety symp-toms among children. Behaviour Research and Therapy, 36, 545–566.

Spielberger, C. D., Gorsuch, R. L., Lushene, R., Vagg, P. R., & Jacobs, G. A. (1973). Manual for the State-Trait Anxiety Inventory for Children. Redwood City, CA: Mindgarden.

Strauss, C. C. (1987). Modification of trait portion of State-Trait Anxiety Inventory for Children—Parent form. Gainesville: University of Florida.

Weems, C. F., Silverman, W. K., Saavedra, L. M., Pina, A. A., & Lumpkin, W. P. (1999). The discrimination of children’s phobias using the Revised Fear Survey Schedule for Children. Journal of Child Psychology and Psychiatry, 40, 941–952.

Wood, J. J., Piacentini, J. C., Bergman, R. L., McCracken, J., & Barrios, V. (2002). Concurrent validity of the anxiety disorders section of the Anxiety Disorders Interview Schedule for DSM-IV: Child and parent ver-sions. Journal of Clinical Child and Adolescent Psychology, 31, 335–342.

Author Biographies

Bradley T. Erford, PhD, is a professor in the school counseling program of the Education Spe-cialties Department in the School of Education at Loyola University Maryland.

Julie Lutz is a graduate of the school counseling mas-ter’s degree program at Loyola University Maryland.



Documents

Choosing Assessment Instruments Counseling and Development