Anxiety Rating for Children - Revised Reliability and Validity

7/30/2019 Anxiety Rating for Children - Revised Reliability and Validity

http://slidepdf.com/reader/full/anxiety-rating-for-children-revised-reliability-and-validity 1/18

Pergamon

Journal of Anxiety Diwrd crs, Vol. 10 , No. 2, pp. 97-114.1996

Copyright &, 1996 Elsevier Science Ltd

Printed in the USA. All rights mse twd

0887-618396 $15.00 + .Ml

SSDI 0887-61%5(95)00039-9

Anxiety Rating for ChiMren - Revised:

Reliability and Validity

GAIL A. BERNSTEIN,M.D.

Division of Child and Adolescent Psychiatry. University of Minnesota Medical School

RossD. CROSBY. PH.D.

Department of Psychiatry, University of Minnesota Medical Scho ol

AMY R. Pt%RWIFN,.A.

Division of Child and Adolescent Psychiatry. University of Minnesota Medical School

CARRIE M. BORCHARDT, M.D.

Division of Ch ild Md Ado lescen t Psychiatry, University of Minnesota Medical Scho ol

Abstract - The purpose of this investigation was to define the psychometric proper-

ties of the Anxiety Rating for Children - Revised (ARC-R), a clinician rating scale

for the assessment of anxiety symptoms in chiidren and adolescents. The ARC-R is

comprised o f an Anxiety subscale and a Physiological subscale. In a clinical sample(N = 22). the test-retest reliability and interrater reliability were investigated. In a

nonoverlapping clinic sample of school refusers (N = 199). the internal reliability and

convergent, divergent, and discriminant val idity were evahtated. Test-retest (r = .93)

and interrater reliability (r = .95) were excellent. There was good internal reliability

of items (Cronbach’s alpha = .80). The Anxiety subscale of the ARC-R correlated

Dr. Bernstein’s effort on this manuscript was supported in part by Grant R29 MH46534 from the

National Institute of Mental Health.

The authors thank John Hopwood, M.A. and Suzy Peterson, B.A. for their assistance n admin-

istering the rating scales and Lois Laitinen, M.B.A., M.M. for manuscript preparation.

Dr. Crosby is currently at NCS Assessments,Minnetonka, MN.Amy Perwien is currently a graduate student in the Department of Clinical and Health

Psychology at the University of Florida.

Requests for reprints should be sent to Gail A. Bernstein, M.D., Director, Division of Child

and Adolescent psychiatry, Box 95 UMHC, 420 Delaware St SE, Minneapolis, MN 55455.

97



98 G. A. BERNSTEIN ET AL.

somewhat higher with self-report anxiety instruments (r = .62) than with self-report

depression instruments (r = .54-S6). The Anxiety subscale of the instrument dis-

criminated between children with and without an anxiety disorder. Th is new instru-

me nt was shown to be a reliable measure of clinician-rated anxiety. The validity of

the ARC-R Anxiety subscale was demonstrated. The potential uses of this instrument

for clinical and research purposes are highlighted .

INTRODUCTION

Anxiety disorders are among the most prevalent disorders in child and ado-

lescent psychiatry (Costello, 1989; Kashani & Grvaschel, 1990). However,

only in very recent years has attention been focused on anxiety disorders in

children and adolescents (Last, 1992). As more attention has shifted to the area

of childhood anxiety disorders, methods of assessment need to be improvedand expanded.

To adequately assess or anxiety in children and adolescents, it is critical to

obtain information with a variety of types of instruments. The methods that are

currently available for assessment of childhood anxiety disorders include

structured and semistructured diagnostic interviews (Chambers et al., 1985;

Herjanic 8z Reich, 1982; Hodges, McKnew, Burbach, & Roebuck, 1987;

Silverman & Nelles, 1988). self-report instruments (Bernstein & Garfinkel,

1992; Ollendick, 1983; Reynolds & Richmond, 1978; Spielberger, 1973), and

parental report instruments (Achenbach, 1991; Wirt, Lachar, Kliendinst, &

Seat, 1977). The Anxiety Rating for Children - Revised (ARC-R) is the only

clinician rating scale, to our knowledge, which was specifically developed to

assessanxiety symptoms in children and adolescents.

The ARC-R uses symptom clusters similar to those in the Hamilton

Anxiety Rating Scale (Hamilton, 1959) which was designed for use in adults.

The Hamilton Anxiety Rating Scale is used as an outcome measure in many

treatment studies of anxiolytic medications (Maier, Buller, Philipp, & Heuser,

1988). It has recently been demonstrated to be reliable and valid in an adoles-

cent sample (Clark & Donovan, 1994).

It has been noted that interview measures are often advantageous becausethey minimize differences in subjects’ interpretations of questions (Clark &Donovan, 1994). These scales are important assessment tools because they inte-

grate. both the clinician’s experience and expertise and the child’s report of anx-

iety symptoms. In addition, the ARC-R provides the clinician with a measure

of the severity of cognitive and somatic aspects of anxiety rather than being

based solely on DSM-N (American Psychiatric Association, 1994) criteria.

While parents can serve as a valuable source of information, the symptoms

experienced by children and adolescents with anxiety disorders are often inter-

nally or subjectively felt. Therefore, it is imperative that the children be inter-viewed about their anxiety symptoms. In a review of parent-child agreement in

clinical assessment, Klein (1991) concluded that the concordance between

children’s and parents’ reports of anxiety disorders is generally poor. In a study

by Herjanic and Reich (1982), children reported significantly more subjective



ARC-R: RELIAB ILITY AND VALID ITY 99

symptoms, such as anxiety and depression, than the parents described in their

children. These factors underscore the importance of obtaining information

relevant to anxiety from the child’s perspective.Although anxiety may be manifested and reported by children and adoles-

cents as somatic symptoms (Beidel, Christ, & Long, 1991; Last, 1991), few

anxiety instruments systematically assess for somatic complaints. The ARC-

R, which includes a physiological subscale, provides the clinician with infor-

mation about the child or adolescent’s somatic symptoms, in addition to

information about psychological anxiety. Of the anxiety instruments designed

for children and adolescents, only the Revised Children’s Manifest Anxiety

Scale (RCMAS; Reynolds & Richmond, 1978) has a physiological subscale.

The physiological items of the RCMAS and ARC-R differ, with the ARC-R

providing a more comprehensive assessment of somatic complaints. Beidel et

al. (1991) found that children with anxiety disorders endorse a variety of dif-

ferent physical complaints.

The purpose of this investigation was to define the psychometric propertiesof the ARC-R in clinical samples and to examine the utility of the ARC-R as

an assessment instrument and as a screening measure for anxiety. It was pre-

dicted that reliability and validity data would support the use of this measure in

children and adolescents. Based on previous research of gender differences on

anxiety measures (Bernstein & Garfinkel, 1992; Ollendick, 1983; Ollendick,

Ring, & Frary, 1989; Reynolds & Richmond, 1985), it was further hypothe-sized that females would score higher than males on the ARC-R.

Development of the Instrument

The original ARC (Erbaugh, 1984) was developed for use as a clinician rat-

ing scale of anxiety in children and adolescents. The Anxiety subscale of the

original ARC included the following items: anxious mood, cognitive, tension,

fears, separation anxiety, depressed mood, and sleep disturbance. The Physio-

logical subscale included: muscular, sensory, cardiovascular, respiratory, gas-

trointestinal, genitourinary, and autonomic. A behavioral observation item wasalso included in the original ARC. The items included in each subscale were

consistent with the concept of measuring anxiety that is used in the Hamilton

Anxiety Rating Scale (Hamilton, 1959). The ARC grouped symptoms together

by category.

The depressed mood item of the original ARC was deleted in the ARC-R

because the content overlapped with the content of the Children’s Depression

Rating Scale - Revised (CDRS-R; Poznanski, Freeman, & Mokros, 1985). It

was felt that the revised version of the ARC would be a purer measure of anxi-

ety without the depression item. In addition, the sleep disturbance item wasdeleted due to poor psychometric properties (e.g., corrected item-to-total cor-

relation of .32 for sleep disturbance; .44-.68 for other items). Due to interest in

the investigation of somatic complaints, six physiological anxiety items were

revised in 1992. The genitourinary item was excluded because the content did



100 0. A. BBBNSTBlN Br AL.

not seem to be applicable to children and adolescents. After these changes, the

instrument was referred to as the ARC-R.

The ARC-R is intended to be administered by clinicians and mental healthprofessionals. The ARC-R allows the clinician to interview the child in a

semistructured fashion and rate the severity of the child’s anxiety symptoms

throughout the interview. In the ARC-R, a number of specific questions for

children and adolescents focusing on anxiety symptoms occurring within the

past week are included for each item. Each item is rated by the clinician

according to symptom severity on a Likert scale. Ratings include: not present

(0), mild (1). moderate (2), severe (3), and very severe (4). Criteria for each of

the severity ratings are included. As an example, the separation anxiety item

and the associated rating criteria are provided in Table 1.In the ARC-R, the Anxiety subscale score is based on five anxiety items (anx-

ious mood, cognitive, tension, fears, separation anxiety), and the Physiological

subscale score is based on six physiological items (muscular, sensory, cardio-

vascular, respiratory, gastrointestinal, autonomic). The sum of the two sub-

scales is the total Anxiety score. There are 11 items with a possible score of 0

TABLE 1

SEPARATIONNXIB~Y ITEM FXOMARC-R

When you’re not with your folks, are you afraid or worried about something bad happening to

them? Do you worry about something bad happening to you when you’re apart?

What has this been like in the past week?

How many times have you been absent in tbe last week? What is it like for you when you know

it’s time to go to school? Do you get stomachaches and headaches? How do you feel when you’re

on the way to school? Have you been leaving school to go home during the school day?

When you’re home, do you stay close to your mother (parent)? Do you follow your mom or dad

around tbe house or yard?

How do you feel when your parent(s) go out without you? Does it make you nervous or afraid to

be.alone in the house?Do you sleep in a room alone? Do you feel afraid or nervous about sleeping alone? Do you sleep

with a brother or sister, or with your parent(s)? Do you go into your parent(s)’ room during the

night?

How do you feel about sleeping away from your home and family (e.g., at friends’ or relatives’

homes, at camp)?

0 = NOT PRESENT - not at all anxious; no diff tculties with separations

I = MILD - feels anxious about some usual separations from patent figures. No social impairment.

2 = MODERATE -quite anxious about routine separations; some social impairment or restriction

of ageappropriate activities

3 = SEVERE - intense dependency and separation anxiety; functioning impaired due to anxiety,

marked social impairment or restriction

4 = VERY SEVERE - panic attacks or tantrums in context of separation. Intense separation

anxiety, extreme dependency, grossly disabling and resulting in incapacity for age-appropriate

functioning




to 4 for each item. The range of total scores is 0 to 44. Administration of the

instrument requires approximately 30 min.

METHOD

The psychometric properties of the ARC-R were studied in two separate

studies. In one study, the stability of the ARC-R across administrations (test-

retest reliability) and between raters (interrater reliability) was examined in 22

children and adolescents with anxiety and/or depressive disorders. In a second

study of 199 children and adolescents in the School Refusal Clinic, the conver-

gent and divergent validity of the Anxiety subscale of the ARC-R was exam-

ined via correlation analyses with other measures of anxiety and depression.

The discriminant validity of the ARC-R Anxiety subscale was examined in

diagnostic subgroups of these outpatients.

Study 1: Test-Retest and Interrater Reliability

Subjects. Twenty-two subjects participated in the reliability study. Written

informed consent for participation was obtained from each subject and a par-

ent. Subjects were informed that the purpose of the study was to learn about

and to define the test properties of the ARC-R. Nineteen of the subjects com-

pleted both interviews.Approximately two-thirds of the subjects (N = 14) were child and adoles-

cent outpatients who had previously been evaluated in the School Refusal

Clinic. Approximately one-third of the subjects (N = 8) were inpatients on the

Child and Adolescent Psychiatry Unit. Inclusion criteria for inpatients were

admission diagnoses (including provisional and rule-out diagnoses) of anxiety

disorder, mood disorder or adjustment disorder with emotional features. The

sample included 13 females (59%) and 9 males (41%). Mean age was 15.2 +

2.1 years with a range of 11 to 18 years. Racial composition was 21 Caucasians

and 1 Asian.

Procedure. Two B.A.-level research assistants were trained by the first author

in administration of the ARC-R interview. The training involved several com-

ponents including: explanation of the ARC-R, observations of live ARC-R

interviews, and practice administrations of the instrument to patients. In the

reliability study, ARC-R interviews were completed with one research assis-

tant giving the interview and the other observing the interview in the same

room. Both raters independently and simultaneously scored the interviews. All

interviews were tape recorded.

Interviewers were not blind to the purpose of the study. However, multipleinterviews were scheduled during the week to decrease the likelihood that an

interviewer would remember information from a particular subject. Subjects

were scheduled for two interviews administered by the same interviewer, 2-5

days apart. The same interviewer was used to minimize changes in scores



102 G. A. BERNSTHN FaT AL.

based on different interviewers. The 2- to 5&y interval was chosen to mini-

mize changes in anxiety level merely as a function of time and because the

ARC-R measures current anxiety (i.e., within the past week). Subjects werepaid at the end of each interview.

Study II: Internal Reliability and Validity

Subjects. Subjects were 84 females (42.2%) and 115 males (57.8%) ranging in

age from 6 to 17. The mean age was 13.1 f 2.4 years. Demographics of the

School Refusal Clinic sample have been described in previous publications

(Bernstein, 1991; Bernstein, Svingen, & Garfinkel, 1990, Bernstein & Garfinkel,

1992).

Procedure. Data from 199 outpatients evaluated in the School Refusal Clinic

over 8 years (1984-1992) were included. Approval from the Institutional

Review Board to review records and existing data was obtained. As part of a

comprehensive 3-hr outpatient assessment, the children and adolescents were

administered the original ARC and the CDRS-R, clinician rating scales, by a

masters-level psychologist. The children and adolescents also completed self-

report rating scales for anxiety and depression. The same psychologist, who

was blind to clinical information about the subjects, administered all the

ARCS, CDRS-Rs and self-report measures. Only the first 26 outpatients hadthe Physiological subscale of the ARC administered due to time constraints in

the Clinic. Parents were not present during the testing.

The clinician rating scales and the self-report instruments were given in the

following order: Children’s Depression Inventory (CDI; Kovacs, 1981),

Revised Children’s Manifest Anxiety Scale (RCMAS; Reynolds & Richmond,

1978), Visual Analogue Scale for Anxiety - Revised (WA-R; Bernstein &

Garfinkel, 1992), CDRS-R, ARC, and Children’s Depression Scale (CDS;

Lang & Tisher, 1978).

Prior to initiating the self-report instruments, the psychologist informally

assessed each child’s reading level by having the child read some of the items

aloud. If reading difficulty was apparent, the psychologist read all the items to

the child. The psychologist remained in the room to answer questions during

the administration of the self-report instruments.

To evaluate the discriminant validity of the ARC-R anxiety subscale, a sub-

set of the patients (N = 127) was assigned DSiU-III-R (American Psychiatric

Association, 1987) diagnoses based on independent review of 95% of the

charts by the first author and either a child and adolescent psychiatry resident

or medical student. The charts contained extensive clinical data because the

focus in the School Refusal Clinic is on thorough diagnostic assessment foranxiety and mood disorders. Information reviewed included the outpatient

evaluation summary and treatment notes. Diagnoses were based on data in the

charts documenting the presence of symptoms and not based on specific diag-

noses in the chart. The raters were blind to scores on the ARC-R and to scores



ARC-R: RELJABlLlTy AND VALIDll’Y 103

on other rating scales. After the first chart reviews, interrater agreement for

assigning subjects to diagnostic groups (anxiety disorder, anxiety and depres-

sive disorders, depressive disorder, and no anxiety or depressive disorder) was85% (100 of 118). Disagreements in diagnostic group assignment were

resolved by a second independent chart review by each rater and subsequent

discussion between the raters after the second chart review.

Gf the 127 cases reviewed, 73 (57.5%) had an anxiety disorder. Of the sub-

jects with an anxiety disorder, 30 had a comorbid depressive disorder (major

depression or dysthymia). Thirty-one (24.4%) had a depressive disorder only,

and 23 (18.1%) had no anxiety or depressive disorder. In the no anxiety or

depressive disorder group, eight (34.8%) had oppositional defiant disorder, six

(26.1%) had conduct disorder, one (4.3%) had attention-deficit hyperactivity

disorder, and eight (34.8%) had no diagnosis. The discriminant validity analy-

sis included three groups: anxiety disorder with or without comorbid depres-

sive disorder, depressive disorder only, and no anxiety or depressive disorder.

It was expected that any subject with an anxiety diagnosis, regardless of thepresence of a comorbid depressive disorder, would have an elevated ARC-R

Anxiety subscale score. Therefore, all subjects with an anxiety disorder, with

or without a depressive disorder, were grouped together.

Psychometric Instruments. Revised Children’s Manifest Anxiety Scale

(RCMAS; Reynolds & Richmond, 1978): The 37 statements on this self-reportanxiety instrument require “yes” or “no” responses. The RCMAS includes

three anxiety subscales and a Lie subscale. Adequate construct validity

(Reynolds, 1980) and test-retest reliability (Wisniewski, Mulick, Genshaft, &

Coury, 1987) have been demonstrated. In the adolescent group, the RCMAS

was also found to be a valid measure of anxiety (Lee, Piersel, Friedlander, &

Collamer, 1988).

Visual Analogue Scale for Anxiety - Revised (VAA-R; Bernstein &

Garfinkel, 1992). Each of the 11 items in this anxiety self-report is followed by

a continuous line on which respondents are asked to place a mark to show how

“jittery/nervous” or “steady” they feel. Reliability and validity have been stud-

ied and values are in the acceptable ranges (Bernstein & Garfinkel, 1992).

Children’s Depression Inventory (CDI; Kovacs, 1981). This is a 27-item

self-report measure of depression. One month test-retest reliability was good in

a nonclinical sample (Kovacs, 1981). The instrument has high internal consis-

tency (Saylor, Finch, Spirito, & Bennett, 1984). Convergent validity (Sham,

Naylor, & Alessi, 1990) and discriminant validity (Moretti, Fine, Haley, 8

Marriage, 1985) have been demonstrated.

Children’s Depression Scale (CDS; Lang & Tisher, 1978). This self-report

depression scale includes 66 statements which are written on individual cards.Each card is sorted into one of five boxes (labeled from “very right” to “very

wrong”). The test-retest reliability at 7-10 days was good in a nonclinical sam-

ple (Tisher & Lang, 1983). Adequate internal consistency and concurrent

validity have been demonstrated (Knight, Hensley, & Waters, 1988; Rotundo



104 G.A.BERNSWNETAL.

& Hensley, 1985). The CDS differentiates between children with and without a

diagnosis of depression (Knight et al., 1988).

Children’s Depression Rating Scale - Revised (CDRS-R; Poxnanski et al.,1985): This is an l&item clinician-rated scale for depression. High test-retest

reliability has been demonstrated (Poznanski et al., 1984). The instrument cor-

relates highly with other measures of depression supporting the convergent

validity of the scale. The CDRS-R discriminates depressed versus nonde-

pressed children (Poxnanski et al., 1984).

Statistical Analysis

Test-retest reliability was determined by Pearson correlation coefficients

between scores obtained at the two test administrations. Interrater reliability forsubscale and total scores was assessed using intraclass correlation coefficients

(Bartko & Carpenter, 1976). The internal reliability of the instrument was

assessedusing Cronbachs alpha. The association between measures was assessed

using Pearson correlations. Differences in ARC-R Anxiety subscale scores

between diagnostic groups were determined using one-way analysis of variance.

Receiver operating characteristic (ROC) analysis (Hanley & McNeil, 1982)

was performed to examine the diagnostic utility of the ARC-R Anxiety sub-

scale. This analysis is useful in characterizing the ability of a diagnostic instru-

ment to distinguish patients with a psychiatric disorder (“true positives”) fromthose without the disorder (“true negatives”) (Hsiao, Bartko, & Potter, 1989;

Murphy et al., 1987).ROC analysis evaluates the test performance of a diagnostic instrument

across the full range of test scores. For any given cutoff score on a diagnostic

instrument, the following values can be calculated: (a) the sensitivity, or “true

positive rate,” is the proportion of subjects with the disorder scoring at or

above that cutoff, (b) the specificity, or ‘true negative rate,” is the proportion of

subjects without the disorder scoring below that cutoff, and (c) the cumphnent

of sensitivity (i.e., one minus sensitivity), or the ‘false negative rate,” is the

proportion of subjects with the disorder scoring below that cutoff, and (d) the

complement of specificity, or “false positive rate,” is the proportion of subjectswithout the disorder scoring at or above that cutoff.

The ability of a diagnostic instrument to distinguish patients with a disorder

from those without the disorder can be graphically displayed using an ROC

curve. An ROC curve plots sensitivity (true positive rates) on the vertical axis

and one minus specificity (false positive rates) on the horizontal axis across all

possible cutoff scores. The diagonal line on the ROC curve, where true positive

rates are equal to false positive rates, represents the line of no information. The

greater the distance between the diagnostic test’s ROC curve and the line of noinformation, the better the diagnostic performance of the instrument. Statistical

methods are available for estimating the area under the ROC curve (AUC) and

its standard error (Hanley & McNeil, 1982). The AUC provides an estimate of

the probability that a randomly chosen subject with an anxiety diagnosis will




have a higher ranking on the ARC-R Anxiety subscale than a randomly chosen

subject without an anxiety diagnosis (Hanley & McNeil, 1982).

In the present study, RGC analysis was used to determine the ability of theARC-R Anxiety subscale to distinguish subjects with an anxiety diagnosis

from those subjects without an anxiety diagnosis, Subjects with an anxiety dis-

order with or without a comorbid depression were included because it was

expected that all subjects with an anxiety disorder regardless of the presence of

a depressive disorder would have an increased score on the ARC-R Anxiety

subscale. The AUC and standard error were calculated on the basis of the

binormal curve (Hanley and McNeil, 1982).

RESULTSStudy I: Test-Retest and Interrater Reliabili ty

Test-retest reliability data were obtained at a mean of 3.1 f .9 days on 19 sub

jects. For the Anxiety subscale score, the Pearson correlation coefficient compar-

ing scores at the first test administration with scores at the second administration

was .97 (Table 2). For the Physiological subscale score, the Pearson correlation

coefficient was .82. Pearson correlation coefficient was .93 for the total Anxiety

score. All correlations are at p < .OOl level. Group mean total Anxiety score at

the first testing was 13.5 + 9.6 and 12.3 + 8.3 at the second administration.

Interrater reliability, based on the first test administration, showed an intra-

class correlation coefficient of .97 for the Anxiety subscale score, .93 for the

Physiological subscale score, and .95 for the total Anxiety score (Table 2).These correlations are significant at p < .OOl level.

Study II: Internal Reliability and Validity

Average interitem correlation on the Anxiety subscale was .43 (range =

.35-66); corrected item-to-total correlations ranged from .47 (separation anxi-

ety) to .68 (anxious mood). Cronbach’s alpha was .79 for the five Anxietyitems (N = 199).

TABLE 2

TlS”bhZlEST AND hTBRRATER hIAB iLITY

Test-Retest Reliability* Interrater Re.liabilityb

N= 19 N= 19

Anxiety Physiological Total Anxiety Physiological Total

Subscale Subscale SCOE. Subscale Subscale score

.9l .82 .93 .91 .93 .95

WI Pearson conelation coefficients at p < .OOl

bul intrachiss comehtions at p < .OCJI



106 G. A. BeRNSTEIN ET AL.

Average interitem correlation for the Physiological subscale was .28 (range

= .08 to SO); corrected item-to-total correlations ranged from .34 (gastroin-

testinal) to -57 (muscular). Cronbach’s alpha was .69 for the six Physiologicalitems (N = 26). Cronbach’s alpha was .80 when the two subscales were com-

bined (N = 26).

All analyses of the validity of the measure used the subscales included in

the ARC-R. The ARC-R is basically a subset of items from the ARC. The

ARC was revised to form the ARC-R by deleting the depression, sleep distur-

bance, genitourinary, and behavioral observation items.

The validity results are based on the ARC-R Anxiety subscale, not the total

ARC-R scale. Pearson correlation coeffkients between the ARC-R Anxiety

subscale (sum of the five anxiety items) and subjects’ scores on Anxiety andDepression rating scales scores are summarr ‘zed in Table 3. The correlations

between the ARC-R Anxiety subscale and scores on the self-report anxiety

instruments were higher (r = .62) than the correlations between the ARC-R

Anxiety subscale and scores on the self-report depression instruments

(r = 54-56). However, the difference between these correlations is not signif-

icant based on the z statistic. All correlations in Table 3 are at p < .OOl level.

To explain the .67 correlation between the Anxiety subscale of the ARC-R and

CDRS-R, the analysis was repeated with anxiety disorder only patients and

resulted in a correlation of r = .58. For depression only patients the correlation

was r = .66.Pearson correlation coefficients between the ARC-R Physiological subscale

(includes six items) and anxiety measures were not significant, although the

correlation between the RCMAS Physiological subscale and the ARC-R

Physiological subscale approached significance (r = .35, p = .093). Significant

TABLE 3

CORRELATIONS~ BETWEEN THE AICOEIY SUBSCALE OF THE ARC-R AND

ANXE TY AND DEPRIWION RATHWX S~ALEX IN A CLINIC SAMPLEI

r N

Anxiety Scales

Revised Children’s Manifest Anxiety Scale

Subscales

Physiological

Worry/Oversensitivity

Fear/Concentration

Visual Analogue for Anxiety - Revised

Depression Scales

Children’s Depression InventoryChildren’s Depression Scale

Children’s Dcpmsion Rating Scale - Revised

.62 185

45

.61

.52

.62 99

.54 18556 175

.67” 187

aAll Pearson correlation coeffkients at p < .OO .

br = .58 for subjects with anxiety disorder only (N = 43).



ARC-R: RELIABILIT Y AND VALID ITY 107

correlations were found between the ARC-R Physiological subscale and the

CD1 (r = .41, p = 048) and the CDRS-R (r = 52, p = .007). The correlation of

the Anxiety and Physiological subscales of the ARC-R was 52 (p = 406,N = 26).

Mean Anxiety subscale scores were significantly higher for females (6.8 f

4.0) than males (5.3 + 3.4) (F = 8.06, df = 1, 197, p = .005) (Table 4). Mean

Physiological subscale scores were also higher for females (5.5 f 2.7) than

males (3.8 f 2.9), although the results did not reach statistical significance

(F = 2.41, df = 1, 24, p = .134). Similarly, females scored significantly higher

than males on all other rating scales (Table 4).

The mean score for the entire sample (N = 199) on the Anxiety subscale

was 5.9 f 3.7, with a range from 0 to 18. The mean score on the Physiological

subscale (N = 26) was 4.6 f 2.9, with a range from 0 to 9.

Analysis of variance (ANOVA) showed a significant difference in anxiety

subscale score among the three diagnostic groups (F = 9.96, df = 2, 124,

p = .OOOl). The mean score for the anxiety disorder group was highest at 6.2 it

3.6. The mean score for the depression group was 4.5 f 2.8 and the mean score

for the no anxiety or depressive disorder group was 3.0 f 2.0.

Following ANOVA, the Tukey-HSD Multiple Range Test was completed to

evaluate pairwise comparisons between groups. The anxiety disorder group

scored significantly higher than each of the other two diagnostic groups on the

anxiety subscale. There was no significant difference on Anxiety subscale scorebetween the depressive disorder and no anxiety or depressive disorder groups.

TABLE 4

COMPARISON OF FEMALES AND hhLE.S ON RATIN G SCALES

Females Males

M SD N M SD N F 4f P

Anxiety Scales

ARC-R Anxiety

Subscale 6.8 4.0 84 5.3 3.4 115 8.06 1,197 .005

ARC-R Physiological

Subscale 5.5 2.7 12 3.8 2.9 14 2.41 1,24 .134

RCMAS 11.5 7.4 76 8.3 4.9 109 12.60 1, 183 <.OOl

VAA-R 5.3 2.1 42 4.0 1.9 64 10.96 1,104 .OOl

Depression Scales

CD1 12.7 9.2 79 8.2 4.9 106 18.73 1, 183 401

CDS 64.5 26.4 74 51.7 23.6 101 11.40 1.173 <.oOl

CDRS-R 42.2 13.5 79 36.7 10.9 108 9.71 1, 185 .002

Note. ARC-R = Anxiety Rating for Children - Revised. RCMAS = Revised Childr en’s Manifest

Anxiety Sca le, VAA-R = Visu al Analog ue Sca le for Anxiety - Revised, CD1 = Child ren’s

Depression Inventory, CDS = Children’s Depression Scale, CDRS-R = Children’s Depression

Rating Scale -Revised.



108 G. A. BERNSTEIN ITI - AL.

The ROC analysis evaluating the ability of the ARC-R Anxiety subscale to

distinguish subjects with an anxiety diagnosis from those without an anxiety

diagnosis is presented in Figure 1. The figure presents sensitivity (true positiverate) and one minus specificity (false positive rate) values for cutoffs between

1 and 12. The AUC is .70 (SD = .05), which is significantly higher than the

line of no information (p < .oOl).

The ROC curve can be used to determine, for a given cutoff score, what

percentage of subjects with and without an anxiety disorder scored at or above

(or conversely, below) that point. For example, the values corresponding to a

cutoff score of 4 are: sensitivity = .75; one minus specificity = .42. This would

indicate that 75% of subjects with an anxiety disorder scored 4 or higher (i.e.,

sensitivity = .75) while only 25% scored below 4. Similarly, 58% of all sub-jects without an anxiety disorder scored below 4 (i.e., specificity = .58), while42% scored 4 or above. A cutoff value of 3 would result in a sensitivity of .82

and a specificity of .31; a cutoff value of 5 would result in a sensitivity of .67

and a specificity of .6 1.

DISCUSSION

The ARC-R, to the best of our knowledge, is the only clinician rating scale

for anxiety that is specifically designed for use in children and adolescents.

0.4 WC = .70

SD=.06

0.0 0.2 0.4 0.6 0.8 1.0

(7 - Specificity)

FIG. 1. ARC-R AMuen Smsax.e ROC CURVE.



ARC-R: RELIABILITY AND VALID ITY 109

The test-retest reliability of r = .93 for the total Anxiety score indicates stabili-

ty of the measure over a short period of time for which the instrument assesses.

Test-retest reliability was evaluated by using older children and adolescents(11-18 years old). Reliability of the ARC-R may have been influenced by the

subjects’ ages because reliability has been shown to increase with age

(Edelbrock, Costello, Dulcan, Kalas, & Conover, 1985; Schwab-Stone, Fallon,

Briggs, & Crowther, 1994). Interrater reliability of .95 indicates agreement of

raters when using the instrument, The instrument also has good internal relia-

bility (Cronbach’s alpha of .80). These properties indicate that the ARC-R is a

reliable measure of clinician rated anxiety.

Correlations between the Anxiety subscale of the ARC-R and RCMAS

(r = .62) (Table 3) and between the anxiety subscale of the ARC-R and the

VAA-R (r = .62) suggest all three anxiety scales are measuring similar con-

structs and support the convergent validity of the instrument. Gf the three sub-

scales on the RCMAS, the ARC-R Anxiety subscale correlates best with the

Worry/Oversensitivity subscale (r = .61), which has been identified as the sub-

scale on the RCMAS that best identifies children with anxiety disorders

(Mat&on, Bagnato, & Brubaker, 1988). In a study of referred children, only

the wonyloversensitivity factor of the RCMAS significantly distinguished the

anxiety group from the psychiatric control group (Mattison et al., 1988).

The correlations between the Anxiety subscale of the ARC-R and self-

report anxiety instruments (r = .62) were higher than the correlations betweenthe Anxiety subscale of the ARC-R and self-report depression instruments

(r = .54-.56) (Table 3), providing some support for the divergent validity of

the instrument. Although the correlation between the ARC-R Anxiety sub-

scale and CDRS-R was r = .67, this may be explained by the observations that

both are clinician ratings and that there is a substantial rate of comorbidity of

anxiety and depression in this sample. Scoring high on both anxiety and

depression instruments may reflect the child or adolescent experiencing both

anxiety and depressive symptoms (Bernstein & Garfinkel, 1992). More symp-

tomatic children and adolescents often have high scores on multiple measuresas a function of their severity (Bernstein, 1991). If only subjects with pure

anxiety disorders were included, the correlation between the ARC-R Anxiety

subscale and CDRS-R was lower (r = S8).

Furthermore, anxiety instruments do not always measure pure anxiety symp-

tomatology, nor do depression instruments measure pure depressive symptoms

(Bernstein & Garfinkel, 1992). Moderate correlations between anxiety and

depression instruments are consistently reported in assessment of children and

adolescents (Strauss, Lease, Last, & Francis, 1988). Nevertheless, the finding

that there are moderate correlations between the ARC-R Anxiety subscale and

the depression scales suggest a potential shortcoming of this instrument.

Further investigation of the relationship between the ARC-R and measures of

depression in a larger, pure anxious sample is needed to evaluate the divergent

validity of the ARC-R.



110 G. A. B@RNSTElN G T AL.

The Physiological subscale of the ARC-R only correlated significantly with

the CD1 (r = .41) and the CDRS-R (r = .52). Due to the small sample size used

to evaluate the Physiological subscale (N = 26), these correlations and othercorrelations between the Physiological subscale and measures of anxiety and

depression should be viewed as preliminary. However, it appears that somatic

complaints may be symptoms of depression as well as symptoms of anxiety. In

a study by McCauley, Carlson, and Calderon (1991) exploring the relationship

between anxiety, depression, and somatic complaints, the presence and severi-

ty of depression predicted somatic complaints, even when anxiety was con-

trolled. Furthermore, a substantial number of the subjects who met criteria for

depression had moderate to severe somatic complaints. The authors concluded

that the association between depression and somatic complaints could not beaccounted for by anxiety.

Females scored significantly higher than males on the Anxiety subscale.

Females also were higher than males on the Physiological subscale, although

this finding was not statistically significant. The gender differences on the ARC-

R are consistent with gender findings on other anxiety measures (Bernstein &

Garilnkel, 1992; Ollendick et al., 1989; Reynolds & Richmond, 1985).

Children and adolescents with a DSM-III-R anxiety disorder had signifi-

cantly higher scores on the Anxiety subscale than those without an anxiety

disorder. This finding provides data for the discriminant validity of the

Anxiety subscale of the ARC-R. However, a limitation of this study is the useof retrospective chart reviews for establishing the diagnoses of the subjects.

The methodology would have been stronger if structured psychiatric inter-

views had been used for diagnoses. Therefore, the conclusions about discrim-

inant validity need to be viewed tentatively.

The diagnostic utility of the ARC-R Anxiety subscale in screening for anxi-

ety disorder in psychiatric patients is suggested by the ROC analysis. The

Anxiety subscale was shown to significantly differentiate between psychiatricpatients with and without anxiety disorders. Cutoff points of 4 or 5 appear

most appropriate in providing an optimal balance between sensitivity andspecificity. Further study is needed to determine the diagnostic utility of the

ARC-R in a nonpsychiatric setting.Somatic complaints are often a manifestation of anxiety. Because it mea-

sures somatic symptoms, the ARC-R may prove to be particularly useful withthe changes made from DSM-III-R (American Psychiatric Association, 1987)

to DSM-ZV (American Psychiatric Association, 1994). Criteria for separation

anxiety disorder, generalized anxiety disorder, and panic disorder emphasize

physical symptoms associated with anxiety. Therefore, with DSM-ZV, more

attention will be focused on somatic symptoms as we diagnose children and

adolescents with anxiety symptoms.

Studies evaluating somatic symptoms in anxious children suggest that fur-

ther investigation, with an instrument such as the ARC-R, will be fruitful.

Several studies have noted that somatic complaints are common in children




with separation anxiety disorder and panic disorder (Last, 1991; Livingston,

Taylor, & Crawford, 1988). In an investigation including 76 children, anxious

children reported significantly more physical symptoms than normal controls,and several of the symptoms differentiated the two groups (Beidel et al.,

1991). As to children and adolescents with recurrent abdominal pain, higher

levels of anxiety in these children compared to those in normal children have

been reported (Hodges, Kline, Barbero, 8z Woodruff, 1985; Walker & Greene,

1989; Wasserman, Whitington, & Rivara, 1987).

In our experience with the ARC-R, some youth appear to minimize their

anxiety symptoms on the five psychological anxiety items, but readily endorse

multiple symptoms on the six physiological items. Perhaps they find it more

acceptable to report physical symptoms than emotional symptoms. The use of

only 26 outpatients in the evaluation of the Physiological subscale is a short-

coming of the validation study. However, data on the psychometric properties

of this subscale are currently being collected. The entire ARC-R instrument is

currently being used as an outcome measure in a treatment study of school-

refusing adolescents with comorbid anxiety disorder and major depression. It

will be of interest in the study to examine the relationship between change on

the Anxiety subscale and change on the Physiological subscale.

The ARC-R is a reliable instrument for clinician-rated anxiety in children

and adolescents. The validity of the Anxiety subscale has been established.

This instrument fills the void of a much needed clinician rating scale for anxi-ety. It provides the advantage of assessing both psychological and physiologi-

cal symptoms of anxiety. Its psychometric properties justify the use of this

rating scale as a clinical instrument for assessing anxiety and demonstrate its

usefulness as a screening instrument. As a research instrument, the ARC-R

may also prove to be an effective tool in evaluating the efficacy of treatments

for anxiety in children and adolescents.

The ARC-R complements the other anxiety instruments currently available by

providing a perspective that is not bound by DW-ZV criteria, as in sbrucRued and

semistru~ interviews. It also overcomes some of the potential weaknesses ofself-report and parental report measures because it integrates the clinician’s exper-

tise with the child or adolescent’s report of symptoms. The ARC-R would be most

useful as part of a comprehensive assessment which includes a diagnostic inter-

view, self-report measures of anxiety and depression, and parental report measures.

REFERENCES

Achenbach, T. M. (1991). M anual for the chil d behavior checklist/4-(18) and 1991 profile.

Burlington, VT: Universi ty of Vermont Department of Psychiatry.

American Psychiatric Association (1987). Diagno stic and statistical manual of mental disorders

(3rd ed.. rev.). Washington, DC: Author.

American Psychiatric Association (1994). Diagnostic and statistical manual of mental disorders

(4th ed.). Washington. DC: Author.

Bartko, J. J.. & Carpenter. W. T. (1976). On the methods and theory of reliability. Journal of

NervousandMentalDiseose, 163.307-317.



112 G. A. BERNSTEIN El- AL.

Beidel, D. C., Christ, M. A. Cr., & Long, P. J. (1991). Somatic complaints in anxious children.

Journal of Abnormal Child Psychology, 19.659670.

Bernstein, G. A. (1991). Comorbidity and sever ity of anxiety and depressive disorders in a clinic

sample. Journal of the American Academy of Child and Adolescent Psychiatry, 30.43-50.

Bernstein, G. A., & Garfnkel, B. D. (1992). The Visual Analogue Scale for Anxiety - Revised:

Psychometric properties. Journa l of Anxiety Disorde rs, 6,223-239.

Bernstein, G. A., Svingen, P. H., & Garfinkel, B. D. (1990). School phobia: Patterns of family

functioning. Journal of the American Academy of Child and Adolescent Psychiatry, 29.24-30.

Chambers, W. J., Puig-Antich. J.. Hirsch, M., Paez, P., Ambrosini. P. J., Tabrizi, M. A., & Davies,

M. (1985). The assessmentof a ffective disorders in children and adolescents by semistructured

interview: Test-retest reliability of the schedule for affective disorders and schizophrenia for

school-age children, present episode version. Archives of General Psychiatty, 42,6%-702.

Clark, D. B.. & Donovan, J. E. (1994). Reliability and val idity of the Hamilton Anxiety Rating

Scale in an adolescent sample. Journal of the American Academy of Child and Adolescent

Psychiatry, 33,354-360.

Costello, E. J. (1989). Child psychiatric disorders and their correlates: A primary care pediatric

sample. Joumal of the American Academy of Child and Adolescent Psychiatry, 28.851-855.

Edelbrock, C. , Costello, E. J., Dulcan, M. K.. Kalas, R, & Conover, N. C. (1985). Age differences

in the reliability of the psychiatric interview of the child. Child Development, 56265-275.

Erbaugh, S. E. (1984). Anxiety Rating for Children. Unpublished instrument.

Hamilton, M. (1959). The assessment of anxiety states by rating. British Journal of Medical

Psychology, 32,50-55.

Hanley, J. A., & McNeil, B. J. (1982). The meaning and use of the area under a Receiver

Operating Characteristic (ROC) curve. Diagnostic Radiology, 143.29-36.

Hetjaoic, B., & Reich, W. (1982). Development o f a structured psychiatr ic interview for children:

Agreement between child and parent on individual symptoms. Journal of Abnormal ChildPsychology, 10,307-324.

Hodges, K., Kline, J. J., Barbero, G., & Woodruff, C. (1985). Anxiety in children with recurrent

abdominal pain and their parents. Psychosomatics, 26859-866.

Hodges, K.. McKnew, D.. Burbach, D. J., & Roebuck L. (1987). Diagnostic concordance between

the Child Assessment Schedule (CAS) and the Schedule for Affective Disorders and

Schizophrenia for School-Age Children (K-SADS) in an outpatient sample using lay inter-

viewers. Joumal of the American Academy of Child and Adolescent Psychiatry, 26.654-661.

Hsiao, J . K., Bartko, J. J., & Potter, W. 2. (1989). Diagnosing diagnoses: Receiver operating char-

acteristic methods and psychiatry. Atchive of Genem l Psych iatry, 46.664667 .

Kashani, J. H., & Grvaschel, H. (1990). A community study of anxiety in children and adoles-

cents. Ame rican Journa l of Psychiatry, 147.313-318.Klein, R. G. (1991). Parent-child agreement in clinical assessment of anxiety and other psy-

chopathology: A review. Journa l of Anxiety Disorde rs, 5,187-l 98.

Knight, D., Hensley, V. R. & Waters, B. (1988). Validation of the Children’s Depression Scale and

the Children’s Depression Inventory in a pmpubettal sample. Journal of Chitif Psychology and

Psychiatry and Allied Discipline s. 29,853-863.

Kovacs. M. (1981). Rating scales o assess epression n school-aged children. A&I Paedop@tiotrica,

46.305-315.

Lang, M., & Tisher, M. (1978). Chitdrens Depression Scale. Melbourne: Australian Council for

Educational Research.

Last, C. G. (1991). Somatic complaints in anxiety disordered children. Journal ofAnriety Disonicrs,

5125-138.

Last, C. G. (1992). Anxiety disorders in childhood and adolescence. In W. M. Reynolds (Ed.),

Internalizing disorders in &&en and adolescents. New York: John Wiley and Sons.

Lee, S. W., Pier&, W. C., Friedlander, R. & Collamer, W. (1988). Concurrent validi ty o f the

Revised Children’s Manifest Anxiety Scale (RCMAS) for adolescents. Educational and

Psychological Measurement, 48.429-433.




Livingston, R. Taylor, J. L., & Crawford, S. L. (1988). A study of somatic complaints and psychiatric

diagnosis in childten. Journal of the American Academy of Child ad Aabkmwt Psych&y, 2’7,

185-187.

McCauley, E., Carbon. G. A., & Calderon, R. (1991). The role of somatic complaints in the diag-

nosis of depression in children and adolescents. Journul of the American Academy of Child

and Adolescent Psych&y, 38.631-635.

Maier, W., Buller, R., Philipp, M., % Heuser, I. (1988). The Hamilton Anxiety Scale: Reliability,

val idity and sensitivity to change in anxiety and depressive disorders. Joum~l ofA$xtive

Disorders, 14,61.

Mattison, R. E., Bagnato, S. J., & Brubaker, B. H. (1988). Diagnostic util ity of the Revised

Children’s Manifest Anxiety Scale in children with DSM-III anxiety disorders. Jourrud of

Anriefy Disonlers, 2.147-155.

Mote& M. M., Fine, S., Haley, G., Bi Marriage, K. (1985). Childhood and adolescent depression:

Child-report versus parent-report information. Journal of the American Academy of Child

Psychiatry, 24,298-302.

Murphy, J., Berwick, D., Weinstein, M., Borus. J., Budman. S., 8i Klerman, G. (1987).

Performance of screening and diagnostic tests. Archives of General Psych&y, 44.550-555.

Ollendick, T. H. (1983). Reliability and val idity of the Revised Fear Survey Schedule for Children

(FSSC-R). Behuviour Resea rch and Therapy, 21,685-692.

Ollendick. T. H., King, N. J., & Frary, R. B. (1989). Feats in children and adolescents: Reliability

and generalizability across gender, age and nationality. Behnviour end Resea rch The rapy, 27,

19-26.

Poznanski, E. 0, Freeman, L. N., & Moktos, H. B. (1985). Children’s Depression Rating Scale -

Revised. Psychopharmacology Bulletin, 4.979-989.

Poznanski, E. 0.. Grossman, J. A., Buchsbaum, Y., Banegas, M., Freeman, L., & Gibbons, R.

(1984). Preliminary studies of the reliability and val idity o f the Children’s Depression RatingScale. Joumol of the American Academy of Child Psychiatry. 23,191-197.

Reynolds, C. R. (1980). Brief reports: Concurrent val idity of What I Think and Feel: The Revised

Children’s Manifest Anxiety Scale. Joumnt of Consulting and Clinical Psychology, 48,

774-775,

Reynolds, C. R., & Richmond, B. 0. (1978). What I think and feel: A revised measure of chil-

dren’s manifest anxiety. Journal of Abnormal Child Psychology, 6.271-280.

Reynolds, C. R., & Richmond, B. 0. (1985). Revised Childre n’s Manifest Anxiety Sca le (RCMAS)

menuul. Los Angeles, CA: Western Psychological Services.

Rotundo, N., & Hensley, V. R. (1985). The Children’s Depression Scale. A study of its volidiry.

Joum ~l of Child Psychology and Psychiatry, 26.917-927.

Saylor, C. F., Finch, A. J., Spirito, A., & Bennett, B. (1984). The Children’s Depression Inventory:A systematic evaluation of psychometric properties. Journal of Consulting and Cfinical

Psychology, 52,955-967.

Schwab-Stone, M., Fallon, T.. Briggs, M., Crowther, B. (1994). Reliability of diagnostic reporting

for children aged 6-1 I years: A test-retest study of the Diagnostic Interview Schedule For chil-

dren -Revised. Ame rican Journa l of Psychiatry, 151.1048-1054 .

Shain, B. N., Naylor, M. & Alessi. N. (1990). Comparison of self-rated and clinician-rated mea-

sures of depression in adolescents.Ame rican Jourm al of Psychinrry, 147.793-795.

Silverman, W. K., & Nelles, W. B. (1988). The Anxiety Disorders Interview Schedule for

Children. Journal of the American Academy of Child and Adolescenr Psychintry. 27,772-778.

Spielberger, C. (1973). Manual for the State-Tra it Anxiety Inventory for Children. Palo Alto:

Consulting Psychologists Press.

Strauss, C. C., Lease, C. A., Last, C. G., & Francis, G. (1988). Overanxious disorder: An examina-

tion of developmental differences. Journal ofAbnomw[ Child Psychology, 16,433-443.

Tisher, M., & Lang, M. (1983). The Children’s Depression Scale: Review and further develop-

ments. In D. P Cantwell & G. A. Carlson (Eds.), Affective disorders in childhood and ndofes-

cence: An update (pp. 181-203). New York: Spectrum Publications.



114 0. A. BlXNS’IElN ET AL.

Walker, L. S., & Greene, J. W. (1989). Childreo with recurrent abdominal pain and their parents:

More somatic complaints, anxiety, and depression than other patient families? Journal of

Pediatric Psychology, 14.23 l-243.

Wasserman, A. L., Whitington, R F., & Rivara, F. P (I 988). Psychogenic basis for abdominal pain

in children and adolescents. Journal of the American Academy of Child and Adolescent

Psychiatry, 27,179-l 84.

Wisniewski, J. J., Mulick, J. A., Genschaft. J. L., & Coury, D. L. (1987). Test-retest reliability of

the Revised Children’s Manifest Anxiety Scale. Perce ptaal and Motor Stilk. 65.67-70.

Wti. R. D., Lachar, D., Klieodinst, J. K., & Seat, F? D. (1977). Multidimensional description of

chi ld person ality: A man aal for the Person ali?y Inventory for Children. Las Angeles: Western

Psychological Services.

Documents

Anxiety Rating for Children - Revised Reliability and Validity