This article was downloaded by: [Adams State University]On: 24 November 2014, At: 15:18Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,37-41 Mortimer Street, London W1T 3JH, UK

Journal of Clinical Child PsychologyPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/hcap19

Factors influencing elementary school teachers' ratingsof ADHD and ODD behaviorsJack Stevens & Alexandra L. QuittnerPublished online: 07 Jun 2010.

To cite this article: Jack Stevens & Alexandra L. Quittner (1998) Factors influencing elementary school teachers' ratings ofADHD and ODD behaviors, Journal of Clinical Child Psychology, 27:4, 406-414, DOI: 10.1207/s15374424jccp2704_4

To link to this article: http://dx.doi.org/10.1207/s15374424jccp2704_4

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) containedin the publications on our platform. However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of theContent. Any opinions and views expressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon andshould be independently verified with primary sources of information. Taylor and Francis shall not be liable forany losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use ofthe Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Journal of Clinical Child Psychology 1998, Vol. 27, No. 4,406-414

Copyright O 1998 by Lawrence Erlbaum Associates, Inc.

Factors Influencing Elementary School Teachers' Ratings of ADHD and ODD Behaviors

Jack Stevens and Alexandra L. Quittner Department of Psychology, Indiana University-Bloomington

Howard Abikoff New York University Medical Center

Examined factors that influence teachers' ratings of children with either attention def- icit hyperactivity disorder (ADHD) or oppositional demnt disorder (ODD). 105 teachers watched 2 videotapes-1 depicting a normal child and the other a child with either ADHD or ODD--and rated each child using 2 different questionnaires. Results indicated that teachers accurately rated the child on the ADHD versus ODD tape as having significantly more inattention and hyperactivity but significantly less oppositionality. However, effect sizes indicated the presence of a unidirectional, neg- ative halo efSect of oppositional behavwrs on ratings of hyperactivity and inattention. Teachers appeared less biased in theirjudgments when using a welboperationalized rating scale. Finally, knowledge, education, and experience with children with ADHD generally had no effect on the accuracy of teachers' ratings.

Elementary school teachers play a major role in the assessment of children's academic and behavioral problems. Due to their extensive contact with chil- dren in a variety of structured and unstructured set- tings and their knowledge of age-appropriate skills and behaviors, teachers provide important informa- tion for both clinical and research purposes (Atkins, Pelham, & Licht, 1985; Shelton & Barkley, 1995). Most often, educators are asked to evaluate children with psychological problems by completing standard- ized rating scales (Sandoval, 1981). These ratings are then used to make decisions concerning diagnosis, treat- ment, and educational placement (see Brown, 1985). Teacher ratings are also frequently used in a research context as inclusionary criteria, as a means of monitor- ing treatment progress, and as indicators of long-term outcomes (DuPaul, Guevremont, & Barkley, 1991).

Despite evidence indicating that teachers' ratings can discriminate between children and adolescents with higher versus lower levels of psychological diffi- culties (e.g., Dalley, Bolocofsky, & Karlin, 1994), teachers are not always accurate and objective raters of

We thank Dr. Janice Bizzari for her support and assistance. We also thank the principals and teachers for their time and conscientious effort.

Alexandra L. Quittner is now at the Department of Clinical and Health Psychology, University of Florida.

Requests for reprints should be sent to Jack Stevens, Department of Psychology, Indiana University, Bloomington, IN 47405-1301, E-mail: jhsteven@indiana.edu, or to Alexandra L. Quittner, Depart- ment of Clinical and Health Psychology, University of Florida, P.O. Box 100165, Gainesville, FL 32160, E-mail: aquittne@hp.ufl.edu

childhood behavior. Often they lack the time or ability to notice specific children's behaviors. Alternatively, teachers may have frequent opportunities to observe children's conduct but may be biased by certain char- acteristics of the students. For example, Stevens (1980) found that ethnicity and socioeconomic status produced negative halo effects on teachers' ratings. Specifically, the videotaped behaviors of African American and poor children were evaluated as more deviant than Caucasian and middle-class children, de- spite identical rates of disruptive behaviors.

Negative halo effects may be associated not only with certain demographic characteristics but also with the presence of certain childhood behaviors. These bi- ases have been found to produce errors in teachers' judgments, particularly in the area of attention deficit hyperactivity disorder (ADHD). Mann et al. (1992) noted that the diagnosis is "based more on an assess- ment of developmentally inappropriate intensity, fre- quency, andor duration of the behavior rather than its mere presence. Such judgments increase the possibil- ity of observer bias" (p. 1539).

Researchers have found that although teachers' rat- ings can reliably differentiate children with and with- out attention deficits (Atkins, Pelham, & Licht, 1985), including even those who exhibit hyperactivity (Brown, 1985), teachers often do not distinguish chil- dren with ADHD from those with symptomatology of the other disruptive behavior disorders--conduct dis- order (CD) and ODD. For example, Schachar, Sandberg, and Rutter (1986) investigated the relation between two respondents-teachers and blind observ-

Dow

nloa

ded

by [

Ada

ms

Stat

e U

nive

rsity

] at

15:

18 2

4 N

ovem

ber

2014

TEACHERS' RATINGS OF ADHD AND ODD BEWAVIORS

ers-on measures of ADHD and ODD. They found a negative halo effect of oppositional behaviors on hy- peractivity ratings but no halo effect of hyperactivity on oppositional behaviors. Thus, regardless of their ac- tivity level, children exhibiting defiance or aggression were judged as having ADHD.

Abikoff, Courtney, Pelham, and Koplewicz (1993) found a similar unidirectional bias in which oppositional behaviors exerted a negative halo effect on ratings of ADHD symptomatology. Specifically, oppositional behaviors in a child on a classroom video- tape spuriously elevated teachers' reports of the sever- ity of his ADHD behaviors. However, hyperactive and inattentive behaviors did not lead teachers to rate an- other child as being defiant or aggressive. This study attempted to replicate these findings, testing the hy- pothesis that ODD behaviors lead teachers to rate chil- dren as hyperactive or inattentive but that ADHD behaviors do not cause teachers to rate these children as oppositional.

A central issue remains unclear from the Schachar et al. (1986) and Abikoff et al. (1993) studies. Spe- cifically, the factors that might lead teachers to appear less biased in their ratings have not been identified. Certain aspects of the assessment measures or particu- lar characteristics of the teachers may affect their abil- ity to differentiate between the two disorders. This study sough~t to address this question by assessing three possible determinants of the bias in teachers' ratings of disruptive blehavior disorders: measurement character- istics of the rating scales, teachers' knowledge of and education in ADBD, and teachers' professional expe- rience with these children.

First, we hypothesized that two properties of rating scales may affect teachers' ability to distinguish ADHD from ODD: level of confounding of symptoms across the two disorders and the extent to which items are behaviorally anchored. Several commonly used teacher rating scale factors (e.g., Hyperactivity Factor of the Conners Teachers Rating Scale; Conners, 1990) have individual items that reflect both ADllD and ODD (e.g., "disturbs other children") behaviors. Therefore, it is not surprising that teachers' ratings of- ten do not differentiate between these disorders when individual items include symptoms applying to both. Some researchers have found that pure items can help teachers distinguish between different types of prob- lem behaviors (Ellen, 1989; Milich Idt Fitzgerald, 1985). However, other studies (e.g., Abikoff et al., 1993; Schachar et al., 1986) have found no evidence that omitting confounded items reduced the unidirec- tional bias. 'These results should be revisited to reex- amine whether items that reflect either ADHD or ODD prove useful in making this distinction. In the current study, we expected to find less bias in teacher ratings when utilizing specific items that reflected only hyper- activity and inattention as opposed to aggression.

We also examined whether teachers' ratings would be less biased when more behaviorally anchored items were administered (Abikoff et al., 1993). Qluestion- naires often contain vague items that require a global rating on some characteristic (e.g., "difficulty in learn- ing"; Vaughn, Riccio, Hynd, & Hall, 1997). These items are often ambiguous with respect to the target be- havior and may be interpreted in different ways, inter- fering with teachers' ability to make more fine-grained distinctions in behavioral ratings. In contrast, descrip- tive items that include greater operationaliiation of the problem behaviors (e.g., "does not sit still", "blurts out answers") should lead to greater accuracy of' teacher ratings. Thus, rating scales that employ concrete as op- posed to global items may be less subject to inegative halo effects (Mimtz & Collins, 1985).

Abikoff et al. (1993) found evidence of the unidi- rectional bias even when teachers rated children using the Diagnostic and Statistical Manual of Mental Dis- orders (3rd editi~on, rev.; DSM-IZZ-R) ADHD diagnos- tic category symptoms, which were specific and well operationaked. In that study, however, a categorical (all or non~e) approach to assessment was utilized for the DSM-III-R symptoms, which did not permit a thorough examination of differences in the magnitudes of teacher ratings. A continuous distribution of ADHD ratings would permit greater detection of differences in teachers' ratings. In this study we compared the impact of using twlo different continuous behavioral measures, one with and one without global items. We bypothe- sized that teachers would be less biased in their ratings when using a rating scale that required behaviorally anchored rather than global judgments.

A second factor that may influence teachers' ability to distinguish between ADHD and ODD is their knowledge of and educational background in ADHD. Little is known about how teachers' knowledge of the disorder is associated with their ratings or their use of particular intervention strategies (Greene, 1995). To date, no study has systematically evaluated the relation between knowledge of ADHD and teachers' ability to discriminate this childhood disorder from others. We hypothesized that greater knowledge of the disorder would heighten teachers' awareness of ADHD symptomatology, resulting in better distinctil~ns be- tween ADHD and ODD.

Finally, we examined whether professional experi- ence with children with ADHD, their parents, a,nd vari- ous health care professionals would lead to more accurate ratiugs. Interactions with students with ADHLI, as well as in-service presentations and conver- sations with medical and mental health professionals about children with ADHD may be related to more positive perceptions of these children, as was found in a study examining teachers' attitudes about mainstreaming students with learning difficulties (Bender, Vail, & Scott, 1995). We hypothesized that

Dow

nloa

ded

by [

Ada

ms

Stat

e U

nive

rsity

] at

15:

18 2

4 N

ovem

ber

2014

STEVENS, QUITTNER. & ABIKOPF

teachers with greater professional experience with children with ADHD and their caretakers would be less susceptible to negative halo effects when rating children displaying oppositional behaviors.

In sum, the central objective of this study was to ex- amine whether a unidirectional, negative halo effect was present in teachers' ratings of ADHD and ODD behaviors. Furthermore, we investigated the role three factors-characteristics of the rating scales, knowl- edge of ADHD, and experience with ADHD-might play in increasing the accuracy of teachers' ratings.

Method

Participants

One hundred and eight elementary school teachers were recruited from eight public schools through let- ters and phone calls to principals. In addition, three ele- mentary school teachers returning for a graduate degree in education also participated. Individual teach- ers were not compensated for their participation; how- ever, schools in which teachers participated were offered either a small monetary donation or an in-service presentation on ADHD after completion of the study. Six of the 11 1 teachers failed to complete 75% of each of the standardized ratings scales de- scribed in the following, and therefore their data were excluded from the analyses.

The final sample for the current study consisted of 105 elementary school teachers from a medium-sized Midwestern city. The majority of teachers were women (90%) and Caucasian (5% were African Amer- ican). Fifty-seven of the participants taught exclu- sively regular education students, 8 teachers taught only special education students, and the remaining 40 taught both regular and special education students. On average, teachers had considerable teaching experi- ence (ME 14.75 years, SD = 9.14). Seventy-seven had a master's degree, and two held a doctorate.

Stimuli

The three videotaped stimuli used in this study are described briefly in the following paragraphs. A more detailed description of the development and validation of the videotapes can be found in Abikoff et al. (1993).

Each 10-min videotape highlighted the activities of one specific 10-year-old boy, along with a teacher and several peers as they engaged in a structured academic activity (e.g., completing worksheets). The target child in each tape exhibited different types and frequencies of disruptive childhood behaviors in order to depict a child who was either "normal," had "pure" ADHD, or had "pure" ODD. Specifically, the two "pathology"

tapes were designed so that the target child in the ADHD tape engaged in behaviors characteristic of ADHD and did not behave in an oppositional fashion. In contrast, the target child in the ODD tape displayed oppositional behaviors characteristic of the disorder but exhibited a rate of ADHD behaviors in the normal range.

Several steps were taken to ensure the validity of the videotapes. Two independent sets of observers, each using either the revised Stony Brook classroom observation code (COC; Abikoff, Gittelman-Klein, & Klein, 1977) or the Classroom Observations of Con- duct and Attention Deficit Disorders (COCADD; Atkins et al., 1985; Atkins, Pelham, & Licht, 1989) and blind to the condition of the target child, were asked to rate the frequency of ADHD and ODD behaviors. Their ratings were in the expected direction. For exam- ple, the rate of hyperactive behaviors was judged to be twice as high for the target child in the ADHD as op- posed to the normal or ODD tape. The frequency and types of interactions amang classmates and teacher were also controll~d on each tape to remove any biases. Finally, to minimize biases related to nonspecific char- acteristics (e.g., attractiveness of the child), the target child in each psychopathology tape was played by the same actor; a different child played the "normal" child.

Procedure

Individuals with elementary school teaching expe- rience were invited to participate in a study of teachers' judgments of childhood behaviors. The study was con- ducted in a large room (e.g., library, conference room) at each school. After signing informed consent state- ments, groups of teachers, ranging in size from 1 to 19, were assigned to one of four conditions.

A between-subject design was employed in which teachers viewed, in counterbalanced order, two 10-rnin videotapes designed by Abikoff and colleagues (1993). All teachers viewed a videotape of a normal fourth-grade boy in a classroom setting and either a videotape of a boy with pure ADHD or a videotape of a boy with pure ODD. Forty-seven teachers viewed the normal and ADHD tapes (i,e., 17 saw the normal tape first and 30 saw the ADHD tape first). The remaining 58 teachers viewed the normal and ODD tapes (i.e., 28 saw the normal tape first and 30 saw the ODD tape first). Thus, the four conditions were: (a) nor- maVADHD, (b) ADHDInormal, (c) normaVODD, and (d) ODDInormal.

Prior to presentation of the two tapes, teachers were given three packets of questionnaires, and the investi- gators read a page of instructions telling teachers that they will be presented with two videotapes showing children in a fourth-grade classroom containing chil- dren with mixed learning levels and that they can think

Dow

nloa

ded

by [

Ada

ms

Stat

e U

nive

rsity

] at

15:

18 2

4 N

ovem

ber

2014

TEACHERS' RATINGS OF ADHD AND ODD BEHAVIORS

of each tape as a 10-min "slice of life" in the school day. They were instructed to watch a particular young- ster on the tape and were not informed of the diagnostic status (i.e., "normal," "ADHD," and "ODD') of the target children. Full instructions can be found in Abikoff et al. (1993).

After viewing the first tape, teachers were told to extrapolate from what they observed on the tape to questions involving behaviors that were not present on the tape, as they would in a real situation. After view- ing each tape, teachers completed two different rating scales-the Conners Teacher Rating Scale (CTRS-28; Goyette, Comers, & Ulrich, 1978) and the SNAP-N Rating Scde (Swanson, personal communication, De- cember 13, 1996). Finally, teachers completed two questionnares in a third envelope that assessed their knowledge of and experience with ADHD. Teachers were not told that the individuals appearing in the tapes were actors until completion of the study. The entire procedure lasted approximately 45 rnin.

Measures

Conners Teacher Ratings Scale. The CTRS- 28 is a shortened version of the 39-item Conners Teacher Questionnaire (Conners, 1969) and is one of the most frequently used teacher rating scales. Teachers rate the target child's behavior on a 4-point scale ranging from not at all to very much. The CTRS-28 yields three factors (i.e., Conduct Problem, Hyperactivity, and Inattentive-Passive) and a Hyperkinesis Index, which contains 10 of the most fre- quently endorsed items by teachers (Goyette et al., 1978). Good internal consistency coefficients (.74-.94) were obtained for the four subscales of the CIXS-28 for teachers rating either pathology tape in this study.

SNAP-IV Rating Scale. A 39-item version of the SNAP-IV Rating Scale was also completed by teachers to rate the presence of ADHD and ODD be- haviors in the target child. Teachers rated the child's behavior on a 4-point scale that was similar to the one used on the CTRS-28. In this version of the SNAP-IV, there were the nine DSM (4th ed.; DSM-ZV) symptoms of ADHD-Inattentive Type, the nine DSM-ZV symp- toms of ADHD-Hyperactive/Impulsive Type, and the eight DSM-Wsymptoms of ODD. The SNAP-IV was chosen because its items utilized more specific opera- tional definitions and required more descriptive (e.g., "fidgets with hands or feet or squirms in seat") as op- posed to global (e.g., "childish and immatureD') judg- ments than did the CTRS-28. In addition, the SNAP-IV specified behaviors according to DSM-N criteria, which reflects the field's most recent under-

standing of ADHD symptoms. Good internal consis- tency coefficients (i.e., .84 to .95) were obtained with this sample.

Knowledge of ADHD. Teachers' knowledge of ADHD was assessed using the 17-item Knowledge scale from the Attention Deficit Hyperactivity Disor- der Knowledge and Opinion Survey-Revised; Bennett, Power, Rostain, and Carr, 1996). This true-false test in- cluded questions concerning the etiology, assessment, treatment, and prognosis of the disorder. Correct re- sponses were coded "1," and incorrect or bllank re- sponses were coded "0." Item-total correlations in the present sample of elementary education teachers were poor, ranging from -.22 to 30. Internal consistency was also low (Cronbach a = 22).

Experience with ADHD. A 23-item scale, the ADHD Experience Scale, was developed for this study. This questionnaire expanded on a similar measure de- veloped by Anastopoulos (personal commu~nication, January 23,1997) andcontained three major sections:

1. Educational opportunities to learn about ADHD (e.g., in-service presentations).

2. Professional experiences with ADHD (e.g., number of previous students with ADHD, con- ferences with parents, number of contacts with health care professionals about these students.

3. Attitudes about ADHD (e.g., how distressing these behaviors are).

Two subscales were created from this measure: a six-item Educational Background scale (Cronbach a = .68) and an eight-item Professional Contact scale (Cronbach a = .75).

Results

Preliminary Analyses

There were no systematic differences in teachers' characteristics across the four conditions. Chi-square tests indicated that the number (of teachers with special education experience or graduate degrees was similar across conditions,x2(3, N= 105)= 1.72 and3.54,p> .30. Three separate muItivariate analyses of variance (MANOVAs) revealed no significant differences across conditions in educational background, profes- sional contact, and knowledge of ADHD, years of expe- rience, or ratings of distress in h~andling ADHDl behav- iors (Hotelling's Trangedfrom0.090to 1 .475,~ > .05).

As expected, paired t tests indicated that teachers dif- ferentiated between the normal and the two pathology

Dow

nloa

ded

by [

Ada

ms

Stat

e U

nive

rsity

] at

15:

18 2

4 N

ovem

ber

2014

STEVENS, QUITTNER, & ABIKOFF

Table 1. Teachers' Mean CTRS-28 and SNAP-IV Ratings of Each Tape

Type of Child Depicted on Tape - --

Normal ADHD ODD

Scale M SD M SD M SD Effect Size Indexs

CTRS-28 HA 0.19 0.08 1.91 0.57 1.66 0.55 0.27 IN-PA 0.36 0.42 1 .56 0.61 1.36 0.58 0.17 HI 0.27 0.28 1.73 0.57 1.78 0.54 0.03 CP 0.03 0.08 0.74 0.53 2.05 0.57 1.23

SNAP-IV HYAMP 0.17 0.17 1.72 0.66 1.10 0.66 0.50 INATT 0.38 0.46 1.90 0.67 1.38 0.72 0.38 ODD 0.03 0.21 0.77 0.67 2.13 0.62 1.08

- - Note: N= 105 for normal tape, N= 47 for ADHD tape, and N= 58 for ODD tape. CTRS-28 = Conners Teachers Rating Scale (28-itemversion); HA =Hyperactivity factor; IN-PA = Inattentive-Passive factor; HI = Hyperkinesis Index; CP = Conduct Problem factor; HYIIMP = DSM-IV ADHD Hyperactivity/Impdsivity scale; INATT = DSM-IV ADHD Inattention scale; ODD = DSM-IVOppositional Defiant Disorder scale. %ffect size = differences in ratings between the ADHD and ODD tape controlling for the order of the normal tape, special education experience, and ratings of the normal tape. Effect sizes < S O are modest, .SO-.70 are moderate, and > .70 are large.

tapes. As can be seen in Table 1, teachers rated the target child in the ADHD tape versus the normal tape as having greater symptomatology on the CTRS-28 Hyperactiv- ity, Inattentive-Passive, andHyperkinesis scales andon the SNAP-IV Hyperactivity-Impulsivity, and Inatten- tion scales; t(46) ranged from 12.68 to 21.47, p < .001. Similarly, teachersratedthe target child intheODD tape versus the normal tape as having more severe oppositional behaviors on the Conners Conduct Prob- lem scale and on the SNAP-IV ODD scale, t(57) =27.83 and 24.76, p < .001.

Ratings of Behavior on the ADHD Versus ODD Tapes

To test the presence of a negative halo effect (Hy- pothesis I), a 2 (Type of Tape: ADHD vs. ODD) x 2 (Order of the Normal Tape: First vs. Second) x 2 (Type of Teacher: Special Education/Inclusion vs. Regular Education) analysis of covariance (ANCOVA) was conducted on each of the four CTRS-28 scales and three SNAP-IV scales. To account for differences in individual response styles across teachers, their ratings on the normal tape were used as covariates. Across all scales, we found no significant effects of order of tape or teachers' special education experience as well as no significant two-way or three-way interactions. F(1,96) ranged from .OO to 3.35, p > .05.

Ratings of hyperactive and inattentive behaviors. As expected, no significant differences were found be- tween ratings of the ADHD versus ODD tape on the In- attentive-Passive scale, F(l, 96) = 2.84, p > .09, or on the Hyperkinesis Index, F(l, 96) = .052, p > 30. Con-

trary to our prediction, the ADHD tape was rated significantly higher than the ODD tape on the CTRS-28 Hyperactivity Factor, F(1, 96) = 7.17, p < .01. Note, however, that the mean CTRS Hyperactivity Factor scores were still above the clinical cutoff of 1.5 on both tapes. Finally, the child in the ADHD versus ODD tape was judged to have greater symptomatology on the SNAP-IV Inattention and Hyperactiv- ity-Impulsivity scales, F(1, 96) = 13.76 and 23.92 re- spectively, p < .001.

Ratings of oppositional behaviors. As predicted, the average CTRS-28 Conduct Problem and SNAP-IV ODD scores were significantly lower for teachers watching the ADHD versus ODD tape, F(1, 96) = 145.66 and 103.24 respectively, p < .001.

Effect sizes for judgments of ADHD versus ODD behaviors. The effect size of type of tape (ADHD vs. ODD) was calculated1 after controlling for order, special education experience, and behavior ratings on the normal tape (see Table 1). As mentioned previ- ously, teachers' ratings of the child on the ADHD ver- sus ODD tape were significantly different on two of the four CTRS-28 scales (Hyperactivity Factor, Conduct Problem Factor) and on all three of the SNAP-IV scales. Thus, although several statistically reliable dif- ferences were found, the magnitude of the main effects for type of tape was considerably less for measures of inattention and hyperactivity than for measures of oppositional behavior. In sum, there was substantially less discrepancy between the two pathology tapes in ratings of ADHD versus ODD behaviors, suggesting

'~ffect size indexes were calculated using the proportion of vari- ance in ratings accounted for by the type of tape. See Cohen (1992) for further details on effect sizes.

Dow

nloa

ded

by [

Ada

ms

Stat

e U

nive

rsity

] at

15:

18 2

4 N

ovem

ber

2014

TEACHERS' RATINGS OF ADHD AND ODD BEHAVIORS

that oppositional behaviors did exert a negative halo ef- fect on ratings of hyperactivity and inattention.

Impact of Operationalization of the Behavioral Measures on Inattention and Hyperactivity Ratings

Teachers were expected to evidence less bias in their ratings of inattention and hyperactivity as operationalization of the rating scale increased. This hypothesis was supported. As mentioned previously, differences in the predicted direction were obtained on both of the well-operationalized ADHD scales of the SNAP-IV but on only one of the three ADHD scales of the less behaviorally anchored CTRS-28. Thus, item content and wording of the questions ap- peared to contribute significantly to the bias.

Two pauticular problems with the CTRS-28 were noted: confounding of content across the scales and the global wording of some items, First, in terms of over- lapping content, both the Hyperactivity factor and the Hyperkinesis Index of the CTRS-28 contained items that assessied both ADHD and ODD behaviors simulta- neously (e.g., "disturbs other children"). A measure of pure inattention and hyperactivity, not confounded by items reflecting aggression and noncompliance, might reduce the occurrence of biased judgments of children with ODD. The five InattentionlOveractivity (10) items from the IOWA Conners Teachers Rating Scale (Loney & Milich, 1982) that were empirically related to hyperactivity but not aggression were subjected to an analysis of c~variance.~ After partialling out ratings of the normal tape, significantly higher ratings on these five items for the target child on the ADHD versus ODD tape were found, M = 10.8 and 8.3 respectively, F(1,96) = 16.76, p < ,001. This indicated that the bias was weaker when using a scale with items reflecting only hyperactivity and inattention.

Second, problems with the wording of the inatten- tion items were noted in an item analysis of the CTRS-28 Inattentive-Passive scale. The Inatten- tive-Passive scale contains two items reflecting social skills as opposed to inattention (i.e., "appears to be eas- ily led by other children" and "appears to lack leader- ship"). Two items also required global rather than descriptive judgments (i.e., "childish and immature" and "difficulty in learning"), which may be more sus- ceptible to negative halo effects (Mintz & Collins,

2 ~ n our study, "restless in the 'squirmy' sense," "makes inappro- priate noises when slhe shouldn't," "distractibility or attention span a problem," "excitable, impulsive," and "fails to finish things that slhe starts" were utilized. The items frrom the original (1982) scale were as follows: "fidgeting," "hums and makes other odd noises," "inatten- tive, easily distracted," "excitable, impulsive," and "fails to finish things he stants (short attention span)."

1985). To assess the impact of these problemaltic items on teachers' ratings, these four items were omitted from the CTRS-28 Inattentivepassive scale, and the remaining four items were subjected to an analysis of covariance. As expected, teachers rating the ADHD versus ODD tape endorsed significantly moire severe inattention om this modified scale, F(1,96) = 8.89, p < .01. In sum, negative halo effects appear to be at least partly due to measurement problems in these com- monly used ratings subscales.

Factors Related to Teachers' ADHD Ratings of the Pathology Tapes

Teachers with greater knowledge about and more professional experience with ADHD were expected to rate the two pathology tapes more accurately. To ex- amine the relation between these teacher chmacteris- tics and their ratings of hyperactive and inattentive behaviors, hierarchical regression analyses were per- formed separately for the PDHD and ODlD tapes. Three-step hierarchical regressions were eomputed with the CTRS-28 InattentivePassive and Hyperac- tivity scales and the SNAP-IV Inattention and Hyper- activity-Impulsivity scales as dependent variables. Ratings of the normal tape andl order of the tape presen- tation were entered on the first and second steps, re- spectively. Next, five variables reflecting teachers' backgrounds and experience were entered on the third step. These teacher factors were: (a) ADHD Educa- tional Background, (b) ADHD Professional Contact, (c) ADHD Knowledge soore, (d) Special Elducation Experience, and (e) Distress Ratings (on ;I 5-point scale) caused by ADHD behaviors.3

The regression analyses indicated that neither teacher's individual rating style (Step 1) nor the order of tape presentation (Step 2) was significantlly related to any of the scale scores for teachers rating either pa- thology tape. The total R2 ranged from .02 to .07. The only exception was that highes hyperactivity ratings of the normal tape were significantly related ito higher CTRS-28 hyperactivity ratings of the ODD tape O) < .05; total R2 = .19). On the third step, ADHD Profes- sional Contact, ADHD Knwwledge Test score, and Level of Distress caused by ADHD behaviors were not significantly associated with ratings of the M)HD tape (p > .05). However, two variables were significantly related to teachers' ratings of the ADHD tape, but in the direction opposite to our prediction:

3~ulticollinearity among the predictor variables did nc>t appear to be a problem. For ratings of the ADHD tape, wrrelati~ons among these variables = -0.26 to 0.57. For ratings of the ODD tape, wrrela- tions among these variables =-0.27 to 0.44. Multicollinearity is more likely when correlations are above .70 (Hanushek & Jackson, 1977).

Dow

nloa

ded

by [

Ada

ms

Stat

e U

nive

rsity

] at

15:

18 2

4 N

ovem

ber

2014

STEVENS, QUITTNER, & ABIKOFF

1. Teachers with more special education experi- ence endorsed less symptomatology on the CTRS-28 InattentivePassive scale.

2. Teachers with more ADHD education had sig- nificantly lower CTRS-28 Hyperactivity rat- ings (p < .05; total R~ = .26 and .18, respec- tively; A R~ coefficients = .21 and .l3).

On the third step of the analyses of the ratings of the ODD tape, ADHD Professional Contact, Special Edu- cation Experience, and Level of Distress caused by ADHD behaviors were not significant predictors. However, as predicted, higher scores on the ADHD Knowledge test were associated with significantly lower CTRS-28 InattentivePassive ratings of the ODD tape. Moreover, ADHD Educational Back- ground was a significant predictor for all four scales (p < 05; total Rz ranged from .18 to .31; A R2 coefficients ranged from "10 to .28). Contrary to Hypothesis 3, teachers with more ADHD education endorsed more inattention and hyperactivity on the ODD tape.

Discussion

We obtained support for our first hypothesis that the presence of oppositional behaviors would exert a uni- directional, negative halo effect on teachers' ratings of hyperactivity and inattention. In this study, teachers generally rated the child on the ADHD versus ODD tape as having significaqtly greater hyperactivity and inattention and significantly less oppositionality. However, an examination of the effect sizes for type of tape indicated inaccuracies in teachers' judgments. Small differences in ratings of hyperactive and inatten- tive behaviors were found between pathology tapes, in contrast to the large differences obtained in oppositional ratings between tapes. Thus, teachers of- ten erroneously rated the child exhibiting only oppositional behaviors as having substantial symp- toms of ADHD.

These findings differ somewhat from those reported by Abikoff etal. (1993). In this study, differences in the ratings of hyperactive and inattentive behaviors be- tween tapes were small but statistically significant. However, in the Abikoff et al. (1993) study, nonsignificant differences in ratings of the two tapes were generally found, suggesting a stronger negative halo effect. One possible explanation for these discrep- ant results is that in recent years teachers have become more aware that there are different types of childhood behavior disorders. The data from the Abikoff et al. (1993) study were callected almost 8 years ago, and in the meantime teachers may have had greater access to information about child behavior problems, which may have attenuated the bias. Similsuly, in the past decade a strong shift in educational philosophy with regard to

mainstreaming has occurred, resulting in more children with various problems in regular education classrooms (Stainback & Stainback, 1995; Westwood, 1997). This shift has given a broad spectrum of teachers greater ex- posure to children with ADHD, which may have de- creased the strength of the unidirectional bias in this study.

We also found strong support for the second hypoth- esis. As expected, teachers evidenced less bias when completing a well-operationalized rating scale, such as the SNAP-IV, and exhibited greater bias on the less well-operationalized CTRS-28. More fine-grained analyses indicated that individual items reflecting both ADHD and ODD behaviors from the CTRS-28 Hyperkinesis scale (e.g., "disturbs other children") ap- peared to contribute to these inaccuracies. However, when items reflecting only ADHD or ODD symptomatology were utilized, teachers accurately en- dorsed greater inattention and hyperactivity for the child in the ADHD versus ODD tape. Note that Abikoff et al. (1993) still foundevidence of a unidirectional bias when utilizing ascalecontaining only hyperactivity and inattention items, Because that study found generally stronger evidence of a bias than the present one, purer item content may not have played asr significant arolein attenuating this bias. Given the less extreme bias found in the current study, purer items have appeamd to redu~e this problem.

In addition, teachers were not able to differentiate the two symptomatic children on the CTRS-28 Inat- tentive-Passive scale, suggesting that this measure of inattention lacks discriminant validity. Other research- ershave also found that the Conners Teacher Rating Scales lacked specificity and were not able to identify those children who had attention deficits per se from those who exhibit hyperactivity or aggression (Ullmann, Sleator, & Sprague, 1985). However, when we excluded items that reflected social skills or were global in nature, teachers correctly rated the target child with ADHD versus ODD as more inattentive. This is consistent with our premise that item$ reflect- ing specific behaviors, rather than global characteris- tics, improve the accuracy of teachers' ratings.

Finally, we found no support for our third hypoth- esis. Having greater knowledge and experience with ADHD was not generally associated with more accu- rate ratings. The lack of a relation betwean knowl- edge of ADHD and teacher ratings should be inter- preted with caution, because the knowledge measure we used had significant limitations. Fir& item-total correlations were low, suggesting that individual items were not tapping the same central construct. Second, the majority of guestbns focused an factual information about the prevalence, etidagy, treatment, and prognosis of the disordw, as opposed to knowl- edge about relevant symptoms, T@aohers7 under- standing of the specific behaviors that do anid do not

Dow

nloa

ded

by [

Ada

ms

Stat

e U

nive

rsity

] at

15:

18 2

4 N

ovem

ber

2014

TEACHEXIS' RATINGS OF ADHD AND ODD BEHAVIORS

constitute ADHD, rather than general facts, may be a better indicator of their ability to recognize and rate ADHD symptoms accurately. However, the design of this stu4y precluded administration of this type of measure because the videotaped stimuli would have influenced those responses.

We also foundlittle evidence that greater exposure to educational materials about ADHD was related to more accurate ratings. In fact, our findings were in the oppo- site direction, indicating that teachers exposed to more information about ADHD tended to rate the child with ODD as bavingmoreinattentionand hyperactivity. This finding held true regardless of the operationalization of theratings scales. Perhaps teachers withmoreeducation relevant to ADHD recognize that the behavior of chil- dren with ODD is problematic but overgeneralize all be- havior disorders to the one they "know bestY-ADHD. Many h v e implied that the extensive coverage of ADHD by the media and other sources has led to an overdiagnosis of the disorder (Hancock, 1996). Thus, children with diverse problems (e.g., learning disorders, aggression) may often be incorrectly labeled ,as ADHD.

Sirnildy, more contact with experts on ADHD and children with this disorder was not related to the accu- racy of teachers' ratings of the ODD tape. It is possible that exposure to professionals and parents does not lead to bemr differentiation of specific childhood disorders. For example, contact with children with ADHD and their caretakers may sengitize teachers to the key symp- toms of the disorder, but it may also result in greater ac- ceptance of children's behavior problems, and hence, less extreme ratings. In addition, it is widely acknowl- edged that health care professionals often hold different views with regard to diagnosis and appropriate treat- ment. Therefore, contact with these professionals may not result in better-informed teachers.

Two limitations of this study are worth noting. First, it is not clear how generalizable findings based on rat- ings of videotaped segments are to the more typical classroom setting. Teachers can rarely focus on one stu- dent for extended period of time, as they did when viewing these videotapes. Moreover, teachers typically evaluate their students in a variety of situations, from structured academic activities to less organized free play. Examining teachers' ratings with children in ac- tual settings, as their behavior varies across situations, would increase the generalizability of our findings. No- tably, however, our results converge with those investi- gators who found a unidirectional bias in a classroom setting (Schachar et al., 1986). Second, in this study we did not investigate teachers' knowledge of ODD, an- other factor that may have influenced their ratings. Greater knowledge of ODD could have increased the accuracy of their ratings of children displaying only oppositioinal behaviors.

Two important implications may be drawn from this investigation. First, clinicians and researchers

would be advised when selecting rating scales to uti- lize well-operationalized questionnaires, which appear to facilitate differentiation of childhood behavior dis- orders. Mote that Conners (1997) has recently revised both the parent and teacher questionnaires to reflect the better-operationalized DSM-N criteria. R~esearchers should further investigate whether improving the operationalization of questionnaires increases their discriminant validity, not only in the domain of child- hood disruptive behaviors but also across all areas of child and adult psychopathology.

Second, factual knowledge and professioinal experi- ence with ADMD do not appear to be sufficient for im- proving teachers' recognition of ADHD and ODD symptomatology. This is an important issue because teachers are often the earliest and most frequent source for identifying children with behavior problems. Two important steps should be taken. First, a psychometrically sound measure of teachers' knowl- edge of A O H D that includes questions relevant to the discrimination of different childhood disord~rs should be developed. Second, through college and continuing education courses teachers should be trained specifi- cally to recognize the symptoms of ADHD, ODD, and other psychol~ogical disorders (Abikoff et al., 1993). Other factors besides knowledge and experience should also be considered, such as teachers' personali- ties and attitudes toward these students. For example, teachers' preference for structured versus unstructured classrooms may influence their ratings of chiildren with hyperactivity (Eddowes, Aldridge, & (Julpepper, 1994). Given the national move toward the mainstreaming of children with a variety crf disabili- ties, it is vitally important that teachers have the knowledge and experience relevant to facing these new challenges.

References

Abikoff, H., Courtney, M., Pelham, W. E., & Kopl~ewicz, H. S. (1993). Teachers' rating of disruptive behaviors: The influence of halo effects. Journal of Abnormal Child Psychology, 21, 519-533.

Abikoff, H., Gittelman-Klein, R., L Klein, D. F. (1977). Validation of a classroom observation code for hyperactive children. Jour- nal of Consulting and Clinical Psychology, 4,772-783.

Atkins,M. S., Pelham, W. E., &Licht,M. H. (1985). Acomparisonof objective classroom measures and teacher ratings; of attention deficit disorder. Journal of Abnormal Child Psychology, 13, 155-167.

Atkins, M. S., Pelham, W. E., & Licht, M. H. (1989). The differential validity of teacher ratings of inattentiontovemtivity and ag- gression. JoumalofAbnormal ChildPsychology, 17, 423--435.

Bender, W. N., Vail, C. O., &Scott, I<. (1995). Teachers' attitudes to- ward increased mainstreaming: Implementing effective instruc- tion for students with learning disabilities. Journar' of Learning Disabilities, 28, 87-94.

Bennett, D. S., Power, T. J., Rostain, A. L., &Cam, D. E. (1996). Par- ent aciceptability and feasibility of ADHD interventions: As-

Dow

nloa

ded

by [

Ada

ms

Stat

e U

nive

rsity

] at

15:

18 2

4 N

ovem

ber

2014

STEVENS, QUITTNER, & ABIKOFF

sessment, correlates, and predictive validity. Journal of Pediat- ric Psychology, 21,643-657.

Brown, R. T. (1985). The validity of teacherratings in differentiating between two subgroups of attention deficit disordered children with or without hyperactivity. Educational and Psychological Measurement, 45,661-669.

Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159.

Conners, C. K. (1969). A teacher rating scale for use in drug studies with children. American Journal of Psychiatry, 126, 884-888.

Conners, C. K. (1990). Conners Rating Scalesmanual. Toronto, On- tario, Canada: Multi-Health Systems, Inc.

Comers, C. K. (1997). Conners' Ratings Scales-Revised North Tonawanda, NY: Multi-health Systems, hc .

Dalley, M. B., Bolocofsky, D. N., & Karlin, N. J. (1994). Teacher-ratings and self-ratings of social competency in adoles- cents with low- and high-depressive symptoms. Journal of Ab- normal Child Psychology, 22,477-485.

DuPual, G. J.,Guevremont, D. C., & Barkley, R. A. (1991). Attention deficit-hyperactivity disorder in adolescence: Critical assess- ment parameters. Clinical Psychology Review, 11, 231-245.

Eddowes, E. A., Aldridge, J., & Culpepper, S. (1994). Primary teach- ers' classroom practices and their perceptions of children's at- tention problems. Perceptual and Motor Skills, 79, 787-790.

Ellen, Arthur S. (1989). Discriminant validity of teacher ratings for n d , learningdisabled, and emotionally-handicapped boys. Journal qfScho01 Psychology, 27,15-25.

Goyette, C. H., Conners, C. IS., & Ulrich, R. F. (1978). Normative data on the Revised Conners Parent and Teacher Rating Scales. Journal ofAbnorma1 Child Psychology, 6,221-236.

Greene, R. W. (1995). Students with ADHD in school classrooms: Teacher factors related to compatibility, assessment, and inter- vention. School Psychology Review, 24, 81-93.

Hancock, L. (1996, March 18). Mother's Little helper. Newsweek, I2?,51-56.

Hanushek, E. A., &Jackson, J. E. (1977). Statistical methods for so- cial scientists. New York: Academic.

Loney, J., & Milich, R. (1982). Hyperactivity, inattention, and ag- gression in clinical practice. In M. Wolraich & D. K. Routh (Eds.), Advances in developmental and behavioral pediatrics (Vol. 12, pp. 221-251). New York: Plenum.

Mann, E. M., Ikeda, Y., Mueller, C. W., Takahashi, A., Tao, K. T., Hnmris, E., Li, B. L., &Chin, D. (1992). Cross-cultural differ- ences in rating hyperactive-disruptive behaviors in children. American Journal of Psychiatry, 149, 153S1542.

Milich, R., & Fitzgerald, G. (1985). Validation of inattentionloverac- tivity and aggression ratings with classroom observations. Jour- nal of Clinical and Consulting Psychology, 53, 139-140.

Mintz, L. I., & Collins, B. (1985). Qualitative influences on the per- ception of movement: An experimental study. Journal of Ab- normal Child Psychology, 13, 143-153.

Sandoval, J. (1981). Format effects in two teacher rating scales of hy- peractivity. Journal ofAbnorma1 Child Psychology, 9,203-21 8.

Schachar, R., Sandberg, S., &Rutter, M. (1986). Agreement between teachers' ratings and observations of hyperactivity, inattentive- ness, and defiance. Journal ofAbnorma1 Child Psychology, 14, 331-345.

Shelton, T. L., & Barkley, R. A. (1995). The assessment and treat- ment of attention-deficit/hyperactivity disorder in children. In M. C. Roberts (Ed.), Handbook of pediatric psychology (pp. 633-654). New York: Guilford.

Stainback, W., & Stainback, S. (1995). Contemplating inclusive edn- cation from a historical perspective. In R. Villa & J. Thousand (Eds.), Creating an inclusive school (chap. 2). Alexandria, VA: Association for Supervision and Curriculum Development.

Stevens, G. (1980). Bias in attributivns of positive and negative be- havior in children by school psychologists, parents, and teach- ers. Perceptual and Motor Skills, 50, 1283-1290.

Ullmann, R. K., Sleator, E. K., & Sprague, R. L. (1985). A change in mind: The Conners Abbreviated Rating Scales reconsidered. Journal of Abnormal Child Psychology, 13,553-565.

Vaughn, M. L., Riccio, C. A., Hynd, G. W., &Hall, J. (1997). Diag- nosing ADHD (predominately Inattentive and Combined Type subtypes): Discriminant validity of the Behavior Assessment System for Children and the Achenbach Parent and Teacher Rating Scales. Journal of Clinical Child Psychology, 26, 349-357.

Westwood, P. (1997). Commonsense methods for children with spe- cial needs. New York: Routledge.

Wilson, C. C. (1980). Behavioral assessment: Questionnaires. School Psychology Review, 9, 58-66.

Manuscript received November 11,1997 Final revision received April 13, 1998

Dow

nloa

ded

by [

Ada

ms

Stat

e U

nive

rsity

] at

15:

18 2

4 N

ovem

ber

2014