Journal of Psychoeducational Assessment2014, Vol 32(1) 88 –92

© 2014 SAGE PublicationsReprints and permissions:

sagepub.com/journalsPermissions.nav jpa.sagepub.com

Test Review

Test Review

Lord, C., Luyster, R. J., Gotham, K., & Guthrie, W. (2012). Autism Diagnostic Observation Schedule, Second Edition (ADOS-2) Manual (Part II): Toddler Module. Torrance, CA: Western Psychological Services, 2012.Lord, C., Rutter, M., DiLavore, P. C., Risi, S., Gotham, K., & Bishop, S. Autism Diagnostic Observation Schedule, Second Edition. Torrance, CA: Western Psychological Services, 2012.

Reviewed by: Adam McCrimmon, University of Calgary, Calgary, AB, Canada; Kristin Rostad, Alberta Children’s Hospital, Calgary, AB, CanadaDOI: 10.1177/0734282913490916

Test Description

The Autism Diagnostic Observation Schedule, Second Edition (ADOS-2; Lord, Luyster, Gotham, & Guthrie, 2012; Lord, Rutter et al., 2012) is a newly updated, semistructured, standardized mea-sure of communication, social interaction, play/imagination, and restricted and/or repetitive behav-iors published by Western Psychological Services. Used in clinical and research settings, the ADOS-2 is often referred to as the “gold standard” measure of observational assessment for autism spectrum disorder (ASD; Kanne, Randolph, & Farmer, 2008). It is a Level C measure that can be administered and interpreted by appropriately credentialed professionals from medicine, psychol-ogy, or a related discipline in approximately 40 to 60 min, depending on the module selected and the specific behavior demonstrated by the examinee. Extensive prior experience with individuals with ASD is strongly recommended, and administrators must have completed clinical training pro-vided by an independent trainer or through the test publisher.

The ADOS-2 kit is a large container consisting of most of the required toy items and materials (some materials must be supplied and replenished by the examiner), and 10 protocols for each of the 5 modules. The manual is expansive and well laid out, beginning with an overview of the measure, guidelines for module selection, administration and coding procedures, instructions for each module, and case examples to assist with interpretation.

Measure Development

The ADOS-2 was designed to improve the accuracy and effectiveness of the diagnostic algorithms of the original ADOS (Lord, Rutter, DiLavoire, & Risi, 1999). Specific changes include revision and expansion of the diagnostic algorithms used in Modules 1 to 3 and the addition of a Toddler Module. A new comparison score in Modules 1 through 3 was added to allow for comparability over time. Minor modifications to the administration instructions for some tasks were also made; however, experienced ADOS administrators will notice that the majority of the administrative procedures remain identical to the previous version of the measure. The protocols have also been updated to include an enhanced description of administration and the behavior(s) being observed.

Administration and Scoring

General Description and Administration

The ADOS-2 is designed to be individually administered by one examiner. It consists of five modules, the selection of which is based primarily on the examinee’s expressive language level

490916 JPA32110.1177/0734282913490916Journal of Psychoeducational Assessment X(X)Test Reviewresearch-article2013

by guest on November 1, 2015jpa.sagepub.comDownloaded from

Test Review 89

as determined by formal language assessment and/or observation by the examiner, and second-arily on their chronological age and/or relevance of tasks to the examinee’s interests and abilities. Each module consists of specified tasks designed to elicit communicative, social, and behavioral characteristics consistent with ASD. The format of the ADOS-2 differs from the majority of other standardized assessment measures in that the assessment is dynamic and examiners should not constantly refer to the manual during administration so that social interaction with the examinee is natural and maintained.

The specific activities and subsequent behavioral coding within each module differ; however, there is some overlap and the general administration and scoring principles remain consistent. Administration necessitates delivery of specific tasks in a semistructured manner whereby exam-iners deliberately modify their behavior according to a hierarchy of structured and unstructured social presses and observe the examinee’s behaviors in response. Administration of tasks is flex-ible and does not have to follow a set sequence. Behavioral and language observations should be noted throughout the administration, particularly regarding social and communicative behaviors related to ASD diagnostic criteria. The tasks are standardized in that examiners are provided instructions regarding task administration and the hierarchy of behaviors to be employed. The tasks are designed to provide structure to the interactions between the examiner and examinee and should not be robotically administered without regard for the nature of the interactions. Coding should occur directly following the administration using the behavioral notes taken by the examiner. A parent/caregiver who is familiar with the child must be present in the room dur-ing administration of the Toddler Module, Module 1, and Module 2.

Toddler module. The Toddler Module is designed for children 12 to 30 months of age who do not consistently use phrase speech (i.e., flexible nonechoed, three-word utterances that include a verb). It consists of 11 primary activities and 4 secondary tasks that inform 41 coded items pertaining to Social Affect (SA) and Restricted and Repetitive Behavior (RRB). Two scoring algorithms were created: All Younger/Older with Few to No Words, and Older with Some Words.

Module 1. Module 1 is designed for children 31 months of age or older who do not consis-tently use phrase speech. It consists of 10 activities that inform 34 coded items pertaining to SA and RRB. There are now two algorithms: Few to No Words, and Some Words.

Module 2. Module 2 is designed for children of any age who use phrase speech but who have not developed verbally fluent speech (i.e., expressive language skills of a typical 4-year-old). It consists of 14 activities that inform 29 coded items pertaining to SA and RRB. Two algorithms now exist: Younger than 5 years, and Aged 5 Years or Older.

Module 3. Module 3 is suited for verbally fluent children and adolescents for whom action-figure toy play is age appropriate (approximately 16 years or younger). It consists of 14 activities that inform 29 coded items pertaining to SA and RRB.

Module 4. Module 4 is suited for verbally fluent older adolescents and adults. It consists of 10 to 15 activities that inform 32 coded items. As Module 4 was not revised with the ADOS-2, its coded items are structured for separate communication, reciprocal social interaction, and RRB domains.

Scoring

Behavioral and language indicators noted during the assessment inform specific coded items for each module pertaining to (a) Language and communication, (b) reciprocal social interaction,

by guest on November 1, 2015jpa.sagepub.comDownloaded from

90 Journal of Psychoeducational Assessment 32(1)

(c) play and imagination, (d) stereotyped behaviors and restricted interests, and (e) other behav-iors. Coding is a primary activity of the ADOS-2 and is the basis for clinical decision making when the codes are transferred to the algorithm and summed. As such, familiarity with coding concepts and conventions are integral to effective and appropriate ADOS-2 administration. Specifically, a code of 0 indicates no evidence of abnormal behavior as specified by the code, 1 indicates mildly abnormal or slightly unusual behavior, 2 indicates definite abnormality of behavior, and 3 indicates markedly abnormal behavior. Other codes (4, 7, 8, and 9) are used to identify specific language ability, atypical behaviors that are present but not of the specified type, or an item that cannot be scored for a number of possible reasons. Additional information pertain-ing to coding can be found in the manual and through specialized training programs offered by the test publisher or test developers.

Technical Adequacy

Standardization

Given the equivalence of ADOS and ADOS-2 items, the authors provide psychometric data on the ADOS validation sample (98 children; 29 Module 1s, 23 Module 2s, 26 Module 3s, and 20 Module 4s) as well as an ADOS-2 extended validation sample (1,574 children; 847 Module 1s, 329 Module 2s, and 398 Module 3s) and an ADOS-2 replication sample (1,282 children; 620 Module 1s, 205 Module 2s, and 457 Module 3s). Across all of these samples, the majority of participants received a diagnosis of Autism (45%-76%), followed by nonautism ASD (7%-29%), and nonspectrum (17%-26%). Participants were predominantly male (57%-86%) and Caucasian (71%-91%). For the Toddler Module validation sample, 182 children were seen for a total of 360 assessments (i.e., most children had two assessments). The majority of the participants were male (76%) and Caucasian (80%). Of the 182 children, 25% had an ASD diagnosis, 20% had a non-spectrum disorder, and 55% were typically developing. The majority of participants in the ADOS, ADOS-2 extended validation, and Toddler Module validation samples were obtained through clinics in Illinois and Michigan. The ADOS-2 replication sample data were obtained from 10 sites across the United States.

Item Selection and Algorithm Development

Items (codes) were selected based on their ability to differentiate between clinical diagnoses (Lord, Rutter et al., 2012). Theoretical importance was also considered, such as the inclusion of items and behaviors considered to be clinically meaningful (e.g., self-injurious behaviors). The ADOS-2 standardization sample was divided into five developmental cells that correspond to the five new ADOS-2 algorithms for Modules 1 through 3 to determine which codes to include. Codes were considered based on their ability to distinguish spectrum from nonspectrum diagno-ses (e.g., no more than 20% of autism cases could have an assigned rating of 0 on a code) as well as for their comparability across modules. An exploratory multifactor item-response analysis was used to organize the codes into new domains: SA and RRB. A confirmatory factor analysis veri-fied the goodness of fit, and a logistic regression supported the predictive value of scores from each of the two new factors. Revised algorithms were created based on verbal skills for Module 1 and chronological age for Module 2.

Reliability

Internal consistency. Cronbach’s α values for Modules 1 through 3 were high for the SA domain (.87-.92) and moderate for the RRB domain (.51-.66). The same pattern was true for the Toddler

by guest on November 1, 2015jpa.sagepub.comDownloaded from

Test Review 91

Module, with the highest alphas in the SA domain (.88-.90) as compared to the RRB domain (.50). For Module 4, Cronbach’s αs exceeded .75 for the communication domain, .85 for the social interaction domain, and .47 for the RRB domain.

Test–retest reliability. Test–retest reliability was calculated for Modules 1 through 3 using a sample of 75 participants who were administered the same module twice within an average of 10 months. Analyses indicated that SA, RRB, and overall total scores had correlations ranging from .68 to .92. Test–retest reliability for the Toddler Module was calculated for 39 children who received two administrations within a 2-month span. The SA, RRB, and overall total correlations ranged from .64 to .88.

Interrater reliability. For the ADOS validation, ADOS-2 extended validation, and Toddler Module validation samples, examiners had to achieve research reliability (i.e., 80% or greater exact agreement with other reliable raters). Across the five modules, interrater reliability for item coding was 71% or higher with the majority of mean weighted kappas exceeding .60. Correla-tions were calculated for SA, RRB, and overall total and ranged from .79 to .98 across the five modules. Agreement in diagnostic classification ranged from 92% to 98% in Modules 1 through 3 and from 87% to 97% in the Toddler Module (Module 4 not calculated).

Validity

Content and construct validity. Codes were selected for inclusion based on their unique con-tribution to the original ADOS algorithm (i.e., no interitem correlations greater than .70). A two-factor solution was identified based on exploratory and confirmatory factor analysis: SA, consisting of the formerly known reciprocal social interaction and communication items, and RRB. For Modules 1 through 3, logistic regressions indicated that both the SA and RRB domains made significant independent contributions to the prediction of diagnosis. However, the overall total score produced the highest predictive value, supporting the use of this score in diagnostic decisions. Codes for the Toddler Module were selected based on their differential distributions across diagnostic groups as well as their relatively low correlations with other items, chronologi-cal age, and measured intelligence. Exploratory factor analysis revealed that once the codes were split into All Younger/Older with Few to No Words and Older with Some Words, chronological and verbal mental age no longer significantly contributed to any factor, supporting the develop-ment of these two separate algorithms.

According to item–total correlations, almost all of the items across the five algorithms used for Modules 1 through 3 and in the Toddler Module correlated more strongly with their assigned domain than with each other (and those that did not typically differed by a correlational value of .05 or less). Item and domain correlations for Module 4 ranged from .50 to .88.

Predictive validity. Receiver operating characteristic curves evaluated the predictive validity of the ADOS-2 to identify individuals on the autism spectrum accurately. When comparing the ADOS to the ADOS-2, sensitivity and specificity values were largely comparable or improved with the new algorithms, particularly in Modules 1 and 2 where dual algorithms were introduced based on expressive language and chronological age (sensitivity ranging from 60%-95% and specificity ranging from 75%-100%). Sensitivity and specificity for the Toddler Module were at 86% or above.

Commentary and Recommendations

The ADOS-2 has several features that support its continued recognition as a “gold standard” assessment measure for ASD. The manual is more expansive and detailed than the previous

by guest on November 1, 2015jpa.sagepub.comDownloaded from

92 Journal of Psychoeducational Assessment 32(1)

measure. The new protocols allow for additional space for behavioral notes and include additional information on hierarchies of examiner prompts and important behaviors to be observed during specific tasks. The behavioral descriptions attached to codes have been clarified. A new compari-son score has been included. The revised algorithms, including the splitting of algorithms based on language ability or age in Modules 1-3 and the re-conceptualization from three domains into two (SA and RRB), allow for enhanced diagnostic clarity as coding is less affected by language impairment and cognitive functioning and the factor structure is in alignment with the forthcom-ing DSM-5 framework. As a result, the sensitivity and specificity of the measure has been improved. Finally, the inclusion of the Toddler Module allows for administration to a wider client population, thus enhancing the measure’s effectiveness as a diagnostic tool.

Despite the positive attributes, the ADOS-2 also contains some limitations. Module 4 was not updated. Although the authors indicate that a lack of sample size was responsible, it remains problematic for researchers and clinicians who may use that module. The training videos offered by the test publisher have not been updated, with the exception of the Toddler Module. As subtle nuances and administrative procedures have changed from the ADOS, updated training tools would be beneficial. The standardization sample was primarily male, Caucasian, and American, which may limit the generalizability of the results to a broader population due to such factors as gender or culturally appropriate social behaviors (e.g., eye contact). The RRB domain had lower internal consistency and test–retest values; however, the authors explained that this is likely due to the limited opportunity to observe these behaviors during administration, as they are lower base rate behaviors. Finally, there is no appropriate module for older individuals who are lower functioning.

Overall, the ADOS-2 is an effective clinical and research instrument for the ASD population. Although the measure does present with some limitations, it remains a primary diagnostic tool for this population and will enhance clinical and research utility.

Declaration of Conflicting Interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publi-cation of this article.

Funding

The author received no financial support for the research, authorship, and/or publication of this article.

References

Kanne, S. M., Randolph, J. K., & Farmer, J. E. (2008). Diagnostic and assessment findings: A bridge to aca-demic planning for children with autism spectrum disorders. Neuropsychology Review, 18(4), 367-384.

Lord, C., Luyster, R. J., Gotham, K., & Guthrie, W. (2012). Autism diagnostic observation schedule, second edition (ADOS-2) manual (Part II): Toddler module. Torrance, CA: Western Psychological Services.

Lord, C., Rutter, M., DiLavore, P. C., & Risi, S. (1999). Autism diagnostic observation schedule: Manual. Los Angeles, CA: Western Psychological Services.

Lord, C., Rutter, M., DiLavore, P. C., Risi, S., Gotham, K., & Bishop, S. (2012). Autism diagnostic observa-tion schedule, second edition. Torrance, CA: Western Psychological Services.

by guest on November 1, 2015jpa.sagepub.comDownloaded from