The Pennsylvania State University

The Graduate School

College of Education

A STRUCTURAL ANALYSIS OF THE SOCIAL SKILLS IMPROVEMENT SYSTEM

RATING SCALES, PARENT FORM: MEASUREMENT INVARIANCE ACROSS RACE

AND LANGUAGE FORMAT

A Dissertation in

School Psychology

by

Brian P. Schneider

2012

Submitted in Partial Fulfillment

of the Requirements

for the Degree of

Doctor of Philosophy

December 2012

ii

The dissertation of Brian Schneider was reviewed and approved* by the following:

James C. DiPerna

Associate Professor of Education

Dissertation Advisor

Chair of Committee

Professor in Charge of the Program of School Psychology

Robert L. Hale

Professor of Education

Jonna M. Kulikowich

Professor of Education

Keith B. Wilson

Special Member

Professor of Education

Southern Illinois University

*Signatures are on file in the Graduate School

iii

ABSTRACT

The purpose of the current study was to evaluate aspects of structural validity for the

Social Skills Improvement System Rating Scales, Parent Form (SSIS-PF). Data were obtained

from the SSIS-PF standardization sample. Confirmatory factor analysis (CFA) was applied to

examine the instrument’s first-order and higher-order measurement structures. Resulting baseline

measurement models were subsequently analyzed for invariance across two variables.

Specifically, measurement invariance was examined as a function of race/ethnicity using

subsamples of African American, Latino, and Caucasian children. Invariance then was examined

as a function of the language in which rating scales were written (i.e., English or Spanish). For

both analyses, multi-sample CFA procedures were used to examine invariance at the configural,

metric, and structural levels. Initial analyses provided support for the first-order measurement

structure of the SSIS-PF, though there was some evidence of a lack of discriminant validity

between select subscales (Cooperation and Responsibility). The instrument’s higher-order

measurement structure showed evidence of reduced fit to standardization data relative to the first-

order model. Follow-up analysis of the higher-order measurement structure of the SSIS-PF was

conducted, and an alternative structure was identified. Results of the invariance analyses with

first-order baseline models suggested that the SSIS-PF demonstrates configural, metric, and

structural invariance as a function of race/ethnicity. Configural and metric invariance were

supported in the language format analysis, but structural invariance was not observed across

English and Spanish language groups. Results are discussed in reference to implications for the

use of the SSIS-PF as well as broader considerations for cross-cultural social skills assessments.

iv

TABLE OF CONTENTS

LIST OF FIGURES ................................................................................................................. vi

LIST OF TABLES ................................................................................................................... vii

ACKNOWLEDGEMENTS ..................................................................................................... ix

Chapter 1 Introduction ............................................................................................................ 1

The Need for Cross-cultural Social Skills Research ........................................................ 1 Demographic Trends: The U.S. Hispanic/Latino Population........................................... 2

Goals and Objectives ....................................................................................................... 4

Chapter 2 Literature Review ................................................................................................... 5

Perspectives on Social Competence ................................................................................. 6 Divergent Perspectives on Social Competence ........................................................ 7

Joining Perspectives: Comprehensive Models of Social Competence ..................... 7

The Issue of Context ................................................................................................ 8 Emphasizing the Social Skills Dimension of Social Competence ................................... 9

Defining Social Skills............................................................................................... 9

Toward a Taxonomy of Social Skills ....................................................................... 11 Social Skills in Context ............................................................................................ 13

Cultural Influences on Social Skill Development ............................................................ 15

Traditional Latino Parenting Practices ..................................................................... 16

Within-group Differences: The Effects of Acculturation ......................................... 17

Cultural Interaction and the Effects of Schooling .................................................... 19

Social Skills Assessment .................................................................................................. 21

Cross-cultural Social Skills Assessment .................................................................. 23

Social Skills Improvement System .................................................................................. 23

Social Skills Rating System ..................................................................................... 25

Structural Investigations of the SSRS ...................................................................... 26

Cross-cultural Applications of the SSRS ................................................................. 27

Research Questions .......................................................................................................... 28

Chapter 3 Method ................................................................................................................... 30

Participants ....................................................................................................................... 30

Measures .......................................................................................................................... 30

SSIS Rating Scales ...................................................................................................... 30

SSIS Rating Scales, Spanish Format .......................................................................... 33

Procedures ........................................................................................................................ 34

Design and Analysis ......................................................................................................... 34

Structural Analyses ..................................................................................................... 34

Parceling Technique ................................................................................................... 35

Assessing Model Fit .................................................................................................... 43

Analysis of Invariance ................................................................................................ 44

v

Chapter 4 Results .................................................................................................................... 46

Preliminary Analyses and Data Preparation .................................................................... 46

Analysis of SSIS-PF Invariance by Race/Ethnicity ......................................................... 59

Analysis of SSIS-PF Higher-Order Factor Structure ....................................................... 71 Analysis of SSIS-PF Invariance by Language Format..................................................... 80 Analysis of the Higher-Order Factor Structure for the SSIS-PF English Language

Format and Spanish Language Format ..................................................................... 86

Chapter 5 Discussion .............................................................................................................. 90

Overview .......................................................................................................................... 90

Primary Findings .............................................................................................................. 90

Factor Structure of the SSIS-PF ............................................................................... 90

SSIS-PF Measurement Invariance: Race/Ethnicity .................................................. 92

SSIS-PF Measurement Invariance: Language Format ............................................. 93

Interpretation of Primary Findings in the Context of Prior Research .............................. 94

SSIS-PF Structure and the Social Skills Construct .................................................. 94

Measuring Social Skills across Cultural Groups ...................................................... 96

Limitations and Future Directions ................................................................................... 98

Data Limitations ....................................................................................................... 98

Design Limitations ................................................................................................... 100

Implications for the Use of the SSIS-PF in Research and Practice .................................. 103

Implicaitons for Cross-Cultural Social Skills Assessment ............................................... 104

Conclusions ...................................................................................................................... 105

References ................................................................................................................................ 107

vi

LIST OF FIGURES

Figure 1: First-order baseline model for the analysis of invariance by race/ethnicity,

Model 1.1. ........................................................................................................................ 63

Figure 2: Higher-order basleline model for the analysis of invariance by race/ethnicity,

Model 1.2. ........................................................................................................................ 64

Figure 3: Revised baseline model for the analysis of invariance by race/ethnicity,

Model 1.3. ........................................................................................................................ 68

Figure 4: Modified SSIS-PF higher-order factor structure, Model 2.2 ................................... 74

Figure 5: Modified SSIS-PF higher-order factor structure, Model 2.3 ................................... 75

Figure 6: Modified SSIS-PF higher-order factor structure, Model 2.4. .................................. 77

Figure 7: Modified SSIS-PF higher-order factor structure, Model 2.5 ................................... 78

Figure 8: Revised baseline model for the analysis of invariance by language format,

Model 3.2 ......................................................................................................................... 82

Figure 9: Revised SSIS-PF higher-order factor structure, Model 4.1, fit to data from

English langauge-format subsample ................................................................................ 88

Figure 10: Revised SSIS-PF higher-order factor structure, Model 4.1, fit to data from

Spanish language format subsample ................................................................................ 89

vii

LIST OF TABLES

Table 1: Frequency Distributions for Demographic Variables from the SSIS Rating

Scales Standardization Sample that were Included in the Current Study ........................ 31

Table 2: Summary of Item-Parcel Assignment for Analysis of SSIS-PF Measurement

Invariance by Race/Ethnicity and Language Format ....................................................... 36

Table 3: Means and Standard Deviations for Item-Parcels and Control Variables used in

Analysis of Invariance by Race/Ethnicity ........................................................................ 47

Table 4: Means and Standard Deviations for Item-Parcels and Control Variables used in

Analysis of Invariance by Language Format ................................................................... 48

Table 5: Bivariate Correlations of SSIS Rating Scale Latent Factor Indicators and Control

Variables for Full Race/Ethnicity Invariance Sample and African American

Subsample ........................................................................................................................ 49

Table 6: Bivariate Correlations of SSIS Rating Scale Latent Factor Indicators and

Control Variables for Caucasian and Latino Subsamples in Race/Ethnicity

Invariance Analysis. ......................................................................................................... 51

Table 7: Bivariate Correlations of SSIS Rating Scale Latent Factor Indicators and Control

Variables for Full Language Format Invariance Sample. ................................................ 53

Table 8: Bivariate Correlations of SSIS Rating Scale Latent Factor Indicators and

Control Variables for English Language Format and Spanish Language Format

Subsamples in Language Format Invariance Analysis ................................................... 55

Table 9: Internal Consistency Coefficients and Intercorrelations for SSIS-PF Social Skills

Subscales based on Race/Ethnicity Item Parcels and Manual Reported Item-Level

Data .................................................................................................................................. 57

Table 10: Internal Consistency Coefficients and Intercorrelations for SSIS-PF Social

Skills Subscales based on Language Format Item Parcels and Manual Reported

Item-Level Data .............................................................................................................. 58

Table 11: Global Fit Statistics for SSIS-PF Baseline Models fitted to Samples with and

without Multivariate and Univariate Outliers .................................................................. 60

Table 12: Global Fit Statistics for SSIS-PF Measurement Model at Successive Stages of

Invariance Analysis by Race/Ethnicity ............................................................................ 62

Table 13: Estimated Latent Factor Correlations for SSIS-PF 7-Factor Baseline

Measurement Model by Race/Ethnicity Group ................................................................ 66

viii

Table 14: Unstandardized and Standardized Factor Loadings by Race/Ethnicity for

Configural Invariance Model ........................................................................................... 70

Table 15: Global Fit Statistics for SSIS-PF Higher-Order Measurement Model at

Successive Stages of Post-Hoc Model Fitting ................................................................. 72

Table 16: Global Fit Statistics for SSIS-PF Measurement Model at Successive Stages of

Invariance Analysis by Language Format ........................................................................ 84

Table 17: Unstandardized and Standardized Factor Loadings by Language Format for

Configural Invariance Model ........................................................................................... 85

Table 18: Global Fit Statistics for Proposed SSIS-PF Higher-Order Measurement Model

with Language Format Samples ....................................................................................... 87

ix

ACKNOWLEDGEMENTS

This project would not have been possible without the support of many individuals. In

particular, I would like to thank my dissertation advisor and committee chair, Dr. James DiPerna.

Your guidance, advice, and encouragement have been invaluable. I would also like to thank the

other members of my committee, Dr. Robert Hale, Dr. Jonna Kulikowich, and Dr. Keith Wilson,

for your expertise, thoughtful consideration, and constructive feedback. In addition, special

thanks to Ms. Becky Holter and Dr. Shirley Woika for all that you do for the Penn State School

Psychology program and its students.

I would like to express my sincerest gratitude and appreciation to Dr. Frank Gresham and

Dr. Stephen Elliott for supporting my request to use the SSIS standardization data for this project.

Additional thanks to Pearson, Inc. for approving that request and preparing the SSIS Rating

Scales standardization dataset for use in this project. It is my hope that the current research will

serve to promote further investigations and applications of this already widely used assessment

instrument.

Finally, heartfelt thanks to my parents, Greg and Nancy Schneider, for your constant love

and encouragement; to Chris, Steve, and Melissa, for being there, always; and to all of the friends

and family members who have helped and supported me along the way.

1

Chapter 1

Introduction

Outcomes associated with children and adolescents’ development and exhibition of social

skills have been well documented. Deficits in social competence and social skills have been

linked with a number of negative outcomes including elevated rates of problem behavior (Lane,

Carter, & Piers, 2006; Skoulos & Shick Tryon, 2007; Webster-Stratton, Reid, & Hammond,

2001), increased risk for depression (Bell-Dolan, Reaven, & Peterson, 1993; Ward, Sylva, &

Gresham, 2010), difficulty with the development and maintenance of peer relationships (Ladd,

1999), and future need for psychiatric services (Cowen, Pederson, Babigan, Izzo, & Trost, 1978).

In contrast, children and adolescents who demonstrate appropriate social skills tend to have better

outcomes. For example, prosocial behavior has been linked with greater peer acceptance, and

positive peer relationships have been linked with a variety of benefits for individual development

(Gifford-Smith & Brownell, 2003; Ladd, 1999; Ladd, Karchenderfer, & Coleman, 1996; Parker

& Asher, 1993; Weiner, 2004). Research also has documented the positive effects of social skills

training interventions for different at-risk populations (Ang & Hughes, 2002; Barrera & Schulte,

2010; Gresham, Cook, Crews, & Kern, 2004; Gresham, Van, & Cook, 2006; Najaka, Gottfredson,

& Wilson, 2001; Reichow & Volkmar, 2010; Webster-Stratton, Reid, & Hammond, 2001).

Finally, in the school setting, social skills have been shown to bear positive influence on students’

learning-related behavior and academic achievement (Wentzel, 1993; Zins, Bloodworth,

Weissberg, & Walberg, 2007).

The Need for Cross-cultural Social Skills Research

Social skills are generally considered to be an important protective factor for all

individuals. Still, cross-cultural social skills research is relatively limited. As such, the degree to

2

which previously cited findings generalize across different cultural groups, even those within the

United States, remains unclear. What is clear is that racial and ethnic minority group

representation within the U.S. school-age population is increasing at a steady rate (KewalRamani,

Gilbertson, Fox, & Provasnik, 2007). Moreover, students from racial and ethnic minority groups

continue to be overrepresented in various demographic risk categories and continue to show

chronic academic underachievement, elevated rates of school dropout, and poorer long-term

outcomes (KewalRamani et al., 2007).

Exploring possible ways to improve academic and developmental outcomes for minority

students is an important research objective. As such, the benefits associated with the acquisition

and use of social skills need to be more closely examined from a cross-cultural perspective.

Research aimed at better understanding the social skills construct within specific groups is also

needed. And, as a prerequisite for accomplishing these larger research goals, methods of

assessing social skills within specified populations must be shown to produce scores that are

appropriately reliable and valid (American Educational Research Association, American

Psychological Association, and National Council on Measurement in Education, 1999).

Developing new and distinctive assessment models for different groups is one alternative that

may be considered in attempting to comply with established measurement standards when

working with diverse populations. However, a more parsimonious alternative is to first examine

the validity of scores produced by existing social skills assessment models when used with

specific subgroups from the larger population.

Demographic Trends: The U.S. Hispanic/Latino Population

A significant portion of recent U.S. population growth has taken place within the

Hispanic or Latino1 subpopulation (KewalRamani et al., 2007). As a heterogeneous group, the

1 In order to remain consistent, the term Latino will be used throughout the document when referring to individuals who self-identify

as either Hispanic or Latino. Though sometimes used interchangeably, the term Latino has been applied more frequently in recent scholarly literature.

3

nativity of Latino Americans can be traced back to a number of different countries and

geographic regions including Mexico, Central and South America, and the Caribbean.

Collectively, Latinos comprise the largest minority population within the United States,

representing approximately 15.8% of the total U.S. population (U.S. Census Bureau, 2009). A

significant proportion of U.S. Latinos are foreign born, with 11% of the under-18 Latino

population and 40% of the total Latino population having been born somewhere outside the

United States (KewelRamani et al., 2007). Moreover, roughly 77% of Latinos in the U.S. report

speaking a language other than English in the home (KewelRamani et al., 2007). These

demographic data suggest that many Latino Americans maintain, to varying degrees, a number of

the traditional values, beliefs, and practices of their native cultures. Being embedded in the larger

and, at times, incongruous ‘American’ majority culture, unique cultural influences may affect

outcomes for Latino students in U.S. schools.

Additional characteristics of the Latino American population, independent of ethnically

influenced ‘culture’, also must be considered when comparing outcomes across groups. For

example, Latinos are overrepresented in risk categories defined by poverty status (25% of all

Latino families) and low parental education (41% of Latino mothers and fathers did not complete

high school; KewelRamani et al., 2007). It is often difficult to separate out effects that are

influenced by cultural factors from those that are driven by other factors that simply covary with

group membership. Nevertheless, recent contributions to cross-cultural and culture specific

research have prompted theorists to replace the once commonly accepted developmental risk

framework for minority children with more nuanced ecocultural systems frameworks that

recognize the positive, negative, and neutral influences of cultural group membership on

individual outcomes (Fuller & Garciá-Coll, 2010; Rogoff, 2003).

4

Goals and Objectives

In light of the demographic trends highlighted above, it is clear that U.S. school systems

must work to accommodate increasing diversity within the student body they stand to serve. In

reference to the Latino population(s) in particular, schools must be aware of cultural and

linguistic variation among students and recognize that existing models of instruction, assessment,

and intervention may need to be modified in order to best serve these groups. Educational

researchers also have a responsibility to facilitate this process by prioritizing cross-cultural and

group-specific research.

The primary objective of the current study is to evaluate the structural validity of the

Social Skills Improvement System Rating Scales (Gresham & Elliott, 2008). Specifically,

structural invariance of the measurement model is examined as a function of both student

race/ethnicity and the language in which the scales are written (i.e., language format;

Spanish/English). Through this process two broader research objectives also are addressed. The

first is to investigate the degree to which the social skills construct can be operationalized in a

way that is not bound by cultural norms. The second is to evaluate the appropriateness of using

standardized behavior rating scales and norm-referenced scores within culturally diverse

populations. Before outlining the methodology used for the current investigation, a review of

pertinent literatures is presented in the following chapter.

5

Chapter 2

Literature Review

Motivated by findings linking skilled social behavior with a variety of positive outcomes,

the late 1970s and early 1980s saw a notable increase in the amount of research pertaining to

social skill development (Gresham, 1986). As an emerging research domain, initial attempts to

conceptualize social skill yielded two competing theoretical perspectives: a trait model and a

molecular model (McFall, 1982). The trait model was described as emphasizing an individual’s

underlying capacity for performing in socially relevant contexts. In contrast, the molecular model

emphasized linkages between discrete, observable behaviors and identifiable social outcomes.

Recognizing that the two models were not entirely incompatible, Gresham (1986) argued for a

model that incorporated both perspectives.

According to Gresham (1986), a trait perspective is useful for understanding the higher

order construct of social competence, which he cited as an evaluative term to be used broadly for

the purpose of describing an individual’s social behavior as appropriate/inappropriate or

successful/unsuccessful. In defining the components of social competence, however, Gresham

(1986) opted for a more molecular approach. Specifically, he asserted the importance of the

behavioral construct of social skills, defined functionally as behaviors that maximize the

likelihood of reinforcement and decrease the likelihood of punishment within a given situation

(Gresham, 1981; Gresham & Reschly, 1987).

Gresham’s (1986) focus on the behavioral components of social competence was driven

by practical considerations. Behavioral constructs are particularly useful for the applied practices

of assessment and intervention. It follows that Gresham and colleagues’ early work led to the

development of a detailed assessment for intervention paradigm (Gresham, 1981; Gresham, 1986;

6

Gresham & Reschly, 1986, 1987; Elliott & Gresham, 1987), and ultimately to the publication of

the Social Skills Rating System (SSRS; Gresham & Elliot, 1990). Recently, the SSRS was

revised and incorporated into the comprehensive Social Skills Improvement System (SSIS;

Gresham & Elliott, 2008), which includes assessment and intervention components at the

universal, selected, and targeted levels. The current study examines specific measurement

properties of the newly published SSIS Rating Scales across race/ethnicity and language format

groups. However, before describing the study objectives in more detail, a review of the broader

literatures on social competence, social skills, and cultural influences on social skill development

is provided.

Perspectives on Social Competence

Social competence is widely recognized as a critically important developmental

construct. In general, social competence is viewed broadly as referring to one’s ability to

function successfully within social contexts (Gresham, 1986; McFall, 1982; Merrell, 1999;

Odom, McConnell, & Brown, 2008). The most appropriate means of conceptualizing social

competence as a psychological construct has been a long-standing topic of debate (see Gresham,

1986; McFall, 1982; Merrell & Gimpel, 1998). Greenspan (1981) offered a tripartite model of

social competence in an attempt to consolidate three divergent approaches that had been

frequently applied to conceptualize the construct. As described by Greenspan (1981; see also

Gresham 1986), skill-oriented approaches are those that prioritize process variables associated

with social interaction. Individuals who have developed an intuitive understanding of the rules

and scripts for appropriate interpersonal interaction are, from a skill-oriented perspective, deemed

socially competent. Outcomes-oriented approaches view social competence retroactively on the

basis of important social achievements. Positive achievements (i.e., outcomes) are taken as an

indication that an individual is socially competent. Finally, content-oriented approaches focus on

the exhibition of specific behaviors presumed to be predictive of successful social functioning.

7

From this perspective, individuals who demonstrate appropriate behaviors relative to situational

demands are said to demonstrate social competence.

Divergent Perspectives on Social Competence. In the fields of social and cognitive

psychology, child development, special education, and school psychology, divergent strands of

research continue to reflect the three social competence orientations identified by Greenspan

(1981). Researchers adopting a social-cognitive perspective have expounded on the nature of

cognitive processes that preempt social behavior. For example, social information-processing

(SIP) theory (Crick & Dodge, 1994) corresponds with a skill-oriented perspective in

conceptualizing behavior as a product of the interaction between the processing of situational

information and pre-existing cognitive structures that have developed over time through prior

experience. Developmental researchers have tended to align with outcomes-oriented

conceptualizations of social competence, emphasizing the importance of outcomes such as peer

acceptance and peer relationships as indicators of present levels of social competence and

predictors of long term adjustment (Ladd, 1999; Ladd, Herald, & Andrews, 2005). Finally,

researchers in applied fields such as special education and school psychology have tended to

adopt content-oriented approaches, focusing on the acquisition and performance of specific skills

that promote positive interactions and reinforcement in social contexts (Gresham & Elliott, 1990;

Merrell, 2001).

Joining Perspectives: Comprehensive Models of Social Competence. The

aforementioned distinctions are not rigid, and across research fields, comprehensive models of

social functioning that incorporate aspects of all three conceptual perspectives tend to have the

most explanatory power. For example, a recent study in the field of child development

investigated a three-component developmental cascade model, which analyzed the cyclical and

reciprocal relations between SIP, peer rejection, and aggression across the early school years

(Lansford, Malone, Dodge, Pettit, & Bates, 2010). The model provides a very useful illustration

8

of interactions between skill (e.g., SIP), outcome (e.g., peer rejection), and content (e.g.,

aggression) factors, and shows how all three function together to contribute to patterns of social

development and the emergence of social dysfunction. In a recently published book chapter,

Odom, McConnell, and Brown (2008) also advocate for a combination of conceptual approaches,

arguing that social competence involves both the selection of appropriate behavioral strategies

and the subsequent attainment of social goals. Neither appropriate strategy selection, nor social

goal attainment, in isolation would suffice an indicator of social competence. Moreover, as a

means of attaining social goals, individuals draw on complex repertoires of cognitive, emotional,

and behavioral competencies (Greenspan, 1981; Merrell, 1999; Odom, McConnell, & Brown,

2008), which suggests that skill and content factors contribute to individual outcomes. In sum,

the compatibility of all three social competence perspectives is evident, and inter-disciplinary

distinctions appear to be an artifact of the specific goals of researchers in different fields.

The Issue of Context. Though social competence is often framed as a characteristic of

the individual, comprehensive conceptualizations of social competence also recognize the

influence that contextual factors may have on an individual’s behavior. For example, Odom,

McConnell and Brown (2008) have categorized factors contributing to social competence across

two broad dimensions: those originating within the child (e.g., neurology, temperament,

cognition), and those influencing the child from the outside (e.g., family, school, peers, culture).

These latter factors represent features of the ecological context in which individuals are situated.

In reference to specific events, features of the immediate context are also relevant, as

these may dictate an individual’s selection of alternate behavioral strategies. A skilled, effective

strategy in one context may be considered unskilled or ineffective in a different context.

Therefore, socially competent individuals may not enact uniform behavioral solutions in pursuit

of social goals across unique contexts (McFall, 1982). In fact, noting that situational variables

represent a critical dimension that must be considered within any valid assessment of social

9

behavior, McFall (1982) advocated for an assessment approach that emphasized analysis of tasks

as opposed to analysis of individuals. Though task-based/role-play assessment research and

practice has faded over the past few decades (Matson & Wilkins, 2009), contextual

considerations in the conceptualization of social behavior remain important. For example,

Sheridan and Walker (1999) outlined an ecological-contextual model of social behavior, rooted in

social cognitive theory, which contends that social behaviors are expressed as a product of the

interaction between three sets of factors: characteristics of the child, characteristics of the

individual(s) with whom the child interacts, and features of the context in which interactions take

place. In sum, a general consensus suggests that contextual factors are influential in

understanding social competence and, more narrowly, in determining the expression social

behavior. A more thorough examination of contextual considerations in the assessment of social

skills is presented later.

Emphasizing the Social Skills Dimension of Social Competence

As noted, social competence is a complex multi-dimensional construct, the assessment of

which presents a number of obstacles. All facets of social competence cannot be readily

observed. However, the discrete behaviors demonstrated by individuals in social situations

represent one facet of social competence that researchers have been able to operationally define

and reliably measure. The proliferation of social skills research over the past several decades

attests to this fact (Matson & Wilkins, 2009; Merrell & Gimpel, 1998).

Defining Social Skills. Although a number of different conceptual approaches have been

applied in attempts to define the social skills construct (Gresham, 1986), several common

construct features are generally agreed upon by researchers in the field (Merrell & Gimpel, 1998).

For one, social skills are fundamentally interactive and include behaviors associated with

interaction initiation and interaction response (Merrell & Gimpel, 1998). Social skills are also

understood to be situation specific, as dictated by the context in which they are employed

10

(Sheridan & Walker, 1999). Finally, from a behavioral perspective – which is the perspective

that has been most frequently applied in the scholarly literature – social skills are discrete,

observable, learned behaviors that maximize reinforcement in the social setting (Gresham, 1986).

The distinctive features listed above have provided a useful structure for research in the

social skills domain. However, on the basis of these features alone, the social skills construct

remains somewhat broad and unfocused. For this reason, a social validity approach to social

skills research has become increasingly popular. The social validity approach maintains the same

basic definitional premises, but further stipulates that social skills predictive of important social

outcomes should be the primary focus of investigation (Gresham, 1986; Merrell & Gimpel, 1998;

Sheridan & Walker, 1999). Social validity approaches prioritize those specific skills that are

shown to be important in particular contexts. Although traditional behavioral definitions of social

skills have been described as optimal for the purpose of assessment (Caldarella & Merrell, 1997),

social validity definitions in particular have been identified as most useful in the assessment-for-

intervention process (Elliott, Gresham, Frank, & Beddow, 2008).

As it relates to the current study, the SSIS Rating Scales manual defines social skills as

“learned behaviors that promote positive interactions while simultaneously discouraging negative

interactions when applied to appropriate social situations” (Gresham & Elliott, 2008, p. 1). Thus,

from a conceptual standpoint, the SSIS adopts a relatively straightforward behavioral definition

of social skills as identified by their situation-specific function. In application, however, certain

key features of the SSIS Rating Scales are clearly oriented to the social validity perspective. For

example, the categorical structure of the social skills scale is designed to tap into specific domains

of behavior (e.g., communication, cooperation, responsibility) that are important across a variety

of settings. The SSIS Rating Scales also call for ratings of both behavior frequency and behavior

importance. The importance ratings, specifically, allow for the prioritization of intervention

efforts as informed by the ratings of the adult figures (e.g., parents, teachers) who are most aware

11

of the different types of social skills required in specific settings. More detailed information

regarding the design and structure of the SSIS Rating Scales will be provided at the end of the

current review.

Toward a Taxonomy of Social Skills. Theoretical understanding of psychological

constructs is often aided by the identification and clarification of taxonomic structures.

Taxonomies derived through empirical analysis are particularly useful in refining theory

(Achenbach, 1995; Caldarella & Merrell, 1997). For example, in reference to the study of

problem behavior, Achenbach (1995) outlined an empirically based paradigm for assessment and

taxonomy predicated on the quantitative, multivariate analysis of large-sample assessment data.

In application, Achenbach was able to use objective data to demonstrate the existence of various

“syndromes of co-occurring problems” (p. 262), thus contributing to the refinement of theory in

reference to childhood problem behavior. Unfortunately, and despite continued research in the

field, consensus regarding a uniform taxonomy of social skills remains elusive.

In recognition of the need for a taxonomy of positive behaviors, Caldarella and Merrell

(1997) conducted a meta-analytic review to inform the development of a common social skills

taxonomy. Their meta-analytic procedure involved identifying studies in which unique

dimensions of social skills had been derived through quantitative analysis (e.g., factor analysis,

cluster analysis, etc.). Having identified 21 relevant works, reviewers then looked for

commonalities across studies in terms of social skills factors and the specific items that had been

used as indicators of each factor. Results revealed five commonly occurring dimensions of social

skills.

A peer relationship dimension was represented by items referencing pro-social peer

interactions, friendship initiation and maintenance, and sensitivity to social cues. A self-

management dimension included behaviors indicative of self-control, compliance, and tolerance.

An academic skills dimension was found to include behaviors showing compliance and task

12

orientation. A compliance dimension was represented by items referencing rule-following and

good citizenship. Finally, an assertion skills dimension consisted of behaviors showing proper

interaction initiation and situation appropriate assertiveness (Caldarella & Merrell, 1997).

Collectively, the five social skills dimensions reported by Caldarella and Merrell (1997)

along with the behaviors that comprise each dimension provide a useful blueprint for research,

assessment, and intervention purposes. However, these findings are not without limitations.

First, only those dimensions that had been previously identified through published quantitative

research investigations were eligible for consideration. Moreover, only those dimensions that had

appeared in numerous studies made the final listing. Factors existing in less than one-third of the

reviewed studies were automatically excluded. Thus, while the resulting taxonomy accurately

describes trends in previous research, it cannot be said to represent a definitive listing of all

pertinent domains of social behavior.

A bigger limitation of Caldarella and Merrell’s (1997) study stems from the qualitative

nature of the meta-analysis. All reviewed studies were looked at independently, and qualitative

descriptions of the emergent social skills dimensions were compared. This process produced

taxonomic distinctions that were not fully differentiated. That is, upon comparison of the

behavioral characteristics of each social skills dimension put forth in the taxonomy, considerable

overlap across posited dimensions was observed. For example, items listed on the compliance

dimension (e.g., “follows rules”, “follows instructions/directions”) were virtually identical with

items listed on the self-management (e.g., “follows rules, accepts imposed limits”) and academic

dimensions (e.g., “listens to and carries out teacher directions”). Thus, although each of the

individual studies included in the review applied appropriate multivariate analyses in order to

extract clear factors/clusters of social skills behaviors, the more subjective meta-analytic

procedure did not retain clear distinctions between the various factors/clusters.

13

Empirical taxonomies generated through multivariate analysis of assessment data are

clearly helpful in terms of promoting the advancement of scientific knowledge. The process by

which such taxonomies are developed and refined illustrates the reciprocal contributions of theory

development and measurement design. As explained by Achenbach (1995), the process typically

starts with the operationalization of theoretical hypotheses into an initial measurement model.

Once the measurement model is specified, data can be collected and analyzed, allowing for an

examination of the degree to which patterns in the observed data are consistent with original

theory. Subsequent iterations of the process allow for further refinement of both the

measurement model and guiding theory. This methodology has already been applied successfully

within the study of childhood problem behaviors (Achenbach, 1995). Still, it remains to be seen

how well such an approach would serve to inform the study of social skills. At present, there

does not appear to be a commonly posited structure defining the scope of the social skills

construct/domain. As, such, assessment instruments have been created in broad and varied ways

(Matson & Wilkins, 2009). There are general similarities in the structure and content of

commonly measured social skills domains (Gresham et al., 2004; Gresham, 2011). However, the

vast collection of contemporary social skills assessment instruments also show variability in

reference to a number of key features including: target population (e.g., broad versus specific age

groupings), scope and description of behavior domains, and the degree to which behaviors are

defined as context-specific versus context-free (Matson & Wilkins, 2009).

Social Skills in Context. From a behavioral perspective, social skills are identified on

the basis of function. The molecular function of specific social skills is likely to vary, but

collectively social skills have been defined as behaviors that promote positive interactions

(Gresham & Elliott, 2008) and lead to desirable social outcomes (Merrell, 1999). Drawing from

this functional approach, it is clear that social skills should not be identified on the basis of the

topography of behavior, but rather on the probability that the exhibition of a particular behavior

14

will be associated with reinforcement in the social context. Sheridan and Walker (1999)

suggested that any accurate conceptualization of social skills must allow for the potential effects

of contextual factors on the expression of behavior. Contextual variation in the expression of

social skills can occur as a function of ecological factors, the presence/absence of particular social

actors, or at different points in the reciprocal process of social interaction (Sheridan & Walker,

1999).

The effect of contextual factors on the topography of behavior clearly complicates social

skills assessment. Certain methods of assessment can be used to counteract context-specific

effects on behavior. For example, qualifying direct behavioral observations within the context of

the environment in which they are taken promotes the reliable interpretation of data collected in

this manner (Norton, Washington, Peters, and Hayes, 2010). It is also plausible that certain

behaviors and categories of behavior function as social skills across a multitude of contexts. For

example, the aforementioned Caldarella and Merrell (1997) meta-analysis identified five primary

domains2 of social skills that were frequently represented across reviewed studies: peer

relationships, self-management, academic, compliance, and assertion. These five categories were

observed across multiple independent samples. Similarly, qualitative research has reported

“substantial overlap” in the types of behaviors identified as important social skills by teachers,

parents, and student respondents from the second and fifth grades (Warnes, Sheridan, Geske, &

Warnes, 2005). Specifically, behaviors associated with compromise, empathy, assistance, trust,

loyalty, and social engagement were consistently identified as important social behaviors by all

groups of respondents. Still, other reviews have pointed out variation in the social skills construct

as a function of gender, developmental status, and cultural group membership (see Merrell &

Gimpel, 1998).

2 Caldarella and Merrell (1997) use the term ‘dimensions.’ However, given that the observed categories do

not appear to represent a set of unitary, latent constructs, the term domain is applied in the current

discussion.

15

In light of cited findings, the effect of contextual factors on the exhibition of social skills

cannot merely be assumed or dismissed through subjective analysis. Rather, context-behavior

interactions must be empirically examined, and their effects parsed out. At present, research

examining relationships between cultural context (e.g., cultural routines, traditions, beliefs,

expectations) and the development of social skills within specific groups is limited (Matson &

Wilkins, 2009). Cultural factors, by definition, influence norms for social behavior. As such, and

given that socially skilled behavior has been widely viewed as a context-dependent construct,

investigations of the ways in which variation in cultural context affect the composition and form

of social skill behaviors are clearly warranted.

Cultural Influences on Social Skill Development

From an ecological perspective, individual development occurs within contexts framed

by complex and interactive systems (Bronfrenbrenner, 1986). Cultural researchers have applied

ecological frameworks as a means of exploring the many ways in which culture influences

individual development (Rogoff, 2003; Weisner, 2002). Most straightforward, perhaps, are broad

trends observed in reference to group membership. However, the fluid nature of cultural identity

and cultural affiliation within groups represents a more subtle dimension of cultural influence that

also must be considered (Fuller & García Coll, 2010; Halgunseth, Ispa, & Ruddy, 2006). Finally,

influences that emerge as individuals come into contact with various social institutions (e.g.,

schools) that may or may not operate according to the same cultural frameworks governing

interactions in the home or local community settings also warrant consideration (Fuller & García

Coll, 2010; Warzon & Ginsburg-Block, 2008). As noted previously, the body of research on

social skills as they exist within and across culturally diverse populations is relatively limited

(Matson & Wilkins, 2009). As such, the following sections review research in reference to

emerging themes in culturally focused developmental research (see Fuller & Garcia Coll, 2010)

16

in an attempt to establish links between eco-cultural developmental factors and the process of

social skill acquisition and maintenance for Latino children in the U.S.

Traditional Latino Parenting Practices. The developmental experiences of Latino

children and adolescents are directly influenced by the culturally bound practices of their parents

(Fuller & García Coll, 2010; Halgunseth et al., 2006). In a review of research on parenting

practices within Latino families, Halgunseth et al. (2006) identified three traits that Latino parents

often strive to cultivate in their children: familismo, respeto, and educación. Familismo refers to

the prioritization of family interests over those of the individual. Respeto refers to the expectation

that individuals will act in accordance with their own social roles and show respect for the roles

of others with whom they interact. Finally, educación refers to the cultivation of social

responsibility and emotional maturity. Halgunseth et al. (2006) argue that these traits represent

goals that often dictate the parenting strategies selected and employed by Latino parents.

It is worth noting that traits such as familismo, respeto, and educación are not

intrinsically unique to Latino cultures. Instead, it is the degree of emphasis placed on such traits

and the culturally bound parenting practices designed to cultivate their development that render

them uniquely Latino. For example, Okagaki and Frensch (1998) found that Latino parents

prioritized their children’s development of autonomous and conforming behaviors, and monitored

these aspects of child development more closely than did European-American or Asian-American

parents. By contrast, Asian American parents were found to place more emphasis on cognitive

development and educational achievement than were parents from the other two groups (Okagaki

& Frensch, 1998). These results do not necessarily suggest that certain cultures devalue

particular aspects of child development (e.g., setting social development in opposition to

academic development). However, there does appear to be a limit on the degree to which parents

can emphasize all of the various aspects of child development. As such, differential competence

orientations emerge to reflect the specific developmental features prioritized by members of

17

different cultural groups (Rogoff, 2003). For Latino populations, research suggests that

individual competencies, including interpersonal/social competence, are emphasized in

proportion to their perceived importance within a broader interdependent, community-oriented

context (Fuller & García-Coll, 2010; Halgunseth et al., 2006; López, Correa-Chávez, Rogoff, &

Gutiérez, 2010; Okagaki & Frensch, 1998).

In terms of specific socialization practices, research findings showing frequent use of

parenting strategies such physical guidance, verbal direction, and rule setting, have been cited as

evidence that Latino American parents tend to adopt more authoritarian roles than do their Euro-

American counterparts (Halgunseth et al., 2006; Livas-Dlott, Fuller, Stein, Bridges, Mangual

Figueroa, and Mireles, 2010). However, authoritarian parenting in the Latino context does not

carry the same stigma that it has traditionally held within more Eurocentric contexts. Rather,

many of the Latino parenting practices that have been classified under the ‘authoritarian’

archetype include a distinctly positive qualitative feature termed cariño (i.e., caring; Livas-Dlott

et al., 2010). The presence of cariño distinguishes these Latino parenting practices from the

traditional Eurocentric view of authoritarian parenting as cold and lacking compassion

(Baumrind, 1989, as cited in Livas-Dlott et al., 2010). At present, much of the research on Latino

parenting practices has been descriptive and ethnographic, and more research is needed in order

to clarify the effects of Latino parenting practices on various child outcomes. Given the traits that

Latino parents often strive to cultivate within their children (e.g., familismo, respeto, educación),

research examining the discrete social skill behaviors that children employ in demonstration of

such traits is one specific area where additional research is needed.

Within-group Differences: The Effects of Acculturation. Individual differences in

cultural identity and cultural affiliation are influenced by a variety of factors including personal

history, nativity, generational status, language status, geographic locale, community

demographics, and the availability and composition of social support networks. The Latino

18

population of the United States shows considerable within-group variability across a number of

such factors (Fuller & García Coll, 2010). Moreover, cultural identity and cultural affiliation are

not static traits; rather they fluctuate across the lifespan and as a function of developmental

experience. Researchers interested in understanding cultural change as it takes place for

individuals immersed within novel environments (e.g., first generation Latino immigrants in the

United States) have begun to more closely examine the construct of acculturation, defined as “the

process of adopting goals and practices due to exposure to a new culture” (Halgunseth et al.,

2006, p. 1282). Thus, as a caveat to the previous section’s descriptions of Latino parenting

practices, it is important to note that the degree to which Latino children and youth are actually

exposed to such ‘traditional’ cultural practices may vary significantly as a function of different

factors including familial levels of acculturation.

Consider, for example, the results of the ethnographic study conducted by Livas-Dlott et

al. (2010) which were broadly interpreted in support of the hypothesis that Latina mothers tend to

adopt power-assertive strategies when attempting to gain compliance from their children (as

opposed to inductive strategies). Interestingly, when data were disaggregated according to groups

defined by the specific repertoires of parenting strategies employed, trends suggested that Latina

mothers who did incorporate inductive strategies were more likely to be second generation and

more likely to have graduated high school. Sample size limitations precluded testing such trends

for statistical significance, but the implication is that more acculturated Latina mothers may

expand their parenting repertoires to include practices commonly employed in the culturally

diverse communities in which they reside (Livas-Dlott et al., 2010).

In a separate study, statistically significant positive relationships (concurrent and

predictive) were found between immigrant Latino parents’ ratings of positive parenting practices

and family cohesion and their children’s self-rated social problem-solving skills and social self-

efficacy (Leidy, Guerra, & Toro, 2010). As a follow-up to the larger study, 12 immigrant Latina

19

mothers participated in a focus group in order to discuss barriers they identified as impediments

to family cohesion and positive parenting practices. Not surprisingly, barriers included

generational differences in level of acculturation, parents’ inability to actively partake in their

children’s education, limited availability of social support due to immigration history, and

perceptions of vulnerability and discrimination due to residency status. Taken collectively,

results from the pair of studies conducted by Leidy et al. (2010) suggest that traditional Latino

parenting practices do serve to cultivate social competence within Latino children, but that such

practices are inevitably affected by factors related to acculturation and exposure to a novel

culture.

Cultural Interaction and the Effects of Schooling. As children enter school, their

ability to utilize learned skills in order to achieve success in the academic setting varies according

to both the level to which their skills have been developed and the degree to which those skills are

recognized as functional assets in the new setting (Phelan, Davidson, & Cao, 1991). For many

Latino students, school entry represents entry into a novel cultural context. Thus, in addition to

traditional academic and social requirements, Latino students often have the added task of

acclimating to a culturally unique interactive setting. A lack of continuity between home and

school contexts places these students at-risk for poorer outcomes, particularly when the

functionality of acquired skills is compromised.

Language minority status represents one obvious cultural factor that inhibits the

functionality of students’ skills in the school setting. Research findings indicate poorer academic

achievement outcomes for English language learner (ELL) students (see Genesee, Lindholm-

Leary, Saunders, & Christian, 2006; Suárez-Orozco, Gaytán, Bang, Pakes, O’Connor, & Rhodes,

2010). Though research is somewhat limited, language minority status also has been identified as

a risk factor for students’ social-emotional outcomes (e.g., Dawson & Williams, 2008). A study

conducted by Spomer and Cowen (2001) found that Latino students exhibit unique profiles of

20

social competence as a function of language status. Specifically, teacher ratings of a clinic

referred sample showed non-ESL students to have higher overall competence scores than their

ESL peers, with strengths identified in the domains of Assertive and Peer Social Skills.

Interestingly, ESL students evidenced higher levels of teacher-rated Frustration Tolerance,

showing that, although language minority status often represents a barrier to be overcome within

the school setting, it cannot be solely defined as a risk factor. Rather, the ‘risk’ associated with

language minority status develops as a product of the interaction between students’ skills and

their functionality within specific settings (e.g., school).

Another study, conducted by Edl, Jones, and Estell (2008), compared teacher rated

academic and social competence across groups of European American regular education students,

Latino regular education students, and Latino bilingual education students. Though results varied

across time points with respect to differences on specific academic and social competence

outcomes, discriminant function analysis and follow-up statistical contrasts consistently showed

the greatest differences to exist between European American regular education students and

Latino bilingual education students, with the European American students rated as more

competent. Fewer differences were observed between European American and Latino regular

education students (Edl et al., 2008). Collectively, these results offer support for the hypothesis

that the degree of continuity between students’ home and school cultures (e.g., ethnicity,

language use) is an important predictor of teacher rated interpersonal competence in the school

setting.

More research examining the differential effects of cultural consonance and mismatch

across the home and school settings is needed (Galindo & Fuller, 2010). Still, schools that are

better able to adapt to students’ cultural needs remain more likely to promote positive outcomes

for diverse student populations. A recent study of pre-kindergarten students’ adjustment

outcomes found that the frequency and quality of classroom based Spanish language interactions

21

predicted significantly better teacher-rated social skills for Spanish speaking children (Chang et

al., 2007). Similarly, an intervention study conducted with at-risk Latino children and their

parents showed statistically significant social skills gains and concurrent reductions in problem

behavior for students who participated in a mentoring program that included educational

components attended by both students and parents (Barron-McKeagney, Woody, & d’Souza,

2001). Though specific effects were not parsed out in the analyses, the joint participation of

students and parents likely contributed to student gains at post-test, with parental involvement

promoting social skill development in the home setting as well. For both studies (i.e., Barron-

McKeagney et al., 2001; Chang et al., 2007), positive outcomes were associated with processes

that functioned to bridge the gap between home and school cultures.

As evidenced throughout the current review, the dynamic nature of cultural influences on

the social development of Latino students must be considered when evaluating existing research

with this diverse population. Additional consideration must also be made in reference to the

measurement properties of instruments used for the collection of research data. One of the

studies cited above (i.e., Spomer & Cowen, 2001) included an independent analysis of the

validity of scores obtained in reference to their research sample. Other studies tended to utilize

measurements that had not been validated for use with their sample populations. In order to

promote the interpretability of future cross-cultural research, studies examining the reliability and

validity of scores from measurement instruments with specific populations are needed.

Social Skills Assessment

Common methods of assessing social skills include direct behavior observations,

behavior rating scales, clinical interviews, teacher nomination procedures, sociometric

techniques, and self-report measures. From a clinical perspective, multi-method and multi-

informant assessment of students’ social skills is recommended as best practice (see Merrell,

2001; Sheridan & Walker, 1999). Assessment information from multiple informants allows for

22

an examination of an individual’s social skills across contexts, in the presence of various

audiences, and from a variety of perspectives. Similarly, the use of multiple methods of

assessment provides a means of cross-validating information yielded through any single method.

In addition, there is also research to suggest that multi-method assessment promotes a more

complete assessment of the broader construct of social competence (see Odom, McConnell, &

Brown, 2008).

Advocating for a multi-method approach to social skills assessment does not reduce the

need, from a measurement perspective, for gathering evidence to support the validity of data

obtained from individual methods. The current study is specifically concerned with measurement

validity as it relates to the use of standardized behavior rating scales in the assessment of

children’s social skills. Behavior rating scales have been recommended as a cornerstone of social

skills assessment (Elliott, Gresham, Frank, & Beddow, 2008; Elliott, Malecki, & Demaray, 2001;

Merrell, 2001). Social skills rating scales are recognized for convenience of administration and

strong psychometric properties, while at the same time offering a means of collecting data across

settings, from multiple informants, and about a variety of behaviors that are directly applicable

for intervention planning (Elliott, et al., 2008; Merrell, 2001). However, various critiques of

social skills rating scales have also been offered including their lack of sensitivity to small

changes in behavior, the need to qualify ratings based on the perspective of the rater, and the

related complexities involved in aggregating ratings from multiple raters (Elliott et al., 2001;

Elliott et al., 2008; Gresham, 2011). It is also important to note that social skills rating scales’

sensitivity to contextual factors is influenced by the level of specificity with which operational

definitions are applied at the item level (Elliott et al., 2008; Matson & Wilkins, 2009). Given the

widespread use of social skills rating scales, there is a continual need for new investigations

designed to examine evidence supporting (or refuting) the validity of scores produced by such

instruments.

23

Cross-cultural Social Skills Assessment. In a recent chapter discussing diversity

characteristics within the scope of social skills assessment, Norton et al. (2010) continue to

recommend a multi-method/multi-informant assessment approach as a means of identifying the

effects of diversity characteristics on the expression of social behaviors. Specifically, these

authors recommend a combination of clinical interview, direct observation, and self-/other-report

behavior ratings. They also noted that, in the context of cross-cultural assessment, it is important

that clinicians guard against misinterpretation of assessment data by gathering additional

information relative to the cultural norms of the individual and the local norms of the

environment in which behaviors are exhibited. Such information gathering is likely to assist in

guarding against misinterpretation of directly observed behaviors and those reported through

clinical interviews.

In reference to behavior rating scales, clinicians typically make interpretations on the

basis of norm-referenced scores. As such, when using behavior rating scales, clinicians must

consider the design features and technical properties of each instrument in relation to the

individual being evaluated. Specific design features that should be evaluated when considering

an instrument for cross-cultural use include appropriateness of content, language and dialect, and

reading level required of respondents (Norton et al., 2010). Additional technical properties that

must be considered include representativeness of the standardization sample and whether there is

evidence to support measurement equivalence/invariance (Knight & Hill, 1998), which refers to

the degree to which a measurement instrument operates in consistent ways for individuals from

different cultural groups (Vandenberg & Lance, 2000).

Social Skills Improvement System (SSIS)

The Social Skills Improvement System (SSIS; Gresham & Elliott, 2008) Rating Scales

represent a comprehensive revision to the original Social Skills Rating System (SSRS). In an

effort to update, improve, and expand the SSRS, several areas were targeted for revision.

24

Specific revision goals included the addition of new subscales, improved alignment across forms

(i.e., teacher, parent, student), and new procedures for linking assessment results with

intervention procedures. Similar to the SSRS, the SSIS Rating Scales are available in three rater

versions (parent, teacher, and student) and for three age ranges (3-5, 5-12, 12-18)3. After

revision, the SSIS Rating Scales include a Social Skills scale comprised of seven independent

subscales (Communication, Cooperation, Assertion, Responsibility, Empathy, Engagement, and

Self-Control), a Problem Behaviors scale comprised of five semi-overlapping subscales

(Externalizing, Bullying, Hyperactivity/Inattention, Internalizing, and Autism Spectrum), and a

nine-item Academic Competence scale (teacher version only). Parent and teacher forms are

scored on a 4-point scale for frequency (Never, Seldom, Often, Almost Always) and a 3-point scale

for importance (Not Important, Important, Critical; Gresham & Elliott, 2008). More specific

information on the psychometric properties of the SSIS Rating Scales is provided in the Method

section.

The SSIS Rating Scales also feature Spanish language parent and student versions, which

were not available for the SSRS. The Spanish translations were developed in three stages

(Gresham & Elliott, 2008). First, three Spanish speaking psychologists completed independent

item translations. The independent translations were then submitted to a psychological testing

company specializing in test translation, and item retention decisions were made in reference to

content consistency and reading level. Finally, through the standardization process, item-total

correlations and internal consistency reliabilities of Spanish form scores were compared with

English form scores to establish evidence of equivalence across form language (Gresham &

Elliott, 2008). Again, a more thorough description of the psychometric properties of the SSIS

Spanish forms is provided in the Method section.

3 The SSIS Rating Scales, Student Form is only available for 8-12 and 12-18 age ranges.

25

Given its relatively recent publication, independent analysis of the measurement

properties of scores from the SSIS Rating Scales are limited. However, a large body of research

has been conducted with its predecessor, the SSRS. Though differing substantially in terms of

format and content, previous analyses of the SSRS may provide insight into areas that should be

closely examined when evaluating the SSIS. As such, the following section reviews the extant

literature findings referencing the SSRS. A brief review of basic psychometric properties of the

scale is provided first. Results from a group of studies which looked more specifically at aspects

of structural validity in reference to SSRS scores are then presented. Finally, research looking at

the validity of SSRS scores as a function of cultural and language group membership is

examined.

Social Skills Rating System (SSRS). The SSRS (Gresham & Elliott, 1990) was widely

cited as one of the most comprehensive and technically adequate instruments available for the

assessment of children’s social skills (Bracken, Keith, & Walker, 1998; Demaray & Ruffalo,

1995; Merrell & Gimpell, 1998). Through the development and standardization process, the

SSRS authors amassed evidence to support the internal consistency and short-term stability of

scores across the domains of social skills, problem behavior, and academic competence (Gresham

& Elliott, 1990). Validity evidence for SSRS scores was also provided in the form of moderate

correlations with other rating scales designed to measure similar and related constructs (Gresham

& Elliott, 1990). Additional studies conducted by independent researchers have supported the

reliability (e.g., Pedersen, Worrell, & French, 2001) and validity (e.g., Flanagan, Alfonso,

Primavera, Povall, & Higgins, 1996) of SSRS scores with independent samples. The SSRS has

also been used as a criterion measure when seeking to establish concurrent validity evidence for

different instruments (e.g., Crowley & Merrell, 2000; Merydith, 2001). In addition to evidence

supporting appropriate psychometric properties, reviewers have recognized the SSRS for its

integrated multi-rater assessment format (i.e., teacher, parent, student; Merrell, 1999) and

26

straightforward application for intervention planning (Bracken et al., 1998; Merrell & Gimpell,

1998). In sum, the collection of available research suggests that the SSRS is a psychometrically

sound rating scale instrument, which produces reliable scores that are valid for interpretation and

application within an assessment for intervention framework.

Structural Investigations of the SSRS. Although a majority of studies have tended to

evaluate the psychometric properties of the SSRS by looking at global indicators of score

reliability and validity, several recent studies have applied more sophisticated exploratory and

confirmatory analytic methods in order to examine the structural integrity of the SSRS

measurement model. Results of these investigations have offered mixed support for the rating

scales’ proposed measurement structure, with variation occurring as a function of respondent

(teacher v. parent) and developmental grouping (preschool v. school-age). For example, the

factor structure of the teacher and parent versions of the SSRS failed to replicate for a sample of

African American preschoolers attending Head Start (Fantuzzo, Manz, & McDermott, 1998;

Manz, Fantuzzo, & McDermott, 1999). In a separate study, the factor structure of the SSRS

teacher version did replicate for a clinical sample of Dutch children with ADHD, although the

factor structure of the parent version was not supported with data from the same sample (Van der

Oord, Van der Meulen, Prins, Oosterlaan, Buitelaar, & Emmelkamp, 2005). Through

confirmatory factor analysis (CFA) Walthall, Konold, and Pianta (2005) found evidence to

support the SSRS teacher version measurement model with an independent sample of school-age

children. However, CFA results in two separate investigations of the SSRS parent version again

failed to replicate the rating scales’ original factor structure with pre-school (Whiteside,

McCarthy, & Miller, 2007) and school-age samples (Van Horn, Atkins-Burnett, Karlin, Ramey,

& Snyder, 2007), respectively. Though the collection of independent studies of the SSRS cited

here does not speak directly to the validity of the revised and expanded SSIS Rating Scales, it

does underline the need for evidence supporting the new rating scales’ structural adequacy.

27

Cross-cultural Applications of the SSRS. Despite its status as one of the most

frequently recommended and widely used instruments in the domain of social skills assessment,

questions have been raised regarding the validity of the SSRS when used with various

racial/ethnic groups (Fantuzzo et al., 1998; Manz et al., 1999; Van der Oord et al., 2005;

Whiteside et al., 2007). Researchers have suggested that cultural differences between study

participants and the SSRS standardization sample may have contributed to observed structural

differences (Manz et al., 1999). The same researchers went on to suggest that “conventional test

construction methods often do not sufficiently represent economically and ethnically diverse

populations” (p. 305), and therefore may not be appropriate for use with such populations.

Two of the previously cited SSRS studies (Van Horn et al., 2007; Walthall et al., 2005)

applied multi-group confirmatory factor analytic methods to objectively examine measurement

invariance of SSRS scores across groups defined by race/ethnicity. In their analysis of the SSRS

teacher elementary form, Walthall et al. (2005) found that the general form of the SSRS

measurement model exhibited invariance across groups of White and non-White students,

providing tentative support for invariance of the SSRS teacher form measurement model as a

function of students’ racial status. Van Horn et al.’s (2007) evaluation of measurement

invariance for the SSRS parent elementary form was even more comprehensive in that the authors

examined invariance at multiple levels and across more clearly identified racial/ethnic groups:

White, non-Hispanic; African American; and Hispanic. Multi-group CFA results indicated that a

modified version of the original SSRS showed both configural and item-level invariance across

groups. However, the invariant model differed substantially from the original measurement

model proposed by the authors of the SSRS. As such, Van Horn et al. (2007) cautioned against

the use of SSRS normative scores with children from different racial and ethnic groups. Such

results further underscore the need for evidence supporting the structural integrity of the revised

28

SSIS Rating Scales, and suggest that specific attention should be given to the examination of

invariance across groups defined by race/ethnicity.

As it relates to the current study, it is also important to note that the Hispanic sample in

the Van Horn et al. (2007) study was comprised of parent participants who provided data through

English language interviews. Although a substantial proportion of the study’s sample (7%)

elected to complete interviews in Spanish, the authors reported that such data “were eliminated

because later analyses showed measurement differences for those interviews” (p. 172, Van Horn

et al., 2007). Specifics regarding the nature of measurement differences as a function of

interview language (i.e., English vs. Spanish) were not provided. Nevertheless, the fact that

differences were found to exist calls into question the adequacy of SSRS scores obtained from a

Spanish translation of the original rating scales. Given that the SSIS Rating Scales include a

published Spanish version, invariance as a function of language format also requires examination.

Research Questions

The SSIS Rating Scales represent a new and promising instrument for the assessment of

students’ social skills. However, in light of previous findings questioning the viability of the

SSRS measurement model when applied with minority group populations (e.g., Fantuzzo et al.,

1998; Manz et al., 1999; Van Horn et al., 2007), the cross-cultural utility of the SSIS Rating

Scales cannot merely be assumed. Focused, multi-group confirmatory factor analyses are needed

to evidence the structural integrity of the measurement model as a function of race/ethnicity.

With the creation of the new SSIS Rating Scale Spanish language forms (parent and child

versions), invariance as a function of language format must also be examined. In reference to

both grouping factors (i.e., race/ethnicity, language format), structural analysis of the

measurement model is needed in order to examine invariance in the full model and the viability of

individual subscales. The current study aims to address this need with respect to the parent

version of the SSIS Rating Scales. Specific research questions include:

29

1. Does the SSIS Rating Scale, Parent Form (SSIS-PF) measurement model demonstrate

adequate fit with standardization data?

2. Does the best fitting measurement model for the SSIS-PF demonstrate invariance across

race/ethnicity groups?

3. Does the best fitting measurement model for the SSIS-PF demonstrate invariance across

English and Spanish language formats?

4. Are norm-based scores produced by the SSIS-PF appropriate for use with individuals across

race/ethnicity groups?

5. Are norm-based scores produced by the SSIS-PF appropriate for use with individuals across

English and Spanish language formats?

30

Chapter 3

Method

Participants

Data examined in the current study were collected during the SSIS Rating Scale

development and standardization process.4 The full SSIS-PF standardization sample consisted of

N = 4,368 (English Version, n = 3,882; Spanish Version, n = 486). However, in order to control

for possible developmental differences, only data referencing children from the middle age group

(i.e., 5- to 12-year-olds) were considered. Through the SSIS standardization process, a stratified

norm sample was developed in alignment with March 2006 U.S. population estimates for

racial/ethnic group membership across the following groups: African American, Caucasian,

Latino,5 and Other (Gresham & Elliott, 2008). For the current study, data were obtained from the

combination of SSIS-PF English and Spanish language format samples. Demographic data for

the study sample is presented in Table 1.6

Measures

SSIS Rating Scales. The current study examined various measurement properties of

scores from the SSIS-PF (Gresham & Elliott, 2008). SSIS Rating Scales were developed as a

means of evaluating the behavior of children and adolescents across three interrelated domains:

social skills, problem behavior, and academic competence. The current investigation focuses on

the Social Skills domain only, which is comprised of seven subscales: Communication,

Cooperation, Assertion, Responsibility, Empathy, Engagement, and Self-Control. The SSIS–PF

4 Standardization data from the Social Skills Improvement System (SSIS). Copyright © 2007 NCS Pearson,

Inc. Used with permission. All rights reserved. 5 The terms Caucasian and Latino are used here to maintain consistency throughout the document. The

SSIS authors use the terms White and Hispanic, respectively, to refer to these race/ethnicity categories. 6 In reference to the distribution of participants across race/ethnicity categories, all groups are mutually

exclusive. Participants classified as ‘Other’ were removed from the sample prior to analysis.

31

Table 1.

Frequency Distributions for Demographic Variables among Participants from the SSIS Rating

Scales Standardization Sample that were Included in the Current Study.

AA

CA

LA

Total

(English)

Spanish

n (%) n (%) n (%) n (%) n (%)

Sexa

Girls 157 (16) 578 (58) 200 (20) 1000 (50) 164 (51)

Boys 155 (16) 586 (59) 200 (20) 1000 (50) 156 (49)

SESb

1 33 (11) 77 (7) 20 (14) 127 (8) 112 (36)

2 117 (38) 285 (25) 52 (36) 462 (28) 106 (34)

3 135 (43) 396 (34) 52 (36) 548 (34) 45 (15)

4 27 (9) 404 (35) 21 (15) 482 (30) 46 (15)

Region

NE 30 (10) 269 (23) 23 (16) 325 (20) 13 (4)

NC 125 (40) 266 (23) 21 (15) 404 (25) 28 (9)

SO 100 (32) 434 (37) 33 (23) 602 (37) 129 (42)

WS 57 (18) 193 (17) 68 (47) 288 (18) 139 (45)

Note. AA = African American; CA = Caucasian; LA = Latino; SES = socioeconomic status; Region = Geographic

Region; NE = Northeast; NC = North Central; SO = South; WS = West. a Data referencing participant ‘sex’ were not included in the dataset used for analyses. Information reported in the

table was taken directly from the SSIS Rating Scales Manual (Gresham & Elliott, 2008). b Maternal education was used as a proxy for SES during the SSIS Rating Scales standardization process. The

following scaling procedure was used to quantify SES via maternal education: 1 = 11th

grade or less; 2 = 12th

grade

or GED; 3 = 1 to 3 years of college; 4 = 4 or more years of college.

32

items are scored on a 4-point scale for frequency (Never, Seldom, Often, Almost Always) and a 3-

point scale for importance (Not Important, Important, Critical; Gresham & Elliott, 2008).

Analyses for the current study were conducted using frequency data only.

Through the standardization process, the authors of the SSIS gathered evidence to

support the reliability and validity of scores (see Gresham & Elliott, 2008).7 Score reliability was

examined via internal consistency, stability, and inter-rater reliability. Internal consistency

coefficient alpha estimates for the Social Skills subscales ranged from .83 - .92 (Social Skills

Scale, α = .97). Two month test-retest stability coefficients ranged from .68 - .86 across the seven

Social Skills subscales (Social Skills Total Scale, r = .84). Finally, interrater reliability

coefficients ranged from .35 - .70 (Social Skills Scale, r = .62).

Multiple forms of evidence supporting the validity of scores produced by SSIS Rating

Scales also were provided in the technical manual. Construct validity was examined through the

analysis of internal structure. Consistent with guiding theory, moderate to large negative

correlations were observed between scores on the Social Skills and Problem Behaviors scales and

subscales. The Social Skills and Problem Behaviors scales showed a large negative correlation (r

= -.49). Large positive correlations between Social Skills subscales also were observed (r = .42 -

.78). Confirmatory factor analysis (CFA) was conducted during the scale development phase of

the standardization process. CFA results were not thoroughly explained in the technical manual.

Those that were presented indicated “modest overall fit” (Gresham & Elliott, 2008, p. 31) with

the standardization data.

Additional validity evidence in the form of correlations with scores from other rating

scales designed to measure similar constructs also was provided. For Parent Form samples

7 Comprehensive reliability and validity data are offered in the technical manual (Gresham & Elliott, 2008).

Unless otherwise noted, all reliability and validity statistics reported in text refer to data collected from the

5- to 12-year-old Parent Form subsample.

33

specifically, concurrent relationships were examined between scores on the SSIS Social Skills

scale and scores on the SSRS Social Skills scale (r = .73), the Behavior Assessment System for

Children, Second Edition (BASC-2) Adaptive Skills scale (r = .62), the Vineland-II Socialization

scale (r = .44), and the Home and Community Social Behavior Scales (HCSBS) Social

Competence scale (r = .77). Finally, SSIS scores were shown to accurately differentiate between

non-clinical and clinical samples (e.g., Autistic, Attention Deficit/Hyperactivity Disorder,

Emotional Disturbance, and Intellectual Disability groups). Given the focus of the current study,

it should be noted that the validity studies which examined the SSIS Rating Scales’ relationships

with other measures and ability to differentiate between clinical and non-clinical groups were

conducted in reference to predominantly Caucasian student samples. Thus, the generalizability of

such validity evidence across race/ethnicity groups is unknown.

SSIS Rating Scales, Spanish Format. The Spanish language versions of the SSIS

Rating Scales were developed through the application of systematic translation procedures. After

translation was complete, preliminary analyses were conducted to examine the reliability of

scores produced from the Spanish version of the instrument. Item-total correlations and internal

consistency reliability coefficients were calculated and compared with those observed for scores

from the English version. For the SSIS-PF Spanish format, item-total correlations for Social

Skills subscales ranged from .32 - .70, and internal consistency coefficient alpha estimates ranged

from .75 - .84 (Social Skills Scale, α = .95). Though not tested for statistical significance, the

authors concluded that item-total correlations and internal consistency coefficients were similar

across language format. Thus, the reliability of scores produced on the Spanish format SSIS

Ratings Scales was tentatively supported. Specific examinations of Spanish format score validity,

however, were not conducted.

34

Procedures

SSIS Rating Scales standardization data were collected from a national sample of 4,700

children ages 3 through 18 years. Data collection was conducted from September 2006 through

October 2007. Teacher participants were recruited by site coordinators at 115 data collection

sites across 36 states. Participating teachers distributed data collection packets to all students in

their respective classes. Consent forms, demographic information, and Parent Form data were

collected first. Upon obtaining informed consent and Parent Form data, additional Teacher and

Student Form data were collected in alignment with targets for demographic group

representation. Known Spanish-speaking parents were provided with English and Spanish

versions of all data collection forms. Additional Spanish forms were distributed to individuals

identified as Spanish-speaking via returned English language consent forms. For more specific

details regarding SSIS Rating Scale standardization data collection please refer to the technical

manual (Gresham & Elliott, 2008).

Design and Analysis

The current study examined measurement invariance of the SSIS-PF as a function of

student race/ethnicity and rating scale language format (i.e., English/Spanish). Multiple stages of

data analysis were applied to investigate different forms of measurement invariance. The same

procedures were used for investigating invariance across both sets of grouping variables (i.e.,

race/ethnicity and language format). Due to sampling restrictions (i.e., only Latino participants

completed Spanish-language SSIS Rating Scales), the studies of invariance by race/ethnicity and

language format were conducted separately.

Structural Analyses. Primary analyses examined invariance in the SSIS-PF

measurement model across groups defined by race/ethnicity and language format, respectively.

Specifically, iterative CFA procedures were used to examine invariance in the SSIS-PF

35

measurement model across race/ethnicity and language format groups. The CFAs were

conducted using MPlus 6.11 software (Muthén & Muthén, 2011). Item-level data (i.e., 4-point

frequency ratings) were not available for analysis.8 Instead, item parcels were used as indicators

of latent factors. Though not ideal for the analysis of measurement invariance, item-parcels have

the advantage of approximating continuously scaled and normally distributed indicators (Hau &

Marsh, 2004). Modified maximum likelihood (MLM) estimation methods were used for all CFA

analyses.

Parceling Technique. Two unique sets of item parcels were created for use in the

current study. In both cases, items were assigned to parcels according to item-total subscale

correlations, and all parcels were created with as few items as possible (i.e., two items where

possible, three where necessary).9 However, slightly different procedures were used to create

parcels according to the parameters of the two separate invariance studies.

Because three groups were involved in the analysis of invariance as a function of

race/ethnicity, specifying parcels on the basis of systematic differences in item-total correlations

across groups was not feasible. As such, the parcels used in the race/ethnicity invariance analyses

were created by pairing items according to the magnitude of item-total correlations on each SSIS-

PF subscale for the full standardization sample. This strategy applies a rationale similar to that

adopted for Cattell’s radial item parceling technique (1956, as cited in Bandalos & Finney, 2001).

Essentially, items with the strongest relationships to the underlying factor (subscale) were paired

together first, and the process continued such that the final parcel was created with items

demonstrating the weakest presumed factor relationships. Results of the systematic item-

parceling process are presented in Table 2.

8 Citing company policy, Pearson, Inc., publisher of the SSIS Rating Scales, agreed to release

standardization data in parcel format only. 9 For 7-item subscales, one parcel was comprised of three items.

36

Table 2

Summary of Item-Parcel Assignment for Analysis of SSIS-PF Measurement Invariance by Race/Ethnicity and Language Format

Item-Total Correlation Parcel Assignment

Item English Spanish Difference Language Race

Communication

Item #4 .50 .53 -.03 2 2

Item #10 .40 .48 -.08 3 3

Item #14 .50 .55 -.05 3 1

Item #20 .40 .32 .08 1 3

Item #24 .46 .51 -.05 2 2

Item #30 .50 .53 -.03 1 1

Item #40 .43 .48 -.05 3 3

(continued)

37

Table 2 (continued)

Summary of Item-Parcel Assignment for Analysis of SSIS-PF Measurement Invariance by Race/Ethnicity and Language Format

Item-Total Correlation Parcel Assignment

Item English Spanish Difference Language Race

Cooperation

Item #2 .64 .64 .00 4 4

Item #7 .66 .66 .00 5 4

Item #12 .56 .61 -.05 6 6

Item #17 .62 .63 -.01 5 5

Item #27 .58 .45 .13 4 5

Item #37 .55 .60 -.05 6 6

(continued)

38

Table 2 (continued)

Summary of Item-Parcel Assignment for Analysis of SSIS-PF Measurement Invariance by Race/Ethnicity and Language Format

Item-Total Correlation Parcel Assignment

Item English Spanish Difference Language Race

Assertion

Item #1 .51 .41 .10 7 7

Item #5 .43 .41 .02 8 9

Item #11 .46 .38 .08 7 8

Item #15 .47 .55 -.08 9 8

Item #25 .43 .44 -.01 8 9

Item #35 .43 .46 -.03 9 9

Item #45 .52 .41 .11 7 7

(continued)

39

Table 2 (continued)

Summary of Item-Parcel Assignment for Analysis of SSIS-PF Measurement Invariance by Race/Ethnicity and Language Format

Item-Total Correlation Parcel Assignment

Item English Spanish Difference Language Race

Responsibility

Item #6 .65 .63 .02 10 11

Item #16 .55 .54 .01 11 12

Item #22 .63 .67 -.04 12 11

Item #26 .69 .70 -.01 12 10

Item #32 .51 .51 .00 11 12

Item #42 .67 .65 .02 10 10

(continued)

40

Table 2 (continued)

Summary of Item-Parcel Assignment for Analysis of SSIS-PF Measurement Invariance by Race/Ethnicity and Language Format

Item-Total Correlation Parcel Assignment

Item English Spanish Difference Language Race

Empathy

Item #3 .61 .49 .12 13 15

Item #8 .70 .65 .05 14 13

Item #13 .54 .60 -.06 15 15

Item #18 .65 .58 .07 13 14

Item #28 .68 .61 .07 14 14

Item #38 .70 .66 .04 15 13

(continued)

41

Table 2 (continued)

Summary of Item-Parcel Assignment for Analysis of SSIS-PF Measurement Invariance by Race/Ethnicity and Language Format

Item-Total Correlation Parcel Assignment

Item English Spanish Difference Language Race

Engagement

Item #9 .57 .51 .06 16 17

Item #19 .70 .72 -.02 18 16

Item #23 .60 .59 .01 17 17

Item #29 .50 .45 .05 17 18

Item #33 .64 .56 .08 16 16

Item #39 .57 .60 -.03 18 17

Item #43 .52 .49 .03 17 18

(continued)

42

Table 2 (continued)

Summary of Item-Parcel Assignment for Analysis of SSIS-PF Measurement Invariance by Race/Ethnicity and Language Format

Item-Total Correlation Parcel Assignment

Item English Spanish Difference Language Race

Self-Control

Item #21 .64 .58 .06 20 19

Item #31 .62 .53 .09 19 20

Item #34 .62 .53 .09 19 20

Item #36 .55 .50 .05 20 21

Item #41 .57 .54 .03 21 21

Item #44 .49 .48 .01 21 21

Item #46 .63 .56 .07 20 19

Note. Item content has been omitted in compliance with terms of the license agreement with Pearson, Inc. Item numbers reported in table, however, correspond

with the actual numbering of items on the SSIS-PF. Item-Total Correlations refer to those observed for the 5-12 age group of the SSIS-PF standardization

sample, as reported the SSIS Rating Scales Manual (Gresham & Elliott, 2008). Parcel assignment for Language Format (Language) was determined according to

the consistency of item-total correlations across English and Spanish forms. Parcel assignment for Race/Ethnicity (Race) was determined according to the

magnitude of item-total correlations for aggregated English form standardization data.

43

For the invariance analysis across language format groups, item-parcels were

systematically developed in order to minimize the likelihood that measurement noninvariance

would be obscured at the level of individual latent factor indicators. To do this, item-total

correlation difference scores were calculated for each item across the two groups (i.e., English

and Spanish). Once each item’s difference score had been calculated, parcels were created by

grouping together the two (or three) items with the closest difference scores. A statistical

simulation study conducted by Meade and Kroustalis (2006) showed that measurement

noninvariance was least likely to be obscured when differentially functioning items were grouped

within a single parcel. Thus, the process of grouping items into parcels according to the

magnitude of item-total correlation difference scores was done intentionally to isolate potential

sources of noninvariance. In effect, items that were most dissimilar across groups in terms of the

magnitude of their relationship to the presumed latent factor (i.e., subscale) were targeted as

potential sources of noninvariance and grouped together. Finally, in instances when difference

score parceling was inconclusive (e.g., 3+ difference scores of equal magnitude), qualitative

analysis was used to assign items to specific parcels on the basis of item-content (see Table 2).

Assessing Model Fit. Model fit was evaluated through examination of global fit

statistics and modification indices. Fit indices and criteria for determining fit were as follows:

root mean square error of approximation (RMSEA) and 90% confidence interval (RMSEA ≤ .06;

90% CI upper limit ≤ .10); comparative fit index (CFI > .95); Tucker-Lewis Index (TLI > .95);

and standardized root mean squared residual (SRMR < .08; Byrne, 2012; Hu & Bentler, 1999).

In addition to assessing the adequacy of global model fit, the statistical and practical significance

of estimated model parameters also was assessed. T-tests were used to examine statistical

significance of parameter estimates. Practical significance was assessed through a review of

standardized model parameters (i.e., factor loadings and factor correlations). Where necessary,

44

model respecificaiton was guided by an inspection of modification indices and other pertinent

technical output. The need for model respecification was informed by three separate

considerations: (a) the adequacy of initial global fit statistics, (b) the projected significance of

newly estimated parameters, and (c) the substantive meaningfulness of new parameters.

Analysis of Invariance. The multi-step process outlined by Byrne (2012) was applied in

separate sets of analyses in order to systematically test for invariance in the SSIS-PF

measurement model as a function of race/ethnicity (African American, Caucasian, Latino) and

language format (English, Spanish), respectively. Recommendations offered by Vandenberg and

Lance (2000) also were used as a reference during analysis. As a preliminary step in the analysis

of invariance, the viability of the implied measurement model for the SSIS Rating Scales10

was

examined with data from a randomly selected subsample from each dataset. Through this first set

of structural analyses, a best fitting measurement model was specified and retained as a baseline.

The baseline model was then independently fitted to datasets for each group, and the need for

group-specific model respecification was examined. Next, increasingly restrictive constraints

were applied to the baseline model in order to test for configural, metric, and structural

invariance, respectively. The baseline model was first specified for configural invariance (i.e.,

model parameters constrained to match baseline configuration, magnitude of individual loadings

free to vary across groups) across all groups. Borrowing from procedures used by Van Horn et

al. (2007), the magnitude of unconstrained factor loadings for all indicators were compared at this

stage to examine potential loci of measurement invariance. Next, further model constraints were

applied, specifying invariance for the individual indicator-loadings across groups. Finally,

10

The term implied is used here to acknowledge the fact that the SSIS Rating Scales Manual does not

explicitly define/depict a measurement model for the instrument. CFA conducted during the scale

development process used a first-order model with all seven first-order factors specified to covary with

each other. However, the use of a global Social Skills score also implies the presence of a unitary higher-

order social skills factor. As such, both first-order and higher-order models were examined in the current

study.

45

structural components of the model (i.e., factor variances and covariances) were constrained for

equality. At each stage, global fit statistics, individual model parameters, and modification

indices were examined. The statistical significance of differences in model fit between

constrained (nested) and unconstrained models was examined through chi-square difference

testing. Model improvements were made where appropriate. All resulting modifications were

carried through to each successive stage of analysis.

46

Chapter 4

Results

Preliminary Analyses and Data Preparation

Means, standard deviations, and correlations for all variables are presented in Tables 3 -

8. Prior to the initiation of invariance testing, data were screened to check the assumptions of

normality, linearity, heteroscedasticity, and multicollinearity. Bivariate scatterplots supported

linear relations between all variables, and an examination of the bivariate correlation matrix

supported the relative uniqueness of all indicators. Skew and kurtosis did not yield evidence of

extreme non-normality. However, moderate negative skew was observed for all indicator

distributions. The negative skew of indicator distributions also contributed to a moderate level of

heteroscedasticity. As such, the robust MLM estimator was used for all SEM analyses to control

for potential deviations from the assumption of multivariate normality.

Given that item-parcels were used as latent factor indicators for all models, preliminary

comparisons were performed in order to assess the degree to which the use of item-parcels, as

compared to single-item indicators, might influence results. First, internal consistency

reliabilities of the seven social skills subscales were calculated with item-parcel data and then

compared with the corresponding internal consistency reliability statistics reported in the SSIS

Rating Scales Manual. Second, subscale total-scores were calculated with item-parcel data.

Subscale total-score correlations were then compared with manual reported subscale

intercorrelations. As shown in Tables 9 and 10, differences in subscale internal consistency

reliabilities and subscale correlations were generally small, which provides some justification for

the use of item parcels as latent factor indicators in the current analysis. Bivariate correlations

47

Table 3

Means and Standard Deviations for Item-Parcels and Control Variables used in Analysis of Invariance by Race/Ethnicity.

Race/Ethnicity Groups

Full Sample African American Caucasian Latino

M SD M SD M SD M SD

P1 2.21 0.52 2.25 0.57 2.20 0.49 2.22 0.59

P2 2.42 0.55 2.43 0.59 2.41 0.54 2.51 0.60

P3 2.23 0.48 2.13 0.51 2.26 0.46 2.23 0.54

P4 2.15 0.57 2.17 0.62 2.13 0.55 2.26 0.60

P5 2.08 0.55 2.07 0.61 2.07 0.53 2.20 0.6

P6 2.31 0.52 2.29 0.59 2.30 0.50 2.43 0.52

P7 2.20 0.60 2.19 0.69 2.21 0.57 2.21 0.66

P8 2.11 0.58 2.09 0.64 2.11 0.56 2.09 0.62

P9 2.13 0.50 2.15 0.55 2.11 0.48 2.20 0.57

P10 1.95 0.67 1.92 0.74 1.95 0.64 2.08 0.74

P11 2.30 0.60 2.21 0.66 2.51 0.58 2.43 0.62

P12 2.24 0.55 2.13 0.62 2.26 0.52 2.29 0.58

P13 2.36 0.58 2.30 0.63 2.37 0.57 2.42 0.59

P14 2.10 0.64 2.03 0.68 2.11 0.62 2.17 0.67

P15 2.22 0.56 2.27 0.61 2.20 0.55 2.29 0.60

P16 2.09 0.65 2.10 0.72 2.10 0.62 2.10 0.75

P17 2.22 0.55 2.24 0.57 2.21 0.54 2.22 0.60

P18 2.20 0.54 2.20 0.56 2.21 0.52 2.17 0.62

P19 1.73 0.63 1.78 0.68 1.71 0.61 1.83 0.66

P20 1.50 0.65 1.38 0.74 1.51 0.61 1.60 0.74

P21 1.83 0.55 1.71 0.63 1.86 0.51 1.86 0.64

Age

107.28

27.38

107.66

28.09

106.47

27.39

112.94

25.09

SES 2.86 0.94 2.59 0.90 2.97 0.93 2.51 0.91 Note. P1-P21 are latent factor indicator parcels developed for the analysis of invariance by race/ethnicity (see Table 2). Age is child-age in months. SES is

socioeconomic status. Sample sizes are: Full Sample (N = 1619); African American (n = 312); Caucasian (n = 1162); Latino (n = 145).

48

Table 4

Means and Standard Deviations for Item-Parcels and Control Variables used in Analysis of Invariance by Language Format.

Language Format Groups

Full Sample English Spanish

M SD M SD M SD

P1 2.33 0.56 2.34 0.54 2.28 0.66

P2 3.44 0.56 2.42 0.55 2.53 0.58

P3 2.17 0.51 2.15 0.49 2.30 0.58

P4 2.14 0.59 2.10 0.48 2.35 0.63

P5 2.17 0.55 2.13 0.53 2.37 0.60

P6 2.33 0.53 2.31 0.52 2.45 0.57

P7 2.25 0.55 2.20 0.54 2.45 0.53

P8 2.16 0.57 2.15 0.55 2.21 0.66

P9 2.08 0.61 2.04 0.60 2.25 0.65

P10 2.12 0.63 2.09 0.61 2.28 0.68

P11 2.24 0.56 2.24 0.55 2.28 0.60

P12 2.20 0.60 2.16 0.58 2.39 0.66

P13 2.12 0.64 2.07 0.63 2.34 0.63

P14 2.25 0.63 2.24 0.62 2.27 0.68

P15 2.38 0.55 2.37 0.54 2.42 0.60

P16 2.04 0.64 2.01 0.63 2.16 0.66

P17 2.26 0.53 2.26 0.52 2.27 0.57

P18 2.25 0.58 2.22 0.57 2.38 0.61

P19 1.55 0.68 1.50 0.65 1.81 0.74

P20 1.81 0.61 1.75 0.59 2.11 0.64

P21 1.88 0.63 1.86 0.62 1.97 0.70

Age

108.31

26.85

107.28

27.38

113.45

23.46

SES 2.73 1.00 2.86 0.94 2.07 1.05 Note. P1-P21 are latent factor indicator parcels developed for the analysis of invariance by language format (see Table #). Age is child-age in months. SES is

socioeconomic status. Sample sizes are: Full Sample (N = 1941); English Format (n = 1619); Spanish Format (n = 320).

49

Table 5

Bivariate Correlations of SSIS Rating Scale Latent Factor Indicators and Control Variables for Full Race/Ethnicity Invariance

Sample and African American Subsample.

Parcel Indicators – Race/Ethnicity Invariance Analysis

P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12

P1 -- .360 .516 .461 .472 .526 .446 .413 .436 .484 .519 .454

P2 .375 -- .370 .390 .332 .476 .318 .345 .346 .425 .363 .407

P3 .560 .361 -- .394 .451 .497 .360 .438 .363 .390 .392 .484

P4 .505 .391 .444 -- .658 .562 .384 .363 .347 .587 .593 .520

P5 .510 .358 .498 .644 -- .591 .326 .398 .355 .566 .490 .572

P6 .554 .422 .534 .562 .595 -- .448 .359 .432 .611 .547 .587

P7 .351 .201 .329 .269 .250 .298 -- .512 .582 .393 .384 .328

P8 .384 .344 .418 .336 .343 .401 .523 -- .550 .467 .387 .421

P9 .441 .327 .378 .325 .338 .394 .555 .522 -- .370 .451 .400

P10 .482 .374 .424 .568 .533 .544 .264 .411 .343 -- .644 .544

P11 .485 .364 .454 .559 .535 .539 .266 .375 .366 .584 -- .564

P12 .445 .350 .508 .544 .594 .561 .235 .366 .317 .523 .531 --

P13 .446 .425 .431 .393 .363 .454 .328 .504 .414 .467 .427 .394

P14 .435 .408 .435 .377 .402 .481 .350 .544 .414 .538 .429 .426

P15 .502 .418 .431 .497 .442 .527 .331 .505 .440 .541 .457 .432

P16 .459 .311 .365 .260 .247 .317 .431 .443 .483 .333 .298 .237

P17 .487 .330 .420 .313 .329 .412 .448 .499 .519 .380 .343 .315

P18 .528 .332 .412 .330 .307 .394 .388 .438 .485 .359 .320 .321

P19 .493 .346 .447 .524 .550 .525 .187 .339 .312 .557 .450 .481

P20 .421 .276 .412 .410 .476 .466 .208 .342 .271 .528 .407 .408

P21 .524 .351 .489 .472 .503 .529 .299 .414 .388 .561 .499 .491

Age .023 -.001 .087 .038 .059 .072 -.077 -.024 -.040 .102 .147 .129

SES .091 .010 .160 .065 .041 .073 .026 .023 -.010 -.003 .040 .098

50

Table 5 continued

Parcel Indicators – Race/Ethnicity Invariance Analysis Control Variables

P13 P14 P15 P16 P17 P18 P19 P20 P21 Age SES

P1 .435 .422 .499 .480 .481 .578 .452 .392 .517 -.004 .043

P2 .479 .413 .414 .299 .331 .314 .344 .260 .335 -.087 .089

P3 .426 .406 .435 .316 .433 .415 .410 .403 .478 .003 .185

P4 .412 .367 .478 .264 .337 .291 .543 .459 .490 .033 .031

P5 .326 .362 .413 .286 .348 .325 .571 .473 .465 .065 .017

P6 .487 .486 .476 .364 .452 .424 .509 .448 .521 .025 .116

P7 .409 .457 .467 .496 .527 .475 .318 .293 .377 -.051 -.010

P8 .504 .508 .593 .412 .533 .473 .378 .377 .445 -.092 .051

P9 .453 .452 .531 .524 .546 .541 .320 .314 .390 -.070 -.009

P10 .524 .576 .542 .363 .425 .395 .562 .556 .628 .046 .110

P11 .409 .474 .511 .354 .434 .332 .447 .569 .496 .068 .047

P12 .447 .407 .453 .237 .356 .341 .532 .426 .541 .127 .125

P13 -- .741 .651 .445 .521 .438 .460 .424 .567 .050 0.87

P14 .728 -- .585 .425 .527 .427 .546 .428 .545 -.001 0.69

P15 .626 .635 -- .432 .526 .464 .454 .405 .547 -.040 .101

P16 .437 .426 .396 -- .675 .639 .321 .309 .384 -.101 .019

P17 .489 .485 .452 .652 -- .595 .363 .379 .423 -.072 -.017

P18 .465 .438 .451 .625 .609 -- .341 .335 .432 -.141 .075

P19 .405 .463 .503 .281 .342 .348 -- .630 .686 .082 .070

P20 .342 .388 .398 .294 .355 .312 .616 -- .584 .144 .049

P21 .475 .527 .511 .363 .403 .411 .665 .585 -- .039 .121

Age .036 .050 .007 -.009 -.068 -.057 .099 .120 .050 -- -.138

SES .027 .032 -.024 .027 -.016 .023 .042 .074 .112 -- Note. Correlations for the full sample (n = 1619) are presented below the diagonal. Correlations for the African American subsample (n = 312) are presented

above the diagonal. P1-P21 are latent factor indicator parcels developed for the analysis of invariance by race/ethnicity. Age is child-age in months. SES is

socioeconomic status.

51

Table 6

Bivariate Correlations of SSIS Rating Scale Latent Factor Indicators and Control Variables for Caucasian and Latino Subsamples in

Race/Ethnicity Invariance Analysis.

Parcel Indicators – Race/Ethnicity Invariance Analysis

P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12

P1 -- .392 .464 .492 .484 .515 .350 .217 .410 .458 .490 .389

P2 .378 -- .466 .474 .411 .527 .315 .423 .327 .356 .416 .331

P3 .603 .347 -- .315 .493 .459 .453 .348 .414 .482 .429 .492

P4 .523 .375 .489 -- .672 .552 .284 .219 .391 .563 .591 .501

P5 .530 .355 .521 .633 -- .620 .335 .229 .376 .575 .548 .608

P6 .573 .385 .566 .561 .591 -- .367 .354 .426 .531 .583 .534

P7 .316 .141 .298 .224 .207 .230 -- .462 .556 .367 .324 .343

P8 .402 .333 .423 .346 .344 .426 .537 -- .518 .335 .297 .367

P9 .448 .319 .385 .304 .324 .374 .547 .515 -- .384 .423 .363

P10 .488 .357 .429 .561 .542 .519 .198 .405 .325 -- .644 .484

P11 .479 .357 .476 .544 .547 .528 .214 .384 .328 .552 -- .496

P12 .460 .336 .514 .565 .603 .557 .180 .346 .284 .523 .519 --

P13 .438 .401 .413 .390 .364 .428 .282 .506 .390 .437 .386 .365

P14 .439 .407 .431 .391 .414 .471 .292 .556 .393 .513 .400 .426

P15 .493 .398 .432 .502 .434 .535 .265 .483 .396 .538 .435 .432

P16 .467 .306 .361 .260 .246 .303 .401 .450 .477 .321 .280 .228

P17 .496 .332 .418 .310 .332 .394 .401 .479 .496 .357 .311 .303

P18 .525 .339 .412 .356 .313 .395 .337 .422 .461 .355 .317 .317

P19 .495 .344 .473 .523 .548 .533 .124 .336 .293 .555 .44 .458

P20 .438 .273 .427 .399 .481 .470 .157 .320 .238 .508 .383 .391

P21 .542 .353 .483 .499 .536 .546 .241 .403 .389 .534 .495 .461

Age .023 .014 .109 .015 .053 .079 -.095 .006 -.047 .096 .155 .139

SES .127 -.017 .150 .117 .081 .076 .046 .013 .012 -.008 .041 .089

52

Table 6 continued

Parcel Indicators – Race/Ethnicity Invariance Analysis Control Variables

P13 P14 P15 P16 P17 P18 P19 P20 P21 Age SES

P1 .550 .471 .563 .380 .450 .459 .564 .432 .509 .085 .025

P2 .472 .405 .539 .376 .318 .341 .346 .336 .414 .044 .137

P3 .540 .500 .488 .497 .449 .417 .429 .315 .503 .144 .068

P4 .380 .312 .484 .258 .285 .269 .470 .378 .320 .187 -.118

P5 .434 .404 .549 .174 .279 .253 .508 .454 .424 .046 -.102

P6 .563 .538 .592 .311 .461 .356 .500 .485 .484 .096 .068

P7 .444 .470 .446 .459. 584 .509 .302 .304 .448 -.005 -.041

P8 .493 .548 .473 .470 .565 .462 .296 .403 .425 -.084 -.002

P9 .506 .477 .509 .439 .608 .526 .399 .383 .420 .048 0.076

P10 .532 .609 .566 .344 .439 .326 .555 .576 .588 .249 -.166

P11 .547 .502 .515 .299 .396 .339 .517 .374 .515 .253 011

P12 .434 .442 .441 .306 .348 .320 .570 .426 .509 .067 .021

P13 -- .763 .683 .456 .546 .484 .508 .441 .553 .081 .028

P14 .717 -- .614 .544 .563 .505 .497 .490 .643 .044 .032

P15 .615 .660 -- .440 .484 .408 .597 .526 .560 .032 -.060

P16 .434 .410 .378 -- .614 .644 .313 .313 .410 .076 .065

P17 .475 .464 .424 .653 -- .646 .359 .440 .448 -.023 .047

P18 .473 .433 .456 .618 .609 -- .319 .222 .495 -.013 .034

P19 .375 .464 .502 .263 .332 .358 -- .625 .641 .140 -.078

P20 .293 .352 .381 .288 .340 .322 .619 -- .570 .064 -.106

P21 .427 .499 .504 .349 .398 .391 .677 .583 -- .036 .008

Age .025 .065 .013 .009 -.073 -.037 .094 .119 .057 -- -.093

SES .004 .017 -.038 .026 -.018 .002 .069 .102 .107 -.003 -- Note. Correlations for the Caucasian subsample (n = 1162) are presented below the diagonal. Correlations for the Latino subsample (n = 145) are presented

above the diagonal. P1-P21 are latent factor indicator parcels developed for the analysis of invariance by race/ethnicity. Age is child-age in months. SES is

socioeconomic status.

53

Table 7

Bivariate Correlations of SSIS Rating Scale Latent Factor Indicators and Control Variables for Full Language Format Invariance

Sample.

Parcel Indicators – Race/Ethnicity Invariance Analysis

P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12

P1 --

P2 .325 --

P3 .479 .406 --

P4 .426 .394 .542 --

P5 .385 .411 .534 .689 --

P6 .466 .433 .581 .606 .598 --

P7 .373 .314 .407 .342 .360 --

P8 .370 .256 .300 .238 .242 .296 .512 --

P9 .401 .361 .406 .383 .349 .426 .529 .443 --

P10 .438 .409 .520 .625 .587 .586 .367 .303 .439 --

P11 .374 .368 .528 .581 .584 .565 .315 .225 .378 .562 --

P12 .452 .423 .523 .663 .589 .589 .350 .298 .429 .788 .553 --

P13 .405 .411 .472 .499 .484 .532 .436 .337 .531 .565 .466 .569

P14 .425 .425 .393 .385 .350 .423 .387 .354 .542 .460 .366 .469

P15 .500 .426 .441 .447 .403 .503 .398 .370 .497 .501 .404 .516

P16 .438 .326 .364 .307 .285 .348 .447 .440 .487 .372 .264 .367

P17 .504 .346 .426 .318 .319 .418 .458 .425 .483 .367 .306 .374

P18 .483 .345 .407 .305 .301 .411 .481 .429 .540 .386 .297 .378

P19 .346 .289 .469 .487 .420 .461 .286 .206 .358 .503 .399 .506

P20 .403 .372 .545 .574 .528 .536 .331 .280 .431 .566 .494 .577

P21 .434 .328 .489 .496 .428 .491 .330 .283 .378 .490 .465 .513

Age .015 -.003 .092 .065 .057 .071 -.106 -.003 .043 .121 .123 .142

SES .151 .000 .053 .017 -.028 .031 -.026 .003 -.038 -.043 .056 -.017

54

Table 7 continued

Parcel Indicators – Race/Ethnicity Invariance Analysis Control Variables

P13 P14 P15 P16 P17 P18 P19 P20 P21 Age SES

P1

P2

P3

P4

P5

P6

P7

P8

P9

P10

P11

P12

P13 --

P14 .639 --

P15 .618 .672 --

P16 .438 .468 .439 --

P17 .421 .474 .481 .634 --

P18 .451 .492 .477 .670 .666 --

P19 .443 .336 .385 .343 .345 .339 --

P20 .568 .444 .487 .372 .375 .360 .606 --

P21 .462 .410 .438 .346 .382 .382 .558 .580 --

Age .071 .003 .020 -.037 -.078 -.012 .130 .116 .049 --

SES -.072 .013 .025 -.032 .016 .002 -.002 -.020 .074 -.075 -- Note. Correlations for the full sample (n = 1941). P1-P21 are latent factor indicator parcels developed for the analysis of invariance by language format. Age is

child-age in months. SES is socioeconomic status.

55

Table 8

Bivariate Correlations of SSIS Rating Scale Latent Factor Indicators and Control Variables for English Language Format and

Spanish Language Format Subsamples in Language Format Invariance Analysis.

Parcel Indicators – Race/Ethnicity Invariance Analysis

P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12

P1 -- .329 .464 .432 .347 .410 .395 .365 .418 .398 .300 .501

P2 .330 -- .492 .460 .493 .463 .508 .282 .361 .456 .441 .531

P3 .492 .380 -- .581 .559 .634 .481 .309 .434 .609 .536 .611

P4 .439 .372 .522 -- .684 .626 .394 .209 .359 .670 .621 .678

P5 .412 .385 .518 .680 -- .672 .414 .176 .317 .656 .632 .662

P6 .490 .422 .561 .595 .574 -- .427 .254 .441 .602 .584 .621

P7 .384 .268 .378 .311 .327 .363 -- .413 .530 .399 .403 .401

P8 .374 .249 .294 .242 .256 .304 .537 -- .480 .272 .187 .286

P9 .407 .355 .388 .373 .341 .412 .518 .432 -- .366 .382 .430

P10 .458 .393 .489 .607 .561 .576 .346 .307 .446 -- .571 .759

P11 .396 .350 .527 .574 .476 .561 .296 .233 .375 .560 -- .530

P12 .452 .392 .491 .651 .559 .575 .321 .299 .417 .792 .560 --

P13 .417 .404 .449 .477 .462 .527 .407 .333 .520 .555 .466 .549

P14 .423 .419 .380 370 .330 .420 .375 .344 .542 .463 .361 .462

P15 .501 .419 .438 .443 .391 .505 .381 .371 .487 .395 .416 .507

P16 .451 .307 .332 .285 .261 .322 .438 .431 .484 .362 .242 .351

P17 .516 .339 .427 .330 .337 .409 .465 .430 .491 .372 .324 .373

P18 .509 .341 .404 .309 .300 .410 .483 .435 .542 .384 .316 .367

P19 .349 .276 .449 .487 .404 .466 .256 .202 .349 .501 .408 .500

P20 .400 .352 .518 .559 .510 .535 .281 .255 .406 .561 .488 .564

P21 .448 .325 .471 .491 .413 .486 .322 .284 .378 .499 .457 .520

Age .019 -.001 .088 .050 .048 .072 -.133 -.006 .037 .111 .129 .152

SES .153 .010 .110 .069 .037 .073 .029 -.007 .002 .002 .098 .035

56

Table 8 continued

Parcel Indicators – Race/Ethnicity Invariance Analysis Control Variables

P13 P14 P15 P16 P17 P18 P19 P20 P21 Age SES

P1 .419 .442 .501 .420 .464 .419 .393 .502 .397 .013 .116

P2 .417 .460 .464 .393 .384 .340 .307 .428 .332 -.057 .074

P3 .533 .446 .453 .468 .430 .387 .496 .610 .539 .060 .012

P4 .540 .457 .471 .357 .281 .227 .417 .568 .499 .069 .045

P5 .508 .443 .453 .335 .262 .245 .398 .516 .467 .024 -.039

P6 .520 .438 .497 .428 .459 .383 .396 .504 .493 .010 .021

P7 .496 .459 .482 .453 .457 .421 .303 .425 .328 -.061 .002

P8 .347 .391 .359 .470 .402 .400 .203 .366 .267 -.006 .095

P9 .529 .554 .543 .472 .462 .495 .318 .459 .349 .013 -.014

P10 .571 .453 .524 .383 .353 .355 .467 .548 .437 .121 -.067

P11 .474 .381 .352 .351 .230 .206 .366 .543 .491 .076 -.056

P12 .611 .509 .56i4 .396 .390 .376 .469 .577 .473 .033 -.011

P13 -- .617 .613 .480 .440 .441 .484 .564 .494 .064 -.059

P14 .651 -- .652 .519 .478 .470 .392 .514 .403 -.081 .020

P15 .623 .676 -- .474 .481 .482 .408 .518 .370 .016 -.006

P16 .420 .457 .430 -- .579 .607 .391 .494 .412 -.022 .018

P17 .422 .473 .480 .649 -- .651 .408 .431 .311 -.128 .072

P18 .443 .498 .475 .680 .673 -- .321 .395 .316 -.068 .032

P19 .416 .324 .380 .321 .334 .329 -- .539 .551 .104 -.044

P20 .552 .436 .486 .333 .370 .336 .604 -- .579 .153 -.010

P21 .450 .411 .453 .326 .399 .392 .556 .580 -- .063 .054

Age .058 .017 .018 -.049 -.070 -.013 .120 .090 .039 -- -.091

SES -.022 .019 .046 -.012 .006 .035 .074 .060 .109 -.046 -- Note. Correlations for the English Language Format subsample (n = 1612) are presented below the diagonal. Correlations for the Spanish Language Format

subsample (n = 320) are presented above the diagonal. P1-P21 are latent factor indicator parcels developed for the analysis of invariance by language format.

Age is child-age in months. SES is socioeconomic status.

57

Table 9

Internal Consistency Coefficients and Intercorrelations for SSIS-PF Social Skills Subscales

based on Race/Ethnicity Item Parcels and Manual Reported Item-level Data.

COMM

COOP

ASRT

RESP

EMP

ENG

SC

Coefficient

Alpha

Communication -- .69 .55 .66 .63 .58 .63 .74

Cooperation .704 -- .47 .78 .57 .42 .67 .83

Assertion .525 .455 -- .48 .59 .65 .46 .75

Responsibility .658 .770 .470 -- .62 .44 .67 .84

Empathy .602 .548 .577 .591 -- .60 .60 .86

Engagement .586 .422 .626 .435 .587 -- .48 .83

Self-Control .593 .645 .413 .674 .577 .430 -- .84

Chronbach’s

Alpha

.704

.828

.767

.772

.852

.825

.819

Note. COMM = Communication; COOP = Cooperation; ASRT = Assertion; RESP = Responsibility; EMP =

Empathy; ENG = Engagement; SC = Self-Control. SSIS-PF Internal Consistency Coefficients and Intercorrelations

based on Race/Ethnicity item parcels are presented below the diagonal. Statistics reported above the diagonal are

taken directly from SSIS Rating Scales Manual.

58

Table 10

Internal Consistency Coefficients and Intercorrelations for SSIS-PF Social Skills Subscales

based on Language Format Item Parcels and Manual Reported Item-level Data.

COMM

COOP

ASRT

RESP

EMP

ENG

SC

Coefficient

Alpha

Communication -- .69 .55 .66 .63 .58 .63 .74

Cooperation .673 -- .47 .78 .57 .42 .67 .83

Assertion .548 .456 -- .48 .59 .65 .46 .75

Responsibility .640 .803 .469 -- .62 .44 .67 .84

Empathy .630 .607 .580 .649 -- .60 .60 .86

Engagement .576 .436 .656 .450 .577 -- .48 .83

Self-Control .600 .661 .434 .660 .596 .463 -- .84

Chronbach’s

Alpha

.659

.831

.754

.848

.848

.851

.789

Note. COMM = Communicaiton; COOP = Cooperation; ASRT = Assertion; RESP = Responsibility; EMP =

Empathy; ENG = Engagement; SC = Self-Control. SSIS-PF Internal Consistency Coefficients and Intercorrelations

based on Language Format item parcels are presented below the diagonal. Statistics reported above the diagonal are

taken directly from SSIS Rating Scales Manual.

59

between latent factor indicators (i.e., item parcels) and select control variables (i.e., child age and

socio-economic status) also were examined prior to model testing (see Tables 5-8). Given the

large sample sizes, statistically significant correlations were observed between a number of the

latent factor indicators and control variables. The magnitude of observed indicator-control

correlations tended to be small. Nevertheless, expanded models with pathways controlling for the

effects of SES and age on the indicator parcels were examined following the completion of

invariance analyses.

No missing values were observed in the dataset. Data were screened for the presence of

univariate and multivariate outliers. Univariate outliers were identified through the calculation of

z-scores for all indicator variables. Several cases contained one or more outlying data points, all

of which were observed at the negative end of indicator distributions (z-scores < -3.5).

Multivariate outliers were identified through the calculation of Mahalanobis distance statistics

(D), which were examined for significance with a Chi-square test (df = 21, p = .001). Initial

baseline models were then fit to three different datasets. A first dataset was comprised of the

random sample for that particular analysis. To create a second dataset, all cases identified as

multivariate outliers were removed from the random sample. Finally, to create a third dataset, all

cases identified as univariate outliers also were removed from the sample. Results were

compared to determine whether the removal of outliers significantly influenced model fit. A

comparison of global fit statistics for initial models fitted to the three datasets is presented in

Table 11. As can be seen in the table, changes in global fit statistics associated with the different

datasets were relatively minor. As such, the following sections describe only those analyses for

which models were fitted to the complete datasets.

Analysis of SSIS-PF Invariance by Race/Ethnicity

As the first step in the analysis of invariance by race/ethnicity, a random sample of cases

(n = 845) was selected to establish a baseline measurement model for the SSIS-PF. A first-order

60

Table 11

Global Fit Statistics for SSIS-PF Baseline Invariance Models fitted to Samples with and without Multivariate and Univariate Outliers.

MLM 2 df Scaling Correction

Factor for MLM

RMSEA

[90% C.I.]

CFI TLI SRMR

SSIS-PF Invariance by Race/Ethnicity

Model 1.1

Random Sample 494.637 168 1.242 0.048

[0.043 – 0.053]

0.959 0.949 0.039

Multivariate Outliers Removed 579.146 168 1.114 0.055

[0.05 – 0.060]

.954 .942 0.041

All Outliers Removed 569.436 168 1.116 0.055

[0.050 – 0.060]

0.951 0.939 0.043

SSIS-PF Invariance by Language Format

Model 3.1

Random Sample 560.275 168 1.201 0.050

[0.045 – 0.054]

0.959 0.949 0.036

Multivariate Outliers Removed 585.872 168 1.121 0.052

[0.048 – 0.057]

0.959 0.948 0.037

All Outliers Removed 559.915 168 1.120 0.051

[0.046 – 0.055]

0.959 0.949 0.036

Note: RMSEA [90% C.I.] = Root Mean Squared Error of Approximation with 90% Confidence Interval; CFI = Confirmatory Fit Index; TLI = Tucker-Lewis

Index; SRMR = Standardized Root Mean Squared Residual. Sample Sizes for Race/Ethnicity Samples are: Random Sample (N = 845); Multivariate Outliers

Removed (N = 807); All Outliers Removed (N = 800). Sample Sizes for Language Format Samples are: Random Sample (N = 949); Multivariate Outliers

Removed (N = 913); All Outliers Removed (N = 906).

61

measurement model with seven distinct social skills domains, Model 1.1,11

was supported by

global fit statistics (MLM 2[168] = 494.637; RMSEA = 0.048, 90% C.I. 0.043 – 0.053; CFI =

0.959; TLI = 0.949; SRMR = 0.039; Figure 1). All freely estimated model pathways were

statistically significant and demonstrated large effect sizes.12

Modification indices did suggest a

number of plausible modifications to further improve model fit, many of which proposed

secondary factor-loadings for indicator parcels. Such a pattern suggests a potential lack of

unidimensionality for SSIS-PF Social Skills factors. This issue is considered more directly in the

analysis of group specific baseline models. However, given evidence of strong global fit, and in

an effort to maintain parsimony and adherence to the original measurement model, further

modification was not performed at this stage.

After completing the first analysis, a second baseline model was specified to test the

plausibility of the unitary Social Skills factor for the SSIS Rating Scales (see Figure 2). Model

1.2, which includes a second-order factor subsuming all seven first-order social skills domains,

showed evidence of reduced fit to the sample data (MLM 2

[182] = 836.604; RMSEA = 0.065,

90% C.I. 0.061 – 0.070; CFI = .918; TLI = .905; SRMR = 0.062; Figure 2). Direct comparison of

Models 1.1 and 1.2 showed a statistically significant reduction in global fit associated with the

more restricted Model 1.2 (ΔMLM 2[14] = 350.967, p < .01). Again, results suggested several

model modifications with the potential to improve the fit of a second-order factor solution. Given

that criteria for global fit were not met, modifications to Model 1.2 were applied in step-wise

fashion. A detailed account of the modification process is provided in the ‘Analysis of SSIS-PF

11

To aid the reader, all models are labeled with two numbers. The first number indicates the set of

analyses (e.g., 1 = Analysis of Invariance by Race/Ethnicity). The second number indicates chronological

sequence. Tables 12, 15, 16, and 18 provide a summary of global fit statistics for each model in each of the

four analyses, respectively. 12

The default settings for MPlus specify unit loading identification (ULI) constraints on the first indicator

of each latent factor in a measurement model. In order to examine the significance/effect (and invariance)

of pathways that were initially fixed, secondary analyses were run with ULI constraints specified for the

second indicator of each factor.

62

Table 12

Global Fit Statistics for SSIS-PF Measurement Model at Successive Stages of Invariance Analyses by Race/Ethnicity.

Model Description/Modification Sample MLM 2 df Scaling

Correction

for MLM

RMSEA

[90% C.I.]

CFI TLI SRMR

Model 1.1 7-Factor, First Order

Random 494.637 168 1.242 0.048

[0.043 – 0.053]

0.959 0.949 0.039

Model 1.2 7-Factor, Second Order

Random 836.604 182 1.239 0.065

[0.061 – 0.070]

0.918 0.905 0.062

Model 1.3 6-Factor, First Order

(Final Baseline)

Random 520.472 174 1.243 0.049

[0.044 – 0.053]

0.957 0.948 0.040

Model 1.3 Group Baseline AA 311.38 174 1.238 0.050

[0.041 – 0.059]

0.956 0.947 0.041

Model 1.3 Group Baseline

CA 712.524 174 1.178 0.052

[0.048 – 0.056]

0.953 0.944 0.042

Model 1.3 Group Baseline

LA 284.402 174 1.117 0.066

[0.052 – 0.080]

0.933 0.919 0.051

Model 1.4 Configural Invariance Full 1309.836 522 1.177 0.053

[0.049 – 0.056]

0.952 0.942 0.043

Model 1.5 Metric Invariance Full 1346.006 552 1.165 0.053

[0.048 – 0.055]

0.952 0.945 0.047

Model 1.6 Structural Invariance Full 1394.409 594 1.165 0.050

[0.047 – 0.053]

0.951 0.945 0.076

Model 1.7 Structural Invariance and

Control Variables

Full 1416.140 594 1.142 0.051

[0.047 – 0.054]

0.953 0.940 0.069

Note. RMSEA [90% C.I.] = Root Mean Squared Error of Approximation with 90% Confidence Interval; CFI = Confirmatory Fit Index; TLI = Tucker-Lewis

Index; SRMR = Standardized Root Mean Squared Residual. Sample sizes are: Random Sample (N = 845); African American (n = 312); Caucasian (n = 1162);

Latino (n = 145); Full Sample (N = 1619).

63

Figure 1. First-order baseline model for the analysis of invariance by race/ethnicity, Model 1.1.

Standardized factor loadings resulting from MLM estimation with random sample are reported. P1 – P21

are parcels used in the race/ethnicity analysis.

Communication

P1

Assertion

Responsibility

Cooperation

Empathy

Engagement

Self-Control

P2

P3

P4

P5

P6

P8

P7

P9

P11

P12

P10

P15

P14

P13

P16

P18

P17

P19

P21

P20

.762

.555

.721

.776

.786

.792

.660

.763

.745

.765

.720

.710

.833

.855

.756

.782

.804

.772

.789

.737

.807

.898

.587

.625

.713

.696

.536

.960

.724

.710

.548

.691

.636

.868

.794

.849

.723

.518

.552

.765

.785

.782

64

Figure 2. Higher-order baseline model for the analysis of invariance by race/ethnicity, Model 1.2.

Standardized factor loadings resulting from MLM estimation with random sample are reported. P1 – P21

are parcels used in the race/ethnicity analysis.

Communication

P1

Assertion

Responsibility

Cooperation

Empathy

Engagement

Self-Control

P2

P3

P4

P5

P6

P8

P7

P9

P11

P12

P10

P15

P14

P13

P16

P18

P17

P19

P21

P20

.747

.560

.732

.769

.779

.805

.661

.769

.737

.766

.725

.705

.832

.851

.762

.775

.809

.773

.784

.737

.810

Social Skills

.956

.913

.737

.947

.789

.703

.846

65

Higher-Order Factor Structure’ section that follows the current section. A viable second-order

factor solution was ultimately produced. However, the new model reflects a significant revision

to the model implied by the published version of the SSIS-PF. Given the exploratory nature of

post-hoc model fitting, this revised model requires independent validation before firm assertions

may be made regarding its adequacy. Therefore, as it relates to the current study, the decision

was made to retain the first-order measurement model, Model 1.1, as the initial baseline model

for the analysis of SSIS-PF measurement invariance across racial groups.

Next, independent baseline models were tested for each of the three race/ethnicity

subgroups. Model 1.1 showed adequate fit for both the Caucasian and African American

samples. Fit statistics for the Latino sample were not as strong when compared against those for

the other two groups.13

In addition to comparatively weaker global model fit, the estimated

baseline solution for the Latino sample also failed to produce a positive definite factor covariance

matrix. Technical output indicated a problem with the Empathy factor for the Latino group.

Upon further inspection, results for the Latino group did show stronger correlations between

Empathy and several of the other social skills domains when compared with the same correlations

for African American and Caucasian groups. However, there was no clear rationale for

implementing specific modifications to address the Empathy factor directly.

Estimated factor correlations for Model 1.1 are presented in Table 13. Note that even

though the estimated factor covariance matrices for the African American and Caucasian groups

were shown to be positive definite, several potentially excessive factor correlations were

observed for all three racial subgroups. Factor correlations of substantial magnitude suggest a

13

Small sample size (n = 145) likely hindered model estimation for the Latino group.

66

Table 13

Estimated Latent Factor Correlations for SSIS-PF 7-Factor Baseline Measurement Model by

Race/Ethnicity Group.

African American

COMM COOP ASRT RESP EMP ENG SC

Communication --

Cooperation .872 --

Assertion .794 .657 --

Responsibility .860 .946 .701 --

Empathy .793 .657 .786 .787 --

Engagement .796 .557 .854 .596 .787 --

Self-Control .782 .802 .603 .855 .736 .575 --

Caucasian

COMM COOP ASRT RESP EMP ENG SC

Communication --

Cooperation .909 --

Assertion .699 .566 --

Responsibility .859 .980 .580 --

Empathy .731 .681 .691 .732 --

Engagement .752 .526 .764 .537 .673 --

Self-Control .792 .828 .506 .818 .659 .585 --

Latino

COMM COOP ASRT RESP EMP ENG SC

Communication --

Cooperation .879 --

Assertion .753 .585 --

Responsibility .871 .952 .669 --

Empathy .904 .697 .805 .829 --

Engagement .776 .462 .911 .597 .748 --

Self-Control .848 .716 .669 .895 .815 .605 -- Note. COMM = Communication; COOP = Cooperation; ASRT = Assertion; RESP = Responsibility; EMP =

Empathy; ENG = Engagement; SC = Self-Control. Sample sizes are: African American (n = 312); Caucasian (n =

1162); Latino (n = 145).

67

lack of discriminant validity between subscales.14

Given that the estimated correlation between

the Cooperation and Responsibility factors was highest across all groups (African American =

.946; Caucasian = .980; Latino = .952), a subsequent modification was applied such that the six

indicators of Cooperation and Responsibility were specified to load onto a single latent factor (see

Figure 3). The resulting 6-factor model, Model 1.3, was tested as a new baseline with the original

random sample (MLM 2[174] = 520.472, RMSEA = 0.049, 90% C.I. 0.044 – 0.053; CFI = 0.957;

TLI = 0.948; SRMR = 0.040; Figure 3), and then with each of the racial group subsamples.

Differences in global fit statistics for racial group baseline models based on Model 1.1 and Model

1.3 were negligible. Moreover, the Model 1.3 solutions yielded positive definite factor

covariance matrices for all three groups.

The Latino model continued to show the weakest fit of the three group-specific baseline

models. Two modification indices suggested that fit could be improved with the specification of

additional parameters for the Latino baseline model. The first involved a secondary loading for

Parcel 10 on the Empathy factor. The second involved a correlation between residuals for Parcel

18 and Parcel 20. Neither proposed modification was justified from a theoretical perspective.15

Also, given the relatively small predicted improvement in model fit associated with proposed

modifications and the goal of retaining fully invariant baseline models for the initiation of

invariance testing, no modifications were applied to the Latino baseline model at this stage in the

analyses.

14

Evidence supporting the discriminant validity of SSIS-PF subscales was included in the SSIS Rating

Scales’ Manual, with subscale intercorrelations ranging from .42 - .78. However, such values reflect

correlations based on subscale scores that are measured with less than perfect reliability. In SEM/CFA,

latent factors are presumed to be measured without error. Thus, observed factor intercorrelations reflect

‘true’ domain overlap. 15

Although SSIS-PF items are not reproduced in text, item content for specific parcels can be determined

by referencing Table 2. Paraphrasing of item content is used, where necessary, in order to provide an

appropriate description of analyses.

68

Figure 3. Revised baseline model for the analysis of invariance by race/ethnicity, Model 1.3. All

indicators for Cooperation and Responsibility (i.e., Parcels 4-6 and 10-12) assigned to a single latent

factor . Standardized factor loadings resulting from MLM estimation with random sample are reported.

P1 – P21 are parcels used in the language format analysis.

Communication

P1

Assertion

Cooperation and Responsibility

Empathy

Engagement

Self-Control

P2

P3

P4

P5

P6

P11

P10

P12

P8

P9

P7

P15

P14

P13

P16

P18

P17

P19

P21

P20

.759

.552

.726

.762

.772

.788

.747

.716

.713

.660

.763

.745

.832

.854

.759

.782

..805

.771

.793

.737

.803

.897

.611

.725

.696

.535

.678

.723

.794

.691

.537

.550

.764

.821

.781

.710

69

Next, Model 1.3 was respecified for configural invariance across groups. The resulting

Model 1.4 includes parameter constraints specifying that the pattern of variable relationships

remain consistent across African American, Caucasian, and Latino subsamples. The magnitudes

of individual model parameters were freely estimated across groups at this stage. Results of the

multi-group analysis suggested adequate fit for Model 1.4 (MLM 2

[522] = 1309.836; RMSEA =

0.053, 90% C.I. 0.049 – 0.056; CFI = 0.952; TLI = 0.942; SRMR = 0.043). Thus, the general

structure of the measurement model was found to be invariant across racial groups. A follow-up

examination of the standardized and unstandardized path coefficients for Model 1.4 indicated

relative consistency across groups even though the paths had not been constrained for

equivalence (see Table 14). There was only one pathway for which the absolute difference in

unstandardized factor loadings across groups was > 0.20.16

Perhaps more instructively, none of

the standardized factor loadings exhibited absolute differences > 0.10.

Given evidence supporting the configural invariance of the measurement model, further

constraints were applied specifying the magnitude of factor loadings as equal across the three

racial groups. The resulting Model 1.5 continued to demonstrate relatively good fit to sample

data (MLM 2

[552] = 1346.006; RMSEA = 0.052, 90% C.I. 0.048 – 0.055 ; CFI = 0.952; TLI =

0.945; SRMR = 0.047). None of the equality-constrained factor loadings were identified as

contributing to misfit of the model (i.e., all corresponding M.I. values < 10.0). Moreover, a direct

comparison of Models 1.4 and 1.5 found a non-significant difference in global fit (ΔMLM 2[30]

= 27.630, p > .05), which supports retention of the more parsimonious model (Model 1.5). Thus,

results support metric invariance for the modified 6-factor SSIS-PF measurement model across

African American, Caucasian, and Latino race/ethnicity groups.

16

Unstandardized factor loadings for Parcel 8 on the Assertion factor were 1.148, 0.931, and 0.922 for

Caucasian, African American, and Latino subsamples, respectively.

70

Table 14

Unstandardized and Standardized Factor Loadings by Race/Ethnicity for Configural Invariance

Model.

African American

Caucasian

Latino

Scale/Parcel

Estimate

Std

Estimate

Std

Estimate

Std

Communication

Parcel 1 -- .741 -- .787 -- .706

Parcel 2 .774 .555 0.719 .522 .866 .600

Parcel 3 .784 .653 0.871 .731 .887 .679

Cooperation

/Responsibility

Parcel 4 -- .755 -- .762 -- .737

Parcel 5 .967 .744 0.993 .784 1.093 .792

Parcel 6 .978 .772 0.921 .766 .909 .768

Parcel 10 1.252 .797 1.127 .730 1.296 .764

Parcel 11 1.053 .745 .993 .707 1.068 .760

Parcel 12 .978 .737 .901 .725 .923 .699

Assertion

Parcel 7 -- .726 -- .666 -- .706

Parcel 8 .931 .727 1.148 .771 .922 .694

Parcel 9 .843 .770 .945 .737 .917 .753

Empathy

Parcel 13 -- .851 -- .809 -- .860

Parcel 14 1.049 .820 1.149 .855 1.102 .835

Parcel 15 .883 .778 .934 .787 .959 .802

Engagement

Parcel 16 -- .795 -- .792 -- .756

Parcel 17 .820 .821 .889 .812 .877 .837

Parcel 18 .768 .777 .814 .771 .859 .788

Self-Control

Parcel 21 -- .823 -- .830 -- .814

Parcel 22 .974 .729 .863 .719 .996 .729

Parcel 22 .950 .835 .829 .824 .951 .802 Note. Results based on technical output for Model 1.4.

71

As a final step in the analysis of invariance, the structural elements of the model (i.e.,

factor variances and covariances) also were constrained to be equal across racial groups. The

resulting model, Model 1.6, continued to demonstrate adequate fit to the sample data (MLM

2

[594] = 1394.409; RMSEA = 0.050, 90% C.I. 0.047 – 0.053 ; CFI = 0.951; TLI = 0.949; SRMR

= 0.076), though an increase in the value of SRMR is noteworthy, given that this fit index is

particularly sensitive to misspecification of factor covariances (Hu and Bentler, 1998). Still, none

of the newly constrained parameters were identified as contributing to model misfit, and a direct

comparison of Models 1.5 and 1.6 indicated a non-significant difference in global fit (ΔMLM

2

[42] = 48.403, p > .05). Therefore, based on the full set of analyses, the 6-factor model shown in

Figure 3 was found to demonstrate measurement invariance across African American, Caucasian,

and Latino subgroups.

At the conclusion of the analysis, pathways controlling for the effects of SES and age

were added to the final invariance model. Each of these pathways were permitted to vary freely

across groups (i.e., paths were not constrained for invariance). A handful of corresponding

standardized path coefficients reached the minimum threshold for a small effect size (r = .10;

Cohen, 1992). Still, there was negligible change in global model fit when control variable

pathways were included in the structural equation (MLM 2[594] = 1416.140; RMSEA = 0.051,

90% C.I. 0.047 – 0.054; CFI = 0.953; TLI = 0.940; SRMR = 0.069).

Analysis of SSIS-PF Higher-Order Factor Structure

After completing the planned analysis of SSIS-PF measurement invariance as a function

of racial group membership, a set of follow-up analyses were conducted. The primary objective

of the follow-up analyses was to investigate the higher-order factor structure of the SSIS-PF.

Using the implied second-order measurement model for the SSIS Rating Scales (Model 1.2/2.1)

as a starting point, post-hoc model fitting was carried out in order to generate a ‘best-fitting’

model for a random sample of cases (see Table 15). Modifications were performed in a step-wise

72

Table 15

Global Fit Statistics for SSIS-PF Higher Order Measurement Model at Successive Stages of Post-Hoc Model Fitting.

Model Description/Modification Sample MLM 2 df Scaling

Correction

Factor for MLM

RMSEA

[90% C.I.]

CFI TLI SRMR

Model 2.1 7-Factor, Second Order

Random 836.604 182 1.239 0.065

[0.061 – 0.070]

0.918 0.905 .062

Model 2.2 Assertion/Engagement

residual covariance

Random 715.450 181 1.238 0.059

[0.055 – 0.064]

0.933 0.922 0.056

Model 2.3 Assertion/Engagement on

Proactive Social Skills

Random 715.450 181 1.238 0.059

[0.055 – 0.064]

0.933 0.922 0.056

Model 2.4 Empathy alone; Rename

Responsive Social Skills

Random 618.605 180 1.237 0.054

[0.049 – 0.058]

0.945 0.936 0.047

Model 2.5 Crossloading for

Communication

Random 559.564 179 1.238 0.050

[0.045 – 0.055]

0.952 0.944 0.042

Model 2.5 Cross-validation

Holdout 515.011 179 1.194 0.049

[0.044 – 0.054]

0.957 0.949 0.040

Model 2.5 Group Baseline AA 319.966 179 1.233 0.050

[0.041 – 0.059]

0.955 0.947 0.042

Model 2.5 Group Baseline CA 751.172 179 1.178 0.052

[0.049 – 0.056]

0.951 0.942 0.044

Model 2.5 Group Baseline* LA 291.329 179 1.120 0.066

[0.052 – 0.079]

0.931 0.920 0.054

Note. RMSEA [90% C.I.] = Root Mean Squared Error of Approximation with 90% Confidence Interval; CFI = Confirmatory Fit Index; TLI = Tucker-Lewis

Index; SRMR = Standardized Root Mean Squared Residual. Sample sizes are: Random Sample (n = 845); Holdout (n = 774); African American (n = 312);

Caucasian (n = 1162); Latino (n = 145). Descriptions marked with an asterisk (*) indicate those analyses for which the factor covariance matrix was not positive

definite.

73

fashion. Once a ‘best-fitting’ model was determined, data from a holdout sample (i.e., those

cases not selected for the initial random sample) were used for cross validation. Finally, given

successful cross-validation, the model was independently fitted to sample data from each of the

three racial subgroups as a preliminary step in the assessment of invariance.17

As a first step, all modification indices produced during the initial analysis of Model 2.1

were examined. Output suggested that the fit of Model 2.1 could be substantially improved

through the inclusion of a freely estimated residual covariance between the Assertion and

Engagement factors. Review of item content for both factors provided conceptual justification

for the proposed residual covariance, as items for both Assertion and Engagement reflect social

skills in the form of proactive behaviors. Given empirical and conceptual support, the

modification was applied (see Figure 4). The resulting Model 2.2 showed evidence of

significantly improved model fit relative to Model 2.1 (ΔMLM 2[1] = 106.215, p < .01), although

global fit statistics still did not meet a priori criteria (MLM 2

[181] = 715.450; RMSEA = 0.059,

90% C.I. 0.055 – 0.064; CFI = .933; TLI = .922; SRMR = 0.056; Figure 5). Next, Model 2.2 was

re-specified in an equivalent format to enhance the overall interpretability of the model. In the

resulting Model 2.3 (see Figure 5), the Assertion and Engagement factors were dropped as

indicators of the original second-order Social Skills factor, and subsequently specified as

indicators of a new second-order factor, Proactive Social Skills. Being equivalent, global fit for

Model 2.3 was identical to that of Model 2.2.

In examining Model 2.3, results suggested the inclusion of a covariance between the

residual for the first-order Empathy factor and the second-order Social Skills factor. Notably, the

standardized estimated parameter change for the proposed covariance was large and negative

17

Given that the revised second-order measurement model was developed through post-hoc model fitting

and has not been independently validated, a full assessment of measurement invariance with the new model

was not appropriate.

74

Figure 4. Modified higher-order factor structure of the SSIS-PF, Model 2.2. Baseline model respecified

to include residual covariance between first-order factors of Assertion and Engagement. Standardized

factor loadings resulting from MLM estimation with random sample are reported. P1 – P21 are parcels

used in the race/ethnicity analysis.

Communication

P1

Assertion

Responsibility

Cooperation

Empathy

Engagement

Self-Control

P2

P3

P4

P5

P6

P8

P7

P9

P11

P12

P10

P15

P14

P13

P16

P18

P17

P19

P21

P20

.745

.559

.734

.769

.780

.804

.666

.751

.753

.765

.723

.707

.832

.852

.761

.778

.815

.764

.788

.739

.806

Social Skills

.948

.931

.694

.966

.770

.659

.853

.624

75

Figure 5. Modified higher-order factor structure of the SSIS-PF, Model 2.3. Specification of a second

higher-order factor, Proactive Social Skills. Standardized factor loadings resulting from MLM estimation

with random sample are reported. P1 – P21 are parcels used in the race/ethnicity analysis.

Communication

P1

Assertion

Responsibility

Cooperation

Empathy

Engagement

Self-Control

P2

P3

P4

P5

P6

P8

P7

P9

P11

P12

P10

P15

P14

P13

P16

P18

P17

P19

P21

P20

.745

.559

.734

.769

.780

.804

.666

.751

.753

.765

.723

.707

.832

.852

.761

.778

.815

.764

.788

.739

.806

Social Skills

Proactive Social Skills

.948

.931

.966

.770

.853

.915

.869

.759

76

(- 0.849). A large, negative covariance between the residual for Empathy and its higher-order

factor suggests that the Empathy domain is somewhat unique when compared with the other four

domains that contribute to Social Skills. Based on item content, Communication, Cooperation,

Responsibility, and Self-Control domains all reference behaviors through which an individual

demonstrates compliance with norms for social exchange (e.g., following rules, assuming

personal responsibilities, responding appropriately to the actions of others). Although not

completely unrelated, the Empathy items primarily reflect the act of being perceptive to the

feelings of others (i.e., making efforts to both understand and respond to others’ feelings). Given

this distinction, a new Model 2.4 was specified by dropping Empathy as an indicator of Social

Skills, and subsequently renaming the second-order Social Skills factor as Responsive Social

Skills (see Figure 6). Again, global fit statistics showed evidence of improved fit (MLM 2

[180] =

618.605; RMSEA = 0.054, 90% C.I. 0.049 – 0.058; CFI = .945; TLI = .936; SRMR = 0.047),

which was found to be statistically significant (ΔMLM 2

[1] = 84.99, p < .01).

In examining the need for further model re-specification, output for Model 2.4 suggested

a potential cross-loading for the first-order Communication factor on the second-order Proactive

Social Skills factor. The cross-loading makes conceptual sense when considering that

communication is a primary means of initiating and participating in social interactions.

Moreover, items contributing to the other two Proactive Social Skills domains also contain

elements of communicative behavior. As such, the cross-loading was specified in Model 2.5 (see

Figure 7). Resulting fit statistics indicated strong global fit for the new model (MLM 2

[179] =

559.564; RMSEA = 0.050, 90% C.I. 0.045 – 0.055; CFI = .952; TLI = .944; SRMR = 0.042),

which represented a significant improvement over Model 2.4 (ΔMLM 2

[1] = 68.50, p < .01).

Given these findings and a lack of substantive rationale for further model modification, Model 2.5

77

Figure 6. Modified higher-order factor structure of the SSIS-PF, Model 2.4. Empathy dropped as an

indicator of higher-order Social Skills factor. Standardized factor loadings resulting from MLM

estimation with random sample are reported. P1 – P21 are parcels used in the race/ethnicity analysis.

Communication

P1

Assertion

Responsibility

Cooperation

Empathy

Engagement

Self-Control

P2

P3

P4

P5

P6

P8

P7

P9

P11

P12

P10

P15

P14

P13

P16

P18

P17

P19

P21

P20

.745

.556

.738

.770

.782

.801

.659

.765

.743

.762

.722

.711

.835

.852

.758

.776

.818

.762

.792

.742

.800

Responsive Social Skills

Proactive Social Skills

.948

.937

.980

.852

.913

.870

.797

.734

.716

78

Figure 7. Modified higher-order factor structure of the SSIS-PF, Model 2.5. Communication specified to

have a cross-loading on Proactive Social Skills factor. Standardized factor loadings resulting from MLM

estimation with random sample are reported. P1 – P21 are parcels used in the race/ethnicity analysis.

Communication

P1

Assertion

Responsibility

Cooperation

Empathy

Engagement

Self-Control

P2

P3

P4

P5

P6

P8

P7

P9

P11

P12

P10

P15

P14

P13

P16

P18

P17

P19

P21

P20

.749

.560

.730

.773

.784

.797

.665

.756

.749

.762

.720

.712

.837

..850

.758

.778

.812

.767

.795

.745

.794

Responsive Social Skills

Proactive Social Skills

.958

.655

.993

.851

.893

.899

.781

.708

.660

.382

79

was retained as the ‘best fitting’ second-order model for the SSIS-PF based on the randomly

selected sample of standardization data.

Model 2.5 represents a higher-order structural model for the SSIS-PF that appears to be

both conceptually defensible and empirically justified. However, given the exploratory nature of

the post-hoc model fitting process, further validation of this substantially revised model is

needed. As a first step in the validation process, a cross-validation analysis was carried out using

data from the holdout sample (n = 774) that had not been used during the model modification

process. When fitted to data from the holdout sample, Model 2.5 continued to show evidence of

strong global model fit (MLM 2[179] = 515.011; RMSEA = 0.049, 90% C.I. 0.044 – 0.054; CFI =

.957; TLI = .949; SRMR = 0.040). Thus, results tentatively support Model 2.5 as an appropriate

representation of the higher-order structure for the SSIS-PF.

Finally, given the current investigation’s focus on determining measurement invariance,

an initial assessment of the adequacy of Model 2.5 with each of the three racial subgroups also

was performed. Consistent with results from the analysis of first-order factors, Model 2.5 showed

adequate fit for both the African American and Caucasian subsamples, but weaker fit for the

Latino subsample (see Table 15). Moreover, results of the CFA for Model 2.5 with the Latino

subsample yielded a factor covariance matrix that was not positive definite. Specifically, results

indicated a negative residual variance for the Responsibility factor, further implicating lack of

discriminant validity between SSIS-PF Social Skills domains – and between Cooperation and

Responsibility in particular. Though the factor covariance matrices for the African American and

Caucasian groups were positive definite, results for these groups also indicated a high degree of

overlap between the two domains in question. Thus, even though Model 2.5 demonstrated strong

fit when tested with two independent and randomly selected samples, the need for further

examination and potential revision of the higher-order factor structure of the SSIS-PF is

indicated.

80

Analysis of SSIS-PF Invariance by Language Format

Before presenting the results of the analysis of invariance by language format, it is

important to note three key differences that distinguish the ‘language format’ invariance analysis

from the ‘race/ethnicity’ invariance analysis. First, the language format sample includes both

English format and Spanish format cases. The race/ethnicity sample included English format

cases only. Second, the parcel indicators for latent factors were constructed independently for the

two sets of analyses (see Method). As such, parcels are not necessarily comprised of the same

items across the two invariance studies. Third, given that the race/ethnicity invariance analysis

suggested a need for substantial revision to the implied higher-order factor structure of the SSIS-

PF, the analysis of measurement invariance by language format was restricted to an examination

of first-order models only. Aside from these noted differences, both studies were carried out

according to the same set of procedures.

For the initial step in the analysis of invariance by language format, a baseline

measurement model for the SSIS-PF was tested with a random sample of cases (n = 949). The 7-

factor first order measurement model18

, Model 3.1, was supported by global fit statistics (MLM

2

[168] = 560.275; RMSEA = 0.050, 90% C.I. 0.044 – 0.054 ; CFI = 0.959; TLI = 0.949; SRMR =

0.036), although potential for improvement through the specification of additional freely

estimated parameters was indicated. In particular, modification indices supported the re-

specification of Parcel 11 as an indicator of Cooperation rather than Responsibility. The two

items from Parcel 11 appear to be similar in content to several of the items already included on

the Cooperation scale (i.e., behaviors referencing rule following and compliance/task-

completion). Moreover, items from the two remaining parcel indicators of Responsibility also

appear to comprise a cohesive domain (i.e., behaviors referencing awareness and ownership of

18

Model 3.1 is structurally identical to Model 1.1 from the race/ethnicity invariance analysis. However,

item-parcel indicators of latent factors were created differently for the two sets of analyses, resulting in

variation of observed model fit.

81

responsibility for one’s own actions). As such, the modification was applied (see Figure 8). The

resulting Model 3.2 showed evidence of strong model fit (MLM 2

[168] = 481.104; RMSEA =

0.044, 90% C.I. 0.040 – 0.049 ; CFI = 0.967; TLI = 0.959; SRMR = 0.034). In addition, all

freely estimated model pathways were statistically significant and demonstrated large effect sizes.

Further model re-specification was not considered necessary at this stage in the analysis.

The group-specific baseline models for English language and Spanish language

subsamples were analyzed next. Model 3.2 showed good fit for both the English language

sample (MLM 2[168] = 756.605; RMSEA = 0.047, 90% C.I. 0.043 – 0.050 ; CFI = 0.964; TLI =

0.955; SRMR = 0.035) and the Spanish language sample (MLM 2[168] = 269.892; RMSEA =

0.044, 90% C.I. 0.034 – 0.054; CFI = 0.968; TLI = 0.960; SRMR = 0.044). Despite evidence of

strong global fit, technical output for Model 3.2 reported a factor covariance matrix for the

Spanish language sample that was not positive definite. An error message suggested that the

problem involved the Empathy factor.19

However, upon further inspection, no substantive

rationale was found to support making additional modifications to the baseline model. As such,

and given the strong global fit for both English language and Spanish language samples, the

decision was made to retain Model 3.2 as the primary baseline model for both groups. The non-

positive definite factor covariance matrix for the Spanish language subsample was acknowledged

as a significant limitation moving forward.

Multi-group analysis of the baseline model specified for configural invariance between

English language and Spanish language groups, Model 3.3, continued to show evidence of strong

global model fit (MLM 2[336] = ; 1026.602; RMSEA = 0.046, 90% C.I. 0.043 – 0.049 ; CFI =

0.965; TLI = 0.956; SRMR = 0.036). Thus, configural invariance of the SSIS-PF as a function of

19

The Empathy factor also was identified as the source of error when a non-positive definite factor

covariance matrix was observed for the Latino baseline model in the analysis of invariance by

race/ethnicity.

82

Figure 8. Revised baseline model for the analysis of invariance by language format, Model 3.2. Parcel

11 reassigned to Cooperation factor. Standardized factor loadings resulting from MLM estimation are

reported. P1 – P21 are parcels used in the language format analysis.

Communication

P1

Assertion

Responsibility

Cooperation

Empathy

Engagement

Self-Control

P2

P3

P4

P5

P6

P8

P7

P9

P11

P12

P10

P15

P14

P13

P16

P18

P17

P19

P21

P20

.659

.528

.724

.820

.787

.765

.724

.671

.740

.730

.882

.905

.811

.805

.808

.812

.800

.825

.696

.798

.738

.892

.572

.568

.730

.679

.563

.880

.727

.767

.522

.744

.722

.786

.816

.767

..815

.504

.568

.746

.822

.843

83

language format was supported. In the subsequent analysis, specification of cross-group equality

constraints for factor loadings in a metric invariance model, Model 3.4, also indicated strong

global model fit (MLM 2[350] = 1044.487; RMSEA = 0.045, 90% C.I. 0.042 – 0.049; CFI =

0.964; TLI = 0.957; SRMR = 0.037). None of the equality constrained parameters were

identified as contributing to model misfit, and a direct comparison of Models 3.3 and 3.4

indicated a non-significant difference in global fit (ΔMLM 2

[14] = 14.481, p > .05).

For the final step in the assessment of measurement invariance by language format, cross-

group equality constraints were specified for all structural parameters. The resulting Model 3.5

continued to show strong fit to the sample data (MLM 2[379] = 1095.640; RMSEA = 0.044, 90%

C.I. 0.041 – 0.047; CFI = 0.963; TLI = 0.959; SRMR = 0.050). However, a direct comparison of

Models 3.4 and 3.5 indicated significantly weaker global fit for the more restricted model

(ΔMLM 2[28] = 50.8107, p < .01). Stated differently, the fit of Model 3.5 is significantly

improved through the introduction of additional parameters allowing for the unconstrained

estimation of factor variances and covariances across language format groups (i.e., Model 3.4).

Thus, while the modified SSIS-PF baseline Model 3.2 demonstrated evidence of configural

invariance (Model 3.3) and metric invariance (Model 3.4), the structural components of the model

were not shown to be invariant as a function of language format (see Tables 16 and 17).

Finally, having concluded the analysis of invariance by language format, unconstrained

pathways controlling for the effects of SES and age were added to Model 3.4. Again, several of

the newly introduced parameters reached the minimum threshold for a small effect size (r = .10;

Cohen, 1992). Still, there was negligible change in global model fit when control variable

pathways were included in the structural equation (MLM 2[350] = 1034.441; RMSEA = 0.045,

90% C.I. 0.042 – 0.048; CFI = 0.967; TLI = 0.952; SRMR = 0.034).

84

Table 16

Global Fit Statistics for SSIS-PF Measurement Model at Successive Stages of Invariance Analyses by Language Format.

Model Description/Modification Sample MLM 2 df Scaling

Correction for

MLM

RMSEA

[90% C.I.]

CFI TLI SRMR

Model 3.1 7-Factor, First Order

Random 560.275 168 1.201 0.050

[0.045 – 0.054]

0.959 0.949 0.036

Model 3.2 Parcel 11 to load on

Cooperation (Baseline)

Random 481.104 168 1.202 0.044

[0.040 – 0.049]

0.967 0.959 0.034

Model 3.2 Group Baseline

English 756.605 168 1.21 0.047

[0.043 – 0.050]

0.964 0.955 0.035

Model 3.2 Group Baseline* Spanish 269.892 168 1.201 0.044

[0.034 – 0.054]

0.968 0.960 0.044

Model 3.3 Configural Invariance*

Full 1028.289 336 1.205 0.046

[0.043 – 0.049]

0.965 0.956 0.036

Model 3.4 Metric Invariance* Full 1044.487 350 1.197 0.045

[0.042 – 0.049]

0.964 0.957 0.037

Model 3.5 Structural Invariance Full 1095.640

378 1.196 0.044

[0.041 – 0.047]

0.963 0.959 0.050

Model 3.6 Metric Invariance and

Control Variables*

Full 1034.441 350 1.187 0.045

[0.042 – 0.048]

0.967 0.952 0.034

Note. RMSEA [90% C.I.] = Root Mean Squared Error of Approximation with 90% Confidence Interval; CFI = Confirmatory Fit Index; TLI = Tucker-Lewis

Index; SRMR = Standardized Root Mean Squared Residual. Sample sizes are: Random Sample (n = 949); English Format (n = 1619); Spanish Format (n = 320);

Full Sample (N = 1941). Descriptions marked with an asterisk (*) indicate those analyses for which the factor covariance matrix was not positive definite.

85

Table 17

Unstandardized and Standardized Factor Loadings by Language Format for Configural

Invariance Model.

English Format

Spanish Format

Scale/Parcel

Estimate

Std

Estimate

Std

Communication

Parcel 1 -- .671 -- .599

Parcel 2 .833 .540 .917 .621

Parcel 3 .962 .705 1.128 .759

Cooperation

Parcel 4 -- .814 -- .821

Parcel 5 .866 .771 .965 .823

Parcel 6 .852 .768 .912 .805

Parcel 11 .849 .731 .880 .740

Assertion

Parcel 7 -- .707 -- .725

Parcel 8 .912 .635 1.007 .585

Parcel 9 1.175 .756 1.265 .751

Responsibility

Parcel 11 -- .884 -- .837

Parcel 12 .965 .896 1.010 .879

Empathy

Parcel 13 -- .804 -- .796

Parcel 14 .984 .806 1.089 .792

Parcel 15 .869 .809 .985 .796

Engagement

Parcel 16 -- .796 -- .774

Parcel 17 .839 .810 .868 .788

Parcel 18 .971 .844 .942 .789

Self-Control

Parcel 21 -- .726 -- .681

Parcel 22 1.008 .815 1.052 .829

Parcel 22 .963 .740 .975 .711 Note. Results based on technical output for Model 3.3.

86

Analysis of the Higher-Order Factor Structure for the SSIS-PF English Language Format

and Spanish Language Format

After completing the planned analysis of SSIS-PF measurement invariance as a function

of language format, follow-up analyses were carried out to investigate the higher-order factor

structure of the SSIS-PF for language format groups (see Table 18). The model to be tested,

Model 4.1, was constructed by combining the first-order structure of the language format

invariance baseline model (Model 3.2)20

and the higher-order structure of the ‘best fitting’ model

produced through the first set of follow-up analyses (Model 2.5). Model 4.1 was then

independently fitted to both the English language format and Spanish language format datasets.

Results indicate that Model 4.1 shows strong global fit for both the English language format

(MLM 2[179] = 844.132; RMSEA = 0.048, 90% C.I. 0.045 – 0.051; CFI = 0.959; TLI = 0.952;

SRMR = 0.034; see Figure 9) and Spanish language format (MLM 2

[179] = 309.340; RMSEA =

0.045, 90% C.I. 0.039 – 0.058; CFI = 0.959; TLI = 0.952; SRMR = 0.049; see Figure 10).

However, results for the Spanish language format again produced a factor covariance matrix that

was not positive definite, identifying a specific issue with the Communication factor (i.e., a

negative residual variance). As stated previously, such results suggest a need for further

investigation and refinement of the SSIS-PF higher-order factor structure, particularly as it relates

to the scales’ Spanish language format.

20

The first order structure of Model 3.2 was used given that Model 4.1 was tested using the same parcel

indicators for latent factors that had been used in the language format invariance analysis.

87

Table 18

Global Fit Statistics for Proposed SSIS-PF Higher Order Measurement Model with Language Format Samples.

Model Description/Modification Sample MLM 2 df Scaling

Correction

Factor for MLM

RMSEA

[90% C.I.]

CFI TLI SRMR

Model 4.1 Modified 7-Factor, Second

Order

Random 528.591 179 1.205 0.045

[0.041 – 0.050]

0.965 0.957 0.037

Model 4.1 Cross-validation*

Holdout 549.564 179 1.236 0.046

[0.041 – 0.050]

0.963 0.956 0.039

Model 4.1 Group Baseline English 844.132 179 1.213 0.048

[0.045 – 0.051]

0.959 0.952 0.037

Model 4.1 Group Baseline* Spanish 309.340 179 1.196 0.049

[0.039 – 0.058]

0.959 0.952 0.049

Note. RMSEA [90% C.I.] = Root Mean Squared Error of Approximation with 90% Confidence Interval; CFI = Confirmatory Fit Index; TLI = Tucker-Lewis

Index; SRMR = Standardized Root Mean Residual. Sample sizes are: Random Sample (n = 949); Holdout (n = 992); English Format (n = 1619); Spanish Format

(n = 320). Descriptions marked with an asterisk (*) indicate those analyses for which the factor covariance matrix was not positive definite.

88

Figure 9. Revised higher-order factor structure adapted to include first-order structure from the language

format invariance analysis, Model 4.1. Standardized factor loadings for MLM estimation with English

language format subgroup are reported. P1 – P21 are parcels used in the language format analysis.

Communication

P1

Assertion

Responsibility

Cooperation

Empathy

Engagement

Self-Control

P2

P3

P4

P5

P6

P8

P7

P9

P10

P12

P11

P15

P14

P13

P16

P18

P17

P19

P21

P20

.671

.545

.701

.813

.769

.770

.711

.639

.751

.730

.884

.896

.803

.805

.811

.728

.816

.738

.728

.816

.738

Responsive Social Skills

Proactive Social Skills

.947

.659

.885

.878

.928

.884

.794

.766

.669

.408

89

Figure 10. Revised higher-order factor structure adapted to include first-order structure from the

language format invariance analysis, Model 4.1. Standardized factor loadings for MLM estimation with

Spanish language format subgroup are reported. P1 – P21 are parcels used in the language format

analysis

Communication

P1

Assertion

Responsibility

Cooperation

Empathy

Engagement

Self-Control

P2

P3

P4

P5

P6

P8

P7

P9

P10

P12

P11

P15

P14

P13

P16

P18

P17

P19

P21

P20

.598

.631

.750

.814

.820

.816

.716

.596

.753

.738

.835

.881

.792

.794

.798

.776

.784

.791

.680

.829

.712

Responsive Social Skills

Proactive Social Skills

.934

.740

.942

.877

.984

.861

.875

.808

.703

.353

90

Chapter 5

Discussion

Overview

The primary objectives for the current study were to examine: (a) the measurement model

of the SSIS-PF, and, if necessary, alternative measurement models for the instrument; (b)

invariance of the SSIS-PF measurement model across race/ethnicity subgroups; and (c)

invariance in the SSIS-PF measurement model across language format subgroups. Results

provided mixed support for the general measurement model of the SSIS-PF. Analysis of the

higher-order structure, in particular, revealed discrepancies between the original model and the

best-fitting model from the current study. However, first-order measurement models were

supported and demonstrated invariance across groups defined by race/ethnicity and language

format.

Primary Findings

Factor Structure of the SSIS-PF. Both first-order and higher-order factor structures

were tested for the SSIS-PF. The full seven-factor first-order structure was upheld through two

iterations of the same analysis (i.e., one for each of the invariance studies). However, results

from both iterations indicated a lack of discriminant validity between several social skills

domains, most significantly the Cooperation and Responsibility domains. Discriminant validity

concerns were mitigated through model modification. In the race/ethnicity analysis, the

Cooperation and Responsibility indicators were collapsed onto a single factor. In the language

format analysis, reassignment of one indicator from the Responsibility factor to the Cooperation

91

factor produced an optimal first-order model.21

Thus, with minor modification, the proposed

first-order measurement structure of the SSIS-PF was supported.

By contrast, the higher-order factor structure of the SSIS-PF showed clear evidence of

reduced fit relative to its first-order counterpart. Thus, the hypothesis of a unitary Social Skills

construct subsuming all seven first-order domains was not supported. Systematic respecification

of the original higher-order model, guided by a combination of empirical and theoretical

considerations, did ultimately yield a revised higher-order structure that explained the data quite

well. The observed adequacy of the alternate model must be qualified with an acknowledgement

of the exploratory nature of post-hoc model fitting (i.e., potentially capitalizing on random

variation in the sample data), although successful cross-validation with an independent holdout

sample counter-balances such concerns.

The alternate higher-order factor structure of the SSIS-PF frames the social skills

construct in a more complex, multi-dimensional conceptualization than that implied by the

published version of the rating scales. In the revised model, two separate higher-order social

skills factors were indicated. A first higher-order factor, Responsive Social Skills, was shown to

encompass behaviors that reflect an awareness of and adherence to accepted norms for social

exchange. Primary domains under the Responsive Social Skills factor included Cooperation,

Responsibility, and Self-Control. A second higher-order factor, Proactive Social Skills, was

reflected in behaviors used to initiate, redirect, or otherwise influence social context. Primary

domains under the Proactive Social Skills factor included Assertion and Engagement. Two first-

order domains, Communication and Empathy, did not align directly with either of the newly

posited higher-order factors, albeit for different reasons. The Communication domain actually

demonstrated substantive loadings on both higher-order factors. Although less than ideal from a

21

Due to differences in the item-composition of indicator parcels, the same indicator reassignment could

not be specified during the race/ethnicity analysis.

92

measurement standpoint, the dual loading appears to be conceptually justified when considering

that communicative behaviors are applied in both responsive and proactive ways. In contrast, the

Empathy domain was not assigned to either of the higher-order factors, with results suggesting

that Empathy instead represents a distinct aspect of social behavior.

SSIS-PF Measurement Invariance: Race/Ethnicity. Given that preliminary findings

suggested a need for revision to the proposed higher-order structure of the SSIS-PF, subsequent

invariance analyses were restricted to an examination of first-order models only. For the analysis

of invariance as a function of race/ethnicity, the six-factor baseline model demonstrated

configural invariance, metric invariance, and structural invariance across groups of African

American, Caucasian, and Latino participants. Collectively, such results indicate that the SSIS-

PF is consistent in the way in which it measures first-order social skills domains across the three

race/ethnicity groups under study.

It is important to note that the use of a six-factor model – as opposed to the original

seven-factor model – was necessary due to the significant redundancy between Cooperation and

Responsibility factors across each of the race/ethnicity samples. Despite this modification, the

remaining six factors consistently reflected the same social skills domains across all three groups.

Moreover, the indicators used to reflect the six social skills domains also demonstrated

relationships of consistent magnitude with the hypothesized social skills domains regardless of

whether the participants were African American, Caucasian, or Latino.

The observation of invariance at the structural level (i.e., invariant factor variances and

covariances) suggests that the amount of intra-group variability observed for each social skills

domain and pattern of relationships among the full set of social skills domains also were

equivalent across race/ethnicity groups. Such structural invariance is not necessarily critical to

the assertion of general measurement invariance or, subsequently, the validity of scores for

members of different groups. Groups could reasonably be expected to differ in the amount of

93

within-group variation for a particular construct, or in the way in which multiple constructs relate

to one another (Vandenberg & Lance, 2000). Nevertheless, the finding of structural invariance in

the current analysis is informative in that it suggests that first-order social skills domains from the

SSIS-PF relate to each other in similar ways across race/ethnicity groups. This finding has

implications for the measurement of higher-order social skills construct(s) as well. From an

SEM/CFA perspective, first-order domains serve as indicators of higher-order constructs.

Therefore, invariance of structural parameters within first-order measurement models (i.e., factor

variances and covariances) indicates potential invariance for higher-order measurement structures

(i.e., second-order factor loadings), provided that the second order model has been appropriately

conceptualized and sufficiently validated.

SSIS-PF Measurement Invariance: Language Format. For the analysis of invariance

by language format, the seven-factor baseline model demonstrated configural invariance and

metric invariance. Thus, the first-order measurement structure and the magnitude of indicator

loadings for the revised baseline model of the SSIS-PF were invariant across English language

and Spanish language groups. Such results suggest that the Spanish translation of the SSIS-PF

retains the same first-order measurement structure as the original English format. Moreover, the

factor indicators also reflect the same domains to the same degree regardless of the language in

which scale items are presented. The study did not support structural invariance of the SSIS-PF

across language formats, which suggests that the pattern of variances and covariances of social

skills domains were not equivalent for English language and Spanish language groups. As noted

previously, such structural invariance is not necessarily critical to the assertion of general

measurement invariance, at least as it pertains to the first-order measurement model under study.

Still, the lack of invariance for SSIS-PF structural parameters across language format groups is

noteworthy given that structural invariance was supported across race/ethnicity groups.

Unfortunately, due to design limitations, it remains unclear as to whether the difference in

94

structural parameters between language format groups is linked directly to the language of the

measurement instrument/respondents, broader cultural differences of the participants included in

the respective language format samples, or simply the use of discrepant baseline models in the

two analyses.

Interpretation of Primary Findings in the Context of Prior Research

SSIS-PF Structure and the Social Skills Construct. As noted previously, the seven

factor first-order measurement structure of the SSIS-PF was tentatively supported, which fits with

general findings suggesting that the broad set of behaviors traditionally viewed under the label

‘social skills’ can be reliably classified into a set of unique domains (Caldarella & Merrell, 1997).

However, prior research also has been inconsistent in its application of social skills

measurement/classification systems (Matson & Wilkinson, 2009). Despite the efforts of a few

researchers (e.g., Caldarella & Merrell, 1997), there remains no uniformly accepted taxonomy of

social skills. As a result, distinctions between social skills domains have not been clearly defined,

which may explain the mixed evidence of discriminant validity among several first-order factors

from the SSIS-PF.

With some modification (e.g., indicator reassignment), concerns regarding discriminant

validity for the SSIS-PF subscales were reduced. And, it should be emphasized that issues of

discriminant validity are not unique to the SSIS-PF, but rather seem to represent a more pervasive

obstacle within the field of social skills assessment. For example, Caldarella and Merrell’s

(1997) preliminary attempt to develop a taxonomy of social skills was similarly limited, as their

taxonomic categories, based on the five most commonly identified social skills ‘dimensions’

observed in prior research, also showed evidence of considerable overlap. Thus, taken in the

context of the larger body of research, current findings underscore the need for additional

research – particularly that which applies focused, systematic, multivariate analytic procedures

(Achenbach, 1995) to refine understanding of first-order social skills domains.

95

Further study of a higher order structure for the social skills domain also is warranted,

given current results suggesting that a unitary social skills factor may not adequately account for

all identified first-order social skills domains on the SSIS-PF. In fact, with its expansion, the

authors of the SSIS Rating Scales may have unintentionally introduced the instrument’s

assessment of a secondary higher-order social skills factor, labeled Proactive Social Skills in the

current study. The observed higher-order Proactive Social Skills factor is particularly noteworthy

when considered in concert with findings reported in a relatively recent CFA study of the SSRS,

which asserted the presence of a previously unidentified first-order ‘Extroversion’ factor (Van

Horn et al., 2007). The emergence of a Proactive Social Skills factor for the SSIS-PF may reflect

the revised instrument’s expanded coverage of an ‘extroverted’ social skills domain (e.g.,

Assertion and Engagement). Supporting this interpretation, items assigned to the Proactive Social

Skills factor of the SSIS-PF are similar in content to those assigned to the Extroversion factor of

the SSRS identified by Van Horn et al. (2007). The construct validity of the newly posited

Proactive Social Skills factor is further bolstered by the presence of comparable factors (e.g.,

Assertiveness, Interest/Participation, Assertiveness-Prosociablity, etc.) on a variety of alternative

social skills measures (see Matson & Wilkins, 2009).

Finally, current results also indicate the need to further examine the role of empathy as it

relates to the broader domain of social skills. Though the Empathy factor from the SSIS-PF did

share significant relationships with both the Responsive and Proactive Social Skills factors, the

best fitting measurement model was observed with Empathy being included as a stand-alone

factor. Therefore, empathy appears to reflect a unique domain of behavior within (or perhaps

related to) the social skills domain. It has been suggested previously that empathy, as a domain

of social skills, tends to emerge at a later stage of development once individuals have acquired

more advanced cognitive and emotional perspective-taking abilities (Merrell & Gimpel, 1998).

Therefore, developmental characteristics of the sample used in the current study, being comprised

96

of children between the ages of 5 and 12, also may have influenced current findings relative to the

Empathy domain.

Measuring Social Skills across Cultural Groups. An overarching goal for the current

investigation was to examine the degree to which the social skills construct, as operationalized by

the SSIS-PF, is generalizable to the Latino population(s) of the United States. Pursuit of this goal

prompted separate analyses of measurement invariance by race/ethnicity and language format,

respectively.

The current analysis of invariance by race/ethnicity is the first to have been conducted

with the SSIS Rating Scales. However, similar studies were conducted with the instrument’s

predecessor, the SSRS. One such study looked at invariance in the teacher version of the SSRS

across groups of White and Non-White participants (Walthold et al., 2005), and a second study

examined measurement invariance in the parent version of the SSRS as a function of several

different grouping variables including race (i.e., African American, Caucasian, and Hispanic; Van

Horn et al., 2007). Both previous studies were consistent in their support for measurement

invariance across racial groups, which is also consistent with findings from the present study of

the SSIS-PF. Thus, collectively, results suggest that social skills can be measured reliably and

consistently across broadly defined racial/ethnic groups.

In reference to language, current results support configural and metric invariance in the

first order SSIS-PF measurement model across English and Spanish language format groups.

Thus, the nature of the seven first-order SSIS-PF domains and the behaviors selected to reflect

these domains remained intact through the translation process. However, a lack of structural

invariance for the SSIS-PF across language format groups suggests potential differences in the

way first-order social skills domains hold together for individuals from English speaking and

Spanish speaking groups.

97

Interpretation of structural variability for the language format analysis is complicated due

to the limited extant research on standardized assessments of social skills with Spanish-speaking

populations, particularly as it relates to the nature of underlying constructs. Ethnographic

research findings do suggest that traditional Latino child rearing practices emphasize the

cultivation of specific, socially relevant qualities that may not have direct correlates in other

cultures (e.g., familismo, respeto, educación; Halgunseth, 2006). However, these terms also have

yet to be adequately operationalized for measurement purposes. Still, it may be that qualities

such as familismo, respeto, and educación actually constitute unique combinations of social

skills domains and/or items already represented on the SSIS-PF. For example, the Empathy

domain was identified as a source of substantial redundancy in the original first-order

measurement model for both the Spanish language format group in the language invariance

analysis and the Latino subsample of the English language format group in the race/ethnicity

study. Thus, it may be that Empathy is a more pervasive element of the social skills construct as

cultivated in Latino/Spanish-speaking cultures when compared with non-Latino/English-speaking

cultures in the United States. At present, such an interpretation is a working hypothesis that

needs to be empirically tested in future studies.

Despite the complexities of interpretation, the collection of current results indicates that

the social skills construct can be measured with consistency across cultural groups defined by

race/ethnicity and language. Such findings are clearly promising in terms of potential uses for the

SSIS-PF as a measurement instrument. However, there also appears to be some inconsistency

with eco-cultural developmental theory (e.g., Rogoff, 2003) in terms of the lack of observed

variation in the nature of the social skills construct across groups. Specifically, the behavioral

construct of social skills likely would be expected to show some variation across groups as a

function of differences in community based practices, beliefs, and traditions. To explain this

apparent inconsistency, it is helpful to consider that several factors may have contributed to

98

findings of measurement invariance in the current study. First, the SSIS-PF utilizes parent

judgments of behavior frequency, as opposed to direct observations of operationally defined

behaviors. Moreover, all SSIS-PF items are not necessarily designed to assess behaviors at the

molecular level (Elliott et al., 2008). Therefore, differences in the topography of specific

behaviors enacted by individuals from different groups may not result in measurement differences

on the SSIS-PF as long as the behaviors achieve the same function and are judged as such by

parent respondents. Second, the variability in cultural identity and affiliation for individuals

within groups, and the similarities among individuals from different groups based on their

exposure/interactions with the same social networks and community-based institutions also may

have reduced the likelihood of observing measurement non-invariance. Thus, while current

findings of measurement invariance for the SSIS-PF are important and have various implications

for research and practice, further study of social skills within and across various cultural groups is

still needed.

Limitations and Future Directions

Limitations of the current study can be broadly classified into two categories. A first set

of limitations arises from the format and composition of the raw data file that was used for

analysis. A second set reflects shortcomings relative to the general design of the study. Both sets

of limitations need to be addressed through future research.

Data Limitations. Perhaps the most significant limitation of the current study is the fact

that item-level data were not available for CFA. Instead, items were combined into parcels,

which in turn, were used as the observed variables in all analyses. Parcel indicators are actually

well suited to CFA when examining rating scales that employ ordinal item-level scores. The

process of grouping items into parcels typically results in observed variable distributions that

approximate normality more closely than their individual item counterparts (Hau & Marsh, 2004).

As was the case in the current study, traditional methods of estimation (e.g., ML, MLM) can

99

often be applied when item parcels are used as indicators. However, grouping items into parcels

also has several drawbacks. Most notably, when parcel indicators are used, the adequacy of

individual items cannot be directly examined (e.g., item reliability, factor loadings, factor

assignment). Moreover, item parceling precludes an examination of invariance at the item level.

In the current study of invariance, item-level analysis would have facilitated a direct comparison

of parameters for the original English-format items from the SSIS-PF and their translated

Spanish-format counterparts. Instead, the use of item-parcel indicators in the analysis of

measurement invariance may have reduced the likelihood of detecting true non-invariance at the

indicator level for both the race/ethnicity and language format studies (Meade & Kroustalis,

2006).

Steps were taken to address limitations related to the use of item parcels. First, all parcels

were developed according to specific procedures designed to either optimize parcel

unidimensionality or minimize the likelihood that non-invariance would be obscured at the

indicator level. In addition, as a means of assessing the effects of item parceling on subscale

measurement properties, item-generated and parcel-generated subscale internal consistency

reliability coefficients and intercorrelations were compared. Minimal differences were observed

upon comparison, providing some added justification for the use of item parcels. Nevertheless,

future investigations of invariance with item-level data from the SSIS Rating Scales are

warranted. Through such efforts, researchers would be able to examine the instrument’s general

measurement model more closely in terms of the adequacy of individual items and their

assignment to first-order domains. Similarly, item-level analysis would permit a more precise

examination of metric invariance, with the potential to inform understanding of similarities and

differences in specific social skills behaviors and their respective contributions to first-order

domains across groups.

100

A second limitation pertaining to the dataset used for the current study involves the

observed sample sizes of certain subsamples. The full sample was appropriately large in

reference to recommendations for CFA, and all groups were proportionately represented in the

sample in accordance with March 2006 U.S. population estimates (Gresham & Elliot, 2008).

However, while the Caucasian subsample remained sufficiently large when group data were

disaggregated, the African American and Latino/Spanish-format subsamples were only

moderately sized, and the Latino/English-format subsample was small. Thus, the stability of

parameter estimates and accuracy of global fit statistics for these latter groups cannot be asserted

with as much confidence as can those for the Caucasian group. All findings concerning the

Latino/English-format subsample specifically should be qualified as tentative in light of the small

sample size for this group.

An additional limitation relative to the dataset used for analysis involves the lack of select

demographic data. First, the dataset provided by the SSIS publisher did not include the sex of the

student participants. It is important to note that the standardization sample for the SSIS Rating

Scales was stratified according to sex, though, with equal representation of girls and boys in the

sample. Thus, it is likely that the distribution was similar in the current sample. Perhaps more

directly relevant to the current study, information was not available regarding the language status

and acculturation of participants. In order to examine the social skills construct across cultures

more precisely, data accounting for these more continuous elements of ‘culture’ should be

considered in future analyses.

Design Limitations. As initially conceptualized, the identification of a best-fitting

measurement model for the SSIS-PF was sought to serve as a baseline for the analysis of

invariance by race/ethnicity. Similarly, the final model from the race/ethnicity invariance

analysis was intended to be used as the baseline for a within group assessment of invariance by

language format. However, due to the collective impact of a number of factors – several of which

101

have been mentioned in the preceding section – adjustments to the original design and sequence

of analyses were required. As a result, analyses did not build from one to the next as efficiently

as possible.

First, a lack of item-level data hindered the study’s pursuit of a single ‘best- fitting’

measurement model for the SSIS-PF. The subsequent need to employ two different sets of item

parcels for the race/ethnicity and language format invariance studies, respectively, meant that

findings from one study were not necessarily directly comparable to the next. For example, the

baseline models tested for invariance differed across the two studies with respect to the number of

first order factors that were included. In addition, sample size restrictions precluded the use of

specific analyses intended to parse out the effects of language format and racial/ethnic group

membership on structural parameters for the SSIS-PF measurement model. As such, another

direction for future research is a study through which measurement invariance can be examined

across race/ethnicity and language format via a single set of iterative analyses (e.g., Non-

Latino/English-format; Latino/English-format; Latino/Spanish-format).

A second caveat to the interpretation of results for the current study concerns the nature

of measurement invariance as a psychometric property that is inferred on the basis of a collection

of evidence. Indices of global fit for equality-constrained models are typically used as primary

sources of evidence. However, comparison of unconstrained model parameters can also be

informative. Relatedly, it is important to note that testing for invariance in model-specified

patterns of parameters across groups does not guarantee detection of differences in specific

parameters, particularly when more than two groups are included in the analysis. Although

omnibus results generally supported SSIS-PF measurement invariance as a function of

race/ethnicity at the configural, metric, and structural levels, conflicting evidence should not be

dismissed. Specifically, the observation of potential group differences for model-implied factor

correlations at the baseline stage of the race/ethnicity invariance analysis, and the noted decrease

102

in absolute model fit when equality constraints were placed on structural parameters at a later

stage in the same study,22

are findings that require further investigation.

In general, findings supporting the invariance of the first-order measurement model of the

SSIS-PF across race/ethnicity and language format are promising and should encourage a variety

of research extensions. However, an acknowledgement of the possibility that relationships

between specific social skills factors may actually differ across race/ethnicity groups also is

justified. The lack of structural invariance for the SSIS-PF as a function of language format also

raises the question of potential race/ethnicity differences in structural parameters, given the

respective demographic composition of the English language format and Spanish language format

samples.

A final caveat regarding study design concerns the difference between full measurement

invariance and partial measurement invariance. The current study was designed to examine full

measurement invariance. As such, although group-specific modifications to the baseline SSIS-PF

model were considered, the goal was to retain equivalent models for each group across all stages

of the analysis where possible. In adhering to this goal, optimal group-specific measurement

models were not explicitly sought. Rather, analyses reflect the degree to which a common

measurement model demonstrated equivalence across groups. Furthermore, at various stages in

the analysis, group-specific models demonstrated less than adequate model fit and/or yielded

solutions that were otherwise problematic (e.g., observed factor covariance matrix that was not

positive definite). Such instances were addressed whenever possible. However, in order to

complete the set of analyses as planned, it was necessary at times to move forward with analyses

despite less than optimal findings at preliminary stages. As such, group-specific research should

22

Although chi-square difference testing revealed no significant difference in model fit when structural

constraints were added to the model, a notable increase in the magnitude of the SRMR absolute fit index

was observed.

103

also be conducted to further inform understanding of the social skills construct as it exists within

specific cultural groups.

The preceding list of design limitations also represent caveats to the interpretation of

results. In addition to those already mentioned, there are several other caveats to the current

study that should be addressed in future research. First, the present study focused solely on the

parent form of the SSIS Rating Scales. Therefore, independent studies of the teacher and self-

report forms of the instrument are needed. Similarly, studies that examine invariance along other

dimensions (i.e., gender, age, time) also would be informative. Finally, given the relative lack of

independent work with the SSIS Rating Scales, studies that examine other aspects of

measurement validity (e.g., predictive validity, sensitivity to change, etc.) should be completed as

well.

Implications for the Use of the SSIS-PF in Research and Practice

As it relates to the general factor structure of the SSIS-PF in its published format,

evidence tentatively supports the validity of a seven-factor structure for the instrument. Two

iterations of the full first-order model showed adequate fit to sample data. Moreover, all

indicators demonstrated strong positive loadings on their respective factors. Thus, the seven first-

order SSIS-PF factors (i.e., Communication, Cooperation, Assertion, Responsibility, Empathy,

Engagement, Self-Control) appear to reflect meaningful domains under the ‘social skills’

construct, and the indicators used to tap into these domains appear to be have been appropriately

selected. A lack of discriminant validity among several of the first-order domains does temper

support for the seven-factor structure despite evidence of strong global model fit. Thus, further

examination and potential revision of item content for highly correlated domains (e.g.,

Cooperation and Responsibility) is warranted.

The higher-order factor structure implied by the published version of the SSIS-PF is also

in need of further examination and potential revision. The existence of a unitary ‘Social Skills’

104

factor subsuming all seven first-order domains was not supported in the current study. Instead,

based on exploratory analysis, a more complex higher-order structuring of the social skills

construct with two distinct second-order factors was shown to be more appropriate. In terms of

practical implications, the validity of using a single Social Skills score based on the aggregate

total of ratings across all seven domains of the SSIS-PF is called into question. If the revised

higher-order structure identified through the current study were to be replicated in future studies,

revision to the recommended procedures for SSIS-PF scoring and interpretation would likely be

required.

Implications for Cross-Cultural Social Skills Assessment

Current results are promising in terms of supporting first-order measurement invariance

for the SSIS-PF across groups defined by race/ethnicity and language format. It is worth noting,

however, that measurement invariance is only meaningful to the extent that an instrument’s

general measurement structure has been well validated. As such, initial focus on continuing to

generate evidence regarding the validity of the general measurement model of the SSIS-PF

should be the highest priority in future studies.

When moving beyond this specific instrument, current results suggest that the social

skills construct, at least in terms of behavioral indicators and first order domains, can be

meaningfully operationalized in ways that remain consistent across cultures defined by broad

race/ethnicity categories (i.e., African American, Caucasian, Latino) and language groups (i.e.,

English, Spanish). At this point, further study of the structural relationships among first-order

and higher-order social skills factors across cultural groups is needed before firm assertion can be

made in reference to similarities and differences. Similarly, studies designed with more

specificity in reference to cultural groupings also are needed (e.g., accounting for SES,

acculturation, language status, etc.).

105

Conclusions

Current evidence supports the first-order measurement structure of the SSIS-PF. All

seven first-order social skills domains (i.e., Communication, Cooperation, Assertion,

Responsibility, Empathy, Engagement, Self-Control) were supported, and all indicators

(comprised of two- and three-item sets) demonstrated appropriately strong relationships with their

respective domains. Within the context of the SSIS integrated assessment-for-intervention

paradigm, valid assessment at the item and domain levels is critical in terms of appropriately

guiding the intervention process. Thus, current evidence generally supports the instrument’s

utility for such purposes. It is important to note, however, that results also indicated limited

discriminant validity among several first-order social skills domains. Redundancy in first-order

measurement is not necessarily overly detrimental to the SSIS assessment-for- intervention

framework. Such results, however, do underscore the absence of a consistent and well-validated

classification structure for the social skills construct/domain (Gresham, 1986; Caldarella &

Merrell, 1997; Matson & Wilkins, 2009).

In contrast to results supporting the first-order measurement structure of the SSIS-PF, current

findings did not support the presence of a single, unitary social skills factor subsuming all seven

of the instrument’s first-order domains. In terms of practical implications, such results suggest

that the calculation of a single norm-referenced score may not be the most appropriate means of

quantifying an individual’s social skills as rated by parents on the SSIS-PF. Instead, exploratory

analysis suggests that the SSIS-PF may actually reflect two distinct higher-order domains:

Responsive Social Skills and Proactive Social Skills. More research is needed before conclusions

can be drawn regarding the higher-order measurement structure of the SSIS-PF. In terms of

methodological implications, the potential for iterative (exploratory and confirmatory)

multivariate analysis to move the field closer to consensus in reference to the most appropriate

means of conceptualizing the social skills construct should be noted.

106

Finally, as it relates to measurement invariance of the SSIS-PF and, more broadly, to the

study of social skills across cultures, current results should be interpreted with some caution. The

study of invariance is predicated on the validity of the general measurement model under study.

And, while the first order model of the SSIS-PF was supported in the current study, results also

indicate the need for further examination of the instrument’s higher-order structure. Limitations

notwithstanding, the first-order measurement structure of the SSIS-PF was shown to be invariant

across race/ethnicity and language format groups, indicating that the instrument can be used with

confidence to assess first-order social skills domains within and across each of the groups under

study. More broadly, such results indicate that narrow domains of social behavior (e.g.,

cooperation, engagement, self-control, etc.) can be objectively measured in consistent ways

across African American, Caucasian, and Latino individuals, as well as English-speaking and

Spanish-speaking groups. Based on results of the current study, though, conclusions and

implications concerning the existence of a broader, unitary social skills construct across groups

must be reserved until further research has been conducted.

107

References

Achenbach, T. M. (1995). Empirically based assessment and taxonomy: Applications to clinical

research. Psychological Assessment, 7, 261-274. doi: 10.1037/1040-3590.7.3.261

American Educational Research Association, American Psychological Association, and National

Council on Measurement in Education (1999). Standards for educational and

psychological testing. Washington, DC: American Educational Research Association.

Ang, R. P., & Hughes, J. N. (2001). Differential benefits of skills training with antisocial youth

based on group composition: A meta-analytic investigation. School Psychology Review, 31,

164-185.

Bandalos, D. L., & Finney, S. J. (2001). Item parceling issues in structural equation modeling. In

G.A. Marcoulides (ed.). New developments and techniques in structural equation modeling

(pp. 269-296). Mahwah, NJ: Lawrence Erlbaum.

Barrera, M., & Schulte, F. (2010). A group social skills intervention program for survivors of

childhood brain tumors. Journal of Pediatric Psychology, 34, 1108-1118. doi:

10.1093/jpepsy/jsp018

Barron-McKeagney, T., Woody, J. D., & D’Souza, H. J. (2001). Mentoring at-risk Latino

children and their parents: Impact on social skills and problem behaviors. Child and

Adolescent Social Work, 18, 119-136. doi: 10.1023/A:1007698728775

Bell-Dolan, D. J., Reaven, N. M., & Peterson, L. (1993). Depression and social functioning: A

multidimensional study of the linkages. Journal of Clinical Child Psychology, 22, 306-315.

doi: 10.1207/s15374424jccp2203_1

Bracken, B. A., Keith, L. K., & Walker, K. C. (1998). Assessment of preschool behavior and

social-emotional functioning: A review of thirteen third-party instruments. Journal of

Psychoeducational Assessment, 16, 153-169. doi: 10.1177/073428299801600204

108

Bronfenbrenner, U. (1986). Ecology of the family as a context for human development: Research

perspectives. Developmental Psychology, 22, 723-742. doi: 10.1037/0012-1649.22.6.723

Byrne, B. A. (2012). Structural Equation Modeling with Mplus: Basic Concepts, Applications,

and Programming. New York: Taylor & Francis Group, LLC.

Caldarella, P., & Merrell, K. W. (1997). Common dimensions of social skills of children and

adolescents: A taxonomy of positive behaviors. School Psychology Review, 26, 264-278.

Chang, F., Crawford, G., Early, D., Bryant, D., Howes, C., Burchinal, M., Barbarin, O.,

Clifford, R., & Pianta, R. (2007). Spanish-speaking children’s social and language

development in pre-kindergarten classrooms. Early Education and Development, 18, 243-

269. doi: 10.1080/10409280701282959

Cowen, E. L., Pederson, A., Babigan, H., Izzo, L. D., & Trost, M. A. (1978). Long-term follow-

up of early detected vulnerable children. Journal of Consulting and Clinical Psychology,

41, 438-446. doi: 10.1037/h0035373

Crick, N. R., & Dodge, K. A. (1994). A review and reformulation of social information-

processing mechanisms in children’s social adjustment. Psychological Bulletin, 115, 74 –

101. doi: 10.1037/0033-2909.115.1.74

Crowley, S. L., & Merrell, K. W. (2000). Convergent and discriminant validity of the

Internalizing Symptoms Scale for Children. Journal of Psychoeducational Assessment, 18,

4-16. doi: 10.1177/073428290001800101

Dawson, B. A., & Williams, S. A. (2008). The impact of language status as an acculturative

stressor on internalizing and externalizing behaviors among Latino/a children: A

longitudinal analysis from school entry through third grade. Journal of Youth and

Adolescence, 37, 399-411. doi: 10.1007/s10964-007-9233-z

Demaray, M. K., & Ruffalo, S. L. (1995). Social skills assessment: A comparative evaluation of

six published rating scales. School Psychology Review, 24, 648-671.

109

Edl, H. M., Jones, M. H., & Estell, D. B. (2008). Ethnicity and English proficiency: Teacher

perceptions of academic and interpersonal competence in European American and Latino

students. School Psychology Review, 37, 38-45.

Elliott, S. N., & Gresham, F. M. (1987). Children’s social skills: Assessment and classificaiton

practices. Journal of Counseling and Development, 66, 96-99.

Elliott, S. N., Gresham, F. M., Frank, J. L., & Beddow, P. A., III. (2008). Intervention validity of

social behavior rating scales: Features of assessments that link results to treatment plans.

Assessment for Effective Intervention, 34, 15-24. doi: 10.1177/1534508408314111

Elliott, S. N., Malecki, C. K., & Demaray, M. K. (2001). New directions in social skills

assessment and intervention for elementary and middle school students. Exceptionality, 9,

19-32.

Fantuzzo, J., Manz, P. H., & McDermott, P. (1998). Preschool version of the Social Skills Rating

System: An empirical analysis of its use with low-income children. Journal of School

Psychology, 36, 199-214. doi: 10.1016/S0022-4405(98)00005-3

Flanagan, D. P., Alfonso, V. C., Primavera, L. H., Povall, L., & Higgins, D. (1996). Convergent

validity of the BASC and SSRS: Implications for social skills assessment. Psychology in

the Schools, 33, 13-23. doi: 10.1002/(SICI)1520-6807(199601)33:1<13::AID-

PITS2>3.0.CO;2-X

Fuller, B., & Garciá-Coll, C. (2010). Learning from Latinos: Contexts, families, and child

development in motion. Developmental Psychology, 46, 559-565. doi: 10.1037/a0019412

Galindo, C., & Fuller, B. (2010). The social competence of Latino kindergartners and growth in

mathematical understanding. Developmental Psychology, 46, 579-592. doi:

10.1037/a0017821

110

Genesee, F., Lindholm-Leary, K., Saunders, W. M., & Christian, D. (Eds.). (2006). Educating

English language learners: A synthesis of research evidence. New York: Cambridge

University Press.

Gifford-Smith, M. E., & Brownell, C. A. (2003). Childhood peer relationships: Social acceptance,

friendships, and peer networks. Journal of School Psychology, 41, 235-284. doi:

10.1016/S0022-4405(03)00048-7

Greenspan, S. (1981). Defining childhood social competence: A proposed working model.

Advances in Special Educaiton, 3, 1-39.

Gresham, F. M. (1981). Assessment of children’s social skills. Journal of School Psychology, 19,

120-133. doi: 10.1016/0022-4405(81)90054-6

Gresham, F. M. (1986). Conceptual Issues in the Assessment of Social Competence in Children.

In P. S. Strain, M. M. Guralnick, & H. M. Walker (Eds.), Children’s social behavior:

development, assessment, and modification. (pp. 143-179). Orlando, FL: Academic Press.

Gresham, F. M. (2011). Social behavioral assessment and intervention: Observations and

impressions. School Psychology Review, 40, 275-283.Gresham, F. M., Cook, C. R., Crews,

S. D., & Kern, L. (2004). Social skills training for children and youth with emotional and

behavioral disorders: Intervention validity considerations and future directions. Behavioral

Disorders, 30, 32-46.

Gresham, F. M., & Elliott, S. N. (1990). Social skills rating system. Minneapolis, MN: NCS

Pearson, Inc.

Gresham, F. M., & Elliott, S. N. (2008). Social skills improvement system: Rating scales manual.

Minneapolis, MN: NCS Pearson, Inc.

Gresham, F. M., & Reschly, D. J. (1986). Social skill deficits and low peer acceptance of

mainstreamed learning disabled children. Learning Disability Quarterly, 9, 23-32. doi:

10.2307/1510398

111

Gresham, F. M., & Reschly, D. J. (1987). Dimensions of social competence: Method factors in

the assessment of adaptive behavior, social skills, and peer acceptance. Journal of School

Psychology, 25, 367-381. doi: 10.1016/0022-4405(87)90038-0

Gresham, F. M., Van, M. B., & Cook, C. R. (2006). Social skills training for teaching

replacement behaviors: Remediating acquisition deficits in at-risk students. Behavioral

Disorders, 31, 363-378.

Halgunseth, L. C., Ispa, J. M., & Ruddy, D. (2006). Parental control in Latino families: An

integrated review of the literature. Child Development, 77, 1282-1297. doi: 10.1111/j.1467-

8624.2006.00934.x

Hau, K. & Marsh, H. W. (2004). The use of item parcels in structural equation modeling: Non-

normal data and small sample sizes. British Journal of Mathematical Statistical

Psychology, 57, 327-351. doi: 10.1111/j.2044-8317.2004.tb00142.x

Hu, L. & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis:

Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55. doi:

10.1080/10705519909540118

Hu, L. & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to

underparameterized model misspecification. Psychological Methods, 1998, 3, 424-453. doi:

10.1037/1082-989X.3.4.424

KewalRamani, A., Gilbertson, L., Fox, M., & Provasnik, S. (2007). Status and trends in the

education of racial and ethnic minorities (NCES 2007-039). National Center for Education

Statistics, Institute of Education Sciences, U.S. Department of Education. Washington,

D.C.

Knight, G. P., & Hill, N. E. (1998). Measurement equivalence in research involving minority

adolescents. In V. C. McLoyd and L. Steinberg (Eds.), Studying Minority Adolescents:

112

Conceptual, Methodological, and Theoretical Issues (pp. 183-210). Mahwah, NJ: Lawrence

Erlbaum Associates.

Ladd, G. W. (1999). Peer relationships and social competence during early and middle childhood.

Annual Revue of Psychology, 50, 333-359. doi: 10.1146/annurev.psych.50.1.333

Ladd, G. W., Herald, S. L., & Andrews, R. K. (2005). Young children’s peer relations and social

competence. In B. Spodek, & O.N. Saracho (Eds.), Handbook of research on the education

of young children, (2nd ed., pp. 23-54). Mahwah, NJ: Lawrence Erlbaum Associates

Publishers.

Ladd, G. W., Kochenderfer, B. J., & Coleman, C. C. (1996). Friendship quality as a predictor of

young children’s early school adjustment. Child Development, 67, 1103-1118. doi:

10.2307/1131882

Lane, K. L., Carter, E. W., Pierson, M. R., & Glaeser, B. C. (2008). Academic, social, and

behavioral characteristics of high school students with emotional disturbances or learning

disabilities. Journal of Emotional and Behavioral Disorders, 14, 108-117. doi.

10.1177/10634266060140020101

Lansford, J. E., Malone, P. S., Dodge, K. A., Pettit, G. S., & Bates, J. E. (2010). Developmental

cascades of peer rejection, social information processing biases, and aggression during

middle childhood. Development and Psychopathology, 22, 593-602. doi:

10.1017/S095479410000301

Leidy, M. S., Guerra, N. G., & Toro, R. I. (2010). Positive parenting, family cohesion, and child

social competence among immigrant Latino families. Journal of Family Psychology, 24,

252-260. doi: 10.1037/a0019407

Livas-Dlott, A., Fuller, B., Stein, G. L., Bridges, M., Figueroa, A. M., & Mireles, L. (2010).

Commands, competence, and cariño: Maternal socialization practices in Mexican American

families. Developmental Psychology, 46, 566-578. doi: 10.1037/a0018016

113

López, A., Correa-Chávez, M., Rogoff, B., & Gutiérez, K. (2010). Attention to instruction

directed to another by U.S. Mexican-heritage children of varying cultural backgrounds.

Developmental Psychology, 46, 593-601.

Manz, P. H., Fantuzzo, J. W., & McDermott, P. A. (1999). The parent version of the preschool

Social Skills Rating Scale: An analysis of its use with low-income, Ethnic Minority

Children. School Psychology Review, 28(3), 493-504.

Matson, J. L. & Wilkins, J. (2009). Psychometric testing methods for children’s social skills.

Research in Developmental Disabilities, 30, 249-274. doi: 10.1016/j.rid.2008.04.002

McFall, R. M. (1982). A review and reformulation of the concept of social skills. Behavioral

Assessment, 4, 1-33. doi: 10.1007/BF01321377

Meade, A. W., & Kroustalis, C. M. (2006). Problems with item parceling for confirmatory factor

analytic tests of measurement invariance. Organizational Research Methods, 9, p. 369-403.

doi: 10.1177/1094428105283384

Merrell, K. (1999). Behavioral, Social, and Emotional Assessment of Children and Adolescents.

Mahwah, NJ: Lawrence Erlbaum Associates.

Merrell, K. W. (2001). Assessment of children’s social skills: Recent developments, best

practices, and new directions. Exceptionality, 9, 3-18. doi: 10.1207/S15327035EX091&2_2

Merrell, K. W., & Gimpel, G. A. (1998). Social skills of children and adolescents:

Conceptualization, assessment, treatment. Mahwah, NJ: Lawrence Erlbaum Associates

Publishers.

Merydith, S. P. (2001). Temporal stability and convergent validity of the Behavior Assessment

System for Children. Journal of School Psychology, 39, 253-265. doi: 10.1016/S0022-

4405(01)00064-4

Muthén, L.K. and Muthén, B.O. (2011). Mplus (Version 6.11). Los Angeles, CA: Muthén &

Muthén.

114

Najaka, S. S., Gottfredson, D. C., & Wilson, D. B. (2001). A meta-analytic inquiry into the

relationship between selected risk factors and problem behavior. Prevention Science, 2,

257-271. doi: 10.1023/A:1013610115351

Norton, P. J., Washington, C. S., Peters, J. E., & Hayes, S. A. (2010). Diversity considerations in

assessing social skills. In D.W. Nangle, D.J. Hansen, C.A. Erdley, & P.J. Norton (Eds.),

Practitioner’s guide to empirically based measures of social skills (pp. 135-153). New

York: Springer Publishing Co.

Okagaki, L., & Frensch, P. A. (1998). Parenting and children’s school achievement: A

multiethnic perspective. American Educational Research Journal, 35, 123-144. doi:

10.2307/1163454

Odom, S. L., McConnell, S. R., & Brown, W. H. (2008). Social competence of young children. In

W. H. Brown, S. L. Odom, & S. R. McConnell (Eds.), Social competence of young

children: Risk, disability, & intervention (pp. 3-29). Baltimore, MD: Paul H. Brookes

Publishing Co.

Parker, J. G., & Asher, S. R. (1993). Friendship and friendship quality in middle childhood: Links

with peer group acceptance and feelings of loneliness and social dissatisfaction.

Developmental Psychology, 29, 611-621. doi: 10.1037/0012-1649.29.4.611

Pedersen, J. A., Worrell, F. C., & French, J. L. (2001). Reliability of the Social Skills Rating

System with rural Appalachian children from families with low incomes. Journal of

Psychoeducational Assessment, 19, 45-53. doi: 10.1177/073428290101900103

Phelan, P. Davidson, A. L., and Cao, H. T. (1991). Students’ multiple worlds: Negotiating the

boundaries of family, peer, and school cultures. Anthrpology & Education Quarterly, 22,

224-250. doi: 10.1525/aeq.1991.22.3.05x1051k

Reichow, B., & Volkmar, F. R. (2010). Social skills interventions for individuals with autism:

Evaluation for evidence-based practices within a best evidence synthesis framework.

115

Journal of Autism and Developmental Disorders, 40, 149-166. doi: 10.1007/s10803-009-

0842-0

Rogoff, B. (2003). The Cultural Nature of Human Development. New York: Oxford University

Press.

Sheridan, S. M., & Walker, D. (1999). Social skills in context: Considerations for assessment,

intervention, and generalization. In C.R. Reynolds & T.B. Gutkin (Eds.), The handbook of

school psychology (3rd

ed., pp. 686-708). New York: Wiley.

Skoulos, V., & Shick Tryon, G. (2007). Social skills of adolescents in special education who

display symptoms of oppositional defiant disorder. American Secondary Education, 35,

103-115.

Spomer, M. L., & Cowen, E. L. (2001). A comparison of the school mental health referral

profiles of young ESL and English-speaking children. Journal of Community Psychology,

29, 69-82. doi: 10.1002/1520-6629(200101)29:1<69::AID-JCOP5>3.3.CO;2-M

Suárez-Orozco, C. S., Gaytán, F. X., Bang, H. J., Pakes, J., O’Connor, E., & Rhodes, J. (2010).

Academic trajectories of newcomer immigrant youth. Developmental Psychology, 46, 602-

618. doi: 10.1037/a0018201

U.S. Census Bureau. (2009) State & County QuickFacts: USA QuickFacts. Retrieved from

http://quickfacts.census.gov/qfd/states/00000.html

Van der Oord, S., Van der Meulen, E. M., Prins, P. J. M., Oosterlaan, J., Buitelaar, J. K., &

Emmelkamp, P. M. G. (2005). A psychometric evaluaton of the social skills rating system

in children with attention deficit hyperactivity disorder. Behaviour Research and Therapy,

43, 733-746. doi: 10.1016/j.brat.2004.06.004

Van Horn, M. L., Atkins-Burnett, S., Karlin, E., Ramey, S. L., & Snyder, S. (2007). Parent

ratings of children’s social skills: Longitudinal psychometric analyses of the social skills

116

rating system. School Psychology Quarterly, 22, 162-199. doi: 10.1037/1045-

3830.22.2.162

Vandenberg, R. J. & Lance, C. E. (2000). A review and synthesis of the measurement invariance

literature: Suggestions, practices, and recommendations for organizational research.

Organizational Research Methods, 3, 4-70. doi: 10.1177/109442810031002

Walthall, J. C., Konold, T. R., & Pianta, R. C. (2005). Factor structure of the social skills rating

system across child gender and ethnicity. Journal of Psycholoeducational Assessment, 23,

201-215. doi: 10.1177/073428290502300301

Ward, S., Sylva, J., & Gresham, F. M. (2010). School-based predictors of early adolescent

depression. School Mental Health, 2, 125-131. doi: 10.1007/s12310-010-9028-3

Warnes, E. D., Sheridan, S. M., Geske, J., & Warnes W. A. (2005). A contextual approach to the

assessment of social skills: Identifying meaningful behaviors for social competence.

Psychology in the Schools, 42, 173-187. doi: 10.1002/pits.20052

Warzon, K. B., & Ginsburg-Block, M. (2008). Cultural continuity between home and school as a

predictor of student motivation: What we know, what we need to learn, and implications

for practice. In C. Hudley & A.E. Gottfried (Eds.) Academic motivation and the culture of

school in childhood and adolescence (pp. 121-145). New York: Oxford University Press.

Whiteside, S. P., McCarthy, D. M., & Miller, J. D. (2007). An examination of the factor structure

of the Social Skills Rating System parent elementary form. Assessment, 14, 246-254. doi:

10.1177/1073191107302062.

Webster-Stratton, C., Reid, J., & Hammond, M. (2001). Social skills and problem-solving

training for children with early-onset conduct problems: Who benefits? Journal of Child

Psychology and Psychiatry, 42, 943-952. doi: 10.1111/1469-7610.00790

Weisner, T. S. (2002). Ecocultural understanding of children’s developmental pathways. Human

Development, 45, 275-281. doi: 10.1159/000064989

117

Weiner, J. (2004). Do peer relationships foster behavioral adjustment in children with learning

disabilities? Learning Disability Quarterly, 27, 21-30. doi: 10.2307/1593629

Wentzel, K. R. (1993). Does being good make the grade? Social behavior and academic

competence in middle school. Journal of Educaitonal Psychology, 85, 357-364. doi:

10.1037/0022-0663.85.2.357

Zins, J. E., Bloodworth, M. R., Weissberg, R. P., & Walberg, H. J. (2007). The scientific base

linking social and emotional learning to school success. Journal of Educaitonal and

Psychological Consultation, 17, 191-210. doi: 10.1080/10474410701413145

VITA

Brian Schneider

Education

Ph. D., School Psychology, Pennsylvania State University, University Park, PA, 2012

M. Ed., School Psychology, Pennsylvania State University, University Park, PA, 2009

B.A., Psychology, Muhlenberg College, Allentown, PA 2006

Fellowship

Specialization in Culture and Language Education (SCALE) Fellowship, 2008 – 2012

Professional Positions

School Psychologist, Owen J. Roberts School District, Pottstown, PA, 2012 – present

School Psychologist, Chester County Intermediate Unit, Downingtown, PA 2011 – 2012

Pre-Doctoral Intern, CORA Services, Inc., Philadelphia, PA 2010 – 2011

Professional Certification

School Psychologist (Pennsylvania)

Research Interests

Cross-Cultural Social Skills Assessment

Language and literacy development of English Language Learners

Multicultural issues in education

Early literacy development

Professional Presentations Schneider, B. P., & DiPerna, J.C. (2012, February). A Structural Analysis of the Social Skills

Improvement System Rating Scales, Parent Form: Measurement Invariance by Race/Ethnicity.

Poster presented at the National Association of School Psychologists Annual Convention,

Philadelphia, PA.

Schneider, B. P., & DiPerna, J. C. (2009, February). The home literacy environment’s effect on

emergent literacy outcomes. Poster presented at the National Association of School Psychologists

Annual Convention, Boston, MA.

Professional Memberships

National Association of School Psychologists, 2011 – present

National Association of School Psychologists, Student Member 2007 – 2011

American Psychological Associate, Student Affiliate, 2009 – 2010

Association of School Psychologists of Pennsylvania, 2006 – 2009