Explaining Gender and Racial Disparities in Academics Using Report Card Comments:

Bringing to Life “The Other Side of the Report Card” with Social-Emotional Learning

Dominic C. Moceri, M.S., Gwyne W. White, B.A.,Jazmin A. Reyes, M.S., and Maurice J. Elias, Ph.D.

Rutgers, The State University of New JerseyDepartment of Psychology

Presented at the Society for Community Research and Action 13th Biennial Conference, June 17th, 2011, Chicago, IL

• Academic Disparities & Implementation Challenges

• Social-Emotional Learning Indicators Project

• Results for one academic year at one school

• Conclusions, future studies and implications for education

Outline

• Girls outperform boys in all major subjects but perform equal to (or weaker than) boys on standardized tests

• Black students receive lower standardized test scores than white students

• NCLB of 2001• National priority to close the achievement gap between

• Races, Ethnicities• Income levels• Zip codes

The Problem of Academic Disparities

(AAUWEF, 1998; Good, et al., 2003; Pomerantz et al, 2002; Snyder & Dillow 2011; USDOE, 2005)

• Psychosocial intervention challenges• How to implement outside of well-controlled conditions?• How to sustain over time?• How to scale up?

• School specific implementation challenges• Budget cuts!• Overburdened teachers!• Inconsistent data collection!

Implementation Challenges

(Elias, 2007; Elias, et al., 2003; Proctor et al., 2009)

• aka Bringing to Life the Other Side of the Report Card

• Goals1) Promote social-emotional learning (SEL) in schools to address

the academic disparities2) Evaluate comments on report cards to address the

implementation challenges• Especially, for SEL skills

Social-Emotional Learning Indicators Project (SEL-IP)

Social-Emotional Learning (SEL)

• “capacity to recognize and manage emotions, solve problems effectively, and establish positive relationships with others” with prosocial attitudes, values and judgments (Zins & Elias, 2006, p. 1)

• “based on the understanding that • (1) many different kinds of problem behaviors are caused by the same or

similar risk factors, and • (2) the best learning emerges from supportive relationships that make

learning both challenging and meaningful” (CASEL, 2005, p. 5)

• Outcomes• mastery of academic subject material• standardized achievement test scores

• $$$ on academic abilities and achievements, but little on evaluating the validity of behavioral ratings

• Opportunity• most parent-teacher conversations focus on students’ behaviors• natural part of a teacher’s job

• no extra work• usually completed on a quarterly basis

• allows for the analysis of time-lagged relationships

Comments on Report Cards

• Replicating Academic Discrepancies• H1: G > B and Wh > Bl for academic grades• H2a: G ≤ B for standardized test score• H2b: Wh > Bl for standardized test scores

• SEL-IP Specific Hypotheses• H3a: H1 & H2 will disappear (or at least be greatly reduced)

after controlling for (+) and (-) comments on the report cards.• H3b: SEL comments > other comments for H3a

Hypotheses

• HS in Northern NJ• High SES district• ~2000 enrollment• 34.1% White, 55.1% Black, 4.4% Hispanic, 2.7% Asian

• Dataset for 2008-2009• Classes: n = 1318 (L) & 1117 (M)• HSPA: n = 449 (L) & 477 (M)• Demographics

• 50.1% female• 61.8% Black, 38.2% White (filtered)• 19.9% free/reduced lunch• 12.1% IEP

Participants

• Normal report cards completed by teachers for 2008-2009• Academic grades• Comments

• Standardized test scores for 2008-2009

• Creating Comment Variables• Each student could receive 2 comments per class per quarter

• 25 eligible comments• We created 6 key categories

• 1st: what conceptually go together• 2nd: tested using Cohen’s alpha & correlations• 3rd: drop variables that fail step 2• 4th: add # of comments for a given class across the year

Materials & Coding

Items for Categories

Pos1: General Academic

Pos2: SEL Type

Ambiv1: Trying Hard

Neg1: Behavior

Neg2: Work

Neg3: Absences & Tardies

Demonstrates academic excellence

Prepared & exhibits interest

Shows improvement

Interferes w/ class – friction

Needs to bring materials/ assignments

Often absent or late

Demonstrates critical thinking

Contributes to class

Tries hard finds subject difficult

Discipline & disruptive behavior

Tests and quizzes

Grade affected by absences

Is interested & eager to learn

Is improving Grade affected by tardies

Constructive influence in class

Frequencies & Correlations: Language

1 2 3 4 5 6

Pos2: SEL Type 41% .05+ -.11*** -.07** -.12*** -.12***

Pos1: General Academic .05+ 6% -.07** -.04+

Ambiv1: Trying Hard -.11*** -.07** 16%

Neg1: Behavior -.07** 3%

Neg2: Work

-.12*** 13%

Neg3: Absences & Tardies -.12*** -.04+ 10%

+ p < .10 * p < .05, ** p < .01, *** p < .001

Frequencies & Correlations: Math

1 2 3 4 5 6

Pos2: SEL Type 46% -.20*** -.08** -.05+ -.18***

Pos1: General Academic 15% -.14** -.10**

Ambiv1: Trying Hard -.20*** -.14*** 27% -.06*

Neg1: Behavior -.08** 2%

Neg2: Work

-.05+ -.06* 14%

Neg3: Absences & Tardies -.18*** -.10 12%

+ p < .10 * p < .05, ** p < .01, *** p < .001

Variables Step 1 (β) Step 2 (β)

Step 1: Demographics Gender Ethnicity Econ Dis IEP

.19*** .42*** -.06* -.08***

.13*** .32*** -.03 -.04+

Step 2: Comments SEL-Type Pos Academic Neg Behavior Neg Work Absences/Tardies Trying Hard

.31*** .17*** -.07*** -.14*** -.13*** -.08***

ΔR2 .25*** (med) .19*** (med)

Total R2 (n = 1196) .43*** (large)

DV = Language/Literature GPA

+ p < .10 * p < .05, ** p < .01, *** p < .001

Variables Step 1 (β) Step 2 (β)

Step 1: Demographics Gender Ethnicity Econ Dis IEP

.06* .35*** -.06+ -.14***

.04+ .19*** -.03 -.07**

Step 2: Comments SEL-Type Pos Academic Neg Behavior Neg Work Absences/Tardies Trying Hard

.36*** .25*** -.07*** -.23*** -.17*** -.18***

ΔR2 .16*** (med) .36*** (large)

Total R2 (n = 1017) .53*** (large)

DV = Math GPA

+ p < .10 * p < .05, ** p < .01, *** p < .001

Variables Step 1 (β) Step 2 (β) Step 3 (β)

Step 1: Demographics Gender Ethnicity Econ Dis IEP

.10* .52*** -.04 -.35***

.07 .44*** -.02 -.27***

.06 .41*** -.03 -.27***

Step 2: Comments SEL-Type Pos Academic Trying Hard

.14** .11* -.21***

.11* .10* -.20***

Step 3: Grades Lang GPA .09

ΔR2 .42*** (large) .07*** (small) .00

Total R2 (n = 278) .50*** (large)

DV = Language HSPA

+ p < .10 * p < .05, ** p < .01, *** p < .001

Variables Step 1 (β) Step 2 (β) Step 3 (β)

Step 1: Demographics Gender Ethnicity Econ Dis IEP

-.03 .63*** -.09+ -.33***

-.04 .52*** -.08+ -.28***

-.05 .44*** -.09* -.22***

Step 2: Comments SEL-Type Pos Academic Neg Behavior Neg Work Trying Hard

.11* .12** -.14*** -.11* -.17***

.01 .06 -.12** -.01 -.08+

Step 3: Grades Math GPA .34***

ΔR2 .55*** (large) .08** (small) .05*** (small)

Total R2 (n = 216) .67*** (large)

DV = Math HSPA

+ p < .10 * p < .05, ** p < .01, *** p < .001

• Both• Comments reduced disparities

• GPA• Comments add to prediction

• Medium to Large effect size• SEL-Type = Largest Beta for Comments

• HSPA• Comments add to prediction, even after adding GPA

• Small effect size• LA: Trying Hard , SEL-Type, Pos Acad; GPA not sig after comments• MA: Trying Hard & Neg Behaivor

Results Summary

• Limitations• Comments are not ideal measure of SEL-skills.• Some comments had low frequencies for this academic year.• Teachers are raters for both grades and comments.• Many students w/ IEPs excluded.

• Future Studies• Can report card comments predict changes in grades?• What are the ideal comments?• Expert SEL coders

Limitations & Future Studies

Implications

• SEL can be measured with report cards• May lead to comments being a screening tool• Parents are right to focus on comments during teacher

meetings

• SEL-type comments and other comments help explain academic disparities• For LA grades, MA grades and LA standardized tests

ReferencesAmerican Association of University Women Educational Foundation. (1998). Gender gaps: Where schools still fail our children.

Washington, DC: Author.Collaborative for Academic, Social, and Emotional Learning. (2005). Safe and sound: An educational leader’s guide to evidence-

based social and emotional learning (SEL) programs (Illinois Edition). Retrieved from http://www.casel.org/pub/safeandsound.php

Elias, M. J. (2007). From model implementation to sustainability: A multisite study of pathways to excellence in social-emotional learning and related school programs. In A. M. Blankstein, P. D. Houston, R. W. Cole (Eds.), Sustaining Professional Learning Communities: The Soul of Educational Leadership Series (pp. 59-95). Thousand Oaks, CA: Paul Chapman Publishing.

Elias, M. J., Zins, J. E., Graczyk, P. A., & Weissberg, R. P. (2003). Implementation, sustainability, and scaling up of social-emotional and academic innovations in public schools. School Psychology Review, 32, 303–319.

Good, C., Aronson, J., & Inzlicht, M. (2003). Improving adolescents' standardized test performance: An intervention to reduce the effects of stereotype threat. Journal of Applied Developmental Psychology, 24, 645-662.

Pomerantz, E. M., Altermatt, E. R., & Saxon, J. L. (2002). Making the grade but feeling distressed: Gender differences in academic performance and internal distress. Journal of Educational Psychology, 94, 396-404.

Proctor, E. K., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, B. (2009). Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health, 35, 24-34.

Snyder, T.D., & Dillow, S.A. (2011). Digest of Education Statistics 2010 (NCES 2011-015). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. Washington, DC.

United States Department of Education. (2005, August). How No Child Left Behind benefits African Americans. Retrieved April 20, 2011, from http://www2.ed.gov/nclb/accountability/achieve/nclb-aa.html

Zins, J.E., & Elias, M.E. (2006). Social and Emotional Learning. In G.G. Bear & K.M. Minke (eds.) Children's needs III (pp. 1-13). National Assocation of School Psychologists.

Thank You

Correspondence concerning this presentation should be addressed to Dominic C. Moceri, M.S., dmoceri@eden.rutgers.eduPsychology Department at Rutgers, The State University of New Jersey

Symposium Title:Moceri, D. C., White, G. W., Reyes, J. A., Elias, M. J., & Weissberg, R. (2011, June). Breaking Barriers to Academic Success: The Role of Social-Emotional and Character Development in Creating Change. Symposium to be presented at the Society for Community Research and Action 13th Biennial Conference, Chicago, IL.

Presentation Title:Moceri, D. C., White, G. W., Reyes, J. A., & Elias, M. J. (2011, June). Explaining Gender and Racial Disparities in Academics Using Report Card Comments: Bringing to Life “The Other Side of the Report Card” with Social-Emotional Learning. Paper to be presented at the Society for Community Research and Action 13th Biennial Conference, Chicago, IL.

Funding provided by NoVo Foundation and the Collaborative for Academic, Social, and Emotional Learning (CASEL). The content of this presentation does not necessarily reflect their views, policies, or endorsements.