The School Improvement Grant Program: Analysis of Performance in America’s

Great City Schools

Council of the Great City Schools Fall 2014

PurposeExamine trends in performance for schools

across the country that received SIG awards as part of the American Recovery and Reinvestment Act of 2009 (ARRA)

Analyze performance for schools receiving grant awards (SIG Award Schools) compared with:◦SIG Eligible Schools – those schools deemed

eligible for SIG awards, but not receiving any funding in Cohort 1 or Cohort 2 of the award cycle;

◦Non-SIG Eligible Schools – those schools across the country not eligible for SIG funding due to higher levels of student achievement.

MethodologyGrades 3-8 Trends

◦ Change in Percentage of Students At or Above Proficient◦ Percentage of Schools Increasing the Percentage of

Students At or Above Proficient by level of improvement: No Improvement 1% to 4%, 5% to 9%, 10% or more

◦ Change in Percentage of Students Below Basic2012-2013 Sample

◦ 13 CGCS States◦ 21 CGCS Districts◦ States were excluded based on three criteria:

1. Fall Testing Dates2. Changes in State Assessments (Content and/or Cut Scores)3. No Data or Poor Data Quality

GRADES 3-8 TRENDS

Percentage of Students Proficient in Math by School Type, Pre & Post

SIG Funding

09-10 - Base-line

10-11 11-12 12-1320%

30%

40%

50%

60%

70%

80%

27.4%

33.4%38.0% 38.1%

49.1%51.7% 52.9% 52.7%

64.6%66.8% 68.1% 68.0%

SIG Awarded Schools (n=173)

Random Sample of SIG Eligible and Non-Awarded Schools (n=626)

Random Sample of Schools Across the State (n=984)

Academic Year

Perc

en

tag

e o

f S

tud

en

ts A

t or

Ab

ove

Pro

fici

en

t

Percentage of Students Proficient in Reading by School Type, Pre &

Post SIG Funding

09-10 - Base-line

10-11 11-12 12-1325%

30%

35%

40%

45%

50%

55%

60%

65%

70%

30.6%33.8%

36.9% 37.6%

47.5%49.2%

51.4% 51.4%

64.7% 66.1% 67.4% 67.3%

SIG Awarded Schools (n=181)Random Sample of SIG Eligible and Non-Awarded Schools (n=628)Random Sample of Schools Across the State (n=986)

Academic YearPerc

en

tag

e o

f S

tud

en

ts A

t or

Ab

ove

Pro

fici

en

t

Percentage of Schools Improving in Mathematics by Category and School

Type, 2010 to 2013

0%

10%

20%

30%

40%

50%

60%

34.5%

27.6%

18.0%20.0%

35.6%

20.7%

16.7%

27.0%

19.6%

8.8%

14.7%

56.9%

27.0%

14.9%12.2%

45.9%

Random Sample of Schools Across States (n=986)Random Sample of SIG Eligible Non-Award Schools (n=627)Non-CGCS SIG Award Schools (n=102)CGCS SIG Award Schools (n=74)

Perc

enta

ge o

f S

chools

Im

pro

vin

g

Percentage of Schools Improving in Reading

by Category and School Type, 2010 to 2013

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

35.1%

30.0%

21.6%

13.2%

31.5%

24.2%

22.6%21.7%

19.4%

24.3%

17.5%

38.8%

23.4%24.7%

22.1%

29.9%

Random Sample of Schools Across States (n=989)Random Sample of SIG Eligible Non-Award Schools (n=628)Non-CGCS SIG Award Schools (n=103)CGCS SIG Award Schools (n=77)

Perc

enta

ge o

f S

chools

Im

pro

vin

g

Percentage of Students Below Basic in Math by School Type, Pre % Post SIG

Funding

09-10 - Base-line

10-11 11-12 12-1310%

15%

20%

25%

30%

35%

40%

45%

41.9%

35.9%

31.7% 31.9%

24.1% 23.0% 22.0% 22.6%

16.4% 15.3% 14.4% 14.7%

SIG Awarded Schools (n=156)Random Sample of SIG Eligible and Non-Awarded Schools (n=535)Random Sample of Schools Across the State (n=686)

Academic Year

Perc

en

tag

e o

f S

tud

en

ts B

elo

w B

asi

c (L

ow

est

Perf

orm

an

ce L

evel)

Percentage of Students Below Basic in Reading by School Type, Pre & Post

SIG Funding

09-10 - Base-line

10-11 11-12 12-1310%

15%

20%

25%

30%

35%

33.7%

30.5%

27.5% 27.5%

22.6%21.5%

19.6% 19.9%

13.0% 13.0%11.8% 12.5%

SIG Awarded Schools (n=156)Random Sample of SIG Eligible and Non-Awarded Schools (n=537)Random Sample of Schools Across the State (n=687)

Academic YearPerc

en

tag

e o

f S

tud

en

ts B

elo

w B

asi

c (L

ow

est

Perf

orm

an

ce L

evel)

PurposeThe Council of the Great City Schools examined how

member districts were implementing School Improvement Grants (SIG) that were funded through the American Recovery and Reinvestment Act of 2009 (ARRA). What were the effects of the program on student achievement?

Districts were chosen for case studies based on state test scores in math and reading, following an analysis of Cohort 1 data. Some districts were chosen because they showed increases in scores; others were chosen because they showed no changes or decreases.

The Council’s research team interviewed central-office staff and school-based personnel who were involved in the design and/or implementation of the grant between 2009-2013.

Research QuestionsThe research team was interested in the following research questions

1. What was the political and organizational context of the district during the SIG implementation? What were the districts’ instructional areas of focus during the study period?

2. What were the school goals and objectives beyond state and district objectives during that period, and what was the process for setting those goals?

3. What kind of interventions were put in place to improve academic performance in the SIG schools?

4. How were the grant-funded schools held accountable for student achievement? What measures were used?

5. What professional development was available for teachers and administrators to address the academic needs of students and special populations performing poorly?

6. What are school and district plans for sustaining programs and processes implemented with SIG funding?

Districts InterviewedClevelandColumbusDenverMiami-Dade CountyMilwaukeePhiladelphiaSan FranciscoSeattle

KEY FINDINGS

Political and Organizational ContextPrior to SIG, respondents reported that there were:

◦ Few support structures in place for low performing schools

◦ No clear direction or organization

◦ Frequent changes in leadership, and

◦ Few high-quality interventions in the lowest performing schools.

Post SIG, respondents reported that there were:

◦ Many schools that developed turnaround plans

◦ Inconsistent initiatives across buildings--few districts developed cohesive plans to address the needs of all SIG schools Schools were often siloed within

the districts Turnaround schools could opt out

of district curriculum Inconsistent performance

◦ Little consistent direction or organization across schools

◦ State intervention was irregular and often not coordinated with the district.

Goals and ObjectivesSchool goals included: Building a strong support team Building teacher buy-in and ownership throughout the

turnaround process Becoming better users/consumers of data Improving student achievement Building relationships with the community Improving parent engagement Improving school climate and morale Increasing student attendance and decreasing student

suspension rates Setting higher expectations for students by increasing

the rigor of instruction Enhancing curriculum materials Providing professional development on instructional

practices and data uses

StaffingDistricts used SIG funding to address personnel concerns: Hiring turnaround principals Working with teacher unions to:

◦ Manage staff turnover process–ensuring low performing schools attracted high quality teachers, and

◦ Extend school days and professional development hours while working on a joint understanding of the unique needs of low performing schools.

Developing unique administrative structures to support low performing schools (i.e., specific school regions or “chancellors district”-like structures )

Hiring:◦ instructional supervisors/coaches◦ reading and math specialists◦ social workers/counselors

Engaging parents and the community Ensuring the fidelity of grant implementation

InterventionsSchools targeted grant funds on student learning by: Increasing school partnerships with community organizations

◦ AVID◦ City Year◦ College Summit◦ Teach for America◦ Peace Corps◦ Communities in Schools

Reducing class sizes Hiring part-time tutors to support struggling students Implementing a new and more rigorous curriculum--often

with a literacy focus Extending school-day time Adding after-school, intercession, and summer enrichment

programs Providing incentives for teachers to improve student

performance Increasing professional development hours for teachers

Professional Development

Schools supported staff by:Providing extensive professional

development to support SIG initiativesFocusing on data useDeveloping an embedded professional

development model, e.g., co-teaching with veteran or “strong” teachers

Improving tools to support teachers (i.e., dashboards, planning tools, etc.)

Allotting time for feedback from teachers and other school leaders

Increasing professional development hours with an emphasis on job-embedded support

AccountabilityStates and districts held schools accountable by:Conducting classroom walkthroughs with

school, district, and state leaders– ◦But there was inconsistent implementation within

buildings◦And classroom observations were less punitive

and more informative and supportiveUsing assessments to improve classroom

instruction and determine interventions in addition to teacher evaluations

Implementing more focused weekly supports and review systems in low performing schools

CONCLUSIONS

Challenges and Opportunities

Challenges◦ Grant was a temporary

solution for larger systemic issues lack of high quality

intervention programs difficulty recruiting and

retaining high quality teachers

◦ Loss of staff that were hired through SIG

◦ Once funding is gone, few plans for support remained across school buildings from district and state leaders

OpportunitiesDistricts and schools may continue to:

◦ Foster partnerships with organizations to support schools

◦ Collaborate with central office staff and seek support

◦ Focus on data to inform instruction

◦ Engage parents and the community

◦ Provide support to teachers through professional development.

25

When It Worked and When It Didn’tWhen It Worked A clear coherent districtwide

plan for turning around low performing schools.

Central office supported low performing schools.

Schools provided flexibility in making staff changes/removing poor performing teachers.

Well coordinated and targeted interventions and supports for struggling students.

Leveraging data to identify professional development for teachers.

Teachers had clear understanding of challenges and commitment needed to succeed.

When It Didn’t Disconnected districtwide plans

that often resulted in the lack of a coordinated strategy.

State and central office administrators focused on grant compliance, not coordination.

Redundant or contradictory state and local intervention efforts

Schools had difficulty removing poor performing staff or hiring stronger teachers.

Excess flexibility for the capacity of the school.

Little evaluation of intervention efforts, and/or leaders were not always clear about the benefits of intervention programs.

Weak instructional interventions

Questions and Answers