Embed Size (px)
DESCRIPTION
Early Findings from the Implementation and Impact Study of Early College High School. Larry Bernstein - RTI Ryoko Yamaguchi and Fatih Unlu - Abt Associates Julie Edmunds - SERVE Elizabeth Glennie - RTI John Willse, UNC - Greensboro and Nina Arshavsky -SERVE SREE Conference 2010 - PowerPoint PPT Presentation
Citation preview
RTI International is a trade name of Research Triangle Institutewww.rti.org
Early Findings from the Implementation and Impact Study of Early College High School
Larry Bernstein - RTIRyoko Yamaguchi and Fatih Unlu - Abt Associates
Julie Edmunds - SERVE Elizabeth Glennie - RTI
John Willse, UNC - Greensboro and Nina Arshavsky -SERVE
SREE Conference 2010March 4, 2010
www.rti.org
Too many students not graduating from high school Of those graduating, many not prepared for
postsecondary education and world of work Of every 100 students entering 9th grade in public HS in
North Carolina, only 70 graduate within 5 years Only 42 of them enroll in college Only 19 of them complete a two-/four-year degree
within 6 years of graduating from HS NC response: extensive public-private effort to
redesign HSs to make them more effective for all students
Background/Context
www.rti.org
Model
National Early College High School (ECHS) Initiative – Gates and other funders
Goal to increase number of students graduating from HS prepared for college/work
Targeted at traditionally underrepresented populations of students
Study focuses on North Carolina’s effort, supported by North Carolina New Schools Project
NC has over 60 schools across the state In NC, ECHSs are small schools, located on college
campuses, grades 9-12 or 9-13
www.rti.org
ECHS Design Principles
Improved student achievement
Increased graduation rates
Increased enrollment in collegePersonalization
Academic and affective supports
Supportive relationships
Intermediate Outcomes Long Term Outcomes
Increased student attendance
Improved attitudes toward self and school
Increased frequency of higher level courses
Increased aspirations toward college
Professionalism
Ongoing professional development
Collaboration among staff
Collective responsibility and decision-making
College Ready
Articulated program of study, grades 9-12 or 13 leading to Associate’s degree
College readiness activities
Powerful Teaching and Learning
High-quality, rigorous, and relevant instruction
Student collaboration and discussion
Formative and multiple assessments
Common standards
Increased graduation from college
Improved behavior
Purposeful Design:
Autonomous governance
Located on college campus
Small size
Flexible use of time
Integration with college
www.rti.org
Study Overview
Partnership between SERVE, NCDPI, NC New Schools Project, Duke University, UNCG, Abt Associates, RTI International
Funded through federal IES grant Four-year experimental study comparing students who
applied to and were randomly accepted into ECHS with students who applied for and were randomly not accepted (attended regular HS)
Participating schools use lottery to select students out of eligible applicant pool; many lotteries stratified to place priority on target population
www.rti.org
Study Goals
Need to determine if and how ECHS is effective:
1. Examine impact of model on student outcomes including: attitudes, attendance, achievement, course-taking, and school leaving/dropout rates
2. Determine whether impact varies by student characteristics, such as: gender; race/ethnicity; poverty status; first generation college status; and prior achievement
3. Determine whether specific program components are associated with better student outcomes
www.rti.org
Overview of Methodology
Experimental design– Schools determine eligible population– Lottery used to allocate spots– Lottery may be stratified if needed; analyses incorporate
weights to reflect differential probabilities of selection
Outcome data collected on both treatment and control students
Implementation data collected primarily on enrolled students– Some information on control group students’ experiences
also collected
www.rti.org
Lottery Sample
2005-2006 2006-2007 2007-2008
(Study Begins)
2008-2009
2009-2010
# of Sites
1 2 5 11 15
# of Students
78 213 433 1,117 1,403
Total: 34 cohorts in 20 sites 3,244 students
www.rti.org
Data Collected on both ECHS (Treatment) and Traditional (Control)
Extant Data from School Records
Test scores Course-taking Attendance Dropouts Discipline
Original Student Opinion Survey
Attitudes and engagement Behavior School experiences:
– Rigorous and relevant instruction; assessment
– Affective and academic support
– Relationships– Expectations
www.rti.org
Data Collected only on ECHS Staff and Students Implementation Surveys—All Staff and Students
– Students—same content as Student Opinion Survey – Staff—questions on implementation of the design principles – Completed annually
Site visits– Interviews with staff and students– Observations– Interviews with college faculty– Occurs once
www.rti.org
School extant data: 718 9th grade students in 8 cohorts in 6 sites
Student Opinion Survey: 575 9th Graders in 10 sites
Implementation data: Surveys from 937 students and 95 staff members in
11 schools Site visits to 9 schools; 11 more planned
Samples
www.rti.org
Background Characteristics of Current Ninth Grade Sample
Treatment Control
Black 22% 21%
Hispanic 6% 5%
Retained 1%* 5%*
First Generation 44% 49%
8th Grade Math Pass Rate
85%* 78%*
8th Grade Algebra Take Up
28% 22%
* significant at p ≤ .05; will need to account for differences in analyses.
Sample: 718 students in 8 cohorts in 6 schools
www.rti.org
Ninth Grade Results
Impacts on: College preparatory course-taking and progression Students’ behavior and attitudes Students’ high school experiences
www.rti.org
Why focus on course-taking?
Big difference between ECHS and Traditional: expectation that everyone goes to college
Students who don’t take Algebra I by end of 9th grade will find it much harder to complete college-preparatory course of study
Higher level math courses not required for graduation but test scores included in school accountability → traditional schools may steer students away from those courses
Therefore math is particularly sensitive indicator Look at course-taking patterns in other subjects as well
www.rti.org
Course-taking Findings
By end of 9th grade, significantly more ECHS students taking and successfully progressing through college preparatory courses
ECHS overall reducing performance gaps due to SES characteristics → leveling of playing field
www.rti.org
Impact on Algebra I
*Significant at p≤.05
www.rti.org
Impact on Geometry
www.rti.org
Impact on Algebra II
*Significant at p≤.05
www.rti.org
Impact on College Prep Mathematics Course-taking
*Significant at p≤.05
www.rti.org
Impact on Civics and Economics
*Significant at p≤.05
www.rti.org
Impact on English I
www.rti.org
Minimizing Gaps in Course Progression
Group Minority Non-Minority Gap
Treatment 79.4% 81.5% -2.1%
Control 57.3% 70.9% -13.6%
Algebra I Progression Rates for Minority and Non-Minority Students
www.rti.org
Gaps in Course Progression by Group
Min. vs. Non-Min. 1st. Gen. vs. Non-1st Gen.
FRL vs. Non-FRL
Alg. I. Eng. I Alg. I Eng. I Alg. I Eng. I
www.rti.org
Impact on Students’ Behavior, Attitudes and Experiences
ECHS students missed significantly fewer days of school (unexcused)
ECHS students were less likely to be suspended ECHS students reported significantly higher levels
of academic engagement and greater self-efficacy in math
Overall, ECHS students reported significantly more positive experiences than control students
www.rti.org
Impact on Student Attendance
*Significant at p≤.05
www.rti.org
Impact on Student Behavior
*Significant at p≤.05
www.rti.org
Impact on Student Attitudes
Any overall impacts in favor of ECHS Impacts vary by school ECHS students report higher levels of academic
engagement than students in control group (effect size = .3)
Positive impact on self-efficacy in math (effect size = .2)
No impact on other outcomes: – Self-efficacy in English – Persistence (e.g. continuing to do work, etc.)
www.rti.org
Looking at Implementation
Collecting data on Design Principles: 1. College Ready
2. Powerful Teaching and Learning
3. Personalization
4. Professionalism
5. Purposeful Design
Data from Student Opinion Survey Data from Implementation Survey and site visits
www.rti.org
Implementation Findings
Overall, ECHS students experience more positive school environments than students in control group
Level of implementation overall high but varies by school
No apparent pattern in level of implementation by design principles
www.rti.org
Students’ Experiences
Scale Effect Size
High Expectations +.70
Relationships +.40
Challenge +.38
Relevant Instruction +.51
Rigorous Instruction +.51
Academic and Social Support +1.07
Source: Student Opinion Survey, administered to treatment and control
www.rti.org
Conclusions
• North Carolina's ECHS model is creating a more positive school environment
• Students in this environment have fewer absences and fewer suspensions
• ECHS students more likely to be on-track for college (defined as taking and succeeding in the required courses)
• Results suggest that ECHS model also most effective for target populations of students traditionally underrepresented in college
