Upload
goro
View
23
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Pennsylvania Charter Schools: What’s Working and What’s Not. Christopher Nelson, Ph.D. The Evaluation Center Western Michigan University [email protected] http://www.evaluation.wmich.edu. Purposes of the Talk. Identify key strengths and weaknesses Identify key policy issues - PowerPoint PPT Presentation
Citation preview
Pennsylvania Charter Schools: What’s Working and What’s Not
Christopher Nelson, Ph.D.The Evaluation CenterWestern Michigan [email protected]
http://www.evaluation.wmich.edu
The Evaluation Center Western Michigan University
2
Purposes of the Talk Identify key strengths and
weaknesses Identify key policy issues Generate discussion of options for
addressing weaknesses and shoring up strengths
The Evaluation Center Western Michigan University
3
Outline Background and data sources Evaluative criteria Key findings Questions and discussion
Background and Data Sources
The Evaluation Center Western Michigan University
5
Background on the Evaluation Act 22 mandates evaluation after 5 years Contract awarded through competitive
bidding process Current report is culmination of 4 years of
field research and secondary data analysis One of the longest sustained charter school
evaluations Evaluation report focuses mainly on
aggregate trends and patterns
The Evaluation Center Western Michigan University
6
Data Sources Site visits to nearly all charter
schools Interviews Focus groups Observations
The Evaluation Center Western Michigan University
7
Data Sources (cont’d) Analysis of secondary data
PSSA Demographic information Financial data
Charter school annual reports In-depth information on school missions
and practices
The Evaluation Center Western Michigan University
8
Data Sources (cont’d) Documentation provided by schools
Mission statements Internal evaluations
Interviews with district officials Interviews with other stakeholder
groups
Evaluative Criteria
The Evaluation Center Western Michigan University
10
Factual Claims +
Criteria of Merit & Worth
=
Evaluative Judgments
Evaluative Criteria
Evaluative judgments combine facts and values
Thus, data alone cannot fully address questions of merit and worth
The Evaluation Center Western Michigan University
11
Evaluative Criteria . . .According to Act 22
“Improve pupil learning . . .”
“Provide expanded choices . . .”
“Increase learning opportunities for all pupils . . .”
“Encourage . . . different and innovative teaching practices . . .”
“Create new professional opportunities for teachers . . .”
“Hold charter schools accountable . . .”
Key Findings
Student Achievement
The Evaluation Center Western Michigan University
13
Key FindingsStudent Achievement
Student achievement as an evaluative criterion
Key issues Aggregate trends Consistency of the charter school effect Correlates of success
The Evaluation Center Western Michigan University
14
1000
1100
1200
1300
1400
1500
1600
Charter schools All PA public schools
Key FindingsStudent Achievement
Charter school students have typically scored well below the average PA student
But score levels say more about student composition than school effectiveness
150 point difference
The Evaluation Center Western Michigan University
15
Key FindingsStudent Achievement
Charter school students have scored only slightly below those in demog. similar noncharter schools
But matching on demographics might mask less tangible differences (e.g., “interventionist” parents)
1000
1100
1200
1300
1400
1500
1600
Charter schools Demographically & geographicallysimilar schools
36 point difference
The Evaluation Center Western Michigan University
16
Key FindingsStudent Achievement
Average charter school student has gained 15 points per year, net of changes in student composition
-100
-50
0
50
100
1998 1999 2000 2001 2002
Ave
rage
filte
red
scor
e
1997 cohort
1998 cohort
1999 cohort
2000 cohort
Zero indicates parity with comparison schools
The Evaluation Center Western Michigan University
17
Key FindingsStudent Achievement Yet, there is considerable variation
among schools – some positive
Com
posi
te F
ilter
ed S
core
Year1999 2000 2001 2002
-400
-200
0
200
400
The Evaluation Center Western Michigan University
18
Key FindingsStudent Achievement Yet, there is considerable variation
among schools – others negative
C
om
posi
te F
iltere
d S
core
Year1998 1999 2000 2001 2002
-400
-200
0
200
400
The Evaluation Center Western Michigan University
19
Key FindingsStudent Achievement Yet, there is considerable variation
among schools
C
ompo
site
filt
ered
sco
re
Year1998 1999 2000 2001 2002
-400
-200
0
200
400
The Evaluation Center Western Michigan University
20
Key FindingsStudent Achievement – the Bottom Line
Achievement appears to be a source of modest strength
But considerable school-to-school variation
Not yet clear what the drivers of success are
Key Findings
Choice and Satisfaction
The Evaluation Center Western Michigan University
22
Key FindingsChoice and Satisfaction
Consumer satisfaction as an evaluative criterion
Key issues Satisfaction with charter schools Number of charter school options Are charter school options any different than those
in noncharter public schools? – the question of “innovation”
The Evaluation Center Western Michigan University
23
Key FindingsChoice and Satisfaction
Parents, students, and teachers are generally satisfied
However, unable to make comparisons with noncharter schools
I am satisfied with the instruction offered
4% 5%14%
24%
53%
0%
20%
40%
60%
80%
100%
Stronglydisagree
Disagree Neitheragree ordisagree
Agree Stronglyagree
The choice “halo” effect?
The Evaluation Center Western Michigan University
24
Key FindingsChoice and Satisfaction
Typical charter school has a self-reported waiting list of 28% of current enrollment
Yet, schools report turnover rates of 0 - 40% per year Dissatisfaction, counseling out, or
realization of poor fit?
The Evaluation Center Western Michigan University
25
Key FindingsChoice and the “Supply” of Options
The number of charter school options has grown considerably
0
10
20
30
40
50
60
70
80
1997-98 1998-99 1999-2000 2000-01 2001-02
Num
ber
of S
choo
ls
0
5,000
10,000
15,000
20,000
25,000
30,000
Num
ber of Students
SchoolsStudents
The Evaluation Center Western Michigan University
26
Key FindingsChoice and the “Supply” of Options
Yet, those options remain concentrated in a few regions of the Commonwealth
Only 18 of 67 counties (27%) have charter schools
Of those, charter enrollment exceeds 1% in only 10
County2001/02
Enrollment(% total)
Philadelphia 7.9
Clinton 4.4
Beaver 4.2
Bucks 3.8
Chester 2.5
Charter EnrollmentTop 5 Counties
The Evaluation Center Western Michigan University
27
Key FindingsChoice and Innovation
Many charter schools offer new choices to particular education markets
Non-traditional grade groupings (67%)
Unique mission foci Non-traditional use of
time and calendar Smaller schools &
classes Cyber schools
Non-traditional grade groupings (67%)
Unique mission foci Non-traditional use of
time and calendar Smaller schools &
classes Cyber schools
The Evaluation Center Western Michigan University
28
Key FindingsChoice and Innovation
However, there is little evidence that charter schools are creating wholly new educational approaches
Barriers to innovation Slack time Slack resources Challenges of start-up Consumer preferences Problems with CAB
definition
The Evaluation Center Western Michigan University
29
Key FindingsChoice and Innovation – the Bottom Line
Parents, students, and teachers appear to be quite satisfied with their charter schools
Number of charter school options has grown considerably but remains geographically concentrated
Many charter schools offer new options in particular markets, but few appear to be producing wholly new educational approaches
Key Findings
Equity and Access
The Evaluation Center Western Michigan University
31
Key FindingsEquity and Access
School choice . . . Engine of desegregation? Sorting machine?
Key issues Types of communities served Types of students selected (self-
selected?) from these communities
The Evaluation Center Western Michigan University
32
Key FindingsEquity and Access – School Location
Charter schools tend to be in areas with larger schools, lower test scores, and higher concentrations of low income and nonwhite students.
Variable Nonchartering Chartering
Median total district enrollment (2001-02) 1,749 3,446
Median per-pupil expenditure (2000-01) $8,319 $8,304
Median aggregate PSSA score (2001-02) 1341 1221
Median % low income students (2001-02) 20.3 63.9
Median % nonwhite students (2001-02) 4.9 67.0
The Evaluation Center Western Michigan University
33
Key FindingsEquity and Access – (Self?) Selection
Declining relative percentage over time
Considerable school-by-school variation
Charter-Host Demographica Differences
-20
-10
0
10
20
30
1997-98 1998-99 1999-00 2000-01 2002-02
Ch
arte
r-H
ost
(%
)%Nonwhite
More nonwhite students
The Evaluation Center Western Michigan University
34
Key FindingsEquity and Access – (Self?) Selection
Slightly more low income students
Relatively stable over time
Considerable school-by-school variation
Charter-Host Differences
-20
-10
0
10
20
30
1997-98 1998-99 1999-00 2000-01 2002-02
Ch
arte
r-H
ost
(%
)%Nonwhite
%FRL
The Evaluation Center Western Michigan University
35
Key FindingsEquity and Access – (Self?) Selection
Fewer students with IEPs
IEP students in charter schools tend to have less severe disabilities
Considerable school-by-school variation
Charter-Host Demographica Differences
-20
-10
0
10
20
30
1997-98 1998-99 1999-00 2000-01 2002-02
Ch
arte
r-H
ost
(%
)%Nonw hite
%FRL
%IEP
The Evaluation Center Western Michigan University
36
Key FindingsEquity and Access – The Bottom Line
On the whole, charter schools have target communities of need
In the aggregate, little systematic evidence of cream-skimming
Possible exception of special education Outlier schools might raise some red flags Negative trend on race Some anecdotal evidence of cream-skimming
The Evaluation Center Western Michigan University
37
Key FindingsEquity and Access – The Bottom Line
But, findings limited by limitations in using enrollment data
Family self-selection vs. school selection
Key Policy Issues
Achievement
The Evaluation Center Western Michigan University
39
Key Policy Issues & OptionsAchievement
What are the correlates of success?
How can we reduce variability on the low end?
What is the potential for success?
Study correlates of success
Facilitate identification and dissemination of best practices
The Evaluation Center Western Michigan University
40
Key Policy Issues & OptionsAccountability as Possible Correlate
Accountability lies at the heart of the charter concept
PA has done better than most states in reporting and technical assistance
Yet, there is still room for improvement
The Evaluation Center Western Michigan University
41
Key Policy Issues & OptionsAccountability as Possible Correlate
Continue improvements to charter school annual report format
Make information public to facilitate market accountability
Continue providing technical assistance in developing clear, measurable goals
The Evaluation Center Western Michigan University
42
Key Policy Issues & OptionsAccountability as Possible Correlate
Consider assessment of LEA capacity for oversight (wills, skills, & bills) Consider the impact of “just-in-time”
oversight by many LEAs Consider targeted state oversight
based on past performance & compliance
Key Policy Issues
Choice and Equity
The Evaluation Center Western Michigan University
44
Key Policy Issues & OptionsChoice & Equity
Is choice good in itself or as tool for increasing achievement?
Is innovation reasonable for charter schools?
What counts as “innovation”?
Possible trade-off between choice and equity
Slippery relationship between satisfaction and achievement
The Evaluation Center Western Michigan University
45
Slippery Relationship Between Achievement & Satisfaction
r = -0.08 (p = 0.64)
Ave
rage a
nnual
gain
(filter
ed PS
SA sc
ores)
Mean satisfaction level (parents, teachers, and students)3 3.5 4 4.5
-100
-50
0
50
100
Key Policy Issues
Cross-Cutting Issues
The Evaluation Center Western Michigan University
47
Key Policy Issues & OptionsCross-Cutting Issues
Balancing achievement, choice, and equity – the challenges of multi-attribute evaluation systems
Cost-effectiveness How do charter schools stack up against other
investments in educational improvement? Actual vs. potential charter school
performance Addressing questions of fact and value
The Evaluation Center Western Michigan University
48
Final Thought “By providing important data and
raising key policy issues, this report has sought to make a sound contribution to the debate over the Commonwealth’s charter schools”