Upload
frederica-quinn
View
214
Download
0
Tags:
Embed Size (px)
Citation preview
Mehmet Dali Öztürk, Ph.D.
Layers of Evaluation & Meaningfulness of Results
3CSN and ELI Collaborate: Data Collection Workshop
September 6, 2013
Least Meaningful
More Meaningful
Most Meaningful
Output Analysis
Outcome Analysis
Impact Assessment
Meaningfulness of ResultsLayers of Evaluation
Layers of Evaluation• Impact assessment focuses on determination of
changes in outcomes that can be attributed to polices, programs, initiatives and projects for schools, individuals, families, or communities. For example, the change in school graduation rates due to a specific program in comparison to schools who did not receive the same program.
• Outcomes focus on intended and expected results of
polices programs, initiatives and projects. For example, the percent increase in schools meeting state standards.
• Outputs are the types and quantities of
engagements and efforts. For example, the number of mentoring hours produced by program personnel.
Program X
Output Outcome ImpactNumber of students in
program
Number of students
completing program
Percent gain in standard test scores in math
Significant changes in percent gain in standard math test scores at the
focus schools where program is implemented
Percent gain in standard test
scores in reading
Significant changes in percent gain in standard test scores in reading at the focus schools where program is implemented
Percent of students
demonstrating a grade increase in math on report
cards
Significant changes in percent of students
demonstrating a grade increase in math on
report cards at the focus schools where program
is implemented
Percent of students
demonstrating a grade increase in
reading on report cards
Significant changes in percent of students
demonstrating a grade increase in reading on
report cards at the focus schools where program
is implemented
1. Are there differences in outcome variables?
2. Are the differences significant?
3. Are the differences meaningful?
4. What do these differences mean with regard to the overall community?
Providing a Context for the Evaluation Process
6
Program Effectiveness/Scale Rating System, which was developed to be a tool to help policy-makers better understand the impact of programs, strategies, and innovations.
Program Effectiveness Scale/Rating System
By placing evaluation results into a format that is easy to understand, the Program Effectiveness Scale can help policy-makers make objective judgments on the worth of a program or strategy.