Upload
aileen-george
View
216
Download
0
Embed Size (px)
Citation preview
Evaluation of NDPC-SD
AKA CIPP Evaluators: Elaine Carlson and Tom Munk
Assist in summative evaluation of the center Helped develop standardized logic model Helped develop summative evaluation plan Helped develop evaluation instruments Will conduct data analysis of summative
evaluation data
Center to Improve Project Performance
NDPC-SD Logic Model
Inputs• Funding• Expertise• Lessons learned • Evidence-based
practices• Research base• Time• Materials and
technology• OSEP priorities• SEA/LEA needs• State readiness
Goals
• 1. Increase the awareness of policymakers, administrators, and practitioners about dropout prevention, reentry, and school completion.
• 2. Increase the number of states that set and meet reasonable and rigorous performance targets for Indicators 1 and 2.
Other Federal activities, OSEP policy environment, Clemson’s experience with the previous NDPC-SD, general education reforms, and economic trends that influence labor markets.
External Factors/Context
Activities Outputs Direct Intermediate Long-term
Outcomes
• A. Provide knowledge development.
Increase the awareness of policymakers,
administrators, and practitioners about school completion.
• Number and quality of comprehensive reviews completed on schedule
• Number of contacts with general TA providers on incorporating school completion models and practices
Decrease the number of students with disabilities who drop out of school.
• Number of training activities developed and delivered
• Number of forums held• Number/type/content/timing of TA activities• Type/number/audience of TA&D activities• Type/number/content of materials produced• Number of APR support materials developed• Amount of TA delivered and outcomes reported• Number of high-quality data systems developed
Increase the number of providers incorporating school completion in the school improvement context.
Improve and increase SEA/LEA use of data-based improvement planning, including developing and improving data systems to track students at risk of dropping out.
Implement and evaluate effective, comprehensive school-completion models, practices, and systems.
Increase the number of students with disabilities who re-enter high school or enter other credentialing or training programs.
• 3. Help SEAs and LEAs develop and improve data systems to track students at risk of dropping out.
• 4. Help SEAs and LEAs implement and evaluate effective, comprehensive school-completion models, practices, and systems for students with disabilities.
• C. Provide technical assistance & dissemination.
Improve and increase SEA/LEA use of SPP/APR indicators to drive school completion activities.
Improve SEA/LEA school-completion policies and procedures.
Increase the number of students with disabilities who graduate.
Increase the number of states that propose
SPP/APR activities for IDEA Part B Indicator1 and Indicator 2 that are aligned with their performance targets for
those indicators.
• Number of SPR, APR, and performance data analyses completed
• Number/attendance of advisory committee meetings
• Number/type/intensity of collaboration activities with OSEP TA & D network and Comprehensive Centers
• Extent of participation in Exiting CoP
• B. Provide leadership and coordination.
1. Increase awareness of policymakers, administrators and practitioners about dropout, school completion and reentry
2. Increase the number of states that set and meet reasonable and rigorous performance targets for Indicators 1 and 2
3. Help SEAs and LEAs develop and improve data systems to track students at risk of dropping out
4. Help SEAs and LEAs implement and evaluate effective, comprehensive school-completion models, practices, and systems for students with disabilities
NDPC-SD Goals
Knowledge Development
Leadership and Coordination
Technical Assistance and
Dissemination
NDPC-SD Activities
Number & quality of comprehensive reviews completed on schedule
Number of contacts with general TA providers on incorporating school completion models and practices
Knowledge Development Outputs
Number of SPR, APR, and performance data analyses completed
Number/attendance of advisory committee meetings
Number/type/intensity of collaboration activities with OSEP TA & D network and Comprehensive Centers
Extent of participation in Exiting CoP
Leadership & Coordination Outputs
Number of training activities developed and delivered
Number of forums held Number/type/content/timing of TA activities Type/number/audience of TA&D activities Type/number/content of materials produced Number of APR support materials developed Amount of TA delivered and outcomes
reported Number of high-quality data systems
developed
TA & Dissemination Outputs
Increase the number of providers incorporating school completion in the school improvement context
Increase the awareness of policymakers, administrators, and practitioners about school completion
Improve and increase SEA/LEA use of data-based improvement planning, including developing and improving data systems to track students at risk of dropping out
Direct Outcomes
Improve SEA/LEA school-completion policies and procedures
Improve and increase SEA/LEA use of SPP/APR indicators to drive school completion activities
Increase the number of states that propose SPP/APR activities for IDEA Part B Indicator1 and Indicator 2 that are aligned with their performance targets for those indicators
Intermediate Outcomes
Decrease the number of students with disabilities who drop out of school
Increase the number of students with disabilities who re-enter high school or enter other credentialing or training programs
Increase the number of students with disabilities who graduate
Long-Term Outcomes
User surveys for Big IDEAs, website, teleseminars
TA/training surveys TA log (who, what, where, when…) Counts of activities for monthly reports School Implementation Survey SEA and LEA Impact Surveys
Data Collection Instruments
Purpose
1. Determine where a school is in the development and implementation of its school-completion initiative. The first administration will establish a baseline. Subsequent administrations should show the school progressing through the stages of implementation.
2. Assesses school practices supporting school completion
3. Determine how much knowledge of the school-completion initiative individual respondents have
School Implementation Survey
Completed by individuals in schools implementing a school-completion initiative based on the NDPC-SD framework
Collects school-level information about: Training School team Planning process School data collection and analysis Selection of evidence-based practices Professional development Evaluation of the initiative Implementation and progress of interventions
School Implementation Survey
Survey was first administered in Georgia & Maryland in fall 2009 to establish baselines
Will be administered in all new training sites in the future
Will be re-administered annually
School Implementation Survey Data Collection
1. School performance data2. Demographic data3. Information about policies &
procedures4. Academic engagement data 5. Information about instructional
systems6. Behavioral engagement data7. Affective engagement data8. Information about professional
development
School Implementation Survey Data: The Most Commonly Used Data in Planning
School Implementation Survey Data: School Has Identified Suitable EBPs
No Don't know In progress Yes Not applicable No response0
25
50
75
100
125
150
175
200
225
250
275
300
17
46
187
134
07
Num
ber
of
Responses
(N = 391 respondents)
School Implementation Survey Data: School Has Determined Level of Implementation
(N = 391 respondents)
No Don't know In progress Yes Not applicable No response0
25
50
75
100
125
150
175
200
225
250
275
300
21
46
202
116
0 6
Nu
mb
er
of
Resp
on
ses
School Implementation Survey Data: School Has Set Timelines for Implementation
(N = 391 respondents)
No Don't know In progress Yes Not applicable No response0
25
50
75
100
125
150
175
200
225
250
275
300
30
54
197
102
08
Nu
mb
er
of
Resp
on
ses
School Implementation Survey Data: Implementation: Interventions in Use
0
50
100
150
200
250
300
350 340320
233247 255 257
225 220
147
202
268
167
253
(N = 391 respondents)
Num
ber
of
responses
Should we split our sample into 2 groups: School Team Leader
Everyone else who is surveyed in the school
Does the School Implementation Survey appear to be measuring what we need?
How could we improve its effectiveness?
Questions for the Committee:School Implementation Survey
Under development! We need to assess the impact of NDPC-SD’s technical assistance on SEA and LEA practices and policies around school completion
State and Local Impact Survey(s)
One survey or two?Who should complete it/them?Draft survey – what to add, delete or revise?
Questions for the Committee: SEA and LEA Impact Survey(s)