Upload
janel-hopkins
View
220
Download
6
Embed Size (px)
Citation preview
Progress Monitoring and RtI: Questions from the Field
Edward S. ShapiroDirector,
Center for Promoting Research to PracticeLehigh University, Bethlehem, PA
Progress Monitoring Role and RtI
Primary method of evaluating student outcomes of instructional intervention
Typical methods use General Outcomes Measurement
Most common is CBM Establish expected rates of gain Establish expected levels of performance Outcomes used to establish response to
intervention
Questions From the Field What should be the expected level of
improvement to set goals? Are these measures linked to state
assessments? What do I use as a valid measure of reading
beyond 5th grade instructional level (and for those situations where ORF does not reflect reading performance)?
What do I use for PM in math? What do I use for PM at middle and
secondary levels?
PA – statewide effort to establish expected levels of performance for students with LD
3 year effort Extensive training lead by PaTTAN
and IU Statewide data collection and
reporting Outcomes establish average rates of
improvement across students with LD
Outcomes of PM in PA2004 -2005
1,963 special education students monitored
149 districts, 24 IU’s LD = 1,502 (76.5%); SED = 88
(4.5%); MR = 223 (11.4%) 1,654 (84.3%) = reading; 599 =
math computation (30.5%) ; 60 = math concepts (3.1%)
FOR LD: 1,277 Reading; 447 Math computation; 46 = math concepts
Comparison of PA PM to Fuchs Study in Reading-Instructional Level
Grade Fuchs Study
Reg Ed
All SpED Students
2004 –2005
LD2004 2005
All SpED Students2003 –2004
LD2003 – 2004
1 2.0 1.07 1.15 1.02 .85
2 1.5 1.18 1.18 1.40 1.10
3 1.0 1.08 1.09 .88 1.06
4 0.85 1.19 1.23 1.08 1.19
5 0.5 1.07 1.07 1.06 .91
6 0.3 1.17 1.27 1.07 1.22
Comparison of PA PM to Fuchs Study in Compute-Instructional Level
Grade Fuchs StudyReg Ed
PDE Benchmarks All SpED Students
2004 -2005
LD
2004 2005
1 n/a .3 .29 .27
2 0.3 .3 .31 .32
3 0.3 .3 .35 .33
4 0.70 .45 .26 .27
5 0.70 .45 .19 .20
6 0.45 .19 .22
PM Conclusions from PA
Validated selection of expected rates of gain in reading
More data needed in math Statewide impact of training in PM
How are these measures related to outcomes of state high stakes assessments? State assessments are linked to curriculum standards GOM measures, especially in reading, show strong
predictions to state assessments Across states, correlations between .6 and .8 (over 15
states) Sensitivity/Specificity indices show very good true
positive, true negative; values for false positive/false negative, not quite as good
Measures meet expected standards for screening but may not be as desirable for high stakes decisions
We may need to do better than screening level measures for these decisions
How are these measures related to outcomes of state high stakes assessments?
Research Need Examine degree to which additional
measures beyond CBM contribute to improving prediction to state assessment
Recent study by Compton et al. (Journal of Ed Psych)
Balance additional measurement with unnecessary, over assessment
What do I use to measure reading beyond the 5th grade instructional level?
ORF known to lose sensitivity to instructional change beyond 5th grade instructional level
Skills in reading at this level require assessment of comprehension and critical thinking
Current GOM in reading comprehension (Maze), not been carefully evaluated for use at this level
What do I use to measure reading beyond the 5th grade instructional level?
Research Needs Measurement tool needed that shows
sensitivity to reading instruction at this level
Measurement tool needs to have credibility with reading personnel
Measurement tool needs to be efficient Late developing reading problems?
Progress Monitoring in Math: Can math be assessed using a GOM? Early grades, multiple skill computational assessment
appears to be adequate Beyond early grades, computational assessment
relationships to overall outcomes in math performance
Credibility problems – linkage to math curriculum is questioned, especially those using curricular such as Everyday Math
Use of Concepts/Applications progress monitoring partially addresses problem, but relationship to state assessment not as strong as reading (correlations in .5- .6 range)
Progress Monitoring in Math: Can math be assessed using a GOM?
Does math require a specific subskill mastery method of assessment? No validated metrics for progress
monitoring at this time How will we establish response-to-
intervention for math difficulties without GOM?
What will we use to establish expected levels of performance and rates of gain?
What do we use for PM at middle and secondary school levels?
PM research efforts focused on elementary level and basic skill development
What validated measures do we have for middle and secondary school students experiencing academic skills problems?
Some initial work (Chris Espin & colleagues, Gerry Tindal & colleagues, Anne Foegen & colleagues, in math) in areas such as written language, algebra, but much more development needed
Positives about PM
Feasible to be used at elementary level
Statewide implementation and training is feasible
Strong GOM in reading (ORF) up through 5th grade
Questions about PM Credibility of GOM in reading beyond 5th
grade instructional level PM in math remains significant question –
GOM or SS PM as SS – validated measures? PM at middle and secondary level remains a
question Need to recognize the high stakes nature of
the questions we are asked to answer using PM data