View
19
Download
1
Category
Tags:
Preview:
DESCRIPTION
Assessing Response to Intervention. A collaborative project between the Florida Department of Education and the University of South Florida. FloridaRtI.usf.edu. Advance Organizer. Team Activity - Review Day Four Intervention Plan Why Monitor Progress? Graphing Conventions Goal Setting - PowerPoint PPT Presentation
Citation preview
FloridaRtI.usf.edu
A collaborative project between the Florida Department of Education and the University of South Florida
Assessing Response to Intervention
Advance Organizer
Team Activity - Review Day Four Intervention Plan
Why Monitor Progress?
Graphing Conventions
Goal Setting
Interpreting Graphs
Decision Making
Review of Problem-Solving Steps
Team Activity-Review Day 4 Intervention Plan
1. How did the intervention plan your team wrote on Day 4 differ from intervention plans typically written at your school? How was the plan similar?
2. For which components of the intervention plan did your team provide the most descriptive and specific details?
3. For which components of the intervention plan could your team have provided additional detail to clarify what needed to happen for implementation to occur?
4. How has the writing of intervention plans changed since your team attended the Day 4 training? How have the plans remained the same?
Beliefs Survey
Your project ID is:
• Last 4 digits of SS#
• Last 2 digits of year of birth
Response to Interventionin Context
Identify
the Problem
Monitor
ProgressAnalyze
the ProblemImplement
Intervention
Evaluate
Intervention
Effectiveness
Timeline
Select/DesignIntervention
• Unless we monitor progress, we cannot determine the rate at which the gap is closing.
• Continuous feedback improves instructional planning (formative assessment)
• Allows earlier decisions about what to do-Increase time, decrease time, revisit problem-solving, etc.
• Provides measure of intervention effects-Getting better, staying the same, or getting worse
• We don’t know the effectiveness of an intervention until we implement it and monitor progress
Why Monitor Progress?
Randy Allison, 2004
Why Monitor Progress?• Provides clear idea of expectations of performance over
time.
• Student outcomes improve when performance is assessed regularly
• Allows visual comparison to a standard
• Data collection provides an objective data base for decision-making
• Know if learning is being enabled, and if so, under what conditions
• Continuous feedback on performance enhances motivation for many
Randy Allison, 2004
Outcome of Monitoring: Diagnose Conditions that Enable Learning
• Identify conditions under which student desired learning and behavior is accelerated and you have determined what enables learning.
• If conditions are configured like X, Y, or Z, then…what benefits result for the student?
Hi
LoTime
X Y Z
... . .. . . .... .......
. .......
Randy Allison, 2004
Why Not Just UsePre - Post Test Comparisons?
• We must measure and demonstrate more than improvement. We must show the rate necessary to attain benchmarks within a time frame.
• May be unreliable because of small amount of data collected
• May be significant time lag between pre-test and post-test
• Not sensitive to small changes in direction of performance in a timely manner
• More difficult to analyze patterns of performanceRandy Allison, 2004
Basics of CBM - Scientifically Based Progress Monitoring
• Research Efforts Led By Stanley Deno, Beginning in 1971 with Federal Funding in 1978 to Provide Viable Progress Monitoring toward IEP Goals
• Almost 30 Years of Continuous Research
• Mid 1990s Witnessed Move to Standard, High Quality Assessment Materials
• A Number of Members of the CBM “Family” including DIBELS
M. Shinn
Why CBM…or any Other Measure?
• Quick and inexpensive
• Linked to instruction and curriculum
• Frequently repeatable
• Sensitive to small increments of growth
• Reliable and valid
• Can be used for multiple purposes and to answer different questions
Jenkins, Deno, and Mirkin
Academic Measures -CBM-
Reading Fluency Words read correctly
Reading Comprehension
Correct words inserted when the seventh word is left out
Spelling Correct words spelled and correct letter sequences
Written Expression
Words written or correct word sequences written
Math Computation
Correct digits written from math computation problems
Math Applications
Correct digits written from math application problems
From M.Shinn
Behavioral Measures
• Office discipline reports
• Behavioral incidents
• Suspension/Detention records
• Observations
• Self-assessments
• Surveys
• Attendance data
• Teacher checklists
• Screening instruments
• Rating scales
Sam
Digits correct per minute
7
6
5
4
3
2
1
12
11
10
9
8
12
1 2 3 4 5 6 7 8Days
Why use graphs?
• Teachers are able to make sound decisions about the instruction being delivered to students based upon data, not guesswork
• Parents are kept well informed about their child’s progress with specific information about how their child is responding to instruction. Parents may assist in making suggestions for instructional adjustments.
• Students know what is expected of them. They receive specific feedback about their performance along the way rather than only at the end of the marking period. Goal setting and progress monitoring are some of the most effective strategies to improve academic engaged time.
General Outcomes /Specific Skills
• The general outcome expectations for students are comprised of many requisite subskills. Both general outcomes and specific skills may be measured.
• What subskills would be necessary for:
Long division?
Fluent reading?
Compliance with teacher direction?
Graph Components
Skill
equalincrements
Time - equal increments
Instructional Change Line
Goal
Intervention(Group or Individual)
Baseline
Aim Line
Trend Line
Skill
Time2 wks 2 wks
Keep increments consistent!
Skill
Time2 wks 2 wks
Keep increments consistent!
In both conditions, the rate of skill acquisition is 1/week.
Dissimilar x-axis increments give the impression that learning rate is increased during second time interval
Skill
Time2 wks
Keep increments consistent!
Similar x-axis increments reveal true progress over time
2 wks
Goal Setting
• Set goals which are ambitious, but reasonable -- rate required to reach goal is 25-50% above typical student rate
• Goal has two components Level of performance desired
Time within which that level will be attained
Goal Setting
• Measure difference between desired and current performance
• Divide by number of weeks
• Compare to standard to determine reasonable & ambitious growth rate (an increase of 25-50% of typical rate)
Goal SettingDesired - Current
Number of Weeks
110 wcm - 60 wcm
20 weeks
50 wcm
20 weeks
2.5 wcm/week2.0 wcm/week is typical
Compare to rate for typical peers: in 25-50% range?
Goal Setting
To determine an increase of 25-50%-
For lower end of range - multiply typical rate X 1.25For upper end of range - multiply typical rate X 1.50
Examples:
If typical rate = 2.00 words correct per minute/week Ambitious range would be 2.5 - 3.0 words correct per minute/week
If typical rate = 3.00 digits correct per minute/weekAmbitious range would be 3.75 - 4.50 digits correct per minute/week
Goal Setting
If:
Goal is 3.5 words correct per minute/week &2.0 words correct per minute/week is typical
Is this in ambitious range?
Goal is .75 digits correct per minute/week &.5 digits correct per minute/week is typical
Is this in ambitious range?
2.0 letter sounds correct per minute/week is typical
What is ambitious range?
ExampleGrowth Rates
• Realistic growth rates (words/ week)
• Grade 1 2.00 words
• Grade 2 1.50 words
• Grade 3 1.00 words
• Grade 4 .90 words
• Grade 5 .50 words
• Grade 6 .30 words
• Ambitious growth rates (words/week)
• Grade 1 3.00 words
• Grade 2 2.00 words
• Grade 3 1.50 words
• Grade 4 1.10 words
• Grade 5 .80 words
• Grade 6 .65 words
Fuchs, Fuchs, 1993
Attend to:
• Level • Slope / Rate
• Variability
Decision Rules: What is a “Good” Response to Intervention?
• Positive Response
Gap is closing
Can extrapolate point at which target student(s) will “come in range” of target--even if this is long range
• Questionable Response
Rate at which gap is widening slows considerably, but gap is still widening
Gap stops widening but closure does not occur
• Poor Response
Gap continues to widen with no change in rate.
Performance
Fall
Positive Response to Intervention
Expected Performance
Observed Performance
Winter Spring
Performance
Time
Positive Response to Intervention
Expected Trajectory
Observed Trajectory
Decision Rules: What is a “Good” Response to Intervention?
• Positive Response
Gap is closing
Can extrapolate point at which target student(s) will “come in range” of target--even if this is long range
• Questionable Response
Rate at which gap is widening slows considerably, but gap is still widening
Gap stops widening but closure does not occur
• Poor Response
Gap continues to widen with no change in rate.
Performance
Fall
Questionable Response to Intervention
Expected Performance
Observed Performance
Winter Spring
Performance
Time
Questionable Response to Intervention
Expected Trajectory
Observed Trajectory
Decision Rules: What is a “Good” Response to Intervention?
• Positive Response
Gap is closing
Can extrapolate point at which target student(s) will “come in range” of target--even if this is long range
• Questionable Response
Rate at which gap is widening slows considerably, but gap is still widening
Gap stops widening but closure does not occur
• Poor Response
Gap continues to widen with no change in rate.
Performance
Fall
Poor Response to Intervention
Expected Performance
Observed Performance
Winter Spring
Performance
Time
Poor Response to Intervention
Expected Trajectory
Observed Trajectory
Performance
Time
Response to Intervention
Expected Trajectory
Observed Trajectory
Positive
Questionable
Poor
DecisionsWhat to do if RtI is:
• Positive
• Continue intervention with current goal
• Continue intervention with goal increased
• Fade intervention to determine if student(s) have acquired functional independence.
DecisionsWhat to do if RtI is:
• Questionable Was intervention implemented as intended?
• If no - employ strategies to increase implementation integrity
• If yes -
Increase intensity of current intervention for a short period of time and assess impact. If rate improves, continue. If rate does not improve, return to problem solving.
DecisionsWhat to do if RtI is:
• Poor Was intervention implemented as intended?
• If no - employ strategies in increase implementation integrity
• If yes -
Is intervention aligned with the verified hypothesis? (Intervention Design)
Are there other hypotheses to consider? (Problem Analysis)
Was the problem identified correctly? (Problem Identification)
Intervention Integrity Decisions
Evidence based intervention linked to verified hypothesis planned
Evidence based intervention implemented
Student Outcomes (SO)
Assessed
Treatment Integrity (TI) Assessed
Data-based Decisions
Continue Intervention
Implement strategies to promote treatment integrity
Modify/change Intervention
+SO +TI
-SO +TI
-SO -TI
From Lisa Hagermoser Sanetti, 2008 NASP Convention
Response to Interventionin Context
Identify
the Problem
Monitor
ProgressAnalyze
the ProblemImplement
Intervention
Evaluate
Intervention
Effectiveness
Timeline
Select/DesignIntervention
Progress Monitoring Resources
• Interventioncentral.org
• Studentprogress.org
• Dibels.uoregon.edu
www.interventioncentral.com
www.studentprogress.org
dibels.uoregon.edu
School Level Data Review Worksheet
Your project ID is:
• Last 4 digits of SS#
• Last 2 digits of year of birth
• Read the case study
• Answer six questions using data provided
Review
Consensus
Infrastructure
Implementation
Review
The steps of problem solving as an integrated, fluid, self-
correcting process.
Step 1 - What’s the Problem?
In order to identify a problem, you’ve got to start with three pieces of data-
• Benchmark level of performance
• Student level of performance
• Peer level of performance
Is this an individual student problem or a larger systemic problem?
Decision Making Rubricfor use with
School-Wide Screening
adapted from:
Heartland AEA 11, Improving Children’s Educational Results
Are over 20% of students
struggling?
Are between 5% and 20% of Are 5% or fewer
and developgroup
intervention
Examine instruction,
curriculum, and environment for
needed adaptations
Develop small group
intervention
Go to problem definition
Go to intervention evaluation
studentsstruggling?
studentsstruggling?
Step 1 - What’s the Problem?
0
10
20
30
40
50
60
70
80
90
100
Sept Oct Nov Dec Jan Feb
% Compliance
35%
Benchmark75%
= Peer Group = Aim Line
BASELINE
The problem is occurring because ________________.
If ____________ would occur, the problem would be reduced.
Goal: The development of hypotheses about probable causes for the identified problem.
Assessments are then conducted to gather information to determine which are most / least likely
Prediction statement:
Step 2- Why is it occurring?
Assessment:How Do We Confirm Hypothesis?
DOMAINSR
ReviewI
InterviewO
ObserveT
Test
IInstruction
CCurriculum
EEnvironment
LLearner
Step 3- What are we going to do about it?
• Effective teaching strategies consider both what to teach and how to teach it.
• Making good decisions will increase student progress.
• It is critical that the instruction be matched to the problem.
Howell & Nolet, 2000
Step 4- Is it working?
Goal
ClassroomIntervention I
Making instructional decisions based on the review and analysis of student data
Progress monitoring always includes graphing
Progress Monitoring
ClassroomIntervention 2
Traditionalvs.
Response to Intervention
Intervention
Intervention
Consider ESETraditional
Intervention
Intervention
Intervention Consider ESE
Ifnecessary
Response to Intervention
RegularEducation
MonitorProgress
MonitorProgress
What is the Problem?Why is itoccurring?Is it working?What are we going to do about it?
Tiers of Service Delivery
I
II
III
Problem Identification
Problem Analysis
Intervention Design
Responseto
Intervention
What is RtI?
RTI is the practice of (1) providing high-quality instruction/intervention matched to student needs and (2) using learning rate over time and level of performance to (3) make important educational decisions to guide instruction.
National Association of State Directors of Special Education, 2005
Perceptions of Skills Survey
Your project ID is:
• Last 4 digits of SS#
• Last 2 digits of year of birth
Final Thoughts
Problem Solving &
Response to Intervention
Training Evaluation
No project ID needed
Thanks!
Resources
http://www.oswego.edu/~mcdougal/web_site_4_11_2005/index.html (Graphing made easy)
http://www.jimwrightonline.com/php/chartdog_2_0/chartdog.php (Chart Dog : on-line graphing tool)
http://www.studentprogress.org/
http://www.progressmonitoring.org/
Resources
• Armstrong, J., & Anthes, K. (2001). How data can help: Putting information to work to raise student achievement. American School Board Journal, 188, 38-41
• Coburn, C., & Talbert, J. (2006). Conceptions of evidence use in school districts: Mapping the terrain. American Journal of Education, 112, 469-495.
• Kerr, K., Marsh, J., Ikemoto, G., Darilek, H., & Barney, H. (2006). Strategies to promote data use for instructional improvement: Actions, outcomes, and lessons from three urban districts. American Journal of Education, 112, 496-520.
• Young, V. (2006). Teachers’ use of data: Loose coupling, agenda setting, and team norms. American Journal of Education, 112, 521-548.
Recommended