FloridaRtI.usf.edu
A collaborative project between the Florida Department of Education and the University of South Florida
Intervention Design
Advance Organizer
Update: Consensus, Infrastructure, Implementation
Data Review: Problem ID, Problem Analysis
Linking Problem Analysis to Intervention
Intervention Design
Intervention Content
Intervention Plan
Intervention Integrity, Support, Documentation
Integrating Tiers of Intervention
Intervention Selection/Design & Implementation in Context
Identify
the Problem
Monitor
Progress
Analyze
the ProblemImplement
Intervention
Evaluate
Intervention
Effectiveness
Timeline
Select/DesignIntervention
Review
Consensus
Infrastructure
Implementation
Discussion - Consensus
• What activities are occurring to facilitate consensus building among school and district staff?
• What activities need to occur to better facilitate consensus building among school and district staff?
• What consensus building activities should be given top priority between now and the end of the school year?
Discussion - Infrastructure
• What activities are occurring to facilitate the development of the infrastructure needed to implement a PS/RtI model?
• What activities need to occur to better facilitate the development of the infrastructure needed to implement a PS/RtI model?
• What infrastructure development activities should be given top priority between now and the end of the school year?
Discussion - Implementation
• Given answers to the questions addressing consensus & infrastructure, what are implications for facilitating implementation of PS/RtI model in your building?
• What components of model can be targeted for implementation between now and end of school year given the levels of consensus and infrastructure?
• What components of model would need to be targeted for implementation later to allow more time to build support from staff and capacity (e.g., assessments, technology, training, policies, procedures) to implement?
Review Previous Skill Assessments
• Problem ID Gap between desired performance and current
performance
• Problem Analysis Most likely barriers to desired performance
• Using what you now understand of Problem ID and Problem Analysis, consider your Tier One data, identify a problem and analyze it. Later, you’ll link your Problem Analysis to the development of a comprehensive intervention plan.
ProblemAnalysis
InterventionDesign
The problem is occurring because ______________.
If ___________ would occur, then the problem would be reduced.
Intervention Development
• Interventions derived from effective problem-solving strategies
• Research/knowledge used to develop hypotheses.
• Verified hypotheses lead naturally to interventions
• Interventions will be evidence-based if hypotheses are evidence-based
Types of Interventions
• Skill Deficit Student lacks skills to successfully complete
task
• Performance Deficit Factors interfering with student’s capability of
performing the skill
Intervention Development:Verified Hypotheses
• H: Only 55% of second grade students are achieving reading fluency benchmark because the curriculum lacks fluency focus.
• P: If we modify the curriculum to focus on reading fluency, then more students will achieve benchmark.
• Intervention: Modification of second grade curriculum to include more reading fluency focus
Intervention Development:Verified Hypotheses
• H: Brandon is unable to stay in his seat because he lacks the self-monitoring skills necessary for self-control
• P: If we teach Brandon self-monitoring skills, then he will display improved self-control.
• Intervention: Self-instructional and self-monitoring training
Intervention Development:Verified Hypotheses
• H: Ernesto is unable to complete arithmetic seatwork on time because he has not acquired math facts at the automatic level
• P: If we provide Ernesto with strategies to increase fact fluency, then he will complete work on time.
• Intervention: Provide Ernesto with strategies to increase fact fluency Provide Ernesto more time until strategies to increase
fact fluency have had time to work
Intervention Development:Verified Hypotheses
• H: Gail is not attending school daily because she believes that all of the students she goes to class with exclude her from social activities
• P: If we help Gail to restructure her beliefs about being excluded, then attendance will improve.
• Intervention: Cognitive restructuring regarding belief system Identify students who would be willing to include Gail
in their activities Ensure adult prompting and monitoring of social
activities
Intervention Development:Verified Hypotheses
• H: Susanna is unable to comprehend at 5th grade level because she is disfluent in 5th grade reading material.
• P: If we improve her fluency, then comprehension will improve.
• Interventions: Limit comprehension expectations to current fluency level
Develop hypotheses to explain low rate of fluency
Tiers of InterventionI & II
Tier I Intervention• Tier One- Examining “Universal” Interventions
• Questions:
What percent of students are achieving district benchmarks? Effectiveness of instruction
How are these students doing compared to grade level benchmarks? GAP analysis
• Hypotheses
Ho: These students have not had access to an effective learning environment.
Ho: These students have not been engaged in an effective learning environment.
Tier 1 Intervention• Consider altering whole group instruction, curriculum
materials, instructional routine, independent practice (e.g., literacy/math centers)
• Breadth of skill focus might vary
• Group students based on skill data (data come from many sources)
• Differentiate instruction based on grouping
• Organize students based on skill performance Higher performing, more students, Lower performing, fewer students
• Same amount of time, different use of that time
Tier 2 Intervention
• Tier Two- Examining “Supplemental” Interventions
• Hypotheses:
Ho: Student requires additional time for direct instruction
Ho: Focus of the curriculum must narrow
• Assessment:
DIBELS, CBM, district assessments
• Interventions:
Increase AET (90-120-180)
e.g., K-3 Academic Support Plan
Narrow focus to fewer, barrier skills
District Supplemental Curriculum
Characteristics of Tier 2 Interventions
• Available in general education settings
• Opportunity to increase exposure (academic engaged time) to curriculum
• Opportunity to narrow focus of the curriculum
• Sufficient time for interventions to have an effect (10-30 weeks)
• Often are “standardized” supplemental curriculum protocols – identified through a problem solving process
Tier 2 Intervention
• First resource is TIME (AET) HOW much more time is needed?
• Second resource is CURRICULUM WHAT does the student need?
• Third resource is PERSONNEL WHO & WHERE will it be provided?
Tier 2: Getting TIME
• “Free” time--does not require additional personnel Staggering instruction
Differentiating instruction
Cross grade instruction
Skill-based instruction
• Standard Protocol Grouping
• Reduced range of “standard” curriculum
• After-School
• Home-Based
Tier 2: Curriculum
• Standard protocol approach
• Focus on essential skills
• Most likely, more EXPOSURE and more FOCUS of core instruction
• Linked directly to core instruction materials and benchmarks
• Criterion for effectiveness is ≈70% of students receiving Tier 2 will reach benchmarks
Tier 2: Personnel
• EVERYONE in the building is a potential resource
• Re-conceptualize who does what
• Personnel deployed AFTER needs are identified
• WHERE matters less and less
• REMEMBER, student performance matters more than labels, locations and staff needs.
• A school cannot deliver intensive (Tier 3) services to more than ≈7% of the population
Outline – Implementing An RtI System
• Tier 2 Decision Making –
Identify less than proficient students
Administer additional brief assessments to examine performance profiles
Group students with like performance profiles for supplemental instruction
Provide supplemental instruction based on skill needs
Outline – Implementing An RtI System (continued)
• Tier 2 Decision Making – Monitor progress
Review student progress monitoring data at scheduled intervals
How successful are students in response to Tier 2 Interventions?
• ≈70 - 80% is a good criterion
Modify supplemental instruction as necessary
Move students across tiers as data warrant
Job-Alike
• What is your role with respect to the PS/RtI model at your school?
• What is working well with PS/RtI?
• What is not working well?
• What is the best part when it comes to implementing PS/RtI at your school?
Intervention Content
Why Intervene?
Problem Definition & Problem Analysis have revealed information the problem solving team will use to determine what and how the students need to be taught.
The purpose of Intervention is to create an
instructional match
Instructional Match =
Explicit Evidence-Based Instructionin
Targeted Skillswithin a
Supported Learning Environment
Criteria for Interventions
• Evidence-based
• Delivered with Integrity
• Scaled for Intensity
• Implemented for Sufficient Time
• Evaluated Frequently
• Integrated Across Tiers
Evidence-Based
SCIENTIFICALLY BASED RESEARCH -
• means research that involves the application of rigorous, systematic, and objective procedures to obtain reliable and valid knowledge relevant to education activities and programs
• Section 9101(37) of ESEA, as amended by the No Child Left Behind Act (NCLB) of 2001
Evidence-Based
• Employs systematic, empirical methods that draw on observation or experiment
• Involves rigorous data analyses that are adequate to test the stated hypotheses and justify the general conclusions
• Relies on measurements or observational methods that provide reliable and valid data across evaluators and observers, across multiple measurements and observations.
• Is evaluated using experimental or quasi-experimental designs… with appropriate controls… with a preference for random-assignment experiments
• Section 9101(37) of ESEA, as amended by the No Child Left Behind Act (NCLB) of 2001
Evidence-Based
• Ensures that experimental studies are presented in sufficient detail and clarity to allow for replication or, at a minimum, offer the opportunity to build systematically on their findings
• Has been accepted by a peer reviewed journal or approved by a panel of independent experts through a rigorous scientific review
• Section 9101(37) of ESEA, as amended by the No Child Left Behind Act (NCLB) of 2001
Practitioner journals or education magazines are not the
same as peer-reviewed academic journals
More
More
Less
Less
Measurement Frequency
Measurement Precision
Depth of Problem Analysis
Instructional Time
Measurement Focus
Evidence Base
Group Size
Principles of Intervention Design
Intervention is…
• Planful- procedures to be applied are specified clearly and completely
• Environmentally Focused- actions taken modify the environment not the individual
• Goal Directed- the team writes an ambitious, yet attainable goal statement prior to intervention design
Principles of Intervention Design
Intervention should be designed to:
• Adjust what is being taught
and / or
• how it is taught
Intervention Plan
Components of an Effective Intervention Plan
Address:
• Persons Responsible
• Skills Targeted
• Implementation Arrangements
• Measurement Strategy
• Decision Making Rule
In essence - who, what, when, where
Designing anIntervention Plan
Person(s) Responsible:These should include names of those implementing the intervention, supporting the intervention, collecting the data and making decisions about the effectiveness of the intervention.
Targeted skills & Instructional strategies:
Specific information about
What & How to teach here
Implementation Arrangements:
Determine: Where, When-(frequency, length of time), Materials
Measurement Strategy:
Measurement conditions (how, what, where, monitoring schedule)
Decision-making Plan
Determine frequency of data collection, number of data points/decision rule - More on this during Day 5
Persons Responsible
Specifically named individuals that will be responsible for:
implementing the intervention,
supporting the intervention,
assessing the integrity of the intervention,
and monitoring the effectiveness of the intervention
Target Skills & Instructional Strategies
Explicitly define the skills to be taught and strategies that will be utilized.
Select evidence based instructional strategies that are based on the defined problem and verified hypotheses in order to maximize likelihood of success.
Implementation Arrangements
• Where
• When
• Frequency
• Length of time
• Materials
Intervention Plan
• Must include:Who is responsible?What will be done?When will it occur?Where will it occur?
Design an Intervention Plan:Measurement Strategy
Record information about:
• Who is responsible for the on-going collection of data
• The method of data collection-(ex.:probes/graphs, frequency counts/graphs)
• The measurement conditions- (ex.: environmental factors are consistent)
• The monitoring schedule-(ex.: establishing consistent measurement intervals)
Design an Intervention Plan:Decision-making Plan
How do we decide if a plan is or isn’t working?
Decisions will be made based on the following:
• Level of skill
• Rate of progress
• Decision rule
Decision Rules: What is a “Good” Response to Intervention?
• Positive Response
Gap is closing
Can extrapolate point at which target student will “come in range” of peers--even if this is long range
• Questionable Response
Rate at which gap is widening slows considerably, but gap is still widening
Gap stops widening but closure does not occur
• Poor Response
Gap continues to widen with no change in rate.
Performance
Time
Positive Response to Intervention
Expected Trajectory
Observed Trajectory
Decision Rules: What is a “Good” Response to Intervention?
• Positive Response
Gap is closing
Can extrapolate point at which target student will “come in range” of peers--even if this is long range
• Questionable Response
Rate at which gap is widening slows considerably, but gap is still widening
Gap stops widening but closure does not occur
• Poor Response
Gap continues to widen with no change in rate.
Performance
Time
Questionable Response to Intervention
Expected Trajectory
Observed Trajectory
Decision Rules: What is a “Good” Response to Intervention?
• Positive Response
Gap is closing
Can extrapolate point at which target student will “come in range” of peers--even if this is long range
• Questionable Response
Rate at which gap is widening slows considerably, but gap is still widening
Gap stops widening but closure does not occur
• Poor Response
Gap continues to widen with no change in rate.
Performance
Time
Poor Response to Intervention
Expected Trajectory
Observed Trajectory
Performance
Time
Response to Intervention
Expected Trajectory
Observed Trajectory
Monitoring Plan for Determining Student Progress
• Must include:Who is responsible?What data will be collected and
how often?How will we decide if the plan
effective?
Decision Rules: Linking RtI to Intervention Decisions
• Positive, Questionable, Poor Response• Intervention Decision Based on RtI (General Guidelines)
Positive• Continue intervention until student reaches benchmark
(at least).• Fade intervention to determine if student has acquired
functional independence. Questionable
• Increase intensity of current intervention for a short period of time and assess impact. If rate improves, continue. If rate does not improve, return to problem solving.
Poor• Return to problem solving for new intervention
Intervention Selection/Design & Implementation in Context
Identify
the Problem
Monitor
Progress
Analyze
the ProblemImplement
Intervention
Evaluate
Intervention
Effectiveness
Timeline
Select/DesignIntervention
Intervention Integrity, Support, &
Documentation
Problem-Solving and Treatment Fidelity
• Response to Intervention is based on the actuality of interventions (Tier 1,2,3) delivered as intended
• We CANNOT do RtI if the intervention was not implemented or was implemented poorly
• Therefore, intervention integrity must be ensured and documented if we are to use RtI
• It will become part and parcel of the procedural safeguard process for students with disabilities.
Problem-Solving and Treatment Fidelity
• Strategies that improve fidelity Follow-up by a consultant/support staff
Presentation of student data illustrating response to intervention
Review of intervention implementation plans
Frequency — range from daily to weekly initially(Noell, Witt, Slider, Connell, Gatti, Williams, Keonig, Resetar, &
Duhon, in press)
Problem-Solving and Treatment Fidelity
• Teacher responsiveness to implementing interventions
Understands the “need” for intervention
Perceives self as possessing skills to implement OR has the social support to implement while acquiring skills
(Bev Showers and colleagues)
• Based on the research, the following intervention protocol should be considered: Ensure that teacher/parent understands need
Evaluate skill of parent/teacher and determine level of support
Delineate intervention in stepwise fashion
Create implementation schedule for intervention
• Time of day, times per day, etc.
Create intervention support/fidelity schedule
Problem-Solving and Treatment Fidelity
• IF YOU CANNOT CREATE A SUPPORT SCHEDULE DO NOT DO INTERVENTION UNTIL ONE CAN BE ESTABLISHED
• It is critical that building administrators understand that the support schedule is as or more important than the intervention schedule
Problem-Solving and Treatment Fidelity
Intervention Integrity Decisions
Evidence based intervention linked to verified hypothesis planned
Evidence based intervention implemented
Student Outcomes (SO)
Assessed
Treatment Integrity (TI) Assessed
Data-based Decisions
Continue Intervention
Implement strategies to promote treatment integrity
Modify/change Intervention
+SO +TI
-SO +TI
-SO -TI
From Lisa Hagermoser Sanetti, 2008 NASP Convention
Intervention Documentation Worksheet
Documentation Plan
• Must include:Who is responsible?What will be done?When will it occur?How will data be shared?
Support Plan
• Must include:Who is responsible?What will be done?When will it occur?Where will it occur?
Intervention Support
• Intervention plans should be developed based on student need and skills of staff
• All intervention plans should have intervention support
• Principals should ensure that intervention plans have intervention support
• Teachers should not be expected to implement plans for which there is no support
Critical Components of Intervention Support
• Support for Intervention Integrity
• Documentation of Intervention Implementation
Intervention and eligibility decisions and outcomes cannot be supported in an RtI model without these two critical components
Intervention Support Plan
• Before Intervention Implementation Review data Review steps to intervention Determine logistics
• First 2 weeks 2-3 meetings/week Review data Review steps to intervention Revise, if necessary
Intervention Support
• Second Two Weeks Meet twice each week
• Following weeks Meet at least weekly Review data Review steps Discuss Revisions
• Approaching benchmark Review data Schedule for intervention fading Review data
Teacher Support Activities
• Holding teachers in high regard; honoring them and considering them worthy of esteem
• Providing teachers with access to tools that work
• Advocating for effective professional development activities
• Actively seeking, and expecting input in: curriculum, instruction, and assessment professional development planning & decision making developing and implementing classroom and individual
interventions
• Having high expectations of teacher’s work
Henninger, Koerner, Morgan, 2008 NASP Conference
Teacher Support Activities
• Acknowledging and rewarding the work and capabilities of teachers as professionals
• Validating teacher’s concerns
• Encourage teachers to work together and recognizing teacher collaboration
• Recognizing strengths and weaknesses
• Understanding and acknowledging pressures on teachers
• Listening to teacher’s concerns
• Providing specific supportsHenninger, Koerner, Morgan, 2008 NASP Conference
Intervention Plan Evaluation
Part One
Your project ID is:
• Last 4 digits of SS#
• Last 2 digits of year of birth
One Confirmed Hypothesis:
• Only 58% of the first grade students are meeting benchmark at the middle of the year because students are not provided opportunities to practice reading connected text on a first grade level.
Intervention Plan Evaluation
Part Two
Your project ID is:
• Last 4 digits of SS#
• Last 2 digits of year of birth
One Confirmed Hypothesis:
• Only 58% of the first grade students are meeting benchmark at the middle of the year because students are not provided opportunities to practice reading connected text on a first grade level.
Comprehensive Intervention Plan Development - Team Activity
Intervention
Infrastructure
Using Existing Data to Predict Intervention Needs
• Previous referral history predicts future referral history
• How do we interpret teacher referrals?
• Previous intervention history predicts future intervention history
• How do we use this information to establish an infrastructure for change?
Data-Driven Infrastructure:Establishing a Building Baseline
• Code referrals (reasons) for past 2-3 years
Identifies problems teachers feel they do not have the skills/support to handle
Referral pattern reflects skill pattern of the staff, the resources currently in place and the “history” of what constitutes a referral in that building
Identifies likely referral types for next 2 years
Identifies focus of Professional Development Activities AND potential Tier II and III interventions
Present data to staff. Reinforces “Need” concept
Data-Driven Infrastructure:Establishing a Building Baseline
• Assess current “Supplemental Interventions” Identify all students receiving supplemental
interventions For those interventions, identify
• Type and Focus (academic, direct instruction, etc)• Duration (minutes/week)• Provider
Aggregate• Identifies instructional support types in building• This constitutes Tier II and III intervention needs
Data-Driven Infrastructure:Establishing a Building Baseline
• Identify current progress monitoring assessment practices Determine if they meet needs/criteria
Strengthen with additional methods, if necessary
• Identify how data are used, organized and interpreted currently Determine how to use technology to support
Training Evaluation
No project ID needed
Final Thoughts
Problem Solving &
Response to Intervention
Resources
http://www.fcrr.org/Interventions/index.htm (Interventions for Struggling Readers)
http://ies.ed.gov/ncee/wwc/ (What Works Clearinghouse)
http://www.justreadflorida.com/instrreading.asp (F.A.I.R.)
http://www.justreadflorida.com/CAR-PD (CAR-PD)
http://www.fldoe.org/bii/curriculum/sss/ (Sunshine State Standards)
http://www.interventioncentral.org/ (Intervention Central)
Resourceshttp://www.fpg.unc.edu/~nirn/resources/type.cfm?typeID=4
(Articles and Reports)
http://www.fpg.unc.edu/~nirn/resources/detail.cfm?resourceID=61 (Top 5 Strategies To Enable the Use of Evidence-based Programs)
http://www.fpg.unc.edu/~nirn/resources/detail.cfm?resourceID=31 (Implementation Research: A Synthesis of the Literature)
http://www.fpg.unc.edu/~nirn/resources/detail.cfm?resourceID=65 (Implementation in the Real World: Taking Programs and Practice to Scale)
http://www.fpg.unc.edu/~nirn/resources/detail.cfm?resourceID=59 (Measuring Fidelity of Implementation: What it is, Why, How to do it, and How to use it)