FloridaRtI.usf.edu

A collaborative project between the Florida Department of Education and the University of South Florida

Intervention Design

Advance Organizer

Update: Consensus, Infrastructure, Implementation

Data Review: Problem ID, Problem Analysis

Linking Problem Analysis to Intervention

Intervention Design

Intervention Content

Intervention Plan

Intervention Integrity, Support, Documentation

Integrating Tiers of Intervention

Intervention Selection/Design & Implementation in Context

Identify

the Problem

Monitor

Progress

Analyze

the ProblemImplement

Intervention

Evaluate

Intervention

Effectiveness

Timeline

Select/DesignIntervention

Review

Consensus

Infrastructure

Implementation

Discussion - Consensus

• What activities are occurring to facilitate consensus building among school and district staff?

• What activities need to occur to better facilitate consensus building among school and district staff?

• What consensus building activities should be given top priority between now and the end of the school year?

Discussion - Infrastructure

• What activities are occurring to facilitate the development of the infrastructure needed to implement a PS/RtI model?

• What activities need to occur to better facilitate the development of the infrastructure needed to implement a PS/RtI model?

• What infrastructure development activities should be given top priority between now and the end of the school year?

Discussion - Implementation

• Given answers to the questions addressing consensus & infrastructure, what are implications for facilitating implementation of PS/RtI model in your building?

• What components of model can be targeted for implementation between now and end of school year given the levels of consensus and infrastructure?

• What components of model would need to be targeted for implementation later to allow more time to build support from staff and capacity (e.g., assessments, technology, training, policies, procedures) to implement?

Review Previous Skill Assessments

• Problem ID Gap between desired performance and current

performance

• Problem Analysis Most likely barriers to desired performance

• Using what you now understand of Problem ID and Problem Analysis, consider your Tier One data, identify a problem and analyze it. Later, you’ll link your Problem Analysis to the development of a comprehensive intervention plan.

ProblemAnalysis

InterventionDesign

The problem is occurring because ______________.

If ___________ would occur, then the problem would be reduced.

Intervention Development

• Interventions derived from effective problem-solving strategies

• Research/knowledge used to develop hypotheses.

• Verified hypotheses lead naturally to interventions

• Interventions will be evidence-based if hypotheses are evidence-based

Types of Interventions

• Skill Deficit Student lacks skills to successfully complete

task

• Performance Deficit Factors interfering with student’s capability of

performing the skill

Intervention Development:Verified Hypotheses

• H: Only 55% of second grade students are achieving reading fluency benchmark because the curriculum lacks fluency focus.

• P: If we modify the curriculum to focus on reading fluency, then more students will achieve benchmark.

• Intervention: Modification of second grade curriculum to include more reading fluency focus

Intervention Development:Verified Hypotheses

• H: Brandon is unable to stay in his seat because he lacks the self-monitoring skills necessary for self-control

• P: If we teach Brandon self-monitoring skills, then he will display improved self-control.

• Intervention: Self-instructional and self-monitoring training

Intervention Development:Verified Hypotheses

• H: Ernesto is unable to complete arithmetic seatwork on time because he has not acquired math facts at the automatic level

• P: If we provide Ernesto with strategies to increase fact fluency, then he will complete work on time.

• Intervention: Provide Ernesto with strategies to increase fact fluency Provide Ernesto more time until strategies to increase

fact fluency have had time to work

Intervention Development:Verified Hypotheses

• H: Gail is not attending school daily because she believes that all of the students she goes to class with exclude her from social activities

• P: If we help Gail to restructure her beliefs about being excluded, then attendance will improve.

• Intervention: Cognitive restructuring regarding belief system Identify students who would be willing to include Gail

in their activities Ensure adult prompting and monitoring of social

activities

Intervention Development:Verified Hypotheses

• H: Susanna is unable to comprehend at 5th grade level because she is disfluent in 5th grade reading material.

• P: If we improve her fluency, then comprehension will improve.

• Interventions: Limit comprehension expectations to current fluency level

Develop hypotheses to explain low rate of fluency

Tiers of InterventionI & II

Tier I Intervention• Tier One- Examining “Universal” Interventions

• Questions:

What percent of students are achieving district benchmarks? Effectiveness of instruction

How are these students doing compared to grade level benchmarks? GAP analysis

• Hypotheses

Ho: These students have not had access to an effective learning environment.

Ho: These students have not been engaged in an effective learning environment.

Tier 1 Intervention• Consider altering whole group instruction, curriculum

materials, instructional routine, independent practice (e.g., literacy/math centers)

• Breadth of skill focus might vary

• Group students based on skill data (data come from many sources)

• Differentiate instruction based on grouping

• Organize students based on skill performance Higher performing, more students, Lower performing, fewer students

• Same amount of time, different use of that time

Tier 2 Intervention

• Tier Two- Examining “Supplemental” Interventions

• Hypotheses:

Ho: Student requires additional time for direct instruction

Ho: Focus of the curriculum must narrow

• Assessment:

DIBELS, CBM, district assessments

• Interventions:

Increase AET (90-120-180)

e.g., K-3 Academic Support Plan

Narrow focus to fewer, barrier skills

District Supplemental Curriculum

Characteristics of Tier 2 Interventions

• Available in general education settings

• Opportunity to increase exposure (academic engaged time) to curriculum

• Opportunity to narrow focus of the curriculum

• Sufficient time for interventions to have an effect (10-30 weeks)

• Often are “standardized” supplemental curriculum protocols – identified through a problem solving process

Tier 2 Intervention

• First resource is TIME (AET) HOW much more time is needed?

• Second resource is CURRICULUM WHAT does the student need?

• Third resource is PERSONNEL WHO & WHERE will it be provided?

Tier 2: Getting TIME

• “Free” time--does not require additional personnel Staggering instruction

Differentiating instruction

Cross grade instruction

Skill-based instruction

• Standard Protocol Grouping

• Reduced range of “standard” curriculum

• After-School

• Home-Based

Tier 2: Curriculum

• Standard protocol approach

• Focus on essential skills

• Most likely, more EXPOSURE and more FOCUS of core instruction

• Linked directly to core instruction materials and benchmarks

• Criterion for effectiveness is ≈70% of students receiving Tier 2 will reach benchmarks

Tier 2: Personnel

• EVERYONE in the building is a potential resource

• Re-conceptualize who does what

• Personnel deployed AFTER needs are identified

• WHERE matters less and less

• REMEMBER, student performance matters more than labels, locations and staff needs.

• A school cannot deliver intensive (Tier 3) services to more than ≈7% of the population

Outline – Implementing An RtI System

• Tier 2 Decision Making –

Identify less than proficient students

Administer additional brief assessments to examine performance profiles

Group students with like performance profiles for supplemental instruction

Provide supplemental instruction based on skill needs

Outline – Implementing An RtI System (continued)

• Tier 2 Decision Making – Monitor progress

Review student progress monitoring data at scheduled intervals

How successful are students in response to Tier 2 Interventions?

• ≈70 - 80% is a good criterion

Modify supplemental instruction as necessary

Move students across tiers as data warrant

Job-Alike

• What is your role with respect to the PS/RtI model at your school?

• What is working well with PS/RtI?

• What is not working well?

• What is the best part when it comes to implementing PS/RtI at your school?

Intervention Content

Why Intervene?

Problem Definition & Problem Analysis have revealed information the problem solving team will use to determine what and how the students need to be taught.

The purpose of Intervention is to create an

instructional match

Instructional Match =

Explicit Evidence-Based Instructionin

Targeted Skillswithin a

Supported Learning Environment

Criteria for Interventions

• Evidence-based

• Delivered with Integrity

• Scaled for Intensity

• Implemented for Sufficient Time

• Evaluated Frequently

• Integrated Across Tiers

Evidence-Based

SCIENTIFICALLY BASED RESEARCH -

• means research that involves the application of rigorous, systematic, and objective procedures to obtain reliable and valid knowledge relevant to education activities and programs

• Section 9101(37) of ESEA, as amended by the No Child Left Behind Act (NCLB) of 2001

Evidence-Based

• Employs systematic, empirical methods that draw on observation or experiment

• Involves rigorous data analyses that are adequate to test the stated hypotheses and justify the general conclusions

• Relies on measurements or observational methods that provide reliable and valid data across evaluators and observers, across multiple measurements and observations.

• Is evaluated using experimental or quasi-experimental designs… with appropriate controls… with a preference for random-assignment experiments

• Section 9101(37) of ESEA, as amended by the No Child Left Behind Act (NCLB) of 2001

Evidence-Based

• Ensures that experimental studies are presented in sufficient detail and clarity to allow for replication or, at a minimum, offer the opportunity to build systematically on their findings

• Has been accepted by a peer reviewed journal or approved by a panel of independent experts through a rigorous scientific review

• Section 9101(37) of ESEA, as amended by the No Child Left Behind Act (NCLB) of 2001

Practitioner journals or education magazines are not the

same as peer-reviewed academic journals

More

More

Less

Less

Measurement Frequency

Measurement Precision

Depth of Problem Analysis

Instructional Time

Measurement Focus

Evidence Base

Group Size

Principles of Intervention Design

Intervention is…

• Planful- procedures to be applied are specified clearly and completely

• Environmentally Focused- actions taken modify the environment not the individual

• Goal Directed- the team writes an ambitious, yet attainable goal statement prior to intervention design

Principles of Intervention Design

Intervention should be designed to:

• Adjust what is being taught

and / or

• how it is taught

Intervention Plan

Components of an Effective Intervention Plan

Address:

• Persons Responsible

• Skills Targeted

• Implementation Arrangements

• Measurement Strategy

• Decision Making Rule

In essence - who, what, when, where

Designing anIntervention Plan

Person(s) Responsible:These should include names of those implementing the intervention, supporting the intervention, collecting the data and making decisions about the effectiveness of the intervention.

Targeted skills & Instructional strategies:

Specific information about

What & How to teach here

Implementation Arrangements:

Determine: Where, When-(frequency, length of time), Materials

Measurement Strategy:

Measurement conditions (how, what, where, monitoring schedule)

Decision-making Plan

Determine frequency of data collection, number of data points/decision rule - More on this during Day 5

Persons Responsible

Specifically named individuals that will be responsible for:

implementing the intervention,

supporting the intervention,

assessing the integrity of the intervention,

and monitoring the effectiveness of the intervention

Target Skills & Instructional Strategies

Explicitly define the skills to be taught and strategies that will be utilized.

Select evidence based instructional strategies that are based on the defined problem and verified hypotheses in order to maximize likelihood of success.

Implementation Arrangements

• Where

• When

• Frequency

• Length of time

• Materials

Intervention Plan

• Must include:Who is responsible?What will be done?When will it occur?Where will it occur?

Design an Intervention Plan:Measurement Strategy

Record information about:

• Who is responsible for the on-going collection of data

• The method of data collection-(ex.:probes/graphs, frequency counts/graphs)

• The measurement conditions- (ex.: environmental factors are consistent)

• The monitoring schedule-(ex.: establishing consistent measurement intervals)

Design an Intervention Plan:Decision-making Plan

How do we decide if a plan is or isn’t working?

Decisions will be made based on the following:

• Level of skill

• Rate of progress

• Decision rule

Decision Rules: What is a “Good” Response to Intervention?

• Positive Response

Gap is closing

Can extrapolate point at which target student will “come in range” of peers--even if this is long range

• Questionable Response

Rate at which gap is widening slows considerably, but gap is still widening

Gap stops widening but closure does not occur

• Poor Response

Gap continues to widen with no change in rate.

Performance

Time

Positive Response to Intervention

Expected Trajectory

Observed Trajectory

Decision Rules: What is a “Good” Response to Intervention?

• Positive Response

Gap is closing

Can extrapolate point at which target student will “come in range” of peers--even if this is long range

• Questionable Response

Rate at which gap is widening slows considerably, but gap is still widening

Gap stops widening but closure does not occur

• Poor Response

Gap continues to widen with no change in rate.

Performance

Time

Questionable Response to Intervention

Expected Trajectory

Observed Trajectory

Decision Rules: What is a “Good” Response to Intervention?

• Positive Response

Gap is closing

Can extrapolate point at which target student will “come in range” of peers--even if this is long range

• Questionable Response

Rate at which gap is widening slows considerably, but gap is still widening

Gap stops widening but closure does not occur

• Poor Response

Gap continues to widen with no change in rate.

Performance

Time

Poor Response to Intervention

Expected Trajectory

Observed Trajectory

Performance

Time

Response to Intervention

Expected Trajectory

Observed Trajectory

Monitoring Plan for Determining Student Progress

• Must include:Who is responsible?What data will be collected and

how often?How will we decide if the plan

effective?

Decision Rules: Linking RtI to Intervention Decisions

• Positive, Questionable, Poor Response• Intervention Decision Based on RtI (General Guidelines)

Positive• Continue intervention until student reaches benchmark

(at least).• Fade intervention to determine if student has acquired

functional independence. Questionable

• Increase intensity of current intervention for a short period of time and assess impact. If rate improves, continue. If rate does not improve, return to problem solving.

Poor• Return to problem solving for new intervention

Intervention Selection/Design & Implementation in Context

Identify

the Problem

Monitor

Progress

Analyze

the ProblemImplement

Intervention

Evaluate

Intervention

Effectiveness

Timeline

Select/DesignIntervention

Intervention Integrity, Support, &

Documentation

Problem-Solving and Treatment Fidelity

• Response to Intervention is based on the actuality of interventions (Tier 1,2,3) delivered as intended

• We CANNOT do RtI if the intervention was not implemented or was implemented poorly

• Therefore, intervention integrity must be ensured and documented if we are to use RtI

• It will become part and parcel of the procedural safeguard process for students with disabilities.

Problem-Solving and Treatment Fidelity

• Strategies that improve fidelity Follow-up by a consultant/support staff

Presentation of student data illustrating response to intervention

Review of intervention implementation plans

Frequency — range from daily to weekly initially(Noell, Witt, Slider, Connell, Gatti, Williams, Keonig, Resetar, &

Duhon, in press)

Problem-Solving and Treatment Fidelity

• Teacher responsiveness to implementing interventions

Understands the “need” for intervention

Perceives self as possessing skills to implement OR has the social support to implement while acquiring skills

(Bev Showers and colleagues)

• Based on the research, the following intervention protocol should be considered: Ensure that teacher/parent understands need

Evaluate skill of parent/teacher and determine level of support

Delineate intervention in stepwise fashion

Create implementation schedule for intervention

• Time of day, times per day, etc.

Create intervention support/fidelity schedule

Problem-Solving and Treatment Fidelity

• IF YOU CANNOT CREATE A SUPPORT SCHEDULE DO NOT DO INTERVENTION UNTIL ONE CAN BE ESTABLISHED

• It is critical that building administrators understand that the support schedule is as or more important than the intervention schedule

Problem-Solving and Treatment Fidelity

Intervention Integrity Decisions

Evidence based intervention linked to verified hypothesis planned

Evidence based intervention implemented

Student Outcomes (SO)

Assessed

Treatment Integrity (TI) Assessed

Data-based Decisions

Continue Intervention

Implement strategies to promote treatment integrity

Modify/change Intervention

+SO +TI

-SO +TI

-SO -TI

From Lisa Hagermoser Sanetti, 2008 NASP Convention

Intervention Documentation Worksheet

Documentation Plan

• Must include:Who is responsible?What will be done?When will it occur?How will data be shared?

Support Plan

• Must include:Who is responsible?What will be done?When will it occur?Where will it occur?

Intervention Support

• Intervention plans should be developed based on student need and skills of staff

• All intervention plans should have intervention support

• Principals should ensure that intervention plans have intervention support

• Teachers should not be expected to implement plans for which there is no support

Critical Components of Intervention Support

• Support for Intervention Integrity

• Documentation of Intervention Implementation

Intervention and eligibility decisions and outcomes cannot be supported in an RtI model without these two critical components

Intervention Support Plan

• Before Intervention Implementation Review data Review steps to intervention Determine logistics

• First 2 weeks 2-3 meetings/week Review data Review steps to intervention Revise, if necessary

Intervention Support

• Second Two Weeks Meet twice each week

• Following weeks Meet at least weekly Review data Review steps Discuss Revisions

• Approaching benchmark Review data Schedule for intervention fading Review data

Teacher Support Activities

• Holding teachers in high regard; honoring them and considering them worthy of esteem

• Providing teachers with access to tools that work

• Advocating for effective professional development activities

• Actively seeking, and expecting input in: curriculum, instruction, and assessment professional development planning & decision making developing and implementing classroom and individual

interventions

• Having high expectations of teacher’s work

Henninger, Koerner, Morgan, 2008 NASP Conference

Teacher Support Activities

• Acknowledging and rewarding the work and capabilities of teachers as professionals

• Validating teacher’s concerns

• Encourage teachers to work together and recognizing teacher collaboration

• Recognizing strengths and weaknesses

• Understanding and acknowledging pressures on teachers

• Listening to teacher’s concerns

• Providing specific supportsHenninger, Koerner, Morgan, 2008 NASP Conference

Intervention Plan Evaluation

Part One

Your project ID is:

• Last 4 digits of SS#

• Last 2 digits of year of birth

One Confirmed Hypothesis:

• Only 58% of the first grade students are meeting benchmark at the middle of the year because students are not provided opportunities to practice reading connected text on a first grade level.

Intervention Plan Evaluation

Part Two

Your project ID is:

• Last 4 digits of SS#

• Last 2 digits of year of birth

One Confirmed Hypothesis:

• Only 58% of the first grade students are meeting benchmark at the middle of the year because students are not provided opportunities to practice reading connected text on a first grade level.

Comprehensive Intervention Plan Development - Team Activity

Intervention

Infrastructure

Using Existing Data to Predict Intervention Needs

• Previous referral history predicts future referral history

• How do we interpret teacher referrals?

• Previous intervention history predicts future intervention history

• How do we use this information to establish an infrastructure for change?

Data-Driven Infrastructure:Establishing a Building Baseline

• Code referrals (reasons) for past 2-3 years

Identifies problems teachers feel they do not have the skills/support to handle

Referral pattern reflects skill pattern of the staff, the resources currently in place and the “history” of what constitutes a referral in that building

Identifies likely referral types for next 2 years

Identifies focus of Professional Development Activities AND potential Tier II and III interventions

Present data to staff. Reinforces “Need” concept

Data-Driven Infrastructure:Establishing a Building Baseline

• Assess current “Supplemental Interventions” Identify all students receiving supplemental

interventions For those interventions, identify

• Type and Focus (academic, direct instruction, etc)• Duration (minutes/week)• Provider

Aggregate• Identifies instructional support types in building• This constitutes Tier II and III intervention needs

Data-Driven Infrastructure:Establishing a Building Baseline

• Identify current progress monitoring assessment practices Determine if they meet needs/criteria

Strengthen with additional methods, if necessary

• Identify how data are used, organized and interpreted currently Determine how to use technology to support

Training Evaluation

No project ID needed

Final Thoughts

Problem Solving &

Response to Intervention

Resources

http://www.fcrr.org/Interventions/index.htm (Interventions for Struggling Readers)

http://ies.ed.gov/ncee/wwc/ (What Works Clearinghouse)

http://www.justreadflorida.com/instrreading.asp (F.A.I.R.)

http://www.justreadflorida.com/CAR-PD (CAR-PD)

http://www.fldoe.org/bii/curriculum/sss/ (Sunshine State Standards)

http://www.interventioncentral.org/ (Intervention Central)

Resourceshttp://www.fpg.unc.edu/~nirn/resources/type.cfm?typeID=4

(Articles and Reports)

http://www.fpg.unc.edu/~nirn/resources/detail.cfm?resourceID=61 (Top 5 Strategies To Enable the Use of Evidence-based Programs)

http://www.fpg.unc.edu/~nirn/resources/detail.cfm?resourceID=31 (Implementation Research: A Synthesis of the Literature)

http://www.fpg.unc.edu/~nirn/resources/detail.cfm?resourceID=65 (Implementation in the Real World: Taking Programs and Practice to Scale)

http://www.fpg.unc.edu/~nirn/resources/detail.cfm?resourceID=59 (Measuring Fidelity of Implementation: What it is, Why, How to do it, and How to use it)