Examining Strategies to Increase Behavior Support Plan

Implementation: Contextual Fit, Training, Performance Feedback.

Jeffery Barker, Mary Jo Brackett, Sally Geary, Renee Hathorn, Stacy Henderson, Brian McKenzie, Colin Ryan, Alicia Teays, and Roland Wilson.Portland State University

Introduction• Approximately 1% to 5% of students exhibit

chronic and intense externalizing behavior as manifested by: aggression, property destruction and antisocial behavior (Sugai et al., 2000).

• These students account for 50% of referrals.• When plan implementation deteriorates,

treatments/interventions may be less effective

Introduction Cont.

• Federal law has attempted to address the difficulty of educating students who require more support

• The IDEA (1997)and IDEIA (2004) require use of function-based positive behavior support plans based on functional behavior assessments (Ingram, Lewis-Palmer,& Sugai, 2005).

Implementation Fidelity

• Professionals in education are experiencing cuts in funding and dwindling resources via time and adequate training.

• Function-based behavior plans must be feasible and implemented as intended

• The best developed plans make no difference if they are not implemented with integrity.

Barriers to Implementation• Lack of training• Lack of time• Difficulty implementing behavior plans in their

setting. – Bambara, Goh, Kern, & Grace 2012

• Potential Solutions– Implementing a quality BSP requires an active team process.– The use of performance feedback shows promise in

promoting implementation of PBSPs (Solomon, Klein, Politylo, 2012).

Reinforcement & Feedback

• Studies show that reinforcement for teachers can be a factor in implementation fidelity (Cossairt, Hall & Hopkins, 1973).

• Research indicates that frequent performance feedback results in a higher level of implementation fidelity (Jones, Wickstrom & Friman, 1997).

Research Questions

• How does implementation fidelity vary as a function of:a) Performance Feedbackb) Contextual Fit to Implementationc) Training of Implementation?

• Do student outcomes appear to be related to implementation fidelity?

Methods• Settings and Participants• Measures

• Contextual Fit• Training• Performance Feedback

–Reinforcement

Participants/Settings• 9 Students

– 7 students with IEP’s 4 in inclusive settings, 3 in self- contained settings– 2 students not identified 1 in transition/day treatment program 1 in general education environment

• Implementers– 3 general education teachers– 3 SPED teachers– 5 SPED assistants

Settings

• Schools– 4 elementary schools– 3 middle schools– 1 high school– 1 post secondary

Measures: Contextual Fit• Purpose

– Han and Weiss (2005) found that in order for a program or plan to be successful, it must be acceptable to the staff implementing it.

– Assessment needed to determine the fit of the plan within the implementer's environment, resources, and values.

• Adapted from the “Self Assessment of Contextual Fit” survey developed by Horner, Salentine, & Albin (2003)– Chosen for it’s comprehensive assessment of contextual

fit and for its ease of use– Major changes included:

• Condensing eight categories down to three• Pairing/combining questions to fit into these categories• Eliminating categories and questions due to redundancy.

Measures: Contextual Fit Cont.• Category Example

– “Knowledge of elements and Skills Needed to Implement the Behavior Support Plan.”

• Question Examples– “I am aware of the elements of this behavior support plan and

the steps that I am expected to do to implement this plan.” 1 2 3 4 5 6

Strongly Moderately Barely Barely Moderately StronglyDisagree Disagree Disagree Agree Agree Agree

– “I have the skills needed to implement this behavior support plan.”

1 2 3 4 5 6Strongly Moderately Barely Barely Moderately StronglyDisagree Disagree Disagree Agree Agree Agree

Measures: Training

• Allen and Forman (1984), Sarakoff et al. (2004), and Digennaro-Reed et al. (2010) found that multiple strategies can be effective in teacher training.

• These include: -Didactic training -Video Modeling

-Prompting -Modeling

-Feedback -Role Playing

-Reinforcement

Measures: Training Cont.

• Training was measured through self-report on a Likert-scale questionnaire designed by the research team to reflect the key components found in the literature review.

Measures: Training Cont.

Give the implementer a copy of the BSP to look at on their own

Verbal description provided to implementer

Model/Demonstrate the intervention

Have implementer practice doing the intervention

Provide feedback to implementer while they practice implementation

1 2 3 4 5

Name Date Rate the extent that you trained your intervention implementer to implement the targeted intervention:Implementer name: Intervention: The student was present when the implementer was provided the description or opportunity to model or practice the intervention? Y NThe description/model/practice occurred in the targeted setting where the intervention is supposed to occur? Y N

Measurements: Performance Feedback

• Five aspects of feedback were measured.1. Frequency

• Ruhl (2004) and Solomon (2012) found that high frequency of feedback increased implementation fidelity.

• Scoring:1 Never☐ 2 Once a Week☐3 Twice Weekly☐4 Daily☐5 Hourly☐

Measurements: Performance Feedback Cont.

2. Mode of feedback delivery• Mortenson (2004) found that this can be

tailored to fit specific teaching environments. • Scoring:

1 No Feedback☐2 Email☐3 Written Note☐4 Call/Skype☐5 In Person☐

Measurements: Performance Feedback Cont.

3. Time Lapse• Codding et al. (2005) found that decreased time

between observation and feedback delivery increased improvements in implementation fidelity.

• Scoring:☐1 No Feedback Given

2 End of Week ☐3 End of Day ☐4 Immediate ☐5 Real Time☐

Measurements: Performance Feedback Cont.

4. Type of feedback• Hagermoser Sanetti et al. (2007) found that

combinations of types of feedback increased implementation fidelity.

• Scoring:☐1 Descriptive feedback (vague in nature, no specific details noted,

either strengths or needed improvements)☐2 Descriptive feedback (specific in nature, strengths and/or

improvements needed noted)☐3 Numeric feedback (feedback scaled on numeric scale)☐4 Graphic feedback (vague in nature, not explained or hard to

understand meaning of graphs)☐5 Graphic feedback (specific in nature, easy to understand, clearly

shows successes or areas in need of improvement)

Measurements: Performance Feedback Cont.

5. Reinforcement– Studies have shown that both positive (Cossairt et al.,

1973) and negative (DiGennaro et al., 2005) reinforcement increased implementation fidelity.

– Scoring:☐1 No Reinforcement Involved

2 Reinforcement Involved, but very limited (present 1-3 ☐times, not rewarding to individual)

3 Reinforcement Involved, limited (present 4-6 times, ☐marginal level of reward to individual)

4 Reinforcement Involved, moderate (present at most ☐sessions, rewarding to individual)

5 Reinforcement Involved (present at all sessions where ☐implementation fidelity met, highly rewarding to individual)

Measures: Student Outcomes

Opportunity Staff score Staff initials EH scoreP3 to P4 0 1 2 0 1 2 1st 5 minutes P4 0 1 2 0 1 2P6 to P7 0 1 2 0 1 21st 5 minutes of P7 0 1 2 0 1 2P7 to HR 0 1 2 0 1 21st 5 minutes of HR 0 1 2 0 1 2

• Each researcher created their own student outcomes measurement tool specifically designed to fit their individual case.

• These tools were used daily to collect student outcome data.• Example:

Random Acts of Koolness! Date:_________________________0 – Not Kool at all 1- Kool, but had to be reminded 2- Way Kool!

Measures: Implementation Fidelity

Date Used White Board

Gave reminders and prompts

Allowed Breaks Checked in and out

0 1 2

0 1 2

0 1 2

0 1 2

0 1 2 0 1 2

0 1 2

0 1 2

0 1 2

0 1 2

0 1 2

0 1 2

Total Points = ________________ Points Possible = 24

Today _________________%Goal _________________%

• Each researcher created their own implementation fidelity measurement tool specifically designed to fit their individual case.

• These tools were used daily to collect implementation fidelity data. • Example:

Implementation IntegritySuccesses:

__________________________________________________________________________________Parent

Signature______________________________________________________________________________

Procedures• Each Researcher performed a Functional

Behavioral Assessment to access their targeted student, which was instrumental in developing a Behavior Support Plan, and developed a plan to support Implementation Fidelity consisting of:– Contextual fit– Training– Performance Feedback and Reinforcement – Implemented the plan and collected data. – Compiled and reviewed data, making adjustments

when necessary

Research Design• Descriptive Case Study

–Collected data• Interviews, existing records, and

observational studies–9 Single case studies–Only 2 of the 9 includes baseline data

• Examine trends across case studies related to implementation fidelity x implementation supports

Results

Individual Case Data & Mean(Impl. Supports, Fidelity & Outcomes)

re ma al cr sh bm sg rw j MeanContextual Fit

80 81.6 96.6 100 96.6 98.3 99.5 100 93 94Training

60 60 100 60 40 60 60 100 60 67Performance

Feedback 60 72 60 64 48 68 60 92 64 64.4Implementation Supports (% In

Place)66.6 71.2 85.5 72 61.5 75.4 63.1 97.3 72.4 74

Implementation Fidelity (% of impl) 94 83 79 99 96 100 87 100 92.5 92.3

Student Outcomes (% of points

earned)67.9 66.7 85 76 76.3 100 96.1 89.4 94.5 83.5

Discussion

High Implementation Fidelity

• Mean Implementation fidelity was 92% across cases with low of 79%

• Paying attention to Implementation Supports seemed to have an impact:–Contextual Fit–Training–Performance Feedback &

Reinforcement

Contextual Fit• Researcher/Practitioners designed

interventions that had high levels of Contextual Fit as rated by implementers:

• Feasibility: Educators were confident they could implement plan in environment

• Effective: Plan was appropriate in response to problem behavior and could result in positive behavior

Training

• Ratings on Training Provided scoring rubric ranged from “2 – Verbal Description provided to implementer” to “5 - Provide feedback to implementer while they practice implementation” – w/ Demonstration and Modeling: Implementation was

96% with fidelity– Even w/ only “Verbal Description without Modeling”

implementation remained high

• Possibly due to high ratings on Contextual Fit and Performance Feedback

Performance Feedback• Ranges conducted

• Hourly/Daily/Weekly/Biweekly–All levels showed consistent implementation

fidelity suggesting that varying levels of Performance Feedback can be used to support implementation.

–This is particularly true with interventions that have high contextual fit and effective training up front.

Reinforcement• Implementers rated levels of

reinforcement received during intervention

• 55% reported no reinforcement involved. Implementation fidelity averaged (91%).

• 33% reported moderate reinforcement involved. Implementation fidelity averaged (92%).

– Implementers reported it had been a rewarding experience…. Simply having someone check in and provide verbal feedback

Implications for Practice• When contextual fit is good and when training and

regular feedback are given implementation fidelity is more likely to be achieved

– Training- Verbal description of the intervention may be sufficient to support fidelity of the plan (when the plan has strong contextual fit & is paired w/ performance feedback)

– Performance feedback is important, in some cases, as little as weekly feedback may be sufficient, if a plan has strong contextual fit

Limitations

• This was only a case study, not an experimental study– So we cannot isolate the specific impact that training,

contextual fit or performance feedback had on implementation

• In most cases ratings of student outcomes and implementation fidelity were designed to be practical to a classroom situation, which could lead to some subjectivity and bias in ratings

• Case study data was constrained by time, no long term data.

Future Research• Conduct an experimental study to determine what the

precise impact of contextual fit, training & performance feedback has on implementation.

• What is the most effective and efficient frequency for providing performance feedback (weekly, etc.)?

• What would the function be of having a structured feedback loop in a behavior support plan?– Most efficient use of time

• How does overall class structure affect implementation fidelity– Peer experience, relationships

Conclusion

• As discussed in our Literature review, our research also found– Designing a Behavior Support Plan with Contextual

Fit, Increases the likelihood of implementation fidelity– Performance feedback in combination with training

also increases likelihood of implementation fidelity– Frequent performance feedback with individuals

administering an intervention often results in a higher level of implementation fidelity

– Higher implementation fidelity results in greater desired behavior success with student

Questions?

Thank You!