Building the Capacity of Schools, Districts and States to Implement School-wide PBIS Rob Horner...

Building the Capacity of Schools, Districts and States to Implement School-wide PBIS

Rob HornerUniversity of Oregonwww.pbis.org

Goals

• Define current status of PBIS implementation in the U.S.

• Summarize features of Schools that are successful at implementing and sustaining PBIS with functional outcomes for students.

• Define features of Districts that establish the capacity to implement PBIS at scales of social significance.

• Define features of States that establish capacity to implement PBIS at scales of social significance

Why SWPBIS?

• The fundamental purpose of SWPBIS is to make schools more effective and equitable learning environments.

Predictable

Consistent

Positive

Safe

SWPBIS: Building Effective Schools

Main Messages• PBIS works.

Effective (academic, behavior)

Equitable (all students succeed) Efficient (time, cost)

Main Message: Build Capcity• Schools

• Implement with high fidelity at all three tiers• Expect more from your districts and states

• District/ Region• Build Training, Coaching, Evaluation and Technical Expertise needed• Build capacity to sustain PBIS• Adapt to geography and size

• States• Provide functional leadership• Implement with a full “slice” of the educational system• Align initiatives• Provide the data systems, training, coaching and evaluation needed

School-wide Positive Behavioral Interventions and Supports (SWPBIS)

• The social culture of a school matters.

• A continuum of supports that begins with the whole school and extends to intensive, wraparound support for individual students and their families.

• Effective practices with the systems needed for high fidelity and sustainability

• Multiple tiers of intensity

What is School-wide Positive Behavior Intervention and Support (PBIS)?

• School-wide PBIS is:• A multi-tiered framework for establishing the social culture

and behavioral supports needed for a school to achieve behavioral and academic outcomes for all students.

• Evidence-based features of SWPBIS• Prevention• Define and teach positive social expectations• Acknowledge positive behavior• Arrange consistent consequences for problem behavior• On-going collection and use of data for decision-making• Continuum of intensive, individual intervention supports. • Implementation of the systems that support effective

practices

Establishing a Social Culture

Common Vision/Values

Common Language

Common Experience

MEMBERSHIP

SYSTEMS

PRACTICESDAT

A

Culturally KnowledgeableStaff Behavior

Culturally Relevant Support for

Student Behavior

OUTCOMES

Culturally Equitable Academic &Social Competence

Culturally ValidDecisionMaking

School-wide PBIS

Standardized Assessments

Experimental Research on SWPBIS

Bradshaw, C.P., Koth, C.W., Thornton, L.A., & Leaf, P.J. (2009). Altering school climate through school-wide Positive Behavioral Interventions and Supports: Findings from a group-randomized effectiveness trial. Prevention Science, 10(2), 100-115

Bradshaw, C.P., Koth, C.W., Bevans, K.B., Ialongo, N., & Leaf, P.J. (2008). The impact of school-wide Positive Behavioral Interventions and Supports (PBIS) on the organizational health of elementary schools. School Psychology Quarterly, 23(4), 462-473.

Bradshaw, C. P., Mitchell, M. M., & Leaf, P. J. (2010). Examining the effects of School-Wide Positive Behavioral Interventions and Supports on student outcomes: Results from a randomized controlled effectiveness trial in elementary schools. Journal of Positive Behavior Interventions, 12, 133-148.

Bradshaw, C.P., Reinke, W. M., Brown, L. D., Bevans, K.B., & Leaf, P.J. (2008). Implementation of school-wide Positive Behavioral Interventions and Supports (PBIS) in elementary schools: Observations from a randomized trial. Education & Treatment of Children, 31, 1-26.

Bradshaw, C., Waasdorp, T., Leaf. P., (in press). Effects of School-wide positive behavioral interventions and supports on child behavior problems and adjustment. Pediatrics.

Horner, R., Sugai, G., Smolkowski, K., Eber, L., Nakasato, J., Todd, A., & Esperanza, J., (2009). A randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior support in elementary schools. Journal of Positive Behavior Interventions, 11, 133-145.

Horner, R. H., Sugai, G., & Anderson, C. M. (2010). Examining the evidence base for school-wide positive behavior support. Focus on Exceptionality, 42(8), 1-14.

Ross, S. W., Endrulat, N. R., & Horner, R. H. (2012). Adult outcomes of school-wide positive behavior support. Journal of Positive Behavioral Interventions. 14(2) 118-128.Waasdorp, T., Bradshaw, C., & Leaf , P., (2012) The Impact of Schoolwide Positive Behavioral Interventions and Supports on Bullying and Peer Rejection: A Randomized Controlled Effectiveness Trial. Archive of Pediatric Adolescent Medicine. 2012;166(2):149-156 Bradshaw, Pas, Goldweber, Rosenberg, & Leaf, 2012

Number of Schools Implementing SWPBIS since 2000January, 2014

19,960

Number of Schools Implementation SWPBIS (Tier I) by StateJanuary 2014

14 States with more than 500

schools

Total number of schools using SWPBIS

Total number of schools measuring fidelity

Schools at Tier I fidelity

Number of PBIS schools (Green) Implementing, (Red) measuring fidelity and (Blue) at Tier I fidelity by state

>75%-------------------

ConnecticutFloridaIllinoisIowa

KentuckyMichigan

MinnesotaMissouri

North CarolinaOregon

South CarolinaVermont

Wisconsin

Building Capacity: Schools• Focus on “core features” that deliver valued outcomes.

• PBIS is a framework for organizing practices that deliver core features. The core features should be documented to produce valued outcomes.

PBISSelection and teaching of school-wide Expectations

School-wide Expectations

Improved Social and Academic Competence for Students

Framework

Practice

CoreFeature

ValuedOutcomes

Schools

• Define and distinguish between• Practices• Core features• Valued outcomes

Procedures Core Features

Valued Outcomes

Core Features

Effective Procedur

e

Effective Procedur

e

Effective Procedur

e

Effective Procedur

e

Values

Science

TechnologyTechnology Cultural/ Contextual

Fit

Implications• Certify, and Promote “core features”

• Do not certify people• Do not certify manuals or programs

• Measure “Core Features”… use for decision-making• Measure fidelity by assessing if “core feature” is in place• Provide examples of multiple practices (ways) to achieve core

features• Focus on “contextual fit” variables that guide selection of

effective practices.

Building Capacity: Schools

• Anticipate implementation error patterns

Primary Prevention:School-/Classroom-Wide Systems for

All Students,Staff, & Settings

Secondary Prevention:Specialized Group

Systems for Students with At-Risk Behavior

Tertiary Prevention:Specialized

IndividualizedSystems for Students

with High-Risk Behavior

~80% of Students

~15%

~5%

SCHOOL-WIDE POSITIVE BEHAVIOR

SUPPORT

27

Main Ideas:1.Invest in prevention first2.Multiple tiers of support intensity3.Early/rapid access to support

Remember that the multiple tiers of support refer to our SUPPORT not Students.

Avoid creating a new disability labeling system.

Reading

Behavior

Math

Health

Building Capacity: Schools

• Measure “fidelity of implementation”• As a DV to assess implementation practices• As an IV to improve level of adoption.

• Fidelity measures should focus on the “core features” of any practice.

Iowa Checklist 01-05, PK-6 % Fully & Partially Implemented

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

05-A

ug-0

3

05-N

ov-0

3

23-F

eb-0

4

22-J

an-0

4

01-F

eb-0

5

02-J

un-0

5

12-A

ug-0

4

24-N

ov-0

4

01-M

ar-0

5

12-S

ep-0

2

31-O

ct-0

2

28-F

eb-0

3

21-A

pr-0

3

01-S

ep-0

3

05-N

ov-0

3

05-A

ug-0

3

11-S

ep-0

3

07-N

ov-0

3

06-F

eb-0

4

01-S

ep-0

3

01-N

ov-0

3

01-M

ar-0

4

03-A

ug-0

4

08-N

ov-0

4

08-M

ar-0

5

03-J

un-0

5

1 1 1 2 2 2 3 3 3 4 4 4 4 4 4 5 5 5 5 6 6 6 7 7 7 7

Start Up Full Implementation Start Up Part Implementation

School-wide PBISTiered Fidelity InventoryOSEP Technical Assistance Center on Positive Behavioral Interventions and Supports

2014

Algozzine, B., Barrett, S., Eber, L., George, H., Horner, R., Lewis, T., Putnam, B., Swain-Bradway, J., McIntosh, K., & Sugai, G (2014). School-wide PBIS Tiered Fidelity Inventory. OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports. www.pbis.org.

October 2014

PBIS Tiered Fidelity Inventory• Assesses PBIS implementation at all three tiers.

Building Capacity: Schools

• Focus on “efficiency” of practices• Time• Money• Expertise of personnel• Match with existing organizations/ systems.

• 1. Efficiency for adoption

• 2. Efficiency for sustained performance

NOTE:Differences in Efficiency across Multiple

Tiers of Support

Time Cost of aDiscipline Referral(Avg. 45 minutes per incident for student 30 min for Admin 15 min for Teacher)

1000 Referrals/yr

2000 Referrals/yr

Administrator Time

500 Hours 1000 Hours

Teacher Time 250 Hours 500 Hours

Student Time 750 Hours 1500 Hours

Totals 1500 Hours 3000 Hours

0

300

600

900

1200

1500

Tota

l O

ffic

e D

iscip

line R

efe

rrals

95-96 96-97 97-98 98-99School Years

Kennedy Middle School

Pre PBIS Year 1 Year 2 Year 3

What does a reduction of 850 office referrals and 25 suspensions mean? Kennedy Middle School

Savings in Administrative time

ODR = 15 min Suspension = 45 min

13,875 minutes231 hours

29, 8-hour days

Savings in Student Instructional time

ODR = 45 min Suspension = 216 min

43,650 minutes728 hours

121, 6-hour school days

Building Capacity: Schools

• Use Implementation Science• Implementation Drivers• Stages of Implementation• Improvement Cycles

WHEN

Stages

WHO

Teams

HOW

Drivers

WHAT

Interventions

HOW

Cycles

Performance Assessment (fidelity)

Coaching

Training

Selection

Integrated & Compensatory

Systems Intervention

Facilitative Administration

Decision Support Data System

AdaptiveTechnical

Com

pete

ncy

Driv

ers O

rganization Drivers

Leadership Drivers

Consistent Uses of Innovations

Reliable Benefits

Integrated & Compensatory

Implementation Drivers An Active Implementation Framework

Implementation Drivers An Active Implementation Framework

Stages of Implementation

• Exploration• Installation• Initial Implementation• Full Implementation• Innovation• Sustainability

Implementation occurs in stages:

Fixsen, Naoom, Blase, Friedman, & Wallace, 2005

2 – 4 Years

Stages of ImplementationFocusFocus StageStage DescriptionDescription

Exploration/Adoption

Decision regarding commitment to adopting the program/practices and supporting successful implementation.

Installation Set up infrastructure so that successful implementation can take place and be supported. Establish team and data systems, conduct audit, develop plan.

Initial Implementation

Try out the practices, work out details, learn and improve before expanding to other contexts.

Elaboration Expand the program/practices to other locations, individuals, times- adjust from learning in initial implementation.

Continuous Improvement/ Regeneration

Make it easier, more efficient. Embed within current practices.

Steve Goodman

Improvement Cycles

Main Messages

• Sustained and High Fidelity Implementation of SWPBIS requires active District Support.

• Especially for Tiers II and III

-----------------------------------------------------------

Student = unit of impactSchools = unit of analysisDistrict = unit of implementation

Leadership Team

Active Coordination

FundingVisibility Political

Support

Training Coaching Evaluation

Local School/District Teams/Demonstrations

BehavioralExpertise

Policy

Sugai et al., www.pbis.org

Implications for Schools• Build commitment from Administration, Faculty, Students and

Families that attention to social culture is important• Implement SCHOOL-WIDE, multi-tiered systems.

• Build on what you already do well• Never stop doing what already works• Always implement the smallest change that produces the largest

effect.• Never adopt something new without defining what you will STOP

doing to create the resources needed for new adoption.

• Measure fidelity of implementation as well as impact• Measure fidelity frequently, and use the information to guide

improvement.

• Report outcomes to families, faculty, community and administration.

Implications for Schools• Expect more support from your district (or regional unit)

• Initial personnel orientation• Data systems

• Fidelity• Universal Screen and Progress Monitor• Standardized Assessments

• Support for Tier II, and Tier III implementation• Role of school psychologist, counselor, social worker

Tier II Tier III

Increased structure Assessment: FBA, Mental Health, Academic, Physical

Increased frequency of recognition/ feedback

Comprehensive support plan

Self-assessment Fidelity measures

Link home and school Outcome measures

Building Capacity: Districts/ Regions

Building Capacity: Districts/ Regions• Three different conditions:

• Stand alone district

• Urban district

• Clusters of rural / small districts

Common Goals:

Different Organizational Challenges

Building Capacity: Districts/ Regions• Initial Implementation

• Build commitment (focus on valued outcomes)• Establish leadership team• Invest in Exemplars… but build capacity as you do this• Invest in building district capacity to

• Implement with fidelity• Implement with depth• Implement with breadth (scale)• Implement with sustainability

• Full Implementation• Use of evaluation data • Iterative commitment events.

Building Capacity: District/ Region

•Real implementation means providing the technical assistance to establish durable systems.

• Selection of Personnel• Training• Coaching• Performance Feedback• Data systems for effective decision-making• Problem solving by teams and administration• Effective engagement of families and community

Position Description

Faculty Evaluation

Annual Orientation

“Preference is given for individuals with demonstrated

knowledge and skill in implementation of school-wide

academic and behavior supports”

Building Capacity: District/ Region

• Data systems• Fidelity of implementation• Universal Screening• Progress Monitoring• Standardized student outcomes

• Stages of Implementation• Exploration• Installation• Initial implementation• Full implementation

Measure District Capacity

District Capacity Assessment www. Scalingup.org

Building Capacity: States• Lead with clarity

• Establish a leadership team with the goal of improving the capacity for implementation

• Implement to change the full system• Focus on a slice of the full system as your implementation target

• Guide adoption of practices• Define core features expected in schools

• Align initiatives to avoid competition and conflict• Braid initiatives at the point of common budget

• Provide that data systems needed for capacity development• Fidelity, and Impact at the school level• Implementation capacity at the district level

• Invest in functional capacity for implementation• Training, Coaching, Evaluation, Technical Expertise

Building Capacity: States• State Implementation Stages

• Exploration• Initial Implementation (Exemplars).

• Scaling paper (100-200 schools)• Evaluation data

• Reinvestment• State capacity• Evaluation data

• Policy change

State

District/ Region

Schools

Building Capacity: States• 1. Selection of effective practices

• 2. Establish expectations• Schools should create a coherent social culture that promotes

learning.• Students should graduate with academic AND social skills

• 3. Establish iterative improvement system• Report on social culture of school

• 4. Build the training, coaching and evaluation capacity at the state level.

• 5. Align initiatives and expectations to promote efficiency and outcomes.

Oregon Promising Practices•

• Standard Operating Procedure: Promising Practices• • Promoting Educational Effectiveness in Oregon:• Standard Operating Procedure for Identifying and

Implementing Educational Innovations• --------------------------------------------------------------------------• Practices may be (a) Standard, (b) Emerging, (c) Scaleworthy

or (d) Not recommended

Criteria for Selection an Educational Practice:

1.Practice addresses a major educational goal

2.Procedures are operationally defined

3.Practice include a professional development protocol

4.Practice include a measure of fidelity and procedures for

improvement5.Practice has been validated as effective in a peer-reviewed

publication6.Practice has been demonstrated as feasible and effective in

at least 50 schools in Oregon7.Practice is documented to as, or more efficient than current

alternatives.

Cascade of Competence

Schools

Districts

Regions

State

State Conferences

State Dept Trainers

Local Content Specialists

National Trainers

District/Regional Trainers

Local Coaching

Evaluation/Strategic Planning

Alignment: Align at the common budget point

Effective Procedur

e

Effective Procedur

e

Effective Procedur

e

Effective Procedur

e

Core Features

Valued Outcomes

Core Features

Core Features

Alignment

Teacher Effectiveness

PBISSchool-

wide Support

Restorative Practices

Define and teach

positive behavior

Appropriate Classroom Behavior

1. Expectations2. Recognition

3. Consequences4. Data System

Consequences1. Questions

2. Restore3. Teach

Building a Coherent Decision System

• Building State capacity to gather information• Documenting outcomes for students• Documenting fidelity• Documenting capacity

School-level decision-making

Individual Decision-systems

Cum

ulati

ve M

ean

OD

Rs

Cumulative Mean ODRs Per Month for 325+ Elementary Schools 08-09

PBIS Tiered Fidelity Inventory• Assesses PBIS implementation at all three tiers.

Assessing Capacity• www.sisep.org• State Capacity Assessment• District Capacity Assessment

An Example

Implementation Fidelity (SET)Elementary and Middle 2009-10

Implementation Fidelity (SET)By Factor for Elem and Middle 2009-10

Summary• Implementation at scale is possible

• Consider the cluster of core features needed for scaling• Admin support, Technical capacity, 100-200 demonstrations• Small demonstrations may be necessary but insufficient• Build in system for adapting the program to fit the local context while

retaining the core features.

• Consider an implementation plan with established procedures for improving efficiency of implementation

• Measure fidelity of implementation as a part of effective practice.

• Sustained implementation requires continuous regeneration

• Always emphasize, measure and report on valued outcomes

Reflection• Schools

• 1. Do we have a regular way to assess if we are using PBIS?• 2. Do we have a regular way to assess if we are benefiting students• 3. Do we have clear expectations for the District/Regions

• District• 1. Does our district have the “capacity” to select and implement effective

practices…. Like PBIS.• 2. Does our district have the capacity to sustain effective practices (data ,

training, coaching, evaluation)

• State/ Commonwealth (build district capacity…• 1. Do we have a way to help districts/ state offices select effective practices and

align federal/state initiatives?• 2. Do we have a way to Train/Coach/ Evaluate across the three types of

districts/regions.• 3. Do we have “Decision Systems” that promote implementation and

improvement.