Using Information for Decision Making Identifying Interventions that support Targeted and Intensive...

Using Information for Decision Making

Identifying Interventions that support Targeted and Intensive Students

Susan Barrettwww.pbis.org

www.pbismaryland.org

Monitor Objectives & Outcomes

• Big Idea: – The staff determine what they want to

answer, what data will answer the question, the simplest way to get that data, and then write an objective for where they want to be in the future.

Questions

How does the team use to data to:

• Get 80% staff buy-in (staff survey, TIC)

• Keep 80% buy-in (BIG 5, communication)

• for problem solving across all teams/committees

• Utililize the three-tiered model of prevention (integrate student services, academic support, teacher support)

SYSTEMS School wide Non classroom Classroom Individual students

PRACTICES define (behav. expect. & routines) teach acknowledge correct follow up & feedback consensus & collaboration

INFORMATION office discipline referrals PBS self assessment survey all staff (parents/students?) input academic progress attendance direct observation school improvement goal progress

SupportingStaff Behavior

SupportingDecisionMaking

SupportingStudent Behavior

Positive Behavior Support

Why Collect Discipline Information?

• Decision making

• Professional Accountability

• Decisions made with data (information) are more likely to be (a) implemented, and (b) effective

Key features of data systems that work.

• The data are accurate• The data are very easy to collect (1% of staff

time)• Data are used for decision-making

– The data must be available when decisions need to be made (weekly?)

– Difference between data needs at a school building versus data needs for a district

– The people who collect the data must see the information used for decision-making.

What data to collect for decision-making?

• USE WHAT YOU HAVE– Office Discipline Referrals/Detentions

• Measure of overall environment. Referrals are affected by (a) student behavior, (b) staff behavior, (c) administrative context

• An under-estimate of what is really happening

• Office Referrals per Day per Month

– Attendance– Suspensions/Expulsions– Vandalism

Office Discipline Referral Processes/Form

• Coherent system in place to collect office discipline referral data– Faculty and staff agree on categories– Faculty and staff agree on process– Office Discipline Referral Form includes needed

information• Name, date, time• Staff• Problem Behavior, maintaining function• Location

Observe Problem Behavior

Warning/Conference with Student

Use Classroom Consequence

Complete Minor Incident Report

Does student have 3 MIR slips

for the same behavior in the same quarter

•Preparedness•Calling Out•Classroom Disruption•Refusal to Follow a Reasonable Request (Insubordination)•Failure to Serve a Detention•Put Downs•Refusing to Work•Inappropriate Tone/Attitude•Electronic Devices•Inappropriate Comments•Food or Drink

•Weapons•Fighting or Aggressive Physical Contact•Chronic Minor Infractions•Aggressive Language•Threats•Harassment of Student or Teacher•Truancy/Cut Class•Smoking•Vandalism•Alcohol•Drugs•Gambling•Dress Code•Cheating•Not w/ Class During Emergency•Leaving School Grounds•Foul Language at Student/Staff

Write referral to office

Administrator determines

consequence

Administrator follows through

on consequence

Administrator provides teacher

feedback

Write the student a

REFERRAL to the main office

•Issue slip when student does not respond to pre-correction, re-direction, or verbal warning

•Once written, file a copy with administrator

•Take concrete action to correct behavior (i.e. assign detention, complete behavior reflection writing, seat change)

SIDE BAR on Minor Inc ident Repor t s

•Issue slip when student does not respond to pre-correction, re-direction, or verbal warning

•Once written, file a copy with administrator

•Take concrete action to correct behavior (i.e. assign detention, complete behavior reflection writing, seat change)

SIDE BAR on Minor Inc ident Repor t s

Is behavior office

managed?

ClassroomManaged

Office Managed

No Yes

Minor Incident Reports

• Overall Design

– Smaller than referral

– In triplicate

– Replaced Detention Forms

• Pre-Referral Documented Step

• Flexible Consequences

– Ex: Detention, reflection, parent signature, etc.

• Administrative Intervention Before Referral

When Should Data be Collected?

• Continuously

• Data collection should be an embedded part of the school cycle not something “extra”

• Data should be summarized prior to meetings of decision-makers (e.g. weekly)

• Data will be inaccurate and irrelevant unless the people who collect and summarize it see the data used for decision-making.

Organizing Data for “Information”

• Counts are good, but not always useful

• To compare across months use “average office discipline referrals per day per month”

0

10

20

30

40

50

60

70

Refe

rrals

Aug Sept Oct Nov Dec Jan Feb Mar Apr May Jun

School Month

Total Ref versus Ref/Day/MoNV High School

0

1

2

3

4

5 M

ean R

efe

rrals

per

Day

Aug Sept Oct Nov Dec Jan Feb Mar Apr May Jun

School Month

Total Ref versus Ref/Day/Mo

Using Data for On-Going Problem Solving

• Start with the decisions not the data

• Use data in “decision layers”– Is there a problem? (overall rate of ODR)– Localize the problem

– (location, problem behavior, students, time of day)

– Get specific

• Don’t drown in the data

• It’s “OK” to be doing well

• Be efficient

Is there a problem?

• Office Referrals per Day per Month

• Attendance

• Faculty Reports

Interpreting Office Referral Data:Is there a problem?

• Absolute level (depending on size of school)– Middle Schools (>5 per day)– Elementary Schools (>1.5-2 per day)

• Trends– Peaks before breaks?– Gradual increasing trend across year?

• Compare levels to last year– Improvement?

0

0.5

1

1.5

2

Ave R

efe

rrals

per

Day

Sept Oct Nov Dec Jan Feb Mar Apr May Jun

School Months

Office Referrals per Day per MonthLast year

0

5

10

15

20

Ave R

efe

rrals

per

Day

Sept Oct Nov Dec Jan Feb Mar Apr May Jun

School Months

Office Referrals per Day per MonthLast year

0

5

10

15

20

Ave R

efe

rrals

per

Day

Sept Oct Nov Dec Jan Feb Mar Apr May Jun

School Months

Office Referrals per Day per MonthLast year

0

5

10

15

20

Ave R

efe

rrals

per

Day

Sept Oct Nov Dec Jan Feb Mar Apr May Jun

School Months

Office Referrals per Day per MonthThis Year

0

5

10

15

20

Ave R

efe

rrals

per

Day

Sept Oct Nov Dec Jan Feb Mar Apr May Jun

School Months

Office Referrals per Day per MonthLast Year and This Year

0

5

10

15

20

Ave R

efe

rrals

per

Day

Sept Oct Nov Dec Jan Feb Mar Apr May Jun

School Months

Office Referrals per Day per MonthLast Year and This Year

Is There a Problem? #1Maintain - Modify - Terminate

0

5

10

15

20

Ave R

efe

rrals

per

Day

Sept Oct Nov Dec Jan Feb Mar Apr May

School Months

Office Referrals per Day per MonthThis Year

Is There a Problem? #4Maintain - Modify - Terminate

0

5

10

15

20

Ave R

efe

rrals

per

Day

Sept Oct Nov Dec Jan Feb Mar Apr May Jun

School Months

Office Referrals per Day per MonthLast Year and This Year

What systems are problematic?• Referrals by problem behavior?

– What problem behaviors are most common?

• Referrals by location?– Are there specific problem locations?

• Referrals by student?– Are there many students receiving referrals or only a

small number of students with many referrals?

• Referrals by time of day?– Are there specific times when problems occur?

Referrals by Problem Behavior

0

10

20

30

40

50

Num

ber

of R

efe

rrals

Lang Achol ArsonBombCombsDefianDisruptDressAgg/fgtTheftHarassProp D Skip Tardy Tobac Vand Weap

Types of Problem Behavior

Referrals per Prob Behavior

Referrals by Problem Behavior

0

10

20

30

40

50

Num

ber

of R

efe

rrals

Lang Achol ArsonBombCombsDefianDisruptDressAgg/fgtTheftHarassProp D Skip Tardy Tobac Vand Weap

Types of Problem Behavior

Referrals per Prob Behavior

Referrals per Location

0

10

20

30

40

50

Num

ber

of O

ffic

e R

efe

rrals

Bath RBus A Bus Caf ClassComm Gym Hall Libr Play G Spec Other

School Locations

Referrals by Location

0

10

20

30

40

50

Num

ber

of O

ffic

e R

efe

rrals

Bath RBus A Bus Caf ClassComm Gym Hall Libr Play G Spec Other

School Locations

Referrals by Location

Referrals per Student

0

10

20

Num

ber

of R

efe

rrals

per

Stu

dent

Students

Referrals per Student

0

10

20

Num

ber

of R

efe

rrals

per

Stu

dent

Students

Students per Number of Referrals

Referrals by Time of Day

0

5

10

15

20

25

30

Num

ber

of R

efe

rrals

7:00 7:30 8:00 8:30 9:00 9:30 10:0010:3011:00 11:3012:0012:30 1:00 1:30 2:00 2:30 3:00 3:30

Time of Day

Referrals by Time of Day

Referrals by Time of Day

0

5

10

15

20

25

30

Num

ber

of R

efe

rrals

7:00 7:30 8:00 8:30 9:00 9:30 10:0010:3011:00 11:3012:0012:30 1:00 1:30 2:00 2:30 3:00 3:30

Time of Day

Referrals by Time of Day

Decision-making

• Is there a problem?

• What areas/systems are involved?

• Are there many students or a few involved?

• What kinds of problem behaviors are occurring?

• When are these behaviors most likely?

• What is the most effective use of our resources to address this problem?

Designing Solutions

• If many students are making the same mistake it typically is the system that needs to change not the students.

• Teach, monitor and reward before relying on punishment.

• An example (Kartub et al, JPBI, 2000)

IF...

FOCUS ON...

More than 40% of students receive one or more office

referrals More than 2.5 office referrals per student

School Wide System

More than 35% of office referrals come from non-classroom settings

More than 15% of students referred from non-classroom settings

Non-Classroom System

More than 60% of office referrals come from the classroom

50% or more of office referrals come from less than 10% of classrooms

Classroom Systems

More than 10-15 students receive 5 or more office referrals

Targeted Group Interventions / Classroom Systems

Less than 10 students with 10 or more office referrals Less than 10 students continue rate of referrals after

receiving targeted group settings Small number of students destabilizing overall

functioning of school

Individual Student Systems

Targeted Group Supports

• For those students who exhibit difficulties despite proactive school-wide prevention efforts

• Likely to be student with both academic & behavioral challenges

• Approximately 10% of school population

Remember…

Without school-wide prevention,

we cannot reliably identify

targeted-level students.

Halls Ferry Elem entary School

YEAR 1 YEAR 2 YEAR 3 YEAR 4 YEAR 5

High Fiv e Approach - school wide social skill lessonsCentral Data SystemProduced school-wide expectations v ideoCafeteria routine and lessons

Playground routine and lessonsProduced bathroom expectations & routines videoProduced indoor recess expectation & routines video

New comer's ClubStudy Skills - Homew ork Support

Social Skills Club

FBA TrainingCoaches T raining

MentoringChamps Theater

Function-BasedStudent SupportTeam

ClassroomBus expectations

Present to Board

District w ide w eb based data systemDistrict level leadership teamDistrict level collaborative teamDistrict w ide netw orking system

Universal School-w ide Systems

Secondary / Targeted Group

Tertiary / Individual Student

District Level Systems

Using Data to Select Interventions

Who are the students who need additional support? (referrals by student, targeted:2-5 referrals, intensive: 6 or more referrals)

What are the interventions? (link to SW)What is the referral process? (teacher support-quick, easy, )What is the time frame students get access to the

intervention? (72 hours)What is the process to determine function?• FACTS, student questionnaire, role of academics, hypothesis

statement, team process?

Does the SST have access to the data for ongoing monitoring of intervention?

~80% of Students

~15%

~5%

Tertiary Prevention:Interagency PartnershipsCommunity Partnerships

Court/ Institutional LiaisonsSpot Light On Schools/ P.O.s

Maryland’s Tomorrow ProgramHomeless ProgramNW/SW IEP Team

Individual Student IEP/ 504 PlansStudent Therapeutic SupportStudent Threat Assessments

Abuse and Neglect Reporting Program (CPS Liaisons)Traumatic Loss Teams

Secondary Prevention:Project Attend/ FACE

School Resource Officer ProgramD.A.R.E./ S.A.D.D. ProgramsStudent Assistance Programs

Wellness CentersHealth Action Plans/ Appraisals

Peer Helper/ Mediation ProgramsPupil Personnel Home Visits/ Social Histories

Instructional Support Teams (IST)Functional Behavior Assessments

Behavior Intervention Plans Educational/ Psychosocial Assessments

Psychological AssessmentsStudent Case Management

Social Skills, Anger Management, Conflict ResolutionParent Presentations/ Trainings

Alternative School/Program Placements

Primary Prevention:School Improvement Planning

School Emergency Safety PlansPositive Behavior Plans/ CodesCharacter Education Programs

Parent/ Family ResourcesStudent Support Teams

Health Screenings/ ProtocolsSchool Nurse/ Assistant ServicesEssential Guidance Curriculum

School Counselor ServicesPupil Personnel Services

Residency/ Attendance Officer ServicesSchool Social Work ServicesSchool Psychologist Services

Baltimore County2004-2005

32 Schools: 14 ES, 12 MS, 6 HS

Programs PurposeExpected Outcome

Target Group Outcome Data (Is it working?)

Recommendations:1. What programs groups can we eliminate?2. What programs groups can we combine?3. What programs groups need to be supported for improved outcomes and sustained functioning?

Targeted and Intensive /Programs/InitiativesName of school: County: School Year 2005-06

Programs PurposeExpected Outcome

Target Group Outcome Data (Is it working?)

Homework Club

Recommendations:1. What programs groups can we eliminate?2. What programs groups can we combine?3. What programs groups need to be supported for improved outcomes and sustained functioning?

Targeted and Intensive /Programs/InitiativesName of school: County: School Year 2005-06

Conduct Brief Functional Assessment

Is the behavior maintained by escape

from social interaction?

Is the behavior related to lack of academic skills?

Is the behavior maintained by peer

attention?

Escape Motivated BEP

Reduce adult interaction

Use escape as a reinforcer

BEP + Academic Support

Increase academic support

Peer Motivated BEP

Allow student to earn reinforcers to share with peers

Horner, Hawken, Marsh

D A T A

S W IS o r C e ntra lD a ta S y s te m

S y s te m a ticD a ta E ntry

E ffic ie nt R e fe rra lF o rm

C o m pre he ns iv e

M u ltip le D a taS o u rc e s to Id e ntifyS e c o nd a ry /T e rtia ry

T eac her referralS c reening, etc .

O ffic e Disc ip lineR eferrals

T ime-o ut,Bud d y R o o ms , etc .

Detentio n

IS S

G rap hicd emo ns tratio n

M ultip le teammemb ers trained

M ultip le rep o rts(d aily average)

M atc hes Data Entry

C o mp rehens ive

C hec klis t

Defined p ro c ess

Daily

L. Newcomer

Summary of PBIS “BIG IDEAS” Systems (How things are done) Team based problem solving Data-based decision making Long term sustainability

Data (How decisions are made) On going data collection & use ODR’s (# per day per month, location, behavior, student) Suspension/expulsion, attendance, tardies

Practices (How staff interact with students) Direct teaching of behavioral expectations On-going reinforcement of expected behaviors Functional behavioral assessment

Using Data

• School-Wide– What’s happening?– Are we meeting our goals?– Are we doing what we said we would?

• Individual Students– What’s happening?– What do we do next?– Are we doing what we said we would?

School-Wide Analysis:What’s Happening

• Can we predict our problems?

• Why are they occurring?

• What can we do to prevent?

• Is it working?

School-Wide Data Analysis:What are our Goals?

• What do we want?

• Is it happening?

• If Yes - what next?

• If No - what next?

School-Wide Analysis:Are we Doing it?

• Are we doing what we said we would?

• What are our barriers?

• What do we have to do to make it work?