Embed Size (px)
Citation preview
Iden%fica%on and Interven%on for Tier 2 Using the Social, Academic,
and Emo%onal Behavior Risk Screener (SAEBRS)
Barbara Mitchell, Ph.D. Steve Kilgus, Ph.D.
University of Missouri October, 2015
Session Outcomes • By the end of this session you will be able to… – Iden%fy poten%al benefits of using a universal screening instrument.
– Choose a screening instrument that is appropriate and relevant for your seQng & context.
– Create a process for regularly using data based methods to iden%fy students who need addi%onal supports.
What Do We Know?
• Approximately 1 in every 4 to 5 youth in the U.S. meets criteria for a mental disorder with severe impairment across their life%me (Merikangas et al., 2010).
• Among those affected only 30% actually receive services (U.S. Public Health Service, 2000)
What Do We Know?
• The most common condi%ons include – Anxiety disorders (31.9%) – Behavior disorders (19.1%) – Mood disorders (14.3%) – Substance use disorders (11.4%)
• Approximately 40% of individuals meet criteria for mul%ple disorders.
(Merikangas et al., 2010)
What Do We Know? • The median age of onset occurs during school-‐age years – 6 years for anxiety – 11 years for behavior – 13 years for mood – 15 years for substance use disorders.
(Merikangas et al., 2010)
What Do We Know?
• Academic success is linked with social & behavioral skills
• Early iden%fica%on with interven%on can decrease the likelihood of academic failure – Prevent onset
• Preven%ve supports reduce the need for more intensive supports later. – Minimize impact of risk
The Good News
• There is great poten%al to reduce the number of new cases of SEB disorders and improve the lives of youth who experience these challenges.
We are limited only by inefficient and ineffective systems!
(NRC & IOM, 2009, p. 16)
Three Levels of Implementa%on A Con%nuum of Support for All
Tier One • All students • Preven%ve, proac%ve
Tier One • All seQngs, all students • Preven%ve, proac%ve
Tier Two • Some students (at-‐risk) • High efficiency • Rapid response
Tier Two • Some students (at-‐risk) • High efficiency • Rapid response
Tier Three • Individual Students • Assessment-‐based • High Intensity
Tier Three • Individual Students • Assessment-‐based • Intense, durable procedures
Academic Systems Behavioral Systems
Who is At-‐Risk? Externalizing Behaviors • Aggression to others or things • Hyperac%vity • Non-‐compliance • Disrup%ve • Arguing • Defiance • Stealing • Not following direc%ons • Calling out
Who is At-‐Risk?
Internalizing Behaviors • Exhibits unusual sadness • Sleeps a lot • Is teased or bullied by peers • Does not par%cipate in games • Very shy or %mid • Acts fearful • Does not stand up for self • Withdrawn • Avoids social interac%ons
Tier 2 Student Iden%fica%on
Tiered Fidelity Inventory (TFI) • Item 2.3 Screening:
• Tier II team uses decision rules and mul%ple sources of data (e.g., ODRs, academic progress, screening tools, aeendance, teacher/family student nomina%ons) to iden%fy students who require Tier II supports.
• pbis.org
Tier 2 Student Iden%fica%on
Benchmarks for Advanced Tiers (BAT) • Item 7. The school uses a data based process for iden%fying students who may need Tier 2 and Tier 3 supports.
• Screening • Request for Assistance, Nomina%on • Progress Monitorin , ODRs • Other
• pbis.org
Iden%fica%on Process • Teacher nomina%on/referral – brief, rapid access
• Exis%ng school data – ODR, classroom minors, aeendance, nurse/counselor visits
• Universal screening instrument
Allows for early interven0on? Iden0fies internalizing & externalizing?
Using a Screening Instrument Advantages • Fast, efficient, and
respecjul • Include all children and
youth of interest • If we make an error, the
error tends to iden%fy students who are not at-‐risk
• Informs schools about the student popula%on
• Find groups of students with common needs
• Facilitates resource mapping of services
Poten<al Problems • Behavior is oken viewed as
purposeful rather than environmental
• Reac%ve rather than proac%ve with respect to behavior
• Impression that kids will “grow out of it”
• Concern about profiling/s%gma%zing
• Fear of costs and poten%al to iden%fy large # of students
• Systems skill set
School-‐Based Preven%on
• What Works … • Building school capacity to ini%ate and sustain an interven%on
• Communica%ng and consistently enforcing behavioral norms
• Comprehensive social skills instruc%onal programs – self-‐control, stress-‐management, responsible decision-‐making, social problem-‐solving, and communica%on skills
(University of Oregon Ins%tute on Violence and Destruc%ve Behavior)
Screening Instruments
• Systema%c Screening for Behavior Disorders (SSBD; Walker & Severson, 1990)
• Strengths & Difficul%es Ques%onnaire (SDQ; Goodman, 2001)
• Behavioral & Emo%onal Screening System (BESS; Kamphaus & Reynolds, 2007)
• Social, Academic, and Emotional Behavior Risk Screener (SAEBRS; Kilgus, von der Embse, Chafouleas, & Riley-‐Tillman 2014)
SAEBRS Overview
Social, Academic, & Emo<onal Behavior Risk Screener (SAEBRS)
• Brief, 19-‐item teacher ra%ng scale • “Teacher ra%ng of all students on common behavioral criteria” (Severson et al., 2007)
• One broad scale and three subscales – Total Behavior (19 items) – Social Behavior (6 items) – Academic Behavior (6 items) – Emo%onal Behavior (7 items)
SAEBRS
• Intended for use in surveillance of both protec<ve and risk factors
• Subscales = domains of func<oning – Items = con%nuum of behavior – Maladap<ve à Adap<ve
• Items = sample from the universe of item content – Do not represent the en%rety of behavior within each domain
SAEBRS Conceptual Map
Total Behavior
Social Behavior
Academic Behavior
Externalizing Problems Social Skills Aeen%onal
Problems Academic Enablers
Emo%onal Behavior
Internalizing Problems
Emo%onal Competencies
Social Behavior
• Behaviors that promote or limit a student’s ability to maintain age appropriate rela%onships with peers and adults
Social Skills Externalizing Problems
Coopera%on with peers Arguing
Polite and socially appropriate respones toward others
Temper outbursts
Disrup%ve behavior
Impulsiveness
Academic Behavior
• Behaviors that promote or limit one’s ability to be prepared for, par%cipate in, and benefit from academic instruc%on
Academic Enablers AJen<on Problems
Interest in academic topics Difficulty working independently
Preparedness for instruc%on Distractedness
Produc%on of acceptable work
Academic engagement
Emo:onal Behavior
• Ac%ons that promote or limit one’s ability to regulate internal states, adapt to change, and respond to stressful/challenging events
Emo<onal Competencies Internalizing Problems
Adaptable to change Sadness
Posi%ve aQtude Fearfulness
Worry
Difficulty rebounding from setbacks
SAEBRS Evidence • Elementary, middle, and high school levels • Urban and rural • Southeast, southwest, midwest • Series of studies
– 1Kilgus, Chafouleas, & Riley-‐Tillman, 2013 – 1Kilgus, Sims, von der Embse, & Riley-‐Tillman, 2015 – 2Kilgus, Sims, von der Embse, & Taylor, in press – 2von der Embse, Pendergast, Kilgus, & Eklund, in press – 3Bowman, Kilgus, & Christ, under review – 3Kilgus, Eklund, von der Embse, & Taylor, under review – 3Pendergast, von der Embse, Kilgus, & Eklund, under review – 3Eklund, Kilgus, von der Embse, Beardmore, & Tanner, under review
Reliability Internal Consistency Inter-‐Rater
Social Behavior .89-‐.94 .41
Academic Behavior .90-‐.93 .47
Emo%onal Behavior .77-‐.83 -‐-‐
Total Behavior .93-‐.94 .48
Validity SB AB EB TB
SSIS .82-‐.90 .69-‐.76 .61-‐.89
BESS .79-‐.85 .86-‐.88 .69-‐.75 .93-‐.94
SRSS -‐.84 -‐.74 -‐.61 -‐.84
SIBS -‐.50 -‐.50 -‐.77 -‐.67
ODRs -‐.54 -‐.33 -‐.24 -‐.42
ISS/OSS .28-‐.39 .19-‐.27 .06-‐.07 .24-‐.24
ORF .31 .48 .28 .41
SWAT .30-‐.34 .52-‐.52 .28-‐.32 .45-‐.45
*Logis%c regression analyses suggest unique individual contribu%on of each subscale to the predic%on of outcomes
Diagnos%c Accuracy • Evidence collected across 4 studies • Outcomes – Social Skills Improvement System (SSIS) Ra%ng Scales – BASC-‐2 Behavioral and Emo%onal Screening System (BESS)
– Student Risk Screening Scale (SRSS) – Student Internalizing Behavior Screener (SIBS)
• Consistency in cut score recommenda%ons – Across studies – Across grade levels – Across %me within a year
At-‐Risk Students (Sensi%vity)
81
91 90 90
19
9 10 10
0
10
20
30
40
50
60
70
80
90
100
Social Academic Emo%onal Total
Percen
t of S
tude
nts
Missed
Detected
Kilgus, Eklund, von der Embse, & Taylor, under review
Goal = .80 (Carran & Scoe, 1992; Metz, 1978; Petscher et al., 2011)
86 84
73
93
14 16
27
7
0
10
20
30
40
50
60
70
80
90
100
Social Academic Emo%onal Total
Percen
t of S
tude
nts
Missed
Detected
Not At-‐Risk Students (Specificity)
Goal = .70 (Hintze & Silberglie, 2005; Kilgus et al., 2014)
Kilgus, Eklund, von der Embse, & Taylor, under review
SAEBRS Interpreta%on & Use
Preparing to Screen
• Iden%fy key players – Data collectors, data analyzers, problem solving team
• Get staff on board – Clarify screening’s role within PBIS model – Be explicit regarding how screening data will connect to preven%on/interven%on efforts
– Emphasize teacher role (observers and catalysts) • Community outreach – Talk to parents – Coordinate with outside service providers
Administra%on • Parental opt-‐out process (Chafouleas, Kilgus, & Wallach, 2010) – Being used to inform decisions with general educa%on seQng
– Teachers (not students) are responsible for ra%ng observable behavior
• Iden%fy %me and seQng – One hour – computer lab session – One day – rota%ons with subs%tute teacher coverage – One week – at leisure or during planning %me
• Teacher completes for each student – 1-‐3 minutes per student
The SAEBRS form was created by Stephen P. Kilgus, Sandra M. Chafouleas, T. Chris Riley-Tillman, and Nathaniel P. von der Embse Copyright © 2013 by Stephen P. Kilgus. All rights reserved. Permission granted to photocopy for personal and educational use as long as the names of the creators and the full copyright notice are included in all copies.
Social, Academic, & Emotional Behavior Risk Screener
Teacher Rating Scale
Your Name:
Student Date of Birth:
Student Name:
Student Grade:
Today’s Date:
Using the following scale, identify how frequently the student has displayed each of the following behaviors during the previous month. Circle only one number for each behavior.
0 = Never, 1 = Sometimes, 2 = Often, 3 = Almost Always
Social Behavior
Arguing 0 1 2 3
Cooperation with peers 0 1 2 3
Temper outbursts 0 1 2 3
Disruptive behavior 0 1 2 3
Polite and socially appropriate responses toward others 0 1 2 3
Impulsiveness 0 1 2 3 Academic Behavior
Interest in academic topics 0 1 2 3
Preparedness for instruction 0 1 2 3
Production of acceptable work 0 1 2 3
Difficulty working independently 0 1 2 3
Distractedness 0 1 2 3
Academic engagement 0 1 2 3
The SAEBRS form was created by Stephen P. Kilgus, Sandra M. Chafouleas, T. Chris Riley-Tillman, and Nathaniel P. von der Embse Copyright © 2013 by Stephen P. Kilgus. All rights reserved. Permission granted to photocopy for personal and educational use as long as the names of the creators and the full copyright notice are included in all copies.
Emotional Behavior
Sadness 0 1 2 3
Fearfulness 0 1 2 3
Adaptable to change 0 1 2 3
Positive attitude 0 1 2 3
Worry 0 1 2 3
Difficulty rebounding from setbacks 0 1 2 3
Withdrawal 0 1 2 3
Scoring
• Score items (while reverse scoring nega%ve items)
• Sum item scores within each scale – SB (0-‐18) – AB (0-‐18) – EB (0-‐21) – TB (0-‐57)
Posi<ve Items
Nega<ve Items
Never 0 3
Some%mes 1 2
Oken 2 1
Almost Always 3 0
Interpreta%on At-‐Risk Not At-‐Risk
SB 0-‐12 13-‐18
AB 0-‐9 10-‐18
EB 0-‐17 18-‐21
TB 0-‐36 37-‐57
• TB score takes precedence in student iden%fica%on – Students who receive Tier 2 services
1. Iden%fy at risk students(TB)
2. Look within subscales to determine area(s) of need
FastBridge Learning
FBL Administra%on
FBL Individual Report
FBL Group Report
SAEBRS Informing Interven%on
Determine the level at which to implement interven%on
(Kilgus & Eklund, 2015)
Universal Screening
School-‐wide Base Rate < 20%,
but Classroom Base Rate ≥ 20%
School-‐wide Base Rate < 20% & Classroom Base Rate ≤ 20%
School-‐wide Base Rate ≥ 20%
System Support (Tier 1)
Classroom Support (Tier 1)
Individual/Small Group Support
(Tier 2)
System Support (Tier 1) School-‐wide Base Rate > 20%
• Two op%ons – Increase fidelity of current universal plan
– Revise universal plan • Use SAEBRS subscale data to inform Tier 1 recommenda%ons – Specific to domains of concern
School-‐wide Base Rate ≥ 20%
System Support (Tier 1)
System Support (Tier 1) School-‐wide Base Rate > 20%
• Social Behavior – Review and revision of school-‐wide expecta%ons or reinforcement plan (ensure integrity)
• Academic Behavior – Connect screening data to academic data – Consider school-‐wide instruc%on of enablers
• Emo<onal Behavior – Adopt school-‐wide social-‐emo%onal learning curriculum
Classroom Support (Tier 1) Classroom Base Rate > 20%
School-‐wide Base Rate < 20%,
but Classroom Base Rate ≥ 20%
Classroom Support (Tier 1)
Teacher Grade
# of students screened
# of students at-‐risk At-‐ Risk
Shaffer 5 25 14 56% Triggs 4 26 13 50% Ells 2 26 7 27%
Memphis 1 28 7 25% Barree 2 25 5 20% Cassidy 4 21 4 19% Ulrich 4 28 5 18%
Classroom Support (Tier 1) Classroom Base Rate > 20%
• Social Behavior – Support teacher classroom management
• Classroom Checkup (Reinke, Herman, & Sprick, 2011) – Group con%ngency interven%ons
• Good Behavior Game • Academic Behavior – Supports teacher instruc%onal prac%ces (e.g., environmental structure, instruc%onal pacing)
– Explicit instruc%on of various academic enablers (e.g., organiza%on, engagement, note taking)
• Emo<onal Behavior – Classroom specific adop%on of SEL curricula
Individual/Group Support (Tier 2) Classroom Base Rate < 20%
School-‐wide Base Rate < 20% &
Classroom Base Rate ≤ 20%
Individual/Small Group Support
(Tier 2)
Teacher Grade
# of students screened
# of students at-‐risk At-‐ Risk
Franks 10 29 5 17% Garree 11 21 3 14% Hollister 9 26 3 12% Innings 12 23 2 9% Vargas 12 24 2 8% Williams 12 27 2 7% Norton 9 21 1 5% Jenkins 11 22 1 5% Kasper 12 24 1 4%
Individual/Group Support (Tier 2) Classroom Base Rate < 20%
Con<ngency Management Skill Instruc<on
Social Behavior CICO or CCE Social Skills Instruc%on (e.g., SSIG
Academic Behavior Academic Behavior CICO (Turtura et al., 2014)
Academic Enablers Instruc%on (e.g., AIMS);
Homework Club
Emo<onal Behavior Internalizing-‐oriented CICO (Dart et al., 2015)
Social-‐Emo%onal Learning (e.g., Strong Kids)
Linking to Other Assessment
Problem Analysis
Universal Screening
Skill Assessment
Brief FBA
Tier 2 Interven<on
Skill Instruc%on
Con%ngency Management
Progress Monitoring
*Goal – support Tier 2 modifica0on and thus more effec0ve interven0on
Problem Analysis • Brief FBA – Ex. Func%onal Assessment Checklist for Teachers and Staff (FACTS)
• Skill Assessment (iden%fy acquisi%on and performance deficits) – Social Behavior
• SSIS Ra0ng Scales – Academic Behavior
• Academic Competence Evalua0on Scales (ACES) – Emo%onal Behavior
• Devereux Student Strengths Assessment (DESSA)
Progress Monitoring • Mul%ple op%ons – Interven%on permanent products (e.g., Daily Progress Reports)
– Direct Behavior Ra%ngs (DBR) • Directbehaviorra%ngs.org
• FastBridge DBR currently under development (Winter 2016 release)
• Track general outcome measures (GOMs) of behavioral func%oning – Disrup%on Behavior – Academic Engagement – Respecjul Behavior
Why use the SAEBRS?
• Focus on both posi%ve and nega%ve behaviors – Promotes a focus on prosocial behavior as well
• Affords mul%ple scores, including scales and subscales – Implica%ons for interven%on
• Brief – Shorter than several alterna%ve screeners
• Strong psychometrics – Extent of diagnos%c accuracy evidence rivals other screeners
Future Direc%ons
• Mul%ple ga%ng procedure – Gate 1 = Teacher nomina%on (3 forms) – Gate 2 = SAEBRS (Total Behavior score)
Sensi<vity Specificity PPV NPV
Study 1 Elementary .83 .95 .73 .97
Middle .79 .95 .72 .96
Study 2 Elementary .70 .96 .72 .95
Middle .81 .96 .73 .97
Kilgus, Eklund, von der Embse, & Taylor, under review
Data Integra%on
• Tradi%onal vs. Preven%on-‐Oriented Screening • Already collec%ng data on – Aeendance • Days absent, tardies, # of moves
– Academic outcomes • Growth on CBM’s • Benchmark assessment data • Standardized test scores (AIMS) • Grades
– Office Discipline Referrals
Session Outcomes • By the end of this session you will be able to… – Iden%fy poten%al benefits of using a universal screening instrument.
– Choose a screening instrument that is appropriate and relevant for your seQng & context.
– Create a process for regularly using data based methods to iden%fy students who need addi%onal supports.
Student Iden%fica%on Process
• Teacher nomina%on • Exis%ng school data • Universal screening instrument
Allows for early interven0on?
Iden0fies internalizing & externalizing?
Screening Implementa%on Examples • Parents complete ra%ng ques%onnaire during Kindergarten registra%on
• Parents and/or students complete ra%ngs when new family registers for school
• Incoming 9th graders complete screening ques%onnaire when they create course schedule; risk scores used to assign advisory courses
• Classroom teachers complete screenings in the spring prior to student transi%on to new building
30% At Risk (consider for Tier 2/3 support)
Goal 15-‐20%
70% Not At Risk Goal 80-‐90%
Of those who were screened approximately 75 students scored in “At-‐Risk” range for social, academic or behavioral concerns
South Elementary -‐ SAEBRS
Screening Results – Fall 2015
Systems of Support for Teachers
-‐ In Class Coaching
- On-going Feedback
- Goal Setting
-‐ Data-Based Performance Feedback
- Peer Observations & Data Collection
- Professional Development
Long-‐Term Goals:
• Internally delivered • Team based approach
• Available for all
• Provided proac%vely • Posi%ve, non-‐evalua%ve support
Contact Informa%on
• Barbara Mitchell, [email protected]
• Steve Kilgus, [email protected]
• Websites – hep://www.pbis.org/ – hep://pbismissouri.org – hep://ebi.missouri.edu/
