Systematic Screenings for Behavior Screening …...2013/06/07 · 5/2/2013 Systematic Screenings for Behavior Screening Tools: Data Based Decisions… One Step Further Kathleen Lynne

5/2/2013

Systematic Screenings for Behavior Screening Tools:

Data Based Decisions… One Step Further

Kathleen Lynne Lane, Ph.D., BCBA-D University of Kansas Wendy Oakes, Ph.D.

Arizona State University

Agenda Comprehensive, Integrated, Three-Tiered (CI3T) Models of Prevention

Behavior Screening tools In CI3T Models

Data-informed Decision Making

1

5/2/2013

Goal: Reverse Harm Specialized Group Systems for Students At‐Risk

Goal: Prevent Harm School/Classroom‐Wide Systems for All Students, Staff, & Settings

Academic Behavioral Social

Comprehensive, Integrated, Three‐Tier Model of Prevention (Lane, Kalberg, & Menzies, 2009)

Tertiary Prevention (Tier 3)

Secondary Prevention (Tier 2)

Primary Prevention (Tier 1)

≈

≈

≈

PBIS Framework

Validated Curricula

Lane & Oakes

Goal: Reduce Harm Specialized Individual Systems for Students with High‐Risk

CI3T Training Series Project SUPPORT & INCLUDE

Lane & Oakes

1:Two‐Hour

Support Session

2: Full Day

Training Trainers

3: Two‐Hour

Support Session

4: Full Day

Training Trainers

5: Two‐Hour

Support Session

6: Full Day

Training Trainers

Primary Primary

Secondary Secondary Secondary Secondary Sessions

Tertiary Tertiary Tertiary Tertiary Tertiary Prevention Series

Primary Prevention Series

12/7/11 1/9/12 1/23/12 2/21/12 3/22/12 4/19/12

2/6/12 3/5/12

11/7/11 1/12/12 6/7/12

1/17/12 1/30/12 3/1/12 4/16/12

2

are available?

Lane & Oakes

5/2/2013

Essential Components of Primary Prevention Efforts

Systematic Screening Academic Behavior

Treatment Integrity

Social Validity

Lane & Oakes State of Tennessee DOE Technical Assistance Grant IRB # 090935

What screening tools

See Lane, Menzies, Oakes, and Kalberg (2012)

3

5/2/2013

Student Risk Screening Scale (Drummond, 1994)

The SRSS is 7-item mass screener used to identify students who are at risk for antisocial behavior.

Uses 4-point Likert-type scale: never = 0, occasionally = 1, sometimes = 2, frequently = 3

Teachers evaluate each student on the following items - Steal - Low Academic Achievement - Lie, Cheat, Sneak - Negative Attitude - Behavior Problems - Aggressive Behavior - Peer Rejection

Student Risk is divided into 3 categories Low 0 – 3 Moderate 4 – 8 High 9 - 21 (SRSS; Drummond, 1994)

4

Lane & Oakes

5/2/2013

Student Risk Screening Scale (Drummond, 1994)

SRSS Scoring 7 - Items are totaled for one score Score is compared to categories for risk

status

Oakes & Lane, 2012

At Risk Total Score = 9 - 21

Some Risk Total Score = 4 - 8

Low Risk Total Score = 0 - 3

5

5/2/2013

Student Risk Screening Scale Middle School Fall 2004 - Fall 2011

n = 12

n = 20

n = 507

Per

cent

age

of S

tude

nts

N=534 N=502 N=454 N=476N=477N=470 N=524 N= 539

Fall Screeners Lane & Oakes

SAMPLE DATA: SRSS Middle School Study 1: Behavioral & Academic Characteristics of SRSS Risk Groups

Variable Risk

Low (n = 422) M (SD)

Moderate (n = 51) M (SD)

High (n = 12) M (SD)

Significance Testing

ODR 1.50 (2.85)

5.02 (5.32)

8.42 (7.01)

L<M<H

In-School Suspensions

0.08 (0.38)

0.35 (1.04)

1.71 (2.26)

L<M<H

GPA 3.35 (0.52)

2.63 (0.65)

2.32 (0.59)

L>M, H M=H

Course Failures Lane & Oakes

0.68 (1.50)

2.78 (3.46)

4.17 (3.49)

L<M, H M=H

(Lane, Parks, Kalberg, & Carter, 2007)

6

5/2/2013

Elementary Level Externalizing AUC 0.952 Results: ROC Curves

1.0

AUC = 0.952

0.8

Sens

itivi

ty

0.6

0.4

0.2

0.0

0.0 0.2 0.4 0.6 0.8 1.0

1 - Specificity

Lane, K. L., Little, M. A., Casey, A. M., Lambert, W., Wehby, J. H., Weisenbach, J. L., & Phillips, A., (2009). A comparison of systematic screening tools for emotional and behavioral disorders: How do they compare? Journal of Emotional and Behavioral Disorders, 17, 93-105.

Elementary Level Internalizing AUC .802

Results: ROC Curves 1.0

AUC = .802

0.8

0.6

Sens

itivi

ty

0.4

0.2

0.0

0.0 0.2 0.4 0.6 0.8 1.0

1 - Specificity

7

student.

5/2/2013

STUDENT RISK SCREENING SCALE-IE CURRENTLY UNDER DEVELOPMENT

TEACHER NAME

0 = Never

Ste

al

Lie

, C

he

at,

Sn

ea

k

Beh

avio

r P

rob

lem

Pe

er

Re

jec

tio

n

Lo

w A

cad

emic

Ac

hie

vem

ent

Ne

ga

tive

Att

itu

de

Ag

gre

ssiv

e B

ehav

ior

Em

oti

on

ally

Fla

t

Sh

y; W

ith

dra

wn

Sad

; D

epre

sse

d

An

xio

us

Ob

sess

ive-

Co

mp

uls

ive

Be

hav

ior

Lo

nel

y

Se

lf-I

nfl

icts

Pa

in

1= Occasionally

2 = Sometimes

3 = Frequently

Use the above scale to rate each item for each

Student Name

Validation Study

(Lane & Oakes, 2012)

Convergent Validity: SRSS-E7, SRSS-I5, & SRSS-IE12 with the SSBD

Lane, K. L., Oakes, W. P., Harris, P. J., Menzies, H. M., Cox, M. L., & Lambert, W. (2012) Initial evidence for the reliability and validity of the Student Risk Screening Scale for internalizing and externalizing behaviors at the elementary level. Behavioral Disorders, 37, 99‐122.

Note. SSBD refers to the Systematic Screening for Behavior Disorders (Walker & Severson, 1992). SRSS-IE5 refers to the version with 5 times retained. SRSS-IE12 refers to the original 7 items from the SRSS developed by Drummond (1994) combined with the new five items constituting the SRSS-IE5. The SRSS-E7 refers to the original 7 items constituting the SRSS.

8

5/2/2013

SSiS – PSG (Elliott & Gresham, 2007)

• Four key areas are assessed: • Prosocial Behavior • Motivation to Learn • Reading Skills • Math Skills

• Three levels: • Preschool • Elementary • Secondary

A comprehensive, multi-tiered program for improving social

behavior.

Focuses on keystone classroom behaviors

and skills.

(Copyright NCS Pearson, 2007)

9

5/2/2013

PSG Actions

Students Scoring a 1 in any area

& Suggested Action

Students Scoring a 2 or 3 in any

area & Suggested

Action

Examining your screening data …

… implications for primary prevention efforts … implications for teachers … implications for student-based interventions

See Lane, Menzies, Bruhn, and Crnobori (2011)

10

5/2/2013

Social Skills Improvement System – Performance Screening Guide Spring 2012 – Total School

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

43.35 47.96 56.12 55.42

45.60 47.55 36.73 38.24

11.04 4.49 7.14 6.34

Perc

ent o

f Stu

dent

s

Adequate progress Moderate Difficulties Significant Difficulties

N = 54

N = 223

N = 212

N = 22

N = 233

N = 235

N = 35

N = 180

N = 275

N = 31

N = 187

N = 271

Reading Skills Math Skills Prosocial Motivation to Behavior Learn n = 489 n = 490 n = 490 n = 489

Subscales

Student Risk Screening Scale Middle School Fall 2004 - Fall 2011

n = 12

n = 20

n = 507

Per

cent

age

of S

tude

nts

N=534 N=502 N=454 N=476N=477N=470 N=524 N= 539

Fall Screeners Lane & Oakes

11

5/2/2013




Teacher-Level Considerations

1. Instructional Considerations 2. General Classroom Management 3. Low-intensity Strategies

12

5/2/2013

Student Risk Screening Scale

Comprehensive, Integrative, Three-tiered (CI3T) Models of Support

Assess, Design, Implement, and

Evaluate

Basic Classroom Management Effective Instruction

Low Intensity Strategies

Behavior Contracts Self-Monitoring

- -Functional Assessment-Based

Interventions

Schoolwide Positive Behavior Support


Higher Intensity Strategies

Assessment

13

5/2/2013

Consideration #1: Essential Components of Classroom Management Classroom Climate Physical Room Arrangement Routines and Procedures Managing Paper Work

Lane & Oakes

Consideration # 2 Instructional Considerations How motivating is my classroom? Control – Challenge – Curiosity –

Contextualization Am I using a variety of instructional

strategies? How am I differentiating instruction? Content – Process – Product

Lane & Oakes

14

5/2/2013

Consideration #3 Low-Intensity Strategies Active Supervision Proximity Pacing Appropriate use of Praise Opportunities to Respond Instructive Feedback Incorporating Choice

Lane & Oakes

Self-Assessment How am I doing with … basic classroom managementstrategies? Instructional considerations? Low-intensity strategies?

15

5/2/2013

Consider a book study … Build school site capacity

Active Supervision

Behavior Specific Praise

Increased OTRs

Choice




16

5/2/2013

Sample School District Student Risk Screening Scale Fall 2012

100% n = 17

80%

70%

82.52

13.64

3.84

n=1397

High Risk 60%

84.53

12.28 3.19

n=530

Moderate50%

81.80

12.60

5.60

n=409

Risk

n = 28

80.92

16.08

3.00

n=458

n = 65 n = 20

90% n = 231 n = 77 n = 91n = 63

Perc

enta

ge o

f Stu

dent

s

Low Risk 40%

30%

20%

10%

0% 2012 Fall MES OES RES

N = 627 N = 500 N = 566 Total School

Sample School District Student Risk Screening Scale Comparing Fall 2012 to Winter 2013

n = 65 n = 63100%

90%

80%

70%

60%

82.52

13.64 3.84

n = 1397

n = 231

High Risk50% 84.80

11.44 3.75

n = 1423

n = 192

Perc

enta

ge o

f Stu

dent

s

Moderate Risk40% Low Risk

30%

20%

10%

0% 2012 Fall Total 2013 Winter Total

N = 1693 N = 1678 School

17

5/2/2013


Goal: Reverse Harm Specialized Group Systems for Students At-Risk







≈

≈

≈

PBIS Framework

Validated Curricula


Assess, Design, Implement, and

Evaluate





Interventions




Assessment

18

5/2/2013

A Step-by-Step Process

Step 1: Construct your assessment schedule Step 2: Identify your secondary supports Existing and new interventions

Step 3: Determine entry criteria Nomination, academic failure, behavior screening

scores, attendance data etc.

Step 4: Identify outcome measures Pre and post tests, CBM, office discipline data, GPA

etc.

Step 5: Identify exit criteria Reduction of discipline contacts, academic success,

reduction of truancies and absences etc.

Step 6: Consider additional needs

Procedures for Monitoring: Assessment Schedule

Aug Sept Oct Nov Dec Jan Feb Mar Apr May

School Demographics

Student Demographics X X X X X X X X X X

Student Outcome Academic Measures

Benchmarking - AIMSweb X X X

Report Card Course Failures

X X X X

Student Outcome Behavior Measures

Screener - SRSS X X X

Discipline: ODR X X X X

Attendance (Tardies/ Unexcused Absences)

X X X

Referrals

SPED and Support-TEAM X X X

Program Measures

Social Validity (PIRS) X X X

Schoolwide Evaluation Tool X

CI3T Treatment Integrity X

19

5/2/2013

Student Risk Screening Scale with academic and behavioral data

Student Risk Screening Scale with academic and behavioral data

20

5/2/2013

After reviewing your assessment schedule Make a master list of all “extra” supports Create an intervention grid Select additional supports with sufficient

evidence to support their use Enlist needed professional development

to assist with implementation

41

A Step-By-Step Process Step 1: Construct your assessment schedule Step 2: Identify your secondary supports Existing and new interventions



Step 4: Identify outcome measures Pre and post tests, CBM, office discipline data, GPA etc.




21

5/2/2013

Sample Secondary Intervention Grid

Support Description Schoolwide Data: Entry

Criteria

Data to Monitor Progress

Exit Criteria

Behavior A written agreement

Contract between two parties used to specify the contingent relationship between the completion of a behavior and access to or delivery of a specific reward. Contract may involve administrator, teacher, parent, and student.

Self Students will monitor Monitoring and record their

academic production (completion/ accuracy) and on‐task behavior each day.








22


5/2/2013



Criteria


Exit Criteria

Behavior A written agreement Behavior: SRSS ‐Contract between two parties used mod to high risk

to specify the contingent Academic: 2 or relationship between the completion of a behavior and access to or delivery

more missing assignments with

of a specific reward. in a grading period Contract may involve administrator, teacher, parent, and student.

Self‐ Students will monitor Students who score monitoring and record their

academic production (completion/ accuracy) and on‐task behavior each day.

in the abnormal range for H and CP on the SDQ; course failure or at risk on CBM








23


5/2/2013



Criteria


Exit Criteria

Behavior A written agreement Behavior: SRSS ‐ Work Contract between two parties used mod to high risk completion,

to specify the contingent AND or other relationship between the completion of a behavior and access to or delivery

Academic: 2 or more missing

behavior addressed in

of a specific reward. assignments with contract Contract may involve in a grading period administrator, teacher, parent, and student.

Self‐ Students will monitor Students who score Work monitoring and record their in the abnormal completion

academic production range for H and CP and accuracy (completion/ accuracy) on the SDQ; course in the and on‐task behavior failure or at risk on academic area each day. CBM of concern;

passing grades








24


5/2/2013



Criteria


Exit Criteria

Behavior A written agreement Behavior: SRSS ‐ Work Successful Contract between two parties used mod to high risk completion, Completion of

to specify the contingent Academic: 2 or or other behavior contract relationship between the completion of a behavior and access to or delivery

more missing assignments with

behavior addressed in

of a specific reward. in a grading period contract Contract may involve administrator, teacher, parent, and student.

Self‐ Students will monitor Students who score Work Passing grade on the monitoring and record their in the abnormal completion report card in the

academic production range for H and CP and accuracy academic area of (completion/ accuracy) on the SDQ; course in the concern and on‐task behavior failure or at risk on academic area each day. CBM of concern;

passing grades








25

5/2/2013

An illustration Support Description Schoolwide Data:

Entry Criteria Data to Monitor Progress:

Exit Criteria

Small group Small group Students who: AIMSweb Meet AIMSweb Reading reading Behavior: reading PSF and reading benchmark instruction instruction (30 Fall SRSS NWF progress at next screening with Self- min, 3 days per at moderate (4 -8) or monitoring time point. Monitoring week). Students

monitored their participation in the reading instructional tasks. Students used checklists of reading lesson components each day to complete and compare to teachers’ rating. K – 1.

high (9 – 21) risk Academic: Fall AIMSweb LNF at the strategic or intensive level

probes (weekly).

Daily self-monitoring checklists

Low Risk on SRSS at next screening time point.

Small group Reading Instruction with Self-Monitoring

Lane, K.L., & Oakes, W. P. (2012). Identifying Students for Secondary and Tertiary Prevention Efforts: How do we determine which students have Tier 2 and Tier 3 needs? In preparation.

26

5/2/2013

Altmann, S. A. (2010). Project support and include: the additive benefits of self-monitoring on students’ reading acquisition. Unpublished master’s thesis, Vanderbilt University.

Treatment integrity Social validity Monitor student progress

Altmann, S. A. (2010). Project support and include: the additive benefits of self-monitoring on students’ reading acquisition. Unpublished master’s thesis, Vanderbilt University.

27

5/2/2013


Goal: Reverse Harm Specialized Group System for Students At-Risk







≈

≈

≈

PBIS Framework

Validated Curricula


Assess, Design, Implement, and Evaluate





Interventions




Assessment

28

5/2/2013

Tertiary Intervention Grid Support Description School-wide Data:

Entry Criteria Data to Monitor

Progress Exit Criteria

Functional Individualized Students who: Data will be The function-Assessment- interventions Behavior collected on based Based developed Scored in the high risk both the (a) intervention Intervention by the

behavior specialist and school leadsership team

category on the Student Risk Screening Scale (SRSS), or scored in the clinical range on one following Strengths and Difficulties (SDQ) subscales: Emotional Symptoms, Conduct Problems, Hyperactivity, or Prosocial Behavior, or earned more than 5 office discipline referrals (ODR) for major events during a grading period ; and/or Academic identified at highest risk for school failure: recommended for retention; or scored far below basic on state-wide or district-wide assessments

target (problem) behavior and (b) replacement (desirable) behavior identified by the team on an on-going basis. Weekly teacher report on academic status ODR data collected weekly

will be faded once a functional relation is demonstrated using a validated single case methodology design (e.g., withdrawal design) and the behavioral objectives specified in the plan are met.

29

5/2/2013

Andrew’s Intervention Outcomes

Baseline MaintenanceWithdrawal Reintroduction Intervention

Figure 1. The symbols used above represent the following: ● liaison collected data; □ teacher collected data; ▲ generalization data collected in Andrew’s Criminal Justice Class; ♦ generalization data collected in Andrew’s English class

What other additional needs to we have in our building?

30

5/2/2013

Before Thinking About Tier 2 for Students … … implications for primary … implications for teachers … implications for student-based interventions

Lane & Oakes See Lane, Menzies, Bruhn, and Crnobori (2011)

Recommendations to Consider

Recommendation #1: Build Stakeholders’ Expertise

Recommendation #2: Develop the Structures to Sustain and Improve Practices

Recommendation #3: Conduct Screenings in a Responsible Fashion

Recommendation #4: Consider Legal Implications- know your state laws

(Lane & Oakes, 2012)

31

Questions:

5/2/2013

Moving Forward

[email protected] [email protected]

32

Documents

Systematic Screenings for Behavior Screening …...2013/06/07 · 5/2/2013 Systematic Screenings for Behavior Screening Tools: Data Based Decisions… One Step Further Kathleen Lynne