Doing SWPBS in Norwaymarieann+sorly.pdf · •Parent Management Training – Oregon (PMTO) - Research •Risk- and protective factors •Behavior problems, social skills deficits

11.11.2010 © The Norwegian Center for Child Behavioral Development, M-A. SørlieSlide 1

Doing SWPBS in Norway

Implementation and Outcomes of

the PALS Model

Mari-Anne Sø[email protected]

The Duch SWPBS National ConferenceArnhem 11.-13. November, 2010

11.11.2010 © The Norwegian Center for Child Behavioral Development, M-A. SørlieSide 2

Characteristics of PALS

• PALS - a Norwegian acronym for “positive behavior, supportive learning environment and interactions in school”.

• PALS is a culturally adapted & extended model of the PBIS/SW-PBS program, developed in US (Sprague & Walker, 2005).

• PALS is a school-wide, multi-level, multi-component and multi-year intervention model.

The PALS National Conference 2010(780 persons)


• Main goals are to effectively prevent and cope with student problem behavior, and to promote social and academic competence in schools.

• Reduce the need for intensive individual & segregating interventions.

Characteristics of PALS/SWPBS

11.11.2010 Side 4

Characteristics of PALS/SWPBS• The general idea behind is to

replace a reactive and punishment oriented approach to problem behavior in schools with effective positive behavior support, which is structured, proactive, inclusive, and encouraging, yet flexible.


The PALS multi-theoretical & empirical foundation (Arnesen et al. 2006)

•Theory•Ecological system theory•Social interaction learning theory•Functional behavior assessment & therapy•Theory of child development (psychopathology)•Theory of innovations (change) in organizations

- Intervention-/treatment models•Positive Behavioral Intervention & Support (PBIS)/•School-Wide Positive Behavior Support (SWPBS)•Parent Management Training – Oregon (PMTO)

- Research•Risk- and protective factors•Behavior problems, social skills deficits & learning problems in school•Effective school-based interventions• Implementation quality.

Empiricallysupported

interventions

The PALSprinciples

Academic andsocial

competence

Targeting problems and

resources

School-wide and team-based

interventions

Arnesen, A., Ogden, T. & Sørlie, M-A. (2006). Positiv atferd og støttende læringsmiljø i skolen [Positive Behaviour and Supporting Learning Environment in School]. Oslo. Universitetsforlaget. The book also published in Denmark and Sweden in 2008.

Ogden, T., Sørlie, M-A., Arnesen, A. & Meek-Hansen (2011). The PALS school-wide positive behavior support model in Norwegian primary schools. Implementation and evaluation. Invited book chapter. – to be published in England early 2011.

Action and skillsorientedapproach

Implementingwith fidelity

Multi-modalinterventions

Targeting problems and

resources

Action and skillsorientedapproach

Matching interventions to

risk level

Implementingwith fidelity

Why multi-modal and school wide interventions implemented with high fidelity?

• School programs might fail because they tend to be too narrowly focused, have only limited intervention periods, include too few sessions, or are only partly implemented at one grade level or in a few classes (Fray, 2002; Sørlie, 2000).

• Positive treatment or intervention outcomes are strongly related to high implementation quality (fidelity) – the higher fidelity, the better effects (Sørlie et al. 2010) .

• Wilson et al. (2001, meta-analysis) found that school-based interventions or strategies, implemented in isolation, had small effects, and they therefore recommend that schools should develop comprehensive and school-wide packages of prevention strategies, and implement these with high fidelity.


Why team-based implementation of PALS?

• School based programs implemented by external professionals where the teachers are given a passive role when the intervention components are presented to the students, have generally produced only small and temporary effects (Ogden, 2001).

• Thus, in PALS/SWPBS an implementation team is established at each school, with participants from staff, administration, parents, and school psychological services.

• The teams plan and implement interventions, introduce PALS to parents and staff, adapt the PALS handbook (Arnesen & Askeland, 2006) to the school’s situation, monitor the process and outcomes on a monthly basis, and coordinate the school-wide assessment of risk and protective factors.

11/11/2010 © The Norwegian Center for Child Behavioral Development, M-A. SørlieSide 8


Characteristics of PALS• As in the US model, pre-defined core components and research supported

interventions are implemented in a step-wise manner at universal (primary prevention), selected (secondary prevention), indicated (tertiary prevention) level, matched to the students risk level, and adjusted to each school’s culture.

• Model extensions: The family based treatment program “Parent Management Training” (PMTO) is included as an intervention component in the Norwegian model• That is; If needed, PMTO is offered to parents of high-risk students (indicated

level).- Additionally in the Norwegian model, teachers of high-risk students receive

supervision & behavior management training based on PMTO principles (indicated level).

• PALS – is one of the very few school intervention models per date combining evidence based primary prevention and evidence based treatment of conduct disorder.

UNIVERSAL LEVEL (all/low risk) - Establish a set of few school-wide rules (rule matrix).- School-wide IT-based problem behavior registration system (SWIS). - School-wide procedures for assessment, positive encouragement and

reactions to positive behavior & predictable mild consequences toproblem behavior.

- Staff skills training in proactive classroom management, effective studentsupport, and good directions.

- Social competence promotion (classroom based student training); -Teaching in school-rules & expected behavior- Social skills training (the Second Step Program, since 07-08).

SELECTED LEVEL (moderate risk) (org. in short time small groups) - Adjusted social skills training & supervision. - Emotional regulation and effective problem solving skills (e.g. the SNAPprogram).

- More frequent and goal oriented home-school cooperation.

INDICATED LEVEL (high risk) (individually organized)- Individual & intensive social skills training with coach (e.g. the Check-in-Check-out program).

- PMTO-based supervision & behavior management training for teachers of high-risk students.

- Parent Management Training (the PMTO treatment program).

The PALS Intervention & Stepwise Implementation Pyramid


Stepwise three-year school implementation

• Year 1: Emphasize on establishing uniform school-wide and classroom systems for behavior & competence support.

• Year 2: Emphasize on combined universal interventions and selected interventions targeting students at moderate risk.

• Year 3: Emphasize on indicated interventions targeting high-risk students.

New schools are recruited once a year (Spring) by individual applications to NCCBD.


Step-wise diffusion of PALS in primary schools

• 2002: 4 schools, 3 municipalities, 3 counties, 1 PALS consultant (trained at the University of Oregon).

• 2005: 6 schools. Our center responsible for nation-wide training, quality control, program implementation and further research on PALS.

• 2006: 51 schools, 21 consultants, whereof 4 regional & 2 national coordinators.

• 2007: 91 schools, 30 consultants, whereof 6 regional & 3 national coordinators.

• 2008: 103 schools, 34 consultants.• 2009: 135 schools, 41 consultants.• 2010: 153 schools, 62 municipalities,15 counties, 51 consultants,

whereof 9 regional & 3 national coordinators.


• Formal support from the chief municipal education officer, committed participation from at least 80% of the school staff and the principal are required prior to implementation.

• Additionally, a standardized site-assessment of the school's PALS Intervention Readiness and their willingness to contribute to formal research are prerequisites in the Norwegian version of PBIS/SWPBS.

• Each school is individually trained & supervised by a local PALS consultant over a period of two school years (totally 40 hours; 2h/month).

• Thereafter: Sustained external support and continued regional & national booster sessions for PALS teams & consultants, on a quarterly (á 3-5 hours) and yearly basis (2-3 days national PALS conference).

Implementation strategies - PALS


• One-year PALS consultant training & supervision program.

• Candidates are recruited from the local School Psychological Services or the Regional Special Education Support System.

• The PALS Consultants Readiness is assessed prior to training.

• Certification is based on video-taped sessions in minimum 3 schools.

• The pre-specified coding system and PALS readiness tools are developed by the PALS National Coordinator Team at NCCBD (with some consultation from the University of Oregon). Per date implemented in Norway only.

Implementation strategies - PALS


Testing the PALS model empirically

Results from ”The PALS Pilot Study” (2002-2005) and ”The Positive Behavior Support Study” (2007-2012)


The PALS Pilot Study (02-05)• Quasi-experimental pre-post design

• T1 - at the beginning of year 1 of the three-year implementation period

• T2 - at the end of year 2 (20 months after baseline) NB! PALS was not fully implemented.

• Participants:• Intervention group: 4 elementary schools with high level of teacher-reported

problem behavior (P-schools)

• Comparison group: 4 neighboring schools (C-schools) with self-initiated interventions to promote positive student behavior or positive learning conditions.

• Informants: 735 students in 3.–7. grade (8-12 years) and 82 teachers working at least half time (92% of the total sample).

• Baseline comparisons: No significant group differences found, except (as expected) in prevalence of behavior problems (more in P-schools).

Arnesen, A., Ogden, T. & Sørlie, M-A. (2006). Positiv atferd og støttende læringsmiljø i skolen [Positive Behaviour and Supporting Learning Environment in School]. Oslo. Universitetsforlaget.

The book also published in Denmark and Sweden , 2008.

Problem Behavior in School Environment & Classrooms

• Significant decrease in teacher observed problem behavior over time in all schools. The reduction was, however, greater in the P-schools than in the C-schools.

• The decrease was statistically and practically significant both on common school arenas (hallways, stairs, schoolyard etc.) and in the classroom context.

• Intervention effects in the modest range (ES=.59 &.49)

Lærerobs. problematferd i skolemiljø

2 Post-test1 Pre-test

27

26

25

24

23

22

21

20

P-skoler

S-skoler

Lærerobs. problematferd i klasserom


Estim

ated M

argina

l Mea

ns

38

36

34

32

30

P-skoler

S-skoler

Teacher observations in school environment

Teacher observations in classrooms

Sørlie & Ogden 2007


Severely behavior problematic students

• Significant and large difference between P- and C-schools at post-test in number of severely behavioral problematic students in class.

• Reduction most evident in number of students with severe externalizing problems. In fact, the number increased over time in the C-schools (ES = .71).

• Also number of students with internalizing problems increased in the C-schools, while the number slightly decreased in the P-schools.

Gj.snittlig antall særlig

atferdsproblematiske i klassen


Est

ima

ted

Ma

rgin

al M

ea

ns

2,2

2,0

1,8

1,6

1,4

1,2

1,0

P-skoler

S-skoler

Mean number of severelybehavior problematic

students


Immigrant students

Fremmedspråkselever

Sosial kompetanse, lv


100

90

80

70

P-skoler

S-skoler

Fremmedspråkselever

Skolefaglig kompetanse


29

28

27

26

25

24

P-skoler

S-skoler

• Significant differences between immigrant students in the P- & C-schools at post.

• The effects especially eye-striking at the social competence domain (ES = 1.73). As expected, no intervention effects on social competence found for native students after two years.

• On the academic skills domain immigrant students in P-schools had a more positive developing trend over time than all other students in the P-and C-schools.

Ogden, Sørlie & Amlund Hagen (2007)

Social competence

Academic competence


Conclusions after two years with PALS

• PALS stood out as a promising intervention model• for school-wide prevention of behavior problems (both high &

low frequent problem behaviors) • for strengthening teachers' sense of collective efficacy (i.e.

mutual capability to organize and execute courses of actions required to produce student success).

• PALS seemed to have some interesting differential effects (i.e. works better for some than others). • Better effects in schools with high problem pressure• Better effects in high-implementation schools• Better effects on externalizing than internalizing problems• Better effects on pro-social skills and academic achievement in

immigrant than native students.


The Positive Behavior Support Study 2007-2012

• Due to several limitations of the first PALS study, a large-scale longitudinal effectiveness study was initiated by NCCBD in 2007.

Participants• 65 primary schools • 10.000 students (grade 4 -7)• 2.500 teachers, assistants and after-school personnel• 120 school administrators, whereof 65 principals.• 3.000 parents (a 30% random sample)


Main research questions

• How effective is PALS in the short- and long-term process of preventing and reducing behavior problems and promoting social competence in students compared to schools doing business-as-usual (BAU) and schools implementing PALS Compressed (PC), a “low dose version” of PALS?

• Is PALS more effective for certain subgroups of students as compared to others? (e.g. high-moderat-low risk students, boys-girls, native - immigrant students, 4th - 7th graders)

• Are there school differences in outcomes?

• If so; What can explain the variation in outcomes?

Baseline comparisons• Of more than 50 school, staff, student and learning environment

variables explored at baseline (T1), group differences were found on only 8 variables, indicating equivalent research groups.

• Comparisons with national averages indicated that the schools in the research sample are representative of Norwegian primary schools (1st –7th grade).

• Most participant teachers were middle aged (60% older than 35 years) and experienced teachers (14% had worked less than 5 years in school) and 8 out of 10 were female.

• On average, 4.5% of the students had been referred to school psychological services, 5.3% received special education, and 7.6% had minority background.

11.11.2010 Side 24 © The Norwegian Center for Child Behavioral Development, M-A. Sørlie

Sørlie, 2010 (in prep.)


Why is PALS effective?Testing influential elements

(a) Implementation quality(b) Collective efficacy in school


PALS implementation quality vs. outcomes

• Implementation quality: Are all the program components and methods put into practice as described by the developer (e.g. in handbook/manual) and in accordance with the theoretical and empirical principles?

• A significant and inverse relationship between teacher-rated implementation quality and level of problem behavior in P-schools at post-test • r = -.30 (problem behavior in classroom) and -.51 (problem

behavior on school common premises), p< .01

• Regression analyses showed that high program implementation quality was significantly associated with the largest reduction in problem behavior over time.

• Conclusion: Optimal effects of PALS are dependent on high implementation quality (i.e. high program and intervention fidelity).



Addressing a major assumption in PALS/SWPBS

Is it really true that:Differences between schools in level of problem behavior are primarily related to psychosocial characteristics of the schools.

• If so, school-wide efforts to promote supportive learning climate and supporting teaching practices are needed.

• Prior research¹ indicates that modifiable aspects of the school context, such as the policy and practice of the teachers as a group, strongly shapes the normative culture of a school, and that the teachers degree of consensus of opinion significantly affects the students academic outcomes.

• But what about effects on student behavior? Research is scarce.

¹e.g. Goddard, Hoy, & Woolfolk Hoy (2004); Gottfredson, Gottfredson, Payne, & Gottfredson (2005); Hattie (2008); Welch (2003).

11.11.2010 © The Norwegian Center for Child Behavioral Development, M-A. SørlieSide 28Side 28

Collective efficacy in school

The belief that we as a teacher group is capable of reducing problem behavior and

promoting student performance


Significant and marked higher perceived collective efficacy among teachers in the P-schools than in the C-schools at post-test (pre-test scores lacking). PALS-school differences in

outcomes were not only related to implementation quality, but were also significantly related to collective efficacy. That is; the better outcomes,

the higher perceived collective efficacy.

Sørlie & Ogden (2007)

Skolens mestringskompetanse (post)

(Teacher collective efficacy, Goodard 2002)

S-skolerP-skoler

62

61

60

59

58

57

Teacher Collective Efficacy (Goddard, 2002)

Findings from the PALS Pilot Study 02-05


Findings that would utterly support the PALS/SWPBS assumption

• Perceived teacher collective efficacy is a stable school level characteristics over time in ordinary schools (PALS not implemented).

• Collective efficacy predicts school differences in level of problem behavior in ordinary schools (PALS not implemented), both concurrently and across time.

Sørlie & Torsheim, in press

© The Norwegian Center for Child Behavioral Development, M-A. SørlieSide 3111/11/2010 Side 31

Data background

• The relationship between perceived teacher collective efficacy and student problem behavior was examined in a two-wave study.

• Participants were the principals and 1.100 teachers in 48 of the 65 schools participating in the ongoing large-scale longitudinal effectiveness study of PALS model.

• Questionnaires completed with approximately 6 months lag• T1: Apr.-May 2007 (end school year 06-07)• T2: Sept.-Oct. 2007 (beginning of school year 07-08)• That is; Both questionnaires were completed prior to the

implementation of PALS (double pre-test).


Presenter

Presentation Notes

T1: Apr.-May and T2: Sept.-Oct. 07

11/11/2010 © The Norwegian Center for Child Behavioral DevelopmentSide 32


Summing up

• Teacher collective efficacy is a stable school level characteristics.• How much problem behavior there is in a school is significantly

related to the teachers’ perceived collective efficacy.• The teachers’ collective efficacy beliefs seem in fact to have

greater impact on the students academic and social learning outcomes than do structural conditions, socio-demographic variables, and teacher self efficacy.

• High teacher collective efficacy may prevent problem behavior in school.

• Collective efficacy stands out as an important target in future prevention of behavior problems in school.

• Results support that teacher collective efficacy is a “driving force”-candidate in PALS.

• The results need to be confirmed in other studies before firm conclusions can be drawn.


Facilitators contributing to implementation of evidencebased programs in Norway

• A genuine interest and commitment at the political and administrative level for national implementation of evidence based methods (e.g. PMTO, MST, The Incredible Years, FFT, MTFC, PALS).

• Establishing a national center for training, implementation and high quality research on evidence-based programs.

• Research on outcomes, implementation and the intervention processes.

• Therapist and practitioner recruitment strategy through the ordinary service systems.

• Long-term financial support.


Facilitators contributing to implementation of evidencebased programs in Norway, cont.

• Comprehensive national systems for program training and quality assurance, including monitoring of implementation quality (i.e. program & treatment adherence), productivity, program outcomes and sustainability.

• Permanent professional networks for booster training, supervision and consultation.

• The ability of the program developers and stakeholders to motivate and inspire Norwegian practitioners (e.g. through articles, books, anthologies, presentations, lectures, media, committee work, yearly national and inclusive conferences with international experts as key speakers).

• Positive feedback from families, schools, and the media.


Supplementsto Sørlieʼs key note


Some facts about schooling in Norway

• Norway rated several years in row as the best country in the world to live in.• Population about 4.8 million living in 19 counties (430 municipalities). • All social services & public education is inclusive and near free of charge.• All children have a legal right to 13 years of schooling, special education by need and free

school choice.• More than 1/4 of the population over 16 has an education at university and/or university

college level.• Compulsory primary and lower secondary schooling lasts for 10 years, and children start

school the year they become 6.• Compulsory education is divided into two main stages: Primary school (grades 1–7) and

lower secondary school (grades 8–10).• 3.100 primary and lower secondary schools (4.7% private), 620.000 pupils (2.2 % in private

schools), 64.000 teachers.• Primary and lower secondary education is founded on the principle of equity, inclusion and

adapted education for all pupils in a school system based on the same National Curriculum.• About 1% in segregated school settings.

• Norway has a very scattered population and many of the schools are small;• 35% have less than 100 pupils. 53% attend schools with more than 300 pupils. Few

schools have more than 750 pupils.• Education costs 6.6 % of the gross domestic product. Average for the OECD countries is

5.9 % (2003).

http://www.udir.no/upload/Brosjyrer/Education_in_Norway.pdf

Testing new variants of the PALS model

PALS Compressed (low dose)PALS SecondaryI-PALS



Ongoing testing of new variants of the PALS model

• PALS Compressed (”Low dose”)• Effective prevention and responding to student problem behavior.

• The program developed by 2 national PALS coordinators-

• Testing and formal evaluation of short- and long-term effectiveness in 17 elementary schools (Sørlie head researcher).

• 30 hours (4x4 days) non-cost course during a school year for the entire school staff in 4 municipalities (3-7 schools per site).

• Combination of lectures, demonstration, training, and “home work”.

• Covers all key features (components & interventions) of the PALS model.

• No external supervision, implementation support, booster sessions or technical program support are offered, except of SWIS (2 schools).


Testing new variants of the PALS model, cont.

• PALS Secondary• PALS model for elementary is adapted for use secondary school (8th-10th

grade) and piloted in 3 schools (since 2007).

• Same key features (components & interventions) for implementation defined as for elementary, however intervention material and tools adjusted to match the grade levels.

• PALS Secondary is now implemented in 32 schools.

• The National PALS coordinating team responsible for program development, supervision and implementation.

• Unfortunalty, not followed by formal evaluation (lack of recourses).


Testing new variants of the PALS model, cont.

• I-PALS • Promote the students academic competence in PALS-schools by a) early

and systematic identification of ”weak readers”, and by b) research validated and differentiated reading support.

• Establish an integrated three-level system of both effective behavioral and academic support.

• The model extension is developed in US. Now in preparation and adaptation to the Norwegian context by 2 national PALS coordinators at NCCBD.

• Planned piloted and evaluated from 2011 (by the same 2 coordinators).


Level 1Universal Interventions(for all students)• Direct instruction of behavioral expectation

• Positive acknowledgment• “Second Step”

Level 1Universal Interventions(for all students)• Preventive, proactive• Differentiated instruction• Research validated

curriculum

Inc

rea

se N

um

be

rs o

f S

tud

en

tsM

ore Intensive Support

The respons to intervention conceptual system (RIT) for behavioral and academic instruction with general and special education integrated at all three levels

Level 2Selected Interventions(for some students: at-risk)• Simple functional behavior

assessment/Behavior intervention plans

• Small group intervention with individual features

• “Check In/Check Out”

Level 2Selected Interventions(for some students: at-risk)• Some individualizing• Small group interventions• High efficiency• Rapid response

Level 3Indicated Interventions(for individual students)• Wraparound intervention• Complex multiple life domain

Functional behavior assessmentand Behavior intervention plans

• SNAP (Stop Now an Plan)

Level 3Indicated Interventions(for individual students)• Assessment based• Resource intensive

SOME

ALL

Kartlegging alle elever

Behavioral Instruction(with fidelity measures)

Academic Instruction(with fidelity measures)

Integrated School-wide Modell for Intervention and Support

FEW

Screen all students


Implementing PIBS/SWPBS or another evidence based program?

Some advices to schools and consultants


Advices to schools and program consultants Introductory phase

• Inform and involve all key persons in the decision of implementing the choosen program (school leader, staff, parents, eventually the students).

• Check if the program is congruent with the goals, policy, practice and philosophy of the school.

• Consider how suitable the program is in relation to the needs, available recourses and capacity, and discuss evt. needs for adjustment.

• Consider the quantity and quality of the staff training offered.

• Consider carefully if there are (willingness to spend/reordering of priorities) sufficient recourses to implement the whole program with sufficient fidelity.

• Clarify at each school who is responsible for evaluation and adjustment of the implementation quality.

• See to that time is set aside to discussion, problem-solving, and flexible adaptation of the program.


Performance phase

• Frequently consider the implementers skills and satisfaction.

• Give practical and emotional support to the program implementers.

• Keep up a positive atmosphere that is open for communication, exchange of ideas, academic process and disagreement.

• Perform en route internal evaluations of the program, the implementation system, achieved changes and barriers.

• Establish internal routines to monitor the implementation quality.

Advices to schools and program consultants, cont.


After-program phase

• Use information and experiences from the implementation to consider how the quality of the program can be improved and/or sustained.

• Integrate the program as a stable part of the schools ordinary routines, activities and tasks.

• Institutionalize the program

• Develop a realistic time plan for longitudinal implementation and sustained outcomes.

• Inform involved and interested persons in the local environment about the program and the results (actively use the local press in all phases).

Advices to schools and program consultants, cont.

Documents

Doing SWPBS in Norwaymarieann+sorly.pdf · •Parent Management Training – Oregon (PMTO) - Research •Risk- and protective factors •Behavior problems, social skills deficits