Upload
vannhan
View
226
Download
0
Embed Size (px)
Citation preview
Tools for Evaluating Social Validityin a Rapidly Expanding PBS
ProgramJulie Graden, Jill Norman, Molly Gadd,
Michelle Marchant, & K. Richard Young
Brigham Young University
Presentation Outline
• Overview of the Peaceable Schools Project• Obstacles to collecting Social Validity in a
growing program• Review of the outcomes of our Social Validity data• Implications for the future
Peaceable Schools Program:An Overview
Our Priority. . .
To create a positive school environmentthat nurtures the social, emotional and
academic well-being of all studentsthrough the use of Positive Behavior
Support (PBS) practices.
“If schools attend to the emotional andbehavioral needs of students on a broadscale, it is likely that they will create theconditions necessary for socialcompetence and academic success ofstudents.”
(Young, Marchant, & Wilder, 2003)
School-Wide Systems
Approach
Individual
Classroom Non-Classroom
Model
Why Measure Social Validity?“It is important to assess social acceptability ofintervention procedures…in order to ensure that allrelevant parties (e.g., teachers, parents, and otherinterventionists) agree that the procedures arereasonable for the classroom, home, or wherever theintervention procedures take place…Namely, if anintervention is viewed as socially acceptable there ishigher probability that it will be implemented withtreatment integrity than if the intervention procedureswere initially viewed to be unacceptable.”
- Lane & Beebe-Frankenberger, 2004
Obstacles to MeasuringSocial Validity
• Developing instruments of social validitymeasurement
• Collecting data
Developing Instruments ofSocial Validity Measurement
• ISQ (Indicators of School Quality)• SVQ (Social Validity Questionnaire)
ISQ: Indicators of School Quality
The ISQ is a comprehensive survey system forschool administrators to evaluate and monitorschool improvement efforts. It summarizes theperceptions of parents, teachers, students, andother school staff regarding more than 30 crucialcharacteristics of the school in 7 major areas.
Development
• Center for the School of the Future (CSF)Utah State University– http://www.schoolquality.org/– Original Instrument
• Peaceable Schools Program, BYU– Supplement
Instrument:Student
Survey ResultsWestside:2002-2003
School Climate Westside:2002-2003
Social Development Westside:2002-2003
Social Validity QuestionnaireSVQ
• Development– Questions addressing school-wide PBS and
specific interventions– Administration
Social Validity Analysis
• Four areas being measured: school-wide,classroom, non-classroom, individual
• Components being measured: procedures,outcomes
• Populations being surveyed: parents,students, teachers/staff
Research QuestionsWhat are• educators’ perceptions of procedures?• educators’ perceptions of outcomes?• parents’ perceptions of outcomes?• students’ perceptions of outcomes?
--School B
--School AIndividual
0.973.40School B
0.834.07School ANon-classroom
0.854.22School B
0.934.22School AClassroom
0.854.00School B
1.004.17School ASchoolwide
SDM
Educators’ Perceptions of Peaceable Schools Project’s Procedures
0.493.75 Teachers
1.532.64 Staff
0.953.19School B
0.864.41 Teachers
0.83.95 Staff
0.834.18School AIndividual
--Non-classroom
0.394.24 Teachers
0.654.27 Staff
0.474.25School B
0.854.53 Teachers
0.784.07 Staff
0.824.3School AClassroom
0.413.93 Teachers
1.373.01 Staff
0.743.48School B
0.774.37 Teachers
0.743.96 Staff
0.754.17School ASchoolwide
SDM
Educators’ Perceptions of Peaceable Schools Project’s Outcomes
0.833.91School B
0.784.23School AIndividual
--School B
--School ANon-classroom
0.794.08School B
0.674.37School AClassroom
0.734.08School B
0.694.28School ASchoolwide
SDM
Parents’ Perceptions of Peaceable SchoolsProject’s Outcomes
0.592.62School B
0.632.59School A
SDM
Students’ Perceptions of Peaceable Schools Project’s Outcomes
General Results
• For both schools the lowest rankingquestion was “The Peaceable Schoolsproject required too much teacher time toimplement.”
• The highest ranking questions were “Myadministrator supports and values the PSP”and “It is important to praise appropriatestudent behavior”.
Methods of Collecting Data
1. Paper/pencil - first year School A: 55% School B: 87.5%
2. Survey Monkey - second year School A: 76% School B: 57%
Benefits of Technology
Technology:• Is fast and easy• Produces a better return rate• Makes data storage less cumbersome• Allows for easy manipulation of data• Cuts costs• Eliminates errors of human calculation
Drawbacks of Technology
Technology:• Requires specific skills and equipment• Distances us from data• May distance us from the people
Creating Social Validity Surveys
• Four areas: schoolwide, classroom, non-classroom, individual
• Components: procedures, outcomes• Populations: parents, students,
teachers/staff• Benefits of technology
Future Research Questions
• How are social validity data influencingprogram change?
• How does technology impact return rate –among different groups?
• How does the social validity of electronicsurveys compare to paper pencil surveys?
• Does the PBS model become more sociallyvalid over time?