Evidence Based School Counseling

Chapter 2

Data Based Decision Making

• Relies on evidence to define problems• Requires that goals are stated in ways that will

allow for data to be gathered• Used quantitative data analysis techniques to

describe problems and to direct activities

Definitions

• Achievement Data– Standardized test scores, grade point averages, SAT and

ACT scores, graduation rates, AP test scores• Achievement Related Data– Course enrollment patterns, discipline referrals,

suspension rates, attendance, parent involvement, homework completion rates

• Competency-related Data– Six or four year academic plans, job shadowing

participation rate, peer mediation sessions, knowledge of graduation requirements.

Models of Data-Based Decision-Making Models

• Whole School Reform- Reynolds and Hines (2001)– SC leads an interdisciplinary team through the

DBDM process to define problems and decide on interventions

• All DBDM focus on the use of data to – Define problems– Set goals– Target interventions

The DBDM Team

• Team is developed based on – Whole School reform initiative– Component of the School Counseling Program

• Questions– Does the team include all the needed perspectives to:

• correctly identify problems and potential solutions• Correctly identify strategies and barriers to intervention

implementation

– Do team members have the necessary data literacy skills– Do team members show the capacity for effective

collaboration

General Model of DBDM

• Describe the Problem• Generate vision data• Committing to benchmarks• Identify where and how to intervene• Selecting interventions• Evaluating interventions• Monitor problem data

Describe the Problem

• Important data to use include the following– Achievement data– Achievement related data– Guidance curriculum competency data– School Climate survey data – Relevant student health and well-being data– Needs Assessment data– Demographic Data(pg. 20 DCH for process model)

Generating Vision Data

• Reflect on the future goal toward which school efforts and resources need to be directed.– Goals need to be stated in concrete and

measureable terms – What should the data look like in 3-5 years. – Should be ambitious and attainable• Ambitious goals serves student interests and engages

the passion of adults in the school• An attainable goal is likely to motivate efforts to change

Committing to benchmarks

• Breaking down the goal expressed as vision data into yearly goals– Reflect equal increments of change needed each

year to reach the vision– Sometimes DBDM team may create unequal

increments

Identifying Where and How to Intervene

• Seven levels of programmatic interventions: – Individual– Group– Classroom– Grade level– Schoolwide– Home/family– Community/societyMay be at several levels.

Selecting Interventions

• (see chapter 4 for details )• Interventions selected must be supported by

strong research evidence. • Research programs carefully • Not just the packaging of interventions

Evaluating Interventions

• (see chapter 5 for a detailed description)• Evaluation should occur at three levels– Know whether the participants learned the knowledge

or skills the intervention intended to teach or whether their attitudes shifted

– Know whether the participants changed their behavior in the problem data in ways that predict success

– Measure the actual change in problem data and to compare it with the benchmark targets

Monitoring Problem data

• Monitored each year. • The DBDM team reviews the problem data in

comparison to the stated benchmarks, examination data and decide whether the current strategies and approaches need to be continued, modified or abandoned.

Enabling Conditions

• Collaborative Culture• Collaborative Structures• Widespread Data Literacy• Access to useful data

• (see pg, 25)

Practical Considerations in Using dataChapter 3

• School counselors use data for a variety of purposes: – To ensure that every student receives the developmental

instruction that is described in professional standards (ASCA, 2003 Campbell & Dahir, 1997).

– To make decisions regarding which areas of need require additional support or intervention (Hatch, Holland & Meyers, 2003; Hayes, et al 2002).

– To measure the effectiveness of their activities and interventions and to share their successes with the school community

– To evaluate the effectiveness of their programs and for program inprovement

Types of Data• Student Achievement Data

– Passing rates for state achievement tests– Standardized achievement test data– SAT and ACT scores– Algebra passage rates– GPA– Drop-out rates and graduation rates– College acceptance rates– Completion of college prep requirements– College freshman remediation rates– Advanced Placement test scores

• Data should be mined to determine where resources are needed.

Achievement-Related Data

• Course enrollment patters• Discipline referrals• Suspension rates• Alcohol, tobacco, and other drug use• Attendance rates• Parent involvement• Extracurricular activities• Homework completion rates

Standards and Competency-Related data

• Each school should have it’s own set of measurable competencies and student learning objectives

• Quantitative indicators of competency attainment– Percentage of students who know credit requirements for

graduation– Demonstrate knowledge of study skills and how to use an

academic planner– Use test-taking strategies– Can identify the steps in setting goals– Demonstrate conflict resolution skills, and– Believe that it is important to come to school every day.

Data Relationships

• (see pg 32 Table 3.1)– What achievement-related data must be moved to

address the problem and what competencies must be developed to move the achievement related data

– Next determine what standards and competences students lack.

Collecting data for Program Planning

• Getting access to data is a first step• Existing data– Departments of education websites for data

reporting• New data– Successful college applications– Scholarship support

Analyzing Data

• Data should be entered in a comparable metric

• Raw data is converted to: – Percentages– Ratios– And/or probabilities

• Important to use appropriate comparison groups and comparisons across multiple measures

Disaggregation of Data

• Disaggregate by the following groupings:– Gender– Race/ethnicity– Socioeconomic status– Language– Special education placement– English language learner status– Grade level– Achievement quartile– Teacher/classroom

Developing Data-Based Action Plans

• Each plan should contain– Competencies addressed– Description of the activity– Data driving the decision to address the

competency– A time line in which the activity is to be completed– Who is responsible for delivery– Means of evaluating student success– Expected results for students

Chapter 5

• Evaluating School Counseling Interventions and Programs

What is Evaluation

• Evaluation-The use of scientific method (hypothesis testing) to improve local decision making by determining whether it was likely that implementing an intervention resulted in desired changes in behavior and performance.– Used to improve decision making in a single setting

• Research-The use of scientific method to determine whether an intervention brings about changes in affect, cognition, and /or behavior. – Used to identify practices that are theoretically effective

across settings

Student-Focused

versus System-Focused

School Counseling Program Activities

MentoringStudents

PhoneContact

StudySkillsGroup

SmallGroup

ClassroomGuidance Behavior

Management

Bully ProofingProgram

Tutoring

Data Driven Interventions

70% Attendance Rate for Low SES Students

IndividualCounseling

Student Focused

Student Focused Interventions

Interventions designed to directly help students gain knowledge and skills in the areas of academic, career, and personal/social development in order to help them better navigate the educational system

System Focused Interventions

Interventions designed to help the system (school) change in order to better meet the needs of the students.

Examples:•Change educator attitudes, expectations, and priorities•Reduce with adult resistance to change•Change policy•Change practice

Activity

Scenario:Free/reduced lunch students do not pass Math at the same rate as their non-free/reduced lunch peers.

Brainstorm:Student Focused InterventionsSystem Focused Interventions

Designing Interventions to Help All Students Meet High Academic Standards

Design Student Focused Interventions to Reach All Students

Who is Served by the School Counseling Program?

ACTIVITY:• Who Typically Receives School

Counseling Program Services?

• Do school counselors filter information?

Funnel of Continuous Interventions

DATA

ACTION PLANAll Students

Large group/Classroom

Small Group

Individual

Referral

Some Students

Few Students

A Student

Continuum of Interventions to Reach All Students

Small Group Interventions• Counseling, study group, homework club, etc.• For students needing extra help to master

indicators• School counselor or appropriate professional

Continuum of Interventions to Reach All Students

Individual Support or Counseling Interventions• Counseling, mentoring, etc.• For students still needing extra help or

support to master indicators• School counselor or appropriate professional

Continuum of Interventions to Reach All Students

Referral/Additional Interventions• Mental health agency, medical

organization, academic tutoring, community service, etc.

• For students and their families needing extra, outside help to enable students to master indicators

• Appropriate professionals

Creating Pre and Post Measures

Ways to Collect Data

• Paper/pencil measures– Likert scale– Multiple choice– True/False– Short answer– Fill in the blank

• Skill demonstration measures– Role play– Demonstration– Presentation– Verbal questions

Activity

Design Pre and Post Measures

Use Data to MonitorStudent Progress

Paradigm Shift

From:

To:

Monitoring Only Process and Tallying Services Delivered

Focusing on Results Tied to the Academic Goals of the School

Types of Data

• Process• Perception • Results

Types of Data

• Process• Perception • Results

Process Data - Examples

• Six counseling groups with 8 students each were held

• 1,350 6-8th grade students received the “Time to Tell” guidance lesson

• All high school students seen individually to prepare 4 year plan.

Adding Process Data

Activity Process Data

Perception Data Results Data

Question or Demonstration

Pre PostHow will student

behavior change?

Pre Post

Date

# %

Date

# %

Date

# %

Date

# %

Perception Data• “What others think, know or demonstrate” data. • Metacognitive Awareness• Measures competency achieved, knowledge gained

or attitudes beliefs of students – Pre-post– Competency achievement– Surveys– Evaluations

• Measures what students are perceived to have gained in knowledge

Perception Data - Examples

• Competency Achievement– Every student in grades 9-12 completed a 4 year plan – Every 10th grade student completed an interest inventory

• Knowledge Gained– 89% of students demonstrate knowledge of promotion/ retention

criteria– 92% can identify Early Warning Signs of violence

• Attitudes or Beliefs– 74%of students believe fighting is wrong– 29% of parents say their child feels safe at school– 58% of teachers say students behavior appropriately in class

Adding Perception Data

Activity Process Data

Perception Data Results Data

Question or Demonstration

Pre PostHow will student

behavior change?

Pre Post

Date

# %

Date

# %

Date

# %

Date

# %

Results Data

• “So WHAT” data• Response to Intervention or Behavior

Responsiveness• How has student behavior changed• Proof activity has (or has not) positively

impacted students ability to utilize the knowledge, attitudes and skills to effect behavior– Attendance– Behavior– Academic achievement

Results Data - Examples

• 42 students on the retention list avoided retentions

• Graduation rates improved 14% over three years

• Attendance improved among 9th grade males by 49%

Adding Results Data

Activity Process Data

Perception Data Results Data

Question or Demonstration

Pre PostHow will student

behavior change?

Pre Post

Date

# %

Date

# %

Date

# %

Date

# %

Activity

Fill in the following DATA columns on your Planning Tool

Process Data 1. What will you count to show what you did?

Perception Data 1. What question on a survey or test, or what demonstration will use to see if students (or adults) think the student has mastered the indicator?

2. What date will you collect the PRE data?3. What date will you collect the POST data?

Results Data 1. What behavior will the students exhibit that will demonstrate they have mastered the indicator?

2. What date will you collect the PRE data?3. What date will you collect the POST data?

Putting These Ideas To Work:Homework

1. With the input of your Advisory Council, faculty and students, complete the design of a continuum of student focused interventions for each of your prioritized indicators

2. Prepare for April training. Complete/revise/ review Essential Learnings Reflection and decide upon systemic change goals (changing policies, and/or practices to help the school better meet student need). Fill in systemic change section on the last page of the today’s Design Tool.