Creating a Systemic, Data-Driven & Accountable School Counseling Program

Creating a Systemic, Data-Driven & Accountable School Counseling

Program

What are the Elements of a Systemic, Data-Driven School Counseling Program?

What Makes a School Counseling Program Accountable?

is driving the education agenda . . .

It’s Part of the ASCA National Model

ACCOUNTABILITY

Results ReportsSchool Counselor Performance Evaluation

The Program Audit

Results Data

Closing the Gap

Systemic Impact

Program Effectiveness

MSCA 2010

Accountability is not just:

Counting tasks…….

Reporting numbers…….

Accounting for time…….

Needs assessments…….

Pre- and Posttests……….

These are important to DO but take the next step!

With our colleagues focused on school

improvement, the question we might be asked is:

How does the school counseling program impact student achievement?

Using data provides us with the tools needed to show how students are different because of what school

counselors do.

Transforming Perception and Focus

The new vision of school counseling is to integrate the work of professional school counselors and programmatic interventions into the mission of schools.

Erford, Transforming the School Counseling Profession 2/e Copyright ©2007 by Pearson Education, Inc. All rights reserved.

The Need for Systemic Data-Driven School Counseling Programs

The ASCA National Model supports data-driven approaches as they:– Create “urgency for change”– Expose “evidence of access and equity issues for

focused advocacy and interventions” By focusing on systemic transformation, the

subsystems are changed to meet the needs of students, rather than attempting to transform or fix students.

A systemic data-driven approach focuses on the structure of schools and supports embedding school counseling interventions across multiple levels of the educational program.


A New Vision School Counseling Program Definition

The new vision for all students is systemic in that it:– Promotes holistic development – Nurtures the uniqueness of each individual– Supports their aspirations and goals for the future– Supports the belief that all students not only can dream

of success, but can achieve it as well


Composition of the School Counseling Program Advisory Committee (SCPAC)

SCPAC serves as a sounding board and steering committee.

The professional school counselor must seek to include individuals who can influence and hold the confidence of school system decision makers.

Influential members of the SCPAC will ease the way for necessary programmatic changes.


Composition of the SCPAC (cont.)

Ideal members of the SCPAC– Principal– Professional school counselor(s)– Influential teachers and parents– Political linkages to parent-teacher organizations– Influential school resource person (school

psychologist, special education teacher, reading specialist)

– Influential community organization and business leaders


Role of the SCPAC The SCPAC should convene at least twice annually. Primary roles of the SCPAC:

– Review the results of the needs assessment.– Make recommendations for program development.– Review accountability data and outcomes research

generated by staff.– Locate internal and external funding sources for

program development.


Example of a Systemic Data-driven Program Goal

Program Planning: Needs Assessment

What is the BURNING Question ??

Systemic Assessment Systemic assessments identify the needs of students and the

larger school community by reaching every subsystem of the school community to ensure that the needs of all diverse populations are included.

Methods of collecting information include:– School and district data bases– Adequate Yearly Progress reports– Surveys– Focus groups– Interviews– Observations

By examining various data elements, professional school counselors can understand what influences access, attainment, and achievement.


Needs Assessment

Purposes of needs assessment:– Helps professional school counselors understand the needs

of various subpopulations of a school community.

– Helps establish the priorities that guide the construction of a comprehensive developmental school counseling program.

– “The emphasis in a needs assessment is not so much on what presently exists, but rather on what the present condition is compared to identified goals and objectives” (Wiles & Bondi, 1984).


Data Driven Needs Assessment Aggregated data: results are lumped together to show total

grade level or schoolwide (average) results.– Helpful in understanding how the average students perform in a given

class, grade, or school.– Tells very little about the diversity of learner performance or needs.– Tells nothing about how various subgroups or subpopulations

performed. Disaggregated data: results are broken down by

subpopulations so that performance differences between and among groups can be analyzed.– Analysis usually involves intergroup differences (male vs. female, race,

ethnicity)– Helps to provide hard evidence of gaps in student performance, rather

than relying on perceptions.– Provides direction for the types of strategies and interventions needed to

close these achievement gaps.


Perceptions-based Needs Assessments

Professional school counselors are often interested in what teachers, parents, and students perceive as primary needs to be addressed in a developmental way.


Frequency of Conducting a Needs Assessment

It is best to follow a continuous cycle of assessing programmatic needs.

Breaking down needs into areas of academic, career, and personal-social development (Campbell & Dahir, 1997; ASCA, 2003a) can guide the rotation of needs assessments.

Assessing needs is part of a much bigger endeavor: implementing curricular and programmatic changes to continuously improve the comprehensive developmental counseling program.


Populations to be Assessed It is most practical and efficient to seek out those stakeholders

who are informed and likely to respond, such as teachers, administrators, students, and parents.

Although more difficult to obtain large responses, valuable information can be garnered from community organizations, local businesses, and the general public as well.

When considering the return rate, a higher return rate leads to a lower sampling error, which leads to greater confidence in the accuracy of the results.

The highest priority needs should be those agreed to by all or most populations assessed.


Design Issues in an Efficient Needs Assessment

Stone and Bradley (1994) recommended seven methods for determining needs:

1) Questionnaires and inventories2) Analysis of records3) Personal interviews4) Counseling statistics5) Classroom visits6) Use of outside consultants7) Systematic evaluation of the guidance program

What is most important is that the needs assessment uses objective methods for data gathering and analysis.


Steps to Form an Efficient Needs Assessment

1) Decide what you need to know.2) Decide on the best approach to derive what you need to

know.3) Develop the needs assessment instrument or method.4) Enlist the support of colleagues and a few individuals

from the target groups to review and try out items for understanding.

5) Implement the final version on the target groups.6) Tabulate, analyze, and interpret the results.7) Translate the results into programmatic goals and

objectives.


Design Issues in an Efficient Needs Assessment (cont.)

It is important to take the population into consideration when designing a scale:– For teachers—should ask for grade level, number of students in

the class, etc.– For parents—should ask for the names of the parent’s children in

case responses require contact by the counselor.– For students—should ask for student’s grade and homeroom

teacher’s name. Questions or response stems should be short, to the point,

and easily understood. Reading level of the items should be appropriate for the

target audience.Erford, Transforming the School Counseling Profession 2/e Copyright ©2007 by Pearson Education, Inc. All rights reserved.

Response Formats If the purpose is to determine the importance or frequency of a

potential problem, it is generally best to use a multipoint scale with 3-5 choices such as:

1) Rarely2) Sometimes3) Frequently4) Most of the time5) Almost always

It is important to provide a clear description of each choice. Be wary of response choices such as:

– “Never” and “always”, as they are rarely ever the case.– “Yes” and “no”, as they force an all or nothing response.


Design Issues in an Efficient Needs Assessment (cont.)

It is important to include a frequency count in a needs assessment in order for professional school counselors to not only assess the importance of an issue, but also determine how many, and which students are in need of a specific service.

Computing the results of a needs assessment is probably best accomplished by assigning a number value to each response category and averaging all responses for a given item.


Group Work What information or data do you need to answer

the question? Does this information already exist? Do you need to collect data-collection instruments? Where are your data sources from to answer your

burning question? What kinds of questions will you ask in your needs

assessment ? And to whom?

Program Planning: Programs Goals & Objectives,


Converting Needs to Program Goals and Objectives

If the needs assessment was designed correctly, translating the results into goals and learning objectives is relatively easy.

Step 1: Prioritize the needs in order of importance and their relationship to existing components of the program.

Step 2: Match up or translate the needs into goals included in the national standards.

Step 3: Operationalize the goals through development of learning objectives.


Goals National Goals

– Provide key mandates such as closing access, attainment and achievement gaps, and ensuring safe and drug free schools.

State– Consider national goals and add issues specific to their states

School Districts– Adapt state goals to specific needs of their districts.

Individual Schools– Add additional goals to meet the needs of the students they

serve. By being systemic and data-driven, school counseling

programs link to the goals of education and the mission of schools at the national, state, district and building level.


Goals of the School Counseling Program

Professional school counselors apply the data collected from the systemic assessment to create specific goals for their program.

Goals:– Are developed as general statements of a desired outcome based on the data.– Give direction to the program while allowing flexibility and creativity in

implementation.– Need to be written so that the results can be linked to other schoolwide

measures of student achievement and produce concrete measures.– Can be prioritized to reflect the most intense needs of the school community.– Reflect wider concerns, such as the AYP status of the school.


Aligning with National, State, School Standards

Curriculum begins with a statement of standards which are further described through a series of outcomes, which are broken down into measurable objectives.

Objective 1

Objective 2

Objective 3

Objective 4

Objective 6

Objective 5

Outcome 1

Outcome 2

Outcome 3

Standard 1


Program Integration The achievement of all students is a schoolwide issue and therefore

the ethical and professional responsibility of all school personnel; the educational program and the school counseling program do not stand alone.

The systemic data-driven school counseling program is integrated into the mission of schools by aligning program goals, development, implementation, and evaluation with the educational program.

Integration of the school counseling program with the educational program has the advantage of also integrating the school counseling program into school improvement plans and initiatives.

For successful integration that truly supports the educational program, professional school counselors need to be familiar with the academic goals and standards of their district and state and any other established curriculum that is used in the school.


District and School Policies Policies:

– Create a “way of doing things”– Set the tone for the culture and climate of the school– Operationalize ethical and legal mandates– Help to implement goals

District and school policies interface with the mission, goals, development, implementation, and evaluation of the systemic data-driven school counseling program.

The practices and procedures used to implement policy can either advantage or disadvantage all students or specific groups of students.

Professional school counselors can be leaders and advocates who examine the impact of day-to-day interpretation and application of school policy.


GROUP PROJECT

Think of your group project and how it relates to the existing school policy.

If this were a real situation, what kind of policy would you look for?

What would be some equitable policies/procedures relating to your burning question?

Content of the School Counseling Program

It is critical that specific interventions be integrated into the wider educational program. The professional school counselor can accomplish this by:– Knowing what is being taught in the academic curriculum

and when; assisting teachers in areas where counseling expertise can support student learning.

– Including the Skills for Success (Maryland State Department of Education, 1998) as programmatic goals to integrate with the academic program.

– Contributing to student learning and development in specific disciplines.


Delivery – Implementation at Multiple Levels of Programmatic Intervention

Individual Group Classroom Grade-level Schoolwide Home/family Community/

society


Schoolwide

The schoolwide level of intervention has the greatest potential for removing systemic barriers and creating the conditions for learning at all other levels.

By examining the interrelatedness and interdependence of schoolwide data, appropriate interventions can be made to impact learning and promote culturally responsive practices.


Grade Level This level of intervention attends to the context of grade-

level experiences and transitions at developmental benchmarks.

Emphasis on examining policies that impact specific grade levels is essential.

The ability to analyze and use disaggregated data helps the professional school counselor discern the influence of grade level and transition points on students.

Examples include: transition activities, post secondary planning, resume writing, and public relations and speaking skills.


Classroom This level of intervention is used in classrooms to

align the school counseling program with the academic curriculum and calendar in collaborative classroom instruction.

Data is used to identify classrooms and teachers in need of specific assistance to transform the culture and climate of the learning environment.


Group This level of intervention responds to specific and

targeted needs. It allows professional school counselors to provide

more intensive and in-depth work with individuals. Group is a critical level of intervention because it

provides social modeling, peer support, and learning through a developmentally appropriate forum.

Professional school counselors should consider:– Policies and procedures that govern small group

interventions and counseling.

– Offering small groups annually, including both proactive and reactive formats.


Individual

This level of intervention is used to respond to crisis situations and issues particular to specific individual students.

Issues identified at this level provide insight into issues in the larger school that can then be further examined and dealt with in the form of preventative interventions.

Issues at the individual level involve schoolwide policies such as: confidentiality, informed consent, duty to warn, and parent/guardian permissions for counseling.


Family This level of intervention uses data to

intentionally involve parents/guardians in the school counseling program as equal partners in promoting student achievement.

Intentional outreach to marginalized parent/guardian populations promotes access and equity and assists them in taking an active and empowered role in the school.


Community This level of intervention is used to actively

involve all community stakeholders in improving student achievement.

Community involvement can increase the likelihood that students will experience consistent expectations across the multiple contexts of their lives.

The community level of intervention includes professional school counselors being advocates in the larger societal community.

Using data creates urgency and drives decisions; using inquiry helps to focus, evaluate, and revise interventions.


Community (cont.) Administrative support and sanctioning

(preferable codified in policy) of the collaborative efforts of the professional school counselor and program interventions can help prevent school counseling from becoming marginalized and superfluous to the mission of schools.

Examples of community interventions include:– Community outreach– Public relations– Community Advisory Boards


Program Planning: The Calendar

Planning the School Counseling Program

Strategic planning begins with the vision, mission, data, and subsequent goals of the program.

Creation of a calendar for the school counseling program is an essential tool that demonstrates the systemic nature of the program and organizes interventions into an intentional order.


Outline of Activities for a School Counseling Calendar

Ongoing Activities– Team/IEP meetings

Group Level– Groups that are beginning,

continuing, or ending this month

Classroom– Targeted interventions with

classrooms in need Grade-level

– Grade-specific interventions Schoolwide

– Data collection, analysis, presentation

Family Level– Parent/Guardian groups– Information/skill building workshops

Community Level– Community Advisory Boards– Special presenters

Professional Development– Conferences– In-house trainings

Upcoming Events– List of activities that need to be

accomplished to ensure timely and thorough planning for each intervention.


Think of YOUR Project…

August September October November December

January February March April May June July

Now consider…

What calendar events affect your burning question? What is TIME sensitive? When are your issues most prevalent during the

calendar year? When would it be appropriate to target certain

interventions to certain groups? When would it be appropriate to target _____ level?

Accountability & Program Evaluation


Accountability: Evaluating Programs and Assessing Outcomes

It is a professional and ethical responsibility for professional school counselors to evaluate their program to ensure that the services provided to students, school personnel, and parents are of high quality and effective in meeting intended needs.

Without accountability data to back up service provision, school counseling services are often among the first “nonessential services” to go during budget cutbacks.


Accountability: Purposes1) To measure the effectiveness of the total guidance program.2) To collect data that will be meaningful in determining what

modifications are indicated.3) To determine the level of program acceptance and support from

students, staff, parents, and the community.4) To obtain information that can be used to inform the public

about the guidance program.5) To collect data that will become an important part of the

counselor’s evaluation.6) To analyze the program budget and compare expenditures to

future program needs.(Stone & Bradley, 1994)


Accountability

Accountability, or responsibility for professional actions, involves 3 facets:

1) Having a set of duties that are well-defined2) Conducting an evaluation of performance on those

duties 3) Reporting results of the evaluation to stakeholders

Numerous contemporary factors have called for greater accountability from professional school counselors and an increased focus on assessment.


Advantages & Disadvantages of Accountability Studies

ADVANTAGES Data is better than perception when

guiding decision-making. Accountability helps demonstrate

necessity, efficiency, and effectiveness of program.

Studies can help identify professional development needs.

Professional school counselors can network to share program results.

Conducting accountability studies is a professional responsibility and demonstrates commitment to improvement.

DISADVANTAGES Outcome measures takes some training

and skill to develop. Quality outcome research and

evaluation takes time and resources. Many do not understand the nature and

purpose of accountability because of misperceptions or previous “bad” experiences.

Data is sometimes “over interpreted” or given undue meaning.

Comprehensive evaluations are seldom conducted, leaving the “big picture” incomplete.


5 Questions to Address Accountability in the School Counseling Program

1) Is a comprehensive, standards-based program in place? (program evaluation or audit)

2) What are the needs of the school’s student population when compared to these standards? (needs assessment)

3) What services were implemented to address the identified needs and standards? (service assessment)

4) What was the result of the implemented services? (results or outcomes studies)

5) How well is the professional school counselor performing? (performance evaluation/appraisal)


ACCOUNTABILITY MODELS

SOARING– Dr. Sylinda Gilchrist– http://www.mdschoolcounselors.com/Soaring

%20Model.pdf MEASURE

– Dr. Carolyn Stone

– http://www.prenhall.com/stone/resources.html

http://www.prenhall.com/stone/resources.html

http://www.prenhall.com/stone/resources.html

Program Evaluation

Program evaluation: a systematic examination of a program’s content.

Program evaluation asks 2 questions:– Does the school district have a written

comprehensive guidance program?– Is the written program of the district being

implemented fully in the school buildings of the district? (Gysbers & Henderson, 2000, p. 262)


Program Evaluation (cont.) A program audit should be conducted at the end of the year

and reports should address:– Program strengths– Areas in need of improvement– Long and short-term goals

ASCA suggested that program criteria be evaluated on the following response choices:– None (not in place)– In Progress (begun, but not completed)– Completed (but perhaps not implemented)– Implemented (fully implemented)– Not Applicable (when the criteria do not apply)


Service Assessment Service assessments are often requested by guidance supervisors and

demanded by superintendents and school boards to document how counselors are spending their time.

Two types are commonly used:– Event-topic counts

• Involve documenting each time an individual is contacted or provided with a counseling service and the nature of the topic addressed.

– Time logs• Involve documenting and categorizing activities for every minute

of the workday. Unfortunately, service assessments give no information about the quality

or effectiveness of counselor interventions.


Results or Outcomes Evaluation

Answers the question, “How are students different as a result of the program?” (Johnson & Johnson, 2003)

Assessment is not a discrete component, but rather an integrated part of a continuous process for program improvement.

The evaluation procedure: QuestionsEvidenceInterpretationConclusionsChange


Results or Outcomes Evaluation (cont.)

Assessment of school counseling programs IS:– a way to answer important

program-related questions. – the responsibility of an

accountable professional school counselor.

– a cooperative endeavor with other SCPAC members and stakeholders.

– ongoing and evolving, a means to a better end -better education for all students.

Assessment of school counseling programs IS NOT:– the evaluation of an individual

professional school counselor.

– an assault on the professional school counselor’s freedom.

– a mandate for standardized tests or curricula.

– all figured out.


Sources of EvidencePeople Students Teachers Staff Administration Parents Employers Graduates Community resource people

Products Portfolios Performances Use of rating from external

examiners Observations Local tests Purchased tests Student self-assessments Surveys Interviews Focus groups Student work


Think of YOUR Project…

What types of assessments are you going to have for each intervention you do?

How will you measure success of the intervention?

Think of data pieces you can use to answer these questions…

Practical Program Evaluation Considerations

To be of practical value, assessment must be connected to real program concerns as well as the core values of the school or program.

Suggestions:– Avoid overwhelming the data collectors.– Focus on only one or several important questions at a time.– Always select measures that will yield reliable and valid scores for the

purposes under study.– Involve the relevant stakeholders.– Use a variety of approaches.– Do not reinvent the wheel—use what you are already doing to generate

useful data.– Don’t be afraid to call upon outside experts to consult on the development

and evaluation of a program.– Start small and build upon what is found to work.– Remember: different can still be effective.


Evaluating the Systemic Data-driven School Counseling Program

Serves several purposes:– Demonstrates accountability and establishes the credibility

of professional school counselors and the school counseling program.

– Allows counselors to determine combinations of interventions that proved successful in goal attainment.

– Demonstrates where goals were not met, revealing the need for further examination.

– Demonstrates how the school counseling program contributes to the achievement of AYP, district, state, and national goals for the equitable achievement of all students.


Implications for Professional School Counselors

Professional school counselors must:– Be self-reflective practitioners– Be able to assess and evaluate their own skill base– Develop and implement an ongoing professional

development plan that will allow them to remain relevant Such commitment insures professional school

counselors will develop the competence to implement the systemic data-driven accountable programs that equitably meet the needs of all students.


Documents

Creating a Systemic, Data-Driven & Accountable School Counseling Program