An Evidence-Based Practice Approach to School Counselor Education John Carey, Carey Dimmitt, Natalie Kosine National Center for School Counseling Outcome

An Evidence-Based Practice An Evidence-Based Practice Approach to School Counselor Approach to School Counselor

EducationEducation

John Carey, Carey Dimmitt, Natalie KosineJohn Carey, Carey Dimmitt, Natalie KosineNational Center for School Counseling Outcome National Center for School Counseling Outcome

ResearchResearchUniversity of Massachusetts, AmherstUniversity of Massachusetts, Amherst

Tim PoyntonTim PoyntonSuffolk UniversitySuffolk University

Evaluation and Research Evaluation and Research Competencies for School Competencies for School

CounselorsCounselors

Knowledge of Evaluation and Research Knowledge of Evaluation and Research Methods are critical knowledge for school Methods are critical knowledge for school counselors to have in order to participate counselors to have in order to participate in school reform efforts, demonstrate in school reform efforts, demonstrate accountability, and effectively define accountability, and effectively define evidence-based practices for local useevidence-based practices for local use

Data Conceptual and Practical methods for using data

SchoolCounselors

Prior Research – School Prior Research – School Counselors and AccountabilityCounselors and Accountability

Early research from the 1970’s focused on “time analysis” procedures

Answers the questionHow does a counselor spend his or

her time?

Still important today, but not in isolation

Prior Research – School Prior Research – School Counselors and AccountabilityCounselors and Accountability

Literature from the 1970’s describes additional data sources for accountability, but relatively few school counselors actually engaged in “formal evaluation activities” that used quantitative data with relatively large numbers of students (Vacc, 1981).

WHY?

Relevance

Accountability TodayAccountability Today

The ASCA National Model is the “gold standard” for school counseling programs, and is driven by the NCLB act

Asserts that counselors must demonstrate “how students are different as a result of the school counseling program in measurable terms” (ASCA, 2003)

School Counselor Competence School Counselor Competence in Research and Evaluationin Research and Evaluation

Early research arguing that school counselors “tend to confuse accountability with research” (Atkinson, Furlong, and Janoff, 1979) is no longer true

Several authors assert that lack of training in graduate school (Wilson, 1985; Fairchild, 1993; Vacc, 1981; Myrick, 1984, 1997) and little knowledge of assessment and evaluation procedures (Myrick, 1984, 2003) significantly hinder counselors’ accountability efforts

School Counselor Competence School Counselor Competence in Research and Evaluationin Research and Evaluation

Articles are appearing that describe various research methods to engage in accountability efforts, such as single-subject designs (Foster et al, 2002) and using school–wide data (Dimmitt, 2003; Hayes et al, 2002)

Why School Counselors Why School Counselors Underutilize the Research Underutilize the Research

LiteratureLiterature School Counselor Education ProgramsSchool Counselor Education Programs

– School Counselor Education Programs School Counselor Education Programs neglect training in research design, neglect training in research design, statistics and measurement statistics and measurement

– School counselors have few opportunities to School counselors have few opportunities to engage in research during graduate training engage in research during graduate training

– School Counselor Education programs School Counselor Education programs underemphasize research approaches that underemphasize research approaches that are the most appropriate for school are the most appropriate for school counselors counselors


LiteratureLiterature School Counselor Education School Counselor Education

ProgramsPrograms– School Counselor Education program do School Counselor Education program do

not infuse research throughout the not infuse research throughout the curriculum curriculum

– School Counselor Education programs School Counselor Education programs fail to teach that research is an fail to teach that research is an indispensable foundation for practiceindispensable foundation for practice


LiteratureLiterature Public Schools Present barriers to Public Schools Present barriers to

Research Research – Parents are reluctant to allow student to Parents are reluctant to allow student to

participate in research participate in research – Classroom organization precludes strong Classroom organization precludes strong

research designs research designs – School Counselor Role does not include School Counselor Role does not include

research research – Reward structures do not encourage Reward structures do not encourage

researchresearch


LiteratureLiterature School Counselor Attitudes towards School Counselor Attitudes towards

ResearchResearch– School Counselors do not perceive School Counselors do not perceive

research as valuable and relevant to research as valuable and relevant to their work their work

– School Counselors commonly fear School Counselors commonly fear statistics statistics

– Gender stereotypes may inhibit research Gender stereotypes may inhibit research interest in female school counselors interest in female school counselors

Why School Counselors Why School Counselors Underemphasize EvaluationUnderemphasize Evaluation

School Counselor Education ProgramsSchool Counselor Education Programs– Neglect evaluation training in favor of research Neglect evaluation training in favor of research

trainingtraining State Departments of EducationState Departments of Education

– Do not require evaluation competence for Do not require evaluation competence for certificationcertification

School CounselorsSchool Counselors– Fear the results of evaluationFear the results of evaluation– Lack evaluation skillsLack evaluation skills

Public SchoolsPublic Schools– Workload precludes evaluationWorkload precludes evaluation

The Uses of DataThe Uses of Data

Accountability

Using Data

Decision Making

Evidence-Based PracticeEvidence-Based Practice All of the prior conceptual work on school All of the prior conceptual work on school

counselor accountability, evaluation and counselor accountability, evaluation and and research has missed the markand research has missed the mark

Accountability has historically been viewed Accountability has historically been viewed as the reason school counselors should as the reason school counselors should engage in data collection activitiesengage in data collection activities

In EBP, accountability is the end product of In EBP, accountability is the end product of a process counselors can engage to a process counselors can engage to identify areas in need of services, select identify areas in need of services, select appropriate interventions to address those appropriate interventions to address those needs, and evaluate the program’s needs, and evaluate the program’s effectiveness at meeting those needseffectiveness at meeting those needs

A Model of Evidence-Based A Model of Evidence-Based PracticePractice

Client Values andExpectations

Practitioner’s Individual Expertise

Best Evidence

EBP

Adapted from:

Shlonsky and Gibbs (2004), “Will the Real Evidence-Based Practice Please Stand Up? Teaching the Process of Evidence-Based Practice to the Helping Professions. In Brief Treatment and Crisis Intervention, 4(2), 137-153.

A Model of Evidence-Based A Model of Evidence-Based Practice in School Counselor Practice in School Counselor

EducationEducation

Interventionand Program EvaluationCompetence

OutcomeResearch

Competence

Assessment of InterventionTargets and Goals

EBP


EducationEducation


OutcomeResearch

CompetenceEBP


“Knowing how studentschanged”


EducationEducation


EBP


OutcomeResearch

Competence

“Knowing what generally works”


EducationEducation

EBP


OutcomeResearch

Competence


“Knowing what isneeded”


EducationEducation


OutcomeResearch

Competence


EBP

An Evidence-Based Practice An Evidence-Based Practice CompetenciesCompetencies

Our recommendations Our recommendations will be framed across will be framed across topical areas which are topical areas which are domains of knowledge domains of knowledge school counselors need school counselors need to be effective to be effective practitioners in the 21practitioners in the 21stst centurycentury

Topical AreasTopical Areas

o Assessment of Assessment of Intervention Intervention Target and Goals Target and Goals Competence Competence

o Outcome Outcome Research Research CompetenceCompetence

o Intervention and Intervention and Program Program Evaluation Evaluation CompetenceCompetence


Developing the knowledge Developing the knowledge and skills to identify areas and skills to identify areas for program improvementfor program improvement

Requires developing competencies in:Requires developing competencies in:

1.1. Analysis of Institutional dataAnalysis of Institutional data

2.2. Processes for obtaining planning Processes for obtaining planning information from stakeholdersinformation from stakeholders

3.3. Quantitative needs assessment Quantitative needs assessment techniquestechniques


o Assessment of Assessment of Intervention Intervention Target and Goals Target and Goals CompetenceCompetence




Developing the knowledge and Developing the knowledge and skills to identify and evaluate skills to identify and evaluate published research literaturepublished research literature

Requires developing competencies Requires developing competencies in:in:

1.1. Documentation and replicability Documentation and replicability of interventionsof interventions

2.2. Strength of the research designStrength of the research design

3.3. Psychometric properties of Psychometric properties of instruments and measuresinstruments and measures


o Assessment of Assessment of Intervention Intervention Target and Goals Target and Goals




Competencies (continued)Competencies (continued)

4.4. Evaluate analytic and meta-Evaluate analytic and meta-analytic reviews of outcome studiesanalytic reviews of outcome studies

5.5. Summarize the results of major Summarize the results of major school-counseling related outcome school-counseling related outcome studiesstudies

6.6. Conceptual knowledge of statistical Conceptual knowledge of statistical significance and effect sizesignificance and effect size

7.7. Conduct literature searches and Conduct literature searches and consult panels and other consult panels and other information sources for evidence-information sources for evidence-based practicesbased practices






Requires developing the Requires developing the knowledge and skills to knowledge and skills to generate local knowledge of generate local knowledge of school counseling intervention school counseling intervention and program effectivenessand program effectiveness

Requires developing competencies in:Requires developing competencies in:

1.1. Strong evaluation designs Strong evaluation designs appropriate for educational appropriate for educational settingssettings

2.2. Selection and use of sound Selection and use of sound instruments and measuresinstruments and measures


o Assessment of Assessment of Intervention Intervention Target and Goals Target and Goals Competence Competence




Competencies (continued)Competencies (continued)

3.3. Data analysis techniques, Data analysis techniques, including the use of technology including the use of technology tools to assist in data analysistools to assist in data analysis

4.4. Interpretation of the results of Interpretation of the results of data analysisdata analysis

5.5. Accurate dissemination and Accurate dissemination and reporting of resultsreporting of results






EducationEducation


OutcomeResearch

Competence


EBP

Curriculum ImplicationsCurriculum Implications A course that helps counselors develop A course that helps counselors develop

skills in all three EPB domains is very helpfulskills in all three EPB domains is very helpful Its important at counselors are also taught Its important at counselors are also taught

how to enact an integrated EBP approach how to enact an integrated EBP approach that can be implemented in a school (Data that can be implemented in a school (Data Based Decision-Making, Action Research, Based Decision-Making, Action Research, Logic Modeling)Logic Modeling)

EBP skills should be integrated into all EBP skills should be integrated into all professional courses and practica (reading professional courses and practica (reading meta-analytic reviews and original research, meta-analytic reviews and original research, evaluating research, writing briefs, evaluating research, writing briefs, designing evaluation approaches)designing evaluation approaches)

Questions and DiscussionQuestions and Discussion

Thank You!Thank You!

For additional information,For additional information,please visit the Center for School please visit the Center for School Counseling Outcome Research Counseling Outcome Research

website:website:

http://www.cscor.org

Documents

An Evidence-Based Practice Approach to School Counselor Education John Carey, Carey Dimmitt, Natalie Kosine National Center for School Counseling Outcome