Embed Size (px)
Citation preview
C O M P R E H E N S I V E P R O G R A M M A T I C
E V A L U A T I O N : E X A M I N I N G T H E
E F F I C A C Y O F S C H O O L
C O U N S E L I N G A D V I S O R Y
B O A R D S
L U C Y L . P U R G A S O N , P H D , W E S T E R N W A S H I N G T O N U N I V E R S I T Y
D I A N A G R U M A N , P H D , W E S T E R N W A S H I N G T O N U N I V E R S I T Y
OPENING QUESTIONS
•Why go to the trouble of forming and maintaining an advisory board?
• Is it just for window dressing or public relations?
•How does it differ from other collaborative relationships?
OBJECTIVES• Understand the different rationales for creating and
implementing an advisory board
• Provide a theoretical framework for parent involvement in advisory boards
• Discuss potential best practices for board creation, implementation, and evaluation
• Provide audience members with tools to develop an action plan
• Develop a path forward for efficacy research on advisory boards
WHAT IS AN ADVISORY BOARD?From the ASCA National Model (ASCA, 2012, p.47):Definition:
“Representative group of stakeholders selected to review and advise on the implementation of the school counseling program. The council meets at least twice a year and maintains an agenda and minutes for each meeting.” Scope:
Advising on program goalsReviewing program resultsMaking recommendations about the school counseling programAdvocating and engaging in public relations for the counseling
program Advocating for funding and resources (Johnson & Johnson, 2001)
WHAT ELSE MAY BE IMPORTANT?We believe there is great value in:
–On-going trust –Historical knowledge –Multiple perspectives in one room –Authentic, independent assessment of outcomes –Dynamic problem-solving –Current feedback from the field
CONTINUUM OF STAKEHOLDER FEEDBACK AND CONTROL Consultation AdvisoryAdvisory Board ofWith Stakeholders Group Board
Directors
Ideas and Input Many POV Data-BasedFiscal &
From Individuals + Feedback Loop +Accountability + Hiring Responsibility
PR Independent Review
Commitment
PURPOSES OF THE BOARD TACTICAL BOARD DEVELOPMENT
• Improve public relations
• Increase clout or buy-in for the school counseling program
• Complete a specific task, such as:
– Increase college/career readiness
– Select an SEL curriculum for use with elementary students
– Increase parent awareness of the school counseling program
MUTUAL BOARD DEVELOPMENT
• Allows for multiple perspectives and the voices of underrepresented populations
• Is a reciprocal process that is beneficial for members and for the school counseling program
• Draws on community cultural capital resources
• Provides multiple channels of communication and feedback loops
OVERLAPPING SPHERES OF INFLUENCE (EPSTEIN, 2010)
School
Community Family
EPSTEIN’S MODEL OF PARENTAL INVOLVEMENT(EPSTEIN ET AL., 2002)
Parenting
Can help schools understand families’ backgrounds, cultures, and
expectations
Volunteering
Recruitment and training of interested family and community
members
Learning at Home
Involving families with academic learning
ADVISORY BOARD: 5 OF THE 6 FOCUS AREAS
ParentingIdentify parents and community members
that represent diverse perspectives and
backgrounds
Communicating
Board members can help disseminate key program information
Volunteering
Draw on community and social capital
knowledge
Learning at Home
Involving families with academic learning
Decision Making
Provide input and perspectives on key
program components and data driven issues
Collaborating with the
Community Opportunity for
multiple stakeholders to be at the table
ACTIVATING COMMUNITY CULTURAL WEALTH (YOSSO, 2005)
Aspirational
Aspiration and
hope
Familial
Cultural
and/or family knowle
dge and
history
Linguistic
Communicate
in differen
t languages or styles
Social
Networks and Commu
nity Resour
ces
Navigational
Navigating social institutio
ns
Resistant
Challenge
inequities
DEFINING EFFECTIVENESS • RAMP requires utilizing a school counseling advisory
board
• Criteria include:
–Wide representation amongst members
–Examination of school data
–Sole focus on school counseling
–Detailed description of board development and program guidelines
–2+ meetings a year
Are these sufficient to measure effectiveness?
MEASURING EFFECTIVENESS PROCESS AND PERCEPTION
• Who is on the board?
• How many times do you meet?
• What is the content of the meetings?
• How did you recruit?
• How much data/what types of data did you gather and share?
• Were members’ perspectives considered?
• Was the meeting organized?
• Was there adequate access to materials?
OUTCOME
• How effective was the school counseling program in improving student academic and SEL learning?
• Did the board meet the goals established at the beginning of the year?
• How did the school counseling program change as a result of the board recommendations?
BEST PRACTICE CONSIDERATIONS (SEE HANDOUT)• Statement of Purpose
• Recruitment
•Meeting Structure
• Assessment and Accountability
• Acknowledgement of Contributions
ACTION PLAN FOR YOUR BOARD (SEE HANDOUT)
•With a neighbor review the action planning handout
• Brainstorm potential plans for your program
•Opportunity to share
COLLECTING DATA: ELEMENTS OF QUALITATIVE PROGRAM EVALUATION AND PERFORMANCE MEASUREMENT
1) Identify an Issue/Create a Focus
Could be related to wanting to understand what is working well or may be related to an unachieved goal.
2) Identify Questions
What do you want to ask members about the issue?
Link to the advisory board mission, goals, objectives, activities, and anticipated outcomes
3) Collecting Data - Design an interview protocol (list of questions)
Can be asked through an interview format or through surveys with open-ended questions.
4) Invite or Select Members to Interview
Convey that information is important and tell why it is important.
Explain how the information will be used.
FRIENDS National Resource Center for Community-Based Child Abuse Prevention, 2009
ANALYZING DATA: ELEMENTS OF QUALITATIVE PROGRAM EVALUATION AND PERFORMANCE MEASUREMENT 5)Analyzing Data
Look for data that stand out
Identify themes
• Focused on identifying words or phrases that summarize big ideas
• Are their direct quotes that are appropriate illustrations of key themes?
• Have you captured all of the perspectives?
Find “critical friends” to offer perspectives
6) Sharing DataImportant to educate others about the value and merit of your work
Look for compelling direct quotes to illustrate important themes
Go back to members who shared their experiences and allow them to expand or provide clarification
Look for opportunities to quantify some of the qualitative data (if it is mentioned by more than one member)
Remember to include process and perception data FRIENDS National Resource Center for Community-Based Child Abuse
Prevention, 2009
THE MOST SIGNIFICANT CHANGE APPROACH (MCDAVID, HUSE, & HAWTHORN, 2012)
• Qualitative method for assessing performance
• Interviewing members
• Asking members to identify positive or negative changes they have observed over a fixed time
• Members are asked to indicate which change is the most significant and why
• Provide authentic views of members’ experiences and perspectives
FUTURE RESEARCHPotential Research Questions• What are the components of an effective advisory board?
• How are school counseling programs changed by the implementation of advisory boards?
Research Methodology• Identify exemplary school counseling advisory boards
• Conduct interviews with key stakeholders
• Review advisory board documents (i.e. meeting minutes, end of year reviews, action plans)
REFERENCES AND RESOURCES
American School Counselor Association (2012). The ASCA National Model: A Framework for School Counseling Programs
(3rd ed.). Alexandria, VA: Author.
Epstein, J. L. (2011). School, family, and community partnerships: Preparing educators and improving schools (2nd ed.). Boulder, CO:
Westview Press.
FRIENDS National Resource Center for Community-Based Child Abuse Prevention (2009). Using qualitative data in program
evaluation: Telling the story of a prevention program. Retrieved from: http://friendsnrc.org/joomdocs/qualitativedata.pdf
Johns Hopkins University (2015). National Network of Partnerships Schools. Retrieved from:
http://www.csos.jhu.edu/p2000/index.htm
Johnson, C. D., & Johnson, S. K. (2001). Results-based student support programs: Leadership academy workbook. San Juan
Capistrano, CA: Professional Update.
McDavid, J. C., Huse, I., & Hawthorn, L. R. L. (2012). Program evaluation and performance measurement: An introduction to
practice (2nd ed.). Thousand Oaks, CA: Sage.
Yosso, T. J. (2005). Whose culture has capital? A critical race theory discussion of community cultural
wealth. Race, Ethnicity, and Education, 8, 69-91.
QUESTIONS?
We enjoyed our conversation today and appreciate your ideas and feedback. Please feel free to email us at:
Lucy Purgason [email protected]
Diana [email protected]
