61
Quality in Afterschool April 28, 2015

Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Embed Size (px)

Citation preview

Page 1: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Quality in AfterschoolApril 28, 2015

Page 2: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Agenda

1. Welcome 2. Context setting3. Research4. Believe It – Build It: MN’s Guide to Effective

Practices in Afterschool 5. Quality Assessment and Process Examples6. Funder Table Talks 7. Funder Table Talk Report Back, Commitments

and Questions

Page 3: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Context Setting

Sheila OehrleinMinnesota Department of Education

Page 4: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Research on Quality

Deborah Moore & Kari Denissen CunnienUniversity of Minnesota & Ignite Afterschool

Page 5: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Measuring Youth Program Quality

Facilitator: Deborah Moore, Director – Youth Work Learning Lab

The Youth Work Learning Lab Presents…

Page 6: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Data Paralysis

The Youth Work Learning Lab - School Of Social Work

Page 7: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

The gauntlet of evaluating in youth programs

Hard for youth workers to measure and collect (time, resources and expertise)

Can’t help determine changes needed in program or by staff

Hard to use many tools across diverse program environments

Difficult for programs to evaluate causal information

Different funding sources require different measures

Using poorly designed tools

Tools are too expensive

Tools don’t help staff be more clear about what to “do”

Not enough learning and training support

Other accountability “burdens” never reduced – keep piling on The Youth Work Learning Lab -

School Of Social Work

Page 8: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

I start with these ideas

Data should help us understand something we do not know, but it is better if it relates to our practical experience

Data should not be a full time job for youth programs

Data should be easy to use for change

Data is only one way of understanding the world, others matter too…

The Youth Work Learning Lab - School Of Social Work

Page 9: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Quality Measures Are Not Your Typical Outcome

Based on solid, consistent and compiling research.

Looks at settings and/or behaviors vs. youth internal states (i.e. harder to measure youth outcomes).

Based on things where we have higher levels of control (our practices).

Based where we have the most commonality across the diverse ways we do our work (how we work with youth, not content).

The Youth Work Learning Lab - School Of Social Work

Page 10: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Researchers Agree on Program Quality Elements

The National Research Council & Institute for Medicine list the following key features for quality programs:

Physical and psychological safety Appropriate structure Supportive relationships Opportunities to belong Positive social norms Support for efficacy and mattering Opportunities for skill-building Integration of family, school and community

efforts

- Community Programs to Promote Youth Development, 2002

Page 11: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Research Says Quality Matters

• Empirical evidence linking quality with positive impact

(Durlak, 2007; Vandell, 2005; 2007; 2009; 2011;

Sambrano, 2005 Catalano, 2005, Eccles & Gootman, 2002; Roth, Brooks-Gunn, Murray & Foster, 1998).

• Negative consequences of poor quality programs

(U.S. Dept. of Health, 2006; Belle, 1999).

• If we assess program quality, we can improve it. (Smith, 2012).

Page 12: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

What does high quality look like?

Creating the space for young people to make decisions and have a choice about things that matter to them.

http://www.youtube.com/watch?v=Z2Z17g-2rIc

The Youth Work Learning Lab - School Of Social Work

Page 13: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

BELIEVE IT. BUILD IT.

Kari Denissen CunnienIgnite Afterschool

Page 14: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool
Page 15: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool
Page 16: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

We Believe Statements

Building Blocks

Continuous Program Improvement Cycle

Page 17: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Intentional Program

Design

Supportive Relationships &Environment

Youth Voice & Leadership

Community & Family

Engagement

Responsiveness to Culture & Identity

Organizational Management, Staff Support & Youth Safety

Page 18: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Safe & Active, Focused & Explicit

Are these effects “meaningful”?• Aspirin on heart disease, d = .03• Early child care & math achievement, d = .09-.12• Class size reductions on math achievement, d = .23Durlak, J.A., & Weissberg, R.P. (2010). American Journal of Community PsychologyAbove charts modified from a presentation by Deborah Vandell

Page 19: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

A community approach

Gambone, M.A., Klem, A.M. & Connell, J.P. (2002). Finding Out What Matters for Youth: Testing Key Links in a Community Action Framework for Youth Development. Philadelphia: Youth Development Strategies, Inc., and Institute for Research and Reform in Education.

Page 20: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Human Development Approach

“the quality of a human life involves multiple elements whose relationship to one another needs close study….one of the appealing features…is its complexity: it appears well equipped to respond to the complexities of human life and human striving.”**

*Measure of America, A Project of the Social Science Research Council. Retrieved from: http://www.measureofamerica.org/human-development/**Nussbaum, Martha C. (2011) Creating Capabilities: The Human Development Approach.

Human development is defined as the process of enlarging people’s freedoms and opportunities and improving their well-being. Central to the human development approach is the concept of capabilities. Capabilities—what people can do and what they can become— are the equipment one has to pursue a life of value.*

Page 21: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Continuous Program Improvement

Page 22: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Youth Program Quality Assessment (YPQA)

Deborah MooreUniversity of Minnesota

Page 23: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

David P. Weikart Center for Youth Program Quality · QUALITY COACHING

The YPQA Measures Differently

1. A validated instrument designed to assess the quality of youth programs at the point of service (POS--where youth and adults interact) and identify volunteer/staff training needs.

2. A set of items that measures youth access to key developmental experiences.

3. A tool which produces scores that can be used for comparison and assessment of progress over time.

4. Identifies staff training needs and encourages improvement and creation of a professional learning community

Page 24: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Higher scores are associated with higher levels of youth engagement

Youth self-reports of: o Belongingo Interesto Challenge o Learning

Page 25: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

YPQA practices connects to social and emotional skills…

o Efficacyo Communicationo Empathyo Problem Solving

Page 26: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Youth Program Quality Assessment (Measures)

Pick a card and have a chat about why that practice matters to young people…

Page 27: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Top 5 Things Funders Should Know About Using the YPQA

o Observational assessment is scary AND affirming for youth workers, tread with care.

o The YPQA is useful as a self-assessment process where staff observe their own program or when external assessors observe (but these are not the same).

o Assessment is the Beginning – for the data to have power, it must be placed in the context of continuous quality improvement (emphasis on improvement) and data must get in the hands of those that work directly with youth.

o There are advantages for using one tool (creates common language across staff and programs, allows for community to aggregate data, focuses shared training on mutual improvement goals across programs, etc.)…but some choice can also be good.

o Quality assessment works BEST if it is about learning and improvement. Funders help when they focus on encouraging participation and improvement.

o It takes time- to change habits, to build organizational capacityThe Youth Work Learning Lab -

School Of Social Work

Page 28: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

More Info or Ideas?

To download the YPQA free of charge go to:

Weikart Center for Youth Program Quality- Download the YouthPQA or http://www.cypq.org/downloadpqa

For info on classes and consulting using the YPQA go to:

Deborah Moore - Director Youth Work Learning LabUniversity of MN – Peters Hall1404 GortnerSt. Paul, MN 55108612-625-7813 [email protected]

Page 29: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Academic Skills PQA

Kara Bixby & Nora RobinsonSaint Paul Public Schools Foundation

Page 30: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool
Page 31: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

31 2014 Saint Paul Public Schools Foundation

Youth Program ScenarioExtended day learning class for 6th

graders.Conducting an architecture project

to promote learning of STEM concepts.

Objective is to learn about angles and proportions.

Group discussion: What will you have the class do to learn these skills in a high-quality way?

Page 32: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

32 2014 Saint Paul Public Schools Foundation

Key QuestionsHow do we help youth programs

provide high-quality academic opportunities?

What do they look like in out-of-school time?

What practices are most important?

Page 33: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Defining Academic Skill-Building

Academic Support

Intentionality

Academic Skill-

Building

=LEARNIN

G

Page 34: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

34 2014 Saint Paul Public Schools Foundation

Overview of the ToolMeasures the quality of academic

skill-building opportunities.Expansion of skill-building scale

in Youth Program Quality Assessment (YPQA).

Alignment with Youth Program Quality Intervention (YPQI).

Research-based and validated.Assists in identifying areas where

staff practice can be improved.

Page 35: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

35 2014 Saint Paul Public Schools Foundation

Settings for UsePrograms that provide academic

supports for youth21st Century Community Learning

CentersSchoolsExtended day learning opportunitiesCamps and museum programsCollective impact models with

academic goals

Page 36: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

36 2014 Saint Paul Public Schools Foundation

Why Use?

Geared toward program improvement and performance management.

Focused on practices shown to be effective for academic skill-building.

Offers a higher standard of performance.

Helps programs show the research-based practices they are using.

Page 37: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

37 2014 Saint Paul Public Schools Foundation

Why Use? (cont.)

Includes items specific to social and emotional learning.

Focused on skill development.Additional point of information

and data for academics.Prepares young people to benefit

from school/core instruction.

Page 38: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

38 2014 Saint Paul Public Schools Foundation

Additional Information

Download tool at Weikart Center website:

http://www.cypq.org/downloadpqa

Twin Cities info session to be scheduled Fall 2015

Page 39: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

39 2014 Saint Paul Public Schools Foundation

Questions?

Kara BixbyResearch & Evaluation Director

[email protected]

651-325-4210

Nora RobinsonTutoring Partnership Director

[email protected]

651-325-4244

Page 40: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Quality Mentoring Assessment Path (QMAP)

Mai-Anh KapankeMentoring Partnership of Minnesota

Page 41: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Quality Mentoring Assessment

Path (QMAP)

Page 42: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

2007 Conceptual work with a Quality Advisory Committee (QAC)

2008-’09 Tool Development and QAC beta testers

2009-’10 36 programs piloted - officially launched October 2010

2013 QMAP expansion - Connecticut, Nebraska and Washington

2014 QMAP expansion - Maryland

2015: Ohio enters agreement to use QMAP

MILESTONES…

Page 43: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool
Page 44: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

QMAP is: Continuous improvement Built around a cycle Accommodates varying

program models Guided by quality

assurance goals Confidential

QMAP is not: Accreditation process One-time evaluation Used with start-up

programs A way to rank or score

programs Public reporting system

WHAT IS QMAP?

Page 45: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool
Page 46: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Divided into three sections Program Design & Management Program Operations Program Evaluation

QMAP ONLINE TOOL

Page 47: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Links to free resources for every item

Includes research and templates

RESOURCES

Page 48: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

2 - 4 hours Develop priorities for

Improvement & Innovation Plan with program staff, not just a score or ratings

Plan becomes the platform for technical assistance with individual programs

ON-SITE REVIEW

Page 49: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool
Page 50: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Summary Packet sent: Cover letter Final Improvement &

Innovation Plan Statement of

Accuracy QMAP Benefits

Overview

FOLLOW-UP

Page 51: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Dedicated Partners

Complete the QMAP process

Expert Partners

100% of outlined best practices in place

QUALITY PARTNER RECOGNITION

Page 52: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Smaller programs are honored to receive the same level of recognition as their peers from larger programs

It’s a great way to share program information with staff, Boards and top leadership of multi-service organizations

QMAP helps programs leverage funding

QMAP builds the field toward quality

When programs have best practices in place, our families, kids and volunteers benefit

“MESSAGES FROM THE FIELD”

Page 53: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Expanded Learning Solution Plan

Jaimee BohningNorthside Achievement Zone

Page 54: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

NAZ SEAL OF EFFECTIVENESS

• Partners following evidence-based Solution Plans

• Track real-time results for programs and individuals

• Blueprint for replication

Page 55: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Collective learning and performance accountability

• Monthly convening

• Includes senior NAZ staff, Board members, Parent Advisory Board members, and rotation of partner leaders

• Analysis to find technical and adaptive solutions

• Building culture of accountability: partners are accountable to each other and to families

RESULTS NAZROUNDTABLE ANALYSIS OF DATA

Page 56: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

HIGH TOUCH, HIGH TECH PARTNERSHIP

Page 57: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

EXPANDED LEARNING SOLUTION PLAN

• AGREED UPON RESULT:“Expanded Learning enrolled scholars demonstrate accelerated progress to achieve grade-level performance in math and reading.”

Page 58: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

EXPANDED LEARNING SOLUTION PLAN

Solution Plan “buckets”• Organization Commitment• Reading Skills • Math Skills• Supporting Scholar Engagement• Transition Preparation and Support• Continuous Quality Improvement• Partnerships with Schools and Community Organizations• Family Engagement

Page 59: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

EXPANDED LEARNING SOLUTION PLAN

• We aim to answer three key questions:– How much?– How well?– Is anybody better off?

What is working and how can we do more of it?Focus on scalability and replication

What isn’t working?Why not?Is it still important? If so, how do we improve?If not, how do we cease resource allocation?

Page 60: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Table Talk

1.RFP Application Process 2.Site Visits3.Communicating quality to your Stakeholders

Page 61: Quality in Afterschool April 28, 2015. Agenda 1.Welcome 2.Context setting 3.Research 4.Believe It – Build It: MN’s Guide to Effective Practices in Afterschool

Table Talk Report Back

1.What were key highlights from each table?

2.What will you take away from today and bring back to your organization?