Carrie E. Markovitz, PhD

Program Evaluation: Challenges and Recommendations

July 23, 2015

2

• Produce evidence that program is meeting intended outcomes

• Support and inform programmatic decisions:• Program design and implementation• Program investments• Program improvement• Implementing change

• Required by program’s funders to inform future funding opportunities

Footer Information Here

Why is Evaluation Important?

3Footer Information Here

Common Challenges in Program Evaluation

• Knowing where to begin• Lack of evaluation capacity within programs

• Lack of knowledge among staff• Lack of resources (staff support, staff time, and/or funding)• Lack of infrastructure (data collection systems, tools/instruments,

surveys/assessments, etc.)

• Managing an evaluation and/or external evaluator

Successful Approaches to Evaluation

• Define the program• Ensure a consistent understanding of program • Only evaluate program components that are ready for

evaluation

• Define the evaluation’s purpose and scope• Develop a culture of evaluation within your

organization• Prepare staff to participate in an evaluation• Develop adequate data collection systems• Communicate findings back to program staff,

beneficiaries, and other stakeholders• Use your evaluation results

• (When appropriate) Hire an external evaluator

5Footer Information Here

One Action Step for AmeriCorps Grantees

Don’t be afraid to start small• Try to do something each year and build on your successes

– Develop a logic model of your program– Survey of your members and/or service recipients– Invest in data systems for regular collection of information for

performance measurement– Develop a relationship with an evaluator and/or university

research department• Evaluation is not a “one and done” type of activity• Make evaluation a part of your organizational thinking

– “We want to invest in this new approach/ How will we know if it’s working?”

6

• Make evaluation part of your regular program activities

• Develop a long-term research agenda:– A series of intentional or planned program evaluations and/or

research activities– Similar to a strategic plan, a research agenda spans over

several years• Set aside funds each year to continue to build evaluation capacity• Use evaluation findings to improve program practices

• Look for supplemental funding options for evaluation• Federal grants/ Foundation funding• University support (financial and in-kind)

Footer Information Here

Recommendations for Building Evaluation Capacity

For further information contact…

Carrie Markovitz, Ph.D.

Principal Research Scientist

NORC at the University of Chicago

markovitz-carrie@norc.org

(301) 634-9388

Thank You!