Upload
kari-brill
View
304
Download
0
Tags:
Embed Size (px)
Citation preview
Program Development and Evaluation: A View from On High Nonprofit Summit – June 30, 2011
David C. Diehl, Ph.D. Assistant Professor, Program Planning and Evaluation
University of Florida
Does your job ever feel like this?
David C. Diehl, Ph.D. B.S. Psychology, University of Wyoming
Ph.D. Human Development, Cornell University
Dissertation: Early Reading Project with Head Start Families
Current Projects:
– Under One Sky: Evaluation of Youth Foster Care and Adoption Program
– United Way Strengthening Families: Process Evaluation of Family Strengthening/Protective Factors with Local United Ways
– Specialty Crop Research Initiative: Evaluation of Effort to Get Better Tasting Fruits to Supermarkets
– Climate Change Project: Evaluation of Effort to Educate Farmers on Climate Change
– Cooperative Extension: Evaluation of Educational Outreach Efforts throughout Florida (financial education, nutrition, etc.)
3 Children, 1 Wife… (10 years of wedded bliss)
Feature model for Wii character at the right…
The important thing is not to stop questioning. Curiosity has its own reason for existing.
- Albert Einstein
Always start with questions.
- David Diehl
Why do you do the work you
do?
What are your greatest
evaluation challenges?
Where can you have the greatest impact?
• What is your passion?
• What do you love?
• What are your gifts/skills?
• When are you the most creative?
When you think
of „evaluation,‟
what are the
first words or
images that
come to mind?
Stream of
Consciousness (Do NOT think)
Some words I
would like you to
start thinking of…
Learning
Creativity
Usefulness
Under My Control
Interesting
and so on…
Why plan programs?
If it were easy, it would
already be done…
What was the US
poverty rate in 2010?
What was the US
poverty rate in 1968?
What was the US
poverty rate in 1959?
14.3%
12.8%
22.4%
Blacks 55% in 1959
Whites 22% in 1959
“It‟s been four decades
since Bobby Kennedy
crouched in a shack along
the Mississippi Delta and
looked into the wide,
listless eyes of a hungry
child. And when Kennedy
turned to the reporters
traveling with him, with
tears in his eyes, he asked a
simple question about
poverty, „How can a country
like our allow it?‟”
Why evaluate programs?
Outcomes-Focused Planning
1. Understand the Situation
2. Define Participant Outcomes
3. Identify “What Works”
4. Create a Program Logic
Model
5. Implement the Program
6. Evaluate and Monitor Progress
7. Learn, Modify, and Improve
Key Principles
Measure, Learn, Improve, Communicate
Aim for Lasting Community Change
Uncover Underlying Issues
Be Specific about Intended Results/Outcomes
Engage the Right People for the Purpose at Hand
Base Decisions on Facts
Source: United Way Worldwide (2009). Outcomes-Focused Planning
Principle 1:
Base decisions on facts
“Numerous studies of Scared
Straight have demonstrated that
the program does not deter
future criminal activities. In
some studies, rearrest rates
were similar between controls
and youths who participated in
Scared Straight. In others,
youths exposed to Scared
Straight actually had higher
rates of rearrest than youths not
involved in this intervention.” - From “Youth Violence: A Report of the
Surgeon General”
We’re #1!! We’re #1!! (although we now have dropped to #7)
Suppose I wanted to create a program to address alcohol use at UF…
(who should I engage in the planning?)
Principle 2:
Engage the right people for the purpose at hand
Principle 3: Be specific about intended results/outcomes
“What gets measured gets done, (which) is a very powerful logic. If we measure something, it makes it important, people pay attention to it, you get it accomplished… The implication of this, however, is dramatic because it means if you measure the wrong thing, the wrong thing gets done.”
— Michael Patton
Principle 4: Uncover underlying issues
“It's so much easier to suggest solutions when you don't know too much about the problem.”
- Malcolm Forbes
If …., then …
If we have these
resources, then we will be able to carry out
these activities
If we carry out these activities,
then we will be able to reach the following people
If we reach these
people in these ways, then we will affect their learning/ attitudes.
If people are learning
and changing
their attitudes, then they
will change their
behavior
If people change their
behavior, social
conditions will also change
Resources/
Inputs
Outputs/
Activities
Short-Term
Outcomes
Long-Term
Outcomes
Intermediate
Outcomes
Assumptions
External Factors
Principle 5: Aim for lasting community change
http://makerealchange.org/
“Experts in his field had figured out how to educate one disadvantaged child, or one classroom full of kids, but the benefits were localized, and usually temporary. And no one had any idea how to change a whole school system or a whole housing project, or for that matter a whole neighborhood. So, in the middle of the 1990's, that's what Geoffrey Canada decided he would do. And now, 10 years later, he has become a very different kind of do-gooder, one with a mission both radically ambitious and startlingly simple. He wants to prove that poor children, and especially poor black children, can succeed -- that is, achieve good reading scores, good grades and good graduation rates -- and not just the smartest or the most motivated or the ones with the most attentive parents, but all of them, in big numbers.”
- New York Times Magazine
Principle 6: Measure, learn, improve,
communicate
http://learningstore.uwex.edu/assets/pdfs/G365
8-1W.PDF
“Being an evaluator is not easy. I’m not referring to the technical problems we face in our work, but to how people react to us and why. Telling someone that you’re an evaluator is like telling them you’re a cross between a proctologist and an IRS auditor. The news evokes a combination of fear, loathing, and disgust, mixed with the pity reserved for people who go where others don’t want them to go.”
-David Chavis, President of the Association for the Study and Development of Community
“Hi, I‟m Dr. Diehl, your proctologist.”
How far you go in life depends on your being tender with the young, compassionate with the aged, sympathetic with the striving, and tolerant of the weak and strong. Because someday in your life you will have been all of these.
- George Washington Carver