Evaluation Bloopersand How to Make the Most of Your Mistakes

Panel presentation at the 2015 Eastern Evaluation Research Society Conference, Absecon, NJ

Cheryl M. Ackerman

@AckermanCheryl

University of Delaware

Your Fearless Leaders

Jennifer Hamilton

@limeygrl

Westat

Isaac Castillo

@Isaac_outcomes

DC Promise Neighborhood Initiative

Let’s Get Real: Evaluation Problems & Solutions

We all make mistakes

Open discussion and therapy session

Turning water into wine, lemons into lemonade…

Introduction

Evaluation design

Evaluation administration

Data analysis, cleaning, & reporting

Working with people

Take-aways & Wrap-up

Structure

1. Mistakes happen. Own them and move on. Bad things happen to good evaluators.

2. Courses devoted to mistakes

3. don’t make assumptions, and test/check assumptions

Participant Take-Aways

4. Forgive others for mistakes. Help people through them.

5. Value people sharing mistakes –especially with junior evaluators.

6. Have / find a support system to lean on and share with. 12 step program for evaluators.

Participant Take-Aways

7. Document as much as possible. Transparency is important.

8. Mentor or orient others. Don’t be the evaluator that doesn’t share. Ask for mentoring. Say yes to people that ask you to be a mentor.

9. Multi-level set of control and build in time for review.

Participant Take-Aways

10. Recognize and present on challenges/mistakes. Talk about what didn’t work.

11. On going professional development. Don’t assume you know everything

12. Build in time. And recognize that time spent time in beginning can save time on back end.

Participant Take-Aways

13. Self-evaluation and self-reflection is important. Create time and space to review what worked and didn’t work for each evaluation.

14. Share mistakes with others and use them as a way to talk about solutions and potential remedies.

Participant Take-Aways

Our mistakes change with experience

We all make mistakes, so ask and accept help

We all see mistakes, so provide help (nicely)

Wrap-Up

Acknowledgements