Upload
jonah-hill
View
25
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Using your evaluation: Communicating, reporting, improving. Who What (Content) How (Format) When. Why communicate?. - PowerPoint PPT Presentation
Citation preview
1© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
1© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Using your evaluation: Communicating, reporting, improving 1. Who
2. What (Content)
3. How (Format)
4. When
2© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Why communicate?
Success is judged by how much our evaluations and the findings are USED. We don’t conduct evaluations for them to sit on the shelf. Evaluation provides opportunity for learning and we are educators!
“The proper function of evaluation is to speed up the learning process by communicating what might otherwise be overlooked or wrongly perceived. The evaluator, then, is an educator. His success is to be judged by his success in communication; that is by what he leads others to understand and believe. Payoff comes from the insight that the evaluator’s work generates in others.”
- L. J. CronbachCronbach, L.J. (1982). Designing evaluations of educational and social programs. San Francisco: Jossey-Bass, p. 8.
3© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Myths
• One report is enough.
• People read written reports.
• Complex analysis and big words impress people.
• Oral reports have the same effect as written reports.
• Describing limitations weakens report.
• Everything should be reported.
• The audience knows why they are getting the report.
4© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
There are lots of ways to communicate our evaluation results. Which have you used?
• Written report - long, short, summary
• Success story
• Impact statement; spotlight
• Elevator story
• Press release
• Media appearance
• At a public meeting
• Memo, email, fax, postcard
• Newsletter
• Personal discussion
• Bulletin, brochure
• Display/exhibit
• Audio/video presentation
5© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Hopefully, you had a plan for using your information when you started your evaluation.
It would include: WHO? do you want to reach
WHAT? will you report
HOW? will you do it
WHEN? will you report
“Start with the end in mind”
6© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
# 1 WHO do you want to reach?
• Program staff• Youth participants• Parents• Volunteers • Your tenure committee• Funders• Program Committee• Collaborating agencies;
partners• Schools boards, parent-
teacher organizations
• County board/City council (elected and appointed officials)
• Community at large• Church organizations• State legislators• Professional organizations • The Extension and
research community• Businesses; business
groups• Police – law enforcement
Potential Users
7© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
#2 WHAT will you say, share, report, communicate??
• What does the target audience care about?
• What do you want the audience to learn – understand?
• What type of information resonates with the audience: numbers, quotes, stories?
• What data will the target audience find compelling, trustworthy?
• What is your story?
8© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
# 3 HOW will you communicate – what format(s) will you use?
• Report• Impact statement• Executive summary• Email, memo• Personal discussion• Display, exhibit• Press release• Newsletter, bulletin• Poster• Slide presentation• Published article• ??????
Overview
Cooperative Extension- 72 county offices- Family Living Programs- 4-H Youth Development- Community, Natural
Resources andEconomic Development
- Agriculture and NaturalResources Education
- Geologic and NaturalHistory Survey
- 1.4 million educationalcontacts per year
Continuing Education- Partnerships with 26
UW System campuses- UW-Learning
Innovations- School for Workers- 162,000 enrollments per
year
Broadcasting & MediaInnovations- Wisconsin Public
Television- Wisconsin Public Radio- Instructional
CommunicationsSystems
- Nearly 1 million viewersand listeners per week
Small BusinessDevelopment Centers- 13 campus-based
centers- Wisconsin Innovation
Service Center- 15,000 hours of
counseling per year
[email protected](608) 262-3786
University of Wisconsin-Extension programs serving the98th Assembly District
Representative Marc Duff
Waukesha County Extension — The Waukesha County UW-ExtensionCooperative Extension office has developed partnerships with numerousorganizations to deliver educational programs addressing local needs. The Waukesha County Extension office developed a Hunger in Waukesha
County Nutrition Survey for area communities. These communities alsoreceive nutrition education services through the Waukesha County NutritionEducation Program at sites like the WIC clinics, elderly nutrition sites andfood pantries.
For the past two years, budgeting, nutrition and food safety have been taughtthrough Waukesha County Extension’s Nutrition Education Program at theWaukesha County Jail. The program also links inmates to resources that willbe available to them once they are released.
The Nutritional Education Program also works with individuals with drug andalcohol abuse problems who are living at the Waukesha County HuberFacility. The program focuses on healthy eating and the effects of drugs andalcohol on the body, food safety and budgeting.
The Parenting the First Year newsletter is distributed through CommunityMemorial Hospital in Menomonee Falls with funding from the local Kiwanisclub. The Parenting the Second and Third Year newsletter is distributedthrough Community Memorial under a grant from the Child AbusePrevention Fund (CAP). The project currently is being evaluated in order tomeasure the value to parents. As a result of this partnership, parentingeducation programs such as Home Alone, Parenting Your Teen andDiscipline vs. Punishment are also offered.
The Waukesha County Extension office provided leadership to a collaborationof 14 agencies/organizations serving children and families to conduct a two-to-three-year program to disseminate research and conduct comprehensivetraining about early learning and brain development of infants and children.
Waukesha County Extension offers a Grant Opportunities Newsletter, whichidentifies grants available to the community from a variety of state and federalagencies as well as local foundations. The list includes federal agencies, suchas the Department of Health and Human Services, Department of Housing andUrban Development, Department of Labor and Employment and TrainingAdministration, and state entities including the Department of WorkforceDevelopment, Department of Administration - Division of Housing, TEACH,Department of Transportation, Department of Natural Resources and theWisconsin Advanced Telecommunications Foundation.
The Waukesha County Extension-Urban Initiative Program in Sussex willaddress neighborhood revitalization, building a sense of community, crimeand public safety. In 2001 the program received a $10,000 CommunityDevelopment Block Grant which will allow expansion of the partnership andimplementation of an action plan in at-risk, troubled neighborhoods.
9© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
• Monthly?
• Quarterly?
• Annually
• Whenever I have a chance
• When requested?
• ???
# 4 When will you communicate?
10© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Be prepared with an “Elevator story”
A succinct, attention grabbing story that can be communicated in a short elevator ride.
Be armed and ready with your best 30-second story! Your story is important. Your job is to make sure decision makers know it. You never know when opportunity might knock!
11© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Success stories and annual accomplishment reports are ways we report in UWEX
They demonstrate that we are responsible and accountable for our public dollars.
See the guidelines and samples that are posted on this web site.
12© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin-Extension, Program Development and Evaluation
Success story and reporting outlines tie back to the logic model : SRRE
13© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Success Story
Situation:
Response:
Results:
Evidence:
Response
Results
Evidence
INPUTS and OUTPUTS: What did Extension do? What did we contribute? Who participated/benefited? #s of key demographics of participants
OUTCOMES: What changed? For whom? What is the value of the change? What does this mean? What was learned?
What evaluation did you conduct – how do you know that the results are accurate and credible?
Situation What is the situation – the problem, concern, issue that needed to be addressed? Local, regional or state information about the issue. Succinct and compelling.
SRRE
14© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Ingredients of a Good Success Story
• Captures attention of reader• Provides a compelling, convincing story• Gives specific outcomes
(e.g., money saved, skills increased, practices changed, actions taken)
• Includes numbers and narrative• Based on reliable, credible information • Presents balanced, fair assessment• Uses succinct, clear writing
15© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Use graphics and design to make the information interesting and easy to understand
• Charts and graphs
• Overheads
• Pamphlets
• Reports
16© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Discuss limitations of the evaluationWritten reports: • Be explicit about your limitationsOral reports: • Be prepared to discuss limitations• Be honest about limitations• Know the claims you cannot make
– Do not claim causation without a true experimental design
– Do not generalize to the population without random sample and quality administration (e.g., <60% response rate on a survey)
17© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Formal evaluation reports typically include… • Abstract/executive summary
• Introduction– Purpose of the evaluation; key questions– Program background, description
• Methods/procedures– Data sources– Data collection procedures– Sampling– Limitations
• Results• Discussion• Conclusions/recommendations• References• Appendices
18© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Reporting results to the media
All Media:
• Avoid using too many statistics.
• Focus on the key points.
• For quotes, speak more globally about the issue.
• Always give the source and timeliness of your stats. It’s the “news peg.”
Steve Busalacchi
Director, News & Information
Wisconsin Medical Society
19© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Reporting results to the media
Radio and TV:• Do not offer exact statistics
– ear cannot track. “73.6% of respondents”vs.“Nearly three quarters of those surveyed”
• Don’t go into great detail. Have backup info ready.
Steve Busalacchi
Director, News & Information
Wisconsin Medical Society
20© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
As you write and talk, remember these tips for effective communications
• Tailor the message to the topic and the audience
• Avoid jargon and technical terms• Be clear, concise• Use active voice• Eliminate wordiness• Check writing, grammar• Be accurate, balanced, impartial• Be timely• Use graphics, quotes, photos, real stories• Consult a communications specialist• Write-rewrite-rewrite
21© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
You’ve spent time and resources doing your evaluation…make sure it isn’t ignored
Here are some tips: • Engage stakeholders in doing the evaluation (in the design, data
collection, in analysis, interpretation of results), then they “own” the evaluation and are more likely to use it and share it with others.
• Get the information to the right people – target people who should care about the evaluation
• Address issues that people think are important – while you may think everything in the evaluation is interesting and important, your audience may not. Customize your communications to each audience.
• Keep it in front of people; keep talking about the evaluation and what was learned
• Be timely: Share information when it is most likely to be used; when the ‘time is right’ and people are ‘ready to listen’.
22© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
And, remember…“Think like a wise man, but communicate in the language of the people.”
− William Butler Yeats