Upload
howard-avery
View
21
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Reasonable ROI Tracking – Measuring Internal and External Impact. Marie Beason Indiana Grantmakers Alliance. Section I: Introduction. Thesis of the Presentation. - PowerPoint PPT Presentation
Citation preview
Reasonable ROI Tracking – Measuring Reasonable ROI Tracking – Measuring Internal and External ImpactInternal and External Impact
Marie Beason
Indiana Grantmakers Alliance
Thesis of the Presentation
Expectations for companies as responsible corporate citizens are rising, and stakeholders are demanding results. Clearly defined goals with strong social impact makes for a more compelling story to tell your stakeholders, resulting in greater impact for the business.
Why Measure?
Increasing Expectations• Internal and external stakeholders are demanding more “results”
• Companies are expected to help address issues, not just give money
• Employees expect their company to be a leading corporate citizen
• Community Leaders expect a commitment to the community
• Customers/Clients expect a company to meet standards
Competitive Advantage• Most companies measure only outputs, not outcomes
• Most companies are unable to articulate how CI affects their business (reputation and/or employee recruitment, retention, morale, etc.)
• Trend is toward defining specific goals for major programs; opportunity still exists to be a leader in an industry or geographic area
Program Improvement• Findings can be relevant, timely and useful
Which Programs Need to be Measured?
Program Content Resources
Signature Program: Teacher Training in Grand Ville
Targeted support of teachers in 3 public schools in poor communities
$100,000 annually
50 volunteers
Employee contributions
Major Focus Areas: Education, Social Services
Support of critical needs in education and social services in Grand Ville
$75,000 annually
Business / Client-DrivenSupport for events, organizations based on client requests to sales reps around the country
$25,000 annually
Employee Engagement (Volunteerism, Dollars for Doers, Matching Gifts, etc.)
Support for matching gifts and dollars for doers
$50,000 annually
APC focuses primarily on its headquarters community and where its employees live and work
Multiple Stakeholders withVarying Needs for Results
Companies need to identify priority stakeholders, issues of concern to those stakeholders and then determine level of measurement needed for each stakeholder
Barriers to Measurement
• Different programs have different expectations of “results”; lack of internal consensus on what to measure
• Portfolio of programs – major initiatives require more rigorous outcomes measurement
• Limited budgets or support for deep measurement and evaluation
• Most companies have not engaged senior management effectively to gain support for increased measurement
• Limited internal knowledge/capacity regarding measurement and evaluation
• Most companies have limited staff and do not have the capacity or knowledge to measure their programs
What Can Be Measured?
InputsInputs
Monitoring:What is the quantity
and quality of the resources we have
invested in this program? Are they
aligned with our program goals?
Performance measurement:
What is the quantity and quality of the products or
services we have delivered?
OutputsOutputs
Outcome measurement:What are the
results for participants,
grantees, employees, the business, and
other stakeholders?
OutcomesOutcomes
Impact study:What are the
social and larger business impacts attributable to the
program/ company?
ImpactImpact
Methods of Measurement
Monitoring Performance measurement
Outcome measurement Impact study
Documenting Research
Tracking investment and progress Determining causality
Minimum evaluation expertise required High level evaluation expertise required
Low cost High cost
Strategies for Measurement
Low-Cost Strategies
•Develop a program model with clear and measurable goals
•Ensure that program vision, goals and objectives align
•Focus on measuring inputs and outputs
•Create a tracking system to regularly monitor and report progress (inputs and outputs)
Medium-Cost Strategies
•Develop a program model with clear and measurable goals
•Create a measurement framework that lays out key measurement questions, indicators, data sources, data collection methods
•Focus on measuring inputs, outputs and short-term outcomes
•Utilize data collection tools that have been developed or could be developed easily
•Create a tracking system to regularly monitor and report progress
Higher Cost Strategies
•Develop a program model with clear and measurable goals
•Hire an external measurement expert to design data collection plans, systems and tools
•Provide PD for grantee’s internal staff around data collection and measurement
•Fund peer-to-peer exchanges re: measurement
1. What Can Be Measured?
InputsInputs
All the resources needed to support a program:•People•Dollars•Marketing•Lines of business
The specific activities and/or programs that serve a particular target audience:•Signature programs•Focused giving•Employee engagement activities
OutputsOutputs
Affects on target audience:•Community awareness, perception of company•Grantee knowledge, attitudes, behaviors•Employee morale, retention, engagement, team work, leadership, productivity
OutcomesOutcomes
The long-term and aggregate effect of the program on overall target audience and the company:•Social impact•Business impact (Reputation among key stakeholders)
ImpactImpact
3. What Level of Information Is Needed?
Signature
Major Issue or Focus Area
Giving
Business-Driven Giving
Employee Programs
Pro
gra
ms
Inputs Outputs Short-term Outcomes Long-term Outcomes Impact
Levels of Information Needed by Stakeholders
Program ContentKey Stakeholder(s)
ReachedLevel of Information
Needed
Signature Focused AllImpact; Long-term
outcomes
Major Issue or Focus Area Giving
Focused on several issues
Regional Stakeholders; Community Partners;
Employees
Outputs; short-term outcomes
Local Area GivingBroad: based on
local needsLocal community leaders,
employeesInputs and outputs, some
short-term outcomes
Client-Driven Giving
Broad: based on client requests
Clients, Business Partners
Inputs
Employee Programs
Broad: based on employee
interests and company priorities
Employees, Community Partners
Varies by program; primarily inputs and
outputs, some short-term outcomes
3. What Tools Can Be Used to Measure?
In House:• Financial databases/reporting• HR surveys and data (recruitment,
retention, morale, productivity, performance)
• Gifts databases• Grantee reports• Line of Business systems and
reports• Existing data (consider marketing
and sales information)• Media Hits• Consumer data
External:• Document/File Review• Surveys• Interviews• Focus Groups• Core Capacity Assessment Tool
(CCAT)
Program Content ResourcesLevel of
MeasuresMeasurement
Tools
Signature Program: Teacher Training in Grand Ville
Targeted support of teachers in 3 public schools in poor communities
$100,000 annually
50 volunteers
Employee contributions
Outcomes
• Teacher survey• Employee survey• Media hits
Major Focus Areas: Education, Social Services
Support of critical needs in education & social services in HQ community
$75,000 annually Outputs
• Financial databases
• Gifts database• Grantee reports
Client-DrivenSupport for events, organizations based on client requests
$25,000 annually Inputs• Financial
databases
Employee Engagement (Volunteerism, Dollars for Doers, Matching Gifts, etc.)
Support for matching gifts and dollars for doers
$50,000 annuallyOutputs - Outcomes
• Financial databases
• HR, survey• Media hits
How to Measure… An Example: APC
4. What Resources Do You Currently Haveor Can You Leverage?
Skills and resources(Also consider other departments)
• People (time and expertise)• Money• Technology• Partners• Leadership
Existing data/ tools(Also consider other departments)
• Marketing reports• Demographic analyses• Community data, reports
Company culture• HR data• Department data
Broader context
5. Who Participates in Measurement?
MeasurementMeasurement
Community Stakeholders,
Grantees, Volunteers
C-Suite Executives
Senior Leaders and Managers
Community Involvement
Staff
Implementer / Coordinator
ReviewerReviewer
Recipients of Findings
6. How Much Will It Cost?
Monitoring
1 – 2%
Performance measurement
3 – 5%
Outcome measurement
5 – 10%
Impact study
10% +
OutcomesOutcomesStrategiesStrategiesInputsInputs
The key “learning-based” question
7. Evaluating Your Program
Which resources were most important for providing “high-quality” service?
Which strategies (program qualities) were most
important for achieving the desired outcomes (including
business outcomes)?
What was the quantity and quality of the
resources we used to implement our
programs?
How much service did we provide and what was the quality of that service?
How were our employees involved (to what extent, in
what ways)?
Did our business or employees change?
Did our service recipients change?
If so, how much and in what ways?
The key “learning-based” question
Key takeaways
Maximize the resources you already have• People (time and expertise)• Money• Technology• Partners• Leadership• Existing data/ tools (Program participation, Program Reach, Marketing data,
Company data, local data about the region and community)
Leverage the resources of other departments• HR• Marketing
Select your measurement level based on your resources, the program you want to measure, and the type of information you need for your targeted audiences
Resources
CEP website (foundations and strategy reports): www.effectivephilanthropy.org/index.php?page=publications
DHFS Evaluation Resource Guide. Wisconsin Department of Health and Family Services. Office of Strategic Finance Evaluation Section. Updated June 2006. http://dhs.wisconsin.gov/aboutDHS/OPIB/policyresearch/EvaluationResourceGuide06-06.pdf
Diversity in Philanthropy website: www.diversityinphilanthropy.org/news/case_studies/pdfs/DPP-Evaluation-Case-Study2.pdf
Davidson, Jane. ‘Evaluation Methodology Basics: The Nuts and Bolts of Sound Evaluation’: www.sagepub.com/authorDetails.nav?contribId=532276
GEO website (see report on Foundations and Evaluation and the Change Agent Project): www.geofunders.org/publications.aspx
Joint Committee on National Standards: www.jcsee.org Kellogg Foundation Program Evaluation and Logic Model Manuals:
www.wkkf.org/~/media/475A9C21974D416C90877A268DF38A15.ashx Mertens, Donna; and Ginsberg, Pauline. The Handbook of Social Research Ethics:
www.sagepub.com/refbooksProdDesc.nav?prodId=Book230293 RWJF Evaluation Series: http://www.rwjf.org/files/research/50349.quality.checklist.final.pdf