Measuring Success in Patient Advocacy Initiatives
In an increasingly challenging donor environment, funders want more meaning reporting of success and outcomes by nonprofits. This webinar provides insights and knowledge that can mean the difference between scaling up - or dialing down - key initiatives.
- Measuring Success in Patient Advocacy Initiatives July 9, 2014
- Welcome! Lori Melanon, Senior Director, Corporate Affairs, Onyx Pharmaceuticals Ken Berger, President & CEO, Charity Navigator Carmen Perez, Manager, Measurement and Standards, CECP
- Progress in Cancer: Measuring Success in Patient Advocacy Initiatives A Metrics Study Lori Melanon Senior Director, Corporate Affairs Onyx Pharmaceuticals
- Measuring Impact in the Evolving World of Nonprofit Organizations (NPOs) Increased pressure to measure impact Fierce competition for funding Measuring success a daunting challenge
- Progress in Cancer Study Overview Objectives Evaluate how diverse NPOs in cancer measure impact against goals to assess trends and best practices Rx4good Methodology Primary Research: Group Self-Assessment 24 NPOs participated in self-assessment of metrics best practices via 8-item Survey Monkey questionnaire Secondary Research: Assessment of Websites and Annual Reports 45-question survey evaluated how 35 NPOs report on diverse measures of progress against goals 1 2
- Study Sample Websites and annual reports of 35 NPOs assessed: Self-assessments completed by 24 (69%) of the 35 organizations
- Group Demographics The majority of the 35 assessed organizations have been operating for 20+ years with annual revenue of $1M$5M Age of Organization Annual Revenue $30M 11% (n=4) N/A 3% (n=1) 20+ Years 49% (n=17) 10-20 Years 43% (n=15) 5-10 Years 6% (n=2) 1-5 Years 2% (n=1)
- Primary Research Findings NPO Self-Assessment of Metrics Best Practices
- Survey Revealed NPOs Perspective on Metrics Reporting 75% of NPOs have changed their view about metrics during last three years increased understanding of value of metrics funder expectations 96% Metrics are essential for evaluating success against goals 33% The work we do is difficult to measure 25% Metrics are important but we dont have the resources to measure our programs 8% We would rather invest in programs than in measuring results
- Key Obstacles to Measuring Impact Lack of Resources Determining Outcomes Change We have limited time, staff and funding to do all the things we want to doAnd it's hard to increase staff size without an increase in funding. Measuring impact takes so much staff and time that the programs do not get the attention they deserve. It is the wrong way around. Not all of our programs are easy to track, especially grassroots and advocacy initiatives. It can be can be a challenge to determine the best way to measure impact. It can also be challenging to determine if behavior has really changed due to a program or an event.
- NPOs Measure Impact in Many Ways, Led by Progress Achieved Against Goals 96% 92% 92% 92% 79% 75% 75% 67% 67% 54% Progress Achieved Against Goals Numbers Reached, Attended Events Actions Taken by Stakeholders as a Result of Programs Social Media or Web Engagement Media Coverage Funding Received Perceptions Changed or Reinforced Sustainability of Programs or Initiatives Message Delivery Funding Given Q: What are the ways in which you measure your impact now? (Check all that apply)
- Metrics Ranked Most and Least Valued by NPOs Mirror Those Attributed to Funders Q: Please rank the following metrics options in order of importance to your ORGANIZATION/FUNDERS, with one being the most important, 10 least important. NPO Both Funders Numbers Reached, Attended Events Actions Taken By Stakeholders as a Result of Programs Progress Achieved Against Goals Perceptions Changed or Reinforced NPO Both Funders Funding Received Funding Given Message Delivery Media Coverage Sustainability of Programs MOST IMPORTANT* *Ranked 1 or 2 as most important by the majority of survey respondents *Ranked 9 or 10 as least important by the majority of survey respondents LEAST IMPORTANT*
- Organizations Revealed Best Practices for Measurement Ongoing review of progress against goals Broad program evaluation & follow-up Digital reporting Tracking research advances
- Secondary Research Findings How NPOs Publicly Report Progress Against Goals via Websites and Annual Reports
- Cancer NPOs Have a Strong Commitment to Reporting Results 91% Publish an annual report 89% Report annual revenue & operating expenses 60% Clearly state organizational goals 46% Report progress against each goal
- and a Strong Commitment to Transparency Approximately 80% identify corporate and individual sources of grants Majority report funding given and funding received (65% and 70%, respectively) Dollar size, number and impact of grants both given and received are often disclosed
- Strong Social Media Presence, Yet Less Often Measured or Reported 97% 97% 91%91% 60%49% 46%Measure impact on progress based on social media or web engagement 29%Quantify growth of social media engagement
- Website Research Shed Light on NPO Priority Metrics Audience reach/ event attendance (91%) Legislative/ policy work (74%) Scientific progress (63%) Survey patients about impact of programs (23%) Media coverage (43%) Message delivery (9%) Website hits (29%) Unique visitors (31%) MORE COMMON LESS COMMON
- Key Takeaways and Study Implications Metrics increasingly valued by NPOs, driven largely by funder expectations Tasked with finding more immediate measures of success Limited resources and knowledge on how to best convey impact Variety of reporting techniques; more outputs than outcomes Funders and NPOs largely aligned in metrics they deem most/least important More focused on reporting results than websites/annual reports indicate NPO demographics have marginal bearing on metrics activities Effective metrics reporting not necessarily linked to size or revenues Diverse stakeholders and funding obstacles will continue to challenge metrics reporting, but best practices and cost-effective strategies exist to showcase impact
- Your Guide to Intelligent Giving Where the Heart Meets the Mind Critical Friend to Charities
- The Core Challenge As I See It* there is virtually no credible evidence that most nonprofit organizations actually produce any social value. *The End of Charity by David Hunter Philadelphia Social Innovations Journal
- The Nonprofit Marketplace Hewlett Fdn THE ROADMAP TO A SOLUTION TO THE CHALLENGES INSTITUTIONAL FUNDERS supply $ & technical assistance for meaningful information BENEFICIARIES are engaged & empowered to provide meaningful information Institutional Funders use a set of standardized NP reports of meaningful information Beneficiaries receive feedback on how their meaningful information was utilized
- WHAT IS MEANINGFUL INFORMATION FOR MEASURING A CHARITYS SUCCESS? RESULTS GOVERNANCE FINANCIAL HEALTH
- How Does This Fit into the Evolution of our Rating System? CN 1.0 Financial Health Launched in 2002 with 1,100 charities CN 2.0 Governance Launched Sept 20, 2011 with 5,500 charities CN 3.0 Results Reporting Methodology released and data collection begun Jan 2013, with a goal of 10,000 charities rated by end of 2016
- Results Reporting Is An assessment of how charities use their results internally and share them with stakeholders, including donors We are looking to see that you are reporting on results measures, and showing how your organization learns and improves based on those measures (i.e. learning and improving over time is more important than a snapshot of results)
- How Charity Navigator Evaluates Results Reporting FIVE ELEMENTS: 1. Alignment of Mission, Solicitations and Resources 2. Results Logic and Measures 3. Validators 4. Constituent Voice 5. Published Evaluation Reports
- One Element Will Impact Ratings Next Year Element One: Alignment of Mission, Solicitations and Resources
- Element Two: Results Logic and Measures Is the organizations causal logic, theory of change, plausible? If applicable, is there an indication of how much of the action is required to produce the pre-defined outputs and outcomes? Is this logic based on reasonable evidence? Are there specified measures (indicators) to be collected and a plan to do so?
- Element Three: Validators Have your charitys results been vetted by another organization? Not every charity and cause area will have a validator. That will not diminish your rating.
- Element Four: Constituent Voice How well does your charity collect and publish feedback from your primary constituents (the people who are meant to be the direct recipients of benefits created by the organizations actions). May not apply to every cause area, but will apply to most.
- Element Five: Published Evaluation Reports Does your charity publish evaluation reports that cover the results of its programs at least every five years? Are those reports based on recognized techniques to understand your results? Does you charity explain what, if anything, it is changing as a result of the findings in the evaluation report?
- WHAT WE HAVE LEARNED SO FAR Out of Approximately 1,000 Human Service Charities Evaluated So Far: ~ 70% of Charities have a clearly defined program ~ 15% of Charities have a clear definition of how much of an action is required to produce the pre-defined outputs and outcomes ~2% of Charities have at least 1 of the other 3 elements of Results Reporting (Validators, Constituent Voice or Published Evaluations)
- A DREAM THAT YOU CAN PLAY A ROLE IN MAKING A REALITY RESULTS STATEMENTS, TODAY TOMORROW ?????
- To Learn More About How We Measure Success:
- To Learn More In General About Results Measurement: Books /Articles/ Studies 1. The Battle for the Soul of the Nonprofit Sector, Berger, Penna and Goldberg, Philadelphia Social Innovation Journal 2. Money Well Spent by Paul Brest, et. Al 3. Working Hard and Working Well, by David E. K. Hunter 4. Billions of Drops in Millions of Buckets by Goldberg 5. Leap of Reason by Mario Marino 6. The Nonprofit Outcome Toolbox by Dr. Robert Penna 7. Charity Navigators webinar on how to use our site 8. Saving Philanthropy Video 9. Independent Sectors Charting Impact 10. PerformWell Web Site
- Email: email@example.com Blogs: www.kenscommentary.org blog.charitynavigator.org Twitter: kenscommentary LinkedIn: http://lnkd.in/jKeK5U Register for Newsletter, Etc.: https://www.charitynavigator.org/index.cfm?bay=my.login KEEPING IN TOUCH
- Evaluation: Corporate Perspective July 2014 Carmen Perez firstname.lastname@example.org 212.825.1581
- THE CEO FORCE FOR GOOD
- 1.23% 1.06% 0.98% 0.76% 1.58% 0.96% 0.76% 1.12% 1.25% 1.10% Utilities (n=18) Technology (n=25) Materials (n=16) Industrials (n=25) Health Care (n=24) Financials (n=48) Energy (n=11) Consumer Staples (n=20) Consumer Discretionary (n=26) Communications (n=10) 2013 Industry Giving Comparisons Median Giving as a % of Pre-Tax Profit For Pharmas: 16.1%
- Pharmaceuticals Giving at a Glance, 2013 79% of all companies run a corporate foundation 86% of all companies match employee gifts 62% of all companies give internationally 64% of all companies make non-cash gifts 78% of Pharma companies 89% of Pharma companies 100% of Pharma companies 78% of Pharma companies N=261 for full sample. Pharma n=9.
- A Majority of Companies are Measuring the Societal Value of their Contributions 18% 35% 47% Very Experienced (5 Years or More) Moderately Experienced (3-4 Years) Slightly Experienced (2 Years or Less) 76% Measure Outcomes and/or Impacts N=160. N=119. 24% Do Not Measure Outcomes and/or Impacts
- Nearly a Third of Companies Focused On Outcome/Impact Measurement in a Strategic Philanthropic Program 4 2 For companies only evaluating grants larger than a specific threshold, the average threshold was approximately $105,000 Other descriptions include companies that: Only evaluate foundation grants Evaluate for a combination of strategic programs, grant size, and cause area(s) Evaluate for as many grants as possible without any specific direction Description of Impact Measurement Scope % of Companies All grants, regardless of grant size 23% Only grants made for a strategic philanthropic program(s) 31% Only grants larger than a specific threshold 16% Only grants larger than a specific threshold AND made to a specific cause area(s) 12% Only grants made to a specific cause area (or cause areas) 8% Other 10%
- How much do companies report spending on evaluation? 4 3 48%of companies spent money in 2013 to evaluate grants 5% of 2013 contributions were dedicated to measuring the outcomes and/or impacts associated of their grants Note: Spending money includes contributions to grantees (earmarked for eva...