Upload
prosper-conley
View
217
Download
2
Tags:
Embed Size (px)
Citation preview
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Down the Rabbit Hole of Data:Tools for Effective Data Collection, Analysis, and Application
Demetrius Glover, Ph.D. - VP of Strategic Information
AgendaPlan for Today1. Introduce the Problem: The Data Rabbit Hole2. United Way of Central Indiana Example
A. Strategic PlanB. Community Impact PlanC. Goals and MetricsD. DashboardE. Data Integration
3. Framework Activity4. Q&A
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Vice President of Strategic InformationUnited Way of Central Indiana, Indianapolis IN
1. Research2. Data Analytics3. Strategy4. Planning5. Leadership
IntroductionDemetrius Glover, Ph.D.
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Visionfor United Way of Central Indiana
Strategy
Planning
Decision Making
Goal Setting
Data Analysis
Drives . . .
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
My Dreamfor United Way of Central Indiana
“UWCI is held accountable to data driven strategies and plans”
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Areas of ResearchWhat is Strategic Information?
Strategic Information Department
Operations
Fundraising
Community
Impact
• Community Assessments• Program Evaluations• Dashboarding
• Fundraising Analytics• Prospect Research• Statistical Projections and Modeling• Survey Development and Analysis
• Human Resources Research• Marketing Research• Constituent Engagement• Database Management
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Problem:
The Rabbit Hole of Data
Data Analysis PitfallsDonor-Staff Conflict
$$$Major Donors
GrantorsFoundations
Knowledgeable Leadership
Experienced StaffProgram ManagersService Providers
Data Analysis PitfallsPublic Knowledge v Expert Knowledge
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Data Analysis PitfallsAspirational Goals v Achievable Goals
GoalsGREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Satisfying Your DonorsReasons for Giving
A feeling of social responsibility
A feeling of giving back to your community
Identifying with a certain cause
A belief that your giving can achieve change
Helping individuals meet their needs
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
57%
62%
65%
70%
74%
Motivations for Charitable Giving
Motivations for Charitable Giving
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Satisfying Your DonorsReasons for Giving
Other
Energizes and inspires you to get involved
Brings the community together to focus on problems
Is an organization that makes sure its programs are effective
Enables you to make the greatest difference
Gets visible results in community
Lets you know what is being done with donoations
Makes sure your money is well spent
0% 5% 10% 15% 20% 25% 30% 35% 40%
5%
5%
7%
9%
11%
14%
21%
28%
Most Important Factor when Deciding to Give to a Charity
Most Important Factor when Deciding to Give to a Charity
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
“When you are studying something to simply find the answer, you are doing research. If you care whether the answer is yes or no, up or down, left or right, you are doing propaganda.”
-Frank Ridzi, Ph.D.
The Need to Produce ResultsSpin becomes a requirement
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
The Myth of CausationCausation v Correlation
Immediate Impact and Outcomes:How soon should we expect
change?
&
Significant Difference: How much difference is
significant?
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Comparison GroupNot so logical comparisons
“This is not apples-to-apples or even apples-to-crabapples; it’s Mars and Unicorns.”
-Heather Girton
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Putting it all togetherScale and Scope – Hypothetical Example
Aspirational Goal
Quality Programs
• Graduation Coach Program
Transformational Outcomes
• Reduces the Number of students that fall behind and DropoutCommunity Level Change
•100% of students graduate high school on time
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
The Secret to Survival in Nonprofit Research?Understand the Realities of the Industry
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Case Study:United Way of Central Indiana
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
2012 Strategic PlanSource of the Problem
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
2012 Strategic PlanStrategy 1: Integrate income and health strategies into our existing education and basic-needs priorities
Sub-Strategies1. Reinforce emphasis on education2. Continue to support a strong human services network3. Continue unrestricted support to certified agencies4. Conduct community assessments to define income and
health priorities5. Revise the outcome section of the agency evaluation
process6. Recruit high-impact agenciesGREAT RIVERS
Great InnovationsTWENTY FIFTEEN
2012 Strategic PlanStrategy 2: Identify and focus on definable niches in education, income, health, and basic needs through which UWCI can make a measurable difference
Sub-Strategies1. Measure and report outcomes of certified agencies2. Integrate income and health strategies with education
strategies3. Engage and convene community partners4. Engage all counties with education/income/health
strategies5. Establish some competitive funding opportunities6. Maximize the impact of donor investmentsGREAT RIVERS
Great InnovationsTWENTY FIFTEEN
2012 Strategic PlanProblems, Limitations & Contradictions
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Alignment WorkgroupStrategy Team
Jay Geshay, MBA Sr VP of Community Impact and
Fundraising
Christie Gillespie, MPA VP of Community Impact
Demetrius Glover, Ph.D. VP of Strategic Information
Denise Baird, Ph.D. Professor of Sociology at Franklin College
Community Impact PlanAlignment Part 1“The purpose of this project is to clearly define the community goals of United Way of Central Indiana and align all of our work with those goals.But we must remember that community impact is more than funding agencies. Funding a program, a collection of programs, or even a collection of agencies will not bring about community impact because programs and agencies can never have enough scale and scope to help enough people to move the big community indicators.The ultimate goal is to align all of our work with our community impact goals. Our programs, partnerships, collaborations, messaging; everything should be aligned.”
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Community Impact PlanAlignment Part 2“Take public policy for example, when our Public Policy VP first saw the Framework he said, ‘This is great. Now I know exactly what to work toward at the policy level.’ Policy change can have more community impact than any particular program, because public policy impacts millions of dollars of government funding, but before we could be successful in bringing about substantial childcare quality legislation, we had to develop a program, provide substantial funding, raise awareness around the issue, and produce results through our quality childcare initiative, so that at the State House, United Way had enough creditability on that issue to effectively advocate for real change. That’s how alignment brings about community impact.”GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Areas of Community Impact
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Correlation between Three Areas Education, Income and Health
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
The New U: Community Impact Strategy
• Set measureable goals
• Align all of our work
• Engage strategic partners
• Report on our progress
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
New DashboardReport On Our Progress – Front
Education Goals
80%of Central Indiana child care centers are high-quality
90%of Central Indiana 3rd graders are at grade-level reading ability
90%on-time high school graduation rate for all Central Indiana students
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
• Reduce percentage of financially unstable families in Central Indiana to 25%
Income Goal
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
• Reduce the childhood obesity rate in Central Indiana by 12% by 2025
• Improve state health rankings for Central
Indiana counties
Health Goals
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
• Ensure that the basic needs of Central Indiana residents are met
Basic Needs Goal
15.7%of people in our service area are unsure of how they will get their next meal.
7% of the needs of Central Indiana callers to 2-1-1 remained unmet (of the 248,415 needs)
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
New DashboardReport On Our Progress - Back
Community Impact FrameworkGoals and Metrics
Community Impact FrameworkOverview
Community Goals• Articulates our aspirational long-term community goals
Focus• Explains why UWCI has chosen the goal and how UWCI seeks to accomplish it
Community Metrics• Community wide metrics that measure our progress toward the community
goals
Program Metrics• Program level metrics collected from agencies that align with community
metrics
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
EIH&B FrameworkEducation Community Goals & Metrics
Goal 1: 90% on-time high school graduation rate for all Central Indiana students
Measure: Cohort High School Graduation Rate
Goal 2: 90% of Central Indiana 3rd graders to grade-level reading
Measure: 3rd Grade ISTEP ELA
Goal 3: 80% of Central Indiana childcare centers are high quality
Measure: Percent of Central Indiana daycare centers at Paths To Quality Levels 3 or 4
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
EIH&B FrameworkEducation Program Metrics
1. # and % of children enrolled in childcare centers rated PTQ 3 or 4
2. # and % assessed for kindergarten readiness using ISTAR-KR
3. # and % of students passing ISTEP English Language Arts (ELA)
4. # and % of students with a 95% school attendance rate
5. # and % of students reporting suspension or expulsion
6. # and % of students progressing to the next grade level
7. # and % of students passing End of Course Assessment (ECA)
8. # and % of students graduating with their 4-year cohort
9. # and % of K-12 students with Individualized Education Plans (IEP) who complete annual goals
EIH&B FrameworkIncome Community Goals & Metrics
Goal: Reduce percentage of financially unstable families in Central Indiana to 25%
Financial Stability is defined as spending less than 30% of monthly income on housing
Community Metrics• Percent of Central Indiana households spending more than
30% of income on housing• Average family income• Average per capita income• Unemployment rate in Indianapolis-Carmel MSA
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
EIH&B FrameworkIncome Program Metrics
1. # and % of clients placed in jobs
2. # and % of clients placed still employed at 90 days
3. # and % of clients reporting increased total income*
4. # and % of clients reporting increased net worth*
5. # and % of clients reporting improved credit score*
6. # and % of clients reporting that they earn a living wage
7. # and % of successfully completing adult education, post-secondary degrees/certifications, or other training programs
*Maintaining increased/improved income, net worth, and credit score should also be considered positive outcomes
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
EIH&B FrameworkHealth Community Goals & Metrics
Goal 1: Reduce the Central Indiana child obesity rate by 12%
Measure: Childhood obesity rate
Goal 2: Improve state health rankings for Central Indiana counties
Measure(s): Adult obesity rate, Tobacco usage rate, Rates of physical inactivity, Excessive alcohol use rate, Mental health provider ratio, Rate of uninsured, and the other metrics in the Robert Wood Johnson Foundation County Health Rankings
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
EIH&B FrameworkHealth Program Metrics
1. # and % of clients maintaining healthy weight
2. # and % of clients participating in physical activity sessions or exercise programs
3. # and % of clients participating in nutrition improvement programs
4. # and % of clients progressing with treatment/therapy plan
5. # and % of clients receiving recommended health screenings
6. # and % of clients receiving or maintaining health insurance*
7. # and % of uninsured clients receiving treatment**
8. # and % of uninsured clients receiving medication***Will also collect data on the number of clients receiving assistance to maintain health insurance**Will also collect data on the number of uninsured clients receiving assistance to receive treatment or medication
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
EIH&B FrameworkBasic Needs Community & Metrics
Goal: Ensure that the basic needs of Central Indiana residents are met
Community Metrics1.Food insecurity rate
2.Number of homeless children
3.Percent of unmet needs for 2-1-1 callers
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
EIH&B FrameworkBasic Needs Program Metrics
1. # of clients receiving emergency shelter*
2. # of clients receiving mortgage/rental assistance*
3. # of clients receiving utility assistance*
4. # of clients receiving meals*
5. # of clients provided transportation*
6. # of clients legal engagements resolved*
*Will collect data on the number of clients served and the number of opportunities filled (i.e. the number of times a service is delivered even if it is to the same client); both should be considered as desired outputs
Impact on Agency AlignmentEvaluations
Low
Alig
nmen
tH
igh
Low Current StandardsHigh
High Alignment /Low Current Standards
High Alignment / High Current Standards
Low Alignment /Low Current Standards
Low Alignment /High Current Standards
Alignment Matrix
Alignment Decisions
High Alignment /Low Current Standards
High Alignment / High Current Standards
Low Alignment /Low Current Standards
Low Alignment /High Current Standards
Alignment Matrix
Invest in Capacity Building to improve strength of an aligned agency
Ideal Agencies; increase funding to support aligned outcomes
Highly at risk for decertification
Potentially at risk for decertification
How is United Way of Central Indiana going to measure this?
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
BackgroundOrigin
• 2011, United Ways across the state agreed that a community reporter would be an effective tool
• 2012, a collection of United Ways in Indiana suggested an initial list of metrics to the Indiana Association of United Ways (IaUW)
• IaUW agreed to provide leadership and funding for a single, web-based tool
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Key Things to RememberIaUW Community Report Card
• Displays a collection of community level Education, Income, and Health metrics for each county and UW service area in Indiana
• These stats will be collected, calculated, and regularly updated by the Polis Center in the same way and at the same time for each county and UW service area in Indiana
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Key BenefitsIaUW Community Report Card
• UW branded resource for community data
• Stats align with UWW Messaging
• Multiple stats for Education, Income and Health
• Complex data calculations readily available
• Identify successes and areas for improvement
• County and UW Service Area Comparisons
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
The Data Integration Plan!
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
That Data Integration Plan will:Allow for Centralized Data Collection and Analysis
• Client level data of United Way of Central Indiana network• Empower United Way research analysts to perform
complex analyses of the data provided by its agencies, programs, and partnerships
• Be a community resource to understand community needs, available resources, and need-service gaps
• Measure the degree to which United Way and its partners are meeting the needs of the community
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
That Data Integration Plan will:Answer the Most Important Questions• Evaluate the effectiveness of individual agencies,
programs, and community efforts • “Connect the dots” between separate programs and
agencies to support horizontal alignment• Measurement of the collective impact of aligned
community efforts• Determine the critical combination of services that
collectively produce transformational outcomes• Determine strategic opportunities to move the needleGREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Data Integration Plan:Diagram
UWCI Community ImpactDatabase
Social/Economic Data
2-1-1 Call Data
Family Financial Stability Data
Student Level Education Data
Agency Outcomes
UWCI Online Assessment
Tool
powered by
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
What are the Challenges?
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Challenges:Things that may derail the UWCI Plan• Resistance to Data Sharing
A. Controlling the messaging of one’s dataB. Data is a product with fundraising value
• Privacy ConcernsLegitimate concerns about client privacy (e.g.: FERPA and HIPAA)
• Resistance to accountability to outcomesCulture of delivering service outputs
• Competing Data Integration Initiativese.g.: Big Data Initiative; Management Performance Hub
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Sounds Great! How long is this going to take?
Realistic TimelinesAlignment Standard Implementation Timeline
Realistic Timelines: Data Integration Plan
Discussion
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Closing Remarks:Conclusions
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Demetrius Glover, Ph.D.United Way of Central IndianaVice President, Strategic [email protected]
For More Information:Contact
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
GREAT RIVERSGreat InnovationsTWENTY FIFTEEN
Down the Rabbit Hole of Data:Tools for Effective Data Collection, Analysis, and Application
Demetrius Glover, Ph.D. - VP of Strategic Information