Embed Size (px)
Citation preview
Measuring and Improving
System Outcomes
• NAEH Conference, Washington DC
• July 17, 2012
• Katharine Gale Consulting
• Berkeley, CA
• (510) 710-9176,
• www.focusstrategies.net
1
Overview
• What are we measuring?
• What does the data look like?
• How can we use the information for system
improvement?
• at the program level
• at the sector/program type level
• at the system level
Measuring System Performance
Program
Program
Program
Program
Program
Program
Evaluation
Measuring Program Performance
Program
Program
Program
Program
Program
Program
Evaluation
From the new CoC Regs
• "The Continuum of Care must...
• Consult with recipients and subrecipients to
establish performance targets appropriate for
population and program type, monitor recipient
and subrecipient performance, evaluate
outcomes, and take action against poor
performers
• Evaluate outcomes of projects funded under the
Emergency Solutions Grants program and the
Continuum of Care program, and report to HUD"
Key Elements of
Performance
• How many household's secure permanent
housing?
• How quickly?
• Do they gain jobs and/or income?
• Do they experience homelessness again?
Also, consider cost per outcome
Setting your Benchmarks
• How is your system performing new?
• How widely variant are outcomes from program to
program?
• What is aspirational but achievable?
9
Benchmarking Example: Exits to Permanent Housing from Transitional
Housing Programs including average, median and 75th percentile*
*Example based on performance range in 08-09, not actual programs’ performance
Benchmarks by program type
Samples ways to
look at these key
outcomes
Source: Sacramento HMIS and HPRP data, 2010, chart prepared by Focus Strategies
Average Rate of Exits to Permanent Housing - Families
Source: Data from 14 Continuums in seven states that prepared Evaluators for NAEH Performance
Improvement Clinics in 2011-2012
Average Rate of Exits to Permanent Housing - Singles
Source: Data from 14 Continuums in seven states that prepared Evaluators for NAEH Performance
Improvement Clinics in 2011- 2012
Averages are only
part of the story.
Outcome vary
across programs!
Average lengths of stay by outcome
- system wide and by program type
Source: Montana HMIS, 2010 data, Chart prepared by Focus Strategies
Snapshots are only part of
the story. Outcomes vary
across time!
Source: Montana HMIS data, Chart prepared by Focus Strategies
Source: Montana HMIS data, Chart prepared by Focus Strategies
Rates of Return within 12 months after
gaining permanent housing
Source: Data averaged from seven Continuums in four states that prepared Homeless System
Evaluators for NAEH Performance Improvement Clinics in 2011-2012
Returns to Homelessness -
system wide
Source: Montana HMIS data, Chart prepared by Focus Strategies
What about cost?
Investments by Program Type
Families
Source: CA county, Data from provider budgets (only partial participation)
Average Cost Per Exit and Per PH Exit
Source: Data from 14 Continuums in seven states that prepared Homeless System Evaluators for NAEH
Performance Improvement Clinics in 2011-2012
Caveat: costs, like
outcomes, vary
across programs!
What to do with this information
• How can we use the information for system
improvement?
• at the program level
• at the sector/program type level
• at the system level
At the program level
• Programs need to know their performance
• Have the data...
• Know how to read the data..
• Trust the data!
• Opportunities to learn/change/improve
• Training
• Technical assistance
• Time
Sample HMIS Report:
Emergency Shelters EMERGENCY
SHELTERS Jan 1, 2010 - Dec 31, 2010 Dec 2010 Nov 2010
Jan 1, 2009 - Dec 31, 2009 GOAL
Outcome Measures
People obtaining permanent housing 24.2% 26.2% 21.5% 21.3% {30%}
Exiting to streets or shelter 15.3% 18.5% 23.4% --- {<30%}
Exit with employment income 16.1% 17.9% 21.7% 15.6% {20%}
Adults entering with no income who exit with an
income 4.4% 3.7% 3.6% 5.7% {15%}
Efficiency/Process Measures
Exit to Known Destination 67.7% 74.7% 71.8% 57.8% {85%}
% of those who obtain
permanent housing who do so within 60 days 58.1% 17.4% 0.00% 46.8%
{50% <60 days}
9 Source: Alameda County HCD InHOUSE HMIS, 2/8/11
At the program level
• Incentives for change
• resources (increases, new funds, awards)
• recognition
• getting to keep grants
• Opportunities to try something new (and
latitude for experimentation.)
* OK if some programs decide to move on...
At the program-type/sector level
• Understanding of expectations/models
• Opportunities to collaborate, or specialize
• Peer learning exchanges
• Training and technical assistance
• Time...
At the system level
• Clarity about what buying and expecting
• Information for allocation decisions, across
programs and program-types
• Better outcomes!!
• A story to tell!
