Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
From Homelessness to Housing:
Exploring Trends and Improving Data to
Strengthen Transition Projects’ Services
Prepared for Transition Projects, Inc.
By
Daniel Benson
Anna Brunner
Sara Nowakowski
Sara Sanders
Karina Virrueta
Workshop in Public Affairs
Spring 2017
©2017 Board of Regents of the University of Wisconsin System
All rights reserved.
For an online copy, see
http://www.lafollette.wisc.edu/outreach-public-service/workshops-in-public-affairs
The Robert M. La Follette School of Public Affairs is a teaching and research department
of the University of Wisconsin–Madison. The school takes no stand on policy issues;
opinions expressed in these pages reflect the views of the authors.
The University of Wisconsin–Madison is an equal opportunity and affirmative-action educator and employer.
We promote excellence through diversity in all programs.
Table of Contents
List of Tables ...................................................................................................................................v
List of Figures ..................................................................................................................................v
Foreword ........................................................................................................................................ vi
Acknowledgements ....................................................................................................................... vii
Executive Summary ..................................................................................................................... viii
Introduction ......................................................................................................................................1
Background ......................................................................................................................................1
Risk Factors for Homelessness ....................................................................................................2
Programs Supporting Homeless Individuals to Housing .............................................................3
Placements and Rental Assistance ...................................................................................................4
Transition Projects’ Data .............................................................................................................4
Analysis of Placement and Subsidies ..........................................................................................5
Overview ..................................................................................................................................6
Housing Status .........................................................................................................................7
Service Provisions ....................................................................................................................9
Patterns in Being Placed or Receiving a Subsidy ............................................................... 9 Patterns in Receiving Multiple Placements and Multiple Subsidies ................................ 10 Patterns in Receiving an Ongoing Subsidy ....................................................................... 11
Transition Projects’ Data-Collection and Use ...............................................................................12
Discussion ......................................................................................................................................13
Recommendations ..........................................................................................................................14
Recommendation 1 ....................................................................................................................15
Promote an understanding of the importance of data ............................................................15
Create learning forums ...........................................................................................................15
Recommendation 2: Formalize data-collection processes .........................................................16
Example of Improving Data-Collection with Services ..........................................................17
Recommendation 3 ....................................................................................................................17
Process of Evaluating Findings ..............................................................................................18
Continuous Improvement .......................................................................................................18
Recommendation 4 ....................................................................................................................18
Thought Leadership ...............................................................................................................19
Conclusion .....................................................................................................................................19
Appendix A: Notes on Data Cleaning and Recoding ....................................................................20
Appendix B: TPI Demographics ....................................................................................................23
Appendix C: Placements and Subsidies Analysis Tables ..............................................................25
Appendix D: Housing Status Analysis Tables ...............................................................................27
Appendix E: Multiple Subsidy and Multiple Placements Analysis Tables ...................................29
Appendix F: Ongoing Subsidy Analysis Tables ............................................................................32
Appendix G: Survey Questions and Select Results .......................................................................33
Appendix H: Suggestions for Integrating Data into Decision-Making .........................................38
Works Cited ...................................................................................................................................39
v
List of Tables
Table 1. Number of Participants Who Received One or More Placements and/or Rental
Subsidies, 2007-2016 .................................................................................................................6
Table 2. Combining Intensive Support and Stable Housing Findings, 2007-2016 .......................11 Table 3. Demographics of All TPI Participants, TPI Participants from 2007-2016, and TPI
Participants in January 2015 ....................................................................................................24 Table 4. TPI Placements per Year, with Demographic Data .........................................................25 Table 5. TPI Subsidies per Year, with Demographic Data ............................................................25
Table 6. Receive Placement Services Regression Coefficients .....................................................26 Table 7. Receive a Subsidy Regression Coefficients .....................................................................26 Table 8. Housing Status Regression Coefficients Conditional on Receiving a Subsidy ...............27 Table 9. Housing Status Descriptive Statistics ..............................................................................28
Table 10. Multiple Subsidy Regression Coefficients ....................................................................29 Table 11. Multiple Subsidies Descriptive Statistics ......................................................................30
Table 12. Multiple Placements Regression Coefficients ...............................................................31 Table 13. Multiple Placements Descriptive Statistics....................................................................31 Table 14. Ongoing Subsidy Regression Coefficients ....................................................................32
Table 15. Ongoing Subsidy Descriptive Statistics .........................................................................32 Table 16. Q1: On average, how many hours a week do you typically spend entering data? (%) .33
Table 17. Q2: This week, how many hours did you spend on data entry? (%) .............................33 Table 18. Q3: How would you rate the quality (considering accuracy, completeness, usefulness)
of the data you enter? (%) ........................................................................................................33
Table 19. Q4: On average, how many hours a week do you spend using data (including
analyzing, cleaning, and using data to make decisions)? (%)..................................................34
Table 20. Q5: This week, how many hours did you spend using data (including analyzing,
cleaning, and using data to make decisions)? (%) ...................................................................34
Table 21. Q6: How would you rate the quality (considering accuracy, completeness, usefulness)
of the data you use? (%)...........................................................................................................34
Table 22. Q7: What is your position at Transition Projects? (n) ...................................................35 Table 23. Data Culture Questions, % Agree or Strongly Agree ....................................................35
List of Figures
Figure 1. Number of Placements and Subsidies, 2007-2016 ...........................................................6 Figure 2. Distribution of Subsidy Length .......................................................................................7 Figure 3. Proportion of Non-White Placement and Subsidy Participants .......................................7 Figure 4. Estimates of Various Subsidy Lengths on Stable Housing Outcomes compared to One-
Month Rental Assistance ...........................................................................................................8 Figure 5. Incorporating Data into Organization Decision-Making Model ....................................14
vi
Foreword
This report is the result of collaboration between the Robert M. La Follette School of Public
Affairs at the University of Wisconsin–Madison and Transition Projects, Inc, a nonprofit housing
organization in Portland, Oregon. The project provided graduate students at the La Follette
School the opportunity to improve their policy analysis skills while contributing to the capacity
of Transition Projects to better serve its community.
The La Follette School provides students with a rigorous two-year graduate program leading to a
master’s degree in public affairs. Students study policy analysis and public management as well
as concentrated study in at least one policy area. The authors of this report all are in their final
semester of their degree program and are enrolled in the Public Affairs 869 Workshop in Public
Affairs at UW–Madison. Although studying policy analysis is important, there is no substitute
for engaging actively in applied policy analysis as a means of developing policy analysis skills.
The Public Affairs 869 Workshop gives graduate students that opportunity.
Transition Projects is a national leader in providing life-changing services to people in need.
Transition Projects engaged with La Follette School of Public Affairs students to perform
rigorous research using data collected over the last decade in the process of delivering thousands
of services. This report includes an analysis of administrative data as well as interviews and other
research with the goal of helping Transition Projects improve services for individuals and
families in Portland.
I am grateful to Transition Projects for partnering with the La Follette School on this project. The
staff of Transition Projects have been exceptionally generous with their time to support this
project, including collaborating on data analysis. The students have collectively contributed
hundreds of hours to this project and in the process developed critical insights about state
policies and programs. The La Follette School is grateful for their efforts and hopes that this
report proves valuable for Transition Projects to improve the outcomes for the people it serves
and inform similar programs nationally.
J. Michael Collins
Professor of Public Affairs
May 2017
Madison, Wisconsin
vii
Acknowledgements
This report is the result of collaborative efforts from numerous contributors beyond the authors.
The La Follette School team would like to thank all those who provided support and expertise
throughout this project. The final product serves as a testament to these individuals’ commitment
to the pursuit of knowledge and the drive to induce positive, practical results. These individuals
provided critical advice and vital insight throughout our efforts, the results of which would not
have been possible without their guidance.
These dedicated individuals include, but are certainly not limited to:
Ryan Dunk, Logistics Coordinator and our client contact at Transition Projects;
Tony Bernal, Christian Heinlein, David Katz-Wigmore, Megan O’Keefe, and the entire
Transition Projects staff;
J. Michael Collins, our project advisor at the Robert M. La Follette School of Public Affairs,
whose expertise and unending encouragement served as indispensable resources for our team
throughout this project;
Jennifer Chang, Senior Policy Coordinator at the Portland Housing Bureau, for her critical
insight into Portland’s homeless crisis and Transition Projects’ place within the community;
Lisa Hildebrand, Senior University Relations Specialist for the La Follette School of Public
Affairs, for her editorial support; and,
Professors Rourke O’Brien, Hilary Shager, Timothy Smeeding, and our student colleagues at the
La Follette School of Public Affairs, who served as critical sounding boards for ideas and
provided invaluable counsel that was instrumental to ensuring the quality of this report.
viii
Executive Summary Effective organizations use quality data to learn more about their participants, better target their
services, and inform their decisions. Challenged by uniquely high and increasing levels of
homelessness, local leaders and homeless service providers in Portland, Oregon, are grappling
with how to better address the needs of the growing homeless population. One of the foremost
homeless service providers in the region, Transition Projects, Inc., has begun to explore how it
can best utilize its wealth of data as a resource to improve its homeless services.
Using program data, this report provides a descriptive analysis of Transition Projects’ key
services, with a focus on placement assistance and rental-subsidy services. Aligning with
prevailing research, some findings within the analysis suggest a systematic relationship between
level of assistance and stable housing; however, the strength of the relationship depends on the
person and his/her situation.
Inconsistencies within Transition Projects’ data challenge the validity of these findings and
prevent a more robust analysis of the organization’s services. To better understand the limitations
in the data, a survey explored how staff collect, use, and feel about data. This report uses
findings and observed limitations in the data, results from the survey, and prevailing research on
effective data use to provide the following actionable recommendations:
1. Further integrate data into Transition Projects’ culture: Promote a shared understanding
across the organization about the value of data. Encourage employees to share best practices
for collecting data.
2. Formalize data-collection processes: Discuss and address inconsistencies in current data-
collection and entry processes. Involve employees from all levels in defining data and
establishing consistent procedures, facilitate buy-in, accountability, and improved data
quality.
3. Evaluate data in terms of programmatic intentions and integrate into decision-making: Consider analysis findings, potential programmatic reasons that may explain them, and
alignment with organization intentions. Consider the outcomes of interest and think about how
to better collect existing data points to facilitate evaluation.
4. Garner sector-wide support for building the evidence base for homelessness programs:
Capitalize on Transition Projects’ position as a regional leader to foster venues for
information sharing across the county and the sector. Emphasize the current limitations of
existing data systems, and provoke discussions of improved strategies to more precisely
evaluate the effectiveness of assistance programs.
Quality data-collection and utilization can improve organizational decision-making, prompt
thoughtful and regular evaluations of programs, increase the efficiencies of service, and further
an organization’s mission. By implementing sustained changes to how the organization collects
and utilizes data, Transition Projects can improve its data to more effectively and efficiently
move people from homelessness to housing.
1
Introduction
Using quality data can help an organization learn about itself and be more effective. Quality data
allows an organization to “monitor program outcomes, expand system reporting capabilities, and
improve the validity of data analysis results” (OCPD, 2007). By evaluating its program
effectiveness and resource allocation, an organization has greater capacity to identify positive
outcomes and opportunities for improvement.
Quality data can also increase an organization’s credibility by establishing an evidence base that
demonstrates program outcomes and impacts. If an organization can highlight its successes and
point to data that verifies those successes, it can paint a more convincing picture for requesting
additional funding to expand programs. Alternatively, data can provide evidence of possible gaps
in services, such as high-risk populations that an organization is not reaching given its current
resource capacity.
Five factors maximize the utility of data an organization collects: (1) validity, (2) completeness,
(3) consistency, (4) accuracy, and (5) verifiability. Poor quality data that does not meet all five
components can impede an organization’s ability to understand its impact, possibly obstructing
its decision-making processes and hindering its mission. In short, “if [the data] is flawed […]
then decisions based on that data are more likely to be flawed” (Fisher, 2001).
Transition Projects (TPI) is a nonprofit homeless service provider based in Portland, Oregon.
The organization is concerned with identifying effective services that help individuals and
families maintain their housing. Additionally, TPI is interested in an evaluation of its own
services and how to enhance its data-collection processes.
This report details known risk factors associated with homelessness and identifies promising,
evidence-based programs for helping individuals experiencing homelessness re-establish housing
stability. To better target its own homelessness services, TPI is also interested in understanding
who it serves, who receives more intensive services, and how these services and service
intensities relate to outcomes. This report provides an analysis of TPI participant and service
data. However, because data limitations hinder the analysis, validity and interpretation of the
findings is limited. To further understand the data limitations as well as TPI’s current use and
attitudes toward data, this report also presents findings from a staff survey. Finally, the report
provides recommendations for TPI to move toward incorporating data into decision-making and
address data limitations, and, as a result, help TPI become a more effective homeless service
provider.
Background
Transition Projects (TPI) is a nonprofit agency with a mission of moving people from
homelessness to housing. Established in 1969, TPI is the largest homeless service provider in
Portland, Oregon, employing over 200 staff across nine program sites. Annually, TPI finds
permanent housing for 800 to 1,000 households, shelters several thousand people, and helps
more than 10,000 people meet their basic needs. The agency administers more than $2 million in
short- and long-term rental assistance to formerly homeless households and offers supportive
2
services such as case management, support groups, and classes to help people maintain their
housing (TPI, 2016).
Recent and dramatic increases in Portland’s population have exacerbated the need for TPI’s
homeless services. Since 2010, Portland’s population increased by 11 percent, from about
566,700 to 612,200 people in 2016 (ACS). This spike in population has put an increased strain
on the area’s housing market, which cannot keep up with demand. As a result, Portland’s rental
vacancy rates are low and gross rents are increasing. Between 2009 and 2015, the median gross
rent increased by 25 percent, to $971 per month (ACS, 2015).
Portland is also facing an affordable housing shortage. The number of people who are cost-
burdened by their rent, defined as those who pay more than 30 percent of their income for
housing, is increasing. The waitlist for public housing at Home Forward, the housing authority
for Portland, doubled in the last five years while the waitlist for Section 8 housing assistance
vouchers quintupled (Home Forward, 2010 2015). Additionally, the affordable housing shortage
has led to an increase in people facing homelessness so severe that the city of Portland
announced a State of Emergency on Housing and Homelessness in 2015.
Risk Factors for Homelessness
Nearly 3.5 million people nationally and 3,800 individuals in Portland and Multnomah County,
Oregon, experience homelessness in any given year (Smock, 2015). However, the risk of
homelessness is not experienced equally by all people – certain demographic sub-groups have a
greater risk of becoming homeless.
The main cause and primary risk factor of homelessness is economic hardship, such as losing a
job. However, a person does not need to be unemployed to be under threat of homelessness.
Underemployment and the low wages that come with it can similarly put individuals and families
at risk of becoming homeless. This is especially true in metropolitan areas where there is little
affordable housing and wages do not cover even the most basic living expenses (NPACH). The
most frequent causes of homelessness among participants in the 2015 point-in-time (PIT)
estimate were unemployment and inability to pay rent (Smock, 2015). Additionally, high
housing costs in Multnomah County have resulted in a need for 25,000 housing units that would
be affordable to the lowest-income renters (Smock, 2015).
While unemployment and underemployment are regarded as the leading causes of homelessness,
the correlated risk factors of mental health problems and substance abuse play critical roles in
elevating the odds that an individual may become homeless. The prevalence of mental health
issues is three to four times higher among homeless individuals, and the rate of alcohol abuse co-
occurring with one or more psychiatric conditions is five times greater (Shelton et al., 2015). In
Multnomah County, 57 percent of homeless individuals have disabling conditions, including
mental health, substance abuse, HIV/AIDS, developmental disabilities, and chronic health
conditions. In total, 49 percent of the homeless population in this area has substance abuse
problems and 36 percent has mental health problems (Smock, 2015).
Men and those identifying as black or African-American are also disproportionately more likely
to be homelessness. Over 60 percent of all sheltered homeless individuals are male and, despite
3
making up only 13 percent of the total U.S. population, 40 percent of the sheltered homeless
population was black or African-American in 2014 (Johnson et al., 1997) — more than three
times the rate of the general population. Multnomah County has a comparable proportion of male
homelessness (69 percent). Black or African-Americans are 24 percent of the homeless
population (Smock, 2015), which is about five times the rate of the area’s black and African-
American population overall (about 5 percent) (ACS).
While the number of veterans who are homeless decreased by 47 percent between 2009 and
2016, veterans still are more burdened with homelessness than any other demographic sub-
group, totaling about nine percent of the homeless population (HUD, 2016). According to a
Portland-Multnomah-Gresham Continuum of Care (CoC) PIT estimate, veterans make up a
slightly larger proportion of the homeless at 12 percent (Smock, 2015).
Programs Supporting Homeless Individuals to Housing
Since the McKinney-Vento Homeless Assistance Act of 1987, a range of innovative programs
have emerged nationally, including rapid re-housing, transitional housing, and permanent
supportive housing (National Coalition for the Homeless, 2006).
The U.S. Department of Housing and Urban Development (HUD) defines rapid re-housing as an
intervention that “rapidly connects families and individuals experiencing homelessness to
permanent housing through a tailored package of assistance that may include time-limited
financial assistance and targeted supportive services” (HUD Exchange, 2014). Transitional
housing is “a project that is designed to provide housing and appropriate supportive services to
homeless persons to facilitate movement to independent living within 24 months, or a longer
period approved by HUD” (HUD). Permanent supportive housing is permanent housing with
indefinite leasing or rental assistance paired with supportive services to assist homeless persons
with a disability or families with an adult or child with a disability achieve housing stability
(HUD Exchange, 2017).
HUD’s Family Options Study (2016) finds that these interventions may not be equally
appropriate for all homeless populations. For instance, temporary assistance provided by
transitional housing and eviction prevention may be a more effective solution for individuals and
families who are experiencing or under threat of homelessness due to a sudden and temporary
psychological, social, or economic shock (Gubits et al., 2016; Curtis et al., 2013). In these cases,
temporary assistance may go a long way toward re-establishing housing stability. Short-term
assistance, on the other hand, may provide fewer positive outcomes for individuals and families
with greater needs, such as chronic health problems or long-term unemployment. More long-
term assistance provided by permanent supportive housing can lead to greater housing stability
(Gubits et al., 2016). Additionally, a long-term subsidy, longer than 18 months, can increase
positive outcomes such as adult and child well-being (Gubits et al., 2016).
More recently, homelessness programs have tried new approaches to stabilizing housing for
individuals and families. Specifically, there has been an increased emphasis on Housing First
programs as well as rental subsidy programs for those struggling to pay for increasing rental
costs. The Housing First intervention “[provides] permanent, independent housing without
prerequisites for sobriety and treatment [… and] removes some of the major obstacles in
4
obtaining and maintaining housing for consumers who are chronically homeless” (Stefancic and
Tsemberis, 2007).
Due to drastic increases in rental costs, particularly in cities like Portland, rental subsidy
programs have become more prevalent in supporting those who are unable to afford rent. Studies
have shown some individuals or families are homeless only because their income was
insufficient to maintain housing (Smock 2015). Rental assistance programs can be used for a
one-time eviction-prevention payment or for ongoing subsidies for an individual or family.
Improving the efficacy and efficiency of homeless services in the Portland-Multnomah-Gresham
CoC necessitates a clear understanding of those who utilize homeless services in the greater
Portland area and how effective these services are in ensuring stable housing. In turn, a complete
picture of the area’s homeless population requires a robust data-collection system built upon a
data-friendly organizational culture. TPI, in its mission to move people from homelessness to
housing, requires understanding both the homeless populations they serve and effective use of its
collected data.
Placements and Rental Assistance
This section presents the following: a brief description of TPI’s data-collection system; an
overview of the demographic groups to whom TPI provides placement and rental assistance
services and how these have changed over time; and a deeper analysis into patterns in participant
characteristics and their relationship to housing status for whom housing status after receiving a
subsidy is known.
Transition Projects’ Data
TPI collects a wide range of data about its participants. As required by HUD and its national
CoC program, all participant data is entered in the Homeless Management Information System
(HMIS). HMIS has emerged as a valuable resource for homeless service providers, allowing
organizations to analyze participant, service, and housing data and track unduplicated data across
many projects in a CoC.
TPI’s HMIS software collects a wide range of data on each participant such as information on:
basic demographics, including race, ethnicity, and veteran status; health and disability status;
income and welfare; location and type of services received; and current or most recent housing
status. TPI uses this data to track participant interactions and demonstrate outputs and outcomes
to funders.
For this analysis, TPI provided data on about 18,200 participants who received services between
2000 and 2016. Due to inconsistencies in older data, this analysis uses data from only 2007 to
2016. That data was merged and cleaned to result in about 17,300 usable records (Appendix A).
See Appendix B for demographic information on all TPI participants during these years. In the
following analysis, a statistical method, called an OLS regression, was used to identify
significant characteristics related to the outcome of interest. This method permits identification
of relationships while controlling for program and participant characteristics, including time
trends, race, and service provisions.
5
TPI is interested in understanding who it serves, who receives more intensive services, and how
these services and service intensities relate to outcomes. However, due to significant data
limitations, the analysis focuses on two TPI services: placement assistance and subsidy
provision.
No causal relationships can be inferred between participants and TPI programs due to an
inability to control for factors such as self-selection and need, case manager discretion, and
resource availability.
Analysis of Placement and Subsidies
Research has shown the effectiveness of programs that prioritize rapid, permanent housing
placement and/or provide rental subsidies for people and families who are homeless or at-risk of
becoming homeless; both strategies are vital tools in TPI’s service toolbox. TPI’s placement
services include participants who self-resolve into housing, participate in rapid re-housing or
permanent supportive housing programs, and/or receive eviction prevention funds. Rental
assistance subsidies are a primary component of these programs. The subsidies are rent
payments, made by case managers, to landlords on behalf of participants. Some placement and
subsidy programs require TPI to follow up and track the participant’s housing status three, six,
and 12 months after receiving a subsidy.
This analysis begins with a brief overview of TPI’s placement and subsidy programs from 2007
to 2016. The analysis proceeds with an exploration of this service data and its relationship with
outcome data – housing status – to identify participants more likely to receive the services,
characteristics related to service provisions, and associations between these service provisions
and housing status. Specifically, the analysis is designed to answer the following three questions:
1. What participant characteristics and/or services are significantly associated with being in
stable housing at follow-up?
2. What participant characteristics are associated with service provisions, namely receiving
more intensive support?
3. Do the identified patterns provide possible explanations for participant characteristics
associated with being in stable housing?
The significant data limitations that restricted analysis to only two of TPI’s services further
hamper the validity of findings and interpretations for questions 1 and 3. More specifically,
because subsidy data is the only service data that includes housing status, the findings for
questions 1 and 3 are conditional on participants receiving a subsidy. Further, without an
appropriate comparison (or control) group, interpretation cannot account for stable housing
findings for those who do not receive a subsidy.
As a result, these findings and interpretations about housing status are presented with less
confidence. However, although causality cannot be inferred, understanding identified patterns,
potential programmatic reasons that may explain them, and if the patterns align with organization
intentions can be a useful step to guide thinking, incorporate data into decision-making, and help
TPI be more effective.
6
Overview
Between 2007 and 2016, TPI provided nearly 8,000 placements to 6,300 individual participants
and over 7,400 rental assistance subsidies to 5,500 participants (Appendix C). On average, TPI
provided almost 800 placements and over 550 subsidies each year (Figure 1).
Figure 1. Number of Placements and Subsidies, 2007-2016
Source: Transition Projects HMIS Data, 2007-2016
This discrepancy in the number of services provided and participants served shows some
participants were placed multiple times and/or received multiple subsidies. As Table 1 shows,
less than one-quarter of participants were placed or received a subsidy more than once.
Receiving multiple placements or multiple subsidies may indicate a higher level of service
provisions for some participants.
Table 1. Number of Participants Who Received One or More Placements and/or Rental Subsidies, 2007-2016
Number of Subsidies Received
Number of Participants
% of Total Subsidy
Participants
Number of Times Placed
Number of Participants
% of Total Placement Participants
1 4,337 78.5 1 4,921 78.0
2 906 16.4 2 1,054 16.7
3 220 4.0 3 259 4.1
4 49 1.0 4 56 1.0
5+ 14 <1 5+ 19 <1
Source: Transition Projects HMIS Data, 2007-2016 (Subsidies n = 5,526 participants; Placements n = 6,309 participants)
Similarly, the length of the subsidy a participant receives varies considerably. While the average
subsidy length is about four months, subsidy length ranges considerably from one month to
nearly seven years (Figure 2). The majority of subsidies provided one month of rental assistance
(64 percent); just over one-third of all subsides (36 percent) were two months or longer.
Conversations with TPI staff suggest this may result from a lack of available financial resources
to give participants longer-term, ongoing subsidies.
729
714
740 702650
861
701806 870
1,203
690
728
703631 630
869
665
927 892
682
2007 2008 2009 2010 2011 2012 2013 2014 2015 2016
Placements Subsidies
7
Figure 2. Distribution of Subsidy Length
Source: Transition Projects HMIS Data, 2007-2016 (n=6,561)
Since 2007, TPI has served more men, domestic violence survivors, and veterans experiencing
homelessness, with the sharpest increase (over 160%) occurring for veterans (Appendix C). TPI
has also served more non-white participants; however, when compared to the total number of
individuals served, the proportion who are non-white has decreased slightly (Figure 3).
Figure 3. Proportion of Non-White Placement and Subsidy Participants
Source: Transition Projects HMIS Data, 2007-2016 (Subsidies n = 5,526 participants; Placements n = 6,309 participants)
Housing Status
Of the approximately 5,500 subsidies provided, the overall rate of subsidy-receiving participants
in stable housing at follow-up is 80 percent. This section seeks to identify patterns in participant
characteristics and service provisions and their relationship to housing status. The analysis
utilizes data for only those participants given subsidies requiring follow-up. This is problematic
for the validity of the findings and interpretation presented. Each of the following estimates is the
relative higher (or lower) rate of likelihood of being in stable housing associated with each
factor, conditional on receiving a subsidy and controlling for other factors.
64%
15%
9% 7%2% 2%
1 month 2 to 6months
7 to 12months
13 to 18months
19 to 24months
Over 2 years
% o
f P
art
icip
ants
Subsidy Length
27%32%
23%
27%32%
29%
2007 2008 2009 2010 2011 2012 2013 2014 2015 2016
Placements Subsidies
8
Conditional on receiving a subsidy and receiving a follow-up, several demographic
characteristics are associated with being in stable housing at statistically significant levels
(Appendix D). Males and non-white participants are less likely to be in stable housing at follow-
up; males are 9 percentage points less likely, and non-white participants are 2 percentage points
less likely. Conversely, veterans are more likely to be in stable housing at follow-up (3
percentage points).
Additionally, conditional on receiving a subsidy, there are interesting patterns between service
provisions and stable housing. For example, being placed more than once is positively associated
with being in stable housing at follow-up (3 percentage points). This may suggest that it takes
time to find the best fit between a placement and a participant’s needs or characteristics.
However, multiple placements can also be interpreted as an ineffective use of resources which
does not align with being more likely to be in stable housing, suggesting more data is needed to
investigate this finding.
Subsidy length is also significantly related to housing status. When compared to receiving one
month of rental assistance, receiving a subsidy for two to six months is negatively related to
stable housing (-6 percentage points). While a subsidy of seven to 13 months is not any different
than receiving one month of rental assistance, a subsidy for 14 or more months is positively
related to being in stable housing (7 percentage points).
Figure 4. Estimates of Various Subsidy Lengths on Stable Housing Outcomes compared to One Month Rental Assistance
1 Month
Source: Transition Projects HMIS Data, 2007-2016 (n=6,561)
Note: Parameter estimates with 95% confidence limits
These findings suggest a longer-term subsidy (14 months or longer) is more positively related to
stable housing than receiving one month of rental assistance. Because subsidy length is
influenced by participant selection and need, case manager discretion, and resource availability,
these findings only identify patterns and trends, and do not confirm that a longer subsidy
contributes to being in stable housing. However, research such as the Family Options Study
9
(2016) does support the pattern stating that families that receive a long-term housing subsidy
(defined as 18 or more months) are significantly less likely to be homeless (Gubits et al., 2016).
These findings also suggest subsidies lasting between two and six months have a small but
statistically significantly negative relationship with stable housing than one month of rental
assistance. Though this negative relationship is unusual and unexpected, the findings may
support the notion that some individuals and families require only short-term financial assistance
to overcome a sudden and temporary shock that leads to homelessness (Gubits et al., 2016).
However, it is counterintuitive that a longer subsidy would be worse than a shorter one,
suggesting the finding could be the result of inconsistent data-collection and entry.
Together, these findings are consistent with previous research finding that the relationship
between financial assistance and stable housing depends on the person and his/her situation
(Gubits et al., 2016). However, data limitations challenge the validity of the findings and
underscore a need for improved data quality.
Service Provisions
Because levels of assistance and participant characteristics have varying relationships with a
participant’s likelihood to be in stable housing, it is worthwhile to investigate which participants
are more likely to receive the services as well as the levels of services that may lead to stable
housing. This section provides a descriptive analysis of participant characteristics and their
relationship to various service provisions, including:
1. Conditional on being a TPI participant, the likelihood of being placed and/or receiving a
subsidy,
2. Conditional on being placed and/or receiving a subsidy, the likelihood of receiving more
intensive support, specifically, a longer subsidy, multiple subsides, or multiple
placements.
After being analyzed on their own, each of these findings is subsequently considered in line with
the stable housing findings from above: Does understanding who receives services and the levels
of services explain why particular sub-groups are positively or negatively related to being in
stable housing? These interpretations are purely conjecture and are intended to prompt further
questions and discussion.
Patterns in Being Placed or Receiving a Subsidy
When compared to all TPI participants, several characteristics are more strongly correlated with
being placed or receiving a subsidy (Appendix C). As previously
noted, those who received a placement are also very likely to
have received a rental subsidy, and vice versa. In fact, this is the
strongest predictor for both being placed and receiving a subsidy
(nearly 90 percentage points in both cases).
Analysis also showed that veterans are more likely to receive
placement assistance than non-veterans (7 percentage points), but
are less likely to receive a subsidy (4 percentage points less).
While this could be due to a funding source that is not growing at
Patterns in Being Placed or Receiving a Subsidy
More likely to be placed: Veterans
More likely to receive subsidy: Domestic Violence Survivors
Less likely to be placed: Males
Less likely to receive subsidy: Veterans
10
the same rate as veteran participation, conversations with TPI staff revealed a recent focus on
increasing services for veterans. Because this shift only recently occurred, the data may not
reflect the impacts yet.
Males are slightly less likely to be placed (1 percentage points less). Domestic violence survivors
are slightly more likely to receive a rental subsidy (2 percentage points). While these findings are
statistically significant, they are relatively small and suggest no demographic sub-group is being
targeted for participation in these services. This is consistent with TPI’s client-driven practice for
determining service allocation.
Though receiving placement or subsidy services does not appear to target particular participants,
service provision is much more nuanced; participants may receive varied intensities of placement
and subsidy services. For example, a participant may receive more than one subsidy or housing
placement and/or a longer subsidy. The next sections explore these nuances, identifying
participant characteristics related to receiving more intense services and considering how these
nuances fit with housing-status findings.
Patterns in Receiving Multiple Placements and Multiple Subsidies
Although most participants are placed only once and receive only one subsidy, some receive a
higher level of service, such as being placed multiple times or receiving multiple subsidies.
Previously, multiple placements were also found to be positively associated with being in stable
housing at follow-up.
Conditional on being placed and/or receiving a subsidy, several
characteristics are more strongly associated with being placed or
receiving a subsidy more than once (Appendix E). There is a
strong relationship between being placed multiple times and
receiving multiple subsidies; if a participant received multiple
placements, he/she is 70 percentage points more likely to receive
multiple subsidies, and vice versa. This aligns with the fact that
the two services are largely provided in tandem.
Veterans and non-white participants are more likely to receive multiple subsidies (about 3
percentage points for both), but less likely to be placed multiple times (4 percentage points less
and 2 percentage points less, respectively). While this may indicate increased organizational
capacity to offer financial support and more effective placement, data restrictions and limitations
in the analysis make it difficult to interpret the findings as a positive or negative outcome. For
example, the data does not include the monetary value of a subsidy, and analysis does not
indicate whether a participant received multiple short-term or multiple long-term subsidies;
limited funding and varying funding cycles may call for multiple bursts of lower value or
shorter-term rental assistance. Similarly, the analysis could not control for the length of time
between additional placements. A participant returning for placement services a short time after
being placed initially would indicate a very different situation than someone who returned years
later.
Patterns in Receiving Multiple Placements/Subsidies
More likely to receive >1 subsidies: Veterans, Non-whites
Less likely to be placed >1 time: Veterans, Non-whites
11
It is more difficult to interpret the positive relationship between multiple placements/multiple
subsidies and stable housing at follow-up. For example, veterans and non-white participants have
the same patterns in receiving these multiple services, yet their trends for stable housing are
opposite: veterans are more likely to be in stable housing at follow-up, non-white participants are
less likely. This may indicate that the strength of the relationship between more intense services
and stable housing is dependent on the person and his/her situation. But, because of the above
limitations, further research is required to explore this puzzling finding.
Patterns in Receiving an Ongoing Subsidy
A longer-term subsidy is another indicator of a higher level of service and is positively
associated with housing status if the subsidy is 14 months or greater. Data limitations do not
permit analysis of participant characteristics related to receiving a 14-month or longer subsidy;
however, analysis of participant characteristics associated with receiving a subsidy for two or
more months reveals some interesting trends (Appendix F).
Conditional on receiving a subsidy, males are less likely to receive
an ongoing subsidy (3 percentage points less), while domestic
violence survivors are significantly more likely to receive an ongoing
subsidy (4 percentage points). These findings hold when controlling
for more demographic factors such as race and prior residence.
The above finding about the positive relationship between length of subsidy and stable housing
suggests that participants who experience more intensive services are more likely to be in stable
housing at follow-up. The finding that male participants are less likely to receive an ongoing
subsidy of two months or more may partly explain why they are less likely to be in stable housing.
However, there are some inconsistencies
in these trends. For instance, domestic
violence survivors are much more likely
to receive an ongoing subsidy but not
more likely to be in stable housing at
follow-up. This may suggest that
domestic violence survivors may be
receiving subsidies for 2 to 14 months or
that they may require additional services
rather than longer-term subsidies.
In summary, though causality cannot be
inferred from these results, certain
participant characteristics are
significantly related to receiving more
intensive support: multiple subsidies,
multiple placements, or longer subsidies. When considering this finding alongside housing-status
findings, a main theme emerged: service intensity is associated with housing status. Table 2
presents a combination of findings for service provision and stable housing.
Patterns in Receiving an Ongoing Subsidy
More likely: Domestic violence survivors
Less likely: Males
12
However, this is not the case for all participant characteristics or service provisions, suggesting
that the impact of more intensive services depends on the person and his/her situation. There
appear to be systematic relationships between greater intensity of service and stable housing,
although, the strength of the relationship depends on the person and his/her situation. Addressing
the data limitations encountered and finding opportunities to increase the quality of data can
permit a more robust analysis in the future and enable TPI to better target services.
Transition Projects’ Data-Collection and Use
In the above sections, this report sought to analyze TPI’s data to identify trends that may help the
organization improve its service delivery and programs, with the goal of improving TPI’s
effectiveness. However, numerous limitations were encountered that hindered the validity of the
findings and interpretations. To further understand the data limitations as well as TPI staff’s
current use and attitudes toward data, the following section presents findings from a staff survey.
Understanding data and data measurement is critical for making progress toward specified
outcomes. However, only the items that are measured, and measured well, ultimately receive
attention (Hatry, 2006). One key component of data-collection involves integrating a focus on
data into the organization’s culture. This ensures that data is collected, entered, and used
correctly and ultimately effectively. In doing so, it is also important to consider staff perceptions
and feelings about data and its use (Partnership for Public Service, 2012). The survey was
distributed to TPI staff to assess the existing culture regarding data-collection and use.
Approximately 60 staff members (25 percent of all TPI staff) responded to the survey. This
section summarizes some of the major themes. For more information regarding the survey
questions and results, see Appendix G.
Data use is relatively frequent among TPI employees. The majority of all employees spend up to
ten hours a week entering and using data for various tasks. About three out of four (74 percent)
employees surveyed perceive the quality of data they enter to be excellent, but the quality of data
used is rated equally often as neutral (46 percent) or excellent (46 percent). This suggests some
difference in judgment between the work individual employees do in entering data versus the
data that is ultimately part of the TPI data set.
All employees agreed that they want TPI to invest in good data quality and improve data usage
(82 percent). However, other opinions and ideas about data and the data system varied between
groups. While the majority of employees generally agreed that they were comfortable using the
data system, administrators were much more likely to be comfortable with it. Case managers and
shelter staff were more likely to have issues using the data system.
Summary of Survey Findings
Data use is frequent; however, perspectives on data quality vary.
Employees agree about investing in good data quality and improving data usage.
Many case managers and shelter staff find data entry takes too much time and seek additional resources to support their use of the data system.
Employees value data-collection, but also stress a desire to maintain client-driven focus.
13
Case managers described data entry as “tedious” and “frustrating,” and noted that “[the system]
changes frequently and it does not always produce an accurate snapshot of participants’
situations.” Half of all case managers surveyed agreed that data entry takes too much time. One-
quarter of shelter staff similarly felt that data entry is time consuming. One shelter employee
reported that “the data options available do not capture all the things [they] do in [their] role.”
Both case managers and shelter staff stated interest in improved data training as a strategy to help
lessen the time used in data entry and reduce duplicate and erroneous data points.
Administrators and shelter staff were more likely to feel that the most important parts of their
jobs require using data; case managers agreed with this statement only 33 percent of the time.
However, in the comments to this question, all employees shared similar perspectives on the
value of data. All survey participants echoed the idea that data is important to securing funding
and providing good services to TPI participants, but also identified compassion and interest in
the individuals as important. All survey participants indicated their hope that data could be used
to help improve TPI’s performance, increase their understanding of data’s role in providing
services to participants, and identify opportunities to better direct resources.
Case managers often have very different perspectives on the usefulness of data compared to
administrative staff and shelter staff. Case managers were more likely than the other groups to
agree that it takes too much time to enter data (53 percent) and that the most important purpose
of data is to satisfy external reporting requirements (67 percent). Many case managers, despite
demonstrating resistance to using data, describe finding value and wanting to use data to help
participants in open-ended questions.
While there is some disagreement among TPI employees about the value of being a data-driven
organization, most survey respondents found value in having data available. Many shelter staff
and case managers reported going beyond the required data-entry tasks to help participants find
the best possible programs and services. It is also encouraging to see an overwhelmingly positive
response toward investing in better data quality and use, signaling willingness across the
organization to make changes.
Discussion
TPI, in its mission to move people from homelessness to housing, requires understanding the
participants served as well as staff perceptions about data-collection and use. Analysis of TPI’s
placements and rental-subsidy programs identified relationships among participant
characteristics, service provisions, and outcomes. Though causality cannot be inferred from these
results, the level of assistance a participant receives is associated with his/her housing status.
Even controlling for other factors, there appear to be systematic relationships between assistance
and stable housing, but the strength of the relationship depends on the person and his/her
situation. However, inconsistencies within Transition Projects’ data challenge the validity of
these findings and prevented a more robust analysis of the organization’s services. Considering
the identified patterns can help guide thinking and organizational decision-making, but
improvement to TPI’s data-collection processes can ensure the accuracy of collected data and
help build TPI’s evidence base.
14
The survey findings of TPI’s data culture identified possible areas for improving data-collection.
While employees are concerned that increased data use may move the organization away from
being client-driven, they also would like TPI to invest in better data quality and use. These
survey findings informed the creation of actionable items in the report’s recommendations.
Alongside these recommendations are suggestions for how TPI can build upon the analysis
findings to position itself for better understanding its participants and service outcomes, and, as a
result, become an even more effective homeless service provider.
Recommendations
Incorporating a focus on data into an organization is a rather onerous process, from establishing
ways to collect useful data to integrating data into decision-making. A decision-making model
with structured steps is illustrated below. It simplifies an ideal decision-making process used by
an organization that efficiently collects and uses data. The model is circular because if the
process is repeated it leads to continuously improved services. Although the following
recommendations emphasize smaller, actionable steps for TPI, the model is provided purely for
reference.
Figure 5. Incorporating Data into Organization Decision-Making Model
Plan/Brainstorm
What do we want to know (outputs/outcomes/impact)? What data do we need to get
there?
Build
Build and standardize processes that allow
collection of useful and appropriate data.
Collect & Analyze Data
Run analyses, which may require hiring an analyst or
contracting outside the organization.
Evaluate
Compare results to programmatic goals. Are we performing how we want to
be?
Integrate
How can we share what we learned and be a thought leader in the
community?
15
Recommendation 1: Further integrate data into Transition Projects’ culture
To work toward the model above, TPI should continue integrating a focus on data into the
existing culture by promoting a shared understanding of the importance of data and by creating
learning forums.
All organizations have a culture, or a shared set of values that drive the mission, and TPI’s
values include being client-driven, compassionate, and equitable (Transition Projects, 2016).
Further integrating data as an organizational value can help TPI more effectively serve
participants and further its mission. From the survey, it is apparent that staff support data use.
TPI also has demonstrated capacity and support for data measurement, and its employees have
responded positively to investing in data quality.
These are among the key cultural components an organization needs to work toward integrating a
data focus (Hatry, 2006).
Promote an understanding of the importance of data
One way to sustain changes in data use is to reinforce the reason why quality data is important
and how it is used. Some staff expressed mixed feelings about data entry, fearing data use
conflicts with the organization’s client-driven approach. Additionally, staff may fear that data is
being used to evaluate their work. By being transparent about how data is used and educating
staff about the benefits of collecting data, TPI can help staff overcome some of these fears.
Additionally, the limitations of data should be made clear to all staff outcomes tell only what
happened, not why (Hatry, 2006). The literature includes a myriad of other strategies for
integrating data into the existing culture; however, these recommendations appear to be most
relevant to TPI, given survey findings.
Create learning forums
Another strategy to integrate data as an organizational value involves creating learning forums.
Learning forums provide opportunities to discuss data findings. The survey revealed that some
shelter staff integrate service data with participant meetings to guide decision-making about
service referrals. Additionally, learning forums may encourage employees to share best practices
for collecting data. Providing an opportunity for staff members and others to share how they are
using data may identify new techniques that could be applied across the organization. At the
Recommendations for Transition Projects
1. Further integrate data into Transition Projects’ culture.
2. Formalize data-collection processes.
3. Evaluate data in terms of programmatic intentions and integrate into decision-making.
4. Garner sector-wide support for building the evidence base for homelessness programs.
16
same time, this will increase staff buy-in to data-collection processes; staff will feel they are
valued and have a greater stake in how TPI uses data (Partnership for Public Service, 2012). By
encouraging TPI as an organization to learn from its employees, goals and processes can become
clearer to all stakeholders.
Recommendation 2: Formalize data-collection processes
This report examined TPI’s placements and rental-subsidy programs and identified relationships
among participant characteristics, service provisions, and outcomes. The La Follette School team
received more data from TPI than it used in the analysis. The following list highlights several
data limitations, grouped by whether these limitations were caused by inconsistencies in data
entry, data-collection, or data use. Addressing these limitations will increase TPI’s capacity to
produce a more robust analysis. The recommendation concludes by suggesting a specific
formalization process for collecting data on services.
Inconsistency in Data Entry
Service categories were tracked in three separate areas. Focusing on
one service was difficult because there was no clear pattern by which to
categorize them. Additionally, two specific areas for entering service
descriptions included over 50 unique service options, and combinations
of the two services entered per participant were inconsistent.
Subsidies were ultimately used by the La Follette Team for this report,
but there were some difficulties with this data as well. Subsidy start and
end dates were recorded inconsistently across TPI. Some were recorded
with an end date at the end of a month, while others had identical start
and end dates.
Inconsistency in Data-Collection Disability data was ultimately not able to be used for the analysis. Although TPI collects
data on disability diagnosis, it was lacking for most of the participant data. Additionally,
disabilities were recorded as whether a participant had a disabling condition, but lacked
information on the severity or current status of the disability. Lastly, due to changes in
disability definition and collection around 2014, the information was not able to be
incorporated into the report.
Inconsistency in Data Use
The purpose of data was unclear for many case managers and shelter staff surveyed.
This lack of clarity can lead to fear or resistance to using data in the future.
Decision-making was a common ideal goal for data use, but the suggested decisions
varied widely. Some case managers feared decisions based on data may be punitive,
performance-based decisions that would impact their job status. Shelter staff were
interested in seeing data used to make programmatic decisions, like planning new
shelters. Others wanted to know more about how the data help TPI as an organization
make decisions. Clarity in decision-making can make data goals more concrete and help
orient data-collection toward meeting agency and program goals.
Inconsistent Data
Race
Ethnicity
Disability
Services
Subsidy
End Date
17
As described in the analytical priorities for data-collection
to the left, additional data-collection could contribute to a
more robust analysis of participant characteristics, services,
and their impact on outcomes. For example, participant
income would add more depth to the analysis of subsidy
intensity, while causality could be potentially inferred if
housing status was recorded for all TPI participants.
By building capacity to track some analytic priority areas
and addressing some of the preceding inconsistencies, TPI
can strengthen its ability to perform more robust analyses
in the future. The following sub-sections and Appendix H provide more specific suggestions and
techniques for improving data-collection processes.
Example of Improving Data-Collection with Services
One approach to creating a more consistent data entry process might include creating clear
consensus around service definitions and the services most important for tracking across the
organization. Including all levels of employees in this discussion can provide several benefits.
For example, administrators, case managers, and shelter staff have unique and diverse
perspectives about different services and integrating these unique perspectives can help TPI gain
a more in-depth understanding of the most relevant service categories (Arygris and Schon,
1996).
Including all levels of staff also helps improve employee buy-in about data-collection, which in
turn improves the chances for better data quality (Partnership for Public Service, 2012). After
agreeing on service categories, disseminating the definitions across the agency and providing
TPI staff training can help maintain data quality. The checklist in Appendix H provides concrete
steps for TPI to use for other data-collection improvements.
Recommendation 3: Evaluate data in terms of programmatic intentions and integrate into decision-making TPI should consider its programmatic goals when evaluating the data findings in this report and
future analyses and should incorporate this evaluation into programmatic decisions. This report’s
analysis focused on three questions: 1. What participant characteristics and/or services are significantly associated with being in
stable housing at follow-up?
2. What participant characteristics are associated with service provisions, namely receiving
more intensive support?
3. Do the identified patterns provide possible explanations for participant characteristics
associated with being in stable housing?
These questions can serve as guidance for TPI when it begins to explore how to link its data with
its programmatic intentions. Integrating collected data, programmatic intentions, and decision-
Analytical Priorities for Data-Collection
Participant Income
Subsidy Value
Placement Intervention Type
Grant Allocation by Subgroup
Housing Status for Each Participant
18
making processes can help TPI leverage required data-collection to further its mission. This
recommendation concludes with a specific process for evaluating service data.
Process of Evaluating Findings
Staff can use this analysis to better understand who is currently served, who is likely to use
services and programs, and who is likely to experience positive outcomes, such as stable
housing. Specifically, TPI can consider why certain patterns are significant and how they align
with programmatic goals and expectations; doing so can help TPI be more intentional with its
programming and services.
The unique relationship between multiple placements and multiple subsidies is an example of a
finding worth investigating. Participants who come back for multiple placements are very likely
to receive multiple subsidies, and vice versa. This may make sense because rental assistance
often accompanies TPI’s placement services; but, this might spur case managers to ask why
participants are returning for additional placements? Further, in addition to receiving a subsidy,
participants who received more than one placement were more likely to be in stable housing. Is
receiving more than one placement supposed to be related to stable housing? Is there a
programmatic reason behind this finding? Or are participant characteristics influencing this
finding? Assessing program data in this manner is helpful for identifying program and service gaps. After
TPI understands more about the reach of its services, it can develop a strategy to better target
them.
Continuous Improvement
Upon developing a strategy, TPI can continue to evaluate how its programs and services are
performing. Routinizing this process allows for continuous process improvement, as illustrated
in the decision-making model above. One way TPI can institutionalize this process is by
incorporating findings into a strategic plan and then measuring program data against the strategic
plan’s goals. Another way would be to hold routine meetings to review and discuss program data
findings.
Recommendation 4: Garner sector-wide support for building the evidence base for homelessness programs
TPI’s knowledge of the network of service providers and city collaborations working to end
homelessness can be essential to continue developing the sector’s evidence base. For instance,
Portland’s city-wide initiative “A Home for Everyone” values using data to guide decision-
making. This creates a venue for TPI to share findings about both the opportunities for using data
and the fact that current organizational data may be restricted in its ability to identify causal
effects. More specifically, this data analysis was restricted by the inability to identify a
comparison group for investigating causal effects of subsidy length on housing status.
Investigating this question and others like it requires more rigorous, controlled evaluations.
19
Thought Leadership
TPI can use its position as a member, and often a leader, in collaborations to highlight the
importance of controlled studies for more precisely evaluating and identifying the most critical
and effective assistance programs. Current data can be used as evidence to encourage funder
support for additional, more rigorous analysis, such as experimental, or even randomized
designs, that may have the most potential to develop an evidence base regarding the impact of
programs.
Conclusion
Analysis of TPI’s placements and rental subsidy programs described relationships among
participant characteristics, service provisions, and housing stability outcomes. Some participant
sub-groups were found to receive more (or less) intensive services when compared to other
participants. However, due to data limitations, findings related to housing status were
inconclusive, counter-intuitive, or potentially false – limiting their validity. Improving data-
collection and further integrating data into the organization’s culture present opportunities to
improve TPI’s ability to perform more robust analyses in the future and continue to build the
evidence base. The report provided recommendations for how TPI can build upon the findings
and better evaluate its assistance programs, monitor successes, and learn from its data to become
a more effective homeless service provider.
Data can be a useful tool to increase knowledge about TPI and its participants, and how TPI can
better serve its community. However, more data for the sake of having data can be as hindering
as collecting no data or poorly collecting data. Being cognizant of the organization’s needs and
the variables that are necessary to further TPI’s mission and goals can help deter data overload
and meaningless data-collection. Also, finding avenues within current HMIS requirements to
collect data of interest can mitigate overloading staff.
TPI is a leader within the Portland community. The area can greatly benefit from TPI’s lead in
improving data quality and reliability, in turn increasing the ability to analyze data across
programs and organizations. TPI has a strong organizational culture. If it can leverage some of
the findings and recommended strategies, TPI can greatly increase its capacity for outcome
evaluation and move Portland closer to finding a home for everyone.
20
Appendix A: Notes on Data Cleaning and Recoding
Data Cleaning
To create the data set used for analysis, demographic data were first cleaned as described in the
following sections. After demographic data were finalized, they were merged onto the
placements and subsidy follow-up data sets. Only participants with demographic data are
considered in our analysis of TPI placements and rental subsidies.
Race
The original HMIS data records race and ethnicity across three variables. Two variables record
race, and a third records if a participant is Hispanic. When participants come to TPI, they are
allowed an open-ended response to the race category, which allows for participants to describe
their race as they like. As a result, these variables are not exclusive, because participants can be
double-counted in various categories. To simplify the categories, we recoded the 8-category race
variable into the following:
White, Non-Hispanic White
AND
Non-Hispanic
Non-White,
Non-Hispanic
Black or African American
Asian
American Indian or Alaska Native
Native Hawaiian or Other Pacific Islander
Other
AND
Non-Hispanic
Hispanic Hispanic
Disability
The HMIS data file on disability was originally in a long format, with multiple entries for each
participant. Disability type was captured in one categorical variable. From this variable, we
created indicators for each disability type. To convert the data to a wide format (one entry per
participant), we collapsed the data set to collect a count of each disability type for each
participant. As a result, our disability data is slightly different from how TPI uses it. Rather than
showing the disabilities a participant has over each visit, we have their cumulative disabilities up
to their most recent visit to TPI, so our counts for disability type may differ from TPI counts.
Disability data were not available for all TPI participants.
Prior Residence
We recoded the prior residence variable into a three-category variable, using HMIS definitions.
The following table shows the kinds of responses that fall into each of these three larger
categories.
21
Literally Homeless Place not meant for habitation (e.g., a vehicle, an abandoned
building, bus/train/subway station/airport, or anywhere
outside)
Emergency shelter, including hotel or motel paid for with
emergency shelter voucher
Save Haven
Interim housing
Institutional Situation Foster care home or foster care group home
Hospital or other residential non-psychiatric medical facility
Jail, prison, or juvenile detention facility
Long-term care facility or nursing home
Psychiatric hospital or other psychiatric facility
Substance abuse treatment facility or detox center
Transitional and
Permanent Housing
Situation
Hotel or motel paid for without emergency shelter voucher
Owned by client, with or without ongoing housing subsidy
Permanent housing for formerly homeless (such as: a CoC
project, HUD legacy programs, or Housing Opportunities for
Persons With Aids (HOPWA))
Rental by client, with or without subsidy
Residential project or halfway house with no homeless
criteria
Staying or living in a friend’s or family’s room, apartment, or
house
Transitional housing for homeless persons
Placements
For the purposes of this report, a participant is considered to have been placed in housing only
when he or she appears in TPI’s placement data. Conversely, any TPI participant who did not
appear in the placement data is not considered in any analysis using placement or housing status.
Within the data provided, some participants were recorded as having been placed in housing
multiple times in a single day. Multiple same-day placements were considered duplicative entries
and were dropped for the purposes of the analysis. Additionally, within the provided data, the
number of placement services recorded for 2016 diverged significantly from TPI’s internal
reporting on placements.
Multiple Placements
Multiple placements refers to whether a participant was placed in housing more than once within
the 10 years of TPI data analyzed. A participant who placed in housing more than once at any
point between 2007 and 2016 was recorded as having been placed multiple times and marked
with a 1. Participants who were placed only once in these 10 years were marked with a zero for
this variable. As stated above, some participants were recorded as having been placed multiple
22
times on a single day. These were considered to be data-entry errors, and one of the entries was
dropped for analysis purposes.
Subsidy Length
Subsidies in this report are only rental subsidies provided in follow-up data. Any TPI participant
who did not appear in the follow-up data is not considered in any analysis using subsidy or
housing-status data. Using the information provided on subsidy start and end dates, subsidy
lengths were counted as the number of days between start and end dates. Because some one-
month subsidies are recorded as ending either on the same day as the start date or at the end of
the month, any subsidy length of 30 days or less was considered a one-month subsidy.
Otherwise, subsidy-length days were converted to subsidy-length months. Based on observed
patterns in the number of participants with subsidies of varying lengths, subsidy lengths were
grouped into 1 month, 2- to 6-month, 6- to 14-month, and more than 14 months.
Multiple Subsidy
Multiple subsidy refers to the number of unique subsidy start dates for each TPI participant. Each
of these unique subsidy start dates was counted for each participant, resulting in a count of
subsidies. From here, an indicator was created to signal if a participant received one subsidy or
more than one subsidy. A 1 indicates that a participant had more than 1 subsidy, while a 0
indicates the participant had only 1 subsidy.
Housing Status
For the purpose of this report, only the housing status at the most recent participant follow-up
was considered. For some participants, housing status changes between follow-up dates, so our
analysis lacks this particular nuance. To determine stable housing, an indicator variable was
created, with a value of 1 for participants in stable housing and 0 if participants were not in
stable housing.
23
Appendix B: TPI Demographics
Overall, TPI serves a majority white population. However, non-white populations are
overrepresented in homeless populations across Oregon and within TPI populations. Most TPI
participants overall and within the timeframe of our analysis were men. Interestingly, in January
2015, TPI served a more equal population of men and women, which differs from the gender
breakdown of homeless individuals in the January Point in Time (PIT) estimate (70 percent
male, 30 percent female).
Roughly 40 percent of the TPI population has a disability, and many individuals have more than
one disability. The most common disabilities among TPI participants were addiction (including
drug, alcohol, and both drug and alcohol addictions), chronic health conditions, developmental
conditions, HIV/AIDS, and mental health problems.
The majority of TPI participants came to TPI after a period of being literally homeless, which
includes sleeping on the street or living in a place unfit for habitation. The next most common
prior residence was transitional and permanent housing solutions.
Domestic violence survivors and veterans were not uncommon in TPI. Nearly one-quarter of all
TPI participants were domestic violence survivors, and about one in five TPI participants were
veterans.
Using January 2015 data, TPI participants can be compared to the broader homelessness
demographics of its CoC. In this month, TPI served 195 participants. Like the CoC, TPI serves a
disproportionately high population of non-white individuals compared to the racial makeup of
Multnomah County and the city of Portland. Women made up 51 percent of TPI participants for
January 2015, a higher proportion than the proportion of homeless women on the PIT estimate.
Veterans made up 22 percent of the TPI population in January 2015.
A table summarizing the demographics of TPI overall, TPI in the last decade, and TPI in January
2015 follows.
24
Table 3. Demographics of All TPI Participants, TPI Participants from 2007-2016, and TPI Participants in January 2015 TPI Overall (n=18,346)
TPI, 2007-2016 (n=16,949)
TPI January 2015 (n=195)
Race White 62 58 55 Non-White 24 23 18 Hispanic 8 8 7 Gender Male 62 61 49 Female 38 39 51 Disabled (total) 42 40 26 Addiction 23 21 10 Chronic Health Condition 16 15 9 Developmental 15 13 5 HIV/AIDS 15 13 6 Mental Health Problem 28 27 18 Other 8 9 1 Prior Residence Literally Homeless 57 56 49 Institutional Situation 12 11 21 Transitional and Permanent Housing Solution
31 312 29
Domestic Violence Situation <1 <1 0 Other 1 1 2 Domestic Violence Survivor 28 27 26 Veteran 23 23 22
Source: TPI HMIS Data, 2007-2016 (n=18,346)
25
Appendix C: Placements and Subsidies Analysis Tables
The following tables (4 and 5) support the description of placements and rental-subsidy data as
described in the Placements and Rental Assistance section. Between 2007 and 2016, TPI
provided nearly 8,000 placements to 6,300 individual participants and more than 7,400 rental
assistance subsidies to 5,500 participants. Participants often receive both placements and
subsidies over time.
Table 4. TPI Placements per Year, with Demographic Data
Year Unique
Participants Total
Placements Male Male (%) Nonwhite
Nonwhite (%) Hispanic
Hispanic (%) Veterans
Veterans (%)
DV Survivor
DV Survivor
(%)
2007 719 729 395 55 195 27 40 6 152 21 150 21 2008 675 714 371 55 206 31 43 6 110 16 169 25 2009 687 740 441 64 203 30 35 5 164 24 186 27 2010 662 702 453 68 214 32 40 6 194 29 175 26 2011 611 650 384 63 198 32 40 7 120 20 178 29 2012 824 861 553 67 260 32 64 8 231 28 229 28 2013 663 701 407 61 202 30 36 5 195 29 206 31 2014 762 806 502 66 213 28 64 8 231 30 198 26 2015 820 870 556 68 210 26 46 6 335 41 197 24 2016 1160 1203 612 53 270 23 61 5 396 34 214 18
All Years 7583 7976 4674 62 2171 29 469 6 2128 28 1902 25
Source: TPI HMIS Data, 2007-2016 (n=6,309)
Table 5. TPI Subsidies per Year, with Demographic Data
Year Unique
Participants Total
Subsidies Male Male (%) Nonwhite
Nonwhite (%) Hispanic
Hispanic (%) Veteran
Veteran (%)
DV Survivor
DV Survivor
(%)
2007 478 690 280 59 128 27 29 6 99 21 106 22 2008 508 728 278 55 143 28 34 7 70 14 121 24 2009 470 703 295 63 123 26 26 6 102 22 128 27 2010 440 631 286 65 140 32 23 5 101 23 109 25 2011 441 630 264 60 138 31 31 7 67 15 129 29 2012 659 869 424 64 191 29 51 8 183 28 184 28 2013 494 665 315 64 138 28 29 6 167 34 142 29 2014 672 927 461 69 164 24 65 10 261 39 165 25 2015 715 892 504 70 176 25 38 5 321 45 161 23 2016 649 682 427 66 191 29 40 6 241 37 159 24
All Years 5526 7417 3534 64 1532 28 366 7 1612 29 1404 25
Source: TPI HMIS Data, 2007-2016 (n=5,526)
26
The following regression tables (6 and 7) show how participant characteristics predict the
likelihood of being placed or receiving a subsidy, when compared to all TPI participants.
Approximately 12,000 participants had complete data records of gender, race, ethnicity, and if
they are a veteran or domestic violence survivor.
Table 6. Receive Placement Services Regression Coefficients
In Placement In Placement In Placement In Placement
In Subsidy 0.885*** 0.909*** 0.912*** 0.905*** (253.79) (268.17) (262.10) (231.08) Male 0.013*** 0.016*** -0.014** (4.01) (4.83) (-2.89) Nonwhite 0.005 0.004 0.006 (1.36) (1.15) (1.43) Hispanic -0.004 -0.004 0.012
(-0.58) (-0.59) (1.28) Veteran 0.072*** (11.13)
Domestic Violence Survivor -0.010 (-1.91)
Constant 0.076*** 0.043*** -0.007* 0.002 (32.87) (15.13) (-2.24) (0.28) Year Fixed Effect No No Yes Yes
Observations 18608 16493 16493 11936 * p < 0.05, ** p < 0.01, *** p < 0.001
mean coefficients; t-score in parentheses
Source: TPI HMIS Data, 2007-2016 (n=18,608)
Table 7. Receive a Subsidy Regression Coefficients
In Subsidy In Subsidy In Subsidy In Subsidy
In Placement 0.824*** 0.878*** 0.866*** 0.882*** (173.76) (206.16) (189.76) (194.13) Male -0.005 -0.010** 0.006 (-1.51) (-3.04) (1.41) Nonwhite 0.004 0.004 0.002 (1.16) (1.16) (0.38) Hispanic -0.004 -0.001 -0.009 (-0.55) (-0.17) (-1.03) Veteran -0.036*** (-5.82) Domestic Violence Survivor 0.015** (2.96) Constant 0.017*** 0.020*** 0.010** 0.006 (14.79) (8.17) (3.02) (1.15) Year Fixed Effect No No Yes Yes
Observations 18608 16493 16493 11936 * p < 0.05, ** p < 0.01, *** p < 0.001
mean coefficients; t-score in parentheses
Source: TPI HMIS Data, 2007-2016 (n=18,608)
27
Appendix D: Housing Status Analysis Tables
The following tables (8 and 9) support the analysis described in the Housing Status section.
Approximately 4,800 participants had complete data records including gender, race, ethnicity,
prior residence, and if they are a veteran or domestic violence survivor.
Table 8. Housing Status Regression Coefficients Conditional on Receiving a Subsidy
Housing Status at Last Follow-Up
Housing Status at Last Follow-Up
Housing Status at Last Follow-Up
Housing Status at Last Follow-Up
2- to 6-Month Subsidy -0.046** -0.048** -0.047** -0.055*** (-3.09) (-3.25) (-3.17) (-3.51)
7- to 13-Month Subsidy 0.006 0.002 0.001 0.003 (0.39) (0.14) (0.07) (0.19) 14+ Month Subsidy 0.075*** 0.068** 0.064** 0.066** (3.49) (3.17) (2.91) (2.92) Given Placement -0.048 -0.048 -0.064
(-1.19) (-1.15) (-1.13) Placed more than 1 time 0.037*** 0.033** 0.034** (3.50) (3.07) (3.07) Male -0.061*** -0.085*** (-5.62) (-5.92) Nonwhite -0.030** -0.024* (-2.58) (-1.98) Hispanic -0.006 0.006 (-0.21) (0.22) Veteran 0.029* (2.16) Domestic Violence Survivor -0.015 (-1.02) Constant 0.610*** 0.647*** 0.698*** 0.713*** (38.30) (15.08) (15.45) (11.90) Year Fixed Effect Yes Yes Yes Yes
Observations 5513 5513 5295 4837 * p < 0.05, ** p < 0.01, *** p < 0.001
mean coefficients; t-score in parentheses
Source: TPI HMIS Data, 2007-2016 (n=5,513)
28
Table 9. Housing Status Descriptive Statistics
Not in Stable Housing
In Stable Housing
Total
Male 0.723 0.634 0.651 (0.448) (0.482) (0.477) Nonwhite 0.318 0.287 0.293 (0.466) (0.453) (0.455) Hispanic 0.0413 0.0431 0.0428 (0.199) (0.203) (0.202) Veteran 0.270 0.289 0.285 (0.444) (0.453) (0.452) Domestic Violence Survivor 0.243 0.297 0.287 (0.468) (0.479) (0.477) 2- to 6-Month Subsidy 0.173 0.157 0.160 (0.378) (0.364) (0.367) 7- to 13-Month Subsidy 0.112 0.126 0.123 (0.316) (0.332) (0.329) 14+ Month Subsidy 0.0381 0.0717 0.0653 (0.192) (0.258) (0.247) Given Placement 0.995 0.990 0.990 (0.0736) (0.102) (0.0971) Placed more than 1 time 0.396 0.465 0.452 (0.489) (0.499) (0.498)
Observations 1035 4478 5513 * p < 0.05, ** p < 0.01, *** p < 0.001
mean coefficients; t-score in parentheses
Source: TPI HMIS Data, 2007-2016 (n=1,279)
29
Appendix E: Multiple Subsidy and Multiple Placements Analysis Tables
The following tables (10-13) support the analysis described in the Service Provision section.
There are some characteristics more correlated with being placed or receiving a subsidy more
than one time. Approximately 4,600 participants had complete data records including gender,
race, ethnicity, prior residence, and if they are a veteran or domestic violence survivor.
Table 10. Multiple Subsidy Regression Coefficients
Given more than 1 Subsidy
Given more than 1 Subsidy
Given more than 1 Subsidy
Given more than 1 Subsidy
Given more than 1 Subsidy
Placed > 1 time 0.556*** 0.578*** 0.738*** 0.729*** 0.728*** (43.80) (43.80) (62.12) (58.81) (58.20) Male -0.003 -0.018 -0.012 (-0.34) (-1.84) (-1.16) Nonwhite 0.027*** 0.030*** 0.027** (3.43) (3.44) (3.12) Hispanic -0.017 -0.013 -0.015 (-0.88) (-0.67) (-0.75) Veteran 0.025** 0.028** (2.61) (2.88) Domestic Violence Survivor
-0.006 -0.002
(-0.55) (-0.17) Literally Homeless -0.035 (-0.87) Institutional Situation -0.016 (-0.40) Transitional and Permanent Housing Solution
0.003
(0.08) Constant 0.218*** 0.141*** 0.020* 0.032*** 0.046 (37.06) (9.46) (2.25) (3.46) (1.10) Year Fixed Effect No Yes Yes No Yes
Observations 6309 6177 5073 4589 4573 * p < 0.05, ** p < 0.01, *** p < 0.001
mean coefficients; t-score in parentheses
Source: TPI HMIS Data, 2007-2016 (n=6,309)
30
Table 11. Multiple Subsidies Descriptive Statistics
One Subsidy More than One Subsidy
Total
Placed > 1 time 0.119 0.848 0.358 (0.324) (0.359) (0.480) Male 0.662 0.637 0.654 (0.473) (0.481) (0.476) Nonwhite 0.242 0.309 0.264 (0.429) (0.463) (0.441) Hispanic 0.0429 0.0368 0.0409 (0.203) (0.188) (0.198) Veteran 0.417 0.338 0.391 (0.493) (0.474) (0.488) Domestic Violence Survivor 0.290 0.328 0.302 (0.454) (0.470) (0.459) Literally Homeless 0.616 0.480 0.572 (0.487) (0.500) (0.495) Institutional Situation 0.125 0.113 0.121 (0.331) (0.317) (0.326)
Transitional and Permanent Housing Solution
0.255 0.402 0.303
(0.436) (0.491) (0.460)
Observations 4337 2188 6525 * p < 0.05, ** p < 0.01, *** p < 0.001
mean coefficients; t-score in parentheses
Source: TPI HMIS Data, 2007-2016 (n=1,247)
31
Table 12. Multiple Placements Regression Coefficients
Placed more than 1 time
Placed more than 1 time
Placed more than 1 time
Placed more than 1 time
Given more than 1 Subsidy
0.425*** 0.483*** 0.806*** 0.795***
(36.78) (40.11) (77.25) (72.91) Male -0.006 0.019 (-0.74) (1.80) Nonwhite -0.023** -0.025** (-2.84) (-2.84) Hispanic 0.023 0.020 (1.13) (0.94) Veteran -0.044*** (-4.25) Domestic Violence Survivor
0.020
(1.78) Constant 0.075*** -0.056*** -0.004 -0.015 (18.43) (-7.15) (-0.54) (-1.43) Year Fixed Effect No Yes Yes Yes
Observations 6309 6177 5073 4589 * p < 0.05, ** p < 0.01, *** p < 0.001
mean coefficients; t-score in parentheses
Source: TPI HMIS Data, 2007-2016 (n=6,309)
Table 13. Multiple Placements Descriptive Statistics
One Placement More than One Placement
Total
Given more than 1 Subsidy 0.0774 0.776 0.327 (0.267) (0.418) (0.469) Male 0.660 0.641 0.654 (0.474) (0.480) (0.476) Nonwhite 0.261 0.269 0.264 (0.439) (0.444) (0.441) Hispanic 0.0375 0.0471 0.0409 (0.190) (0.212) (0.198) Veteran 0.438 0.307 0.391 (0.496) (0.462) (0.488) Domestic Violence Survivor 0.283 0.336 0.302 (0.451) (0.473) (0.459) Literally Homeless 0.609 0.504 0.572 (0.488) (0.501) (0.495) Institutional Situation 0.131 0.103 0.121 (0.338) (0.304) (0.326)
Transitional and Permanent Housing Solution
0.255 0.390 0.303
(0.436) (0.488) (0.460)
Observations 4921 1388 6309 * p < 0.05, ** p < 0.01, *** p < 0.001
mean coefficients; t-score in parentheses
Source: TPI HMIS Data, 2007-2016 (n=1,247)
32
Appendix F: Ongoing Subsidy Analysis Tables
The following tables (14 and 15) support the analysis of ongoing subsidies as described in the
Service Provision section. Of the approximately 7,400 subsidies provided between 2007 and 2016,
88 percent (6,500 subsidies) included an end date, permitting calculation of the length of the subsidy
a participant received. Approximately 5,700 participants had complete data records including
gender, race, ethnicity, prior residence, and if they are a veteran or domestic violence survivor.
Table 14. Ongoing Subsidy Regression Coefficients
Subsidy Received for more than 1 month
Subsidy Received for more than 1 month
Subsidy Received for more than 1 month
Male -0.026* -0.023 -0.034* (-2.02) (-1.44) (-2.06) Nonwhite -0.041** -0.026 -0.024 (-3.10) (-1.90) (-1.74) Hispanic 0.013 0.002 0.003 (0.44) (0.06) (0.11) Veteran 0.002 0.008 (0.15) (0.54) Domestic Violence Survivor 0.038* 0.036* (2.25) (2.14) Literally Homeless 0.052 (0.60) Institutional Situation -0.063 (-0.70) Transitional and Permanent Housing Solution
0.003
(0.03) Constant 0.389*** 0.262*** 0.241** (34.51) (10.37) (2.67) Year Fixed Effect No Yes Yes
Observations 6288 5730 5711 * p < 0.05, ** p < 0.01, *** p < 0.001
mean coefficients; t-score in parentheses
Source: TPI HMIS Data, 2007-2016 (n=6,288)
Table 15. Ongoing Subsidy Descriptive Statistics
1 Month Subsidy Subsidy for 1+ Month Total
Literally Homeless 0.526 0.521 0.523 (0.500) (0.500) (0.500) Institutional Situation 0.126 0.0975 0.111 (0.332) (0.297) (0.315) Transitional and Permanent Housing Solution
0.343 0.380 0.362
(0.475) (0.486) (0.481) Male 0.661 0.691 0.676 (0.474) (0.462) (0.468) Nonwhite 0.312 0.270 0.291 (0.464) (0.444) (0.454) Hispanic 0.0381 0.0404 0.0392 (0.192) (0.197) (0.194)
Observations 4210 2278 6488 * p < 0.05, ** p < 0.01, *** p < 0.001
mean coefficients; t-score in parentheses
Source: TPI HMIS Data, 2007-2016 (n=1,427)
33
Appendix G: Survey Questions and Select Results
Response rate: 68 out of 274, 25 percent
Table 16. Q1: On average, how many hours a week do you typically spend entering data? (%)
0 Hours 0 and 5 hours 5 and 10 hours More than 10
Administration 13 40 27 20
Case Manager 0 40 53 7
Shelter Staff 5 45 37 13
Total 5 45 37 13
Source: La Follette School Team Survey of TPI Staff, 2017
Table 17. Q2: This week, how many hours did you spend on data entry? (%)
0 Hours 0 and 5 hours 5 and 10 hours More than 10
Administration 27 47 0 28
Case Manager 0 50 36 14
Shelter Staff 3 49 34 14
Total 8 48 27 17
Source: La Follette School Team Survey of TPI Staff, 2017
Table 18. Q3: How would you rate the quality (considering accuracy, completeness, usefulness) of the data you enter? (%)
Poor Neutral Excellent
Administration 0 7 93
Case Manager 7 20 73
Shelter Staff 5 30 66
Total 4 22 74
Source: La Follette School Team Survey of TPI Staff, 2017
34
Table 19. Q4: On average, how many hours a week do you spend using data (including analyzing, cleaning, and using data to make decisions)? (%)
0 Hours 0 and 5 hours 5 and 10 hours More than 10
Administration 13 17 13 27
Case Manager 7 86 7 0
Shelter Staff 19 50 17 14
Total 15 57 14 14
Source: La Follette School Team Survey of TPI Staff, 2017
Table 20. Q5: This week, how many hours did you spend using data (including analyzing, cleaning, and using data to make decisions)? (%)
0 Hours 0 and 5 hours 5 and 10 hours More than 10
Administration 27 40 0 33
Case Manager 7 79 14 0
Shelter Staff 25 44 22 8
Total 22 51 15 12
Source: La Follette School Team Survey of TPI Staff, 2017
Table 21. Q6: How would you rate the quality (considering accuracy, completeness, usefulness) of the data you use? (%)
Poor Neutral Excellent
Administration 0 53 47
Case Manager 7 40 53
Shelter Staff 13 45 42
Total 9 46 46
Source: La Follette School Team Survey of TPI Staff, 2017
35
Table 22. Q7: What is your position at Transition Projects? (n)
Title n
Administration 15
Case Manager 15
Shelter Staff 38
Total 68
Source: La Follette School Team Survey of TPI Staff, 2017
Q8: Use the space below to provide any additional comments about data use or the data culture of TPI. Sample responses TPI needs to be really honest with itself regarding critical data entry problems at shelters. Low
performers must be coached or moved, and the managers do not have the time to coach. Something that will help us develop a data culture is training on data systems.
Table 23. Data culture questions, % agree or strongly agree
Administration Case Managers
Shelter Staff
Q9: I think data should have a large role in how decisions are made.
67 40 58
Q12: I am comfortable using the data system at TPI. 93 73 74
Q13: The most important part of my job requires using data.
73 33 68
Q15: I want TPI to invest resources to ensure good data quality and improve data use.
87 93 76
Q16: It takes too much time to enter data. 33 53 26
Q18: The most important purpose of data collection is to satisfy external reporting requirements.
53 67 45
Q19: It is clear how data are used by TPI to make decisions.
40 40 61
Source: La Follette Team Survey of TPI Staff, 2017
36
Q10: What decisions do you think should be made using data?
Most. What services we fund more, where we conduct outreach, how we modify our service
practices. Everything we do is essentially informed by the information regarding our clientele.
The expansion of shelters planning to be opened. Where funding is allocated. What other
services clients most need referrals to, including medical care, housing, food assistance, etc. and
therefore follow-up education for social workers.
Q11: Please use the space below to expand or clarify any of your answers from this section (Q9
and Q10—using data for decision-making).
Again, data is a good indicator on how well the organization is doing as a whole.
Demographic data can inform TPI about participants and their needs.
Q14: Please use the space below to expand or clarify any of your answers from this section (Q12
and Q13—using data at TPI).
The most important part of my job requires being compassionate and patient when assisting
clients with maintaining safe housing, and data entry is a key part of funding that safe housing.
I find the interaction with participants to be the most important part of my job. But I realize that
the data is essential in our ability to continue doing the work that we do in preparing grant
documentation and showing progress. Data shows us how effective our interactions are.
Q17: Please use the space below to expand or clarify any of your answers from this section (Q15
and Q16—attitudes toward data).
TPI is working on implementing better data training for staff, but we have a long way to go. Staff
are often entering more data than necessary by duplicating their responses which results in
people spending much more time on data entry. This can be resolved with better data quality
training.
Entering data is tedious, regardless of the length of time it takes.
We need to have someone to enter all data so that it isn’t the responsibility of case managers so
they can more effectively utilize their time.
37
Q20: Please use the space below to expand or clarify any of your answers from this section (Q18
and Q19—importance of data).
Although data is very important to collect for external funders, it is also important to collect for
our own information so that the agency can determine what is working well, what isn't and
where participants need the most support.
I think TPI can have too much faith in the data at times. There is plenty of user error and I don’t
believe the reports are that accurate a lot of the time. There is still a lot of variance around
definitions of what we are reporting i.e. does going into transitional housing count as a housing
placement? Data and reporting can be unclear or skewed at times when multiple people are
pulling the same data and coming up with different numbers. Managers and directors pulling
their own numbers can skew the clarity of our data.
Q20: What additional data would be helpful for you to use that is not already collected by TPI?
I think we need more accurate data collection, not necessarily more data.
I think we have too much data and that makes it hard [to] clean data.
Q21: Should TPI be a data-driven organization? What steps need to be taken to reach that goal?
I feel that TPI is already data-driven in many ways. If there were clear-cut guidelines on what
data to put in place when there is not a direct response, our data could be more clear and
concise. This giving us the opportunity to see clearly indicate the next step in the data-driven
process.
I disagree, I think data can only capture a small part of a human-based phenomenon. I think
hard data should be a part of what helps make decisions but we should not rely solely on hard
data. I believe that anecdotes, experiences, opinions etc. are all things that are very relevant and
important to include when working with people in a supportive and service-based organization.
38
Appendix H: Suggestions for Integrating Data into Decision-Making
Integrating a focus on data into the existing processes and organizational culture is a lengthy and
often difficult process. The following are specific, actionable items to help organizations collect,
evaluate, and use data.
PLAN: Identify a leader for the improvement process. Leaders should be able to:
Spend a few hours preparing for meetings, including reviewing data with staff.
Attend and regularly participate in scheduled meetings about data improvement.
Follow up and follow through with decisions made on data improvement.
Be comfortable working with data and employees across TPI, and be invested in improving
data-collection processes.
INTEGRATE: Working with TPI staff and administrative staff, the leader should:
Clarify decision-making goals and the role of data in making decisions by TPI administrators.
Ensure all staff understand the role of data in decision-making.
Examine the findings from the data. Are services meeting the goals of TPI? Are participants
getting enough help?
Use these findings and re-examine these services or characteristics of interest.
EVALUATE: With the leader’s assistance, identify program goals of interest
Start with a program, or even one service or participant characteristic that everyone uses or
finds valuable to facilitate buy-in from all employees.
Discuss the goals for this service or characteristic—what do you most want to know?
As a team, discuss how data can be collected and evaluated to learn what you want to know.
Work together to adjust the data-collection process to ensure data accuracy and quality for
answering your questions.
COLLECT & ANALYZE DATA: Start using the new processes for data-collection, and:
Collect feedback from staff on the new data process—what works? What doesn’t?
Examine patterns in the data to help identify categories for service or participant
characteristics of interest.
Develop a routine for cleaning and assessing data quality.
BUILD: Invite TPI staff to the conversation. Meet with all employees to:
Gain perspectives of data-collection from all sides.
Clarify goals for improving data-collection.
Find agreed-upon meaning and use for data collected. For example, is a “no” response a
missing data point, or is it equal to 0?
Standardize the data-collection process together, and train all staff to use the new process.
Additional strategies can be found in Harry Hatry’s 2011 report “A Guide to Data-Driven
Performance Reviews,” publicly available via the Urban Institute.
39
Works Cited
Argyris, Chris, and Donald Schon. 1996. "What is an Organization that it may Learn?" In
Organizational Learning II: Theory, Method, and Practice, by Chris Argyris and Donald
Schon.
Burt, Martha. 2010. "Life After Transitional Housing for Homeless Families." U.S. Department
of Housing and Urban Development. Urban Institute. Accessed April 2017.
http://www.huduser.gov/portal/publications/LifeAfterTransition.pdf.
Carman, Joanne. 2009. "Nonprofits, Funders, and Evaluation: Accountability in Action." The
American Review of Public Administration 39 (4): 374-390.
City of Portland, Oregon. 2013. A Home for Everyone: A United Community Plan to End
Homelessness to Portland/Multnomah County. Accessed April 2017.
https://www.portlandoregon.gov/phb/article/443314.
Curtis, Marah A., Hope Corman, Kelly Noonan, and Nancy E. Reichman. 2013. Life
Shocks and Homelessness. Demography 50(6), 2227-2253.
Fisher, Craig, and Bruce Kingma. 2001. "Criticality of Data Quality as Exemplified in
Disasters." Information and Management 39: 109-116.
Gubits, Daniel, et al. 2016. Family Options Study: 3-Year Impacts of Housing and Services
Interventions for Homeless Families. Accessed April 2017.
https://www.huduser.gov/portal/sites/default/files/pdf/FamilyOptionsStudySummary
Report.pdf.
Harbarger, Molly. 2016. Multnomah County Opens Women and Couples Homeless Shelters in
Southeast Portland. Accessed April 2017.
http://www.oregonlive.com/portland/index.ssf/2016/11/multnomah_county_opens_
women_a.html.
Hatry, Harry. 2006. Performance Measurement: Getting Results. Urban Institute Press.
Hatry, Harry. 2011. A Guide to Data-Driven Performance Reviews. Urban Institute.
Johnson, TP, SA Freels, and JA Parsons, et al. 1997. "Substance Abuse and Homelessness:
Social Selection or Social Adaptation." Addiction 92: 437-445.
National Coalition for the Homeless. 2006. McKinney-Vento Act. Accessed April 2017.
http://www.nationalhomeless.org/publications/facts/McKinney.pdf.
NPACH. 2017. Homelessness Risk Factors. Accessed April 2017.
http://www.npach.org/homelessness-risk-factors.html.
Partnership for Public Service. 2012. "From Data to Decisions II: Building an Analytics
Culture."
Sermons, M. William, and Peter Witte. 2011. State of Homelessness in America: A Research
Report on Homelessness. National Alliance to End Homelessness and Homeless
Research Institute.
Shelton, Katherine, Pamela J. Taylor, Adrian Bonner, and Marianne van den Bree. 2015. "Risk
Factors for Homelessness: Evidence from a Population-Based Study." Psychiatric
Services 60: 465-472.
Smock, Kristina. 2015. 2015 Point-in-Time: Count of Homelessness in
Portland/Gresham/Multnomah County, Oregon. Kristina Smock Consulting.
40
Stefancic, Ana, and Sam Tsemberis. 2007. "Housing First for Long-Term Shelter Dwellers with
Psychiatric Disabilities in a Suburban County: A Four-Year Study of Housing Access and
Retention." The Journal of Primary Prevention 28 (3): 265-279.
Transition Projects. 2016. About Transition Projects. Accessed April 2017.
http://www.tprojects.org/transitionprojects/.
Transition Projects. 2016. Housing and Homelessness: A Snapshot. Accessed April 2017.
http://www.tprojects.org/LiteratureRetrieve.aspx?ID=162627.
U.S. Department of Housing and Urban Development. 2016. Continuum of Care (CoC) Program
Eligibility Requirements. Accessed April 2017.
https://www.hudexchange.info/programs/coc/coc-program-eligibility-requirements/.
U.S. Department of Housing and Urban Development. 2011. "Part 91- Consolidated Submissions
for Community Planning and Development Programs."
U.S. Department of Housing and Urban Development. 2012. "Portland, Oregon: Innovative
Homeless Service Model at Bud Clark Commons."
U.S. Department of Housing and Urban Development. 2006. Predicting Staying in or Leaving
Permanent Supportive Housing that Serves Homeless People with Serious Mental
Illness." HUD Office of Policy Research.
U.S. Department of Housing and Urban Development. 2014. Rapid Re-Housing Brief. HUD
Exchange.