Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
Job Search and Hiring with Two-sided Limited Information about
Workseekers’ Skills∗
Eliana Carranza† Robert Garlick‡ Kate Orkin§ Neil Rankin¶
September 25, 2018
Abstract
Firms typically make hiring decisions with limited information about workseekers’ skills and
productivity. Similarly, workseekers make job search and employment decisions with limited
information about their own skills and productivity. These information frictions on both the
demand and supply sides of the labor market can distort job search and hiring decisions, lowering
total employment, total earnings, and firm productivity. We test for the existence of information
frictions using a series of randomized controlled trials in urban South Africa. We directly
assess workseekers’ skills in six domains and experimentally vary whether workseekers receive
information about their own skills and whether they can signal this information to firms. We
find that giving information to workseekers has large effects on their beliefs about their skills,
no effects on their job search behavior, and limited effects on their employment outcomes.
Helping workseekers signal their skills to firms substantially increases their employment rate and
earnings. Employment effects do not differ substantially by measured skills or other plausible
correlates of productivity but are higher for workseekers who receive more positive reports
conditional on the skills. We conclude that information frictions exist on both sides of the labor
market but only the demand-side frictions appear important for labor market outcomes.
∗We are grateful for exceptional work by Emmanuel Bakirdjian, Nilmini Herath, Shelby Carvalho, Allegra Cock-burn, Lukas Hensel, Wim Louw, Svetlana Pimkina, Ashley Pople, Sri Ramesh, Caitlan Russell, Wendy Trott, andLaurel Wheeler as well as multiple staff at the Harambee Youth Employment Accelerator, the Abdul Latif JameelLatif Poverty Action Lab Africa Office (J-PAL Africa), the Universities of Cape Town (UCT), Duke, Oxford, andStellenbosch, the Centre for Study of African Economies. The project is conducted in collaboration with the WorldBank Jobs Group and Africa Gender Innovation Lab, and is partly funded by the Private Enterprise in Low IncomeCountries Consortium. The findings, interpretations, and conclusions in this paper are entirely those of the authors.This study has been approved by the Commerce Faculty Ethics in Research Committee at UCT, the Duke Institu-tional Review Board for the Protection of Human Subjects in Non-Medical Research (protocol # D0368) and theDepartment of Economics Research Ethics Committee at Oxford (protocol # ECONCIA15-055). The pre-analysisplan is available at https://www.socialscienceregistry.org/trials/1631.†World Bank, [email protected]‡Duke University, [email protected]§University of Oxford, [email protected]¶University of Stellenbosch, [email protected]
1
1 Introduction
Many countries face high and rising unemployment and underemployment, particularly for young
people (World Bank, 2018). This challenge occurs across both developed and developing countries in
Latin America, the Middle East, North Africa, South Europe, and sub-Saharan Africa. This poses
major economic, political and social challenges: periods of unemployment can permanently harm
youths’ employment trajectories and contribute to crime and political instability (Altonji et al.,
2015; Blattman and Annan, 2016; Freeman, 1999). At the same time, firms in many developing
countries have low labor productivity and low growth rates and report that they cannot find and
hire suitable workers (Hardy and McCasland, 2017; Hsieh and Olken, 2014).
Limited information about workseekers’ skills can contribute to this problem by distorting
labor supply, labor demand, and firm-worker matching, particularly but not exclusively for young
people. On the demand side, firms who observe noisy proxies for workseekers’ skills may hire fewer
workers and pay lower wages if they face some combination of concave production technology in
skill-adjusted labor use, minimum wages, firing costs, or risk aversion. On the supply side, noise
in workseekers’ beliefs about their own skills will distort their job search behavior and potentially
their employment outcomes, though the direction of these distortions is theoretically ambiguous.
Both demand- and supply-side distortions can reduce the quality of firm-worker matches, reducing
output and earnings even if there are no employment effects. Both the demand- and supply-side
distortions will be larger when formal education is weakly correlated with skills and for young
workseekers who cannot use prior work experience to certify their skills. The latter effect can lead
to inefficiently low hiring of young workseekers from a social welfare perspective, as firms do not
want to incur the cost of making first-time hires and allowing the market to learn about workseekers’
skills through experience.
We use a series of experiments in urban South Africa to test the existence of demand- and supply-
side information frictions. We assess roughly 7,000 young workseekers’ skills in six domains using
validated psychometric tests. We randomly assign workseekers to either not see their assessment
results, see their assessment results, or see their assessment results and be able to communicate
these results to firms. (In ongoing work, we also conduct an audit study that experimentally
manipulates how much information firms see about applicants’ skills.) We work with a nationally
recognised assessment and placement agency whose branding helps to give the assessment results
credibility with both workseekers and firms. We measure workseekers’ beliefs about their own skills,
beliefs about the labor market, search behavior, and employment outcomes before and three to six
months after the treatment.
We find evidence that information frictions exist, particularly on the demand side, and distort
labor market outcomes. Workseekers who learn their assessment results update their beliefs about
their own skills and beliefs about returns to search but do not substantially change their job
2
search behavior or employment outcomes. Workseekers who learn their assessment results and can
share the results with firms update their beliefs in similar ways but also experience a 17% rise in
employment and 32% rise in earnings. Their search behavior is mostly unchanged, suggesting these
effects are driven by reductions in demand-side information frictions. We also explore heterogeneity
in the effect of information frictions on workseekers with different skill levels, which is theoretically
ambiguous. We find little evidence that the treatment effects differ by skill, by education, or by
prior work experience.
We build on a growing literature on information frictions in labor markets. One set of papers
documents patterns in panel data consistent with information frictions: the relationship between
wages and skills that are observed by the econometrician but unobserved by firms becomes stronger
with tenure, consistent with employers learning about skills over time (Altonji and Pierret, 2001;
Farber and Gibbons, 1996; Kahn and Lange, 2014; MacLeod et al., 2017). This part of the literature
tests for information frictions using earnings or wage data conditional on employment. This is less
informative in labor markets with high unemployment, informal employment, and self-employment.
In these markets, information frictions may distort the employment margin, stopping firms from
beginning to learn about workseekers’ skills over time. A more recent set of papers show that
experimentally providing more information about workseekers’ skills can change their job search
behavior and employment outcomes (Abebe et al., 2016; Bassi and Nansamba, 2017; Pallais, 2014).
Similarly, Alfonsi et al. (2017) argue that vocational training helps workseekers more than on-the-
job training because it carries a credential employers can observe.
Our work also relates to a literature on firm-side responses to information frictions. Firms using
skills assessments or algorithmic recommendations to make hiring decisions can raise fill rates,
raise productivity and lower turnover (Autor and Scarborough, 2008; Hoffman et al., 2018; Horton,
2017). This helps motivate the idea that information frictions can reduce the quality of firm-worker
matches in addition to any employment effects. Firms may also hire through networks (Beaman
et al., 2018; Ioannides and Loury, 2004), use reference letters as signals of productivity (Abel et al.,
2016), or use apprenticeship programmes as screening mechanisms to identify high-skilled workers
(Hardy and McCasland, 2017). These mechanisms may alleviate information frictions but all have
limitations. Networks can systematically exclude some types of workers and reference letters are
least useful for new labor market entrants.
We contribute to the literature in three ways. First, we separately vary workseekers’ information
about their own skills and their ability to signal this to firms, which allows us to show that demand-
side information frictions are more important than supply-side information frictions. Existing work,
other than Bassi and Nansamba (2017), studies only one side of the market. This is important
because the incidence of and optimal policy response to information frictions differs for demand- and
supply-side frictions. If only one side of the market faces information frictions, then market-based
or state provision of information will be more effective when targeted at that side of the market.
3
If only one side of the market faces information frictions, agents on the other side may be able to
extract rents, either by workseekers misreporting their skills or firms failing to pay workseekers in
line with their skills (Adhvaryu et al., 2017).
Second, we measure the full decision process for workseekers, from job search through hiring to
retention. Existing work, other than Abebe et al. (2016) and Pallais (2014), studies downstream
outcomes and may miss the effect of information frictions on upstream outcomes. For example,
studying the effect of information frictions on earnings trajectories misses effects on employment
and on search behavior. We find that downstream supply-side decisions like job search are weakly
affected by information frictions but demand-side decisions like hiring are sensitive to information
frictions.1
Third, we study how the effect of information frictions differs by workseekers’ skills. The
distribution of effects is theoretically ambiguous and depends on specific modeling choices. We
estimate heterogeneous treatment effects of better information for workseekers with higher and
lower skills. We also provide workseekers with coarse assessment results (reporting their terciles,
not cardinal scores) and use a regression discontinuity design to compare treatment effects on
workseekers with similar skills who receive more or less positive reports. We find little evidence
of heterogeneity by skills other than a positive employment effect of crossing the threshold into
a higher tercile. This result, combined with little evidence of heterogeneity by prior experience
and education, shows that information frictions do not differentially affect high- and low-skilled
workseekers.
With these advances, we provide a more comprehensive description of labor markets with in-
formation frictions. Our finding that demand-side information frictions can be quantitatively im-
portant can help to inform the design of policies to improve labor market functioning. The class of
policies we study are relatively cheap and easy to implement. Most other policies which try to re-
duce youth unemployment, such as wage subsidies, vocational training, entrepreneurship training,
and public works programs are either expensive or do not robustly increase employment (Blattman
and Ralson, 2015; Card et al., 2015; McKenzie, 2017).
We work in a developing country context with high unemployment and with young workseekers.
We argue this is a particularly salient context and population to study. Formal education is weakly
correlated with measured skills and there are many small firms that cannot afford complex in-house
skills assessment systems (Hsieh and Olken, 2014; Pritchett, 2013). Information frictions may also
have particularly serious impacts for young workseekers, who cannot use past work experience
and reference letters as signals of employment (World Bank, 2018). We do not claim our results
generalize to all labor markets. But the indirect evidence of information frictions documented in
the US suggests that our results are relevant even in markets with lower unemployment.
1We work with a sample of workseekers that are actively searching at baseline. Hence we capture effects ofinformation frictions on intensive margin search but may understate effects on extensive margin search.
4
We divide this paper into five substantive sections. We present a simple conceptual description
of a labor market with limited information in section 2. We describe the context in which we work,
our trial, and our sample in section 3. We report average treatment effects in section 4 and explore
heterogeneous treatment effects in section 5 motivated by the conceptual framework.
2 Conceptual Framework
We consider a labor market facing two-sided limited information: firms and/or workseekers may
imperfectly observe workseekers’ skills. We show that these information frictions can distort job
search decisions and job offer decisions, ultimately distorting employment, earnings, productivity,
and revenue. We first describe the full-information labor market and then introduce information
frictions on the supply and demand sides. We begin with a very simple static model and discuss
extensions at the end of this section.
On the supply side, each workseeker i has one heterogeneous attribute, skill Si, and homogeneous
time endowment, T . Based on these attributes, she makes two sequential decisions. Her first
decision is how to allocate time between search for formal work Wi and other activities Oi. Job
search yields utility UW (Si,Wi), discussed in more detail below, while time spent on other activities
yields utility UO(Si, Oi). These utility functions include pecuniary and non-pecuniary benefits and
costs such as wages, monetary costs of job search, and leisure.2 She then receives a job offer with
probability p(Si,W∗i ) with an associated wage Y (Si,W
∗i ). Her second decision is to accept or reject
the job offer.
In a full information world, she chooses W ∗i and O∗i = T −W ∗i to equate the marginal return
from job search and the marginal return from other activities
∂UW (Si,W∗i (Si))
∂Wi=∂UO (Si, Ti −W ∗i (Si))
∂Wi. (1)
If she receives a job offer, she accepts any offer with wage Y (Si,W∗i ) > Y (Si,W
∗i ). Her decision de-
cisions may depend on her heterogeneous skill Si in two ways. First, W ∗i (·) will be a non-degenerate
function of Si if skill and time are non-separable in the utility functions UW (·, ·) and/or UO(·, ·).For example, the return to job search may higher for workseekers with higher skills, in which case
the optimal time allocation to searching for formal work will be increasing in skill. Second, her
wage and/or reservation wage will depend on skill if firms’ pay a skill premium (discussed below)
or workseekers with higher skills have different outside options.
On the demand side, each firm f has a single fixed attribute, productivity Af . The firm produces
2The utility function UW (·, ·) can be interpreted as the reduced-form of three structural functions: a job productionfunction that maps skills and search time into a probability of securing a formal job, a wage function that maps skillsinto earnings conditional on securing a formal job, and a utility function over wage-financed consumption and timeallocated to other activities.
5
output Qf using effective labor LEf and other inputs Kf with a constant elasticity of substitution
(CES) production function that depends on productivity Af . Firms face fixed output prices and
costs of capital. Effective labor LE =∑
i∈f Si is the sum of the skills of all workers employed by
the firm. Absent information frictions, firms’ optimal choice of labor satisfies p× ∂Qf (L∗,K∗)∂L = w,
where w is the wage. Workers are paid their marginal revenue product, so workers with higher
skills receive higher wages. This model gives rise to a threshold rule, where each firm hires workers
in a skill band that depends on the firm-specific technology Af .
In the full-information equilibrium, workseekers’ time allocation and firms’ input choices gen-
erate a wage for each skill level Y ∗(Si,Wi ∗ (Si)) in which all workseekers who receive job offers
accept them.
We now introduce supply-side information frictions. Workseekers observe a noisy proxy Si =
F (Si, εi), where εi captures the information friction. Workseekers allocate time between job search
and other activities based on Si but their payoffs from job search depend on Si.3 Hence, the new
indifference condition is
E[∂UW (Si,W
∗i (Si))
∂Wi
]=∂UO (Si, Ti −W ∗i (Si))
∂Wi(2)
where the expectation is taken over the distribution of εi. The optimal time allocations in conditions
(1) and (2) will be equal if both utility functions are linear in skill and time. Otherwise, the time
allocations will generally be different. If, for example, utility from job search is a concave function
of time allocated to job search, then information frictions will lower the optimal time allocation to
job search. This will in turn lower the formal employment rate and lower earnings.
This framework illustrates that the effect of information frictions will depend on the structure
of the utility functions. The framework does not generate an unconditional prediction about which
types of workseekers (in terms of skills) will be affected most by information frictions. The relative
effects on workseekers with different skills depend on the nature of information friction, captured
by F (·, εi), and on the structure of the utility functions. Even when the information friction is
identical for all workseekers, Si = Si + εi with E[εi|Si] = 0, a nonlinear utility function means that
the information friction may distort decisions differently at different levels of skills.
This structure still applies if skills are multidimensional, provded at least one skill is unobserved.
This structure still applies if workseekers can search in multiple sectors or using multiple strategies:
information frictions will then distort time allocations between search and other activities and
between different types of search.
Labor demand is determined by the decisions of profit-maximizing firms.We assume firm f faces
3With two-sided information frictions their payoff from job search will depend on ˜Si = G(Si, νi), where νi capturesthe information friction facing the firm. The wedge between search-relevant and wage-relevant skills remains, exceptin the special case where firms and workseekers face identical information frictions.
6
produces output Qf using labor Lf and other inputs Kf with a constant elasticity of substitution
(CES) production function and firm-specific technology Af . Absent information frictions, firms’
optimal choice of labor satisfies p× ∂Qf (L∗,K∗)∂L = w, where w is the wage.4 We model heterogeneous
skills by specifying output as a function of effective labor LE =∑
i∈f Si, which is the sum of the
skills of all workers employed by the firm. Workers are still paid their marginal revenue product,
so workers with higher skills receive higher wages. This model gives rise to a threshold rule, where
each firm hires workers in a skill band that depends on the firm-specific technology Af .
Demand-side information frictions occur when firms cannot perfectly observe workseekers’ skills.
The concavity of the CES production means that the expected marginal revenue product from each
work seeker is lower than their expected skills:
E
[p×
∂Qf
(LE∗,K∗
)∂LE∗
]< p×
∂Qf
(E[LE∗] ,K∗)
∂LE∗ . (3)
If wages are completely flexible and all firms face information frictions, then wages will fall at each
level of skills. If there is some floor on wages then the lower wages may not fully offset the fall in
expected revenue and firms will reduce total labor demand.5 Concavity of the production function
is sufficient but not necessary to generate this result. The same result holds if firms are risk-averse,
which may be an appropriate assumption for small firms that cannot easily insure against negative
shocks from low-skilled workers damaging equipment or alienating customers.
This simple framework illustrates that information frictions on either the demand or supply side
of the labor market can generate lower employment and earnings. The framework also illustrates an
important distinction. Eliminating demand-side information frictions can change employment and
earnings without any change in workseekers’ behavior. Eliminating supply-side information frictions
can change employment and earnings only by changing workseekers’ behavior. Our experiment
and data collection are designed both to test for the existence of information frictions and to
differentiate demand- and supply-side frictions. Note that the framework does not deliver a clear
prediction about the incidence of distortions across the skill distribution. Workseekers with higher
skills can experience larger or small distortions under different parameterizations of the production
and utility functions. We view the incidence of distortions as an empirical question and test for
heterogeneous effects of improved information provision on workseekers with different skills.
This framework matches many features of countries with high youth unemployment. Formal
school qualifications are weakly correlated with measured skills and productivity so workseekers
do not learn about their skills during their schooling and cannot signal skills to firms through
4 We assume firms have no market power in the labor or product market so p and w do not depend on the firm’sinput choices. This assumption does not affect the core insights of the framework.
5This wage floor may reflect minimum wages or positive reservation wages from workseekers’ outside options.Even a wage floor at zero may distort hiring decisions if information frictions are very large and there is a positiveprobability that some workseekers will have negative marginal revenue products.
7
schooling qualifications (Fedderke, 2005; Pritchett, 2013; Soderbom and Teal, 2004). In the specific
South African context, Lam et al. (2012) show that measured skills are weakly correlated with grade
progression in schools in low-income neighborhoods, consistent with the idea that years of schooling
are a weak signal of skill. Firms rely more on other signals, such as reference letters and referrals,
which exclude young workseekers who lack references from previous work and people without access
to referral networks (Beaman et al., 2018; Calvo-Armengol and Jackson, 2004; Dustmann et al.,
2015; Ioannides and Loury, 2004). Small and new firms, which cannot easily afford complex work
seeker assessment systems, risk making mistakes during hiring and hence grow more slowly and
struggle to challenge incumbents (Hsieh and Olken, 2014).
This framework can be easily generalized to relax a number of simplifying assumptions. We
focus on single-dimensional skills. But the core results are unchanged if skills are multidimensional,
provided at least one dimension is imperfectly observed. We focus on a single-dimensional measure
of search effort: time. But the core results are unchanged if we allow pecuniary costs of search and
perfect credit markets. With pecuniary credit constraints, distortions due to information frictions
are more difficult to characterize and depend on the joint distribution of true skills, noise, and
access to credit. We assume firms have no market power in the labor or product markets. Relaxing
this assumption does not change the prediction that information frictions distort search and hiring
but does make it harder to characterize welfare effects. Perhaps most importantly, we focus on
a static framework that does not allow firms or workseekers to learn. This framework can be
extended to a multiperiod model where firms observe workers’ skills after hiring them, as in Altonji
and Pierret (2001). As long as revelation is not instantaneous or firms incur hiring or firing costs,
the predictions of the framework are unchanged. This simplification is motivated by the fact that
learning cannot occur for unemployed workers and empirical work in the US shows that firms learn
slowly about employed workers’ skills (Arcidiaono et al., 2010; Lange, 2007).
3 Experiment and Setting
We implement a three-arm randomized controlled trial with 6,891 workseekers in South Africa’s
Gauteng province. The trial is directly designed to test predictions from the conceptual framework
in section 2.
We run the trial with the Harambee Youth Employment Accelerator, a large South African
NGO that provides assessment and placement services to large and medium-sized firms. Haram-
bee recruits young workseekers, assesses them, and selects a fraction of assessed workseekers for
further training and placement at corporate partners. Workseekers are eligible for recruitment and
assessment if they are age 18-29, have no criminal record, are South African citizens or have legal
permission to work in South Africa, have graduated from a high school in a disadvantaged neigh-
borhood, have not been formally employed in the past 12 months, and are actively searching for
8
work. Harambee recruits workseekers through advertising in traditional and social media, referrals,
and some door-to-door recruitment.
These criteria yield a sample of unemployed and underemployed youths from low-income homes
who have limited access to traditional signals of productivity: university qualifications, references
from prior employment, or family connections. This is exactly the sample that our framework
suggests should face information frictions. We deliberately focus on a theory-relevant population
rather than seeking a representative sample. We do not claim our results would generalize to the
broader population in South Africa or in other countries. We show in table 12 that respondents
in our sample have a similar employment rate to the province in which we work but have higher
education and are searching more actively. Our sample contains 62% women, perhaps because
employment rates conditional on education are higher for men in South Africa.
We conduct six skills assessments with workseekers and randomly divide workseekers into four
groups:
1. A control group of 2,275 workseekers, who receive no feedback on their assessment results.
2. A private treatment group of 2,114 workseekers, who receive feedback on their assessment
results in two ways: a roughly one-hour group briefing with a psychologist and printed reports
showing their assessment results and briefly explaining the assessments. These reports contain
no identifying information about the work seeker and no branding (figure 1). This treatment
is designed to improve workseekers’ information about their own skills and hence test for
supply-side information frictions.
3. A public treatment group of 2,248 workseekers, who receive the same treatment as the private
group and 30 color-printed reports that contain the work seeker’s name and national identity
number and the brand of the agency and the World Bank (figure 2). This treatment is
designed to improve workseekers’ information about their own skills and their ability to
inform prospective employers about these skills by sharing the reports and hence test for
both supply- and demand-side information frictions. Workseekers in this group can scan or
photocopy their reports and contact the implementing agency to receive additional reports.
They can also direct prospective employers to https://www.assessmentreport.info/, which
provides more information about the skills assessments, sample questions, and research linking
these assessments to workplace productivity.
4. A placebo treatment group of 254 workseekers, who receive the same reports as members of
the public group but with their own assessment results omitted (figure 6). This treatment
is designed to test if any public treatment effect is driven by the information in the skills
assessments or some other feature of the reports. For example, employers may be more likely
to hire workseekers with reports because the branding makes applications more salient or
9
because just attending the assessment process conveys a positive signal of workseekers’ dedi-
cation. This treatment group is deliberately small because it is designed to test a competing
mechanism rather than a core prediction of the conceptual framework.
All reports place workseekers in terciles, relative to the population of assessed workseekers,
rather than reporting their absolute scores. We piloted versions of the report with only absolute
scores and only relative scores. The absolute scores provided readers with less information, as they
could not easily anchor absolute scores to real outcomes. Providing both the absolute and relative
scores led to complicated reports that overwhelmed and confused users.
We assess workseekers’ skills in six domains. Detailed information on all six assessments is
available at https://www.assessmentreport.info, including sample questions.
1. Numeracy, which focuses on practical arithmetic and pattern recognition. We calculate a
single numeracy score using the inverse variance-weighted average of two numeracy assessment
scores. One assessment is developed by a South Africa-based adult education agency and is
registered with the South African Qualifications Authority. The other assessment is developed
by a large retail chain and used in their applicant screening process.
2. Literacy and communication, which focuses on comprehension of spoken and written passages.
The assessment is developed by a South Africa-based adult education agency and is registered
with the South African Qualifications Authority.
3. Concept formation, which measures fluid intelligence and has a similar design to the Ravens
Progressive Matrices test. The assessment is developed by a South Africa-based adult edu-
cation agency and is registered with the Health Professionals Council of South Africa.
4. Grit, which a self-reported measure of how much determination candidates show when they
work on challenging problems. Our assessment is adapted from the survey-based measure
discussed in Duckworth et al. (2007).
5. Focus, which measures a candidate’s ability to distinguish relevant from irrelevant information
in potentially confusing environments. Our assessment is a shortened and computerized
version of the widely-used Stroop Test (Stroop, 1935).
6. Planning, which measures how candidates behave when faced with complex, multi-step prob-
lems. Our assessment is adapted from a test proposed by Gneezy et al. (2010).
The first three assessments were developed and validated by South African psychologists. We
developed the final three assessments by piloting measures used by psychologists and behavioral
economists in other contexts, administering the assessments with a sample of more than 100 respon-
dents from our target population, verifying that the individual items are appropriately positively
10
Figure 1: Sample Private Report
REPORT ON CANDIDATE COMPETENCIES-Personal Copy-
This report contains results from the assessments you took at Harambee in Phase 1 and Phase 2. These results can helpyou learn about some of your strengths and weaknesses and inform your job search.
You completed assessments on English Communication (listening, reading and comprehension) and Numeracy today inPhase 2. In Phase 1, you completed a Concept Formation assessment which asked you to identify patterns.
1. The Numeracy tests measure various maths abilities. Your score is the average of the two maths tests you did today at Harambee.
2. The Communication test measures English language ability through listening, reading and comprehension.3. The Concept Formation test measures the ability to understand and solve problems. Candidates with high scores
can generally solve complex problems, while lower scores show an ability to solve less complex problems.
You also did some games and questionnaires to measure your soft skills:4. The Planning Ability Test measures how you plan your actions in multi-step problems. Candidates with high
scores generally plan one or more steps ahead in solving complex problems. 5. The Focus Test looks at your ability to pick out which information is important in confusing environments.
Candidates with high scores are able to focus on tasks in distracting situations.6. The Grit Scale measures candidates’ determination when working on difficult problems. Candidates with high
scores spend more time working on the problems rather than choosing to pursue different problems.
Your results have been compared to a large group of young South African job seekers who have a matriccertificate, are from socially disadvantaged backgrounds and have been assessed by Harambee.
You scored in the MIDDLE THIRD of candidates assessed by Harambee for Numeracy, MIDDLE THIRD forCommunication, LOWER THIRD for Concept Formation, LOWER THIRD for Planning Ability, MIDDLETHIRD for Focus and TOP THIRD for the Grit Scale.
DISCLAIMERPlease note that this is a confidential assessment report and is intended for use by the person specified above. Assessment results are not infallible and may not be entirely accurate.
Planning Ability
Notes: This figure shows an example of the reports given to workseekers in the private treatment arm. The reportscontain the work seeker’s assessment results but no identifying information and no branding. This is spacefiller textused to align the position of the figures across pages. It contains no information and should be left hidden usingwhite text.
11
Figure 2: Sample Public Report
REPORT ON CANDIDATE COMPETENCIESname.. surname..ID No. id..This report provides information on assessments conducted by Harambee Youth Employment Accelerator (harambee.co.za), a SouthAfrican organisation that connects employers looking for entry-level talent to young, high-potential work-seekers with a matric orequivalent. Harambee has conducted more than 1 million assessments and placed candidates with over 250 top companies in retail,hospitality, financial services and other sectors. Assessments are designed by psychologists and predict candidates’ productivity andsuccess in the workplace. This report was designed and funded in collaboration with the World Bank. You can find more informationabout this report, the assessments and contact details at www.assessmentreport.info. «name» was assessed at Harambee on 13September, 2016.
«name» completed assessments on English Communication (listening, reading, comprehension), Numeracy, and Concept Formation:1. The Numeracy tests measure candidates’ ability to apply numerical concepts at a National Qualifications Framework (NQF) level,
such as working with fractions, ratios, money, percentages and units, and performing calculations with time and area. This score isan average of two numeracy tests the candidate completed.
2. The Communication test measures a candidate's grasp of the English language through listening, reading and comprehension. Itassesses at an NQF level, for example measuring the ability to recognise and recall literal and non-literal text.
3. The Concept Formation Test is a non-verbal measure that evaluates candidates’ ability to understand and solve problems. Thosewith high scores are generally able to solve complex problems, while lower scores indicate an ability to solve less complexproblems.
«name» also completed tasks and questionnaires to assess their soft skills: 4. The Planning Ability Test measures how candidates plan their actions in multi-step problems. Candidates with high scores gener-
ally plan one or more steps ahead in solving complex problems. 5. The Focus Test assesses a candidate’s ability to distinguish relevant from irrelevant information in potentially confusing
environments. Candidates with high scores are generally able to focus on tasks in distracting surroundings, while candidates withlower scores are more easily distracted by irrelevant information.
6. The Grit Scale measures whether candidates show determination when working on challenging problems. Those with high scores generally spend more time working on challenging problems, while those with low scores choose to pursue different problems.
«name»’s results have been compared to a large benchmark group of young (age 18-34) South Africans assessed by Harambee.All candidates have a matric certificate and are from socially disadvantaged backgrounds. The benchmark group is 5,000 forcognitive skills and 400 for soft skills. «name» scored in the «tercile_num» THIRD of candidates assessed by Harambee for Numeracy, «tercile_lit» THIRD forCommunication, «tercile_cft» THIRD for Concept Formation, «tercile_tol» THIRD for Planning Ability, «tercile_troop»THIRD for Focus and «tercile_grit» THIRD for the Grit Scale.
DISCLAIMER: This is a confidential assessment report for use by the person specified above. The information in the report should only be disclosed on a “need to know basis” with the prior understanding of the candidate. Assessment results are not infallible and may not be entirely accurate. Best practice indicates that any organisation’s career management decisions should depend on factors in addition to these assessment results. Harambee cannot accept responsibility for decisions made based on the information contained in this report and cannot be held liable for the consequences of those decisions.
Notes: This figure shows an example of the reports given to workseekers in the public treatment arm. The reportscontain the assessment results, the workseeker’s name and national identity number, and the logo of the World Bankand the implementing agency. Each work seeker received 20 of these reports and guidelines on how to request morereports.
12
correlated with each other, and shortening the test by selecting the items that best predict the
average score.6 All assessments are conducted on desktop computers, so the assessment results
will be driven in part by candidates’ computer skills.
This design yields six cardinal skills measures. We divide candidates into terciles separately for
each skills measure and display these terciles on the reports. We use both the terciles and cardinal
scores in analysis, which we discuss in section 4.
We interact with workseekers over multiple stages:
1. Workseekers apply on date T1 to be assessed by Harambee, after being recruited.
2. Workseekers are invited to a first assessment on date T2. They complete the concept formation
assessment, a career placement tool (see https://www.shadowmatch.com for details), and
receive some basic job search training (e.g. how to write a CV). Surveys are completed at a
computer workstation. Candidates receive some basic training on how to use a mouse and
keyboard, but the computer skills required are very minimal.
3. Workseekers are invited to a second assessment on date T3. They complete the other five
assessments and a self-administered baseline survey completed at a computer workstation.
This asks questions about their demographics, employment, job search, beliefs about their
skills, self-esteem, beliefs about the labor market returns to skills and search effort, risk
preferences, and time preferences.
4. At the end of day T3, workseekers receive any treatment to which they have been assigned.
5. On day T3 +2 or T3 +3, workseekers are invited to participate in a short text message survey.
This surveys asks about perceived numeracy skills and their self-esteem.
6. Starting 2 months after T3, workseekers complete a detailed follow-up survey by phone. This
survey asks about their current and past employment, job search, beliefs about their skills,
and beliefs about the labor market returns to search effort. The date that we first attempt
to survey workseekers is randomized between 2 and 4 months.
The time lags between T1, T2, and T3 are determined by Harambee’s operations and may last
several month. The assessment dates are based on Harambee’s perception about labor demand
in specific neighborhoods. The time lags will be smaller if workseekers apply during periods of
high labor demand or live in neighborhoods where there is high labor demand. This recruitment
schedule means that some workseekers will complete stage 1 but not proceed to stage 2 or stage 3.
6For the first 23% of the workseekers we assessed, we used self-reported measures of control and flexibility insteadof the focus and planning assessments. The control scale is a self-reported measure of how candidates react whenconfronted with new and confusing problems. The flexibility scale is a self-reported measure of how candidatestackle problems with several potential solution approaches. We adapted both scales from an instrument developedby Hepner and Petersen (1982) and validated them for our target population.
13
Hence our sample at T3, relative to T1, is likely to contain fewer workseekers who are at the margin
of employment and unemployment (they get jobs in the meantime) and who are at the margin
between searching and discouragement (they withdraw from search). Our randomization is based
on workseekers who reach stage 3 so this attrition may affect the generalizability of our results but
not their internal validity.
We implemented the treatment for 84 days between 22 September 2016 and 13 April 2017. We
conducted the phone follow-up survey between 30 November 2016 and 5 September 2017.
Harambee invites some workseekers after date T3 for further training and placement at a corpo-
rate partner. These workseekers are selected based on strong performance in the concept formation
and numeracy assessments and their results in the career placement tool. Only a small fraction of
workseekers are invited for further training and placement and these invitations are independent
of treatment assignment.
We randomize treatment at the day level using a sequential blocking algorithm:
1. We divide the first 18 days of the trial intro 2 blocks of 9 days. We then randomly assign 3
days from each block to control, private treatment, and public treatment.
2. Near the end of block 2, we repeat the randomization process for a third block. We then
repeat this process near the end of blocks 3 to 9 for blocks 4 to 10.
3. If we cannot implement one day of the intervention (e.g. due to power failures or network
problems that mean the baseline cannot be administered) in block B, then we add another
day to block B+1. We then randomly assign one extra day from block B+1 to the same
treatment status as the dropped day from block B.
The first 7 blocks each contain 9 or 10 days. The last 3 blocks contain 6, 13, and 7 days as the
exact end date of the intervention was subject to ongoing negotiation with Harambee. All placebo
treatment days are in block 9. Table 13 shows that the treatment groups are balanced on key
baseline characteristics.
We report summary statistics for key baseline and follow-up variables in Tables 1 and 2. At
both baseline and endline, respondents are actively searching and expect to continue searching.
The baseline employment rate is 38% and, conditional on working, respondents earn an average
of USD90 per week and only 10% of them work full-time. The endline employment rate drops to
32% and employed workers now work slightly more hours and earn roughly the same. Very few
workers have any sort of written contract and 44% work for themselves or family members. This
clearly shows that most of our respondents are strongly attached to the labor market but have low
employment rates and are seldom in “high quality” jobs.
The attrition rates for the text message and phone surveys are respectively 18.6 and 4.1%.
Attrition does not differ by treatment arm, as we show in Appendix Table 15. Attrition does
14
Table 1: Baseline Summary StatisticsBaseline Variable # Obs Mean Std Dev. Min 10th pctile 90th pctile MaxAge 6891 23.6 3.3 18.0 19.8 28.3 35.3Male 6891 0.382 0.486 0 . . 1Completed diploma or degree 6891 0.167 0.373 0 . . 1Completed high school 6891 0.610 0.488 0 . . 1Self-esteem index 6891 0.192 .998 -3.598 -1.347 1.251 1.424# job applications in past 30 days* 6815 9.9 18.6 0 2 20 400Hours of job search in past 7 days* 6699 16.7 19.6 0 2 48 96# planned job applications in next 30 days* 6840 17.8 25.5 0 4 36 190Worked in the past 7 days 6891 0.378 0.485 0 . . 1Hours worked in past 7 days* 2508 13.9 15.0 0 3 40 72Total earnings in past 7 days* 2116 559.9 712.1 0 100 1400 4000Hourly wages in past 7 days 2076 64.3 84.9 0 7.5 160 500
This table shows summary statistics for selected baseline variables. Variables marked with * have been winsorized at the 99thpercentile. Earnings and wages are reported in South African Rands; 1 Rand ≈ USD0.16 in purchasing power parity terms.
Table 2: Endline Summary StatisticsEndline Variable # Obs Mean Std Dev. Min 10th pctile 90th pctile Max
Self-esteem index 6609 -0.056 0.858 -3.598 -1.17 .905 1.424Any job search activity in last 30 days 6608 0.918 0.274 0 . . 1Any job search activity in last 7 days 6608 0.692 0.462 0 . . 1# job applications submitted in last 30 days 6577 12.799 21.467 0 1 27 200Hours spent searching for work in past 7 days 6601 9.935 14.196 0 0 25 125Worked in past 7 days 6607 0.323 0.468 0 . . 1Worked since treatment 6607 0.690 0.462 0 . . 1Worked in 1st calendar month after treatment 6604 0.482 0.500 0 . . 1Worked in 2nd calendar month after treatment 6607 0.454 0.498 0 . . 1Hours worked for all jobs in past 7 days 2121 28.4 21.1 0 4 56 91Total earnings from all jobs in past 7 days* 2112 590.4 928.7 0 2 1500 5300Hourly wages from all jobs in past 7 days* 2097 31.216 52.981 0 .111 77.8 375Have written permanent job contract 2100 0.035 0.183 0 . . 1Employed in wage job 2100 0.563 0.496 0 . . 1Self-employed 2100 0.251 0.434 0 . . 1
This table shows summary statistics for selected endline variables. Variables marked with * have been winsorized at the 99thpercentile. Earnings and wages are reported in South African Rands; 1 Rand ≈ USD0.16 in purchasing power parity terms.
differ over some baseline characteristics: male respondents are more likely to attrite and, in the
text message survey, attrition is higher for those with lower numeracy and concept formation skills
(Appendix Table 15).
4 Average Treatment Effects
We estimate average treatment effects using the following regression model:
Yid = Privateid · β1 + Publicid · β2 + Sd + Xid · µ+ εid (4)
where Privateid and Publicid are treatment group indicators for work seeker i who completed the
intervention on date d, Sid is a vector of randomization block fixed effects, and Xid is a vec-
15
tor of control variables from the baseline survey and assessment process.7 Standard errors are
heteroskedasticity-robust and clustered by intervention date – the unit at which treatment is as-
signed. We report only intention-to-treat estimates and do not construct a measure of treatment
take-up such as using public reports with job applications, as our conceptual framework predicts
that both treatments can affect outcomes through multiple mechanisms.
Building on the conceptual framework in section 2, we test if the average effects of the public and
private treatment are separately equal to zero (β1 = 0 and β2 = 0), equal to each other (β1 = β2),
and jointly equal to zero (β1 = β2 = 0). We report p-values for these tests using the cluster-
and heteroskedasticity-robust standard errors. To account for false discovery rates in multiple
hypothesis testing, we also report the corresponding q-values that control the false discovery rate
with each family of outcomes, following Benjamini et al. (2006).8
4.1 Summary index results
We begin by estimating the public and private treatment effects on seven outcome families, three to
four months after treatment. We classify all outcomes into seven families and construct a summary
index for each family, as an inverse-covariance weighted average of the outcomes in the family
Anderson (2008). We examine beliefs about own skills, beliefs about returns to job search, beliefs
about future earnings, search effort, search effectiveness, employment status, and employment
quality.9. This gives us a general overview of the types of outcomes affected by treatment, and
guides our discussion of individual outcomes within these families in the next subsections.
We find that both treatments have large effects on beliefs about own skills, small effects on
beliefs about the returns to job search, and little effect on beliefs about earnings, search effort, search
effectiveness, and employment quality. The public treatment substantially increases employment
status but the private treatment does not. These results show that workseekers do face limited
information about their skills. Providing new information about their skills changes their beliefs
but only changes employment outcomes when they can communicate this information to firms. The
lack of results on search suggests that the key margin is changes in firm behavior, though we add
some caveats to this interpretation in subsection 4.3.
7We prespecified baseline controls for numeracy, communication, concept formation, and grit scores; the firstprincipal component of focus and planning scores; self-esteem score; expected numeracy score; expected tercile of thecommunication score; expected tercile of the concept formation score; age; gender; an indicator for having educationabove the sample median; and an indicator for working in the week before baseline. We selected these controls from alarger list by running a LASSO algorithm for each pre-specified outcome and including all controls that were selectedby the LASSO for at least 50 percent of the outcome measures (Tibshirani, 1996).
8We do not correct for adjust testing across outcome families. These measure different types of outcomes thatmay be affected in different ways by our treatments.
9We first standardize each outcome to have mean zero and standard deviation one–if necessary, we recode their signto make the direction of all outcomes uniform and replace missing values with zeroes. We then calculate a weightedaverage of the demeaned primary outcomes in each domain, where the weights equal the diagonal elements of theinverse covariance matrix. Each index has mean zero but their standard deviations vary; indices whose componentsare more strongly correlated will have lower variance.
16
Table 3: Effects of Public and Private Treatments on Outcome Indices(1) (2) (3) (4) (5) (6) (7)
Beliefs about . . . Search . . . Employment . . .Skill Search Earnings Effort Effective Status Quality
Public 0.706∗∗∗ 0.079∗∗ 0.002 0.011 0.030 0.132∗∗∗ 0.018(0.036) (0.025) (0.029) (0.031) (0.034) (0.025) (0.027)
Private 0.558∗∗∗ 0.045∗ 0.012 0.013 0.031 0.047 0.014(0.033) (0.026) (0.027) (0.033) (0.031) (0.029) (0.025)
# observations 6367 6367 6367 6367 6367 6367 6367# clusters 81 81 81 81 81 81 81R-squared 0.293 0.055 0.306 0.083 0.046 0.115 0.015Control mean 0.000 0.000 -0.000 -0.000 0.000 0.000 0.000
p: Pub = 0 0.000 0.003 0.950 0.739 0.369 0.000 0.515p: Pvt = 0 0.000 0.091 0.655 0.698 0.311 0.108 0.563p: Pub = Pvt 0.000 0.189 0.672 0.936 0.977 0.001 0.900p: Pub = Pvt = 0 0.000 0.010 0.871 0.918 0.527 0.000 0.769
Outcomes are expressed in standard deviations of summary indices (Anderson, 2008). All re-gressions include randomization block fixed effects and baseline values of the outcome; education;all assessment results; beliefs about numeracy, communications & concept formation results; age;gender; and employment status. Heteroskedasticity-robust standard errors clustered by treatmentday are shown in parentheses. Lower rows of the table show p-values for linear combinations ofcoefficients.
4.2 Effects on Employment
The public treatment increases both the employment status index and each component of the index
(Table 4). Respondents are 5 percentage points more likely to have worked in the week before the
phone survey and at any point since the baseline, off bases of respectively 30 and 67% (columns 1
and 2). They worked 16 percent more hours in the preceding week (column 3). And they found
work soon after treatment: they are more likely to have worked in the first or second month after
treatment (columns 5 and 6).10 The private treatment effects are uniformly positive but always
smaller than the public treatment effects, seldom statistically significantly different to zero, and
often statistically significantly smaller than the public effects.
The public treatment does not affect the full employment quality index but does affect several
components (Table 5). The public treatment raises earnings in the past week by 32% and hourly
wages by 20% (columns 1 and 2). The public treatment has no effect on the formality of the job
(columns 3 and 4) and little effect on the probability of wage employment versus self employment
10Given variation in the length of time between baseline and phone survey, we only observe 3,230 workseekers in thesample four months after treatment, enabling us to calculate effects on employment one, two and three months aftertreatment. For the remainder, the phone survey is three months after baseline, which only enables us to calculateeffects on employment after one and two months. The effects after three months are similar in size to those after oneand two months but are not statistically significantly different from zero because the sample is smaller.
17
Table 4: Effects of Public and Private Treatments on Employment Quantity
(1) (2) (3) (4) (5) (6)Worked . . . Hours worked Worked in . . . month
past week since baseline in past week First Second Third
Public 0.052∗∗∗ 0.052∗∗∗ 1.339∗ 0.035∗∗∗ 0.058∗∗∗ 0.038(0.012) (0.010) (0.571) (0.011) (0.014) (0.022)
Private 0.011 0.031∗ 0.525 0.027 0.009 0.010(0.012) (0.013) (0.513) (0.013) (0.015) (0.021)
# observations 6365 6365 6356 6362 6365 3230# clusters 81 81 81 81 81 81R-squared 0.041 0.089 0.034 0.095 0.074 0.057Control group mean 0.309 0.671 8.794 0.465 0.437 0.403Control mean | employed 28.672
p: Pub = 0 0.000 0.000 0.021 0.002 0.000 0.082q: Pub = 0 0.001 0.001 0.014 0.002 0.001 0.031p: Pvt = 0 0.348 0.021 0.309 0.035 0.565 0.618q: Pvt = 0 0.771 0.142 0.771 0.142 0.792 0.792p: Pub = Pvt 0.002 0.097 0.152 0.457 0.001 0.210q: Pub = Pvt 0.005 0.193 0.236 0.354 0.004 0.267p: Pub = Pvt = 0 0.000 0.000 0.070 0.006 0.000 0.204q: Pub = Pvt = 0 0.001 0.001 0.044 0.007 0.001 0.109
All regressions include randomization block fixed effects and baseline values of the outcome; educa-tion; all assessment results; beliefs about numeracy, communications & concept formation results;age; gender; and employment status. Heteroskedasticity-robust standard errors clustered by treat-ment day are shown in parentheses. Lower rows of the table show p- and q-values for linearcombinations of coefficients. The q-values adjust for multiple testing over all outcomes in the samedomain following Benjamini et al. (2006).
or family employment (columns 5 and 6).11 The private treatment substantially increases earnings
but has no effect on formality or employment type.
The earnings and wage effects may occur at the extensive margin – more workseekers are
employed and hence have positive earnings – or the intensive margin – workseekers who would
have jobs anyway earn more. We show that these effects are separately identified using a simple
derivation. Setting earnings for the non-working equal to zero and restricting attention to the
11The outcome in column 5 is a binary variable equal to one if the respondent has wage employment and zero ifthe respondent has self employment, family employment, or no employment. The outcome in column 6 is equal toone if the respondent has self employment and zero otherwise.
18
Table 5: Effects of Public and Private Treatments on Employment Quality
(1) (2) (3) (4) (5) (6)In past week . . . Contract type . . . Employment type
earnings wages written permanent wage self
Public 49.635∗∗∗ 1.782∗ 0.020 0.004 0.024∗ 0.010(17.892) (0.830) (0.010) (0.003) (0.012) (0.008)
Private 43.855∗∗∗ 1.009 0.017 0.005 0.022 -0.012(18.183) (1.013) (0.009) (0.003) (0.011) (0.007)
# observations 6347 6333 6333 6333 6333 6333# clusters 81 81 81 81 81 81R-squared 0.027 0.022 0.014 0.004 0.023 0.021Control mean 155.163 8.731 0.120 0.009 0.172 0.081Control mean | employed 504.858 28.645 0.392 0.029 0.562 0.265
p: Pub = 0 0.007 0.035 0.055 0.266 0.049 0.187q: Pub = 0 0.116 0.211 0.211 0.500 0.211 0.454p: Pvt = 0 0.018 0.322 0.072 0.148 0.043 0.080q: Pvt = 0 0.158 0.210 0.158 0.177 0.158 0.158p: Pub = Pvt 0.789 0.465 0.771 0.710 0.903 0.001q: Pub = Pvt 1.000 1.000 1.000 1.000 1.000 0.014p: Pub = Pvt = 0 0.007 0.104 0.091 0.307 0.069 0.004q: Pub = Pvt = 0 0.038 0.167 0.166 0.241 0.152 0.038
All regressions include randomization block fixed effects and baseline values of the outcome; educa-tion; all assessment results; beliefs about numeracy, communications & concept formation results;age; gender; and employment status. Heteroskedasticity-robust standard errors clustered by treat-ment day are shown in parentheses. Lower rows of the table show p- and q-values for linearcombinations of coefficients. The q-values adjust for multiple testing over all outcomes in the samedomain following Benjamini et al. (2006).
control and public treatment groups, and we can write
E[Earn|Public = 1]− E[Earn|Public = 0]︸ ︷︷ ︸≡A
= E[Earn|Public = 1,Work = 1]− E[Earn|Public = 0,Work = 1]︸ ︷︷ ︸≡B
·Pr[Work = 1|Public = 1]︸ ︷︷ ︸≡C
+ E[Earn|Public = 0,Work = 1]︸ ︷︷ ︸≡D
· (Pr[Work = 1|Public = 1]− Pr[Work = 1|Public = 0])︸ ︷︷ ︸≡F
Term A is the average treatment effect on earnings, term B is the average treatment effect on
earnings conditional on working, term C is the level of employment in the public treatment group,
term D is the average earnings for employed members of the control group, and term F is the
treatment effect on employment. Term B is not identified in standard experiments, as introduce
bias when we condition on the potentially selected sample of working respondents. However, the
other four terms are identified, so we can recover B = (A−D · F )/C. If the earnings effect occurs
19
only at the extensive margin, then B = 0; if the earnings effect occurs only at the intensive margin,
then F = 0.
Applying this idea, we find that the average public treatment effect on weekly earnings for
workers employed in the public treatment group is 66 South African Rands (standard error 27),
equal to 12% of baseline weekly earnings for those with jobs. The effect for hourly wages is 0.8
South African Rands (standard error 1.2), equal to 2% of baseline hourly wages for those with
jobs. Taken together, these results show that the public treatment affects earnings by increasing
employment and hours but has little effect on hourly wages. To the extent that wages are a measure
of match quality, it does not appear that the public treatment is substantially improving match
quality.
4.3 Effects on Beliefs and Search
In this section we explore the mechanisms driving the employment effects. Our conceptual frame-
work predicts that the private treatment can change employment outcomes by changing workseekers
beliefs about their skills, hence changing their expected returns to search and/or expected earn-
ings, and hence changing their search effect and/or effectiveness. The public treatment can work
through the same mechanisms but may also change employment by changing firms’ responses to
workseekers’ search activities.
We first discuss treatment effects on respondents’ beliefs about their ability. We compare
workseekers’ beliefs three or four months after treatment, at the time of the phone survey, to their
actual test scores. Then we estimate effects on the proportion of respondents who place themselves
in the correct tercile on the six skills measures.12 Our results, in Table 6, show robust and large
increases in the accuracy of respondents beliefs about their skills that are slightly larger for the
public treatment. Both treatments increase the number of domains in which respondents place
themselves in the correct tercile for all assessments (column 1) and decrease the number of domains
in which they place themselves in a higher tercile or lower tercile (columns 2 and 3). Effects on
over-confidence are larger than on under-confidence, but this may be driven by the higher levels of
over- than under-confidence at baseline. The public effects are uniformly and significantly larger
than the private effects.
We find evidence suggesting that the larger impact of the public treatment may arise from
reinforcement of effects on beliefs when workseekers find work. For one test score, numeracy, we
collected the respondents’ beliefs about which tercile they fall into immediately after treatment
via SMS. In Table 7, we observe that immediately after treatment, the public and private treat-
ment effects on beliefs are nearly identical (column 1). The difference in treatment effects on
beliefs between public and private treatments only arises after three to four months (column 2).
12The measures range from 0 for those who are inaccurate on all skills to 3 for those who are accurate on all skills.
20
Tab
le6:
Tre
atm
ent
Eff
ects
onB
elie
fs(1
)(2
)(3
)(4
)(5
)(6
)(7
)(8
)B
elie
fab
out
#te
rciles
#te
rciles
#te
rciles
Bel
ief
ab
out
Exp
ecte
d#
month
sB
elie
fab
out
Exp
ecte
dfu
ture
skill
index
acc
ura
teov
erco
nfiden
tunder
confiden
tse
arc
hin
dex
toget
work
earn
ings
index
earn
ings
Public
trea
tmen
t0.7
06***
0.9
51***
-0.6
40***
-0.3
06***
0.0
79***
-0.0
28***
0.0
02
-89.9
52
(0.0
36)
(0.0
48)
(0.0
32)
(0.0
35)
(0.0
25)
(0.0
10)
(0.0
29)
(437.2
58)
Pri
vate
trea
tmen
t0.5
58***
0.7
41***
-0.5
40***
-0.1
93***
0.0
45*
-0.0
15
0.0
12
-40.7
59
(0.0
33)
(0.0
47)
(0.0
37)
(0.0
33)
(0.0
26)
(0.0
12)
(0.0
27)
(380.4
62)
#O
bs
6367
6367
6367
6367
6367
6099
6367
6355
#cl
ust
ers
81
81
81
81
81
81
81
81
R-s
quare
d0.2
93
0.2
78
0.4
83
0.1
16
0.0
55
0.0
46
0.3
06
0.0
58
p:
Public
=0
00
00
0.0
03
0.0
06
0.9
50
0.8
38
q:
Public
=0
-0.0
01
0.0
01
0.0
01
-0.0
18
-1.0
00
p:
Pri
vate
=0
00
00
0.0
91
0.1
98
0.6
55
0.9
15
q:
Pri
vate
=0
-0.0
01
0.0
01
0.0
01
-0.2
27
-1
p:
Public
=P
riva
te0
00.0
04
0.0
02
0.1
89
0.2
24
0.6
72
0.9
07
q:
Public
=P
riva
te-
0.0
01
0.0
06
0.0
05
-0.3
65
-1
p:
Public
=P
riva
te=
00
00
00.0
10
0.0
23
0.8
71
0.9
79
q:
Public
=P
riva
te=
0-
0.0
01
0.0
01
0.0
01
-0.0
41
-1
All
regre
ssio
ns
incl
ude
random
izati
on
blo
ckfixed
effec
tsand
base
line
valu
esof
the
outc
om
e;ed
uca
tion;
all
ass
essm
ent
resu
lts;
bel
iefs
ab
out
num
eracy
,co
mm
u-
nic
ati
on
&co
nce
pt
form
ati
on
resu
lts;
age;
gen
der
;and
emplo
ym
ent
statu
s.H
eter
osk
edast
icit
y-r
obust
standard
erro
rscl
ust
ered
by
trea
tmen
tdate
are
show
nin
pare
nth
eses
.L
ower
row
sof
the
table
show
p-
and
q-
valu
esfo
rlinea
rco
mbin
ati
ons
of
coeffi
cien
ts.
The
q-v
alu
esadju
stfo
rm
ult
iple
test
ing
over
all
outc
om
esin
the
sam
efa
mily
follow
ing
Ben
jam
ini
etal.
(2006).
21
Our interpretation is that as the public reports increase likelihood of employment, workseekers
receive feedback that the information on the report is accurate thus reinforcing their beliefs.13 Nei-
ther treatment significantly changes overall self-esteem in either the text message or phone survey
(columns 3 and 4). Effects on beliefs about ability do not spill over onto general self-worth, either
immediately after treatment or three to four months later.
Table 7: Reconciling Differential Belief Effects(1) (2) (3) (4)
Correct numeracy belief in . . . Above-median self-esteemText survey Phone survey Text survey Phone survey
Public treatment 0.317 0.231 -0.003 0.002(0.015) (0.016) (0.014) (0.015)
Private treatment 0.333 0.203 0.010 -0.000(0.017) (0.016) (0.015) (0.017)
# respondents 5088 5088 0.4861 0.4861# clusters 81 81 81 81Control mean 0.400 0.404 0.487 0.481p: Pub = Pvt 0.219 0.039 0.414 0.896p: Pub effects equal 0.000 0.777p: Pvt effects equal 0.000 0.585p: Differences equal 0.014 0.480
All regressions include randomization block fixed effects and baseline values of the outcome; education; all assessment re-sults; beliefs about numeracy, communications & concept formation results; age; gender; and employment status. Sample isrestricted to respondents with valid responses from both survey rounds. Round-specific models are estimated as a system.Heteroskedasticity-robust standard errors clustered by treatment day are shown in parentheses. Lower rows of the table showp–values for linear combinations of coefficients.
Effects on workseekers’ beliefs about their returns to search effort are smaller but significant,
and similar for both treatment groups. We asked workseekers the number of applications they
expect to submit in next 30 days and how long they believe they will take to find a formal job
if they submit that many applications, as well as half and twice as many applications. We then
calculate the number of months the workseekers expect it will take for them to get a job, per job
application. The public treatment substantially reduces the expected number of months required to
get a job (Table 6 column 5), while the private treatment has a smaller and statistically significant
effect. (Effects on time to employment with more and fewer applications have the same signs but
are not precisely estimated.)
Neither treatment has any effect on any measure of expected earnings conditional on full-time
employment. The public treatment effects on beliefs are qualitatively consistent with the actual
treatment effects: the treatment increases both actual and expected employment over short time
horizons while having little effect on either actual or expected wages conditional on hours worked.
Consistent with our summary index results, we find no significant effects of either the public
or private treatment on any of the component measures of the job search effort and effectiveness
13Our evidence favors this explanation, but there are a number of other potential reasons. Workseekers may believethe information on the public reports is more credible because it is branded and color-printed. Or information on thepublic reports may be more salient as workseekers may continue to see the information on the reports as they attachthem to job applications. We cannot directly rule out these explanations but we provide some evidence against themin the appendices when we discuss the placebo reports.
22
indices (Appendix Tables 16 and 17). There are no large or statistically significant effects on
whether workseekers searched for jobs in the week prior to the phone survey, hours of search in the
last week, spending on search in the last week, the number of applications submitted in the last 30
days.14 Effects on all measures of search effectiveness are fairly precisely estimated zeros, including
on the number of employer responses of any form, number of offers received, the ratio of employer
responses to the number of applications, and the ratio of offers to number of applicants.
It is possible that by the time of the phone survey, some workseekers may have stopped searching
altogether or be searching less as a result, even if they had searched harder or more effectively
immediately after treatment. Indeed, in table 4 column 4, we showed a significant effect of the
public treatment on employment one month later (and potentially sooner). To test this idea, we
compare search and employment effects across workseekers surveyed in the first and second halves
of the survey period (Table 8). We find that search effects decrease over time, though the differences
are small and imprecisely estimated (columns 1 and 2). This result is robust to instrumenting the
survey date with the randomly assigned target date for the survey.
These results show that allowing workseekers to signal their skills to firms by providing them
with certified skill assessments substantially increases their employment rate and earnings. This
shows that demand-side information frictions can distort hiring decisions and reduce employment
and earnings. Providing information on its own does change work seeker beliefs but this has limited
to no effects on their job search effort or effectiveness or their employment outcomes. Supply-side
information frictions seem to result in only limited distortions in job search decisions and outcomes.
5 Heterogeneous Treatment Effects
5.1 Heterogeneity by Skill
Our conceptual framework predicts that employment effects may differ by skill level, though the
precise incidence of information frictions through the skill distribution depends depend on specific
modeling choices. We explore this hypothesis by constructing an index of the numeracy, commu-
nication, and concept formation scores following Anderson (2008). We then regress employment
outcomes on the skills index using a local linear regression, separately by treatment group, and
show the fitted values in figures 3, 4, and 5.
These results show a strong relationship between the skill index and all employment status
measures. The employment quantity index, hours worked in the preceding 7 days, employment in
the preceding 7 days and employment since treatment are weakly increasing in skill throughout most
of the distribution, though the relationship is sometimes decreasing in the tails, for all treatment
groups. In addition, there are public treatment effects for all these measures: all are higher for
14This may be partially because overall search levels are high. Even in the control group 70 percent of workseekerswere actively searching.
23
Figure 3: Effects of Public and Private Treatments on Employment by Skills
−.1
0.1
.2.3
Em
plo
ym
ent quantity
index
−1 −.5 0 .5 1 1.5Skills index
Control Private Public
−.1
0.1
.2.3
Em
plo
ym
ent qualit
y index
−2 −1 0 1 2Skills index
Control Private Public
Notes: This figure shows the fitted values from local linear regressions of selected employment outcomes on the skillsindex. Regressions are truncated at the 5th and 95th percentiles of the skills index. Vertical lines show the 33rd and67th percentiles of the skills index. Employment index construction is described in the text and follows Anderson(2008). This is filler text used to align figures. This is filler text used to align figures. This is filler text used to alignfigures. This is filler text used to align figures.
24
Figure 4: Effects of Public and Private Treatments on Employment by Skills
.25
.3.3
5.4
.45
Work
ed in last 7 d
ays
−1 −.5 0 .5 1 1.5Skills index
Control Private Public
.6.7
.8.9
Work
ed s
ince tre
atm
ent
−1 −.5 0 .5 1 1.5
Skills index
Control Private Public
Notes: This figure shows the fitted values from local linear regressions of selected employment outcomes on the skillsindex. Regressions are truncated at the 5th and 95th percentiles of the skills index. Vertical lines show the 33rd and67th of the skills index. This is filler text used to align figures. This is filler text used to align figures. This is fillertext used to align figures. This is filler text used to align figures. This is filler text used to align figures.
25
Figure 5: Effects of Public and Private Treatments on Employment by Skills
68
10
12
Hours
work
ed in last 7 d
ays
−1 −.5 0 .5 1 1.5Skills index
Control Private Public
150
200
250
300
Tota
l earn
ings in last 7 d
ays
−1 −.5 0 .5 1 1.5Skills index
Control Private Public
Notes: This figure shows the fitted values from local linear regressions of selected employment outcomes on the skillsindex. Regressions are truncated at the 5th and 95th percentiles of the skills index. Vertical lines show the 33rd and67th of the skills index. Hours and earnings are both reported for the preceding seven days, asked separately aboutmain and othe jobs and then combined, winsorized at their 99th percentiles, and set to zero for respondents who didnot work in the preceding seven days.
26
Table 8: Dynamic Effects and Survey Timing(1) (2) (3) (4)
Search Effort Search Effective Work Status Worked in past 7 daysPublic × Early 0.047 0.067 0.152∗∗∗ 0.052∗∗
(0.053) (0.044) (0.036) (0.018)Public × Late -0.042 -0.017 0.112∗∗ 0.054∗∗
(0.042) (0.036) (0.040) (0.018)Private × Early 0.058 0.066 0.047 0.017
(0.051) (0.044) (0.040) (0.021)Private × Late -0.049 0.007 0.047 0.007
(0.040) (0.035) (0.039) (0.016)Late Phone Survey 0.037 -0.020 0.018 0.012
(0.048) (0.033) (0.042) (0.018)# respondents 6367 6367 6367 6365# clusters 81 81 81 81R2 0.041 0.017 0.115 0.041p: Average Public = 0 0.453 0.191 0.000 0.000p: Public × Late = 0 0.381 0.127 0.000 0.006p: Public × Early = 0 0.328 0.644 0.007 0.003p: Average Private = 0 0.230 0.326 0.272 0.622p: Private × Early = 0 0.258 0.137 0.250 0.417p: Private × Late = 0 0.230 0.849 0.240 0.670
All regressions include randomization block fixed effects and baseline values of the outcome; education; all assessment re-sults; beliefs about numeracy, communications & concept formation results; age; gender; and employment status. Sample isrestricted to respondents with valid responses from both survey rounds. Round-specific models are estimated as a system.Heteroskedasticity-robust standard errors clustered by treatment day are shown in parentheses. Lower rows of the table showp–values for linear combinations of coefficients.
the public group than the private or control groups for most of the skills distribution. In contrast,
the relationship between the private and control groups varies over the distribution and is often
fairly similar. On earnings, there is a less strong relationship between skills and earnings. This is
consistent with a labor market where entry-level wages for workseekers with limited education and
little work experience may be fairly fixed regardless of work seeker ability.
However, there is little evidence of heterogeneous treatment effects: the gap between the public
and other groups is similar at all levels of the skills distribution. We test this idea more formally
by estimating models of the form
Yid = Privateid ·β1 +Publicid ·β2 +Privateid ·Highid ·γ1 +Publicid ·Highid ·γ2 +Sd+Xid ·µ+εid (5)
where Highid is an indicator equal to one for workseekers with above-median values of the skills index
and is included in the control vector Xid. We report the estimated coefficients of this regression in
columns 1 and 2 of Table 9. We gain find little evidence of heterogeneous treatment effects by skill.
5.2 Heterogeneity by Skill or Signal?
Treatment effects may differ by skill for two reasons. First, workseekers with different skills receive
different information about themselves and are able to send a different signal to firms. Second,
workseekers with different skills may use information and technology differently or face different
information frictions. For example, more skilled workseekers may be better at understanding the
27
Tab
le9:
Het
erog
eneo
us
Tre
atm
ent
Eff
ects
by
Skil
lan
dP
rod
uct
ivit
yM
easu
res
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
Sta
tus
ind
exQ
uality
ind
exS
tatu
sin
dex
Qu
ality
ind
exS
tatu
sin
dex
Qu
ality
ind
exS
tatu
sin
dex
Qu
ality
ind
exP
ub
lic×
Hig
h-0
.034
0.0
15
-0.0
58
0.0
49
-0.0
39
0.0
86
0.0
36
0.0
05
(0.0
49)
(0.0
61)
(0.0
52)
(0.0
69)
(0.0
67)
(0.0
76)
(0.0
66)
(0.0
81)
Pu
blic
0.1
40∗∗∗
0.0
67∗
0.1
42∗∗∗
0.0
40
0.1
38∗∗∗
0.0
42
0.1
18∗∗∗
0.0
74∗∗
(0.0
34)
(0.0
37)
(0.0
34)
(0.0
39)
(0.0
33)
(0.0
31)
(0.0
26)
(0.0
34)
Pri
vate×
Hig
h-0
.040
0.0
72
-0.0
32
-0.0
29
-0.0
66
-0.0
89
-0.0
00
0.0
23
(0.0
44)
(0.0
60)
(0.0
54)
(0.0
67)
(0.0
61)
(0.0
58)
(0.0
84)
(0.0
97)
Pri
vate
0.0
78∗∗
0.0
27
0.0
64∗
0.0
66
0.0
82∗∗
0.0
98∗∗
0.0
59∗∗
0.0
59
(0.0
36)
(0.0
44)
(0.0
37)
(0.0
46)
(0.0
33)
(0.0
40)
(0.0
28)
(0.0
36)
Het
erogen
eity
mea
sure
Skill
ind
exE
mp
loyab
ilit
yin
dex
Base
lin
eem
plo
ym
ent
Deg
ree/
dip
lom
a#
resp
on
den
ts6367
6367
6367
6367
6367
6367
6367
6367
#cl
ust
ers
81
81
81
81
81
81
81
81
R2
0.0
81
0.0
15
0.0
82
0.0
15
0.0
81
0.0
15
0.0
81
0.0
14
All
regre
ssio
ns
incl
ud
era
nd
om
izati
on
blo
ckfi
xed
effec
tsan
db
ase
lin
evalu
esof
the
ou
tcom
e;ed
uca
tion
;all
ass
essm
ent
resu
lts;
bel
iefs
ab
ou
tnu
mer
acy
,co
mm
un
icati
on
s&
con
cep
tfo
rmati
on
resu
lts;
age;
gen
der
;an
dem
plo
ym
ent
statu
s.H
eter
osk
edast
icit
y-r
ob
ust
stan
dard
erro
rscl
ust
ered
by
trea
tmen
td
ay
are
show
nin
pare
nth
eses
.C
olu
mn
s1-4
show
inte
ract
ions
bet
wee
ntr
eatm
ent
ind
icato
rsan
din
dic
ato
rsfo
rab
ove-
med
ian
valu
esof
the
skill
ind
exor
late
nt
pro
bab
ilit
yof
emp
loym
ent
ind
ex.
Colu
mn
s1-4
show
inte
ract
ion
sb
etw
een
trea
tmen
tin
dic
ato
rsan
din
dic
ato
rsfo
rw
ork
ing
at
base
line
or
havin
ga
un
iver
sity
deg
ree/
dip
lom
aat
base
lin
e.
28
information, may convey information about themselves better in interviews, or use the reports more
effectively. We call these two effects respectively treatment heterogeneity effects and treatment effect
heterogeneity.
Identifying treatment heterogeneity effects requires comparing workseekers with similar skills
who receive different information. We use a regression discontinuity to compare workseekers at the
margin of being placed in different terciles.15 We stack the two cutoffs so each workseeker may be
used twice in the analysis. Given the low power of regression discontinuity designs, these results
should be treated with caution.
In the public group, crossing a threshold increases the employment status index, probability
of current employment, and current earnings, though the first two effects are only marginally
statistically significant (Table 10). There are no similar effects for the private or control groups.
We see no effects on any measures of search behavior. These results are consistent with a role for
treatment heterogeneity effects, in which workseekers who can send more positive reports to firms
benefit more.
Table 10: Effect of Treatment Heterogeneity by SkillIndex of employment . . . In past 7 days . . .status quality worked earnings
Public 0.147 -0.018 0.058 103.2(0.085) (0.073) (0.037) (37.7)
p: Pub = 0 0.084 0.805 0.117 0.006# observations 4496 4496 4322 4476# clusters 81 81 81 81Private -0.016 -0.129 -0.042 -92.0
(0.061) (0.127) (0.029) (67.9)p: Pvt = 0 0.793 0.310 0.148 0.176# observations 4228 4228 4040 4216# clusters 81 81 81 81Control -0.075 0.021 -0.004 -20.3
(0.088) (0.063) (0.045) (39.0)p: Control = 0 0.394 0.739 0.929 0.603# observations 4550 4550 4368 4542# clusters 81 81 81 81
Outcomes in columns 1 - 2 are expressed in standard deviations of domain-level indices (Anderson, 2008). Results are obtainedby stacking the top-middle and middle-low tercile cutoffs for the numeracy assessment, recentering each cutoff at zero, andestimating discontinuities separately by treatment group. All results are generated using local linear regressions with triangularkernels and the robust bandwidth estimator from Calonico et al. (2014). Heteroskedasticity-robust standard errors clusteredby treatment day are shown in parentheses. Lower rows of the table show p-values for linear combinations of coefficients.
Identifying treatment effect heterogeneity requires comparing workseekers with different skills
who receive the same information. We thus compare workseekers in the top and bottom half of
each tercile. We focus only on numeracy scores for consistency with the previous analysis, though
results are again robust to including other scores. We find little evidence of either public or private
treatment effect heterogeneity: there are no significant differences in treatment effects between
workseekers with higher or lower skills, within a tercile.
15We only use the numeracy assessment for this exercise because it is the least coarse score. Results are similarwhen we add the grit and concept formation scores, which are less coarse than the communication score. When weuse all three scores, we stack all six cutoffs.
29
Table 11: Treatment Effect Heterogeneity by SkillIndex of employment . . . In past 7 days . . .status quality worked earnings
Public 0.105 0.012 0.041 53.6(0.047) (0.046) (0.021) (37.8)
Public * High -0.002 0.002 -0.003 4.3(0.059) (0.065) (0.027) (42.5)
Private -0.024 -0.020 -0.024 13.8(0.050) (0.042) (0.022) (35.2)
Private * High 0.067 0.056 0.040 60.9(0.056) (0.067) (0.026) (45.0)
High 0.001 -0.005 0.007 -22.4(0.045) (0.039) (0.021) (28.4)
# respondents 6367 6367 6365 6347# clusters 81 81 81 81R-squared 0.105 0.010 0.038 0.013Control mean 0.000 -0.000 0.309 152.9Control mean | employed 504.9p: Pub = 0 0.025 0.775 0.062 0.127p: Pub × High = 0 0.973 0.975 0.912 0.920p: Pvt = 0 0.631 0.634 0.275 0.695p: Pvt × High = 0 0.232 0.403 0.124 0.176
Outcomes in columns 1 - 2 are expressed in standard deviations of domain-level indices (Anderson, 2008). All regressions includerandomization block fixed effects and baseline values of the outcome; education; all assessment results; beliefs about numeracy,communications & concept formation results; age; gender; and employment status. Heteroskedasticity-robust standard errorsclustered by treatment day are shown in parentheses. Lower rows of the table show p-values for linear combinations of coefficients.
5.3 Heterogeneity by Other Potential Productivity Correlates
Even if treatment effects differ only weakly by skill, they may differ with other covariates that
either predict workplace productivity or are perceived by firms to predict workplace productivity.
We estimate the parameters of equation (5) replacing Highid with indicators for working at baseline,
having a university degree or diploma at baseline, and being above the median value of a latent
employability index. We construct the latter index by regressing endline employment on a vector
of baseline covariates using the control group observations and using linear predictions as the
employability index. We select the baseline covariates using a LASSO algorithm and, following
Abadie et al. (2014) we use out-of-sample estimation in the control group to avoid overfitting.
We show in Table 9 that there is little evidence of heterogeneity in treatment effects across any
of these measures. None of the interaction terms is large or statistically significantly different from
zero. We repeat this exercise for a longer list of prespecified baseline markers of heterogeneity –
age, gender, baseline employment history, other education measures, and time and risk preferences
– and do not find robust evidence of heterogeneity. (Results available on request.)
6 Conclusion
Firms typically make hiring decisions with limited information about workseekers’ skills and produc-
tivity. Similarly, workseekers make job search and employment decisions with limited information
about their own skills and productivity. These information frictions on both the demand and supply
30
sides of the labor market can distort job search and hiring decisions, lowering total employment, to-
tal earnings, and firm productivity. We study information frictions using a randomized controlled
trial and longitudinal survey in urban South Africa. South Africa is marked by extremely high
unemployment, particularly amongst youths, and a weak education system. This suggests that
information frictions may be particularly important in this market.
We directly assess workseekers’ skills and experimentally vary whether workseekers receive in-
formation about their own skills and whether they can signal this information to firms. We find
that giving information to workseekers has large effects on their beliefs about their skills, some early
effects on their job search behavior (which do not persist) and limited effects on their employment
outcomes. Helping workseekers signal their skills to firms substantially increases their employment
rate and earnings, driven partly by increased search effort but also by workseekers being more able
to signal skills to firms. We conclude that information frictions exist on both sides of the labor
market but only the demand-side frictions appear important for labor market outcomes.
Our results contribute to a growing literature on information frictions in the labor market.
We add to a small body of research that directly manipulates information about workseekers’
skills. Relative to this literature, we separately vary workseekers’ information about their own
skills and their ability to signal this to firms, which allows us to separate supply- and demand-side
information frictions. We provide more detailed measures of job search and employment quality
than most research in this area, which allows us to show that changes in search behavior do not
explain information-induced changes in employment outcomes. Our findings show that information
frictions can substantially distort individual workseekers’ employment outcomes. In ongoing work,
we build on this to test if information frictions reduce total labor demand or simply help some
workseekers at others’ expense. This ongoing programme of work will more thoroughly characterize
the employment and welfare effects of information frictions on both sides of the labor market.
References
Abebe, G., S. Caria, M. Fafchamps, P. Falco, S. Franklin, and S. Quinn (2016): “TheCurse of Anonymity or Tyranny of Distance? The Impacts of Job-Search Support in UrbanEthiopia,” Working Paper 22409, National Bureau of Economic Research.
Abel, M., R. Burger, and P. Piraino (2016): “The Value of Reference Letters -ExperimentalEvidence from South Africa,” Working Paper, Harvard University.
Alfonsi, L., O. Bandiera, V. Bassi, R. Burgess, I. Rasul, M. Sulaiman, and A. Vitali(2017): “Tackling Youth Unemployment: Evidence from a Labor Market Experiment in Uganda,”Manuscript, University College London.
Altonji, J., L. Kahn, and J. Speer (2015): “Cashier or Consultant? Entry Labor MarketConditions, Field of Study, and Career Success,” Journal of Labor Economics, 34, 361–401.
31
Altonji, J. and C. Pierret (2001): “Employer Learning and Statistical Discrimination,” Quar-terly Journal of Economics, 116, 313–335.
Anderson, M. (2008): “Multiple Inference and Gender Differences in the Effects of Early In-tervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects,”Journal of the American Statistical Association, 103, 1481–1495.
Arcidiaono, P., P. Bayer, and A. Hizmo (2010): “Beyond Signaling and Human Capital:Education and the Revelation of Ability,” American Economic Journal: Applied Economics, 2,76–104.
Autor, D. and D. Scarborough (2008): “Does Job Testing Harm Minority Workers? Evidencefrom Retail Establishments,” Quarterly Journal of Economics, 123, 219–277.
Bassi, V. and A. Nansamba (2017): “Screening and Signaling Non-Cognitive Skills: Experimen-tal Evidence from Uganda,” Manuscript, University of Southern California.
Beaman, L., N. Keleher, and J. Magruder (2018): “Do Job Networks Disadvantage Women?Evidence from a Recruitment Experiment in Malawi,” Journal of Labour Economics, 36, 121–153.
Benjamini, Y., A. Krieger, and D. Yekutieli (2006): “Adaptive Linear Step-Up ProceduresThat Control the False Discovery Rate,” Biometrika, 93, 491–507.
Blattman, C. and J. Annan (2016): “Can Employment Reduce Lawlessness and Rebellion? AField Experiment with High-Risk Men in a Fragile State,” American Political Science Review,110, 1–17.
Blattman, C. and L. Ralson (2015): “Generating Employment in Poor and Fragile States: Ev-idence from Labor Market and Entrepreneurship Programs,” Manuscript, Columbia University.
Calvo-Armengol, A. and M. O. Jackson (2004): “The Effects of Social Networks on Employ-ment and Inequality,” American Economic Review, 94, 426–454.
Card, D., J. Kluve, and A. Weber (2015): “What Works? A Meta-Analysis of Recent Ac-tive Labor Market Program Evaluations,” Working Paper 21431, National Bureau of EconomicResearch.
Duckworth, A., C. Peterson, M. Matthews, and D. Kelly (2007): “Grit: Perseveranceand Passion for Long-term Goals,” Journal of Personality and Social Psychology, 92, 1087–1101.
Dustmann, C., A. Glitz, U. Schonberg, and H. Brucker (2015): “Referral-based Job SearchNetworks,” Review of Economic Studies, 83, 514–546.
Farber, H. and R. Gibbons (1996): “Learning and Wage Dynamics,” Quarterly Journal ofEconomics, 111, 1007–1047.
Fedderke, J. (2005): “Technology, Human Capital and Growth,” Working Paper 27, EconomicResearch Southern Africa.
Freeman, R. (1999): “The Economics of Crime,” in Handbook of Labor Economics Volume 5, ed.by O. Ashenfelter and D. Card, Elsevier, 3529–3572.
Gneezy, U., A. Rustichini, and A. Vostroknutov (2010): “Experience and Insight in TheRace Game,” Journal of Economic Behavior and Organization, 75, 144–155.
32
Hardy, M. and J. McCasland (2017): “Are Small Firms Labor Constrained? ExperimentalEvidence from Ghana,” Working Paper, New York University - Abu Dhabi.
Hepner, P. and C. Petersen (1982): “The Development and Implications of a PersonalProblem-Solving Inventory,” Journal of Counseling Psychology, 29, 66–75.
Hoffman, M., L. Kahn, and D. Li (2018): “Discretion in Hiring,” The Quarterly Journal ofEconomics, 133, 1–36.
Horton, J. (2017): “The Effects of Algorithmic Labor Market Recommendations: Evidence froma Field Experiment,” Journal of Labor Economics, 35, 345–285.
Hsieh, C.-T. and B. Olken (2014): “The Missing ‘Missing Middle’,” Journal of EconomicPerspectives, 28, 89–108.
Ioannides, Y. M. and L. D. Loury (2004): “Job Information Networks, Neighborhood Effects,and Inequality,” Journal of Economic Literature, 42, 1056–1093.
Kahn, L. and F. Lange (2014): “Employer Learning, Productivity, and the Earnings Distribu-tion: Evidence from Performance Measures,” Review of Economic Studies, 84, 1575–1613.
Lange, F. (2007): “The Speed of Employer Learning,” Journal of Labor Economics, 25, 1–35.
MacLeod, W. B., E. Riehl, J. Saavedra, and M. Urquiola (2017): “The Big Sort: CollegeReputation and Labor Market Outcomes,” American Economic Journal: Applied Economics, 9,223–261.
McKenzie, D. (2017): “How Effective Are Active Labor Market Policies in Developing Countries?A Critical Review of Recent Evidence,” Working Paper 8011, World Bank Policy Research.
Pallais, A. (2014): “Inefficient Hiring in Entry-Level Labor Markets,” American Economic Re-view, 104, 3565–3599.
Pritchett, L. (2013): The Rebirth of Education: Schooling Ain’t Learning, Washington, DC:Center for Global Development.
Soderbom, M. and F. Teal (2004): “Size and Efficiency in African Manufacturing Firms:Evidence from Firm-level Panel Data,” Journal of Development Economics, 73, 369–394.
Stroop, J. R. (1935): “Studies of Interference in Serial Verbal Reactions,” Journal of ExperimentalPsychology, 18, 643–662.
Tibshirani, R. (1996): “Regression Shrinkage and Selection via the Lasso,” Journal of the RoyalStatistical Society Series B (Methodological), 58, 267–288.
World Bank (2018): World Development Report 2018: Learning to Realize Education’s Promise,World Bank.
33
A Extra Tables and Figures
Table 12: Comparison between our population and nationally representative statistics
South Africa Gauteng Gauteng 18-29 Our SampleAge 28.914 31.461 23.776 23.646
(19.606) (19.099) (3.326) (3.299)Male 0.489 0.504 0.512 0.382Currently Employed 0.290 0.375 0.346 0.378Currently searching 0.102 0.151 0.302 0.984< Grade 12 education 0.737 0.612 0.429 0.011Grade 12 education 0.184 0.255 0.449 0.610> Grade 12 education 0.072 0.120 0.116 0.379
National statistics are calculated from the Quarterly Labour Force Surveys (QLFS), averaging over all 2016 waves. Calculationsuse post-stratification weights supplied by Statistics South Africa
Table 13: Balance Test Statistics
Baseline variable F-test statistic p-value
From clustered SEs From permutation testHours searched in past 7 days 0.01 0.988 0.98Worked in past 7 days 1.75 0.181 0.226Male 1.47 0.236 0.306Grit assessment score 2.91 0.061 0.096Flexibility assessment score 1.02 0.382 0.55Control assessment score 2.58 0.107 0.232Concept formation score 0.00 0.999 0.996Literacy assessment score 0.83 0.442 0.504Numeracy assessment score 0.51 0.602 0.64Self-esteem scale 2.51 0.088 0.124Notes: This table shows balance tests constructed by regressing each of the selected baseline characteristics on indicators for the public and private treatment groups and testing if these coefficients are jointly zero. The tests are conducted using heteroskedasticity-robust standard errors clustered by treatment date and by permuting the treatment assignments within blocks.
34
Table 14: Post-Treatment Survey Attrition Rate by Treatment Group
(1) (2)Text Message Survey 1st Phone Survey
Control group 0.170 0.040(0.013) (0.006)
Public treatment group 0.177 0.039(0.011) (0.004)
Private treatment group 0.182 0.044(0.010) (0.004)
Placebo treatment group 0.142 0.047(0.032) (0.026)
p: Control = Pvt. 0.481 0.632p: Control = Pub. 0.670 0.855p: Pvt. = Pub. 0.785 0.388p: Control = Pvt. = Pub. 0.778 0.681p: Control = Plc. 0.414 0.787p: Pvt. = Plc. 0.238 0.888p: Pub. = Plc. 0.297 0.746p: Control = Pvt. = Pub. = Plc. 0.641 0.841# observations 6891 6891# clusters 84 84
Notes: This table shows the fraction of each treatment group that does not complete each follow-up survey round.Heteroskdasticity-robust standard errors clustered by treatment date are shown in parentheses.
35
Table 15: Post-Treatment Survey Attrition Rate by Baseline Characteristics
(1) (2)Text Message Survey 1st Phone Survey
Numeracy score -0.029∗∗∗ 0.003(0.006) (0.003)
Literacy score 0.009 0.004(0.006) (0.003)
Concept formation score -0.019∗∗∗ 0.002(0.006) (0.003)
Grit score -0.002 -0.007∗∗
(0.005) (0.003)Other scores 0.001 -0.002
(0.004) (0.003)Perceived numeracy score -0.000 -0.000
(0.000) (0.000)Perceived literacy score 0.014 -0.003
(0.010) (0.004)Perceived concept formation score 0.010 -0.003
(0.009) (0.004)Self-esteem index 0.006 0.002
(0.004) (0.002)Completed at most high school -0.008 -0.003
(0.012) (0.005)Age -0.002 0.001
(0.001) (0.001)Male 0.048∗∗∗ 0.014∗∗∗
(0.010) (0.005)Worked in last 7 days -0.005 -0.001
(0.008) (0.005)Missing (Other scores) -0.027 -0.027∗∗∗
(0.041) (0.004)Missing (Perceived numeracy score) -0.008 -0.036∗∗∗
(0.101) (0.011)Missing (Perceived literacy score) -0.005 0.051
(0.069) (0.039)Missing (Perceived concept formation score) 0.011 -0.035∗
(0.056) (0.020)Constant 0.171∗∗∗ 0.029
(0.056) (0.028)
F: All coefficients jointly zero 5.3 6.5p: All coefficients jointly zero 0.000 0.000Adj R2 0.014 0.005# observations 6891 6891# clusters 84 84Control group mean 0.170 0.040
Notes: This table shows the relationship between not completing each survey round and pre-specified baseline covari-ates in the full sample. Heteroskdasticity-robust standard errors clustered by treatment date are shown in parentheses.*; **; and *** denote significance at the 10; 5; and 1 percent levels respectively.
36
Table 16: Effects of Public and Private Treatments on Search Effort(1) (2) (3) (4)
Any search Hours of search Apps submitted Spending on searchlast 7 days last 7 days last 30 days last 7 days
Public -0.02 0.252 0.624 3.001(0.014) (0.368) (0.684) (4.058)
Private -0.006 -0.31 0.078 1.02(0.014) (0.362) (0.648) (4.801)
# observations 6366 6350 6336 6350# clusters 81 81 81 81R-squared 0.031 0.024 0.062 0.036Control mean 0.695 11.083 11.891 122.81p: Public = 0 0.152 0.496 0.364 0.462p: Private = 0 0.702 0.393 0.904 0.832p: Public = Private 0.243 0.137 0.394 0.657p: Public = Private = 0 0.293 0.326 0.599 0.748
All regressions include randomization block fixed effects and baseline values of the outcome; education; all assessment results;beliefs about numeracy, communications & concept formation results; age; gender; and employment status. Heteroskedasticity-robust standard errors clustered by treatment day are shown in parentheses. Lower rows of the table show p- and q-values forlinear combinations of coefficients. The q-values adjust for multiple testing over all outcomes in the same domain followingBenjamini et al. (2006).
Table 17: Effects of Public and Private Treatments on Search EffectivenessNumber of Number of Number of responses Number of offers
employer responses offers per app per apps submittedPublic 0.049 0.007 0.001 0.001
(0.047) (0.017) (0.004) (0.002)Private 0.041 0.017 -0.006 0.002
(0.042) (0.017) (0.004) (0.003)# observations 6351 6350 5732 5731# clusters 81 81 81 81R-squared 0.021 0.011 0.010 0.009Control mean 0.772 0.182 0.097 0.026p: Public = 0 0.298 0.677 0.779 0.624p: Private = 0 0.298 0.677 0.779 0.624p: Public = Private 0.863 0.568 0.095 0.772p: Public = Private = 0 0.488 0.610 0.200 0.759
All variables are measured over the 30 days prior to endline. All regressions include randomization block fixed effects andbaseline values of the outcome; education; all assessment results; beliefs about numeracy, communications & concept formationresults; age; gender; and employment status. Heteroskedasticity-robust standard errors clustered by treatment day are shownin parentheses. Lower rows of the table show p- and q-values for linear combinations of coefficients. The q-values adjust formultiple testing over all outcomes in the same domain following Benjamini et al. (2006).
37
B Placebo Treatment
Firms might respond to the existence or branding of the assessment results, rather than the actual
results. For example, the branding might make applications more salient in a large pool or the
report might be taken as a signal of the workseekers’ effort. We test this explanation with a
placebo treatment in which workseekers can inform firms about the assessments and assessment
process but not their own assessment results (Figure 6. We include only three treatment days and
254 workseekers in this treatment group because it is not our main focus. This means the placebo
treatment effects are imprecisely estimated, but we are still able to rule out some possible effects.
Table 18: Public, Private, and Placebo Treatment Effects on Indices
(1) (2) (3) (4) (5) (6) (7)Beliefs about . . . Search . . . Employment . . .
Skill Search Earnings Effort Effective Status QualityPublic treatment 0.720∗∗∗ 0.080∗∗∗ 0.002 0.029 0.025 0.123∗∗∗ 0.079∗∗
(0.037) (0.026) (0.029) (0.031) (0.028) (0.025) (0.030)Private treatment 0.570∗∗∗ 0.046∗ 0.012 0.013 0.013 0.058∗∗ 0.062∗
(0.034) (0.027) (0.028) (0.033) (0.028) (0.027) (0.032)Placebo treatment -0.055 0.127∗∗ 0.004 0.050 -0.006 0.043 -0.036
(0.065) (0.049) (0.064) (0.062) (0.045) (0.046) (0.063)# respondents 6609 6609 6609 6609 6609 6609 6609# clusters 84.000 84.000 84.000 84.000 84.000 84.000 84.000R-squared 0.298 0.056 0.305 0.140 0.028 0.084 0.020Control group mean 0.000 0.000 -0.000 0.000 0.000 0.009 0.006p: Public = 0 0.000 0.003 0.952 0.346 0.379 0.000 0.011p: Private = 0 0.000 0.087 0.666 0.704 0.642 0.031 0.054p: Placebo = 0 0.399 0.010 0.950 0.424 0.897 0.350 0.566p: Public = Private 0.000 0.197 0.681 0.589 0.707 0.020 0.622p: Public = Private = 0 0.000 0.010 0.879 0.631 0.670 0.000 0.024p: Public = Placebo 0.000 0.363 0.973 0.730 0.549 0.094 0.095p: Public = Placebo = 0 0.000 0.001 0.997 0.556 0.664 0.000 0.030
All regressions include randomization block fixed effects and baseline values of the outcome; education; all cognitiveand noncognitive assessment results; beliefs about all cognitive assessment results; age; gender; and employmentstatus. Heteroskedasticity-robust standard errors clustered by treatment day are shown in parentheses. Lower rowsof the tables show p- and q-values for linear combinations of coefficients. The q-values adjust for multiple testingover all outcomes in the same family following Benjamini et al. (2006). *; **; and *** denote significance at the 10;5; and 1 percent levels respectively.
We report the placebo treatment effects on indices in Table 18 and compare them to public
treatment effects. We find small and statistically insignificant effects on beliefs about skill, beliefs
about earnings, search effort, and search effectiveness, employment status, and employment quality.
We can reject equality with the public treatment effect for the first and last two indices. The only
treatment effect that exceeds 0.05 standard deviations is on beliefs about returns to search. This
is driven by a large increase in the number of applications workseekers plan to submit and a large
decrease in the length of time they expect to take to get a job. Neither effect is precisely estimated
or statistically significant at conventional levels. We conclude that there is little evidence to support
the mechanisms underlying the placebo report.
38
Figure 6: Sample Placebo Report
REPORT ON ASSESSMENT PROCESSname.. surname..ID No. id..
This report provides information on assessments conducted by Harambee Youth Employment Accelerator(harambee.co.za), a South African organisation that connects employers looking for entry-level talent to young, high-potential work-seekers with a matric or equivalent. Harambee has conducted more than 1 million assessments and placedcandidates with over 250 top companies in retail, hospitality, financial services and other sectors. Assessments aredesigned by psychologists and predict candidates’ productivity and success in the workplace. This report was designedand funded in collaboration with the World Bank. You can find more information about this report, the assessments andcontact details at www.assessmentreport.info. «name» was assessed at Harambee on «date».
«name» completed assessments on English Communication (listening, reading, comprehension), Numeracy, and ConceptFormation:
1. The Numeracy tests measure candidates’ ability to apply numerical concepts at a National Qualifications Framework (NQF) level, such as working with fractions, ratios, money, percentages and units, and performing calculations with time and area. This score is an average of two numeracy tests the candidate completed.
2. The Communication test measures a candidate's grasp of the English language through listening, reading andcomprehension. It assesses at an NQF level, for example measuring the ability to recognise and recall literal and nonliteral text.
3. The Concept Formation Test is a nonverbal measure that evaluates candidates’ ability to understand and solveproblems. Those with high scores are generally able to solve complex problems, while lower scores indicate an abilityto solve less complex problems.
«name» also completed tasks and questionnaires to assess their soft skills:
4. The Planning Ability Test measures how candidates plan their actions in multi-step problems. Candidates with high scores generally plan one or more steps ahead in solving complex problems.
5. The Focus Test assesses a candidate’s ability to distinguish relevant from irrelevant information in potentiallyconfusing environments. Candidates with high scores are generally able to focus on tasks in distracting surroundings,while candidates with lower scores are more easily distracted by irrelevant information.
6. The Grit Scale measures whether candidates show determination when working on challenging problems. Those with high scores generally spend more time working on challenging problems, while those with low scores choose to pursue different problems.
DISCLAIMER: This is a confidential assessment report for use by the person specified above. The information in the report should only be disclosed on a “need to know basis” with the prior understanding of the candidate. Harambee cannot accept responsibility fordecisions made based on the information contained in this report and cannot be held liable for the consequences of those decisions.
Notes: This figure shows an example of the reports given to workseekers in the placebo treatment arm. The reports contain noassessment results, the workseeker’s name and national identity number, and the logo of the World Bank and the implementingagency. Each work seeker received 20 of these reports and guidelines on how to request more reports.
39