Evaluating Active Labor Market Policies using a Spatial ...conference.iza.org/conference_files/empohe2016/vom berge_p23723.pdf · Evaluating Active Labor Market Policies using a Spatial

Evaluating Active Labor Market Policies using a

Spatial Regression Discontinuity Design∗

Daniel Heuermann† Johannes F Schmieder§ Philipp vom Berge‡

University of Regensburg Boston University, Institute for Employment

NBER, IZA Research (IAB)

PRELIMINARY- Please do not cite or circulate-

December 2015

Abstract

We evaluate the e�ectiveness of active labor market programs (ALMP) for long-term un-employed workers using a spatial regression discontinuity design and a large georeferenceddata set on workers and unemployed persons in Germany. This approach allows for preciseestimation of the e�ects of di�erent policies, such as public employment programs, trainingprograms, wage subsidies, or coaching, on subsequent employment prospects of job seekers.Unemployed individuals who exhaust their regular unemployment bene�ts (UB1) are eligibleto a second tier of bene�ts (UB2) administered by local job centers which provide job searchsupport in the form of ALMPs. We exploit the fact that individuals are assigned strictlygeographically to these job centers to estimate the e�ect of these policies using a regressiondiscontinuity design at the county borders, where policies change discontinuously while theconditions of regional labor markets vary smoothly in space. The structure of the data allowsus to disentangle short- and long-term e�ects and, hence, to control for potential lock-in e�ects,as well as to shed light on dynamic employment e�ects over longer periods. Our preliminaryresults, based on outcomes over a 24 month horizon, indicate that, at least in the short termparticipating in ALMPs does not raise the number of days employed and if anything leads toa slight decrease.

∗We would like to thank Stefan Bender, Kevin Lang, Jeremy Magruder, Joachim Möller, Daniele Paser-

man, and seminar participants at Boston University, University of Regensburg, the IAB and the University of

Goettingen for helpful comments. All errors are our own.† [email protected]§ [email protected]§

‡ [email protected]

1 Introduction

Persistently high unemployment rates after the �nancial crisis in 2008 have led to renewed

interest in policies to support the unemployed. Extensions of unemployment bene�ts have

played a major role as a strategy, in particular in the US and in several Southern European

countries (see Schmieder and von Wachter, 2015, for a recent survey). These policies are often

complemented by a paralled expansion of active labor market programs (ALMPs), which cover

all public measures aimed at actively raising the re-employment chances of the unemployed.1

Today, all OECD countries implement a di�erentiated set of instruments and programs. These

can broadly be classi�ed into job placement services, training programs, public employment

schemes, and wage subsidies. The objective of these measures is to reduce skill mismatch,

avoid skill depreciation, and lower the entry barriers into the labor market for the unemployed.

On the downside, these programs are very expensive. On average, the OECD countries

spend 0.6 per cent of their GDP on ALMPs (OECD 2013). In Germany, direct spending

on ALMPs for long-term unemployed persons amounts to 3.5 billion Euro annually (BIAJ,

2013). When additionally taking into account administrative costs for sta� and rent, this

number rises to 8 billion Euro, equalling one per cent of overall public expenditure. Given

these �nancial dimensions, ALMPs exert substantial pressure on public households. It is,

hence, of vital interest for societies and policy makers to assess whether they achieve their

main objective of raising re-employment prospects of the unemployed. Given limited public

budgets and the resulting need to design cost-e�ective sets of instruments and program, it

is of particular importance to understand their relative e�ectiveness with regard to di�erent

target groups.

Fuelled by a rising availability of suitable data sets, the recent expansion of ALMPs has led

to a surge of studies investigating their e�ectiveness. In their survey of the literature, Card et

al. (2015) identify 154 microeconometric evaluations authored between 2007 and 2014, which

add to the 97 studies originating from the time before 2007 (see Card et al., 2010). The results

1See Pavoni and Violante (2007) for a theoretical analysis of how such programs may interact with unem-ployment bene�ts.

1

from their meta-analysis show that the average impacts of ALMPs are close to zero in the

short run, but rise over longer intervals and that the size of the e�ects varies by program and

target group.

Methodologically, the key problem in any evaluation of ALMPs lies in the non-random

selection of persons into treatment. This problem has been approached in the literature by

numerous identi�cation strategies, including randmized trials. However, two key challenges

for the identi�cation of treatment e�ects from ALMPs, that have not been properly addressed

so far, arise from the potential existence of threat and spillover e�ects. Threat e�ects refer

to a situation where the mere potential of being obligatorily sent to an ALMP alters the

employment probability of unemployment by changing their job search intensity. Spillovers

from ALMPs occur if the geographic proximity to ALMPs participants changes the level of

information and/or motivation of unemployed and, as a result, their job search intensity. The

uniting feature of both e�ects is that the impact of ALMPs may well extend to persons who

are not actively taking part in any policy or program. In this case, any identi�cation strategy

that compares program participants to non-participants is likely to yield bias results.

In this paper we propose a novel identi�cation approach based in a spatial regression

discontinuity design, which is capable of controlling for threat e�ects and spillovers when

addressing the e�ectiveness of ALMPs. Drawing on a large, georeferenced sample of unem-

ployed persons in Germany, we exploit the fact that treatment intensity varies discontinuously

in space due to the substantial degrees of freedom that job centers have when deciding on the

set of instruments and policies they use. As a result of this autonomy, unemployed persons

belonging to di�erent job center districts have di�erent ex-ante probabilities of participat-

ing in ALMPs. Given that no other variables change discontinuously at job center borders,

the key idea of our identi�cation approach is di�erence in employment probabilities between

workers close to the borders can plausibly only arise from di�erences in the sets of policies

implemented by boch job centers. The key identifying assumption underlying this approach

is that all other variables that determine employment chances behave smoothly at the border.

Below, we provide evidence that this assumption is satis�ed.

2

We employ the spatial RD approach to separately evaluate the e�ectiveness of training

measures, wage subsidies, public employment schemes, individual counselling, and mobility

support. In order to take account for potential lock-in e�ects and to measure medium-term

e�ects, we provide evidence over the course of two years. Overall, our results are sobering. For

all policies together, we �nd evidence for a signi�cantly negative e�ect of program participation

on employment probabilities. This negative e�ect rises in size over the course of 24 months

after participation. At the same time, participation in ALMPs signi�cantly raises the number

of days individuals continue to receive bene�t payments over the next two years.

The paper is structured as follows. The next section explains the institutional background

and describes the data. In section three we derive the spatial regression discontinuity design.

Section four contains preliminary results based on the spatial regression discontinuity design

and section �ve concludes.

2 Institutional Background, Data and Sample

2.1 Institutional Background

The implementation of ALMPs in Germany is organized in a two-tier system, which follows the

systematic split into a regular unemployment insurance (UI) branch, also called unemployment

bene�ts 1 (UB1) and second tier called unemployment bene�ts 2 (UB2), which is a means

tested form of bene�ts with features both of classical UI systems and welfare systems. Since

a series of labor market reforms in the early 2000s ('Hartz reforms'), legal competence for the

second tier rests with the so called job centers. Active and passive labor market policies in the

UI branch are implemented by the local employment agencies, which are organized in larger

geographic units. These cover, on average, three job center districts.

In this paper we are concerned with the e�ectiveness of ALPMs with respect to UB2

recipients. For this group, 408 job centers in Germany are responsible for implementing

both passive and active labor market policies. Job centers were created in 2005 with the

Hartz-reforms and are legally independent public corporations. They are provided with global

budgets and, as a result, have substantial degrees of freedom of how to actively support UB2

3

recipients. While the legal instruments the job centers can use are de�ned in the Social Security

Codes II and III, no pre-de�ned budget exists for the use of speci�c instruments. In addition,

according to �16f in the Social Security Code II, job centers are legally allowed to spent part

of their budget freely, i.e., on any type of support for unemployed they deem helpful. Finally,

job centers have access to various federally funded program, which further increase their room

for maneuvre. Job center policies are in most cases steered by the 'Traegerversammlung',

which is e�ectively a governing board consisting of members of the Local Labor Agency and

the county administration. This board de�nes the overall strategy and the priorities to be

set within each �scal year. Given the �exible budgets, case workers can decide freely and on

a case-by-case basis which instruments to be implemented for each person, according to her

personal needs.

Three features of this overall setting are of key importance for our identi�cation approach.

First, the borders of the job center districts in almost all cases coincide with the borders

of the 402 counties ('Kreise und kreisfreie Städte') in Germany. The counties themselves

have no local authority on labor market or economic policies, but are mainly responsible for

administrative issues. In addition, while in the United States access to schools is an important

local amenity that leads to sorting along school district borders, in Germany education is

funded and administered through the German states. There are di�erences across states in

terms of educational quality, but there is very little variation across counties within states.

Furthermore there are no local taxes that are county speci�c which might lead to sorting

around the borders.

The second important institutional feature is that due to the degrees of freedom job centers

are endowed with, the resources available for ALMP policies and the relative emphasis on

di�erent types of ALMPs vary substantially at the county borders. And �nally the third

feature is that, being e�ectively NUTS III regions with an average population of 200,000

persons, job center districts are rather small. Especially in densely populated areas their

borders cut through local labor markets and commuting zones. As a result of these three

features, the only variable that changes discontinuosly is the intensity of ALMPs and the

4

particular mix of the set of instruments. In contrast, close to county borders all individual

and labor market variables can be expected to evolve smoothly over space. We explain how

we exploit this discontinuity in our spatial RD in more detail in Section 3.

2.2 Employment and County Level Data

The �rst data source we draw on are social security data provided by the German Federal

Employment Agency. The data contains daily information on employment and unemployment

histories on individual level. The employment data is collected from employment noti�cations

to the social security administration, which are required by law for all jobs liable to social

security contributions. These cover roughly 85 percent of employment in Germany. Self-

employed and public servants are not contained in the data. The data on unemployment are

taken from the German unemployment insurance system. It contains information on bene�t

receipt, registeration as unemployed, and participation in ALMPs. The full data set covers

daily information on employment status, wages, bene�t receipt, participation in ALMPs, as

well as numerous covariates on individual level including precise information on home and

employer addresses. From these data, we have generated summary statistics to gain �rst

insights into the relative importance of di�erent types of instruments and policies. Table 1

contains numbers of participants in the �ve most frequent types of measurs. During the period

between 2006 to 2008, 38 per cent of all participants in ALMPs were taking part in measures

of public employment, while 31 per cent have undergone training measures. The third row

in the table shows that 16 per cent of participants were supported under the free funding

opportunity contained in �16f (SGB II). Finally, 11 per cent of participants were placed into

public employment schemes, while with close to two per cent, wage subsidies play only a minor

role.

We complement these information taken from individual level data by aggregate measures

on ALMP activities for each job center. The �rst data source we use encompasses annual

information on the activities of each job center, which is provided by the statistical branch of

the Federal Employment Agency on its website. Important in the present context, this data

5

contains information on the absolute and relative numbers of individuals receiving ALG II,

as well as on ALMP expenditure by job center from 2005 on. In addition to this data, we

have obtained the annual budgets provided to each job center. These budgets are listed in

a reply of the German government to an inquiry from the leftist party 'Die Linke', which is

publicly available in the collection of electronic documents in the online library of the German

parliament (Bundesregierung 2011). In Table 2, we have merged this budgetary information

for the years 2005 to 2008 with information on unemployment by job center region. The upper

part of the table contains the budget available for each unemployed person for integration

measures ('Eingliederungstitel') like training measures, public employment schemes, and wage

subsidies. The lower part of the table shows the annual sums spent for administrative costs

('Verwaltungskosten'), which mainly cover sta� costs and rent.

The table yields two notable facts. First, it shows that the job centers di�er pronouncedly

with respect to the budgets available for each unemployed. In 2005, the smallest amount

available for each unemployed was 221 Euro (Job Center Munich Land), while the largest

amount was 2,823 Euro (Job Center Landau). Secondly, with a rise from 583 to 1,055 Euro,

the sum of integration funds per unemployed has more than doubled over the period of four

years. During the same period, administrative funds have risen from 6,7 to 8,5 billion Euros.

The more procounced growth of integration budgets in contrast to administrative budgets

provides further evidence for the notion that job centers have been able to diversify their

policies over time.

2.3 Sample of UB2 Recipients

From the individual data set we take all individuals that have received bene�ts from UB2

for at least one day between 2005 and 2013. Given the German UI system, an UB2 entrant

may either come from receiving UB1, after she has exhausted her bene�ts, or directly from

employment, e.g., if she did not qualify for UB1 due to a very short employment spell. An

individual may also enter UB2 after any other spell of non-participation in the labor market

(e.g. education, staying at home, living abroad, etc).

6

For this sample we have obtained the exact home addresses as of June 30th for each year,

either as they are recorded in the employment data or in the unemployment data. Using

GIS software we have geocoded all addresses and translated them into exact latitude and

longitude coordinates. Using these geographic coordinates, we have determined the closest

county border for each all individuals and have computed the shortest distance to this border

in kilometers. In addition, we have generated concentric circles around each person and

calculated the average attributes of all bene�t recipients within these circles. Important for

us, beyond demographic characteristics, these attributes encompass information on average

participation of each person's individual reference group in measure of ALMP.

3 Methods

3.1 Distance-to-Border Regression Discontinuity Design

We are interested in estimating the e�ect of active labor market programs on individual

employment outcomes. For every individual i, let c indicate the county of residence and n the

closest neighboring county. We denote a border between the two counties as b, and a subscript

b indicates the closest county border for that observation. Let the Tic indicate a treatment

such as average expenditure on active labor market programs, or expected days spend in active

labor market programs. To the extent that job centers treat all employees in their respective

county identically, this would only vary on the county level, however if job center policies vary

over space there may well be intra county variation. Consider the following model how ALMP

treatment may a�ect employment outcomes:

yic = βTic +Xicπ + εic (1)

Since Tic is likely correlated with the state of the labor market in county c, as well as with

other county characteristics, estimating this equation using OLS likely leads to signi�cant

omitted variable bias.

To overcome the omitted variable bias we employ a spatial regression discontinuity design,

7

where we use the discontinuity at the boundary as an instrument for the treatment variable.

Let distib be the distance from the closest border. For each possible treatment variable, we

multiply this distance with −1 if individual i is in a county that has a lower value of the

treatment Tic then the county on the other side of the border, so that:

Distib =

−distib if Tic < Tin

distib if Tic ≥ Tin

We then estimate the following equation:

yic = α + δD(Distib ≥ 0) + f(Distib) + θb + εic (2)

Since we have many borders and thus individuals with Distib ≥ 0 may be far away from

individuals with Distib < 0, we also include border �xed e�ects θb so that the identi�cation

comes only from individuals living close to either side of a border. The coe�cient δ captures

how the outcome variable varies at the borders. Estimating equation (2) using the treatment

variables Tic as an outcome as well as employment outcomes, yields a Wald estimator of the

treatment e�ects in equation (1), by dividing the reduced form by the �rst stage coe�cient.

3.2 Spatial Fixed E�ects Estimator

The distance to border design yields consistent estimates of the causal e�ect of ALMP, as

long as individuals are smoothly distributed throughout space, so that at each point close to

a border we get a similar number of individuals just on the other side of the border. This may

be violated in practice, since border usually run in between towns and villages, not through

towns, so that many individuals who live close to the border may not have a direct counterpart

on the other side of the border who is actually nearby. We get around this by using the Spatial

Fixed E�ects Estimator used by Magruder (2012), which in turn is based on Conley and Udry

(2010) and Goldstein and Udry (2008).

Consider the data generating process:

8

yic = βTic + εic (3)

The problem is that E[εic|Tic] 6= 0, since treatment is correlated with many other factors

that vary through space. However, suppose that we can write:

εi = f(lati, loni) + ui

where E[ui|lat, lon] = 0 and f(.) is a continuous function in space (that is of longitude lon

and latitude lat). Let R(i) be the set of all individuals within a radius of r of individual i.

Consider all individuals i′ in a radius r. By continuity of f() we get that

limr→0

1

nR(i)

∑i′∈R(i)

f(lati′ , loni′) = f(lati, loni)

This yields

limr→0

yic − 1

nR(i)

∑i′∈R(i)

yi′

= limr→0

βTic − 1

nR(i)

∑i′∈R(i)

Ti′c′

+ εic −1

nR(i)

∑i′∈R(i)

εi′

= β lim

r→0

Tic − 1

nR(i)

∑i′∈R(i)

Ti′c′

+ limr→0

uic − 1

nR(i)

∑i′∈R(i)

ui′

Let Ti = limr→0

{[Tic − 1

nR(i)

∑i′∈R(i) Ti′c′

]}and yi = limr→0

{yic − 1

nR(i)

∑i′∈R(i) yi′

}and

we get that the estimator

β =Cov(yi, T )

V ar(T )

is a consistent estimator for β, since limr→0

{uic − 1

nR(i)

∑i′∈R(i) ui′

}= 0. This suggests

a straightforward estimator, we simply transform the treatment and outcome variables of

interest, by subtracting for each person the average of treatment / outcome variable in a radius

r from the treatment / outcome of that person. Once we have the transformed variables Ti

and yi, we regress the transformed outcome variable on the transformed treatment variable

9

to obtain an unbiased estimate of β.

The intuition behind this transformation is straightforward. Since factors εic in�uencing

the outcome apart from the treatment variable vary smoothly through space, subtracting

the average of individuals in a small neighborhood will di�erence out the in�uence of these

factors, but also di�erences in the treatment. Only for individuals at the border where Ti

varies discontinuously is not all the variation di�erenced out and thus the observations close

to the cuto� provide the identi�cation.

While we think the spatial FE estimator is preferred to the border distance RD, the latter

has been used in the past and is very intuitive. We will present results for both estimators

below.

3.3 Dynamic Regression Discontinuity Design

The previous literature has suggested that the time horizon is important for evaluating the

e�ects of ALMPs, since in the short run lock-in e�ects can dominate any positive employment

e�ects. To develop a deeper picture of the dynamic e�ects, we estimate how the treatments and

outcomes of UB2 recipients vary at the county border using a dynamic regression discontinuity

design:

yict = αt + δtD(Distib ≥ 0) + ft(Distib) + εict for t = −12, ..., 24 (4)

where the subscript t indexes the month since entry into UB2. We estimate this equation

separately for each month relative to the entry into UB2. The estimates for αt trace out

how the dependent variable changes over the months before and after entry into UB2 for

individuals just next to the border on the side with less ALMPs, while the estimate of δt

provide an estimate of how much the pro�le shifts, relative to individuals on the other side.

4 Results

In this section we present results based on a preliminary dataset of a 25 percent sample of

the IEB, which covers the years up until 2009. We will focus on individuals who entered UB2

10

during 2006 and 2007, which allows us to follow them up to 2 years after entry into UB2.

This is an important caveat, since the literature has found that active labor market programs

often lead to a lock-in e�ect initially where individuals stay unemployed longer, since they

are in a program and therefore not actively looking for a job. The positive e�ects may then

take some time to materialize. We have now obtained a 100 percent version of the IEB data

covering the time period until 2013, which will allow us to follow UB entrants from 2006 to

2009 for at least 4 years or even longer (for the earlier entrants) and will make it possible to

analyze lock-in vs. long-term e�ects in detail. We are currently working on cleaning this data

and geocoding the addresses for all years.

Another caveat is that in our current data we only have geocoded locations for 2008.

We currently extrapolate these locations to the earlier years. This will undoubtedly introduce

measurement error. In principle since we are looking at UB2 entrants from years prior to 2008,

this could also lead to more systematic biases since the location in 2008 itself is endogeneous.

It seems somewhat unlikely to us that individuals would move (especially while unemployed)

in response to ALMP policies. When we restricted the data to individuals that did not move

between counties from 2006 to 2008 (county of residence is available in all years), this did

not change our results. In any case, in our new dataset we will have addresses for all years,

allowing us to sidestep this issue.

We next explore the validity of the border discontinuity design, show the variation of

ALMP around the borders, and then present estimates based on the Distance-to-Border RD

design and the Spatial FE Design.

4.1 Validity of RD

Using our preliminary data, we �rst calculate for each county and each year the average time

individuals who have entered UB2 in that year spend in an ALMP. We then calculated for each

individual who entered UB2 during 2006 and 2007 the distance to the closest county border.

If ALMP in the county where the UB2 recipient lived is smaller than in the next closest

county, we multiplied this distance with minus one. We restrict the data to individuals who

11

live within 10 kilometers of an inner-German county border. We drop individuals where the

closest border is the one to neighboring country (e.g., France, Austria, Poland). Furthermore,

we drop all borders between counties where the two neighboring counties are not in the same

state in order to avoid confounding e�ects stemming from di�erences in state level policies (in

practice this restriction makes no di�erence).

Figure 2 shows the density of observations around the county borders. Each dot consists

of all individuals living in a one kilometer bin of distance to the border. A �rst noteworthy

feature is that the density increases towards the border from both sides. This is due to a

simple geometric feature: For example consider a county that is a perfect circle with a 10

km radius. The area of a band that is between 0 and 1 km from the border is given as:

A2 = π(10km)2 − π(9km)2 = 59.7km2, which is more than twice as large as for example a

band that is between 5 and 4 km: A2 = π(5km)2−π(4km)2 = 28.3km2. Furthermore counties

that are small enough can not have anyone living for example 10 km from the closest border.

A second noteworthy feature is that the density increases up to the last kilometer from the

border and then dips for the last bin. This is because in practice most county borders do not

run through cities or towns, but in between them. While they are often very close, this means

that fewer people live right next to a border than for example 1 kilometer away from it. And

�nally a third feature of this plot is that it is asymmetric, with somewhat more mass to the

right, especially in the 2 to 5 km distance range. The reason for this excess mass appears

to be that several large cities in Germany, such as Munich or Hamburg, are also their own

counties. Since these cities often have higher spending on ALMPs per unemployed, they tend

to end up on the right of the cuto� in these graphs and due to their large populations create

the hump on the right. While the shape looks unusual for an RD density plot, there is no

obvious discontinuity in the density at the cuto�.

An important test for the plausibility of the identi�cation assumptions in the RD design is

that predetermined characteristics evolve smoothly at the cuto�. Figure 3 shows how average

characteristics of UB2 entrants in our sample evolve around the border. Panel (a) shows the

average years of schooling of the UB2 recipients. Interestingly, education declines towards the

12

border, which likely re�ects the fact that the borders tend to be in more rural areas, while

individuals living further away from the county border are more likely to live in cities. The

shape is reversed for experience, which seems to be driven by the age of the UB2 entrants

(not shown), where individuals further away (who are likely more urban) are younger. The

share of women among the entrants is essentially �at throughout space (notice the scale) and

individuals close to the border have spend somewhat more time in regular UI bene�ts prior

to entering UB2, also likely due to their higher experience and therefore longer eligibility for

bene�ts. Importantly, all characteristics evolve very smoothly around the cuto� and do not

suggest that there is any sorting at the cuto�s but instead that individuals on both sides are

quite comparable to each other.

4.2 Estimates based on Distance-to-Border RD Design

We now turn to estimates of the e�ect of ALMPs using the Distance-to-Border RD Design

outlined in section 3. As a �rst step, we document that ALMP does indeed increase signif-

icantly at county borders. Figure 4 (a) shows the average number a new entrant into UB2

spends in ALMP over the subsequent 2 year period. As can be seen, to the left of the border,

UB2 entrants spend about 46 days in various active labor market programs. Right at the

border there is a sharp jump to around 62 days, suggesting that individuals who live just on

the other side of the border spend on average around 18 days (or close to 50 percent) more

in ALMPs than individuals who are entering UB2 in counties with more restrictive ALMP

policies than their neighbors. Figure 4 (b) shows the number of days individuals receive UB1

bene�ts over the next 2 years. Since UB2 recipients are typically not eligible for UB1 bene�ts

when they enter UB2 and not very likely to requalify for UB1 over the relatively short period

of 2 years, it is not surprising that the level of UB1 receipt in this �gure is low and that there

is no discontinuity at the border.2 By contrast, panel (c) shows the number of days spend

in UB2 over the next 2 years. Quite strikingly, time spent in UB2 increases discontinuously

at the border by almost 10 days, suggesting that more intense ALMP policies by job centers

2Individuals who had very low earnings prior to becoming unemployed may be eligible for UB1, but sincetheir UB1 bene�ts are so low they may also qualify for UB2 bene�ts.

13

keep UB2 recipients longer in UB2. This is consistent with panel (d), which shows that the

number of days individuals are employed decreases by around 9-10 days over the next two

years. While this could re�ect the lock-in e�ect that individuals who are put into ALMP are

by de�nition not working and thus kept back from the labor market, these results suggest that

at least over a 2 year horizon a higher intensity of ALMPs does not improve labor market

outcomes, despite their relatively large costs. This could of course be di�erent over longer

time periods and could mask substantial heterogeneity in e�ectiveness of di�erent individual

ALMP policies, something which we address below.

4.3 Estimates based on Spatial FE Design

The Distance-to-Border RD Design is intuitive and allows for a straightforward graphical

representation, however it relies crucially on the assumption that the density of UB2 entrants

is continuous through space and in particular around the county borders. We know however

that except for some larger cities, county borders typically do not cut directly through towns

and cities, but instead may pass close to them. To see that this could be problematic for

the RD design, consider a single border with a city on each side of the border. Both cities

may be exactly one kilometer from the border, but they may be at di�erent points along the

border and in practice not very close to each other. The Distance-to-Border RD would put

both cities at the one kilometer distance from the border and use them for comparison, even

though the actual distance may be quite large and they man not actually be in the same labor

market or easy to commute between.

The spatial FE estimator on the other hand does not su�er from this shortcoming. Given

a radius r for the spatial FE transformation, each individual is only compared to other indi-

viduals who are at most r kilometers away from them. By choosing a smaller and smaller r

we are getting arbitrarily close to the ideal of only comparing individuals who live right next

to each other but on di�erent sides of a county border.

We therefore apply the spatial FE transformation to our data for di�erent radii, 10km,

5km and 2km, and present both graphical evidence as well as regression estimates. For the

14

graphical evidence, we show the average values of the transformed variables in bins of distance

to the closest border as before. This highlights how the transformation works and shows how

only observations close to the border contribute to identi�cation. Figure 5 shows the average

days spent in ALMP by distance to border after the spatial FE transformation with a 10km

radius. Given the 10km radius, the outer most points should be close to zero, except for

measurement error. This suggests that with a 10km radius speci�cation these points do not

contribute to identi�cation in the spatial FE regressions. As we approach the border from

the left, the average days spent in ALMP of individuals falls relative to the average of all

individuals in a 10km radius. This is because the average increasingly contains individuals

on the other side of the border where they spend more time in ALMP. At the border the

di�erence is almost 10 days. On the other side we see a reverse pattern, although not quite

symmetric. Just on the right of the border average days in ALMP are about 5 days more

than in a 10 km radius. In fact this asymmetry suggests a violation of the smooth density

assumption that would be required for the Distance-to-Border RD design, but that does not

constitute a problem for the spatial FE design. Interestingly, we still see a jump of around 15

days at the border. The other 3 panels show similar patterns consistent with Figure 4 before.

Again, there is no jump and almost no variation for UB1 receipt and about an 8 day jump

for time spent in UB2 and employment.

Table 3 shows spatial FE estimates, that is estimates of the equation

yic = βTic + εic (5)

where yic is an outcome variable with the spatial FE transformation de�ned in section 3.2

and Tic is the transformed treatment variable. As a treatment variable we use the average

time a UB2 recipient in a given county spends in ALMP. Since days spent in ALMP may vary

throughout the county, we �rst show in column 1 the e�ect of average days in ALMP in a

county on the actual change in ALMP at the county border spent in ALMP. The top panel

shows the estimates for a 2 km radius, the middle panel for a 5 km radius and the bottom

one for a 10 km radius. The coe�cient in the top Panel of 0.81 implies that at the border

15

individuals who are in a county that sends on average UB2 recipients for one additional day

into ALMP spend 0.81 days extra in ALMP. The coe�cient is relatively precisely estimated,

though note that we cannot reject a coe�cient of 1. For larger radii the precision increases

and the coe�cient is very close to 1.

Column (2) of Table 3 shows the e�ect of being in a county with an additional day of

ALMP intensity on days receiving UB2. The result con�rms the previous graphical evidence,

that additional time in ALMP increases time spent in UB 2. In particular, one additional day

spent in ALMP increases time in UB 2 by about 0.4 to 0.5 days. Similarly column (3) shows

that days spent in employment decline for each day in ALMP by about 0.4 to 0.5 days, with

relatively precisely estimated point estimates. The coe�cients on UB1 are in magnitude close

to zero and except for the 10 km radius not statistically signi�cant.

A particularly attractive feature of the border discontinuity design is that, unlike in most

other RD designs, there are a very large number of discontinuities, namely at each between

county border. Since the mix of di�erent ALMPs varies across job centers, each border

constitutes a slightly di�erent experiment since the mix of ALMPs changes di�erently at each

one. This allows us to separately identify the e�ects of di�erent ALMPs, which would not

be possible with a single experiment or a single border. For that purpose we estimate the

following equation:

yic = β1TEducProgic + β2T

EmpProgic + β3T

OtherALMPic + β4T

PSAic + εic (6)

where we allow for di�erential e�ects of 4 major groups of ALMP policies.

Table 3 shows estimates of equation (6). It is clear that with a 2km radius we lack precision.

With a 5km radius, we �nd that education, employment and other ALMPs increase time

in UB2, while quite strikingly time assigned to PSAs, privat job search agencies, appears

to reduce time in UB2 almost one for one, though with a relatively large standard error.

Interestingly, and warranting caution, this positive e�ect of PSA on UB2 recipiency does not

seem to translate into more employment and disappears when going to the 10 km radius. The

most robust result on employment appears to be that the other ALMPs have a negative e�ect

16

on employment in all three speci�cations. The closest to a positive employment e�ect seems

to be associated with the actual employment programs. Given the nature of these programs,

where the unemployed are directly placed into speci�cally created jobs, a positive e�ect would

be unsurprising and, in fact, expected. A coe�cient of less than 1 would still suggest that

the employment program is crowding out regular, i.e. unsubsidized, forms of employment.

These speci�cations are a bit too imprecise to draw strong conclusions, but we are hopeful

that these will be more informative once we estimate these with our much larger and more

comprehensive dataset that should provide quite a bit of extra precision.

4.4 Dynamic Estimates

We now turn to estimates of the dynamics of ALMP assignment and subsequent labor market

outcomes. For this purpose we estimate equation (4) for each month from 12 months before

UB2 entry to 24 months afterwards and plot the resulting estimate for the individuals to the

left and to the right of the county border. Figure 6 shows these dynamic results for some of

the main outcomes. Panel (a) shows days spent in ALMP. Since many UB2 entrants received

UB1 before (and exhausted UB1), many were also in ALMP prior to UB2 entry. However, the

levels were essentially identical, giving us further con�dence in the border RD design since it

shows that there are no di�erences in terms of UB1 policies that could confound our results.

After UB2 entry the gap opens up and individuals in the high intensity ALMP counties spend

about 50 percent more time in ALMPs than in the low intensity counties just on the other side

of the border. Interestingly the di�erence appears immediately and remains pretty constant

from around 6 months onwards and it does not appear that it is concentrated at a particular

time of the UB2 spell. Panel (b) shows a similar �gure for education programs again.

In Panel (c) we look at days employed as an outcome variable. Prior to entering UB2,

individuals spent about 20 days per month in employment (or more plausibly about 2/3 of

individuals were employed), which drops in the months before entering UB2. Again reassur-

ingly the pre-trends and levels are almost identical and furthermore over the �rst 7-8 months

there is no di�erence after entering UB2. Interestingly there thus does not appear to be a

17

lock-in e�ect early on. After about 8 months there is a small gap with the group on the

low ALMP intensity side of the border returning quicker to work with about 0.5 additional

days per month in employment. This di�erence remains constant throughout the 24 months

window. While this is still consistent with lock-in e�ects, since ALMP levels are also higher

throughout the entire 24 months period, there is no sign of a reversal towards the end that

would point towards positive long-run e�ects of the policies.

Panel (d) of Figure 6 shows that the negative employment e�ects of ALMP also translate

into lower earnings of around 10 to 15 Euro per month from month 10 onwards, with no sign

of a reversal towards the end.

5 Conclusion

Active labor market programs are a major policy tool with many governments and UI agencies

putting high hopes in these programs to help job seekers �nd back into work faster. The spatial

regression discontinuity design proposed in this paper provides a powerful method to identify

the causal e�ects of such ALMP programs on labor market outcomes.

While the results presented here are preliminary and investigate only short to medium

run impacts, they appear somewhat sobering at this stage. At least over a 2 year horizon,

individuals who are assigned to job centers that invest signi�cantly higher resources into a

variety of ALMP policies do not appear to bene�t from these programs but rather spend more

time on unemployment bene�ts 2, spend less time in employment and have lower monthly

earnings.

In currently ongoing work we are expanding our sample (from a 25 percent sample to the

100 percent) and time frame (adding 4 more years, allowing for a longer time horizon). Fur-

thermore, we will have better measures for ALMP participation as well as local expenditures

on ALMP.

18

References

BIAJ, �Jobcenter 2013: Eingliederungsmittel und Verwaltungsmittel,� Technical Report, Bre-mer Institut für Arbeitsmarktforschung und Jugendberufshilfe (2013).

Bundesregierung, �Antwort der Bundesregoerung auf die kleine Anfrage der Fraktion DIELINKE,� Bundestagsdrucksache 17/5304, Bundesregierung (2011).

Card, David, Jochen Kluve, and Andrea Weber, �Active Labour Market Policy Evaluations:A Meta-Analysis,� The Economic Journal, 120 (548), (2010), F452�F477.

, , and , �What Works? A Meta Analysis of Recent Active Labor Market ProgramEvaluations,� Working Paper 21431, National Bureau of Economic Research July (2015).

Conley, Timothy G and Christopher R Udry, �Learning about a new technology: Pineapplein Ghana,� The American Economic Review, (2010), 35�69.

Goldstein, Markus and Christopher Udry, �The pro�ts of power: Land rights and agriculturalinvestment in Ghana,� Journal of Political Economy, 116 (6), (2008), 981�1022.

Magruder, Jeremy R., �High Unemployment Yet Few Small Firms: The Role of Central-ized Bargaining in South Africa,� American Economic Journal: Applied Economics, 4 (3),(2012), 138-66.

OECD, �Public expenditure on active labour market policies 2013/1.�

Pavoni, Nicola and G. L. Violante, �Optimal Welfare-to-Work Programs,� Review of Economic

Studies, 74 (1), (2007), 283-318.

Schmieder, Johannes F and Till von Wachter, �E�ects of Unemployment Insurance: NewEvidence and Interpretation,� mimeo, (2015).

19

Table 1: Share of ALMPs by Type

2006 2007 2008 All Years

Training 31.1 29.9 32.1 31.1Flexible Funding 16.1 20.4 18.1 18.2External Placement Service 11.4 8.2 13.9 11.1Public Employment 37.7 39.5 34.1 37.7Wage Subsidy 1.9 1.9 1.8 1.9

20

Table 2: Descriptive Statistics of Job Center Budgets

Year Mean SD Max Min

Integration funds per unemployed by jobcenter2005 1,061 583 221 2,8232006 1,586 751 417 4,7002007 1,852 852 722 4,9192008 2,388 1,055 594 6,884Administrative funds by job center (in Million Euro)2005 6,715 116,999 206,781 8142006 7,688 13,472 239,716 1,0762007 8,032 13,644 233,596 1,0802008 8,549 16,359 267,963 1,099

21

Table 3: RD Estimates of the E�ect of ALMP - Employment Outcomes - Treat-ment Mean ALMP - Controlling for Spatial FE

(1) (2) (3) (4)Days in Days receiving Days employed Days receiving

ALMP over UB 2 over over UB 1 overnext 2 years next 2 years next 2 years next 2 years

Spatial FE Radius 2 km

Average days in ALMP 0.81 0.51 -0.42 0.044[0.084]** [0.20]* [0.21]* [0.068]

Observations 101840 101840 101840 101840Mean of Dep. Var. 56.8 525.7 310.8 37.8


Average days in ALMP 0.97 0.45 -0.43 0.017[0.038]** [0.092]** [0.095]** [0.031]



Average days in ALMP 1.03 0.39 -0.47 0.048[0.022]** [0.055]** [0.057]** [0.018]**


Notes: Coe�cients from RD regressions. Local linear regressions (di�erent slopes) on eachside of cuto�. Standard errors clustered on day level (* P<.05, ** P<.01)).

22

Table 4: RD Estimates of the E�ect of ALMP - Employment Outcomes - Treat-ment Mean ALMP - Controlling for Spatial FE

(1) (2) (3) (4)Days in Days receiving Days employed Days receiving

ALMP over UB 2 over over UB 1 overnext 2 years next 2 years next 2 years next 2 years


Days in education program 0.93 1.81 -0.70 -0.053[0.34]** [0.83]* [0.87] [0.28]

Days in employment program 0.76 0.15 1.17 -0.13[0.26]** [0.64] [0.66] [0.22]

Days in other ALMP program 0.70 0.63 -1.03 0.16[0.16]** [0.40] [0.41]* [0.13]

Days in PSA 1.07 -0.90 -0.52 0.022[0.41]** [0.98] [1.02] [0.33]



Days in education program 1.05 0.79 -0.41 0.20[0.16]** [0.39]* [0.41] [0.13]

Days in employment program 1.08 0.77 0.10 -0.16[0.12]** [0.30]* [0.31] [0.10]

Days in other ALMP program 0.93 0.50 -0.65 0.057[0.076]** [0.18]** [0.19]** [0.062]

Days in PSA 0.75 -1.03 -0.48 -0.049[0.19]** [0.45]* [0.47] [0.15]



Days in education program 1.06 0.97 -0.81 0.15[0.099]** [0.24]** [0.25]** [0.081]

Days in employment program 1.13 0.33 -0.21 0.032[0.075]** [0.19] [0.19] [0.062]

Days in other ALMP program 0.97 0.35 -0.52 0.013[0.046]** [0.11]** [0.12]** [0.038]

Days in PSA 1.00 -0.22 -0.24 0.083[0.11]** [0.26] [0.27] [0.087]


Notes: Coe�cients from RD regressions. Local linear regressions (di�erent slopes) on each side ofcuto�. Standard errors clustered on day level (* P<.05, ** P<.01)).

23

Figure 1: Spending on ALMP per Recipient of Unemployment Bene�ts 2 (UB2) in2009

Notes: The maps shows the average spending on ALMP per UB2 recipient in 2009 on the countylevel. Spending is measured in Euro. Source: Statistics of the German Federal Employment Agency.

24

Figure 2: Density Around County Borders

010

000

2000

030

000

4000

050

000

Obs

erva

tions

−10 −5 0 5 10Distance from County Border in KM

Bandwidth: 1 KM, Treatment: mean_next_massnahme

Number SGBII Spells around County Border

Notes: Each bin contains the number of individuals within a 1 kilometer bandwidth of distancefrom the nearest county border. For each individual the distance is measured as the distance to thenearest county border. If the adjacent border has a higher average intensity of ALMP, the distanceis multiplied with -1, so that the observations to the left correspond to individuals who are on theside with a lower propensity to send the long term unemployed into active labor market programs.

25

Figure 3: Smoothness of Observable Characteristics Around County Borders

12.2

12.4

12.6

12.8

13E

duca

tion

year

s



Education years around County Border

(a) Average years of schooling

1313

.514

14.5

Pot

entia

l exp

erie

nce



Potential experience around County Border

(b) Average years of experience

.5.5

05.5

1F

emal

e



Female around County Border

(c) Share female

9510

010

511

011

512

0N

umbe

r of

day

s A

LG I

in p

revi

ous

3 ye

ars



Number of days ALG I in previous 3 years around County Border

(d) Average days receiving UI bene�ts in previous 3years

Notes: Each bin contains all individuals within a 1 kilometer bandwidth of distance from thenearest county border. Panel (a) shows the average years of schooling in each bin. Panel (b) theyears of experience observed in the data. Panel (c) the share of individuals who are female, andPanel (d) the number of days individuals have received regular unemployment bene�ts (UB1) inthe 3 years prior to entering unemployment bene�ts 2 (UB2). For each individual the distance ismeasured as the distance to the nearest county border. If the adjacent border has a higher averageintensity of ALMP, the distance is multiplied with -1, so that the observations to the left correspondto individuals who are on the side with a lower propensity to send the long term unemployed intoactive labor market programs.

26

Figure 4: Variation of Days spent in ALMP and Labor Market Outcomes aroundBorders

4050

6070

Num

ber

of d

ays

in p

rogr

am o

ver

next

2 y

ears



Number of days in program over next 2 years around County Border

(a) Days spent in ALMP

3436

3840

Num

ber

of d

ays

ALG

I ov

er n

ext 2

yea

rs



Number of days ALG I over next 2 years around County Border

(b) Days in UB1 over next 2 years

520

525

530

535

540

545

Num

ber

of d

ays

ALG

II o

ver

next

2 y

ears



Number of days ALG II over next 2 years around County Border

(c) Days receiving UB2 over next 2 years

295

300

305

310

315

320

Num

ber

of d

ays

empl

oyed

ove

r ne

xt 2

yea

rs



Number of days employed over next 2 years around County Border

(d) Days employed over next 2 years

Notes: Each bin contains all individuals within a 1 kilometer bandwidth of distance from the nearestcounty border. For each individual the distance is measured as the distance to the nearest countyborder. If the adjacent border has a higher average intensity of ALMP, the distance is multipliedwith -1, so that the observations to the left correspond to individuals who are on the side with alower propensity to send the long term unemployed into active labor market programs.

27

Figure 5: Variation of Days spent in ALMP and Labor Market Outcomes aroundBorders using Spatial FE Design

−10

−5

05

next

_mas

snah

me_

resi

d



next_massnahme_resid around County Border

(a) Average days spent in ALMP

−2

−1

01

2ne

xt_d

aysa

lg_r

esid



next_daysalg_resid around County Border

(b) Days in UB1 over next 2 years

−10

−5

05

10ne

xt_d

ayss

gb2_

resi

d



next_dayssgb2_resid around County Border

(c) Days receiving UB2 over next 2 years

−5

05

1015

next

_day

sem

p_re

sid



next_daysemp_resid around County Border

(d) Days employed over next 2 years

Notes: Each bin contains all individuals within a 1 kilometer bandwidth of distance from thenearest county border. For each individual the outcome variable is equal to the observed value forthat individual minus the average value of that variable of all individuals within a 10 km radius ofthat individual. For each individual the distance is measured as the distance to the nearest countyborder. If the adjacent border has a higher average intensity of ALMP, the distance is multipliedwith -1, so that the observations to the left correspond to individuals who are on the side with alower propensity to send the long term unemployed into active labor market programs.

28

Figure 6: Dynamic Estimates of the E�ects of ALMP Intensity on Labor MarketOutcomes - Border Distance RD Estimates

11

.52

2.5

33

.5

−12 −6 0 6 12 18 24Months since ALG II entry

Side of border with low ALMP intensity Side of border with high ALMP intensityNumber low ALMP: 170045, Number high ALMP: 233232

Days in any ALMP by Months since ALG II Entry

(a) Days spent in any ALMP

.4.6

.81



Days in Education Program by Months since ALG II Entry

(b) Days spent in Education ALMP

10

12

14

16

18

20



Days per month employed by Months since ALG II Entry

(c) Days employed

20

03

00

40

05

00

60

0



Total monthly earnings by Months since ALG II Entry

(d) Monthly Earnings

Notes: The �gures present estimates from a distributed lag model of how days spent in ALMP, em-ployment and monthly earnings vary at the borders between high and low ALMP counties. Verticalbars indicate statistical signi�cance.

29

Documents

Evaluating Active Labor Market Policies using a Spatial ...conference.iza.org/conference_files/empohe2016/vom berge_p23723.pdf · Evaluating Active Labor Market Policies using a Spatial