Active Labor Market Policies:

An Introduction into their Rationale and Evaluation

(with an emphasis on transition countries)

Hartmut Lehmann (University of Bologna and IZA)

Forlì – politica economica 2014-2015

2

Structure of Lecture • Active Labor Market Policies (ALMP): types of

programs, scope, their rationale in OECD countries and issue of applicability to transition countries;

• ALMP in Macedonia: A cautionary note;

• A brief history of the evaluation of ALMP in transition countries:– Macroeconometric studies;– Microeconometric studies;

• Case of random social experiments: pros and cons

Structure of Lecture, cont.

Country studies:• A randomized social experiment in Hungary (Micklewright and Nagy

2005);• The public works program in Slovenia and the Heckman correction

model (Vodopivec 1999);• ALMP in Poland and exact matching models (Kluve, Lehmann and

Schmidt 2008);• The “Beautiful Serbia“ program and propensity score matching

(Bonin and Rinne 2007);• Lock-in effects of Slovak subsidized jobs: a natural experiment (van

Ours 2004) .• Economic Impacts of Professional Training in the Informal Sector:

The Case of the Labor Force Training Program in Côte d‘Ivoire (Verner and Verner 2005)

3

What do we mean by ALMP here?

• We use narrow definition:• ALMP are policies directed at the unemployed or those

workers threatened by redundancies.

4

5

ALMP in OECD: archetypical programs and generic purpose

6

Scope of ALMP: expenditures in EU-15 states

Figure 2:

Average expenditures (in % of GDP) on ALMP in the EU15

0

0,5

1

1,5

2

2,5

7

Scope of ALMP: expenditures in new EU states

Figure 1:

Average expenditures (in % of GDP) on ALMP in the new EU member states

00,10,20,30,40,5

Estonia

Romania

Latvi

a

Czech

Rep

ublic

Lithu

ania

Slovak R

epub

lic

Poland

Bulgar

ia

Hungar

y

8

ALMP in OECD countries – micro aspects

• ALMP in mature OECD countries like b, c, d and e seek to integrate marginal social groups or to re-integrate marginalized groups into the labor market.

• large majority of the labor force are continuously employed, but certain groups with relatively loose labor market attachment and/or very low human capital experience great difficulties in finding permanent employment

• ALMP can increase the human capital of such "marginal" persons and can strengthen their labor market attachment thus boosting the probability of employment or re-employment.

• ALMP is also meant to fight poverty by increasing productivity of workers and boost their wage income.

ALMP in OECD countries – macro aspects

• ALMP are meant to increase labor demand in times of recessions (purpose is obvious and trivial, effect on overall unemployment can be quite complex, though – see Calmfors (1994).

• Also, ALMP are meant to “cheat the Phillips curve“, i.e. to lower the trade-off between unemployment and inflation.

• Bailey and Tobin (1977): in times of close to full employment, conventional Keynesian aggregate demand management policies will only push up inflation if those who are unemployed are hard to employ (because of lacking human capital and/or loose labor market attachment).

• Recipe: provide training or selected public employment to those difficult to employ since these persons will not contribute to inflationary pressures, i.e. these persons find themselves on flatter “Phillips curves“ than workers with high human capital and strong labor market attachment.

9

10

ALMP in OECD countries, “business cycle perspective”

Integration or re-integration of "problem persons" effective labor supply, ceteris paribus equilibrium wage employment; Relates also to persons who have lost their job because of structural shocks or a deep recession (long-term unemployed). ALMP which successfully increase the effective labor supply, during the expansionary phase of the business cycle, can contribute to the dampening of inflationary pressures and/or help in the solution of partial hysteresis of unemployment due to long-term unemployment.

ALMP and unemployment hysteresis

11

12

General questions asked when evaluating ALMP measuresDid the ALMP measure lower the overall unemployment rate? Did the schemes target the groups identified as those having problems leaving unemployment? Did participation in a scheme enhance individuals' productivity, expressed in higher wages? Did the measure increase the average re-employment probability of participants? Have distortive effects, e.g. substitution - dead weight- displacement of output- and fiscal substitution effects, been minimized?

13

Applicability of ALMP measures developed in another context to

transition countries Reiterating the main principles of ALMP in OECD

countries• Historically, ALMP in mature OECD countries like b, c, d and e

seek to integrate marginal social groups or to re-integrate marginalized groups into the labor market.

• The large majority of the labor force are continuously employed.

14

Applicability of ALMP to transition countries

Labor markets in transition economies are in general characterized by:– a low demand for labor– a stagnant unemployment pool – rising long-term unemployment – tougher competition for jobs among the unemployed than in

most mature OECD countries – Strong labor market attachment of a significant component of

the unemployed (and even of the long-term unemployed)– Large stock of accumulated human capital among the

unemployed (and even the long-term unemployed)N.B. These stylized facts certainly given in first ten years of

transition and relevant in Balkan economies even today.

15

Applicability of ALMP to transition countries – example training

• Unemployed in CEE countries have potential for adapting relatively quickly to new tasks. They may, therefore, be the typical target group for measures like further training and retraining.

• However, targeting the “standard” target groups (unskilled, low-educated, older workers, etc.) for further training and retraining might not be efficient – see stylized facts.

• In actual fact, in most transition countries we observe “creaming effects.”

16

ALMP and regional mismatch in transition

ALMP measures like b, c, and d are means of reducing mismatch by skill but also by region.

1. Taking the workers to the work:

FOCUS: helping the unemployed move to regions with better employment opportunities.

• Further training and retraining schemes • direct interventions by governments (e.g. subsidizing public

housing in high opportunity regions; government contributions to moving expenses.)

17

ALMP and regional mismatch in transition, cont.

2."bringing work to the workers",

FOCUS: creating jobs in high unemployment regions.

• investment grants or subsidies to firms if they locate or undertake new capital investment in such regions.

In transition countries both strands of regional policies are problematic because:

• High barriers to worker mobility• Investment policies are prohibitively expensive (especially as most governments in transition economies have budgetary problems)

18

ALMP in a transition country:

Polish ALMP measures in some detail

19

• Public Works;• Intervention Works;• Further training and re-training;• Start-up loans.

Polish ALMP measures that we look at

20

Public WorksLocal authorities employ those with uninterrupted unemployment spells of more than six months on public projects. Purpose of projects:• expand or maintain the public infrastructure ( latter more common);• environmental protection or amelioration.

The duration of these jobs cannot exceed six months and it is the expressed intention of the government to rotate them among the long-term unemployed.People employed on public works might receive wages that are above the minimum wage.

21

Intervention Works (wage/job subsidies)

Firms (private or state-owned) can approach the local employment council and ask for subsidized additional work places.

In order to qualify for this scheme the firm has to have more than 10 employees and must not have released more than 10 percent of its workforce in the last six months.

Subsidized employment is not to exceed six months. The state pays a wage subsidy to the firm equal to the level of benefits and often firms or local employment councils pay additional wages to these workers.

22

Intervention works, aims

Intervention works have two aims.

1. by hiring an unemployed person on a subsidized job he or she can enhance or regain human capital that might enable him or her to subsequently enter a regular job.

2. entrepreneurs can learn about the productivity of a worker without paying him or her a full wage.

23

Intervention works, structure of incentives

• Incentives to the firm are structured in such a way that ensures the longest possible employment relationship. The longer a previously unemployed worker is kept in an intervention works slot the higher the cumulative subsidy going to the firm will be.

• Workers have an incentive to hold on to such a subsidized job for at least 6 months as, in the 1990s at least, an employment relationship of this length entitled workers to another round of 12 months benefit receipt.

24

Further training and re-training

Private and public agencies are paid a fee to train some of the unemployed who in turn are paid an allowance (115 percent of benefits) while on the course.

Main objectives: solve skill mismatch and augment human capital. By increasing the human capital of the unemployed in skills that employers in the expanding sectors want, the chances of the unemployed to enter a regular job are meant to increase and bottlenecks in the supply of certain skilled workers are meant to be eliminated.

25

Start-up Loans: set-up

Subsidies (in the form of credits) to the unemployed given by LLOs to start-up their own businesses.

If after 24 months of founding a business it is still operative, 50 percent of the loan will be written off. Employment offices seem to examine applicants well as many of the started businesses presumably survive for more than two years.

26

Start-up Loans: small displacement of output effects

• Most of the started businesses are in the services sector which at least in the first ten years of transition was underdeveloped in Poland.

• Displacement of output effects which, e.g. in the case of the

British Enterprise Allowance Scheme, were estimated to be approximately 50 percent should, therefore, be relatively small.

27

Evaluation of ALMP:

Why is it important to have outside researchers evaluate ALMP?

What do we mean by evaluation

• Evaluation does not mean an evaluation of the adminstrative process connected to the implementation of the ALMP measure.

• Evaluation means that we want to know whether the ALMP measure has improved labor market prospects at the macro but also at the individual level.

• Pitfalls (2 examples):– Number of participants is not a measure of efficacy;– Before- after comparisons are not a reliable measure of success of a

program.

28

29

Evaluation of ALMP:

- Macroeconometric approach- Microeconometric approach

30

Macroeconometric evaluation of ALMP

• Flow analysis of administrative macro data - aim: establish the overall effect of an ALMP measure on outflows from unemployment to employment (sensible when measure is “large”).

• Potential strong point of such an approach: take account of dead weight loss and substitution effects.

• Link to general equilibrium analysis of the effects of ALMP on the natural rate of unemployment or on the NAIRU (non-accelerating inflation rate of unemployment)

Determination of employment and the NAIRU – theoretical underpinnings of evaluation

31

Determination of employment and the NAIRU

• Employment schedule downward sloping (like labor demand curve);

• Wage setting schedule: higher aggregate employment causes pressure for higher real wages – reasons:– In framework of trade union models;– In framework of efficiency wage models.

Determination of employment and the NAIRU: model extension with ALMP

Determination of employment and the NAIRU: model extension with ALMP

• On horizontal axis now: regular employment ;• Open unemployment, u0, and program participation,

r0, make up that fraction of the labor force not regularly employed;

• participants in ALMP shifts RR-curve to the left and fraction of program participants is increased by r;

• Open unemployment is reduced to u1;

• This is gross effect of program expansion;• To get to net effect, we need to account for indirect

effects via shifts of WS and RES.

Effects of ALMP expansion on WS and RES:

Effects of ALMP expansion on WS and RES:Matching effects

• ALMP expansion improves matching efficieny via three channels:– Attenuating mismatch (retraining and further training);– Promotion of active search behavior of unemployed (e.g.,

Restart program in Britain);– Working in labor market program substitutes for regular

employment (solves asymmetry of information).

• Positive effects: shifting to the right of RES and of WS (from A to B): regular employment expands for sure, effect on real wage ambigous.

Effects of ALMP expansion on WS and RES:Matching effects

• Negative effects: lock-in effects during program participation and ex ante effects of reduced search behavior (especially if program has high value);

• Micro studies that compare, e.g., employment rates of treated and controls might have an upward bias of treatment effect because controls reduce active search behavior before program starts (wait to participate).

• Net effect: positive effects minus negative effects.

Effects of ALMP expansion on WS and RES:competition effects for insiders

• Policies targeted at outsiders (like the long-term unemployed) might raise the competitiveness of these unemployed relative to insiders;

• This results in downward wage pressure, i.e. WS shifts down and new equilibrium is at D regular employment rises as real wages go down;

• Job search assistance programs (like Restart) generate the fastest competition effects.

Effects of ALMP expansion on WS and RES:dead weight (DW) and substitution effects

• Definitions of DW and substitution effects:– DW: hirings from target group that would have

occurred in absence of program;– Substitution: jobs created for certain category

replace jobs of other categories because of changes in the relative wage structure.

• Both effects shown by a leftward shift of RES (at given real wage regular employment declines): new eqilibrium at C regular employment and real wages decline.

Effects of ALMP expansion on WS and RES:dead weight (DW) and substitution effects

• Most relavant in private sector employment incentive schemes;

• Also given in the public sector, though where we can have fiscal displacement (Example: municipality holds back on hiring two janitors, hires instead two subsidized unemployed, this frees money for hiring a police officer).

Effects of ALMP expansion on WS and RES:other effects

• Reduced welfare losses and reduction in wage constraints: equilibrium moves from A to E;

• Productivity effects: two scenarios possible: (a) RES shifts up with equilibrium at F

(b) RES and WS shift up with equilibrium at G.

Macroeconometric Evaluation

42

Relating the unemployment rate to outflows

• By definition we get the identity:

• In the steady state (we can drop subscripts) this becomes the equation:

• In Britain, e.g., Pissarides (1985) finds that I are relatively constant and the rise of U over time is caused by a fall in o;

• In transition countries - against predictions – I are not particularly large, o are particularly small (relative to OECD countries).

1t t t tU U I O

, inf int

.

I IU where I is lows ounemployment

O oU

and ois theoutflowrate fromunemployment

44

Macroeconometric evaluation of ALMP

• Flow analysis of administrative macro data - aim: establish the overall effect of an ALMP measure on outflows from unemployment to employment (sensible when measure is “large”).

• Potential strong point of such an approach: take account of dead weight loss and substitution effects.

45

Macroeconometric evaluation of ALMP

• Heuristic model:

where O=outflows from U;x1=variables controlling for the state of the labor market; x2= ALMP measures.

Working horse of this model: “augmented” matching function;Data in industrialized countries: quarterly time series at

national level (if stationary, little problems regarding estimation);

Haskel and Pissarides (1988), Layard et al. (1991), Lehmann (1993), Dauth et al. (2010).

1 2 1 2 2O = f ( ; ), > 0, 0 0x x f f for

46

Macroeconometric approach, cont.

“Augmented” Matching Function

The "working horse" giving economic content to models of outflows from unemployment to employment is the matching function where in its simplest form unemployed job searchers are matched with vacant jobs. In this context ALMP can be thought of as measures which facilitate this matching process. When ALMP are added to the stock of unemployment and vacancies as factors potentially determining job matching we speak of an "augmented" matching function. The following briefly motivates one theoretical derivation of an estimable "augmented" matching function.

47

Short theoretical derivation of an estimable augmented matching function

0. > f ,f U), (V, f = H 21 (1) H is the number of unemployed being hired in non subsidized jobs in a given period; U and V are respectively the stock of registered unemployed and of notified vacancies, both measured at the beginning of the period; U is the search effective stock of the unemployed, 0=<=<1. Equation (1) says that the number of hirings depends on the number of vacancies and the fraction of the unemployed who are fit for the labor market.

Macroeconometric approach, cont.

48

Macroeconometric approach, cont.

• • with and (2)

• is the search effectiveness index, c is an index of search effectiveness in absence of search-enhancing labor market schemes, and M is the weighted sum of the search-enhancing employment measures E.

• One way to think about search effectiveness is to relate it to the duration of unemployment: the longer the duration the lower the search effectiveness because of actual lack of productivity or because of perceived (by employers) lack of productivity on the part of the unemployed (e.g., lack or loss of human capital).

• ALMP are a means to boost this search effectiveness.

M), + (1 c = , ii

n=i

1=i

= M 1 = ii

49

Macroeconometric approach, cont.

• Log-linearizing equation (1) and adding a constant term we get

• Ln (U)=ln + ln U. But ln = ln [c(1+M)]. We know that, for small values of x, ln (1+x) x.

For small values of αM the equation becomes

• Not imposing the same coefficient on lnU and lnc, adding seasonal dummmies and a time trend plus an error term, we get an estimable function:

(3) U)(ln + Vln +ln = Hln 210

(4) = whereM, + c)ln + (ln U + Vln + Hln 233210

3

0 1 2 3 4 51

ln ln ln ln ( /100) (5)i ii

H V U c ALMP t sd

Instrumenting ALMP

• Nice example of instrumenting ALMP: Boeri (1997) who uses expenditure levels of ALMP assigned at the beginning of the year as an instrument on the actual expenditure.

• Intuitive idea is that while assigned levels are highly correlated with the actual expenditures on ALMP, they are not correlated with unforeseen shocks that affect outflows from unemplyoment.

• Mathematically, let Z be the instrument, then E(Z,ALMP)0 but

E(Z,)=0. In the bivariate case the OLS and the IV estimators are:

50

1

2

1

( )

n

i ii

ols n

ii

O ALMP

ALMP

1

1

n

i ii

IV n

i ii

O Z

Z ALMP

51

Boeri (1997) Table 4: Czech Republic (cursive entries are standard errors) c1 c2 gamma 1

52

Boeri (1997) Table 4: Poland – (1) outflows to all jobs; (2) outflows into non-subsidized jobs only

c1 c2 gamma 1

Detecting dead weight loss and substitution effects

• Under certain set-ups analyzing flows out of unemployment into employment allows us to get at dead weight loss (DWL) and substitution effects of ALMP.

• Definitions:– DWL occurs if, e.g., the government subsidizes the hiring of the long-term

unemployed, but an employer has the intention to hire long-term unemployed no matter whether the government provides a subsidy or not. Of course, the employer is happy to take the subsidy anyway. Hence, this means that the funds provided by the government have not changed the behavior of this economic agent these funds are lost to society.

– Substitution occurs in various ways: (a) when an employer hires the long-term unemployed instead of the short-term unemployed;

(b) When an employer fires unsubsidized workers and hires the subsidized long-term unemployed workers in their place.

53

Detecting dead weight loss & substitution effects, cont.

• Let us have two duration-specific outflow rates from unemployment,

Ost and Olt , where the first variable is the outflow rate from short-term unemployment (u< 1 year) and the second term is the outflow rate from long-term unemployment (u>= 1 year). Let ALMPlt be a measure targeted at the long-term unemployed. Then, we can estimate the following system of equations:

Assume that error terms are contemporaneously correlated (SURE).Then substitution effect is given if:

and DWL:

54

ln ( )

ln ( )st st lt st

lt lt lt lt

O f labor market state ALMP

O f labor market state ALMP

0,st

0.lt

55

Macroeconometric evaluation of ALMP in transition countries

• This type of evaluation has fallen from grace – what are the reasons?– Too many loose ends with methodology;– Hard to tease out some correlation between any measure

of ALMP and the unemployment at national level;– James Heckman has set the evaluation agenda (micro

guy).

Microeconometric evaluation of ALMP

56

57

Microeconometric studies on effectiveness of ALMP

This approach looks at post-treatment labor market outcomes (treatment here is participation in ALMP), mainly labor market status (employment, unemplyoment, inactivity) but sometimes also earnings, of persons who have been on an ALMP scheme and compares them to those hypothetical labor market outcomes that would have occurred if the same persons had not participated.

problem of counterfactual

The need of a control group• If I want to evaluate the impact of a measure I need a control group

as a benchmark;• Example: clinical trials – we have the treated (those who get the

new medicament) and a control group (those who get a placebo);• We have an outcome like, e.g., duration of life;• Clearly, the outcome of the treated relative to the outcome of the

controls is the treatment effect here;• Note: in clinical trials researchers can fix all other factors that have

an impact on the outcome of interest, they vary only the dosage of the treatment (in our case full dosage versus zero dosage) true causal effect of treatment can be identified;

• With social and economic policies it is a lot more difficult to fix all factors apart from the treatment.

58

59

Problem of counterfactualConsider labor market outcome Y and binary treatment D, participation in an ALMP or not, for an individual:

Unit level effect is never directly observable. Why?

,1

,0

1

0

DifYY

DifYY

01 YY

60

Problem of counterfactual• Need measure that summarizes individual impacts

appropriately:• ATET = Average Treatment of the Treated:

• Not identified without random assignment! Why?

Can never be observed

Can be observed but is not necessarily the same as • Only if we have a random social experiment , is it true

(but even here only under certain assumptions) that

)1|()1|()1|()1|( 0101 DYEDYEDYYEDE

0 0( | 1) ( | 0)E Y D E Y D

0( | 1)E Y D

0( | 0)E Y D 0( | 1)E Y D

Problem of counterfactual

61

• If we have random assignment to an ALMP, then we can identify the Average Treatment Effect of the Treated (ATET), since:

• So, with a random social experiment it is relatively easy to establish an average causal effect of an ALMP. All I need to do is to take a group of similar persons, assign half of these to the ALMP randomly and the other half to non-participation. Then compare the average outcome of participants to the average outcome of non-participants; any difference in the two average outcomes is then the causal effect that we wish to establish.

• There are very few social experiments in labor economics, though. In most of the evaluation work researchers use “observational data“ and try to get:

1 0 1 0( | 1) ( | 1) ( | 1) ( | 0)E Y D E Y D E Y D E Y D

0 0( | 1) ( | 0)E Y D E Y D

The problem of selection bias• Without random assignment to treatment, why is:

• The reason normally given is selection bias. This means that persons selecting themselves into an ALMP might have some unobservable characteristics that are different from the unobservable characteristics of non-participants. If these unobservable characteristics also have an impact on the labor market outcome, Y, then the difference in the average outcome of participants and the average outcome of non-participants does not give us the true causal effect of the ALMP on Y.

• If participants are for example more motivated than non-participants, then (at least) part of the difference has to be attributed to the greater motivation rather than to the ALMP.

62

0 0( | 1) ( | 0)E Y D E Y D

The problem of selection bias (cont.)• Let us demonstrate selection bias by looking at a very simple

regression framework:

• where D=1 if a person participates in an ALMP and D=0 if a person does not participate.

• Under the error term, , all unobserved characteristics that have an impact on the wage are subsumed. If one of those unobserved characteristics is motivation (i.e. motivation has an impact on wages) and if more motivated people are more likely to participate in the program then we have E(D, )>0. If we use OLS estimation then will have an upward bias.

• Under random assignment E(D, )=0 and OLS gives an unbiased estimate of .

63

0 1Wage yrseduc D

64

Methods of micreoconometric impact evaluation

• Randomized social experiments;

• Methods with observational data:– Hazard rate analysis/multinomial logit;– Exact matching; – Matching on the propensity score, i.e. on the probability

to participate in a measure;

65

Rationale for microeconometric evaluation

• Rationale: how sensible given that I can‘t catch general equilibrium effects?

• What do evaluation studies say about the effectiveness of various ALMP measures?

• Table on next page looks at overall effects – one important caveat: need to look at subpopulations since effectiveness can be quite heterogeneous.

Efficacy of ALMP programs

66

Type of program A. Costs

B. Evidence on Effectiveness

a. Public employment services (“job

brokerage”) and administration

A. relatively cheap

B. highly effective* (unanimous)**

b. Labor market training A. relatively very expensive

B. effective (mixed)***

c. Employment incentives / Start-up incentives A. relatively expensive

B. effective (mixed)

d. Direct job creation / Public sector

employment

A. relatively expensive

B. ineffective (unanimous)

Note: * “effective” means that the average employment or reemployment probability of a person participating in the indicated measure is increased.** “unanimous” means that virtually all studies show the indicated effect.

*** “mixed” means that some studies show the indicated effect but other studies do not.

“Problems“ with ALMP in lower income transition countries

• The results in the above table derived mainly from the new member states of the EU;

• Like in developing countries, in lower income transition countries we might find:

– Poor adminstrative structures to implement and monitor policies;– Labor markets are often segmented;– Under-employment rather than unemployment;– A large informal sector.

67

ALMP in times of world depressions: A topic for discussion!

• The discussion of the rationale and efficacy of ALMPs in the literature relates to “normal times“, i.e. periods with moderate vascillations of the business cycle.

• Are requirements for ALMPs different in times of large depressions (like the great depression of the 1930‘s and the global crisis now) than in “normal times?“– example “public works“ – in normal times they get a bad

press as unable to raise the probability of participants to find regular employment after participation;

in depressions they might be needed as a social policy to prevent wide-spread poverty.

68

The case of social experiments - prodecure

• Reiterating the procedure in case of a training program:– a group of persons are selected or choose themselves to participate in

the program;– From this group persons are randomly selected to get training and

(consequently) those not selected are randomized out of the training program;

– Then, a comparison of the average labor market outcome after the training is over of those randomly selected in and those randomly selected out gives me the average treatment effect of the treated.

69

The case of social experiments - pros

• A randomized social experiment for sure establishes a causal effect of the program, i.e. it solves the problem of the counterfactual and establishes the average treatment effect of the treated (ATET) – this under certain assumptions (no randomization and substitution biases: see cons slide);

• It requires relatively simple statistics and econometrics and the underlying assumptions are not very restrictive;

• The results are easily understandable for policy makers and pundits and do not require an advanced degree in econometrics;

• There are some arguments for randomization even on ethical grounds (better to have random assignment than selection by labor officials that serves the interests of the officials).

70

The case of social experiments - cons

• A randomized social experiment (RSE) can establish the average effect of the program but not, e.g., the median effect or other moments of a distribution;

• Using a RSE is essentially a “black box“ procedure, i.e. it does not take into account the cumulative knowledge that we have about the functioning of labor markets;

• Ethical considerations – denying a person treatment who really needs it is not ethical;

• Randomization bias – arises when participation in program changes because the program has random selection into the treatment;

• Substitution bias – arises when those who have been randomly denied treatment look for close substitutes;

• With randomization and substitution biases RSE does not give a causal effect.

71

Country studies

• Hungary• Slovenia• Poland• Slovakia

72

73

Randomized social experiment in Hungary

• Micklewright and Nagy (2005) on job search monitoring and unemployment in Hungary;– 1st randomized experiment to my knowledge;– UB recipients randomly assigned to intense and very

frequent interviewing on job search or to minimal interviewing;

– Econometrics can be kept simple but treatment effect is identified;

– Result: only women age 30 or over had shorter duration of unemployment after treatment; o.w. no treatment effect.

Effect of Public Employment Scheme in Slovenia (Vodopivec 1999)

• This public works scheme provides jobs in public administration or the non-profit sector;

• Maximum duration: one year;• While in most transition countries public works jobs have a

shorter duration and a low human capital content, in Slovenia persons on scheme can acquire human capital in a substantial way;

• Compared to most transition countries the educational level of participants is on average far higher in Slovenia (see table on next slide);

• While everybody can participate in the program, “problem groups“ are the explicit target groups of the program.

74

Vodopivec (1999)

75

76

(Vodopivec 1999) – empirical strategy

• EXITni= individual's labor market status after spending n months

searching for a job. • For those who participated in the public works program, the

start of the searching time was set to zero at the moment when they finished their participation in the public works program. For those who did not participate in public works, the start of the searching time coincided with the registration at the employment office.

• The variable EXITni - three values: 0, if after n months individual still unemployed; 1, if after n

months individual employed; and 2, if after n months individual inactive (out of labor force).

77

Model individual's labor force status after n months of job search as: EXITni = X’i1 + PWi2 + i (1) where Xi - a vector of personal characteristics (gender, ethnicity, and age) and human capital characteristics (education, work experience, health condition), PWi is a dummy representing past participation in public works (PWi = 1 if an individual participated in public works, 0 otherwise), and 1 is a parameter vector and 2 is parameter (scalar) to be estimated. By assumption, E(i) = 0 and Var(i) =

2.

Vodopivec (1999) – empirical strategy

78

Vodopivec (1999)Sample selection problem and possible bias of coefficients

We might get biased estimates of the impact of public works on chances to find a job if there is a problem of selection.

Individuals opting to participate in public works may differ from those opting not to in many aspects, some of which may be unobservable. If these unobservable characteristics also affect job prospects individual, the equation (1) is misspecified and the estimated coefficient 2 biased.

This bias can be negative or positive!

79

Remedy to self-selection problem Heckman two-stage procedure: In the first stage, equation of participation in public works is estimated, with regressors derived from the process and circumstances described above. The outcome of that stage is a new variable (the inverse Mills ratio, ), to be used as one of the regressors in the second stage -- that is, in the estimation of equation of exit from unemployment.

Vodopivec (1999)

80

Participation equation: PWi = Xi 1 + Zi2 + ui (2)

where Xi : personal and human capital variables, Zi : factors capturing criteria for selection for public works (number of dependents, for example). This estimation produces a new variable - inverse Mills ratio

i = ( Xi 1 + Zi2 )/( Xi 1 + Zi2 ), for participants of the public works, i = - (Xi 1 + Zi2 )/(1 - ( Xi 1 + Zi2)), for non-participants

Vodopivec (1999)

81

Assuming joint normality for distribution (i , ui) with the correlation , for participants,

E(EXITi PWi =1) = Xi1 + 2 + E(i PWi =1) = Xi1 + 2 + {(Xi 1 + Zi2 )/(Xi 1 + Zi2)},

and for non-participants:

E(EXITi PWi =0) = Xi1 + E(i PWi =0) = Xi1 + {- (Xi 1 + Zi2)/(1 - (Xi 1 + Zi2))},

Vodopivec

82

Vodopivec (1999)• The difference in expected value of EXIT between the

participants and non-participants is thus

• E(EXITi PWi =1) - E(EXITi PWi =0) = 2 + {( Xi 1 + Zi2 )/( Xi 1 + Zi2) (1 - ( Xi

1 + Zi2 ))} (3)

• By including the selectivity correction term in the estimation of equation (1), the bias presented by the second term of the right-hand-side of equation (3) is purged from the estimates.

• Programs selection rules produce a variable to be used as an instrument identifying the selection equation.

83

Vodopivec (1999)

• Results: – Immediately following participation in Slovenian public

works scheme higher probability to find regular employment than non-participants;

– This positive effect dissipates as participants stay on in unemployment.

• Main problem: bi-variate normal distribution of the two errors is just assumed

Polish ALMP and MatchingKluve, Lehmann and Schmidt (2008)

1. Background: The Polish Labor Market

2. Data & MethodsData: retrospective data at individual levelMatching: imitation of a social experiment

3. The Analysis of Matched SamplesComposition of matched samples and timing of interventionsDistribution of pre-treatment LM histories

4. Empirical ResultsHeterogeneity of labor market outcomesAverage treatment effects and their interpretation

5. Conclusion

Background: Polish labor marketLegacy of state-wide planning systems: Dramatic rise of openunemployment (Góra and Lehmann 1992; Góra and Schmidt

1998)

Unemployment Rates, Poland 1994, by AgeMales 15-24 25-34 35-44 45-54 55-64 All Ages

30.4% 11.3% 10.3% 7.6% 6.2% 12.7%

Characterization as “stagnant pool” (Boeri 1994):

LTU Shares, Poland 1994, by AgeMales 15-24 25-34 35-44 45-54 55-64 All Ages 24.9% 36.0% 37.1% 44.8% 38.4% 33.4%

GDP shows U-curve in transition (Blanchard 1997) In sample period - 1992-1996 – GDP in Poland already growingBut employment declining until 1994, after that modest rise

Background: Polish labor marketMeasures of ALMP to combat U and LTU:

- Provision of human capital: Further training and re-training

- Wage subsidies: intervention works - Public works- Loans to set up own business

Note:- To participate need to be registered with labor office.- for wage subsidies and public works:

>=6 months employment, benefit receipt eligibility restored

Background: Polish labor marketEvaluation of effectiveness of measures imperative, difficult task:

- non-experimental data prevalent in Europe, and CEE- as transition begun, no "all-purpose" data sets available- important to control for changing macro environment

This study: Evaluation of ALMP interventions "training" and"intervention works" in Poland 1992-1996, w/ emphasis on:

- Application of covariate matching, role of LM histories- Specifically adapted (to the case of retrospective data) variant of

matching estimator, "moving window" technique- Treatment effect heterogeneity wrt/ subsets of participants

Data and MethodsData: "Polish Labour Force Survey" PLFS

– collected quarterly– rotating panel– started in May 1992

18th wave (August 1996): Monthly information on individual labor market histories, January ‘92 – August ‘96 (56 months)

– Condensation into quarterly sequences– Elimination of individuals with “too early” (before January ’93) or

“too late” (after November ’95) treatment participation– Labor market outcomes: ”employed = 1”, ”unemployed = 2”, ”not in

the labor force = 0”, and ”treatment participation”

Matching using a "moving window" (sample B)

Matching over identical individual LM histories using a "moving window" (sample C)

Data and Methods: MatchingConsider binary treatment D, treatment participation or not

Unit level effect never directly observable

Need measure that summarizes individual impacts appropriately:

ATET

Not identified without random assignment, consider vector of covariates X identifying assumption: assignment mechanism D is independent of potential outcomes Y0,Y1 conditional on X.

,1

,0

1

0

DifYY

DifYY

01 YY

)1|()1|()1|()1|( 0101 DYEDYEDYYEDE

Data and Methods: MatchingGiven unconfoundedness, i.e. selection on observables:

)0,|()1,|(

)1,|()1,|()1,|(

01

01

DXYEDXYE

DXYEDXYEDXE

Analyzing Matched SamplesSample Composition:

- For each of the two treatments, consider 3 matched samples

- Increasing requirements on X- Focus on role of LM histories

- Why? Recent literature (e.g. Heckman and Smith 1999, 2004) emphasizes correlation of outcome before and after intervention, and stresses role of LM dynamics (“LF status dynamics”).

- Employment histories: Card and Sullivan (1988)

- “Ashenfelter's dip”

Analyzing Matched SamplesWe use all records with at least one spell of unemployment in thesample period

Sample A:- Control unit matched to treated if LM history observed w/out

gaps- Contents of LM history not used, no restrictions on covariates

Sample B:+ match control unit if identical in: age, sex, education, marital

status, and region

Sample C:+ match control if displays identical 4-quarter (12-month) pre-

treatment LM history at exact same point in time as treated

Analyzing Matched Samples

Analyzing Matched SamplesThe timing of interventions:

- For comparability, timing structure same for all samples (12+x+9)- Sequences have to be complete- Information which LM history only used for sample C

- Monthly information condensed into quarterly sequences of LF status: 0=inactive, 1=employed, 2=unemployed

- Duration of treatment variable for individual matches:- Pre-treatment history ends at beginning of treatment (month

"start")- Post-treatment outcomes start at the end of treatment (month

"stop")

["moving window" figures]

97

Matching using a "moving window" (sample A)

98

Matching over identical individual pre-treatment labor market histories using a "moving window" (sample C)

Analyzing Matched SamplesIndividual pre-treatment histories:

12-month pre-treatment history can be summarized in overall 81 quarterly sequences ("0000" to "2222")

For illustration condensed into 11 coarse categories:

- "not active" (many 0s, category 1=0000)- "unemployed" (many 2s, category 6=2222)- "employed" (many 1s, category 11=1111)

Analyzing Matched SamplesDistribution of pre-treatment LM history by sample:

Intervention Works: Training:

101

Matching using a "moving window" (sample B)

102

Matching over identical individual pre-treatment labor market histories using a "moving window" (sample C)

Analyzing Matched SamplesDistribution of pre-treatment LM history by

sample:

- Sample B: strong deviation between treated and matched comparison units

- Workers in IW display particularly many "unemployed" histories

- For both measures, comparisons in sample B are "too successful"

- Inclusion of covariates when moving from sample A to sample B does not help much in balancing LF status sequences

Empirical Results

Distribution of LF states after treatment:

- Accordingly, the 9 months succeeding treatment can be summarized in overall 27 quarterly states ("000" to "222") and 9 coarse categories:

- "inactive": many 0s, category 1=000- "unemployed": many 2s, category 5=222- "employed": many 1s, category 9=111

EmpiricalResults

Empirical Results- loss of treated observations while moving from B to C

does not substantially affect distribution of success after treatment

- without consideration of pre-treatment histories in B the matched comparison units are, on average, „too successful“

- among treatment participants, the „more successful“ are on average in „training“, not in „intervention works“

Empirical Results

- For intervention works the "unemployed" sequences dominate the picture for all samples.

- Comparison units in B are, on average, quite successful, but not so in C.

- A similar, albeit not so pronounced picture emerges for training:- in B the "employed" sequences dominate the picture among

comparisons, while "unemployed" sequences are more prevalent among treated

- this order reverses with sample C as comparison.

Empirical Results

Empirical Results

Empirical Results

Empirical Results

Empirical ResultsATET (sample C) is weighted average of pre-

treatment history-specific effects:

Empirical ResultsIntervention Works:

Empirical Results

Training:

Empirical ResultsBenefit receipt, sample C:

ConclusionsAnalysis of LM interventions involves key problem of non-experimental construction of appropriate comparison situation

Specific evaluation context here: rapidly changing macro conditions , so authors use"moving window“ treatment start / stop for non-participants

Role of Labor Force status histories: contain indispensable information re/ selection and help reduce overt bias

Results suggest training might be promising treatment for participants, while intervention works is not: "benefit churning"

118

Quasi-natural experiment with Slovak ALMP

• Van Ours (2004) looks at “lock-in effect“ of subsidized jobs in Slovakia, taking advantage of a “natural experiment“ – extension of Publicly Useful Jobs (PUJ) from 6 to 9 and then 12 months, while Socially Purposeful Jobs (SPJ) retain their duration of 24 months.

• Two countervailing forces as far as the effect of subsidized jobs on gaining regular employment: (+) signal of higher willingness to work/higher productivity and (-) with length of subsidized job fall in job search intensity (i.e. lock-in effect);

• Result: increase in duration of PUJ lowers transition into regular jobs (increased lock-in).