Active labour market measures and

entrepreneurship in Poland

Rafał Trzciński

Impact Evaluation Spring School

Hungary, 18.05.2011

case 1 – Evaluation of @lternative II case 1 – Evaluation of @lternative II projectproject• Objective of the project: preventing unemployment among

young people.• The territorial scope: 59 poviats of Poland (NUTS 4) with

unemployment rate above 20% and high unemployment among young people.

• Eligibility:• young unemployed (27 or younger), • registered at the labour office.

• Total number of beneficiaries: 5 657.• Budget: € 4 090 702.• Type of services: pre-training assistance (recruitment, needs

assessment, guidance); vocational training services linked with ECDL, both at basic and advanced level, as well as related areas; practical training (temporary employment/ on the job training organised under the agreements signed with employers; training allowances; vocational guidance and post training assistance; job-broking.

• Period of implementation: 2005-2006.

?

Selectionto the project

Self-selection to the project

Problem: High unemployment

rate among young people

@lternative IIproject

Efect: lower unemploy-

ment

Factor x1

Factor xn

SELECTION BIAS

Evaluation problemEvaluation problem

The selection bias problem in The selection bias problem in control/comparison group control/comparison group approachapproachTo estimate the impact of the

intervention we cannot simply compare beneficiaries (treated) with those who did not participate in the project (non-treated)

This is because of factors affecting both participation and outcomes.

If we don't control for those factors, we can overestimate or underestimate the impacts(picking the winners/ picking the losers).

DData used in the evaluationata used in the evaluationPULS System, which:•is used for services for the unemployed•is present in approximately 90% of Poviat Labour Offices in Poland (2006)•provides a wide range of data on each unemployed person (socio-demographics, employment characteristics, previous qualification improvement, skills etc. ),•includes a detailed history of unemployment and other activities on each person (registration in the office, deregistration, trainings, use of the benefits, etc.).

Data collectionData collection• We collected data from 55 of the 59

Poviat Labour Offices involved in the project.

• In total we managed to identify 5 065 participants of the @lternative II project (90% of all beneficiaries).

• Moreover we collected data on 126 633 persons (non-treated), which meet the formal conditions for eligibility for the project (registration in the labour office, age condition).

VariablesVariablesSocio-demographic characteristics Sex Age Marital status Single parenting Number of children Education Poviat

Employment characteristics Profession (ten categories) Number of days of work Number of professions (in total)Number of days being unemployed before participating in the projectNumber of days receiving the unemployment benefit before…Number of job offers during the one year period before…Number of days participating in subsidised workNumber of days of permanent unemployment (during the two years period before…)

Previous qualification improvementNumber of training courses, in which the person participated during the one year prior to participation in the projectTotal number of days spent on trainingHaving a work placement before participating in the project Motivation to find a job

Percentage of showing up in the Poviat Labour Office,Having the right to unemployment benefit

SkillsPossession of driving license (B category)

Back to the selection Back to the selection problem…problem…

Counterfactual action

Bearing in mind the Bearing in mind the assumptions…assumptions…

Conditional Independence Assumption

Population A

Population BCounterfactual

action

• We assume that if we can control for observable differences in characteristics between the beneficiares and non-treated population, the outcome (observable change) that would result in the absence of treatment (counterfactual action) is the same in both populations.

• Ergo, we assume that unobservables do not affect the outcomes!

Eligible non participants(N=126 633)

Control group(N=5 065)

ps= 0,6ps= 0,5

ps= 0,8ps= 0,1

ps= 0,2

ps= 0,3ps= 0,2

ps= 0,01

ps= 0,4

ps= 0,9

Beneficiaries(N=5 065)

ps= 0,8ps= 0,3

ps= 0,9ps= 0,4

ps= 0,1

Propensity score matching (1-1; nearest Propensity score matching (1-1; nearest neighbour)neighbour)

What we have achieved What we have achieved usingusing PSM?PSM?

ImpactImpact

Source: Ex-post evaluation of Phare 2003 Economic and Social Cohesion – Human Resources Development component, PAED, Warsaw 2007

ImpactImpact

Cost-benefit analysisCost-benefit analysis

• Objective of the project: encouraging business activities among unemployed people.• Beneficiaries: unemployed people (with

priority to young job-seekers).• Type of services: initial business training;

guidance on conducting economic activities; training allowance; relevant specialised training; coaching after setting up a business.• Time of implementation: 2004-2005.• Evaluation framework: the same

approach as in the @lternativa II exaple (the same methodology, source of data, analysis...).

(Counter)example 2 – (Counter)example 2 – Entrepreneurship promotion projectEntrepreneurship promotion project

Impact?Impact?

Source: Ex-post evaluation of Phare 2002 Economic and Social Cohesion – Human Resources Development component, PAED, Warsaw 2006

Lessons learned/points for the Lessons learned/points for the discussiondiscussion

What data we were lacking in both examples?◦ Missing covariates? (Are our assumptions plausible?)◦ Missing outcome variables?

What do we know and what we don't know after completing the evaluation (towards theory based impact evaluation)?

How we could modify the plan of the evaluaton to get more insight on impacts (targeting issue)?

What is the avaibility of systems such as PULS in other EU countries (looking for possibilities of implementing IE)?

What is the utility of data collected in public statistics? Do we need new data systems for IE or maybe we need to modify existing ones? (towards more systematic discussion on IE planning).

Thank you!!!