Upload
lyle
View
40
Download
0
Embed Size (px)
DESCRIPTION
Evolution of Uval's work program: from support to evaluation. Italian Evaluation Unit Open Days 3 – 5 July 2006 Laura Raimondo. Until April 2006 Ministry of Economy and Finance. Department for Development Policies. CSF MA. UVAL’s tasks. UVAL founded in 1982 Reformed in 1998 - PowerPoint PPT Presentation
Citation preview
Open Days, Rome 3 July
Evolution of Uval's work program: from support to evaluation
Italian Evaluation Unit Open Days3 – 5 July 2006
Laura Raimondo
Open Days, Rome 3 July
Until April 2006
Ministry of Economy and Finance
Open Days, Rome 3 July
Department for Development Policies
CSF MA
Open Days, Rome 3 July
UVAL’s tasks
• UVAL founded in 1982• Reformed in 1998• Re-reformed in 2002• Main tasks:
– Promotion of data, information and knowledge for public policies
– Support to administrations to modernize and innovate public action,
– Evaluation of programs and capacity building for evaluation
– Ex-ante assessement of projects
Open Days, Rome 3 July
UVAL’S work program
• Activities (not staff) are organized in four areas– Methods and studies for public policy– Modernization and administrative innovation– Program evaluation – Project appraisal
• 104 activities• Support to other parts of the Department and to
the Network of the Italian Evaluation Units is still a large part of work (e.g., project appraisal, support to programming for 2007-2013)
Open Days, Rome 3 July
The Evaluation Unit (UVAL)
Area 1 Area 2 Division 3 Division 4
Metodi e studi per le politiche pubbliche
Trasferimento metodi e sostegno attuativo
Program Evaluation Project Evaluation
Tendenze e analisi delle politiche di
sviluppo territoriale
Sostegno attuativo ai servizi DPS e alle
amm.regionali e centrali
Coordination CSF evaluation system
Development of guidelines and standards for the
assessment of infrastructure and integrated projects
Rapporto territorialeIntese e APQ,
impostazione e performance.
Valutazione QCS e controllo qualità valutazioni dei PO
Progetto conti pubblici territoriali e
finanza pubblica
Attuazione Premialità QCS.
Metodi e indirizzi per la valutazione e l'utilizzazione
della valutazione
Support for project cycle enhancement
Supporto e formazione Rete dei nuclei
Gruppi di lavoro (GdL) del QCS.
Conduzione e committenza di valutazioni su strumenti, piani e
programmi
Certified assessment of specific sectorial programmes
Comunicazione e pubblicazioni.
Contributo indirizzi e raccordi attività DPS
Metaevaluation of projects
Area 1 Area 2 Area 3 Area 4
Methods and studies for public
policies
Modernization and innovation of
administrative actionProgram Evaluation Project Evaluation
Trends and analysis of territorial
development policies
Support for implementation for DDPservices and for
regional and central adm.
Coordination of CSF evaluation system
Development of guidelines and standards for the
assessment of infrastructure and integrated projects
Annual reportInstitutional agreements
and FPA, support to design and
implementation
CSF evaluation and control of evaluation quality of OP
Project of territorial public accounts and
public financesImplementation of CSF performance reserve.
Methods and guidelines for evaluation and use of
evaluation
Support for project cycle enhancement
Support and training to the Network of regional and central evaluation units
CSF working groups Conduction and commissioning of evaluation on instruments, plans
and programmes
Certified assessment of specific sectorial programmes
Communication and publications Input to DDP guidelines
Metaevaluation of projects
Open Days, Rome 3 July
Program evaluation
• Evaluation capacity development– Started with Mezzogiorno & structural funds– Now expanding to whole country & all development
program– Huge task: guidelines, groups, observation of
practice & processes, common work
• Evaluations– Slowly growing – Legitimity (who asks for evaluations)– Resources (financial and staff)
Open Days, Rome 3 July
Evaluation: where we started from
• A “continental” public service: No orientation to results
• Absence of social partners and policy makers
• Push for evaluation from EU Structural Funds:
– No national reinterpretation of general framework
– Predominance of EC evaluative needs
– Rigid framework• Compliance at best purely formal
• No interactions
• Low quality of evaluations
• Little (if any) use of evaluations
Open Days, Rome 3 July
Evaluation capacity strategy initially relied on:
• EU evaluation requirements
+• New context of policy making
– More ambitious development policy– More emphasis on local knowledge– More emphasis on change– More responsibilities to regional governments– Greater importance to administrative capacity
Open Days, Rome 3 July
Close links between administration capacity and development policy
• Development policy requires administrative capacity
• Evaluation capacity is part of administrative (and political) capacity
• Evaluation results improve development policies if useful for key actors
• More than one way to get there
Open Days, Rome 3 July
A piecemeal strategy
• developed over time
• gradually introducing innovations
• knowledge from day-to-day work
• taking into account achievements and disappointments
We are still working on it
Open Days, Rome 3 July
Some evaluation capacity building activities UVAL worked on
• Drafting national Guidelines (2001-2002)
• Efforts to open up the market (2001)
• Setting up networks and discussion groups (from 2001)
• Creation of evaluation units (2000-2002)
• Meta-evaluation activities (from 2002)
• Self assessment exercises (2002-2003)
• Utilization of evaluation, evaluative knowledge, and implicit knowledge in discussing strategic lines for next programming period and for nat’l interventions (2005)
• Orientation to ex-ante evaluation process and products and organization of training and discussion groups (2006)
Open Days, Rome 3 July
UVAL role in the evaluation of Ob.1 CSF
•Organizing the evaluation:
• Suggested four segments (main contract, environment, cultural resources, macro model)
•Evaluation questions:
• Collected MA’s point of view• Drafted Terms of Reference• Dealt with EC, evaluator, MA
•Steering Group:
• Read and commented report drafts and reports• Informal dealings, formal meetings and feedback• Protected evaluators’ independence
Open Days, Rome 3 July
A part of the Network of Italian Evaluation Units
• Not ALL evaluation units:
– Evaluation Units dealing with Development Policies– Within executive branch – Some Ministries and all Regions– There are others: Regional elected assemblies,
Universities, other Ministries.
• Mix of tasks (actually evaluation only a small part)
– ex ante analysis, project appraisals, feasibility studies– direct evaluations (only one so far)– management of mid-term evaluations – Various roles in programming activities– Interaction with local authorities and coalitions on projects
• UVAL plays a proactive role in the Network, but has no formal “central” role.
Open Days, Rome 3 July
The National Evaluation System
– UVAL – Ministry of Economy and Finance (Coordinator)
– INEA – Ministry for agricultural policies– ISFOL – ESF Evaluation Unit – Ministry of
Labour– Evaluation Units of MA’s
Initially built to deal with evaluation of Objective 1 regions’ strategy and OPs
Now on the way to be extended to all Italian Regions’ regional policies
Open Days, Rome 3 July
External : Mid-term evaluation of CSF and OPExternal : Mid-term evaluation of CSF and OP
Internal:Internal:
Evaluation of the performance reserve Evaluation of the performance reserve systemsystem
Thematic evaluations (social inclusion, urban Thematic evaluations (social inclusion, urban development, research)development, research)
Self-evaluationSelf-evaluation
Different ways to organise evaluation
Open Days, Rome 3 July
EVALUATION OF EVALUATIONS, MID-TERM EVALUATION, UP-DATE
•Design•Early Reports
•Fieldworks•Intermediate Products and in-depth studies
•Fieldworks•Evaluation Questions•Organization of evaluation update•EC reads Evaluation reports•SNV reads reports for lessons and utilization
•Fieldworks•Updating Mid-Term Evaluation Reports
•SNV reads intermediate evaluation products •Working in team with Ob.1 Units (Questionnaire and Meeting in Oct).
EV
AL
UT
ION
O
F
EV
AL
UA
TIO
NS
MID
-TE
RM
E
VA
LU
AT
ION
OF
OP
an
d C
SF
Mid-Term Evaluation Report
20062005200420032002
Up-dating Mid-term :Strategy of CSF
•SNV reads preliminary products•Work with Ob.1 Units •Methodological talks SNV – Evaluators – MA’s – Units•SNV makes reports to CSF’s MC