State Planning Organization

Internship Programme in UVER

State Planning Organization DG Regional Development and Structural AdjustmentDepartment of Monitoring and Evaluation

5-16 November 2007

Zeynep ERSAYINVolkan İdris SARI

State Planning Organization

Content

Definition of monitoring

Overview of major studies in Monitoring and Evaluation Department of SPO

Internship Subjects

Perspectives

State Planning Organization

Definition of monitoring (parallel approach)

Monitoring can be defined as collection, analysis and use of information for the purpose of management and decision-making systematically. Monitoring tracks the actual performance or situation against what was planned or expected according to pre-determined targets (CMM, 2006)

Continuous function that uses the systematic collection of data on specified indicators to provide management and stakeholders of an ongoing intervention with indicators of the extent of progress and achievement of objectives and progress in the use of allocated funds (OECD, 2002)

State Planning Organization

Regional Programme Implementations and RDAs

İzmir

Çukurova

State Planning Organization

Overview of major studies in M&E Dep. of SPO

Ongoing studies:

Design and coordination of the monitoring system for EU Regional Development Programmes (RDPs)

Establishment of monitoring and coordination of the monitoring system for RDAs

Design of the monitoring system for Public Investments (small-scale, local)

Contribution to design of the monitoring system for National Development Plan (large-scale, national)

Future studies:

Effectiveness evaluation at project level

Common monitoring system: programmes designed, implemented and monitored at regional and local level.

Review of the project fiche to incorporate future plan

Forecasting system and database integration

State Planning Organization

Internship Subjects

Data Collection

Data Integration

Monitoring and Information System

Project Fiche

Data Anaysis

Project Monitoring and Reporting

Verification of Public Investments

Data Interpretation

Benchmarking and Ranking

Effectiveness Evaluation

State Planning Organization

Data Collection

Italian case Turkish case Opportunities Data collected from four main different DBs (APQ, Monit, AVCP, Public tenders) Importance of the data quality

New DB for the Public Investment Projects No effective monitoring system for public investments For EU programmes, monitoring system based on declaration and validation SPO is extending this system for public interventions

Italian experience can be “the good example” for the development of Turkish system

Standardized data collection and

systematic data analysis

State Planning Organization

Data Integration

Italian case Turkish case Opportunities Integration of data collected at project level and tender database Relevant data collected from different actors for further analysis Statistical tools and definition of unique key field Textual analysis of project titles

M&E Department intends to integrate data at central, regional and local levels Tender DB is maintained by Public Procurement Institution No integration yet

Integration of the different DBs increases the quality, variety and reliability of the analysis

State Planning Organization

Monitoring Information System

Italian case Turkish case Opportunities EU Funds APQ Effectiveness Evaluation Completamenti Integration of systems

Not functional for public investments yet Common monitoring system for EU RDPs MIS for KÖYDES (Village infrastructure) System design for monitoring of Public Investment

• System for Project proposal launched

• System for provincial projects designed

• System design for public investments ongoing

Importance of the integrated MIS

State Planning Organization

Project FicheSec. Italian case Turkish case Differences

1 Project Details Project Details New fields can be added

2

Time scale Planning stages, other approval stages, other activities, Implementation stages

- Will be incorporated

3 Economic plan ExpendituresFuture perspective will be incorporated

4 Financial plan Financial planAvailability of data on sources of funds

5Accounting data

Liabilities, Payments, Savings

- Will be considered

6 Targets Indicators Similar

7 Other information

8 Problems and needs

State Planning Organization

Data Analysis

Italian case Turkish case Opportunities Several types of analysis made by UVER Promotion of different aspects of projects and programmes at different levels Territorial analysis and spatial interactions

Short-run: this kind of studies can be applied to EU RDPs within the limits of available data and procedures

Long-run: Can be applied in Turkey only after developing a sound data collection and monitoring system for public investments

Integrating different DBs and using other source of data enable the monitoring and evaluation unit:

• to understand past behaviors of the system

• to produce analyses for identifying problems and needs in current situation

• to promote possible future interventions

State Planning Organization

Project Monitoring and Reporting

Italian case Turkish case Opportunities Continuous data collection with project fiche (core data of the system) Low quality problem in early stages of the system DPS more focused on data collection and procedures Monitoring has to have financial outputs

Project reporting without IS, on paper for public investment New system is on progress Lessons learnt in EU Programmes (KÖYDES, Provincial investments) Similar data quality problems Discussions on monitoring of the Annual Investment Plan

Review of the project fiche During discussion M&E Department is designing data collection and monitoring mechanisms An asset for the design process

State Planning Organization

Verification of Public Investments

Italian case Turkish case Opportunities Two-level control in the verification of the public investments Every single programme under EU funds has MA, CA and AA. UVER is an AA for some of the National Operational Programmes

Different management structure of the EU funds in Turkey For grants: first level of control - PIUs and CFCU Overall control: Treasury Auditors Ex-ante approval: ECD For other tenders: verification and auditing procedures are similar to Italian case

For the Instrument for Pre-Accession 2007-13, verification procedures will be very similar with Member states

State Planning Organization

Data Interpretation

Italian case Turkish case Opportunities Forecasting system Early-warning system

Data interpretations are limited to current data collections Not allowing forecasting For EU programmes, because of the nature of the system there are no need for forecasting

Short-run: forecasting for the DAs projects

State Planning Organization

Benchmarking and Ranking

Italian case Turkish case Opportunities Producing inputs to decision makers for the evaluation of the public administration According to developments in methodology, the analysis will be carried out periodically within the limits of data sets Rating of the administrations Three group of areas for the ranking: critical one is institutional contest

New concept was developed with strategic planning perspective for the evaluation of the public bodies There is no specific analysis on the project implementation performance of the administrations

Partnership between different institutions during data collection

State Planning Organization

Effectiveness Evaluation

Italian case Turkish case Opportunities Based on outcomes and results of the projects by Eff. Area of UVER Integrated information system for monitoring and evaluation Positive and negative assessment Problems in performance indicators and baseline analysis Differences between EU and national evaluation perspective

No ex-post evaluation for the completed projects under investment plans Final evaluation of the EU RDPs will be carried out Similar problems in baseline analysis and selection of performance indicators

Project- and programme-level approaches to the evaluation Performance indicators can be shared Future studies can be focused on this specific issue

State Planning Organization

Perspectives

Presentation of project work in “Final Conference”

Dissemination of internship experiences within SPO

Setup of a contact list of participants for exchange

Definition of a project fiche for future collaboration between SPO and DPS

State Planning Organization

GRAZIE PER TUTTO...ARRIVEDERCI…

Zeynep ERSAYIN – ztokoglu@dpt.gov.tr

Volkan İdris SARI – isari@dpt.gov.tr