interventions 2000-2006 ISPA) - Work Package C

European Commission DG REGIO

Ex post evaluation of cohesion policy interventions 2000-2006 financed by the Cohesion Fund (including former ISPA) - Work Package C - Cost benefit analysis of environmental projects

Final Inception Report

March 2010

European Commission DG REGIO

Ex post evaluation of cohesion policy interventions 2000-2006 financed by the Cohesion Fund (including former ISPA) - Work Package C - Cost benefit analysis of environmental projects

Final Inception Report

March 2010

COWI A/S

Parallelvej 2

DK-2800 Kongens Lyngby

Denmark

Tel +45 45 97 22 11

Fax +45 45 97 22 12

www.cowi.com

Document no. D1

Version 0.2

Date of issue 27/03/10

Prepared BGHA/PGM

Checked TSKY

Approved JKP

Ex post evaluation of cohesion policy interventions 2000-2006 financed by the Cohesion Fund (including former ISPA) - Work Package

C - Cost benefit analysis of environmental projects

C:\Documents and Settings\TSKY\Desktop\Inception report revised\Revised_Inception_report_270310.docx

1

.

Table of Contents

1 Introduction 2

1.1 Purpose of Inception Report 2

1.2 Outline of Inception Report 3

1.3 Summary of proposal 3

1.4 Issues discussed at kick off meeting 8

2 Methodological considerations 12

2.1 CBA method considerations 12

3 Description of tasks 20

3.1 Task 0 Inception phase 21

3.2 Task 1 Selection of 10 major environment projects 21

3.3 Task 2 Ex Post analysis 25

3.4 Task 3 Assess CBA as a method 31

4 Management issues 35

4.1 Time table 35

4.2 Planning quality assurance 35

4.3 Conflicts of interest 37

4.4 External experts 37

Table of Appendices

Annex 1 Brief project descriptions completed for Project No 26 and Project No 36/37




2

.

1 Introduction

Background In January 2010 DG REGIO accepted COWI’s proposal in response to the call

for tenders by open procedure Nº 2009CE160CAT052 launched by DG

REGIO, European Commission. It regards the provision of services under con-

tract "Ex post evaluation of cohesion policy interventions 2000-2006 financed

by the Cohesion Fund (including former ISPA) - Work Package C - Cost bene-

fit analysis of selected environment projects" (Reference: REGIO.C.4/JS D

(2009) 680291).

Deliverable 1 This Inception Report is the first out of nine deliverables for the current study

under the contract.

1.1 Purpose of Inception Report

The Inception Report provides feedback on all the issues raised and discussed

at the kick off meeting (KOM) with DG Regio on 13th January 2010 in re-

sponse to the proposal submitted by COWI A/S.

In particular, the Inception Report includes:

• A further elaboration of the methodology to be applied in assessing first

the 20 selected projects and then in more detail 10 selected projects as

compared to what was initially included in the COWI proposal. Identifica-

tion of the types of primary data to be collected at project level is ad-

dressed based on the brief project descriptions (Task 1) and further elabo-

rated through the pilot cases (Task 2.0). Likewise further considerations

concerning other information sources are included in the Inception Report.

(Chapter 3)

• A first test of the template to be used for the brief descriptions of the 20

selected projects; the test includes a thorough description of Project No 27

(2000HU16PPE003) and Project No 36/37

(2005SI16CPE001/2004SI16CPE001), enclosed in Annex 1




3

.

1.2 Outline of Inception Report

Chapter 1 presents a summary of the proposal, including the objectives and the

background for the evaluation. Further, the chapter presents the purpose of this

deliverable and comments the issues discussed at the kick off meeting.

Chapter 2 provides an elaborated discussion on various methodological consid-

erations to be taken into account while conducting the study.

Chapter 3 provides a detailed presentation of the tasks and sub-tasks to be un-

dertaken during the study, including timing, manning and reference to deliver-

ables.

Chapter 4 addresses other management issues such as an updated timetable,

presentation of the experts, and conflicts of interest.

Annex 1 includes brief descriptions for two pilot cases, namely the Hungarian

project (No 27) and the combined Slovenian projects (No 36/37), based on the

template for brief descriptions of 20 selected projects.

1.3 Summary of proposal

This section will present a brief summary of the proposal, as an introduction to

this study.

1.3.1 Study objectives

The Terms of References specifies the objective of the project in a very clear

manner: it is “to carry out ex post cost-benefit analyses for selected environ-

ment projects financed by the Cohesion Fund and ISPA, and to learn from these

analyses for the current and future programming periods". This objective is fur-

ther elaborated via the formulation of three fundamental questions, which are:

• What were the impacts of the examined projects?

• How can ex post cost-benefit analyses contribute to the practice of ex

ante cost-benefit analyses?

• What are the potentials and limits to carry out an ex post cost-benefit

analysis to identify and/or analyse the impact of projects? Is it an ap-

propriate tool for impact analysis?

We acknowledge that these three fundamental questions in practice imply a

double objective of the project: the first objective is to deliver actual cost bene-

fit analysis (CBA) results for ten environmental infrastructure investment pro-

jects, while the second objective is to deliver an assessment of the usefulness of

the CBA method in the given context - and in this context, to provide recom-

mendations for its future use. Figure 1-1 below illustrates this understanding of

the project objectives.

Two objectives




4

.

The figure highlights the preparation of CBA of 10 projects (the first funda-

mental question) and the assessment of CBA as a method (the second and third

fundamental questions). The project is proposed to consist of four tasks - Tasks

0-3 - where the two last tasks are crucial for the preparation of CBA of 10 pro-

jects and also the assessment of CBA as a method (see Chapter 3 for further

details of the tasks).

Figure 1-1 Project overview

In addition, the figure also demonstrates the interrelation between the tasks and

the selection of the ten projects for detailed analysis, and the process of assess-

ment of CBA as a method.

1.3.2 Background of the study

When assessing the ex ante CBAs for the 10 selected environment projects

within this evaluation and when providing ex post CBAs, it is of course central

to understand the needs of the Cohesion Fund and ISPA officials as well as

beneficiaries for providing CBA results. These needs are well-described in the

Terms of References, where it clearly is stated why the European Commission,

Directorate-General for Regional Policy is undertaking an ex post evaluation of

cohesion policy interventions financed by the Cohesion Fund (including the

former Instrument for Structural Policies for Pre-accession (ISPA) during the

period 2000-2006 in 17 countries (Greece, Ireland, Portugal and Spain, Cyprus,

Cohesion Fund and

ISPA objectives and

needs




5

.

Czech Republic, Estonia, Hungary, Latvia, Lithuania, Malta, Poland, Slovakia

and Slovenia, Bulgaria and Romania, and Croatia).

The ex post evaluation is an important instrument to inform national and re-

gional authorities, the general public, the European Parliament and other stake-

holders involved about the outcomes of the 2000-2006 cohesion policy inter-

ventions. The evaluation will establish the effect of the Cohesion Fund and

ISPA on economic and social cohesion and lessons for the future. Results and

interim results of this evaluation will be used for the discussion of a next pro-

gramming period of cohesion policy.

The overall ex post evaluation will consist of a set of three interlinked evalua-

tions (in the following: "work packages"):

1 Work package A: Contribution to EU transport and environment policies;

2 Work package B: Cost benefit analysis of transport projects;

3 Work package C: Cost benefit analysis of environment projects.

As stated above, this evaluation regards work package C (CBA of environ-

mental projects) of the ex post evaluation of the Cohesion Fund and ISPA.

In addition to understanding the needs for CBA results, it is also important for

carrying out the CBAs to understand the objectives of the Cohesion Fund and

ISPA - i.e. the reasons for providing funding. Table 1-1 provides an overview

and compares the main features and differences between the two financial in-

struments.

Two financial in-

struments




6

.

Table 1-1 Main features of Cohesion Fund and ISPA

COHESION FUND ISPA

Eligibility and conditionality

MS with GNP less than 90% of EC average Convergence programme (Art 2.4) Conditional assistance (Art 6) Mid-term Review

Candidate Countries of Central and Eastern Europe

Overall allocation

€2.8 billion p.a. 2004-06, equal to 38 Euro per capita p.a.

€1.4 billion p.a. 2004-06, equal to 11 Euro per capita p.a. (Romania & Bulgaria)

Co-Financing Ratio (subject to revenue-generation and other financing)

Grant Rate 80-85 %, on projects of min €10 million (except in duly justified cases)

Grant Rate 75 % (85% as exception), on projects of min €5 million

Expenditure eligibility

On receipt of complete application includ-ing: land purchase costs, VAT in limited cases, pursuant to new Commission Finan-cial Regulation 16/2003

At the date of signature of the Financing Memorandum (FM)

Decision Procedures

Commission Decision Commission Decision and Signature of FM, After Favourable ISPA Committee opinion

Implementing Procedures

Fully Decentralized to Member States

• National law based on EC pro-curement Directives

• Ex-post control by the Commis-sion

Partly Decentralised (EC Delegations)

• External Aid Rules (PRAG)

• Ex-ante approval

In the period 2000-2006, the Cohesion Fund and ISPA co-financed 750 envi-

ronment infrastructure projects, totalling €15.8 billion of investment. These

projects contribute to the achievement of the objectives of Community policy

of environment, in line with the priorities of the environment action pro-

grammes. For all these projects an ex ante cost benefit analysis are required to

be carried out. Each project was prepared by the Member States and adopted by

the Commission.

In the Terms of References a project list included 40 environment projects fi-

nanced by the Cohesion Fund and ISPA. This is the gross list of projects among

which the ten projects that will be further analysed must be selected. Among

the projects a vast majority are wastewater or combined water and wastewater

projects - all highly prioritised areas in the EU environmental policy. Nine of

the projects relate to solid waste projects. The projects are implemented in

Southern Europe (Spain, Greece, and Portugal), Central Europe (Ireland) or

Eastern Europe (Bulgaria, Czech Republic, Estonia, Latvia, Lithuania, Poland,

Romania, Slovenia and Slovakia).

The environmental basis is therefore focussed on integrated water management

and solid waste while the geographical scope is very broad. Projects within

solid waste, water and wastewater are infrastructure projects generating direct

The original list of

40 projects - Inte-

grated water man-

agement and solid

waste




7

.

environmental services such as handling and/or treatment of waste and provi-

sion of facilities to clean discharged water and/or to enable a public supply of

drinking water. Major strategic concerns in the feasibility study and the appli-

cations should be the investment alternatives. Some technologies requires rela-

tively low capital costs but are expensive during the operation phase (e.g.

chemical treatment of wastewater) other alternatives are expensive upfront but

connect with low operation costs. A relevant social and economic aspect is to

create affordable charges of the publicly supplied environmental services.

Therefore willingness to pay analyses are important studies connected to the

cost-benefit analysis for environmental infrastructure projects.

A major challenge - that will thoroughly be dealt with during the present pro-

ject - arise from analysing the infrastructure projects in the CBA framework,

We will have to deal with non-monetary effects connected to environmental

impacts, recreational areas and positive and negative side effects of the envi-

ronmental service. Hence, the present project will assist the Commission in as-

sessing the already implemented projects both in financial Terms of References

but also with respect to the positive and negative benefits which are difficult to

quantify in monetary Terms of References. Further discussions of these chal-

lenges are presented in Chapter 2.

1.3.3 Methodology and tasks

As presented above in figure 1-1 Project Overview, then the study was pro-

posed to consist of four major tasks, consisting of:

• Task 0: the inception phase,

• Task 1: Selection of 10 major environmental projects,

• Task 2: the Ex Post project analysis, and

• Task 3: the assessment of the CBA method.

The following chart displays the major outputs and deliverables expected from

these four tasks.




8

.

Figure 1-2 Major outputs from the four tasks

Details of each task and sub-tasks are presented in Chapter 3.

1.4 Issues discussed at kick off meeting

The following table provides an overview of all the issues raised by DG Regio

at the kick off meeting and a short clarification on the issues. Relevant issues

are further detailed in the following chapters.

Table 1-2 Issues addressed at kick off meeting

Issue raised at Kick of

Meeting (KOM) Clarification of issue

Introduction

1. Client's success crite-

ria

An overall success criterion of DG Regio is that the project deliverables are of high

quality. This concerns both analyses, methods applied, and the report layout, language

etc. Hence, it is of utmost importance that the procedures of project management and

quality are fully complied with − e.g. that sufficient time for quality assurance of all de-

liverables is given.

The specific success criteria are that 10 CBAs are provided for the selected environ-

ment projects that the CBA method originally used for such projects is assessed, and

that recommendations for improving or using the CBA method are provided.

DG Regio is in general seeking to upgrade its capacity to carry out evaluations. This

can be seen in the light of the increased focus of EC President Barroso's new Com-

mission on accountability. Such upgrading will require increased experience with

evaluations and the exploring of new and better ways to do this.

(Ref. Project Management and Quality Plan (PMQ Plan) p. 3)

2. Overview of COWI

team roles and re-

sponsibilities

The following figure displays the organisation and hierarchy among actors relevant for

the project.




9

.



It should be noted that some of the of the core team consultants of the proposal have

been replaced by others. We have replaced Arne Bernt Hasling, Bodil Bjerg, Carsten

Skov, and Niels Bisgaard Pedersen with Szabolcs Szekeres (external consultant to

COWI A/S and CBA expert) and Tine Skyggebjerg, COWI A/S as project assistant. In

this manner a very focused and dedicated core team has been established.

The project manager, Birgitte Holt Andersen, interfaces with the Project Officer, Adam

Abdulwahab of DG Regio. Peter G Madsen will be the technical coordinator and hence

lead all tasks related to the CBA methodology.

(Ref. Project Management and Quality Plan p.3 and 7)

Task 1

3. Content of CD/material

received on 20 pro-

jects

Status of material received from DG Regio.

Received from DG Regio Comments

DVD received at KOM (13/1/2010) sup-posedly to contain all material of the origi-nal 40 projects

No ex ante material was included for any of the 40 projects

DVD received 28/1/2010 for 11 of the 20 selected projects

Nearly completed material for 11 projects

Status of additional 8 projects We have received material for all 19 projects.

Romania was excluded from the list of 20 projects without a substitute, therefore the

total list is 19 projects.

4. Selection of 20 pro-

jects

Done and agreed

The final list of selected 19 projects is presented under sub-task 1.1. The list agreed on

a the KOM was subject to changes after Steering Group Meeting on 3 March , and

these alterations are included in this updated list.

5. Project description

template

2 examples of project descriptions are included (Project 27 and 36/37). For details see

annex.




10

.



6. All elements men-

tioned in ToR to be

included in the tem-

plate, e.g. the source

of information

Elements are included in project description template. Additionally to first draft versions

more details on financial issues and short review of available material will be included

as well as contact details for the projects.

7. Financial overview

should include other

sources of IFI

It is included in project description template

8. Project descriptions to

include contact person

(s) in the organisation

responsible for the

project and the result

of first contact to the

project

It is included in project description template

9. Role of scoreboard

unclear

The scoreboard has been replaced by a more pragmatic approach to the selection of

the ten case studies. For instance, a key criterion for selecting cases is availability of

information, which is checked during the elaboration of the short profiles. The approach

is described in detail in chapter 3.2 under Task 1.2

Task 2

10. Option/technological

solutions: it would be

interesting to have this

ex post, independently

of options analysed ex

ante.

Pitfalls: (1) availability

of cost of technologies

at the time of decision,

(2) spending too much

effort

Improvements of the ex-ante CBAs will look into the alternative options to the one se-

lected by the project applicants, and in this context look for possible "donor biases" −

i.e. a possible tendency that capital-intensive solutions are chosen in favour of less

capital − but more operational cost-intensive projects. Such analysis of alternative op-

tions will happen, to a large extent, as an integrated part of the ex-post analysis (It is to

be undertaken during Task 2.2).

In practice, the quality assessments of the ex-ante CBAs will therefore distinguish be-

tween the original ex-ante CBAs and the "improved" ex-ante CBAs.

(Ref. Sub-Task-Description 2.1-C; Also, this issue is addressed in the project descrip-

tions)

11 Qualitative scoring

system: DG Regio

prefer quantified re-

sults and causal links if

monetisation is not

feasible

Acknowledging the exploratory nature of the analysis − hereunder when specifying the

impacts of water and solid waste projects − particular attention will be paid to transpar-

ency regarding the use and development of the CBA methodology. Furthermore, build-

ing on both quantitative and qualitative assessments implies a need to present the

combined results in a narrative way − in addition to the calculations. The approach to

do this will be a part of the internal guidelines developed in Task 2.0.

(Further details in description of sub-task 2.1-C)

12 Unit costs issues:

WP10 methodology

should be further re-

fined. An distinction is

required between ac-

tual and estimated

figures

We will establish unit costs for the 10 projects, i.e. disaggregating costs to the different

types of key components ("level 3" costs). In carrying out this task, we will apply the

methodology developed by Work Package 10 of the ex-post evaluation of the ERDF in

the 2000-06 programming period.

(Ref. SubTask-Description 2.2-C)

13 Adam Abdulwahap

(PO of DG Regio) to

participate in one of

the mission for the

Noted




11

.



Pilot Projects

Task 3

14 Task 3 is less devel-

oped that the other

parts of the offer. The

task will require un-

usual expertise and is

more than drawing

information on the 10

project's experience.

COWI needs to elabo-

rate how we intend to

provide answers to the

questions in ToR

In accordance with the ToR the assessment will cover:

• The strengths and weaknesses of the CBA methodologies applied by the Member

States in general, i.e. in additions to the ones that guided the ex-ante CBAs.

• Effectiveness of CBA as a tool supporting the project generation and project deci-

sion of Member States and Commission in general.

• Utility of ex-post CBAs from the point of view of project holders, Member States

and the Commission in general.

• The extent that ex-post CBA is an appropriate tool for evaluating impact.

• Rigorous treatment of environmental costs and benefits that cannot be expressed

in monetary terms, including how they can be taken into account in decision-

making − based on finding in the identified material.

• Relevance and potential utility of data obtained from CBAs in macroeconomic

modelling.

Furthermore, the assessment will address the possible links between CBA methodolo-

gies and other tools for environmental assessments, such as EASEWASTE − a PC

tool for environmental assessment of waste systems developed by the Technical Uni-

versity of Denmark based on life cycle assessment principles.

(Ref. Task-Description 3.3-C)

Next steps and Project Plan

15 Confirmation of time-

table The updated timetable is presented in Section 4.1.

Inception Report

16 Data collection primary

data: what data and

how to collect it

This is addressed in the project descriptions

17 Inclusion of Opera-

tional management

plan incl.: responsibili-

ties of each member of

the consortium, how to

deal with language

issues and who is car-

ing out the work at the

ground

Details are elaborated in relevant sub-tasks-descriptions, see Chapter 3 for details.

18 QA plan Quality assurance is included in the planning of the project. Details on the issue of

quality assurance is addressed in Chapter 4.2.

Other issues

19 Liason with other WPs Initial contact has been established with WP A, Project Manager Tom Robinson of RGL

Forensics.

20 External experts Three experts have confirmed their participation in a expert group (Ref. Chapter 4.3)




12

.

2 Methodological considerations

As the inception of the study has progressed, the study team has further re-

viewed and discussed the methodology for conducting CBAs within the envi-

ronment sector and a detailed understanding of this is presented in this chapter.

2.1 CBA method considerations

The discussions that follow address the peculiarities of the CBA methods of

analysis in relation to the specific project types under consideration. Hence,

these sections will not cover the standard aspects common to all CBAs, e.g. the

requirement to make forecasts with and without the project, to use shadow

prices where required, or to use discounting. All elements in the Commission

CBA Guide will be observed.

Waste is by-product of normal life. What can we do with it? We could pile it

up in our backyard, but that might have unpleasant consequences. The pile of

waste would soon become an unpleasant eyesore, and also a possible health

hazard, by being the source of bad smells, and, in some climates, a magnet for

insects. The runoffs could threaten the soil and groundwater. For this reason

people have preferred to carry their waste to disposal lots far from their own

backyard, but not so far so as to make the transport too expensive.

But this merely shifts the problem from the backyard of individuals to the by-

ways of their villages or towns. The problems of aesthetics, smells, runoffs,

and disease vectors remain, but at different locations. The solution to these

problems is offered by modern solid waste projects that seek to dispose of

waste in a sanitary way, and avoid the unpleasant external effects, as economi-

cally as possible.

The above description guides us through the formulation of the CBA method-

ology of solid waste projects. Some key observations can be made:

1 There is a willingness to pay for the removal of waste. People prefer to

pay to have waste removed, rather than having it in their backyards. Pay-

ment can be in cash, or through the effort of carrying the waste oneself.

However because considerable external costs are involved in solid waste

handling and disposal, a focus only on willingness to pay falls short of

measuring the full benefits of a project.

Solid waste projects




13

.

2 Properly designed and operated solid waste projects will reduce or elimi-

nate external environmental costs (aesthetics, smells, soil and water con-

tamination). These have to be considered in the CBA.

3 In some cases solid waste projects may earn revenues through recycling

waste or energy generation.

4 Solid waste projects have capital and operating costs, which may include

costs directly borne by clients, such as packaging (typically plastic bags)

and sometimes transport to collection points.

It is rare to find a solid waste project with significant health benefits, because

people generally manage to remove waste far enough from their immediate sur-

roundings to prevent exposure to health hazards. Two aspects are important

when designing and evaluating a solid waste project:

- Cost minimization, through the comparison of alternatives in dimen-

sions such as choice of technology, geographical area of coverage, and

timing and phasing.

- Quantification of the externalities associated with the project alterna-

tives. These are aesthetics, smells, and soil and groundwater contami-

nation. The quantification of these externalities can be achieved

through the techniques of hedonic pricing or contingent valuation, or

by costing remediation measures.

Data are needed for the project, for the situation without the project, and ideally

for one or more project alternatives, in the following categories, which can be

further subdivided as needed, depending on the degree of detail available:

- capital and operating costs;

- revenues from recycling;

- externalities.

The analysis of waste water projects is very similar to that of solid waste pro-

jects. For wastewater originating in households, the technological alternatives

are: (1) connecting to a sewage system or (2) the construction of an on site

sewage facility. Industrial waste-water sometimes must be treated separately

from municipal sewage systems because that allows the treatment to be tailored

to the characteristics of the waste water generated.

The analysis of external costs is important for this type of project, and it needs

to be conducted for all considered alternatives. Even the proposed projects

themselves could have external costs, albeit, it is hoped, lesser ones than that of

its alternatives. The external costs usually relate to the discharge from the pro-

ject (liquid or sludge disposal), and will vary depending on the number of

stages of cleaning applied (primary, secondary, or tertiary). The external costs

Waste water projects




14

.

are incurred trough the project alternatives’ effects on bodies of water, on the

land, or on the atmosphere.

This type of project could in some cases generate direct benefits as well, if it

can produce treated water for irrigation or sludge for soil treatment. These liq-

uids contain nitrogen, phosphorous and potassium, which are valuable fertiliz-

ers. But water and sludge must be treated to keep toxic or infectious compo-

nents below threshold values. Consequently, it could happen that they have no

commercial value.




- Capital and operating costs

- Revenues from recycling

- Externalities

The consumption of potable water involves fewer externalities than waste dis-

posal. For this reason willingness to pay by users plays a larger role in the

analysis of this kind of project, as in this case willingness to pay captures a high

proportion of the benefits enjoyed by users.

Despite this, the comparison of alternatives is still the key element of analyses

of this project type as well, given that projects or their alternatives seldom re-

sult in great changes in the quality or quantity of water consumed (the only

cases in which willingness to pay play a role in the analysis).

The externalities that might have to be considered are those that would be gen-

erated in providing and using the service. The construction and operation of

water supply systems could have adverse environmental impacts, and the dis-

posal of used water by consumers could create external effects, especially in the

absence of adequate sewage services of some kind. The health consequences of

potable water projects are seldom an explicit factor, for alternative methods of

providing potable water nearly always exist.

Water is often a scarce resource, and its opportunity cost must be estimated.

It is usual to review tariff policies when analyzing potable water projects, be-

cause when tariffs fail to reflect true costs, demand for water will be larger than

it would otherwise be, which might in turn be the main driver for the project.

So setting the tariffs right could indeed become an alternative to the project,

and needs to be looked at.

It is also customary to review water losses in potable water systems, for stem-

ming such losses might be more economical than building new adduction

works. However, donor bias often conspires against this option.

Potable water pro-

jects




15

.




- Capital and operating costs

- Willingness to pay for changes in water consumption.

- Externalities

2.1.1 Quantification of environmental externalities

Assigning monetary values to external costs to projects in the fields of waste

disposal, waste water and potable water usually concern damage to soil, water

or the atmosphere. These can be quantified directly or indirectly.

The direct methods are:

- Hedonic pricing. Under this method the external cost is quantified by

its observable effects on something that is affected by the externality

and that has a market price. The most typical example is the reduction

in the value of houses that are subject to odours near a waste treatment

site. The value of the externality can be established econometrically.

- Contingent valuation. Under this method members of a sample of af-

fected people are asked about their willingness to pay for the removal

of an external cost that affects them. The sample is stratified, and dif-

ferent strata are confronted with different hypothetical costs. The re-

sponse rates can be used to derive a monetary value for the external-

ity.

Indirect methods An indirect method is estimating the cost of remediation that removes the

external cost. This method has an important weakness, however: it relies on the

(untested) assumption that the cost of remediation is less than the cost of the

externality.

It will be impossible to conduct studies of this kind during this ex-post evalua-

tion study. But omitting the quantification of external costs would weaken the

analysis of those projects that result in a perceptible change in external costs.

The only practical solution to this is to use the method of “benefit transfer”.

“Benefit transfer” is the method of attributing to a project benefits calculated

for a similar project elsewhere. To apply this method care has to be taken to:

- Ensure that the externality is comparable.

- Make adjustments to systematic differences in willingness to pay (this

is important if the quantification of the externality was done in a dif-

ferent country.)

Direct methods




16

.

- Scale the results as needed: take into account the number of people

exposed, the size of the area affected, etc.

It will be necessary to apply this method for projects in which it and its alterna-

tives differ perceptibly in the extent of external costs caused or avoided. To

this end it will be necessary to:

- identify studies in the literature quantifying similar externalities; and

- identify project data with which to do the necessary adjustment and

scaling.

2.1.2 The impact of regulations

Potable water, solid waste and waste water projects are often subject to regula-

tion and standards that constrain the design of projects. These constraints re-

duce the feasible set of alternatives to those that are compliant with the regula-

tions in force. This must be taken into account when performing CBA of indi-

vidual projects.

A legitimate question might arise about the desirability of having such regula-

tions. To answer that question, of course, the cost of meeting the standards

would have to be compared with the cost of alternatives that are not compliant.

Non-compliant alternatives might differ significantly from the compliant ones

in both costs and benefits- But this is not the task at hand in the current study.

The current study only needs to be careful not to compare complying projects

with non complying alternatives.

2.1.3 Special considerations of Ex-Post evaluations

CBAs are forecasts of the future. An ex-post evaluation is usually conducted

after some time has passed since a project’s completion, and can serve either

the purpose of ascertaining the actual performance of the project, or to analyze

the quality of the original analysis, or both.

It is important to note that the two purposes require different approaches to un-

dertaking the ex-post evaluation. Therefore a single ex-post evaluation cannot

serve both objectives at once without adjustment. The evaluation done for one

of the purposes cannot automatically be used for the other.

The advantage of an ex-post evaluation is the benefit of (partial) hindsight. The

ex-ante evaluation is largely based on forecasts, which are subject to uncer-

tainty (which may or may not have been acknowledged at the time). The ex-

post evaluation has the benefit of being able to observe the actual values of

some variables, without any uncertainty (typically investments costs), and ob-

serve the trends of other variables (typically benefits and operating costs) that

reduces the uncertainty of the forecasts required for the rest of the useful life of

the project.

Objectives




17

.

If the purpose of the ex-post evaluation is ascertaining the performance of the

project, the task at hand is simply that of performing an appropriate CBA, using

the new, more reliable data. This is the simpler of the two possible purposes.

If the purpose is to assess the quality of the original analysis (beyond simply

checking if it followed the right methodology), the ex-post evaluation task is

more difficult. In that case it is not enough to compare the forecast to what ac-

tually happened, as that is an unfair test. The question that has to be answered

is whether the recommendation of the original study was correct or not on the

basis of the information available at the time. This distinction derives from the

fact that CBAs are forecast of uncertain events, and nobody can be required to

own a crystal ball.

Margin of error Not all studies acknowledge their expected margins of error, but a margin of

error is there none the less. It is a very rare project indeed that has a zero prob-

ability of being unfeasible. Almost no projects would ever be undertaken if

they were required to guarantee a positive return under all circumstances. It

does happen that events unfold in such a manner that a project with high ex-

ante probability of acceptable returns end up delivering an unacceptable one ex-

post.

For this reason, a low ex-post return is not necessarily proof of a wrong ex-ante

decision. To show that in regards to public sector projects, which are usually

evaluated under the assumption of risk neutrality, one would have to show that

with the information available at the time the expected value of the return indi-

cator (typically NPV) was unacceptable. This is a more difficult task, because

one would have to find out what information was available at the time when the

original CBA was conducted. (This comment refers more to the forecast of

variables required in the analysis, rather than to the methodologies of the origi-

nal analyses, which are easer to qualify.)

We interpret the current study to require both types of analysis, but with a

greater weight being placed on evaluating the actual consequence of having

undertaken the project.

We see that for all three project types the comparison of the costs of alterna-

tives is a key element of the analyses to be performed. This requires that the

following steps be undertaken for each project:

1. Determine what alternatives have been considered, and establish if they

have been reasonably well chosen.

2. If not, then reasonable alternatives to be used in the analysis will have to

be proposed.

3. Actual costs of the project undertaken will have to be ascertained. If the

data are unavailable or incomplete, they will have to be estimated.

4. The cost of actual or supposed alternatives will have to be estimated.

Methods




18

.

5. Data on utilization rates (or demand for the services) of the projects will

have to be obtained; this cannot be replaced by estimates. However, the

utilization rates (or demands) for the remaining useful time of the projects

will be estimated.

6. Similarly operating costs of the projects should be ascertained, but their

future values will have to be estimated.

Many of the estimates alluded to above will be engineering estimates. They

will be based on reasonable alternatives, taking acceptable quantities of mate-

rials and local unit costs values. Nevertheless, they will be subject to error.

2.1.4 Margins of error of the CBAs performed.

Even though some actual project data should be available in all cases, it should

be clear from the foregoing discussion that the CBA analyses will rely heavily

on estimates. To quantify the consequences of using estimated data on the con-

clusions of the analyses, the Monte Carlo simulation method will be used. This

will combine the uncertainties involved and derive confidence intervals for the

conclusions.

Monte Carlo simulation (MCS) is one technique that helps to reduce the uncertainty

involved in estimating future outcomes. MCS can be applied to complex, non-

linear models or used to evaluate the accuracy and performance of other models.

It can also be implemented in risk management, portfolio management, pricing

derivatives, strategic planning, project planning, cost modelling and other fields.

To this end the following steps will be followed:

1. Computational models will be constructed for the three project types,

allowing enough flexibility in them to accommodate variations in pro-

ject structure.

2. Sensitivity analyses will be conducted for each project to identify the

variables with greatest impact on the results of the analysis.

3. Margins of error will be estimated for all estimations made, and will be

represented by probability distributions. Very careful attention will be

given to choosing the margin of error of variables ranking high in the

results of the sensitivity analysis. Likely correlations between vari-

ables will be taken into account.

4. Using the Monte Carlo method the probability distributions of the pro-

ject indicators will be obtained. This will provide margins of error for

the conclusions.

5. The expected opportunity loss of each alternative will be computed

when more than one alternative to the situation without the project can




19

.

be quantified. This will assess the probability that each alternative

analyzed has of being the best.




20

.

3 Description of tasks

The following section presents the detailing of the task to be undertaken during

this study. For each task the purpose, the objective and the method is described.

It is detailed who will be the main persons undertaking the assignment and po-

tential risks will be discussed. It is envisaged that the description of these task

will be updated according to the progress of the study, as more information be-

comes available the task and approaches will be adjusted accordingly. This in-

formation is available in a dynamic document, the Project Management and

Quality Plan that is used as internal steering tool. This document can be made

available as documentation to the Commission upon request.

The following figure displays and overview of the tasks to be undertaken.

Figure 3-1 Overview of tasks to be undertaken

As described in the introduction, the objective of the assignment is twofold.

Firstly, it will deliver an actual cost benefit analysis (CBA) results for 10 se-

lected environment projects − within the topics: water and solid waste − fi-

nanced by the Cohesion Fund and ISPA during 2000-2006. The results will

both be used for evaluating the impacts of the respective projects, and they will

be used as lessons for the design and selection process of similar projects in the

next programming period of cohesion policy. Secondly, the evaluation will as-

sess the usefulness of the CBA method in the given context.

The scope of work to fulfil these two objectives has both individual and over-

lapping features, which have been considered in the refinement of the sub-

tasks.




21

.

3.1 Task 0 Inception phase

Task

No

Name of Task Start End Responsible (in bold) Days

0 Inception Week 2 Week 6 Birgitte Holt Andersen 6

Including subtasks Peter G. Madsen 3

Szabolcs Szekeres 3

Tine Skyggeberg 5

Raphael Zayat 2

Csil 3

Total 22

A Objectives

To start up the project and to achieve a firm mutual understanding of the objectives, method and

expected output of the evaluation. Furthermore, it is an aim to select 20 out of 40 projects for fur-

ther review − giving preference to projects that are completed, while also ensuring coverage of

the topics water and solid waste, and of both Cohesion Fund countries and ISPA countries.

B Work to be performed

Kick of Meeting (KOM)

Selection of 20 projects − out of 40

A 3 page description of 2 of the 20 projects to test the '3-page template' and to provide further

insight as input to CBA methodology. Since we at present (8 February 2010) have not yet re-

ceived all necessary available information for the projects financed by the Cohesion Fund, 2

ISPA projects have been selected: No 27 Hungarian water project and No 36/37 Slovenian solid

waste project.

Preparation of Inception Report reflecting updated and more comprehensive methodology.

C Methodology

Discussion with DG Regio at KOM.

First view at available material as provided by DG Regio concerning the 40 projects, but particu-

larly for the selected 20 projects.

Internal coordination, engagement with Consortium partners, brainstorming on methodological

challenges for Tasks 1, 2 and 3.

D Contingency (Identification of problem and proposed solution)

None foreseen.

E Deliverables & Meetings Due date

D1: Inception Report. 15 February 2010.

The three sub-tasks includes kick off meeting, quick appraisals of 20 projects

and an Inception Report has not been described separately, but their description

is included in the table above.

3.2 Task 1 Selection of 10 major environment projects

Task 1 concerns the selection of 10 environment projects from a gross list of 40

projects. This will be done in two steps. Firstly, 20 of the 40 projects will be

selected for brief assessments. This selection did already take place during the

kick-off part of the study based on the following criteria: all 9 completed pro-

jects are included, coverage of both water and solid waste projects, and cover-

age of both Cohesion Fund and ISPA beneficiary countries. Secondly, the 10




22

.

projects will be selected under continued consideration of the just-mentioned

criteria. Foremost, however, the final selection will be pragmatic and select

projects where it appears that good data are accessible. Hence, we will not

claim to have analysed a representative selection of environment projects.

Details of the sub-tasks are presented below.

3.2.1 Task 1.1 Project description

Task

No


1.1 Project descriptions for 20

projects

Week 5 Week 9 Birgitte Holt Andersen 2

Peter G. Madsen 4

Szabolcs Szekeres 4

Birgitte Grønbech 8

Tine Skyggeberg 7

Jonas Stravinskas 2

Raphael Zayat 4

Tecninvest 16

Local experts 6

Total 53

A Objectives

The objective of this task is to provide a brief description of each of the selected 20 water or solid

waste projects − mainly to serve as the basis for which to select the final 10 projects. In addition,

the descriptions will give some insight into the quality of the 20 project applications in general −

and particular the ex-ante CBAs.


1 Development of a common template for the brief project descriptions (a draft version of the tem-

plate is attached to Inception Report that also contains tests of it for 2 projects).

2 Assign responsible core team members and local correspondents (if appropriate) to each of the

20 projects (list enclosed on next page)

3 For each of the 20 projects:

• Check available material on CD provided by DG Regio and make a list of content.

• Completing project template based on available material.

• Identify project contact person and make a first contact to address further issues according

to the template.

• Complete brief project description according to the template.

C Methodology

The template should contain identification of the variables that will need to be analysed and data

collected for within Task 2.

The list overleaf indicates which team members are responsible for completing the brief descrip-

tion of each project.


Lack of, or poor quality of, ex-ante CBA data: We will live with a few incomplete project descrip-

tions, but need to have sufficient completed ones to make a good choice of 10 projects for Task

2.

Difficulties in getting access to project contact person: DG Regio should be able to get access to

a contact person through national representatives




23

.


D2: Project descriptions for 20

projects.

15 March 2010.

Project no. 34 from Portugal is highlighted with grey, as this project is elimi-

nated from list of the final 10. The reason for this was that this project has al-

ready undergone a thorough internal evaluation.

Figure 3-2 Overview of 20 projects

List of selected 20

projects




24

.

3.2.2 Task 1.2 Selection of ten major projects

Task

No


1.2 Selection of 10 pro-

jects


Peter G. Madsen 3


Szabolcs Szekeres 1

Csil 5

Jesper Karup Pedersen 1

Total 16

A Objectives

The objective is to make a decision on which 10 projects out of the 20 projects are to be further

analysed and assessed. The aim is to end up with 10 projects for which sufficient data on costs

and benefits are accessible, and the ex-ante CBA is complete enough to enable the ex-ante/ex-

post comparison. In this vein, a selection criterion is likely to address the projects' coverage of

alternative options to the chosen. In any case, the selection will not necessarily be representa-

tive.


1 Review the 20 brief project descriptions

2 Applying the selection criteria (se below) to obtain a proposal for the 10 projects to proceed with,

and discuss this proposal at an internal meeting (video conference).

3 Discuss the proposal with DG Regio at meeting scheduled for the end of March − leading to a

final selection.

C Methodology

The key judgment criterion for the selection of the 10 out of 20 projects is the quality of informa-

tion available and the completeness and quality of the ex-ante CBA. Without a good ex-ante

CBA the ex-post exercise is almost meaningless (at best, it is terribly expensive). A key outcome

of the brief project descriptions as completed in Task 1.1 is an overall and broad assessment on

the quality of the ex-ante CBA's, and a listing of key indicators and methods used. A second key

judgment criterion is the evaluation issue on which a specific ex-post analysis will shed light.

This is an important element for Task 3 of this project. Hence, the template includes a heading:

"objectives of a possible ex-post evaluation".

In making the final selection of the 10, we should obviously start by seeing which project have

enough information to allow us to do an ex-post evaluation with a reasonable amount of effort.

The ones that do should then be grouped by main methodological features to be tested. This will

make for a few groups, populated by no more than two-three projects. We will know them well,

having written briefs about them, and we will be able to select based on two main criteria men-

tioned above.


The risk is that it will be difficult to find 10 projects with sufficient data, in which case we will have

to select the relatively best ones and compensate through assumptions regarding the ex-ante

situation. Furthermore, there is a risk of deselecting good projects because there is poor ex-ante

information: this risk will be reduced via consulting the available progress reporting from the 20

projects.


Minutes of Meeting with DG Re- Meeting date + 1 week.




25

.

gio scheduled for end of March.

3.3 Task 2 Ex Post analysis

Task 2 concerns the ex-post − but also reviews the ex-ante − CBAs for the 10

selected environmental projects. The result fulfils primarily the first objective

of actually delivering CBAs, but the ease with which the results have been pro-

duced (and the difficulties encountered) will be a central element in the assess-

ment of the CBA as a method. The task is divided into four sub-tasks, where

the first is the pilot case analysis, the second is a revision of ex ante CBAs for

each of the ten projects, then the CBAs will be carried out and the final sub-

task is to compare the ex ante CBAs with the ex post CBAs. Details of the sub-

tasks are provided below.

3.3.1 Task 2.0 Pilot Case analysis

Task

No


2.0 Pilot case analyses Week 12 Week 17 Birgitte Holt Andersen 5

Peter G. Madsen 5

Szabolcs Szekeres 1


Tine Skyggeberg 2

Jesper Karup 1

Raphael Zayat 5

COWI water & waste experts 4

Csil 5

Local experts 0

Total 32

A Objectives

The objective of the pilot case analyses is twofold:

1) to review the ex-ante and carry out the ex-post CBAs for 2 of the 10 projects,

2) to test and resultantly revise/improve the methodology and guidelines according to the lessons

learnt.


1 Select two projects for pilot case analysis. The two cases should be selected from the project

types with the highest number of projects in the sample of 10, to maximize the usefulness of the

specific lessons to be learnt.

2 Finalise and agree on the methodological approach to be tested via the two pilot cases. This will,

for example, comprise the development of internal documents/toolkits such as data collection

guidelines, checklists and interview guides for local correspondents for the field studies that might

be necessary. While each of the 10 projects will have a method of analysis deigned specifically

for it, taking into account all of its aspects, the methodologies developed for these two projects

will be the models for the remaining projects of the same type, so attention will be paid to their

general usefulness. (Please see the descriptions of Tasks 2.1 to 2.3 for further)

3 Carry out the two sample CBAs, both updating the ex-ante analysis as far as feasible, and con-

ducting the ex-post evaluation.




26

.

4 Internal assessment by the project team of the results of the pilot analyses.

5 Revise/improve the methodologies employed, as needed, for use in Tasks 2.1 to 2.3

6 Revise/improve the CBAs for the 2 pilot projects if necessary (possibly requiring extra missions to

the project locations).

C Methodology

Please see Tasks 2.1 to 2.3 for methodological elements. Internal guidelines to data collection

etc. will, however, be drafted prior to carrying out these elements for the 2 pilot cases.

The selection of the 2 pilot cases will be done based on the outcome of Task 1.2. Those respon-

sible for carrying out the work will be same team members who did the Brief description (to en-

sure continuity) enforced by another core team member and a local expert in case of language

problems.


It is of utmost importance that 2 good projects are chosen: 4 of the most promising projects with

respect to CBA analysis are scrutinised to make the final selection.


D3:1st Progress report April. 15 April 2010.

D4: First Intermediate report. 14 May 2010.

SG + Expert meeting. 26 or 27 May 2010.

3.3.2 Task 2.1 Revision of ex ante CBA for ten projects

Task

No


2.1 Ex-ante project analysis Week 18 Week 34 Birgitte Holt Andersen 5

Peter G. Madsen 5


Szabolcs Szekeres 7

Tine Skyggebjerg 2

Jesper Karup 1

Raphael Zayat 1

COWI water and waste experts 8

Csil 2

Tecinvest 4

Local experts 10

Total 53

A Objectives

The objective of Task 2.1 is to review the quality of the 10 ex-ante analyses. In particular the re-

view will consider if:

• some important element is lacking (e.g. options analysis, risk analysis, calculation of invest-

ment residual value at end year);

• methodology used contains errors (e.g. misapplication of discounted cash flows)

• assumptions are unrealistic or poorly justified (e.g. assumptions on future trends and fore-

casts, the choice of a discount rate);

• results achieved are weak or inconsistent with the rationale for EU funding (e.g. overprofita-

bility, FRR(K)>financial discount rate so that the project is likely to be over-funded)

Furthermore, if a given ex-ante analysis does not cover the essential information or contains er-




27

.

rors, efforts will be made to produce an "improved ex-ante CBA" to be able to have a basis for

comparison with the given ex-post CBA (to be produced by the study team in Task 2.2). It must

be underlined, however, that it is not at all the objective to fully reconstruct the ex-ante CBAs.

Instead we should use most of our resources to produce good ex-post CBAs (Task 2.2). Finally, it

must be acknowledged that it is not an objective to blame the people who produced the ex-ante

CBAs − they might well have followed the guidelines given at the time of application − i.e. our

critical approach is focused on learning lessons to improve ex-ante CBAs in future project appli-

cations (such lessons will also be learnt from Tasks 2.2 and 2.3).


1 Review of available material for the 10 projects − in particular the respective feasibility studies −

to get in-depth pictures of the coverage and quality of the items introduced by the 3-page descrip-

tions made in Task 1.1, and of whether the requirements to information of the adopted ex-post

CBA methodology are fulfilled (see Task 2.2)

2 Assessment of the coverage and quality of the ex-ante CBAs, e.g. analysed impacts of water and

solid waste projects, use of conversion factors, description of baselines etc.; and rough examina-

tions of the intervention logics (that also will comprise a common framework for the experts in-

volved in the study)

3 Establishment of "improved ex-ante CBAs" − when necessary

C Methodology

The internal guidelines developed and revised as part of the pilot case analyses (Task 2.0) will be

implemented. The information collected will be inspired by the items covered in the 3-page de-

scriptions (Task 1.1) and the Terms of Reference for the present study. The key methodological

reference for this methodological assessment will be the DG REGIO CBA Guide.

Regarding the rough examinations of the intervention logics for the 10 projects, the terminology

used in DG Budget "Evaluating EU Activities − A practical guide for the Commission services" will

be adopted. The approach is to assess which costs and benefits (hereunder in the form of out-

puts, results or impacts) are or should have been covered by the ex-ante analysis, and to assess

which of these items are or should have been quantified, and which only can be analysed in a

qualitative manner.

Note that the improvements of the ex-ante CBAs also will look into the alternative options to the

one selected by the project applicants, and in this context look for possible "donor biases" − i.e. a

possible tendency that capital-intensive solutions are chosen in favour of less capital − but more

operational cost-intensive projects. Such analysis of alternative options will happen, to a large

extent, as an integrated part of the ex-post analysis (Task 2.2).

In practice, the quality assessments of the ex-ante CBAs will therefore distinguish between the

original ex-ante CBAs and the "improved" ex-ante CBAs. In this manner we will aim to conduct a

“improved” ex-ante CBA based on the information available at this point and with the newest

methodology including application of the software PROPLAN presented at the Steering Group

meeting March 3rd.


There is at present much uncertainty regarding the quality of the ex-ante CBAs: much emphasis

in the 2 pilot case analyses (Task 2.0) will be given to establishing the level of ambition regarding

the "improvement" of ex-ante CBAs.


D3: Input to Progress reports

June, July, Sept 2010.

Mid June, mid July, mid September.

D7-8: Input to final reporting. Jan-March 2011.




28

.

3.3.3 Task 2.2 Carrying out the ex post CBAs

Task

No

Name of Task Start End Responsible (in bold) Day

s

2.2 Ex-post cost benefit analysis Week 18 Week 34 Birgitte Holt Andersen 5

Peter G. Madsen 5

Szabolcs Szekeres 7


Tine Skyggebjerg 2

Jonas Stravinskas 8

Jesper Karup 1

COWI water and

waste experts

20

Raphael Zayat 5

Local experts 36

Csil 4

Tecinvest 4

Total 101

A Objectives

The objective of the ex-post CBAs − which is one of the two main objectives of the evaluation − is

to deliver actual CBA results for the 10 selected environment projects. While the analysis will fol-

low the DG Regio’s "Guide to Cost Benefit Analysis of Investment Projects", it is acknowledged

that guides coverage of water and solid waste projects is limited and can be further developed.

Hence, it is also an objective of this project to explore and, if possible, further develop the appli-

cable CBA methodology, including developing CBA references for the sectors under examina-

tion. Such exploratory approach might imply that the analysis of the 10 different projects might

not be fully comparable − i.e. they might address the different specifics of the single projects,

while at the same time a consistent CBA methodology will be pursued. Furthermore, while the 10

CBAs in themselves are central for the evaluation of the 10 projects, the results are also central

for assessing the use of CBA as a method in the context of environment projects (Task 3).


1 Missions will be carried out to each of the 10 project sites (if need be, we may carry out one fur-

ther mission to selected project sites). The information will partly be collected via the internal

documents/toolkits developed and tested in Task 0 ,and partly via "learning-as-you-go" − i.e. ad-

dressing specific cost and benefit issues identified during the visits to the project sites, hereunder

the discussions with the different project holders.

2 Cost benefit analysis will be carried out according to the methodology presented in the CBA

guide and other international standard when necessary− and although not all costs and benefits

may be quantifiable, calculations of economic returns and risk and sensitivity analyses, and unit

costs will be made. In this context we will need to formulate the baselines for the analysis, which

might or might not be easily derived from the ex-ante options analysed (Task 2.1).

We will, in the formulation of baselines, look into the feasibility of presenting (technical) alterna-

tives in a qualitative way and/or via expert assessments ("back of an envelope assessments") or

via discussions with project holders, to obtain some kind of baseline for each of the 10 projects,

and we will look into the feasibility (i.e. within the limits of the study resources) of making some

quantitative baseline calculations. Furthermore, we will address the extent to which the different

alternatives for solving the water and the solid waste problems have similar benefits or not (e.g.

because they lead to similar increases in water supply or similar reductions in waste) − i.e. so

that the analyses might have a tendency of becoming cost-effectiveness analyses. (In practice

many decisions, e.g. at local level, are likely to be biased against the cheapest and easiest solu-




29

.

tions, rather than of fully assessed costs and benefits).

3 Qualitative analysis of costs and benefits will focus on those impacts that cannot be quantified.

The analysis will mainly involve thorough narrative descriptions of the assessed costs and bene-

fits so that they are comprehensible for policy-makers and other stakeholders. The aspect that

costs might be easier to quantify than benefits imply an imbalance that might give rise to the risk

that policy-makers and others resist or hesitate to take action. This imbalance will be addressed.

Furthermore, an approach to combining the quantitative and qualitative assessments will be de-

veloped. Finally, the assessment will look into the possible appearance of unintended effects −

some of which might be quantified.

C Methodology

In accordance with the ToR, the analysis will − in addition to the elements and methodology

(hereunder the handling of conversion factors, transfers etc.) in the DG Regio guideline − specifi-

cally address:

• Utilisation,

• Validity of assumptions made during the ex-ante analyses,

• Risk mitigation measures, including consequences of mitigated and (both foreseen and un-

foreseen) non-mitigated risks,

• Costs and benefits that cannot be expressed in monetary forms,

• Meeting environmental requirements,

• The contribution of accompanying actions which are outside the project but intended to en-

hance the project success (qualitative approach only).

• Unintended effects, and

• The calculation of the margin of error of the ex-post CBA.

In addition, we will establish unit costs for the 10 projects, i.e. disaggregating costs to the differ-

ent types of key components ("level 3" costs). In carrying out this task, we will apply the method-

ology developed by Work Package 10 of the ex-post evaluation of the ERDF in the 2000-06 pro-

gramming period.

Acknowledging the exploratory nature of the analysis − hereunder when specifying the impacts of

water and solid waste projects − particular attention will be paid to transparency regarding the

use and development of the CBA methodology. Furthermore, building on both quantitative and

qualitative assessments implies a need to present the combined results in a narrative way − in

addition to the calculations. The approach to do this will be a part of the internal guidelines de-

veloped in Task 2.0.


The availability of data to carry out the analysis is for some projects likely to be limited: be trans-

parent about the extent to which qualitative assessments only can be made.


D3: Input to Progress reports June, July, Sept

2010.

Mid June, mid July, mid September.

D7-8: Input to final reporting Jan-March 2011




30

.

3.3.4 Task 2.3 Comparing ex ante and ex post CBAs

Task

No


2.3 Comparing ex-ante

CBA with ex-post CBA


Peter G. Madsen 5

Szabolcs Szekeres 7


Tine Skyggebjerg 2

Jonas Stravinskas 3

Jesper Karup 1

COWI water and waste experts 9

Raphael Zayat 5

Csil 2

Tecinvest 1

Local experts 14

Total 58

A Objectives

The objective of comparing the ex-ante and ex-post CBA is in accordance with ToR in particular

to address:

• The appropriateness of both the financial and economic discount rates to the projects and

whether all relevant factors were taken into account;

• Reasons for any difference between financial and economic net present values or rates of

returns calculated ex-ante and ex-post;

• Reasons for possible time and cost overruns. In carrying out this task, we will take into ac-

count the methodology and findings of the abovementioned Work Package 10;

• An examination of the ex-ante risk assessment and review of the actual risk bearers, given

the actual outcomes during project implementation.

• How can ex-post CBAs contribute to the practice of ex-ante CBAs?


Answering the above questions via qualitative desk study analysis and interviews to stakeholders

to identify and interpret reasons for ex-ante/ex-post deviations.

C Methodology

In the comparison ex-ante/ex-post reasons for deviations will be explored and discussed. An im-

portant distinction will be between exogenous and endogenous factor. It is in fact useful to dis-

cuss if deviations from the estimates were normally predictable and under the control of the

project management (so in this case a proper discussion about to what extent it was not taken

into due consideration ex-ante and how this affected the performance of the project will be pro-

vided), or, in the case they were not, to elaborate on the degree of resilience of the project to ex-

ternal shocks and actual responses (if any) put in place by the project management. In both cas-

es a discussion about how the risk analysis had or would have made a difference in dealing with

effects of uncertainties will be highlighted.


As mentioned several times, the ex-ante CBAs (although some might have been "improved"

within the present study) might be incomplete, and so difficult to compare with the ex-post CBAs:

for such projects, most emphasis will be on answering the last of the above questions.




31

.


D5: Second Intermediate Report. 15 October 2010.

SG+Expert meeting. 27 or 28 October 2010.

3.4 Task 3 Assess CBA as a method

Task 3 concerns the assessment of the CBA as a method. This will partly be

based on the results of Task 2 and partly on other relevant experiences, from

academics and other experts. The result thus primarily fulfils the second objec-

tive, although it will also shed light on the quality and thus the value of the 10

specific project CBAs. The task is divided into four sub-tasks, including a

seminar with the member states, and assessment of CBA as method in the ten

cases, compare our CBA experience to other experiences, and to provide the

Commission with our recommendations in the final report.

3.4.1 Task 3.1 Seminar with the Member States

Task

No


3.1 Seminar with Member States Week 46 Week 47 Birgitte Holt Andersen 4

Peter G Madsen 5

Szabolcs Szekeres 1


Tine Skyggeberg 2

Jonas Stravinskas 3

Raphael Zayat 1

Jesper Karup 1

Csil 2

Other 6

Total 28

A Objectives

The objective is partly to validate the findings of the study so far, and partly to receive information

and comments that can be used in the assessment of the CBA as method for environment pro-

jects (water and solid waste projects), and thus for making recommendations to DG Regio and

other stakeholders.


1 Develop seminar programme, identify and invite participants from Member State authorities, pro-

ject holders, and EC officials.

2 Execute seminar with introductions by study team members and scientific experts. One of the

project analyses carried out may serve as a case illustrating the challenges and opportunities in

applying CBAs for environment projects.

3 Producing seminar programme.

C Methodology

The methodology is described under B above. Note: The seminar might be organised together

with a similar seminar within Work Package B (CBA of transport projects) − hence there might be

a need to coordinate the methodologies.





32

.

None foreseen: it is expected to be feasible to attract the right participants.


D6: Seminar with Member

States.

Early December, 2010.

3.4.2 Task 3.2 Assessment of CBA as a method for the ten cases

Task

No


3.2 Assessment of CBA as

method based on 10

project analyses

Week 43

2010

Week 6

2011

Birgitte Holt Andersen 4

Peter G Madsen 5

Szabolcs Szekeres 3


Tine Skyggeberg 2

Jonas Stravinskas 3

Jesper Karup 1

COWI water&waste experts 4

Csil 13

Raphael Zayat 1

Total 38

A Objectives

The objective of the task (together with Task 3.3) − which is the second of the two main objec-

tives of the evaluation − is to assess the use of CBA as method of appraisal for environmental

projects. Note: the study team will in practice have this objective in mind all the time while carry-

ing out Task 2.


The task will be carried out as a desk study − and will mainly be based on the information gener-

ated via Task 2.

C Methodology


• The strengths and weaknesses of the CBA methodologies applied by the Member States −

i.e. the ones that guided the ex-ante CBAs (assessments of Member State guidelines used

at present are made in Task 3.3).

• Effectiveness of CBA as a tool supporting the project generation and project decision of

Member States and the Commission − mainly based on the ex-ante analysis in Task 2.1.

• Utility of ex-post CBAs from the point of view of project holders, Member States and the

Commission − investigated when carrying out Task 2.2.

• Rigorous treatment of environmental costs and benefits that cannot be expressed in mone-

tary terms, including how they can be taken into account in decision-making − to a large ex-

tent based on the approach developed in Task 2.2 to combine quantitative and qualitative

assessments.


None foreseen.




33

.


Provide input to D7: Draft Final

report.

15 January 2011 (week 2 of 2011).

3.4.3 Task 3.3 Assessment of CBA method compared to other experiences

Task

No


3.3 Assessment of CBA as

method based on other

experiences

Week 43

2010

Week 6

2011

Birgitte Holt Andersen 4

Peter G Madsen 5

Szabolcs Szekeres 3


Tine Skyggeberg 2

Jonas Stravinskas 3

Raphael Zayat 1

Jesper Karup 2

Csil 4

Others

Total 26

A Objectives

The objective of the task (together with Task 3.2) − which is the second of the two main objec-

tives of the evaluation − is to assess the CBA as method for environment projects based on other

experiences than those gained in Task 2.


1 The other experiences to build the assessment upon will be identified. It is likely, however, that

much of the appropriate material may have been identified already during the previous tasks.

2 The task will then be carried out as a desk study of the identified material.

C Methodology


• The strengths and weaknesses of the CBA methodologies applied by the Member States in

general, i.e. in additions to the ones that guided the ex-ante CBAs.

• Effectiveness of CBA as a tool supporting the project generation and project decision of

Member States and Commission in general.

• Utility of ex-post CBAs from the point of view of project holders, Member States and the

Commission in general.

• The extent that ex-post CBA is an appropriate tool for evaluating impact.

• Rigorous treatment of environmental costs and benefits that cannot be expressed in mone-

tary terms, including how they can be taken into account in decision-making − based on find-

ing in the identified material.

• Relevance and potential utility of data obtained from CBAs in macroeconomic modelling.

Furthermore, the assessment will address the possible links between CBA methodologies and




34

.

other tools for environmental assessments, such as EASEWASTE − a PC tool for environmental

assessment of waste systems developed by the Technical University of Denmark based on life

cycle assessment principles.


None foreseen


Provide input to D7: Draft Final

report

15 January 2011 (week 2 of 2011)

3.4.4 Task 3.4 Recommendations and final report

Task

No


3.4 Recommendations and

reporting

Week 6 2011 Week 10 2011 Birgitte Holt Andersen 7

Peter G Madsen 5

Szabolcs Szekeres 3


Tine Skyggeberg 2

Jesper Karup 2

Raphael Zayat 1

Jonas Stravinskas 3

Csil 4

Total 33

A Objectives

The objective of the final task is to prepare the Final Report covering all elements of all tasks,

answering the underlying questions of the evaluation and presenting findings and recommenda-

tions in an operational way, so that DG Regio and other stakeholders may make good use of the

evaluation carried out − and also to finalise the report on the basis of comments received from

DG Regio and the Steering Group.


All tasks above provide inputs to this task. Furthermore, the meeting with DG Regio and the

fourth and last meeting of the Steering Group will provide valuable inputs as well. Much attention

will be paid to communication. The draft Final Report and also the Final Report will include an

executive summary and short summaries of the CBAs of the 10 projects carried out.

C Methodology

The task will then be carried out as a desk study.


None foreseen.

E Deliverables &

Meetings

Due date

D8: Final Report. 18 March 2011 (week 11 of 2011).




35

.

4 Management issues

This chapter contains some additional issues that have been raised by either the

Commission or the consultants in the inception phase.

4.1 Time table

The time table has been updated according to agreements.

One important point in regards to the time table, is the deliverable containing

the 20 project descriptions. Deadline for these is March 15th, however, due to

difficulties in accessing the relevant material, then a number of these has been

postponed. This means that deadline for the projects from Ireland (Dublin),

Portugal (Porto), Slovak (Presov), and Spain (four project in Zaragoza, Madrid,

Tajo Basin and Valencia) has been postponed until March 17th.

Figure 4-1 Updated timetable

4.2 Planning quality assurance

Quality assurance has been thoroughly included in the planning process. The

following table display the planning of the deliverables leaving additional time




36

.

for quality assurance by adding an internal deadline before submitting deliver-

ables to the Commission.

Table 4-1 Planning of deliverables

Date Internal Deliverable Project review team

15/02/2010

Revised ver-

sion 26/3/2010

13/02/2010 D1: Inception Report Core team, Jesper Karup

Language QA by COWI language

expert

15/03/2010 13/03/2010 D2: Project descriptions of 20

projects

Core team, Jesper Karup, External

experts,


expert

15/04/2010 14/04/2010 D3:1st Progress report − April PM will make quality assurance of

the monthly progress reports

14/05/2010 12/05/2010 D4: 1st Intermediate report Jesper Karup Pedersen, External

experts,


expert

15/06/2010 14/06/2010 D3: 2nd Progress report −

June

PM will make quality assurance of


15/07/2010 14/07/2010 D3:3rd Progress report July PM will make quality assurance of


15/09/2010 14/09/2010 D3:4th Progress report Sept PM will make quality assurance of


15/10/2010 13/10/2010 D5: 2nd Intermediate report Jesper Karup Pedersen, external

experts,


expert

Early Decem-

ber

D6: Seminar with member

States

Core team

15/01/2011 10/01/2011 D7: Draft Final Report Jesper Karup Pedersen, external

experts,


expert

18/03/2011 13/03/2011 D8: Final Report Jesper Karup Pedersen, external

experts,


expert

TBD D9: 2 presentations at meet-

ings in Brussels (dates to be

decided)

Core team




37

.

A large part of the quality assurance will be conducted as per-reviews within

the core team, so that deliverables prepared by one senior expert will be criti-

cally reviewed by another senior expert. In addition, to ensure sufficient quality

assurance on the ten CBAs, then time has been allocated to the Quality assur-

ance team which includes Jesper Karup Pedersen as technical expertise and as

well a COWI language expert.

TASK 0 TASK 1 TASK 2 TASK 3 Total

0 1.1 1.2 2.0 2.1 2.2 2.3 3.1 3.2 3.3 3.4

Birgitte Holt Andersen 6 2 3 5 5 5 5 4 4 4 7 50

Peter G Madsen 3 4 3 5 5 5 5 5 5 5 5 50

Birgitte Grønbech 8 3 4 4 4 4 3 2 2 6 40

Jesper Karup Pedersen 1 1 1 1 1 1 1 2 2 11

Tine Skyggeberg 5 7 0 2 2 2 2 2 2 2 2 28

COWI water expert 2 4 12 5 2 25

COWI solid waste expert 2 4 8 4 2 20

Szabolcs Szekeres 3 4 1 1 7 7 7 1 3 3 3 40

Jonas Stravinskas 2 8 3 3 3 3 3 25

Raphael Zayat 2 4 5 5 5 5 1 1 1 1 30

Csil 3 0 5 5 2 4 2 2 13 4 4 44

Tecinvest 16 4 4 1 25

Scientific advisors 6 6 6 18

Experts>10years 4 0 5 8 1 18

Experts 5-10years 2 0 5 28 13 48

COWI Language Expert (2) (6) (2) 10

22 53 16 38 53 101 58 28 38 32 33 472

4.3 Conflicts of interest

At the meeting with the Steering Group March 3rd, the issues of conflicts of in-

terests were addressed, as COWI has in some instances been involved in prepa-

ration of ex-ante studies for environmental projects. In this regards, the consult-

ant will on a case-by-case basis indicate and address any potential conflicts of

interest and ensure that the consultants working on the case are objective in

their assessment. Should it become necessary, we will include additional con-

sultants to undertake work, where COWI or the consultants are not able to han-

dle projects objectively.

In regards to the particular discussion of the Lithuanian case Nemunas, it has

been checked, and COWI has not been involved in the ex ante studies.

4.4 External experts

The following external experts are proposed to be associated with the project:

Name of expert Key expertise




38

.

Professor Giles Atkinsons,

The London School of Economics and Political Science, Department of Geography and Environ-ment UK [email protected] Ph: +44 20 7955 6809

green accounting; sustain-able development; environ-mental valuation; environ-mental policy; environ-mental economics

Prof. Sándor Kerekes, vice rector re-sponsible for academic affairs of Corvinus University Budapest.

MSc in Chemistry in 1971 and PhD in management in 1984 and DSc in 2003 from Hungarian Academy of Sci-ences.

Prof. Runar Brännlund,

Professor in economics. Department of Economics, Umeå University; Professor in forest economics, Depart-ment of Forest Economics, Swedish University of Agricultural Sciences. E-mail: [email protected]

Master in Economics and Statistics, Umeå University 1981 Ph.d in Forest economics, Swedish University of Agri-cultural sciences 1988

Not available




39

.

Annex 1 Brief project descriptions completed for Project No 26 and Project No 36/37

C:\Documents and Settings\TSKY\Desktop\Inception report revised\D2 27 HU 2000HU16PPE003_FINALXX.docx

Document no.

Version

Date of issue

Prepared

Checked

Approved

Table of Contents

1 Project Summary 1

1.1 Socio economic context and objective 1

1.2 Project identification 2

2 Assessment of the available ex-ante documentation 2

2.1 Project feasibility and alternative options 2

2.2 Financial analysis 2

2.3 Economic analysis 3

2.4 Risk assessment 3

3 Assessment of the available ex-post documentation 4

3.1 Construction period 4

3.2 Operation period 5

4 Tasks of a possible ex-post evaluation 5

5 Recommendations 6

6 Sources of information 6

1 Project Summary

1.1 Socio economic context and objective

Szeged is a city of nearly 170,000 inhabitants in southern Hungary. It boasted one of the oldest sewer systems in the country, dating from the 19th Century, which had, however, become obsolete and inadequate more than 100 years later. Only 72% of the city’s dwellings were connected to it, and 40% of the sewage collected was discharged raw into the Tisza River, with the rest receiv-ing only mechanical treatment.

Project objective With this project the Hungarian government sought to remedy this situation and, at the same time, help meet the EU pre-accession requirements, and con-

European Commission, DG Regio

Ex post evaluation of cohesion interventions

2000-2006

Brief Project Description, Project no. 27: Szeged Municipal Wastewater Treatment and Collection Project, Hungary

(2000HU16PPE00)

COWI A/S

Parallelvej 2


Denmark

Tel +45 45 97 22 11

Fax +45 45 97 22 12

www.cowi.com

Environmental pres-sure

Ex post evaluation of cohesion interventions 2000-2006 – Project no. 27 Hungary


2 / 6

.

tribute to the improvement of the environment of the surrounding region (sur-face, subsurface waters and soil).

1.2 Project identification

Project units The project includes (a) the expansion of the existing waste-water treatment plant to include full biological treatment of up to 60,000 m3/day of sewage; (b) increasing the length of the sewer network from 310 km to 718 km; and (c) in-creasing the number of household connections to 15,470, thereby expanding coverage of the system to 100% of the population of Szeged and of four addi-tional towns (nearly 181,000 inhabitants in all).

The financial memorandum was signed by the EU on 22 December 2000. Pro-ject execution started on 19 September 2003 and concluded on 30 June 2008.

2 Assessment of the available ex-ante documentation

2.1 Project feasibility and alternative options

Project feasibility The document entitled “Financial and economic analysis 2000HU16PPE003” provides a good ex-ante analysis and description of the project. While it out-lines the proposed methods of achieving the objective of an environmentally compliant design in detail, it points out that the actual design option was left for bidders to specify. The design capacity of the WWTP is 60,000 m3/day, while forecast utilization for 2034 was 50,000 m3/day. But these are average daily numbers that do not consider the fluctuations imposed by storm waters.

Alternative options In the course of the ex-post evaluation a quick review of alternative technical solutions would be looked at, and the cost impact of alternative plant sizes should receive a brief review.

2.2 Financial analysis

Total investment cost was forecast to be EUR 84 m. The Financial Rate of Re-turn of the project was reported to be - 0.39% and the Net Present Value of the project cash flows was given -31.85 million EUR (without mention of the dis-count rate used). The same indicators, taking into account the intended ISPA grant, are 8.77 % and 8.98 million EUR respectively. The period of operation was considered to be 30 years.

Tariffs The average monthly water tariff that could be charged was expected to be 14.20 EUR, amounting to 2.9 % of average household income, which was deemed affordable for the population in Szeged and Dél-Ujszeged.

Missing information Some of the relevant Annexes in the cited document are missing, and those covering the same topics in others are not easily legible. These will have to be obtained.

Financial return and cash flow



3 / 6

.

In the course of the ex-post evaluation the utilization rates and tariff revenues will have to be updated, as well as the actual capital costs incurred.

2.3 Economic analysis

The ex-ante analysis presents both cost-efficiency and cost-benefit analyses of the project. For the cost-efficiency analysis, the present value of investment and operating costs is related to the reduction of pollutant discharges that the project was forecast to bring. As no other alternative was evaluated, there is nothing within the project documentation with which the result could be com-pared.

Benefits The CBA’s approach is essentially sound, although it may not be correct in every respect. Five benefits were quantified:

(1) The decrease of the environmental pollution load, valued at the level of fines that the Government was expected to levy on them. This is an inter-esting approach. It assumes that the fines correspond to the damages caused. (86.1 % of the benefits)

(2) Savings on unemployment allowance. This is not correct as stated, but can be viewed as a proxy for the using the shadow price of labour. (2.9 % of the benefits)

(3) Increase in the value of real estate. This is meant to convey the value of the benefits enjoyed by those who are connected to sewer service. It is danger-ous way to calculate this and the details could not be checked. (4.7 % of the benefits)

(4) Recreational activities made possible at the Tisza River; (1.6 % of the bene-fits)

(5) Biogas generation (4.7 % of the benefits)

No other shadow prices were used, but VAT taxes were eliminated from the calculation. The results of the CBA calculation were: IRR of 9.88%, NPV at 5% discount rate of EUR 29m, and B/C of 1.32,

Missing information The corresponding Annex is missing. In the course of the ex-post evaluation unit benefits for the above will have to be verified and an estimate made of the benefits of reduced ground water contamination. It is interesting to note one observation found in the “Annex III Environmental impact assessment” (page 7) “…the pollution load from the Town is not extremely high on the recipient even without treatment, but it is quite comparable with the Maros [a nearby af-fluent] in some parameters”.

2.4 Risk assessment

No formal sensitivity analysis calculations were found in the documentation available. Two sources of risk were mentioned: the risk of non payment of

Type of economic analysis

Use of conversion factors

Sensitivity / Risk analysis



4 / 6

.

tariffs and that of the usage rate being below forecast. But no specific mitiga-tion measures were postulated.

In the course of the ex-post evaluation a full sensitivity analysis and a risk analysis will have to be undertaken.

3 Assessment of the available ex-post documentation

3.1 Construction period

This section describes the extent to which the investments proposed in the ex-ante documentation have actually been realized, in physical terms, during the investment phase of the project.

Only the two most easily summarized physical targets will be mentioned.

Target Executed Deviation

New sewer construction 408 km 402 km -6

Number of new sewer con-

nections

15,470 14,327 -1143

The shortfall in connections is proportionally much higher than the shortfall in sewer construction. The effects of this on the project’s performance should be investigated in the course of the ex-post evaluation.

Budget execution This section compares the planned and observed investment expenses of the project. Very detailed information exists in a completion report available in Hungarian. In this brief note only the totals are quoted.

Budget Revised Deviation

Total expenses EUR 84 m EUR 95.6 m EUR 11.6 m

The cost overrun is proportionately higher than it seems, because of the reduc-tion in the actually completed work in physical terms.

The Hungarian language Closing Report contains detailed information on the budget execution, as it reports on the results of several audits carried out on this project.

Schedule review This project was plagued by delays. It became financially eligible on December 22, 2000, but the project actually started only on September 19, 2003. It was originally scheduled to be completed at the end of 2005, but it was finished June 30, 2008.

The Hungarian language Closing Report presents a detailed chronology and analysis of the delays. This could be incorporated into an ex-post evaluation.

Physical quantities of work undertaken



5 / 6

.

3.2 Operation period

No data on the operational period is available. However, the Hungarian lan-guage closing report notes that the effluent quality objectives of the project were met, and that raw sewage is no longer discharged into the Tisza River.

4 Tasks of a possible ex-post evaluation

This project raises a number of interesting questions, besides the basic one of establishing the actual return on the project:

• Quantification of benefits. The use of fines to estimate environmental damages poses a question worth exploring. However this could only be done by reference to results from the literature.

• The benefit of diminished groundwater pollution was not quantified. A review of the literature on this matter might offer useful guidance for future CBAs in this field.

• Technical alternatives were not explicitly presented, but creating some realistic ones would not be too difficult, so it could be interesting to see how much difference doing so could have made. These include alter-ative technologies and alternatives of scaling.

• The difficulties of project execution are well documented. Their analy-sis might provide conclusions of general applicability.

Capital costs

All categories Ascertain quantities and actual costs.

Formulate alternatives of technology and

scaling.

Forecast periodic reinvestment costs.

Utilization rate

Verify and revise forecast. Considering that

users pay for water and sewer jointly, con-

sider the impact of possible tariff changes.

Benefits

User benefits that derive from having a

sewer connection

Estimate properly.

Operating costs

All categories Verify and forecast.

Externalities

Water pollution (discharges) Find alternative estimates of this cost

Ground water pollution Find alternative estimates of this costs



6 / 6

.

5 Recommendations

This project raises a number of interesting questions, and should be considered for inclusion in the final group of 10, subject to the selection criteria.

It should be mentioned, however, that given that some key benefits can only be estimated by reference to the literature, the results obtained are not likely to be precise.

6 Sources of information

“Annex II” 26/06/1999. File name: Annex II 2000HU16PPE003.pdf

“Environmental impact assessment”, Szeged, February 2000. File name: Annex III Environmental impact assessment.pdf

“Feasibility Study”, Szeged, February 2000. File name: Feasibility study 2000HU16PPE003.pdf

“Financial and economic analysis”, SVM Plc., Tóth and Partner Ltd, Szeged 2000. File name: Financial and economic analysis 2000HU16PPE003.pdf

“Log Frame Analysis of Szeged Municipal Wastewater Treatment and Collec-tion Development”, undated. File name: Log frame analysis 2000HU16PPE003.pdf

Maps, undated. File name: Maps 2000HU16PPE003.pdf

“ZÁRÓJELENTÉS”, Környezetvédelmi és Vízügyi Minisztérium, Fejlesztési Igazgatóság. File name: Final report___Hungarian.TIF.

C:\Documents and Settings\TSKY\Desktop\Inception report revised\D2 36_37_SI_2004(5)SI16CPE001_FINALXX.docx

Table of Contents

1 Project summary 1

1.1 Socio economic context and objective 1

1.2 Project identification 2

2 Assessment of available ex-ante documentation 2

2.1 Project feasibility and alternative options 2

2.2 Financial analysis 3

2.3 Economic analysis 3

2.4 Risk assessment 4

3 Assessment of available ex-post documentation 4

3.1 Construction period 4

3.2 Operation period 5

4 Tasks of possible ex-post evaluation 5

5 Recommendations 6

6 Sources of information 6

1 Project summary

1.1 Socio economic context and objective

In 2002, in the Celje waste contribution area 75,461 tons of municipal waste

were collected from 64,687 households [= 185,132 inhabitants] (87%) and the

public services sector, such as offices, schools, kindergartens, market halls and

parks (13%). The waste was composed of biodegradable waste (29%), paper

and cardboard (18%), glass (5%), plastics (10%), metals (7%), and other waste

(31%). The forecast of a stable population implies that the total amount of mu-

nicipal waste generated is expected to remain unchanged. However, the existing

capacity of landfills will soon be reached, and they are inappropriately located

and technically inadequate (unsealed, no gas drainage, exposed to floods or

close to groundwater etc.).

Furthermore, to comply with new regulation requirements - especially demands

for land-filling of biodegradable waste and separate collection - new invest-

ments were needed.

European Commission, DG Regio

Ex post evaluation of cohesion interventions

2000-2006

Brief Project Description, Project no. 36/37: Regional Waste Management Centre Celje, Slovenia

(2004(5)SI16CPE001)

COWI A/S

Parallelvej 2


Denmark

Tel +45 45 97 22 11

Fax +45 45 97 22 12

www.cowi.com

Environmental and

socio economic pres-

sure

Regulation compli-

ance

Ex post evaluation of cohesion interventions 2000-2006 – Project no. 36/37 Slovenia


2 / 7

.

Project objective The construction of the new Regional Waste Management Centre Celje

(RWMC Celje) aimed at compliance with this new regulation. At the same

time, emphasis was on waste recycling and on mechanic-biological and thermal

treatment of municipal waste in order to generate fuel for electricity and heat

production.

1.2 Project identification

Project units The RWMC units comprise from Stage 1 of the project a composting plant for

separately collected biodegradable waste (capacity of 5,000 tons per year); a

sorting plant for separately collected plastic, metal, paper and cardboard (capac-

ity of 5,000 tons per year); a dismantling facility for bulky waste; and the ex-

pansion of the Bukovžlak landfill for the rest of waste (final capacity of

1,878,000 m3). Stage 2 of the project comprises equipment for mechanic-

biological treatment of waste (capacity of 61,500 tons per year) and equipment

for thermal treatment of waste (capacity of 25,000 tons per year).

Network effects The following additional infrastructure developments are included in the

project: access roads and bridge, asphalt area, car wash facility, and a platform

for treatment and temporarily storing of structural waste.

2 Assessment of available ex-ante documentation

2.1 Project feasibility and alternative options

Project feasibility As mentioned above, the demands to both stages of the project are to a large

extent induced by stricter regulation requirements for municipal biodegradable

waste reduction and for separate waste collection. Hence, there is a reliable,

stable demand for the new waste management facilities.

Alternative options The presentation of the alternative options in Stage 1 is somewhat unclear. For

Stage 1 an alternative location to Bukovžlak has apparently been analysed, and

a number of technologies for the treatment of collected biological waste as well

as the rest of the waste are presented - however without information about the

technologies that have actually been chosen. For Stage 2, three options are pre-

sented: (1) mass incineration of rest of waste, capable of incinerating waste wa-

ter treatment plant sludge; (2) mechanic-biological treatment of rest of waste

and provisions for further handling of light fractions and waste water treatment

plant sludge; (3) mechanic-biological treatment of rest of waste and incinera-

tion of light fractions and waste water treatment plant sludge within a local

thermal treatment facility of limited capacity. A selection scoring system ad-

dressing impacts on quantity of waste; environmental, economic and social as-

pects; indirect impacts; and feasibility was applied - leading to a choice of the

third option. Finally, a "do nothing" option is described that implies a need ei-

ther to reduce the quantity of waste or transport it across state borders - which

is assessed to be an irrational solution.



3 / 7

.

2.2 Financial analysis

Cash flows The cash flow analysis is assessed to have been proficiently made. It has good

coverage of both investment costs and expected operating income and costs

over a period of 30 years.

Financial return The expectations to income and costs lead to an estimate of the financial return

on the investment cost: In Stage 1, FIRR/C is 0%, and FIRR/K is 4.2 %. In

Stage 2, FRR/C is -1.58%, and FIRR/K is 2.94%. All financial return figures

are below the applied discount rate of 6% (used in Stage 2) and 8% (used in

Stage 1).

Sources of finance The investment was financed by the participating municipalities, with co-

financing from the Cohesion Fund, and with means of the budget of the Repub-

lic of Slovenia stemming from environmental levies. The total eligible costs of

the two stages amounted to EUR29.3 million (Stage 1: EUR 8.9 million, Stage

2: EUR 20.3 million), of which the Cohesion Fund contributed 80%. The fig-

ures seem to differ slightly between the different available document, but all are

at this level.

Finally, it appears convincing that the project will be financially sustainable.

The cumulated cash flows expected were positive - apart from relatively small

negative figures for the first few years of operation (2009-12).

2.3 Economic analysis

The Stage 1 feasibility study does not comprise a CBA, using the argument that

there is no generally agreed methodology for quantifying and valuing environ-

mental benefits from waste management facilities. Instead, a kind of a cost-

effectiveness analysis was applied. It can be argued that such analysis is suit-

able for deciding upon the cheapest option to fulfil a given regulation require-

ment. However, the analysis provided does not seem to go beyond the cost cal-

culations of the financial analysis - i.e. a socioeconomic cost calculation seems

to be missing.

The Stage 2 feasibility study does in contrast attempt to carry out a CBA by

including the added value to (regional) construction and business services com-

panies, and a reduction in the consumption of fossil fuels.

However, these benefit calculations seem to be a wrong or missing application

of conversion factors. Although, the added value to the (regional) companies

can be understood as socioeconomic costs being lower than the amount paid to

these companies by the project; but this somehow assumes that the companies

have over-capacities - e.g. measures via over-employment. Similarly, the re-

duced consumption of fuels should already have been part of the financial

analysis. Hence, the socioeconomic benefits should, for example, limit them-

selves to reduced greenhouse gas emissions.

Quantitative impacts Hence, the attempts to quantify impacts are limited and they are assessed not to

have been made in a correct way.

Financial sustainabil-

ity

Type of economic

analysis

Use of conversion

factors



4 / 7

.

Qualitative impacts In addition to the abovementioned impacts the two feasibility studies briefly

discuss a number of impacts in a qualitative manner: protection of surface and

ground water, less area used for waste disposal, health risk to the population,

and employment (that already is covered by the financial analysis - although no

use of shadow wages have been made). Elements such as noise, emissions and

odours are not fully addressed.

Discount rate While Stage 1 does not make use of a discount rate, the analysis in Stage 2 uses

a real economic discount rate of 5% (i.e. slightly below the EC recommenda-

tion of 5.5%). This rate is, however, lower than the 6% used in the financial

analysis (8% in Stage 1). Hence, it could appear that relatively more uncer-

tainty or less importance of future financial costs and benefits have been as-

sumed.

Economic return Hence, only calculations of economic returns is provided in the Stage 2

feasibility study: ERR of 8.61%, B/C ratio of 1.33, and ENPV of 9,518,000

Euro.

2.4 Risk assessment

Sensitivity analysis For both stages the risk assessment is limited to the financial analysis - i.e. no

sensitivity analyses are made for the economic analysis. Scenarios were made

for +/-10% changes in the critical variables: co-financing rate, amount and

prices of sale, selected investment and operation cost items.

Risk analysis A number of risks are mentioned in the Stage 2 feasibility study: assuring status

as producer of electricity, cancellation or non-acknowledgement of bonuses at

electric energy sale, and disagreement regarding the realisation of the project.

However, neither the sizes/distributions of the risks nor the impact of the

size/distribution of the financial return are provided.

3 Assessment of available ex-post documentation

3.1 Construction period

The below table shows the extent to which the waste treatment plant invest-

ments proposed in the ex-ante documentation have actually been realized, in

physical terms, during the investment phase of the project:

Target Executed by 2007 (Stage

1) and 2008 (Stage 2)

Composting plant for bio-

degradable waste

Capacity of 5,000 tons pa ≈ 50%

Sorting plant Capacity of 5,000 tons pa ≈ 90%

Bukovžlak landfill Capacity of 1,878,000 m3 ≈ 90%

Mechanic-biological

treatment equipment

Capacity of 61,500 tons pa ?

Physical quantities of

work undertaken



5 / 7

.

Thermal treatment equip-

ment

Capacity of 25,000 tons pa ≈ 70%

From the details provided by the "indicators of physical execution" provided in

the progress reports it is not straightforward to get a full picture of the progress

made for most of the above main investment items. However, it appears that the

satisfactory progress has been made.

Budget execution The below table shows that the Stage 2 is expected to be within budget, while

Stage 1 actually has been cheaper to implement than originally expected.

Budget Revised Deviation

Stage 1 EUR 17.0 m EUR 14.8 m EUR -2.2 m

Stage 2 EUR 29.0 m EUR 29,0m 0

Total expenses EUR 46.0 m EUR 43.8 m EUR -2.2 m

Schedule review The progress reporting does not reveal any problems will keeping to the time

schedule. However, DG Regio emphasises that both stages have been pro-

longed somewhat in time.

3.2 Operation period

No data for the operational period is available from the used documentation,

but since the project has started its trial period, this is expected to be the case.

4 Tasks of possible ex-post evaluation

This project raises a number of interesting questions to be considered in an ex

post evaluation:

Capital costs/alternative options

Technical options Since the presentation of the alternative technical op-

tions is unclear, it would be useful to revisit these.

Baseline The baselines of "do nothing" and "fulfilling regulation

requirements at minimum costs" need to be clearer

elaborated.

Utilization rate

Demand analysis Although, a stable demand for waste treatment is ex-

pected, the expectations to the capacity needs should

be revisited.

Benefits

Benefits to the inhabitants The feasibility studies do not address the inhabitants'

willingness-to-pay for better waste management - and

so how this compares with the tariffs to be paid.



6 / 7

.

Operating costs

All categories Verify and forecast.

Use of conversion factors There is a need to covert some of the financial costs

into socioeconomic measures.

Externalities

Greenhouse gas emissions Find estimates of costs and benefits.

Noise and odours Find estimates of costs and benefits

Ground water protection Find estimates of costs and benefits

5 Recommendations

This project raises a number of interesting questions, and should be considered

for inclusion in the final group of 10, subject to the selection criteria.

6 Sources of information

City Municipality of Celje (2004a), Feasibility Study including Cost-Benefit

and Financial Analysis.

City Municipality of Celje (2004b), Regional Waste Management Centre Celje

(RWMC Celje), Cohesion Fund - 2000-006, Application for Assis-

tance.

City Municipality of Celje (2005a), Regional Waste Management Centre Celje,

Stage 2 - Mechanic Biological and Thermal Treatment of Municipal

Waste, Feasibility Study.

City Municipality of Celje (2005b), Regional Waste Management Centre Celje,

Stage 2 - Mechanic Biological and Thermal Treatment of Municipal

Waste, Cohesion Fund - 2000-006, Application for Assistance.

Republic of Slovenia, Government Office for Local Self-Government and Re-

gional Policy (2005), Additional clarifications in relation to the project

CCI 2005/SI/16/C/PE/001: Regional Waste Management Centre -

Stage 2.

Republic of Slovenia, Government Office for Structural Policies and Regional

Development (2004a), Additional clarification - part II in relation to

the project 2004/SI/16/C/PE/001 RWMC Celje.

Republic of Slovenia, Government Office for Structural Policies and Regional

Development (2004b), 2004SI16CPE001 Cohesion Fund Slovenia -

Regional Waste Management Centre Celje. Answers to comments and

requests for additional clarification.



7 / 7

.

Republic of Slovenia, Ministry of Finance (2007), Application for the 4th in-

terim payment for CF project Regional Waste Management Centre

Celje.

Republic of Slovenia, Ministry of Finance (2008), Application for the 4th in-

terim payment for CF project Regional Waste Management Centre

Celje - Stage 2 - Mechanic Biological and Thermal Treatment of Mu-

nicipal Waste.