Project methodology & application– for TER

By the External ConsultantD. Tsamboulas

Objective identify projects prioritization/

categorization, support elaboration of a medium and

long-term investment strategy in the region concerned

encourage the realization of projects that have good chances of implementation and fall within the TER Master Plans objectives.

Phases of Methodology PHASE A – Identification

PHASE B – Forecasting

PHASE C – Evaluation

PHASE D – Prioritisation

Identification Phase Identification of the projects that

worth further analysis and evaluation according to their.. Relevance Readiness Viability

…countries complete TEMPLATES 1 and 2

TEMPLATE 1 – Identified Projects

TEMPLATE 2B – Rail and related infrastructure Project Fiche

TEMPLATE 2C – Maritime/ Port Fiche

Forecasting Phase Any official forecasts or official estimations could

serve in verifying and finalize consultants’ forecasts.

Alternative demand scenarios are to be produced in the framework of WP3, in a qualitative macro-scale based on the expected economic development of the countries concerned as well as other characteristics.

If forecasted data are not collected, then WP3 results will be used. For any forecasted data provided, consistency with the macro-level forecasts (elaborated in WP3) will be investigated.

Evaluation Phase Selection of Criteria – 3 hyper-criteria

CLUSTER A: Socio-economic return on investment (CA)

CLUSTER B: Functionality and coherency of the network (CB)

CLUSTER C: Strategic/ Political concerns regarding the network (CC):

Quantification of Criteria - Scores Weighting/ Hierarchy of Criteria – Delphi/Paired

Comparison Total Performance of Project

(=> to assist Prioritization on the next Phase)

Selection and Quantification of Criteria -11. Degree of urgency A: Immediate requirement (in the next 2 years-

until 2005), B: Very urgent (between 2005 and 2010), C: Urgent (between 2010 and 2015), D: May be postponed for some years (between 2015 and 2020), E: To be reconsidered later (after 2020)

2. Cost effectiveness A: Excellent (IRR more than 15%), B: Very good

(13-15%), C: Good (10-13%), D: Acceptable (4,5-10%), E: Low (less than 4,5%)

Selection and Quantification of Criteria -23. Relative investment costs (costs/GDP) Rehabilitation/upgrading of railways: A: less than (min cost

of this project type/GDP)%; …(intermediate values to be calculated assuming linearity, see next figure)… E: more than (max cost of this project type/GDP)%

New Railway Line: A: less than (min cost of this project type/GDP)%; …(intermediate values to be calculated assuming linearity , see next figure)…E: more than (max cost of this project type/GDP)%

Figure 1

ED= DC= CB= BA=1 and

A=5, B=4, C=3, D=2, E=1

X1: the min cost of the project

type observed in the country (in million € or $).

X2: the max cost of the project

type observed in the country (in million € or $)

X3: the considered project cost

(in million € or $)

Country’s GDP given in million € or $

Selection and Quantification of Criteria -3 4. Level of transport demand

Railways: A: present traffic more than 140 trains a day; B: present traffic from 100 to 140 trains a day; C: from 60 to 100 trains a day; D: from 25 to 60 trains a day; E: less than 25 trains a day

5. Financing feasibility A: Excellent, B: Very Good, C: Good, D: Medium, E:

Low

Selection and Quantification of Criteria -46. Relative importance of international demand of

traffic (passengers) A: more than 30 % of total traffic; B: from 25 to 30 % of

total traffic; C: from 15 to 25 % of total traffic; D: from 7 to 15 % of total traffic; E: less than 7 % of total traffic

7. Relative importance of international demand of traffic (goods)

The same as 6.

8. Alleviation of bottlenecks A: Satisfactory, B: Adequate, C: Medium, D: Inadequate,

E: Unsatisfactory

Selection and Quantification of Criteria -59. Interconnection of existing networks A: Missing Link, B: Natural Barrier, C: Improve the

connection, D: No influence, E: Averse effects on rest of network

 10.Technical interoperability of network A: No interoperability problems, B: Minimal

interoperability problems, C: Tolerable Interoperability problems, D: Serious interoperability problems, E: Unsolvable interoperability problems

 

Selection and Quantification of Criteria -611.Border effects A: No border problems, B: Minimal border problems, C:

Tolerable border problems, D: Serious border problems, E: Unsolvable border problems

 

12.Political commitment A: Strong, B: High, C: Medium, D: Adequate, E: Low 

13. Regional and international cooperation A: Satisfactory, B: Adequate, C: Medium, D: Inadequate,

E: Unsatisfactory 

Selection and Quantification of Criteria -714. Historical/ heritage issues

A: No effects, B: Minimal effects, C: Tolerable/ Reversible effects, D: Serious effects, E: Irreversible effects

 15. Economic impact A: Strong impact, B: High impact, C: Medium

impact, D: Low impact, E: No impact

 

Criteria Scores A value is 5 (the highest) in terms of score.

Respectively for value E, is 1 (the lowest).

Therefore: where:

J = A, B or C and i = 1,….,5

The template for criterions scores is TEMPLATE 3.

5,1JiC

TEMPLATE 3 Project Criteria Scores

Criterion Scores from Country Experts Good communication between the externals and

the country experts is necessary. For instance, war effects or weather that

destroyed sections of transport infrastructure. If the external consultants for some reason will not identify them as “missing links” in criterion CB4,

then country experts must do it, when reviewing the criterion scores.

Weighting/ Hierarchy of Criteria Country experts have received TEMPLATE 4

with proposed default set of weights, derived by the consultants, using Paired Comparison Matrix.

The sum of criteria weights should be 1.

  Therefore: and where: J= A, B or C and i = 1,….,5

1,0JiW 15

1

C

AJ iJiW

Paired Comparison Paired comparison approach is a scaling approach. Only one question to be answered is “is this

criterion more important than the other?”. This means that the paired comparison matrix (see

Table I next) can be filled with zeros and ones, where one represents “is more important”.

By adding these values over the column, a measure is obtained for the degree to which a criterion is important compared to all other criteria, if finally these measures are standardised (see Formula I next), a set of criteria weights is created.

W1 W2 … WN W1 W2 … WN

Table I An example of Paired Comparison matrix

Standardised score wi = (I) scoresraw

wscoreraw i

''

..''

TEMPLATE 4 Project Criteria Weights

Criteria Weights from the Country Experts As an example, if country A wishes to put high priority

for sections of the network destroyed by war or weather. Then, the experts have to classify them as missing links, and in the weighting they have to put high values in the criterion CB4, as well as criterion CC2 .

Another example is when a country wishes to promote a link that it considers important as a domestic link: in such a case it has to put a very low weight to criteria (CB1), (CB2), (CC1).

Furthermore, if country experts provide their own weights, with the proper justification then they will be used instead of default weight introduced by the external consultants.

Projects Total Score/ Performance -1 To derive the project’s total score in each

country we use the following relationship:

T.S.Project/Country =

where:CJi [1,5]

WJi [0,1]J = A, B or C andi = 1,….,5

TSProject/Country [1,5]

C

AJ iJiJi WC

5

1

*

Projects Total Score/ Performance -2 For Total Score per Project, we use

Country/ Spatial Weights (SW).

SWCountry = % of projects length in the country/ total project’s length.

So the Total Score per project will be:

T.S.Project = T.S.Project/Country * SWCountry

Prioritization Phase The combination of the criterions scores

and priorities puts each project in one of the four priority categories.

If the project scores between 4-5 then it belongs to priority category I.

If the project scores 3 then it belongs to priority category II.

If the project scores 2 then it belongs to priority category III.

If the project scores 1 then it belongs to priority category IV.

Priority Categories I: projects, which may be funded and implemented

rapidly, including on-going projects up to 2010. II: projects requiring some additional investigations

for final definition before likely financing, or planned for implementation up to 2015

III: projects requiring further investigations for final definition and scheduling before possible financing, or planned for implementation up to 2020.

IV: projects to be implemented in the long run, including the projects where insufficient data exists.

Prioritization Results If a project results i.e. to be in priority category II

according to TER Methodology but according to Van Miert prioritization belongs in another Priority Class (i.e. A, B or C) then Van Miert’s prioritization will be followed, at least for the EU member states (current and the ones to be members in 1/5/2004).

On the other hand, in the unlikely case that the priority of a project differs with the national priority, a more thorough analysis on the underlying assumptions will take place.

Assumptions for Criteria Quantification

Criterion CA1 Degree of urgency Where the countries indicated the time plan of the projects elaboration,

the time plan was used for the quantification of this criterion Where time plan was not mentioned but in the goals -or the expected

benefits of the project-, the project was described as necessary for compliance with EU directives/ policies, or for decreasing unemployment or for other very important reason and at the same time the total implementation period of the project was between 1-5 years, the score given was 5=A 

Where time plan was not mentioned but in the goals -or the expected benefits of the project-, the project was described as necessary for compliance with EU directives/ policies, or for decreasing unemployment or for other very important reason and at the same time the total implementation period of the project was between 5-10 years, the score given was 4=B

For any other case (of implementation years), the score given was 3=C If there weren’t any available data to support the quantification of this

criterion, then the score given was the lowest 1=E unless there was a good justification and then the score given was 2=D, assuming that the project merits some consideration.

Criterion CA2 Cost effectiveness If the IRR was available then the quantification was

done as described earlier. If no IRR was available –without any justification-

the score given was the lowest 1=E If the IRR was not available with a strong

justification like i.e. the feasibility study is not completed yet but it is on-going the score given was the 3= C, assuming that the project merits consideration.

For any other case of missing IRR, the score given was 2=D.

Criterion CA3 Relative investment cost Where the min and max values of a country for

each project type were not available (and they weren’t available in all countries that so far have sent data) the following assumption was made: the min and the max values from the group of projects presented were used.

Criterion CA4 Level of transport demand If traffic data were available then the quantification

was done as described earlier. If no traffic data were available –without any

justification- the score given was the lowest 1=E If traffic data were not available with a strong

justification like i.e. the traffic study is not completed yet the score given was the 3= C, assuming that traffic is such to justify the project consideration.

For any other case of no traffic data the score given was 2=D.

Criterion CA5 Financing feasibility Where the expected IRR was very high, the private sector

financial participation was also very high and if the financial study was completed and accepted, the score would be 5=A.

Where the expected IRR was medium towards high, the private sector financial participation was also medium towards high and if the financial study was completed and accepted, the score would be 4=B.

Where the expected IRR was medium, the private sector financial participation was also medium and if the financial study was completed and accepted, the score would be 3=C. 

In any other case the score was 2=D, except where no IRR or no feasibility study was available or IRR was really low and there were no private funds as well, then the score given was 1=E.

Criteria CB1 & CB2

Relative importance of international demand of traffic (passengers & freight)

If traffic data were available then the quantification was done as described earlier.

If no traffic data were available –without any justification- the score given was the lowest 1=E

If traffic data were not available with a strong justification like i.e. the traffic study is not completed yet the score given was the 3= C, assuming that international traffic is such to justify the project consideration.

For any other case of no traffic data the score given was 2=D.

Criterion CB3 Alleviation of bottlenecks If traffic data “before” and “after” (through forecasting),

were available then their comparison indicated whether alleviation of bottlenecks took place. Based on these results the scores are produced.

If no traffic data were available –without any justification- the score given was the lowest 1=E

If no traffic data were available “before” and “after”, with a strong justification like i.e. the traffic study is not completed yet the score given was the medium one 3=C, assuming that the difference in traffic “before” and “after” the project implementation is such to justify the project consideration.

For any other case of missing traffic data the score given was 2=D.

Criterion CB4 Interconnection of existing networks If in the projects description was mentioned that

this project (road or rail) will alleviate a missing link the score given obviously is 5=A.

If in the projects description was mentioned that this project (road or rail) will improve the condition of the current situation the score given obviously is 3=C.

If nothing of the above was mentioned the quantification was done based on unique characteristics of each project. However for all considered projects the score given was 3=C and in some cases 5=A.

Criterion CB5 Technical interoperability of network If a project was cross-border and based on the

descriptions of the participating countries interoperability problems exist, then 3=C or lower.

4=B is used only in not so severe interoperability cases, where data exists and it can be verified.

However all projects considered were inside country’s borders so the score given to all was 5=A.

Criterion CC1 Border effects If a project was cross-border and based on the

descriptions of the participating countries border problems exist was 3=C or lower.

4=B is used only in cases when border problems exist, but are not so critical.

However, all the projects considered were inside country’s borders so the score given to all was 5=A.

Criteria CC2, CC3, CC4,

strategically sensitive criteria For the three strategically sensitive criteria

Political commitment (CC2)

Regional and international cooperation (CC3)

Historical/ heritage/ environmental issues (CC4)

the score given to all projects was 5=A, since the consultant believes it is rather impossible for any country -and concerning any type of project- not to be politically committed to it and also not to try for the best regional cooperation as well as to ignore environmental/ heritage issues.

In cases that a country expert assigns a different value then the default value will be changed.

Criterion CC5 Economic impact Where a) the expected IRR was high, b) the travel time was

reduced, c) safety was increased, d) access to ports or other terminals was easier and e) socio-economic parameters (economic development, unemployment) of the region were expected to benefit from the project, the score given was 5=A.

Where a) the expected IRR was medium towards high, b) the travel time was reduced, c) safety was increased and d) socio-economic parameters (economic development, unemployment) of the region were expected to benefit from the project, the score given was 4=B. 

Where a) the expected IRR was medium and b) the travel time was reduced score given was 3=C.

For any other case the score given was 2=D, except where no data were available and then the score given was 1=E.

Example of Evaluation Methodology for TER – using assumptions

Lithuanian Project: Modernization of Telecommunications on the Rail Corridor IXB

Example steps Complete Project Fiche – see next Derive Criteria Scores Use default set of Criteria Weights Derive Project Total Score Prioritize Project

TEMPLATE 2B – Rail and related infrastructure Project Fiche

Criteria Scores-1

1. Degree of urgencyBased on the data of TEMPLATE 2B the project is already under construction and the total construction time 2 years, therefore project’s implementation is characterized as A: immediate requirement.

CA1=5

2. Cost effectivenessBased on TEMPLATE 2B, there is no IRR mentioned so the project’s cost effectiveness is characterized as E: Low (IRR higher than 4,5%).

CA2=1

Criteria Scores-23. Relative investment costs (costs/GDP) Based on the data of TEMPLATE 2B, country’s GDP and Figure 1

the project’s relative investment cost is characterized as A.

CA3=5 (or taken directly from Figure 1 -see example next)

4. Level of Transport DemandBased on the data of TEMPLATE 2B, in two out of three section the level of transport demand is more that 60 and less than 100 trains a day (64 and 93 trains), therefore the project’s level of transport demand is characterized as C: from 60 to 100 trains a day.

CA4=3

X1: 3,1 million €

X2: 500 million €

X3: 7 million €

GDP =12.039 millions €

Therefore: 

(X1/GDP)% = 0,026 %

(X2/GDP)% = 4,153%

(X3/GDP)% = 0,058 %

Criteria Scores-35. Financing Feasibility Since in TEMPLATE 2B no IRR is available and there are no private

funds as well, then project’s financing feasibility is characterized as E: Low.

CA5=4

6. Relative importance of international demand of traffic (passengers)Based on TEMPLATE 2B, there are no international traffic data–without any justification-, therefore the project’s relative importance of international demand of traffic is characterized as E: less than 7 % of total traffic

CB1=1

Criteria Scores-47. Relative importance of international demand of traffic

(goods) Based on TEMPLATE 2B, there are no international traffic data–

without any justification-, therefore the project’s relative importance of international demand of traffic is characterized as E: less than 7 % of total traffic

CB2= 1

8. Alleviation of BottlenecksComparing total traffic data “before” and “after” and based on expert’s opinion the project’s alleviation of bottlenecks is characterized as A: Satisfactory.

CB3=5

Criteria Scores-5

9. Interconnection of existing networks Since it is a modernization of a line, the project will just

improves connection so based on expert’s opinion the project’s interconnection of existing networks (in this case existing lines) is characterized as C: Improve the connection.

CB4= 3

10. Technical interoperability of networkSince it is an inside-border project, based on expert’s opinion the project’s technical interoperability in the network is characterized as A: No interoperability problems.

CB5=5

Criteria Scores-6

11. Border effects The project is inside borders, therefore regarding the border

effects is characterized as A: No border problems.

CC1= 5

12. Political CommitmentThe political commitment is characterized as A: Strong.

CC2=5 (Consultant Default Value)

13. Regional and International Cooperation The regional cooperation (since there is no international

cooperation) is characterized as A: Satisfactory.

CC3=5 (Consultant Default Value)

Criteria Scores-7

14. Historical/ heritage Issues There are no effects on historical heritage, therefore the

project scores A: No effects.

CC4= 5 (Consultant Default Value)

15. Economic ImpactNo data were available (no IRR, no travel time data, no cost data after the project etc.), so score it is expected to be E: No impact.

CC2=1

See TEMPLATE 3 completed next..

TEMPLATE 3 Criteria Scores

TEMPLATE 4 Criteria Weights

Project’s Total Score In our case is only one country so spatial

weighting was unnecessary Based on methodology described earlier the

calculation of Total Score is presented in TEMPLATE 5. (It is the weighted sum of criteria scores or else TEMPLATE 5 is the result of multiplying TEMPLATES 3 and 4)

TEMPLATE 5 Project Total Score

Prioritization of Project The Project Total Score is:

T.S. =3,36

Therefore the project belongs in Priority category:

II:projects requiring some additional investigations for final definition before likely financing, or planned for implementation up to 2015.

First evaluation for submitted TER projects See Matrix next results

for the countries that sent data, namely: Bosnia & Herzegovina, Bulgaria, Georgia, Hungary,Lithuania, Moldova, Romania, Slovakia,Slovenia and Turkey

Countries from which data are pending Austria, Greece, Italy, Czech Rep., Poland,

Belarus, Croatia, Serbia & Montenegro, F.Y.R.O.M, Russia Federation, Ukraine

HU

NG

AR

Y

HU

NG

AR

Y

LIT

HU

AN

IA

LIT

HU

AN

IA

Notes on the Matrix In total the TER projects are: 79 with

13% (else 11 projects) belong in Priority Category I

1% (else 1 project) belong in Priority Category I - II

63% (else 49 projects) belong in Priority Category II

23% (else 18 projects) belong in Priority Category III

None belong in Priority Category IV