Cost time project performance in megaprojects in time project performance in megaprojects in general and ... • t is behind schedule ... – Oil and gas projects, e.g. refineries,

Cost‐time project performance in megaprojects in general and nuclear in

particular

Dr Giorgio Locatelli PhD CEng FHEA

Lecturer in Infrastructure Procurement and ManagementSchool of Civil Engineering ‐ University of Leeds

[email protected]

1

Agenda

• Introduction: do we need this?

• Megaprojects Performance: how bad are

they?

• Reasons and remedies behind cost and

time escalation

• The “cost action experience”

• Take home messages

Technical Meeting on the Economic Analysis of HTGR and SMR ‐ IAEA 25 to 28 August 2015 2

Introduction: do we need this?Let’s introduce the elephant


The Rickover Effect

A paper reactor [new reactor concept] has the following characteristics: • it is simple• it is small• it is cheap• it is lightweight• it can be built very quickly• very little development is required and • it will use off the shelf components; it is in the study phase and not being built now.

By contrast a real reactor has the following characteristics: • it is complicated• it is large; it is heavy• it is being built now• t is behind schedule• requires an immense amount of development on apparently trivial items;• takes a long time to build because of its engineering development problems.

Hyman G. Rickover in pp. 158‐159 The Rickover Effect (1992) by Theodore Rockwell, Naval Institute Press.


Introduction: do we need this?Are nuclear projects intrinsically evil?


R² = 0.5522

4.00

4.50

5.00

5.50

6.00

6.50

28‐8‐76 18‐2‐82 11‐8‐87 31‐1‐93 24‐7‐98 14‐1‐04 6‐7‐09

YEAR

S

CONSTRUCTION STARTED

CONSTRUCTION TIME FOR THE STANDARD 1 GW KOREAN PWR

HANBIT

HANUL

KORI

SHIN‐KORI

SHIN‐WOLSONG

Megaprojects Performance:Introduction

• A megaproject is an extremely large‐scale investment project costing hundreds millions of euro and having considerable impacts on communities, environment and shareholder’s value.

• There are several types of megaprojects, including:– Civil infrastructural projects:

• Rail and rapid transit projects • Bridge and tunnel projects• …

– Oil and gas projects, e.g. refineries, long pipelines, or large LNG regasification plants.– Power plants:

• Nuclear power • Large wind firms (e.g. Great Gabbard in UK),• Large solar plants (e.g. Andasol in Spain).

– Airport projects– Aerospace projects– …


Megaprojects Performance:Transportation infrastructure

• Inaccuracy in cost forecasts in constant prices is on average 44.7% for rail, 33.8% for bridges and tunnels, and 20.4% for roads.

• For the 70‐year period accuracy has not improved.

• Average inaccuracy for rail passenger forecasts is –51.4%, with 84% of all rail projects being wrong by more than ±20%.

• For roads, average inaccuracy in traffic forecasts is 9.5%, with half of all road forecasts being wrong by more than ±20%.

• For the 30‐year period accuracy has not improved.


Flyvbjerg, B., 2006. From Nobel Prize to projectmanagement: Getting risks right. Project ManagementJournal, 37, pp.5–15.

Megaprojects Performance:Large Dams

• Database encompasses 245 large dams, built between 1934 and 2007, on five continents, in 65 different countries

• Considering their cost overrun– 3 out of 4 large dams suffered a cost overrun.– Actual costs were on average 96% higher than estimated costs; the median was 27% .– Differences among regions are not significant. – The typical forecasted benefit‐to‐cost ratio was 1.4. In other words, planners expected

the net present benefits to exceed the net present costs by about 40%. – Project type (e.g., hydropower, irrigation, or multipurpose dam) or wall type (earthfill,

rockfill, concrete arch, etc.) does not influence cost overrun.– Irrespective of the year or decade in which a dam is built there are no significant

differences in forecasting errors. There is little learning from past mistakes. – The rate of cost overrun outliers increases with increase in dam size measured either in

installed hydropower generation or wall height.


Ansar, A. et al., 2014. Should we build more largedams? The actual costs of hydropower megaprojectdevelopment. Energy Policy, 69, pp.43–56.

Megaprojects Performance:Large Dams

• Regarding schedule overrun – 8 out of every 10 large dams suffered a schedule overrun.– Actual implementation schedule was on average 44% (or 2.3 years) higher than the

estimate with a median of 27% (or 1.7 years). Like cost overruns, the evidence is overwhelming that implementation schedules are systematically biased towards underestimation

– There is less variation in schedule overruns across regions than cost overruns. – Large dams built everywhere take significantly longer than planners forecast. – There is no evidence for schedule estimates to have improved over time.


Ansar, A. et al., 2014. Should we build more largedams? The actual costs of hydropower megaprojectdevelopment. Energy Policy, 69, pp.43–56.

Megaprojects Performance:IPA database (318 projects)

• The 35% of the projects succeeded.

• By contrast, the failures are truly miserable projects:

– 40% constant currency overrun

– very expensive in absolute terms.

– Delay by 28 % – 15% slower than a

competitive schedule. – they averaged only 60% of

planned production in the first year.

• The oil and gas production sector fares the worst; 78% of megaprojects in this industrial sector are classified as failures.


Merrow, E.W., 2011. Industrial megaprojects:Concepts, strategies, and practices for success, Wiley.com.

Megaprojects Performance:Megaproject cost action


Reasons behind cost and time escalationInadequacy of the methodologies used

• A normal distribution of error with an average close to zero should be expected, however the actual distribution is not normal with an average error much greater than zero (actual costs are usually over budget).

• It is reasonable to expect an improvement over time of assessment methods due to more sophisticated forecasting models and modern informatics tools. However, over time, the estimations do not improve.

It is not the model that accounts for most of the differences


“Project”, “Human” and “Robotic” problems


Reasons behind cost and time escalationProject characteristics

Statistical Results of IPA 318 projects database

• Regulatory Climate and Stability: permitting problems cost overruns.• Clear and coherent business objective• Quality and reliability of Basic data (necessary to develop the FEED)• Radical new technology• Project team characteristics• Quality of the Front End Loading, and Front End Engineering and Design.• Site remoteness • Contractual forms• Incentives (including Schedule Incentives)• Government role• Risk Management strategy• Labour Availability• Project governance



Reasons behind cost and time escalationOptimism bias and strategic manipulation

• (Wachs 1990) interviewing government officials, consultants and planners, noted that estimations were biases. They manipulated forecasts to achieve values, not justified in technical terms, but acceptable for their superiors to implement the project.

• (Lovallo & Kahneman 2003) show that cognitive biases and organizational pressures push managers to provide optimistic forecasts.

• (Flyvbjerg et al. 2005) incorporates the results of previous contributions adding other reasons:

– Opportunism. This reason explains the phenomenon in terms of personal interest.– Optimism bias. The most common psychological explanation is the presence of a certain

“optimism” which induces promoters to consider each assumption positively. Such optimism is misleading for the promoters themselves, and not an intentional error.


Locatelli, G. & Mancini, M., 2010 Small–medium sizednuclear coal and gas power plant: A probabilistic analysisof their financial performances and influence of CO2 cost.Energy Policy, 38(10), pp.6360–6374.

Still, a man hears what he wants to hearAnd disregards the rest

SIMON & GARFUNKEL ‐ The Boxer

Reasons behind cost and time escalationOK, let’s test you

• I will ask 10 questions – general knowledge

• I know that you don’t know all the answer. On something you have little idea• THIS IS LIKE IN THE ECONOMICS OF NEW NPP – A LOT OF UNKNOW

• You need to answer anyway, providing a confidence interval. • The interval should be your 90% confidence interval• YOU WANT THAT THE CORRECT ANSWER IS IN WITH 90% CHANCE

• I know that providing as interval “– infinite; + infinite” you are sure• The point is 90% of the time, no more no less

Example of questions:• How many habitants in Japan?• What’s the weight of an adult Lion? (KG or pounds)


Take pen and paper



1. The year in which Ludwig van Beethoven was born2. Length of the Nilo River3. Number of passenger in 2014 year landing at the JFK Airport in NY4. Time it takes the sunlight to reach the earth (second) 5. Average diameter of Jupiter6. Number of knives, forks and spoon in the white house7. Number of actively spoken language in the world8. Gestation period (in days) of an Asian Elephant9. Number of babies born every day worldwide10. Length of time a snail can sleep if it isn’t disturbed (days)


Adapted from: Russo and shoemaker a static brain, an odd trivia fact

Ready to see the answers?



1. The year in which Ludwig van Beethoven was born 17702. Length of the Nilo River 4187 miles or 6738 Kilometres3. Number of passenger in 2014 year landing at the JFK Airport in NY 53,254,3624. Time it takes the sunlight to reach the earth (second) 492 s (8 min, 12 s)5. Average diameter of Jupiter 86881 miles or 139,822 Kilometers6. Number of knives, forks and spoon in the white house 13,0927. Number of actively spoken language in the world 6,0008. Gestation period (in days) of an Asian Elephant 645 days9. Number of babies born every day worldwide 353,00010. Length of time a snail can sleep if it isn’t disturbed (days) 1095 days


Adapted from: Russo and shoemaker a static brain, an odd trivia fact

Reasons behind cost and time escalationOverconfidence

We are confident that our judgments are correct


Reasons behind cost and time escalationSo, in each mathematical model


Mathematical model

DCFROA…

Inputs Outputs

Reasons behind cost and time escalationStakeholders mistakes

Merrow’s experience

• Greed• Schedule pressure• Poor bidding phase between a resource holder • Reduction on the upfront cost, leading to poor quality FEED• Unrealistic cost estimations (see previous section)• Poor risk allocation • Excessive pressure on the project manager and “blame culture”



So, Let’s have a “robot society”Robots are honest, mathematically minded


Reasons behind cost and time escalationWinner course

• The winner's curse is a phenomenon that occur in common value auctions with incomplete information, such as bidding in projects.

• In auction, the winner will tend to overpay. In a project the sponsor will award the bid to the lowest offer or “best value for money”

• The winner may "cursed" in one of two ways:1) the winning bid exceeds the value of the auctioned asset such that the winner is worse

off in absolute terms; 2) the value of the asset is less than the bidder anticipated, so the bidder may still have a

net gain but will be worse off than anticipated.

• Among many bids the robot will select the “most convenient” that is likely to be, again, the most optimistic! EVEN IF ESTIMATES ARE HONEST – “Incomplete informations”


Reasons behind cost and time escalationRegression toward the mean

If a variable is extreme on its first measurement, it will tend to be closer to the average on its second measurement

EXAMPLE: PMP Exam to select the “5 most knowledgeable team members” – The certification exam has 200 multiple‐choice questions (4 answers, 1 right)– 100 candidates attempt the exam. Each candidate know X question and guess 200 – X– Top 5 candidate scored, on average, 180– If we repeat the exam the Top 5 are likely to change, The original “Top 5” would score

lower.

WHY? The Top 5 where both knowledgeable and lucky. In the second attempt knowledge stays, luckiness, doesn’t

– On average they knew 120, guessed 80. Very lucky, they got 60 right, so 120+60 = 180– In reality they should had got 120 + 80 x 0.25 = 140– 180 – 140 = Lucky effect– Next attempt will be averaged at 140.

This applies also in choosing the best subcontractor based on the “last performance”


The “Megaprojects Cost Action”Introduction

MEGAPROJECT ‐ The Effective Design and Delivery of Megaprojects in the European Union • A network of ~80 researchers from 22 European countries• Funded under the auspices of the ESF COST Action programme by the European

Commission• Looking at what can be learnt across sectors • Started May 2011. End May 2015.

I was the disseminator coordinator and the “Methodology” Working group leader

27Technical Meeting on the Economic Analysis of HTGR and SMR ‐ IAEA 25 to 28 August 2015

Transport Megaprojects Energy Megaprojects

~ 19 yr ~ 10 yr

~ 40% ~ 80%

Nuclear Megaprojects

MochovceHinkley PointFlamanvilleOlkiluoto

The “Megaprojects Cost Action”Performance Comparisons


The “Megaprojects Cost Action”The Meta Cross Case Analysis

• Capture MEGAPROJECT cases in a standard template– Project Stakeholders– Project Management– Project Performance– Project Environment– Project Timeline

• Look across cases to see how performance relates to key features of the MEGAPROJECT Case


megaproject characteristics

megaproject performance

Independentvariables

dependentvariables

The “Megaprojects Cost Action”The key idea


Relating megaproject characteristics and meagproject performance

The “Megaprojects Cost Action”The Protocol

• Delineation of ~50 megaproject characteristics – independent variables binary in nature

• First of A Kind• Financial Support from national government• Offshore project

• Simple expression of megaproject performance against time and cost – dependant variables in binary nature– 10% threshold

• Capture both of the above in an excel matrix

• Perform statistical analysis– Fisher exact test– Machine learning

• Analyse the results


The “Megaprojects Cost Action”In a nutshell

Real worldQualitative, quantitative,

complex, dynamic…

Mathematical / Statistical test

Precise algorithms.

CODING‐ Give definition‐ Rules‐ Thresholds‐ Goal: reduce

arbitrary interpretations

FEEDBACK‐Better definitions‐Missing data‐ New attributes‐ Delete attributes


The “Megaprojects Cost Action”Selected Results

Over budget Delay construction

Delay planning

The project has national public acceptability +++

Environmental groups have been engaged ex‐ante, not ex post +++

The project has a strong regulation system ‐‐‐ ‐‐‐

There is planned a long term stability in usage and value + ++

Financial Support from national government ++

The majority of the national population trust the national authority ++

There was a formal litigation procedure during the contract between Client and EPC ‐‐

The megaproject is composed of more than 1 identical independent unit ‐‐

a) The project is modular ‐ dependent modules ++

Offshore project ++ ‐

The project has an SPE/SPV +++ ‐‐

+++ Strongly Supportive, ++ Supportive, + Weakly Supportive, ‐‐‐ Strongly Opponent, ‐‐ Opponent, ‐Weakly Opponent.33Technical Meeting on the Economic Analysis of HTGR

and SMR ‐ IAEA 25 to 28 August 2015

Take home messages

• Large infrastructural megaprojects are often over budget and late – Megaprojects are overbudget all over the world, even if with different rates. – There is not improvement over the decades. The project performance of today are

roughly similar to the one of ten, twenty, thirty years ago.– The type of technology is not a key factor: This applies to Oil & Gas project as well as to

Dams, even if nuclear power plants can have overbudget above the average.

• Nuclear power plants are megaprojects – All nuclear project delivered in Europe are over budget and late.– Stakeholders play a key role: the same PWR reactors was delivered overbudget and late

in USA, while in Korea are delivered on time and on budget.

• We need more empirical research

• Project attributes matters

• Companies project portfolios of closed projects contains key information!


Understanding cost escalation and delays

What nuclear projects can learn from the megaproject sector

Dr Giorgio Locatelli PhD CEng FHEA

Lecturer in Infrastructure Procurement and ManagementSchool of Civil Engineering ‐ University of Leeds

[email protected]

35Technical Meeting on the Economic

Analysis of HTGR and SMR ‐ IAEA 25 to 28 August 2015

Documents

Cost time project performance in megaprojects in time project performance in megaprojects in general and ... • t is behind schedule ... – Oil and gas projects, e.g. refineries,