Why Large Projects Fail More Often - Pemexuvirtual.pep.pemex.com/nuevo/Why_Large_Projects_Fail_ Final.pdf · Why Large Projects Fail More Often ... • Large projects in the Process

Why Large Projects Fail More OftenMegaproject Failures: Understanding the Effects of Size

Edward W. Merrow, President & CEOIndependent Project Analysis, Inc.

Joint Meeting of the AACEI National Capital Section and AmericanSociety of Mechanical Engineers Section

Wednesday, April 20, 2011

CONFIDENTIAL 2 INDEPENDENT PROJECT ANALYSIS

Framing the Issues

• Large projects in the Process Industries (oil, chemicals, minerals, power, pharmaceuticals) have substantially poorer outcomes than smaller projects

• By the time a project achieves megaproject status, over about $1 billion, the failure rate is two-thirds!

• Many more large projects are being developed than ever before in every industrial sector

• Our market forecast for this decade is for more rapid growth in megaprojects in history

• These projects represent a major source of financial risk to companies


The Key Questions

• Large projects are more difficult on almost every dimension of project management

• Do large projects fail more often simply because they are more difficult—or…

– Do we alter our practices?

– Do the projects not respond to what are good practices for smaller projects?

– Are practices not appropriate for the problems at hand?


Outline

• Databases

• Outcomes Change with Size

• Larger Projects Are More Sensitive to Practices

• Practices Vary with Project Size

• Implications and Recommendations


IPA’s Vantage Point on the Industry

• IPA evaluates capital projects for the process industries around the world

• In a typical year, we evaluate 700-800 projects

• Our evaluations usually occur twice on the front-end and twice at the end of the project, at the end of startup and after one year of operation

• As a result, we have developed very large global databases of capital projects


Depth and Quantitative Nature of Practice and Outcome Measurements

• Up to 3,500 questions per project; 13,000+ total projects, 318 megaprojects over 22 years– Number and type of questions/data depend on project

scope, industry sector, type of project> Greenfield, revamp, turnaround, new technology, pipelines

vs. plant facilities

> Categories are based on quantitative, statistically-validated evaluations of practices and their outcomes

> Data on practices – mixture of quantitative, yes/no, dates, text, likert scales; all projects include lengthy face-to-face interview with project team

• Key practices include clarity of business & project objectives, front end loading, and team integration


$3,250$197 (mean)

$58 (median)Project Cost (in millions 2011 US$)

Megaprojects Database

Process Facilities Database

Number of Owners Represented

Percent of Projects with Any New Technology

Average Execution Duration (months)

Average Authorization Year

Number of Projects

340

16%

22 (mean)

21 (median)

2000

4500

77

36%

43

2005

318

Research Database Description

Characteristics


Industries Represented

Process Facilities DatabaseRefining Refining

27%27%

Chemicals Chemicals 38%38%

MMM MMM 8%8%

PharmsPharms8%8%

Consumer Consumer ChemChem& Pulp/Paper & Pulp/Paper

7%7%

Distribution Distribution 5%5%

Onshore E&P Onshore E&P 4%4%

OtherOther3%3%

Refining Refining 21%21%

Chemicals Chemicals 10%10%

MMM MMM 15%15%

LNG LNG 8%8%

Power Power Generation Generation

3%3%

Distribution Distribution 2%2%

Other Other 2%2% Oil & Gas Production Oil & Gas Production

41%41%

Megaprojects Database


United States Process: 48%

Mega: 13%

EuropeProcess: 19%

Mega: 15%

AsiaProcess: 8%

Mega: 9%

OceaniaProcess: 4%

Mega: 9%

AfricaProcess: 3%

Mega: 9%

South AmericaProcess: 7% Mega: 19%

Geographical DistributionProcess Projects and Megaprojects

CanadaProcess: 5%

Mega: 7% Central Asia Mega: 4%

Middle EastProcess: 4% Mega: 15%

Caribbean Process: 1%

Mexico Process: 1%


Outline

• Database






Most Key Outcomes Degrade with Size

0.02DeclinesOperability

0.001

0.03

Decreases up to $650 million and then increases

Schedule Slip

0.0001Gets worseCost Competitiveness

0.001IncreasesCost Growth

Probability OutcomeAs Project Size Increases


Outcomes Diverge

• We mostly maintain control of projects up to about $600-750 million

• Over $750 million, project outcomes start to degrade very rapidly

• A wide chasm develops between good and poor projects:

– Good projects are genuinely excellent

– Poor projects tend to be poor on most every outcome


Costs grew (real) 25% +

Schedule Slipped 25% +

Execution time (Absolute Measure) 50% +

Severe and Continuing Operational Problems into Year 2 after startup

Yes

Defining Success and Failure

• We deem a project to be a failure if one or more of the following occurred:

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Megaprojects

Failure Rate

Overspent (Absolute Measure) 25% +

Projects <$500 MM


80%

COST INDEX – (Competitiveness)

PRODUCTION(Predictability)

EXECUTION SCHEDULE INDEX

(Competitiveness)

EXECUTIONSCHEDULE SLIP

(Predictability)

COST GROWTH(Predictability)

1.3

1.0

1.2

1.0

1.3

1.2

1.1

0%10%

30%

20%

0%

Failed Projects

Successful Projects (35 percent)

40%

100%

60%

1.1

10%

20%30%

Source: Merrow, Edward W. (2011). Industrial Megaprojects : Concepts, Strategies, and Practices for Success. Hoboken, New Jersey: John Wiley & Sons, Inc.

Megaprojects Split Into Radically Different Groups


80%

COST INDEX

PRODUCTION

EXECUTION SCHEDULE INDEX

EXECUTIONSCHEDULE SLIP

COST GROWTH

1.3

1.0

1.2

1.0

1.3

1.2

1.1

0%10%

30%

20%

0%

Failed Projects

Successful Projects (63 percent)

40%

100%

60%

1.1

10%

20%30%

Non-Megaprojects Separate Very Differently


Outline

• Database






-0%

-2%

-4%

-6%

-8%

-10%

-12%

-14%

$10MM -$25MM

$25MM -$100MM

$100MM -$250MM

$250MM -$500MM

Megaprojects

Effe

ct o

f Tea

m U

nder

stan

ding

Bus

ines

s O

bjec

tives

on

Cos

t Ind

ex

Clear Objectives Are Much MoreImportant for Megaprojects


Project ObjectivesProject ObjectivesProject Objectives Team IntegrationTeam IntegrationTeam Integration Roles &Responsibilities

Roles &Roles &ResponsibilitiesResponsibilities

ProjectImplementation

Process

ProjectProjectImplementationImplementation

ProcessProcess

• Specific project objectives have been developed

• Objectives have been clearly communicated to and are understood by all project team members

• All functions that can influence the project’s outcomes are represented on project team

• Team is adequately staffed; no TBD on org chart

• Roles, responsibilities, and expectations have been clearly defined

• Responsibilities and tasks have been agreed on

• Project team is aligned

• Problem areas have been identified

• Common work process is in place for developing and executing the project

• Process is well understood by project team

Components of Team DevelopmentPB4

Slide 18

PB4 Added this slide

For same reason we present clearly understood objectives, we present Team Integration,

And I want to show a bit more on the basis

Didn't shorten bullets on this, not as many as for FELPaul Bizot, 4/18/2011


Team Integration Is Important for AllBut Crucial for Larger Projects

-30%

-25%

-20%

-15%

-10%

-5%

0%$10MM -$25MM

$25MM -$100MM

$100MM -$250MM

$250MM -$500MM

Megaprojects

Effe

ct o

f Tea

m In

tegr

atio

n on

Cos

t Ind

ex


Front End Loading is a Key Practice

SiteFactors

SiteFactors

EngineeringDefinition

EngineeringDefinition

ProjectExecution

Plan

ProjectExecution

PlanFELFEL

• Equipment layout

• Soils data

• Environmental requirements

• Health & Safety requirements

• Local labor quality and availability

• Local materials availability

• Engineering tasks– Detailed scope– Feedstock/product

properties– Program Definition– Drawings which are

basis for design– Major equipment specs– Cost estimate

• Participation/buy-in of all who can influence project

• Contracting strategy• Team participants & roles• Integrated schedule• Plans for all phases and

resources• Cost/schedule controls

PB3

Slide 20

PB3 Added this slide

Again this audience probably has very weak knowledge of IPA, and minimal "process" knowledge, though quite possibly all of them areinvolved in projects, some of them very large projects

Consolidated the details of this slide somewhat.

Am ambivalent about sharing the details; again this goes back to sharing something of the robustness and comprehensiveness of the measurements. Paul Bizot, 4/18/2011


FEL is Most Important for Megaprojects

0%

10%

20%

30%

40%

Best Practical Good Fair Poor Screening

Effe

ct o

f FEL

on

the

Cos

t Ind

ex

FEL Index

Megaprojects

MISERABLEPOORFAIRGOODBEST

250 - 500

100 - 250

<100MM

Note that FEL is also crucialfor even “largish” projects


Turnover of the Project Leader Always HurtsBut It Destroys Megaprojects

-40%

-35%

-30%

-25%

-20%

-15%

-10%

-5%

0%$10MM -$25MM

$25MM -$100MM

$100MM -$250MM

$250MM -$500MM

Megaprojects

Effe

ct o

f PM

Tur

nove

r on

Pro

ject

Suc

cess

Rat

e


The Effects of Practices Increase with Project Size

• Large projects are very sensitive to practices because they are more complex

• Much more difficult to “work around” surprises because so much more has to be coordinated

• All of this is well known

• Therefore, logic would suggest that larger projects systematically follow better practices than smaller ones


Outline

• Database






Practices Actually Degrade with Size

• Team integration is much poorer for megaprojects (55 percent) than projects under $750 million (74 percent)

• Turnover increases

• One megaproject team in five admits it does not understand the business objectives versus one in ten for smaller projects

• Megaproject teams are much more likely to report being confused about trade-offs among outcomes

• FEL completeness actually declines as projects get large


25%

30%

35%

40%

45%

50%

10 50 100 150 200 250 300 350 400 450 500

Cost 2011$ (million)

Perc

ent o

f Pro

ject

s th

at E

xper

ienc

e Pr

ojec

t Man

ager

Tur

nove

r

Larger Projects Experience More Frequent Project Manager Turnovers

Pr < 0.001

Note: Controlling for Execution Schedule Index

Turnover rate in Megaprojects

Pr < 0.005


Most FEL Elements Degrade as Size Increases

Tends to degradeEngineeringTends to improveExecution PlanningDegradesOperational Health and SafetyDegradesPermittingNo changeSoils DefinitionDegradePlot Plans and ArrangementsDegradeLocal Material CostsDegradesLabor Cost and ProductivityDegradesLabor AvailabilityNo changeLocationChange with SizeElement of Definition (FEL)


0%

10%

20%

30%

40%

50%

60%

70%

Perc

enta

ge o

f Pro

ject

s

FEL Index


How Well Are Megaprojects Defined at Authorization?

Source: Industrial Megaprojects, op. cit.


What Difference Does it Make?

0%

10%

20%

30%

40%

50%

60%

70%

Percent in Group Percent Successful in Group

Perc

enta

ge o

f Pro

ject

s

FEL Index



As Projects Get Large…

• They are more likely to have multiple, sometimes competing, objectives

• Business sponsors tend to be less certain and clear about what they need for a successful venture

• It becomes much more likely that there are JV partners and otherstakeholders to align

• Governments take more interest

• We are more likely to require new technology

• Even if the technology is conventional, we are less likely to have complete basic data

• Permits are more important for large projects, but we are much less likely to have them in-hand at authorization

• The business role is to shape the opportunity in a way that makes project management practicable


Megaproject Development Is More Complex

FEL 2 Scope Development FEL 3 FEED

Venture Shaping Process

• Allocate value • Align stakeholders• Close the business deal• Stabilize the project environment

Project FEL Process

Basic Data Development

Project Timeline

Closure


Excellent FEL Completes a Complex Process

Clarity of Trade-offsAmong Outcomes

Clarity of theBusiness Objectives

ExcellentFEL

Best Practical FEL Doubles Chances of Success

TeamIntegration


0

10

20

30

40

50

60

70

Very Fairly Somewhat Unclear

Under $500MM Megaprojects

Perc

ent o

f Pro

ject

sOnly a Slight Difference in Clarity of Business

Objectives Between Megas and Non-Megas

Team Understanding of Project Business Objectives


0

10

20

30

40

50

60

70

80


Under $500MM Megaprojects

Perc

ent o

f Suc

cess

fulP

roje

cts

But Differences in Effects Are Huge



Causal Path Runs Through Team Integration and Trade-offs to FEL

Clarity of Trade-offsAmong Outcomes

Pr>|t|< .0001


ExcellentFEL


TeamIntegration

Pr>|X2|< .0001

Pr>|t|< .0001

Pr>|z|< .0001

Pr>|X2|< .0001


Team Integration and Clarity of Objectives Go Together

0

10

20

30

40

50

60

70

80


Integrated Not Integrated

Data are for megaprojects only, but are similar for all projects

Perc

ent o

f Pro

ject

s


Pr>|X2|< .0001


Causal Path Runs Through Integration and Trade-offs to FEL

Clarity of Trade-offsAmong OutcomesPr>|X2|< .0001


ExcellentFEL


TeamIntegration

Pr>|t|< .0001

Pr>|t|< .0001

Pr>|z|< .0001

Pr>|X2|< .0001


Trade-offs Are Less Clear for Megaprojects

0

5

10

15

20

25

30

35

40

45


<$500MM Megas

Perc

ent o

f Pro

ject

s

Team Understanding of Cost/Schedule/Operability Trade-offs


Team Integration Is the Critical Enabler

Very Clear Fairly Clear SomewhatClear

Not Very Clear Not Clear At All

FEL

Inde

x

Clarity of the Cost/Schedule/Operability Trade-offs

Fair

Bes

tM

iser

able

Goo

dPo

or

Pr < 0.0001

Pr < 0.91

Source: Industrial Megaprojects, op. cit.

Team Is Not IntegratedTeam Is Integrated


Causal Path Runs Through Integration and Trade-offs to FEL

Clarity of Trade-offsAmong OutcomesPr>|X2|< .0001


ExcellentFEL


+Team

Integration

Pr>|t|< .0001

Pr>|t|< .0001

Pr>|z|< .0001

Pr>|X2|< .0001

All of the relationships on this graph are among the strongest statistical results in project research


What Is Going On?

• As size increases, we see difficulty accelerating in the face ofdegrading practices

• The practices are harder as projects get bigger

– More resources are required to develop

– Those resources are in short supply

• If the project is far from home, personnel are more expensive

• If the project is remote, we do not (or cannot) staff it

• As the demographic shift plays out, experienced people are becoming scarce

• Worst of all, as projects get bigger, the businesses are less sure of what they want and the chances of non-alignment with stakeholders increase


Outline

• Database






Answering the Questions Posed

• Very large projects do not fail simply because of the difficulties stemming from size

• Not only do megaprojects respond well to good practices, they respond better than smaller projects

• Conversely, they degrade very rapidly with practices that are merely mediocre for smaller projects

• But we are actually much less likely (for organizational reasons) to follow those good practices for our megaprojects

• The only practice that is almost unique for megaprojects is Venture Shaping and most companies lack a coherent approach to this process


Some Implications

• Very large project outcomes are so much worse than those of smaller projects that we are systematically over-estimating economies of scale

• As an industry, we are headed for trouble in this decade that will dwarf the difficulties of the previous decade– We have more megaprojects coming in this decade than

ever before—literally hundreds in the next 5 years

– Neither owners nor contractors have the staff needed to properly resource these projects

– Businesses seem no more sure-footed today than in days past, maybe less so

– Businesses are even less project-savvy today than yesterday


Recommendations

• Doing real portfolio management of projects will be more important than ever

• Marginal projects simply must be dropped or there will be little chance of doing anything well

• Resource constraints must figure into the project selection decisions

• Disadvantages of megaprojects in the portfolio need to be recognized and risks hedged

• Nothing will help, however, unless we can achieve stronger business work process for megaprojects


Megaprojects Require a StrongBusiness-Led Shaping Process

• In the past, we have developed each megaproject as a unique event with no systematic business work process

• Business professionals need to adopt a much more disciplined approach to large project development

• That work process must:

– Frame the business opportunity

– Systematically align the stakeholders by distributing the project’s value

– Create a stable platform from which the project can be executed without excessive turbulence in its environment

• Shaping process must be integrated with the FEL process and come to closure at the end of FEL 2 (Scope Development)


Implications for Public Sector Projects

• There is widely held belief that large public sector projects tend to overrun because the estimates are deliberately low-balled

• Our analysis of large private sector projects suggests that no Machiavellian explanation is required

• Large projects have a dismal track record because we have not adjusted our practices to fit the difficulty that the projects present

Thank you!Contact Information:

IPA, Inc.44426 Atwater Drive

Ashburn, Virginia USA 20147+1 703.729.8300

www.ipaglobal.com

Documents

Why Large Projects Fail More Often - Pemexuvirtual.pep.pemex.com/nuevo/Why_Large_Projects_Fail_ Final.pdf · Why Large Projects Fail More Often ... • Large projects in the Process