Glenn.butts mega projects estimates

  • Published on

  • View

  • Download

Embed Size (px)


  • Glenn C ButtsGlenn ButtsFebruary, 2010

    Hope is NOT a Valid Management Strategy

    Mega Projects Estimates -A History of Denial

    The source of knowledge is experience.Experience is of two kinds-our own, and that of others.

    Knowledge gained from our own experience is most expensive.Mystery and ignorance are always coexistent.

    Learn the truth and the mystery will disappear.

    Special Thanks to Kent Linton for help in preparing this presentation. Used with Permission

  • Page 2

    Associated Press March 4, 2009

    Cost overruns plague U.S. space agencyGAO report used NASA as poster child for bad practices in estimating costsWASHINGTON - NASA can land a spacecraft on a peanut-shaped asteroid 150 million miles away, but it doesn't come close to hitting the budget target for building its spacecraft, according to congressional auditors. NASA's top officials know it and even joke about it.

    Unfortunately for us, this is NOT News . . .NASA has a long history of bad cost and schedule estimates

    . . . and Congress doesn't appear to be in a joking mood any longer.

  • Page 3

    Not Limited to NASACost Overrun Everywhere . . . WHY???

    A Worldwide phenomenon that affects both the private and public sectors.Green Book - English Government

    Confidential project documents from 1995 reveal Bechtel willingly hid coststo present a more favorable view of

    Big Dig's financial picture at the behest of top state officials seeking a more

    publicly acceptable bottom line.Source - Boston Globe

  • Page 4

    US NAVY October 22, 2009

    Independence, a new 418-foot warship Top speed in excess of 45 knots, or about 52 mph.

    Eventually, Navy wants to build 55 of them. Early estimates for Littoral Combat Ships were about $220 million each

    Costs escalated because of new requirements and desire to expedite construction. Cost of ships now capped at $460 million each, starting in new fiscal year.

  • Page 5

    Boeing 787 Dreamliner 2009 Status

    Boeings radical plan to build revolutionary new 787 Dreamliner Heavy outsourcing - worldwide Unprecedented use of composite materials Unmatched fuel economy

    Radical New plan didnt work. Communication problems Quality suffered Structural issues Schedules slipped

    Initial delivery date May 2008 Test flight delayed over 2 years

    Still faces 12 months of testing 60 Order cancelations Billions in costs increases, including

    Payments & concessions to airlines Purchasing suppliers Additional management oversight costs

    "The initial plan outran our ability to execute it," Boeing Chief Executive Jim McNerney

    Dreamlifter built to bring large parts to Seattle from around the world for final assembly

  • Page 6

    Boeing 747-8 - 2009 Status

    Boeings conventional plan to build new 747-8 freighter variant Project started in 2005

    2 years after 787 Variant of existing 747

    New wings New Engines New cockpit technologies

    Uses traditional materials instead of composites Orthodox manufacturing process

    Conventional Plan didnt work either. Besieged by design and engineering problems from onset Boeing announced $1 Billion dollar charge as a result of problems in Oct 09 Initial delivery delayed at least a year

    Test Flight Mid January 2010

    No intention to pick on Boeing Airbuss new A380 experiencing similar problems

    We believe that building the 747-8 should have been a straightforward development process on a derivative program.

    Aerospace Analyst Douglas Harned

  • Page 7

    Congress is Not Any Better

    Congressional Visitors Center Boondoggle 580,000 sq. ft. facility estimated cost $71 million in the early 1990s

    Construction began June 2000 with a promised Completion date of 2004 for a Budget of $265 million

    Finally completed five years behind schedule in December 2008 The final cost - a whopping $621 million

    $671 M $71 M - 1 =845% cost growth

    Just One Exampleof Many

  • Page 8

    Underestimates are Nothing New - Quotes from 1964All but One from Apollo Executives Meeting

    "I can think of a lot of programs in the Boeing Company where, if the estimate had been realistic, you wouldn't have had the program, and that is the truth."

    W. M. Allen - President, Boeing

    "You have basically built into the entire system a series of events which very much leads to under-estimating what the program is going to cost."

    J. S. Parker - Vice President, GE

    "As advocates of new programs, government agencies have often encouraged contractors to estimate costs optimistically. Recognizing that headquarters might be shocked out of supporting a program

    where true costs were revealed at the outset. They have sought to disclose cost increases gradually, after programs have

    gained momentum and cancellation has become difficult."Dr. Frederic Scherer - Harvard

    "We can give you estimates all the way to the end of the Saturn-IVB, but do you really want to know it?"

    Donald W Douglas - President Douglas Aircraft

    Underestimates documented back to 1869 probably prior.

  • Page 9

    Congress Complicates Matters

    Another Example:Dick Cheney canceled Marine Corps' troubled V-22 Osprey aircraft not once, but 4 times, only to see Congress reverse the decision.

    Bill Nelson and Co. take town Obamas NASA frontrunner

    Word is that Steve Isakowitz, the frontrunner for the NASA Administratorss job, has been taken down by a group of his opponents on Capitol Hill.

    According to Washington insiders and Hill staff, a group of lawmakers led by Floridas Democratic senior Senator Bill Nelson are taking credit for pushing Isakowitz out of the picture.

    Nelson and other NASA boosters did not like Isakowitzs reputation

    as a tough fiscal manager.

  • Page 10

    We Claim to Seek Accurate Estimates

    Yet, we; Persist in submitting optimistic initial estimates Omit probable scope from our estimates that often become "real." Exclude credible risks from our analysis

    SM&A actively mitigates risks with a likelihood of occurrence of only one in several thousand, and are supported by managment.

    Management mantra if status quo is assumed, we will get the status quo

    Meaning status quo is too expensive There are valid reasons for why things cost what they cost Excuse for under-estimating projects

    Possible conclusions: Accurate estimates are impossible, especially at early stages

    The variables are too complex, unquantifiable, and incalculable - NOT We really dont want accurate estimates;

    The Primary concern is keeping program viable & funded as long as possible Early estimates are deliberately misrepresented and meant to gain approval

  • Page 11

    Early Estimates are Often Deliberate MisrepresentationPolite way of Saying Lying.

    Conclusions from the largest study of its kind: Examined 258 large transportation projects Underestimation has not decreased during past 70 yrs. It was found with overwhelming statistical significance that

    cost underestimation cannot be explained by error. Is best explained by strategic misrepresentation, namely lying,

    with a view to getting projects started. Cost overruns of 50% to 100% in real terms are common.

    Overruns above 100% are not uncommon. Accuracy is gained at each successive stage of progress. Promoters routinely ignore, hide, or otherwise leave out

    important project costs and risks in order to make total costs appear low.

    Cost underestimation occurred in almost 9 out of 10 projects. The likelihood of actual costs being larger than estimated is 86% The likelihood of actual costs being lower than estimated is 14%

    This study looked at comparatively simple projects.Statistics for more complex projects would likely be worse!

  • Page 12

    How Do We Underestimate?- Let Me Count The Ways -

    1. OMIT PROBABLE SCOPE from estimate2. OMIT POSSIBLE RISKS from analysis

    Internal & External

    3. UNREALISTIC, OPTIMISTIC assumptions4. Use historically LOW ESCALATION projections

    RAND Study Reason for 11.2% of Cost Growth

    5. Issue cost estimates in BASE YEAR dollars Estimates should be in then year dollars (escalated to year in which it is spent)

    6. Many estimates NOT PREPARED BY A BONA FIDE ESTIMATOR Everyone's a estimator Being certified no guarantee of having necessary experience

    7. REWARD failure, PUNISH honesty8. NOT ENOUGH TIME to prepare CREDIBLE estimates

    Time often spent doing what if exercises, or splitting dollars into arbitrary buckets

    I reject a system that rewards failure and protects a person from its consequences

    - Barack Obama -

    RAND Study Reasonfor 74% of Cost Growth


  • Page 13

    Escalation Assumptions Should be RealisticNASA New Start Escalation Index

    The current efforts to help revive the economy are likely to produce inflation that could be worse than what the country suffered in the late 1970s.

    Warren Buffett, March 2009








    1959 1964 1969 1974 1979 1984 1989 1994 1999 2004 2009 2014



    ual I



    Annual Average 5%

    Realistic Assumption?RAND - 1981 escalation

    accounts for 11.2% of

    project cost growth

    88% probability assumed

    escalation will be exceeded

  • Forgotten SPACE SHUTTLE Facts

    Initially approved by 1 Vote Congress WAS concerned about estimates

    FY 1972 Estimate DDT&E $5.5 Billion + 20% Contingency = $6.6 Billion

    Contingency Reportedly Removed by Congress Facilities $0.3 Billion 1st Manned Flight projected by June 1976 (4 yrs.)

    Reality - 1982 Total Cost $17.789 Billion

    DDT&E $13.138 Billion Production $ 3.900 Billion Facilities $ 0.703 Billion

    1st Manned flight occurred in April 1981 (9 yrs.)

    Increase WAS Foreseen . . . Considering all of the technological and operational unknowns involved in the shuttle

    development and the fact that no vehicles of similar function have ever been designed before or have ever operated over the range of flight regimes required for the shuttle, prudent extrapolation of prior experience would indicate that estimated development costs may be 30 to 50 percent on the low side. Thus, the estimates of $6.5 billion in RDT&E for the Mk I/Mk II shuttle program may range between $8.5 to $10 billion, reflecting increased program costs of $2.5 to $3.5 billion. Similar uncertainties must be considered to apply to other non-recurring costs such as production and facilities (amounting to about $4 billion). Thus a possible cost uncertainty of about $5 billion for total program costs might be envisioned giving a high estimate of total non-recurring cost of about $15 billion. - October 1971 Report -

    The chance of a Shuttle launch on November 20 1980 is probably 10 to 20%

    Dr. Frosch NASA - AdministratorCongressional Testimony 1980

    Page 14

  • Page 15

    Realistic Assumptions ARE Critical

    Space Shuttle Operations Concept - 1974This is what was estimated.

    Orbiter Processing Facility - TodayThis is what was built.

  • Page 16

    Never Fear!Joint Confidence Levels (JCL) Will Save the Day!!

    In January 2009, NASA adopted a new acquisition strategy policy which requires space flight and IT projects and programs to develop joint cost & schedule probabilistic estimates.

    NASAs commitment to using joint cost and schedule confidence levels has moved NASA along a path towards improving our delivery of projects on time and within budget.

    Paraphrased from Christopher Scolese Acting NASA Administrator's (3-5-09) Congressional Testimony

  • Page 17

    Probabilistic Cost & Scheduling Concept - Nothing NewDidnt Work on 1st Attempt

    PERT: Ignored possibility of unlikely events

    Sold as greatest thing since sliced bread

    Credited for the unmitigated success of Polaris program

    Actually a gimmick to wow Congress Only applied to a few program activities

    Was subsequently expanded to include cost

    Was then used to manage the Trident Program Trident became the Single worst managed

    project in US Military History

    Because it did not work, it fell out of favor in the mid to late 70s

    Source The Polaris System Development, 1972

  • Page 18

    The Real Reason for Polaris' Success

    The Polaris Program: Attracted top talent Created a new office with special privileges Authorized special uniforms Provided first class travel Initiated hassle free expense accounts Gave liberal promotions and awards for success

    The Staff was told to Think big or get out Required minimal paperwork and reports

    Didn't burden talent with bureaucratic paperwork Got personal pledges from contractors &

    personnel that schedules would be met Issued swift & sure ramifications for failure

    Branch heads were dismissed within 24 hours Contractor personnel were prohibited from

    returning to previous assignments Was very receptive to innovative ideas Candid honesty was actively sought

    Lying cardinal crime and treated accordingly!Source The Polaris System Development, 1972

  • Page 19

    Management CAN'T Figure Out WhyS-Curves Flatten Over Time Can You?

    Graphic by David Bearden Aerospace Corp.STEREO Mission

    Initial estimate from cost model vs model output from as built configuration



    $200 $250 $300 $350 $400 $450 $500 $550 $600 $650 $700 $750 $800Total Mission Cost (FY07$M)


    ent L



    d of




    at C


    SDT DesignFinal Design

    $299M Estimate with 20% Reserve

    Final Cost$551M



    $200 $250 $300 $350 $400 $450 $500 $550 $600 $650 $700 $750 $800Total Mission Cost (FY07$M)


    ent L



    d of




    at C


    SDT DesignFinal Design

    $299M Estimate with 20% Reserve

    Final Cost$551M



    $200 $250 $300 $350 $400 $450 $500 $550 $600 $650 $700 $750 $800Total Mission Cost (FY07$M)


    ent L



    d of




    at C


    SDT DesignFinal Design

    $299M Estimate with 20% Reserve

    Final Cost$551M

    Does this help Credibility?

    Answer: Red S-Curve was initial project estimate from cost model based on numerous optimistic assumptions.Blue S-Curve is model estimate based on as built configuration.

  • Page 20

    Possible Solutions

    Hold project managers accountable for the original estimate Project Managers will THEN begin demanding credible estimates Recognize that historically many original assumptions are OPTIMISTIC or

    simply NOT valid Resist the legacy practice of labeling COST growth as "SCOPE" growth Rather than scope "growth" historically, scope "reduction" is often what

    has actually occurred. Real requirements are often trimmed - NOT expanded Space Station was originally touted with 8 functions - but cut down to 1 Shuttle's original requirement was for a payload of 65,000 lbs. Maximum current

    payload capability is 49,000 lbs. Mercury Program deleted two monkey passenger test flights Etc., etc

    OR Include all possible risks in cost projections (internal and external)

    Projects tendency to spend everything available must be constrained Congress budget contraints aggrivate issue they demand that all money is spent.

    Make it fashionable to be austere and frugal and encourage "under run" It's always possible to spend more time and money

  • Page 21

    How English Government Deals With Bad Estimates

    European Solution apply an optimism bias adjustment factor

    There is a demonstrated, systematic, tendency for project appraisers to be over...