Embed Size (px)
Citation preview
^Åèìáëáíáçå oÉëÉ~êÅÜ éêçÖê~ãdo^ar^qb p`elli lc _rpfkbpp C mr_if` mlif`vk^s^i mlpqdo^ar^qb p`elli
Approved for public release; distribution unlimited.
Prepared for the Naval Postgraduate School, Monterey, California 93943
Disclaimer: The views represented in this report are those of the authors and do not reflect the official policy position of the Navy, the Department of Defense, or the Federal Government.
bu`bomq colj qeb
mêçÅÉÉÇáåÖëçÑ íÜÉ
bfdeqe ^kkr^i ^`nrfpfqflk
obpb^o`e pvjmlpfrj
qeropa^v pbppflkp
slirjb ff
Cost and Time Overruns for Major Defense Acquisition Programs: An Annotated Brief
David Berteau, Guy Ben-Ari, Joachim Hofbauer, Gregory Sanders, Jesse Ellman, and David Morrow, Center for Strategic & International Studies
Published: 30 April 2011
NPS-AM-11-C8P14R02-053
Report Documentation Page Form ApprovedOMB No. 0704-0188
Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering andmaintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, ArlingtonVA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if itdoes not display a currently valid OMB control number.
1. REPORT DATE APR 2011 2. REPORT TYPE
3. DATES COVERED 00-00-2011 to 00-00-2011
4. TITLE AND SUBTITLE Cost and Time Overruns for Major Defense Acquisition Programs: AnAnnotated Brief
5a. CONTRACT NUMBER
5b. GRANT NUMBER
5c. PROGRAM ELEMENT NUMBER
6. AUTHOR(S) 5d. PROJECT NUMBER
5e. TASK NUMBER
5f. WORK UNIT NUMBER
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Center for Strategic & International Studies,Defense-IndustrialInitiatives Group,1800 K Street, NW,Washington,DC,20006
8. PERFORMING ORGANIZATIONREPORT NUMBER
9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)
11. SPONSOR/MONITOR’S REPORT NUMBER(S)
12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution unlimited
13. SUPPLEMENTARY NOTES Presented at the Naval Postgraduate School?s 8th Annual Acquisition Research Symposium, 10-12 May2011, Seaside, CA. U.S. Government or Federal Rights License
14. ABSTRACT Cost and time overruns in Major Defense Acquisition Programs (MDAPs) have become a high-profileproblem attracting the interest of Congress, government, and watchdog groups. According to the GAO, the98 MDAPs from FY2010 collectively ran $402 billion over budget and were an average of 22 monthsbehind schedule since their first full estimate. President Obama?s memorandum on governmentcontracting of 4 March 2009 also highlighted this issue. This paper presents findings of research on theroot causes of cost and schedule delays for 92 MDAPs active in 2010 and 12 cancelled programs. Theresults do not establish causality but they do indicate multiple notable correlations. Inaccurate costestimates are responsible for the strongest correlation with net cost growth changes and are associated with40% of the accumulated cost overruns. In addition, the start year has little impact on the compoundannual growth rate of cost overruns. This suggests that relatively better performance of newer programsmay prove illusionary as programs age. Finally, fixed price contracts appear to have relatively smalleroverruns, although this may tell us more about which programs are likely to receive fixed price contractsrather than what effect fixed price contracts may have on program performance.
15. SUBJECT TERMS
16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT Same as
Report (SAR)
18. NUMBEROF PAGES
30
19a. NAME OFRESPONSIBLE PERSON
a. REPORT unclassified
b. ABSTRACT unclassified
c. THIS PAGE unclassified
Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18
^Åèìáëáíáçå oÉëÉ~êÅÜ éêçÖê~ãdo^ar^qb p`elli lc _rpfkbpp C mr_if` mlif`vk^s^i mlpqdo^ar^qb p`elli
The research presented at the symposium was supported by the Acquisition Chair of the Graduate School of Business & Public Policy at the Naval Postgraduate School. To request Defense Acquisition Research or to become a research sponsor, please contact: NPS Acquisition Research Program Attn: James B. Greene, RADM, USN, (Ret.) Acquisition Chair Graduate School of Business and Public Policy Naval Postgraduate School 555 Dyer Road, Room 332 Monterey, CA 93943-5103 Tel: (831) 656-2092 Fax: (831) 656-2253 E-mail: [email protected] Copies of the Acquisition Sponsored Research Reports may be printed from our website www.acquisitionresearch.net
^Åèìáëáíáçå oÉëÉ~êÅÜW `ob^qfkd pvkbodv clo fkclojba `e^kdb - i -
Preface & Acknowledgements
During his internship with the Graduate School of Business & Public Policy in June 2010, U.S. Air Force Academy Cadet Chase Lane surveyed the activities of the Naval Postgraduate School’s Acquisition Research Program in its first seven years. The sheer volume of research products—almost 600 published papers (e.g., technical reports, journal articles, theses)—indicates the extent to which the depth and breadth of acquisition research has increased during these years. Over 300 authors contributed to these works, which means that the pool of those who have had significant intellectual engagement with acquisition issues has increased substantially. The broad range of research topics includes acquisition reform, defense industry, fielding, contracting, interoperability, organizational behavior, risk management, cost estimating, and many others. Approaches range from conceptual and exploratory studies to develop propositions about various aspects of acquisition, to applied and statistical analyses to test specific hypotheses. Methodologies include case studies, modeling, surveys, and experiments. On the whole, such findings make us both grateful for the ARP’s progress to date, and hopeful that this progress in research will lead to substantive improvements in the DoD’s acquisition outcomes.
As pragmatists, we of course recognize that such change can only occur to the extent that the potential knowledge wrapped up in these products is put to use and tested to determine its value. We take seriously the pernicious effects of the so-called “theory–practice” gap, which would separate the acquisition scholar from the acquisition practitioner, and relegate the scholar’s work to mere academic “shelfware.” Some design features of our program that we believe help avoid these effects include the following: connecting researchers with practitioners on specific projects; requiring researchers to brief sponsors on project findings as a condition of funding award; “pushing” potentially high-impact research reports (e.g., via overnight shipping) to selected practitioners and policy-makers; and most notably, sponsoring this symposium, which we craft intentionally as an opportunity for fruitful, lasting connections between scholars and practitioners.
A former Defense Acquisition Executive, responding to a comment that academic research was not generally useful in acquisition practice, opined, “That’s not their [the academics’] problem—it’s ours [the practitioners’]. They can only perform research; it’s up to us to use it.” While we certainly agree with this sentiment, we also recognize that any research, however theoretical, must point to some termination in action; academics have a responsibility to make their work intelligible to practitioners. Thus we continue to seek projects that both comport with solid standards of scholarship, and address relevant acquisition issues. These years of experience have shown us the difficulty in attempting to balance these two objectives, but we are convinced that the attempt is absolutely essential if any real improvement is to be realized.
We gratefully acknowledge the ongoing support and leadership of our sponsors, whose foresight and vision have assured the continuing success of the Acquisition Research Program:
• Office of the Under Secretary of Defense (Acquisition, Technology & Logistics)
• Program Executive Officer SHIPS
• Commander, Naval Sea Systems Command
• Army Contracting Command, U.S. Army Materiel Command
• Program Manager, Airborne, Maritime and Fixed Station Joint Tactical Radio System
^Åèìáëáíáçå oÉëÉ~êÅÜW `ob^qfkd pvkbodv clo fkclojba `e^kdb - ii -
• Program Executive Officer Integrated Warfare Systems
• Office of the Assistant Secretary of the Air Force (Acquisition)
• Office of the Assistant Secretary of the Army (Acquisition, Logistics, & Technology)
• Deputy Assistant Secretary of the Navy (Acquisition & Logistics Management)
• Director, Strategic Systems Programs Office
• Deputy Director, Acquisition Career Management, US Army
• Defense Business Systems Acquisition Executive, Business Transformation Agency
• Office of Procurement and Assistance Management Headquarters, Department of Energy
We also thank the Naval Postgraduate School Foundation and acknowledge its generous contributions in support of this Symposium.
James B. Greene, Jr. Keith F. Snider, PhD Rear Admiral, U.S. Navy (Ret.) Associate Professor
^Åèìáëáíáçå oÉëÉ~êÅÜW `ob^qfkd pvkbodv clo fkclojba `e^kdb - 5 -
Panel 14 – Major Programs: The Good, the Bad, and the Ugly
Thursday, May 12, 2011
9:30 a.m. – 11:00 a.m.
Chair: Vice Admiral W. Mark Skinner, USN, Principal Military Deputy, Assistant Secretary of the Navy (Research, Development, & Acquisition)
An Assessment of the DoD’s 2010 Portfolio of Major Defense Acquisition Programs
Michael Sullivan, GAO
Cost and Time Overruns for Major Defense Acquisition Programs: An Annotated Brief
David Berteau, Guy Ben-Ari, Joachim Hofbauer, Gregory Sanders, Jesse Ellman, and David Morrow, Center for Strategic & International Studies
Straight Talk: Major Program Manager Views of Defense Acquisition Roy Wood and Al Moseley, DAU
Vice Admiral W. Mark Skinner—Principal Military Deputy, Assistant Secretary of the Navy, (Research, Development & Acquisition). Vice Admiral Skinner assumed his duties August 9, 2010.
Skinner was born in Houston, Texas and graduated from the United States Naval Academy in June 1977.
As a flag officer, he was the program executive officer for Tactical Aircraft Programs and commanded Naval Air Warfare Center, Weapons Division, and served as assistant commander, Test and Evaluation, Naval Air Systems Command. Skinner held both operational and shore commands, to include commanding officer Patrol Squadron 47, chief test pilot and commanding officer of Naval Force Aircraft Test Squadron, and program manager for a chief of naval operations special project.
He is a graduate of the Navy Test Pilot School and served in Force Warfare Aircraft Test Directorate, where he was recognized as Directorate Test Pilot of the Year in 1986. Additionally, he received a degree in Financial Management from the Naval Postgraduate School, where he graduated as a Conrad Scholar and was awarded the Department of Navy award for excellence in financial management and the Rear Admiral Thomas R. McClellan award for excellence in administrative sciences.
His awards include Legion of Merit (3 awards), Meritorious Service Medal (4 awards), Navy Commendation Medal (2 awards), Navy Achievement Medal, and other unit deployment citations and ribbons.
^Åèìáëáíáçå oÉëÉ~êÅÜW `ob^qfkd pvkbodv clo fkclojba `e^kdb - 7 -
Cost and Time Overruns for Major Defense Acquisition Programs: An Annotated Brief David Berteau—Senior Adviser and Director, CSIS Defense-Industrial Initiatives Group, covering defense management, programs, contracting, and acquisition. Mr. Berteau’s group also assesses national security economics and the industrial base supporting defense. Mr. Berteau is an adjunct professor at Georgetown University, a member of the Defense Acquisition University Board of Visitors, a director of the Procurement Round Table, and a fellow of the National Academy of Public Administration. He also serves on the Secretary of the Army’s Commission on Army Acquisition and Program Management in Expeditionary Operations. [[email protected]]
Guy Ben-Ari—Deputy Director, Defense-Industrial Initiatives Group at the Center for Strategic International Studies. Mr. Ben-Ari works on projects related to the U.S. technology and industrial bases supporting defense. His current research efforts involve defense R&D policies, defense economics, and managing complex defense acquisition programs. Mr. Ben-Ari holds a bachelor’s degree in political science from Tel Aviv University, a master’s degree in international science and technology policy from the George Washington University, and is currently a PhD candidate (ABD) at the George Washington University.
Joachim Hofbauer—Fellow, Defense-Industrial Initiatives Group at the Center for Strategic and International Studies (CSIS). Mr. Hofbauer specializes in U.S. and European defense acquisition and industrial base issues and their impact on the transatlantic defense market. Before joining CSIS, he worked as a freelance defense analyst in Germany and the United Kingdom. His analysis has been published in several U.S. and German defense publications. Mr. Hofbauer holds a BA in European studies from the University of Passau and an MA with honors in security studies, with a concentration in defense analysis, from Georgetown University.
Gregory Sanders—Fellow, Defense-Industrial Initiatives Group at CSIS. Mr. Sanders gathers and analyzes data on U.S. defense acquisition and contract spending as international defense budgetary and trade trends. He has also studied data visualization and ways to use complex data collections to create succinct and innovative tables, charts, and maps. Mr. Sanders holds an MA in international relations from the University of Denver and a BA in government and politics, as well as a BS in computer science, from the University of Maryland.
Jesse Ellman—Research Associate, Defense-Industrial Initiatives Group at the Center for Strategic and International Studies (CSIS). Mr. Ellman specializes in U.S. defense acquisition issues, with a particular focus on recent U.S. Army modernization efforts. He holds a BA in Political Science from Stony Brook University, and an MA with honors in Security Studies, with a concentration in Military Operations, from Georgetown University.
David Morrow—Research Associate, Defense-Industrial Initiatives Group (DIIG) at CSIS. Mr. Morrow focuses on federal professional services contracting, U.S. naval shipbuilding, and private security contracting. Previously, he interned at the U.S. Department of State’s Office of European Security and Political Affairs and at the U.S.–Russia Business Council. He holds a BA in International Affairs from James Madison University and an MA in European and Eurasian Studies from the George Washington University.
Abstract Cost and time overruns in Major Defense Acquisition Programs (MDAPs) have become a high-profile problem attracting the interest of Congress, government, and watchdog groups. According to the GAO, the 98 MDAPs from FY2010 collectively ran $402 billion over budget and were an average of 22 months behind schedule since their first full estimate. President Obama’s memorandum on government contracting of 4 March 2009 also highlighted this issue.
^Åèìáëáíáçå oÉëÉ~êÅÜW `ob^qfkd pvkbodv clo fkclojba `e^kdb - 8 -
This paper presents findings of research on the root causes of cost and schedule delays for 92 MDAPs active in 2010 and 12 cancelled programs. The results do not establish causality but they do indicate multiple notable correlations. Inaccurate cost estimates are responsible for the strongest correlation with net cost growth changes and are associated with 40% of the accumulated cost overruns. In addition, the start year has little impact on the compound annual growth rate of cost overruns. This suggests that relatively better performance of newer programs may prove illusionary as programs age. Finally, fixed price contracts appear to have relatively smaller overruns, although this may tell us more about which programs are likely to receive fixed price contracts rather than what effect fixed price contracts may have on program performance.
Introduction Cost and time overruns in Major Defense Acquisition Programs (MDAPs) have
become a high-profile problem attracting the interest of Congress, government, and watchdog groups. According to the GAO, the 98 MDAPs from FY2010 collectively ran $402 billion over budget and were an average of 22 months behind schedule since their first full estimate. President Obama’s memo on government contracting of 4 March 2009 also highlighted this issue.
This paper1 presents findings of research on the root causes of cost and schedule delays for MDAPs, incorporating 2010 SAR data.
(20)
‐
20
40
60
80
100
120
‐200% 0% 200% 400%
Real co
st overrun
s (in
2010 billion
s, quantity
adjusted
)
Real cost overruns (in percent, quantity adjusted)
F‐35
CHEM DEMIL‐ACWA
FCS
(2)
‐
2
4
6
8
10
‐25% 0% 25% 50%
Figure 1. Relative Cost Overruns vs. Absolute Cost Overruns for FY2009 MDAPs Note. The sample includes 92 FY2010 MDAPs with a baseline estimate beyond Milestone B in the June 2010 SAR as well as twelve additional cancelled programs, notably including the Future Combat System (FCS). The source for this figure was Selected Acquisition Reports; the analysis was by CSIS Defense-Industrial Initiatives Group.
1 Nicholas Lombardo was a contributing researcher on this report.
^Åèìáëáíáçå oÉëÉ~êÅÜW `ob^qfkd pvkbodv clo fkclojba `e^kdb - 9 -
Problem Definition Past studies on this topic either have not offered rigorous data analysis or were
focused on a critical but still narrow aspect of the problem, such as technical maturity. Meanwhile, Congressional leadership often focuses on different issues such as contract type and competition. As a result, acquisition reform efforts like the Weapon Systems Acquisition Reform Act of 2009 are hampered by an insufficient analytical basis.
For instance, in its annual assessment of selected weapon systems, the Government Accountability Office (GAO) predominantly focuses on knowledge-based factors such as technology, maturity, and associated program decisions as causes for these problems. Former Under Secretary of Defense for Acquisition, Technology, and Logistics, John Young, claimed in a memorandum on 31 March 2009 that many of the allegations of the GAO are based on inadequate analytical methods and that consequently many of the results are misleading.
This disagreement is exemplary of the diverging set of opinions that exists regarding the root causes of MDAP cost overruns and schedule delays. The result amplifies disagreement regarding potential fixes. On the government side, Senator McCain identified the usage of cost plus contracts as a major source for cost increases and Secretary Gates pointed towards the contract structures as a key source of cost and schedule overruns in some MDAPs. Defense contractors, on the other hand, regularly cite the altering of requirements in advanced program stages as an important factor for cost increases.
The currently ongoing process of reforming and fixing the defense acquisition system still lacks the foundation of a detailed evaluation of the causality chain of cost overruns and program delays of MDAPs. This lack of understanding of underlying mechanisms makes the design of adequate solutions inherently difficult and renders them potentially ineffective. This study directly aims at developing the urgently needed knowledge base that will better guide efforts to correct the growing trends of cost increases and schedule overruns.
Methodology This report analyzes a series of variables—namely realism of baseline program cost
estimates, government management and oversight, the role of contractors and lead military Services, levels of competition, and contract structures—to determine what factors might contribute to or be correlated with the observed cost overruns in the execution of MDAPs.
This research draws on three primary data sources:
1. Selected Acquisition Reports (SARs): The SARs track Major Defense Acquisition Programs, reporting on their schedule, unit counts, total spending, and progress through milestones. The unit of analysis is the programs themselves, making it the ideal source for top level analysis.
2. Federal Procurement Data System (FPDS): The FPDS is a database of every government contract, with millions of entries each year. Each entry has extensive data on the contractors, contract type, competition, place of performance, and a variety of other topics as mandated by Congress. Cross-referencing individual contracts with MDAPs is possible using the system equipment codes (which match up with those of the MDAPs). This source provides the most in-depth data on the government contracting process.
3. Department of Defense budget documents: In addition to budget data, these documents provide topical information on each MDAP and its
^Åèìáëáíáçå oÉëÉ~êÅÜW `ob^qfkd pvkbodv clo fkclojba `e^kdb - 10 -
subcomponents. They will primarily be used to categorize projects as well as to support and double check spending figures from the other two sources.
The report focuses on MDAPs from the FY2010 MDAP list. Within this sample group the analysis is limited to 104 MDAPs with cost estimates set at Milestone B or beyond, including MDAPs that were cancelled between 1999 and today. That gate is meant to be a hurdle that requires programs to reach a certain level of technological maturity. As a result Milestone B “is normally the initiation of an acquisition program” (“Acquisition History Project,“ n.d.). This common starting point ensures that only programs in a relatively mature acquisition phase are compared. Cancelled programs are included to avoid the selection bias that results from excluding several of the worst performing proposals from analysis. Figure 1 provides an overview of the cost overruns of these 104 programs.
Unfortunately, full data are not available on all 104 MDAPs when examining contract type and competition, because not all of the programs have at least 50% of the SARs contract value accounted for in 2004–2009 FPDS data. As a result, the “unclear” category is used to signify this missing data in competition and contract type findings. In addition, FPDS totals for program spending are sometimes higher than the funding status according to the SARs. In those cases, the SAR totals are treated as the more reliable figure.
These snapshots provide an adequate starting point for detecting correlations between a series of potentially relevant factors and cost growth. The charts reflect the basic information arranged across a variety of data elements, but they do not constitute a sufficient basis for establishing causality or policy changes, for which further analysis would be needed.
Analysis This analysis focuses on examining the impact of baseline cost estimates, quantity,
and schedule changes, as well as engineering problems, the extent of competition, contract structure, the lead branch of military service, and the identity of the prime contractor on the cost performance on MDAPs.
$124
$50 $63
$203
$51$3
‐$86
‐$3 ‐$4‐$16
‐$3 $0
21%
2% 4%
9% 8%
0%
3% 3% 4%
13%
3% 0%
‐15%
0%
15%
30%
$(120)
$(60)
$‐
$60
$120
$180
$240
Quantity Schedule Engineering Estimating Support Other
Real co
st overrun
s (in percent)
Real co
st overrun
s (in 20
10 billion
s)
Cost Increase Cost Decrease Average Increase Baseline‐Weighted Average Increase
Cost Increase
Cost Decrease
Figure 2. Functional Reasons for Cost Overruns
Note. The source for this figure was Selected Acquisition Reports; the analysis was by CSIS Defense-Industrial Initiatives Group.
^Åèìáëáíáçå oÉëÉ~êÅÜW `ob^qfkd pvkbodv clo fkclojba `e^kdb - 11 -
Breaking down cost growth by functional areas as provided in the SARs identifies variances in the estimating process as the primary driver for cost growth, being responsible for $202.8 billion in cost growth for the 104 MDAPs analyzed.
Another noteworthy observation from Figure 2 is the fact that the cost savings achieved through quantity changes equals approximately two thirds of the cost growth originating from changes in unit numbers. This is not encouraging, as for programs with upfront research and development costs, reducing the number of units lowers the overall program cost but it increases the per-unit cost, effectively curtailing the government’s buying power. In turn, cost increases deriving from increases in the number of units require a higher overall program budget but lower the price per unit.
Nunn-McCurdy breaches, for instance, are based on the growth in the per-unit acquisition cost rather than overall program cost in order to account for this fact. This presentation therefore focuses on quantity-adjusted cost changes. The Selected Acquisition Reports do not list the exact methodology for quantity adjustments; unfortunately, the adjustment is not equivalent to the sum of cost adjustments that are not attributed to quantity changes. This complicates analysis of the functional reasons for cost growth.
‐20%
0%
20%
40%
60%
80%
100%
120%
140%
160%
1985 1990 1995 2000 2005 2010
Real com
poun
d an
nual cost g
rowth ra
te(in
percent, adjusted for q
uantity chan
ges)
Year of last baseline estimate
C‐130 AMP
EFV
Figure 3. Time-Cost Correlation Note. The source for this figure was Selected Acquisition Reports; the analysis was by CSIS Defense-Industrial Initiatives Group.
The next explanatory variable examined for its impact on program performance is the time-cost growth correlation. If cost increases accrue over time, then programs with an older
^Åèìáëáíáçå oÉëÉ~êÅÜW `ob^qfkd pvkbodv clo fkclojba `e^kdb - 12 -
baseline estimate would tend to accumulate relatively higher cost increases. The data for the analyzed programs show that older programs indeed experience larger overruns.
However, Figure 3 shows that when measured in compound annual growth rate2 rather than aggregate relative cost growth, the time-cost growth correlation is almost constant. The C-130 AMP project is distorting this trend because its estimate was not changed when it was given a new baseline in 2010. Notwithstanding C-130 AMP, this growth correlation not only provides further evidence for the assertion that cost growth occurs steadily throughout the program lifespan, but it also suggests that younger programs are not performing better than older programs.
(20)
‐
20
40
60
80
100
120
‐100% 0% 100% 200% 300%
Real co
st overrun
s (in
2010 billion
s, quantity
adjusted
)
Real cost overruns (in percent, quantity adjusted,
service averages are unweighted)
F‐35
CHEM DEMIL‐ACWA
FCS
(2)
‐
2
4
6
8
10
‐25% 0% 25% 50%Air Force ArmyNavy DoD‐wideAir Force Average Army AverageNavy Average DoD‐wide Average
Figure 4. Cost Overruns by Lead Service (I) Note. The source for this figure was Selected Acquisition Reports; the analysis was by CSIS Defense-Industrial Initiatives Group.
The analysis of the correlation between the lead branch of military service responsible for MDAPs and cost growth patterns reveals that programs led by the Army appear to have fewer, smaller overruns, followed by the Navy and then the Air Force, whereas DoD-wide programs tend to accrue significantly larger cost overruns. The picture alters slightly when utilizing baseline-weighted averages with the Navy showing the least overruns followed by the Army, the Air Force, and DoD-wide programs. The considerable difference for the Army’s results—11% on average versus 20% for baseline-weighted averages—is driven by the cancelled Future Combat System. It is important to note that DoD-wide includes both programs managed by DoD agencies and joint programs such as the Joint Strike Fighter.
2 The compound annual growth rate describes the average year-to-year cost growth of program spending since its baseline. Thus if comparing two programs with the same percentage of cost growth since their baseline estimate, the program with an earlier baseline year would have a smaller compound annual growth rate.
^Åèìáëáíáçå oÉëÉ~êÅÜW `ob^qfkd pvkbodv clo fkclojba `e^kdb - 13 -
The outcome of this data analysis might be skewed based on the relatively small sample group utilized in this analysis. For instance, it appears that the DoD-wide category might be heavily influenced by the negative cost developments in the Joint Strike Fighter program. As for the other components, further analyses with larger sample groups are required to validate observed trends.
Any conclusions from Figure 4 identifying superior program management of existing programs by Service are premature, even if additional data and analysis were to confirm this variation in cost performance based on lead Service. A number of other factors may explain the differences, such as a tendency toward less risk-prone MDAPs. Further research will be needed to analyze the underlying causality and detect the true root causes for these trends.
$73$89
$58
$124
‐$3.9 ‐$1.9 ‐$0.5 ‐$4.3
11%
17% 22%
41%
20%
16%
23%
44%
‐15%
0%
15%
30%
45%
60%
75%
$(30)
$‐
$30
$60
$90
$120
$150
Army Navy Air Force DoD‐wide
Real co
st overrun
s (in
percent, quantity
adjusted
)
Real co
st overrun
s (in
2010 billion
s, quantity
adjusted
)
Cost Overruns Cost Savings Average Overrun Baseline‐Weighted Average Overrun
Cost Overruns
Cost Savings
Figure 5. Cost Overruns by Lead Service (II) Note. The source for this figure was Selected Acquisition Reports; the analysis was by CSIS Defense-Industrial Initiatives Group.
Figure 5 supports the conclusion of the previous chart, with the poorest cost performance in DoD-wide managed MDAPs, while Army and Navy MDAPs, depending on what kind of average is utilized, display the smallest cost overruns. In absolute terms, the Air Force shows the lowest total in real cost overruns. Notably, while the Navy performs relatively well on a percentage basis, it also has the largest share of overruns in absolute terms for any of the three Service branches. This can be attributed to the size and duration of many Navy programs.
This comparison provides further support for the assertion that MDAPs managed by the Army and the Navy suffer smaller overruns, while DoD-wide managed MDAPs tend to accrue larger overruns. However, the level of analysis conducted so far does not allow for any firm conclusions on the actual role of any Service’s program management skills in these trends.
^Åèìáëáíáçå oÉëÉ~êÅÜW `ob^qfkd pvkbodv clo fkclojba `e^kdb - 14 -
(20)
‐
20
40
60
80
100
120
‐100% 0% 100% 200% 300%
Real co
st overrun
s (in
2010 billion
s, quantity
adjusted
)
Real cost overruns (in percent, quantity adjusted, prime averages unweighted)
Boeing General Dynamics Lockheed Martin
Northrop Grumman Raytheon Other
F‐35
CHEM DEMIL‐ACWA
FCS
(2)
‐
2
4
6
8
10
‐25% 0% 25% 50%
Boeing Average General Dynamics Average
Lockheed Martin Average Northrop Grumman Average
Raytheon Average Other Average
Figure 6. Cost Overruns by Prime Contractor (I) Note. The source for this figure was Selected Acquisition Reports; the analysis was by CSIS Defense-Industrial Initiatives Group.
Another predictor for program performance could be the identity of the prime contractor for a given program. One striking trend in Figure 6 that is visible for the “big five” U.S. defense companies is the fact that Raytheon on average appears to be associated with significantly better cost performance outcomes than other defense companies. Due to a lack of data granularity, the other companies category includes joint ventures and projects that are split between multiple contractors.
The preliminary character of the analysis does not fully validate any findings of superior management or outcomes. In addition, even if confirmed, it would be premature to start praising any company for better program execution because other factors such as specialization in technologically more mature program areas might be the true drivers behind this trend. As was the case for the breakdown by lead Service, further research will be needed to analyze the underlying causality.
^Åèìáëáíáçå oÉëÉ~êÅÜW `ob^qfkd pvkbodv clo fkclojba `e^kdb - 15 -
$90
$37
$132
$26$9
$50
‐$0.1‐$6.5 ‐$0.9 ‐$1.6 ‐$0.3 ‐$1.2
31%
13%
19%14%
9%
24%30%
20%
29%
17%13%
17%
‐10%
0%
10%
20%
30%
40%
50%
$(30)
$‐
$30
$60
$90
$120
$150
Boeing General Dynamics
Lockheed Martin
Northrop Grumman
Raytheon Other
Real co
st overrun
s (in
percent, quantity
adjusted
)
Real co
st overrun
s (in
2010 billion
s, quantity
adjusted
)
Cost Overruns Cost Savings Average Overrun Baseline‐Weighted Average Overrun
Cost Overruns
Cost Savings
Figure 7. Cost Overruns by Prime Contractor (II) Note. The source for this figure was Selected Acquisition Reports; the analysis was by CSIS Defense-Industrial Initiatives Group.
The comparison between the share of cost growth and the share of contract value for MDAPs, aggregated by prime contractor, correlates with the finding that MDAPs for which Raytheon is the prime contractor appear to exhibit the best cost performance amongst the big five defense companies. When it comes to the remainder of the big five, Figure 7 shows that their average performance varies based on the means used to measure it with different results when the programs are weighted by the baseline estimate than if all of the MDAPs are treated as having an equal weight. Again, this variance gives reason to be cautious in extrapolating from these results.
^Åèìáëáíáçå oÉëÉ~êÅÜW `ob^qfkd pvkbodv clo fkclojba `e^kdb - 16 -
$19 $2 $5 $6 $6
$167
$209
‐$2.5 $0.0 ‐$0.1 $0.0 $0.0 ‐$0.9 ‐$6.5
32%
18%
9%
31%
26%23%
19%22% 19%
9%
35%
15%
31% 31%
‐10%
0%
10%
20%
30%
40%
50%
60%
$(40)
$‐
$40
$80
$120
$160
$200
$240
Full comp. (2+ bids)
Full comp. (1 bid)
Partial comp. (2+ bids)
Partial comp.
(1 bidder)
Follow on to
competed action
No comp. Unclear comp.
Nom
inal Cost O
verrun
s (In
percent, w
eighted by
percent of M
DAP
accoun
ted for in FPDS
, quantity
adjusted
)
Nom
inal co
st overrun
s (in
then
year billion
s, quantity
adjusted)
Cost Overruns Cost Savings Average Overrun Baseline‐Weighted Average Overrun
Cost Overruns
Cost Savings
Figure 8. Cost Overruns by Type of Competition Note. The source for this figure was Selected Acquisition Reports; the analysis was by CSIS Defense-Industrial Initiatives Group.
The type of contract award process could potentially also be correlated with the cost performance of MDAPs. The findings in Figure 8 are decidedly mixed. In absolute dollar terms, competitive contracts produce less cost growth than contracts awarded with no competition or under unclear circumstances. This is driven by the comparative scarcity of competed contract dollars in the sample. As a result, when comparing relative cost overrun rates the results are different. Only partial competition3 with multiple bidders displays a notably better outcome.
Perhaps surprisingly, full and open competition with multiple bidders performs on average worse than no or unclear competition. Only when considering baseline-weighted averages does full and open competition with multiple bidders perform better than no or unclear competition. Based on the SAR’s data, this can be attributed to full and open competition with multiple bidders having the highest percentage of estimating variance of any of the categories. This result is consistent with the hypothesis that bidders may propose lower costs in order to win price-based competitions. However, further study would be needed to determine whether full and open competitions also suffer from a selection bias or other unexplained cause.
3 Partial competition refers to forms of competition other than full and open because the number of bidders is legally limited.
^Åèìáëáíáçå oÉëÉ~êÅÜW `ob^qfkd pvkbodv clo fkclojba `e^kdb - 17 -
$67
$20
$69 $52
$211
‐$1.2 $0.0 ‐$2.2 ‐$0.1 ‐$6.5
20%
27%
28%
35%
19%20%
28%32%
44%
31%
‐5%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
$(25)
$‐
$25
$50
$75
$100
$125
$150
$175
$200
$225
Fixed price Combination Cost‐plus award/incentive
Cost (all other*) Unspecified
Nominal Cost O
verrun
s (In
percent, w
eighted by
percent of M
DAP
accoun
ted for in FPDS
, quantity
adjusted)
Nom
inal co
st overrun
s (in
then
year billion
s, quantity
adjusted)
Cost Overruns Cost Savings Average Overrun Baseline‐Weighted Average Overrun
Cost Savings
Cost Overruns
Figure 9. Cost Overruns by Contract Type Note. *Cost (all other) includes time and materials contracts as well as labor hours contracts. The source for this figure was Selected Acquisition Reports; the analysis was by CSIS Defense-Industrial Initiatives Group.
Contract structure provides another possible determining factor for the performance of MDAPs. One key observation from Figure 9 is that fixed price contracts appear to have on average less cost growth and the cost all other contract types appear to have more, when comparing the share of cost growth and the share of contract value for MDAPs. An interesting finding is the fact that unspecified contract types, while responsible for the majority of cost overruns in absolute terms, perform best when measured based on baseline-weighted averages.
Acquisition reformers often point toward cost-plus contracts as a factor driving cost overruns. This argument is supported by the high average cost overrun percentages of both categories of cost plus contracts. The type of fee structure used also appears relevant, because cost-plus award/incentive contracts have lower relative cost growth than all other forms of cost reimbursement contracting although this is driven in part by the outsized influence of the F-35 project which falls within the cost (all other) category. However, fixed price contracts are more commonly the vehicle of choice for mature technology in full rate production, which are generally considered low risk.
Findings This report provides a foundation for future researchers and reformers grappling with
the problem of cost overruns in major defense acquisition projects. The results discussed below have been validated by the two most recent Selected Acquisition Reports, and together with the underlying data and methodology provide a roadmap for future work.
The strongest correlation with net cost growth is shown in Figure 2: changes in cost estimates are responsible for around 40% of the accumulated cost overruns. Of similar importance, Figure 3 shows that the start year has little impact on the compound annual
^Åèìáëáíáçå oÉëÉ~êÅÜW `ob^qfkd pvkbodv clo fkclojba `e^kdb - 18 -
growth rate of cost overruns. This suggests that the relatively better performance of newer programs may prove illusionary as programs age. Finally, Figure 9 shows that fixed price contracts appear to have relatively smaller overruns, although this may tell us more about which programs are likely to receive fixed price contracts rather than what effect fixed price contracts may have on program performance.
There are three logical avenues for future research to build on these results. First, additional factors could be added to the mix to help allocate responsibility to the underlying characteristics of an MDAP versus the methods chosen to implement it. Second, the dataset could be steadily expanded to include completed projects and to widen the historical scope and sample size of the project. Third, researchers could examine cost growth throughout the history of a select number of programs and also better control for the effects of updated baselines on older projects. Finally, the government could facilitate all three approaches and enable a range of assessments by allowing outside researchers to access the data that underlies the Selected Acquisition Reports.
Reformers and others studying this issue can take the next step by accessing the data, which will be posted at the Defense-Industrial Initiatives Group website (http://www.csis.org/diig) in time for the May 2011 Naval Postgraduate School conference. The authors intend to stay fully engaged with this issue as the root causes underlying the crisis in MDAP cost growth are being identified and addressed.
References Acquisition history project working paper #3: The evolution of DoD Directive 5000.1
Acquisition Management Policy 1971-2003. (n.d.). Defense Acquisition History Project. Retrieved from http://www.history.army.mil/acquisition/research/working3.html
Cost and time overruns in Major Defense Acquisition Programs (MDAPs)
David Berteau, Director Defense-Industrial Initiatives GroupDefense Industrial Initiatives Group
Center for Strategic and International StudiesMay 12, 2011
NPS 8th Annual Acquisition Research Symposium May 11-12, 2010
Relative cost growth versus absolute cost growth120
F‐3510
80
100
ns adju
sted
)
6
8
40
60
Rea
l cos
t ove
rrun
0 bi
llions
, qua
ntity
FCS
4
-
20
(in 2
010
CHEM DEMIL‐ACWA
-
2
(20)-100% 0% 100% 200% 300%
(2)-25% 0% 25% 50%
2Note: The sample includes 92 FY2010 MDAPs with a baseline estimate beyond Milestone B in the June 2010 SAR as well as twelve additional cancelled programs, notably including the Future Combat System (FCS).Source: Selected Acquisition Reports; analysis by CSIS Defense-Industrial Initiatives Group
Real cost overruns (in percent, quantity adjusted)
Functional reasons for cost growth30%$240
$203
21%
13%15%$120
$180
erce
nt)
0 bi
llion
s)
$124
$50 $63
$203
$51
$3%4%
9% 8%
3%3%
4%
13%
3%0%
$60
ost o
verr
uns
(in p
e
over
runs
(in
2010
$3
-$3 -$4-$16
-$3 $0
2%%
0%
3% 0%0%
$(60)
$-
Rea
l co
Rea
l cos
t
-$86-15%$(120)
Quantity Schedule Engineering Estimating Support Other
3Note: The sample includes 92 FY2010 MDAPs with a baseline estimate beyond Milestone B in the June 2010 SAR as well as twelve additional cancelled programs, notably including the Future Combat System (FCS).Source: Selected Acquisition Reports; analysis by CSIS Defense-Industrial Initiatives Group
Cost Increase Cost Decrease Average Increase Baseline-Weighted Average Increase
Ti t l ti (% Ch )Time-cost correlation (% Change)300%
200%
250%
chan
ges)
100%
150%
cos
t gro
wth
rate
uste
d fo
r qua
ntity
0%
50%Rea
l (in
per
cent
, adj
u
-100%
-50%
1985 1990 1995 2000 2005 2010
4Note: The sample includes 92 FY2010 MDAPs with a baseline estimate beyond Milestone B in the June 2010 SAR as well as twelve additional cancelled programs, notably including the Future Combat System (FCS).Source: Selected Acquisition Reports; analysis by CSIS Defense-Industrial Initiatives Group
1985 1990 1995 2000 2005 2010
Year of last baseline estimate
Ti t l ti (C d A l G th R t )Time-cost correlation (Compound Annual Growth Rate)
140%
160%
C-130 AMP
100%
120%
140%
row
th ra
tey
chan
ges)
60%
80%
und
annu
al c
ost g
rdj
uste
d fo
r qua
ntity
20%
40%
Rea
l com
pou
(in p
erce
nt, a
d
EFV
-20%
0%
1985 1990 1995 2000 2005 2010
5
Note: The sample includes 92 FY2010 MDAPs with a baseline estimate beyond Milestone B in the June 2010 SAR as well as twelve additional cancelled programs, notably including the Future Combat System (FCS). WIN-T Increment 2 was changed baseline years in 2010, but the compound annual growth rate was calculated as if the change happened in 2009.Source: Selected Acquisition Reports; analysis by CSIS Defense-Industrial Initiatives Group
Year of last baseline estimate
Cost overruns by lead service (I)10 120
F-35
6
8
80
100
runs
ty
adj
uste
d)
2
4
40
60
Rea
l cos
t ove
rr10
bill
ions
, qua
nti
FCS
--
20
(in 2
0
CHEM DEMIL‐ACWA
(2)-25% 0% 25% 50%
Air Force Army Navy
DoD-wide Air Force Average Army Average
(20)-100% 0% 100% 200% 300%
Real cost overruns (in percent, quantity adjusted,
service averages are unweighted)
6Note: The sample includes 92 FY2010 MDAPs with a baseline estimate beyond Milestone B in the June 2010 SAR as well as twelve additional cancelled programs, notably including the Future Combat System (FCS).Source: Selected Acquisition Reports; analysis by CSIS Defense-Industrial Initiatives Group
DoD wide Air Force Average Army Average
Navy Average DoD-wide Average
service averages are unweighted)
C t b l d i (II)Cost overruns by lead service (II)75%$150
1%
44%45%
60%
$90
$120
verr
uns
ntity
adj
uste
d)
over
runs
ua
ntity
adj
uste
d)
$73$89 $124
17% 22%
41%
20%23%
30%$60
Rea
l cos
t o(in
per
cent
, qua
n
Rea
l cos
t oin
201
0 bi
llion
s, q
$
$58
-$3.9 -$1.9 -$0.5$4 3
11% 16%
0%
15%
$-
$30
(
$3.9 -$4.3
-15%$(30)Army Navy Air Force DoD-wide
7Note: The sample includes 92 FY2010 MDAPs with a baseline estimate beyond Milestone B in the June 2010 SAR as well as twelve additional cancelled programs, notably including the Future Combat System (FCS).Source: Selected Acquisition Reports; analysis by CSIS Defense-Industrial Initiatives Group
Cost Increase Cost Decrease Average Overrun Baseline-Weighted Average Overrun
Cost overruns by prime contractor (I)
100
120 F‐35
10
60
80
100
rrun
s nt
ity a
djus
ted)
6
8
20
40
60
Rea
l cos
t ove
2010
bill
ions
, qua
n
CHEM
FCS
2
4
(20)
-
20
(in 2 DEMIL‐ACWA
(2)
-
(20)-100% 0% 100% 200% 300%
Real cost overruns (in percent, quantity adjusted, prime averages unweighted)
(2)-25% 0% 25% 50%
Boeing Average General Dynamics Average Lockheed Martin Average
8Note: The sample includes 92 FY2010 MDAPs with a baseline estimate beyond Milestone B in the June 2010 SAR as well as twelve additional cancelled programs, notably including the Future Combat System (FCS).Source: Selected Acquisition Reports; analysis by CSIS Defense-Industrial Initiatives Group
Boeing General Dynamics Lockheed Martin Northrop Grumman Raytheon OtherNorthrop Grumman Average Raytheon Average Other Average
C t b i t t (II)Cost overruns by prime contractor (II)50%$150
31%
30% 29%30%
40%
$90
$120
erru
ns
ty a
djus
ted)
ns adju
sted
)
$90
$132
13%
19%
14%
24%30%
20%
17%
13%17%
10%
20%
$30
$60
Rea
l cos
t ove
(in p
erce
nt, q
uant
i
Rea
l cos
t ove
rrun
0 bi
llions
, qua
ntity
$37$26
$9
$50
-$0.1-$6.5
-$0.9 -$1.6 -$0.3 -$1.2
9%
0%
10%
$-
$30 (
(in 2
010
-10%$(30)Boeing General
DynamicsLockheed
MartinNorthrop Grumman
Raytheon Other
9Note: The sample includes 92 FY2010 MDAPs with a baseline estimate beyond Milestone B in the June 2010 SAR as well as twelve additional cancelled programs, notably including the Future Combat System (FCS).Source: Selected Acquisition Reports; analysis by CSIS Defense-Industrial Initiatives Group
Cost Increase Cost Decrease Average Overrun Baseline-Weighted Average Overrun
Cost overruns by type of competition
50%
60%
$200
$240
n FP
DS
,
ed)
$20932%
31%
35%31% 31%
30%
40%
50%
$120
$160
$200
uns
P
acco
unte
d fo
r in
)ost o
verr
uns
s, q
uant
ity a
djus
te
$16718%
9%
26%23%
19%22% 19%
15%10%
20%
30%
$40
$80
$120
omin
al C
ost O
verr
upe
rcen
t of
MD
APqu
antit
y ad
just
ed)
Nom
inal
co
n th
en y
ear b
illio
n
$19 $2 $5 $6 $6
-$2.5 $0.0 -$0.1 $0.0 $0.0 -$0.9-$6.5
9%
10%
0%
10%
$(40)
$-
$40
No
cent
, wei
ghte
d by
(in
-10%$(40)Full comp. (2+
bids)Full comp. (1
bid)Partial comp.
(2+ bids)Partial comp.
(1 bidder)Follow on to
competed action
No comp. Unclear comp.
(In p
erc
10Note: The sample includes 92 FY2010 MDAPs with a baseline estimate beyond Milestone B in the June 2010 SAR as well as twelve additional cancelled programs, notably including the Future Combat System (FCS).Source: Selected Acquisition Reports; 2004-2009 FPDS data; analysis by CSIS Defense-Industrial Initiatives Group.
Cost Increase Cost Decrease Average Overrun Baseline-Weighted Average Overrun
Cost overruns by contract type44%
40%
45%
$200
$225
FP
DS
,
uste
d)
28%
35%
28%
32%31%
25%
30%
35%
$125
$150
$175
uns
P
acco
unte
d fo
r in
)l cos
t ove
rrun
s on
s, q
uant
ity a
dju
$211
20%
27%
19%
20%
15%
20%
25%
$75
$100
$125
omin
al C
ost O
verr
y pe
rcen
t of M
DAP
quan
tity
adju
sted
Nom
ina l
(in th
en y
ear b
illi
$67
$20
$69 $52
-$1 2 $0 0 -$2 2 $0 1 -$6 50%
5%
10%
$-
$25
$50 No
rcen
t, w
eigh
ted
by
-$1.2 $0.0 $2.2 -$0.1 -$6.5-5%$(25)
Fixed price Combination Cost-plus award/incentive
Cost (all other*) Unspecified (In p
er
11
• Cost (all other) includes time and materials contracts as well as labor hours contracts. Note: The sample includes 92 FY2010 MDAPs with a baseline estimate beyond Milestone B in the June 2010 SAR as well as twelve additional cancelled programs, notably including the Future Combat System (FCS).Source: Selected Acquisition Reports; 2004-2009 FPDS data; analysis by CSIS Defense-Industrial Initiatives Group.
Cost Increase Cost Decrease Average Overrun Baseline-Weighted Average Overrun
Fi diFindings
• Changes in cost estimates are responsible for around 40 percent of the accumulated cost overruns.
• Newer programs appear not to perform better than older ro rams when ud ed based on com ound annual p g j g p
growth rate.• Fixed price contracts appear to have relatively smaller
overrunsoverruns.• Data will be posted at the Defense-Industrial Initiatives
Group website (http://www.csis.org/diig).
12
Ab t C IS
At a time of new global opportunities and challenges, the Center for Strategic and International Studies (CSIS) provides strategic insights and policy solutions to decisionmakers in government international
About CSIS
(CSIS) provides strategic insights and policy solutions to decisionmakers in government, international institutions, the private sector, and civil society. A bipartisan, nonprofit organization headquartered in Washington, DC, CSIS conducts research and analysis and develops policy initiatives that look into the future and anticipate change.
Founded by David M Abshire and Admiral Arleigh Burke at the height of the Cold War CSIS wasFounded by David M. Abshire and Admiral Arleigh Burke at the height of the Cold War, CSIS was dedicated to finding ways for America to sustain its prominence and prosperity as a force for good in the world.
Since 1962, CSIS has grown to become one of the world’s preeminent international policy institutions, with more than 220 full-time staf nd a la l n ywith more than 220 full-time staff and a large network of affiliated scholars focused on defense and security, regional stability, and transnational challenges ranging from energy and climate to global development and economic integration.
Former U.S. senator Sam Nunn became chairman of the CSIS Board of Trustees in 1999, and John J. Hamre has led CSIS as its president and chief executive officer since April 2000Hamre has led CSIS as its president and chief executive officer since April 2000
CSIS does not take specific policy positions; accordingly, all views expressed in this presentation should be understood to be solely those of the author(s).
