Secure and optimize development budgets on

projects

Olivier is IAC's reference on reducing development

costs for projects. He leads missions in many

industrial sectors such as aerospace, defense,

medical, rail transport, etc.

Olivier Saint-Esprit

Partnerolivier.saint-esprit@iac.fr+33 (0)6 28 72 07 67

Customer references:

2

Focus on development costs in aeronautics and defense

SPACE DEFENSE AERONAUTICS CONSUMER GOODS

Telecom Payload Radar System Power Distribution System Household appliance

Non-recurring costs 30 M€ 100 M€ 20 M€ 5 M€

Recurring unit cost 1 M€ 0,5 M€ 50 k€ 50 €

Annual production volume 2 units 10 units 20 units 500 000 units

Total production time 5 years 20 years 30 years 5 years

Cumulative recurring costs 10 M€ 100 M€ 30 M€ 125 M€

Thus, in aeronautics, space and defense, development budgets usually represent an average of 40%

to 80% of the total cost that a company must incur to develop and then manufacture a new system.

As a comparison, the ratio of non-

recurring costs sharply decreases for

less capital-intensive industries such as

consumer goods.

In industrial sectors with low production volumes, such as space, defense and aeronautics, medical

imaging or high-tech equipment, the amount of non-recurring costs1 spent by an industrial on a

new program is frequently equivalent to or greater than the sum of recurrent manufacturing costs

accumulated over the entire production life of the equipment.

The table below provides some representative orders of magnitude by sector of activity.

75%Space

Defense

Aeronautics

Consumer Goods

25%

50%50%

40%60%

4%96%

Non-recurring costs Cumulated recurring costs

1 Non-recurring costs refer to expenses incurred during the development of a project, to design, test and prototype a new system or equipment. These costs are spent only once, unlike the recurrent manufacturing costs that need to be incurred for each unit during the serial production phase.

3

Initial budget Budget at completion

GALILEO Satellite positioning system € 5bn € 13bn6

A400M ATLAS Military transport aircraft € 20bn€ € 31bn7

B787 Long-haul aircraft $ 5,8bn $ 32bn88

F-35 Lightning II Multirole combat aircraft $ 233bn > $ 400bn99

The typical cost structure of a product

development in aeronautics and defense is

presented opposite.

Engineering costs usually represent 70% of total

non-recurring costs2.

70%Engineering

Industrialisation

Risk provision

Prototypes

Tests

13%

10%

4%

3%

Securing these budgets is a major challenge

for industrial groups, as any drift affects the

company’s profitability and cash flow. In some

extreme cases such as in the nuclear power

sector, historical players have found themselves

in bankruptcy due to technological failures and

uncontrolled delays3.

A company's competitiveness regarding its

development expenditure is therefore crucial.

It is a matter of winning contracts and preparing

for the future, by preserving the company’s

capacity to carry out self-financed research

activities.

Nevertheless, most of the flagship programs in

aeronautic, space and defense are experiencing

budget and time overruns4.

For several reasons, notably related to increasing

technological complexity of equipment5

many programs are indeed seeing their initial

development budget explode:

2 Certification costs are not considered here; still they represent an important cost in aviation.

3 https://www.lecho.be/entreprises/energie/La-debacle-des-geants-du-nucleaire-Westinghouse-et-Areva/9880880

4 http://www.cad-magazine.com/sites/default/files/articles/pdf/cad192_pp16-18_repere%20siemens.pdf

5 Sophie LEFEEZ, « Toujours plus chers ? Complexité des armements et inflation des coûts militaires », IFRI Focus stratégique, n° 42, février

2013 https://www.ifri.org/sites/default/files/atoms/files/fs42lefeez.pdf

6https://www.latribune.fr/entreprises-finance/industrie/aeronautique-defense/combien-va-couter-le-programme-galileo-a-leurope-546146.html

7 https://fr.wikipedia.org/wiki/Airbus_A400M_Atlas#Retards_et_surco%C3%BBts

8 https://www.seattletimes.com/business/boeing-celebrates-787-delivery-as-programs-costs-top-32-billion/

9 https://www.usinenouvelle.com/article/le-couteux-chasseur-f-35-de-lockheed-martin.N498089

4

01 04

02 05

04

03

In the absence of well-established time reporting methods, information about the full cost of a project and the detail of these costs is not always available nor well documented. This difficulty is reinforced when significant volumes of hours have been outsourced externally;

Despite this situation, companies often devote less attention to securing their development costs, compared to efforts dedicated to optimizing the recurring costs of their products.

Several factors explain this state of affairs:

Few feedback and root cause analysis exercises are actually conducted. When they are, most of the projects concerned are those that have gone off track. Good practices implemented on projects that have met their commitments are rarely clearly highlighted, as compliance with initial estimates is considered as normal;

Finally, it should be pointed out (let’s point out?) that the optimization of study budgets remains a taboo in some companies because engineering teams are still reluctant to apply to themselves the principles of efficiency inherited from Lean Manufacturing, now well integrated by production teams12.

More and more sophisticated predictive methods are now available to evaluate the manufacturing costs of a product (see our recent article on this subject: Predictive costing for your competitiveness10), but these methods are still deficient to estimate a project’s study volumes;

Study costs are sometimes knowingly underestimated at the beginning of a project to win the investment decision with acceptable profitability11. When these biases are noted, the lack of reference points caused by the deficiencies listed in the previous points limits the possibilities of objection;

10 https://www.interactionconsultants.com/en/publication/predictive-costing-to-serve-your-competitiveness

11 The case of the A400M is emblematic of a voluntary underestimation accepted by all parties. When you look at the functions requested on the one hand, and the time and cost that the parties have committed to on the other, "the product is impossible," says one specialist. Valérie Lion, " A400M rattrapé au vol ", L'Express, n°25, 24-30 June 2011, p. 84. 86

12 Lean Engineering is a variation, intended for engineering activities, of the technical principles derived from Lean Manufacturing: to produce exactly the necessary knowledge at the right time (knowledge being perishable), without jolts or unnecessary "stocks" and without wasting time in activities without added value.

5

Nevertheless, an increasing number of companies choose to dedicate more attention to accurate estimation and securitization of their development budgets.

The first 2 steps are then essential to any progress initiative. They consist in ensuring the correct dimensioning of the initial provision, and especially in securing this initial provision to avoid slippages during the execution phase. This comes well before considering the reduction of the amounts engaged.

The estimation of the most accurate budget upstream of a project is the first key to competitiveness. Indeed, a budget that is poorly estimated or insincere bears the seeds of future overruns.

Frank Freiman has modelled the impacts of a poor budget estimation for a project, both upward and downward. The curve that bears his name illustrates that:

• when costs are underestimated, initial manufacturing and planning plans prove impractical. The actual cost then explodes as the project must be reoriented. It results in unplanned additional work, reorganizations and need of additional resources.

• the overestimation of costs is self-fulfilling, leading to waste

1 = Underestimates lead to disaster2 = Realistic estimates minimize final costs

3 = Overestimates become self-fulfilling prophecies

Figure by MIT OCW. Adapted from Freiman, F. R. “The Fast Cost Estimating Models.” AACE

Transactions (1983).

Estimated Cost

0

1 2 3

0

10

10

20

20

The Freeman Curve

Fina

l Pro

grm

Cos

t

66

Despite the quality of the initial estimate,

the cost drift factors of a project remain

important. The missions conducted in recent

years by the IAC teams have allowed to

identify 3 major causes to cost drifts of a

project after its launch. It is a matter of:

Changes in the expression of needs after the start of the project;

Frequently insufficient evaluation of maturity on the technological bricks implemented on the project;

Incorrect estimation of the actual carry over level (or reuse) of solutions from one program to another.

Let’s develop these 3 points in the rest of this

article.

01

02

03

77

1. Evolutions in the expression of needs during a project are the source of many slippagesIn complex system development projects, such

as railway or aeronautical programs for example,

changes frequently occur. Of course, it would

be illusory to imagine abolishing changes

completely because they are intrinsic to

concurrent engineering project methodologies.

They allow for instance to begin the study of

the fuel system on an aircraft program without

waiting for the certification of the engine.

Flexibility is also needed to adapt to late market

developments, for example in the space sector

where development cycles are increasingly

shorter.

Moreover, design work is sometimes initiated

“ahead of schedule” in anticipation of

formalization of the customer need – in certain

cases even before the official notification

or launch of the project. This anticipation is

made in the hope of “buying time” or freeing

up calendar margins on a schedule deemed too

constrained. Although based on good intentions,

these anticipations frequently result in a

particularly high scrap14 or rework15 rate. They

can even lead to impossibilities such that the

project cannot succeed, as it was the case for

the LOUVOIS payroll software launched in 1996

and definitively abandoned in 2013 without ever

having functioned properly16.

However, unpredictable specification changes

remain marginal: the review of specifications in

the preliminary design phase makes it possible

to know their sensitivity and quantify their

evolution risk.

The best approach then consists in anticipating

the occurrence of modifications: adjusting

the development logic accordingly and

communicating to the customer and internal

entities potentially generating modification

requests, the dates by which they must have

been notified.

The impact of a specification evolution during

development depends on 3 parameters:

a) progress of the program

b) typology of the business activities affected

(system engineering, mechanical design,

hardware or software electronic design, etc.)

c) interdependencies between departments.

13 Golub's Law No. 2, a famous unknown computer scientist.

14 Scrap: waste: activities carried out are useless for the project, and the time spent is a dry loss.

15 Rework: by integrating additional activities not initially planned (thus generating additional costs), the work can be reoriented to meet the needs of the project.

16 "It also appears that the functional design of this information system was insufficient to model such complexity. Indeed, the general functional specifications were not even written with enough acuity even though the realization of the software had started. In 2015, four years after the first deployment, the delegated project owner had to write functional specifications of allowances in a hurry, even though they have been calculated since deployment. Unique joint pay software. (2018, February 18). In Wikipedia. Retrieved from https://fr.wikipedia.org/w/index.php?title=Logiciel_

"One of the advantages of setting vague objectives for a project is that you will have no difficulty estimating the corresponding expenses...13".

88

The flexibility matrix presented below is an educational tool that can be used to explain the level of

impact of a specification evolution as the project milestones are crossed.

This matrix can be implemented not only to improve communication between internal teams, but also

proactively for the external client, to highlight the impact of a change in the expression of need and

to contract in advance the additional cost that will be invoiced in case of late evolution.

This tool can also be deployed for suppliers, to make the entire development process more flexible

between project partners.

Low impact Medium impact Strong impactNo impact

Impact on budget and planning

MECHANICAL design

HARDWARE design of

electronic boardsWIRING

SOFTWARE modification of

function

Addition of a SOFTWARE

function

Before KOM

No Impact if relocation of mechanical

interfaces (same volume & shape)

No impact if up to 10 signals

modification on ICD

No impact if:- Pin allocation

modification- Gauge

modification- Routing

modification- Add/removal of wire in the

perimeter of the defined connector

No impact if modification can be achieved in

current processing ressources

AFTER KOM

AFTER PDR B1 standard

AFTER CDR B1 standard

AFTER PDR B2 standardAFTER B1 standard deliveryAFTER CDR B2 standardAFTER B2 standard delivery AFTER B2 standard Flight Clearance

AFTER CDR C-model

After C-model delivery

Enginering department impacted by a change in specification

Prog

ram

mile

ston

es

Example of a flexibility matrix for a program to develop an aeronautical

electrical generation system

9

Average Program Research, Development, Test, and Evaluation Cost Growth from First Full Estimate

2. Assessing the maturity of the technological bricks implemented on the project is also crucialIndeed, a development program is generally built

around Key Technical Elements17 made available by

the company's upstream research departments.

Controlling the level of technological maturity

of these bricks is one of the success factors

of the project.If the development planning is

based on the use of mature bricks, and they

are not in fact, then they prove unusable by the

development teams and additional work must

urgently be carried out.

One of the major causes of budget drift lies

in insufficient prepared incorporation of these

bricks into the program schedules.

The United States Government Accountability

Office (GAO) analyzed 52 weapons programs. It

revealed that most of these programs started

with lower levels of knowledge and maturity than

recommended by best practice. The table18 below

shows that programs that started with immature

bricks recorded an average 34.9% increase in

their development budgets, compared to only

4.8% for projects supported by mature bricks.

Mature technologies

4,8

%

34,9

Immature technologies

0

10

20

30

40

17 A technical element is said to be "key" if the product or system considered for evaluation depends on this technical element to achieve its main functionalities with an acceptable cost and development time, as well as production costs, and if this technical element or its implementation is intrinsically new or only new in their implementation environment.

18 Office, U. S. G. A. (2006). Defense Acquisitions: Assessments of Selected Major Weapon Programs, (GAO-06-391). Retrieved from https://www.gao.gov/assets/250/249468.pdf

“What gets us into trouble is not what we don't know. It's what we know for sure that just ain't so…” Mark Twain

10

Knowledge of the TRL scale (Technology Readiness

Level-developed by NASA in 1974 to provide a

common basis for defi ning the level of maturity of

a technology) is now widespread, but we note that

its eff ective use during the planning and budgeting

phases of projects remains insuffi cient.

Inappropriate implementation of this indicator can

also lead to heavy additional costs to projects

budgets because despite the linearity of this scale,

the eff orts to exceed level 6 are very substantial19.

Real system completed and qualifi ed by successful operational missions

Real system completed and qualifi ed by tests and demonstrations

Demonstration of a prototype system in an operational environment

Demonstration of a prototype or model system/subsystem in a representative environment

Validation of components and/or models in a representative environment

Validation of components and/or models in the laboratory

Analytical or experimental evidence of the main functions and/or characteristics of the concept

Technological concept and/or formulated applications

Basic principles observed or described

TRL 9

TRL 8

TRL 7

TRL 6

TRL 5

TRL 4

TRL 3

TRL 2

TRL 1

System test, launch and

reindustrialisation

System/subsystem

development

Technology Development

Basic technological

research

Technology Demonstration

Research and demonstration of

feasibility

TRL scale from DGA's 2009 Defense and Security Research and Technology

Strategic Plan 20

19 "Actually, there is a huge distance between being technically proven and implementing successfully. This diffi culty in the transition of a new technology is aff ectionately called "Death Valley." Boeing: Technology Readiness and the Valley of Death. (n.d.). Accessed April 11, 2018, at http://www.boeing.com/features/innovation-quarterly/may2017/feature-thoughtleadership-newman.page

20 Armaments Portal: The Strategic Research and Technology Plan (SP R&T) 2009. (n.d.). Retrieved 11 April 2018, from https:// www.ixarm.com/sites/default/fi les/documents/PS_R_T_english_web_0.pdf

11

We have identified 3 main reasons why errors are made in assessing the maturity of key technical

elements despite the use of the TRL scale:

01

03

02Misunderstandings persist frequently between teams on the actual level of

maturity as the R&T teams in charge of the

study and the provision of the technological

bricks are not the ones who carry out the

development: an organizational solution to

this pitfall then resides in the constitution of

integrated Research ↔ Development teams;

Finally, when TRL and/or MRL levels are correctly assessed at the beginning of the project, the development logic and the detailed planning of the resulting activities are not always constructed to

ensure a gradual rise in maturity. A flowchart that highlights the contribution of each activity to this rise

in maturity (see diagram below) allows to organize development accordingly, for example by providing

upstream risk mitigation activities, or by "taking" an activity off the critical path.)

The TRL rating alone is not sufficient to account for the incorporability of a brick within a development: other

indicators must be assessed, at least the MRL21,

or even other indicators capable of restoring

the ease of integration of a component within

a complex system22;

21 There is an established Manufacturing Readiness Level (MRL) scale, adopted by the U.S. Department of Defense in 2005, that addresses the feasibility and affordability of producing the technology at the required scale and rate. There are also integration maturity metrics used to assess an Integration Readiness Level (IRL) scale. And both contribute to a System Readiness Level (SRL). Boeing: Technology Readiness and the Valley of Death. (s. d.). Accesed 11th April 2018, on http://www.boeing.com/features/innovationquarterly/may2017/feature-thought-leadership-newman.page

22 See on this subject: Garg, T., Eppinger, S., Joglekar, N., & Olechowski, A. (2017). Using TRLs and system architecture to estimate technology integration risk. Accessed online 11 April 2018 at: http://web.mit.edu/eppinger/www/pdf/Garg_ICED2017.pdf

ActivitéB

ActivitéC

ActivitéD

ActivitéA

TRL 6

MRL 6 MRL 6

MRL 7 MRL 7

MRL 7

MRL 5

MRL 5

MRL 5

TRL 7

TRL 7 TRL 8

TRL 7 TRL 7

TRL 7TRL 6

ActivitéF

ActivitéE

A simple graphical representation of this type highlights the critical path(s) (in blue) of maturity rise on the different indicators selected - it should be pointed out that the level of maturity resulting from several activities is always equal to the lowest level of maturity.

12

3. The inaccurate estimate of the carry over level generates risks

Finally, it is the incorrect estimation of the actual

carry over (or reuse) level of the solutions from

one program to another that generates risks of

discrepancies between the initial forecast and

the actual figure.

Indeed, it is not often that a new development

is done entirely from scratch, without providing

for the re-use of a physical element or software.

Reuse or carry over assumptions are then

considered when budgeting activities, in order to

materialize the economy of the redevelopment

of these functions or subsets previously

developed. The most mature industries have

defined within the framework of Product Policy

reflections elementary bricks, intended to allow

the design with less delays and less costs for

future products in a logic of modular design

(we also speak of Platforming23 or sometimes

of versioning). In the energy and rail sectors,

the most advanced players have begun their

transition from the "Engineering-to-order24"

world to the "Configure-to-order25".

However, the carry-over of a hardware or

software solution does not spare the company

from re-expending a substantial budget to adapt

and incorporate the re-used solutions, if this

carryover was not thought through during the

initial design steps. Indeed, "technical difficulties

precisely lie in systems integration: the cost

and organizational difficulties are exponential

compared to the additional functions26",

which means that integrating a reuse solution

into a complex system can prove extremely

complex, and ultimately can cost as much as

a new development if this integration was not

initially planned. The Reuse Readiness Level

(RRL27) indicator proposed by NASA aims at

explaining the level of re-usability of a software

in a different application than the one it was

originally developed for.

23 https://www.iac.fr/publication/publication-platforming

24 Engineer-to-order (ETO) approach is one in which a company designs and manufactures a product based on very specific customer requirements. Engineered-to-Order (ETO) | Arena Solutions. (s. d.). Accessed 11th April 2018, on https://www.arenasolutions.com/ resources/articles/engineered-to-order/

25 Configure-to-order (CTO) represents the ability for a user to define the component make-up (configuration) of a product at the very moment of ordering that product, and a vendor to subsequently build that configuration dynamically upon receipt of the order. Configure To Order Manufacturing Software. (s. d.). Accessed 11th April 2018, on http://www.software4manufacturers.com/manufacturing/styles_configure_to_order.aspx?Styles_Shortname=CTO

26 Sophie LEFEEZ, "Always more expensive? Complexity of armaments and inflation of military costs", IFRI Strategic Focus, No. 42, February 2013 https://www.ifri.org/sites/default/files/atoms/files/fs42lefeez.pdf

27 Reuse Readiness Levels (RRLs) - Earth Science Data System Working Groups - ESDSWG - Earthdata Wiki. (n.d.). Accessed 11 April 2018, at https://wiki.earthdata.nasa.gov/download/attachments/49446977/RRLs_v1.0.pdf?version=1&modificationDate=1428606889506&api=v2

13

It is also the foundation of the Design for

Variety28 methodology, aimed at developing

standardized and modular product

architectures. This approach implements two

parameters:

• the Generational Variety Index (GVI) which

represents the necessary redesign intensity

to meet future market needs;

• the Coupling Index (CI) indicating the intensity

of the coupling between the components of

a product.

This methodology will be detailed in a

forthcoming IAC publication on modular

design methodologies.

Thus, we note that a sizable proportion

of budget overruns are related to a poor

upstream assessment of the actual re-

usability of solutions previously developed.

Careful attention to these aspects upstream

of a project together with the use of objective

evaluation tools enables to correctly assess

the reuse assumptions on which the project

budget is built.

28 Martin, M. V., & Ishii, K. (2002). Design for variety: developing standardized and modularized product platform architectures. Research in Engineering Design, 13(4), 213-235. Accessed 11th April 2018, on http://web.mit. edu/deweck/Public/ESD39/Readings/GVImetrics.MartinIshii-2002.pdf

14

To Conclude

The aeronautics, space, telecommunication

and defense sectors are now seeing their

development cycles shorten, due to the

emergence of new players and profound

upheavals in business models.

This acceleration combined with a technological

frenzy29 jeopardizes the ability of industrial

companies to realistically assess and secure

their development budgets.

If numerous factors of uncertainty can never be

eliminated such as the evolution of need during

the project or the development difficulties of

a new technology, good practices consist in

assessing the risk levels incurred as soon as

possible and with an objective approach. When

carried out upstream, these preparatory works

turn out to be a very good investment, helping

to avoid heavy slippages. They moreover open

the door to a serene progress of the project by

increasing the level of confidence of the team

in the submitted budget.

29 "The F-35 Joint Strike Fighter contains an "crazy number" of chips according to a semiconductor expert." The Hunt for the Kill Switch", IEEE spectrum magazine, May 2008. quoted in Sophie LEFEEZ, "Ever more expensive? Complexity of armaments and IFRI Strategic Focus, No. 42, February 2013 https://www.ifri.org/sites/default/files/atoms/files/fs42lefeez.pdf

Methods to optimize project execution such as Lean Engineering methodologies can then be deployed. They will be the subject of a future publication.

15