25
Project Scope Increases and Use of COCOMO II within Earned Value Management System Vicki Love Software Engineering Staff Raytheon Systems Company – IIS – Garland Division 972-205-4297 Copyright 2004, Raytheon Company Page 1 of 25 ALL RIGHTS RESERVED

COCOMO 2004 Conference

Embed Size (px)

Citation preview

Page 1: COCOMO 2004 Conference

Project Scope Increases and Use of COCOMO II within Earned Value Management System

Vicki LoveSoftware Engineering Staff

Raytheon Systems Company – IIS – Garland Division972-205-4297

Copyright 2004, Raytheon Company Page 1 of 18 ALL RIGHTS RESERVED

Page 2: COCOMO 2004 Conference

Project Scope Increases and Use of COCOMO II within Earned Value Management System

Table of Contents

Introduction..........................................................................4Introduction..........................................................................41 Components...................................................................4

1.1 Earned Value Management System.....................................................................41.2 Effort Model........................................................................................................4

1.2.1 Effort Model Calibrations............................................................................61.2.2 Effect of Scope Changes..............................................................................6

2 Case Study.....................................................................62.1 Project Execution.................................................................................................6

3 Real Life Results..........................................................134 Alternatives..................................................................14

4.1 Alternative 0 – Standard Use.............................................................................144.2 Alternative 1 - Estimate Project Growth...........................................................144.3 Alternative 2 - Calibration Process Change.......................................................164.4 Alternative 3 - For Non-Compressed Projects Only – Use Original Productivities to Bid All Changes.................................................................................16

5 Recommendation.........................................................176 Bibliography.................................................................18

Table of FiguresFigure 2-1 Diseconomy of Scale – Effort............................................................................5Figure 2-2 Diseconomy of Scale – Productivity..................................................................5Figure 3-1 Project Change Tracking Table.........................................................................7Figure 3-2 Variance Causes.................................................................................................8Figure 3-3 Project Tracking Table – Second RFC..............................................................9Figure 3-4 SCED EM..........................................................................................................9Figure 3-5 Diseconomy of Scale With Schedule Compression – SCED Step Function -

Effort..........................................................................................................................10Figure 3-6 Diseconomy of Scale With Schedule Compression – SCED Step Function -

Productivity................................................................................................................10Figure 3-7 Diseconomy of Scale With Schedule Compression – SCED Polynomial

Function – Effort........................................................................................................11Figure 3-8 Diseconomy of Scale With Schedule Compression – SCED Polynomial

Function – Productivity.............................................................................................12Figure 3-9 Project Change Tracking Table - Updated for new SCED Calculation...........12Figure 5-1 Project Changes - Alternative 0 versus Alternative 1......................................15Figure 5-2 Project Tracking Table - No Schedule Compression.......................................16

Copyright 2004, Raytheon Company Page 2 of 18 ALL RIGHTS RESERVED

Page 3: COCOMO 2004 Conference

Figure 5-3 Project Changes – Alternative 0 Versus Alternative 3....................................17

Copyright 2004, Raytheon Company Page 3 of 18 ALL RIGHTS RESERVED

Page 4: COCOMO 2004 Conference

1 IntroductionThis paper will explore the challenges of the use of the COCOMO II model along with the Earned Value Management System. The COCOMO II model is used to estimate the cost and schedule of the project and all changes in scope throughout the life of the project based on site specific historical data, project size estimates, and project characteristics. The Earned Value Management System is used in-process to monitor cost and schedule of program and to estimate project deviations from the plan based on actual performance and future work.The project described in this paper is based on the experience of an actual project performed by Raytheon for the United States Government. The specific numbers (size, productivity, hours, etc.) have been changed to protect proprietary information.

2 Components

2.1 Earned Value Management SystemEarned Value Management System (EVMS) is a control system which facilitates management of a project’s technical scope, schedule, and budget by focusing on deviations from the integrated project plans. The high level process followed is to determine the activities required to complete the project and to estimate the cost and schedule for each activity. The actual effort and time spent on the activities is collected, and various metrics are then collected that indicate variance from the plan (estimate).

2.2 Effort ModelA parametric effort model is a parameter driven mathematical model which is used to estimate the effort required to develop a system. The parameters are used to characterize the project. They, along with the size of the project, are used to generate the predicted effort, which is laid into the Earned Value Management System. In general, parametric models assume a non-constant relationship between size of project and the effort required to complete the project; the bigger a project is, the less productive it is. This relationship is called “diseconomy of scale”1. Figure 2-1and Figure 2-2 illustrate diseconomy of scale. On Figure 2-1, the blue line represents the effort associated with projects that vary in size up to one million ESLOC. The effort is computed using a productivity computed from a COCOMO II model of a 400 thousand ESLOC project using the industry calibration. The pink line represents the effort for the same projects, but instead of using a constant productivity, the COCOMO II model is run on each project size. As you can see, the effort is less than the effort computed using a constant productivity on the smaller projects and is more on the larger projects. Figure 2-1 is a graph of productivities on the same group of projects. Again, you can see that the model generates a higher productivity on smaller projects than on larger projects.

Copyright 2004, Raytheon Company Page 4 of 18 ALL RIGHTS RESERVED

Page 5: COCOMO 2004 Conference

Size and Effort

0

250000

500000

750000

1000000

1250000

0 100 200 300 400 500 600 700 800 900

Size (KESLOC)

Eff

ort

(H

ou

rs)

Example Parametric Size/Effort Relationship

Constant Size/Effort Relationship Using 400 KESLOC Productivity

Figure 2-1 Diseconomy of Scale – Effort

Size and Productivity

0

0.75

1.5

0 100 200 300 400 500 600 700 800 900

Size (KESLOC)

Pro

du

cti

vit

y

(ES

LO

C/H

ou

r)

Example Parametric Size/Effort Relationship

Constant Size/Effort Relationship Using 400 KESLOC Productivity

Figure 2-2 Diseconomy of Scale – Productivity

Both of these examples use unconstrained schedules, meaning that the project duration is nominal. When the schedule is constrained, or compressed (meaning the project

Copyright 2004, Raytheon Company Page 5 of 18 ALL RIGHTS RESERVED

Page 6: COCOMO 2004 Conference

development is accelerated), it typically takes more effort to complete the project. When schedule compression is a factor, the deviation from a constant relationship becomes even more significant. This will be discussed later.

2.2.1 Effort Model CalibrationsEffort models are typically calibrated with data from completed projects. This data is usually collected after the projects’ completion and represents an end view of the project, meaning that the characteristics, including size, of each project contained in the calibration set are the characteristics of the project at completion. The implication of this is that each bid using this calibration set is being modeled as if it is an end of project size, schedule, etc. This is generally not true because changes, especially in scope, typically occur throughout the development cycle. This method of calibrating the model does not cause problems when used only to bid a project at its start, but it can cause issues if the model is going to be used in-process to bid all scope changes throughout the life of the project, especially if the project is managed using EVMS.

2.2.2 Effect of Scope ChangesSince the project being bid is priced with the implicit assumption that its size is the expected end size, each scope increase changes the expected end size, and thus expected productivity of the whole program. As each size change drives overall productivity down, additional effort is incurred in areas of the project which aren’t directly affected by change, i.e. a CSCI which does not incur additional work does realize a productivity hit, and thus incurs additional effort.The case study in section 3 will demonstrate the interactions between the effort model and EVMS when the model is calibrated as described in section and is used in-process to bid changes. Section 4, discusses alternative methods of calibration.

3 Case Study

3.1 Project ExecutionThe scenario begins when the project is bid using the COCOMO II model. The project is 475K ESLOC and the schedule is 28 months; the project is just over 85% compressed. The expected productivity for the project is 1.0308 ESLOC/hr. The effort generated by the model is laid into EVMS for the all of the activities required to complete the entire project. Periodic analysis (monthly) is performed on each CSCI at the high level. Any variances from the plan are analyzed and contributing factors identified.The first request for change (RFC) comes in from the customer to add functionality. The customer has requested a not to exceed estimate (NTE) for this functionality. The RFC is evaluated; the additional functionality will add 5000 ESLOC to the project. A quick calculation using the expected project productivity and the ESLOC added by the RFC yields a cost of 4,856 hours (5000 ESLOC/1.030798 ESLOC/hr). This number is turned in. It is accepted, so the formal pricing process using the model begins. The functionality is added to the model, and the delta between the model with the functionality and the model without the functionality is taken as the cost for the additional ESLOC. The cost comes to be 5,404 hours. This is higher than the NTE– what happened? The productivity that was used to generate the NTE was for a smaller project. Due to diseconomy of scale,

Copyright 2004, Raytheon Company Page 6 of 18 ALL RIGHTS RESERVED

Page 7: COCOMO 2004 Conference

the productivity for the overall project dropped when the new functionality was added. The new productivity for the project has dropped to 1.029574. The cost for adding this functionality consists of two parts:

1. Due to Scope Change – those impacts directly associated with additional scopea. These are the additional ESLOC which have been added to the project as a

result of the scope changeb. This cost is calculated by dividing the added ESLOC by the new

productivity.i. In this example it is 5 KESLOC/1.02957 = 4856 hours

2. Productivity Impactsa. This is the additional effort generated as a result of increasing the size of

the project.b. This cost is calculated by subtracting the effort for original KESLOC

using the starting productivity from the effort for the original KESLOC using the productivity after the change.

i. In this example it is:475 KESLOC/1.02957 – 475 KESLOC/1.0308 = 548 hours

Lesson Learned 1If the model is to be used during formal cost submittal, it should also be

used during informal cost estimates; especially for Not To Exceed estimates.

The cost estimation process for the project was changed to use the model for both formal and informal cost estimates. Since size of the project impacts overall costs, all changes are modeled and tracked, including both internal change requests (ICR’s) and external requests for change (RFC’s) (see Figure 3-3).

Description Existing New Starting After Change

Due to Scope

ChangeProductivity

Impacts Starting After ChangeOriginal Estimate N/A 475 N/A 1.03 460,808 N/A N/A 85.5%RFC 1 475.0 5.0 1.03 1.03 4,856 548 85.5% 85.1%ICR 1 480.0 1.0 1.03 1.03 972 110 85.1% 85.1%

KESLOC Productivity Additional Hours Schedule Compression

Figure 3-3 Project Change Tracking Table

RFC 1 is negotiated; the Control Account Managers (CAM’s) now begin rolling the additional hours into the EVMS system. The additional hours due to the scope change are easy; new activities required to incorporate the change are created; the hours are spread across the new tasks. The productivity impacts are harder. Since the original bid was for a smaller program, the original estimated productivity was higher than project will actually realize. But the original effort was spread across the entire project, so all tasks have been allocated too little effort according to the latest model. Thus, as each change request decreases productivity for the project, past and future activities that were not directly

Copyright 2004, Raytheon Company Page 7 of 18 ALL RIGHTS RESERVED

Page 8: COCOMO 2004 Conference

impacted by the change itself need to be compensated with additional effort in order to reflect the new, lower productivity. This raises several issues:

1) Determining how much effort to put where is difficult.2) Per EVMS rules, budget may not be put into the past, so budget to account for

past activities is added into current month in correct WBS, which messes up current month CPI.

3) Looks like “getting well” – which is a big no-no.This brings us to our second lesson learned:

Lesson Learned 2A structured process is needed to ensure that hours generated by

productivity impacts are laid in using consistent rules.

The first thing that is done is to categorize the types of things that may cause variances. Each item is then characterized as a “productivity impact” from a model perspective or not. By doing this, some of the subjectivity in determining whether an activity should receive productivity impact hours from RFC’s is removed. It puts a more controlled and structured process in place to help determine where to put the productivity hours and how much to put where. Otherwise, the tendency would be to make up variances wherever they occur, even if they were not due to a productivity impact. Note that some things may truly be impacting productivity, but unless the item is the type of productivity impact that would be expected by diseconomies of scale, it is not a candidate for the productivity impacts generated by the model. See Figure 3-4 for an example of this.

Variance Cause Candidate for Productivity Hours?

More time spent than anticipated resolving interface issues. Yes

More time spent than anticipated in design reviews, boards, etc. to ensure that enough communication between stakeholders occurred. Yes

Team lead spent more time managing and less time coding than planned. Yes

More time spent integrating with other parts of system than anticipated. Yes

Productivity problem with particular engineer. No

Increased growth in area caused more engineers to be added to staff, thus decreasing team productivity due to ramp up time, additional management, etc. Yes

Software was more complex or required more changes than anticipated. No

Late coming requirements caused more swirl, thus affecting productivity. Yes

Engineers found new and improved ways to implement functionality, which makes addition of future capabilities easier, but took longer to implement than original design. NoPlatform instability impacted engineers' ability to perform work. No

Figure 3-4 Variance Causes

The CAM’s now need to carefully monitor their progress and productivity and note along the way anything that’s impacting their cost or schedule. At the end of each month, the EVMS data is distributed to each CAM. At this time, the CAM’s compare their subjective evaluations with the objective EVMS data. They update their notes as needed,

Copyright 2004, Raytheon Company Page 8 of 18 ALL RIGHTS RESERVED

Page 9: COCOMO 2004 Conference

detailing where productivity impacts have occurred and are likely to occur in the future so that they can defend where and when productivity impacts are rolled into EVMS. If productivity impacts are foreseen in the future, the additional hours are added to the appropriate tasks in the future with appropriate justification provided. If productivity impacts have been incurred in the past, new single day tasks with enough budget to absorb the past productivity impact are added to the current or next month in the proper WBS. Thus, issues number 1 and 3 are fixed (as best as possible); issue number 2 is still a problem that could not be overcome. It could be mitigated by providing additional information with any current month numbers describing what was done, why it was done, and what impact it had on the current month numbers.A new RFC comes in and is evaluated for a NTE estimate. This one adds 2 KESLOC to the system (see Figure 3-5).

Description Existing New Starting After Change

Due to Scope

ChangeProductivity

Impacts Starting After ChangeOriginal Estimate N/A 475 N/A 1.03 460,808 N/A N/A 85.5%RFC 1 475.0 5.0 1.03 1.03 4,856 548 85.5% 85.1%ICR 1 480.0 1.0 1.03 1.03 972 110 85.1% 85.1%RFC 2 481.0 2.0 1.03 0.90 2,216 65,673 85.1% 84.9%

KESLOC Productivity Additional Hours Schedule Compression

Figure 3-5 Project Tracking Table – Second RFC

The productivity impacts are ridiculously high – what happened? The COCOMO II SCED effort multiplier just hit a threshold (see Figure 3-6), so the productivity took a large jump as did the estimated effort. The additional 2KESLOC changed the schedule compression from just over 85% to just under 85%.

< 75% of Nominal75% of

Nominal85% of

Nominal100% of Nominal

>100% of Nominal

Unachievable Very Low Low Nomimal High-Very HighN/A 1.43 1.14 1.00 1.00

Rating LevelEffort Multiplier

SCED EM

Figure 3-6 SCED EM

Recall the diseconomy of scale curves in Figure 2-1 and Figure 2-2. When schedule compression becomes a factor, the deviation of these curves from a constant relationship becomes even more pronounced. The large increase in cost was a result of hitting a boundary for the SCED EM. Figure 3-7 and Figure 3-8 show the same projects as show in Figure 2-1 and Figure 2-1. The difference is that the schedules are constrained to the nominal schedule for a 400 thousand ESLOC project. This means that all of the projects that are bigger than 400 KESLOC are compressed. The red line shows what the schedule compression is for each project. Note the large drop in productivity and increase in productivity at the boundaries; the productivity and effort curves are step functions.

Copyright 2004, Raytheon Company Page 9 of 18 ALL RIGHTS RESERVED

Page 10: COCOMO 2004 Conference

Size and Effort - Step-Function SCED Curve

0

250000

500000

750000

1000000

1250000

0 100 200 300 400 500 600 700 800 900

Size (KESLOC)

Eff

ort

(H

ou

rs)

0.75

0.8

0.85

0.9

0.95

1

Sc

he

du

le

Co

mp

res

sio

n

Example Parametric Size/Effort RelationshipConstant Size/Effort Relationship Using 400 KESLOC ProductivitySchedule Compression Bin by Size

Figure 3-7 Diseconomy of Scale With Schedule Compression – SCED Step Function - Effort

Size and Productivity - Step-Function SCED Curve

0

0.75

1.5

0 100 200 300 400 500 600 700 800 900

Size (KESLOC)

Pro

du

cti

vit

y

(ES

LO

C/H

ou

r)

0.75

0.8

0.85

0.9

0.95

1

Sc

he

du

le

Co

mp

res

sio

n

Example Parametric Size/Effort RelationshipConstant Size/Effort Relationship Using 400 KESLOC ProductivitySchedule Compression Bin by Size

Figure 3-8 Diseconomy of Scale With Schedule Compression – SCED Step Function - Productivity

The step functions present a problem when using COCOMO II in-process as demonstrated in the case study. It is not reasonable that a relatively small addition of 2

Copyright 2004, Raytheon Company Page 10 of 18 ALL RIGHTS RESERVED

Page 11: COCOMO 2004 Conference

KESLOC to the project was going to add almost 68K hours of effort. This brings us to our third lesson learned.

Lesson Learned 3A smooth SCED curve is necessary when using COCOMO II in-process to

estimate costs of changes.

The project’s implementation of COCOMO II was updated to calculate SCED using a polynomial function in order to produce a smooth curve (see Figure 3-9 and Figure 3-10).

Size and Effort - Smooth SCED Curve

0

250000

500000

750000

1000000

1250000

0 100 200 300 400 500 600 700 800 900

Size (KESLOC)

Eff

ort

(H

ou

rs)

0.75

0.8

0.85

0.9

0.95

1

Sc

he

du

le

Co

mp

res

sio

n

Example Parametric Size/Effort Relationship

Constant Size/Effort Relationship Using 400 KESLOC ProductivitySchedule Compression by Size

Figure 3-9 Diseconomy of Scale With Schedule Compression – SCED Polynomial Function – Effort

Copyright 2004, Raytheon Company Page 11 of 18 ALL RIGHTS RESERVED

Page 12: COCOMO 2004 Conference

Size and Productivity - Smooth SCED Curve

0

0.75

1.5

0 100 200 300 400 500 600 700 800 900

Size (KESLOC)

Pro

du

ctiv

ity

(ES

LO

C/H

ou

r)

0.75

0.8

0.85

0.9

0.95

1

Sc

he

du

le

Co

mp

res

sio

n

Example Parametric Size/Effort Relationship

Constant Size/Effort Relationship Using 400 KESLOC ProductivitySchedule Compression by Size

Figure 3-10 Diseconomy of Scale With Schedule Compression – SCED Polynomial Function – Productivity

Once this was done, the model was rerun with the additional ESLOC added by RFC 2 (see Figure 3-11).

Description Existing New Starting After Change

Due to Scope

ChangeProductivity

Impacts Starting After ChangeOriginal Estimate N/A 475 N/A 0.89 531,183 N/A N/A 85.5%RFC 1 475.0 5.0 0.89 0.89 5,625 3,195 85.5% 85.1%ICR 1 480.0 1.0 0.89 0.89 1,126 651 85.1% 85.1%RFC 2 481.0 2.0 0.89 0.89 2,258 1,310 85.1% 84.9%

KESLOC Productivity Additional Hours Schedule Compression

Figure 3-11 Project Change Tracking Table - Updated for new SCED Calculation

For RFC 2, the total additional effort has dropped from 67,889 to 3,568 hours. Note, however, that changing the SCED calculation changed the original cost of the project from 460,808 hours to 531,183 hours and the cost of RFC 1 from 5,404 hours to 8,820 hours. The unbid 70,375 hours from the original proposal and 3,416 hours from RFC 1 are unrecoverable (unless the full 68K hours are turned in as an impact to this small RFC, which would be a very hard sell to either management or customer). Are the additional hours believable? It was decided that a large step function was probably not reflective of reality; a smooth curve was more realistic. Hence, lesson learned 3 was updated to be:

Lesson Learned 3 - UpdatedA smooth SCED curve is necessary when using COCOMO II in-process to estimate costs of changes, and is more reflective of reality even when the

model is not used in-process.

Copyright 2004, Raytheon Company Page 12 of 18 ALL RIGHTS RESERVED

Page 13: COCOMO 2004 Conference

The new calculation of SCED is rolled into the site-wide implementation of the model so that other projects will receive the additional hours at the time of the original bid.

So now we have a process with the following characteristics:

1) Each project is bid with a productivity assuming no further scope growth2) The project is bid with a SCED calculation which produces a smooth productivity

curve rather than a step-function3) Productivity impacts as a result of scope growth and schedule compression are

assessed at the time of the scope increase4) A structured method of determining where/when to roll productivity impacts into

EVMS is in place (though this is still a somewhat subjective process).

But does it really work?

4 Real Life ResultsThe process described in this paper was used on a real project. The project was about 600K ESLOC and took over 3 years for 80+ software engineers to develop (at its peak). The project is still in progress, but after completing 90% of the work and the first (and largest) of three deliveries to the customer, the development portion of the project had realized a productivity within 7% of the model projection. The total project (development, integration, defect workoff) is anticipated to be within 5%. This compares to 12% had the updates to the project been bid not using the model in-process, but instead bid using the productivity of the original proposal. The large difference is mostly due to the very high schedule compression on this project. This level of accuracy is outstanding. In addition, the customer was very satisfied both with the performance of the project and the end product. The following are quotes from the customer office program manager (<> indicates words changed to protect program and agency identification):

“The team was given as little funding as possible and a mandate to develop requirements, design, test, manufacture, and deliver an extremely complex system in <about 3 years>. So complex and with so little time that in fact, the independent auditors assessment was <less than 15% chance> of success. And here we are – watching real operational data flow through the <> system as planned. Not only did the team exceed, the team shattered expectations!”

“The program that carried <the agency> to a iCMM (software maturity) level 3 award.”

“The highest scored program reviewed … in the entire intelligence community as an Information Technology investment!”

Copyright 2004, Raytheon Company Page 13 of 18 ALL RIGHTS RESERVED

Page 14: COCOMO 2004 Conference

From the director of customer agency “<PROJECT> is the model project for <this agency>. It’s on schedule, within budget, has a great team supporting it – it’s the way we’d like all our programs to work!”

The process does work; it produces excellent results from both a numbers perspective and a customer satisfaction perspective, but it’s also not ideal. This project was very fortunate to have a customer who believed in modeling and was both willing to spend the time to understand what was being done and why, and who was willing to accept the results. The process is a difficult one in that detailed analysis is required in order to determine where and when to allocate the productivity hours. In addition, no matter how much analysis is done, it is still very subjective. Finally, the issue of messing up the current month EVMS CPI measures when adding hours into the current month to account for productivity impacts incurred in the past was not solved.

5 AlternativesThere are some possible solutions to these issues. Three alternatives are discussed. The first two involve changing the process at the time of the initial bid. The third involves changing the way hours are laid into EVMS. All alternatives facilitate the in-process use of the model, but cause other potential issues. For completeness, the original method and its issues are repeated in this section as Alternative 0.

5.1 Alternative 0 – Standard UseThe model is calibrated using end of project data. The new project is characterized and sized against current known requirements. Any change delta between the start productivities and end productivity (ie. extra effort) due to increases in size is not included in the baseline. This extra cost is viewed a direct result of the size increase and is included as cost associated with the RFC that drives the scope change. These extra hours are rolled into EVMS as they are assessed during the life of the project. . This raises several issues:

1) Determining how much effort to put where is difficult.2) Per EVMS rules, budget may not be put into the past, so budget is added into

current month in correct WBS, which messes up current month numbers.3) Looks like “getting well” – which is a big no-no.

These can be mitigated as described previously, but not eliminated. In addition, the management effort required to mitigate these issues is significant.

5.2 Alternative 1 - Estimate Project GrowthThe amount of growth on project can be estimated and the project modeled including this amount of growth. There are two ways to accomplish this:

1. Explicitly add the expected amount of growth to the model as SLOC; use the resulting productivities to bid the project without the added SLOC

2. Use the REVL parameter to account for expected growth throughout the life of the program.

Copyright 2004, Raytheon Company Page 14 of 18 ALL RIGHTS RESERVED

Page 15: COCOMO 2004 Conference

Both of these methods do essentially the same thing – they effectively increase the size of the project, and then use the productivities from this larger project to bid the project without the growth included. Let’s see what happens if this is done using our same case study. Figure 5-12 shows what happens for each of the first three scope changes on the case study process for both the method used on the program (as described in section 3) and for Alternative 1.

Change Description ESLOC AddedHours Due to

Scope ChangeProductivity

Impacts (hrs)Total Cost of Change (Hrs)

Hours Due to Scope Change

Productivity Impacts (hrs)

Total Cost of Change (Hrs)

Starting Project Cost (Hours) 475 531,183 N/A 531,183 616,012 0 616,012

RFC 1 5 5,625 3,195 8,820 6,484 0 6,484

ICR 1 1 1,126 651 1,777 1,297 0 1,297

RFC 2 2 2,258 1,310 3,568 2,594 0 2,594

All remaining changes 117 151,734 81,039 232,773 151,734 0 151,734

Total 600 691,926 86,195 778,121 778,121 0 778,121

Alternative 0 Alternative 1

Figure 5-12 Project Changes - Alternative 0 versus Alternative 1

As you can see, since the project is bid with the ending productivity, each scope change throughout the life of the program can be bid with the same productivity as the project was originally bid. This solves all problems associated with the changing productivity. All of the productivity impacts are being bid up-front, thus there is no effort incurred for scope increases which occur during the life of the program outside of those driven by the scope change itself. Since there are no productivity impacts rolled in during the program execution, there aren’t problems rolling additional hours into EVMS.There are other potential issues.

1. Note the difference in the cost of the original proposal. Alternative 1 yields 84,829 more hours than Alternative 0. This is significant. It could potentially be so high that the business may be lost in a competitive situation.

2. There is an issue of what to do when the assumed amount of growth is reached, or what to do if it’s not reached. Once the project reaches the assumed amount of growth, the productivity estimated at the proposal time and used throughout this point is no longer valid. Are further productivity impacts simply estimation error whose cost is absorbed by the contractor, or is an update needed that allows these costs to be bid? What if the project doesn’t realize the anticipated amount of scope increase? Are the extra hours that were bid at the proposal profit to the contractor, or should the customer receive a refund? Further work is needed to answer these questions.

3. The productivity used in proposals is not reflective of actual productivity. This issue is both a public relations problem and a cost issue. The productivity used in the proposal may be appropriate for the purpose of generating costs based on the way the model is calibrated, but it is lower than the past actual achieved productivities on a project this size. In a competitive bid situation the company won’t look as productive as competitors, which could result in loss of business.

If the PR issue can be addressed and there isn’t price to win issue with bidding the lower productivity, this alternative is the recommended method of using the model in-process along with EVMS.

Copyright 2004, Raytheon Company Page 15 of 18 ALL RIGHTS RESERVED

Page 16: COCOMO 2004 Conference

5.3 Alternative 2 - Calibration Process ChangeThis alternative is very similar to the Alternative 1. If the amount of growth across programs comprising the calibration set is consistent, model can be calibrated with start of program sizes. The total effort to complete the projects is used, but the size of the project at the start is used. This has the effect of generating a lower productivity at the start which can be used throughout the life of the program (similar to approach described in section 5.2). The benefit of this over Alternative 1 is that it can be used even if hard data indicating the amount of growth is not available, but it is generally agreed that the historical programs and the program being bid are similar with respect to the amount of growth incurred. The downside of this over Alternative 1 is that it is more subjective in that the scope growth is implicitly included without full understanding of what it is. Thus, the modeler does not have control over how much growth is included in the project size in this Alternative, while Alternative 1 does provide this flexibility. This Alternative raises all of the same issues as Alternative 1, with the addition that due to the subjectivity involved in understanding the amount of scope growth, it may not be clear when or if issue number 2 even occurs.

5.4 Alternative 3 - For Non-Compressed Projects Only – Use Original Productivities to Bid All Changes

If the project is not compressed and is not likely to become very compressed, using the original productivities throughout the life of the project may be okay. The same project changes as described in the case study are applied to a project that experiences little schedule compression using Alternative 0 (see Figure 5-13). The less complicated Alternative 3 does generate slightly fewer hours than the more complicated method used, but the 3.8% difference is probably within the margin error (see Figure 5-14). This small difference probably doesn’t justify the use of the more complicated and controversial other alternatives.

Description Existing New Starting After Change

Due to Scope

ChangeProductivity

Impacts Starting After ChangeOriginal Estimate N/A 475 N/A 1.03 460,808 N/A N/A 107.9%RFC 1 475.0 5.0 1.03 1.03 4,856 548 107.9% 107.5%ICR 1 480.0 1.0 1.03 1.03 972 110 107.5% 107.5%RFC 2 481.0 2.0 1.03 1.03 1,944 220 107.5% 107.3%Rest 483.0 117.0 1.03 0.99 117,967 17,536 107.3% 99.3%

KESLOC Productivity Additional Hours Schedule Compression

Figure 5-13 Project Tracking Table - No Schedule Compression

Copyright 2004, Raytheon Company Page 16 of 18 ALL RIGHTS RESERVED

Page 17: COCOMO 2004 Conference

Change Description ESLOC AddedHours Due to

Scope ChangeProductivity

Impacts (hrs)Total Cost of Change (Hrs)

Hours Due to Scope Change

Productivity Impacts (hrs)

Total Cost of Change (Hrs)

Starting Project Cost (Hours) 475 460,808 N/A 460,808 460,808 0 460,808

RFC 1 5 4,856 548 5,404 4,851 0 4,851

ICR 1 1 972 110 1,082 970 0 970

RFC 2 2 1,944 220 2,164 1,940 0 1,940

All remaining changes 117 117,967 17,536 135,503 113,504 0 113,504

Total 600 586,547 18,414 604,961 582,073 0 582,073Alternative 3 is 3.8% less

than the more complicated method used.

Alternative 0 Alternative 3

Figure 5-14 Project Changes – Alternative 0 Versus Alternative 3

6 RecommendationThe results indicate that the in-process use of the model to generate costs does generate significantly better estimates for compressed projects. If the PR issue can be addressed and there isn’t price to win issue with bidding the lower productivity, Alternative 1 is the recommended method of using the model in-process along with EVMS. If not, Alternative 0 is recommended, but extensive conversations with the customer and with management must occur prior to the start of the project to ensure full understanding of how the costing process will work. If these discussions do not occur, it’s likely that the process will not be accepted.If the project is not compressed and is not likely to become compressed, Alternative 3 is recommended.

Copyright 2004, Raytheon Company Page 17 of 18 ALL RIGHTS RESERVED

Page 18: COCOMO 2004 Conference

7 Bibliography

1. Boehm, Barry, Software Engineering Economics, Prentice Hall, Inc., Englewoods Cliff, NJ, 1981

2. Boehm, Barry, Software Cost Estimation with COCOMO II, Prentice Hall, Inc., Upper Saddle River, NJ, 2000

Copyright 2004, Raytheon Company Page 18 of 18 ALL RIGHTS RESERVED