INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
COSYSMOCOnstructive SYStems Engineering Cost MOdel
Tampa, FL
February 3, 2003Dr. Barry BoehmRicardo Valerdi
University of Southern CaliforniaCenter for Software Engineering (CSE)
INCOSE IW Workshop
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Outline• Workshop Agenda• Background on CSE, COSYSMO, and COCOMO II• COSYSMO Overview
– Operational concept and scope– Prototype demo
• Model Progress to Date– Front end sizing and drivers – Full life cycle sizing and drivers
• New proposed drivers• Action items
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Workshop Agenda• Proposed drivers
– Process maturity– # of recursive levels of design– Technology risk (different ratings on the
viewpoints)– Length of life cycle– Quality Attributes (ie., documentation)– Manufacturability/Producibility– Degree of Distribution
• Discontinuity of members• ISO/IEC 15288 expertise in the working group• Preparation for Delphi Round 2• Preliminary data collection
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
USC Center for Software Engineering (CSE)• Researches, teaches, and practices CMMI-based
Software engineering – Systems and software engineering fully integrated
• Focuses on better models to guide integrated systems and software engineering – Success models: stakeholder win-win, business
cases – Product models: requirements, architectures, COTS– Process models: spiral extensions, value-based RUP
extensions – Property models: cost, schedule, quality
• Applies and extends research on major programs (DARPA/Army, FCS, FAA ERAM, NASA Missions)
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
• Commercial Industry (15)
– Daimler Chrysler, Freshwater Partners, Galorath, Group Systems.Com, Hughes, IBM, Cost Xpert Group, Microsoft, Motorola, Price Systems, Rational, Reuters Consulting, Sun, Telcordia, Xerox
• Aerospace Industry (6)
– Boeing, Lockheed Martin, Northrop Grumman, Raytheon, SAIC, TRW
• Government (8)
– DARPA, DISA, FAA, NASA-Ames, NSF, OSD/ARA/SIS, US Army Research Labs, US Army TACOM
• FFRDC’s and Consortia (4)
– Aerospace, JPL, SEI, SPC
• International (1)
– Chung-Ang U. (Korea)
USC-CSE Affiliates (34)
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
• Commercial Industry (1)– Galorath
• Aerospace Industry (5)– Lockheed Martin, Northrop Grumman*,
Raytheon, SAIC, TRW• Government (1)
– US Army Research Labs• FFRDC’s and Consortia (2)
– Aerospace, SPC
INCOSE Companies actively involved with COSYSMO
*Most recentaddition
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Key Members of the COSYSMO Working Group
Karen Owens, Marilee WheatonEvin StumpGarry Roedler, Gary HafenGary Thomas, John RieffTony Jordano, Don GreenleeChris MillerCheryl JonesBarry Boehm, Elliot Axelband,
Don Reifer, Ricardo Valerdi
Aerospace Corp.Galorath
LMCORaytheon
SAICSPC
US Army/PSSMUSC
underline = will participate in Monday’s Workshop
Italics = SE experience
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
USC-CSE Cost, Schedule, and Quality Models
• Build on experience with COCOMO 1981, COCOMO II – Most widely used software cost models worldwide– Developed with Affiliate funding, expertise, data support
• Collaborative efforts between Computer Science (CS) and Industrial Systems Engineering (ISE) Depts.– 3 CS PhD’s, 2 ISE PhD’s to date– Valerdi an ISE PhD student– Boehm joint appointment in CS, ISE
• COCOMO Suite of models– Cost, schedule: COCOMO II, CORADMO, COCOTS– Quality: COQUALMO– Systems Engineering: COSYSMO
• Uses mature 7-step model development methodology
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
7-step Modeling MethodologyAnalyze Existingliterature
1
2
3
4
5
6
7
PerformBehavioral Analysis
Identify RelativeSignificance
Perform Expert-Judgement, DelphiAssessment
Gather Project Data
Determine BayesianA-Posteriori Update
Gather more data;refine model
A-PRIORI MODEL +
SAMPLING DATA =
A-POSTERIORI MODEL
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Parametric Cost Model Critical PathUsual # Months*
6 Converge on cost drivers, WBS
6 Converge on detailed definitions and rating scales
12 Obtain initial exploratory dataset (5-10 projects)
6 Refine model based on data collection & analysis experience
12+ Obtain IOC calibration dataset (30 projects)
9 Refine IOC model and tool
Critical Path Task
*Can be shortened and selectively overlapped
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
COSYSMO: Overview• Parametric model to estimate system
engineering costs• Covers full system engineering lifecycle• Focused on use for Investment Analysis,
Concept Definition phases estimation and tradeoff analyses– Input parameters can be determined in
early phases
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
COSYSMO
SizeDrivers
EffortMultipliers
Effort
Duration
Calibration
# Requirements# Interfaces# Scenarios# Algorithms
+Volatility Factor
- Application factors-5 factors
- Team factors-7 factors
- Schedule driverWBS guided by EIA/ANSI 632 & ISO/IEC 15288
COSYSMO Operational Concept
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Previous COSYSMO Evolution Path
Inception Elaboration Construction TransitionOper Test & Eval
1. COSYSMO-IP
2. COSYSMO-C4ISR
3. COSYSMO-Machine
4. COSYSMO-SoS
IP (Sub)system
C4ISR System
Physical MachineSystem
System of Systems (SoS)
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Revised View of COSYSMO Evolution Path(Results from last week’s meeting)
Oper Test & Eval
1. COSYSMO-IP
2. COSYSMO-C4ISR
3. COSYSMO-Machine
4. COSYSMO-SoS
Global Command and Control System
Satellite Ground Station
Joint Strike Fighter
Future Combat Systems
Initiate data collection for all and let the amount of data received determine what is included.
Include ISO/IEC 15288 Stages
DevelopConceptualizeTransition to Operation
Operate, Maintain, or Enhance
Replace or Dismantle
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
EIA/ANSI 632
EIA/ANSI 632 - Provide an integrated set of fundamental processes to aid a developer in the engineering or re-engineering of a system
Breadth and Depth of Key SE StandardsSystem life
ISO/IEC 15288
Level of
deta
il
Conceptualize DevelopTransition to
Operation
Operate,Maintain,
or EnhanceReplace
or Dismantle
Processdescription
High levelpractices
Detailedpractices
ISO/IEC 15288 - Establish a common framework for describing the life cycle of systems
Purpose of the Standards:Purpose of the Standards:
IEEE 1
220
IEEE 1220 - Provide a standard for managing systems engineering
Input to 632/1220
Source : Draft Report ISO Study Group May 2, 2000
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
ISO/IEC 15288 Key Terms• System
– a combination of interacting elements organized to achieve one or more stated purposes
• System-of-Interest– the system whose life cycle is under consideration in the context of this
International Standard
• System Element– a member of a set of elements that constitutes a system
– NOTE: A system element is a discrete part of a system that can be implemented to fulfill specified requirements
• Enabling System– a system that complements a system-of-interest during its life cycle
stages but does not necessarily contribute directly to its function during operation
– NOTE: For example, when a system-of-interest enters the production stage, an enabling production system is required
Source: ISO/IEC 15288.Source: ISO/IEC 15288.
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Systemelement
System-of-interest
Systemelement
Systemelement
SystemelementSystem
Systemelement
Systemelement
Systemelement
System
Systemelement
Systemelement
Systemelement
System
Systemelement
Systemelement
SystemelementSystem
Systemelement
Systemelement
System
Systemelement
Systemelement
System
Systemelement
Systemelement System
Systemelement
Systemelement
Systemelement
ISO/IEC 15288 System of Interest Structure
Make orbuy
Source: ISO/IEC 15288.Source: ISO/IEC 15288.
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Outline• Workshop Agenda• Background on CSE, COSYSMO, and COCOMO II• COSYSMO Overview
– Operational concept and scope– Prototype demo
• Model Progress to Date– Front end sizing and drivers – Full life cycle sizing and drivers
• New proposed drivers• Action items
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Recent developments• Developed a project plan• Reduced drivers from 24 to 18• Expanded the model to cover the later phases
of the life cycle• Replaced EIA 632 with ISO 15288• Introduced new drivers to reflect full life cycle
scope • Changed name from COSYSMO-IP (Information
Processing) to COSYSMO• Revised the evolution path to allow the data to
drive the scope of the model
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
4 Size Drivers1. Number of System Requirements
2. Number of Major Interfaces
3. Number of Operational Scenarios
4. Number of Unique Algorithms
• Each weighted by complexity, volatility, and degree of reuse
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Number of System RequirementsThis driver represents the number of requirements that are typically taken from the system or marketing specification. A requirement is a statement of capability containing a normative verb such as “shall” or “will.” It may be functional, performance, feature, or service-oriented in nature depending on the methodology used for specification. System requirements can typically be quantified by counting the number of applicable “shall’s” or “will’s” in the system or marketing specification.
Easy Nominal Difficult
No. of System Requirements
- Well specified - Loosely specified - Poorly specified
- Traceable to source - Can be traced to source with some effort
- Hard to trace to source
- Simple to understand - Takes some effort to understand - Hard to understand
- Little requirements overlap - Some overlap - High degree of requirements overlap
- Familiar - Generally familiar - Unfamiliar
- Good understanding of what’s needed to satisfy and verify requirements
- General understanding of what’s needed to satisfy and verify requirements
- Poor understanding of what’s needed to satisfy and verify requirements
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Number of Major InterfacesThis driver represents the number of shared major physical and logical boundaries between system components or functions (internal interfaces) and those external to the system (external interfaces). These interfaces typically can be quantified by counting the number of interfaces identified in either the system’s context diagram and/or by counting the significant interfaces in all applicable Interface Control Documents.
Easy Nominal Difficult
No. of Major Interfaces
- Well defined - Loosely defined - Ill defined
- Uncoupled - Loosely coupled - Highly coupled
- Cohesive - Moderate cohesion - Low cohesion
- Well behaved - Predictable behavior - Poorly behaved
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Number of Operational ScenariosThis driver represents the number of operational scenarios that a system must satisfy. Such threads typically result in end-to-end test scenarios that are developed to validate the system satisfies all of its requirements. The number of scenarios can typically be quantified by counting the number of end-to-end tests used to validate the system functionality and performance. They can also be calculated by counting the number of high-level use cases developed as part of the operational architecture.
Easy Nominal Difficult
No. of Operational Scenarios
- Well defined - Loosely defined - Ill defined
- Few end-to-end scenarios (< 10)
- Modest no. of end-to-end scenarios (10 < OS < 30)
- Many end-to-end scenarios (> 30)
- Timelines not an issue - Timelines a constraint - Tight timelines through scenario network
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Number of Unique AlgorithmsThis driver represents the number of newly defined or significantly altered functions that require unique mathematical algorithms to be derived in order to achieve the system performance requirements. As an example, this could include a complex aircraft tracking algorithm like a Kalman Filter being derived using existing experience as the basis for the all aspect search function. Another example could be a brand new discrimination algorithm being derived to identify friend or foe function in space-based applications. The number can be quantified by counting the number of unique algorithms needed to support each of the mathematical functions specified in the system specification or mode description document (for sensor-based systems).
Easy Nominal Difficult
No. of Unique Algorithms
- Existing algorithms - Some new algorithms - Many new algorithms
- Basic math - Algebraic by nature - Difficult math (calculus)
- Straightforward structure - Nested structure with decision logic - Recursive in structure with distributed control
- Simple data - Relational data - Persistent data
- Timing not an issue - Timing a constraint - Dynamic, with timing issues
- Library-based solution - Some modeling involved - Simulation and modeling involved
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
12 Cost Drivers
1. Requirements understanding
2. Architecture complexity
3. Level of service requirements
4. Migration complexity
5. Technology Risk
Application Factors (5)
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Requirements understanding This cost driver rates the level of understanding of the system requirements by all stakeholders including the systems, software, hardware, customers, team members, users, etc…
Very low Low Nominal High Very High
Poor, unprecedented system
Minimal, many undefined areas
Reasonable, some undefined areas
Strong, few undefined areas
Full understanding of requirements, familiar system
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Architecture complexity This cost driver rates the relative difficulty of determining and managing the system architecture in terms of platforms, standards, components (COTS/GOTS/NDI/new), connectors (protocols), and constraints. This includes tasks like systems analysis, tradeoff analysis, modeling, simulation, case studies, etc…
Very low Low Nominal High Very High
Poor understanding of architecture and COTS, unprecedented system
Minimal understanding of architecture and COTS, many undefined areas
Reasonable understanding of architecture and COTS, some weak areas
Strong understanding of architecture and COTS, few undefined areas
Full understanding of architecture, familiar system and COTS
2 level WBS 3-4 level WBS 5-6 level WBS >6 level WBS
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Level of service requirementsThis cost driver rates the difficulty and criticality of satisfying the Key Performance Parameters (KPP). For example: security, safety, response time, the “illities”, etc…
Very low Low Nominal High Very High
Difficulty Simple requirements
Low difficulty Moderately complex
Difficult requirements
Really complex
Criticality Slight inconvenience
Easily recoverable losses
Some loss High financial loss
Risk to human life
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Migration complexity This cost driver rates the complexity of migrating the system from previous system components, databases, workflows, etc, due to new technology introductions, planned upgrades, increased performance, business process reengineering etc…
Very low Low Nominal High Very High
Introduction of requirements is transparent
Difficult to upgrade Very difficult to upgrade
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Technology RiskThe maturity, readiness, and obsolescence of the technology(ies) being implemented. This may include
Viewpoint Rating
VL L N H VH
Technology Maturity Level
Technology proven and widely used throughout industry
Proven through actual use and ready for widespread adoption
Proven on pilot projects and ready to roll-out for production jobs
Ready for pilot use Still in the laboratory
Technology Readiness Level
Mission proven (TRL 9)
Concept qualified (TRL 8) Concept has been demonstrated (TRL 7)
Proof of concept validated (TRL 5 & 6)
Concept defined (TRL 3 & 4)
Technology Obsolescence Level
- Technology is the state-of-the-practice- Emerging technology could compete in future
- Technology is stale- New and better technology is on the horizon in the near-term
- Technology is outdated and use should be avoided in new systems- Spare parts supply is scarce
We need to supply guidelines on how to compose the readiness and obsolescence into a single factor.
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
12 Cost Drivers (cont.)
1. Stakeholder team cohesion
2. Personnel capability
3. Personnel experience/continuity
4. Process maturity
5. Multisite coordination
6. Formality of deliverables
7. Tool support
Team Factors (7)
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Stakeholder team cohesion Represents a multi-attribute parameter which includes leadership, shared vision, diversity of stakeholders, approval cycles, group dynamics, IPT framework, team dynamics, trust, and amount of change in responsibilities. It further represents the heterogeneity in stakeholder community of the end users, customers, implementers, and development team.
VL L N H VH
Highly heterogeneous stakeholder communities with diverse objectivesMultiple stakeholders with diverse expertise, task nature, language, culture, infrastructureSystem involving major changes in stakeholder roles & responsibilities
Heterogeneous stakeholder community with converging organizational objectivesSystem involves some changes in stakeholder roles & responsibilities
Common shared organizational objectivesFunctional team
Strong team cohesionHigh stakeholder trust levelClear roles & responsibilities
Virtually homogeneous stakeholder communitiesInstitutionalized, shared project cultureStrong team dynamics
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Personnel capability Systems Engineering’s ability to perform in their duties and the quality of human capital.
Very Low Low Nominal High Very High
15th percentile 35th percentile 55th percentile 75th percentile 90th percentile
Personnel experience/continuity The applicability and consistency of the staff over the life of the project with respect to the customer, user, technology, domain, etc…
Very low Low Nominal High Very High
Less than 2 months 1 year continuous experience, other technical experience in similar job
3 years of continuous experience
5 years of continuous experience
10 years of continuous experience
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Process maturity Maturity per EIA/IS 731, SE CMM or CMMI.
Very low Low Nominal High Very High Extra High
Level 1 (lower half)
Level 1 (upper half)
Level 2 Level 3 Level 4 Level 5
Multisite coordination Location of stakeholders, team members, resources (travel).
Very low Low Nominal High Very High Extra High
SITE: Collocation
International Multi-city and multi-national
Multi-city or multi-company
Same city or metro area
Same building or complex
Fully located
SITE: Communications
Some phone, mail
Individual phone, FAX
Narrowband e-mail
Wideband electronic communication
Wideband electronic communication, occasional video conference
Interactive multimedia
Add Collaboration barriers row (time zone, security, export restrictions, language, culture, competition, Intellectual property, contractual)
Revisit, should include more detailUnder process levels, process improvementcommitment
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Formality of deliverables The breadth and depth of documentation required to be formally delivered.
Very low Low Nominal High Very High
General goals Some conformity Some relaxation Partially streamlined process, occasional relaxation
Rigorous, follows customer requirements
Tool support Use of tools in the System Engineering environment.
Very low Low Nominal High Very High
No SE tools Simple SE tools, little integration
Basic SE tools integrated (throughout the systems process)
Strong, mature SE tools, integrated (integrated with other disciplines)
Strong, mature proactive SE tools integrated with process (Model-based SE and management systems)
Right size of documentation to the life cycle needsThe
Right size
Breadth depth
Reviews?
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Outline• Workshop Agenda• Background on CSE, COSYSMO, and COCOMO II• COSYSMO Overview
– Operational concept and scope– Prototype demo
• Model Progress to Date– Front end sizing and drivers – Full life cycle sizing and drivers
• New proposed drivers• Action items
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
# and diversity of installations/platformsThe number of different platforms that the system will be hosted and installed on. The complexity in the operating environment (space, sea, land, fixed, mobile, portable, information assurance/security). For example, in a wireless network it could be the number of unique installation sites and the number of types of fixed clients, mobile clients, and servers.
Viewpoint N H VH
Sites/installations Few # of installations or many similar installations
Moderate # of installations or some amount of multiple types of installations
Numerous # of installations with many unique aspects
Operating environment Not a driving factor Moderate environmental constraints
Multiple complexities/constraints caused by operating environment
No. of Different Platforms
Few types of platforms (< 5) Modest types of no. of platforms (5 < P <10)
Many types of platforms (> 10)
Homogeneous Mixed Heterogeneous
Typically networked using a single protocol
Typically networked using several consistent protocols
Typically networked using different protocols
Include phase out
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
# of recursive levels in the designNew cost driver. Can capture the number of boundaries in the SOI (System of Interest). Number of recursive levels that require SE. For example, a system with a single level of design may have lower numbers of recursive levels in the design. On the other hand, a system with a system integrator (system of systems level), a prime (system level), a subcontractor (segment level), second-tier subcontractor (subsystem level), and third-tier contractor (component/box level) would have 5 levels of recursive design. Includes internal design/subsystem levels. Note: this driver depends on what level your enabling system lies.Revision: Has system-wide multiplicative effect vs. incremental additive effect. Better handled in two cost drivers: Architecture Complexity (technical aspects) & Multisite Coordination (management aspects).
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
# and diversity of installations/platforms
New size driver. Can capture the number of varied and multiple installations and/or platforms. This is in light of the discussion to cover the full system life cycle. This driver is especially important in later implementation/deployment stages.Revision: Its effect on effort is additive but the amount of added effort per type of installation/platform is a multiplier of the baseline effort. Probably best covered by a formula like:(# of diverse installation/platform types) * (Avg extra % of systems engineering effort per type)
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
The number of different processing platforms that the system will be hosted and installed on. For example, in a network it could be the number of unique installation sites and the number of workstations and servers.
Viewpoint N H VH
Sites/installations Few # of installations or many similar installations
Moderate # of installations or
some amount of multiple types of
installations
Numerous # of installations with
many unique aspects
Operating environment
Not a driving factor
Moderate environmental
constraints
Multiple complexities/const
raints caused by operating
environment
# of Different Platforms
Few platforms (< 5)
Modest no. of platforms
(5 < P <10)
Many platforms (> 10)
Homogeneous Mixed Heterogeneous
Typically networked using a single protocol
Typically networked using several consistent
protocols
Typically networked using
different protocols
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
# of years in operational life cycle
Revision: Another additive factor where the added effort is a multiplier of the baseline effort. Probably best covered by a formula like:(# of years in operational life cycle) * (Avg annual % of continuing systems engineering effort)
• Evolutionary acquisition• # of independently evolving interoperating systems• Upgrade to prior models
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
# and diversity of installations/platforms phased out
Revision: Its effect on effort is additive but
the amount of added effort per
installation/platform being phased out is a
multiplier of the baseline effort. Probably
best covered by a formula like:
(# of diverse installation/platform types) * (Avg extra % of systems engineering effort per type being phased out)
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Rationale: Reliability, availability and maintainability requirements for certain classes of systems often drive their costs. This is true for military as well as commercial systems. A cost driver aimed at assessing the impact of varying these requirements is therefore needed.Quality Attributes: Represents a measure of the extent to which the system must perform its intended requirements (reliability), be available (availability) and facilitate change, modification and repair (maintainability) over time.
Viewpoint Rating
VL L N H VH
Reliability Errors led to slight inconvenience
Errors led to easily recoverable loss
Errors led to moderate recoverable loss
Errors led to high financial loss
Safety critical
Availability < 70% availability
Between 70% and 90%
availability
> 90% availability
24/7 with schedulable
downtime (for PM)
24/7 with negligible downtime
Maintainability
Difficult to change, modify and/or repair
Can change, modify and/or repair with effort
Easily changed, modified and repaired
Reconfigurable Self healing
Quality Attributes
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Rationale: The ability to manufacture/produce the system can drive the costs especially when advances in manufacturing technology are needed to field the system.Manufacturability/Producibility: Represents the ability of contractors to manufacture/produce the system in specific quantity to meet the market demand.
Viewpoint Rating
VL L N H VH
Manufacturability
New technology needed to manufacture system
Facilities needed to manufacture systems
Technology/facilities in place to manufacture system
Can meet 100% of peak manufacturing demand
Have excess manufacturing capability
Producibility Can meet less than 70% of
peak demand
Can meet between 70 and 90% of
peak demand
Can meet 90% of peak demand
Can meet 100% of peak demand
Have excess
production capability
Manufacturability/Producibility
DFMADesign for “x”Design trades
Revisit
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Rationale: The degree of partitioning and distribution of the system impacts its cost. This cost driver how interoperable the system must be.Degree of Distribution: Characterizes how distributed the architecture is.
Viewpoint Rating
VL L N H VH
Distribution Self-contained with limited distribution
Distribution via LANs/direct links
Distribution via LANs/WANs
Network of networks
System of systems
Interoperability Self-contained with no
interoperability
Interoperates with fewer
than 3 systems
Interoperates with between 3 to 5 systems
Interoperates with 5 to 10
systems
Interoperates with more than 10 systems
Degree of Distribution
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Levels & complexity of V&V
RevisitMight be under # of Requirements (under VH)
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
632 - 20(Implement-
ation)632 - 21
(Transition)632 - 31
(Verification)632 - 32
(Readiness)632 - 33b
(Validation)15288
(Operations)
15288 (Maintenance or Support)
15288 (Retirement)
# System Requirements x x x x x# Major Interfaces x x x x x# Unique Algorithms x x x x# Operational Scenarios x x x x x x# Recursive Levels in the Design x x x x # Systems being Phased Out (Covered by Migration Complexity?) x x x# Operators / Maintainers x x# Training Courses x# Installations
# System Elements
# Length of Lifecycle
Size Drivers vs. EIA/ANSI 632 & ISO/IEC 15288 Stages Late in the Life Cycle
LegendBold = existing driverItalics = proposed addition
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Application factors
632 - 20(Implement-
ation)632 - 21
(Transition)632 - 31
(Verification)632 - 32
(Readiness)632 - 33b
(Validation)15288
(Operations)
15288 (Maintenance or Support)
15288 (Retirement)
–Requirements understanding x x x x x–Architecture complexity x x x x x x x–Level of service requirements, criticality, difficulty x x x x x–Migration complexity x x–Technology risk (maturity & obsolescence) x x- Operational Complexity x
Cost Drivers vs. EIA/ANSI 632 & ISO/IEC 15288 Stages Late in the Life Cycle
LegendBold = existing driverItalics = proposed addition
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Team factors
632 - 20(Implement-
ation)632 - 21
(Transition)632 - 31
(Verification)632 - 32
(Readiness)632 - 33b
(Validation)15288
(Operations)
15288 (Maintenance or Support)
15288 (Retirement)
–Stakeholder team cohesion x x–Personnel capability x x x x x x x x
–Personnel experience/continuity x x x x x x x x–Process maturity x x x x x–Multi-site coordination x x x x x x x x–Formality of deliverables x x x x x x–Tool support x x
Cost Drivers vs. EIA/ANSI 632 & ISO/IEC 15288 Stages Late in the Life Cycle
LegendBold = existing driverItalics = proposed addition
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Action Items from Last WG Meeting1. Develop a project Plan2. Address technology maturity/obsolescence3. Refine driver definitions to incorporate
ISO/IEC 15288 definitions4. Incorporate System and People idea5. Refine drivers applicability matrix6. Develop data collection strategy7. Generate Data Collection Form8. Update Stakeholder Cohesion to include
diversity, identification and trust
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Calendar of Activities: 2003
2003 2004
INCOSE 2003
USC CSE Annual Research Review
INCOSEFall Workshop
COCOMO Forum
Conference on Systems Integration
J F M A M J J A S O N D
Paper & tutorial
submitted
Practical Software & Systems Measurement Workshop
Paper submitted
ISPA
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Points of ContactDr. Barry Boehm[[email protected]]
Dr. Elliot Axelband[[email protected]]
Don Reifer[[email protected]]
Ricardo Valerdi[[email protected]]
Websitehttp://sunset.usc.edu
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Backup Charts
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
USC-CSE Research ($10M backlog)
• DARPA/Army: Model applications and extensions for Future Combat Systems
• DARPA: Architectures for mobile distributed systems (DASADA)
• FAA: Acquisition processes; COCOMO security extensions
• NASA: Empirical methods for High Dependability Computing
• NSF: Center for Empirically-Based Software Engineering (with U. of Maryland)
• NSF: Strategic Design (with CMU, Virginia, Washington)
• Industry Affiliates’ program
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
General Affiliate Benefits• Affiliates-only Web portal
– Early access to tools, methods, papers, talks, student resumes
• Tools: COCOMO Suite, Architecture tools, WinWin• Technical Report series• Workshops on Affiliate-prioritized topics• Annual Research Review and Steering Group
meeting• Annual one-day professor-visit• Bilateral visit arrangements; internships• Conferences and special workshops• Monthly LA SPIN meetings• Tutorials and eWorkshops
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Collaboration Modes and Special Benefits• Software architecting assistance
-Aerospace, Hughes, JPL, Northrop Grumman, TACOM, TRW, Xerox
• Software process/cost/quality/cycle time assistance-Aerospace, Litton, Microsoft, Northrop Grumman, Raytheon, SAIC, Sun, TACOM, TRW, Xerox
• Management reviews of critical projects-Litton, Motorola, SAIC, SEI, TRW
• Reviews of corporate research programs-Daimler Chrysler, Draper Labs, Lockheed Martin, SAIC, SEI, SPC, Telcordia, TRW
• Joint research contracts-Aerospace, Lockheed Martin, Northrop Grumman, SEI, SPC, TRW
• Aid in commercializing USC-CSE research -C-Bridge, Galorath, Group Systems.com, Marotz, Price Systems, Rational
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Collaboration Modes and Special Benefits - II
• Special Projects-Aerospace, Auto Club, FAA, Fidelity, IBM, JPL, Litton, Northrop Grumman, Telcordia
• Joint workshops on key topics-Aerospace, Motorola, Rational, DOD/SIS, SEI, SPC
• Focused working groups (COSYSMO) -Aerospace, Galorath, Lockheed Martin, Raytheon, SAIC, SPC, TRW
• Visiting collaborators-Aerospace, Chung-Ang, C-Bridge, IBM, JPL, Litton, Northrop Grumman, SEI, TRW
• Corporate State-of-the-art tutorials-Boeing, Chung-Ang, Daimler Chrysler, Draper, EDS, FAA, Fidelity, IBM, JPL, Litton, Lockheed Martin, Lucent, Motorola, Microsoft, Raytheon, SAIC, SEI, SPC, Sun, TRW, Xerox
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Mapping of Old to New COSYSMO-IP Drivers
Number of System Requirements Number of Major InterfacesNumber of Technical Performance
MeasuresNumber of Operational ScenariosNumber of Modes of OperationNumber of Different PlatformsNumber of Unique Algorithms
Old (7) New (4)
Number of System Requirements Number of Major Interfaces
Number of Operational Scenarios
Number of Unique Algorithms
Siz
e F
acto
rs
Level of Service Requirements
Architecture complexity
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Delphi Round 1 HighlightsRange of sensitivity for Size Drivers
# A
lgo
rith
ms
# R
eq
uir
emen
ts
# In
terf
aces
# T
PM
’s
# S
cen
ario
s
# M
od
es
# P
latf
orm
s
5.57
Relative
Effort
1
2.232.54
2.212.10
6.48
6
4
2
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Requirements understanding Architecture complexity Level of service requirementsLegacy Transition complexity COTS assessment complexity Platform difficulty Required business process
reengineering Technology Maturity Physical system/information subsystem tradeoff analysis complexity
Requirements understanding Architecture complexity Level of service requirementsMigration complexity
Technology Maturity
Ap
pli
cati
on
Co
st F
acto
rs
Mapping of Old to New COSYSMO-IP Drivers
# of TPMs # of Platforms
Old (9) New (5)
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Delphi Round 1 Highlights (cont.)
Range of sensitivity for Cost Drivers (Application Factors)
EMR
1.93
2.812.13
2.432.24
4
2
Req
uir
em
ents
un
d.
Arc
hit
ectu
re u
nd
.
Lev
el o
f se
rvic
e re
qs.
Leg
acy
tra
nsi
tio
n
CO
TS
Pla
tfo
rm d
iffi
cult
y
Bu
s. p
roce
ss r
een
g.
1.741.13
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Number and diversity of stakeholder communities Stakeholder team cohesion Personnel capability Personnel experience/continuity Process maturity Multisite coordination Formality of deliverables Tool support
Old (8) New (7)
Stakeholder team cohesion Personnel capability Personal experience/continuity Process maturity Multisite coordination Formality of deliverables Tool support
Reqs Und
Mapping of Old to New COSYSMO-IP Drivers
Tea
m C
ost
Fac
tors
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
Delphi Round 1 Highlights (cont.)Range of sensitivity for Cost Drivers (Team Factors)
1.28
2.461.91 2.161.94
1.25
To
ol
sup
po
rt
Sta
keh
old
er c
om
m.
Sta
keh
old
er c
oh
esio
n
Per
son
nel
cap
ab
ilit
y
Per
son
al
exp
erie
nce
Pro
cess
mat
uri
ty
Mu
ltis
ite
coo
rd.
Fo
rmal
ity
of
del
iv.
1.841.78
EMR4
2
INCOSE IWFebruary 2003
USC
C S E University of Southern CaliforniaCenter for Software Engineering
COSYSMO/COCOMO II MappingPrevious candidate starting point
Development EIA Stage Inception
Elaboration
Construction
Transition Management
14
12
10
14 Environment/CM
10
8
5
5 Requirements
38
18
8
4 Design
19
36
16
4 Implementation
8
13
34
19 Assessment
8
10
24
24 Deployment
3
3
3
30 TBD TBD TBD
= COCOMOII = COSYSMO-IP
When doingCOSYSMO-IP andCOCOMOII, Subtract grey areasprevent doublecounting.
TBD