Lane: What is an SoS USC-CSSE-2013-001 Page 1

What is a System of Systems and Why Should I Care?

Jo Ann Lane

Daniel J. Epstein Department of Industrial and Systems Engineering

University of Southern California

Email: jolane at usc.edu

Introduction Today’s need for more complex, more capable systems in a short timeframe is leading more

organizations towards the integration of new and existing systems with commercial-off-the-shelf (COTS)

products into network-centric, knowledge-based systems of systems (SoS). With this approach, system

development processes to define the new multi-system architecture, identify sources to either supply or

develop the required components, and eventually integrate and test these high level components are

evolving and are being referred to as SoS Engineering (SoSE).

The goal of this paper is to further describe the concept of an SoS, explain why it is important to conduct

engineering at the SoS level, briefly describe how SoSE tends to differ from single systems engineering

including some of the key challenges associated with SoSE, provide an example of how SoSE can

significantly improve SoS performance, and conclude with a discussion on some key SoS and SoSE

research areas.

Systems of Systems are Everywhere Many today agree that software-intensive systems are ubiquitous. We have software embedded in our

automobiles, household appliances, and even computers and sensors on our bicycles to tell us how far

we have gone and our average speed. It is also easy to see, once one understands the concept of SoS,

that SoSs can also be found everywhere.

SoSs within Our Homes: Just looking within our homes we can see home security systems that are linked

to the security companies as well as to our smartphones. In addition, we can turn our homes into

“smart homes” by integrating our home security systems, air conditioning and power systems, fire alarm

systems, and communications systems to automatically respond to problem situations.

Enterprise-wide SoS: Most business enterprises contain one or more SoSs. For example, most

businesses have integrated many of their back office systems such as employee systems, payroll

systems, and accounting systems. In addition, they may also have an integrated set of customer-facing

systems such as order-entry, pricing systems, billing, service monitoring, inventory management, and

customer help. These types of SoS tend to be relatively static in that the systems are typically always

connected and interoperating with each other to support the organization’s key business functions. An

example of a customer-facing SoS is a healthcare SoS that integrates many of the patient care systems,

as illustrated in Figure 1.

Lane: What is an SoS USC-CSSE-2013-001 Page 2

Laboratory System

Imaging Management

System Pharmacy System

Patient Management

System

Telemetry System

Health Care

Network

Figure 1: Example Heathcare SoS.

Other examples of this type can be found in Friedman’s The World is Flat [Friedman, 2005], where he

describes the Wal-Mart SoS used to manage their supply chain and the United Parcel Service (UPS) SoS

used to manage and track package deliveries.

Regional Area Crisis Response SoS (RACRS): Crisis response systems are varied and expensive. And not

all crises require the same system assets in order to manage or resolve the crisis. For example, a fast

moving fire requires a quick initial response by one or more fire departments to fight the fire, police to

keep bystanders out of the way of the firefighters and to maybe evacuate areas as the fire spreads, and

ambulances if there are fire-related injuries. If the fire is not easily contained by the first responders,

additional resources such as water tankers and unmanned aerial vehicles are deployed to help with the

fire fight and to better ascertain the directions in which the fire is moving. For very large fires,

additional fire departments may be asked to help as well as military fire-fighting personnel in the area.

If a lot of people need to be evacuated, aerial vehicles can be used to help determine the best roads for

evacuation. In addition, some large cities have instrumented their main roadways to monitor traffic and

these can be used either instead of the aerial vehicles or to augment the aerial vehicles. As the number

of agencies, both civilian and military, increase so too does the need for communications systems to

allow the responders to interoperate with each other—firefighters’ lives are at risk if they cannot

communicate with other firefighters in the area and have a relatively good situational awareness of

where the fire is and where it is going.

If the crisis is a hazardous material spill on a busy freeway, the first responders may again be a fire

department (since they are most easily deployed). But this time, they are supported by hazardous

materials specialists to help determine the type of material spilled and the appropriate methods for

removing the materials and decontaminating the area. In some cases, robots may be used to collect

spill samples and/or decontaminate the area. Also, it may also be necessary to evacuate people in the

area. So, again, the police, aerial vehicles, and other systems may be used to help with this activity.

Lane: What is an SoS USC-CSSE-2013-001 Page 3

Figure 2 illustrates some of the systems that may be called upon to respond to various crises as well as a

command center that is used to help coordinate the activities of these systems.

Figure 2: Dynamic Crisis Response SoS.

National and International Defense SoS: Government defense organizations are sometimes required to

work together to perform some of their missions. These joint missions may call upon army, navy,

marine corps, and air force systems to interoperate with each other. In other situations, the US military

organizations are called upon to support international missions and must interoperate with coalition

forces such as we are now doing in Afghanistan. Examples of types of systems that must interoperate in

this environment include command and control, communications, and medical care systems.

Space Exploration SoS: Systems such as the Mars Science Laboratory (MSL)

(http://mars.jpl.nasa.gov/msl/mission/rover/) can also be viewed as an SoS given the variety of research

systems placed upon the exploration rover platform that must interoperate with each other. MSL

science instruments are developed and maintained by various organizations, yet designed to

interoperate with each other to support the MSL missions. For example, the rover Mastcam camera

must interoperate with other science instruments to record conditions under which various Mars

samples are collected.

Key Characteristics of SoS

How is an SoS Different from a System? The earliest references in the literature to “systems within systems” or “system of systems” can be

found in [Berry, 1964] and [Ackoff, 1971]. These 1960-1970 era SoS concepts are early insights into the

evolution of today’s systems of systems. Even though the term “system of systems” was not commonly

Lane: What is an SoS USC-CSSE-2013-001 Page 4

used forty years ago, systems of systems were being developed and deployed. These SoSs are

represented by:

Undersea surveillance and weapons systems such as the Integrated Undersea Surveillance

System (IUSS) [FAS, 2006; IUSSCAA, 2006], Sound Surveillance System (SOSUS)

[GlobalSecurity.ORG, 2005], and Anti-Submarine Warfare (ASW) system [Smithsonian

Institution, 2000] used during the Cold War era to track and evade Russian submarines

Global Positioning System (GPS) [NAVSTAR, 2006] that is today considered both as a SoS and a

constituent-system for other SoSs

Military command and control centers.

As these types of integrated systems became more common, system engineering experts and

researchers began to define and study them as a special class of systems. Also, the term has become a

popular way to represent a strategic and economic approach to enhancing existing system capabilities,

and we now have an abundance of definitions.

In a review of SoS-related publications, [Lane and Valerdi, 2007] shows that the term “system of

systems” means many things to many different people and organizations. In the business domain, an

SoS is the enterprise-wide or multiple enterprise integration and sharing of core business information

across functional and geographical areas. In the military domain, an SoS is a dynamic communications

infrastructure and a configurable set of constituent-systems to support operations in a constantly

changing, sometimes adversarial, environment. In the collaborative research domain, it may be a

collection and sharing of research data and tools such as the Global Earth Observation SoS (GEOSS)

described in [U.S. EPA, 2013].

For some, an SoS may be a multisystem architecture that is planned up-front by a prime contractor or

lead system integrator. For others, an SoS is an architecture that evolves over time, often driven by

organizational needs, new technologies appearing on the horizon, and available budget and schedule.

The evolutionary SoS architecture is more of a network architecture that is reconfigured and grows with

needs and available resources.

Some SoS definitions refer to “emergent behaviors” or a “common purpose” where an SoS can perform

functions that cannot be provided by any of the constituent-systems [Cocks, 2006; DoD, 2006; Eisner,

1993; Kriegel, 1999; Maier, 1998; Sage and Cuppan, 2001; Shenhar, 1994; USAF SAB, 2005]. However, if

one reviews definitions of a system [ANSI/EIA, 1999; Blanchard and Fabrycky, 1998; INCOSE, 2006;

ISO/IEC, 2002; Rechtin, 1991], one sees that many of these definitions indicate that a system is a set of

components working together for a common objective or purpose. So, “emergent behavior” does not

appear to be a system characteristic unique to SoSs. What is controversial in the SoS arena is whether

emergent behaviors should be planned and managed. There are those that would like to see the

development of convergent protocols that could be implemented in a variety of systems to “SoS-

enable” them, allowing these systems to easily come and go in SoSs with little additional effort [USAF

SAB, 2005]. There are others that are concerned that if one is not careful, there may be undesirable

Lane: What is an SoS USC-CSSE-2013-001 Page 5

emergent behaviors (e.g., safety or security problems), and to avoid these problems, emergent

behaviors must be carefully planned, tested, and managed [DoD, 2008].

In any case, users and nodes in the SoS network may be either fixed or mobile. Communications between constituent-systems in the SoS are often some combination of common and custom-defined protocols. Networks may tie together other networks as well as nodes and users. SoS constituent-systems typically come and go over time. These constituent-systems can operate both within the SoS framework and independent of this framework. In a general sense, it is challenging to define clear boundaries of a specific SoS because of its dynamic nature. What is unique to SoSs [Maier, 1998] is that they are comprised of constituent-systems that possess the following characteristics:

Operationally independent, meaning that most or all of the constituent-systems can perform useful functions both within the SoS and outside of the SoS.

Managerially independent, meaning that most or all of the constituent-systems are managed and maintained for their own purposes.

Types of SoS Further SoS research [Maier, 1998; Dahmann and Baldwin, 2008] has identified four types of SoSE management approaches: virtual, collaborative, acknowledged, and directed. These categories, illustrated in Figure 3, are primarily based upon the levels of responsibility and authority overseeing the evolution of the SoS:

Virtual [Maier, 1998]: This type of SoS lacks a central management authority and a clear SoS purpose. It is often ad hoc and the constituent-systems are not necessarily known. An example of this type of SoS is the internet and all of the services that one can find and integrate in an ad hoc manner.

Collaborative [Maier, 1998]: In a collaborative SoS, the constituent-system engineering teams work together more or less voluntarily to fulfill agreed upon central purposes. In this type of SoS, there is no SoS engineering team to guide or manage SoS-related activities of the constituent-systems. An example of this type of SoS might be the regional area crisis response system where each agency that participates in first responder types of situations is responsible for its own systems.

Acknowledged [Dahmann and Baldwin, 2008]: Acknowledged SoS have recognized objectives, a designated manager, and resources at the SoS level, e.g., an SoSE team. But the SoSE team does not have complete authority over the constituent-systems. The constituent-systems maintain their independent ownership, objectives, funding, and development approaches. Figure 3 (Acknowledged) illustrates this using unidirectional arrows between the SoSE team and the constituent-systems. The unidirectional arrow means that the SoSE team can provide guidance to the constituent-systems but that the constituent-systems are not required to comply with SoSE requests or to formally report to SoSE teams. An example of this type of SoS might be a military command and control SoS that has transitioned from a collaborative SoS to an acknowledged SoS due to the importance of the missions supported by the SoS or the complexities of the cross-cutting SoS capabilities.

Directed [Maier, 1998]: A directed SoS is centrally managed by a government, corporate, or Lead System Integrator (LSI) team and built to fulfill specific purposes. The constituent-systems

Lane: What is an SoS USC-CSSE-2013-001 Page 6

maintain their ability to operate independently, but evolution is predominately controlled by SoS management organization. Figure 3 (Directed) illustrates this using bi-directional arrows between the SoSE team and key constituent-systems (but not necessarily all). The bi-directional arrows indicate that the SoSE team has authority (and often funding) to require these constituent-systems (often through some sort of contract) to develop and support SoS capabilities. Examples of this type of SoS might be the health care SoS or the MSL SoS described above where there is a chief information officer or a management team responsible for the overall SoS and its constituent-systems.

Figure 3: Types of SoS.

Key Ways in Which SoSE Differs from Traditional Single Systems

Engineering SoSs are seldom developed as an SoS. Rather, systems are developed over time that can interface with

each other. Typically, as these systems are deployed and start working together as a set of

interoperating systems, they form a collaborative SoS where key capabilities performed at the SoS level

are evolved through collaboration of the constituent-system engineering teams. When a collaborative

SoS becomes strategically important or there are problems in managing and evolving the SoS

capabilities through collaboration, the collaborative SoS may become an acknowledged SoS. It is at this

point that an SoSE team is identified and systems engineering is conducted at the SoS level. So, SoSE

seldom starts with a “clean sheet of paper”. Rather, SoSE begins with assessing the key SoS capabilities

and how well they are currently being performed. Then the SoSE team begins to identify needed

performance and functional changes and alternatives for providing these enhancements over time.

Lane: What is an SoS USC-CSSE-2013-001 Page 7

Most systems engineering standards are oriented towards the initial develop and evolution of single

systems. Most also address SoS engineering and development to some extent and at least imply that

the processes, standards, and models apply to SoSE. Noticing that more and more, defense-related

system capabilities relied upon the interaction of multiple systems developed and maintained by

multiple organizations, the US Department of Defense (DoD) Office of the Secretary of Defense (OSD)

Acquisitions, Technology, and Logistics (AT&L) Systems and Software Engineering (SSE) organization

went a step further and developed an SoSE guidebook, the Systems Engineering Guide for System of

Systems (hereinafter referred to as the DoD SoSE guidebook) [DoD, 2008], that extends the US Defense

Acquisition Guide (DAG). This SoSE guidebook, designed to be used in conjunction with the DAG, was

developed as a result of conversations with 18 SoSE teams.

Through these conversations, it was clear that traditional SE processes must be tailored at the SoS level

to both guide the evolution of the SoS and at the same time allow the CSs to evolve to meet the needs

of their single-system stakeholders. This tailoring was captured in the seven core SoSE activities

described in the DoD SoSE guidebook. A key message of the DoD SoSE guidebook is that SoSE core

elements are built upon the traditional SE activities, but they are both an expansion of the traditional

activities and organized in a very different manner. The key SoSE core activities in the DoD SoSE

guidebook [DoD, 2008] are:

Translating capability objectives: The SoSE team must develop a basic understanding of the

expectations of the SoS capability and then translate the capability into a set of requirements

for meeting the expectations.

Understanding systems and relationships: In a SoS, the focus is on the systems which contribute

to SoS capabilities and their interrelationships instead of the traditional focus on boundaries and

interfaces.

Assessing actual performance to capability objectives: To be able to understand current SoS

performance and ascertain the impact of constituent-system changes, the SoSE team establishes

SoS metrics, defines methods for assessing performance, and conducts evaluations of actual

performance using the metrics and methods.

Developing, evolving, and maintaining an SoS architecture/design: As soon as systems start

interfacing with each other and sharing data, there is an implied architecture for the collection

of systems (or SoS). One of the key responsibilities of an SoSE team is to establish and maintain

a sustainable framework to support the evolution of the SoS to meet user needs. Evolutionary

changes include changes in systems’ functionality, performance, or interfaces. These needed

changes often require systems to migrate from the early “implied” architecture to a more robust

architecture or framework.

Monitoring and assessing changes: The SoSE team must constantly monitor proposed or

potential changes to the constituent-systems and assess their impacts to a) identify

opportunities for enhanced functionality and performance, and b) preclude or mitigate

problems for the SoS and other constituent-systems.

Lane: What is an SoS USC-CSSE-2013-001 Page 8

Addressing new requirements and options: The SoSE team reviews, prioritizes, and determines

which SoS requirements to implement next. Part of this activity is evaluating various options for

implementing the capability and requires the participation of the affected constituent-systems.

Orchestrating upgrades to SoS: This activity is the actual implementation of the desired

capabilities and includes the planning, coordination, integration, and testing of changes in the

constituent-systems to meet SoS needs.

Key SoSE Challenges Several challenges at the SoSE level were mentioned during the interviews conducted as part of

developing the DoD SoSE Guidebook. The following summarizes some of the key SoSE challenges.

Focusing constituent-systems on SoS needs and capabilities: In the DoD environment, most system

development, upgrade, and maintenance funding is at the single-system level. As a result, incremental

system upgrades are specified and prioritized by the single-system stakeholders. SoSE teams must often

negotiate with the single-system stakeholders to get them to support single-system changes required to

support SoS needs and capabilities. With limited funding and tight schedules at the single system level,

this can require single-system stakeholders to defer needed single system changes in order to

accommodate the SoS changes in a timely way.

In the commercial environment, it is often not much easier. While there is typically an organization

Chief Information Officer (CIO) that is responsible for the enterprise systems, many of these enterprise

systems are comprised of one or more Commercial-Off-the-Shelf (COTS) products. The evolution of the

COTS products depends upon the COTS vendor’s perception of market trends—if a requested change is

consistent with other users’ needs, it will get incorporated. Otherwise the request is either deferred or

ignored. In addition, COTS vendors like to evolve their products in ways that require their users to

periodically upgrade their COTS products—classic examples of this are operating systems such as

Windows and database management systems such as Oracle. And finally, COTS vendors often view

other COTS vendors as competitors and are not interested in being interoperable with other COTS

products or being compatible with competitor products.

Coordinating development of new capabilities across constituent-systems: As mentioned above, a new

capability (or the enhancement of an existing capability) can often require changes to multiple systems.

However, each system has its own upgrade cycle which is probably not well aligned to other

constituent-systems in the SoS. So, it is often the case that “pieces” of a given capability (or capability

upgrade) are deployed in the operational environment over a considerable period of time. This requires

systems to be able to interoperate with older versions of some systems until all of the affected systems

have been upgraded.

Another challenge in this area is that some systems belong to multiple SoS. In these cases, these

systems receive requests from multiple SoS, some of which conflict with each other. It is typically up to

the single system to decide which sets of changes to implement, leaving some SoS to pursue other

alternatives for their desired SoS capability.

Lane: What is an SoS USC-CSSE-2013-001 Page 9

Maintaining SoS performance: As SoSs grow in size and complexity over time, performance issues can

arise from the amount of resident data, transfer of data between systems, and system interoperability

issues, requiring the SoS engineers to make architectural adjustments to better tune the SoS. Often

these architectural changes can affect multiple constituent-systems and can be especially difficult to

implement when some of the constituent-systems belong to more than one SoS.

Testing SoS capabilities in an asynchronous development environment: The ability to fully test SoS

capabilities before they are deployed is extremely limited often due to the size and complexity of the

SoS, but also due to the fact that the capability “pieces” are deployed in various systems as they are

implemented. As a result, a key approach to “testing” SoS capabilities is assessing performance in the

operational environment, then making adjustments over time to achieve the desired capability

performance.

Identifying (anticipating) and managing unplanned SoS emergent behaviors: Because SoS capability

testing is often not very rigorous before the constituent-systems are deployed and users often use

systems in ways not anticipated by the developers, emergent system and SoS behaviors are often not

detected until the system is already in use. Some of the emergent behaviors are desirable and may be

enhanced in future SoS upgrades, but others may be extremely undesirable and must be mitigated

immediately. Mitigation activities may include disabling affected features or capabilities in the deployed

systems, developing “quick fixes” to patch the affected systems, or rolling back to a previous version of

one or more constituent-systems. In many SoS environments, it is not reasonable to shut the SoS down

until a fix is available.

SoSE capability cost and schedule analysis: Those responsible for funding decisions at the SoS level are

also concerned about cost and scheduled related to the sustainment and enhancement of existing SoS

capabilities. While existing cost models can be calibrated to support SoSE estimation [Lane, 2009],

actual cost and schedule for various capability alternatives can be impacted by constituent-system

interoperability, system fragility (for those constituent-systems that are close to end of life), the

dependability of the constituent-system stakeholders, and the coordination of multiple development

organizations, each with its own development processes and personnel expertise and productivity.

An Example of the Impact SoS Engineering Can Have Today many crisis response organizations, as illustrated in Figure 4, are incorporating various

technologies in their platforms and systems so that they can better interoperate as an SoS in response

to sudden crises. These capabilities have been very important in fighting recent wild fires in Southern

California [CDFFP, 2011].

Lane: What is an SoS USC-CSSE-2013-001 Page 10

Figure 4: Regional Area Crisis Management System of Systems.

However, an example of what can happen when interoperability is missing is the Cedar Fire that

occurred in San Diego County, California, in 2003. On Saturday, October 25, 2003, an in-experienced

hunter became separated from his friends in the mountains east of San Diego. Disoriented from heat

and dehydration, he set a signal fire so that he would be rescued. The signal fire worked and the hunter

was rescued by local firefighters. However, it was getting dark and the winds were calm, so the

firefighters decided to let the signal fire burn itself out. Unexpectedly strong winds came up during the

night and whipped the fire up, pushing it into the city of San Diego. Fire, police, sheriff deputies,

ambulances, hospitals, and evacuation centers worked together to fight the fire and keep the local

residents out of harm’s way. As the fire continued to quickly grow and spread, additional civil

firefighting resources were called in from surrounding counties, states, and even Canada to help fight

the fire. Even with the additional resources, local and regional responders were severely overwhelmed.

In response, state and federal military organizations who were in the area volunteered to help fight the

fires—they had both firefighting equipment and trained personnel to operate the equipment. However,

they ended up on the sidelines for most of the time watching homes and businesses burn because they

could not communicate with the local responders and it was deemed too risky to have them in the

firefighting areas without communications with the other responders. When the fires finally did reach

the US Marine Corps Air Station Miramar in central San Diego, the marines were “ready and waiting”

and able to stop the fire from burning all the way to the Pacific Ocean.

After the Cedar fire was over, county and state officials conducted studies to determine how to better

fight a fire of this magnitude in the future—this was San Diego’s wake-up call that first responders and

crisis-related systems needed to be more interoperable. As a result of considerable analysis of the “as

is” systems and alternatives for improving interoperability between the systems of multiple agencies at

various levels, changes were made to improve interoperability of the local, regional, and federal crises

Lane: What is an SoS USC-CSSE-2013-001 Page 11

response systems. In addition, county-wide “exercises” were planned and conducted to ensure that

first-responder personnel were able to take advantage of the new communications capabilities, policies,

and procedures in place between different agencies.

A few years later in 2007, San Diego experienced another horrific fire. Hot, dry “Santa Ana” winds

caused power lines in the county backcountry to spark and to quickly whip up fires on many fronts. This

2007 fire, referred to the Witch Creek Fire, could have been as bad as or worse than the 2003 Cedar

Fire. However, with the improved first responder and firefighter communications interoperability,

military and other state and Federal organizations were able to help early and minimize the fire damage.

Figure 5 compares the damage that resulted from these two fires.

Figure 5: Impacts of Interoperability Improvements Four Years Later in San Diego, California

The statistics in Figure 5 show that with the improved firefighting capabilities in San Diego, acreage

burned was reduced by over 25%, structures destroyed reduced by almost 50%, and human casualties

reduced to a small fraction of the Cedar Fire. In addition, the total costs of fighting the Witch Creek fire

were only about one third of the cost to fight the Cedar Fire. At this point in time, no actual numbers

are publically available to show what the improvements in interoperability cost, but the local

government agencies agree that there has been a significant return on their first-responder

interoperability investments in San Diego, California.

Overview of Current USC CSSE SoSE Research Areas The following list identifies some of the current SoSE research at USC CSSE along with references to

journal, conference, and CSSE technical reports that further describes each research area. These

Lane: What is an SoS USC-CSSE-2013-001 Page 12

research areas are primarily investigating and proposing improvements to SoSE and the ability plan and

manage SoSs over time.

SoSE cost modeling and estimation [Lane, 2009]: Describes the COSYSMO for SoSE cost model

and how it can be used to characterize proposed changes within a set of SoS constituent-

systems and determine an estimate of the systems engineering effort required to engineer the

indicated changes. (Note that any related software changes must be further estimated using

the COCOMO® for software cost estimation model for each affected system.)

Constituent-system interoperability [Lane and Valerdi, 2011]: Describes the importance of

constituent-system interoperability in determining the amount of effort to improve and expand

SoS capabilities within a given SoS. Alternatives are also presented for including this influence in

the COSYSMO for SoSE cost model.

SoSE and lean principles [Lane and Valerdi, 2010]: Describes the results of applying a lean

enterprise lens to case study data from 13 SoSE teams. The case study data supports

observations that SoSE teams are developing processes that are consistent with many lean

enterprise principles. This paper provides further insights and recommendations for the

evolution of system of systems processes using lean concepts.

Kanban scheduling to improve visibility and work flow within an SoS development environment

[Turner and Lane, 2013]: Describes a lean Kanban framework that can be used in an SoS

development environment to enable:

o Better status visibility managing multiple concurrent development projects, especially

across deep supplier chains

o More effective integration and use of scarce systems engineering resources

o Earlier delivery of project and enterprise value

o More flexibility while retaining predictability

o Less blocking of product team tasks waiting for systems engineering response

o Lower governance overhead.

SysML tailored for SoSE [Lane and Bohn, 2013]: Presents an approach to tailoring SoS Systems

Modeling Language (SysML) models to support SoS capability engineering and cost

modeling/estimation.

SoS capability engineering [Lane, 2012]: Provides guidance for translating SoS capability

objectives into requirements; defines SoS engineering methods, processes, and tools that

might support this activity; and illustrates how the methods, processes, and tools can be used

and integrated to support SoS engineering using an example SoS.

References Ackoff, R. 1971. Towards a system of systems concepts. Management Science Theory Series 17, no. 11:

661-671.

ANSI/EIA. 1999. ANSI/EIA-632-1988 Processes for engineering a system.

Berry, B. 1964. Cities as systems within systems of cities. The Regional Science Association Papers 13.

Blanchard, B. and W. Fabrycky.1998. Systems engineering and analysis. Upper Saddle River: Prentice Hall.

Lane: What is an SoS USC-CSSE-2013-001 Page 13

Boehm, B., C. Abts, A. Brown, S. Chulani, B. Clark, E. Horowitz, R. Madachy, D. Reifer, and B. Steece. 2000. Software cost estimation with COCOMO II. Upper Saddle River: Prentice Hall.

Boehm, B., R. Valerdi, J. Lane, and A. Brown. 2005. COCOMO suite methodology and evolution. CrossTalk - The Journal of Defense Software Engineering 18, no. 4: 20-25.

California Department of Forestry and Fire Protection, http://www.fire.ca.gov/fire_protection/fire_protection_coop_efforts.php, accessed on 3/5/2011.

Cocks, D. 2006. How should we use the term "system of systems" and why should we care? Proceedings of the 16th Annual INCOSE International Symposium, July 9-13, in Orlando, FL.

Dahmann, J. and K. Baldwin. 2008. Understanding the current state of US defense systems of systems and the implications for systems engineering. Proceedings of the IEEE Systems Conference, April 7-10, in Montreal, Canada.

Department of Defense, 2006. Defense acquisition guidebook, Version 1.6. http://akss.dau.mil/dag/ (accessed on 2/2/2007).

Department of Defense. 2008. Systems engineering guide for system of systems, version 1.0.

Dorner, D. (1996); The logic of failure, Metropolitan Books.

Eisner, H. 1993. RCASSE: Rapid computer-aided systems of systems engineering. Proceedings of the 3rd International Symposium of the National Council of System Engineering (NCOSE) 1: 267-273.

Federation of American Scientists (FAS), Integrated undersea surveillance system (IUSS), http://www.fas.org/irp/program/collect/iuss.htm (accessed on 12/27/2006).

Friedman, T. 2005. The world is flat: A brief history of the twenty-first century. New York: Farrar, Straus and Giroux.

GlobalSecurity.ORG. 2005. Sound surveillance system (SOSUS). http://www.globalsecurity.org/intell/systems/sosus.htm (accessed on 1/20/2007).

Highsmith, J. 2000. Adaptive software development: A collaborative approach to managing complex systems, New York: Dorset House Publishing.

INCOSE. 2006. Systems engineering handbook, Version 3, INCOSE-TP-2003-002-03.

ISO/IEC. 2002. ISO/IEC 15288:2002(E) Systems engineering - system life cycle processes.

IUSS-Caesar Alumni Association (IUSSCAA). 2006. IUSS history. http://www.iusscaa.org/history.htm (accessed on 12/27/2006).

Krygiel, A. 1999. Behind the wizard’s curtain: An integration environment for a system of systems. CCRP Publication Series.

Lane, J. 2009. Cost Model Extensions to Support Systems Engineering Cost Estimation for Complex Systems and Systems of Systems, Proceedings of the Seventh Conference on Systems Engineering Research.

Lane, J. 2012. System of Systems Capability to Requirements, USC-CSSE Technical Report USC-CSSE-2012-505.

Lane, J. and T. Bohn. 2013. Using SysML Modeling To Understand and Evolve Systems of Systems, Systems Engineering Journal, Vol. 16, No. 1.

Lane: What is an SoS USC-CSSE-2013-001 Page 14

Lane, J. and R. Valerdi. 2007. Synthesizing system-of-systems concepts for use in cost estimation. Systems Engineering 10, no. 4:297-307.

Lane, J. and R. Valerdi. 2010. How to Accelerate Understanding and Optimization of System of Systems Engineering through Lean Enterprise Principles, Proceedings of the IEEE Systems Conference, 5-8 April, in San Diego, CA.

Lane, J. and R. Valerdi. 2011. System Interoperability Influence of System on Systems Engineering Effort, Proceedings of the Conference on Systems Engineering Research, 14-16 April, Los Angeles, CA.

Maier, M. 1998. Architecting principles for systems-of-systems. Systems Engineering 1, no. 4: 267-284.

NAVSTAR Global Positioning System Joint Program Office, http://gps.losangeles.af.mil/ , (accessed on 12/6/2006).

Rechtin, E. 1991. Systems architecting: Creating and building complex systems, Upper Saddle River, Prentice Hall.

Sage, A. and C. Cuppan. 2001. On thesystems engineering and management of systems of systems and federations of systems. Information, Knowledge, and Systems Management 2: 325-345.

Shenhar, A. 1994. A new systems engineering taxonomy, Proceedings of the 4th International Symposium of the National Council on System Engineering, National Council on System Engineering 2, pp 261-276.

Smithsonian Institute, National Museum of American History. 2000. Submarine missions: Anti-dubmarine warfare, http://americanhistory.si.edu/subs/work/ missions/warfare/index.html (accessed on 1/20/2007).

Turner, R. and J. Lane. 2013. Goal-Question-Kanban: applying lean concepts to coordinate multi-level systems engineering in large enterprises, accepted for the Conference on Systems Engineering Research scheduled for 19-22 March 2013, Atlanta, Georgia.

United States Air Force (USAF) Scientific Advisory Board (SAB). 2005. Report on system-of-systems engineering for Air Force capability development; Public Release SAB-TR-05-04

United States Environmental Protection Agency (U.S. EPA), Global Earth Observation System of Systems (GEOSS), http://www.epa.gov/geoss/ accessed on 1/1/2013.