Ct*Cr ý

-v-,

~D C

---- -- -. g Adak

STA F VILI*-_pro e fo pu lc rl e 0

unclassified ,SECURITY CLASSIFICATION OF TRIS PAGE

REPORT DOCUMENTATION PAGEIa. REPORT SECURITY CLASSIFICATION lb. RESTRICTIVE MARKINGS

unclassified n/a

2a. SECUR!TY CLASSIFICATION AJUTHORITY 3. DIStRIBUTION! AVAILABIUTY OF REPORTUSAF, AFSC/CA Dist. A. Approved for public release:

2b. DECLASSiFICATION /DOWNGRADING SCHEDULE unlimited distribution.

4. PERFORMING ORGANIZATION REPORT NUMBER(S) 5. MONITORING ORGANIZATION REPORT NUMBER(S)

n/a n/a

6a. NAME OF PERFORMING ORGANIZATION 6b. OFFICE SYMBOL 7a. NAME OF MONITORING ORGANIZATION

AIR FORCE STUDIES BOARD (If applicable)

NATIONAL RESEARCH COUNCIL n/a HQ AFSC/CA

6c. ADDRESS (Cty, State, and ZIP Code) 7b. ADDRESS (City, State, and ZIP Code)2101 Constitution Avenue, N.W.

Washington, D.C. 20418 Andrews AFB, MD 20334

8e. NAME OF FUNDING ISPONSORING 1 'b. OFFICE SYMBOL 9 PROCUREMENT INSTRUMENT IDENTIFICATION NUMBERORGANJIZATION! Of applicable)HQ AFSC n/a ,._.

Sc. ADDRESS (City, State, and ZIP Code) 10 SOURCE OF FUNDING NUMBERS

PROGRAM PROJECT TASK WORK UNITAndrews AFB, MD 20334 ELEMENT NO. NO. NO. ACCESSION NO.

n/a n/a n/a n/a11. TITLE (Include Security Claisfication)

ADAPTING SOFTWARE DEVELOPMENT POLICIES TO MODERN TECHNOLOGY (U)

12. PERSONAL AUTHOR(S) Committee on Adapting Software Development Policies ...

13a. TYPE OF REPORT 1 3b. TIME COVERED 114. DATE OF REPORT (Year. Month, Day) S. PAGE COUNTfinal FROM 6/88 TO 8/88 1989 July 88

16. SUPPLEMENTARY NOTATION

17. COSATI CODES "!1. SUBJECT TERMS (Continue on reverse if necessay and identify by block number)FIELD GROUP I SUBGROUP Ii software software reliability

12 E8 SEE software acquisition

'9. ABSTRACT (Continue on reverse if necessiary aod identify by block number)

-- The problem of developing reliable software that is capable of performing its intendedfunction has persisted since ladvent of digital computer.S, Subject has been reviewed manytimes. Are newer methods of software development now being introduced for large, high-technology systems outstripping conventional software acquisition techniques and policies?

The committee reviewed recent• software studies to learn why they did not adequatelysolve problem. Also, committee assessed current and oast development programs, investi-gated methods for the user and developer to work more closely, evaluated softwareprocess models as alternatives to the waterfall model, evaluated incremental development,

evolutionary development, Drototyping, etc.

20. DISTRIBUTION /AVAILABILITY OF ABSTRACT 21. ABSTRACT SECURITY CLASSIFICATIONiIUNCLASSIFIED/UNUMITED 03 SAME AS RPT, r3OTIC USERS unclassified

22a. NAME OF RESPONSIBLE INDIVIDUAL I22b.TELEPHONE nu*de Area Code) 122c. OFFICE SYMBOLVernon H. Miles, Sr., Director AFSB 202/334-3531' n/a

DD FORM 1473,84 MAR 83 APR edition may be used until exhausted. SECURITY CLASSIFICATION OF THIS PAGEAll other editions are obsolete.

Adapting SoftwareDevelopment Policies toModern Technology

Committee on Adapting Software Development Policiesto Modern Technology

Air Force Studies BoardCommission on Engineering and Technical SystemsNational Research Council

DTIC

S LECTE

NATIONAL ACADEMY PRESSWashington, D.C. 1989 DIS 8TAO A

AVpTovýd for publie r1ehaft;DLMnbut-nn U'litnitod

NOTICE: The project that is the subject of this report was approved by the GoverningBoard of the National Research Council, whose members are drawn from the councils ofthe National Academy of Sciences, the National Academy of Engineering, and theInstitute of Medicine. The members of the committee responsible for the report werechosen for their special competences and with regard for appropriate balance.

This report has been reviewed by a ,%roup other than the authors according toprocedures approved by a Report Review Committee consisting of members of theNational Academy of Sciences, the National Academy of Engineering, and the Institute ofMedicine.

The National Academy of Sciences is a private, nonprofit, self-perpetuating societyof distinguished scholars engaged in scientific and engineering research, dedicated to thefurtherance of science and technology and to their use for the general welfare. Uponthe authority of the charter granted to it by the Congress in 1863, the Academy has amandate that requires it to advise the federal government on scientific and technicalmatters. Dr. Frank Press is president of the National Academy of Sciences.

The National Academy of Engineering was established in 1964, under the charter ofthe National Academy of Sciences, as a parallel organization of outstanding engineers. Itis autonomous in its administration and in the selection of its members, sharing with theNational Academy of Sciences the responsibility for advising the federal government.The National Academy of Engineering also sponsors engineering programs aimed atmeeting national needs, encourages education and research, and recognizes the superiorachievements of engineers. Dr. Robert M. White is president of the National Academy ofEngineering.

The Institute of Medicine was established in 1970 by the National Academy ofSciences to secure the services of eminent members of appropriate professions in theexamination of policy matters pertaining to the health of the public. The Institute actsunder the responsibility given to the National Academy of Sciences by its congressionalcharter to be an adviser to the federal government and, upon its own initiative, toidentify issues of medical care, research, and education. Dr. Samuel 0. Thier is presidentof the Institute of Medicine.

The National Research Council was organized by the National Academy of Sciencesin 1916 to associate the broad community of science and technology with the Academy'spurposes of furthering knowledge and advising the federal government. Functioning inaccordance with general policies determined by the Academy, the Council has become theprincipal operating agency of both the National Academy of Sciences and the NationalAcademy of Engineering in providing services to the government, the public, and thescientific and engineering communities. The Council is administered jointly by bothAcademies and the Institute of Medicine. Dr. Frank Press and Dr. Robert M. White are"chairman and vice chairman, respectively, of the National Research Council.

This report represents work under Contract No. F49620-87-C-0122 between theUnited States Air Force and the Natiohal Academy of Sciences.

Copies of this report are available from:

Air Force Studies BoardNational Research Council2101 Constitution Avenue, N.W.Washington, D.C. 20418(202) 334-3531

and from the Defense Technical Information Center, Alexandria, Virginia.

ADAPTING SOFTWARE DEVELOPMENT POLICIES TO MODERN TECHNOLOGY 11o

AIR FORCE STUDIES BOARD

John L. McLucas, ChairmanQuesTech Inc.

John J. Martin, Vice ChairmanNASA (retired)

Julian Davidson, Chairman Emeritus, Booz Allen and Hamilton, Inc.Josppftne D. Davis, Albany State CollegePaul i. Droulihet, Massachusetts Institute of Technology Lincoln LaboratoryCraig L. Fischer, M.D., Diametrix Inc.Grant L. Hansen, Unisys Corporation (retired)James E. Hubbard, Jr., Charles Stark Draper LaboratoryBenjamin Huberman, The Consultants International GroupErich P. Ippen, Massachusetts Institute of TechnologyLorenz A. Kull, Science Applications International CorporationJohn K. Lauber, National Transporation Safety BoardJames W. Mar, Massachusetts Institute of TechnologyGary D. Mather, Booz Allen and Hamilton, Inc.Robert C. Mathis, Toomay, Mathis & Associates, Inc.Brockway McMillan, Chairman Emeritus, Bell Telephone Labs, Inc. (retired)Hyla S. Napadensky, Napadensky Energetics, Inc.Brian O'Brien, Chairman Emeritus, private consultantOswald G. Villard, Jr., Member Emeritus, Stanford UniversityRobert A. White, University of Illinois, Urbana-Champaign

COMMITTEE ON ADAPTING SOFTWARE DEVELOPMENT POLICIESTO MODERN TECHNOLOGY __-

Accesion For

Walter R. Beam, Chairman NTIS ,RA&IGeorge Mason University DTIC TAB 0

Unxaznno,.•ce d 0]

Robert L. Edge, U.S. Air Force (retired) justioation_ 0David J. Farber, University of PennsylvaniaC. Corde.1 Green, Kestrel Institute ByRichard J. Sylvester, MITRE Corporation DBstribution/Joseph E. Urban, University of Miami - -Willis H. Ware, RAND Corporation Avalability- C odes

Avail and/orDist Special

LIAISON REPRESENTATIVES

Philip S. Babel, Wright-Patterson Air Force Base, OhioBarry W. Boehm, TRW CorporationLieutenant Colonel Terry E. Courtwright, Andrews Air Force Base, MarylandSamuel A. DiNitto, RADC/COE, Griffiss Air Force Base, New YorkWinston W. Royce, Lockheed Aeronautical SystemsMary Shaw, NRC Computer Science and Technology Board \ ,•€'Lieutenant Colonel Michael P. Weldemer, Andrews Air Force Base, Maryland '_.o-

iv ADAP1ING SOFTWARE DEVELOPMENT POLICIES TO MODERN TECHNOLOGY

AIR FORCE STUDIES BOARD STAFF

Vernon H. Miles, Sr., DirectorDonald L. Whittaker, Editor/Staff AssociateTerrie Noble, Administrative AssociateKatherine H. Atkins, Secretary

ADAPTING SOFTWARE DEVELOPMENT POLICIES TO MODERN TECHNOLOGY V

STATEMENT OF TASK

The problem of developing reliable software that is capable of performing its intendedfunction has persisted since the advent of the digital computer. The subject has beenreviewed many times by various groups, including the Air Force Studies Board (AFSB), yetthe Commander of the Air Force Systems Command states that it is his primary problem insystems development. Four years ago the AFSB studied software reliability. The Boardwants to know how effective their previous study was and why it and other recent studies onthe subject have not resolved the problems with software development. The answer may bethat newer methods of software development now being introduced for large, high-technologysystems are outstripping conventional software acquisition techniques and policies. Thesenewer methods are directed at coping with the realization that software cannot be definedand controlled under conventional precepts of baseline management (as outlined in MIL-STDs483, 490, 999, 1521, 1679, and DoD-STD 2167). Three areas where the system may breakdown are:

1. Software architects need access to experimental testing of systems alternatives beforethey are in a position to write sharply stated requirements or make optional designselections.

2. User interfaces are poorly handled or ignored under the present procurement system.

3. Design inadequacies or outright failures in other system elements are corrected bysoftware changes. These occur late in the procurement cycle and are invariably handledin an ad hoc manner outside the normal procurement actions.

Solutions are emerging to handle these difficulties but they cannot be introduced intothe procurement system because they all violate the basic principle of an early, formal def-inition of design. This study may provide new concepts for maintaining control of acquisi-tions during the early phases that do permit software architects to work effectively.

To better understand the problems in software development and help derive solutions,the committee will:

* review recent software studies and determine why they have not resolved the softwareacquisition problem,

* assess some current and past development programs and identify software deficiencies,

" investigate methods for the software user and software developer to work more closelytogether in co-evolution of the task,

"* consider the role of prototyping and evolution in the software development process,

" evaluate software process models as alternatives to the waterfall model (e.g., the spiralmodel of software development, or software first) especially as regards their appro-priateness to high-tech procurements, and

determine how adequate controls (e.g., formal derivation history) for testability,reliability, and survivability can be made part of an incremental development.

vi ADAPTING SOFTWARE DEVELOPMENT POLICIES TO MODERN TECHNOLOGY

Note: This statement of task proved a very serviceable basis for the committee'sstudy. However, we must point out that studies can seldom if ever resolveproblems of these kinds, only suggest means of resolution. The aforementioneddesign inadequacies and failures must be handled by normal procurement actions,for there is no magic alternative. We recognize that there are always ad hocdeviations from the preliminary plans for many complex systems.

The last of the six tasks listed above appears to presuppose a mechanism to beused during incremental development, and is addressed explicitly in Section 3.3.3.

ADAPTING SOFTWARE DEVELOPMENT POLICIES TO MODERN TSCHNOLOGY Vii

CONTENTS

EXECUTIVE SUMMARY a1

1.0 INTRODUCTION W51.1 Background1.2 Study Approach

2.0 BACKGROUND a 72.1 Perspectives2.2 Previous Studies2.3 Ada, STARS, and the Software Engineering Institute

3.0 STRATEGIES FOR SOLUTION a 183.1 Overview3.2 Risk Reduction Objectives3.3 Improving the Acquisition and Development Processes3.4 Strengthening Personnel Resources3.5 Quality of the Software Product- -Acquisition Aspects3.6 Technology

4.0 CONCLUSIONS AND RECOMMENDATIONS .554.1 The Acquisition Process4.2 Acquisition Policy4.3 In-house Expertise4.4 Technology Program4.5 Recommendations in this Report

APPENDIX A: Previous Studies with Brief Summary of Recommendations N 59

APPENDIX B: Schedule of Meetings M 69

APPENDIX C: Briefing Charts n 75

GLOSSARY o 87

ACRONYMS v87

EXECUTWE SUMMARY 1

EXECUTIVE SUMMARY

THE PROBLEM

Today computer software controls primarily to early reduction of risks andthe operation of most weapon systems; to software-knowledgeable management.ten years ago it did not. Accordingly, Many key system *players" in bothsoftware-related development has stead- government and industry are weak inily increased in importance to Air Force software knowledge. Also, softwareSystems Command (AFSC). As an issue, experts are frequently not expert init has arisen more and more frequently systems. Software has not been properlyin recent times, as evidenced by at least recognized as an item that takes a17 "software studies" initiated by the Air relatively long time to develop; insteadForce, Department of Defense (DoD), development of appropriate software hasand U.S. General Accounting Office often been initiated late in the system(GAO) over the past 15 years. The life cycle. Although software has beenCommittee on Adapting Software Devel- used to overcome some hardwareopment Policies to Modern Technology limitations (but with time or costwas asked to re'view these earlier studies overruns), some hardware and systemand find out why, in spite of their limitations are beyond software's abilityefforts, they did not solve software to compensate.development problems. From thisreview, we reached two conclusions: (1) Overall, we addressed three mainthe importance and complexity of soft- areas: software development processes,ware have grown faster than could be the people needed to manage and carryadequately handled by the prevailing out these processes, and the technologymanagement and technical methods, and that underlies process evolution. Each(2) while this growth might have been of these areas demands more attentionpredicted, its consequences for personnel now that software is making a majorand programs were not fully anticipated. transition, joining integrated circuits at

the center of information technology.In addition, the committee was

asked to examine programs past andpresent to determine their software ADA, STARS, AND SEIdeficiencies. We accomplished thisreview with the cooperation of represen- Ada, the Department of Defensetatives from the AFSC product division, computing language, is a combination ofAir Force Logistics Command (AFLC), "good news' and "bad news." It is trulyand the using commands. In doing so, valuable for constructing large softwarewe found few problems for which programs; however, its use for a systemsoftware alone could be blamed, demands a proven compiler, which N

more complex than those for older,Most problems that surfaced as simpler languages. For real-time

software shortcomings were due in fact applications it is comparable to mostto not having well-developed require- other high-order programming languages;ments or to impractical acquisition however, its performance with existingconstraints. Successful acquisition of hardware is less than that possible withsr•ftware-intensive systems was due the use of machine language program-

2 ADAPTING SOFTWARE DEVELOPMENT POLICIES TO MODERN TECHNOLOGY

ming. Waivers from the Ada standard The waterfall model is satisfactorywill make economic and operational only for precedented systems, i.e., thosesense in some cases; but in each case for which substantial implementationcost and performance issues must be experience exists. In other cases, it isadequately understood. highly risky and does not emphasize

safeguards in the definition of require-Three different efforts involving ments, evolution of system design, and

the Defense Advanced Research Projects development or maintenance of effectiveAgency's STARS (software technology design teams. In short, alternativefor available, reliable systems) program models are needed for the developmentsupport this committee's recommenda- of unprecedented systems.tions, particularly in relation to softwareengineering environments (SEEs). We recommend appropriate application ofSTARS, however, should not be expected alternative acquisition processes. Suchto produce near-term results; from an alternative processes should Include aAir Force perspective, STARS results demonstration/validation (Dem/Val)may appear in system proposals within a phase, a competitive first-phase procure-few years, but not before then. ment, or both. Also, a warm, and

therefore responsive, Industrial baseThe Software Engineering Institute should be maintained by funding

(SEI) is a federal contract research application-specific software technologycenter, and should be viewed as an programs.

additional rather than a primaryresource. SEI staff should complement Acquisition Practices. In examiningthose employed directly by the Air acquisition practices, we found thatForce. AFSC should work with SEI to DoD-STD-2167 and MIL-STD-1521 (thetake advantage of SEI's capabilities in waterfall model) and other specificationscontractor assessment and SEEs. and standards often were imposed with

little concern for the characteristics ofthe particular acquisition. Firm fixed

FINDINGS price contracting is still being imposedon development of some unprecedented

Five software issues occupied the systems.committee's attention: (1) the (system)acquisition process, (2) Pcquisition Although many program offices arepractices, (3) software engineering, (4) calling for more user involvement, itin-house software expertise, and (5) appears that a more tailored usersoftware technology, involvement -- the right number of the

right people at the right time -- isThe Acquisition Process. The process required. Tailoring of acquisitionsmodel for software development defined should be universally and intelligentlyby DoD-STD-2167A is the so-called applied, yet there can be more risk than"waterfall" model. This model, often reward in such tailoring; without theheld to be the only acceptable model, experience and knowledge base to guideassumes non-iterative development them, program offices become uncom-between steps. The waterfall model is fortable and ignore tailoring or leave itan open loop model, often characterized to the contractors. Lack of timelyby unrealistic specifications, and mr;a- attention to software development oftensured by documentation rather than puts software advanced development indemonstrable results. Contractors slav- the midst of full-scale system develop-ishly follow this flawed model, which ment.only adds to the problem.

EXECUTIVE SUNflMARY 3

We recommend that AFSC Issue a policy A few varieties of SEE are avail-statement and implement workshops In able today, but too few to meet thetailored acquisition, devoting special projected needs of NASA and theemphasis to software. The handbooks Department of Defense. NASA issupporting Standards 2167A and 2168 investing a large budget in an SEE to beshould be published expeditiously as part developed for use by contractors of itsof this initiative, which should also Space Station program; DoD's Jointaddress user and maintainer involvement. Avionics Integration Working GroupCurrent Air Force regulations suggest plans to provide an SEE, as government-using the :oftware maintainer (e.g., Air furnished equipment, to contractors ofForce Logistics Command [AFLCI) for constituent programs. Large, geographi-Independent Validation and Verification cally distributed system and software(IV&V); this was reported by AFLC to developments demand this kind of com-have worked well and merits wider puter resource, which can also provideconsideration. Also, AFSC should avoid tools to help AFSC's software acquisitionfirm fixed price contracting in the activities.acquisition of unprecedented systems,unless adequate prior risk reduction We believe that AFSC should first,efforts are made. require contractors to use an appropriate

SEE for all but the smallest software-Software Engineering. Software engi- development acquisitions. Second,neering requires disciplined application developers of unprecedented systemsof a proven set of tools and techniques should be required to prove their(collectively referred to when computer- possession of and knowledge concerningbased as a software engineering environ- suitable design exploration tools. Third,meni, or an SEE) during software devel- AFSC should acquire those elements ofopment and later, after delivery with the an SEE that will support computer-basedsystem. Automated tools are needed to documentation, and build up a manage-control information throughout the sys- ment tool set. Fourth, contractorstern life cycle, first by the contractor should be required to deliver softwareand then by the maintainer. Certain engineering dAta in machine-readableSEE tools can also help AFSC program form that can be manipulated by AFSC'smanagers understand and monitor prog- document preparation tools.ress, collect data, and plan programs.Developers' software engineering, which In-house Software Expertise. Inis discussed in AFSC pamphlet 800-43 eyamining the personnel resources AFSC(31 January 86), may consist principally can bring to bear on software issues, weof the developmental tool set used in identified only about 320 software-software development. Such a tool set specialty officers at work in thewill include a compiler, debugger, Command. (The number of software-library, test support, etc. However, for knowledgeable civilians or of systemsunprecedented systems, SEEs must engineers with strong software skillsinclude a design exploration tool set could not be determined but is notincluding one or more very high level believed large enough to meet presentlanguages (VHLL) and simulators for and future needs.) Today few seniortesting. For management purposes, yet people are in this specialty field, thoughanother tool set may be employed, some software-knowledgeable officers arecontaining process models, scheduling employed in more visible specialtiesand tracking software, and acquisition promising better chances for promotion.management communications. The need for expertise is clearly on the

rise and already exceeds the number ofexperienced personnel available, and

4 ADAPTING SOFTWARE DEVELOPMENT POLICIES TO MODERN TECHNOLOGY

there is no Air Force or AFSC program ever, they do have important technicalin operation that can meet that need aspects. For example, many commercial-soon enough. ly available software tools and research

results are slow to be used in the AirWe recommend that experts now In the Force acquisition process. Compared tocommand be identified and that some of technologies believed to be important tothem be organized into one or more the Air Force's technological "edge,"software systems engineering *advisory software technology is underfunded by ateams" based on knowledge and exper- factor of 2 to 4; this contrast should belence. These individuals should be used addressed. We recommend using SEEs asat key stages before, during, and after primary vehicles for technology transfercontract awards involving significant of important software process and man-software components. Further efforts agement technologies. We also recom-should be made to identify and develop mend continuing to fund programmingsystem etgineers with strong software language evolution for VHLLs andskills and software engineers with application-oriented languages appro-system competence. Such individuals are priate to AFSC products.particularly important in AFSC, althoughthey are needed also in industry. These We urge the Air Force to establish aqualified individuals are a scarce production-funded program similar toresource; they should be Intensively MANTECH but for software productionmanaged through appropriate career technology pertinent to particularprogression and special monitoriug. systems or system classes. SoftwareMilitary software specialists should process technologies should be commer-expect a reasonable promotion path, say clalized since that would Improveto grade 0-6 after 20 years in service, available tools at less cost to the%ith opportunity for general officer government. We suggest that thepromotions, software technology base be expanded at

the fastest rate it can sensibly proceed,Software Technology. We agree with over a period of five years or more, toearlier studies that software acquisition bring It Into closer parity with otherproblems are largely managerial; how- critical technology areas.

CHAPTER ONE @ INTRODUCTION 5

1.0 INTRODUCTION

1.1 BACKGROUND 0 evaluated alternative softwaredevelopment process models, and

Today computer software controlsthe operation of most weapons systems; 0 determined how adequate controlsten years ago it did not. Developing for testability, reliability, and sur-reliable software that is able to perform vivability can be made part of anits intended function has been a problem incremental development.since the advent of the digital computer.The Commander of the Air Force Sys- The committee followed a twofoldtems Command states that it is now the approach. First, we identified at leastmost serious obstacle to systems decv- 17 earlier software studies initiated byelopment. At the Commander's request, the Air Force, Department of Defense,the Air Force Studies Board established or General Accounting Office during thethe committee on Adapting Software last 15 years, including one 1984 AirDevelopment Policies to Modern Tech- Force Studies Board study of softwarenology in May 1988 to help the Air reliability. We reviewed these studies toForce devise new precepts of software determine why they did not lead to sol-baseline management so that software ution of software development problemsarchitects can design a more effective (see Appendix A, Previous Studies withand reliable product. The work of the Brief Summary of Recommendations).committee began in early June and endedin late August 1988. Second, we visited various Air

Force bases to hear briefings fromsoftware development managers about

1.2 STUDY APPROACH the deficiencies of current and past AirForce programs. In June, we visited the

To better understand the problems Electronic Systems Division at Hanscomin software development and find solu- Air Force Base to hear presentations ontions, we: software in command and control sys-

tems, and visited the Aeronautical"* reviewed recent software studies to Systems Division at Wright-Patterson Air

determine why they have not solved Force Base to hear presentations onthe software acquisition problem, software in aircraft systems. In July,

members of the committee met at" assessed some current and past Warner Robins Air Logistics Center to

development programs to identify learn some of the views of the personnelsoftware deficiencies, who maintain electronic warfare systems.

During a two-week summer session in" investigated ways that the software Irvine, California, the committee heard

user and developer can work more presentations by Space Systems Division,closely together in co-evolution of Strategic Air Command, Tactical Airthe task, Command, Air Force Space Command, Air

Force Communications Command, and" considered the role of prototyping STARS. The committee then wrote this

and evolution in the software report. (See Appendix B for the Scheduledevelopment process, of Committee Meetings.)

6 ADAPTING SOFTWARE DEVELOPMENT POLICIES TO MODERN TECHNOLOGY

Overall, the committee addressed transition to join integrated circuits as athree main areas: software development central technology for information andprocesses, the people needed to manage control of systems. This study providesand carry out these processes, and the new concepts for maintaining control oftechnology that underlies process evo- acquisitions during the early phases solution. Each area needs more attention that software architects can work effec-now that software is making its major tively.

CHAPTER TWO a BACKOKOUND 7

2.0 BACKGROUND

2.1 PERSPECTIVES the fact that most systems now con-templated or under development are

2.1.1 Introduction predominantly software-controlled.

The central subject of this report This situation is not accidental.is the acquisition of software in Air Software control affords degrees ofForce systems. Software is hardly a flexibility unknown in hardware-definednovel topic in Air Force-sponsored systems. Processes for software devel-studies. A short history of related past opment, modification, and testing muststudies and their major recommendations be made to operate reliably and afford-follows in Section 2.2.1. If computer ably under timely and responsive config-hardware and software technology were uration control. When these processesstagnant fields, there would be no need are mastered, the Air Force will possessfor a new study. But if one field unprecedented abilities to respond tochanges, the environment of the other is threat changes or uncertainties and tochanged. Hardware has evolved at a incorporate new technologies (softwarevery rapid pace that has been unimpeded or hardware) with speed and confidence.by predictions that software may bereaching its limits in advancement. Soft-ware technology, as we shall see, has 2.1.3 Software Research and Develop-also evolved in several important direc- ment Investmenttions. Perhaps most important, theobjectives that software will be expected Until the 1960s, software was ato meet have recently increased dra- small part of system research and devel-matically. The Air Force Systems Corn- opment (R&D) cost, reflecting the smallmand's traditional roles are changing, amount of software in use. Many digitaland its Air Force clients (Air Force systems of that period were "hard-wired"Logistics Command and the using corn- and required no software. Today, how-mands) are changing also. Amid such a ever, for all but the simplest digitalpattern of major change there has been functions in electronics, software controlno difficulty in finding new challenges, is preferred. This approach reflects not

only the ability to alter details orparameters late in development or during

2.1.2 Software Is Ubiquitous field operation, but also the far greaterpower of a stored-program controlled

Although application software has processor over hard-wired logic.been a requisite of Air Force computerutiliz.ation since its early involvementwith commercial computers, only very 2.1.4 Software = System Persinalityrecently have software controlled air-craft (e.g., the later blocks of F-15s and In earlier systems, the 'personality"F-16s) been added to Air Force inven- of the system was fixed by hardwaretories. The 8-12-year or longer acqui- design decisions; in modern systems itssition cycles that have become charac- main principles are defined by software.teristic of the Department of Defense The many modes of a modern airborne(DoD) in peacetime has tended to mask attack radar, for example, are defined

8 ADAPTING SOFTWARE DEVELOPMENT POLICIES TO MODERN TECHNOLOGY

solely by software. The F- 16 system's operate within that subsystem).ground-attack and air-attack modes areselected in its central computer and Given intelligent integration ofdefine operation of both digital and subsystems that are designed to operateanalog subsystems. reliably in a closely integrated system,

integration of software -coutrolled sys-With control of the system oper- ýems ultimately may be simpler than

ation has come increased dependence on intagration of older hardware-intensivesoftware. The most important system systems because most subsystem testingdecisions often map directly into soft- can be done by simulation of the rest ofware decisions. Individuals responsible the system. We are, however, far fromfor a system's software must participate that state of affairs, in part becausein the system-level design considerations. software developers (and others) lackSoftware has become the principal in- knowledge of the physics associated withstrument for implementing the design, the system operation. Incomplete inter-from the top level down to small details. face definition and other shortcomings

further complicate the simulation.

2.1.5 Integration of Software-IntensiveSystems 2.1.6 Important Changes In Computer

and Digital Systems ArenasTraditionally, complex Air Force

systems are integrated from many sub- The trend toward software con-systems. Attempts to reduce numbers of trolled systems is occurring in ansubsystems as a means of reducing sys- industrial environment that featurestem complexity can be only partially rapid increases in computer power andsuccessful, for none of the Air Force's available memory capacity, doublingindustrial suppliers is knowledgeable in capacity every three years, and reducingall aspects of flight control, propulsion cost so that the newer and betterand fuel management, navigation, target systems cost little more than theirdetection, weapon management, commun- predecessors.ications, electronic warfare, and thelike. Also, there is essential analog Although high-level programminghardware (seekers, engines, control languages (HLLs) have existed for moresurface actuators, etc.) whose integration than three decades, the small processinginto digital systems is understood only power and memory capacity of most ofby specialists. The net result is that the computers embedded in weapon sys-integration of the hardware (and soft- tems discouraged use of HLLs. As aware) products of several or many firms result, to this date most of the weaponwill still be required in major systems. system software maintained by the Air

Force Logistics Command (AFLC) is stillIntegration within a software- written in assembly (machine) languages.

dominated system, for comparable func- Air Force determination in the earlytional complexities, is no more difficult 1970s to require that the JOVIALand in some ways simpler than in a language be used to program all systemshardware-dominated system. Certain was premature. HLLs broke into Airtraditional principles still apply: make Force front-rank weaponry with thesubsystem interfaces as "clean' (logically modern J-3B version of JOVIAL in thespecific, unambiguous, uniform) as is BI and F-16. More recently, Ada waspractical, and align hardware and soft- selected by DoD as the HLL of choice.ware interfaces (that is to say, programs Despite a slow start on Ada by the Airdefining a subsystem in detail should Force, the Air Force and its contractors

CHAPTER TWO * BACKGROUND 9

have made the needed ideological Software for numerical analysis haschanges, and most perceive benefits been around so long that it draws littlethrough the use of Ada. research attention. This cannot be said

for symbolic (object, text, data base)After decades of computer industry analysis, because the human-machine

focus principally on computer hardware interface needs substantial improvementevolution, there has recently been a to permit it to deal with semanticallysignificant growth of emphasis on soft- complex problems. This area defines aware. This change has come about, in major area of software research, somepart, because of the relatively mature under the banner of AL.nature of conventional processor design,and the realization that highly paralleled The software technology of mostor concurrently executing processor direct importance to this committee'sassemblages require software advances if objectives is that related to thethey are to become more widely useful, development, testing, integration, and

modification of software -- in essence,Exploratory activities in artificial the manufacturing and testing tools for

intelligence (AI) represent one cutting software programs. Although some ofedge of software engineering. Al-based these are now traditional (e.g., coin-activities are underway that will make pilers, editors, debuggers, operatingimportant contributions to Air Force systems), others are not. There iscomputer application. The anthropomor- critical need for tools that allow systemphic character attributed to Al ("mach- and software developers to describeines that think like people") may have system operation in languages that areunintentionally misinformed some laymen unambiguous, complete, readable byas to the reasonable expectations for humans and computers alike, and thatnext-generation computers. The user- can express concepts of a computeroriented thrust of Al was, however, application in ways meaningful toappropriate for addressing human- application-area experts.computer interfaces, an area whereprogress previously had been quite slow.

2.1.8 Related Changes In System Design

2.1.7 Software-Related Technology System designers must respond tothe trend toward software-defined sys-

Software is unlike other major tems. The most important aspect of thecomponents of Air Force systems, trend, from their point of view, is thebecause it requires design and testing additional system integration that isbut no manufacturing, and because possible and will become desirable to"maintenance" means repair of design users.errors and software enhancements.

For example, in most military air-Software itself has no wear-out craft, real-time integration is a crew

modes. It reveals few parallels to task. Sensor data is presented in thefeatures of hardware that might be used cockpit, and the crew manipulates con-through analogy to improve it. The trols that actuate engine, airframe, andhardware in which it is used, however, other mechanisms. The introduction ofdetermines the ultimate performance many new electronic subsystems sincesoftware can attain, although perfor- World War II has greatly increased crewmance can be degraded by poor software workloads while providing greater firedesign. power and survivability. But crews have

been reduced while crew tasks have in-

10 ADAPTING SOFTWARE DEVELOPMENT POLICIES TO MODERN TECHNOLOGY

creased. Some subsystems are internally take over either of those areas.integrated so that the crew has only toturn them on. Nothing approaching Real-time programs characteristicmaximum subsystem interdependence-- of weapon system software are highdata from each sensor being used by unit-cost items. Because of time-many other subsystems, for example-- urgency of national security operations,has been approached in practice. This stringent real-time limitations willinterdependence is expected in the next continue to be placed on Defense soft-generation of aircraft. wa:e despite the increasing processing

power. This appears certain to keepIntegration must rely heavily on alive machine (assembly) language

software, since hardware lacks the requirements, even where most process-needed flexibilty to respond readily to ing can be done under less severe timingessential change., Integration must be constraints using Ada-based software.under the control of the overall systemdesigners. Integration also affects The issue of providing operatinghardware design since most integration system programs for DoD mission-criticalneeds represent additional computer computer resources has long beenworkloads and memory capacity inside or avoided or ignored. The purpose of anoutside the subsystems. Not only per- operating system is to coordinate andformance, but also testing will become organize the assignment of computersoftware-iptensive. Unless systems engi- system resources such as memory, stor-neers are adequately conversant with the age, and input/output. Commercialcapabilities and limitations of software, operating systems for large mainframesthis increased integration will not be lack the real-time qualities oftensuccessful. .Likewise, software devel- demanded by DoD, and there have beenopers must appreciate and contribute to no operating systems available commer-system design decisions. cially for most of the many small mil-

itarized computers DoD has installedduring the past two decades. Just as

2.1.9 Software Considerations in the important, hardware resources managedDepartment of Defense Versus the by DoD weapon system computers lackCommercial Software Market the interface commonality expected in

the design of commercial operating sys-The DoD is probably the largest tern programs. Thus, few computers

U.S. user of custom software. Aside assigned to time-critical DoD applica-from commercial computer programs used tions have enjoyed the design flexibilitymainly in administrative activities or by possible with a full-capability operatingpersonal computers, most of the software system.acquired by DoD is unique to particularDoD needs. DoD is also a large, though While the flexibility of operatingnot dominant, customer for commercial systems can (if proper care is lacking)software such as operating systems or lead to indeterminacies of performancedata base management programs. The unacceptable in time-critical DefenseU.S. work force of computer program- applications, more and more Defensemers is much more involved in commer- systems will benefit from the utility ofcial data processing and personal the features usually associated withcomputing applications than in DoD operating systems. Some briefersprograms. In commercial data process- showed mild interest in militarization ofing, the COBOL language remains pop- AT&T's popular, portable UNIX operatingular, and C remains popular in personal system. The designers of Ada anti-computing, while Ada seems unlikely to cipated that operating-system-like

CHAPTER TWO a BACKGROUND I I

features would be built into an "Ada critically important.environment" that (like an operatingsystem) would require hardware-unique Accordingly, AFSC managementparts. Although this software feature should not always expect its productdid not emerge as critical during our divisions to see software problems in thestudy, it could become so as all Air same light. It also follows that noForce systems grow in complexity during single procurement method or acquisitionthe next few years. strategy can possibly serve for such a

variety of situations. Thus, while inthis report we propose promotion of

2.1.10 Software In the Air Force incremental development and of proto-typing, neither should be recommended

Probably the most important con- as a universal practice.sideration for the management of soft-ware development in the Air Force is itswide variety of applications. 2.1.11 User Considerations

On the one hand, Electronic AFSC plays the role of smart buyerSystems Division (ESD) and Space in software and systems acquisition.,Systems Division (SSD) manage a large Traditionally, AFSC has included rep-number of software developments based resentatives of the users (for example,on commercially available hardware and present or former fighter piloto) in itsoperating systems operating in surface acquisition cadres or evaluation groupsor underground locations. Aeronautical to provide needed user input.Systems Division (ASD), Munitions Sys-tems Division (MSD), and SSD manage Today's more complex software ismany system developments involving both often expected to embody interests ofhardware and real-time software. Aside its users more broadly, includingfrom systems developed through AFSC, facilities for rapid modification ofcomputer installations serve administra- operational flight programs. Suchtive functions in all commands. For modification is only one area in whichsoftware developed within the Air Force, operating commands and AFLC main-AFSC is near the bot*gm of a list that tainers have strong interest. Theincludes at least AFLC, Tactical Air inherent flexibility of software is madeCommand, Strategic Air Command, and even more manifest by HILL program-Air Force Commimications Command. ming; user commands plan to take

advantage or the vaunted softwareAir Force commands and units view flexibility, and have the personnel to

systems and system functions from articulate :hose desires if not the in-several different perspectives (as user, place resources to carry them out.operator, developer, maintainer, and/orbuyer). AFSC is assigned the role of Even with extensive control mech-buyer for a large share of these systems. anisms, efforts to rapidly modify soft-Our assessment of AFSC's software ware without extensive testing usuallyproblems was complicated by different produce errors, and often disasters.perceptions of need by different Air Control mechanisms and testing alone doForce commands and units. Not surpris- not constitute an acceptable user-ingly, some of these views conflict. The reprogramming capability, especiallydesign considerations for a national where changes may be made indepen-strategic warning system are obviously dently in several combat theaters. Morefar different from those for an avionic robust approaches to providing mission-radar warning system, although both are and theater-dependent functions and

12 ADAPTING SOFTWARE DEVELOPMENT POLICIES TO MODERN TECHNOLOGY

parameters will need to be evolved. Other topics have developed only inrecent years, or have become significantonly in recent years. Thus, they have

2.2 PREVIOUS STUDIES been addressed only by correspondinggroups; included are (1) integrity, the

2.2.1 Discussion definition of which has currently becomeimportant in the context of computer

Appendix A is a summary of 17 security for information systems and isdocuments that were available and now just being threshed out; (2) reusablereviewed for the study. Some were software, which has begun to makewritten by study groups outside the Air technical sense and show realisticForce; others were sponsored by the Air feasibility just in the last few years; (3)Force Studies Board and the Air Force post-deployment lifecycle support, aScientific Advisory Board. Each of the subject with which the Air Force haslatter addressed software matters at had to deal for most of a decade butleast twice directly; in the case of the which has now grown to such magnitudeAFSB, both studies were 3-week summer that it is a major driver for softwarestudies. For each item, a brief summary concerns; and finally (4) tools andor paraphrase of the recommendations is environments for software development.given.

The tools-and-environment issueThe 17 reports are not a complete was addressed by the 1977 Heffron study

data base of everything written on soft- but lay quiescent for six years until theware in the last decade, nor do they Vick group singled it out for majorinclude many private studies for the Air attention. The long interval probablyForce that addressed software, nor do reflects the growth of the technicalthey reflect continuing interactions of foundation for the subject in the Adacomputer-informed people with Air Force community and other groups. The earlymanagers and officers. They do reflect identification of the issue probablya compendium of what study groups reflects the influence of the AT&T Bell(some of which were specifically char- Telephone Laboratories in the 1977tered by the Air Force) believed the activity. At the time, Bell Labs wasproblems and solutions to be. Impor- developing large and complex softwaretantly, the studies involved people who systems.knew the Air Force as an organizationand institution, and could hopefully Excluding the ones just noted, thebridge the gap between the insider six stalwarts that have been consideredenvironment and the outside world. by nearly every study are (1) definition

of requirements, (2) acquisition (pro-Table A-I in Appendix A sum- cedures), (3) the development process,

marizes the recommendations of all (4) personnel (quality and quantity), (5)studies that addressed the software issue policy (acquisition and management), anddirectly, and catalogs the recommenda- (6) management (at both project andtions by subject. A few topi'.s were not oversight levels). While it is dangerousaddressed by more than one study, but to work from an incomplete data base ofthese topics were either of specific documents, observations and conjecturesconcern at the time or were directed about each can illuminate the progressitems for the study group. These topics and the causes of the software problem.include STARS, the DoD softwareinitiative, and the DoD programminglanguage Ada.

CHAPTER TWO a BACKGROUND 13

2.2.2 Definition of Requirements sibility for acquisition of developmentsystems; that remains in the R&D side of

First noted by the Heffron study in the house. The R&D side, working with1977, the problem of defining require- AFSC and the other Joint Logisticsments lay dormant for six years until Commanders (AFLC and Army and Navythe Air Force Scientific Advisory Board counterparts), drafted and redraftedrevived it in 1983. While the problem standards, the latest wrinkle being theundoubtedly was recognized throughout concept of tailoring, that is, deletingthe interval, the lack of activity dozumentation requirements from DoD-probably reflected a gestation period in STD 2167A rather than modifying them.which ideas for trying to improve the Tailoring more broadly can includeup-front requirements process were changes of program phasing as well.developed. It is also possible that theAir Force needed the time to reallyunderstand software requirements, a 2.2.4 The Development Procwssmuch more detailed, tedious, and poten-tially trouble-making phase than for The development process washardware projects. addressed early in the 1978 Walden study

under Air Force Scientific AdvisoryBoard auspices, but it languished for

2.2.3 Acquisition about five years. That committeeapparently concluded that the Air Force

Also noted by the Heffror study, had not used contemporary softwareacquisition had intermittent attention by development methods already in use inother groups. Both DoD and Air Force the private sector, or perhaps the Airpolicies were changing through the Force could not compel its contractorsmid-1970s and mid-1980s, system (and to use such methods. The methodologytherefore, software) responsibility was that the study groups had in mindfragmented in the Air Force. The Air tended to center on software-orientedForce Comptroller held responsibility for research efforts but also on thenon-weapon system (i.e., non-embedded) commercial/industrial segment whereacquisitions, but there was also some quite different motivations influencediffusion of responsibility throughout the decisions and managers.major commands (MAJCOMs); the DeputyChief of Staff for Research and Devel-opment had responsibility for embedded 2.2.5 Personnelsystems. While personnel specialty codes(AFSCs) for computeroriented people Personnel is a chronic problem inexisted, career progression was not and out of the Air Force. This wasperceived as satisfactory. Also, Con- noted by the 1977 Heffron study, andgress was trying to sort out and then not again until the early 1980s. Inlegislate the acquisition of embedded and the interim, the Air Force had beennon-embedded systems. trying to improve its internal manage-

ment of scarce resources, but the issuesIn retrospect, the mid-1970s to identified in the most recent Brooks

mid-1980s seem to be ! period of trying study have persisted through the years.,and shakeout. The Air Force has estab-lished a focal point for computer and The overwhelming growth of com-for command, control, communications, puter-related activity throughout societyand intelligence (C3 1) at the Assistant has kept the pressure on the Air ForceChief of Staff level. But this organ- to retain its people. At the same timeization does not have primary respon- the increasingly pervasive computer lit-

14 ADAPTING SOFTWARE DEVELOPMENT POLICIES TO MODERN TECHNOLOGY

eracy and skill that younger personnel that any reasonably informed computerbring with them, plus the proliferation person would, if told that software wasof personal computers, have acted to in trouble, ideytify the six issues justmake otherwise "computer outsiders! into summarized. On the other hand, theinformed and experienced practitioners. diversity of the groups that produced

the relevant reports, the variability ofThe best efforts of the Air Force the individuals involved, the time span

to achieve an adequate supply of com- over which the studies were conducted,puter software and systems people have ana the changing context in which suc-kept the Air Force from falling hope- ceeding studies were done all collectivelylessly behind. In its own jargon, the argue that the six do indeed constitute aAir Force has barely managed to stay valid, although not necessarily complete,"on the power curve." characterization of the Air Force soft-

ware dilemma.

2.2.6 Policy On the other hand, it would not befair to conclude that the Air Force has

Of all the issues, policy has failed to respond to the recommenda-received the most persistent attention tions. While its response has appearedover the years, probably because any to some observers to be inadequate-study group will believe implicitly that (mostly because the software problemsthere must be something wrong with the still exist) many things have been done.rules of the game or the problems would Communications and computer peoplenot exist., The many recommendations have been consolidated into a commonreflect experimentation on the part of organization. Communications and corn-the studies to try out innovative ideas. puters officers have been consolidated

into a common set. A focal point forsoftware has been created at the Assist-

2.2.7 Management ant Chief of Staff level.

Curiously, the management issue Either the response of the Airwas not mentioned in the earlier studies Force either has not been vigorous oralthough the personnel issue was. sustained enough, or the problem hasEvidently, oversight management and escalated faster than the Air Force hasmonitoring was not considered as critical been able to react. At this point theas technical project management (which Air Force, and AFSC in particular, hasprobably reflected the emphasis on two choices: take additional actionspersonnel). Due to the increasing either on past remedial actions or oncomplexity of weapon systems and their new ones, or learn how to live with andsoftware and also because of the plan around the problem.magnitude of the projects, the impor-tance of management control on budget, In judging the last decade ofschedule, and performance surfaced as a studies and recommendations and the Airdemand for tools and procedures to keep Force response to them, one mustthe management hierarchy informed of remember how vigorously the softwareprogress and alert for incipient prob- aspects of systems have advanced inlems. complexity and scope; and one always

must keep in mind how demanding theAir Force has become for weapon and

2.2.8 Summary system capability that can be providedonly through complex software.

One might be tempted to argue

CHAPTER TWO a BACKGROUND 15

2.3 ADA, STARS, AND THE SOFT- (timing, memory, bus bandwidthWARE ENGINEERING INSTITUTE margins, etc.) is within the

proper bounds. BenchmarksAda, Software Technology for specific to the application are

Available, Reliable Systems (STARS), and needed.the Software Engineering Institute (SEI)are not topics of this study, but because 0 The relevant items in Chaptersthey are clearly related to this subject 13 and 14 of MIL-STD-1815we were asked to provide our views on (the DoD Ada specifications)them. are properly and efficiently

implemented. These chaptersconcern real-time directives to

2.3.1 Ads the compiler, input/output(I/O), foreign (non-Ada) soft-

Ada should be used in applications ware, etc. These features arethat have traditionally emplcytd pro- not now fully tested by thecedur,-oriented languages such as Ada Compiler Validation Capa-JO'N lAL, FORTRAN, and COBOL, or bility and are highly implemen-more machine-oriented languages. tation dependent.Fielded applications that have success-fully and productively used problem- The availability of quick reactionoriented languages such as Atlas for compiler support to provide fixes orautomated test equipment (ATE), LISP, temporary "workarounds." Com-and Prolog for Al and knowledge-based pilers will always have errors, butapplications, should not be forced to if corrections must wait until themove to Ada at this time. next "release" (usually handled at

6- to 12-month intervals), anEven where Ada is the appropriate immature compiler is not usable and

language, the following should be con- even a seasoned one may be ques-sidered before Ada is used on specific tionable in certain applications.programs:

The adequacy of the developmentThe availability of one or more team's skill and knowledge of Adaproduction quality compilers that and the compiler(s) to be used.meets the performance requirements Since Ada usually presents a severeof the program. The program "culture shock" to uninitiatedoffice and/or the prime contractor development teams, learning andshould have adequate benchmarks using Ada for the first time is tooto determine whether. big a risk to take for a critical

system development. Contractorsa Time and table limitations are should demonstrate the proficiency

compiled that can support the of the specific team(s) to be usedsize of the sy-tem, the size of before award.the software teams, the designand implementation strategies, Further, the design approachand any other aspect critical to (e.g., use of special languagethe success of the development features such as tasking, exceptioneffort. We recommend the handling, and generics) should bestated limits be verified with clearly established and followedthe benchmarks. during the design. Also, designers

and programmers must have ano The real-time performance understanding of the real-time and

16 ADAPTING SOFTWARE DEVELOPMENT POLICIES TO MODERN TECHNOLOGY

memory implications of their source software arena as well as that of aero-code. The use of benchmarks or space software.instructors from the compilervendor can help serve this under- The forty-funded software technol-standing. Quality assurance teams, ogy studies began in 1987. Currentarmed with automated tools, can plans call for them to be folded into thehelp maintain consistency with the competing primes activity. Each studydesired approach, as well as involves advanced computer scienceidentify possibly wasteful use of technologies that are all ultimatelyresources (such as timing and focused on needed Ada-oriented applica-memory). tions, many of which are Air Force

applications.The experience and competence inAda of the System Program Office 'Shadow" programs are a daringpersonnel in Ada. This knowledge initiative aimed at simultaneously over-is necessary to insure that the coming resistance to Ada and developingabove issues are properly handled productivity metrics to quantify theand the risks are adequately benefits of Ada. The concept is to fundattended to by the contractor. modest parallel developments in ongoing

development programs planned before theadvent of Ada, to introduce Ada skills

2.3.2 The STARS Program and benefits.

The STARS program, DoD's cor- AFSC procurements should event-porate software technology program, has ually benefit to some degree from therecently sharpened its goals and imple- well-meshed STARS program combination.mented them in terms of three parallel Benefits should be broadly based, with aparts: wide range of software engineering and

computer science initiatives of interest"* "competing primes," to AFSC. All are Ada-oriented, a cor-"* (forty-funded) software technology rect and useful orientation for the times.

studies, and STARS is a program aimed at significant" "shadow" programs., long-term payoffs; its results should

become noticeable in the early 1990s."Competing primes" is the center-

piece of the STARS program. Threeprime system contractcrs have been 2.3.3 Software Engineering Institutefunded, in parallel and competitively, tobuild Ada-oriented SEEs particular to The Software Engineering Institutethree specific application areas that (SEI) is a recently established FFRDCinclude advanced tool sets supporting the (federally funded R&D center) that,software development cycle with particu- along with MITRE and Aerospace Corp-lar emphasis on integrated design sup- oration, can be used to improve AFSCport. Each contractor is to maximize acquisitions. SErs principal role is tothe use of subcontractors, with the aim improve and hasten transition ofof bringing the leading software tech- software technology from its creativenology groups into this competition, thus beginnings to direct use in acquisitionswidening the program's influence out leading to lowered costs, quickerinto the software industry. The strategy schedules, and reduced latent errors.also emphasizes the use and development Like the other DoD activities, SEI isof commercial software tools with the pledged to the proliferation and successintent of influencing the commercial of Ada.

CHAPTER TWO a BACKGROUND 17

Later in this report we recommend process models is the leading researchthat AFSC (I) compel their software effort in this important area. Its toolssuppliers to use software prototypes to for the evaluation of potential softwaredemonstrate architectural design worthi- contractors (developed jointly withness, (2) enforce use of SEEs, (3) dev- MITRE) are also outstanding and willelop its in-house personnel resources, (4) likely play a major role in upgradingaggressively tailor acquisition regula- acquisition practices. The SEI is a newtions, (5) bring the user into the institution that is well positioned to helpprocurement earlier, and (6) improve AFSC improve its acquisition practices.management of software by installing AFSC must, however, establish channelsand using an appropriate process model into SEI and its DoD taskmasters thatfor each acquisition. In one way or will allow AFSC concerns to be under-another, SEI is working on all of these stood and SErs results to move easilyrecommendations. Its work on software into Air Force procurements.

rl ... .... , .. ......

18 ADAPTING SOFTWARE DEVELOPMENT POLICIES TO MODERN TECHNOLOGY

3.0 STRATEGIES FOR SOLUTIONS

3.1 OVERVIEW these risks. Section 3.3 discussesgeneric ways to improve the acquisition

The difficulties of acquiring and development processes. Section 3.4software systems can be eased by suggests ways to improve software prod-addressing three major elements: the ucts. Section 3.5 focuses on ways toprocesses, the products, and the people. retain people and improve their skills.The processes in question are the Section 3.6 discusses, from the perspec-government's acquisition process and the tive of the technologist, software dev-contractor's development process. How elopment technologies, both emergentthese are carried out is a critical and available, that can improve softwaredeterminant of the risks and outcomes development.of product acquisition., The products inquestion are software and its accom-panying documentation. Software 3.2 RISK REDUCTION OBJECTIVESproducts, however, do not stand alone.They have meaning only in the context Risk reduction during softwareof the digital hardware architecture that development is a vital contributor tosoftware causes to operate, and in the procurement of complex, high-quality,functional and performance requirements software-intensive systems. Severalof the entire system in its operpting technologies - tools and techniques - areenvironment. Both processes and emerging thr offer considerable promiseproducts involve people -- multi- in reducing software development risks.disciplinary technical and multi-faceted The purpose of this study is to identifymanagement people. The skill, exper- valuable emerging technologies and toience, and communication ability of formulate a set of recommendations thatthese people in both industry and will bring taese technologies into thegovernment are critical to successful mainstream of software development ofsoftware development. Air Force Systems Command (AFSC) sys-

tems. Some of the software riskUnderlying the software arena is a reduction objectives addressed directly

fast-changing digital hardware technol- or indirectly in this report are:ogy and a complex (and sometimes con-fusing) software technology that is * Bridge the requirements design gap.rapidly evolving. These moving baselines * Improve user involvement in thefor system technology and its imple- requirements process.mentation in vastly different applications * Ensure continuity of designcontinue to keep software on the cutting expertise.edge but also make it prone to technol- 0 Design software for evolution.ogical risk. * Improve latent error reduction.

a Tailor contracting to softwareProposed strategies for dealing with characteristics.

current software development problems 0 Increase investment in softwareare presented in the following sections technology and advanced develop-of this report. Section 3.2 discusses ment.software development risks and someemerging technology that can reduce

CHAPTER THREE s STRATEGIES FOR SOLUTION 19

Software development of a large, search for solutions becomes clearer.complex, and unprecedented system is These uncommon people must be sup-bristling with risks. An unprecedented ported by two classes of tools andsystem is a type or class of system not techniques: (1) design exploration toolsbuil, before, or for which little or no that experimentally evaluate the con-design experience is available, or design sequences of design decisions, and (2)principles or technology are poorly support tools that relieve them fromunderstood by the system designers; any many tedious, mind-numbing tasks betterof these factors magnify acquisition risk. had'Aed by a computer. Both classes ofA primary risk encountered early in dev- tools • covered in this report, includedelopment is translating a well-stated set in discussions on prototyping, iterativeof systen-level requirements into a development, executable specifications,responi;ve software design. The ability and other development issues. A majorto make this ttep skillfully and quickly problem for AFSC is to make theseis vitai to successful software devel- needed tools an accepted and expectedopment. Many downstream difficulties part of software development, as well asand errors arise when this early step is compliant with AFSC acquisition policies.done poorly or too slowly. Yet in 1988 At the same time, AFSC must avoidno methodologies or useful formalisms needless rigidity in its tool specifica-existed for making this step; perhaps tions, since the tools used must supportthere never will be such automated aids software disciplines that are appropriatefor this purely intellectual task, although to the development program.we foresee aids to test the merits of adesign. A second major software procure-

ment problem that affects AFSC is theThe leap from system-level require- uncertain involvement with its user

ments to software design is made by community. This problem is twofold:human beings who perform it more or first, software is that part of the systemless well in proportion to their creative that interfaces with the user, giving thespark. Furthermore, they must make system its perceived operational qualit-their most critical software architectural ies. Design of these qualities is bestleaps without benefit of science, math- done in conjunction with the user, start-ematics, or much help from engineering ing long before system developmentprinciples. All system development begins and continuing throughout theteams need an "architect" who can suc- development and testing phases. Second,cessfully bridge this gap between sys- when the procured system is transferredtems phenomenology and software design from AFSC to Air Force Logistics Coin-architecture. When no such person is mand (AFLC) and the using command, apresent, or when this essential gap is better hand-off is required than existsbadly bridged, great tension arises today. The present methods, which arebetween system design groups and soft- based largely on formal transfer ofware groups. In practice, a tactic for documents and code, probably will notcovering the absence of an adequate work if one-million-lines-of-code-plusarchitecture is to schedule a lot of early systems must be rapidly modified in the(before preliminary design review) depot or field to support post-1995 wardeliverables, giving an appearance, but fighting doctrines.not the reality of progress.

A third element in reducing soft-When it is recognized that this ware development risk is continuity and

requirements-design gap cannot be maintenance of technical understandingbridged by methodologies or techniques and expertise throughout the full lifebut only by skilled, creative people, the cycle of a system's development. This

20 ADAPTING SOFTWARE DEVELOPMENT POLICIES TO MODERN TECHNOLOGY

period extends from the establishment of contrast to the requirements-design gapthe requirements to the successful intellectual problem, mathematically-implementation of the operating code. based or logically-based formalisms areUp to now maintenance of continuity emerging that can improve our ability toand expertise has been attempted rid systems of latent errors and enablethrough a sequence of documents that, increased quality, reliability, productwhen stacked end to end, could be integrity, and safety in software prod-measured in tens of feet. No one was ucts. Section 3.6 discusses some ofpleased with this process, but there were these topics.no better alternatives. Today there arebetter alternatives and the concept of Present methods for system con-transferring software design information tracting owe their origins to hardware-through hard-copy documents should give dominated procurements; generally theyway to more effective procedures using are poorly suited for many softwarecomputers to search for and present procurements. For good results, soft-desired information, ware procurements need early phases in

which design exploration, includingDesigning software that can evolve exploration of ultimately bad ideas, is at

gracefully far into the future is another least tolerated, if not encouraged; incritical concern for AFSC. Current later phases, single-minded, focusedsoftware development methods, procure- productization typical of the processment approaches, financial reward sys- described in DoD-STD-2167 can theL betems, and political imperatives prevent carried out. A demonstration/validationbuilding software able to reach beyond (Dem/Val) phase followed by full scaleimmediate goals. If there were real development (the process often selectedperceived value in building software for for large weapons systems), is a good,evolution into the future and if the high level generic model that can beneeded technical qualities could be adapted to meet the special needs ofdefined, implementation of these changes software development.would greatly alter AFSC institutions.Although hardware may change greatly We want to emphasize that a soft-in kind as technology evolves, underlying ware risk-reduction phase is appropriate,system functions expressed through even for a software-based system notsoftware often evolve more through requiring hardware development or spe-addition than replacement. These cial integration. A planned presiminaryconclusions suggest that, given port- phase prior to full-bore software devel-ability to faster processors, software opment can represent ,he experimenta-may become a more reasonable vehicle tion necessary to arrive at good tech-than hardware for long-term system nical solutions.evolution.

Where software development accom-Due to their great complexity, panics hardware development for an

software products have inevitably suf- unprecedented system, the software risk-fered from latent errors discovered in reduction phase should begin as soon asthe field after formal delivery. The a system architecture has been proposedrelated initiatives of software quality, and should parallel the hardware risk-software reliability, software product reduction work. Development of soft-integrity, and software safety address ware typically does not begin untildifferent perspectives of this problem. hardware has been fully specified. As aCurrent approaches are largely ad hoc, result, trades can be made in only onedepending mainly on the creativity of direction: asking the software to com-the software designers. However, in pensate for hardware or architectural

CHAPTER THREE s STRATEGIES FOR SOLUTION 21

shortcomings. enforces. Aeronautical Systems Division(ASD) employs a Contractor Capability

Whatever processes and methods and Capacity Review during sourceare adopted, it is likely that the cost selection. Furthermore, ASD has evolvedper line of code will continue to a Software Integrity Program to provideincrease. No method known to us will disciplines for acquisition of software.reduce costs of new code that must deal Electronic Systems Division (ESD) hasnot only with today's problems but also evolved Software Reporting Metrics, thewith efficient parallel processing, secure Sof:ware Engineering Exercise (a con-processing, fault-tolerant processing, and tractor assessment test used duringmany other new software, hardware, and source selection), "greybeard" teams, redsystem concepts that will appear. If our teams, (see Glossary) and a Softwarerecommendations are correct, and if they Requirements Audit to address contrac-are implemented within AFSC software tors' requirements and managementdevelopment, as development costs processes.inevitably increase, the financial benefitswill be derived through greater prob- To be most beneficial, all of theseability of timely and successful system efforts must be elements of a coherentdevelopment, longer useful system management strategy which recognizeslifetimes, and fewer cost overruns, that many Air Force systems are on the

leading edge of technology, where thereare uncertainties and where one must

3.3 IMPROVING THE ACQUISITION continually trade off cost, schedule, andAND DEVELOPMENT PROCESSES performance risks. These tradeoffs must

be made in the context of a clearlyCurrent Assessment understood acquisition process and a

properly applied development modelAFSC is participating in several adapted to the needs of the particular

initiatives that represent useful advances development.against software-related acquisitionproblems. The SEI is making progress in Below is a discussion of the fam-its development of techniques to eval- iliar "waterfall model" for hardware anduate contractor software development software development. Included in thiscapability and to provide a needed presentation are the risks of using thissoftware engineering curriculum for the model, comments on the development ofcountry, The STARS program is moving requiaments and how they drive theto further use of Ada libraries and a process, acquisition management issues,"software first" development approach and life cycle software issues. Findingsthrough its "competing prime contractor" and recommendations are included.contracts. DoD-STD2167A, the revisedSoftware Development Standard, wasreleased in 1988 and provides rmore 3.3.1 The Waterfall Modelflexibility in the acquisition approach.A guidebook to help acquisition organ- In acquiring systems with majorizations apply this standard will soon be software elements, the Air Force typ-prepared. ically imposes a process referred to as a

"waterfall model", for both hardware andAFSC product divisions are also software development during the

working hard to lessen risks in software full-scale engineering developmentdevelopments. Every product division is (FSED) phase of the life cycle (Seemoving to use Ada so that it can benefit Figure 1). This process has yielded bothfrom the increased discipline that Ada successes and failures. Successes seem

22 ADAPTING SOFTWARE DEVELOPMENT POLICIES TO MODERN TECHNOLOGY

to be characterized by three significant in cost and schedule estimates withoutelements. These are: first taking the following steps:

I. a stable set of system and software 1. Develop a team of systems, hard-development requirements that is ware, and software engineers whonot significantly different from that know the application area and thefor a previously developed system user's needs and who can com-or systems, municate wit* each other so that

each k,-s an understanding suf-2. a digital system architecture and a ficiently similar to work together

software design that will satisfy on the same system.the requirements,

2. Develop a digital system archi-3. contractor's systems engineering tecture and a softvare design that

teams and software teams corn- can illuminate the unknowns andmunicate effectively with each identify the risks. For someother and have had previous appli- systems, it may be necessary tocation experience in developing a test an implementation of thesimilar system or systems. design in the real environment to

permit adequate risk reduction.A system with these three charac-

teristics is a precedented system. Many 3. Iterate the requirements and theof AFSC's procurements continue to design through trade studies andapply the waterfall model to unprece- user inputs to establish validated,dented systems not satisfying these cost-beneficial, system and softwarecriteria. Furthermore, these acquisitions requirements.are often established through firm fixedprice contracts, geared to schedules Given the satisfactory completionwhich would only be reasonable for of these activities and adequate contin-precedented systems. Schedule and uity of the engineering staff, one has amanpower prediction models are often reasonable basis for cost and schedulebased on data for completed programs estimation for subsequent, more formal(both precedented and unprecented, but implementation. The waterfall modelexcluding failed and cancelled programs). then becomes an appropriate managementConclusions based on these models are concept for the remainder of the devel-sensitive to changes in input assump- opment process.tions; since the inputs are uncertain forunprecedented systems, the credibility of Some application areas in AFSCpredictions is poor. have little difficulty with the waterfall

model, but only because they arestrongly precedented. These include

Risk Reduction for Unprecedented most training software (flight simula-Systems Using the Waterfall Model tors), satellite system software,

unit-under-test software for automiticThe conventional waterfall model test equipment, some air defense

used during FSED usually assumes min- systems, and conventional surveillanceimal risks. Therefore it does not systems. These classes of programsadequately identify system risks and encounter serious difficulties only ifuncertainty, at least not early enough to requirements reach too far beyond thosetrade and reduce these risks. Risks and of previous systems or if too fewunknowns in an unprecedented system application-experienced technical anddevelopment cannot be properly reflected management staff are available to the

I I ,,, |I'10. .... i~ i I

Ii Ii~i.Ii ilU U I I

No

• "'• _

, ® ,, .....--*\ 0l~i~r~i 1* .......

! '0

CHAPTER THREE m STRATEGIES FOR SOLUTION 23

program. development for its contractor. Forsuch systems it is important to verify in

RECOMMENDATION 1: Air Force Sys- advance that the requirements andtems Command, working with the Joint design can be achieved within knownLogistics Commanders organization, and controllable risks.should ensure that development modelsand accompanying rational alternativesto the waterfall model, based on risk Late and Risky Requirements Baselinereduction concepts, are included in theforthcoming Handbook 287 for DoD-STD- During FSED for unprecedented2167A, with supporting direction In AFR systems the Air Force contracts (often800-2 and 800-14. on a fixed-price basis) for software

whose requirements and design riskshave not yet been identified. Software

Other Development Models development requirements are allocatedand derived from operational require-

Table I lists several system sit- ments and system functional require-nations and suggested processes for ments only after contract award. Thedevelopment. For the models depicted in true cost of some requirements may bethe handbook, approaches to reducing unknown. The contractor's digitalrisk and trading off cost, schedule, and systems engineering activity may proposeperformance risks should be described, an overall system architecture tooThe presence of alternative models in ambitious to be feasible or affordable.the handbook should encourage tailoringof acquisitions so that risk elements are When hardware encounters diffi-properly addressed. In addition to the culty, software is a favored work-aroundhandbook itself, AFSC acquisition per- medium, because it is thought "flexible."sonnel should have training in tailoring. The software must accommodate a poorlyManagement oversight will be needed to defined and continually changing missionensure that intelligent compliance is environment. Overall, the softwaretaking place. requirements baseline for unprecedented

systems is completed too late in theprogram, without an adequate resolution

3.3.2 Development of Requirements and of risks and without an adequate basisDesign for estimating development cost and

schedule.Many applications in AFSC are

unprecedented, either because a new RECOMMENDATION 2: AFSC shouldsystem is so different that no one has ensure adequate software risk reductiondone it before or because the particular for unprecedented systems during aparticipants (user, acquirer, contractor, demonstration/validation (Dem/Val) ormanagers technical staff) on the project equivalent phase preceding full-scalehave not done it before. Examples development. For unprecedentedreviewed by the committee included the systems, AFSC should provide policyStrategic Air Command Digital Infor- guidance for compettlve two-phasemahion Network, which had the first procurements, such tl'at software risksmulti-level secure operating system; over are reduced to a practical minimumthe horizon/backscatter radar, which was before proposals are prepared for thethe first large, distributed system with following phase(s). For unprecedentedwhich the contractor team was involved; systems, AFSC should direct an Indepen-and the B-I B Defensive Avionics System, dent assessment of system and softwarewhich also was the first large software risks near the end of the Dem/Val phase

24 ADAPTING SOFTWARE DEVELOPMENT POLICIES TO MODERN TECHNOLOGY

or Phase I of a two-phase procurement, subsequent contract is signed. Thisas part of the basis for follow-on continuity of knowledgeable staff isprocurement decisions. For multiphased essential to risk reduction. One exampleprocurements, AFSC should direct that of how this might be donewould be tocontract mechanisms provide continuity define part of the FSED work to beof essential contractor skill and exper- accomplished during the Dem/Val con-tise between contract phases. tract, but not to be evaluated as part of

the competition for FSED. This stepSoftware requirements, risk iden- would eliminate the confusion and

tification, digital architecture adequacy, schedule delay of restarting the program,software design issues, and operating at the expense of funding additionalenvironment uncertainties should be redundant efforts during the sourceresolved during the Dem/Val phase of selection interval.acquisition. Digital system architectureand key parts of the software should be RECOMMENDATION 3: For key tech-benchmarked and prototyped to validate nologles in systems and application areasthe design and to assess its timing, where operational threats or require-memory sizing, and data bandwidth capa- ments change rapidly, AFSC should fundbility, In some cases a preliminary parallel technology programs at theimplementation of the complete real-time system level to foster a ready industrialsoftware component will be needed. base from which to compete single phaseSometimes the users will need to be system acquisitions.involved during these preliminaryactivities to ensure that the statedrequirements truly represent their needs. Changing Requirements

Where there is no Dem/Val phase, For applications such as electronica competitive two-phase procurement warfare, requirements continue to change(with down-selection at the end of the rapidly, so that a system design basedfirst phase) could be employed for on early requirements may not be ableunprecedented systems. Near the end of to handle new ones. For such importantDem/Val or first-phase procurement, an applications areas it is necessary toAir Force software systems engineering maintain a warm industrial base byadvisory team independent of the pro- funding technology programs that lookgram office should review the effort to beyond the component level to theensure that critical software issues have system level. Requirements and compat-been addressed and that risks are known ible designs for digital systems (includ-and acceptable; otherwise, the next ing the software) may well be imple-phase should not commence. mented, flight tested, and evolved by

several competing companies. When aIt is important to maintain con- new or modified weapon system is

tinuity of the contractor systems engi- needed, requirement-compatible designsneering and software engineering teams and skilled design teams will be availableinto the next, full-scale design phase. to compete and extend their productsMany key personnel will be lost to other for low risk implementation in systems.activities if the technical team remains This technique is followed for jet engineunsupported or without meaningful activ- development and could well apply to theity during lengthy source selections. digital system and software area for keyThe Air Force should structure acquisi- technologies and application areas.tions so that key technical personnel aregiven contract support and provided To summarize: AFSC can developuseful continuing activity until the requirements and contractors can assess

Ij -T

LLL

E~~ F90~0

1.V F'

F~ W,

E ~

CHAPTER THREE a STRATEGIES FOR SOLUTION 25

the risks and impacts of these require- 0 Changing requirements duringments in a single procurement phase for acquisition. Requirem