UNCLASSIFIED Copy 18 of 189 copies

AD-A197 038

IDA PAPER P-2018

GUIDELINES FOR TAILORING DOD-STD-2167AFOR SDS SOFTWARE DEVELOPMENT

0

LRobert J. Knapper

C-Cathy Jo Linn Z

FYTIC John SalasinIELECTE 0

i AUG 1 7 1988 %

0 February 1988

Prepared forStrategic Defense Initiative Organization (SDIO)

DLSTIBUTION STATEMEN4T AApprpved for public relea.i

LDitrution Unlimited

INSTITUTE FOR DEFENSE ANALYSES1801 N. Beauregard Street, Alexandria, Virginia 22311

i S , 8 1'7 ' UNCLASSIFIED IDA Log No. HQ 87-32194

DEFINITIONSIA pWU m Me l owla doments to M the r st of its work.

Rperot are nl mod sthldlWs ad most carelully csusldusd ploduclt IDA plillehe.Teynmally embody rmit @ major prIoecs which (ml hhve a dies bearn on de11 onailet maia programs. or (h) address Wus o1 sl ieat crmrn to the EecuilveBranch. the CmirmS and/or ths public, or (c) address ass that hav significant economiclmpl km. iDA Reput am reviewed by ouitsoide aels of exprs to ense their highquality and rslovncs to Ms problems stulied, ad ty are foloed by tlb Preou at of IDA.

PapersPapers normally address rulatvy redtd tnchalcal or policy lasm. They commlcata

No results of specil analyses, Interim po or phons of a tok, ad hoc or quick reactiwork. Papers ae reviewed t en- that they mast standards similar to thow expeclad of

Iread papers In polsslonal Jourmls.

Memorandum ReportsIDA M dun Reparbt w wed for the canvoesiad of the openers or th omnysts to

* sc~i eubstsnllve work doesInquick rPH sla dleta.d major Inleircfl techical uppedaclMivlet; to mae allabls prelinary and tatatfos results d4 anlyses or @4 wrdnilgroup ad panel Oetlv;lss to forward Iuermail that Is esstiall malyzed and uNva-

taled; o to make a ream of -usrne es, meetlgs. or brieftl, r of dae demloe Ithe courve of a Ivstmtlaf. Reow of Mimadem Reports Is silted to thidr contMetand Intended no.

The remalts of IDA work us also conveyed by brilags and IlonIal memoranda to spoNOmand thrs doeinated by the speons, whon apprprte.

-~~ The work rpte In this document was Ionductdwder contract MDA 90 84 C 003 forthe Departma of Doemese. The pliffatlon of tis IDA document dose not Indlicte ondrse-met by tho Department d Dolens, nor should theontns he consmued N reflecing tlbs

official podtw -4 tht agenc.

'This paper hoe bon revosd by IDA to msre thMt It mate high sandards of eg O l s,Subictlviy, ad tsund antcl mehodology and tha lo conclusion s from *A

* method ly.~I

V Approved for peblic release; distribution unllmitled.

S -.

I:'-,

SECURITY CLASSIFICATION OF THIS PAGE

REPORT DOCUMENTATION PAGE /-.P- /-// 7 3' 6Ia REPORT SECURITY CLASSIFICATION lb. RESTRICTIVE MARKINGS

Unclassified2a SECURITY CLASSIFICATION AUTHORITY 3 DISTRIBUTION/AVAILABILITY OF REPORT

Approved for public release - distribution unlimited2b DECLASSIFICATION/DOWNGRADING SCHEDULE

4 PERFORMING ORGANIZATION REPORT NUMBER(S) 5 MONITORING ORGANIZATION REPORT NUMBER(S)

IDA Paper P-2018

6a NAME OF PERFORMING ORGANIZATION 6 b OFFICE SYMBOL 7a NAME OF MONITORING ORGANIZATION

Institute for Defense Analyses IDA OUSDA- DIMO

6c ADDRESS (City, State, and Zip Code) 7b ADDRESS (City, State, and Zip Code)

1801 N. Beauregard St. 1801 N. Beauregard StreetAlexandria, VA 22311 Alexandria. Virginia 22311

8% Sa NAME OF FUNDING/SPONSORING 8b OFFICE SYMBOL 9 PROCUREMENT INSTRUMENT IDENTIFICATION NUMBERORGANIZATION (if applicable)

., Strategic Defense Initiative Organization SDIO MDA 903 84 C 0031

Sc ADDRESS (City, State, and Zip Code) 10. SOURCE OF FUNDING NUMBERS

SDIO/SYS/BM PROGRAM PROJECT TASK WORK UNIT

Room 1E149 ELEMENT NO. NO. NO. ACCESSION NO.

Pentagon, Washington D.C. 20301-7100 T-R5-42211 TITLE (Include Security Classification)

Guidlines for Tailoring DOD-STD-2167A for SDS Software Development (U)

S 12 PERSONAL AUTHOR(S)Robert J. Knapper, Cathy Jo Linn, John Salasin

13a TYPE OF REPORT 13b ITME COVERED 14 DATE OF REPORT (Year, Month, Day) 5i PAGE COUN

6 Final FROM_ TO 1988 February 53

6 SUPPLEMENTARY NOTATION

IS7 l SUBJECT TERMS (Continue on reverse if necessary and identify by block number)-" 17 COSATI CODES Software Engineering: Software Development; Military Standards and Specifications;

FIELD GROUP sU.RouP Data Item Description (DID); Software; Prototype; DOD-STD-2167A; Request forProposal (RFP); Tailoring; Software Design; Concept Formulation (Cl); InitialPrototype (IP); Advanced Prototype (AP); Full-Scale Development (FSD);

MIL-STD-1 2119 ABSTRACT (Continue on reverse if necessary and identify by block number)

IDA Paper P-2018 provides the SDIO with guidelines for tailoring of DOD-STD-2167A and related Data Item Descriptions(DIDs), including a suggested interpretation of the review requirements in MIL-STD-1521B for use as guidelines in developingprototype and production software. Although not approved for use, revision A of DOD-STD-2167 was chosed as the subjectsince it will supercede the current standard in 1987, and because it more adequately addresses modem software technologyideas. The recommendations in this paper are intended to be reflected in Requests for Proposals (RFPs) distributed to vendorsproposing to develop SDI software and in contracts let for SDI software aquisition.

• 20 DISTRIBUTION/AVAILABILITY OF ABSTRACT 21 ABSTRACT SECURITY CLASSIFICATION

*LNCLASSIFIED/UNLIMITEDO SAME AS RPT. 13DTIC USERS Unclassified

22a NAME OF RESPONSIBLE INDIVIDUAL 22b TELEPHONE (Include Area Code) 22c OFFICE SYMBOL

Dr. Cathy Ju Linn, IDA (703) 824-5520 IDA/CSEDoS.

.4f DD FORM 1473, 84 MAR SECURITY CLASSIFICATION OF TillS PAGE' • 83 APR edition may be used until exhausted

All other editions are obsolete

,: Se , ......... -

UNCLASSIFIED

IDA PAPER P-2018

GUIDELINES FOR TAILORING DOD-STD-2167AFOR SDS SOFTWARE DEVELOPMENT

Robert J. Knapper

Cathy Jo LinnJohn Salasin

February 1988

By

A.v:ict4-., rTy Codes

INSTITUTE FOR DEFENSE ANALYSES

Contract MDA 903 84 C 0031Task T-RS-422

-U A F

UNCLASIFIE

UNCLASSIFIED

TABLE OF CONTENTS

1. PURPOSE 12. SCOPE 13. BACKGROUND 1

3.1 SDI Software Needs 13.2 DOD-STD-2167A 23.3 Related Efforts 4

4. RECOMMENDED TAILORINGS 44.1 Additional Definitions 44.2 General Requirements Tailoring 54.3 Detailed Requirements Tailoring 104.4 Data Item Descriptions 104.5 Documentation and Review Requirements (MIL-STD-1521B) 10

4.5.1 Concept Formulation 114.5.2 Initial Prototypes 124.5.3 Advanced Prototypes 134.5.4 Full-Scale Development 14

5. CONCLUSIONS 14

" APPENDIX A. LIST OF ACRONYMS 15

:..*,APPENDIX B. Applicable Data Item Description Tailorings 17

B.1 Software Development Plan (DI-MCCR-80030A) 17B.2 Software Requirements Specification (DI-MCCR-80025A) 18B.3 Software Product Specification (DI-MCCR-80029A) 18

APPENDIX C. REFERENCES 19

APPENDIX D. DOD-STD-2167A (27 OCTOBER 1987 DRAFT) 21

~vN.L

.1V

0 UNCLASSIFIED

UNCLASSIFIED

AZ

THSPG NETOAL LN

4 viUNCLASSIFIED r

UNCLASSIFIED

PREFACE

This paper presents guidelines for tailoring the DOD-STD-2167A Software DevelopmentStandard to software that may be developed for the Strategic Defense Initiative Organization(SDIO). It is based on a preliminary examination of how DOD-STD-2167A might apply toStrategic Defense System (SDS) software in various stages of research and development. It ,provides general guidelines for use by SDIO, agents, and contractors.

The document, developed under Task Order T-R5-422, is part of IDA's effort to support theSDIO by providing analyses and recommendations for software activities. The document wasreviewed on 16 July 1987 by the members of the following CSED Peer Review: AudreyHook, Jack Kramer, Catherine McDonald, Sarah Nash, and Robert Winner. In addition, the draft 0version was externally reviewed by and comments were received from members of the SDIOcommunity.

0

r.-4

vii 0

~UNCLASSIFIED

Lac iis M 11 11 Ia Xv

* a

UNCLASSIFIED

a'.

0

viii

',

N0

viii IS UNCLASSIFIED

, ,UNCLASSIFIED

ACKNOLEDGENMENTS

The authors would like to express their thanks to the CSED peer review members, AudreyHook, Jack Kramer, Catherine McDonald, Sarah Nash, and Robert Winner for their time andassistance in providing helpful comments for the final paper. Additional comments andguidance in preparing the final paper were gratefully accepted from the CSED Editor, 0

, Katydean Price. CSED staff secretary Jayne Hoblin prepared the figures used in the paper.

Appreciation is also expressed for comments received from reviewers outside of IDA; DannyCohen - ISI, Albert Small - SRS Technologies, Lt Col Ted Mervosh - ESD Hanscom, MajorFrank Maressa - SDIO, James Tremlett - RADC, and LT Marcia Van Wye - USSPACECOM.

ix

U S

.4.

P

1.hi.

Ix S*" UNCLASSIFIED

,"-'--'-,<.. ."."-' -' , .. '-.",.'-."-'.,"-. -. -, -.- ,'-."." .'.'" -" "" "" "',"" "' ." ,'..'.'':, "-,.'" ' ". ',,'- "- ",.'-"",''- '- % :, ', " "-".".- .,'

UNCLASSIFIED

mI

9

i THIS PAGE INTENTIONALLY BLANK"'

tI

x

UNCLASSIFIED

Guidelines for Tailoring DOD-STD-2167Afor SDI Related Software Development P

1. PURPOSE 3

This paper provides the SDIO with a recommended tailoring of DOD-STD-2167A and relatedData Item Descriptions (DIDs), including a suggested interpretation of the review requirementsin MIL-STD-1521B (Technical Reviews and Audits for Systems, Equipments, and ComputerSoftware), for use as guidelines in developing prototype and production software. Althoughstill a draft, Revision A of the Standard was chosen as the subject of this paper since it willsupercede the current standard in 1987 and because it more adequately addresses modernsoftware technology ideas. This paper does not suggest that changes be made to the Standard,but rather should be viewed as an interpretation of the Standard. However, based in part on

recommendations derived from the initial draft of this paper and other work at IDA, somechanges in the Standard have been made. The guidelines in this paper are intended to bereflected in Requests for Proposals (RFPs) distributed to vendors proposing to develop SDI

* ' software and in contracts let for SDI software acquisition.

2. SCOPE

Section 3. of this paper outlines the software needs of the SDI and describes how those needsmay not be fulfilled with an untailored DOD-STD-2167A. (For reference, a copy of theStandard is found in Appendix D.) The specific tailoring recommendations for DOD-STD-2167Aare found in Section 4. and the applicability of MIL-STD-1521B reviews over the range ofpotential SDI software programs is discussed. Appendix B covers the recommendations fortailoring the associated DIDs. Some notes and disclaimers are in Section 5.

3. BACKGROUND

'- 3.1 SDI Software Needs

The software requirements of SDI Battle Management/Command, Control, and Communications(BMiC3) stretch beyond the current state-of-the-practice in performance, capacity, andadaptability. To meet those requirements in the future, software design and implementationwill need to be innovative and must evolve rapidly with the changing edge of technology.Ideas will have to be translated quickly into s ftware prototypes to demonstrate the validity

a of the technology they represent. In addition, libraries of certified' software componentsproduced acquired for SDI or other Government funded programs will be heavily used.

All aspects of software, from early-on studies of theoretical concepts expressed formally as

'Certified in this case refers to software components guaranteed to be free of 'Trolan Horses'

or other hostile entities and which are functionally, correct to their specifications.

UNCLASSIFIED

a'~_ S.' ,.r -, *~a4%., ebi~ Ir. 0* 16 %,;~ IN'~gy. N*,

-' UNCLASSIFIED

abstract equations, represented as finite state automata, etc., through to the development, testing,and deployment of entire systems, are actively being pursued today by the computingcommunity for use in a wide range of applications. Although interested in recognizing and 'U.

supporting emerging technologies, the SDIO may only fund software efforts that result inproducts expressed in executable software, which are applicable to and/or extensible intoStategic Defense System (SDS) subsystems. Ideally, but not exclusively, the software developedfor one phase of the program will be upwardly compatible with succeeding phases. That is,the final software product from one phase becomes the initial specifications for a succeedingphase. This differs from a more traditional approach of funding programs that do not rely onor do not take advantage of prior products as an initial baseline. BM/C3 potential softwareprograms will fall into four general phases:

1. Concept Formulation (CF) - where software is developed to asssist in provingconcepts, defining technology requirements, or for demonstrating the feasibility orappropriateness of technical approaches.

2. Initial Prototypes (IP) - where software is developed to provide an initialdemonstration of components in a system context to assist in defining interfaces

0 and in refining functional requirements.

3. Advanced Prototypes (AP) - where systematic, incrementally advanced, integratablesoftware implementations of the initial prototypes provide detailed functional andperformance requirements for system development.

4. Full-Scale Development (FSD) - where complete operational code, evaluated againstspecific full-scale engineering decision (FSED) criteria, is developed for real-worldapplications.

The SDIO classifies programs in the CF and IP categories as "technology" programs, whilethose in AP (and possibly a small number in IP) are classified as "experimental validation"programs. The important point in the classification scheme is not, however, the relationshipof the categories to a funding classification, but rather the uses to which the results of aprogram are to be applied. As a consequence, the requirements and the applicability ofmilitary standards will vary among the four categories.

3.2 DOD-STD-2167A

The Defense System Software Development Standard, DOD-STD-2167A (27 October 1987 draft),contains the requirements for the development of mission-critical computer resources (MCCR)software. DOD-STD-2167A is intended to establish "uniform requirements for softwaredevelopment that are applicable throughout the system life cycle." Although the Standard isintended for use through the entire life cycle and "is not intended to specify or discourage theuse of any particular software development method", it must be tailored to suit the specificcontract needs of individual programs. In general, the Standard provides an adequatemechanism for the software development process. Some parts however, require a clarification,

'.- additional emphasis, or a specific interpretation to support the software philosophies held by* the SDIO.

FSD software will make heavy use of reusable modules from non-developmental software(NDS) libraries of Government furnished software (GFS) components or certified CommercialOff-The Shelf (COTS) software. Procedures for developing, cataloging, storing, accessing, and

2

- UNCLASSIFIED

C.v

UNCLASSIFIED

controlling certified software components in reusable component libraries will be very complex.A hierarchy of attributes relating to the security certification, fault tolerance, and verifiabilityof each Library entry will have to be established and maintained through an intelligentlibrary management system to make effective reuse of software. A straight-forward top-down

- design approach would not naturally lend itself to constructing software from these reusablemodules. The reuse of software modules often requires an element of the bottom-up approachto design. The Standard supports reuse by requiring the contractor to consiaer using NDSsoftware (para 4.2.4). However, the specific requirement to exploite reusable software is notemphasized. Tailoring is needed to require development methodologies that apply or encourage

this practice.

Similarly, using the classic waterfall life cycle methodology within research activities isconstraining. The waterfall method is an unambiguous method for implementing productionsoftware, beginning with requirements and proceeding Linearly through specifications, design,

- implementation, and maintenance. With this method however, errors in requirements,specification, and design are not detected until the software is implemented. Finding errors solate in the life cycle adversely impacts not only cost but also delivery schedules. Errors inearly phases are especially common in development projects for systems that have not beenpreviously implemented and for which the requirements are not well-defined or understood. Toavoid this pitfall, the SDIO is employing a prototyping discipline whereby key portions of thesystem are developed to ensure the correctness of the system requirements and specifications.Errors detected in the prototyping efforts result in early corrections to the system specification, v

S- saving time and helping to keep costs within budget. The Standard supports the use ofprototyping by specifically requiring that the "software development process shall include thefollowing major activities, which may overlap and may be applied iteratively or recursively..."(para 4.1.1). Also, the contractor is required to use a systematic and well-documented softwaredevelopment method (para 4.2.1). Protoyping is such a method. However, there is no explicit

, requirement for prototyping. The SDIO promotion of a prototyping development methodologyneeds to be refelected in the tailorings.

The SDI software will be developed by a variety of organizations ranging from universities toaerospace firms. The application of DOD-STD-2167A's documentation requirements (para 4.6.4 -and Figure 2.) needs to be tailored appropriately. For example, the impact of a CF activityconducting technology feasibility studies might become reduced if the activity is mired informal documentation. It is reasonable that a minimum of formal documentation may be allthat is needed with an IP prototype. On the other hand, an FSD activity may not only beconstructing an operational product, as opposed to an experimental prototype, but would havewell-defined requirements and specifications to use as the required formal documentation.

V. Along the same lines, the formal reviews and audits as specified in MIL-STD-1521B are lessapplicable for CF and IP activities. The Standard supports this view by allowing the degree towhich formal reviews are required to be determined by the contract (para 4.1.2). This is C.-

,, relatively explicit and should not need further clarification through tailoring. However, some

suggested documentation and review requirements for various activities are shown in Section 04.4.-

9'. Finallv, the Standard does not explicitly require the use of Ada- and Ada program design

language (PDL) as stated in DoD Directives 3405.1 and 3405.2. Even if a waiver for use of alanguage other than Ada will be requested, the Standard needs to be tailored to reflect

40

', Ada is a registered trademark of the U.S. Government, Ada Joint Program Office

S3

UNCLASSIFIED-N N

"*~~ ,. 'C. ** ~ -. e' .. . C %

je e-. - -.- '.w

UNCLASSIFIED

conformity with DoDDs 3405.1 and 3405.2. In addition, the use of SDI Ada/Process DescriptionLanguage (SA/PDL) to describe the design of information processing elements of the SDS willbe required.

3.3 Related Efforts

In addition to applying military standards to the software development process as a whole,other efforts, such as for testing and evaluation, are underway to "standardize" specific aspectsof software development.

A Test and Evaluation Master Plan (TEMP) is being developed to provide guidance for theplanning, execution, and reporting of BM/C3 software test and evaluation (T&E) activities. ThisTEMP will differ from other TEMPs for large and complex software systems, such as theWorld Wide Military Command and Control Information System (WIS), in that this TEMPidentifies the need for formal notations to support the rigorous specification of testingrequirements as part of software requirements and specifications.

Since any system cannot be tested except against its specifications, T&E will be facilitated by iusing a formal notation that provides a grammar for ensuring complete specification of critical

*O software characteristics, their threshold values, and associated testing requirements.

4. RECOMMENDED TAILORINGS,.1**.

p.'. Tailoring is a cost-effective method for applying the requirements to meet the specific needs ofparticular programs. A well-tailored standard will avoid unnecessary activities during theacquisition and will thereby reduce the cost of the overall software development effort.

The tailoring of the Standard involves three general steps: tailoring the General Requirementsin Section 4., tailoring the Specific Requirements in Section 5., and finally tailoring theassociated DIDs. To facilitate an understanding of some of the tailorings, additional definitionshave been suggested for Section 3. All paragraph or figure references that follow are from the -Standard or DIDs. All changes within a paragraph will be denoted with change bars andhighlighted with boldface type.

4.1 Additional Definitions

, Additions to Section 3. DEFINITIONS.

.3.XX Development phases:

13.XX.1 Concept Formulation. Development of software to assist in proving concepts,defining technology requirements, or demonstrating the feasibility or appropriateness

* Iof technical approaches.I 3.XX.2 Initial Prototypes. Development of software to provide an initialdemonstration of components in a system context to assist in defining interfaces andin refining functional requirements.

I3.XX.3 Advanced Prototypes. Developwent of systematic, incrementally advanced,*lintegratable software implementations of initial prototypes to provide detailed

Ifunctional and performance requirements for system development.

I3.XX.4 Full-Scale Development. Development of operational software systems,a .

'~. 4

6 UNCLASSIFIED: .... ; ' -",, 4: *%.":i!: ' :,

eS. 'FL .ir

UNCLASSIFIED

undertaken based on an evaluation of prototype and study results with respect tospecific full-scale engineering decision criteria.

4.2 General Requirements Tailoring

Changes in Section 4. GENERAL REQUIREMENTS. 0

para 4.1.1 Software development process. ... The software development process shall include the

l following major activities, as applicable to the development phase (see Figures 2a-2d)and as specified in the contract:

I a. Preliminary Software Requirements Analysis 0

I b. Preliminary Design (Initial Prototype)

I c. Iterative Prototype Design Change/Refinement/Expansion

0I d. Refined Software Requirements Analysis

e. Detailed Design

f. Coding and Unit Testing0

g. CSC Integration and Testing

h. CSCI Testing

The last item "i", is deleted.•

|para 4.1.2 Formal reviews/audits ... Figures 2a through 2d illustrate the occurrence of' V formal reviews and audits for the four software development phases and shows the

-|relationship of the minimal set of deliverable products to baselines and the DevelopmentalConfiguration.

para 4.1.9 Software development library. ... The contractor shall document and implement 6

. procedures for controlling software and associated documentation residing within the SDL. Thecontractor shall also document and implement procedures for cataloging, storing,accessing, and controlling reusable software. The contractor shall maintain the SDL ...

I para 4.2.4 Non-developmental software. The contractor shall, to the maximum extentpossible, incorporate non-developmental software (NDS), such as reusable software that was

Iinternally developed or obtainable as Government furnished software (GFS), into thedeliverable software. The contractor shall document plans for using NDS. NDS shall be

Icatalogued, stored, and controlled through the procedures for the software devlopmentlibrary. NDS may be ...

5 0

UNCLASSIFIED

* UNCLASSIFIED

*.L~.

<t - n nwu Z- e.J "

LU 0.

UNCLASSIFIED

- - ---- -- - - - - - -

UNCLASSIFIED

hdd

si 0

T0

cz -0

-a 0

w

EE( ~L-r %f0o

0M

al itw2= 2--ac

o F~n E,6,2U

a:(.I

UC

cc _ 0

L UN>SSFE

~~u *c"j'* * *

I. t~UNCLASSIFIED

0 0-

_ _ _ _ _ _ _ _ _ _ _ _ _ !2

01-. c

cc [57f1

3 2 % M M O 0 Zo

000

z 1

;a 'UA 2

-

- e

0. 0 80C

LuL0 0

E.Lm w

2c 6

UL

M 1 0-w

8UNCLASSIFIED

A UNCLASSIFIED

0

-.- DI2

CLC

u *0

LL

sclP-A

hl.

t, .0c = Ina

L il9UNCLASIFIE

UNCLASSIFIED

para 4.2.7 High order language. The contractor shall use the High Order Language (HOL) Ada,Ias required by DoD Directives 3405.1 and 3405.2, to code deliverable software. 4* -,The last sentence in para 4.2.7, "If no HOL .. ", is deleted.

Ipara 4.2.8 Design and coding standards. The contractor shall document and implement the AdaI PDL and Ada coding standards to be used in the development of deliverable software. Inaddition, the contractor shall use SDI Ada Process Description Language to provide aformal description of the design for simulation and evaluation prior to thedevelopment of code. Software coding standards shall comply with the requirements specifiedin Appendix B. .,

4.3 Detailed Requirements Tailoring

Changes in Section 5. DETAILED REQUIREMENTS.

No changes.p .

V: 4.4 Data Item Descriptions

The DIDs associated with DOD-STD-2167A describe a concise and complete set of documents jfor recording information required by the Standard. As with the Standard itself, the DIDsmay be tailored. The Contract Data Requirements List (CDRL), DD Form 1423, incorporatedinto a .ontract identifies the data requirements that shall be developed as specified byapproved DIDs. Not every DID associated with the Standard is applicable to every contracthowever. Appendix B in this paper provides a tailoring of those DIDs requiring modificationto maintain consistency with the Standard as tailored by this paper. Applicability, tailoring,etc. of the other DIDs is contract dependent. In addition, there are items for which DIDs donot exist, e.g. the Program Report for CF activities. Preparation of draft DIDs for these itemsmay be considered as a future effort.

4.5 Documentation and Review Requirements (MIL-STD-1521B)

The SDIO will potentially fund programs at all four phases of development. As stated before,these programs are varying in their scope of activities and deliverable products. Therefore, it isreasonable that the documentation and review requirements across these four phases will alsovary. Fundamental research performing exploratory development of experimental prototypesmay not reach the stage of detailed design, while full-scale engineering development programs

p" will develop completely integrated, operationally tested products.

Specific requirements for documentation and reviews are individually contract dependent andthe guidelines for tailoring MIL-STD-1521B are found in Appendix J of that standard. The AStandard is very clear in paragraph 100.4.1, "The key to tailoring MIL-STD-1521 is to matchthe MIL-STD-1521 requirements against the details of the applicable SOW/Contractual taskrequirements." The following though, may serve as a possible generic solution or as a baseline

10UNCLASSIFIED

11 1 , 1 1 1 1 1 1 1 1 1

UNCLASSIFIED

for BM/C3 software programs. The mapping of the documentation and reviews listed for theIP and AP phases into the DIDs and MIL-STD-1521B is shown in parentheses. p

4.5.1 Concept Formulation

The requirements for CF phase programs are relatively simple. Developing software to proveconcepts or for defining requirements is not aimed at producing an operational implementationbut rather at developing technology into a feasible demonstration with minimal risks. None ofthe documents required by DOD-STD-2167A nor the reviews required by MIL-STD-1521B areapplicable. However, all SDIO funded activities are expected to generate products, either in theform of documents, code, or both. For CF programs the following will be required:

Documentation:

" Program Report consisting of textual documentation describing:

- Capabilities and functionality of the technology developed

Interface with other SDI activities

Benefits provided to SDI

SApproach used to develop this technology

- Test results from simulating the prototype

" Concepts Prototype Code, either as Ada, SA/PDL, or a specifically approved

language

Reviews:

* Concepts Prototype Product Review

4.5.2 Initial Prototypes

IP programs will develop the initial prototypes in SA/PDL, often using the technologydemonstrated in CF programs. Preliminary requirements will be derived directly from CF code.Test plans and procedures will also be developed: testing will be conducted and reported. AnIP program would need the following documents and formal reviews:

Documentation:

* Initial Prototype Requirements Specification (Preliminary Software Requirements

U L11 SI " UNCLASSIFIED

UNCLASSIFIED

Specification)3

* Initial Prototype Development Plan (Preliminary Software Development Plan)

$ Initial Prototype Design Document (Preliminary Software Top-Level DesignDocument)

* Initial Prototype Source and Object code

$ Initial Prototype Test Description (Preliminary Software Test Description)

, Initial Prototype Test Report (Preliminary Software Test Report)

Reviews:

* Initial Prototype Specification Review (Preliminary Software Specification Review)

* * Initial Prototype Design Review (Preliminary Design Review)

* Initial Prototype Product Review

4.5.3 Advanced Prototypes

Programs in the AP phase will involve systematic, incremental growth of IP prototypes andwill result in very nearly the implementation of a complete system or integratable componentcapable of handling real-world operational constraints in the simulation system. The systemdesign will be taken directly from the SA/PDL of the preceeding IP. Structural, functional,and interface organization and procedures will be documented in a software development plan.As the incremental prototypes are produced, tested, and refined, the final requirements and thefinal design documents will be produced. Test plans and procedures will also be produced;testing will be conducted and reported. An AP program should include the followingdocumentation and reviews:

* Documentation:

N. * Software Prototype Development Plan (Refined Software Development Plan)

* Software Prototype Requirements Specifications (Refined Software Requirements0 Specification)

* Software Prototype Top-Level Design Documents (Refined Software Top-LevelDesign Document)

k.,' Software Prototype Detailed Design Document (Software Detailed Design Document)

3Tltems in "( )" are the relevant DOD-STD-2167A DIDs

12 4UNCLASSIFIED

UNCLASSIFIED

* Software Prototype Source and Object code

* Software Prototype Test Description (Refined Software Test Description)

* Software Prototype Test Report (Refined Software Test Report)

Reviews:

* Final Software Prototype Specification Review (Refined Software Specification

Review)

* Final Software Prototype Top-Level Design Review (Refined Preliminary Design O

Review)

Final Software Prototype Design Review (Critical Design Review)

8 Final Software Prototype Test Readiness Review (Test Readiness Review)

Final Software Prototype Product Review

4.5.4 Full-Scale Development

A full-scale engineering development program will result in a completely operational, fullyintegrated, and thoroughly tested software system. The software requirements and detaileddesign will have been refined from AP programs. From this point, depending on the contract,it is reasonable to produce all of the documentation and prepare for all of the formal reviewsspecified in DOD-STD-2167A and MIL-STD-1521B by using the waterfall life cycle method asa model.

p 405. CONCLUSIONS

The 27 October 1987 draft of DOD-STD-2167A and 01 May 1987 drafts of the associatedDIDs were used in this paper. This version of DOD-STD-2167A has been forwarded by theJoint Logistical Commanders Joint Policy Coordinating Group on Resource Management to the

t--% Defense Product Standards Office for release. Any changes made to the Standard prior to thisrelease should not affect this paper.

As mentioned in the previous section, the requirements for documentation and reviews arecontract specific. The SDIO and/or the contracting agency should work with the contractor tocorrectly specify the appropriate documents and reviews needed.

The Standard, although tuned to more modern methods of software development, must still betailored to provide a structure for cost-effective software acquisition. The recommendations inthis paper are untried. However, a good example, along the same lines, of a completelytailored Standard is currently in draft form out of the System Integration Office of the U.S.

f-" Space Command.

13 0

UNCLASSIFIED

,1

11

.4 •-VNLSSFE

UNCLASSIFIED

APPENDIX ALIST OF ACRONYMS

ABM Anti-Ballistic MissleAP Advanced PrototypesBM/C3 Battle Management/Command, Control,

CommunicationsCDRL Contract Data Requirements ListCF Concept Formulation

-COTS Commercial Off-the-ShelfCSC Computer Software Component ,

CSCI Computer Software Configuration ItemDID Data Item DescriptionDOD Department of DefenseFSD Full-Scale DevelopmentFSED Full-Scale Engineering DecisionGFS Goverment Furnished SoftwareHOL High Order LanguageIP Initial PrototypesLLCSC Lower-Level Computer Software ComponentMCCR Mission-Critical Computer ResourcesNDS Non-Developmental SoftwarePDL Program Design LanguageRFP Request for ProposalsSA/PDL SDI Ada/Process Description LanguageSDI Strategic Defense Initiative

SSDIO Strategic Defense Initiative OrganizationSDL Software Development LibrarySDP Software Development PlanSDS Strategic Defense SystemSOW Statement of WorkTLCSC Top-Level Computer Software ComponentWIS World Wide Military Command and Control Information System

his

UCA II

p. %

'po)%

S

t15~UNCLASSIFIED

0

UNCLASSIFIED

i0

I TIS PAGE INTENTIONALLY BLANK

I

" .

..-

.-

. '

: ; ,¢1 60,NCAS•FE

UNCLASSIFIED

APPENDIX B0Applicable Data Item Description Tailorings%4"

B.1 Software Development Plan (DI-MCCR-80030A)

Changes in Section 10. PREPARATION INSTRUCTIONS

para 10.2.5.9 Software development library. This paragraph shall be numbered 3.9 and shall"" describe the software development library (SDL) to be used by the contractor for controlling

the software and associated documentation. This paragraph shall include a description of thep.' contractor's procedures ind methods for establishing and implementing the SDL, the contractor's

. laccess and control procedures for data stored in the SDL, and the contractor's procedures

I for cataloging, storing, and accessing reusable component software.

para 10.2.6.2.1 Software development techniques and methodologies. This subparagraph-U'* shall be numbered 4.2.1 and identify and describe the techniques and methodologies the

contractor plans to use to perform:

* I :a. Preliminary Software Requirements Analysis (including structured requirementsanalysis tools, techniques, or a combination of both)

I b. Preliminary Design (Initial Prototype)

I c. Iterative Prototype Design Change/Refinement/Expansion

| d. Refined Software Requirements Analysis

e. Detailed Design

f. Coding and Unit Testing

g. CSC Integration and Testing

h. CSCI Testing

I para 10.2.6.2.4 Design standards. This subparagraph shall be numbered 4.2.4 and shall-reference the Ada PDL design standards the contractor will use in developing the

Isoftware. This subparagraph shall also reference the SA/PDL used for processdescription.

The last sentence in para 10.2.6.2.4, "If the contractor ..." is deleted.

" I para 10.2.6.2.5 Coding standards. This subparagraph shall be numbered 4.2.5 and shallIreference the Ada coding standards in the contractor will use in developing thesoftware. Any planned modifications to the referenced coding standards shall be documented inthis subparagraph.

The second sentence in para 10.2.6.2.5. "If the contractor .. ' is deleted.

17UNCLASSIFIED

UNCLASSIFIED

B.2 Software Requirements Specification (DI-MCCR-80025A)

para 10.2.5.3.1 Programming language(s). This subparagraph shall be numbered 3.3.1 andI shall reference the Ada programming language required by DoDDs 3405.1 and 3405.2.to be used to implement the CSCI.

para 10.2.5.3.2 Coding standards. This subparagraph shall be numbered 3.3.2 and shall specifyIdirectly or by reference the Ada coding standards under which the CSCI shall be

implemented.

para 10.2.5.3.3 Compiler/assembler. This subparagraph shall be numbered 3.3.3 and shallIspecify the Ada compiler to be used to translate the CSCI implementation.''.

para 10.2.5.4.1 Design standards. This subparagraph shall be numbered 3.4.1 and shall specifyIdirectly or by reference the Ada PDL standards under which the CSCI shall be implemented.

B.3 Software Product Specification (DI-MCCR-80029A)

I para Compiler/assembler. This paragraph shall be numbered 3.5 and shall specify the Adacompiler used to translate the source code.

118

U S-I.'a

* 18~~UNCLASSIFIED -

UNCLASSIFIED

APPENDIX CREFRENCES

1. Defense System Software Development Standard, DOD-STD-2167A, prepared bySPAWAR, 27 October 1987 draft.

2. Software Development Plan, DI-MCCR-80030A, Data Item Description, prepared byNAVY-EC, 01 May 1987 draft.

3. Software Requirements Specification, DI-MCCR-80025A, Data Item Description,prepared by NAVY-EC, 01 May 1987 draft.

4. Software Product Specification, DI-MCCR-80029A, Data Item Description, preparedby NAVY-EC, 01 May 1987 draft.

5. Technical Reviews and Audits for Systems, Equipments, and Computer Software,MIL-STD-1521B, 04 June 1985

6. Computer Programming Language Policy, Department of Defense Directive 3405.1,'Under Secretary of Defense (Acquisition), 02 April 1987.

7. Use of Ada in Weapon Systems, Department of Defense Directive 3405.2, UnderSecretary of Defense (Acquisition), 30 March 1987.

8. Strategic Defense Initiative Ada Process Description Language, Linn, CJ., et al,IDA Paper P-1983, 16 March 1987.

9. Strategic Defense Initiative Software Test and Evaluation Master Plan (TEMP),Version 1.01 Draft, 20 May 1987.

i,*.

19 -UNCLASSIFIED

--

UNCLASSIFIED

7,7,

00

02

THIS PAGE INTENTIONALLY BLANK0

IIIS A

20 ,S N L SSFE

UNCLASSIFID

APPENDIX DDOD-STD-2167A (27 OCTOBER 1987 DRAFT)

toi

21 DRA~r

UNCLSSIFED m

UNCLASSIFIED

Ilk

,1-

*220, . UNCLASSIFIED

OOD-MT2167A27 0dobor 1967

3UPERSEDING

O00-STO-210674 June 1965

DRAFT

MILITARY STANDARD

DEFENSE SYSTEM

SOFTWARE DEVELOPMENT

A.j

AMSC NO. _____AREA MCCR

DISTRIBUTION STATEMENT A. Approd for public release:dfstrbution Is unlimitd.

D00-STD.2107A (DRAFT)

DEpARThIENT OF DEFENSE

Defense System Software Devlopment0

1. This AMlitary Standard is approved for use by all Departments and Agencies of the Department ofDefNse.42. Beneficial comment (recommendations, addition*, deletions) and any petno daawich mybeo

use in Improving this document should be addressed to: Commander, Space and Naval Warfare SystemsCommand. ATTN: SPAWAR -3212. Washington. D.C. 20363-5100, by using the sf-addressed '0Standardization Document Improvement Proposal (DD Form 1426) appearing at the end of this documentor by letter.

40

VI)1 .-

01

0004TD2167A (DRAFT) Wqv.w

FOEWR1. This sdu~ establishes uniform~ requirements for software development that are applicablethroughout the 5rptemf life cycle. The requirements of this Standard Provide the basis for Governmentinsight into a contractor's software develOpment testing, and evaluation efforts. This Standard istobused In conjunction with 000-STD-21M8 Defense System Software Quality Program.isab

Z Thi stndard Is not intended to specify or discourage the use of any particular Software developmentmesho4. The contractor is responsible for selecting software development mnethods that best support theact~eeent of contract requirements.

3.This standard. together with the other 000 and military documents referenced In Section 2f provdsthe means for establishing, evaluating, and maintaining quality In software and asS cated documentation.Guidance on the Implementation of this standard and Its relationship to thwe other documents isprovided In OOO.HOBK-287.

4. Dats Itemn Descriptions (010%) applicable to this standard are listed In Section 4L These DODtdcrbe a set of documents for recording the Information required by this standard. Production ofdeliverable data using automated techniques Is encouraged.

DOo4TD.21 VA (DRFT)

CONTENTS

1. SCOPE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.2 A picato o . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. .. .

1.2.1 System development ...................... 11.2.2 Software quality program ..................... 11.2.3 Firmwar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. .1.2.4 Software developed by Government agencies ..................

-; 1.3 Tailoring of this standard . .

2. REFERENCED DOCUMENTS ................................ 32.1 Government documents ................................. 3

2.1.1 Specifications, standards, and handbooks ........... ......... 32.1.2 Other Government documents, drwings, and publom ............ 3

2.2 Other publications ....................... ...................... 30 2.3 Order of precedence. . ............................. 3

-'" 3. DEFINITIONS ............................... ........ 5

4. GENERAL REQUIREMENTS .......... ................................. 94.1 Software development management ......... .......................... 9

4.1.1 Software development process ........ ......................... 94.1.2 Formal reviews/audits. .. .. .. .... ... ....... .. .. .... .4.1.3 Software development planning. ......................... 94.1.4 Risk management .......... ............................... 94.1.5 Security .......... .................................... 94.1.8 Subcontractor management ........ ........................... 94.1.7 Interface with the software IV&V agent ...... ...................... 94.1.8 Cost/schedule reporting ......... ............................ g4.1.9 Software development library. ................................. .114.1.10 Corrective action process ........ ............................ 11

. 4.1.11 Problemichange report ....... ............................. .11V 4.2 Software engineering ......... ................................. 11• 4.2.1 Software development methods ...... ......................... .11

4.2.2 Software engineering environment ...... ....................... .114.2.3 Safety analysis ......... ................................. .114.2.4 Non-developmental software ........ .......................... 144.2.5 Computer software organization ........ ........................ 144.2.6 Traceability of requirements to design ...................... 14

0 4.2.7 High order language ......... .............................. 140 4.2.8 Design and coding standards ........ .......................... 14

'C 4.2.9 Software development files ........ ........................... 14- 4.2.10 Processing resource and reserve capacity .................... 14

4.3 Formal qualification testing ................................. 16Z 4.3.1 Formal qualification test planning ......................... 150' 4.3.2 Software test environment ........................... 16

4.3.3 Independence in FOT activities.............................. ......

4.3.4 Traceability of requirements to test cases ...... .................... 16

v

CONTENTS - Continued

4.4 Sof ,4e product evaluations .............................."4.4.1 Indepndence in product evaluation activities .................. 16

4.4.2 Final evaluations ................................ 14.4.3 Softwae evaluation records ...........................4.4,4 Evaluation criteria ............................... 17

4.5 Saftwe configuration management .......................... 174,5.1 Configuration identflcato1 ......... ........................... 1745.2 Configuration control .............................. 174,5.3 Configuration status accounting ......................... 174.5A Storage, handling, anddelvey of proect ................. s4.5.5 Engineering Change Proposals ....... .......... 18 '.

4.6 Tr aftloning to software support ............................ 184.6.1 Regenerble and maintainable code ............................ ,s4.6.2 Transition planning .................. ................ is4.8.3 Software transition and continuing support ...... ....................

' 4.6.4 Software support and operational documentation ..... ................. I

. DETAILED REQUIREMENTS ......... ................................. 195.1 System requirements analysis/design ....... ......................... .19

5.1.1 Software development management ....... ....................... 195.1.2 Software engineering ......... .............................. 19 ,

5.1.3 Formal qualification testing ......... ........................... 195.1.4 Software product evaluations ........ .......................... 195.1.5 Configuration management ....... ........................... 19 "

5.2 Software requirements analysis ......... ............................. 215.2.1 Software development management ....... ....................... 215.2.2 Software engineering ........ .............................. 21 215.2.3 Formal qualification testing ........................... 215.2.4 Software product evaluations ........ .......................... 215.2.5 Configuration management ........ ........................... 21

5.3 Preliminary design .......... ................................... 235.3.1 Software development management ....... ....................... 23

_l,., 5.3.2 Software engineering ......... .............................. 23

5.3.3 Formal qualification testing ......... ........................... 23_ 5.3.4 Software product evaluations ........ .......................... 23 ,v 5.3.5 Configuration management ........ ........................... 23

5.4 Detailed design ........... .................................... 255.4.1 Software development management ...... .......................5.4.2 Software engineering ......... .............................. 25

% P-5.4.3 Formal qualification testing ......... ........................... 25'Vs 5.4.4 Software product evaluations ........ .......................... 25

5.4.5 Configuration management ........ ........................... 25-.. 5.5 Coding and CSU testing .......... ................................ 28

5.5.1 Software development management ...... ....................... 28

h. 5.5.2 Software engineenng ........ .............................. 285.5.3 Formal qualification testing ......... ........................... 285.5.4 Software product evaluations ........ .......................... 28

0 5.5.5 Configuration management ........ ........................... 28

0 vi

lem-zal ,Nr NS

DO0.S1D-21e7A (CRAFT)

CONTENTS -Continued

S.* CSC Intertdon and testing ................ . 305.0.1 Software development management ....................... 30

7.1 Software development management ....................... 325.7.2 Software engineering ....... .............................. 325.7.3 Formal qualification testing ...... ........................... 32

15.74 Software product evaluations ...... .......................... 325.7.5 Configuration management ................................. 32

5.8 Sytem integration and testing ....... ............................. 34 t5.8.1 Software development management ........................... 345.&2 Software engineering .................................... 345.8 Formal qualification testing ...... ........................... 345.6.4 Software product evaluations ...... .......................... 345.8.5 Configuration management ................................ 34

6NOFES. .. .. .. .. .. . .. . .. . .. . .. . ... ... ... ... . .. . .... 356. Data requirements list and cross reference ............................6.2 Subject term (key word) listing ........... ............................. 36

iNDEX ............... ............................................. 47

JA

W-0~

'

'" I.

S

I,..

;- vii

". * '' '' "--.r" ' " ' / \*"' " " -:' ... ", "- ',' ', """.

aar

* 0004TO2107A (DRAFT)

.y.CON4TENTS -Continued

FIGURES

I An example at system development r~w n Kt ............... 12 0 e products, evie, audits, and basellne ................. 123 E e of a system Nekdown d CSCI "a-n t .......... .1

5 Evaluation crteft for product at Sofware Requiremewf Analsi . .. .. .. . .. 22

a Evaluation criteria for products of Preiminary Design ................. 247 Evaluation criteria for products of Detailed Design ..... .................. 27a Evaluation criteria for products of Coding and CSU Teating .............. . . ..- 299 Ealuation criteris for products of CSC Integration and Testing............ 3110 Evaluation criteda for products of CSCI Testing ....................... 33

APPENDIXES

Append

A List of acronyms and abbrrAwtions ............. ............. 39B Requirements for software coding standards ......................... 41C Category and priority clasficationa for software problem reporting ............. 430 Evaluation Criteria. .. .. .. .. .. . ... ... .... ... ... ...... 45.

till

%#

VIl

D00OO.T21 67A (CRAFT)

1. SCOPE

* 1.1 £ a. ThS purpo of this stadtfd I to etaulsh requirem, to be applied during t

acquisition, deVsOpM11t, and support of softwart systems.

A" 1 ca This standard applies to the extent specified In the contract clauses, the Statement of

Work (SOW). and the Contract Data Requirements List (CORL).

1.2.1 Svgtem development' Application of this standard must be coordinated with MIL-STD-499,

Engineering Management, for total system development.

-. 1.2.2 fltware ouality ogarm. Application of this standard must be coordinated with DOD-SrD-2168,

- .- IDefense System Software Quality Program, to provide a complete software quality program.

1.2.3 Fw1s1a. This standard applies to the development or support of the software element of

" - firmware. This standard does not apply to the development of the hardwar element of flrmware.

1.2.4 SaMf M developed by Government aaencies. The provisions of this standard may be applied to

Government agencies. When a Government agency perorms software development or support In

accordance with this standard, the term "contractor" refers to that Government agency and the term

* "subcontrlctor rerem to any contractor of that Government agency.

"-:. t.3 T ,,_,dno of this standard. This standard contains a set of requirements designed to be tailoke for

each contract by the contracting agency. The tailoring process intended for this standard is the

deletion of non-applicable requirements.

0

1/,.

:1."

.f

A.

A..A,

-A"S

OO.STD-2107A (RAFT)

2. REFERENCED DOCUMENTS

2.1 G&MMeSM documens

2.1.1 Scecif atlons, standards, and handbooks. Unless otherwise specified, the following specifications,.- standards, and handbooks of the issue listed in that tisue of the Department of Defense Index of

Specifications and Standards (OODiSS) specified in the solicitation form a part of this standard to theextent specified herein.

-. MLITARY STANDARDS

:\ DOD-STD-2168 - Defense System Software Quality Program

0 STD480 - Configuration Control - Engineering Changes, Deviations, and Waivers

MIL-STD-481 - Configuration Control - Engineering Changes, Deviations, and Wavers (Short Form)

, MIL-STD-483 Configuration Management Practices for Systems, Equipment, Munitions, and'.." Computer Software

MIL-STO-490 - Specification Practices

MIL-6SD-499 Engineering Management

A MIL-STD-1521 - Technical Reviews and Audits for Systems, Equipments, and Computer Software

* 2.1.2 Other Government documents. drawings. and publicatfons. None.

(Copies of specifications, standards, handbooks, drawings, and publications required by contractors in- connection with specific acquisition functions should be obtained from the contracting agency or as

directed by the contracting officer.)

2.2 Other oublications. None.

2.3 Order of precedence. In the event of a conflict between the text of this standard and thereferences cited herein, the text of this standard shall take precedence.

Ko 7

0-3/4

o0004fft-2107A (ORAFT)

. - 3. DEFINITIONS

C.

3.1Sa D00TOOI-49A

3.2 Oilwminfaon by the Government that specification content Is acceptable.

"" 3.3 DM0113. See DOO-STD480.

3,4 Co.mp data definition. A statement of the charcteistit of the basic elements of informationoperged upon by hardware in responding to compute Inas tuctlirl Thawet may Include,bait we not limited to, type, range, strcture. and value.

:o 3.5 .mm' harf. Devices capable of accepting and storing computer data, executing a systematic" ' " sequence of operations on computer data, or producing cono aputs. Such devices can perform

s antal Interpretation, computation, communication, control, or othe logical flunctions

3.6 Qaoute.sources. The totality of computer hardwlare, software, personnell, documentation,

supples, and services applied to a given effort.

3.7 Comumr i ,mM (or software). A combination of masoclated computer ibutbucdons and computerdat definitions rmuired to enable the computer hardware to pedor omputtion or control funtions.

3.8 Comouter Software Comoonent (CSC1. A distinct pot of a computer software configurma item(CSCQ. CWs may be further decomposed into other CSCs and Computer Software Unit (CSUs).

3.9 Comoute Software ConfiguratIon Item (CSCIl. A configuration Item for computer software.

-3.10 ComOuter software documentation. Technical data or Information, Induding comouter listings andprintouts, which documents the requirements, design, or details of computer software, explains thecapabiliftes and limitations of the software, or provides operating instructions for using or supportingcomputer software during the software's operational life.

-: I 3.11 Comouter Software Unit (CSU). An element specified in the design of a Computer Software* "Component (CSC) that is separately testable.

3.12 Confiouratlon Identification. See DOD-STD-480.

3.13 Conflauratlon Item. See DOD-ST0-4S0.

3.14 Contracting agency. As used in this standard, contracting agency refers to the *contracting office"as defined in Federal Acquisition Regulation Subpart 2.1, or its designated representative.

3.15 Develoomental Configuration. The contractor's software and associated technical documentationthat defines the evolving configuration of a CSCI during development. It is under the development

I contractor's configuration control and describes the software design and Implementation. TheDevelopmental Configuration for a CSCI consists of a Software Design Document and source codelistings. Any item in the Developmental Configuration may be stored on electronic media.

3.16 Evaluatlg. See DOD-STD-2168.

1 3.17 Firmwar. The combination of a hardware device and computer instructions or computer data thatreside as read-only software on the hardware device. The software cannot be readily modified under

program control.I.'

*', .. ,, ..

DOO-STO-2167A (DRAFT)

&181 F.,M1 ,me-tion Tona MOT) A proem th the nonracing agency to dowermnewheye a configuration item compole with the alloci mquiement for that Item .

3.19 £.P DB. See OooTD48.

.- w ,e . A configuratdon Item for hardware.

3.21 Inde_,ndent Verification and Validation WI&VI. Verification and validati performed by a contrac-• -'+. tar or Government agency that is not responsible for developing the product or performing the activity .,

being evaluated. IV&V is an activity that Is conducted sepatl from the Software development +acivites governed by this standard.

,- 3.22 Non deel''entalsofwm INDS). Deliverable software th Is not developed underthecoract .ft

but in provided by the contractor, the Government. or a third prty. NOS may be referred to asreusable softwae, Government furnished software, or commercially available software, depending on its

I, PL ° source. .

6 3m2 Product Bsin. See OOO4TD-48.

3.24 Rhlease. A configuration management action whereby a pa atcular version of software Is made "available for a specific purpose (e.g., released for test).

- 3.25 Rmb e satwae. Software developed In response to the requirements for one application tha .

can be used, In whole or In pat, to satsy the requirements of another applicatIon.

3.26 Softwam develooment file (SDA. A repository for a ca perint to thedevelopment or support of software. Contents typically Include (either directly or by reference) designconuiderationa and constraints, design documentation and data, schedule and status Information, testrequirements, test cases, test procedures, and test results. 6

3.27 Sotwar development library ISDI). A controlled collection of software, documentation, anda ocated tools and procedures used to facilitte the orderly development and subeequent support of

software. The SOL includes the Developmental Configuration as pat of its contents. A softwaredevelopment library provides storage of and controlled access to software and documentation inhuman-readable form, machine-readable form, or both. The library may also contain management data

pertinent to the software development project,

3.28 Software enoineerina environment. The set of automated tools, firmware devices, and hardware -

necessary to perform the software engineering effort. The automated tools may include but are notlimited to compilers, assemblers, linkers, loaders, operating system, debuggers, simulators, emulators, test .-tools, documentation tools, and data base management system(s). 0

*"-. 3.29 Sofwar ggg The sum of all aotvities that take place to ensure that implemented and fielded W.software continues to fully support the operational mission of the software.

3.30 Software test environment. A set of automated tools, firmware devices, and hardware necessary totest software. The automated tools may include but are not limited to test tools such as simulation .-

0 software, code analyzers, etc. and may also include those tools used in the software engineenng en-vironment.

o 'o

3.31 System Soecification. A system level requirements specification. A system specification may be aSystem/Segment Specification (SSS), Prime Item Development Specification (PIDS), or Critical ItemDevelopment Specification (CIDS).

... .. . . .,S -.,.'

,,.'2-.

000,COT21 7A (tPAFT)

3.32 Alidglon. ThO procea of evalusting softwe to detemine compliance wit specified n mentL

3.33 y The process of evaluating the products of a given software development actvity to

detemne correWI5 and comnsistency with rempct to the products and standards provided as Input to

that actdty.

3.34 Veruig. An Identified and documented body of software. Modifications to a version of software

(reeulting in a new version) require configuration management actions by either the contractor, the

contracting agency, or both.

3.35 Deflnitions of acronyms used in this standard. SeO Appendix A.

' ,

I7/I

Sii

" 7/8

-' 5.

* OOO.SID21?7A (OAFT..

4. GENERAL REUIRnEMErq,

" 4.1 " -flhl--moMe.". t miniA02mern. The contactor shall perform software development managemenr*in compilrce wiM the following requirements.

"- 4.1.1 Software develooment process. The contractor shall Implement a process for managing the

development of the deliverable software. The contractor's software development process for each ,SCIsa be compatible with the contract schedule for formal reviews and audits. The software developmentprocess shall Include the following major activities, which may overlap and may be applied Iteratively or

"" a . System Requirements Analysis/Designb. Software Requirements AnalysisC. Preliminary DesigndL Detailed Designe. Coding and CSU Testingf. CSC Integration and Testingg. CSCI Testing.fh. System integration and Testing.

4.1.2 Formal rivlewsludits. Ouring the software development process, the contractor Shall conduct orsupport formal reviews and audits as required by the contract. Guidance on formal reviews and Mft isprovided in MIL-STO-1521. The relationship of the forma reviews and audit to software and hUdwaMdevelopment is shown in Figure 1. Figure 2 illustrates the occurrence of fomal reiews and alldf forsoftware and shows the relationship of deliverable products to baselines and the Developmental

b configuration.

4. .3 Software davlooment planning. The contractor shall develop plans for conducting the activities..- required by this standard. These plans shall be documented in a Software Development Plan (SOP).

Following contracting agency approval of the SOP, the contractor shall conduct the activities required bythis standard In accordance with the SOP. With the exception of scheduling information, updates to the

I. SOP shall be subject to contracting agency approval.

4.1.4 R isk management. The contractor shall document and implement procedures for risk management.The contractor shall identify, analyze, prioritize, and monitor the areas of the software developmentproject that involve potential technical, cost, or schedule risks.

4.1.5 Aeurit. The contractor shall comply with the security requirements specified in the contract.

~ .' 4.1.6 Subcontractor manaoement. The contractor shall pass down to the subcontractor(s) all contractual"-' requirements necessary to ensure that all software and associated documentation delivered to the

contracting agency are developed in accordance with the prime contract requirements. The contractorshall provide to the subcontractor(s) the baselined requirements for the software to be developed by the

subcontractor(s).

- 4.1.7 Interface with software V&V aqent(sl. The contractor shall interface with the software

Independent Verification and Validation (IV&V) agent(s) as specified in the contract.

4.1.8 Cost/schedule reoortina. The contractor shall prepare and maintain cost/schedule reports at the

I- CSCI level. The cost reports shall indicate budgeted versus actual expenditures and shall conform to the

Work Breakdown Structure (WBS) applicable to the development effort. These reports shall also indicate

to the contracting agency planned, actual, and predicted progress.

* 9

004O4I6?VA OATd

11

101I

000-STD-2167A (DRAFT)

4.1.9 Saftware dvelooment librfr. The contrctor shall estllish a software delIoent library (SD.).The cw r shall documnt and Implement procedures for controling software and associateddocumena-on residi g within the SOL The contrctor shall mainton tt SL for the duration of thecontrnc.

4.1.10 Corrective action process. The contractor shall document and implement a corrective actionprocess for handling all problems detected in the products under configuration control and in thesoftwwre development activities required by the contract The corrective aWon process shall comply

- ., with the following requirements:

a. The process shall be closedl4oop, ensuring that al detected problem ae promptly reported and'A.. entered Into the corrective action process, action Is Initiated on them, resolution Is achieved,status Is tracked and reported, and records of the proWlres am maintained for the life of the

'A- contract.

b. Inputs to the corrective action process shall consist of problem/change reports and otherdiscrepancy reports.

c. Each problem shall be classified by category and by priority. The categories and prortiesidentified in Appendix C shall be included in the category and pioriy clasificatons.

d. Analysis shall be performed to detect trends in the problems reported.

. e. Corrective actions shall be evaluated to: (1) vWfy that problem have been resolved, adversetrends have been reversed, and changes have been correctly Implemented In the appropriateprocesses and products, and (2) to determine whether additional problems have been introduced.

4.1.11 Problem/change reoort, The contractor shall prepare a problem/chnge report to describe eachproblem detected in software or documentatian that has been placed under configuration control. Theproblem/change report shall describe the corrective action needed and the actions taken to resolve theproblem. These reports shall serve as input to the conective action process.

4.2 Software enaineering. The contractor shall perform software engineering in compliance with the-- following requirements.

4.2.1 Software development methods. The contractor shall use systematic and well documented software

, .:-" development methods to perform requirements analysis, design, coding, integration, and testing of the-, deliverable software. The contractor shall implement software development methods that support the* formal reviews and audits required by the contract..'". A4.2.2 Software enaineerino environment The contractor shall establish a software engineering

%environment to perform the software engineering effort. The software engineering environment shallcomply with the security requirements of the contract. The contractor shall document and implementplans for the installation, configuration control, and maintenance of each item of the environment.4.2.3 Safjnly. The contractor shall perform the analysis necessary to ensure that the softwarerequirements, design, and operating procedures minimize the potential for hazardous conditions during theoperational mission. Any potentially hazardous conditions or operating procedures shall be clearlyidentified and documented.

... i' " '

'A * tJp'.~%'~

* 0004TD417A (OAAF)

wasPawnonamm man

* OPYWANI

PRIUMAI viw ocIumoTSWg~jpiCATII#U I1 011514111

gvswmI SOWANS

na"IT (TUIT p inif

SOPIWANII DUNEEM4I

WICIPICag"OU4 4PCTH

GIUVIIUW"XV

OILIIVULOPA

Z4

PLAN

Ams" castail PION REVIE

ALOOF"NA I JWC~l* *IUUU in..h~jruI AISLIN

FiGUE 2 Deiverbleprouct. reiew. aditl. ad bselnes

REVIE

* 12

O0O4TD-21?6A (OAAFr)

ANDWAN ClOFIWADER~r

LIIRATI@

SQUIRACO U0.11 T

o NOPNT TIMO WAREP~NA041111GRNAT1ONv comionami

* MAYII VNOONSUPPLUO trn 4w"M1 usctowo~

a. PUII TEM WIITtATO

SYSTEM ITRATION SA STING r0 INCMPORAS INO VILOPMNTAL

COWIGONPAGUNN

MAY 81 A.DOCLAG11!I

z PRIMll I~t WVCPPOATUQT3.~~~~~SS LINIALII EIIICTO

MAY 1 0111111111111) WIL PTIFYTM IGRTO 2. Delverbl Trdcs eiw.adt.adbslns-cniud

13~w~l

.~PRO U -

4.Z.4 NW R"11O

safwar 04M W d delverable software. The contractor shall dcurrent plan fo Lan N408 Noamay be incoorld by one contrcor wlthOA cOntractIng agency approval only it the NOS IS " '-" :docmesot" in scoardwftc with fth reqiromeit of thissawitTe otaedvlemr fa oNM) nee not ano the design coralderstlolR. coflstralnts6 or data. Incorporation of NOS s"s comply

wihttdata d9 requirements Inth ora

415Compuersfr oganaio Th onraor saldecompose adPariton each CSCI ntoC~ wSofwar Coponnts(OSCs) wad Computer SotaeUnit(SG In accordance with tie

development method(s) documented ;i the Software Devele~riont Plan (SOP). The Contractor shall ensure

and toofecCS and CS.Figure 3 presents a ifustratlon of a system breakdown adCCdecomrpositWon.

4.2.6 TDOeablty of reauirements to deelan. The contractor shall document the traceability of therequirements altocaed from the system specification to each CSCI, It Compute Software Components

* (CSCs) and Computer Software Units (CSUs), and fromi the CSU level to fth Software RequirementsSpcfton (SRSs) and Interface Requirements Specification (IRS).

4.Z?7 Hlgh oder language. The contractor shaill use the High Order Language(s) (HOLs) specified In thecontract to code the deliverable software. It no HOLIs l required by the contract, fth contractor shallobtin contractig agency approval to use a particular language.

428 Design and codina standards The contractor shall document and Implement design anid codingstandards to be used In the devolopme. of deliverable software. Software coding standards shal complywith the requiremfents specified in Appendix El.

4.2.9 Software development files. The contractor shaill document the development of each ComputerSoftware Unit (CSU), Computer Software Compoent (CSC), and CSCI In software development filies(SOFs). The contractor shall establish aseparate SOF for each CSUI or a logically related group ofCSUs; each CSC or a logically related group of OSCe; and each OSCI. The contractor shall documentand Implement procedures for establishing and maintaining SOFs. The contractor shall maintain the SO~sfor the duration of the contract. The SOFs shall be made available for contracting agency review uponrequest SOM may be generated, maintained, and controlled by automated means. To reduce duplication,SOFs should not contain information provided In other documents or SOFa. The set of SOFs shallinclude (directly or by reference) the following information:

a. Design considerations and constraintsb. Design documentation and datac. Schedule and status information

dTetrequirements and responsibilitiese. Test cases, procedures, and results.

* 4.2.10 ProCeasna resource and reserve Maacitv. The contractor shall analyze the processing resourceand reev requirements, such as timing, memory utilization, 1/O channel utilization, identified In thecontract and shall allocate these resources among the CSCIs. The allocation of these resources to a -

e CSCI shall be documented in the Software Requirements Specifications (SRSs) for that CSCI. The ~contractor shall monitor the utilization of processing resources for the duration of the contract andshall reallocate the resources as necessary to satisfy the reserve requirements. Measured resource ,.

utilization at the time of delivery shall be documented in the Software Product Specification (SPS) for ~each CSCI.

14

N. N

000-STD-2167A (DRAFT)

-.

iCOW' I

liiim

w " O U U M IV co =olt 2 )- c

o,,

I I4

4. Fam gUWMMn rtai Theuru or'~rr-sl WW,. conduct-~- 'GT of esch *.C an ft t opue

r#0o neuvln sse prvdb tcotnMgaec.Tecotatrs.TsgW

Softwana Ted Mari (S-TP)._ Folon =* ctinagn agenoval oftoSthe contr shaT

updda to w ST "VII be subec to cotatn agnc aprovalThecntato " contifntorTedl i~ anmeing the tet t Inv Csitso* aneg d ithos ther Inolvrnertn

Cbwhwconduiguration ~I tms.

tomprsefot. Th otaele niomntsalcmlywt wscuiyo h_____________Th e contractor shall docmneveIpemnllasfrO pans % for k configu frao

e3.1iromant shalfbe ton" to deonsratetha this Urnd. ow p5Mgn shM*n~ ajibedcunefe I h4.Ualifictinda sIn (FT atM Te)gnztcr ucinadprsrsosbefrh

Olgar T e ln (topermi ollowt cotecting ancy topamuse Ov e 81Wdo th conrafcorhallocoffc~vactin. he prsos coduciFO tde shail %A, bte xception of t schepduln ino %

4.3.4 Trcebpf ofj e suet to ctestin agny ra The contractor shall doduentiyn the trcaiit ffrequiremest I ( the otae tesureets ttivesressfing the (SRlw and thos eqtairvomeintegifiatin(IS)~ th oarer coigration ptials. tsidb ahldcs dntfe nteSfwr o

4.4. Software teaut evaironen The contractor shall conducts ealutoso eieal software tetearnmndo efrdocumeaiton The secife tneteionmenofthi sandrdi complac ith the seufyrqurmnsolotin

4.4.1 Indepndence in pouteaai act i~tle The organizations, functions, and persons rsosbefrfliln*rsosbefrfliln the evauao requirements of this standard shall have the resources, epniiiy jhet.d

repniiiy uhrtadorganizational freedom to permit obective testingio and to causeh Ihidn i thfcao 1 -intainadvrfcto fcorrective action The persons conducting PO ciiissal~o ethe ealaon wo dpreod thel

* notwabeth eonwhdeeoethpruc or are responsible for the product. This dossntpel ebr 1tes otrp egineembrroingdvlomn team from participating in thOT evlutins Rsonieliy.o

rhefuiment on the software rucmets ecvications (S)adInefcequirements salbasindndspecificionh S

Soteto (DevelTe ntact ShaO dcmnttiP)ce.lt n h T orec S4444

4.objetwre of eacta evaluation.Tecrco shall conducteevaluationsof deliverable itmiscetblfntares andfdosametto as secieed n sction wa deopehisda. n ncmlacewt holwn

requirofeaments.ar

4.4.3 Sofwednei rd evaluation rcdsThcotactort s.al Thpae oranitin fons ndp3snrsonseucftilig h evaluation pefre.We reqiemsaeeno itifstnad sharollm/havge t.reourcl e

* ~initiation and verfallo sevosfnu h corrective action. Thppronodcting The evaluation rof rduc shall b

th ffillen r thetrsotinae roduct alutoeqieed shall bemitndfothlfe assiged tandsecfet.

.46

Sotwr DeelpenTla SD)442EnLfauca ro osbitn ahdeieal tmt h otatn gny h rcoaco shl d-enlycodnt h tm ihaporaeognztos o ia vlain hobjective ~ ~ ~ ~ ~ ~ ~ ~~4 Pfec ia vlainsalb oesreta h eieal tmi cetbe. ntemso

rwrw~x,-u~r~ll I LI Pr J~., 'Y Wk W W W -T-TW VWV -V T~V rWj wn .. r ' '.rv

000-T2167A (DRAFT)

4.4.4 Ealua tania. The contractor shall evaluate the products Identified in section 5 agaInst he

evluaon citerla specified in Figures 4 through 10. Default definitions for the criteria are specified InAppndb 0. Thg =Mt may prps additionl criteria and alternate definitions for any of the

ciftft epecW.ed in Appendft 0. Additional criteda and altemriat definitions ar subject to contracting

_n pa

.5 Software configuration manaaement The contractor shall perform software configurationmanagement In compliance with the following requirements.

4.5.1 Conflouraon identification. The contractor shall document and Implement plans for performingconfiguration Identficaton. Configuration identification shall be conducted In accordance with the

Identification scheme specified in the contract. Configuration Identification performed by the contractor. shall accomplish the following:

Sa. Identify the documentation that establishes the Functional, Allocated, and Product Baselines, andthe Developmental Configuration.

b. Identify the documentation and the computer software media containing code, documentation, orboth that are placed under configuration control.

c. Identify each CSCI and its corresponding Computer Software Components (CSCA) and Computer

Scftware Units (CSUs).

d. Identiy the version, release, change status, and any other Identilication details of eachdeliverable item.

e. Identify the version of each CSCl, CSC, and CSU to which the corresponding softwaredocumentation applies.

f. Identify the specific version of software contained on a deliverable medium, Including allchanges incorporated since its previous release.

4.5.2 Configuration control. The contractor shall document and implement plans for performingconfiguration control. Configuration control performed by the contractor shall accomplish the following:

a. Establish a Developmental Configuration for each CSCI.

b. Maintain current c:opies of the deliverable documentation and code.

c. Provide the contracting agency access to documentation and code under configuration control.

d. Control the preparation and dissemination of changes to the master copies of deliverablesoftware and documentation that have been placed under configuration control so that theyreflect only approved changes.

4.5.3 Configuration status accounting. The contractor shall document and implement plans forperforming configuration status accounting. The contractor shall generate management records and ,,

status reports on all products comprising the Developmental Configuration and the Allocated and ProductBaselines. The status reports shall: %

L. Provide traceability of changes to controlled products. 4%

b. Serve as a basis for communicating the status of configuration identification and associatedsoftware.

17

* 0004STO2107A (CAAFT)

c- Seow a a vehicle for ensuring that delivered documents describe and represent the masciefd

4A5. MMMa tofing. and dlivery gi =leld mad The contractor shall document and Implementmetods and P r ures for the storage. handiling. and delivery o1 software and documentation. The Icontractor "ha maintain master copies of the delivered software and documentation.

4.5.3 Fanengn Change Proooals. The contractor shiall prepare Engineering Change proposals (CpsIn accordance with MILS~TO.483, DOD.S4TII-48, and ML4STD..41. The contractor sh" 9rpr -4Spe cificatin -Change Noticen (SCNs) in accordance with MIL-STD.48 and MIL-STD.480. pq

4.6 Trarauffeng to software suo The contractor shall provide transittion support in compliancwihthe following requirements.

4.6.1 Rilleermhle and maintainable code. The contractor shall provideO to the contracting agencydel~verable Code that Can be regenerated and maintained using commercaly available, Govemeont-

owed, or contractually deliverable support Software and hardware that has been Identified by thecontactng agency.7-

4462 TranaitIn oanr.-. The contractor shall Prepare plans for transitioning fte deliverable softwarefrom development to Support These plans shall be documented In the Computer Resource integr~ated

esupport Document (CR150). II

46.W Software transition and continuing sumart The contracto shall perform installation and checkdoutof tie dellverable software in the support environment designated by the conrti ng agency. The '

Contractor shall provide training and continuing assistance to the contracting agency's support activityas specified In the contract.

4.1.4 Software sunport and ocerational docuMentation. The contractor shall develop and deliver the -

following software support and operational documentation an required by the Contract Data RequirementsList (CORL):

V a. Computer Resources Integrated Support Document (CRISO)b. Computer System Operator's Manual (CSOM)C. Software User's Manual (SUM)d. Software Programmer's Manual (5PM)e. Firmware Support Manual (FSM).

Sa

DO-STD2167A (DRAT) .

f.,.

5. DETAILED REQUIREMENTS .

5.1 MMM wft jaahidlgian. The contractor shall perform the following system requirements 0analylu/delign eivlee.

5.1.1 Softwardevalooment management.

5.1.1.1 The contractor shall support the System Requirements Review (SRR) as specified in the

5.1.2 The contractor shall support the System Design Review (SDR) as specified In the contract.

5.1.2 Software eaineenna.

5.1.2.1 The contractor shall analyze the preliminary system specification and shall determine whetherthe software requirements are consistent and complete. 6

5.1.2.2 The contractor shall conduct analys4-m to determine the beat allocation of system requirementsbetween hardware, software, and personnel order to partition the system Into HWCI*, CSCI, andmanuh! operations. The contractor shall docurr.->' the allocation in a Systemr/Segment Design Document(MD).

5.1.2.3 The contractor shall define a preliminary set of engineering requirements for each CSCL Thecontractor shall document these requirements in the preliminary Softwe Requirements Specficalion(SRS) for each CSCI.

5.1.2.4 The contractor shall define a preliminary set of interface requirements for each external -interface of each CSCI. The contractor shall document these requirements in a preliminary Interface *Requirements Specification (IRS).

5.1.3 Formal aualification testina. The contractor shall define a preliminary set of qualification %requirements for each CSCI. The contractor shall document these requirements in the preliminary .Software Requirements Specification (SRS) for each CSCI. These requirements shall be consistent withthe qualification requirements defined in the system specification.

5.1.4 Software oroduct evaluations. The contractor shall perform evaluations of the following products.using the evaluation criteria specified in Figure 4:

a. The Software Development Plan (SOP)b. The System/Segment Design Document (SSDD)c. The preliminary Software Requirements Specification (SRS) for each CSCId. The preliminary Interface Requirements Specification (IRS).

5.1.5 Qgnflouraton management. The contractor shall place the following documents under configuration ,.control prior to delivery to the contracting agency:

a. The Software Development Plan (SOP)b. The System/Segment Design Document (SSDD)c. The preliminary Software Requirements Specification (SRS) for each CSCI.d. The preliminary Interface Requirements Specification (IRS).

'- 19•

___ __ ___ _r,_____ ____ ___ ____ ____'1 "./. a/./QWy

0004=4167A (OAFT)

;%:.3

2 • _I 1 m .11

[•I Iif

•* lii -JII[

in-.

UA CI)uI,

*1+. • 20

,4. tJ; +i, '.

S + j++: ] :-++I g.+

S +c d +

DODSTD-21 A (DRAFT)

5.2 S, f~i~wM _ri-,rements analysis. The contractor shall perform the following softwar requirements

5.2.1 Sf-. delooment manpaedent. The contractor shall conduct one or more Softwgr

Specification Review(s) (SSR) in accordance with MIL-STD-1521. Following successful completion of anSSR and when authenticated by the contracting agency, the Software Requirements Specifications (SRSs)and mociated Interface Requirements Specification (IRS) will establish the Allocated Baseline for the

CSCI&

5.Z2 Software enaineerin.

5.22.1 The contractor shall define a complete set of engineering requirements for each CSCI. Thecontractor shall document these requirements in the Software Requirements Specification (SRS) for eachCSCL

5.2.2 The contractor shall define a complete set of Interface requirements for each external interface* of each CSCI. The contractor shall document the" requirementa in an Interface Requirements

S(IRS).

5.2.3 Fonnal aualiflcatlon testina. The contractor sha define a complete set of qualificationrequirements for each CSCI. The contractor sa document thee requirements in the SoftwareRequirements Specification (SRS) for each CSCI. S

N.p 5..4 Software product evaluations. The contractor shal perform evluatons of the products identifiedbelow, using the evaluation criteria specified in FIgure 5. The contractor shall present a summary ofthe evaluation results at the Software Specification Review(s).

a. The Software Requirements Speclfication (SRS) for each CSCIb. The Interface Requirements Specification (IRS).

5.2.5 Conflaurstion management. The contractor shall place the Software Requirements Specification(SRS) for each CSCI and the associated Interface Requirements Specification (IRS) under configurationcontrol prior to delivery to the contracting agency.

21

M4 1, V..Sv

OO4TD.2107A (OAAF77

~itj lii0 4

SS

wL'

TV.

0OO8TD-2167A (DRAFT)

5.3 Preliminary desi. The contractor shall perform the following preliminary design ac*vte

5.3.1 Sa._, dMI_!,1m02 menaemrn . The contractor shall conduct one or more Preliminary Design

Review(s) (POR) in accordance with MILSTD-1 521.

5.3.2 Software enalneerna.

5.3.2.1 The contractor shall develop a preliminary design for each CSCI, shall allocate requirementsfrom the Software Requirements Specifications (SRSs) and associated Interface Requirements

Specifications (IRS) to the CSCs of each CSCI, and shall establish design requirements for each CSC.

The contractor shall document this information in the Software Design Document (S00) for each CSCI.

5.3.2.2 The contractor shall develop a preliminary design for the exemal interfaces of each CSCIdocumented In the Interface Requirements Specification (IRS). The contractor shall document this

information in a preliminary Interface Design Document (leD).

5.3.2.3 The contractor shall document any additional engineering Information generated during the

preliminary design process in Section 6 of the Software Design Document (SOD) for each CSCL. The

engineering information shall include rationale, results of analyses and trade-off studies, and any other

Information that aids in understanding the preliminary design.

53.2.4 The contractor shall establish test requirements for conducting CSC integration and tesng.The contractors CSC integration and testing shall Include steesing the software at the limits of its

specified requirements. The contractor shall record the test quiremenrt (directly or by reference) In

the CSC software development files.

5.3.3 Formal ouulification testing. The contractor shall identify the formal qualification tests to be

conducted to comply with the qualification requirements identified in the Software RequirementsSpecification(s) (SRSs). The contractor shall document this information for each CSCI in the Software

p. Test Plan (STP).

5.3.4 Software product evaluations. The contractor shall perform evaluations of the products identified

below, using the evaluation criteria specified in Figure 6. The contractor shall present a summary of

the evaluation results at the Preliminary Design Review(s).

a. The Software Design Document (SOD) for each CSCI

b. The preliminary Interface Design Document (100)c. The Software Test Plan (STP)d. The CSC test requirements.

5.3.5 Conflauration manaoement.

5.3.5.1 The contractor shall incorporate tne Software Design Document (SOD) for each CSCI into the

CSCrs Developmental Configuration prior to delivery to the contracting agency.

5.3.5.2 The contractor shall place the Software Test Plan (STP) under configuration control prior to

delivery to the contracting agency.

5.3.5.3 The contractor shall place the preliminary Interface Design Document (100) under configuration

control prior to delivery to the contracting agency.

B23

-fl4x

000-SID-167A (OPAn~1_____I__-_T_______A

-I".

-2

A 1 owI

In caccc c =iij_____ ____ I

0C V __ _S_

uj a 13 a

~~24

DOO-SD02167A (DRAFT)

S. y. The do shall peIdorm the follwing detailed design activities.

5.4.1 806 ( 01111810MO L MID ggDil The con5trcor shall conduct one or more CdW~ DeigReview(s) (COR) In accordance with MILS1 -1521.

5.4.2 Saflar aauin rina.

5.4.2.1 The contractor shall develop a detailed design for each CSCI, shall allocate requirements fromthe Computer Software Components (CSCs) to the Computer Software Unite (CSUs) of each CSCI, andshall establsh design requirements for each CSU. The contractor shaill document this Information in theSoftware Design Document (SOD) for each CSC.

5.4.2.2 The contractor shall develop the detailed design of the 0801 external Interfaces documented Int Interface Requirements Specification (IRS). The contractor shall document this information in theinterface Design Document (100).

5.4.2.3 The contractor shall document any additional engineering Information generated during thedetailed design process in Section 6 of the Software Design Document (SO) for each CSCI. Theengineering Information shall include rationale, results of analyses and trade-off studies, and any otherIncrmation that ads In understanding the detailed design.

5.4.2.4 The contractor shall establish test responhbilltle. tmt coe (in terms of Inputs, expectedresults, and evaluation criteria), and schedules for CSC Integration and testing. The con~atroe ShaWeo this Information (directly or by reference) In the CSC software deveopiment files.

5.4.2.5 The contractor shall establish test requirements, tes responsibilities, tesit cam (in terms ofinputs, expected results, and evaluation criteria), and schedules for testing all CSUs. The contractors:CSU testing shall include stressing the software at the limits of its specified requirements. Thecontractor shall record this information (directly or by reference) in the CSU software development files.

5.4.3 Formal auatificstion testino. The contractor shall identify and describe the test cases for theformal qualification test identifications in the Software Test Plan (STP). The contractor shall documentthis information in the Software Test Description (STD) for each C0C.

5.4.4 Software oroduct evaluations. The contractor shall perform evaluations of the products identifiedbelow, using the evaluation criteria specified in Figure 7. The contractor shall present a summary ofthe evaluation results at the Critical Design Review(s).

a. The updated Software Design Document (SOD) for each CSCIb. The updated Interface Design Document (100)c. CSC test cases •d. CSU test requirements and test casese A specified percentage of the set of CSU and CSC software development files (SDFs). The

specified percentage shall be as identified in the Software Development Plan (SOP).111R. f. The Software Test Description (STO) for each CSCI.

5.4.5 Conflouration manaoement. 0

5.4.5.1 The contractor shall incorporate the updated Software Design Document (SOD) for each CSCIinto the CSCI's Developmental Configuration prior to delivery to the contracting agency.

5.4.5.2 The contractor shall place the updated Interface Design Document (100) under configurationcontrol prior to delivery to the contracting agency. S

25

0004rtD.217?A (CRAT)

S.4h. TheonS'oG shiall place tho 5cR Wa' Tsd 0esc0pWW (STO) for each CSCI une

~gs~fl @orwa pow to delivelY tO the contracting agency.

A

VIZ.

26

00.04-167A (DRAFT)

- - - - -Q

22

* ooo.TrD-2le7A (DRAFT)

n Ut l perform me following Coding and CSU testing5.5 QCMnna ad cs etn.The conitractor'

*.51 mm d pw'lment manaacmifL No additional requirements.

5.5.2 SoftwMr enaineerin.

5.5.2.1 The contractor shall develop test procedures for conducting each CSU test. The contractor -.sIall record these procedures in the corresponding CSU software development files (SOFs).

5.5..2 The on*actor shall code and test each CSU ensuring tha the algorithms and logic employedby each CU are correct and that the CSU satisfies At specified requirements. The contractor shallrecord the tet results of all CSU testing in the corresponding CSU SOP.

5.5.2.3 The contractor shall make all necessary revisions to the design documentation and code,

perform all necessary retesting, and shall update the SOFs of all CSUs that undergo design or codingchangee based on CSU tests.

5.5.2.4 The contractor shall develop test procedures for conducting each CSC test. The contractor* shall record these procedures in the CSC SOrs.

* 5.5.3 FPmal cualification testing. No additional requirements.

5.5.4 Software poduct evaluaions. The contractor shall perform evaluations of the products identified.,.. below, using the evaluation criteria specified In Fgure &..

'.8.

a. The source code for each CSUb. The CSC test proceduresc. The CSU test procedures and test results asd. A specified percentage of the set of updated software development files (SOFs).

5.5.5 Conflourafton manaoement. A.

5.5.5.1 The contractor shall incorporate the updated Software Design Documents (SODs) and sourcecode listings for each successfully tested and evaluated CSU into the appropriate Developmental *Configuration. ""

5.5.5.2 The contractor shall place the source code for each successfully tested and evaluated CSU "under configuration control.

I

20 a

'-I

000-870-2167A (DRAFT)

An~.2

t.~~( - - J.1 1 1

caof

uj j

29~

V.*e ~000O-T"-07rA MPv:

UP CSC Inhaiomotifiiad teatln", The conrgctof &Wa perfon, the f1ollotWing MW mbegio- s tootngactivities.

&L6.10111111M 20100Ihh- f3ID ma~e~ Th covato " odc orn or mom@ Teoo RoadineezReview() (TVA in rdece with ML-S'T-1 521.

5.8.2 Sofltwfiag fnain0ln.

5.8L2.1 The contractor shall conduct CSC integration and testing. The contractor "hl ensure that thealgmohms and logic employed by each CSC are correct and that the CSC sawIsfie its specified

5.6.2.2 The contractor shall record the test results of all CSC Integraon and testing In thecorresponding 080 software development files (SOF).,

5.6.2.3 The cmnractor shall make all necessary revisions to the design documentation and code,perform all necessary retesting, and update the software development files (SOMs) of all CSUs, CSCs, andCSCIs that undergo design or coding changes based on the results of all testing performed.

5.6.3 Formal oua lfcatlon testing.

* 5.6.3.1 For each formal qualification test case Identified In the Softwere Ted Descripton(s) (STDs) the #.

contractor shall develop set-up procedures, procedures for conducting each tot and pioced foranayzing the test results. These procedures shall be documented In the Softweae Test Description (STO)for each 0801.

5.6.3.2 Prior to conducting Formal Qualification Testing (FOT), the contractor shall conduct te testsdocumented in the Software Test Description (STD) for each CSCI to ensure that the procedures arecomplete and accurate and that the software is ready for FOT. The contractor shIll record the results '

of this activity In tho corresponding CSCI software development fliles (SOFs) and shall update the STDasappropriate.

5.6.4 Software oroduct evaluations. The contractor shall perform evaluatons of the products identified Nbelow, using the evaluation criteria specified in Figure 9. The contractor shall present a summary ofthe evaluation results at the Test Readiness Review.

a. The test results recorded in the software development files (SOFs)b. The updated Software Test Description (STD) for each OSCIc. The updated source code and design documentsd. A specified percentage of the updated software development files (SOFs).

5.6.5 Confiauration management. The contractor shall incorporate the updated Software Design 4Documents (SODs) and source code listngs for each successfully tested and evaluated CSC into thelo

.. .appropriate Developmental Configuration.

C

30C7

0 008T-047A (OAAFT)

IC2.ii II Q

h31

-,--- -. . . .. .. . . .. . -w u-v v . vvr ' ,:: . W ,F u- ._ w- _ , , .*":,m-N -'a .-.

0004rD-21 67A (DRAFT)

S.57 c Wng The contractor sh~all perform the folmWlng CSsti ng aciils

5.7.1 g'U dIaoment man@ement The contractor sa support the Functional Configuration

Audit(i)-(CA) and Physgca Configuration Audit(s) (PCA). The FCA and PCA for a CSCI may be delayed

until adtW systom integraton and testing.

5.7.2 ~re etatne mino.

5.7.2.1 The contractor shall make necessary revisions to the Softwae Design Document(s) (SOOs) and ,code, conduct all necessay retesting, and update the aftwre development files (SOFs) of all CSUs,CSC, and CSCls that undergo design or coding changes based on the results of formal qualification ptesting.

5.7.2.2 The contractor shall make necessary revisions to the Interface Design document (tOO) based onthe results of formal qualification testing and shall prepare the 100 for delivery.

5.7.2.3 Following successful completion of formal qualification testing and prior to FunctionalConfiguration Audit (FCA) and Physical Configuration Audit (PCA), the contractor shall produce theupdated source code for each CSCI. The contractor shall prepare the source code for each CSCI for cdellvery as specified In the Software Requirements Specification (SRS).

5.7.2.4 For each CSCI, the contractor shall prepare a Software Product Speciicato (SPS) for ,delivery.

5.7.3 Formal oualification testing.

5.7.3.1 The contractor shall perform the formal qualification testing (F0T) activiltes in accordance

with the procedures documented in the Software Test Description (STO) for each CSCI.

5.7.3.2 The contractor shall record the results of formal qualification testing in the Software Test'

Report(s) (STRs) for each CSCI.

5.7.3.3 Upon completion of FQT, the contractor shall prepare an up-to-date Software Test Description(STD) for delivery to the contracting agency.

5.7.4 Software oroduct evaluations. The contractor shall perform evaluations of the products identifiebelow, using the evaluation criteria specified in Figure 10.

a. The Software Test Report(s) (STRs) for each CSCI* b. The updated source code and design documentation.

- 5.7.5 Confiouration management. NO

5.7.5.1 The contractor shall identify the exact version of each CSCI to be delivered. The contractor

shall document this information in a Version Description Document (VOO) for each CSCI. ,

5.7.5.2 Following successful completion of the Functional Configuration Audit (FCA) and PhysicatLConfiguration Audit (PCA) and when authenticated by the contracting agency, the Software Product

Specification (SPS) for each CSCI will be incorporated into the Product Baseline. At this point. eact."

CSCI's Developmental Configuration shall cease to exist.

%

e 32

CT

'I'a

ji I 101

E mP 1113il3

0004"S)62167A (CRAFr) .

S y tm Integratian and testing. The conuttrS ' shall perform the following System integration an ,,

Testing act1146ieZ4

5.8.1 &jW dMM- _-meiint mnaaentjlt. The contractor shall support the Functional and Physical

Cofigu~Au di (see 5.7.1).

5.9.2 Saftwer anoineedn. The contractor shall make necessary revisions to design documentation and

code and shall perform all retesting necessary based on system integration and testing.

3.8.3 Formid,,u-illtcation testin a.

5.8.3.1 The contractor shall support the development and documentation of system Integration and test

plas tea cases, and test procedures.

5.8.3.2 The contractor shall support system Integration ad testing actties.

5,8.3.3 The Contractor shall support post test analysis and reporting of system Integration and test

resut.

5.44 Spware ,rmduct evaluations. The contractor shall perform evaluations of the updated source codeand design documentation using the evaluation criteri specified In Figure 10.

5.8.5 ",nflaurstion management The contractor shall prepare necessary changes to beseilned

documentation in accordance with paragraph 4.5.5.

4,.-.

3-

71

.1

0

e" 3'

k -.-.4'"

" ' ' ' -

% "" "% ' ' ' '". . .•.

30041V.O-2187A (ORAFT)

6. NOTES

6.1 Ono amMIIII IV Und crom_ ranm . Whi this standard is used in an acquisition whichItipoatp M & 00 Frm 140, Contract Oats Requirements List (CDRL), the data requirements identifiedbesoW s11 be developed a specified by an approved Dat tem Description (DO Form 1664) anddelivered in accordance with the approved COAL Incorporated Into the contraL When t provisions of

"- the 000O FAR Supplement 27.410- arm invoked and the DO) Form 1423 Isl not used, the data specified

beloi hal be delivered by the contractor in accordance with he contract or purchase orderrequiremnents.

Paragrap b 1 Rmilremn Tide Aoicable DID No.

S5.1.2.2, 51.4, System/Segment Design OI-CMAN-XO0(X5.1.5 Document (SSOO)

4.1.3, 4.2.5, Software Development Plan Di-MCCR-8003044.1, .1.4, 5.1.5, (SOP)

.9. SA.4, 40.1, 40.2.5

I4.2.6, 4.2.10, 4.3.4, Software Requirements DI- R45.1.2.3, 5.1.3, 5.1.4, Specification (SRS)

.1.5, 5.2.1, 5.2.2.1,5.2.3, 5.2.4, 5.2.5,5.3.2.1, 5.3.3, 5.7.2.3,

S40.3.1

4.2., 4.3.4, 5.1.2.4, Interface Requirements Di-MCCR-800265.1.4, 5.1.5, 5.2.1, Specification (IRS)5.Z2.2., 5.2.4, 5.2.5,

S5.3.2.1, 5.3.2.2, 5.4.2.2

5.3.2.2, 5.3.4, 5.3.5.3, Interface Design Document DI-MCCR-800275.4.2.2, 5.4.4, 5.4.5.2, (IDO)5.7.2.2

4

53.2.1, 5.3.2.3, Software Design DI-MCCR-8001253.4, 5.3.5.1, 5.4.2.1 Document (SOD)5.4.2.3, 5.4.4, 5.4.5.1

4.2.10, 5.7.2.4. 5.7.5.2 Software Product Specification DI-MCCR-80029(SPS)

5.7.5.1 Version Description Document DI-MCCR-80013

" (VOD)

35

-- -. -. =., ' f

Doo04r2oITA (MPA r)

N&lilz Data Rgairererf TileDI N

4.&1, 5.A 53. Software Tet Plan DI4-CCR-M14 La

434, S 5 Software Tedt Description 4CCR=400I SS.4.5., 5.6.3.1, 5.8.3.2, (STD)5.6.4, 5.7.3.1. 5.7.3.3

5.7.2, &7.4 Software Test Report DIMCCR-eOi0 7

4.6.4 computer Systern operatoes DI..MCCR401 8Manual (CSOM)

44.4 Software Usota Manual (SUM) DI-MCCR-m0io

4..4 Software Programmer's Manual ODMCCR-sOO21(5PM)

4.6.4 Firmware Support Manual (FSM) Ok,CCR-60022

4.6.2, 4.6.4 Computer Resources Integrated O-MCCR-80024 5Support Document (CRISO)

(Data item descriptions related to this standard, and identified in section 6 will be approved and listedas such In DoD 5000.19-L, Vol. Il, AMSOL Copies of data Rtm descriptions required by the contractorsin connection with specific acquisition functions should be obtained from the Naval Publications and IForms Center or as directed by the contracting officer.)

6.2 Subject term (key wordl listina.

AcquisitionBaselines,CodeCoding and CSU testingComputer

* Computer resourcesComputer softwareComputer software componentComputer software configuration itemComputer software unitConfiguration item

*Configuration managementCSCCSC integration and testing

. 36

kN 6)

S o.04fTD12 GTA (DRAF )

-gcsuoats Item deeaIplinnoDetald dsignDevelopmental configurationEngineering environment

- FhvwsreFonqld Qualification TestingNon-develOpmental software

S' Preliminay design.5. QualIfcation

., ~Requirements analysis'. Risk management

Saty managementSoftware development

Softwane development file

Software development librarySoftware engineeringSoftwa r product evaluation

-. Software requirements analysisSSoftware supportL . System integration and testing

Test environmentTesting

373

.5.

.A.

.4 .-

• 37/38

---------- ''-- -- -- '.--~-.--

oOD.8T01 SIA (DRAFT)

APPENDC A

LIST OF ACRONYMS AND ABBREVIATIONS

10.1 PuZ . This appendix provides a list of al acronyms and abbreviations used in this standard,

wlth the associated meaning.

COR Critical Design Review

CORL Contract Data Requirements UstCI8 Critical Item Development SpecificationCRIE0 Computer Resources Integrated Support DocumentC8C Computer Software Component-C)I Computer Software Configuration ItemCSOM Computer System Operator's ManualCst Computer Software UnitDID Data Item Description0O Department of DefenseDO.I6S .parrn.ant of Defense Index of Specifications and StandardsECP Engineering Change ProposalFAR Federal Acquisition RegulationFCA Functional Configuration AuditFSM Firmware Support ManualFOT Formal Qualification TestingGFS Government Furnished SoftwareHOL High order languageHWCI Hardware Configuration Item100 Interface Design DocumentI/0 Input/OutputIRS Interface Requirements SpecificationIV&V Independent Verification and ValidationNDS Non-developmental softwarePCA Physical Configuration AuditPOR Preliminary Design ReviewPIDS Prime Item Development SpecificationSCN Specification Change NoticeSOD Software Design DocumentSOF Software development fileSOL Software development librarySDP Software Development PlanSCR System Design ReviewSOW Statement of WorkSPM Software Programmer's ManualSPS Software Product Specification 0SRR System Requirements ReviewSRS Software Requirements SpecificationSSDO System/Segment Design DocumentSSR Software Specification ReviewSSS System/Segment SpecificationSTO Software Test DescriptionSTP Software Test PlanSTR Software Test Report

39

* 0004T0.2167A (ORFT

su a.5 .~

Mna

.. R TO .MkM OV6

5%Wop~lnDouwS.D

wo sradw5Srutr

94

DOOSTD-2167A (DRAFT)

APPENDIX B

REQUIREMENTS FOR SOFTWARE CODING STANDARDS

20.1 p . The purpose of this appendix is to specify language independent requirements for

software coding standards.

P 20.2 &Wjg bi. This appendix applies to all deliverable source code developed under the contract.

20.3 Rules and Convention. The following subparagraphs define the requirements for rules andconventions applicable to software coding standards. The contractor shall implement software codingstandards that comply with thes requirements.

20.3.1 PNmtstion tye. The coding standards shal! describe the rules and conventions for the formatof the source code which may include paper listings, listings stored on electronic media, or both. Therulee and conventions for presentation style shall Include standards for

pra. Indentation and spacingb. The use of capitalization

*c. Uniforn presentation of information throughout the source code (e.g., the grouping together of>ad ddt declartons).. -d. Useoofheader

- =e. Layout of source code listingsf. Conditions under which comments ae provided and the format to be usedg. Size of code aggregates (e.g. on the average 100 or at most 200 executable, non-expandable

statements).

20.3.2 Naming. The coding standards shall describe the rules and conventions governing the selection ofidentifiers used in the source code listings (e.g., identifiers for CSUs, variables, parameters, packages,procedures, subunits, and other aggregates of code.) Restrictions on the use of reserve words andkeywords shall be identified.

* .. 20.3.3 Restrictions on the implementation language. The coding standards shall include a description ofany restrictions imposed on the use of constructs and features of the implementation language due toproject or machine-dependent characteristics. Machine-dependent characteristics may includeinput/output features, word length-dependent features, use of floating point arithmetic, etc. Project

" ~. characteristics may include, but are not limited to, safety or security considerations in the operationalenvironment.

- .~-20.3.4 Use of lanouaoe constructs and features. The coding standards shall address the allowed use ofconstructs and features of the implementation language. For example, when Ada is the implementation

.~-language, the coding standards shall address the use of exception handling, goto and abort statements,. unchecked conversion, etc.

20.3.5 Comolexitv. The coding standards shall describe controls and restrictions on the complexity ofcode aggregates.

0 .41142

.OO.T0 2167A (MRAFT)

APPENOO( C

CATEGORY AND PRIORTY CLASSIFICATIONSFOR SOFTwARE PROBLEM REPORTING

• .e 30.1 .]~3v a.e . This appendix contains a category and priority classifcation scheme to be applied to all

problems detected in the deliverable softwae that has been placed under contractor configuration

30.2 w kfcalon by ctlegory. Problems detected during software operation shall be classified by

.. , ,.category as follows:

a. jMzbfl. The software does not operate according to supporting documintation andthe documentation is correct.

,, b. Documentation oroblem. The software does not operate according to supporting documentationbut the software operation is correct.

c. Olssanprblem. The software operated according to supporting documentadon but a deelgndefctency exist. The design deficiency may not always result In a directly observab eoperatona symptom but possesses the potential for creating further problems.

.%" 30.3 Classification by priority. Problems detected In the software shll be classified by priouty as

folows:

a. Priority I A software problem that does one of the following:

(1) Prevents the accomplishment of an operational or mission essential capability specified bybaselned requirements

(2) Prevents the operator's accomplishment of an operational or mission essential capability

i (3) Jeopardizes personnel safety.

b. Priority 2. A software problem that does one of the following:

(1) Adversely affects the accomplishment of an operational or mission essential capabilityspecified by baselined requirements so as to degrade performance and for which no

-I alternative work-around solution is known

(2) Adversely affects the operator's accomplishment of an operational or mission essential, capability specified by baselinel requirements so as to degrade performance and for which

no alternative work-around solution is known.

.,, c. Priorit 3. A software problem that does one of the following:

S'(1) Adversely affects the accomplishment of an operational or mission essential capability

4-% specified by baselined requirements so as to degrade performance and for which analternative work-around solution is known

6 (2) Adversely affects the operator's accomplishment of an operational or mission essentialcapability specified by baselined requirements so as to degrade performance and for which

an alternative work-around solution is known.

"* 43

'4 SJ4~

Appendix C

d. pftf A bAW9Sproblem that 18 an operato InowmlflCe or annoyance and which dodo

rm jmuWaaodo wo ormw capsilly.

a. dat N other snai

o.

00

a.. 000.TD42187A (DRAFT)

APPIENDIX 0 4

EVAi.UATION CRn"ERlA

40.1 Pu This appendix contains a del 5et of definitlons for the evaluation criteria appearing inFigures 4 through 10. These definitions shall be implemented by the contractor if an alternative set has

'" not been proposed In the Software Development Plan and accepted by the contracting agency.I 40.2 Crltda definitions The following definition* are listed In the ordier tha the criteria appear in :

Figures 4 through 10. For convenience, the definitions use the word "document" for the item beinga auat even though In some instances the item being evaluated may be other than a document.

402.1 Internal consistency. Internal consistency as used in this standard means that (I) no twostaements In a document contradict one another, (2) a given term, acronymn, or abbreviation means the jsuie thing throughout the document, and (3) a given item or concept is refered to by the same nameor description throughout the document

40.2.2 Udedl i Understandability, as used In this standard, means tha: (1) the document uses: : nies of grammar, capitalization, punctuation, symbols, and notation consistent with thoese specified In

the U. S. Government Printing Office Style Manual, (2) all tems not contained In the U. S. GovernmentP Prinig 0111ice Style Manual or Merrlr-Wbetea New Inter-atoal diconery (t revision) amdeflned, (3) standard abbreviations listed In MIL-STD-12 e used, (4) all acronyms and abbreviations notRood In ML-STD-12 ame defined, (5) all acronyms and abbrevatons am preceded by the word or termspeled out In full the first time they am used In the document, unim the fig use occurs In a tole,"ltwu, or equation, In which case they are explained In the tet or in a footnote, and (6) all bls,figures, and Illustrations are called out In the text before t appear, In the order In which theyappear in the document.

40.2.3 Treceability to indicate2d documents. Traceability as used In this standard means that thedocument in question is in agreement with a predecessor document to which It has a hierarchicalrelationship. Traceability has five elements: (1) the document in question contains or Implements allapplicable stipulations of the predecessor document. (2) a given term, acronym, or abbreviation means thesame thing In the documents, (3) a given item or concept is referred to by the same name or description

,A in the documents, (4) all material in the successor document has its basis in the predecessor document,that Is, no untraceable material has been introduced, and (5) the two documents do not contradict oneanother.

*' 40.2.4 Consistency with indicated documents. Consistency between documents, as used in this standard,means that two or more documents that are not hierarchically related are free from contradictions withone another. Elements of consistency are: (1) no two statements contradict one another, (2) a given

' .term, acronym, or abbreviation means the same thing in the documents, and (3) a given item or conceptis referred to by the same name or description in the documents.

40.2.5 Aporoorlato analysis. desion. and coding techniaues used. The contract may include provisionsregarding the requirements analysis, design, and coding techniques to be used. The contractor'sSoftware Development Plan (SOP) describes the contractores proposed implementation of these techniques.This criterion consists of compliance with the techniques specified in the contract and SOP.

40.2.6 Acorooriate allocation of sizing and timino resources. This criterion, as used in this standard,means that (1) the amount of memory or time allocated to a given element does not exceed documented

"- constraints applicable to that element, and (2) the sum of the allocated amounts for all subordinateelements is within the overall allocation for an item.

45

!!I-:

%

DOO4411PA (CRFTAppendix t-

40.2.7 . MaJ"a"e in ovoueSO of reoulqrments, This criterion, M used In this standard, mean that (1)every escM rluirenme Is addressed by d I0in one teak (2) tt cses have been selectedor bft4average" sftdn aind boundary" situadon, such as minimum and maxmum values. (3) Istr ahave been selected, such as out-of.bounds velu40, arid (4) tlt cases th exercise combinations ofdifferent functions are Included.

40.3 Additional crterL The following definitions apply to Criteri that are not selNl-xplanatory andtat appear in the NOTES column of Figures 4 through 10. These criteria re not Included in eachfigure, but appear only as appropriate.

40.3.1 Adeguacv of aualtv factors. This criterion applies to the qualty factr requirements In theSoltwae Requirements Specification (SRS). Aspects to be considered am: (1) trade-oft between qualityfactors have been considered and documented, and (2) each quality factor accompanied by a feasiblemethod to be used to evaluate compliance with it is required by the SRS DID.

40.3.2 Testability of Mouirement. A requirement is considered to be httble V an objecive andfeasLible teat can be designed to determine whether the requirement Is met by the softwe

40.3.3 Consistency between data definition and data t- Thi criterion applies primarily to design* documents. It means that each data element is defined in a way tha is consistent with Its usage in the ,software logic.

40.3.4 Adeauacv of test cases. test oroceduM. (test Inauts aoected resut. evalaifon citerfn TesOo es and test procedures should specify exactly what Inputs to provide, what steps to follow, whatoutputs to expect. and what criteria to use in evaJuating the outputL. If any of these elements am notspecified, the test case or test procedure is inadequate.

40.3.5 Comnleteness of testing. Testing Is complete it all test cases and all test procedures have been *performed, all results have been recorded, and all acceptance criteria have been met.40.3.8 Comaleteness of retesting. Retesting consists of repeating a subset of the test cases and test .procedures after software corrections have been made to correct problems found In previous testing,Retesting is considered complete if1 (1) all test cases and test procedures that revealed problems in theprevious testing have been repeated, their results have been recorded, and the results have met *acceptance criteria, and (2) all test cases and test procedures that revealed no problems during theprevious testing, but that test functions that are affected by the corrections, have been repeated, theirresults have been recorded, and the results 'ave met acceptance criteria.

0~

%0 .

'4 ,

40~%

W

V .I 1 % % t " " - " - ' ' % ' ° ' *

--- - - - 1 - -p=o

OOC4TD.216?A (PRAFT)

AOOU .1ON 11,1 U an COMPUTER SOiWARE CONFIGURpATION r.LOCA1U. ALLOGAIO 4,.25 4.2.4Z10, .12.2. CONFIGURATION MANAGEMENT (CM) 2.1. 3.24, 3.34,

5I.,5.42.1,40"2. 4.5. 5.1.5. 5±5. 5.3.5. 5.4.A 5.5.5. 5..5, S5.7.5, 5.8.5ALLO 7TD 6ASELINE 3.1 ,4±105.12.1, 5.122 CONFIGURATION STATUS ACCOUNTING 4.5.3

ow ALINE CONTRACT. CONTRACTUAL 1. 1.3. 4.1-8.4.0,1APROVAL. APPROVE 4.1.3.44,4±7.4.3,4.3.1.4.4.4. AB SPECIFIED IN 41.% 4.1.7.4.2.7. 4.,1.4.6

" ~~AUlI11 Z1.1. 4.1.1. 4.1.&. Flgum 1.42.1. Figur Z. 3.7.1. OUlRATION OF 4.1l 4. 4Xl.0

.72,3 .7..2 .1 iDET1 IN 42.10AUTHIENTICATED. AUTHENTICATON 32 5..1. 5.7.52 IN ACCOROANCE WITH &16AMKJN, ISENED 3., 4.12. 4.1. Fp. 2. 5.8.5. LIFE OF 4.1.10A 4.4.3 I

U 30Ja(1), 30.3.b(1), 30.3J.b0 30.3.01). REWUM BY 4.1.2.4.1.10,42.1.42.730440 F)RE BI4EDrS OF 4.1.8 422.42.44,32

A.LOCATD .14..1-4.5±13. 5 CONTACTAENCY 1 2..2.3.*14, 3.1 322.3.34,FuNCTIONAL 3.164.5.I4.5.3 3.3L 4.1 A 4.1 . 4.1 424. 47.42i,4.34A 41,qiiOOT 3U3L 4.1., 4.5.3,5.7.5.2 4.4.,4 4.44.4, 4 ,52"41 48.3.1.&. 51,

COL COO 3.30 M 4.1.1.s .4±. 7.4±.8. 5.2.5. S.341...2 52..3.5 L5.,.4.5.4.5.3.4.L.1. 4 ...b . 4..1. 5.5, 5..22.5.$53. 5.7.., .7.5 401SA4. 5-&.1I. .552. Pute 6. 5.6 , 5.6.4.o $5.6.S. CORREdTI ACTION, OOIREK ACTION PROCESS

I.72.1, .723, &.7.4A S." 2 .4,. Appe 5, 4.1.10 4.1.11.4. A. 4.. 4.440". COOT11CHEULE REPORTMN 4.1.3

COOINM AND COU TESING 5.* CRITICALDEI N REVIEW !COR) 5.4.1.5.4.4COOWG STANDANS 42.. Apol'db B CRITICAL ITEM DEVELOPMENT SPEClICATON (CO)MCOMPUTER 3.4.3.10. 3.17.4.3, 4.5.1 .b 3.31COAUTER DATA 3.4.3.5.3.7,3.17 CSC INTEGRATION ANO TESTING 4.1.1.1.5.324. 5.4.2.4,COMPUTER MAOWARE 3.5. 3.6.3.7 S.4.4, 5.5Z4,5.5.4.51. SA 6COMPUTER INSTRUClIONS 3.4.3.7,3.17 OSCITSTING 4.1.1.g.4±,5.7, FIpte 10COMPUTER RESOURCES 3.6 CSU TESTING 4.1.1.s.5.4.2.5.5.4.43.5,4.5.5, Figt. 6COMPUTER RESOURCES INTEGRATED SUPPORT DATA RIGHTS 424

DOCUMENT (CRIO) 4.62.4.6.4.iL 6.1 DO FORM 1423 6.1COMPUTER SOFTWARE COMPONENT (CSC) 3.8. 42.5, DO FORM 1664 6.1

4.2..4.5.1 .c. 5.42.1 DELINER. DELIVERALE. DELNERED 322. 4.1.1. 4.12..COMPUTER SOFTWARE CONFIGURATION ITEM (CSCI) 4.1.6,4.2.1. F4pn 2.42.4.4.2.7,42..4.2.10, 4.4,

3.8. 3., 3.13, 3.15. 3.10, 4.1.1. 4.1.1.@. 4.1.8. 42.5. 4.42. 4.5.1 .C. 4.5.1.1. 4.52.b. 4.52.d. 4.5.3.C. 4.5.4.4.6. Fig"a 3. 429. 4±10. 4.3. 4.3.1. 4.3.4. 4.5.1.c. 4.6.1.4.62 4.6.3. 4.6.4, 5.1.5. 5..5. . 5.3.5.2.4.5.1.0. 4.52.iL 5.122. 5.12.3. 5.12.4, 5.1.3. 5.1.4.. 5.3.5.3. 5.4.5.1. 5.4.52. 5.4.5.3. 5.7.2.2. 5.7.2.3. 5.7-2.4,

.15.a. 52.1, 5.22.1. 5.22. 52.3. 5.2.4.8. 52.5, 5.7.3.3. 5.7.5.1.6.1.202.30.1".311, 5.322 5.32.3. 5.3.3. 5.3.4A, 5.3.5.1, 5.42.1, DESIGN AND COOING STANDARDS 4.2.8. Appendx 8 p5.422. .42 5.43.4.4.L 5.4.4.f. 5.4.5.1. 5.4.5.3. DEVELOPMENTAL CONFIGURATION 3.15, 3.27, 4.12.5..1.5...1...3.2. .1.4, 5.7. 5.7.1, 5.72.1. 5.7.2.3. 4.5.1.a. 4.5.2.a. 4.5.3. 5.3.5.1. 5.45.1, 5.5.5.1, 56.5.56.7±4.5.75.1..7.32.5.7.4.&, 5.7.5.1, 5.7.5.2. 5.7.52Fi.o 10 DETAILED DESIGN 4.1.1 .d. 5.4, Figure 7

COMPUTER SOFTWARE UNIT (CSU) 3.8, 3.11, 4.2.3, DOD0TD4W 2.1 .142.3.4.5.1.0 5.4.1 DOO-STD-2168 2.1.1

COMPUTER SYSTEM OPERATOR'S MANUAL (CSOM) ENGINEERING CHANGE PROPOSAL (ECP) 2.1.1, 4.5.54.6.4.b. 6.1 ENGINEERING ENVIRONMENT 32.3.30.42.2

CONFIIGURATION CONTROL 2.1.1. 3.15. 4.1.10, 4.1.11. EVALUATE. EVALUATIONS 3.16. 3.21,3.32. 3.33. 4.1.10.e.42.2, 4.3± 4.5.1.b. 4..2. 4.5.2.c. 4.52 d. 5.1 5. 5.25. 4.4. 4.4.1, 4.42. 4.4.3. 4.4.4. 5.1.4. 52.4, 5.3.4, 5.4.24.5.3.52.5.3.5.3.5.4.52.5.4.53. 5.5.52.30.1 EVALUATE. EVALUATIONS 5.42.5. 5.4.4. 5.5.4, 5.5.5.1.

CONFIGURATION IDENTIFICATION 3.12, 4.5.1.4.5.3.b 5.5.5.2. 5.6.4, 5.6.5. 5.7.4, 5.8.4, Fip. 4-10. Appendx D

47pg

I ° p

OOO.STD-2167A MDAPt)

EVALUATION C1WM. 4 ILIA Fllw4, &12.4 POCIEIRMS 4.2.3Ppme &, &.4 Flom , .4.2.4.5.423. .. 44. Fip SYSTIS 3.27.5.&4. Pipn . 6 5. PFp 9. 5.7.4. Fipe 10, OPEPATIONAL5.L4. Apnltd 0 CAPABILITY 30.3

EVALUATION RECORDS 4.4.3 OCUMENTATION 48.40171NAL INTERFACE 5.1.4. 52.2,5.32.2 5.4.2 ENVIRONMENT 20.3.3FWEDELMACOUITION REGULATION AR) 6.1 Lu 3.10FINAL EVALUATION 4.4.2 MSSION 32,42.3FlM W I 12..17.3. 3.30 RROU IIN4 4.2FM ARIE SUPPORT MANUAL (F51 4.6.4.. 0.1 SYrMPOM 302..FORMAL QUALIFICATION ORGANIZArsONAL

E QUIEMENTS 5.1-,5.2.3,5.3.3 F OM 4.3.3, 4.41ACTIE 4.3. 43..44 2.3A 57.3.1 PHYICAL CONIrILIATION AUDIT (PCA 5.7.1.5.72,3ISTING ) 3.16, 4.3. 43.1.5.1.3. 52.3. 5.3.3., 5.7..2, 5..1

5.4. 5.3, 55.3.1,5.5.32 5.72.1.5.722 5.723. PUA .PI.ANNING 41. 4.2.2.2.44.3.1.4.2.4.41.5.7.32 5.7A, 5.8.3 4.1, 452. 4.2 4., LIA4 5.1.5, LI 5JA 3 .4.0,

FUNCTIONAL CONFIGURATION AUorT (1€cA 5,.,. 5.3.2. 5.44. L.4,4,. 5 l. .140., 4US25.72.5 &7.52"5.Z.1 PRELNARY OEN 41.1.54PIm6ll

* GOVNIINME FURNZHED SOFTWARE (GFS) 3.2 PnBU~llowE V IE LI,.4

HIGH LANGUAGE (HOL) 4.2.7 PRE DEVELOPMENT CIFICAT11N MMHAOWARE CTION ITEM (HWC) 320.4.3 PROBC4 RIEP RT 4.1 10bJ. 4.1.11. 4.4.3

5.12.2 CATEOM 4.1.10.4.11,AppowfC.,V2.INDEPENOENCE 4.3., 4.4.1 PRIIORTY 41.10., 4.1.11. Apmn.C. 30.3INDIEPENOENT VERIFICATION AND VALIDATION (IV&V) PROCEDUIR 327, 4.1.4. 4.1.9 4.23, 42,4.5.4, 5..3.1,

3.21.4.1.7 20.3.2INSTALLATION AND CHECKOUT 4.22.4.32. 4.0.3 TEST 3. ,42.8.., 5..1.5.24. 5.5.4.b. 5.5.4.e, '

INTERFACE DESIGN DOCUMENT (100) 5.32.2. 5.3.4.b. 5.6.3.1, 5532 5.7.1. 55.31, 40.3.4, 40.35.5-1-3.5A5 5.4.4b, 5.4.5.2. 5.72.2. 0.1 40.3.0

INTERFACE REQUIREMENTS SPECIFICATION (IRS) 42.6, PRODUCT BASEUNE of RASELINE4.3.4. 5.12.4. 5.1.4.d. 5.1.5.d. 5.2.1. 5.2.2.2. 5.2.4.b, PRODUCT EVALUATION 3.16. 3.33. 4.1.10.b. 4.4. 4.4.1.52.5, 5.3.2.1. 5.32.2. 5.4.2.2. 6.1 4.4.3. 4.4.4. 5.1.4. 52.4. 5.3.4. 5.4.4, 5.5.4, 5.6.4, 5.7.4.

%1O CHANNEL UTILIZATION 4.2.10 5..4. Figur 4 through 10INTEGRATION 4.2.1,4.3.4.3.1 QUALITYFACTORS 40.3.1

-s. CSC INTEGRATION AND TEST1N QUALIFICATION 4.3.1.5.1.3. 5,2.3, 5.3.3see SYSTEM INTEGRATION AND TESTING see FORMAL QUALIFICATION TESTING

LANGUAGE 20.1.20.3.4,20.3.5 RELEASE 324. 4.5.1.d. 4.5.1 .f* USTINGS 3.10.3.15.5.5.5.1,5,6.20.3.1 REQUIREMENTS 1.1, 1.3. 3.10, 3.25. 3.32. 4.1. 4.16,

MANAGEMENT RECORDS 4.3.3 4.1.10. 42, 42.1. 42.3. 42.4. 4.2.8. 4.3. 4.5. 4.0.MASTER COPIES 4.52.d. 4.5.4 5.12.1. 5.42.S, 5.5.2.5. 5.6.2.1. 402.5, 40.2.7. 40.3.2.MEMORY 4Z10. 402.6 ALLOCATED 3.16, 42.5. 4.2.6, 5.122. 5.3.2.1. 5.4.2.tMETHOOSAAETHOOOLOGY 4.5.4, 40 3.1 BASEUNED 4.1.6, 30.3.a. 30.3.b, 30.3.c

DEVELOPMENT 4±.1,4.2.5 COOING STANDARDS ADPend 8MIL-4T.12 4022 DATA RIGHTS 12,4.6.4.0.1MIL- -,0481 2.1.1.4.5.5 DESIGN 5.32.1.5.42.1MILSTD-463 2.1.1.4.5.5 ENGINEERING 522.1MIL-37-490 2.1.1, 4.5.5 EVALUATION 4.4.1MIL-ST-40 1..1,2.1.1 INTERFACE 5.1.2.4,5.222MIL-TD-1521 2.1.1,4.12. 5.2.1. 5.3.1.541.,58.1 QUALIFICATION 5.1.3,52.3.5.3.3NON'DVELOPMENTAL SOFTWARE (NOS) 322. 4.2.4 QUALITY FACTOR 40.3.1

w.. OPERATING RESERVE 4.2.10INSTRUCTIONS 3.10 SECURITY 4.1.5, 4.2.4.32

K:4..:,r *:

4 - OOD.4417A POAWT)ama

REOUTS*1 SOWAMMQEUSMDMAAL~JYSS 4.1.1.b,4.1.pam .4,.148.1. M l 4.a, 54.. guv 4MT JI.,4 2A4& .&." 2.5324.±S.3.. SO1WARE REQU0114Eh SPECIFICATION 00 4Z&g

4A10. 43A 5123.A51. 1,4 . 5.1.4.o 5Z1,.--E RETIIFIEMENTS SPECIFICATION .2±. 5.24.,, 32.% . 5,3±. S.7±3. .. 1, 40.&1m SOFrTWAR CUPEMENTS SPEC ICATION SOFTWARE SPEC&ICATION IEW SIR) 521. S"4mwTnACfAILUr SOFTWME SUPPOWT 1.1.123, 1.X4. 10, 3.2 327.RE UIREM NTSNYSI 42.1, 40.2.5 ,1.2L, 4 " MI. , 44A. 4A.,on SOFTWARE REOUIMNTS ANALYSS m COMPUTE SIRFWAE INTEGSWATB SU*0RT

M SYSTEM REOIRIwENTS ANALYSuSOESIN DOCUMIENT (Cm= O)REMO CAPAC91Y 4.10 No UeST E5

SRESIOURCES 4A. 4.4.1 SOFT UPWARE T 0ECRlnON M 43A. &4.41

,.CESNG 4Z10 SOFTWARE TEST ENVIRONMENT M30. 4.32SNG AND TMING 4003 SOFiWAR TEST PLAN OMT L 4,.1. A53..4. S."2

R:UI5 11." 3.8Z3. .7.1. 2-85.4Z e..RIEUSAS 3.2 3.25 SFIWArPTW ESTREPORTM (Th) 5.732.5.7.4,. .1

EVI"W0I 21.1. 4.1.1, 4.1Z Fgwn 1, 4±1. FOqui 2. SOFTWAREU BSMANUAL IMUM4 8 .4413..I. 5.1.1 5.2.1. 5Z4, 5.1.53.4.5.4.1. SOURCE COOE 3.15. 5.545 a 55. 2..4.0 3.7.AS.44. 5.1, l.4 5.7.4.. 5,5.4,204. 208.1

CO1NTACTN1 AGENCY REVIEW 42-9.4.4.3 SOURCE COOS UIN 5.5.1.o . .SRISK MANAGEMNT 4.1.4 SPECIICATION CHAN NOTnCE W" "A

SAFElY ANALYSIS 4.2.3.20.3.,30.3 SrATU REPORTS 4MSCHSIULE 3.20.4.1.4. 4.2.9. 5.42.4.5.4.2.5 STREW 4.3,4..1. 53.2.4. 8.4.M 4O±7

CONTRACT 4.1.1 SUBCONTRACTOR 1A.4.1.AREPORTS 4.1.8 SUPPORT

SECURITY 4.1.5,4±2.4.32.20.3.3 DOCUMENTATION 4.5.4,, SOFTWARE DESIGN DOCUMENT (SOD) 5.32.1. 5.32.3, ENVIRONMENT 4.603

51.4., 5.3.5-.1. 5.4±,1. 5.4±3, 5.4.4.a, 5.4.5.1, 1.72.1, PLANNING 4.6.26.1 REVIEWS ANO AUOITS 4.12.42.1, 5.1.1.1.5.1.12

SOFTWARE DEVELOPM1ENT FILE(S) (SO s) 3.26, 4.2.4, 5.7.142.9. 5.3.2.4. 5.4Z4. 5.42.5. 5.4.4, 5.52.1. 5.5.4.C. T OANSITIO 4.6. 4.025.622.5.±3, 5.6.32. 5.6.4.& 5.8.4.d. 5.7.2.1 see SOFTWARE SUPPORT

SOFTWARE DEVELOPMENT UIBRY (SOL) 327, 4.1.9 SYSTEM ANALYSIS su SYSTEM REQUIREMENTSSOFTWARE DEVELOPMENT MANAGEMENT 4.1, 5.1.1, ANALYSIVESM(GN

5±1, 5.3.1.5.4.1. 5.5.1.5.6.1.5.7.1, 5.6.1 SYSTEM DESIGN REVIEW (SOR) 5.1.12SOFTWARE DEVELOPMENT PLAN (SOP) 4.1.3. 4.2.5, SYSTEM INTEGRATION ANO TESTING 4.1.1.h. 5.7.1, 5.8

4.4.1. 5.1.4.L 5.1.5.a. 5.4.4.@, 6.1. 40.1, 402.5 SYSTEM REQUIREMENTS ANALYSSDESIGN 4.1.1.. 5.1SOFTWARE DEVELOPMENT PROCESS 4.1.1.41.2 SYSTEM REQUIREMENTS REVIEW (SRR) 5.1.1.1SOFTWARE ENGINEERING 42. 4.3.3. 5.12, 522.5 .32, SYSTEM/SEGMENT DESIGN DOCUMENT (SSOO) 5.1.2Z,

5.42. 5.3.2.51.2.5.72. 5.62 5.1.4.b, 5.1.5. 6.1

-" SOFTWARE ENGINIERIN2 ENVIRONMENT 3,26. 3.30. SYSTEM/SEGMENT SPECIFICATION (SSS) 3.31422 SYSTEM SPECIFICATION 3.31,4.2.6. 5.1.2.1, .1.3

SOFTW ARE P DUCl" EVALUATIONS 44 44.1, 4.4.3, TAILORING 1.35.1.4. 5.2.4 5.3,4. 5.4.4, 5.5.4. 58 4. 5.7.4. 5.91.4. TARGET C0MPUTER 4.3 ,

[-..4U- 4r 4- 10 TECHNICAL.[" SOFTrWARPJIDUCT SPECIFICATION (SPS) 42,10, DATA 3.10 '

. 5.7Z.4. 5.7.52. 0.1 DOCUMENTATION 3.15;

:,. SOFTWARE PROGRAMMER'S MANUAL (SPM) 4.6.4.d. 6A1 RISK 4.1.4

- SOFTWARE QUALITY PROGRAM 1.2.2.2.1 1

49

6l

V:. :-.-..... : . :- .. . .. ... .. . . . . . . . . . .

S g~~00470-21S7A PlAFTn

TES. yTT L11. Q0. 44 &1 43

CA M.4L.444 5 .M. .. 31427403

co#cuCT 55. 92 .55.4. j40.3.5UM(WONMENT 3.30

sSSOFTWARETEST EIVIONMIDITPRW JPES 3.2L 42.0.5.5.2.1.5524 5.5b

SA4.*. 52. 1. 40.4.40.3-5401-9

5.4.4.d5.±4 -

RESPONUS1LIIS 42*.5.42.4.5.425REI&TS 4223.0.5.4.2.5.5-52.5.

.5.0.2Z 5*.3.& 5.6.&. 5.6.4, 5.7.32.

5*.8Z&40.&'?SCHEDULES 5.4.24.5.42.5

.1' 100O.S 3.2L 3.30V~~~~ oo" &M0t4 AN ci TESTINGAs' CSC; INTEORATIOt4 AND YESTING

* osFONML QUFICATION TESTING (F01)

OTWARE TEST DESCRIPTION

wa.SOPTyWARESTENMiONMENTS.sag SOFTWARE TEST PLAN

soSOFWARE TEST REPORT

So. SySTE IN1!(3ATICN AND TES11NGTMs~ RE.AINESS REVIEW (TRR) 5.6.1TRACEASIY 4.53.402.&. Figure 4.10

REQ(UIREMENTS ToO ESIGN 4.2.6REOUIREMENTS TO TEST CASES 4.3.4

TRAINING 4.6.3V4ALIOATIO 3.21,3324.1.77VEIIATION 3.3.24.1.7.43-34.4.1

VERSION 33.451d ...........

*. 5,VER31ON DESCRIpniON DOCUMENT (V00) 5.7.5.1. 6.1

WORK &REAgDOWN STRUCTURE (VMS) 4.1 .8

-rN-

D0O4T-2167A (DRAFT)

CisodhuW

Ay- AR,CCRMIAVNw~y - EC.MC.TDOMSH.ASAir Fan .02.10,17,26

Nvy -EC

77vje ,c-0022)MC.

Distribution List for IDA Paper P-2018

NAME AND ADDRESS NUMBER OF COPIES

-Sponsor

CAPT David Hart 5 copiesSDIO/S/BMThe PentagonWashington, D.C. 20301-7100

Other

Defense Technical Information Center 2 copies-\ - Cameron Station

Alexandria, VA 22314

LTC Charles Anderson 5 copiesRADC/COGriffis AFB, NY 13440

Dr. James M. Boyle 5 copies-' Math and Computer Science Div.

Argonne National Laboratory9700 S. Cass Ave.Argonne, I1 60439-4844

*Mr. Mark Crosby 1 copySAICMS T-9-31710 Goodridge Dr.

I |Mclean, Va 22102

COL Jim Graham 5 copies• SDIO/S/SE

VThe PentagonWashington, D.C. 20301-7100

Dr. Jung P. Hong 5 copiesLos Alamos National LaboratoryMS K-488P.O. Box 1663Los Alamos, NM 87544

Dr Virginia P. Kobler 5 copiesUS Army Strategic Defense CommandDASD-H-SBYP.O. Box 1500Huntsville, AL 35087-3801

Lt. N. Ann Kuo 5 copiesESD/MD

-'*Z ,Hanscom AFB, MA 01731

' I0

Major Frank Maressa 5 copiesSDIO/S/SAThe PentagonWashington, D.C. 20301-7100

Mr. Dick Martin 5 copiesCarnegie-Mellon UniversitySoftware Engineering InstitutePittsburgh, PA 15213

Ms. Nancy Mathis 5 copiesMITRE4717 University DR., Suite 98Huntsville, AL 35816

Mr. Steve McBurnett 5 copiesCode 5570US Naval Research LaboratoryWashington, D.C. 20375-5000

LTC Ted Mervosh 5 copies~ESD/MD

Hanscom AFB, MA 01731

LTC John Morrison 5 copiesESD/MDHanscom AFB, MA 01731 -4

Mr. Karl Nyberg 1 copyGrebyn CorporationP.O. Box 1144Vienna, VA 22180 4

COL Dick Paul 5 copiesESD/MDHanscom AFB, MA 01731

MAJ James Price 5 copies

SDIO/SThe PentagonWashington, D.C. 20301-7100

Mr. Ron Raposo 5 copiesRADC/COTGriffis AFB, MY 13440

LTC Jon Rindt 5 copiesSDIO/S/BM

The PentagonWashington, D.C. 20301-7100

0 4

Dr. Albert Small 1 copySRS TechnologiesSuite 8001500 Wilson BoulevardArlington, VA 22209

MAJ Peter Sowa 5 copiesSDIO/S/BMThe PentagonWashington D.C. 20301-7100

Mr. J. R. Southern 5 copiesUSA-SDCDASD-H-SBDHuntsville, AL 35807-3801

Dr. Stephen Squires 1 copyDARPA

r 1400 Wilson BoulevardArlington. VA 22209

. Mr William Stout 5 copiesANSER

- '1215 Jefferson Davis Hwy., Suite 800.. Arlington, VA 22202

Mr. H. G. Stuebing 5 copiesNaval Air Development CenterCode 50CWarminster, PA 18974

Dr. Doyle Thomas 5 copiesUSA SDCDASD-H-SBP.O. Box 1500Huntsville AL 35807

Mr. Lawrence A. Tubbs 5 copies• _ USA-SDC

DASD-H-SBP.O. Box 1500Huntsville, AL 35807

- COL William Wisdom 5 copies* SDIO/S/TE

The PentagonWashington, D.C. 80914-5001

LT Marcia Wye 1 copyHQ US SPACE COM/J5BPeterson AFB, CO 80914-5001

0;

CSED Review Panel

Dr. Dan Alpert, Director 1 copy 0Center for Advanced StudyUniversity of Illinois912 W. Illinois StreetUrbana, Illinois 61801

Dr. Barry W. Boehm 1 copyTRW Defense Systems GroupMS 2-2304One Space ParkRedondo Beach, CA 90278

Dr. Ruth Davis 1 copyThe Pymatuning Group, Inc.2000 N. 15th Street, Suite 707Arlington, VA 22201

* Dr. Larry E. Druffel 1 copySoftware Engineering InstituteShadyside Place

. 480 South Aiken Av.Pittsburgh, PA 15231

Dr. C. E. Hutchinson, Dean 1 copyThayer School of EngineeringDartmouth CollegeHanover, NH 03755

Mr. A. J. Jordano 1 copyManager, Systems & Software 0Engineering HeadquartersFederal Systems Division6600 Rockledge Dr.Bethesda, MD 20817

* Mr. Robert K. Lehto 1 copyMainstay302 Mill St.Occoquan, VA 22125

Mr. Oliver Selfridge 1 copy*45 Percy Road

Lexington, MA 02173

f.-..

..t

IDA

General W. Y. Smith, HQ 1 copyMr. Seymour Deitchman, HQ 1 copyMr. Philip Major, HQ 1 copyMs. Karen H. Weber, HQ 1 copyDr. Jack Kramer, CSED 1 copyDr. Robert I. Winner, CSED 1 copyDr. John Salasin, CSED 25 copiesMr. Howard Cohen, CSED 1 copyMs. Audrey A. Hook, CSED 1 copyDr. Joseph Linn, CSED 1 copyDr. Cathy Jo Linn, CSED 5 copiesMr. Robert Knapper, CSED 5 copiesMs. Julia Sensiba, CSED 2 copiesMr. Richard Waychoff, CSED 5 copiesIDA Control & Distribution Vault 3 copies

0

-1' d:.

0

0

I IFF

r,,