SOFTWARECPR - Amazon S3s3-us-east-2.amazonaws.com/mlfpcontent/wp-content/... · • FDA Scientific Reviewers 2.3. THE LEAST BURDENSOME APPROACH We believe we should consider the least

CRISIS PREVENTION AND RECOVERY, LLCSOFTWARECPR

®

FDA Medical Device Software Validation Reference Manual

December 2005 Clickable Table of Contents FDA General Principles of Software Validation FDA General Principles of Software Validation Preamble FDA Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices FDA Guidance for Industry, FDA Reviewers and Compliance on Off-the-Shelf Software Use in Medical Devices FDA Cybersecurity for Networked Medical Devices Containing Off-the-Shelf Software SoftwareCPR List of FDA CDRH ODE Software Submission Guidance Documents SoftwareCPR® Validation & Verification Definitions SoftwareCPR® GPSV Checklist AAMI-BIT Software Test Coverage Article AAMI-BIT Software CAPA Article AAMI-BIT Risk Probability Article FDA Recognition Statement for AAMI SW68 for Device Software SoftwareCPR FDA Software Standards Recognition Comparison SoftwareCPR Training Aide – AAMI SW68 Documentation Requirements SoftwareCPR AAMI SW68 Conformance Assessment Job Aide

www.softwarecpr.com 20 Berkshire Drive

Winchester, MA 01890 781-721-2921

Copyright © Copyright 2005 Crisis Prevention and Recovery, LLC. (CPRLLC), all rights reserved. SoftwareCPR is a division of Crisis Prevention and Recovery, LLC and the SoftwareCPR® logo is a registered trademark. SoftwareCPR® authorizes its clients and SoftwareCPR®.com subscribers use of this document for internal review and training. Any other use or dissemination of this document is expressly prohibited without the written authorization of SoftwareCPR. Individual original FDA documents in their original form without SoftwareCPR® annotations are public information and may be shared without restriction. Legal Disclaimer The training document that follows should only be applied in the appropriate context with oversight by regulatory and software professionals with direct knowledge and experience with the topics presented. The document should not be used as a cookbook since it is not adequate for some situations and may be excessive for others. While SoftwareCPR® attempts to ensure the accuracy of information presented, no guarantees are made since regulatory interpretations and enforcement practices are constantly changing, and are not entirely uniform in their application. Disclaimer of Warranties: The information is provided AS IS, without warranties of any kind. CPRLLC does not represent or warrant that any information or data provided herein is suitable for a particular purpose. CPRLLC hereby disclaims and negates any and all warranties, whether express or implied, relating to such information and data, including the warranties of merchantability and fitness for a particular purpose.

General Principles of SoftwareValidation; Final Guidance for

Industry and FDA Staff

Document issued on: January 11, 2002

This document supersedes the draft document, "General Principles ofSoftware Validation, Version 1.1, dated June 9, 1997.

U.S. Department Of Health and Human ServicesFood and Drug Administration

Center for Devices and Radiological HealthCenter for Biologics Evaluation and Research

Page ii

General Principles of Software ValidationGuidance for Industry and FDA Staff

Preface

Public Comment

Comments and suggestions may be submitted at any time for Agency consideration to DocketsManagement Branch, Division of Management Systems and Policy, Office of Human Resources andManagement Services, Food and Drug Administration, 5630 Fishers Lane, Room 1061, (HFA-305),Rockville, MD, 20852. When submitting comments, please refer to the exact title of this guidancedocument. Comments may not be acted upon by the Agency until the document is next revised orupdated.

For questions regarding the use or interpretation of this guidance which involve the Center for Devicesand Radiological Health (CDRH), contact John F. Murray at (301) 594-4659 or [email protected]

For questions regarding the use or interpretation of this guidance which involve the Center for BiologicsEvaluation and Research (CBER) contact Jerome Davis at (301) 827-6220 or [email protected].

Additional Copies

CDRHAdditional copies are available from the Internet at:http://www.fda.gov/cdrh/comp/guidance/938.pdf or via CDRH Facts-On-Demand. In order toreceive this document via your fax machine, call the CDRH Facts-On-Demand system at 800-899-0381 or 301-827-0111 from a touch-tone telephone. Press 1 to enter the system. At thesecond voice prompt, press 1 to order a document. Enter the document number 938 followedby the pound sign (#). Follow the remaining voice prompts to complete your request.

CBERAdditional copies are available from the Internet at: http://www.fda.gov/cber/guidelines.htm, bywriting to CBER, Office of Communication, Training, and Manufacturers' Assistance (HFM-40), 1401 Rockville Pike, Rockville, Maryland 20852-1448, or by telephone request at 1-800-835-5709 or 301-827-1800.

Page iii


Table of Contents

SECTION 1. PURPOSE .............................................................................................................. 1

SECTION 2. SCOPE ................................................................................................................... 1

2.1. Applicability........................................................................................................................ 2

2.2. Audience ............................................................................................................................. 2

2.3. THE LEAST BURDENSOME APPROACH................................................................... 2

2.4. Regulatory Requirements for Software Validation.......................................................... 3

2.4. Quality System Regulation vs Pre-Market Submissions ................................................. 4

SECTION 3. CONTEXT FOR SOFTWARE VALIDATION ................................................... 5

3.1. Definitions and Terminology ............................................................................................. 53.1.1 Requirements and Specifications................................................................................... 53.1.2 Verification and Validation........................................................................................... 63.1.3 IQ/OQ/PQ...................................................................................................................... 7

3.2. Software Development as Part of System Design............................................................ 7

3.3. Software is Different from Hardware................................................................................ 8

3.4. Benefits of Software Validation......................................................................................... 9

3.5 Design Review................................................................................................................... 9

SECTION 4. PRINCIPLES OF SOFTWARE VALIDATION ............................................... 11

4.1. Requirements ................................................................................................................... 11

4.2. Defect Prevention ............................................................................................................ 11

4.3. Time and Effort ................................................................................................................ 11

4.4. Software Life Cycle.......................................................................................................... 11

4.5. Plans.................................................................................................................................. 12

4.6. Procedures........................................................................................................................ 12

4.7. Software Validation After a Change ............................................................................... 12

4.8. Validation Coverage ........................................................................................................ 12

4.9. Independence of Review.................................................................................................. 12

Page iv


4.10. Flexibility and Responsibility ........................................................................................ 13

SECTION 5. ACTIVITIES AND TASKS ................................................................................ 14

5.1. Software Life Cycle Activities......................................................................................... 14

5.2. Typical Tasks Supporting Validation.............................................................................. 145.2.1. Quality Planning ........................................................................................................ 155.2.2. Requirements.............................................................................................................. 165.2.3. Design......................................................................................................................... 175.2.4. Construction or Coding ............................................................................................. 205.2.5. Testing by the Software Developer ............................................................................ 215.2.6. User Site Testing ........................................................................................................ 275.2.7. Maintenance and Software Changes ......................................................................... 28

SECTION 6. VALIDATION OF AUTOMATED PROCESS EQUIPMENT AND QUALITYSYSTEM SOFTWARE................................................................................................................ 30

6.1. How Much Validation Evidence Is Needed?.................................................................. 31

6.2. Defined User Requirements ............................................................................................ 32

6.3. Validation of Off-the-Shelf Software and Automated Equipment.................................. 33

APPENDIX A - REFERENCES ................................................................................................. 35

Food and Drug Administration References ............................................................................ 35

Other Government References............................................................................................... 36

International and National Consensus Standards .................................................................. 37

Production Process Software References............................................................................... 38

General Software Quality References.................................................................................... 39

APPENDIX B - DEVELOPMENT TEAM ................................................................................ 43

Page 1


General Principles of Software Validation

This document is intended to provide guidance. It represents the Agency’s currentthinking on this topic. It does not create or confer any rights for or on any person anddoes not operate to bind Food and Drug Administration (FDA) or the public. Analternative approach may be used if such approach satisfies the requirements of theapplicable statutes and regulations.

SECTION 1. PURPOSE

This guidance outlines general validation principles that the Food and Drug Administration (FDA)considers to be applicable to the validation of medical device software or the validation of softwareused to design, develop, or manufacture medical devices. This final guidance document, Version 2.0,supersedes the draft document, General Principles of Software Validation, Version 1.1, dated June9, 1997.

SECTION 2. SCOPE

This guidance describes how certain provisions of the medical device Quality System regulation apply tosoftware and the agency’s current approach to evaluating a software validation system. For example,this document lists elements that are acceptable to the FDA for the validation of software; however, itdoes not list all of the activities and tasks that must, in all instances, be used to comply with the law.

The scope of this guidance is somewhat broader than the scope of validation in the strictest definition ofthat term. Planning, verification, testing, traceability, configuration management, and many other aspectsof good software engineering discussed in this guidance are important activities that together help tosupport a final conclusion that software is validated.

This guidance recommends an integration of software life cycle management and risk managementactivities. Based on the intended use and the safety risk associated with the software to be developed,the software developer should determine the specific approach, the combination of techniques to beused, and the level of effort to be applied. While this guidance does not recommend any specific lifecycle model or any specific technique or method, it does recommend that software validation andverification activities be conducted throughout the entire software life cycle.

Where the software is developed by someone other than the device manufacturer (e.g., off-the-shelfsoftware) the software developer may not be directly responsible for compliance with FDA regulations.

Page 2


In that case, the party with regulatory responsibility (i.e., the device manufacturer) needs to assess theadequacy of the off-the-shelf software developer’s activities and determine what additional efforts areneeded to establish that the software is validated for the device manufacturer’s intended use.

2.1. APPLICABILITY

This guidance applies to:

• Software used as a component, part, or accessory of a medical device;• Software that is itself a medical device (e.g., blood establishment software);• Software used in the production of a device (e.g., programmable logic controllers in manufacturing

equipment); and• Software used in implementation of the device manufacturer's quality system (e.g., software that

records and maintains the device history record).

This document is based on generally recognized software validation principles and, therefore, can beapplied to any software. For FDA purposes, this guidance applies to any software related to aregulated medical device, as defined by Section 201(h) of the Federal Food, Drug, and Cosmetic Act(the Act) and by current FDA software and regulatory policy. This document does not specificallyidentify which software is or is not regulated.

2.2. AUDIENCE

This guidance provides useful information and recommendations to the following individuals:

• Persons subject to the medical device Quality System regulation• Persons responsible for the design, development, or production of medical device software• Persons responsible for the design, development, production, or procurement of automated

tools used for the design, development, or manufacture of medical devices or software toolsused to implement the quality system itself

• FDA Investigators• FDA Compliance Officers• FDA Scientific Reviewers

2.3. THE LEAST BURDENSOME APPROACH

We believe we should consider the least burdensome approach in all areas of medical device regulation.This guidance reflects our careful review of the relevant scientific and legal requirements and what webelieve is the least burdensome way for you to comply with those requirements. However, if youbelieve that an alternative approach would be less burdensome, please contact us so we can consider

Page 3


your point of view. You may send your written comments to the contact person listed in the preface tothis guidance or to the CDRH Ombudsman. Comprehensive information on CDRH’s Ombudsman,including ways to contact him, can be found on the Internet at:

http://www.fda.gov/cdrh/resolvingdisputes/ombudsman.html.

2.4. REGULATORY REQUIREMENTS FOR SOFTWARE VALIDATION

The FDA’s analysis of 3140 medical device recalls conducted between 1992 and 1998 reveals that242 of them (7.7%) are attributable to software failures. Of those software related recalls, 192 (or79%) were caused by software defects that were introduced when changes were made to the softwareafter its initial production and distribution. Software validation and other related good softwareengineering practices discussed in this guidance are a principal means of avoiding such defects andresultant recalls.

Software validation is a requirement of the Quality System regulation, which was published in theFederal Register on October 7, 1996 and took effect on June 1, 1997. (See Title 21 Code of FederalRegulations (CFR) Part 820, and 61 Federal Register (FR) 52602, respectively.) Validationrequirements apply to software used as components in medical devices, to software that is itself amedical device, and to software used in production of the device or in implementation of the devicemanufacturer's quality system.

Unless specifically exempted in a classification regulation, any medical device software productdeveloped after June 1, 1997, regardless of its device class, is subject to applicable design controlprovisions. (See of 21 CFR §820.30.) This requirement includes the completion of currentdevelopment projects, all new development projects, and all changes made to existing medical devicesoftware. Specific requirements for validation of device software are found in21 CFR §820.30(g). Other design controls, such as planning, input, verification, and reviews, arerequired for medical device software. (See 21 CFR §820.30.) The corresponding documented resultsfrom these activities can provide additional support for a conclusion that medical device software isvalidated.

Any software used to automate any part of the device production process or any part of the qualitysystem must be validated for its intended use, as required by 21 CFR §820.70(i). This requirementapplies to any software used to automate device design, testing, component acceptance, manufacturing,labeling, packaging, distribution, complaint handling, or to automate any other aspect of the qualitysystem.

In addition, computer systems used to create, modify, and maintain electronic recordsand to manage electronic signatures are also subject to the validation requirements.(See 21 CFR §11.10(a).) Such computer systems must be validated to ensure accuracy, reliability,consistent intended performance, and the ability to discern invalid or altered records.

http://www.fda.gov/cdrh/resolvingdisputes/ombudsman.html

Page 4


Software for the above applications may be developed in-house or under contract. However, softwareis frequently purchased off-the-shelf for a particular intended use. All production and/or quality systemsoftware, even if purchased off-the-shelf, should have documented requirements that fully define itsintended use, and information against which testing results and other evidence can be compared, toshow that the software is validated for its intended use.

The use of off-the-shelf software in automated medical devices and in automated manufacturing andquality system operations is increasing. Off-the-shelf software may have many capabilities, only a fewof which are needed by the device manufacturer. Device manufacturers are responsible for theadequacy of the software used in their devices, and used to produce devices. When devicemanufacturers purchase "off-the-shelf'' software, they must ensure that it will perform as intended in theirchosen application. For off-the-shelf software used in manufacturing or in the quality system, additionalguidance is included in Section 6.3 of this document. For device software, additional useful informationmay be found in FDA’s Guidance for Industry, FDA Reviewers, and Compliance on Off-The-ShelfSoftware Use in Medical Devices.

2.4. QUALITY SYSTEM REGULATION VS PRE-MARKET SUBMISSIONS

This document addresses Quality System regulation issues that involve the implementation of softwarevalidation. It provides guidance for the management and control of the software validation process.The management and control of the software validation process should not be confused with any othervalidation requirements, such as process validation for an automated manufacturing process.

Device manufacturers may use the same procedures and records for compliance with quality system anddesign control requirements, as well as for pre-market submissions to FDA. This document does notcover any specific safety or efficacy issues related to software validation. Design issues anddocumentation requirements for pre-market submissions of regulated software are not addressed by thisdocument. Specific issues related to safety and efficacy, and the documentation required in pre-marketsubmissions, should be addressed to the Office of Device Evaluation (ODE), Center for Devices andRadiological Health (CDRH) or to the Office of Blood Research and Review, Center for BiologicsEvaluation and Research (CBER). See the references in Appendix A for applicable FDA guidancedocuments for pre-market submissions.

http://www.fda.gov/cdrh/ode/1252.html

Page 5


SECTION 3. CONTEXT FOR SOFTWARE VALIDATION

Many people have asked for specific guidance on what FDA expects them to do to ensure compliancewith the Quality System regulation with regard to software validation. Information on softwarevalidation presented in this document is not new. Validation of software, using the principles and taskslisted in Sections 4 and 5, has been conducted in many segments of the software industry for well over20 years.

Due to the great variety of medical devices, processes, and manufacturing facilities, it is not possible tostate in one document all of the specific validation elements that are applicable. However, a generalapplication of several broad concepts can be used successfully as guidance for software validation.These broad concepts provide an acceptable framework for building a comprehensive approach tosoftware validation. Additional specific information is available from many of the references listed inAppendix A.

3.1. DEFINITIONS AND TERMINOLOGY

Unless defined in the Quality System regulation, or otherwise specified below, all other terms used inthis guidance are as defined in the current edition of the FDA Glossary of Computerized System andSoftware Development Terminology.

The medical device Quality System regulation (21 CFR 820.3(k)) defines "establish" to mean "define,document, and implement." Where it appears in this guidance, the words "establish" and “established”should be interpreted to have this same meaning.

Some definitions found in the medical device Quality System regulation can be confusing whencompared to commonly used terminology in the software industry. Examples are requirements,specification, verification, and validation.

3.1.1 Requirements and Specifications

While the Quality System regulation states that design input requirements must be documented, and thatspecified requirements must be verified, the regulation does not further clarify the distinction between theterms “requirement” and “specification.” A requirement can be any need or expectation for a systemor for its software. Requirements reflect the stated or implied needs of the customer, and may bemarket-based, contractual, or statutory, as well as an organization's internal requirements. There can bemany different kinds of requirements (e.g., design, functional, implementation, interface, performance, orphysical requirements). Software requirements are typically derived from the system requirements forthose aspects of system functionality that have been allocated to software. Software requirements aretypically stated in functional terms and are defined, refined, and updated as a development projectprogresses. Success in accurately and completely documenting software requirements is a crucial factorin successful validation of the resulting software.

Page 6


A specification is defined as “a document that states requirements.” (See 21 CFR §820.3(y).) It mayrefer to or include drawings, patterns, or other relevant documents and usually indicates the means andthe criteria whereby conformity with the requirement can be checked. There are many different kinds ofwritten specifications, e.g., system requirements specification, software requirements specification,software design specification, software test specification, software integration specification, etc. All ofthese documents establish “specified requirements” and are design outputs for which various forms ofverification are necessary.

3.1.2 Verification and Validation

The Quality System regulation is harmonized with ISO 8402:1994, which treats “verification” and“validation” as separate and distinct terms. On the other hand, many software engineering journalarticles and textbooks use the terms "verification" and "validation" interchangeably, or in some casesrefer to software "verification, validation, and testing (VV&T)" as if it is a single concept, with nodistinction among the three terms.

Software verification provides objective evidence that the design outputs of a particular phase of thesoftware development life cycle meet all of the specified requirements for that phase. Softwareverification looks for consistency, completeness, and correctness of the software and its supportingdocumentation, as it is being developed, and provides support for a subsequent conclusion that softwareis validated. Software testing is one of many verification activities intended to confirm that softwaredevelopment output meets its input requirements. Other verification activities include various static anddynamic analyses, code and document inspections, walkthroughs, and other techniques.

Software validation is a part of the design validation for a finished device, but is not separately definedin the Quality System regulation. For purposes of this guidance, FDA considers software validation tobe “confirmation by examination and provision of objective evidence that softwarespecifications conform to user needs and intended uses, and that the particular requirementsimplemented through software can be consistently fulfilled.” In practice, software validationactivities may occur both during, as well as at the end of the software development life cycle to ensurethat all requirements have been fulfilled. Since software is usually part of a larger hardware system, thevalidation of software typically includes evidence that all software requirements have been implementedcorrectly and completely and are traceable to system requirements. A conclusion that software isvalidated is highly dependent upon comprehensive software testing, inspections, analyses, and otherverification tasks performed at each stage of the software development life cycle. Testing of devicesoftware functionality in a simulated use environment, and user site testing are typically included ascomponents of an overall design validation program for a software automated device.

Software verification and validation are difficult because a developer cannot test forever, and it is hardto know how much evidence is enough. In large measure, software validation is a matter of developinga “level of confidence” that the device meets all requirements and user expectations for the softwareautomated functions and features of the device. Measures such as defects found in specificationsdocuments, estimates of defects remaining, testing coverage, and other techniques are all used to

Page 7


develop an acceptable level of confidence before shipping the product. The level of confidence, andtherefore the level of software validation, verification, and testing effort needed, will vary dependingupon the safety risk (hazard) posed by the automated functions of the device. Additional guidanceregarding safety risk management for software may be found in Section 4 of FDA’s Guidance for theContent of Pre-market Submissions for Software Contained in Medical Devices, and in theinternational standards ISO/IEC 14971-1 and IEC 60601-1-4 referenced in Appendix A.

3.1.3 IQ/OQ/PQ

For many years, both FDA and regulated industry have attempted to understand and define softwarevalidation within the context of process validation terminology. For example, industry documents andother FDA validation guidance sometimes describe user site software validation in terms of installationqualification (IQ), operational qualification (OQ) and performance qualification (PQ). Definitions ofthese terms and additional information regarding IQ/OQ/PQ may be found in FDA’s Guideline onGeneral Principles of Process Validation, dated May 11, 1987, and in FDA’s Glossary ofComputerized System and Software Development Terminology, dated August 1995.

While IQ/OQ/PQ terminology has served its purpose well and is one of many legitimate ways toorganize software validation tasks at the user site, this terminology may not be well understood amongmany software professionals, and it is not used elsewhere in this document. However, both FDApersonnel and device manufacturers need to be aware of these differences in terminology as they ask forand provide information regarding software validation.

3.2. SOFTWARE DEVELOPMENT AS PART OF SYSTEM DESIGN

The decision to implement system functionality using software is one that is typically made during systemdesign. Software requirements are typically derived from the overall system requirements and design forthose aspects in the system that are to be implemented using software. There are user needs andintended uses for a finished device, but users typically do not specify whether those requirements are tobe met by hardware, software, or some combination of both. Therefore, software validation must beconsidered within the context of the overall design validation for the system.

A documented requirements specification represents the user's needs and intended uses from which theproduct is developed. A primary goal of software validation is to then demonstrate that all completedsoftware products comply with all documented software and system requirements. The correctness andcompleteness of both the system requirements and the software requirements should be addressed aspart of the design validation process for the device. Software validation includes confirmation ofconformance to all software specifications and confirmation that all software requirements are traceableto the system specifications. Confirmation is an important part of the overall design validation to ensurethat all aspects of the medical device conform to user needs and intended uses.

http://www.fda.gov/cdrh/ode/425.pdf

http://www.fda.gov/cdrh/ode/swareval.html

http://www.fda.gov/ora/inspect_ref/igs/gloss.html



Page 8


3.3. SOFTWARE IS DIFFERENT FROM HARDWARE

While software shares many of the same engineering tasks as hardware, it has some very importantdifferences. For example:

• The vast majority of software problems are traceable to errors made during the design anddevelopment process. While the quality of a hardware product is highly dependent on design,development and manufacture, the quality of a software product is dependent primarily ondesign and development with a minimum concern for software manufacture. Softwaremanufacturing consists of reproduction that can be easily verified. It is not difficult tomanufacture thousands of program copies that function exactly the same as the original; thedifficulty comes in getting the original program to meet all specifications.

• One of the most significant features of software is branching, i.e., the ability to executealternative series of commands, based on differing inputs. This feature is a major contributingfactor for another characteristic of software – its complexity. Even short programs can be verycomplex and difficult to fully understand.

• Typically, testing alone cannot fully verify that software is complete and correct. In addition totesting, other verification techniques and a structured and documented development processshould be combined to ensure a comprehensive validation approach.

• Unlike hardware, software is not a physical entity and does not wear out. In fact, software mayimprove with age, as latent defects are discovered and removed. However, as software isconstantly updated and changed, such improvements are sometimes countered by new defectsintroduced into the software during the change.

• Unlike some hardware failures, software failures occur without advanced warning. Thesoftware’s branching that allows it to follow differing paths during execution, may hide somelatent defects until long after a software product has been introduced into the marketplace.

• Another related characteristic of software is the speed and ease with which it can be changed.This factor can cause both software and non-software professionals to believe that softwareproblems can be corrected easily. Combined with a lack of understanding of software, it canlead managers to believe that tightly controlled engineering is not needed as much for softwareas it is for hardware. In fact, the opposite is true. Because of its complexity, thedevelopment process for software should be even more tightly controlled than forhardware, in order to prevent problems that cannot be easily detected later in thedevelopment process.

• Seemingly insignificant changes in software code can create unexpected and very significantproblems elsewhere in the software program. The software development process should besufficiently well planned, controlled, and documented to detect and correct unexpected resultsfrom software changes.

Page 9


• Given the high demand for software professionals and the highly mobile workforce, the softwarepersonnel who make maintenance changes to software may not have been involved in theoriginal software development. Therefore, accurate and thorough documentation is essential.

• Historically, software components have not been as frequently standardized and interchangeableas hardware components. However, medical device software developers are beginning to usecomponent-based development tools and techniques. Object-oriented methodologies and theuse of off-the-shelf software components hold promise for faster and less expensive softwaredevelopment. However, component-based approaches require very careful attention duringintegration. Prior to integration, time is needed to fully define and develop reusable softwarecode and to fully understand the behavior of off-the-shelf components.

For these and other reasons, software engineering needs an even greater level of managerialscrutiny and control than does hardware engineering.

3.4. BENEFITS OF SOFTWARE VALIDATION

Software validation is a critical tool used to assure the quality of device software and softwareautomated operations. Software validation can increase the usability and reliability of the device,resulting in decreased failure rates, fewer recalls and corrective actions, less risk to patients and users,and reduced liability to device manufacturers. Software validation can also reduce long term costs bymaking it easier and less costly to reliably modify software and revalidate software changes. Softwaremaintenance can represent a very large percentage of the total cost of software over its entire life cycle.An established comprehensive software validation process helps to reduce the long-term cost ofsoftware by reducing the cost of validation for each subsequent release of the software.

3.5 DESIGN REVIEW

Design reviews are documented, comprehensive, and systematic examinations of a design to evaluatethe adequacy of the design requirements, to evaluate the capability of the design to meet theserequirements, and to identify problems. While there may be many informal technical reviews that occurwithin the development team during a software project, a formal design review is more structured andincludes participation from others outside the development team. Formal design reviews may referenceor include results from other formal and informal reviews. Design reviews may be conducted separatelyfor the software, after the software is integrated with the hardware into the system, or both. Designreviews should include examination of development plans, requirements specifications, designspecifications, testing plans and procedures, all other documents and activities associated with theproject, verification results from each stage of the defined life cycle, and validation results for the overalldevice.

Design review is a primary tool for managing and evaluating development projects. For example, formaldesign reviews allow management to confirm that all goals defined in the software validation plan have

Page 10


been achieved. The Quality System regulation requires that at least one formal design review beconducted during the device design process. However, it is recommended that multiple design reviewsbe conducted (e.g., at the end of each software life cycle activity, in preparation for proceeding to thenext activity). Formal design review is especially important at or near the end of the requirementsactivity, before major resources have been committed to specific design solutions. Problems found atthis point can be resolved more easily, save time and money, and reduce the likelihood of missing acritical issue.

Answers to some key questions should be documented during formal design reviews. These include:

• Have the appropriate tasks and expected results, outputs, or products been established for eachsoftware life cycle activity?

• Do the tasks and expected results, outputs, or products of each software life cycle activity:

ü Comply with the requirements of other software life cycle activities in terms of correctness,completeness, consistency, and accuracy?

ü Satisfy the standards, practices, and conventions of that activity?

ü Establish a proper basis for initiating tasks for the next software life cycle activity?

Page 11


SECTION 4. PRINCIPLES OF SOFTWARE VALIDATION

This section lists the general principles that should be considered for the validation of software.

4.1. REQUIREMENTS

A documented software requirements specification provides a baseline for both validation andverification. The software validation process cannot be completed without an established softwarerequirements specification (Ref: 21 CFR 820.3(z) and (aa) and 820.30(f) and (g)).

4.2. DEFECT PREVENTION

Software quality assurance needs to focus on preventing the introduction of defects into the softwaredevelopment process and not on trying to “test quality into” the software code after it is written.Software testing is very limited in its ability to surface all latent defects in software code. For example,the complexity of most software prevents it from being exhaustively tested. Software testing is anecessary activity. However, in most cases software testing by itself is not sufficient toestablish confidence that the software is fit for its intended use. In order to establish thatconfidence, software developers should use a mixture of methods and techniques to prevent softwareerrors and to detect software errors that do occur. The “best mix” of methods depends on manyfactors including the development environment, application, size of project, language, and risk.

4.3. TIME AND EFFORT

To build a case that the software is validated requires time and effort. Preparation for softwarevalidation should begin early, i.e., during design and development planning and design input. The finalconclusion that the software is validated should be based on evidence collected from planned effortsconducted throughout the software lifecycle.

4.4. SOFTWARE LIFE CYCLE

Software validation takes place within the environment of an established software life cycle. Thesoftware life cycle contains software engineering tasks and documentation necessary to support thesoftware validation effort. In addition, the software life cycle contains specific verification and validationtasks that are appropriate for the intended use of the software. This guidance does not recommend anyparticular life cycle models – only that they should be selected and used for a software developmentproject.

Page 12


4.5. PLANS

The software validation process is defined and controlled through the use of a plan. The softwarevalidation plan defines “what” is to be accomplished through the software validation effort. Softwarevalidation plans are a significant quality system tool. Software validation plans specify areas such asscope, approach, resources, schedules and the types and extent of activities, tasks, and work items.

4.6. PROCEDURES

The software validation process is executed through the use of procedures. These procedures establish“how” to conduct the software validation effort. The procedures should identify the specific actions orsequence of actions that must be taken to complete individual validation activities, tasks, and workitems.

4.7. SOFTWARE VALIDATION AFTER A CHANGE

Due to the complexity of software, a seemingly small local change may have a significant global systemimpact. When any change (even a small change) is made to the software, the validation status of thesoftware needs to be re-established. Whenever software is changed, a validation analysis shouldbe conducted not just for validation of the individual change, but also to determine the extentand impact of that change on the entire software system. Based on this analysis, the softwaredeveloper should then conduct an appropriate level of software regression testing to show thatunchanged but vulnerable portions of the system have not been adversely affected. Design controls andappropriate regression testing provide the confidence that the software is validated after a softwarechange.

4.8. VALIDATION COVERAGE

Validation coverage should be based on the software’s complexity and safety risk – not on firm size orresource constraints. The selection of validation activities, tasks, and work items should becommensurate with the complexity of the software design and the risk associated with the use of thesoftware for the specified intended use. For lower risk devices, only baseline validation activities maybe conducted. As the risk increases additional validation activities should be added to cover theadditional risk. Validation documentation should be sufficient to demonstrate that all software validationplans and procedures have been completed successfully.

4.9. INDEPENDENCE OF REVIEW

Validation activities should be conducted using the basic quality assurance precept of “independence ofreview.” Self-validation is extremely difficult. When possible, an independent evaluation is alwaysbetter, especially for higher risk applications. Some firms contract out for a third-party independent

Page 13


verification and validation, but this solution may not always be feasible. Another approach is to assigninternal staff members that are not involved in a particular design or its implementation, but who havesufficient knowledge to evaluate the project and conduct the verification and validation activities. Smallerfirms may need to be creative in how tasks are organized and assigned in order to maintain internalindependence of review.

4.10. FLEXIBILITY AND RESPONSIBILITY

Specific implementation of these software validation principles may be quite different from oneapplication to another. The device manufacturer has flexibility in choosing how to apply these validationprinciples, but retains ultimate responsibility for demonstrating that the software has been validated.

Software is designed, developed, validated, and regulated in a wide spectrum of environments, and fora wide variety of devices with varying levels of risk. FDA regulated medical device applications includesoftware that:

• Is a component, part, or accessory of a medical device;• Is itself a medical device; or• Is used in manufacturing, design and development, or other parts of the quality system.

In each environment, software components from many sources may be used to create the application(e.g., in-house developed software, off-the-shelf software, contract software, shareware). In addition,software components come in many different forms (e.g., application software, operating systems,compilers, debuggers, configuration management tools, and many more). The validation of software inthese environments can be a complex undertaking; therefore, it is appropriate that all of these softwarevalidation principles be considered when designing the software validation process. The resultantsoftware validation process should be commensurate with the safety risk associated with the system,device, or process.

Software validation activities and tasks may be dispersed, occurring at different locations and beingconducted by different organizations. However, regardless of the distribution of tasks, contractualrelations, source of components, or the development environment, the device manufacturer orspecification developer retains ultimate responsibility for ensuring that the software is validated.

Page 14


SECTION 5. ACTIVITIES AND TASKS

Software validation is accomplished through a series of activities and tasks that are planned andexecuted at various stages of the software development life cycle. These tasks may be one timeoccurrences or may be iterated many times, depending on the life cycle model used and the scope ofchanges made as the software project progresses.

5.1. SOFTWARE LIFE CYCLE ACTIVITIES

This guidance does not recommend the use of any specific software life cycle model. Softwaredevelopers should establish a software life cycle model that is appropriate for their product andorganization. The software life cycle model that is selected should cover the software from its birth to itsretirement. Activities in a typical software life cycle model include the following:

• Quality Planning• System Requirements Definition• Detailed Software Requirements Specification• Software Design Specification• Construction or Coding• Testing• Installation• Operation and Support• Maintenance• Retirement

Verification, testing, and other tasks that support software validation occur during each of theseactivities. A life cycle model organizes these software development activities in various ways andprovides a framework for monitoring and controlling the software development project. Severalsoftware life cycle models (e.g., waterfall, spiral, rapid prototyping, incremental development, etc.) aredefined in FDA’s Glossary of Computerized System and Software Development Terminology,dated August 1995. These and many other life cycle models are described in various references listedin Appendix A.

5.2. TYPICAL TASKS SUPPORTING VALIDATION

For each of the software life cycle activities, there are certain “typical” tasks that support a conclusionthat the software is validated. However, the specific tasks to be performed, their order of performance,and the iteration and timing of their performance will be dictated by the specific software life cyclemodel that is selected and the safety risk associated with the software application. For very low riskapplications, certain tasks may not be needed at all. However, the software developer should at leastconsider each of these tasks and should define and document which tasks are or are not appropriate for


Page 15


their specific application. The following discussion is generic and is not intended to prescribe anyparticular software life cycle model or any particular order in which tasks are to be performed.

5.2.1. Quality Planning

Design and development planning should culminate in a plan that identifies necessary tasks, proceduresfor anomaly reporting and resolution, necessary resources, and management review requirements,including formal design reviews. A software life cycle model and associated activities should beidentified, as well as those tasks necessary for each software life cycle activity. The plan should include:

• The specific tasks for each life cycle activity;• Enumeration of important quality factors (e.g., reliability, maintainability, and usability);• Methods and procedures for each task;• Task acceptance criteria;• Criteria for defining and documenting outputs in terms that will allow evaluation of their

conformance to input requirements;• Inputs for each task;• Outputs from each task;• Roles, resources, and responsibilities for each task;• Risks and assumptions; and• Documentation of user needs.

Management must identify and provide the appropriate software development environment andresources. (See 21 CFR §820.20(b)(1) and (2).) Typically, each task requires personnel as well asphysical resources. The plan should identify the personnel, the facility and equipment resources for eachtask, and the role that risk (hazard) management will play. A configuration management plan should bedeveloped that will guide and control multiple parallel development activities and ensure propercommunications and documentation. Controls are necessary to ensure positive and correctcorrespondence among all approved versions of the specifications documents, source code, objectcode, and test suites that comprise a software system. The controls also should ensure accurateidentification of, and access to, the currently approved versions.

Procedures should be created for reporting and resolving software anomalies found through validationor other activities. Management should identify the reports and specify the contents, format, andresponsible organizational elements for each report. Procedures also are necessary for the review andapproval of software development results, including the responsible organizational elements for suchreviews and approvals.

Typical Tasks – Quality Planning

• Risk (Hazard) Management Plan• Configuration Management Plan

Page 16


• Software Quality Assurance Plan− Software Verification and Validation Plan

q Verification and Validation Tasks, and Acceptance Criteriaq Schedule and Resource Allocation (for software verification and validation activities)q Reporting Requirements

− Formal Design Review Requirements− Other Technical Review Requirements

• Problem Reporting and Resolution Procedures• Other Support Activities

5.2.2. Requirements

Requirements development includes the identification, analysis, and documentation of information aboutthe device and its intended use. Areas of special importance include allocation of system functions tohardware/software, operating conditions, user characteristics, potential hazards, and anticipated tasks.In addition, the requirements should state clearly the intended use of the software.

The software requirements specification document should contain a written definition of the softwarefunctions. It is not possible to validate software without predetermined and documented softwarerequirements. Typical software requirements specify the following:

• All software system inputs;• All software system outputs;• All functions that the software system will perform;• All performance requirements that the software will meet, (e.g., data throughput, reliability, and

timing);• The definition of all external and user interfaces, as well as any internal software-to-system

interfaces;• How users will interact with the system;• What constitutes an error and how errors should be handled;• Required response times;• The intended operating environment for the software, if this is a design constraint (e.g.,

hardware platform, operating system);• All ranges, limits, defaults, and specific values that the software will accept; and• All safety related requirements, specifications, features, or functions that will be implemented in

software.

Software safety requirements are derived from a technical risk management process that is closelyintegrated with the system requirements development process. Software requirement specificationsshould identify clearly the potential hazards that can result from a software failure in the system as wellas any safety requirements to be implemented in software. The consequences of software failure shouldbe evaluated, along with means of mitigating such failures (e.g., hardware mitigation, defensiveprogramming, etc.). From this analysis, it should be possible to identify the most appropriate measuresnecessary to prevent harm.

Page 17


The Quality System regulation requires a mechanism for addressing incomplete, ambiguous, orconflicting requirements. (See 21 CFR 820.30(c).) Each requirement (e.g., hardware, software, user,operator interface, and safety) identified in the software requirements specification should be evaluatedfor accuracy, completeness, consistency, testability, correctness, and clarity. For example, softwarerequirements should be evaluated to verify that:

• There are no internal inconsistencies among requirements;• All of the performance requirements for the system have been spelled out;• Fault tolerance, safety, and security requirements are complete and correct;• Allocation of software functions is accurate and complete;• Software requirements are appropriate for the system hazards; and• All requirements are expressed in terms that are measurable or objectively verifiable.

A software requirements traceability analysis should be conducted to trace software requirements to(and from) system requirements and to risk analysis results. In addition to any other analyses anddocumentation used to verify software requirements, a formal design review is recommended to confirmthat requirements are fully specified and appropriate before extensive software design efforts begin.Requirements can be approved and released incrementally, but care should be taken that interactionsand interfaces among software (and hardware) requirements are properly reviewed, analyzed, andcontrolled.

Typical Tasks – Requirements

• Preliminary Risk Analysis• Traceability Analysis

− Software Requirements to System Requirements (and vice versa)− Software Requirements to Risk Analysis

• Description of User Characteristics• Listing of Characteristics and Limitations of Primary and Secondary Memory• Software Requirements Evaluation• Software User Interface Requirements Analysis• System Test Plan Generation• Acceptance Test Plan Generation• Ambiguity Review or Analysis

5.2.3. Design

In the design process, the software requirements specification is translated into a logical and physicalrepresentation of the software to be implemented. The software design specification is a description ofwhat the software should do and how it should do it. Due to complexity of the project or to enable

Page 18


persons with varying levels of technical responsibilities to clearly understand design information, thedesign specification may contain both a high level summary of the design and detailed designinformation. The completed software design specification constrains the programmer/coder to staywithin the intent of the agreed upon requirements and design. A complete software design specificationwill relieve the programmer from the need to make ad hoc design decisions.

The software design needs to address human factors. Use error caused by designs that are eitheroverly complex or contrary to users' intuitive expectations for operation is one of the most persistent andcritical problems encountered by FDA. Frequently, the design of the software is a factor in such useerrors. Human factors engineering should be woven into the entire design and development process,including the device design requirements, analyses, and tests. Device safety and usability issues shouldbe considered when developing flowcharts, state diagrams, prototyping tools, and test plans. Also, taskand function analyses, risk analyses, prototype tests and reviews, and full usability tests should beperformed. Participants from the user population should be included when applying thesemethodologies.

The software design specification should include:

• Software requirements specification, including predetermined criteria for acceptance of thesoftware;

• Software risk analysis;• Development procedures and coding guidelines (or other programming procedures);• Systems documentation (e.g., a narrative or a context diagram) that describes the systems

context in which the program is intended to function, including the relationship of hardware,software, and the physical environment;

• Hardware to be used;• Parameters to be measured or recorded;• Logical structure (including control logic) and logical processing steps (e.g., algorithms);• Data structures and data flow diagrams;• Definitions of variables (control and data) and description of where they are used;• Error, alarm, and warning messages;• Supporting software (e.g., operating systems, drivers, other application software);• Communication links (links among internal modules of the software, links with the supporting

software, links with the hardware, and links with the user);• Security measures (both physical and logical security); and• Any additional constraints not identified in the above elements.

The first four of the elements noted above usually are separate pre-existing documents that are includedby reference in the software design specification. Software requirements specification was discussed inthe preceding section, as was software risk analysis. Written development procedures serve as a guideto the organization, and written programming procedures serve as a guide to individual programmers.As software cannot be validated without knowledge of the context in which it is intended to function,systems documentation is referenced. If some of the above elements are not included in the software, it

Page 19


may be helpful to future reviewers and maintainers of the software if that is clearly stated (e.g., There areno error messages in this program).

The activities that occur during software design have several purposes. Software design evaluations areconducted to determine if the design is complete, correct, consistent, unambiguous, feasible, andmaintainable. Appropriate consideration of software architecture (e.g., modular structure) during designcan reduce the magnitude of future validation efforts when software changes are needed. Softwaredesign evaluations may include analyses of control flow, data flow, complexity, timing, sizing, memoryallocation, criticality analysis, and many other aspects of the design. A traceability analysis should beconducted to verify that the software design implements all of the software requirements. As atechnique for identifying where requirements are not sufficient, the traceability analysis should also verifythat all aspects of the design are traceable to software requirements. An analysis of communication linksshould be conducted to evaluate the proposed design with respect to hardware, user, and relatedsoftware requirements. The software risk analysis should be re-examined to determine whether anyadditional hazards have been identified and whether any new hazards have been introduced by thedesign.

At the end of the software design activity, a Formal Design Review should be conducted to verify thatthe design is correct, consistent, complete, accurate, and testable, before moving to implement thedesign. Portions of the design can be approved and released incrementally for implementation; but careshould be taken that interactions and communication links among various elements are properlyreviewed, analyzed, and controlled.

Most software development models will be iterative. This is likely to result in several versions of boththe software requirement specification and the software design specification. All approved versionsshould be archived and controlled in accordance with established configuration managementprocedures.

Typical Tasks – Design

• Updated Software Risk Analysis• Traceability Analysis - Design Specification to Software Requirements (and vice versa)• Software Design Evaluation• Design Communication Link Analysis• Module Test Plan Generation• Integration Test Plan Generation• Test Design Generation (module, integration, system, and acceptance)

Page 20


5.2.4. Construction or Coding

Software may be constructed either by coding (i.e., programming) or by assembling together previouslycoded software components (e.g., from code libraries, off-the-shelf software, etc.) for use in a newapplication. Coding is the software activity where the detailed design specification is implemented assource code. Coding is the lowest level of abstraction for the software development process. It is thelast stage in decomposition of the software requirements where module specifications are translated intoa programming language.

Coding usually involves the use of a high-level programming language, but may also entail the use ofassembly language (or microcode) for time-critical operations. The source code may be eithercompiled or interpreted for use on a target hardware platform. Decisions on the selection ofprogramming languages and software build tools (assemblers, linkers, and compilers) should includeconsideration of the impact on subsequent quality evaluation tasks (e.g., availability of debugging andtesting tools for the chosen language). Some compilers offer optional levels and commands for errorchecking to assist in debugging the code. Different levels of error checking may be used throughout thecoding process, and warnings or other messages from the compiler may or may not be recorded.However, at the end of the coding and debugging process, the most rigorous level of error checking isnormally used to document what compilation errors still remain in the software. If the most rigorouslevel of error checking is not used for final translation of the source code, then justification for use of theless rigorous translation error checking should be documented. Also, for the final compilation, thereshould be documentation of the compilation process and its outcome, including any warnings or othermessages from the compiler and their resolution, or justification for the decision to leave issuesunresolved.

Firms frequently adopt specific coding guidelines that establish quality policies and procedures related tothe software coding process. Source code should be evaluated to verify its compliance with specifiedcoding guidelines. Such guidelines should include coding conventions regarding clarity, style, complexitymanagement, and commenting. Code comments should provide useful and descriptive information for amodule, including expected inputs and outputs, variables referenced, expected data types, andoperations to be performed. Source code should also be evaluated to verify its compliance with thecorresponding detailed design specification. Modules ready for integration and test should havedocumentation of compliance with coding guidelines and any other applicable quality policies andprocedures.

Source code evaluations are often implemented as code inspections and code walkthroughs. Suchstatic analyses provide a very effective means to detect errors before execution of the code. They allowfor examination of each error in isolation and can also help in focusing later dynamic testing of thesoftware. Firms may use manual (desk) checking with appropriate controls to ensure consistency andindependence. Source code evaluations should be extended to verification of internal linkages betweenmodules and layers (horizontal and vertical interfaces), and compliance with their design specifications.Documentation of the procedures used and the results of source code evaluations should be maintainedas part of design verification.

Page 21


A source code traceability analysis is an important tool to verify that all code is linked to establishedspecifications and established test procedures. A source code traceability analysis should be conductedand documented to verify that:

• Each element of the software design specification has been implemented in code;• Modules and functions implemented in code can be traced back to an element in the software

design specification and to the risk analysis;• Tests for modules and functions can be traced back to an element in the software design

specification and to the risk analysis; and• Tests for modules and functions can be traced to source code for the same modules and

functions.

Typical Tasks – Construction or Coding

• Traceability Analyses− Source Code to Design Specification (and vice versa)− Test Cases to Source Code and to Design Specification

• Source Code and Source Code Documentation Evaluation• Source Code Interface Analysis• Test Procedure and Test Case Generation (module, integration, system, and

acceptance)

5.2.5. Testing by the Software Developer

Software testing entails running software products under known conditions with defined inputs anddocumented outcomes that can be compared to their predefined expectations. It is a time consuming,difficult, and imperfect activity. As such, it requires early planning in order to be effective and efficient.

Test plans and test cases should be created as early in the software development process as feasible.They should identify the schedules, environments, resources (personnel, tools, etc.), methodologies,cases (inputs, procedures, outputs, expected results), documentation, and reporting criteria. Themagnitude of effort to be applied throughout the testing process can be linked to complexity, criticality,reliability, and/or safety issues (e.g., requiring functions or modules that produce critical outcomes to bechallenged with intensive testing of their fault tolerance features). Descriptions of categories of softwareand software testing effort appear in the literature, for example:

• NIST Special Publication 500-235, Structured Testing: A Testing Methodology Using theCyclomatic Complexity Metric;

• NUREG/CR-6293, Verification and Validation Guidelines for High Integrity Systems; and• IEEE Computer Society Press, Handbook of Software Reliability Engineering.

Page 22


Software test plans should identify the particular tasks to be conducted at each stage of developmentand include justification of the level of effort represented by their corresponding completion criteria.

Software testing has limitations that must be recognized and considered when planning the testing of aparticular software product. Except for the simplest of programs, software cannot be exhaustivelytested. Generally it is not feasible to test a software product with all possible inputs, nor is it possible totest all possible data processing paths that can occur during program execution. There is no one type oftesting or testing methodology that can ensure a particular software product has been thoroughly tested.Testing of all program functionality does not mean all of the program has been tested. Testing of all of aprogram's code does not mean all necessary functionality is present in the program. Testing of allprogram functionality and all program code does not mean the program is 100% correct! Softwaretesting that finds no errors should not be interpreted to mean that errors do not exist in the softwareproduct; it may mean the testing was superficial.

An essential element of a software test case is the expected result. It is the key detail that permitsobjective evaluation of the actual test result. This necessary testing information is obtained from thecorresponding, predefined definition or specification. A software specification document must identifywhat, when, how, why, etc., is to be achieved with an engineering (i.e., measurable or objectivelyverifiable) level of detail in order for it to be confirmed through testing. The real effort of effectivesoftware testing lies in the definition of what is to be tested rather than in the performance of the test.

A software testing process should be based on principles that foster effective examinations of a softwareproduct. Applicable software testing tenets include:

• The expected test outcome is predefined;• A good test case has a high probability of exposing an error;• A successful test is one that finds an error;• There is independence from coding;• Both application (user) and software (programming) expertise are employed;• Testers use different tools from coders;• Examining only the usual case is insufficient;• Test documentation permits its reuse and an independent confirmation of the pass/fail status of a

test outcome during subsequent review.

Once the prerequisite tasks (e.g., code inspection) have been successfully completed, software testingbegins. It starts with unit level testing and concludes with system level testing. There may be a distinctintegration level of testing. A software product should be challenged with test cases based on its internalstructure and with test cases based on its external specification. These tests should provide a thoroughand rigorous examination of the software product's compliance with its functional, performance, andinterface definitions and requirements.

Code-based testing is also known as structural testing or "white-box" testing. It identifies test casesbased on knowledge obtained from the source code, detailed design specification, and otherdevelopment documents. These test cases challenge the control decisions made by the program; andthe program's data structures including configuration tables. Structural testing can identify "dead" code

Page 23


that is never executed when the program is run. Structural testing is accomplished primarily with unit(module) level testing, but can be extended to other levels of software testing.

The level of structural testing can be evaluated using metrics that are designed to show what percentageof the software structure has been evaluated during structural testing. These metrics are typicallyreferred to as “coverage” and are a measure of completeness with respect to test selection criteria. Theamount of structural coverage should be commensurate with the level of risk posed by the software.Use of the term “coverage” usually means 100% coverage. For example, if a testing program hasachieved “statement coverage,” it means that 100% of the statements in the software have beenexecuted at least once. Common structural coverage metrics include:

• Statement Coverage – This criteria requires sufficient test cases for each program statementto be executed at least once; however, its achievement is insufficient to provide confidence in asoftware product's behavior.

• Decision (Branch) Coverage – This criteria requires sufficient test cases for each programdecision or branch to be executed so that each possible outcome occurs at least once. It isconsidered to be a minimum level of coverage for most software products, but decisioncoverage alone is insufficient for high-integrity applications.

• Condition Coverage – This criteria requires sufficient test cases for each condition in aprogram decision to take on all possible outcomes at least once. It differs from branchcoverage only when multiple conditions must be evaluated to reach a decision.

• Multi-Condition Coverage – This criteria requires sufficient test cases to exercise all possiblecombinations of conditions in a program decision.

• Loop Coverage – This criteria requires sufficient test cases for all program loops to beexecuted for zero, one, two, and many iterations covering initialization, typical running andtermination (boundary) conditions.

• Path Coverage – This criteria requires sufficient test cases for each feasible path, basis path,etc., from start to exit of a defined program segment, to be executed at least once. Because ofthe very large number of possible paths through a software program, path coverage is generallynot achievable. The amount of path coverage is normally established based on the risk orcriticality of the software under test.

• Data Flow Coverage – This criteria requires sufficient test cases for each feasible data flow tobe executed at least once. A number of data flow testing strategies are available.

Definition-based or specification-based testing is also known as functional testing or "black-box" testing.It identifies test cases based on the definition of what the software product (whether it be a unit(module) or a complete program) is intended to do. These test cases challenge the intended use orfunctionality of a program, and the program's internal and external interfaces. Functional testing can beapplied at all levels of software testing, from unit to system level testing.

Page 24


The following types of functional software testing involve generally increasing levels of effort:

• Normal Case – Testing with usual inputs is necessary. However, testing a software productonly with expected, valid inputs does not thoroughly test that software product. By itself,normal case testing cannot provide sufficient confidence in the dependability of the softwareproduct.

• Output Forcing – Choosing test inputs to ensure that selected (or all) software outputs aregenerated by testing.

• Robustness – Software testing should demonstrate that a software product behaves correctlywhen given unexpected, invalid inputs. Methods for identifying a sufficient set of such test casesinclude Equivalence Class Partitioning, Boundary Value Analysis, and Special CaseIdentification (Error Guessing). While important and necessary, these techniques do not ensurethat all of the most appropriate challenges to a software product have been identified for testing.

• Combinations of Inputs – The functional testing methods identified above all emphasizeindividual or single test inputs. Most software products operate with multiple inputs under theirconditions of use. Thorough software product testing should consider the combinations ofinputs a software unit or system may encounter during operation. Error guessing can beextended to identify combinations of inputs, but it is an ad hoc technique. Cause-effect graphingis one functional software testing technique that systematically identifies combinations of inputsto a software product for inclusion in test cases.

Functional and structural software test case identification techniques provide specific inputs for testing,rather than random test inputs. One weakness of these techniques is the difficulty in linking structuraland functional test completion criteria to a software product's reliability. Advanced software testingmethods, such as statistical testing, can be employed to provide further assurance that a softwareproduct is dependable. Statistical testing uses randomly generated test data from defined distributionsbased on an operational profile (e.g., expected use, hazardous use, or malicious use of the softwareproduct). Large amounts of test data are generated and can be targeted to cover particular areas orconcerns, providing an increased possibility of identifying individual and multiple rare operatingconditions that were not anticipated by either the software product's designers or its testers. Statisticaltesting also provides high structural coverage. It does require a stable software product. Thus,structural and functional testing are prerequisites for statistical testing of a software product.

Another aspect of software testing is the testing of software changes. Changes occur frequently duringsoftware development. These changes are the result of 1) debugging that finds an error and it iscorrected, 2) new or changed requirements ("requirements creep"), and 3) modified designs as moreeffective or efficient implementations are found. Once a software product has been baselined(approved), any change to that product should have its own “mini life cycle,” including testing. Testingof a changed software product requires additional effort. Not only should it demonstrate that thechange was implemented correctly, testing should also demonstrate that the change did not adverselyimpact other parts of the software product. Regression analysis and testing are employed to provide

Page 25


assurance that a change has not created problems elsewhere in the software product. Regressionanalysis is the determination of the impact of a change based on review of the relevant documentation(e.g., software requirements specification, software design specification, source code, test plans, testcases, test scripts, etc.) in order to identify the necessary regression tests to be run. Regression testingis the rerunning of test cases that a program has previously executed correctly and comparing thecurrent result to the previous result in order to detect unintended effects of a software change.Regression analysis and regression testing should also be employed when using integration methods tobuild a software product to ensure that newly integrated modules do not adversely impact the operationof previously integrated modules.

In order to provide a thorough and rigorous examination of a software product, development testing istypically organized into levels. As an example, a software product's testing can be organized into unit,integration, and system levels of testing.

1) Unit (module or component) level testing focuses on the early examination of sub-programfunctionality and ensures that functionality not visible at the system level is examined by testing. Unittesting ensures that quality software units are furnished for integration into the finished softwareproduct.

2) Integration level testing focuses on the transfer of data and control across a program's internal andexternal interfaces. External interfaces are those with other software (including operating systemsoftware), system hardware, and the users and can be described as communications links.

3) System level testing demonstrates that all specified functionality exists and that the software productis trustworthy. This testing verifies the as-built program's functionality and performance with respectto the requirements for the software product as exhibited on the specified operating platform(s).System level software testing addresses functional concerns and the following elements of a device'ssoftware that are related to the intended use(s):

• Performance issues (e.g., response times, reliability measurements);• Responses to stress conditions, e.g., behavior under maximum load, continuous use;• Operation of internal and external security features;• Effectiveness of recovery procedures, including disaster recovery;• Usability;• Compatibility with other software products;• Behavior in each of the defined hardware configurations; and• Accuracy of documentation.

Control measures (e.g., a traceability analysis) should be used to ensure that the intended coverage isachieved.

System level testing also exhibits the software product's behavior in the intended operating environment.The location of such testing is dependent upon the software developer's ability to produce the targetoperating environment(s). Depending upon the circumstances, simulation and/or testing at (potential)customer locations may be utilized. Test plans should identify the controls needed to ensure that the

Page 26


intended coverage is achieved and that proper documentation is prepared when planned system leveltesting is conducted at sites not directly controlled by the software developer. Also, for a softwareproduct that is a medical device or a component of a medical device that is to be used on humans priorto FDA clearance, testing involving human subjects may require an Investigational Device Exemption(IDE) or Institutional Review Board (IRB) approval.

Test procedures, test data, and test results should be documented in a manner permitting objectivepass/fail decisions to be reached. They should also be suitable for review and objective decisionmaking subsequent to running the test, and they should be suitable for use in any subsequent regressiontesting. Errors detected during testing should be logged, classified, reviewed, and resolved prior torelease of the software. Software error data that is collected and analyzed during a development lifecycle may be used to determine the suitability of the software product for release for commercialdistribution. Test reports should comply with the requirements of the corresponding test plans.

Software products that perform useful functions in medical devices or their production are oftencomplex. Software testing tools are frequently used to ensure consistency, thoroughness, and efficiencyin the testing of such software products and to fulfill the requirements of the planned testing activities.These tools may include supporting software built in-house to facilitate unit (module) testing andsubsequent integration testing (e.g., drivers and stubs) as well as commercial software testing tools.Such tools should have a degree of quality no less than the software product they are used to develop.Appropriate documentation providing evidence of the validation of these software tools for theirintended use should be maintained (see section 6 of this guidance).

Typical Tasks – Testing by the Software Developer

• Test Planning• Structural Test Case Identification• Functional Test Case Identification• Traceability Analysis - Testing

− Unit (Module) Tests to Detailed Design− Integration Tests to High Level Design− System Tests to Software Requirements

• Unit (Module) Test Execution• Integration Test Execution• Functional Test Execution• System Test Execution• Acceptance Test Execution• Test Results Evaluation• Error Evaluation/Resolution• Final Test Report

Page 27


5.2.6. User Site Testing

Testing at the user site is an essential part of software validation. The Quality System regulation requiresinstallation and inspection procedures (including testing where appropriate) as well as documentation ofinspection and testing to demonstrate proper installation. (See 21 CFR §820.170.) Likewise,manufacturing equipment must meet specified requirements, and automated systems must be validatedfor their intended use. (See 21 CFR §820.70(g) and 21 CFR §820.70(i) respectively.)

Terminology regarding user site testing can be confusing. Terms such as beta test, site validation, useracceptance test, installation verification, and installation testing have all been used to describe user sitetesting. For purposes of this guidance, the term “user site testing” encompasses all of these and anyother testing that takes place outside of the developer’s controlled environment. This testing should takeplace at a user's site with the actual hardware and software that will be part of the installed systemconfiguration. The testing is accomplished through either actual or simulated use of the software beingtested within the context in which it is intended to function.

Guidance contained here is general in nature and is applicable to any user site testing. However, insome areas (e.g., blood establishment systems) there may be specific site validation issues that need tobe considered in the planning of user site testing. Test planners should check with the FDA Center(s)with the corresponding product jurisdiction to determine whether there are any additional regulatoryrequirements for user site testing.

User site testing should follow a pre-defined written plan with a formal summary of testing and a recordof formal acceptance. Documented evidence of all testing procedures, test input data, and test resultsshould be retained.

There should be evidence that hardware and software are installed and configured as specified.Measures should ensure that all system components are exercised during the testing and that theversions of these components are those specified. The testing plan should specify testing throughout thefull range of operating conditions and should specify continuation for a sufficient time to allow the systemto encounter a wide spectrum of conditions and events in an effort to detect any latent faults that are notapparent during more normal activities.

Some of the evaluations that have been performed earlier by the software developer at the developer'ssite should be repeated at the site of actual use. These may include tests for a high volume of data,heavy loads or stresses, security, fault testing (avoidance, detection, tolerance, and recovery), errormessages, and implementation of safety requirements. The developer may be able to furnish the userwith some of the test data sets to be used for this purpose.

In addition to an evaluation of the system's ability to properly perform its intended functions, thereshould be an evaluation of the ability of the users of the system to understand and correctly interfacewith it. Operators should be able to perform the intended functions and respond in an appropriate andtimely manner to all alarms, warnings, and error messages.

Page 28


During user site testing, records should be maintained of both proper system performance and anysystem failures that are encountered. The revision of the system to compensate for faults detectedduring this user site testing should follow the same procedures and controls as for any other softwarechange.

The developers of the software may or may not be involved in the user site testing. If the developersare involved, they may seamlessly carry over to the user's site the last portions of design-level systemstesting. If the developers are not involved, it is all the more important that the user have persons whounderstand the importance of careful test planning, the definition of expected test results, and therecording of all test outputs.

Typical Tasks – User Site Testing

• Acceptance Test Execution• Test Results Evaluation• Error Evaluation/Resolution• Final Test Report

5.2.7. Maintenance and Software Changes

As applied to software, the term maintenance does not mean the same as when applied to hardware.The operational maintenance of hardware and software are different because their failure/errormechanisms are different. Hardware maintenance typically includes preventive hardware maintenanceactions, component replacement, and corrective changes. Software maintenance includes corrective,perfective, and adaptive maintenance but does not include preventive maintenance actions or softwarecomponent replacement.

Changes made to correct errors and faults in the software are corrective maintenance. Changes madeto the software to improve the performance, maintainability, or other attributes of the software systemare perfective maintenance. Software changes to make the software system usable in a changedenvironment are adaptive maintenance.

When changes are made to a software system, either during initial development or during post releasemaintenance, sufficient regression analysis and testing should be conducted to demonstrate that portionsof the software not involved in the change were not adversely impacted. This is in addition to testingthat evaluates the correctness of the implemented change(s).

The specific validation effort necessary for each software change is determined by the type of change,the development products affected, and the impact of those products on the operation of the software.Careful and complete documentation of the design structure and interrelationships of various modules,interfaces, etc., can limit the validation effort needed when a change is made. The level of effort needed

Page 29


to fully validate a change is also dependent upon the degree to which validation of the original softwarewas documented and archived. For example, test documentation, test cases, and results of previousverification and validation testing need to be archived if they are to be available for performingsubsequent regression testing. Failure to archive this information for later use can significantly increasethe level of effort and expense of revalidating the software after a change is made.

In addition to software verification and validation tasks that are part of the standard softwaredevelopment process, the following additional maintenance tasks should be addressed:

• Software Validation Plan Revision - For software that was previously validated, the existingsoftware validation plan should be revised to support the validation of the revised software. Ifno previous software validation plan exists, such a plan should be established to support thevalidation of the revised software.

• Anomaly Evaluation – Software organizations frequently maintain documentation, such assoftware problem reports that describe software anomalies discovered and the specificcorrective action taken to fix each anomaly. Too often, however, mistakes are repeatedbecause software developers do not take the next step to determine the root causes ofproblems and make the process and procedural changes needed to avoid recurrence of theproblem. Software anomalies should be evaluated in terms of their severity and their effects onsystem operation and safety, but they should also be treated as symptoms of processdeficiencies in the quality system. A root cause analysis of anomalies can identify specific qualitysystem deficiencies. Where trends are identified (e.g., recurrence of similar softwareanomalies), appropriate corrective and preventive actions must be implemented anddocumented to avoid further recurrence of similar quality problems. (See 21 CFR 820.100.)

• Problem Identification and Resolution Tracking - All problems discovered duringmaintenance of the software should be documented. The resolution of each problem should betracked to ensure it is fixed, for historical reference, and for trending.

• Proposed Change Assessment - All proposed modifications, enhancements, or additionsshould be assessed to determine the effect each change would have on the system. Thisinformation should determine the extent to which verification and/or validation tasks need to beiterated.

• Task Iteration - For approved software changes, all necessary verification and validationtasks should be performed to ensure that planned changes are implemented correctly, alldocumentation is complete and up to date, and no unacceptable changes have occurred insoftware performance.

• Documentation Updating – Documentation should be carefully reviewed to determine whichdocuments have been impacted by a change. All approved documents (e.g., specifications, testprocedures, user manuals, etc.) that have been affected should be updated in accordance withconfiguration management procedures. Specifications should be updated before anymaintenance and software changes are made.

Page 30


SECTION 6. VALIDATION OF AUTOMATED PROCESSEQUIPMENT AND QUALITY SYSTEM SOFTWARE

The Quality System regulation requires that “when computers or automated data processing systems areused as part of production or the quality system, the [device] manufacturer shall validate computersoftware for its intended use according to an established protocol.” (See 21 CFR §820.70(i)). This hasbeen a regulatory requirement of FDA’s medical device Good Manufacturing Practice (GMP)regulations since 1978.

In addition to the above validation requirement, computer systems that implement part of a devicemanufacturer’s production processes or quality system (or that are used to create and maintain recordsrequired by any other FDA regulation) are subject to the Electronic Records; Electronic Signaturesregulation. (See 21 CFR Part 11.) This regulation establishes additional security, data integrity, andvalidation requirements when records are created or maintained electronically. These additional Part 11requirements should be carefully considered and included in system requirements and softwarerequirements for any automated record `keeping systems. System validation and software validationshould demonstrate that all Part 11 requirements have been met.

Computers and automated equipment are used extensively throughout all aspects of medical devicedesign, laboratory testing and analysis, product inspection and acceptance, production and processcontrol, environmental controls, packaging, labeling, traceability, document control, complaintmanagement, and many other aspects of the quality system. Increasingly, automated plant flooroperations can involve extensive use of embedded systems in:

• programmable logic controllers;• digital function controllers;• statistical process control;• supervisory control and data acquisition;• robotics;• human-machine interfaces;• input/output devices; and• computer operating systems.

Software tools are frequently used to design, build, and test the software that goes into an automatedmedical device. Many other commercial software applications, such as word processors, spreadsheets,databases, and flowcharting software are used to implement the quality system. All of these applicationsare subject to the requirement for software validation, but the validation approach used for eachapplication can vary widely.

Whether production or quality system software is developed in-house by the device manufacturer,developed by a contractor, or purchased off-the-shelf, it should be developed using the basic principles

Page 31


outlined elsewhere in this guidance. The device manufacturer has latitude and flexibility in defining howvalidation of that software will be accomplished, but validation should be a key consideration in decidinghow and by whom the software will be developed or from whom it will be purchased. The softwaredeveloper defines a life cycle model. Validation is typically supported by:

• verifications of the outputs from each stage of that software development life cycle; and• checking for proper operation of the finished software in the device manufacturer’s intended use

environment.

6.1. HOW MUCH VALIDATION EVIDENCE IS NEEDED?

The level of validation effort should be commensurate with the risk posed by the automated operation.In addition to risk other factors, such as the complexity of the process software and the degree to whichthe device manufacturer is dependent upon that automated process to produce a safe and effectivedevice, determine the nature and extent of testing needed as part of the validation effort. Documentedrequirements and risk analysis of the automated process help to define the scope of the evidenceneeded to show that the software is validated for its intended use. For example, an automated millingmachine may require very little testing if the device manufacturer can show that the output of theoperation is subsequently fully verified against the specification before release. On the other hand,extensive testing may be needed for:

• a plant-wide electronic record and electronic signature system;• an automated controller for a sterilization cycle; or• automated test equipment used for inspection and acceptance of finished circuit boards in a life-

sustaining / life-supporting device.

Numerous commercial software applications may be used as part of the quality system (e.g., aspreadsheet or statistical package used for quality system calculations, a graphics package used fortrend analysis, or a commercial database used for recording device history records or for complaintmanagement). The extent of validation evidence needed for such software depends on the devicemanufacturer’s documented intended use of that software. For example, a device manufacturer whochooses not to use all the vendor-supplied capabilities of the software only needs to validate thosefunctions that will be used and for which the device manufacturer is dependent upon the software resultsas part of production or the quality system. However, high risk applications should not be running in thesame operating environment with non-validated software functions, even if those software functions arenot used. Risk mitigation techniques such as memory partitioning or other approaches to resourceprotection may need to be considered when high risk applications and lower risk applications are to beused in the same operating environment. When software is upgraded or any changes are made to thesoftware, the device manufacturer should consider how those changes may impact the “used portions”of the software and must reconfirm the validation of those portions of the software that are used. (See21 CFR §820.70(i).)

Page 32


6.2. DEFINED USER REQUIREMENTS

A very important key to software validation is a documented user requirements specification thatdefines:

• the “intended use” of the software or automated equipment; and• the extent to which the device manufacturer is dependent upon that software or equipment for

production of a quality medical device.

The device manufacturer (user) needs to define the expected operating environment including anyrequired hardware and software configurations, software versions, utilities, etc. The user also needs to:

• document requirements for system performance, quality, error handling, startup, shutdown,security, etc.;

• identify any safety related functions or features, such as sensors, alarms, interlocks, logicalprocessing steps, or command sequences; and

• define objective criteria for determining acceptable performance.

The validation must be conducted in accordance with a documented protocol, and the validation resultsmust also be documented. (See 21 CFR §820.70(i).) Test cases should be documented that willexercise the system to challenge its performance against the pre-determined criteria, especially for itsmost critical parameters. Test cases should address error and alarm conditions, startup, shutdown, allapplicable user functions and operator controls, potential operator errors, maximum and minimumranges of allowed values, and stress conditions applicable to the intended use of the equipment. Thetest cases should be executed and the results should be recorded and evaluated to determine whetherthe results support a conclusion that the software is validated for its intended use.

A device manufacturer may conduct a validation using their own personnel or may depend on a thirdparty such as the equipment/software vendor or a consultant. In any case, the device manufacturerretains the ultimate responsibility for ensuring that the production and quality system software:

• is validated according to a written procedure for the particular intended use; and• will perform as intended in the chosen application.

The device manufacturer should have documentation including:

• defined user requirements;• validation protocol used;• acceptance criteria;• test cases and results; and• a validation summary

that objectively confirms that the software is validated for its intended use.

Page 33


6.3. VALIDATION OF OFF-THE-SHELF SOFTWARE AND AUTOMATED EQUIPMENT

Most of the automated equipment and systems used by device manufacturers are supplied by third-party vendors and are purchased off-the-shelf (OTS). The device manufacturer is responsible forensuring that the product development methodologies used by the OTS software developer areappropriate and sufficient for the device manufacturer’s intended use of that OTS software. For OTSsoftware and equipment, the device manufacturer may or may not have access to the vendor’s softwarevalidation documentation. If the vendor can provide information about their system requirements,software requirements, validation process, and the results of their validation, the medical devicemanufacturer can use that information as a beginning point for their required validation documentation.The vendor’s life cycle documentation, such as testing protocols and results, source code, designspecification, and requirements specification, can be useful in establishing that the software has beenvalidated. However, such documentation is frequently not available from commercial equipmentvendors, or the vendor may refuse to share their proprietary information.

Where possible and depending upon the device risk involved, the device manufacturer should considerauditing the vendor’s design and development methodologies used in the construction of the OTSsoftware and should assess the development and validation documentation generated for the OTSsoftware. Such audits can be conducted by the device manufacturer or by a qualified third party. Theaudit should demonstrate that the vendor’s procedures for and results of the verification and validationactivities performed the OTS software are appropriate and sufficient for the safety and effectivenessrequirements of the medical device to be produced using that software.

Some vendors who are not accustomed to operating in a regulated environment may not have adocumented life cycle process that can support the device manufacturer’s validation requirement. Othervendors may not permit an audit. Where necessary validation information is not available from thevendor, the device manufacturer will need to perform sufficient system level “black box” testing toestablish that the software meets their “user needs and intended uses.” For many applications black boxtesting alone is not sufficient. Depending upon the risk of the device produced, the role of the OTSsoftware in the process, the ability to audit the vendor, and the sufficiency of vendor-suppliedinformation, the use of OTS software or equipment may or may not be appropriate, especially if thereare suitable alternatives available. The device manufacturer should also consider the implications (if any)for continued maintenance and support of the OTS software should the vendor terminate their support.

For some off-the-shelf software development tools, such as software compilers, linkers, editors, andoperating systems, exhaustive black-box testing by the device manufacturer may be impractical.Without such testing – a key element of the validation effort – it may not be possible to validate thesesoftware tools. However, their proper operation may be satisfactorily inferred by other means. Forexample, compilers are frequently certified by independent third-party testing, and commercial softwareproducts may have “bug lists”, system requirements and other operational information available from thevendor that can be compared to the device manufacturer’s intended use to help focus the “black-box”testing effort. Off-the-shelf operating systems need not be validated as a separate program. However,system-level validation testing of the application software should address all the operating systemservices used, including maximum loading conditions, file operations, handling of system error

Page 34


conditions, and memory constraints that may be applicable to the intended use of the applicationprogram.

For more detailed information, see the production and process software references in Appendix A.

Page 35


APPENDIX A - REFERENCES

Food and Drug Administration References

Design Control Guidance for Medical Device Manufacturers, Center for Devices and RadiologicalHealth, Food and Drug Administration, March 1997.

Do It by Design, An Introduction to Human Factors in Medical Devices, Center for Devices andRadiological Health, Food and Drug Administration, March 1997.

Electronic Records; Electronic Signatures Final Rule, 62 Federal Register 13430 (March 20,1997).

Glossary of Computerized System and Software Development Terminology, Division of FieldInvestigations, Office of Regional Operations, Office of Regulatory Affairs, Food and DrugAdministration, August 1995.

Guidance for the Content of Pre-market Submissions for Software Contained in MedicalDevices, Office of Device Evaluation, Center for Devices and Radiological Health, Food and DrugAdministration, May 1998.

Guidance for Industry, FDA Reviewers and Compliance on Off-the-Shelf Software Use inMedical Devices, Office of Device Evaluation, Center for Devices and Radiological Health, Food andDrug Administration, September 1999.

Guideline on General Principles of Process Validation, Center for Drugs and Biologics, & CenterFor Devices and Radiological Health, Food and Drug Administration, May 1987.

Medical Devices; Current Good Manufacturing Practice (CGMP) Final Rule; Quality SystemRegulation , 61 Federal Register 52602 (October 7, 1996).

Reviewer Guidance for a Pre-Market Notification Submission for Blood Establishment ComputerSoftware, Center for Biologics Evaluation and Research, Food and Drug Administration, January 1997

Student Manual 1, Course INV545, Computer System Validation, Division of Human ResourceDevelopment, Office of Regulatory Affairs, Food and Drug Administration, 1997.

Technical Report, Software Development Activities, Division of Field Investigations, Office ofRegional Operations, Office of Regulatory Affairs, Food and Drug Administration, July 1987.

http://www.fda.gov/cdrh/comp/designgd.html

http://www.fda.gov/cdrh/humfac/doit.html







http://www.fda.gov/cber/gdlns/swreview.txt

Page 36


Other Government References

W. Richards Adrion, Martha A. Branstad, John C. Cherniavsky. NBS Special Publication 500-75,Validation, Verification, and Testing of Computer Software, Center for Programming Science andTechnology, Institute for Computer Sciences and Technology, National Bureau of Standards, U.S.Department of Commerce, February 1981.

Martha A. Branstad, John C Cherniavsky, W. Richards Adrion, NBS Special Publication 500-56,Validation, Verification, and Testing for the Individual Programmer, Center for ProgrammingScience and Technology, Institute for Computer Sciences and Technology, National Bureau ofStandards, U.S. Department of Commerce, February 1980.

J.L. Bryant, N.P. Wilburn, Handbook of Software Quality Assurance Techniques Applicable tothe Nuclear Industry, NUREG/CR-4640, U.S. Nuclear Regulatory Commission, 1987.

H. Hecht, et.al., Verification and Validation Guidelines for High Integrity Systems. NUREG/CR-6293. Prepared for U.S. Nuclear Regulatory Commission, 1995.

H. Hecht, et.al., Review Guidelines on Software Languages for Use in Nuclear Power PlantSafety Systems, Final Report. NUREG/CR-6463. Prepared for U.S. Nuclear RegulatoryCommission, 1996.

J.D. Lawrence, W.L. Persons, Survey of Industry Methods for Producing Highly ReliableSoftware, NUREG/CR-6278, U.S. Nuclear Regulatory Commission, 1994.

J.D. Lawrence, G.G. Preckshot, Design Factors for Safety-Critical Software, NUREG/CR-6294,U.S. Nuclear Regulatory Commission, 1994.

Patricia B. Powell, Editor. NBS Special Publication 500-98, Planning for Software Validation,Verification, and Testing, Center for Programming Science and Technology, Institute for ComputerSciences and Technology, National Bureau of Standards, U.S. Department of Commerce, November1982.

Patricia B. Powell, Editor. NBS Special Publication 500-93, Software Validation, Verification,and Testing Technique and Tool Reference Guide, Center for Programming Science andTechnology, Institute for Computer Sciences and Technology, National Bureau of Standards, U.S.Department of Commerce, September 1982.

Delores R. Wallace, Roger U. Fujii, NIST Special Publication 500-165, Software Verification andValidation: Its Role in Computer Assurance and Its Relationship with Software ProjectManagement Standards, National Computer Systems Laboratory, National Institute of Standards andTechnology, U.S. Department of Commerce, September 1995.

Delores R. Wallace, Laura M. Ippolito, D. Richard Kuhn, NIST Special Publication 500-204, HighIntegrity Software, Standards and Guidelines, Computer Systems Laboratory, National Institute of

Page 37


Standards and Technology, U.S. Department of Commerce, September 1992.

Delores R. Wallace, et.al. NIST Special Publication 500-234, Reference Information for theSoftware Verification and Validation Process. Computer Systems Laboratory, National Institute ofStandards and Technology, U.S. Department of Commerce, March 1996.

Delores R. Wallace, Editor. NIST Special Publication 500-235, Structured Testing: A TestingMethodology Using the Cyclomatic Complexity Metric. Computer Systems Laboratory, NationalInstitute of Standards and Technology, U.S. Department of Commerce, August 1996.

International and National Consensus Standards

ANSI / ANS-10.4-1987, Guidelines for the Verification and Validation of Scientific andEngineering Computer Programs for the Nuclear Industry, American National Standards Institute,1987.

ANSI / ASQC Standard D1160-1995, Formal Design Reviews, American Society for QualityControl, 1995.

ANSI / UL 1998:1998, Standard for Safety for Software in Programmable Components,Underwriters Laboratories, Inc., 1998.

AS 3563.1-1991, Software Quality Management System, Part 1: Requirements. Published byStandards Australia [Standards Association of Australia], 1 The Crescent, Homebush, NSW 2140.

AS 3563.2-1991, Software Quality Management System, Part 2: Implementation Guide.Published by Standards Australia [Standards Association of Australia], 1 The Crescent, Homebush,NSW 2140.

IEC 60601-1-4:1996, Medical electrical equipment, Part 1: General requirements for safety, 4.Collateral Standard: Programmable electrical medical systems. International ElectrotechnicalCommission, 1996.

IEC 61506:1997, Industrial process measurement and control – Documentation of applicationsoftware. International Electrotechnical Commission, 1997.

IEC 61508:1998, Functional safety of electrical/electronic/programmable electronic safety-related systems. International Electrotechnical Commission, 1998.

IEEE Std 1012-1986, Software Verification and Validation Plans, Institute for Electrical andElectronics Engineers, 1986.

Page 38


IEEE Standards Collection, Software Engineering, Institute of Electrical and Electronics Engineers,Inc., 1994. ISBN 1-55937-442-X.

ISO 8402:1994, Quality management and quality assurance – Vocabulary. InternationalOrganization for Standardization, 1994.

ISO 9000-3:1997, Quality management and quality assurance standards - Part 3: Guidelines forthe application of ISO 9001:1994 to the development, supply, installation and maintenance ofcomputer software. International Organization for Standardization, 1997.

ISO 9001:1994, Quality systems – Model for quality assurance in design, development,production, installation, and servicing. International Organization for Standardization, 1994.

ISO 13485:1996, Quality systems – Medical devices – Particular requirements for the applicationof ISO 9001. International Organization for Standardization, 1996.

ISO/IEC 12119:1994, Information technology – Software packages – Quality requirements andtesting, Joint Technical Committee ISO/IEC JTC 1, International Organization for Standardization andInternational Electrotechnical Commission, 1994.

ISO/IEC 12207:1995, Information technology – Software life cycle processes, Joint TechnicalCommittee ISO/IEC JTC 1, Subcommittee SC 7, International Organization for Standardization andInternational Electrotechnical Commission, 1995.

ISO/IEC 14598:1999, Information technology – Software product evaluation, Joint TechnicalCommittee ISO/IEC JTC 1, Subcommittee SC 7, International Organization for Standardization andInternational Electrotechnical Commission, 1999.

ISO 14971-1:1998, Medical Devices – Risk Management – Part 1: Application of Risk Analysis.International Organization for Standardization, 1998.

Software Considerations in Airborne Systems and Equipment Certification. Special Committee167 of RTCA. RTCA Inc., Washington, D.C. Tel: 202-833-9339. Document No. RTCA/DO-178B, December 1992.

Production Process Software References

The Application of the Principles of GLP to Computerized Systems, Environmental Monograph#116, Organization for Economic Cooperation and Development (OECD), 1995.

George J. Grigonis, Jr., Edward J. Subak, Jr., and Michael Wyrick, “Validation Key Practices forComputer Systems Used in Regulated Operations,” Pharmaceutical Technology, June 1997.

Guide to Inspection of Computerized Systems in Drug Processing, Reference Materials and

Page 39


Training Aids for Investigators, Division of Drug Quality Compliance, Associate Director forCompliance, Office of Drugs, National Center for Drugs and Biologics, & Division of FieldInvestigations, Associate Director for Field Support, Executive Director of Regional Operations, Foodand Drug Administration, February 1983.

Daniel P. Olivier, “Validating Process Software”, FDA Investigator Course: Medical DeviceProcess Validation, Food and Drug Administration.

GAMP Guide For Validation of Automated Systems in Pharmaceutical Manufacture,VersionV3.0, Good Automated Manufacturing Practice (GAMP) Forum, March 1998:

Volume 1, Part 1: User Guide Part 2: Supplier Guide

Volume 2: Best Practice for User and Suppliers.

Technical Report No. 18, Validation of Computer-Related Systems. PDA Committee onValidation of Computer-Related Systems. PDA Journal of Pharmaceutical Science and Technology,Volume 49, Number 1, January-February 1995 Supplement.

Validation Compliance Annual 1995, International Validation Forum, Inc.

General Software Quality References

Boris Beizer, Black Box Testing, Techniques for Functional Testing of Software and Systems, JohnWiley & Sons, 1995. ISBN 0-471-12094-4.

Boris Beizer, Software System Testing and Quality Assurance, International Thomson ComputerPress, 1996. ISBN 1-85032-821-8.

Boris Beizer, Software Testing Techniques, Second Edition, Van Nostrand Reinhold, 1990. ISBN 0-442-20672-0.

Richard Bender, Writing Testable Requirements, Version 1.0, Bender & Associates, Inc., Larkspur,CA 94777, 1996.

Frederick P. Brooks, Jr., The Mythical Man-Month, Essays on Software Engineering, Addison-Wesley Longman, Anniversary Edition, 1995. ISBN 0-201-83595-9.

Silvana Castano, et.al., Database Security, ACM Press, Addison-Wesley Publishing Company, 1995.ISBN 0-201-59375-0.

Computerized Data Systems for Nonclinical Safety Assessment, Current Concepts and QualityAssurance, Drug Information Association, Maple Glen, PA, September 1988.

Page 40


M. S. Deutsch, Software Verification and Validation, Realistic Project Approaches, Prentice Hall,1982.

Robert H. Dunn and Richard S. Ullman, TQM for Computer Software, Second Edition, McGraw-Hill, Inc., 1994. ISBN 0-07-018314-7.

Elfriede Dustin, Jeff Rashka, and John Paul, Automated Software Testing – Introduction,Management and Performance, Addison Wesley Longman, Inc., 1999. ISBN 0-201-43287-0.

Robert G. Ebenau and Susan H. Strauss, Software Inspection Process, McGraw-Hill, 1994.ISBN 0-07-062166-7.

Richard E. Fairley, Software Engineering Concepts, McGraw-Hill Publishing Company, 1985.ISBN 0-07-019902-7.

Michael A. Friedman and Jeffrey M. Voas, Software Assessment - Reliability, Safety, Testability,Wiley-Interscience, John Wiley & Sons Inc., 1995. ISBN 0-471-01009-X.

Tom Gilb, Dorothy Graham, Software Inspection, Addison-Wesley Publishing Company, 1993.ISBN 0-201-63181-4.

Robert B. Grady, Practical Software Metrics for Project Management and ProcessImprovement, PTR Prentice-Hall Inc., 1992. ISBN 0-13-720384-5.

Les Hatton, Safer C: Developing Software for High-integrity and Safety-critical Systems,McGraw-Hill Book Company, 1994. ISBN 0-07-707640-0.

Janis V. Halvorsen, A Software Requirements Specification Document Model for the MedicalDevice Industry, Proceedings IEEE SOUTHEASTCON '93, Banking on Technology, April 4th -7th,1993, Charlotte, North Carolina.

Debra S. Herrmann, Software Safety and Reliability: Techniques, Approaches and Standards ofKey Industrial Sectors, IEEE Computer Society, 1999. ISBN 0-7695-0299-7.

Bill Hetzel, The Complete Guide to Software Testing, Second Edition, A Wiley-QED Publication,John Wiley & Sons, Inc., 1988. ISBN 0-471-56567-9.

Watts S. Humphrey, A Discipline for Software Engineering. Addison-Wesley Longman, 1995.ISBN 0-201-54610-8.

Watts S. Humphrey, Managing the Software Process, Addison-Wesley Publishing Company, 1989.ISBN 0-201-18095-2.

Capers Jones, Software Quality, Analysis and Guidelines for Success, International ThomsonComputer Press, 1997. ISBN 1-85032-867-6.

Page 41


J.M. Juran, Frank M. Gryna, Quality Planning and Analysis, Third Edition, , McGraw-Hill, 1993.ISBN 0-07-033183-9.

Stephen H. Kan, Metrics and Models in Software Quality Engineering, Addison-Wesley PublishingCompany, 1995. ISBN 0-201-63339-6.

Cem Kaner, Jack Falk, Hung Quoc Nguyen, Testing Computer Software, Second Edition, VsnNostrand Reinhold, 1993. ISBN 0-442-01361-2.

Craig Kaplan, Ralph Clark, Victor Tang, Secrets of Software Quality, 40 Innovations from IBM,McGraw-Hill, 1995. ISBN 0-07-911795-3.

Edward Kit, Software Testing in the Real World, Addison-Wesley Longman, 1995. ISBN 0-201-87756-2.

Alan Kusinitz, “Software Validation”, Current Issues in Medical Device Quality Systems,Association for the Advancement of Medical Instrumentation, 1997. ISBN 1-57020-075-0.

Nancy G. Leveson, Safeware, System Safety and Computers, Addison-Wesley PublishingCompany, 1995. ISBN 0-201-11972-2.

Michael R. Lyu, Editor, Handbook of Software Reliability Engineering, IEEE Computer SocietyPress, McGraw-Hill, 1996. ISBN 0-07-039400-8.

Steven R. Mallory, Software Development and Quality Assurance for the HealthcareManufacturing Industries, Interpharm Press,Inc., 1994. ISBN 0-935184-58-9.

Brian Marick, The Craft of Software Testing, Prentice Hall PTR, 1995. ISBN 0-13-177411-5.

Steve McConnell, Rapid Development, Microsoft Press, 1996. ISBN 1-55615-900-5.

Glenford J. Myers, The Art of Software Testing, John Wiley & Sons, 1979.ISBN 0-471-04328-1.

Peter G. Neumann, Computer Related Risks, ACM Press/Addison-Wesley Publishing Co., 1995.ISBN 0-201-55805-X.

Daniel Olivier, Conducting Software Audits, Auditing Software for Conformance to FDARequirements, Computer Application Specialists, San Diego, CA, 1994.

William Perry, Effective Methods for Software Testing, John Wiley & Sons, Inc. 1995. ISBN 0-471-06097-6.

William E. Perry, Randall W. Rice, Surviving the Top Ten Challenges of Software Testing, Dorset

Page 42


House Publishing, 1997. ISBN 0-932633-38-2.

Roger S. Pressman, Software Engineering, A Practitioner's Approach, Third Edition, McGraw-HillInc., 1992. ISBN 0-07-050814-3.

Roger S. Pressman, A Manager’s Guide to Software Engineering, McGraw-Hill Inc., 1993ISBN 0-07-050820-8.

A. P. Sage, J. D. Palmer, Software Systems Engineering, John Wiley & Sons, 1990.

Joc Sanders, Eugene Curran, Software Quality, Addison-Wesley Publishing Co., 1994. ISBN 0-201-63198-9.

Ken Shumate, Marilyn Keller, Software Specification and Design, A Disciplined Approach forReal-Time Systems, John Wiley & Sons, 1992. ISBN 0-471-53296-7.

Dennis D. Smith, Designing Maintainable Software, Springer-Verlag, 1999.ISBN 0-387-98783-5.

Ian Sommerville, Software Engineering, Third Edition, Addison Wesley Publishing Co., 1989. ISBN0-201-17568-1.

Karl E. Wiegers, Creating a Software Engineering Culture, Dorset House Publishing, 1996. ISBN0-932633-33-1.

Karl E. Wiegers, Software Inspection, Improving Quality with Software Inspections, SoftwareDevelopment, April 1995, pages 55-64.

Karl E. Wiegers, Software Requirements, Microsoft Press, 1999. ISBN 0-7356-0631-5.

Page 43


APPENDIX B - DEVELOPMENT TEAM

Center for Devices and Radiological Health

Office of Compliance Stewart Crumpler

Office of Device Evaluation James Cheng, Donna-Bea Tillman

Office of Health and Industry Programs Bryan Benesch, Dick Sawyer

Office of Science and Technology John Murray

Office of Surveillance and Biometrics Howard Press

Center for Drug Evaluation and Research

Office of Medical Policy Charles Snipes

Center for Biologics Evaluation and Research

Office of Compliance and Biologics Quality Alice Godziemski

Office of Regulatory Affairs

Office of Regional Operations David Bergeson, Joan Loreng

1482 Federal Register / Vol. 67, No. 8 / Friday, January 11, 2002 / Notices

proposal also involves the acquisition ofa nonbanking company, the review alsoincludes whether the acquisition of thenonbanking company complies with thestandards in section 4 of the BHC Act(12 U.S.C. 1843). Unless otherwisenoted, nonbanking activities will beconducted throughout the United States.Additional information on all bankholding companies may be obtainedfrom the National Information Centerwebsite at www.ffiec.gov/nic/.

Unless otherwise noted, commentsregarding each of these applicationsmust be received at the Reserve Bankindicated or the offices of the Board ofGovernors not later than February 4,2002.

A. Federal Reserve Bank of KansasCity (Susan Zubradt, Assistant VicePresident) 925 Grand Avenue, KansasCity, Missouri 64198–0001:

1. Lauritzen Corporation, Omaha,Nebraska; to acquire 1.54 percent, for atotal of 23.03 percent, of the votingshares of First National of Nebraska,Inc., Omaha, Nebraska, and therebyindirectly acquire additional interest inFirst National Bank of Omaha, Omaha,Nebraska; First National Bank, NorthPlatte, Nebraska; Platte Valley StateBank & Trust Co., Kearney, Nebraska;Fremont National Bank & Trust Co.,Fremont, Nebraska; First National Bank& Trust Company, Columbus, Nebraska;First National Bank, Overland Park,Kansas; First National Bank SouthDakota, Yankton, South Dakota; FirstNational of Colorado, Inc., Fort Collins,Colorado, First National Bank, FortCollins, Colorado; Union Colony Bank,Greeley, Colorado; First National Bankof Colorado, Boulder, Colorado; FirstNational of Illinois, Inc., Omaha,Nebraska, and Castle Bank, N.A.,DeKalb, Illinois.

Board of Governors of the Federal ReserveSystem, January 7, 2002.Robert deV. Frierson,Deputy Secretary of the Board.[FR Doc. 02–686 Filed 1–10–02; 8:45 am]BILLING CODE 6210–02–S

DEPARTMENT OF HEALTH ANDHUMAN SERVICES

Food and Drug Administration

[Docket No. 97D–0282]

Medical Devices: General Principles ofSoftware Validation; Final Guidance forIndustry and FDA Staff; Availability

AGENCY: Food and Drug Administration,HHS.ACTION: Notice.

SUMMARY: The Food and DrugAdministration (FDA) is announcing theavailability of the guidance entitled‘‘General Principles of SoftwareValidation.’’ This document providesguidance to medical devicemanufacturers and FDA staff concerningrequirements for validating softwareused within medical devices, in deviceproduction, or in implementing themanufacturer’s quality system.DATES: Submit written or electroniccomments at any time.ADDRESSES: Submit written requests forsingle copies on a 3.5′′ diskette of theguidance document entitled ‘‘GeneralPrinciples of Software Validation’’ tothe Division of Small Manufacturers,International and Consumer Assistance(HFZ–220), Center for Devices andRadiological Health (CDRH), Food andDrug Administration, 1350 Piccard Dr.,Rockville, MD 20850. Send two self-addressed adhesive labels to assist thatoffice in processing your request, or faxyour request to 301–443–8818. See theSUPPLEMENTARY INFORMATION section forinformation on electronic access to theguidance.

Submit written comments concerningthis guidance to the DocketsManagement Branch (HFA–305), Foodand Drug Administration, 5630 FishersLane, rm. 1061, Rockville, MD 20852.Submit electronic comments to http://www.fda.gov/dockets/ecomments.Comments are to be identified with thedocket number found in brackets in theheading of this document.FOR FURTHER INFORMATION CONTACT: JohnF. Murray, Center for Devices andRadiological Health (HFZ–340), Foodand Drug Administration, 9200Corporate Blvd., Rockville, MD 20850,301–594–4659.SUPPLEMENTARY INFORMATION:

I. BackgroundThis final guidance document entitled

‘‘General Principles of SoftwareValidation’’ provides guidance tomedical device manufacturers and FDAstaff concerning requirements forvalidating software used within medicaldevices, in device production, or inimplementing the manufacturer’squality system. It replaces the draftguidance that FDA issued for commenton June 9, 1997, and published in theFederal Register of July 25, 1997 (62 FR40099).

We received responses from 36organizations and individuals, withmore than 650 questions, comments,and specific recommendations forchanges to the guidance. However,further work on the guidance wasinterrupted by other high priority

activities, including implementation ofthe Food and Drug AdministrationModernization Act of 1997, FDA’sresponse to year 2000 softwareconcerns, and two rounds ofimplementation of our first medicaldevice performance standard. Becauseof the delay in issuing this finalguidance, we have chosen to summarizeour response to the comments received.As with any guidance, we will continueto accept comments and may updatethis document in the future.

The following summarizes thecomments we received, and significantchanges we made to the guidance inresponse to those comments:

A. Intended ScopeFrom a few of the comments received,

it appears that some parties may nothave realized the full breadth of thequality system regulation. The softwarevalidation requirement in 21 CFR820.70(i) of the quality systemregulation also applies to automatedtools used to design medical devicesand tools used to develop software.Since the first medical device goodmanufacturing practice regulation waspublished in 1978, there has alwaysbeen an explicit validation requirementfor software used in device productionor used to implement the qualitysystem. When design controls wereintroduced into the quality systemregulation in 1997, that softwarevalidation requirement was extended tosoftware used to design devices, such ascomputer-aided design and softwaredevelopment tools. FDA clearlyaddressed this issue at the end of itsresponse to comment 136 in thepreamble to the quality systemregulation (61 FR 52602 at 52630,October 7, 1996). A copy of the text isincluded at the end of this section.

Some comments objected to thediscussion of validation activitiesduring the predesign ‘‘concept’’ phase ofsoftware development, both because thequality system regulation does not applyto research activities, and because thereis too little information available at thatpoint to make any validation relatedactivity worthwhile. In response tothese concerns, we have removed allreference to validation activities duringthe ‘‘concept’’ phase.

Other comments noted that theguidance covered more than justvalidation issues, and suggestedchanging the title to broaden the scopeof the guidance. We acknowledge thatthe scope of the guidance is somewhatbroader than the scope of validation inthe strictest definition of that term.However, we have chosen not to changethe title of the guidance. Planning,

VerDate 11<MAY>2000 18:14 Jan 10, 2002 Jkt 197001 PO 00000 Frm 00050 Fmt 4703 Sfmt 4703 E:\FR\FM\11JAN1.SGM pfrm07 PsN: 11JAN1

default

default

default

default

default

default

default

Medical Devices: General Principles of

default

Software Validation; Final Guidance for

default

Industry and FDA Staff; Availability

1483Federal Register / Vol. 67, No. 8 / Friday, January 11, 2002 / Notices

verification, testing, traceability,configuration management, and manyother activities discussed in theguidance are important activities thattogether help to support a finalconclusion that software is validated.

Some comments expressed concernsthat the guidance might be applied toorigorously by FDA investigators, andsome pharmaceutical manufacturersraised questions about how theguidance would be applied to their drugmanufacturing operations. The agency’sgood guidance practices (GGPs) clearlystate the role of FDA guidance.Alternative approaches that accomplishfull compliance with the quality systemregulation are acceptable. While it isclearly intended for medical devicemanufacturers, the guidance may alsobe useful to the pharmaceutical industryand other industries regulated by FDA.

Many comments suggested that wemove all discussions regarding use ofoff-the-shelf (OTS) software to theagency’s guidance entitled ‘‘Off-the-Shelf Software Use in Medical Devices.’’In response to these comments, specificcross references to that document havebeen added within the text of thisguidance. However, the OTS guidancedocument deals specifically withpremarket submissions for OTS softwarecontained in medical devices. It is notthe appropriate guidance for OTSsoftware used in manufacturing andquality systems applications.

B. FlexibilityNumerous comments cited overly

restrictive language and lack ofsufficient implementation flexibility inthe draft guidance. For example, manycomments noted that the guidanceimplies use of a ‘‘waterfall’’ as thepreferred life cycle developmentmethodology. Several commentssuggested that more discussion wasneeded regarding ‘‘rapid applicationdevelopment’’ and ‘‘component-basedmethodologies,’’ as well as ‘‘build alittle/test a little’’ as an acceptablemethodology. Other comments asked forspecific examples of available life cyclemodels that could be used. In responseto these comments, and in accordancewith our own GGPs, we have carefullyrewritten the text to remove any director implied use of the words ‘‘shall’’ or‘‘must,’’ except where we describe orreference a regulation. We also haveadded language to specifically state thatincremental developmentmethodologies may be used, and thatactivities and tasks can be performed ina different order, if called for by thechosen life cycle model. However, forease of description, we have retained anorganization of activities based on

‘‘requirements,’’ ‘‘design,’’ ‘‘coding (orconstruction),’’ and ‘‘testing.’’Regardless of the order in which tasksare accomplished, these four categoriesof activities are common to most lifecycle models. We have not includedexamples of the dozens of life cyclemodels that are available. To do socould imply agency endorsement ofcertain life cycle models that areincluded over those models that are notincluded. Instead, you are referred tomany of the textbooks and otherreferences listed at the end of theguidance, which provide details ofmany of these life cycle models.

One group of comments objected toany use of the word ‘‘all’’ whendescribing items to be included inspecification documents, noting that‘‘all’’ is not a quantifiable term. Othercomments suggested use of the word‘‘may’’ rather than ‘‘should.’’ On theother hand, a few comments asked fora specific compliance matrix, so thatmanufacturers would know exactly howto comply with FDA expectations. Wehave not adopted these suggestedchanges. We believe that agencyguidance should identify and encourageuse of approaches known to have beenused effectively, while the manufacturerretains the prerogative to choosealternative approaches that are equallyeffective. Based on variables such asfirm size and structure, device risk,project size, and complexity,manufacturers have the flexibility tochoose different approaches for differentprojects, and to select effectiveapproaches that best fit their specificneeds.

C. Format

Several comments suggested use ofthe framework and format ininternational guidelines such as ISO9000–3, GAMP, IEEE SoftwareStandards and ISO/IEC 12207. We havedrawn information from each of thesesources and many other listedreferences, but unfortunately, there is nosingle format available. We haverewritten the guidance to addressspecific suggestions for wordingchanges and simpler language. Somecomments asked for extensive use ofcharts, analogies, and examples for theconcepts presented in the document.While valuable, such an approach couldeasily triple the size of the guidance.Instead, we suggest referring to any ofthe extensive list of references includedat the end of the guidance for moredetails on specific implementationapproaches.

D. Differences Between Hardware andSoftware

Regarding the discussion ofdifferences between hardware andsoftware, the comments were somewhatdivided. Some comments applauded theagency for recognizing the legitimatedifferences between hardwareengineering and software engineering.Other comments argued that ‘‘softwareis not different’’ and suggested deletionof all or most of this section, eitherbecause it was unnecessary, or becauseit could be misinterpreted by softwaredevelopers who lack sufficientengineering discipline. One commentsuggested emphasizing the similaritiesof the engineering discipline needed tobuild both hardware and software. Wehave chosen to keep this sectionbecause we believe it explains part ofthe rationale for why software must bethoroughly validated, and why thesoftware development process needs tobe carefully controlled and managed.We have also added additionalinformation regarding the impact ofmobility of software professionals onthe long-term maintenance of softwareand the need for thoroughdocumentation.

Some comments objected to thediscussion of standardization and reuseof software components and asked formore recognition of the trend towardincreased use of OTS and component-based development methods. Othercomments objected to the statement that‘‘repairs made to correct softwaredefects establish a new design.’’ Wehave revised the text to address both ofthese concerns.

E. Principles of Software Validation

We reorganized and rewrote thesection regarding ‘‘Principles ofSoftware Validation’’ to address thecomments received. For example, wemoved the subsection dealing withdocumenting software ‘‘Requirements’’to the front of the section to reflect theimportance of requirements in thevalidation process. We clarifiedlanguage regarding ‘‘predetermined’’requirements to allow for incremental orevolutionary development ofrequirements during the developmentproject. However, we have retained theconcept that documented requirementsshould be established prior to formaltesting or other verification activities toprovide ‘‘objective’’ evidence that thoserequirements were met.

The subsection previously entitled‘‘Testing’’ is retitled ‘‘DefectPrevention’’ and is revised to emphasizethe importance of preventing software



defects, as opposed to trying to ‘‘testquality into’’ software.

We have renamed the subsection on‘‘Timing.’’ In response to severalcomments concerning validationcontinuing ‘‘for the entire life cycle,’’ wehave rewritten the text, but haveretained the concept. At each stage ofthe software life cycle, there isinformation available that cancontribute to a conclusion that thesoftware meets user needs and intendeduses. Therefore, the validation processdoes not end when the device isshipped.

We replaced the subsection on‘‘Management’’ with a new subsectiondealing with the ‘‘Software Life Cycle.’’

We have clarified the subsectionsdealing with ‘‘Plans’’ and ‘‘Procedures’’to distinguish between plans that definewhat to do, and procedures thatdescribe how to do it.

The subsection entitled ‘‘PartialValidation’’ is substantially rewrittenand retitled ‘‘Software Validation Aftera Change.’’ Many readers misinterpretedthe statement that ‘‘software cannot bepartially validated’’ and thought weintended all validation testing to berepeated every time any change is made.That is not what we meant. Based on thecomments received, we have rewrittenthe discussion to emphasize the needfor regression analysis after a change,followed by an appropriate level ofregression testing to reestablish thevalidation status of the software. Wehave deleted specific discussion ofretrospective validation and reverseengineering of nonvalidated software,but these issues should be coveredduring the regression analysis.

We have retitled and rewritten thesubsection on ‘‘Amount of Effort.’’ Nowtitled ‘‘Validation Coverage,’’ it stilldescribes an approach that ties the levelof validation and verification effort tothe safety risk and complexity of thesoftware.

We revised the subsection on‘‘Independence of Review’’ to providegreater flexibility and a betterexplanation of its intent.

The subsection previously entitled‘‘Real World’’ is now entitled‘‘Flexibility and Responsibility,’’ andreemphasizes that devicemanufacturers/software developers havea lot of flexibility in how theyimplement their software validationprocess, but the device manufacturer isultimately responsible for the adequacyand effectiveness of the selectedapproach.

F. TerminologySome of the most significant

comments we received had to do with

our basic definition of softwarevalidation. In the previous draftguidance, we relied upon technicaldefinitions used by the NationalInstitute of Standards and Technologyand by the Institute of Electrical andElectronic Engineers. These technicaldefinitions created some confusion withother definitions in our quality systemregulation. Numerous commentsobjected to our use of ‘‘validation’’ as anumbrella term to cover ‘‘design review’’and ‘‘verification’’ as well as validation.They stated that both design review andverification are distinctly separablequality concepts and are not a part ofvalidation. In response to theseconcerns, we have changed thedefinition of software validation to bemore consistent with the quality systemregulation and other internationalquality standards. Our reviseddefinition of software validation isderived directly from the definitions of‘‘validation’’ and ‘‘design validation’’ inthe quality system regulation.

Comments also objected to the title‘‘Typical Validation Tasks’’ at the end ofeach subsection in the section V of theguidance and suggested that they arereally verification tasks. Othercomments objected to possibleinterpretation of these as mandatorytasks. In response to these comments,we have also added text to explain thatthere are typical verification and testingtasks that support an overall conclusionthat software is validated. Thereafter,when we discuss ‘‘Typical TasksSupporting Validation,’’ we do not try todifferentiate between verification tasksversus validation tasks. Instead, we haverevised the text to list ‘‘Typical Tasks.’’While we want to avoid any inferencethat the tasks are mandatory in everycase, the guidance makes the point thatthese are ‘‘typical’’ approaches that arerecommended by software engineeringstandards and textbooks, and widelyused by many software engineeringprofessionals.

Several comments notedinconsistencies in terminology from thatcontained in the quality systemregulation, in two software guidancesissued by the Office of DeviceEvaluation, and in the FDA glossary ofcomputerized system and softwaredevelopment terminology. Thesecomments also suggested use of the term‘‘risk analysis’’ instead of ‘‘hazardanalysis’’ throughout the softwarevalidation guidance. We have revisedthe guidance to incorporate the term‘‘risk analysis’’ throughout. However,we continue to emphasize that whilethere are many different risks (e.g.,economic or time to market), FDA isconcerned about safety risk (hazard). At

their next revision, we expect to updateother software guidance documents andthe FDA glossary with consistentdefinitions of validation, verification,and risk analysis. In addition, we nowuse the term ‘‘user site testing’’ ratherthan ‘‘installation testing’’ to describetesting performed at the user site andoutside the control of the softwaremanufacturer.

Some comments questioned whetherOTS software could be validatedbecause the device manufacturerfrequently does not have access to thesource code. These comments suggestedthat OTS software should be ‘‘qualified’’rather than ‘‘validated.’’ However, webelieve that the evidence developed bya device manufacturer concerning OTSsoftware is a true validation because itdirectly supports a conclusion that thesoftware meets user needs and intendeduses. Where the source code is notavailable, it is incumbent upon thedevice manufacturer to use other means(such as audits, or more extensive blackbox testing) to infer the structuralintegrity of the OTS software. This issueis clearly addressed in comment 136 ofthe preamble to the quality systemregulation (61 FR 52602 at 52630).

Other comments from thepharmaceutical industry suggestedincorporation of widely understoodprocess validation terminology (i.e.,installation qualification (IQ),operational qualification (OQ), andperformance qualification (PQ)) todescribe software validation. Anothercomment suggested use of ‘‘productperformance qualification’’ rather than‘‘design validation.’’ We have added asection that refers to the various typesof qualification, but we have chosen notto adopt ‘‘qualification’’ terminology inexplaining software validationrequirements. Of course, manufacturersmay continue to organize theirvalidation efforts using IQ/OQ/PQterminology, if they wish.

In response to comments, a newsubsection has been added to explainthe differences between ‘‘requirements,’’which may be general in nature, versus‘‘specifications,’’ which are developedto an engineering level of detail.

Several comments objected to use ofundefined terms such as ‘‘microcode’’and ‘‘assertions.’’ We reiterate that theseand many other terms used throughoutthe guidance are specifically defined inthe FDA glossary of computerizedsystem and software developmentterminology, which is available at http://www.fda.gov/ora/inspect_ref/igs/gloss.html.



G. Design Review

As noted above, design reviews arenot a part of validation. In fact, severalcomments noted that results ofverification and validation are inputs todesign reviews—not the other wayaround. To emphasize this point, wemoved the subsection on ‘‘DesignReviews’’ outside the section on‘‘Typical Tasks Supporting Validation.’’We also added information about thedifference between formal designreviews that are mandated by thequality system regulation versus lessformal technical reviews.

H. Traceability

A few comments objected to theguidance regarding ‘‘traceabilityanalysis,’’ especially the discussion atthe end of the subsection on ‘‘Coding.’’Two comments noted that for verycomplex programs with thousands oflines of code or thousands of modules,the traceability analysis would beextremely complex and of little value.One suggested that design review wasan adequate substitute for traceabilityanalysis. We disagree. Traceability is anessential aspect of verification, and it isan important input into design reviews.We therefore do not believe that designreview could be an adequate substitutefor traceability analysis.

One comment stated thatrequirements are not always neatlystructured, and it is very difficult totrace exactly how they are implementedin the design. There are numerousmany-to-one and one-to-manyrelationships to be mapped fromrequirements to design to code. Weagree with this observation; however, itactually further supports the need fortraceability. The larger and morecomplex the project, the more importantthe traceability analysis becomes.Therefore, we have retained thediscussions regarding traceability, andin response to several other comments,we have added traceability of softwarerequirements to the safety risk analysis.

Another comment noted that inherenttraceability can be built intodocumentation and code without havingto have a separate traceabilitydocument. We agree and for that reasonhave avoided use of the most commonlyused term—‘‘traceability matrix.’’ Threecommon approaches are traceabilitymatrix, using computer databases toevaluate traceability, or buildinginherent traceability into the structureof the documentation and code. Theremay be many other approaches totraceability. Software developers haveflexibility in how they want toimplement traceability.

I. Risk AnalysisMany comments questioned the

concept of a software failure modes andeffects analysis (FMEA). They statedthat given the difficulty of predictingspecific software failure modes, FMEAis better used as a system level riskanalysis tool. We have revised theguidance to discuss software riskanalysis within the context of systemsafety. However, while we acknowledgesome limitations in its use, we alsobelieve that software FMEA can be auseful tool, especially for safety criticalaspects of software applications. It mayalso be useful early in the developmentprocess for analyzing safety criticalsoftware requirements.

One comment objected to thesuggestion that risk analysis begin at thestage where requirements are defined.However, to be useful and have animpact on the software developmentprocess, we believe that risk analysisneeds to begin early and needs to beupdated as the project progresses. Inaddition, we have revised variousportions of the guidance to emphasizethat the level of safety risk is a majorfactor in determining the level of effortto be applied in testing and otherverification and validation tasks.

J. PlanningIn response to comments, we have

changed the subsection on‘‘Management’’ to be entitled ‘‘QualityPlanning.’’ It now provides a moregeneral discussion of the softwarevalidation and verification concerns toconsider during quality planning.

Several comments questioned the ideaof early test planning, which wasrecommended in the draft guidance. Forexample, they argued that there isinsufficient information availableduring requirements development to beable to develop a system test plan or anacceptance test plan. We disagree andhave retained the recommendations forearly test planning, but we havespecified that test plans and test casesshould be created as early in thesoftware development process ‘‘asfeasible.’’ One of the important criteria,both for requirements and for design, isthat they be testable. The fact that thereis insufficient information for aparticular test plan is valuable feedbackto the development process that perhapsthe requirements or design processes arenot yet sufficiently complete. Planningis a dynamic activity that should bereexamined and updated as the projectprogresses.

K. RequirementsMany comments objected to use of the

word ‘‘all’’ in describing what is

typically specified in softwarerequirements. We agree thatrequirements frequently do not specify‘‘all’’ that they should. However, that iswidely recognized as one the majorflaws in software development, and itscorrection is one of the most importantmessages intended by this guidance. Inorder to be complete, a softwarerequirements specification should coverall the pertinent issues—not just aselected few.

One comment noted thatrequirements may not always bemeasurable. We have changed the textto state that requirements should be‘‘measurable or objectively verifiable.’’

A few comments noted that ‘‘internalinterfaces’’ and ‘‘all ranges of values thesoftware will accept’’ are a part ofdesign—not requirements. We agreeregarding internal interfaces and havechanged the text accordingly. However,since software requirements are derivedfrom system requirements, there may besome internal system interfacesprescribed from the high level systemdesign that would impact softwarerequirements. Regarding ‘‘ranges ofvalues,’’ we note that there is rarely abright line of demarcation betweenrequirements and design. Softwaredevelopers have flexibility as to wherein their life cycle they wish to coverparticular issues. We rejected mostcomments requesting even greater levelsof detail and specificity regarding staticverification techniques. For example,several comments asked for more detailregarding ‘‘requirements evaluation’’and ‘‘interface analysis.’’ Details onthese techniques are available in manyof the references listed at the end of theguidance. FDA investigators will expectto see a verification procedure thatincludes a means for identifying andresolving incomplete, ambiguous, andconflicting requirements, as required bythe regulation. They will also expect tosee objective documented evidence thatthe verification procedure wasimplemented.

L. Design

We have retained wording about theneed for design specifications to becomplete enough for programmers notto have to make ad hoc decisions. Theintent is to ensure that the code createdis consistent with the designspecification. When programmers orengineers decide to add newfunctionality not identified previouslyin the requirements or design, thosespecifications need to be updated toreflect the actual code created. Theproject manager, design team, and anyfuture maintainers of the software need



to have accurate documentation in orderto do their work.

We have dropped the listing ofspecific approaches to software design,and we have included a more generaldescription of what should be includedin a software design specification. Somecomments considered the previous listto be too prescriptive as well asincomplete.

We recognize that portions of thesoftware are completed and releasedincrementally, and life cycle processesare repeated iteratively. The intent isthat those portions of the software havedesign documentation that is consistentwith the software application that isimplemented. One comment noted thatin a rapid application development(RAD) environment, there is typicallyno formal design document in placeduring coding. We recognize that RADis valuable as a prototyping tool, but itsuse does not preclude the need todocument the specific design, once it isagreed upon.

M. CodingWe have changed the title of this

subsection to reflect that the creation ofa software application can be eitherthrough coding, or through combiningexisting software components, such asOTS software products or functionalcomponents from existing codelibraries.

Comments objected to the idea ofhaving to keep results of allcompilations of the code. In response,we have revised the discussion ofcompiler error checking to state that theresults of the ‘‘final’’ compilation of thecode should be retained to documentany errors that remain uncorrected inthe final software product.

N. Testing by the Software DeveloperWe renamed and revised this

subsection to provide a betterexplanation of the purpose of testing,and to avoid prescriptive languageconcerning use of specific testingtechniques. We have added languageregarding use of incrementaldevelopment and testing methodologies.We expanded the discussion of testingcoverage to explain how differentdegrees of coverage should beconsidered for varying levels of risk,and that the manufacturer has flexibilityto choose the right level of coverage.

One comment noted that the intent oftesting is to find errors, and suggesteda better explanation of this and othertenets of a software testing strategy. Wehave added such an explanation.

Other comments argued thatstatistical testing based on usage profilesis more effective than extensive

structural testing in finding softwaredefects. We agree that statistical testingis one of many valuable testingmethodologies, and we have addedinformation about its use. However, it isimportant to note that statistical testingis an adjunctive approach, rather thanan outright replacement for other typesof testing.

O. User Site TestingBased on several comments, we have

renamed the subsection formerlyentitled ‘‘Installation Testing’’ andmoved it into the section on life cycleactivities. User site testing can be anyone of several types of testing performedby the user or by others at the user site.System level testing performed by thesoftware developer under conditionsthat simulate the user’s environment isan important part of validation for someproducts, and it may substitute for someaspects of user site testing. However, forcertain products such as bloodestablishment software, there arespecific FDA requirements foradditional testing to be performed at theuser site. For manufacturing and qualitysystem software, user site testing isfrequently performed by the devicemanufacturer.

P. Maintenance and Software ChangesSeveral comments objected to the

statement that ‘‘all modifications aredesign changes,’’ noting that somechanges, such as a correction of codingerrors, do not change the intendeddesign. We have made appropriatechanges to the text. However, wecontinue to emphasize that thevalidation of all software changes needsto include a regression analysis and, asappropriate, regression testing to showthat the change has not negativelyimpacted the software.

In response to other comments, wehave added information regardinganomaly evaluation, problemidentification and resolution tracking,and the need to update documentation.

Q. Process and Quality System SoftwareWe have added a new section to the

document dealing with validation ofautomated process equipment andquality system software. This changewas in response to the many commentsthat raised issues and asked for moredetailed information about validatingsuch software, especially OTSautomated equipment and OTSsoftware.

Many comments discussed thedifficulties encountered in trying tovalidate OTS software, and suggested adifferent approach for validation ofmanufacturing and quality system

software. Source code and life cycledocumentation are frequentlyunavailable for review, so structuraltesting is usually not possible. Auditingthe vendor’s software developmentactivities is one possibility, but somesoftware vendors will not agree to beingaudited. One comment suggested thatrisk analysis, design, coding, and unittesting should not apply to qualitysystem software, especially if it ispurchased, and further suggested thatfunctional testing is the most that can beexpected. Several comments suggestedthat for widely used applications, therecan be a reasonable assumption that thevendor validated the software at thetime it was developed, and thatinstallation qualification by the usershould be sufficient. Many of theseissues are addressed in the response tocomment 136 in the preamble of thequality system regulation (61 FR 52602at 52630).

It is not the agency’s intent todiscourage use of OTS computerproducts. The activities described in theguidance can be shared between thevendor and device manufacturer (theuser). However, we believe that theprinciples and activities described inthe guidance are important for anoverall conclusion that software isvalidated for its intended use. Devicemanufacturers are required to havepurchasing controls for the productsand services they receive. Such controlsare an important part of decisionmaking regarding OTS software. Ourexperience is that ‘‘assumptions’’regarding validation by the vendor arenot always well founded. Each OTSsoftware product needs to beindividually evaluated based on theintended use of the software, availablelife cycle documentation, availableverification and validation evidence,and most importantly the device safetyrisk posed by the automated process.Device manufacturers can use multiplesources of information, but areultimately responsible for documentingthe basis for their conclusion that thesoftware is validated for its intendeduse.

Several comments suggestedalternative approaches for certain typesof software, such as operating systemsand certain tools used in softwaredevelopment, such as compilers androbust ‘‘middleware’’ such as Oracle,Documentum, or Lotus Notes. We haveadded suggestions for alternativeapproaches, while still retaining thebasic requirement that the softwaremust be validated for its intended use.

A few comments questioned who isresponsible for validation of OTSsoftware. One questioned FDA’s



authority to regulate software vendors,but argued that device manufacturerscannot be responsible because they lackaccess to source code and life cycledocumentation. Another noted thatvendors frequently change theirhardware and software, resulting inunreasonable FDA expectations forrevalidation of each change. Onecomment asked for more detailsregarding the impact of the supplier’squality system on purchasing decisions.In response to these comments, wereaffirm that FDA holds the devicemanufacturer responsible for thesoftware validation requirement. Thisresponsibility can be further delegatedin part through contracting andpurchasing controls, and monitoredthrough supplier audits or other means,but the device manufacturer isultimately responsible for its decision tochoose a particular software product.The fact that a vendor refuses to provideaccess to its development process ordocumentation does not relieve thedevice manufacturer of thisresponsibility. Likewise, we note thatthe device manufacturer is not obligatedto install every software upgrade offeredby a vendor. Validation of thoseupgrades and support from the vendor,including access to the necessaryvendor documentation, need to play animportant role in the upgrade decision.

Some comments argued that softwarevalidation should be treated more likeprocess validation, which is onlyrequired if the output of the processcannot be fully verified by subsequentinspection and testing. Other commentsasked for clarification of the term‘‘verification by output’’ and askedwhether it negated the requirement forsoftware validation. One commentargued that output of software drivensystems can never be fully verified.Another comment suggested theconsideration of intended use anddependence upon software for properoperation of the process to determinewhether verification could besubstituted for software validation.

In response to these comments, webelieve there are very few exampleswhere ‘‘verification’’ in lieu of softwarevalidation could be justified, and evenin those cases, most manufacturerswould choose to validate the softwarerather than go through repeatedverifications of output. For example,while every aspect of a drawing from acomputer-aided design (CAD) systemcan be independently verified, no userof a CAD system is likely to go to thattrouble or expense for every aspect ofevery drawing. Likewise, becausesoftware itself cannot be fully verified,automated software development tools

used to create medical device softwaremust be validated for their intended use.

Requirements are needed to establishintended use, the degree of dependenceon the software, and therefore thedegree of validation needed. The devicemanufacturer decides whether or not touse OTS software. The ability tovalidate for intended use and vendorsupport for the effort should be a partof that decision. Static analysis andstructural testing are techniques to beused in evaluating source code and lifecycle documentation, when these itemsare available. Otherwise, the devicemanufacturer is dependent uponfunctional testing alone. This issue isdiscussed in response to comment 136in the preamble to the quality systemregulation (61 FR 52602 at 52630). Theimpact on the safety and quality of themedical device is an importantdetermining factor in the approach andlevel of effort to be applied forvalidating automated manufacturingand quality system software, just as it isfor software in a medical device.

R. References

There were numerousrecommendations for additionalreferences. Those and many otherreference books, international standards,and FDA guidance documents havebeen added to the appendix at the endof the validation guidance.

For ease of cross reference, the text ofcomment 136 from the preamble of thequality system regulation is includedbelow:

136. One comment on § 820.70(h),‘‘Automated processes,’’’ (now § 820.70(i)),stated that the section should be revised toreflect that software used in such systemsmust be validated for ‘‘its intended use,’’ notsimply validated. Another comment statedthat most companies buy software currentlyavailable on the market and do not makechanges to the software. It was recommendedthat § 820.70(h) allow for use of outsidepersonnel for validation runs and notnecessarily require the development of asoftware validation procedure. One commentsuggested that the section should allowverification rather than validation of off-the-shelf software. Several comments on‘‘automated processes’’’ stated that the term‘‘data processing systems’’ was unclear andits inclusion rendered the requirement toobroad. Others asked for clarification of‘‘automated data processing systems.’’

FDA has modified the requirement tomandate validation for the intended use ofthe software. In addition, the requirementthat the software be validated by individualsdesignated by the manufacturer has also beendeleted to make clear that validation may beperformed by those other than themanufacturer. However, whether themanufacturer designates its own personnel orrelies on outside assistance to validate

software, there must be an establishedprocedure to ensure validation is carried outproperly.

FDA has maintained the requirement forvalidation because the agency believes that itis necessary that software be validated to theextent possible to adequately ensureperformance. Where source code and designspecifications cannot be obtained, ‘‘black boxtesting’’ must be performed to confirm thatthe software meets the user’s needs and itsintended uses.

FDA emphasizes that manufacturers areresponsible for the adequacy of the softwareused in their devices, and activities used toproduce devices. When manufacturerspurchase ‘‘off-the-shelf’’ software, they mustensure that it will perform as intended in itschosen application.

FDA has amended the requirement to state‘‘When computers or automated dataprocessing systems are used as part ofproduction or the quality system,’’ forclarification. Software used in production orthe quality system, whether it be in thedesigning, manufacturing, distributing, ortracing, must be validated.

II. Significance of GuidanceThis guidance document represents

the agency’s current thinking onsoftware validation. It does not create orconfer any rights for or on any personand does not operate to bind FDA or thepublic. An alternative approach may beused if such approach satisfies theapplicable statutes and regulations.

The agency has adopted GGPs, andpublished the final rule, which set forththe agency’s regulations for thedevelopment, issuance, and use ofguidance documents (21 CFR 10.115).This guidance document is issued as alevel 1 guidance in accordance with theGGP regulations.

III. Electronic AccessIn order to receive ‘‘General

Principles of Software Validation’’ viayour fax machine, call the CDRH Facts-On-Demand system at 800–899–0381 or301–827–0111 from a touch-tonetelephone. Press 1 to enter the system.At the second voice prompt press 1 toorder a document. Enter the documentnumber (938) followed by the poundsign (#). Follow the remaining voiceprompts to complete your request.

Persons interested in obtaining a copyof the guidance may also do so using theInternet. CDRH maintains an entry onthe Internet for easy access toinformation including text, graphics,and files that may be downloaded to apersonal computer with Internet access.Updated on a regular basis, the CDRHhome page includes the civil moneypenalty guidance documents package,device safety alerts, Federal Registerreprints, information on premarketsubmissions (including lists of approvedapplications and manufacturers’



addresses), small manufacturers’assistance, information on videoconferencing and electronicsubmissions, Mammography Matters,and other device-oriented information.The CDRH home page may be accessedat http://www.fda.gov/cdrh. Guidancedocuments are also available on theDockets Management Branch Internetsite at http:/www.fda.gov/ohrms/dockets/default.htm.

IV. CommentsInterested persons may submit to the

Dockets Management Branch (addressabove) written or electronic commentsregarding this guidance at any time.Submit two copies of anycomments,except that individuals maysubmit one copy. Comments are to beidentified with the docket numberfound in brackets in the heading of thisdocument. The guidance document andreceived comments may be seen in theDockets Management Branch between 9a.m. and 4 p.m., Monday throughFriday.

Dated: December 11, 2001.Linda S. Kahan,Deputy Director, Center for Devices andRadiological Health.[FR Doc. 02–690 Filed 1–10–02; 8:45 am]BILLING CODE 4160–01–S

DEPARTMENT OF HEALTH ANDHUMAN SERVICES

National Institutes of Health

Government-Owned Inventions;Availability for Licensing

AGENCY: National Institutes of Health,Public Health Service, DHHS.ACTION: Notice.

SUMMARY: The inventions listed beloware owned by agencies of the U.S.Government and are available forlicensing in the U.S. in accordance with35 U.S.C. 207 to achieve expeditiouscommercialization of results offederally-funded research anddevelopment. Foreign patentapplications are filed on selectedinventions to extend market coveragefor companies and may also be availablefor licensing.ADDRESSES: Licensing information andcopies of the U.S. patent applicationslisted below may be obtained by writingto the indicated licensing contact at theOffice of Technology Transfer, NationalInstitutes of Health, 6011 ExecutiveBoulevard, Suite 325, Rockville,Maryland 20852–3804; telephone: 301/496–7057; fax: 301/402–0220. A signedConfidential Disclosure Agreement will

be required to receive copies of thepatent applications.

Expression, Purification and EfficacyTesting of Synthetic PlasmodiumFalciparum Apical Membrane Antigen1 Expressed in Pichia PastorisStowers et al. (NIAID)DHHS Reference No. E–025–02/0 filed

09 Nov 2001Licensing Contact: Carol Salata; 301/

496–7735 ext. 232; e-mail:[email protected] challenge facing the biotechnology

industry involves finding robustsystems for the expression of largeamounts of recombinant protein. Extratechnological hurdles are faced whenthese proteins are required fortherapeutic usages.

Malaria remains one of the leadingcauses of both morbidity and mortalityin the tropical and sub-tropical world.Currently, there is no malaria vaccine.This invention relates to both of theseissues.

Two recombinant forms of the malariaasexual blood stage antigen ApicalMembrane Antigen 1 (AMA1) wereproduced in Pichia pastoris using totallydefined, synthetic medias and afermentation methodology that has beenreproducibly scaled over a 10-fold rangeto 60L. High levels of secretedrecombinant protein were obtained(300mg/L secreted protein in thesupernatant, and >50mg/L final purifiedbulk protein), and a purification strategydeveloped to remove Host cell-derivedlipids. Highly purified forms of bothtypes of AMA1 produced appear toproduce antibodies in vivo in rabbitsthat block homologous parasites frominvading red blood cells in vitro. Thecombination of the two allelic formsmade appears potent at inducingantibodies capable of blocking theinvasion of many heterologous parasitestrains in vitro, suggesting that thecombination of these two alleles ofAMA1 will provide sufficient coveragefrom the diverse field populations ofparasites. One of the two AMA1’s, basedon the FVO allelic variant of AMA1,was emulsified with complete andincomplete Freund’s adjuvant.

Vaccination of highly susceptibleAotus vociferans monkeys with thisformulation conferred significantprotection from a subsequent lethalchallenge with the virulent FVOPlasmodium falciparum parasite. Five ofeight animals whose primary immuneresponse was directed against AMA1were completely protected. These tworecombinant form of AMA1 may be aneffective malaria vaccine. Theproduction and purificationmethodologies may be suitable to other

therapeutic proteins where large-scale,inexpensive production methodologiesare required.

Two cDNA Clones of Hepatitis E Virus(HEV) That Are Infectious for Primatesand Encode a Virulent and anAttenuated Virus Respectively

Suzanne U. Emerson, Robert H. Purcell,Mingdong Zhang, and Xiang-Jin Meng(NIAID)

DHHS Reference No. E–278–01/0 filed09 Nov 2001

Licensing Contact: Carol Salata; 301/496–7735 ext. 232; e-mail:[email protected] E virus (HEV) is a human

pathogen that is the most importantcause of acute hepatitis in areas wherethe virus in endemic (Southeast andCentral Asia, and parts of Africa). Thisinvention relates to transcripts from thetwo cDNA clones that produced virusfollowing intrahepatic transfection ofchimpanzees. The virus encoded bycDNA with the consensus sequence ofthe wild-type Sar 55 Pakistani strain ofHEV caused liver enzyme elevations(i.e. acute hepatitis) in the chimpanzeeand resulted in seroconversion to anti-HEV at five weeks followinginoculation. The second cDNA differedfrom the first by a two nucleotides, oneof which was located in the codingregion. The nucleotide at this positionand the 18–20 nucleotides surroundingit are highly conserved in all strainssequenced thus far. Two chimpanzeesinoculated with transcripts from thisclone seroconverted to anti-HEV butseroconversion was delayed until week14 and liver enzyme levels did not rise,indicating the virus was attenuated.Viral sequences could be recovered fromthe serum of only one chimp and at onlyone time point by reverse-transcriptionpolymerase chain reaction, indicatingviral replication was inefficient. Anattenuated vaccine would be more costeffective than a recombinant proteinvaccine.

Suppression of CCR5 but Not CXCR4-Tropic HIV–1 Replication in LymphoidTissue by Human Herpes Virus 6

Margolis et al. (NICHD)DHHS Reference No. E–089–01/0 filed

28 Mar 2001Licensing Contact: Carol Salata; 301/

496–7735 ext. 232; e-mail:[email protected]–1 infects cells via a receptor

complex formed by CD4 and acoreceptor, such as CCR5 or CXCR4.The early stages of HIV–1 infection aredominated by CCR5-tropic viralvariants. CXCR4-tropic variantsfrequently emerge at later stages


Guidance for Industry and FDA Staff

Guidance for the Content of Premarket Submissions for Software

Contained in Medical Devices

Document issued on: May 11, 2005

This document supersedes Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices, issued May 29, 1998, and Reviewer Guidance for a Premarket Notification Submission for Blood Establishment Computer Software, issued January 13, 1997. For questions regarding this document concerning devices regulated by CDRH contact David S. Buckles at (301) 443-8517. For questions regarding this document concerning devices regulated by CBER contact Linda Weir at (301) 827-6136.

U.S. Department of Health and Human Services Food and Drug Administration


Office of Device Evaluation Office of In Vitro Diagnostics

Center for Biologics Evaluation and Research

Office of Blood Research and Review

Alan

June 27, 2005 (Updated Dec. 8, 2005 with additional explanation) SoftwareCPR explanatory notes are designated in red text surrounded by a red border to distinguish them as not part of the original FDA document (the border will show up in B&W if printed). SoftwareCPR emphasis of FDA text are shown in yellow highlighting which may not show up if printed in B&W. Disclaimer: explanatory notes represent only SoftwareCPR's understanding and interpretation at the time. FDA's actual requirements and interpretation evolve over time and vary by device, branch, and reviewer. No warranty is made by SoftwareCPR. SoftwareCPR is simply sharing its understanding as one source of input for use by experienced regulatory affairs staff.

Alan

yellow highlighting

Contains Nonbinding Recommendations

Preface

Public Comment Comments and suggestions may be submitted at any time for Agency consideration to the Division of Dockets Management, Food and Drug Administration, 5630 Fishers Lane, Room 1061, (HFA-305), Rockville, MD, 20852. When submitting comments, please refer to the exact title of this guidance document. Comments may not be acted upon by the Agency until the document is next revised or updated.

Additional Copies

CDRH

Additional copies are available from the Internet at:http://www.fda.gov/cdrh/ode/guidance/337.pdf, or to receive this document by fax, call the CDRH Facts-On-Demand system at 800-899-0381 or 301-827-0111 from a touch-tone telephone. Press 1 to enter the system. At the second voice prompt, press 1 to order a document. Enter the document number (337) followed by the pound sign (#). Follow the remaining voice prompts to complete your request.

CBER

Additional copies of this guidance are available from the Office of Communication, Training and Manufacturers Assistance (HFM-40), 1401 Rockville Pike, Rockville, MD 20852-1448 or by calling 1-800-835-4709 or 301-827-1800, or from the Internet at http://www.fda.gov/cber/guidelines.htm.


Table of Contents

Introduction................................................................................................................................ 1

The Least Burdensome Approach............................................................................................... 1

Scope......................................................................................................................................... 2

Relationship to Other Documents................................................................................................. 3

Terminology................................................................................................................................ 3

Level of Concern........................................................................................................................ 4

Determining Level of Concern..................................................................................................... 6

Software-related Documentation................................................................................................. 8

The Special 510(k) Program..................................................................................................... 15

The Abbreviated 510(k) Program.............................................................................................. 16

Additional Topics...................................................................................................................... 17

References................................................................................................................................ 19


1

Guidance for Industry and FDA Staff

Guidance for the Content of Premarket Submissions for Software Contained in

Medical Devices

This guidance represents the Food and Drug Administration's (FDA's) current thinking on this topic. It does not create or confer any rights for or on any person and does not operate to bind FDA or the public. You can use an alternative approach if the approach satisfies the requirements of the applicable statutes and regulations. If you want to discuss an alternative approach, contact the FDA staff responsible for implementing this guidance. If you cannot identify the appropriate FDA staff, call the appropriate number listed on the title page of this guidance.

Introduction This guidance document is intended to provide information to industry regarding the documentation that we recommend you include in premarket submissions for software devices, including stand-alone software applications and hardware-based devices that incorporate software. This document is a result of ongoing efforts to state our recommendations more clearly and ensure they remain current as technology advances. This document also combines into one guidance recommendations previously included in two guidance documents.i

The Least Burdensome Approach The issues identified in this guidance document represent those that we believe should be addressed before your device can be marketed. In developing the guidance, we carefully considered the relevant statutory criteria for Agency decision-making. We also considered the burden that may be incurred in your attempt to follow the guidance and address the issues we have identified. We believe that we have considered the least burdensome approach to resolving the issues presented in the guidance document. If, however, you believe that there is a less burdensome way to address the issues, you should follow the procedures outlined in the guidance, A Suggested Approach to Resolving Least Burdensome Issues, http://www.fda.gov/cdrh/modact/leastburdensome.html.

Alan Kusinitz

two guidance documents.

Alan

The 1997 BECS Guidance and the 1998 version of this guidance. Note the Off-the-shelf software guidance is still current and to be used with this guidance.


2

FDA's guidance documents, including this guidance, do not establish legally enforceable responsibilities. Instead, guidances describe the Agency's current thinking on a topic and should be viewed only as recommendations, unless specific regulatory or statutory requirements are cited. The use of the word should in Agency guidances means that something is suggested or recommended, but not required.

Scope For the purposes of this document, we refer to devices that contain one or more software components, parts, or accessories, or are composed solely of software as “software devices,” including:

• firmware and other means for software-based control of medical devices

• stand-alone software applications

• software intended for installation in general-purpose computers

• dedicated hardware/software medical devices.

• accessories to medical devices when those accessories contain or are composed of software.

This guidance applies to software devices regardless of the means by which the software is delivered to the end user, whether factory-installed, installed by a third-party vendor, or field-installed or -upgraded.

Software not covered by this guidance includes software designed for manufacturing or other process-control functions but not intended for use as a device. For further information or to clarify the requirements for your device, please contact the responsible FDA review division.

This guidance document applies to all types of premarket submissions for software devices, including:

• Premarket Notification (510(k)) including Traditional, Special, and Abbreviated submissions

• Premarket Approval Application (PMA)

• Investigational Device Exemption (IDE)

• Humanitarian Device Exemption (HDE), including amendments and supplements.

Alan

Our experience is that FDA is somewhat flexible in its application of this guidance and the 3 levels provided represent a continuum that can vary by their concern for certain types of devices.

Alan

stand-alone software applications

Alan

If they are medical devices and not exempt from pre-market submission.

Alan

And supplements if software changes are involved seems implied.

Alan Kusinitz

firmware


3

Relationship to Other Documents

FDA Guidance Documents

We intend this document to complement other existing guidance documents that provide recommendations related to software. For example, we recommend that you also refer to the guidance “General Principles of Software Validation”ii for recommendations on software related to a device (including software that is a stand-alone device or that is a component, part, or accessory of a device). We recommend that you refer to the “Guidance for Off-the-Shelf Software Use in Medical Devices”iii in cases where your device uses off-the-shelf software.

Manufacturers of Software Devices should create and maintain software-related documentation in accordance with the requirements of the Quality System Regulationiv (QS regulation) (21 CFR part 820). As with other FDA guidance documents that provide recommendations, please note that following the recommendations of this guidance is not a substitute for compliance with the QS regulation.

Software-Related Consensus Standards

The emergence of consensus standards related to software has helped to improve the consistency and quality of software development and documentation, particularly with respect to critical activities such as risk assessment and management. When possible, we harmonized the terminology and recommendations in this guidance with software-related consensus standards such as ISO 14971v and AAMI SW68.vi

Terminology

Verification and Validation

This document uses the terms "verification" and "validation" (also referred to as “V&V”) as they are defined in the QS regulation.iv

Verification “means confirmation by examination and provision of objective evidence that specified requirements have been fulfilled.” 21 CFR 820.3(aa). In a software development environment, software verification is confirmation that the output of a particular phase of development meets all of the input requirements for that phase. Software testing is one of several verification activities intended to confirm that the software development output meets its input requirements. Other verification activities include:

• walk-throughs

• various static and dynamic analyses

• code and document inspections

Alan

This guidance only identifies information for the submission. Additional software validation information should be on file to demonstrate compliance with 820.30 of the QS Regulation

Alan

Note that FDA staff was involved in AAMI TIR32 which provides reference material on performing adequate software risk/hazard analysis.

Alan

This makes explicit that review and inspections are a valid type of verification activity (and we assume therefore they can be used to demonstrate design and code coverage in some cases). Of course these static test methods can not replace all dynamic tests.


4

• module level testing

• integration testing. Design validation “means establishing by objective evidence that device specifications conform with user needs and intended use(s).” 21 CFR 820.3(z)(2). Use of the term validation in this document is limited to design validation and does not include process validation as defined in 21 CFR 820.3(z)(1). One component of design validation is software validation. Software validation refers to establishing, by objective evidence, that the software conforms with the user needs and intended uses of the device. Software validation is a part of design validation of the finished device. It involves checking for proper operation of the software in its actual or simulated use environment, including integration into the final device where appropriate. Software validation is highly dependent upon comprehensive software testing and other verification tasks previously completed at each stage of the software development life cycle. Planning, verification, traceability, configuration management, and many other aspects of good software engineering are important activities that together help to support a conclusion that software is validated.

Minor and Serious Injuries

For the purposes of this document, we use the term minor injury to mean any injury that does not meet the definition of a serious injury as defined in 21 CFR 803.3(bb)(1). This regulation defines serious injury as an injury or illness that:

i. is life threatening;

ii. results in permanent impairment of a body function or permanent damage to a body structure; or

iii. necessitates medical or surgical intervention to preclude permanent impairment of a body function or permanent damage to a body structure.

For the purposes of this document, the term permanent is defined as “irreversible impairment or damage to a body structure or function, excluding trivial impairment or damage.” 21 CFR 803.3(bb)(2).

Level of Concern

Introduction

The documentation that we recommend you include in a premarket submission generally depends on the device’s Level of Concern. For the purposes of this guidance document, Level of Concern refers to an estimate of the severity of injury that a device could permit or inflict, either directly or indirectly, on a patient or operator as a result of device failures, design flaws,

Alan

Note that our experience has been that for many types of devices module, integration, and system testing can be combined to some degree into common test cases provided a good explanation is provided as to how the test cases address the intent of module, integration and system testing.

Alan

Note that our experience has been that for many types of devices software system testing does not need to be separated from overall device validation although this can be useful.


5

or simply by virtue of employing the device for its intended use. We recommend that you describe the role of the software in causing, controlling, and/or mitigating hazards that could result in injury to the patient or the operator, because this is also a factor in determining the appropriate Level of Concern for your device.

The extent of documentation that we recommend you submit for your Software Device is proportional to the Level of Concern associated with the device. Level of Concern is defined only for use in this context and is not related to device classification (Class I, II or III) or to hazard or risk analysis per se.

Major, Moderate, or Minor Level of Concern

The following sections provide recommendations for determining the Level of Concern that may be appropriate for your Software Device and recommendations for documentation that you should submit for each Level of Concern. We recommend that you determine the Level of Concern before any mitigation of relevant hazards. In other words, the Level of Concern should be driven by the hazard analysis in the absence of mitigations, regardless of the effects of the mitigations on the individual hazards. FDA recommends that you state in your submission the Level of Concern you have determined for your Software Device. It may be Major, Moderate or Minor as defined below. We also recommend that you describe how you arrived at that Level of Concern. The Level of Concern is based on how the operation of the software associated with device function affects the patient or operator. The effect may be direct or indirect.

Major

We believe the level of concern is Major if a failure or latent flaw could directly result in death or serious injury to the patient or operator. The level of concern is also Major if a failure or latent flaw could indirectly result in death or serious injury of the patient or operator through incorrect or delayed information or through the action of a care provider. Moderate

We believe the level of concern is Moderate if a failure or latent design flaw could directly result in minor injury to the patient or operator. The level of concern is also Moderate if a failure or latent flaw could indirectly result in minor injury to the patient or operator through incorrect or delayed information or through the action of a care provider. Minor

We believe the level of concern is Minor if failures or latent design flaws are unlikely to cause any injury to the patient or operator.

Alan

in the absence of mitigations, regardless of the effects of the mitigations on the individual hazards.

Alan

This is an important requirement.

Alan

Note FDA states LOC for certain types of devices in device-specific guidance (e.g., MRI,s glucose meters,..) in ways that seem contrary to this breakdown (such as MIMS/PACS considered Minor LOC and Home Glucose meters considered Moderate LOC). This is due to their opinion of actual risk history and the amount of documentation they want to see.

Alan

Minor is the only LOC where likelihood is stated as a consideration. Since above FDA indicates LOC should be determined prior to mitigations likelihood should be based on external factors such as clinical practice and intended use and users.

Alan

Check device-specific guidances since some provide FDA's designation of LOC and they maybe be counter-intuitive in terms of these definitions, often a lower level then one would expect.


6

Determining Level of Concern

We have provided the following key questions to assist you in determining the Level of Concern. We recommend that you assess the Level of Concern before mitigating any hazard; that is, you should assess your software device against these questions as though you have not implemented hazard mitigations.

If the answer to any question is No, continue on to the next question. As discussed in more detail later, we recommend that you include the basis for your conclusion as to the Level of Concern in your submission. In all cases, we recommend that you assess the Level of Concern within the context of the worst possible, reasonably foreseeable, clinical consequences of failure of the Software Device.

Table 1 Major Level of Concern

If the answer to any one question below is Yes, the Level of Concern for the Software Device is likely to be Major.

1. Does the Software Device qualify as Blood Establishment Computer Software?

(Blood Establishment Computer Software is defined as software products intended for use in the manufacture of blood and blood components or for the maintenance of data that blood establishment personnel use in making decisions regarding the suitability of donors and the release of blood or blood components for transfusion or further manufacture.)

2. Is the Software Device intended to be used in combination with a drug or biologic?

3. Is the Software Device an accessory to a medical device that has a Major Level of Concern?

4. Prior to mitigation of hazards, could a failure of the Software Device result in death or serious injury, either to a patient or to a user of the device? Examples of this include the following:

a. Does the Software Device control a life supporting or life sustaining function?


7

b. Does the Software Device control the delivery of potentially harmful energy that could result in death or serious injury, such as radiation treatment systems, defibrillators, and ablation generators?

c. Does the Software Device control the delivery of treatment or therapy such that an error or malfunction could result in death or serious injury?

d. Does the Software Device provide diagnostic information that directly drives a decision regarding treatment or therapy, such that if misapplied it could result in serious injury or death?

e. Does the Software Device provide vital signs monitoring and alarms for potentially life threatening situations in which medical intervention is necessary?

Table 2 Moderate Level of Concern

If the Software Device is not Major Level of Concern and the answer to any one question below is Yes, the Level of Concern is likely to be Moderate.

1. Is the Software Device an accessory to a medical device that has a Moderate Level of Concern?

2. Prior to mitigation of hazards, could a failure of the Software Device result in Minor Injury, either to a patient or to a user of the device?

3. Could a malfunction of, or a latent design flaw in, the Software Device lead to an erroneous diagnosis or a delay in delivery of appropriate medical care that would likely lead to Minor Injury?

If the answers to all of the questions in Tables 1 and 2 above are No, the Level of Concern is Minor.


8

The review divisions at FDA are available to discuss any questions you may have about the Level of Concern for your Software Device. If you believe the Level of Concern for your device is Major and you have not previously filed a premarket submission for this type of Software Device, we recommend that you contact the appropriate division at FDA to discuss your Software Device before filing your submission.

Software-related Documentation Software-related documentation that you provide in your premarket submission should be consistent with the intended use of the Software Device, the Level of Concern, and the type of submission. This section describes the documentation that we recommend you include in your premarket submission based on the Level of Concern (see Table 3). However, you should follow the recommendations in device-specific guidance, if available for your device. In general, the documentation provided in your submission should:

• describe the design of your device

• document how your design was implemented

• demonstrate how the device produced by your design implementation was tested

• show that you identified hazards appropriately and managed risks effectively

• provide traceability to link together design, implementation, testing, and risk management. The type and extent of documentation that we recommend you submit is summarized in Table 3. Our recommendations are keyed to the Level of Concern of your device. These recommendations are predicated on your effective implementation and management of the QSR, including Design Controls.iv We believe the documents that we recommend submitting will generally be the same documents that you would normally generate during the development of a Software Device. Therefore, in a properly managed and documented medical device software development environment, the documents that you submit in response to the recommendations in this guidance may be copies of your product development documents. We explain the documents that we recommend submitting in the sections following Table 3. In some instances, the recommended documentation for the Level of Concern may take the form of statements in the body of the submission; other documents, such as the Software Requirements Specification, will likely be stand-alone documents copied into the submission.

Alan

If FDA is asked we prefer to propose what we think is adequate documentation to submit rather then ask open ended questions.

Alan

device-specific guidance,

Alan

This is a nice goal but usually special summaries and roadmaps and explanation and rationale are needed to aide FDA in its review and to avoid confusion.


9

Table 3. Documentation Based on Level of Concern

SOFTWARE DOCUMENTATION

MINOR CONCERN

MODERATE CONCERN

MAJOR CONCERN

Level of Concern A statement indicating the Level of Concern and a description of the rationale for that level.

Software Description A summary overview of the features and software operating environment.

Device Hazard Analysis Tabular description of identified hardware and software hazards, including severity assessment and mitigations.

Software Requirements Specification (SRS)

Summary of functional requirements from SRS.

The complete SRS document.

Architecture Design Chart

No documentation is necessary in the submission.

Detailed depiction of functional units and software modules. May include state diagrams as well as flow charts.

Software Design Specification (SDS)


Software design specification document.

Traceability Analysis Traceability among requirements, specifications, identified hazards and mitigations, and Verification and Validation testing.

Software Development Environment Description


Summary of software life cycle development plan, including a summary of the configuration management and

Summary of software life cycle development plan. Annotated list of control documents generated during development process. Include the

Alan

Note this may be several documents from high level to detailed but often doesn't need to be at the highly technical/pseudocode level.

Alan

Note this may be several documents depending on the approach taken internal.


10

SOFTWARE DOCUMENTATION

MINOR CONCERN

MODERATE CONCERN

MAJOR CONCERN

maintenance activities.

configuration management and maintenance plan documents.

Verification and Validation Documentation

Software functional test plan, pass / fail criteria, and results.

Description of V&V activities at the unit, integration, and system level. System level test protocol, including pass/fail criteria, and tests results.

Description of V&V activities at the unit, integration, and system level. Unit, integration and system level test protocols, including pass/fail criteria, test report, summary, and tests results.

Revision Level History Revision history log, including release version number and date.

Unresolved Anomalies (Bugs or Defects)


List of remaining software anomalies, annotated with an explanation of the impact on safety or effectiveness, including operator usage and human factors.

Level of Concern

We recommend that you indicate the Level of Concern for your Software Device, determined before the effects of any mitigations. We recommend that you clearly state which one of the three levels of concern is appropriate for your device and include documentation of the rationale for your decision. We also recommend that your documentation make your decision-making process apparent to FDA.

Software Description

We recommend that you provide a comprehensive overview of the device features that are controlled by software, and describe the intended operational environment. Generally, we

Alan

Note that for most devices FDA doesn't want thousands of pages of test results and data dumps so some flexibility is allowed in the level of detail of results and objective evidence provided even for Moderate and Major LOC. highly technical/pseudocode level. Also for Major only examples of Unit and Integration test are required as indicated later in this guidance.

Alan

Usually a summary of key versions subject to formal test.

Alan

Take care to consider all existing anomalies in the version for which clearance is being requested regardless of how or when they were reported including domestic or overseas. Also be sure the list is accurate and complete at the date of submission. Review pending change and enhancement lists since FDA might consider some of these items actually to be bugs or defects.


11

recommend that you provide the information in paragraph format and highlight major or operationally significant software features. The software description should include information on the following:

• programming language

• hardware platform

• operating system (if applicable)

• use of Off-the-Shelf software (if applicable).

If your device uses Off-the Shelf software, please refer to the FDA guidance document “Guidance for Off-the-Shelf Software Use in Medical Devices.”iii If this information is included in another document, such as the Software Requirements Specification, your submission should contain an annotation and a reference to the document in the submission where this information is located.

Device Hazard Analysis

We recommend that you submit a Device Hazard Analysis for all Software Devices. The Device Hazard Analysis should take into account all device hazards associated with the device’s intended use, including both hardware and software hazards. We recommend that you present the information in tabular form with a line item for each identified hazard. This document can be in the form of an extract of the software-related items from a comprehensive risk management document, such as the Risk Management Summary described in ISO 14971.v In this format, each line item should include:

• identification of the hazardous event

• severity of the hazard

• cause(s) of the hazard

• method of control (e.g., alarm, hardware design)

• corrective measures taken, including an explanation of the aspects of the device design/requirements, that eliminate, reduce, or warn of a hazardous event; and

• verification that the method of control was implemented correctly.

When performing a hazard analysis, we recommend that you address all foreseeable hazards, including those resulting from intentional or inadvertent misuse of the device.

Software Requirements Specification

The Software Requirements Specification (SRS) documents the requirements for the software. This typically includes functional, performance, interface, design, developmental, and other requirements for the software. In effect, this document describes what the Software Device is supposed to do. Examples of some typical requirements that would be included in a SRS are

Alan

If your device uses Off-the Shelf software, please refer to the FDA guidance document “Guidance for Off-the-Shelf Software Use in Medical Devices.”iii

Alan

including those resulting from intentional or inadvertent misuse of the device.

Alan Kusinitz

includes functional, performance, interface, design, developmental,

Alan

Note that likelihood/probability is not mentioned here due to FDA's skepticism of the ability to properly apply likelihood especially for likelihood of defects. If you do include likelihood do not use it to try to justify no risk control needed.


12

described below. For Software Devices that are identified as Minor Level of Concern, we recommend that you provide only the summary functional requirements section from the SRS, including identification of off-the-shelf software. For Software Devices that are identified as Major or Moderate Level of Concern, we recommend that you provide the complete SRS document.

Hardware Requirements

Hardware requirements generally include:

• microprocessors

• memory devices

• sensors

• energy sources

• safety features

• communications. Programming Language Requirements

Programming language requirements include program size requirements or restrictions, and information on management of memory leaks. Interface Requirements

Interface requirements generally include both communication between system components and communication with the user such as:

• printers

• monitors

• keyboard

• mouse. Software Performance and Functional Requirements

Software performance and functional requirements include algorithms or control characteristics for therapy, diagnosis, monitoring, alarms, analysis, and interpretation with full text references or supporting clinical data, if necessary. Software performance and functional requirements may also include:

• device limitations due to software

• internal software tests and checks

• error and interrupt handling

• fault detection, tolerance, and recovery characteristics

Alan Kusinitz

algorithms or control characteristics

Alan

Note that lthese are indented subsections. FDA expects information (summary or detailed depending on the Level of Concern) to include somthing about each of the following itmes : -Harware requirements -Programming Language Requirements - Interfaces - Performance Reqsuirements - Functional Requirements Their intent is defined in each of these subsections.

Alan

A description of algorithms is very important. The higher the level of concern the more detail is expected.


13

• safety requirements

• timing and memory requirements

• identification of off-the-shelf software, if appropriate.

Architecture Design Chart

This document is typically a flowchart or similar depiction of the relationships among the major functional units in the Software Device, including relationships to hardware and to data flows such as networking. It is usually not necessary to include every function call and module in this document; however, there should be sufficient information to allow for review of the organization of the software relative to the functionality and to the intended use of the Software Device. For Moderate and Major Level of Concern devices, detailed information such as state diagrams may be useful to clearly depict the relationships among the software functional units. If the Architecture Design Chart is included in another document such as the SRS then you should include in your submission a statement to that effect and a reference to the location of the Architecture Design Chart in the submission.

Software Design Specification

The Software Design Specification (SDS) describes the implementation of the requirements for the Software Device. In terms of the relationship between the SRS and the SDS, the SRS describes what the Software Device will do and the SDS describes how the requirements in the SRS are implemented. The information presented in the SDS should be sufficient to ensure that the work performed by the software engineers who created the Software Device was clear and unambiguous, with minimal ad hoc design decisions. The SDS may contain references to other documents, such as detailed software specifications. However, the document you submit should, in and of itself, provide adequate information to allow for review of the implementation plan for the software requirements in terms of intended use, functionality, safety, and effectiveness.

Traceability Analysis

A Traceability Analysis links together your product design requirements, design specifications, and testing requirements. It also provides a means of tying together identified hazards with the implementation and testing of the mitigations. We recommend that you submit for review explicit traceability among these activities and associated documentation because they are essential to effective product development and to our understanding of product design, development and testing, and hazard mitigations. The Traceability Analysis commonly consists of a matrix with line items for requirements, specifications and tests, and pointers to hazard mitigations. It is possible to document traceability simply through a shared organizational

Alan

is typically a flowchart or similar depiction of the relationships among the major functional units in the Software Device,

Alan

The SDS may contain references to other documents, such as detailed software specifications. However, the document you submit should, in and of itself, provide adequate information to allow for review of the implementation plan for the software requirements in terms of intended use, functionality, safety, and effectiveness.

Alan

A Traceability Analysis links together your product design requirements, design specifications, and testing requirements. It also provides a means of tying together identified hazards with the implementation and testing of the mitigations.

Alan

Note that FDA is not requesting traceability to the code level.


14

structure with a common numbering scheme; however, we recommend that you include some mechanism, such as a matrix for guiding the reviewer through the information you submit.

Software Development Environment Description

For Moderate and Major Level of Concern Software Devices, the submission should include a summary of the software development life cycle plan. This summary should describe the sponsor’s software development life cycle and the processes that are in place to manage the various life cycle activities. For Major Level of Concern Software Devices, this document should also include an annotated list of the control/baseline documents generated during the software development process and a list or description of software coding standards. As mentioned elsewhere, configuration or change management is a crucial aspect of software development. Changes to the Software Device after initial market release should be subject to positive control, with definitive specification and test plans including well-defined regression testing where appropriate. The description of the development environment should provide information on your configuration management and maintenance plan that addresses these aspects of the software development life cycle. For a Major Level of Concern device, we recommend that you provide sufficient detail to allow for a thorough understanding of the configuration management and maintenance plan. For a Moderate Level of Concern device, we recommend that you provide only a summary of the configuration management and maintenance plans.

Verification and Validation Documentation

The terms “verification” and “validation” described earlier in this document refer to two phases of Software Device testing. This section recommends the type of testing documentation you should include in a premarket submission for a Software Device, based on the Level of Concern.

Minor Level of Concern Devices

For Minor Level of Concern devices, we recommend that you submit documentation of system or device level testing, and, where appropriate, integration testing. The documentation submitted should include system or device level test pass/fail criteria and a summary of the test results. Moderate Level of Concern Devices

For Moderate Level of Concern devices, we recommend that you submit a summary list of validation and verification activities and the results of these activities. We also recommend that you submit your pass/fail criteria. You should ensure that the Traceability Analysis effectively links these activities and results to your design requirements and specifications.

Alan

For a Major Level of Concern device, we recommend that you provide sufficient detail to allow for a thorough understanding of the configuration management and maintenance plan. For a Moderate Level of Concern device, we recommend that you provide only a summary of the configuration management and maintenance plans.

Alan

For Moderate Level of Concern devices, we recommend that you submit a summary list of validation and verification activities and the results of these activities. We also recommend that you submit your pass/fail criteria.


15

Major Level of Concern Devices

For Major Level of Concern devices, we recommend that you submit the information recommended above for Moderate Level of Concern devices and a description of any tests that were not passed. We also recommend that you include any modifications made in response to failed tests and documentation of results demonstrating that the modifications were effective. Documentation provided in your submission should include examples of unit integration testing and a summary of the results.

Revision Level History

Your submission should include the history of software revisions generated during the course of product development. This typically takes the form of a line-item tabulation of the major changes to the software during the development cycle, including date, version number, and a brief description of the changes in the version relative to the previous version. The last entry in the list should be the final version to be incorporated in the released device. This entry should also include any differences between the tested version of software and the released version, along with an assessment of the potential effect of the differences on the safety and effectiveness of the device.

Unresolved Anomalies (Bugs or Defects)

For Moderate and Major Level of Concern Software Devices, the submission should include a list of all unresolved software anomalies. For each anomaly, we recommend that you indicate the:

• problem

• impact on device performance

• any plans or timeframes for correcting the problem (where appropriate). We recommend that you annotate each item with an explanation of the impact of the anomaly on device safety or effectiveness, including operator usage and human factors issues. Typically, this list can be generated as an output of a change control board or similar mechanism for evaluation and disposition of unresolved software anomalies. We recommend that you communicate this list to the end user as appropriate to assist in the proper operation of the device. In all instances where it is practical to do so, you should include any mitigations or possible work-arounds for unresolved anomalies; this recommendation applies to Blood Establishment Computer Software in particular.

The Special 510(k) Program For a premarket submission to qualify for review under the Special 510(k) Program, the device should be a modification of your 510(k) cleared device that you own, where the modification does

Alan

We also recommend that you include any modifications made in response to failed tests and documentation of results demonstrating that the modifications were effective.

Alan

Documentation provided in your submission should include examples of unit integration testing and a summary of the results.

Alan

major changes to the software during the development cycle,

Alan

The last entry in the list should be the final version to be incorporated in the released device. This entry should also include any differences between the tested version of software and the released version, along with an assessment of the potential effect of the differences on the safety and effectiveness of the device.

Alan

Note that FDA is unlikely to clear/approve submissions with unresolved anomalies that are related to safety or effectiveness. It rarely just excepts a promise to fix these.

Alan

We recommend that you annotate each item with an explanation of the impact of the anomaly on device safety or effectiveness,

Alan

We recommend that you communicate this list to the end user

Alan

you should include any mitigations or possible work-arounds for unresolved anomalies; this recommendation applies to Blood Establishment Computer Software in particular.


16

not alter the intended use or the fundamental scientific technology of the devicevii. In a Special 510(k), you should follow the recommendations in this guidance on the documentation to submit, but submit only the documentation related to the modification that prompted the submission. For example, when submitting the documentation of requirements and specifications in a Special 510(k), the documentation should focus on the modifications and may not necessarily include all of the requirements and specifications of the entire device. We recommend that you submit the regression testing performed to verify and validate the modifications. We recommend that you submit your test plans, pass/fail criteria, and summary results rather than test data. In all cases, the type of software-related documentation and the level of detail you provide should be appropriate to the Level of Concern associated with your device in the context of the modifications. Since a Special 510(k) submission relies on your declaration of conformance to design controls, we believe you cannot properly submit a Special 510(k) until you have completed testing or other activities relied on by your declaration (see section 514(c)(1)(B) of the Federal Food, Drug, and Cosmetic Act (Act) (21 U.S.C. 360d(c)(1)(B)).

The Abbreviated 510(k) Program An Abbreviated 510(k) submission must include the required elements identified in 21 CFR 807.87. In an Abbreviated 510(k), FDA may consider the contents of the documentation recommended in this guidance to be appropriate supporting data within the meaning of 21 CFR 807.87(f) or (g). Therefore, we recommend that you submit the documentation described in this guidance.viii If you choose to rely on an FDA-recognized standard for any part of the device design or testing, you may include either a:

• statement that testing will be conducted and meet specified acceptance criteria before the product is marketed; or

• declaration of conformity to the standard.ix Because a declaration of conformity is based on results from testing, we believe you cannot properly submit a declaration of conformity until you have completed the testing the standard describes. For more information, please refer to section 514(c)(1)(B) of the Act and the FDA guidance, “Use of Standards in Substantial Equivalence Determinations.”x If you declare conformance to a standard that recommends specific tests or testing methods for your Software Device, we recommend that you submit documentation of pass/fail criteria and associated test results along with your declaration of conformance. We also recommend that you list deviations from the tests and test methods specified in the standard and explain these deviations in terms of the impact on the safety and effectiveness of the Software Device. A list of FDA recognized consensus standards is available on the CDRH web site.xi

Alan

the documentation should focus on the modifications and may not necessarily include all of the requirements and specifications of the entire device.

Alan

We recommend that you submit the regression testing performed to verify and validate the modifications.

Alan

This includes several software-specific standards including AAMI SW68.


17

Additional Topics

Risk Assessment and Management

Background

Inadequate or inappropriate software development life cycle and risk management activities, inappropriate use of a Software Device, or operational errors can result in a variety of potential failures or design flaws. Among these are unsafe or ineffective delivery of energy, drugs, and life-supporting or life-sustaining functions. The delivery of incorrect or incomplete information causing a misdiagnosis or selection of the wrong treatment or therapy is also a potential failure associated with certain Software Devices. Therefore, the risks associated with potential failures or design flaws are a concern during the review of Software Devices. Risk Assessment and Level of Concern

As mentioned earlier, your assessment of the risks associated with your Software Device should assist you in determining an appropriate Level of Concern. We also recommend that you consider the Level of Concern for other devices of the same generic type or intended use. If you believe a different Level of Concern is appropriate for your device, we recommend that you submit a detailed explanation of your rationale. Risk Management

The risk associated with Software Devices varies over a continuum from negligible to very severe. In general, FDA considers risk as the product of the severity of injury and the probability of its occurrence. However, software failures are systemic in nature and therefore the probability of occurrence cannot be determined using traditional statistical methods. Therefore, we recommend that you base your estimation of risk for your Software Device on the severity of the hazard resulting from failure, assuming that the failure will occur. We also recommend that you use risk identification and control techniques described in consensus standards such as ISO 14971.v

Software Change Management

Design, development, testing, and version control of revisions to the software are as important as development and testing of the software that was reviewed in the premarket submission. We believe the majority of software-related device problems that occur in the field, including software-related device recalls, happen to devices that are running software that has been revised since premarket review. In some instances, revisions that did not require FDA review were implicated in adverse events and recalls.xii We believe this indicates the need for careful control of software revisions.

Alan

The delivery of incorrect or incomplete information causing a misdiagnosis or selection of the wrong treatment or therapy is also a potential failure associated with certain Software Devices.

Alan

In some instances, revisions that did not require FDA review were implicated in adverse events and recalls.


18

Blood Establishment Computer Software

In premarket submissions for Blood Establishment Computer Software, you should submit a complete copy of the User’s Manual as it will be provided to the user, including, but not limited to, a description of all limitations. Additionally, you should submit the documentation you will provide to the user to describe all outstanding anomalies or software defects with corresponding workarounds, where applicable, if these issues are not addressed in the User’s Manual.

Software of Unknown Pedigree (SOUP)

Some or all of the software contained in a Software Device may have been obtained by the submitter from a third party. The type and quality of documentation that accompanies this software can vary considerably. Software for which adequate documentation may be difficult to obtain is referred to as Software of Unknown Pedigree or “SOUP.”

It may be difficult for you to obtain, generate, or reconstruct appropriate design documentation as described in this guidance for SOUP. Therefore, we recommend that you explain the origin of the software and the circumstances surrounding the software documentation. Additionally, your Hazard Analysis should encompass the risks associated with the SOUP regarding missing or incomplete documentation or lack of documentation of prior testing. Nonetheless, the responsibility for adequate testing of the device and for providing appropriate documentation of software test plans and results remains with you.

Virus Protection Software

Software applications designed to protect information systems, including Software Devices, from harmful or malicious code (“viruses,” “worms,” etc.) are becoming more commonplace as devices become increasingly interconnected and therefore exposed to the external information environment. Issues related to installation and testing of virus protection software are beyond the scope of this document. You may contact the CDRH Office of Compliance for more information on this topic.

Interfaces, Networking, and Network Infrastructure

As mentioned above, Software Devices are increasingly interconnected, both through point-to-point interfaces for exchange of specific data with specific devices and by connection to local and wide area networks and the Internet. While data exchange and communication infrastructure such as telephone lines, local area networks, and broadband connections are not regulated as medical devices, connection to these carriers affects the operation of Software Devices, sometimes adversely. An example is a Software Device that is connected to a local area network and ceases to operate properly when a problem occurs with the network interface. We recommend that your software design should take into account both the capabilities and liabilities of the interfaces provided with your device, and in particular that your hazard analysis and mitigations encompass these issues.

Alan

SoftwareCPR is preparing a separate analysis of the impact of this guidance on BECS devices.

Alan

It is our understanding that this means no specific information on virus protection is required in submissions although it would seem that the hazard analysis would list these things as mitigations where relevant.

Alan

and in particular that your hazard analysis and mitigations encompass these issues.


19

Combination Products

Generally, the recommendations of this guidance will apply to the device component of combination products (such as drug-device and biologics-device combinations) when the device component meets the definition of a Software Device. For more information, you may contact the Office of Combination Products or the FDA review division that will have the lead review for your combination product.

References i This document combines the recommendations in “Guidance for FDA Reviewers and

Industry: Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices” issued on May 29, 1998, and “Reviewer Guidance for a Premarket Notification Submission for Blood Establishment Computer Software” issued on January 13, 1997.

ii “General Principles of Software Validation,” http://www.fda.gov/cdrh/comp/guidance/938.html.

iii “Guidance for Off-the-Shelf Software Use in Medical Devices” http://www.fda.gov/cdrh/ode/guidance/585.pdf.

iv 21 CFR 820.30 Subpart C – Design Controls of the Quality System Regulation. v ISO 14971-1; Medical devices - Risk management - Part 1: Application of risk analysis. vi AAMI SW68:2001; Medical device software - Software life cycle processes. vii See “The New 510(k) Paradigm – Alternate Approaches to Demonstrating Substantial

Equivalence in Premarket Notifications – Final Guidance,” available on the FDA Web site at http://www.fda.gov/cdrh/ode/parad510.html.

viii For more information see Device Advice, “How to Prepare an Abbreviated 510(k),” http://www.fda.gov/cdrh/devadvice/3145.html, in particular the section titled “Information Required in an Abbreviated 510(k).”

ix See “Required Elements for a Declaration of Conformity to a Recognized Standard (Screening Checklist for All Premarket Notification [510(K)] Submissions),” http://www.fda.gov/cdrh/ode/reqrecstand.html.

x See “Use of Standards in Substantial Equivalence Determinations,” http://www.fda.gov/cdrh/ode/guidance/1131.html.

xi http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfStandards/search.cfm. xii For information on determining when revisions to software should result in a new premarket

submission, you should consult the relevant FDA guidances such as “Deciding When to Submit

Alan

The CBER guidance is withdrawn and therefore abbreviated 510(k) based on it can longer be submitted which means additional submission information.


20

a 510(k) for a Change to an Existing Device,” http://www.fda.gov/cdrh/ode/510kmod.html. See also 21 CFR 807.81(a)(3).

Guidance for Industry, FDA Reviewers and Compliance on

Off-The-Shelf Software Use in Medical Devices

Document issued on: September 9, 1999

This document supersedes document, Guidance on Off-the-Shelf Software Use in Medical Devices, June 4, 1997.

U.S. Department Of Health And Human Services Food and Drug Administration


Office of Device Evaluation

Page 1 of 25Off-The-Shelf Software Use in Medical Devices

3/7/2002http://www.fda.gov/cdrh/ode/guidance/585.html

Preface Public Comment

Comments and suggestions may be submitted at any time for Agency consideration to Donna-Bea Tillman, Office of Device Evaluation at [email protected] or at 301-443-8517. Comments may not be acted upon by the Agency until the document is next revised or updated. For questions regarding the use or interpretation of this guidance contact Donna-Bea Tillman at [email protected] or at 301-443-8517. Questions regarding the use or interpretation of this guidance for a particular device should be directed to the appropriate ODE review division.

Additional Copies

World Wide Web/CDRH home page: http://www.fda.gov/cdrh/ode/guidance/585.pdf, or CDRH Facts on Demand at 1-800-899-0381 or 301-827-0111, specify number 585 when prompted for the document shelf number.

Table of Contents

1 Overview

1.1 Introduction and Background 1.2 Purpose / Scope 1.3 Definitions 1.4 OTS Software Decision Schematic

Figure 1-1. OTS Software Decision Schematic Table 1-1. Documentation Summary from Figure

2 OTS Software Use

2.1 BASIC DOCUMENTATION for OTS Software 2.2 OTS Software Hazard Analysis 2.3 OTS Software Hazard Mitigation

Figure 2-1. Typical Hazard Analysis and Mitigation Table 2-1. Injury Reduction Countermeasures

2.4 Describe and Justify Residual Risk 2.5 SPECIAL DOCUMENTATION for OTS Software

3 OTS Software Used in Marketing Applications

3.1 Examples



3.1.1 Corneal Topographer 3.1.2 Perineometer 3.1.3 Implantable Medical Device Programmers

3.2 510(k) Issues with OTS Software

3.2.1 OTS Software Changes Requiring a 510(k) 3.2.2 Exemption of Laboratory Information Management Systems

3.3 IDE Issues with OTS Software 3.4 Exemption of Certain Diagnostic Devices 3.5 PMA Issues with OTS Software 3.6 Artificial Intelligence 3.7 Product Labeling

4 Bibliography

5 Appendices

5.1 Operating Systems 5.2 Utilities and Drivers 5.3 Local Area Networks (LANs)

5.3.1 Requirements Analysis 5.3.2 Implementation

5.4 Device Master Files 5.5 Maintenance and Obsolescence

5.5.1 Safety 5.5.2 Design 5.5.3 Verification and Validation 5.5.4 Installation 5.5.5 Obsolescence 5.5.6 Change control

Numbers in square brackets [##] appearing in this guidance refer to citations in the Bibliography (Section 4)

1 Overview 1.1 Introduction and Background

Off-the-shelf (OTS) software is commonly being considered for incorporation into medical devices as the use of general purpose computer hardware becomes more prevalent. The use of OTS software in a medical device allows the manufacturer to concentrate on the application software needed to run device-specific functions. However, OTS software intended for general purpose computing may not be appropriate for a given specific use in a medical device. The medical device manufacturer using OTS software generally gives up software life cycle control, but still bears the responsibility for the continued



safe and effective performance of the medical device.

This guidance document was developed to address the many questions asked by medical device manufacturers regarding what they need to provide in a pre-market submission to the FDA when they use OTS software. The specific response to these questions depends on the medical device in question and the impact on patient, operator, or bystander safety if the OTS software fails. Thus, the answer to the question, "What do I need to document?" may differ and is based on the risk analysis that is an integral part of designing a medical device. The detail of documentation to be provided to FDA and the level of life cycle control necessary for the medical device manufacturer increase as severity of the hazards to patients, operators, or bystanders from OTS software failure increases.

This document lays out in broad terms how the medical device manufacturer can consider what is necessary to document for submission to the agency. A BASIC set of need-to-document items is recommended for all OTS software, and a detailed discussion is provided on additional (SPECIAL) needs and responsibilities of the manufacturer when the severity of the hazards from OTS software failure become more significant.

1.2 Purpose / Scope

This guidance document represents the agency’s current thinking on the documentation that should be provided in premarket submissions for medical devices using OTS software. It does not create or confer any rights for or on any person and does not operate to bind FDA or the public. An alternative approach may be used if such approach satisfies the requirements of the applicable statute, regulations or both. The FDA uses mandatory language, such as shall, must, and require, when referring to statutory or regulatory requirements. The FDA uses non-mandatory language such as should, may, can, and recommend when referring to guidance.

The purpose of this document is to describe the information that generally should be provided in a medical device application involving OTS software. This information is in addition to the documentation described in the Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices [4]. Many of the principles outlined herein may also be helpful to device manufacturers in establishing design controls and validation plans for use of off-the-shelf software in their devices. This guidance discusses key elements reviewers should look for in the submission thereby providing a common baseline from which both manufacturers and reviewers can operate. This should improve predictability of agency interaction with sponsors regarding applications involving OTS software.

The guidance provided in this document reflects a safety-based approach to risk management and is designed to be consistent with international standards on risk management. Existing international standards indicate that the estimation of risk should be considered as the product of the severity of harm and the probability of occurrence of harm. Probabilities of occurrence are calculated based on clinical and engineering considerations. On the clinical side, manufacturers use patient populations, user skill sets, labeling and risk benefit analysis to calculate risk and acceptable risk levels. On the software engineering side, probabilities of occurrence would normally be based on software failure rates. However, software failures are systematic in nature and therefore their probability of occurrence can not be determined using traditional statistical methods.

Because the risk estimates for hazards related to software cannot easily be estimated based on software failure rates, CDRH has concluded that engineering risk management for medical device software should focus on the severity of the harm that could result from the software failure. Hazard Analysis is



defined as the identification of Hazards and their initiating causes [IEC 60601-1-4]. Based on the definition of Risk Analysis in ISO DIS 14971 and EN 1441, hazard analysis is actually a subset of risk analysis; because risk analysis for software cannot be based on probability of occurrence, the actual function of risk analysis for software can then be reduced to a hazard analysis function. Technically speaking, the use of either term risk or hazard analysis is appropriate. However, CDRH has chosen to use the term hazard analysis to reinforce the concept that calculating risk based on software failure rates is generally not justified, and that it is more appropriate to manage software safety risk based on the severity of harm rather then the software failure rates.

1.3 Definitions

Following a safety-based approach to risk analysis, we define:

Hazard – A possible source of danger or a condition which could result in human injury.

Hazard Analysis – Identification of hazards and their initiating causes. [IEC 60601-1-4]

Hazard Mitigation – Reduction in the severity of the hazard, the likelihood of the occurrence, or both.

Major Level of Concern – The Level of Concern is major if operation of the software associated with device function directly affects the patient, operator, and/or bystander so that failures or latent flaws could result in death or serious injury to the patient, operator, and/or bystander, or if it indirectly affects the patient, operator, and/or bystander (e.g., through the action of care provider) such that incorrect or delayed information could result in death or serious injury to the patient, operator, and/or bystander.

Minor Level of Concern – The Level of Concern is minor if failures or latent design flaws would not be expected to result in any injury to the patient, operator, and/or bystander.

Moderate Level of Concern – The Level of Concern is moderate if the operation of the software associated with device function directly affects the patient, operator, and/or bystander so that failures or latent design flaws could result in non-serious injury to the patient, operator, and/or bystander, or if it indirectly affects the patient, operator, and/or bystander (e.g., through the action of the care provider) where incorrect or delayed information could result in non-serious injury of the patient, operator, and/or bystander.

Off-the-Shelf Software (OTS software) – A generally available software component, used by a medical device manufacturer for which the manufacturer can not claim complete software life cycle control.

Risk Analysis – Investigation of available information to identify hazards and to estimate risks. [ISO DIS 14971]

Risk Control – the process through which decisions are reached and implemented for reducing risks to, or maintaining risks within, specified limits. [ISO DIS 14971]

Safety – In the regulation of medical devices, safety means that the probable benefits to health for its intended use when accompanied by adequate directions and warnings against unsafe use, outweigh any probable risks. In this guidance we will use the words "safety and effectiveness" to remind ourselves that safety is only meaningful in the context of the benefit-risk considerations and the labeling.



Serious Injury – as adopted from the Medical Device Reporting (MDR) regulation in the Code of Federal Regulations 21 CFR 803.3 (aa), means an injury or illness that:

1. is life threatening, 2. results in permanent impairment of a body function or permanent damage to a body structure, or 3. necessitates medical or surgical intervention to preclude permanent impairment of a body function

or permanent damage to a body structure.

Permanent – for the purpose of this subpart, permanent means irreversible impairment or damage to a body structure or function excluding trivial impairment or damage.

Other software terminology used in this document is defined in FDA's Glossary of Computerized System and Software Development Terminology [6].

1.4 OTS Software Decision Schematic

The content of the application supporting use of OTS Software in a medical device depends on the results of the hazard analysis. Figure 1-1 provides a schematic of the decision process and a table of contents for Section 2 of this guidance document.

Figure 1-1. OTS Software Decision Schematic





Table 1-1 summarizes the recommended contents for an OTS Software submission based on Figure 1-1.

Table 1-1. Documentation Summary from Figure 1-1

2 OTS Software Use 2.1 BASIC DOCUMENTATION for OTS Software

The OTS Software BASIC DOCUMENTATION is intended to answer the following questions:

1. What is it? - For each component of OTS Software used, specify the following:

l Title and Manufacturer of the OTS Software. l Version Level, Release Date, Patch Number and Upgrade Designation as appropriate. l Any OTS Software documentation that will be provided to the end user. l Why is this OTS Software appropriate for this medical device? l What are the expected design limitations of the OTS Software?

Note: The medical device manufacturer should only use the OTS Software as specified in an appropriate document, i.e., design record. If the version of the OTS Software changes, the appropriate document should be updated to reflect the change.

2. What are the Computer System Specifications for the OTS Software? - For what configuration will the OTS software be validated? Specify the following:

l Hardware specifications: processor (manufacturer, speed, and features), RAM

Minor Level of Concern before mitigationsHazard Analysis Basic Documentation

Minor Level of Concern after mitigationsHazard Analysis Basis Documentation Hazard Mitigations

Moderate Level of ConcernHazard Analysis Basis Documentation Hazard Mitigations Describe and Justify Residual Risk

Major Level of Concern after mitigationsHazard Analysis Basic Documentation Hazard Mitigations Describe and Justify Residual Risk Special Documentation



(memory size), hard disk size, other storage, communications, display, etc. l Software specifications: operating system, drivers, utilities, etc. The software

requirements specification (SRS) listing for each item should contain the name (e.g., Windows 95, Excel, Sun OS, etc.), specific version levels (e.g., 4.1, 5.0, etc.) and a complete list of any patches that have been provided by the OTS Software manufacturer.

3. How will you assure appropriate actions are taken by the End User?

l What aspects of the OTS Software and system can (and/or must) be installed/configured?

l What steps are permitted (or must be taken) to install and/or configure the product? l How often will the configuration need to be changed? l What education and training are suggested or required for the user of the OTS

Software? l What measures have been designed into the medical device to prevent the operation

of any non-specified OTS Software, e.g., word processors, games? Operation of non-specified OTS Software may be prevented by system design, preventive measures, or labeling. Introduction may be prevented by disabling input (floppy disk, CD, tape drives, modems).

4. What does the OTS Software do? – What function does the OTS software provide in this device? This is equivalent to the software requirements in the Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices [4] for this OTS software. Specify the following:

l What is the OTS Software intended to do? The sponsor’s design documentation should specify exactly which OTS components will be included in the design of the medical device. Specify to what extent OTS Software is involved in error control and messaging in device error control.

l What are the links with other software including software outside the medical device (not reviewed as part of this or another application)? The links to outside software should be completely defined for each medical device/module. The design documentation should include a complete description of the linkage between the medical device software and any outside software (e.g., networks).

5. How do you know it works? – Based on the Level of Concern:

l Describe testing, verification and validation of the OTS Software and ensure it is appropriate for the device hazards associated with the OTS software. (See Note 1)

l Provide the results of the testing. (See Note 2) l Is there a current list of OTS Software problems (bugs) and access to updates?

Note 1: FDA recommends that software test, verification and validation plans identify the exact OTS Software (title and version) that is to be used. When the software is tested it should be integrated and tested using the specific OTS Software that will be delivered to the user.

Note 2: If the manufacturer allows the use of the medical device with different versions of OTS Software then the manufacturer should validate the medical



device for each OTS Software version.

6. How will you keep track of (control) the OTS Software? - An appropriate plan should answer the following questions:

l What measures have been designed into the medical device to prevent the introduction of incorrect versions? On startup, ideally, the medical device should check to verify that all software is the correct title, version level and configuration. If the correct software is not loaded, the medical device should warn the operator and shut down to a safe state.

l How will you maintain the OTS Software configuration? l Where and how will you store the OTS Software? l How will you ensure proper installation of the OTS Software? l How will you ensure proper maintenance and life cycle support for the OTS

Software?

2.2 OTS Software Hazard Analysis

A comprehensive risk management approach includes hazard analysis and mitigation that continues iteratively throughout the life of the product. The manufacturer is expected to perform an OTS Software hazard analysis as a part of a medical device (system) hazard analysis.

OTS Software failure, malfunction, or misuse may present a hazard to the patient, operators, or bystanders. Figure 2-1 (next page) summarizes the typical hazard management and mitigation process which would include a hazard analysis of the OTS software component.

The submission should include the following information to document the OTS software hazard analysis:

l A list of all potential hazards identified. l The estimated severity of each identified hazard. l A list of all potential causes of each identified hazard.

Note: A tabular format of the OTS Software hazard analysis or a tabular summary will facilitate review. The hazard analysis for OTS Software may be included in the overall device hazard analysis provided adequate documentation is provided.

If the device with the OTS Software represents a Minor Level of Concern, then the Level of Concern for the OTS Software can be no greater. The hazard analysis for the OTS Software in such a device may simply document the Minor Level of Concern of the device.

Where failure, malfunction, or misuse of the OTS Software poses no possibility of injury to the patient, operators, or bystanders, then the OTS Software is said to present a Minor Level of Concern, and the fulfillment of the BASIC DOCUMENTATION (see section 2.1) will be considered sufficient.

2.3 OTS Software Hazard Mitigation

Hazard mitigation activities may seek to reduce the severity of the hazard, the likelihood of the occurrence, or both. Hazard mitigation interventions may be considered in three categories with the



following order of precedence:

l Design (or redesign) l Protective measures (passive measures) l Warning the user (labeling)

Figure 2-1. Typical Hazard Analysis and Mitigation

These approaches may involve hardware and/or software. These three mitigation approaches are by no means mutually exclusive and may be used concurrently. The most desirable approach is to design in effective controls, i.e., eliminate the need for a hazardous operation or component. Protective measures are considered passive (from the user’s standpoint) since they do not require any action on the part of the user. The least effective approaches depend on some action (or lack of action) on the part of the medical device user.

The submission should include the following information to document the OTS Software hazard mitigation:



1. A list of all identified medical device hazards associated with the OTS Software 2. The steps taken to mitigate each hazard 3. The residual risk

Note: A tabular format of the risk management or a tabular summary will facilitate review. These results will typically be included as a part of the overall medical device Hazard Analysis and Mitigation plan.

One example of a comprehensive approach to injury prevention in public health was developed around ten "countermeasures" [2]. Table 2-1 (see next page) illustrates a generic approach to the hazard mitigation, in this case, to preventing injury-related energy release to patients, operators, or bystanders.

With implementation of each hazard mitigation, the residual risk is assessed as well as assessment of any new hazards which may be introduced.

Acceptable levels of residual risk, based on the severity or the likelihood of the residual risk occurring, will depend on the intended use of the medical device and the function performed by the software. In the case of diagnostic tests, injury includes results which can lead to unnecessary invasive diagnostic testing (e.g., biopsy) or withholding or delaying important diagnostic or therapeutic procedures.

The sponsor will need to describe and justify the residual risk (section 2.4) for Moderate or Major Levels of Concern. Where failure, malfunction, or misuse of the OTS Software is likely to result in death or serious injury to the patient, operators, or bystanders, then the OTS Software is said to present a Major Level of Concern. If the residual risk from the OTS Software presents a Major Level of Concern, the sponsor will need to fulfill SPECIAL DOCUMENTATION (see Section 2.5).

Table 2-1. Injury Reduction Countermeasures

1. Prevent accumulation of the energy.

2. Reduce the amount of the energy delivered.

3. Prevent inappropriate release of the energy.

4. Modify the release of the energy.

5. Separate the patient from the energy in time and space.

6. Provide physical barriers between the energy and the patient.

7. Change the surfaces or basic structures at the interface.

8. Reduce likelihood of misapplication or Increase resistance of the patient.

9. Provide rapid emergency response to injury.

10. Improve medical care and rehabilitation after the injury.



2.4 Describe and Justify Residual Risk

The sponsor should provide a detailed (complete) discussion of the risk which remains.

The risk related to the use of OTS Software should be considered in relation to the risk of the alternatives, e.g., custom developed software. Any experience (data) with the use of the OTS Software in this or a related application should be presented by the sponsor and will be considered by the reviewers. Whether the residual risk is acceptable depends on the specific medical device application.

2.5 SPECIAL DOCUMENTATION for OTS Software

To fulfill SPECIAL DOCUMENTATION for OTS Software of a Major Level of Concern, the medical device manufacturer is expected to:

1. Provide assurance to FDA that the product development methodologies used by the OTS Software developer are appropriate and sufficient for the intended use of the OTS Software within the specific medical device. FDA recommends this include an audit of the OTS Software developer’s design and development methodologies used in the construction of the OTS Software. This audit should thoroughly assess the development and qualification documentation generated for the OTS Software. (See note 2.5.1)

Note: If such an audit is not possible and after hazard mitigation, the OTS Software still represents a Major Level of Concern, the use of such OTS Software may not be appropriate for the intended medical device application.

2. Demonstrate that the procedures and results of the verification and validation activities performed for the OTS Software are appropriate and sufficient for the safety and effectiveness requirements of the medical device. Verification and validation activities include not only those performed by the OTS Software developer, but also include those performed by the medical device manufacturer when qualifying the OTS Software for its use in the specific medical device.

3. Demonstrate the existence of appropriate mechanisms for assuring the continued maintenance and support of the OTS Software should the original OTS Software developer terminate their support.

3 OTS Software Used in Marketing Applications 3.1 Examples

Examples of medical devices using OTS software are described in this section. These examples illustrate the reasoning which leads to defining the Level of Concern for a medical device and thus the kinds of development processes which should be used and the information to be provided in a regulatory submission.

3.1.1 Corneal Topographer

—Minor Level of Concern medical device (see Section 2.1)

Intended Use: A corneal topographer provides images of the abnormalities in the curvature



of the cornea, the simplest being astigmatism.

Description: A corneal topographer consists of a hollow cone which the patient looks into from the base looking towards the interior of the point (like looking into the big end of a megaphone with one eye). The inside of the cone is white with black concentric circles. The concentric circles reflect off the eye and are imaged by a camera with a computer controlled lens situated at the point of the cone looking at the patient’s eye. The shapes of the reflections of the concentric circles are used to develop a topographic map of the cornea curvature which is printed out.

OTS Software: An OTS operating system such as Windows is commonly used to interface the user, the microcomputer hardware platform, the corneal topographer, data storage, and output devices.

OTS Software Level of Concern: A corneal topographer represents no threat of direct harm to the patient. The risk of indirect harm from a misdiagnosis relating to medical device malfunction is small since the worst case is an incorrect image which is considered correct. The OTS Software in this medical device thus represents a Minor Level of Concern (see section 2.2) and should satisfy BASIC DOCUMENTATION (see section 2.1).

3.1.2 Perineometer

—Minor Level of Concern medical device (see Section 2.1)

Intended Use: Perineometers are used to provide feedback to a patient performing muscle strengthening exercises (Kegel exercises) for the treatment of certain types of urinary incontinence.

Description: There are two types of perineometers: those which measure pressure, and those which measure electrical activity (EMG) from muscles. Each device consists of a probe that is placed into either the vagina or the rectum, and a monitoring unit. The pressure devices use an air-filled probe connected to the monitoring unit by a piece of plastic tubing. When the patient performs the exercise, the probe is compressed, and the monitoring unit reports the change in pressure. The electrical devices use an electrode to measure the electrical activity of the target muscles during the exercises, and this information is reported by the monitoring unit.

OTS Software: An OTS operating system, such as DOS or Windows, may be used to record and display the data collected by the monitoring unit.

OTS Software Level of Concern: Perineometers represent no threat of direct injury to the patient, since no energy is applied by the medical device to the patient. The risk of indirect injury due to inaccurate feedback during the exercise session is expected to be small, as these medical devices are only used as an adjunct to exercise therapy, and they are used under clinical supervision. The OTS Software in this medical device thus represents a Minor Level of Concern (see section 2.2) and should satisfy the BASIC DOCUMENTATION (see Section 2.1).

3.1.3 Implantable Medical Device Programmers



—Describe and Justify Residual Risk (see Section 2.5)

Intended Use: An implantable medical device programmer provides interface and two-way communication with an implantable cardioverter-defibrillator (ICD) or cardiac pacemaker.

Description: An implantable medical device programmer consists of an electromagnetic programming head which is placed over the implanted device and provides through-the-skin communication with the implanted device, the personal computer (PC) interface, and the PC hardware and software. The programmer permits the physician-user to:

l query the implant for performance history (device and patient), and, in some systems, for print-out of the recorded electrograms;

l set the adjustable (programmable) characteristics of the implant; l provide the induced shock for system initialization and diagnostic purposes; and l verify implant operating characteristics and status (including battery) via signals from

the implant.

OTS Software: An OTS operating system such as DOS or Windows is used to provide a user interface (sometimes graphical), interface to the PC (hardware platform), and interface with data storage, and output devices.

OTS Software Level of Concern: The on-board software for the implant satisfies the definition of Major Level of Concern software (life supporting/life sustaining) and would need to satisfy the SPECIAL DOCUMENTATION (see Section 2.4). Whether the device programmer can be considered of lesser Level of Concern depends primarily on the protection designed into the implant or the programmer. Steps taken to mitigate the risk might include:

l design of the implant to minimize the possibility of misprogramming to inappropriate operational states;

l design of the programmer interface to minimize the chance of miscommunication including hardening of the hardware against electromagnetic interference (EMI);

l limiting the part of the OTS Software which is utilized in the programming application;

l protecting the PC from use for other applications, including consideration of the following:

l Software design features to protect against adding unwanted software, modification or system use; and

l Hardware design features to protect against unwanted system use.

Other points which might be offered to support use of OTS Software in the programmer might include:

1. documented experience (data) with use of the OTS Software in this application

l What was the system in place to detect and report problems? l What is the rate of problems reported compared to other (perhaps non-OTS Software)

systems?

2. documented experience with the OTS Software in other relevant applications



l What are the reported problems (bug list) and how many are relevant to this application?

l Has there been difficulty in developing work-arounds for the problems relevant to this application?

The review team must decide whether the overall programmer system as implemented satisfies the necessary system safety and effectiveness (see section 2.5).

3.2 510(k) Issues with OTS Software

The conditions under which a new or changed medical device including OTS Software will require a new 510(k) are the same as for a device not involving OTS Software. These conditions are given in CDRH’s guidance Deciding When to Submit a 510(k) for a Change to an Existing Device [3]. The section (B) on Technology Engineering and Performance Changes in the 510(k) guidance is most applicable to OTS Software.

Section B of the guidance includes the following questions:

l B1 Is it (the modification) a control mechanism change? l B2 Is it an operating principle change? l B5 Is it a change in performance specifications? l B8 Is it a change in software or firmware? The types of changes identified in questions B4

through B8 have frequently been called design changes or engineering changes. They encompass everything from the routine specification changes necessary to maintain or improve medical device performance as a result of feedback from users, field or plant personnel, etc., up to and including significant product redesign.

l B8.1 Does the change affect the indications for use? As with an explicit labeling change, if the change affects the indications for use, i.e., if it creates an implied new indication for use, a new 510(k) should be submitted.

l B8.2 Are clinical data necessary to evaluate safety and effectiveness for purposes of determining substantial equivalence? Whenever a manufacturer recognizes that clinical data are needed because bench testing or simulations are not sufficient to assess safety and effectiveness and, thus, to establish the substantial equivalence of a new design, a 510(k) should be submitted.

l B8.3 Do results of design validation raise new issues of safety and effectiveness? All changes to medical device design will require some level of design validation or evaluation to assure that the device continues to perform as intended. The successful application of routine design validation activities will logically result in manufacturers documenting their efforts and proceeding with the design change, i.e., assuring that no issues of safety or effectiveness are raised.

A yes answer to any of these questions in section B will generally require a new 510(k).

3.2.1 OTS Software Changes Requiring a 510(k)

For medical devices where the OTS Software represents a Minor Level of Concern, OTS Software changes would not typically require a new 510(k). However, the manufacturer is responsible for validating the change.

For other medical devices, the decision as to whether a new 510(k) is required depends on the intended



use of the device; the function of the OTS Software; and to what extent the risks due to OTS Software have been mitigated (see guidance on when to submit a 510(k) [3]).

3.2.2 Exemption of Laboratory Information Management Systems

Laboratory information management systems (LIMS) are Class I devices (21 CFR 862.2100, Calculator/Data Processing Module for Clinical Use). They are included in the category of electronic medical devices intended to store, retrieve, and process laboratory data. LIMS may also handle scheduling, billing and other non-device functions. LIMS have been exempted from 510(k) since June 8, 1988. However, compliance with all other requirements is required, including registration, listing, GMP, and MDR.

The LIMS exemption does not apply to applications of artificial intelligence or other algorithms intended to assign a probability of diagnosis for the purpose of guiding therapy or further diagnostic studies.

Such clinical data management functions may be subject to FDA regulations as are blood establishment software systems.

3.3 IDE Issues with OTS Software

The requirements for an IDE are the same whether or not the medical device contains OTS Software. The OTS Software may be a component of a medical device or the OTS Software may be the entire medical device, e.g., diagnostic software. The conditions which would require submission of an IDE are specified in 21 CFR 812 and generally include changes that would affect the patient population for which the medical device is intended; conditions of use of the device (including those recommended or suggested in the labeling or advertising; the probable benefit from the use of the device weighed against any probable injury or illness from such use); or the reliability of the medical device.

Some specific issues related to OTS Software might include initial (beta) testing of an OTS Software medical device in clinical studies. Such a study must comply with applicable IDE requirements. For non-significant risk medical devices, that includes approval by an institutional review board and patient informed consent. For significant risk studies, the initial user testing (beta testing) protocol would be included in an IDE submission to ODE. For example, beta testing of radiation treatment planning software, including any OTS Software modules, would be conducted under a full IDE with FDA approval as a prerequisite.

3.4 Exemption of Certain Diagnostic Devices

If the product incorporating the OTS Software is a diagnostic medical device, it may be exempted from IDE requirements, if it meets the criteria in section 21 CFR 812.2 (c) (3). For example, clinical (beta) testing of a noninvasive diagnostic device that does not require significant risk invasive sampling procedure and that does not introduce energy into the body, is exempted from IRB approval, patient informed consent, and other IDE requirements, if a medically established diagnostic product or procedure is used to confirm the diagnosis.

3.5 PMA Issues with OTS Software

The criteria and requirements for premarket approval applications are in 21 CFR 814. When a



manufacturer submits a premarket approval submission for a medical device, there must be valid scientific evidence (including clinical evidence, if needed) to support a reasonable assurance of safety and effectiveness of the device.

The OTS Software used in a medical device is evaluated in the context of the overall medical device. The extent to which the medical device manufacturer must ensure that the OTS software was developed using appropriate life cycle control depends upon the overall risk of the medical device, the role of the OTS Software, and the Level of Concern associated with possible failures of the OTS Software component.

For example, a commercially available neural network, used by a medical device manufacturer for pattern recognition, would require extensive validation if used in a Pap smear screening device, in computer-assisted radiology, or for computer-assisted analysis of ECG waveforms. The same neural network, used for less critical computer-assisted analysis of EEG waveforms, might require less rigorous software documentation. Likewise, a commercially available personal computer operating system with graphical user interface, would require extensive documentation and evidence of validation when intended for use in a cardiac pacemaker programmer. Less documentation and verification of the OTS operating system would be required for programming an artificial ear.

3.6 Artificial Intelligence

OTS knowledge-based software (for example, artificial intelligence, expert systems, and neural net software) are being developed for a number of medical applications. A typical system accepts clinical findings (sometimes including imaging data) and generates probabilities of disease states and/or recommendations for subsequent data gathering or treatment. The clinician may order a surgical biopsy or other invasive tests or initiate therapy based on the system output. Such systems should be tested and reviewed in a manner consistent with both their safety and effectiveness of their direct effects (recommendations) and indirect effects (missed appropriate diagnostic testing and treatment).

3.7 Product Labeling

FDA recommends that the user’s manual specify the version(s) of the OTS Software that can be used with the medical device. Such specification would not be required for embedded software (i.e., the user does not select the OTS Software and cannot change the software provided by the medical device manufacturer).

The user’s manual should contain appropriate warnings to the user indicating that the use of any software other than those specified will violate the safety, effectiveness and design controls of this medical device and that such use may result in an increased risk to users and patients. Further description of what comprises a warning and how to write it are included in Medical Device Labeling—Suggested Format and Content [5]

When OTS medical device software is delivered on a magnetic/ user installable medium, the package should include labeling that indicates the minimum hardware platforms on which the software is validated to run (processor, memory, disk, interface etc.). The appropriate testing for the user to assure proper installation should also be described in the labeling.

If the hardware on which the OTS Software runs is a stand-alone computer and the user is not "locked out" by hardware or software system features, then the user should be warned against installing any



other software (utilities or applications programs) on the computer.

4 Bibliography 1. Levesen NG: Safeware – System Safety and Computers. Addison-Wesley, New York, 1995, 680 pages. Abs: A good discussion of the problem area by a recognized expert on software safety.

2. Haddon W, Baker SP: Injury protocol. in Duncan, Clark Brain, MacMahon (eds): Preventive Medicine, New York, Little, Brown, 1979. Abs: A readable discussion of basic injury reduction strategies from some of the most experienced in the field.

3. USPHS DHHS FDA CDRH: Deciding When to Submit a 510(k) for a Change to an Existing Device. 510(k) Memorandum #K97-1. January 10, 1997. Abs: CDRH guidance that discusses how to decide when a change to an existing 510(k) requires a new 510(k) submission. Text version is available on the FDA home page at http://www.FDA.GOV/cdrh/ode/510kmod.html .

4. USPHS DHHS FDA CDRH: ODE Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices. May 29, 1998. Abs: This document provides the current guidance in the review of software which comprises part of (or all of) a medical device. Available on the FDA Home Page at http://www.fda.gov/cdrh/ode/software.pdf

5. USPHS DHHS FDA CDRH: Medical Device Labeling—Suggested Format and Content. DRAFT Version 4.2, copies of this work-in-progress are available as of March 4, 1997. Abs: This document provides the current guidance on the policy, format and content of the labeling of medical devices.

6. USPHS DHHS FDA ORA: Glossary of Computerized System and Software Development Terminology. Abs: This document provides a glossary of commonly used computer and software terms.

5 Appendices The purpose of these appendices is to provide background and comment on various OTS software. Based on the Level of Concern, device manufacturers should either use or not use Commercial Off-the-shelf Software (COTS).

5.1 Operating Systems

The operating system software is the primary software program which manages the basic functions of the computer and its associated hardware, including peripherals. The operating system provides a basic user interface, is responsible for managing applications programs and tasks, controlling memory allocation and data storage devices, and providing input/output for the computer as well as any additional peripheral devices which are present.

"Open" hardware (mass market) architecture computers vary widely in architectural and organizational characteristics such as timing, addressing, and processing. Operating systems and application software executing on these platforms should be "robust" enough to perform appropriately in this environment.

OTS driver software packages provide interface functions between the CPU, operating system, and the input/output peripheral. However, the performance and functionality of the OTS driver software may be



affected by the overall system configuration and the OTS hardware. In general, OTS driver software packages can be classified into the following input/output interface types: serial, parallel, video signal, telemetry, LAN, and internal bus. In most cases, a particular software driver derives from a particular interface protocol and contains the data signals, control signals, and timing signals for proper operation.

Since tests for most input/output interface/bus configurations require the particular bus analysis or logic analysis, scope, and knowledge of the particular interface protocol, the validation process for the OTS driver software package should be part of the system interface validation process for higher levels of concern. This includes the verification of the data values in both directions for the data signals; various mode settings for the control signals in both directions (if applicable); and the input/output interrupt and timing functions of the driver with the CPU and operating system.

5.2 Utilities and Drivers

The purpose of this appendix is to provide general recommendations and background for the use of OTS utility and driver software packages in the medical device validation process.

Utility software is generally designed to work with a specific operating system. Unlike applications software, utility software is intended to supplant or enhance functions typically performed by the operating system. Examples of utility programs are memory managers, file managers, and virus checkers. Networking software can also be considered as utility software in that it allows multiple computers to access the same resources. Operating systems can also be designed to support or enable network operations without any additional utility software.

Off-the-shelf operating systems are commonly considered for incorporation into medical devices as the use of general purpose computer hardware becomes more prevalent. The use of OTS operating system software allows device manufacturers to concentrate on the application software needed to run device-specific functions. However, an OTS operating system software is intended for general purpose computing and may not be appropriate for a given specific use in a medical device. Developers of OTS operating systems typically design their systems for general purpose business or consumer computing environments and tasks where software failures and errors are more accepted. This acceptability of errors in the general purpose computing environment may make the OTS operating system software inappropriate for less error-tolerant environments or applications.

The incorporation of OTS operating system software may also introduce unnecessary functions and complexity into a medical device. General purpose functional requirements typically result in the OTS operating system software being large and unwieldy in the attempt to incorporate more functionality into the operating system. This excess functionality is typically never used for specific medical device applications and increases the likelihood that errors may be introduced into the operating system. The basic functions of an OTS operating systems used for medical device applications are typically the graphical user interface environment and the hardware interface functions. There are a number of operating systems used for timing- or resource-critical applications that provide the basic functionality needed to support user and hardware interfaces, but do not have many of the disadvantages of general purpose business or consumer operating systems.

OTS utility software packages can perform the following functions: math functions (fast Fourier transform, sin, cos); display functions (graphic); management functions (copy, delete, store various computer data/files); and the data manipulation function (transfer from one Boolean type or both. The validation for these types of the software should be appropriate to the Level of Concern.



5.3 Local Area Networks (LANs)

The purpose of this appendix is to provide general recommendations and background for the network aspects of OTS Software use. Medical devices, particularly multi-parameter patient monitors and imaging systems, are increasingly networked for clinical work groups, centralized monitoring, and storage of patient medical data and records. LANs and other networks support more and more communication and sharing of images, measurement data, audio, video, graphics, text, etc. This heterogeneous media environment comes at a cost of more processing power, higher bandwidth or network speed, sophisticated object-relational databases, and security and access considerations.

The evaluation of networked medical devices begins with a definition of the technical requirements of the network application and the understanding of those requirements.

5.3.1 Requirements Analysis

1. Speed - The response time required for safe and effective operation determines the LAN data rate (bandwidth) for the medical device system. The CPU processing power and clock speed required at device monitors, workstations, and client machines should be appropriate so that bottlenecks do not occur.

2. LAN Architecture - The size of the LAN (the number of user nodes) and the topology of the LAN should be specified.

l Discuss to what extent the LAN needs to be fault tolerant, e.g., when a workstation fails?

l Discuss to what extent the LAN needs to be scaleable, i.e., can new user nodes be added without degrading system performance?

l Discuss to what extent the main device software needs to be computationally self-sufficient or distributed?

3. Network Operating System (NOS). Whether off-the-shelf or proprietary, this selection should consider the trade-off between robustness and flexibility.

4. Data Integrity - One of the most important issues for any medical device operating in a network is data integrity. The manufacturer should insure that the network system software and hardware incorporate error checking, handling, and correction measures commensurate with the level of concern of the device.

Transmission of data packets and files should include error detection and correction. Error detection methods include parity, checksum, and cyclic redundancy check (CRC).

Transaction rollback after non-committed changes or network failure, supports data integrity in medical device LANs.

Critical data and files may be stored in duplicate at separate locations.

5. Network Management and Security - User authorization and authentication should precede accesses to sensitive patient information.

The above five items are not independent. Decisions made in one item area may affect the performance of the LAN in another area.



5.3.2 Implementation

The speed required by the medical device system dictates the hardware selection, the network interface cards and transmissions protocols. For example, if the conventional Ethernet protocol (maximum transmission speed of 10 Mbps) is too slow for the intended application, then a different transmission protocol will be needed.

Simplicity of the LAN architecture versus fault tolerance is a trade-off that may arise in the implementation of the networked medical device systems. The LAN could be implemented as a linear bus network (perhaps the simplest scheme), but if any connecting link on the bus fails the whole network can fail. A star topology with redundant centralized hub is an example of a more complex but more robust network structure.

Segmentation of high bandwidth applications may be employed to improve LAN performance. Limiting the data traffic to data intensive clusters reduces traffic throughout the overall LAN.

5.4 Device Master Files

Much of the information regarding development and validation of OTS Software may not be readily available to the medical device manufacturer who wishes to use the OTS Software as a device component. Commercial OTS Software vendors who wish to make their OTS Software available for use in medical devices, but do not want to share the confidential and/or proprietary details of their software development and validation with customers (medical device manufacturers) may direct the information in a device master file to the FDA.

The master file should contain information regarding the OTS Software development, validation and known software bugs in support of use of the software by medical device manufacturers. The intended level of risk of potential device applications should guide the OTS vendor in deciding what level of detail to provide in the master file.

The OTS Software vendor should also consider which types of device applications may or may not be appropriate uses of the OTS Software as a component. The vendor can then grant permission to specific device manufacturers to reference the master file in their premarket submissions. Information regarding device master files is contained in DSMA’s "Premarket Approval (PMA) Manual", or via Facts-on-Demand or from the FDA home page (http://www.fda.gov/cdrh/dsma/pmaman/front.html)

5.5 Maintenance and Obsolescence

This appendix addresses relevant maintainability issues with regard to OTS Sotware in medical devices.

Maintenance activities are generally considered to begin subsequent to the establishment and distribution of a medical device product baseline. The distinction between maintenance and product development is an important one. Product development design activities generally lead to a system structure of highly integrated components and logic. Maintenance activities introduce changes into this structure which may lead to a loss in the integrity of the structure. Structure integrity may be affected through changes due to new design requirements, corrections, or environmental adaptations. These types of changes may impact the integrity of the structure organization, architecture, logic, integration, or any combination of these characteristics. Maintenance of products with OTS Software components may be particularly problematic for reasons discussed in the main body of this document, i.e., the sponsor does



not have control of the OTS Software component life cycle process.

In particular, this section identifies general safety and effectiveness, design, verification / validation, change, installation, and decommissioning concerns. These concerns may be applied to all regulated medical device software and stand-alone medical software devices. The appropriate evaluation will depend on the Level of Concern.

Assumptions for this section include:

l Manufacturer Good Software Development Practices (GSDP)s and Good Corrective Action Practices (GCAP) are in place.

l A product baseline exists. l A new product baseline based on a prior product baseline is under CDRH review.

Each concern below corresponds to a product development life cycle phase. The concerns identify fundamental maintenance concerns relevant to all regulated PEMS and stand-alone medical software devices. Guidance in the main body of this document provides the procedural foundation for concerns in this section.

5.5.1 Safety

Introduction of new or modified OTS components to a product baseline may impact the safety of the product. Therefore a safety impact assessment of the medical device must be performed and associated hazards documented in a Failure Modes and Effects Analysis (FMEA) table. Each hazard’s consequence should be provided and expressed qualitatively; e.g. major, moderate, or minor. Traceability between these identified hazards, their design requirements, and test reports must be provided.

Analysis should include the review of release bulletins (known error reports), user manuals, specifications, patches, literature and internet searches for other user’s experience with this OTS Software.

The submission should answer the following questions:

l has a FMEA with traceability to requirements and test reports been provided? l are safety functions isolated from new OTS component(s)? l does the new OTS component affect system safety integrity? l what new human factors conditions are introduced with new OTS components?

5.5.2 Design

Introduction of new or modified OTS Software components to a product baseline may impact the original design of the product. This impact may result from necessary changes to the product structure organization, architecture, logic, integration, or combination of these characteristics.

Problems attributable to structural changes include:

l new system resource requirements, such as shared and/or fixed memory l new timing considerations l new memory organization (e.g., 16 bit to 32 bit to 64 bit words), partitioning l new human factor issues



l new data integrity issues l new software required to create the final code (build tools)

Consequently the submission should answer the following questions:

l How will the new OTS Software component(s) change the performance characteristics? l How will the new OTS Software component(s) change the operational environment? l Is data integrity preserved?

5.5.3 Verification and Validation

As in the establishment of a product baseline, verification and validation (V&V) activities should occur when maintenance changes are made to a product baseline. Analysis of these changes directs necessary V&V activities. New OTS Software components in a product baseline introduce unknown logic paths and complexities into the product. "Black-box" testing of OTS Software components may allow some validation claims to be made. However, the unknown logic paths and complexities of OTS Software components make it important to know that design structure or logic elsewhere in the system is not impacted. This means a full system regression test should be performed. Results of these validation activities should be documented.


l Do test reports provide objective evidence that identified OTS Software component hazards have been addressed?

l Do test reports provide objective evidence that all identified SYSTEM hazards have been addressed?

l Has a system regression test been performed?

5.5.4 Installation

Changes in a product baseline structure resulting from the integration of new OTS Software components may impact installation requirements. This impact can range from minor documentation changes to field upgrades. The reviewer should ascertain the impact of OTS Software component changes on fielded products.

The submission should answer the following question: What is the impact of new OTS Software components on fielded medical device products?

For example: Do new OTS Software components correctly operate within the specifications of medical devices currently fielded?

5.5.5 Obsolescence

Rapid technology changes, economics, and market demand are shrinking product life spans. A direct consequence of these phenomena is that an OTS Software component today may not exist two years from now. Short life spans are a particular characteristic of software because it is relatively easy to change. Obsolescence of OTS Software components can have significant impact on regulated products because the device manufacturer may lose the ability to properly support fielded products. The sponsor needs to support fielded medical device products with OTS Software components.




l Will the old OTS Software component still be available for fielded medical devices? l Is there a retirement plan for OTS Software components to be replaced/eliminated? l Do new OTS Software component(s) replace fielded components?

5.5.6 Change control

The submission must identify the product to be considered. Therefore, the product configuration provided should specify:

l hardware platform (e.g. microprocessor, minimum memory required, addressable word size) l software platform (e.g. operating system, communications, database’s, necessary utilities, etc.) l OTS component(s) other than (b) above (see basic requirements in the main body of this

document) l internally developed application(s)

Uploaded on October 4, 1999 (FOD number changed to 585 on 2/13/02)



Guidance for Industry Cybersecurity for Networked

Medical Devices Containing Off-the-Shelf (OTS) Software

Document issued on: January 14, 2005

For questions regarding this document contact John F. Murray Jr. 240-276-0284, [email protected]. or David S. Buckles 301-443-8517 x174, [email protected]

U.S. Department of Health and Human Services Food and Drug Administration

Center for Devices and Radiological Health Office of Compliance

Office of Device Evaluation

mailto:[email protected]

mailto:[email protected]


Preface

Public Comment Written comments and suggestions may be submitted at any time for Agency consideration to the Division of Dockets Management, Food and Drug Administration, 5630 Fishers Lane, Room 1061, (HFA-305), Rockville, MD, 20852. When submitting comments, please refer to the exact title of this guidance document. Comments may not be acted upon by the Agency until the document is next revised or updated. Additional Copies Additional copies are available from the Internet at: http://www.fda.gov/cdrh/comp/guidance/1553.pdf. To receive this document by fax, call the CDRH Facts-On-Demand system at 800-899-0381 or 301-827-0111 from a touch-tone telephone. Press 1 to enter the system. At the second voice prompt, press 1 to order a document. Enter the document number (1553) followed by the pound sign (#). Follow the remaining voice prompts to complete your request.

http://www.fda.gov/cdrh/comp/guidance/1553.pdf


Guidance for Industry

Cybersecurity for Networked Medical

Devices Containing Off-the-Shelf (OTS) Software

This guidance represents the Food and Drug Administration's (FDA's) current thinking on this topic. It does not create or confer any rights for or on any person and does not operate to bind FDA or the public. You can use an alternative approach if the approach satisfies the requirements of the applicable statutes and regulations. If you want to discuss an alternative approach, contact the FDA staff responsible for implementing this guidance. If you cannot identify the appropriate FDA staff, call the appropriate number listed on the title page of this guidance.

Introduction A growing number of medical devices are designed to be connected to computer networks. Many of these networked medical devices incorporate off-the-shelf software that is vulnerable to cybersecurity threats such as viruses and worms. These vulnerabilities may represent a risk to the safe and effective operation of networked medical devices and typically require an ongoing maintenance effort throughout the product life cycle to assure an adequate degree of protection. FDA is issuing this guidance to clarify how existing regulations, including the Quality System (QS) Regulation, apply to such cybersecurity maintenance activities. FDA's guidance documents, including this guidance, do not establish legally enforceable responsibilities. Instead, guidances describe the Agency's current thinking on a topic and should be viewed only as recommendations, unless specific regulatory or statutory requirements are cited. The use of the word should in Agency guidances means that something is suggested or recommended, but not required. The Least Burdensome Approach We believe we should consider the least burdensome approach in all areas of medical device regulation. This guidance reflects our careful review of the relevant scientific and legal requirements and what we believe is the least burdensome way for you to comply with those

1


requirements. However, if you believe that an alternative approach would be less burdensome, please contact us so we can consider your point of view. You may send your written comments to the contact persons listed on the coversheet to this guidance or to the CDRH Ombudsman. Comprehensive information on CDRH's Ombudsman, including ways to contact him, can be found on the Internet at http://www.fda.gov/cdrh/ombudsman/. Background This guidance outlines general principles that we consider to be applicable to software maintenance actions required to address cybersecurity vulnerabilities for networked medical devices—specifically, those that incorporate off-the-shelf (OTS) software. The guidance is organized in question-and-answer format, providing responses to questions that have frequently been posed to FDA staff. The “I” in the questions and the “you” in the answers are intended to apply to device manufacturers who incorporate OTS software in their medical devices.

The QS regulation, 21 CFR Part 820, applies to software maintenance actions. In addition, FDA has issued several guidance documents on software, including:

General Principles of Software Validation; Final Guidance for Industry and FDA Staff, January 11, 2002, http://www.fda.gov/cdrh/comp/guidance/938.html.

Guidance for Industry, FDA Reviewers and Compliance on Off-the-Shelf Software Use in Medical Devices, September 9, 1999, http://www.fda.gov/cdrh/ode/guidance/585.html.

Guidance for FDA Reviewers and Industry, Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices, May 29, 1998, http://www.fda.gov/cdrh/ode/57.html.

2

http://www.fda.gov/cdrh/ombudsman/

http://www.fda.gov/cdrh/comp/guidance/938.html

http://www.fda.gov/cdrh/ode/guidance/585.html



Questions and Answers 1. Which medical devices are covered by this guidance?

This guidance provides recommendations for medical devices that incorporate off-the-shelf (OTS) software and that can be connected to a private intranet or the public Internet. This guidance is addressed to device manufacturers who incorporate OTS software in their medical devices. However, this information also may be useful to network administrators in health care organizations and information technology vendors. 2. What is a cybersecurity vulnerability?

For purposes of this guidance, a cybersecurity vulnerability exists whenever the OTS software provides the opportunity for unauthorized access to the network or the medical device. Cybersecurity vulnerabilities open the door to unwanted software changes that may have an effect on the safety and effectiveness of the medical device. 3. What is it about “network-connected medical devices” that causes so much

concern? Vulnerabilities in cybersecurity may represent a risk to the safe and effective operation of networked medical devices using OTS software. Failure to properly address these vulnerabilities could result in an adverse effect on public health. A major concern with OTS software is the need for timely software patches to correct newly discovered vulnerabilities in the software. 4. Who is responsible for ensuring the safety and effectiveness of medical devices

that incorporate OTS software?

You (the device manufacturer who uses OTS software in your medical device) bear the responsibility for the continued safe and effective performance of the medical device, including the performance of OTS software that is part of the device.1 5. How should purchasers and users of these medical devices respond to information

about a cybersecurity vulnerability? FDA recommends that purchasers and users of medical devices that may be subject to a cybersecurity vulnerability contact you with their concerns. As Question 4 explains, you are responsible for the performance of OTS software that is part of your device. Although there may be times when it is appropriate for the user to become involved (see Question 9 below), the user should not attempt to make changes without seeking your advice and recommendations.

1 For more information, you should refer to “Guidance for Industry, FDA Reviewers and Compliance on Off-The-Shelf Software Use in Medical Devices,” Sept. 9, 1999 (see Background section of this guidance).

3


•

•

6. What regulations apply to software patches that address cybersecurity

vulnerabilities?

The need to be vigilant and responsive to cybersecurity vulnerabilities is part of your obligation under 21 CFR 820.100 to systematically analyze sources of information and implement actions needed to correct and prevent problems. The preamble to the QS regulation explains that actions taken should “be appropriate to the magnitude of the problem and commensurate with the risks encountered” (61 Fed. Reg. 52633; Oct. 7, 1996). Information in this guidance will remind you of some of the actions that ordinarily will be necessary to address this particular type of software concern. Under 21 CFR 820.30(g), design validation requires that devices conform to defined user needs and intended uses, including an obligation to perform software validation and risk analysis, where appropriate. Software changes to address cybersecurity vulnerabilities are design changes and must be validated before approval and issuance. 21 CFR 820.30(i). 7. Is FDA premarket review required prior to implementation of a software patch

to address a cybersecurity vulnerability?

Usually not. In general, FDA review is necessary when a change or modification could significantly affect the safety or effectiveness of the medical device. 21 CFR 807.81(a)(3), 814.39. a. 510(k). For medical devices cleared for market under the 510(k) program, you may refer to our guidance entitled, “Deciding When to Submit a 510(k) for a Change to an Existing Device.” 2 That guidance explains that a new 510(k) submission to FDA is necessary for a change or modification to an existing medical device if:

The medical device has a new or changed indication for use (e.g., the diseases or conditions the medical device is intended to treat); or The proposed change (e.g, modification in design, energy source, chemical composition, or material) could significantly affect the safety or effectiveness of the medical device.

It is possible, but unlikely, that a software patch will need a new 510(k) submission.3 As with all changes made to devices, you should document the basis of your decisions in the design history file. See 21 CFR 820.3(e), 820.30(j).

2 “Deciding When to Submit a 510(k) for a Change to an Existing Device,” Jan. 10, 1997, http://www.fda.gov/cdrh/ode/510kmod.html. 3 If a submission is necessary, you should refer to “Guidance for FDA Reviewers and Industry, Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices,” May 29, 1998 (see Background section of this guidance).

4


5

b. Premarket Approval Application (PMA). For medical devices approved under PMAs (21 CFR Part 814), a PMA supplement is required for a software patch if the patch results in a change to the approved indications for use or is deemed by the manufacturer to have an adverse effect on the safety and effectiveness of the approved medical device. 21 CFR 814.39. Otherwise, you should report your decision to apply a software patch to your PMA device to FDA in your annual reports. See 21 CFR 814.39(b), 814.84. 8. Should I validate the software changes made to address cybersecurity

vulnerabilities?

Yes. See answer to Question 4. You should validate all software design changes, including computer software changes to address cybersecurity vulnerabilities, according to an established protocol before approval and issuance. 21 CFR 820.30(i). You may refer to the “General Principles of Software Validation; Final Guidance for Industry and FDA Staff” (see Background section) for more information about how to validate software changes. For most software changes intended to address cybersecurity vulnerabilities, analysis, inspection, and testing should be adequate and clinical validation should not be necessary.

9. What else should I do to ensure cybersecurity for networked medical devices? You should maintain formal business relationships with your OTS software vendors to ensure timely receipt of information concerning quality problems and recommended corrective and preventive actions. Because of the frequency of cybersecurity patches, we recommend that you develop a single cybersecurity maintenance plan to address compliance with the QS regulation and the issues discussed in this guidance document. While it is customary for the medical device manufacturer to perform these software maintenance activities, there may be situations in which it is appropriate for the user facility, OTS vendor, or a third party to be involved. Your software maintenance plan should provide a mechanism for you to exercise overall responsibility while delegating specific tasks to other parties. The vast majority of healthcare organizations will lack detailed design information and technical resources to assume primary maintenance responsibility for medical device software and, therefore, will rely on you to assume the primary maintenance responsibility.

10. Do I need to report a cybersecurity patch?

Not usually, because most software patches are installed to reduce the risk of developing a problem associated with a cybersecurity vulnerability and not to address a risk to health posed by the device. In most cases, therefore, you would not need to report a cybersecurity patch under 21 CFR Part 806 so long as you have evaluated the change and recorded the correction in your records. However, if the software patch affects the safety or effectiveness of the medical device, you should report the correction to FDA, even if a software maintenance plan is in effect.

Page 1 of 7

Copyright 2005 SoftwareCPR http://www.softwarecpr.com Nov 28, 2005

FDA CDRH ODE Software Submission Guidance Documents

SoftwareCPR Training Aide This training aide lists FDA software guidance documents related to pre-market submissions (IDEs, PMAs, 510(k)s). Although SoftwareCPR does not guarantee this to be a full list, it is believed to identify most, if not all, such guidances released as of the date cited above. These lists do not include other software guidance unrelated to submissions (e.g., Part 11 Validation Guidance, etc.) The software submissions guidances are divided into the following categories:

• Primary software submission guidances – these provide general requirements for software information in submissions, are not device-specific, and are referenced by many device-specific guidances.

• General submission guidances – guidances that do not specifically focus on software but contain information relevant to software submissions

• Device- specific submission guidances – these provide information on constructing submission for specific types of medical devices; ones listed here contain software guidance beyond a simple reference to a general software submission guidance. Note that if a device-specific guidance conflicts with a general software submission guidance the device-specific guidance takes precedence. Also note that some of these guidances specify the level of concern of the software to allow easier interpretation of the general submission guidance for the specific type of device involved.

The SoftwareCPR.com library at http://www.softwarecpr.com contains a description and highlight of each guidance and a link to the actual guidance as well. Primary Software Submission Guidances

Date Document Link (you may need to right click) 9/9/99 Guidance for Industry, FDA Reviewers and

Compliance on Off-the-Shelf Software Use in Medical Devices

http://www.softwarecpr.com/docs/OTSSswsubmissionguidance.PDF

5/11/05 Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices

http://www.softwarecpr.com/docs/FDASoftwareSubmissionGuidance.pdf

1/11/02 General Principles of Software Validation (GPSV) Note: This guidance is for compliance with the software validation requirements of the Quality System Regulation and not for submissions purposes. It is included on this list since some of its explanatory information on software validation can be helpful in understanding the submission terminology and requirements.

http://www.softwarecpr.com/docs/GPSVfinal.pdf

General Submission Guidances

Date Document Link 5/15/02 Screening Checklist For All Premarket

Notification [510(k)] Submissions

http://www.softwarecpr.com/docs/checklist-f102.pdf

1/31/02 FDA Issues Checklist for Standards Conformance

http://www.softwarecpr.com/docs/reqrecstand.pdf

1/21/02 FDA ODE 510(k) Checklist http://www.softwarecpr.com/docs/510kscreeningchecklist-f102.pdf

http://www.softwarecpr.com


http://www.softwarecpr.com/docs/OTSSswsubmissionguidance.PDF

http://www.softwarecpr.com/docs/FDASoftwareSubmissionGuidance.pdf

http://www.softwarecpr.com/docs/GPSVfinal.pdf

http://www.softwarecpr.com/docs/checklist-f102.pdf

http://www.softwarecpr.com/docs/reqrecstand.pdf

http://www.softwarecpr.com/docs/510kscreeningchecklist-f102.pdf

Page 2 of 7


Shortcut to ActiveWebSite.lnk

3/12/00

Use of Standards in Substantial Equivalence Determinations

http://www.softwarecpr.com/docs/510kuseOfStdsguidance.pdf

10/22/98 Guidance for Industry: Frequently Asked Questions on the new 510(k) Paradigm

http://www.softwarecpr.com/docs/510kQ&A.pdf

8/6/98 Modifications To Devices Subject to Premarket Approval - The PMA Supplement Decision Making Process

http://www.softwarecpr.com/docs/PMAmodifications.pdf

3/20/98 The New 510(k) Paradigm: Alternate Approaches to Demonstrating Substantial Equivalence in Premarket Notifications

http://www.softwarecpr.com/docs/510knewparadigmguidance.pdf

1/1/98 PMA Manual http://www.fda.gov/cdrh/manual/pmamanul.pdf 1/10/97 Deciding when to submit a 510(k) for a

change to existing device http://www.softwarecpr.com/docs/510kmod.pdf

11/21/95 510(k) Requirements During Firm-Initiated Recalls

http://www.softwarecpr.com/docs/510(k)reqs4recallfixes.PDF

Device-specific Software Submission Guidances

Date Document Link 10/4/05 Guidance for Industry and

FDA Staff Class II Special Controls Guidance Document: AFP-L3% Immunological Test Systems – Major level of concern

http://www.softwarecpr.com/docs/FDA-AFP-L3immunoIVD100405.pdf

5/11/05 Guidance for Industry and FDA Staff Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices

http://www.softwarecpr.com/Docs/FDASoftwareSubmissionGuidance.pdf

3/10/05 Guidance for Industry and FDA Staff Class II Special Controls Guidance Document: Instrumentation for Clinical Multiplex Test Systems – Moderate level of concern

http://www.softwarecpr.com/Docs/MultiplexClinicalChemistryGuidance1546.pdf

3/10/05 Guidance for Industry and FDA Staff Class II Special Controls Guidance Document: Drug Metabolizing Enzyme Genotyping System – Moderate level of concern

http://www.softwarecpr.com/Docs/EnzymeGenotypingGuidance-1551.pdf

12/28/04 Guidance for Industry and FDA Staff Class II Special Controls Guidance Document: Assisted Reproduction Laser Systems

http://www.softwarecpr.com/Docs/510kAssistedReproductionLaserSystems-122804.pdf

5/11/04 Guidance for Industry and FDA Staff Class II Special

http://www.softwarecpr.com/Docs/CancelCellDetectionDeviceGuidance0904-Doc1531.pdf


http://www.softwarecpr.com/docs/510kuseOfStdsguidance.pdf

http://www.softwarecpr.com/docs/510kQ&A.pdf

http://www.softwarecpr.com/docs/PMAmodifications.pdf

http://www.softwarecpr.com/docs/510knewparadigmguidance.pdf

http://www.fda.gov/cdrh/manual/pmamanul.pdf

http://www.softwarecpr.com/docs/510kmod.pdf

http://www.softwarecpr.com/docs/510(k)reqs4recallfixes.PDF

http://www.softwarecpr.com/docs/FDA-AFP-L3immunoIVD100405.pdf

http://www.softwarecpr.com/Docs/FDASoftwareSubmissionGuidance.pdf

http://www.softwarecpr.com/Docs/MultiplexClinicalChemistryGuidance1546.pdf

http://www.softwarecpr.com/Docs/EnzymeGenotypingGuidance-1551.pdf

http://www.softwarecpr.com/Docs/510kAssistedReproductionLaserSystems-122804.pdf

http://www.softwarecpr.com/Docs/CancelCellDetectionDeviceGuidance0904-Doc1531.pdf

Page 3 of 7


Controls Guidance Document: Immunomagnetic Circulating Cancer Cell Selection and Enumeration System – software considered Moderate LOC

3/16/04 Guidance for Industry and FDA Staff Class II Special Controls Guidance Document: Factor V Leiden DNA Mutation Detection Systems

http://www.softwarecpr.com/docs/DNAMutationDetectionSystems.pdf

10/28/2003 Class II Special Controls Guidance Document: Arrhythmia Detector and Alarm; Draft Guidance for Industry and FDA

http://www.softwarecpr.com/docs/Arrhythmiadetectorandalarm.pdf

6/19/03 Guidance for Industry and FDA Staff 510(k) Submissions for Coagulation Instruments

http://www.softwarecpr.com/docs/510KSubmissionforCoagulationInstruments.pdf

4/22/03 Class II Special Controls Guidance Document: Optical Impression Systems for Computer Assisted Design and Manufacturing (CAD/CAM) of Dental Restorations; Guidance for Industry and FDA

http://www.softwarecpr.com/docs/OpticalImpression-CAD-CAM.pdf

2/28/2003 Medical Devices; Hematology and Pathology Devices; Reclassification of Automated Blood Cell Separator Device Operating by Filtration Principle from Class III to Class II

http://www.softwarecpr.com/Docs/bldcellsep.pdf

12/13/2003 Class II Special Controls Guidance Document: Cutaneous Carbon Dioxide (PcCO2) and Oxygen(PcO2) Monitors; Draft Guidance for Industry and FDA- Moderate

http://www.fda.gov/cdrh/ode/guidance/1335.pdf

8/14/2002 Class II Special Controls Guidance Document: Dental Sonography and Jaw Tracking Devices; Draft Guidance for Industry and FDA Reviewers

http://www.softwarecpr.com/docs/DentalSonographySpecialControls.pdf

6/12/2002 Implantable Middle Ear Hearing Device; Draft Guidance For Industry and FDA


2/7/02 Class II Special Controls Guidance Document: Medical Washers and Medical Device Washer-Disinfectors; Guidance for

http://www.softwarecpr.com/docs/510kWasherDisinfectorGuidanceFinal1252.pdf


http://www.softwarecpr.com/docs/DNAMutationDetectionSystems.pdf

http://www.softwarecpr.com/docs/Arrhythmiadetectorandalarm.pdf

http://www.softwarecpr.com/docs/510KSubmissionforCoagulationInstruments.pdf

http://www.softwarecpr.com/docs/OpticalImpression-CAD-CAM.pdf

http://www.softwarecpr.com/Docs/bldcellsep.pdf


http://www.softwarecpr.com/docs/DentalSonographySpecialControls.pdf


http://www.softwarecpr.com/docs/510kWasherDisinfectorGuidanceFinal1252.pdf

Page 4 of 7


the Medical Device Industry and FDA Review Staff - Moderate

12/3/01 FDA 510(k) Guidance Ingestible Imaging Capsule SW- Minor


11/28/01 Class II Special Controls Guidance Document: Ingestible Telemetric Gastrointestinal Capsule Imaging System; Final Guidance for Industry and FDA


11/15/01 Class II Special Controls Guidance Document: Indwelling Blood Gas Analyzers


8/3/01 Premarket Notification Submissions forAutomated Testing Instruments Used in Blood Establishments

http://www.softwarecpr.com/docs/pmaautotest-CBER510-k.pdf

4/2/01 Information Disclosure by Manufacturers to Assemblers for Diagnostic X-ray Systems; Final Guidance for Industry and FDA


3/12/01 Class II Special Controls Guidance Document: Pharmacy Compounding Systems; Final Guidance for Industry and FDA


7/27/00 Guidance for the Submission Of Premarket Notifications for Medical Image Management Devices - Minor

http://www.softwarecpr.com/docs/ImagemanagementGuidance72700.pdf

7/6/00 1-Consolidated Annual Report for a Device product line (1-CARD)


6/21/00 Non-invasive Pulse Oximeter - Major


1/24/00 Guidance Document for Premarket Notification Submissions for Nitric> <Oxide> Delivery Apparatus, <Nitric> <Oxide> Analyzer and Nitrogen Dioxide Analyze

http://www.softwarecpr.com/docs/510kNitricoxidedevices.PDf

8/25/99 Electro-optical Sensors for the In Vivo Detection of Cervical Cancer and its Precursors:


6/9/99 Guidance Document for Powered Muscle Stimulator 510(k)s


4/27/99 Guidance for Industry and FDA Reviewers/Staff In Vitro Diagnostic Fibrin Monomer Paracoagulation






http://www.softwarecpr.com/docs/pmaautotest-CBER510-k.pdf



http://www.softwarecpr.com/docs/ImagemanagementGuidance72700.pdf



http://www.softwarecpr.com/docs/510kNitricoxidedevices.PDf




Page 5 of 7


Test 3/18/99 Guidance Document for

Industry and CDRH Staff for the Preparation of Investigational Device Exemptions and Premarket Approval Applications for Bone Growth Stimulator Devices

http://www.fda.gov/cdrh/ode/bgsguide.pdf

12/3/98 Guidance for the Submission of Premarket Notifications for Emission Computed Tomography Devices and Accessories (SPECT and PET) and Nuclear Tomography Systems


11/20/98 Guidance for the Submission of Premarket Notifications For Radionuclide Dose Calibrators

http://www.fda.gov/cdrh/ode/radcalibrators.pdf

11/14/98 Guidance for the Submission Of Premarket Notifications for Magnetic Resonance Diagnostic Devices


11/5/98 Cardiac Monitor Guidance (including Cardiotachometer and Rate Alarm)


9/30/98 Guidance Document for Powered Suction Pump 510(k)s

http://www.fda.gov/cdrh/ode/powerpump.pdf

8/7/98 Guidance for the Content of Premarket Notifications for Hemodialysis Delivery Systems

http://www.softwarecpr.com/docs/HEMODIALYSISDELIVERYSYSTEMS.pdf

8/4/98 Guidance On the content and format of Premarket Notification 510(K) Submissions Of Washers and Washer/Disinfectors - Moderate

http://www.softwarecpr.com/docs/washerdisinfectors1998draft.pdf

7/6/98 Guidance for Industry In Vitro Diagnostic Urea Nitrogen Test System


7/6/98 Guidance for Industry In Vitro Diagnostic Sodium Test System


7/6/98 Guidance for Industry In Vitro Diagnostic Potassium Test System


7/6/98 Guidance for Industry In Vitro Diagnostic Chloride Test System


7/2/98 Guidance for Industry In http://www.fda.gov/cdrh/ode/74.html


http://www.fda.gov/cdrh/ode/bgsguide.pdf


http://www.fda.gov/cdrh/ode/radcalibrators.pdf



http://www.fda.gov/cdrh/ode/powerpump.pdf

http://www.softwarecpr.com/docs/HEMODIALYSISDELIVERYSYSTEMS.pdf

http://www.softwarecpr.com/docs/washerdisinfectors1998draft.pdf






Page 6 of 7


Vitro Diagnostic Creatinine Test System

2/19/98 30-Day Notices and 135-Day PMA Supplements for Manufacturing Method or Process Changes, Guidance for Industry and CDRH

http://www.fda.gov/cdrh/modact/daypmasp.pdf

11/3/97 Electroencephalograph Devices Guidance for 510(k) Content Draft Document Version 1.0

http://www.fda.gov/cdrh/ode/eedev.html

6/4/97 Intrapartum Continuous Monitors for Fetal Oxygen Saturation and Fetal pH

http://www.fda.gov/cdrh/ode/fetal.html

2/28/97 Review Criteria Assessment of Portable Blood Glucose Monitoring In Vitro Diagnostic Devices Using Glucose Oxidase, Dehydrogenase or Hexokinase Methodolog

http://www.softwarecpr.com/docs/PortableGlucoseSubmissionGuidance.htm

3/10/97 Non-invasive Blood Pressure (NIBP) Monitor Guidance

http://www.fda.gov/cdrh/ode/noninvas.html

1/31/97 Third Party Review Guidance For Vitreous Aspiration & Cutting Device Premarket Notification (510(k))


1/13/97 Reviewer guidance for a Premarket notification submission for blood establishment Computer Software

http://www.softwarecpr.com/docs/Bloodbank510kguidance.pdf

9/6/96 Letter to Manufacturers of Prescription Home Monitors for Non-Stress Tests

http://www.fda.gov/cdrh/ode/homenst.html

6/10/96 Data of Commercialization of Original Equipment Manufacturer, Secondary and Generic Reagents for Automated Analyzers

http://www.fda.gov/cdrh/ode/odecl950.html

3/14/96 Thermal Endometrial Ablation Devices


3/7/96 Hysteroscopes and Gynecologic Laparoscopes Submission Guidance for a 510(k)


7/26/95 Guidance Document For the Preparation of Premarket Notification [510(K)] Applications For Powered Tables and Multifunctional Physical Therapy Tables


7/26/95 Guidance Document for the Preperation of Premarket Notification [510(k)] Applications for Immersion



http://www.fda.gov/cdrh/modact/daypmasp.pdf

http://www.fda.gov/cdrh/ode/eedev.html

http://www.fda.gov/cdrh/ode/fetal.html

http://www.softwarecpr.com/docs/PortableGlucoseSubmissionGuidance.htm

http://www.fda.gov/cdrh/ode/noninvas.html


http://www.softwarecpr.com/docs/Bloodbank510kguidance.pdf

http://www.fda.gov/cdrh/ode/homenst.html

http://www.fda.gov/cdrh/ode/odecl950.html





Page 7 of 7


Hydrobaths 7/26/95 Guidance Document for the

Preparation of Premarket Notification [510(k)] Applications for Mechanical and Powered Wheelchairs, and Motorized Three-wheeled Vehicles


7/26/95 Guidance Document for the Preparation of Premarket Notification [510(k)] Applications for Submerged (Underwater) Exercise Equipment


7/26/95 Guidance Document For the Preparation of Premarket Notification [510(K)] Applications For Communications Systems (Powered and Non-Powered) and Powered Environmental Control Systems


7/1/95 Reviewer Guidance for Ventilators

http://www.softwarecpr.com/docs/Ventilator510kdraftguidance1995.pdf

8/1/94 Guidance for Tens 510(k) Content


8/1/94 Galvanic Skin Response Measurement Devices Draft Guidance For 510(K) Content


8/1/94 Biofeedback Devices Draft Guidance for 510(K) Content


7/7/94 Draft Version Neuro Endoscope Guidance


6/19/94 Draft Guidance for Implantable Cardioverter-Defibrillators Version 4.2

None

10/1/93 Reviewer Guidance For Nebulizers, Metered Dose Inhalers, Spacers and Actuators


3/1/93 Guidance on the Content of Premarket Notification [510(k)] Submissions for External Infusion Pumps

http://www.fda.gov/cdrh/ode/odegr823.html





http://www.softwarecpr.com/docs/Ventilator510kdraftguidance1995.pdf






http://www.fda.gov/cdrh/ode/odegr823.html

SoftwareCPR 11-Jan-2002 Page 1 of 3

www.softwarecpr.com

20 Berkshire Drive Winchester, MA 01890

781-721-2921

Validation & Verification Definitions

This training document provides a list of verification and validation terms and the definitions specified in various FDA documents and referenced standards. Validation terminology can be quite confusing because of the general (validation) and specific (process validation, design validation, software validation) uses of the term validation and the variety of definitions used in standards, by FDA, and by the software engineering community. In addition, there are many different approaches to validation and this can add to the confusion. SoftwareCPR suggests that the exact definitions and arguments related to separating verification and validation terminology and activities are not highly productive. It is, however, important to understand the range of meanings and intent in order to perform verification and validation activities that together satisfy the intent of the regulatory requirements. It is also important to be able to articulate why a particular approach covers all relevant activities regardless of the exact definitions and whether or not software verification and validation activities are separated from each other, or even whether or not they are separated from overall device design validation or in manufacturing from equipment qualification or process validation activities. The sequence and organization of activities and validation documents can be based on whatever is the most efficient and effective approach for a given situation as long as it can be articulated how all verification and validation requirements are met.1 The intent of the terms list compiled below is to aide in understanding the range of definitions, the distinctions, and their overlap. The list is organized by document source starting with the Jan 11, 2002 General Principles of Software Validation Guidance issued jointly by CDRH and CBER.

General Principles of Software Validation Guidance “Software verification provides objective evidence that the design outputs of a particular phase of the software development life cycle meet all of the specified requirements for that phase. Software verification looks for consistency, completeness, and correctness of the software and its supporting documentation, as it is being developed, and provides support for a subsequent conclusion that software is validated. Software testing is one of many verification activities intended to confirm that software development output meets its input requirements. Other verification activities include various static and dynamic analyses, code and document inspections, walkthroughs, and other techniques.” “Software validation is a part of the design validation for a finished device, but is not separately defined in the Quality System regulation. For purposes of this guidance, FDA considers software validation to be “confirmation by examination and provision of objective evidence that software specifications conform to user needs and intended uses, and that the particular requirements implemented through software can be consistently fulfilled.”2 In practice, software validation activities may occur both during, as well as at the end of the software development life cycle to ensure that all requirements have been fulfilled. Since software is usually part of a larger hardware system, the validation of software typically includes evidence that all software requirements have been implemented correctly and completely and are traceable to system requirements. A conclusion that software is validated is highly dependent upon comprehensive software testing, inspections, analyses, and other verification tasks performed at each stage of the software development life cycle. Testing of device software functionality in a

1 SoftwareCPR Note: As an example FDA often expects unit, integration, and system testing to be performed as part of software verification. In some cases it makes sense to separate these activities including responsibilities for performing them and protocols associated with them. In other cases it can be most efficient to combine some of these activities in terms of responsibility and into single test protocols but one hould be able to articulate how the combined protocol accomplishes both unit and integration testing for instance. To do so effectively on must understand the distinctions and intent of the different terms. 2 This definition is identical to the definition in the AAMI SW68 Standard Medical Devices – Software life cycle processes

CRISIS PREVENTION AND RECOVERY , LLCSOFTWARE CPR


simulated use environment, and user site testing are typically included as components of an overall design validation program for a software automated device.”

FDA Quality System Regulation Part 820 Process validation means establishing by objective evidence that a process consistently produces a result or product meeting its predetermined specifications. Validation means confirmation by examination and provision of objective evidence that the particular requirements for a specific intended use can be consistently fulfilled. Design validation means establishing by objective evidence that device specifications conform with user needs and intended use(s). Design validation. Each manufacturer shall establish and maintain procedures for validating the device design. Design validation shall be performed under defined operating conditions on initial production units, lots, or batches, or their equivalents. Design validation shall ensure that devices conform to defined user needs and intended uses and shall include testing of production units under actual or simulated use conditions. Design validation shall include software validation and risk analysis, where appropriate. The results of the design validation, including identification of the design, method(s), the date, and the individual(s) performing the validation, shall be documented in the DHF. § 820.75 Process validation. (a) Where the results of a process cannot be fully verified by subsequent inspection and test, the process shall be validated with a high degree of assurance and approved according to established procedures. The validation activities and results, including the date and signature of the individual(s) approving the validation and where appropriate the major equipment validated, shall be documented. (b) Each manufacturer shall establish and maintain procedures for monitoring and control of process parameters for validated processes to ensure that the specified requirements continue to be met. (1) Each manufacturer shall ensure that validated processes are performed by qualified individual(s). (2) For validated processes, the monitoring and control methods and data, the date performed, and, where appropriate, the individual(s) performing the process or the major equipment used shall be documented. (c) When changes or process deviations occur, the manufacturer shall review and evaluate the process and perform revalidation where appropriate. These activities shall be documented. Verification means confirmation by examination and provision of objective evidence that specified requirements have been fulfilled. Design verification. Each manufacturer shall establish and maintain procedures for verifying the device design. Design verification shall confirm that the design output meets the design input requirements. The results of the design verification, including identification of the design, method(s), the date, and the individual(s) performing the verification, shall be documented in the DHF.

FDA ORA Software Glossary data validation. (1) (ISO) A process used to determine if data are inaccurate, incomplete, or unreasonable. The process may include format checks, completeness checks, check key tests, reasonableness checks and limit checks. (2) The checking of data for correctness or compliance with applicable standards, rules, and conventions. Regression Analysis And Testing. A software verification and validation task to determine the extent of verification and validation analysis and testing that must be repeated when changes are made to any previously examined software products.[SoftwareCPR Note: This term and definition are also included in FDA’s Draft Part 11 glossary ] revalidation. Relative to software changes, revalidation means validating the change itself, assessing the nature of the change to determine potential ripple effects, and performing the necessary regression testing. VV&T. validation, verification, and testing.


validation. (1) (FDA) Establishing documented evidence which provides a high degree of assurance that a specific process will consistently produce a product meeting its predetermined specifications and quality attributes. Contrast with data validation. validation, process. (FDA) Establishing documented evidence which provides a high degree of assurance that a specific process will consistently produce a product meeting its predetermined specifications and quality characteristics. validation, prospective. (FDA) Validation conducted prior to the distribution of either a new product, or product made under a revised manufacturing process, where the revisions may affect the product's characteristics. validation protocol. (FDA) A written plan stating how validation will be conducted, including test parameters, product characteristics, production equipment, and decision points on what constitutes acceptable test results. See: test plan. validation, retrospective. (FDA) (1) Validation of a process for a product already in distribution based upon accumulated production, testing and control data. (2) Retrospective validation can also be useful to augment initial premarket prospective validation for new products or changed processes. Test data is useful only if the methods and results are adequately specific. Whenever test data are used to demonstrate conformance to specifications, it is important that the test methodology be qualified to assure that the test results are objective and accurate. validation, software. (NBS) Determination of the correctness of the final program or software produced from a development project with respect to the user needs and requirements. Validation is usually accomplished by verifying each stage of the software development life cycle. See: verification, software. validation, verification, and testing. (NIST) Used as an entity to define a procedure of review, analysis, and testing throughout the software life cycle to discover errors, determine functionality, and ensure the production of quality software. V&V. verification and validation. verification, software. (NBS) In general the demonstration of consistency, completeness, and correctness of the software at each stage and between each stage of the development life cycle. See: validation, software. prototyping; spiral model.

IEEE 1012:1998 Standard for Software Verification and Validation ISO 8402:1994

validation: Confirmation by examination and provisions of objective evidence that the particular requirements for a specific intended use are fulfilled.

NOTES 1—In design and development, validation concerns the process of examining a product to determine conformity with user needs. 2—Validation is normally performed on the final product under defined operating conditions. It may be necessary in earlier stages. 3—“Validated” is used to designate the corresponding status. 4—Multiple validations may be carried out if there are different intended uses. [ISO 8402: 1994]

verification: Confirmation by examination and provisions of objective evidence that specified requirements have been fulfilled.

NOTES 1—In design and development, verification concerns the process of examining the result of a given activity to determine conformity with the stated requirement for that activity. 2—“Verified” is used to designate the corresponding status. [ISO 8402: 1994]

SoftwareCPR® GPSV Lifecycle Requirements Checklist 5-April-04

Page 1 of 18

Company: Project: Assessor:

GPSV Section Requirements (assessor comments if any in parenthesis)

Yes/No/Partal Documents Conform? Yes/No/Partial And Doc Names

1 Introduction

2 GPSV Tasks

2.1 Quality Planning

Is there an established system development methodology?

Do SOP’s define a software development lifecycle model?

Do SOP’s require design and development planning?

Do SOP’s require design and development planning to identify the Following?

tasks? (methods? procedure? acceptance criteria? input/ouput?)

roles, resources and responsibilities? (personnel and equipment)

management review requirements?


Page 2 of 18



formal design review?

risks and assumptions?

role of risk management?

Do SOP’s require design and development planning to identify the Following?

Risk Management planning?

preliminary Risk Analysis during requirements definition?

Re examined after the design has been specified

Software Quality Assurance including Verification &Validation Planning?

Technical Reviews?

Do SOP’s require a plan to define and control the software validation Process?

Do SOP’s require validation planning to address:


Page 3 of 18



specific validation procedures?

validation coverage based on complexity and safety risk?

independent evaluation and review of validation activities?

Do SOP’s require planning to include Configuration Management Planning?

Do SOP’s require Configuration Management planning to address Links across the following:

specifications? design documents?

source code? object code?

tests?

Do SOP’s include Configuration management and Change control procedures?

Do SOP’s require Configuration Management planning to address identification and access to currently approved versions?


Page 4 of 18



Do SOP’s require procedures for problem/anomaly reporting and resolution and define the following:

problem report content and format?

responsibilities?

Do SOP’s require procedures for review and approval of software Development results?

Do these procedures include responsible organizational elements for Review and approval?

Do SOP’s require planning to identify software development support Activities?

2.2 Requirements

2.2.1 Requirements Specifications

Do SOP’s require identification, analysis and documentation of Specification?

Do SOP’s require specifications to define the intended use of device?

Do SOP’s require specifications to define user needs?


Page 5 of 18



Do SOP’s require specifications to define software functions? Does This include:

system inputs and outputs?

functions?

performance requirements?

hardware and environment?

algorithms?

external and user interfaces?

error and error handling?

response times?

operation environment?

ranges? limits? defaults?


Page 6 of 18



requirements for vendors and product qualification and controls for COTS?

Do SOP’s require software requirement specifications to identify Safety related requirements ?

Do SOP’s require identification of potential hazards due to software Failures?

Do SOP’s require identification of mitigations for software failures? Does this include:

hardware?

defensive programming?

2.2.2 Requirements Review

Do SOP’s require verification of requirements specifications using Formal design review?

Do SOP’s address incomplete, ambiguous or conflicting requirements?

Do SOP’s require requirement specifications to be evaluated for:

accuracy?


Page 7 of 18



completeness?

consistency,?

testability, (measurable or objectively verifiable)?

correctness?

clarity?

Do SOP’s require requirements reviews before extensive design effort Begins?

Do SOP’s require requirements to be approved and released?

Do SOP’s require software requirements specifications to be updated Through development of life cycle?

Do SOP’s require traceability analysis to trace software requirement specifications to system requirements?

to risk analysis results? (system hazards)?

2.3 Design


Page 8 of 18



2.3.1 Design Specifications

Do SOP’s require design specifications?

Do SOP’s require design specifications to address human factors?

Are designs evaluated for complexity?

Are designs evaluated to ensure they are not contrary to user intuitive expectations?

Include participants from the user community?

Do SOP’s require design specifications to include the following:

system documentation of function, hardware, software and environment?

hardware to be used? parameters to be measured or recorded?

logical structures and process? data structures and data flow? variables definitions?

error, alarm warning messages, or statement identifying lack of error messages?


Page 9 of 18



supporting software, such as operating systems, drivers?

communication links within software? to other software? hardware? and user?

security measures?

additional constraints?

2.3.2 Design Reviews

Do SOP’s require verification of software design documents using formal design review?

Do SOP’s require design reviews to determine if the design is:

correct? consistent? complete? unambiguous?

feasible? maintainable? accurate? testable?

Do SOP’s require design reviews before implementation?

Do SOP’s require design reviews to evaluate the following:


Page 10 of 18



control flows? data flows?

complexity?

timing? sizing?

memory allocation?

criticality analysis?

Do SOP’s require analysis of design for all communication links within software, to other software, hardware and user?

Do SOP’s require traceability analysis to trace software requirements to the design?

Do SOP’s require traceability analysis to trace design to software requirement?

Do SOP’s require designs to be approved, released, and controlled?

Do SOP’s require Risk Analysis?

2.4 Coding


Page 11 of 18



Do SOP’s require source code evaluation? Does this include:

verification between source code and design specification?

code inspections, code walkthrough?

Do SOP’s require coding guidelines? Do these guidelines include the following:

compiler level of error checking

warning messages and justifications?

coding conventions?

Do SOP’s require procedures and results of source code evaluation to be maintained?

Do SOP’s require source code traceability analysis? Does this include:

design traceable to code?

code traceable to design specification? to risk analysis?


Page 12 of 18



tests traceable to design specification? to risk analysis? to code?

2.5 Developer Testing

2.5.1 Test Planning

Do SOP’s require the development of test plans and test cases? Does this include the following:

schedules? resources? (personnel, tools)

methodologies?, environment?

test cases? (procedures, inputs, outputs, expected results) documentation?

reporting criteria?

Do SOP’s require testing effort to be based on complexity, criticality, reliability and or safety issues?

Do SOP’s require levels of test planning? Unit? Integration? System?

Do SOP’s require test cases based on both user and coder perspectives?


Page 13 of 18



code and non-coded based test?

white box testing? identification of dead code?

statement coverage?, branch coverage?

black box testing? normal? boundary? error guessing? special case?

random data sets? ad hoc?

Do SOP’s require system level testing?

security features?

recovery procedures?

compatibility with other products?

2.5.2 Test Execution

Do SOP’s address recording of test data and test results?


Page 14 of 18



Do SOP’s address errors detected during testing? Does this include:

logging? classifying? reviewing?

resolving errors prior to release of the software?

are these changes made under configuration controls?

regression testing?

2.5.3 Testing Tools

Do SOP’s address specification (intended use) and validation of software tools?

drivers and stubs?

commercial software testing tools?

2.6 User Site Testing

Do SOP’s require User Site testing:


Page 15 of 18



Do SOP’s require the following for User Site testing:

pre-defined written plan?

addresses traceability analysis?

testing protocols, procedures, and acceptance criteria?

objective evidence of the validation process

formal summary of testing?

record of formal acceptance?

Do test plans for User Site testing address the following:

evidence hardware and software are installed and configured as specified?

testing throughout the full range of operating conditions?

sufficient time to encounter faults not apparent during normal activities?


Page 16 of 18



repeat of earlier testing including stress? fault testing?, error messages? safety requirements?

Validation(meeting user needs) not just Verification(meeting specifications)? (for example implications to the user based on algorithm modifications.)

Do SOP’s require evaluation of the ability of the users to understand and correctly interface with the system?

Do SOP’s address system failures detected during User Site testing?

Addresses issue identification and resolution tracking?

Do SOP’s address formal release process?

2.7 Maintenance

Do SOP’s define a software life cycle for software change?

for enhancements? for corrections?

Do SOP’s require software change assessment?

Do SOP’s require validation analysis for a software change?


Page 17 of 18



Does the validation analysis require evaluation of the individual change and potential impact of change on the system?

Do SOP’s address revising software validation plan?

Do SOP’s address anomaly evaluation? Does this including

evaluated for severity and system and safety effects?

root cause analysis? identification of trends? CAPA?

Do SOP’s address problem identification and resolution tracking?

Do SOP’s require documentation to be reviewed and updated?

Specifications, design, test, user manuals ?

Do SOP’s require specifications to be updated before software changes are made?

design documents?

3 Off -the -shelf (OTSS) Software


Page 18 of 18



Do SOP’s ensure development methodologies, used for OTS, are appropriate and sufficient for intended use?

Do SOP’s address assessment of vendor life cycle development and validation documents, system requirements, software requirements, design specifications, validation process, testing protocols, source code, results of validation?

Do SOP’s require validation for OTS to ensure it meets user needs and intended use?

Do SOP’s address use of development tools such as compilers, linker, editor, operation systems?

Page 1 of 3

12/18/02

Software Test Coverage Article1 Alan Kusinitz

Software testing is an important aspect of software verification and software validation as defined by FDA. Along with requirements specification and configuration management it is one of the aspects of software development and design control that receives the most scrutiny by FDA reviewers as well as inspectors. The most difficult aspect of testing (and the hardest to teach) is determining “how much is enough”. One can always do more, use a more rigorous method, add other tools, provide more specificity and detail in specifications and test documents, or perform more extensive reviews of tests and their results. For software related to medical devices it is now quite common to assert that the rigor and detail required is relative to the safety risks involved. FDA says this explicitly in their guidance documents.2 But what exactly does this mean? In a particular situation, how much is enough? Performing adequate testing would be simple if software were simple – one could identify and execute a small number of tests that would thoroughly evaluate all quality-related attributes of the software under all conditions (e.g., stress, fault, etc.). Since even “small” software programs can, however, have a large number of permutations and combinations of inputs, logic, and execution paths, it is generally impossible to “fully” test software. Once this is understood it becomes clear that test planning is a sampling-based activity best done with a risk-based perspective. Random sampling has not been demonstrated to be an adequate approach since the location of software bugs do not follow a normal distribution; further, some bugs are more important to find then others based on safety and effectiveness concerns. This is important for developers, testers, and regulators to recognize. A failure to do so can result in either unrealistic requirements for testing, or

1 Draft of SoftwareCPR® article printed in the May/June issue of the AAMI BIT peer reviewed journal. For more information click www.softwarecpr.com. 2 The FDA’s General Principles of Software Validation Final Jan 2002 and the associated Federal Register notice providing explanation and commentary.

a dangerous false sense of security that if one runs “all” defined tests one is “fully” testing the software.3 There are many approaches to, and tools and methodologies for, testing. In a regulated environment there are additional explicit and implicit requirements for testing. While not trivial, it is relatively easy to develop an understanding of specific approaches, methods, tools, and test documentation requirements. However, there is more to consider. A key factor in planning adequate testing, and determining and demonstrating when one has “done enough”, is determining if test coverage is adequate. This is a different perspective then simply determining if test documentation is adequately detailed, or if test plans and test cases are properly approved and reviewed. Instead, it is a focus on first identifying the various types of coverage that one needs to achieve, then planning tests to based on the coverage identified, and developing a documentation approach (normally including a variety of trace or cross reference tables) that will adequately demonstrate that the intended coverage was achieved. FDA in its guidance documents states clearly that the rigor and degree of test coverage required can vary based on the safety and effectiveness issues involved. But if there is no planned coverage, or way to demonstrate such coverage is achieved, then arguments over how much is enough are futile. One can have hundreds or even thousands of pages of detailed test cases and test results , and yet still be unsure if one has achieved adequate test coverage. An FDA reviewer or inspector faced with examining reams of detailed test case and results documentation can look at that test documentation for the elements of information included, but can not easily determine coverage adequacy without themselves comparing the tests against various aspects that should be covered and asking questions to challenge its adequacy. So, at best, voluminous test documentation alone forces an auditor or inspector to look carefully at the detail. At its worst such an approach suggests no effective process was in place for defining or assuring adequate test coverage and may challenge the inspector to find the holes. In fact, a common auditing technique is to select a few requirements, hazard mitigations, or types of tests and look to see where and how thoroughly they were

3 This is especially dangerous if one assumes that just by repeating all original tests as part of regression testing that the software must then be safe and effective.

Page 2 of 3

addressed in testing. Where a coverage was neither planned prior to testing or verified after testing it is common for deficiencies to exist and be easily detected. So if test coverage is such an important element in assuring that one has done enough testing, then the first question to ask is, what types of coverage should be addressed? The most common aspect of coverage discussed and addressed is coverage of requirements. In other words, is there one or more test case or a portion of a test case to cover each of the documented requirements? If there are requirements without associated test cases, then it would seem clear that testing is not adequate. Planning and evaluating test coverage of requirements can be complex. It can involve extensive use of complex requirements management, test planning, and traceability tools for complex or critical software . For something simple – such as a spreadsheet used to implement QC testing for medical device production – it could be a two column table (see Table 1) with requirements or requirements numbers in the left column and test cases listed in the right column. Several tests could be listed for one requirement or one test could be listed for multiple requirements. If there are no blanks in the test column then some level of requirements coverage will be achieved and can be easily demonstrated.4 REQUIREMENT TEST CASE

1.Calculate Sample Standard Deviation of 20 test results

T1

2 Calculate mean of 20 results T1

.3 Set an error flag if any result cells are outside the range 2.5 – 4.7

T2

Table 1 Requirements/Test Coverage This, however, is only one type of coverage. Even where extensive effort is associated in assuring that testing covers all requirements, this alone is not enough. Some other types of coverage that could be

4 Obviously simply having a test case or test step that addresses a requirement does not necessarily mean that that testing was comprehensive and rigorous, but it does demonstrate a certain level of coverage.

important include, but are not limited to, the following:

• Hazard coverage – testing each hazard and each risk control measure individually and under a range of conditions normal and abnormal.

• Structural Coverage - testing each subsystem, module, object, algorithm, or off-the-shelf component, or even each line of code or path in some cases.

• Platform Coverage - testing on all of the intended operating system or hardware platforms

• Version Coverage – testing of all the versions and variants currently supported or in use

• Test Type Coverage – this would demonstrate that all of the types of testing that were intended to be executed (e.g., stress, usability, multi-user, timing, start up and shutdown, security, boundary, error, etc.) were addressed

• Interface Coverage – testing all of the internal and external interfaces of the software

• Environment testing – testing in all of the intended environments such as hospital types, languages, etc.

• User Type Testing – testing to demonstrate that each type of intended user, for instance, supervisors, lab technicians, patients , can use the device

• Regulatory and Standards Compliance – testing to demonstrate that required functionality from regulations and standards has been verified (e.g., 21 CFR Part 11 computer generated audit trail and security requirements)

Coverage maps similar to Table 1 could be constructed for each relevant type of coverage needed as part of the test planning process. In addition to specific types of test coverage, there is also the concept of compound coverage. For instance, one can have tests for all the requirements, and one could have tests for each relevant test type, but has one tested each of the most important or safety related requirements under stress and across a variety of test types ? Another example is that one can test all requirements but has one tested on all of the relevant versions or variants of the software, or on at least the final version of software? These can all be

Page 3 of 3

important aspects of testing especially for highly critical software or functionality. The concept of coverage of test types receives some of the least attention in formal plans, documentation, and coverage matrices. Often coverage of test types is dealt with informally as the testers write test cases and using their experience add negative, stress, and other types of testing. The purpose of planning and tracking coverage of test types is to insure that test cases are constructed to address a wide range of conditions. Both the FDA’s General Principles of Software Validation Guidance and their Guidance for the Content of Pre-market Submissions for Software Contained in Medical Devices Guidance provide lists of some types of testing to be considered. It is not uncommon for FDA 483 observations and warning letters to mention that certain types of testing were not addressed or were not adequate. It is beyond the scope of this article to identify and explain all possible relevant types of testing, but from the perspective of determining how much is enough, one consideration is : does the testing address all the relevant types of testing for the particular application? Types of testing to be addressed can be identified in a high-level test plan. A cross reference or test trace matrix can then be constructed that has on the left the types of testing to be addressed, and on the right a list of the test cases that cover each test type. This can both improve test coverage and demonstrate at a high level that such testing has been accomplished. In some cases it may be advisable to assure that all safeguards implemented in software receive multiple types of testing and this can be demonstrated using a similar table just for this purpose. Another aspect of coverage has to do with stages and levels of testing. One of the questions that often arises is: what does the FDA mean by unit, integration, system testing and user site testing and how much testing needs to be done at each of these levels? An answer often given is that unit testing is white box structural testing of each module within the system, integration testing is the testing of internal and external interfaces, system testing is functional black box testing of the complete software, and user site testing is usability or beta testing with actual users in their intended environment. While this perspective is useful from a heuristic perspective, it is often too restrictive if translated to mean that each level of testing is planned, documented, and executed separately. Therefore, another aspect of test coverage is to demonstrate how the combination of tests planned and executed in

different stages and levels of testing cover the intent of the levels of testing mentioned in FDA documents. For instance some very important functionality, implemented as risk control measures, could be difficult or impossible to test on the final system and might require special test code or test harnesses or simulators and might better be accomplished at the unit or integration level during development. If this is the case then to assure and demonstrate test coverage of requirements one might have on the left side a list of requirements and on the right side several columns for each level of testing. The fact that there may be no system level tests for certain functionality would not necessarily mean the testing overall was not adequate provided the unit and integration testing that covered that functionality was performed on an appropriate version of the software, and not too early in the lifecycle. In designing regression testing, even for released product, it might be important to include in regression testing not just some functional tests, but some unit or white box tests that focus on particularly critical software components or algorithms. Coverage is so central to the adequacy of testing it could be suggested that for medical device software of minor and moderate levels of concern (as defined in FDA’s guidance documents) and software used in production or the quality system, testing aspects of regulatory reviews and inspections should be focused on determining the approaches taken to assuring test coverage, the types of coverage defined, and that the coverage planned was achieved. This in combination with a checklist of basic elements of test documentation would probably go further towards assuring test adequacy than subjective discussions about how detailed each line of a test protocol is and how detailed each set of results needs to be and could be sufficient for regulatory reviews of all but major level of concern software. From a practitioner’s perspective, determining how much testing is enough for safety-critical software, such as associated with medical devices, is something that keeps us awake at night. The realization that “full” testing is unachievable allows us to focus on risk based testing with defined test coverage requirements focused in areas of greatest risk, yet comprehensive in terms of types of coverage. Having done this we are better able to sleep at night having done our best to protect the public health and being able to demonstrate to regulators the adequacy of our testing.

One view of medical device risk management isthat it is intended to ensure safety. While this isa commendable goal, it does not adequately

represent the complexity of medical devices, their usage,or their potential benefits to public health.

For pharmaceutical products, the complexity of therisk/benefit tradeoff is well understood. Most drugs canhave serious health risks to some percentage of thepopulation due to allergic reactions, drug interactions, orside effects. The percentage of the population that couldbe negatively influenced, and yet is acceptable, variesbased on the overall potential public health benefits andthe target population. For instance, drugs for healthychildren generally have to be “safer” than drugs thatcould aid the terminally ill. The combination of societalvalues and legal and regulatory interpretationsdetermine acceptable risks; this could change over time.

Therefore, it is more accurate to state that the goal ofmedical device risk management is to reduce the risk aslow as reasonably practicable (ALARP), and ensure thatthe medical device has acceptable residual risk given itsintended use and potential positive impact on publichealth. Medical devices must not only be as safe aspossible, they must also be effective for their intendeduse; there is a tradeoff between these that blur thedistinction between safety and effectiveness.

BackgroundCurrent standards for medical device risk management(for example, ANSI/AAMI/ISO 14971) define risk assome combination of the severity of harm and theprobability2 of that harm occurring.

In many risk analysis schemes, there is significantfocus on establishing severities and probabilities ofpotentially hazardous situations and calculatingquantitative risk levels. Unfortunately, probability-basedrisk ratings are prone to misuse.

For example, estimation of the probability of asoftware defect that could lead to a hazard (or any othertype of systematic non-random design failure forhardware as well as software) is often a subjective processlacking an empirical basis. Sometimes the methodologyfor estimating probability and the use of these estimatesis flawed in relation to the intent of the overall riskratings. For instance, if one knows from past similarprojects that the released software defect rate is onedefect per 1,000 lines of code, one could say theprobability of any particular defect is very low.3 Whencombined with severity, all risk ratings could then end upvery low even if death or serious injury were possible ifthe software fails in certain ways.

Taking guesses at probabilities or applying the“wrong” type of probability when calculating probabilityof a hazard can distort risk ratings. As a result, it coulddiminish or eliminate the design focus on risk control ofsignificant hazards or the verification focus on testing offailures that could lead to these hazards.

In some risk management approaches, decisions aremade based on initial risk ratings as to whether riskcontrol is needed. If a risk rating is below a certain value,there is no process requirement in these schemes toidentify risk control measures for that hazard or failuremode. There are two potential problems with such anapproach:1. If probabilities are simply guesses, incorrect guesses

could lead to erroneously low-risk ratings, resulting in

QUALITY MATTERS

Alan Kusinitz

304 Biomedical Instrumentation &Technology

Uses and Misuses of Probability inMedical Device Risk Management

Alan Kusinitz specializes in softwaresafety, project management, validation,quality systems, regulatory compliance,and FDA representation and negotiation.He is the founder and Managing Partnerof SoftwareCPR, which specializes insoftware regulatory compliance andquality improvement.

1From IEC 60601-1-4.2When probability is determined using subjective rankings, rather than empiricalcalculations, it would be more appropriate to use the term likelihood. For thepurpose of this article and for consistency with related standards the termprobability is used and is intended to reflect both quantitative and subjectivequantitative or qualitative estimations.3In addition, the frequency of a defect’s manifestation in actual use depends onthe frequency of execution of the code involved which may also be difficult topredict.

Uses and Misuses of Probability in Medical Device Risk Management

QUALITY MATTERS

no attempt to develop risk control measures for thecorresponding hazards.

2. Setting a risk threshold results in no risk controlmeasures in which some could have been easilyidentified and implemented at no or low cost,resulting in a residual risk that even if acceptable is notas low as it could have been.

The FDA wrote in several of its guidances4 and inresponse to evaluation of several international standardsthat it does not agree that probability can be determinedfor a software failure. Therefore, FDA does not thinkthat probability should be considered in risk analysis forsoftware or hardware systematic5 design errors. TheFDA states that only hazard severity should be used insuch cases.

Since risk management standards, includingANSI/AAMI/ISO 14971, include probability and sincemany risk management practitioners believe probabilityis an important factor to be considered, it appears thatthere is a significant disagreement between the FDA andindustry.

It is this author’s opinion that this apparentdisagreement is due to an oversimplification of the use ofprobability and the lack of standard terminology andmethods for properly applying each different type ofprobability in medical device risk management.

Some people argue that for systematic errors, such ashardware design errors and all software defects, the bestapproach is to simply omit probability and use onlyseverity in risk management. While this may be betterthan misusing probability, it is an oversimplification.Such simplification may not always result in the lowestresidual risk for the device.

Some types of probability are not only beneficial but,at least implicitly, impossible to avoid. For instance,many worthwhile risk control measures do not entirelyeliminate a hazard. These measures merely lower theprobability of the hazardous situation or harm occurring.Even such common risk control measures for datacorruption, such as cyclic redundancy checks; paritychecks; and checksums, are not fool proof and simplyreduce the probability of undetected data corruption and

its related hazards. For hardware, redundancy to avoidsingle points of failure from resulting in a hazardoussituation is often considered adequate even though thisonly lowers the probability of the hazardous situationfrom occurring.

While estimating and properly using the probabilityof a specific software defect may be problematic,estimating the impact of a risk control measure on theprobability of a hazardous situation from occurring maybe quite useful. Therefore, certain types of probabilitycan be used effectively provided their limitations arecarefully considered and factored into the riskmanagement approach.

Let’s assume one agrees that medical device riskmanagement is intended to ensure that both residual riskis as low as reasonably practicable, and residual risk isacceptable.

Then, risk analysis and risk rating schemes mustfoster meaningful relative ratings—not necessarilyaccurate individual ratings—between risks of differentseverities and probabilities of harm. This will ensure thatdevelopment, verification, validation staff, and activitiesfocus greater attention in areas of higher risk.

A scheme that overrates the risk of a hazardoussituation can be almost as bad as one that underrates it. Ifall hazardous situations are treated as of equal risk, thereis no focus on the most probable or severe ones. Twohazardous situations may have the potential to result indeath, yet death may be much more probable in one case(due to fewer contraindications visible to the clinician orpatient) and this distinction should be incorporated intorisk ratings to allow proper focus. Alternatively, in adevice where all hazardous situations have low severitythere still needs to be a way to create some relativeranking or, again, there is no way to focus more attentionon the most significant risks of that particular device.

TerminologyRather than arguing over the benefits or dangers of usingprobability in general, the discussion is better shifted toidentifying different types of probability and discussingthe potential use or misuse of each type.

To effectively distinguish and use different types ofprobability, one must first establish some relatedterminology. Terminology from ANSI/AAMI/ISO14971 provides the following definitions:• Harm - physical injury or damage to the health of

people or damage to property or the environment

September/October 2005 305

4FDA’s Guidance for the Content of Premarket Submissions for SoftwareContained in Medical Devices, FDA’s Guidance for Industry, FDA Reviewers andCompliance on Off-the-Shelf Software Use in Medical Devices, and FDA’s writtencomments in its standard recognition statement on IEC 60601-1-4 dated8/14/1998.5As opposed to random errors such as hardware that wears out and has aknown mean time between failures.

Alan Kusinitz

QUALITY MATTERS


• Hazardous Situation - circumstance in whichpeople, property, or the environment are exposed toone or more hazard

• Hazard - potential source of harm• Risk - combination of the probability of occurrence of

harm and the severity of that harm• Severity - measure of the possible consequences of a

hazard• Systematic failure - design errors that do not occur

randomly and for which a probability of occurrencecannot be calculated

• Residual Risk - risk remaining after protectivemeasures have been taken.

The term hazard is a very imprecise and broad term asdefined and used in ANSI/AAMI/ISO 14971. TheAAMI Medical Device Software Risk ManagementTechnical Information Report (TIR32) expands andfurther subdivides this term into hazards, which arestated close to the level of clinical impact, and causes orcontributing factors. These factors could resultindividually or in some combination into a hazard thatcould lead to a hazardous situation, which couldpotentially result in harm.

Given this breakdown, it is clear that there could bedifferent probabilities for a specific cause, hazard,hazardous situation, and the actual harm. Theprobability of a specific cause could be high. However,due to other checks in the system or a combination ofnecessary conditions required to lead to the hazard, theprobability of the hazard occurring could be very low. Insome cases, even if the probability of a hazardoussituation is high, the probability of actual harm could below due to contraindications and clinical practice thatwould make detection of the failure obvious to the useror patient.

While it is very difficult to be precise in estimatingeach of these probabilities, the distinctions betweenthem can be useful. For instance, one could decide toignore the probability of a particular software defect(cause) and just focus on the severity at that level as theFDA suggests due to the difficulty of defining a credibleprobability. Yet, one could include probability as part ofthe risk of harm occurring. This could be done in theform of some relative ranking determined with inputfrom clinicians.

Another consideration regarding probabilities andrisk ratings is related to the product developmentlifecycle. Is a given risk rating determined before or after

risk control measures are identified? Probabilities ofcauses, hazards, hazardous situations, and harm aregenerally lower after risk control. After all, that is thepoint. If the risk management process is not clear on theevolving nature of risk assessment, and if riskdocumentation is not clear in this regard, there will besignificant confusion.

FDA’s off-the-shelf software guidance uses two typesof risks and establishes different pre-market submissionrequirements for pre-mitigation risk and post-mitigationrisk.

This distinction is extremely useful in establishingeffective risk management methods, avoiding confusionduring product development, and documenting thatadequate risk control measures have been implemented.6

If risk control measures are assumed rather thandocumented in response to initial pre-mitigation riskidentification, regulators and auditors may review riskmanagement documentation and believe risk control isinadequate. Without explicit documented risk analysispre- and post-risk control, internal staff can becomequickly confused when performing risk managementactivities as the project progresses. In a worst casescenario, important risk control measures areimplemented but not identified in the riskdocumentation and then modified or removed insubsequent maintenance without recognition of theirimportance to safety or effectiveness.

Types of ProbabilityBased on this discussion, we could define the probabilityof: • harm (PHarm)• a hazardous situation (PHS)• a specific cause leading to a hazardous situation (PHZ)• a specific software or hardware systematic error (i.e.,

bug, defect) (PSE) • a specific user error (PUE)

These may need further subdivision for differenttypes of users, indications for use, intended uses, ortarget populations. For instance, the probability of harmof a particular device may be much greater if the device isused on a neonatal population than for adults or if usedfor a young healthy population as opposed to a frail,elderly population.

6For example, using a low voltage DC battery rather than AC power to eliminateany chance of electrocution.

September/October 2005 305

One could use an upper or lower case P or some prefixor superscript to distinguish pre- and post-risk controlprobability or one could partition pre- and post-riskcontrol probability into explicitly labeled separatedocuments and tables or parts of specific documents andtables.

We can also define a variety of probabilities associatedwith risk control: • a patient or user detecting a problem prior to harm

(PUD)• an internal risk control measure detecting a failure or

hazard and notifying the user or patient (PRD)• a risk control measure preventing a failure from

resulting in a hazardous situation (PRP)Alternatively, rather than attempt to quantitatively

estimate the probability of a hazard after risk controlmeasures, one could qualitatively group types of riskcontrol measures into those that:• entirely eliminate the hazardous situation or its causes • make the hazardous situation or its causes highly

unlikely• make the hazardous situation or its causes less likely.

In some cases, it could be useful to establish arequirement that for hazardous situations of a certainseverity, risk control measures must fall into one of thesespecific groups as a minimum.

One could also require that software components ofhigher risk or that implement risk control measures aresubject to more rigorous design or verification methods.

One could define additional probabilities. However, itis only important to subdivide probabilities to the extentthat they actually aid in risk identification and control. Ingeneral, it is better to spend less time on refiningprobabilities and risk estimates and more timeidentifying hazardous situations, their potential causes,and effective risk control measures.

Application and ExamplesUsing some of the concepts previously discussed, onecould construct a table to summarize risk analysis andrisk control that addresses pre- and post-risk control riskassessments and internal device risk control measures aswell as external clinical usage information.

One approach that illustrates use of a variety of typesof probability is depicted in Table 1. It is neither thepurpose of this article or the author’s opinion thatextensive estimation of many levels of probabilities isalways useful or appropriate. Table 1 is provided only asan aid to illustrate the use of different types ofprobabilities. Table 2 provides a simpler example wherefewer types of probabilities are estimated but there arestill several useful distinctions; this may be the preferredapproach for many projects.

In both approaches it is important to require riskcontrol measures to be defined regardless of theprobability estimates unless these estimates areempirically defensible and lack of risk control measureswould be legally and ethically appropriate. Theprobability estimates primarily focus efforts in areaswhere harm is most probable and aid in determinatingthe adequacy of the risk control measures in relation tothese risks; probability should not be used to reducefocus on the most severe hazards even if less probable.

Designing a medical device with acceptable risk orperhaps the minimum risk possible given the state of theart requires a thorough understanding of:• clinical intended use of the product • potential harms• potential benefits of the device• target patient population and indications for use• types of intended users• ways that the device could fail• ways a user could make mistakes using the device• methods for designing effective risk control measures

that either prevent thehazards from occurring,reduce their severity, orreduce their probability.

Using informationfrom the detailed riskanalysis and summarytables, one can write anassessment of residualrisk and why it is or isnot acceptable. This

Hazardous Situation

Pre-PHS Severity Pre-HS Risk

Rating

Causes Pre-PSE, PUE

Cause Risk Rating

Pre- PHarm Risk Control

Measures

Post- PHS Residual risk rating

Could include multiple hazard and

severity columns for highest level

hazards to a breakdown of sub-

hazards if this helps identify potential causes. But all

levels of hazards should focus on

things that clearly relate to potential

harm.

Pre-risk control

probability of the

hazardous situation

Severity scale (for low risk devices

may need expansion

to distinguish amongst relatively

low severity hazards)

Pre-risk control

risk rating of

the hazardo

us situation

.

Some combination of

Pre-PHS and

severity

Software, hardware or user error.

Could

include multiple columns for sub-

causes or chains of events.

Pre-risk control

probability of the cause

Pre risk control risk rating for

each cause.

Some combination of Pre-PSE or

PUE and severity.

Probability of harm

should the hazardous situation occur.

Takes into account clinical practice and user

and patient ease of

detection - PUD

Processes, procedures or 'system controls located

within the device or its

label that affect the likelihood that the

cause will bring about

the hazardous situation

Probabilty of the

hazardous situation after risk control

measures are

defined.

Risk rating for each

hazardous situation after risk control.

Some combination of severity

and post risk control

probability of the

hazardous situation and

harm.

Table 1. Example risk table incorporating several types of probabilities

Uses and Misuses of Probability in Medical Device Risk Management

QUALITY MATTERS


final “safety case”is extremelyimportant todemonstra t ingthat appropriateexpertise has beenapplied andconvincing othersthat the specific risk management methods used weresuccessful. This is far better than arguing over thedetails of individual probability estimates, which can besubjective and difficult to defend. The textualexplanation of probability, severity, causes, hazardoussituations, and risk control measures is often moreconvincing and defensible than formulas that combinesubjective probability and severity metrics into overallrisk ratings.

While this article has focused on approaches toeffectively using probability, it is important to note thatsimply omitting probability and assuming that if a

systematic error could lead to a hazardous situation thenit will (i.e., probability=1) is an excellent approach toinitial risk analysis. This can be followed byincorporating probability to address externalmitigations by users and the effectiveness of internalrisk control measures.

Greater attention to the various types of probabilitiesand greater precision in their definition and usageshould lead to a better, common understanding andreduced philosophical disagreement on the appropriaterole probability should play in medical device riskmanagement.

Hazardous Situation Severity Causes Pre- PHarm Risk Control Measures Post- PHS Residual risk rating Could include multiple hazard

and severity columns for highest level hazards to a

breakdown of sub -hazards if this helps identify potential

causes. But al l levels of hazards should focus on

things that clearly relate to potential harm.

Severity scale (for low risk devices

may need expansion to distinguish

amongst relatively low severity

hazards)

Software, hardware or user error.

Could include

multiple columns for

sub-causes or chains of events.

Probability of harm should the

hazardous situation occur.

Takes into

account clinical practice and

user and patient ease of

detection - PUD

Risk control measures located within the

device or its label that affect the likelihood that

the cause will bring about the hazardous

situation or reduce the severity should it occur.

Probabilty of the hazardous situation after

risk control measures are

defined.

Risk rating for each hazardous situation

after risk control. Some combination of severity,

and post risk control probability of the

hazardous situation and harm.

Table 2. Simplified risk table without probabilities of causes and more focus on severity.

Alan Kusinitz

QUALITY MATTERS

FDA Home Page | CDRH Home Page | Search | CDRH A-Z Index | Contact CDRH

510(k)

| Registration | Listing | Adverse Events

| PMA | Classification | CLIA

CFR Title 21

| Advisory Committees

| Assembler | Recalls | Guidance | Standards

New Search Back To Search Results

Recognized Consensus Standards

Recognition List Number: 006 Effective Date: 10/01/2001 Part B: SUPPLEMENTARY INFORMATION Item 7: AAMI / ANSI SW68:2001, Medical device software - Software life cycle processes. (Software) Date of Standard: 2001. Addresses of Standards Organizations: ASSOCIATION FOR THE ADVANCEMENT OF MEDICAL INSTRUMENTATION (AAMI) 1110 N. GLEBE ROAD SUITE 220 ARLINGTON, VA 22201-5762 AMERICAN NATIONAL STANDARDS INSTITUTE (ANSI) 11 WEST 42ND STREET NEW YORK, NY 10036 CDRH Office and Division Associated with Recognized Standards: OFFICE OF DEVICE EVALUATION (ODE)

Devices Affected:

All medical devices containing software

Processes Affected:

510(K), IDE, PMA, HDE

Type of Standard: Horizontal, National

Page 1 of 4FDA > CDRH > Recognized Consensus Standards

12/21/2005http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfStandards/Detail.CFM?STANDAR...

Extent of Recognition:

The attached table defines the relationship between the standard AAMI SW68 and the ODE Guidance for the Content of Pre-market Submissions for Medical Devices Containing Software. Hereon known as the "guidance". The table uses the following two terms: "See Guidance" and "DON'T SUBMIT" defined as follows: "See Guidance" - For the indicated software documentation the manufacturer would normally follow the recommendations contained in the guidance document. "DON'T SUBMIT" - If the manufacturer declares conformance to this standard, the manufacturer is NOT required to submit the indicated software documentation. Whenever AAMI SW68 is used, the medical device manufacturer must document all conformance declaration information in the pre-market submission. CDRH recognizes that no two medical devices are the same, and that it may be appropriate for manufacturers to tailor a standard for their particular needs. However, if the standard is tailored, then the manufacturer is expected to submit a tailoring report in the pre-market submission. The tailoring report should identify specifically (section-by-section, and line-by-line if necessary) which parts of the standard have been modified or deleted. The tailoring report will be reviewed to establish that the manufacturer's application of the standard is adequate for the specific device under review. CDRH has the right to deny recognition to an improperly tailored standard.

SOFTWARE DOCUMENTATION LEVEL OF CONCERN

Minor Moderate Major


See Guidance

See Guidance

See Guidance


See Guidance

See Guidance

See Guidance


See Guidance

See Guidance

See Guidance

Architectural Design Don't Submit

Don't Submit

See Guidance

Design Specification Don't Submit

Don't Submit

See Guidance



Database Updated November 9, 2005

Traceability Don't Submit

Don't Submit

See Guidance

Development Don't Submit

Don't Submit

Don't Submit

Validation, Verification and Testing (VV&T)

Don't Submit

Don't Submit

See Guidance

Revision History Don't Submit

Don't Submit

See Guidance

Unresolved Anomalies

Don't Submit

Don't Submit

See Guidance

Release Version Number

See Guidance

See Guidance

See Guidance

Related CFR Citations and Product Codes:

Other: VARIOUS

Relevant Guidance:

- ODE Guidance for the Content of Pre-market Submissions for Medical Devices Containing Software - Off-the-Shelf Software Use in Medical Devices - Guidance for Industry: General Principles of Software Validation (draft)

FDA Technical Contact:

John F. Murray FDA/CDRH/OC/DOEB 2094 GAITHER ROAD, HFZ-340 Rockville MD 20850 240/276-0284 Email: [email protected]



SoftwareCPR FDA Software Standards Recognition Quick Reference September 2002

The table below provides a comparison of the submission benefits of conformance to each of the FDA recognized software standards. Blank cells indicate that conformance to standards does not affect the submission software documentation requirement. Entry of a standard in a cell indicates that conformance to that standard eliminates the need to provide that documentation in the submission. For instance conformance to AAMI SW68 would eliminate the need to provide Architectural Design information in a submission for a minor or moderate level of concern software and many other items.






Architectural Design

AAMI SW68 UL 1998 12207

AAMI SW68 UL 1998 12207

UL 1998

Design Specification

AAMI SW68 UL 1998 12207

AAMI SW68 UL 1998 12207

UL 1998

Traceability

AAMI SW68 UL 1998 12207

AAMI SW68 UL 1998 12207

Development

AAMI SW68 UL 1998 12207 IEEE 1074

AAMI SW68 UL 1998 12207 IEEE 1074

AAMI SW68 UL 1998 12207


AAMI SW68 UL 1998 12207 IEEE 1012

AAMI SW68 UL 1998 12207 IEEE 1012

Revision History AAMI SW68 AAMI SW68

Unresolved Anomalies AAMI SW68 AAMI SW68

Release Version Number Clearly, AAMI SW68 provides the maximum reduction in submission documentation for minor and moderate level of concern software and is the only standard which allows omission of a list of unresolved anomalies. Note that FDA participated in development of SW68. For major level of concern software UL 1998 most reduces the documentation to be submitted.

Note that the documentation requirements for ISO/IEC 12207 and IEEE/EIA 12207.0 are identical so no separate entries are made for 12207.0. Disclaimer: When preparing a submission or deciding on standards conformance the detailed current FDA recognition information on the FDA web site should be reviewed thoroughly along with the FDA software submission guidance.

SoftwareCPR FDA Software Standards Recognition Quick Reference September 2002

The table below provides a comparison of the submission benefits of conformance to each of the FDA recognized software standards. Blank cells indicate that conformance to standards does not affect the submission software documentation requirement. Entry of a standard in a cell indicates that conformance to that standard eliminates the need to provide that documentation in the submission. For instance conformance to AAMI SW68 would eliminate the need to provide Architectural Design information in a submission for a minor or moderate level of concern software and many other items.






Architectural Design

AAMI SW68 UL 1998 12207

AAMI SW68 UL 1998 12207

UL 1998

Design Specification

AAMI SW68 UL 1998 12207

AAMI SW68 UL 1998 12207

UL 1998

Traceability

AAMI SW68 UL 1998 12207

AAMI SW68 UL 1998 12207

Development

AAMI SW68 UL 1998 12207 IEEE 1074

AAMI SW68 UL 1998 12207 IEEE 1074

AAMI SW68 UL 1998 12207


AAMI SW68 UL 1998 12207 IEEE 1012

AAMI SW68 UL 1998 12207 IEEE 1012

Revision History AAMI SW68 AAMI SW68

Unresolved Anomalies AAMI SW68 AAMI SW68

Release Version Number Clearly, AAMI SW68 provides the maximum reduction in submission documentation for minor and moderate level of concern software and is the only standard which allows omission of a list of unresolved anomalies. Note that FDA participated in development of SW68. For major level of concern software UL 1998 most reduces the documentation to be submitted.

Note that the documentation requirements for ISO/IEC 12207 and IEEE/EIA 12207.0 are identical so no separate entries are made for 12207.0. Disclaimer: When preparing a submission or deciding on standards conformance the detailed current FDA recognition information on the FDA web site should be reviewed thoroughly along with the FDA software submission guidance.

Rev date 4/5/02 www.softwarecpr.com

SoftwareCPR® Copyright 2002 Page 1 of 4

CRISIS PREVENTION AND RECOVERY, LLCSOFTWARECPR


Winchester, MA 01890 781-721-2921

SoftwareCPR Training Aide - AAMI/ANSI SW68 Documentation Requirements.

Copyright © Copyright 2002 Crisis Prevention and Recovery, LLC. (CPRLLC), all rights reserved. SoftwareCPR is a division of Crisis Prevention and Recovery, LLC and the SoftwareCPR logo is a registered trademark. SoftwareCPR authorizes its clients and SoftwareCPR.com subscribers use of this document for internal review and training. Any other use or dissemination of this document is expressly prohibited without the written authorization of SoftwareCPR. Legal Disclaimer The training document example that follows should only be applied in the appropriate context with oversight by regulatory and software professionals with direct knowledge and experience with the topics presented. The document should not be used as a cookbook since it is not adequate for some situations and may be excessive for others. While SoftwareCPR attempts to assure the accuracy of information presented no guarantees are made since regulatory interpretations and enforcement practices are constantly changing, and are not entirely uniform in their application. Disclaimer of Warranties: The information is provided AS IS, without warranties of any kind. CPRLLC does not represent or warrant that any information or data provided herein is suitable for a particular purpose. CPRLLC hereby disclaims and negates any and all warranties, whether express or implied, relating to such information and data, including the warranties of merchantability and fitness for a particular purpose. The ANSI/AAMI SW68 Standard uses the terms “document” and “establish.” Use of either of these terms indicates that documentation is required. The term “establish” is defined in the foreword as:

‘establish’ means to define, document, and implement. Where `establish` is used in relation to processes and activities it implies documented plans or procedures. The standard mentions the word “document” in 27 sections and the word “establish” in 20 sections. There are a total of 441 sections where documentation is required2 for devices where software that can cause a hazard or controls risk (Section 1.3.1). For devices where software cannot cause a hazard and that does not control risk (Section 1.3.2) there are 19 sections where documentation is required.

1 Note: In three sections the words “document” and “establish” were both mentioned, and therefore the total is three less then if you added them individually. 2 Note: SW68 does not specify specific documents, procedures, or their organization. More than 1 documentation requirement can be met in a single document, plan or procedure.



The table below lists sections and text indicating documentation requirements. Note that documentation requirements from the verification process (Section 6.4) are listed with the primary process section to which they relate. SW68 Section Documentation Requirement

5.1.1 Development Process Implementation

5.1.1.1 The software life cycle model and mapping of activities and tasks shall be documented. 5.1.1.2 (6.4.2.1) Each phase transition shall be approved and this approval and the date of the transition shall be documented. 5.1.1.3 The developer shall establish plan(s) for conducting the activities of the software development process. The

plan(s) shall include, as appropriate, specific computer programming languages, standards, methods, tools, actions, and responsibility associated with the development and verification of all requirements including safety and security.

5.1.2 Software Requirements Analysis

5.1.2.1 The developer shall establish software requirements 5.1.2.2 The developer shall establish software requirements for each OTS software component in addition to those

listed in 5.1.2.1. 5.1.2.5 (6.4.2.2) The results of software requirements verification shall be documented. 5.1.3.1 The architecture of the medical device software shall be documented.

5.1.3 Software Architectural Design 5.1.3.2 The developer shall develop and document and architecture for the interfaces between the software items and

components external to the software items (both software and hardware) and for the interfaces between the software items.

5.1.3.3 (6.4.2.3) The software architecture verification shall be documented.

5.1.4 Software Detailed Design 5.1.4.4 The developer shall develop and document a detailed design for any interfaces between the software unit and

external components (hardware or software), and any interfaces between software units. 5.1.4.5 (6.4.2.4) The results of the design verification shall be documented.

5.1.5 Software Coding and Testing 5.1.5.1 The developer shall establish the following:

Each software unit; A strategy and methods for verifying each software unit; Procedures for verifying each software unit.

5.1.5.2 (6.4.2.5) The results of the software coding and testing verification shall be documented.

5.1.6 Software Integration and Testing 5.1.6.1 The developer shall establish an integration plan to integrate the software units and perform integration

testing. The plan shall include the approach, responsibilities and sequence. The plan shall include all OTS software components.

5.1.6.4 The integration and test results shall be documented. Integration test documentation shall include test results, anomalies found, the revision of software tested, relevant hardware and software test configurations, relevant test tools, date tested, and identification of the tester.

5.1.6.6 (6.4.2.6) The results of the software integration verification and testing shall be documented.

5.1.7 Software System Testing 5.1.7.1 The developer shall establish a plan to verify that the software system meets its requirements. The plan shall

include the approach, responsibilities and sequence. The plan shall include all OTS software components. 5.1.7.2 The developer shall establish, for each software requirement, a set of tests (inputs, expected outcomes, test

criteria) and procedures for conducting software system testing. 5.1.7.6 The software system tests, the method by which results are determined to be correct or incorrect, and the test

results shall be documented. Documented test results shall identify the software version tested, hardware, test



tools, revision level or revision date of the test cases, date tested, and identification of the tester. 5.1.7.7 (6.4.2.7) The results of the software system testing shall be documented.

5.1.8 Software Validation 5.1.8.1 The developer shall establish a software validation plan. The plan shall specify the type of tests and activities

that will be used to validate the software. The plan shall include all OTS software components. 5.1.8.4 The software validation tests, the method by which results are determined to be correct or incorrect, and the

test results shall be documented. Documented test results shall identify the software version tested, hardware, test tools, revision level or revision date of the test cases, date tested, and identification of the tester.

5.1.9 Software Release

5.1.9.2 The developer shall document the versions of the software and documentation that are being released. 5.1.9.3 The developer shall document the procedure and environment used to create the released software. 5.1.9.5 The developer shall handle, protect, store, label, package and deliver the software in accordance with

established procedures.

5.2.1 Process Implementation 5.2.1.1 The maintainer shall establish plans and procedures for conducting the activities and tasks of the

Maintenance Process. 5.2.1.2 The maintainer shall establish procedures for receiving, recording, and tracking problem reports and

modification requests. Whenever problems are encountered, they shall be recorded and entered into the Problem Resolution (6.5).

5.2.1.3 The maintainer shall establish procedures to evaluate and implement upgrades, bug fixes and patches for OTS software.

5.2.2 Problem and Modification Analysis

5.2.2.1 The maintainer shall analyze the problem report or modification request for its impact on the organization, existing supported systems, and the interfacing systems for the following: Type; for example, corrective, improvement, preventive, or adaptive to new environment; Scope; for example, size of modification, number of device models impacted, supported accessories, impacted, resources involved, time to modify; Criticality; for example, impact on performance, safety, or security. The analysis shall be documented.

5.2.2.3 The maintainer shall conduct analysis and determine which documentation, software units, and versions thereof need to be modified or deleted, and if any new items will be produced. New, modified and deleted items shall be documented.

5.2.2.6 The maintainer shall document the problem/modification request, the analysis results, the implementation options, and the selected and approved option.

5.2.3 Modification Implementation

5.2.3.1 The maintainer shall utilize the Development Process (5.1) or an established maintenance process to implement the modifications a.) Test and evaluation criteria for testing and evaluating the new and modified items (software units, components, and configuration items) of the system shall be defined and documented. b.) The rationale for the amount of regression testing necessary to ensure that the unmodified items of the system were not affected shall be documented.

5.2.3.2 The complete and correct implementation of the new and modified requirements shall be ensured. It also shall be ensured that the original, unmodified requirements were not affected and that all hazard mitigations (whether intentionally modified or not) remain effective and are not unintentionally compromised. The test results shall be documented.

6.1.1 Process Implementation

6.1.1 The software developer shall establish a plan to conduct the activities and tasks of the software hazard management process. All OTS software shall be included in the software hazard management process.



6.1.3 Mitigation of Hazards

6.1.3 Hardware, software, working environment or user instruction risk control measures shall be defined, documented and reviewed for adequacy.

6.1.4 Verification of Mitigations

6.1.4.1 Each of the mitigations shall be verified and this verification shall be documented. 6.1.4.2 There shall be documented traceability of software hazards as appropriate:

From the hazard to the software functionality; From the software functionality to the specific software cause or failure; From the software cause or failure to the mitigation; and From the mitigation to the verification of the mitigation.


6.2.1 A plan, identifying the documents to be produced during the life cycle of the software product, shall be established. For each identified document, the following shall be addressed: a.) Title or name; b.) Purpose; c.) Intended audience of document; d.) Documentation standards applicable for this document; and e.) Procedures and responsibilities for development, review, approval, modification, and configuration management.

6.2.3 Maintenance

6.2.3.1 Change Control procedures shall be established in accordance with the Configuration Management Process (6.3).


6.3.1.1 A configuration management plan shall be established. The plan shall describe: the items to be controlled, the configuration management activities; procedures and schedule for performing these activities; the organization(s) responsible for performing configuration management and activities; and their relationship with other organizations, such as software development or maintenance.

6.3.1.2 The developer shall establish procedures to control the receipt, installation, and acceptance of each OTS software component.

6.3.2 Configuration Identification

6.3.2 A scheme shall be established for the unique identification of software configuration items and their versions to be controlled for the project. This scheme shall include OTS software components. For each software configuration item and its versions, the documentation that defines the baseline and the version references shall be identified.

6.3.3 Configuration Control

6.3.3.1 The following shall be performed: identification and recording of change requests; analysis and evaluation of the changes; approval or disapproval of the request; and implementation, verification, and release of the modified software item. Each upgrade, bug fix or patch for OTS software shall be evaluated and the evaluation documented.


6.5.1 A problem resolution process shall be established for handling all known problems (including non-conformance) detected in the software products and activities. Plans or procedures shall specify at each stage of the life cycle the aspects of the problem resolution process that will be formal and documented

SoftwareCPR Copyright 2002 Rev Date 4/3/02


ANSI/AAMI SW68-2001: Medical device software – Software life cycle processes

SoftwareCPR Conformance Assessment Job Aide

Purpose This document is a software assessment job aide for assessors qualified to perform conformance

assessments for the ANSI/AAMI SW68-2001 recognized by the US FDA. It serves as a checklist/memory jogger and provides space to record notes if desired.

Usage • This assessment aide serves as an informal checklist and assessor job aide for reference.

• It is not to be used as a mindless checklist. • Not all questions are relevant to all interviewees and projects. • Questions are to be spread over a number of interviewees and not all asked of each

individual. • An assessment would not normally proceed in the order of the sections of this job aide

and many issues would be addressed at one time with the appropriate individual(s) • Notes can be kept in the spaces provided or separately as preferred. • THIS IS ONLY A GUIDE and REMINDER and does not reflect on the form or content

of any final report or recommendations. • THIS DOES NOT SERVE as a list of best practices or a standard assessment scheme. • THIS IS NOT FOR DISTRIBUTION BEYOND CLIENTS AND

SOFTWARECPR.com Subsbcribers.

SoftwareCPR Copyright 2002 Rev Date 4/3/02 Page 2 of 30

®


Winchester, MA 01890 781-721-2921

Copyright © Copyright 2002 Crisis Prevention and Recovery, LLC. (CPRLLC), all rights reserved. SoftwareCPR is a division of Crisis Prevention and Recovery, LLC and the SoftwareCPR logo is a registered trademark. SoftwareCPR authorizes its clients and SoftwareCPR.com subscribers use of this document for internal review and training. Any other use or dissemination of this document is expressly prohibited without the written authorization of SoftwareCPR. Legal Disclaimer The training document example that follows should only be applied in the appropriate context with oversight by regulatory and software professionals with direct knowledge and experience with the topics presented. The document should not be used as a cookbook since it is not adequate for some situations and may be excessive for others. While SoftwareCPR attempts to assure the accuracy of information presented no guarantees are made since regulatory interpretations and enforcement practices are constantly changing, and are not entirely uniform in their application. Disclaimer of Warranties: The information is provided AS IS, without warranties of any kind. CPRLLC does not represent or warrant that any information or data provided herein is suitable for a particular purpose. CPRLLC hereby disclaims and negates any and all warranties, whether express or implied, relating to such information and data, including the warranties of merchantability and fitness for a particular purpose.


SoftwareCPR Copyright 2002 Rev Date 4/3/02 Page 3 of 30

Table of Contents 5 Primary Life Cycle Processes ......................................................................4

5.1 Development Process ..................................................................................................................... 4 5.1.1 Process Implementation........................................................................................................... 4 5.1.2 Software Requirements Analysis............................................................................................. 5 5.1.3 Software Architectural Design ................................................................................................ 7 5.1.4 Software Detailed Design ........................................................................................................ 8 5.1.5 Software Coding and Testing .................................................................................................. 9 5.1.6 Software Integration and Testing........................................................................................... 11 5.1.7 Software System Testing ....................................................................................................... 12 5.1.8 Software Validation ............................................................................................................... 15 5.1.9 Software Release ................................................................................................................... 16

5.2 Maintenance Process .................................................................................................................... 17 5.2.1 Process Implementation......................................................................................................... 17 5.2.2 Problem and Modification Analysis ...................................................................................... 18 5.2.3 Modification Implementation ................................................................................................ 19

6 Support Processes .....................................................................................20 6.1 Software Hazard Management Process ........................................................................................ 20

6.1.1 Process Implementation......................................................................................................... 20 6.1.2 Software Hazard Analysis ..................................................................................................... 20 6.1.3 Mitigation of Hazards ............................................................................................................ 21 6.1.4 Verification of Mitigations .................................................................................................... 21 6.1.5 Hazard Management of Software Changes ........................................................................... 22

6.2 Documentation Process ................................................................................................................ 23 6.2.1 Process Implementation......................................................................................................... 23 6.2.2 Design and Development....................................................................................................... 23 6.2.3 Maintenance........................................................................................................................... 24

6.3 Configuration Management Process............................................................................................. 24 6.3.1 Process Implementation......................................................................................................... 24 6.3.2 Configuration Identification .................................................................................................. 25 6.3.3 Configuration Control............................................................................................................ 25 6.3.4 Configuration Status Accounting .......................................................................................... 26

6.4 Verification Process...................................................................................................................... 26 6.4.1 Process Implementation......................................................................................................... 26

6.5 Problem Resolution Process ......................................................................................................... 27 6.5.1 Process Implementation......................................................................................................... 27 6.5.2 Problem Resolution ............................................................................................................... 29

Result column: P= Planned, S= some aspects, �=complete Documentation Status Column: P= planned, D= drafted, �=complete (and approved if approval required)

Page 4 of 30

5 Primary Life Cycle Processes

5.1 Development Process

5.1.1 Process Implementation SW 68 Section Conformity Requirements Result Document Status Assessor Notes and Observations

5.1.1.1 Software life cycle definition appropriate to scope

Required

5.1.1.1 Software development mapped onto life cycle model

Required

5.1.1.2 (6.4.2.1) Implementation of life cycle model:

Required

a.) Deliverables verified

b.) Phase transition approved and documented

5.1.1.3 Plan for software development process Required

5.1.1.3 Included in plan computer programming languages,

-Standards,

-Methods,

-Tools,


Page 5 of 30

-Actions,

-Responsibility of all requirements

5.1.1.4 Activities to manage OTS software

5.1.2 Software Requirements Analysis Section Conformity Requirements Result Documentation Status Assessor Notes and Observations

5.1.2.1 Establish software requirements

Required

a.) Functional and capability

b.) Interfaces external to software

c.) Safety requirements

d.) Alarms

e.) Security requirements

f.) Human-factors

g.) Data definition


Page 6 of 30

h.) Installation at operation and maintenance site

i.) User documentation developed

j.) User operation

k.) User maintenance

l.) Required national or international standard

5.1.2.2 Establish software requirements for each OTS

Required

a.) Title and manufacturer

b.) Version level, release date, patch number, upgrade designation

c.) System hardware and software

d.) Support risk management process

e.) Interfaces to OTS software Safety functions

5.1.2.3 (5.1.2.1 and 5.1.2.2) Uniquely identified and traceable to verification and validation testing

5.1.2.4 (4.2.1 and A.4.2) Traceable to risk control measures in ISO 14971


Page 7 of 30

5.1.2.5 (6.4.2.2) Software requirements verification

Required

a.) Consistency

b.) Testability

c.) Traceability to device requirements

d.) Feasibility of design

e.) Hazard mitigation

f.) Feasibility of operation and maintenance

g.) Traceability to hazard analysis and verification

5.1.3 Software Architectural Design Section Conformity Requirements Result Documentation Status Assessor Notes and Observations

5.1.3.1 Describe medical device software structure

Required

5.1.3.2 Document architecture for interfaces

Required

5.1.3.3 (6.4.2.3) Software architecture verification

Required


Page 8 of 30

a.) Adequacy for hazard mitigation

b.) Internal consistency Feasibility of interfaces

5.1.4 Software Detailed Design Section Conformity Requirements Result Documentation Status Assessor Notes and Observations

5.1.4.1 Define software units

5.1.4.1 Software units testable

5.1.4.2 Refine software item into lower levels

5.1.4.3 Develop detailed design for software unit

5.1.4.4 Develop and document a detailed design for interfaces between software unit and external components and any interface between software units

Required

5.1.4.5 (6.4.2.4) Verify software detailed design

Required

a.) Coverage of requirements

b.) External consistency with architectural design

c.) Internal consistency between software units


Page 9 of 30

d.) Appropriateness of design methods and standard use

e.) Adequacy of hazard mitigation

f.) Feasibility of testing

g.) Feasibility of operation and maintenance

h.) Implementation of intended events, inputs, outputs, interfaces, logic flow, allocation of timing and sizing budgets, error definition, isolation and recovery

i.) Default state defined

j.) Initialization of variables and memory management

k.) cold and warm resets and stand-by

5.1.5 Software Coding and Testing Section Conformity Requirements Result Documentation Status Assessor Notes and Observations

5.1.5.1 a.) Establish each software unit

Required

b.) Strategy and methods for verifying

c.) Procedures for verifying


Page 10 of 30

5.1.5.2 (6.4.2.5) Verify software code and test procedures as follows:

Required

a.) Code is correct and compliant with requirements

b.) Code correctly implements safety requirements

c.) Consistency with interfaces documented in detailed design

d.) Appropriateness of coding methods and standards uses

e.) Conformance with programming procedures

f.) Appropriateness of verification strategies

g.) Correctness of test procedures

h.) Feasibility of software integration and testing

i.) Maintainability

j.) Proper event sequence

k.) Correct data and control flow

l.) Appropriate allocation timing and sizing budgets


Page 11 of 30

m.) Fault handling

n.) Initialization of variables

o.) Self-diagnostics

p.) Memory management and overflows

q.) Boundary conditions

5.1.6 Software Integration and Testing Section Conformity Requirements Result Documentation Status Assessor Notes and Observations

5.1.6.1 Establish integration plan to integrate software units and perform testing

Required

5.1.6.1 Included in plan the approach and responsibilities sequence

5.1.6.1 Include all OTS software components

5.1.6.2 Integrate software units and test as the aggregates are developed in accordance with integration plan

5.1.6.3 Regression testing

5.1.6.4 Integration test documentation include test results,

Required


Page 12 of 30

-Anomalies found,

-Revision of software tested,

-Relevant hardware and software test configurations,

-Relevant test tools,

-Date tested

-Identification of tester

5.1.6.5 Include in tests test cases which expose software behavior

5.1.6.6 (6.4.2.6) Developer verify integration as follows:

Required

a.) Software units integrated into software item

b.) Hardware and software items and support for manual operations have been integrated in system

c.) Integration tasks have been performed in accordance with an integration plan

5.1.7 Software System Testing Section Conformity Requirements Result Documentation Status Assessor Notes and Observations


Page 13 of 30

5.1.7.1 Plan to verify software system meets requirements

Required

5.1.7.1 Included in plan is approach,

-Responsibilities,

-Sequence,

-OTS software components

5.1.7.2 Establish for each software requirement a set of tests and procedures for conduction software system testing

Required

5.1.7.3 For medical devices required to comply with national or international standards, relevant software system testing to demonstrate compliance

5.1.7.4 Errors detected during testing logged,

-Classified,

-Reviewed,

-Resolved prior to release

5.1.7.5 When changes are made during software system testing regression testing conducted to assure no unintended side effects


Page 14 of 30

5.1.7.6 Software system tests documented

Required

5.1.7.6 Documented test results shall identify the software or hardware version tested,

-Test tools,

-Revision level or revision date,

-Date tested,


5.1.7.7 (6.4.2.7) Developer verify software system as follows:

Required

a.) Traceabilty to software requirements

b.) Static test coverage of the code

c.) Conformance to expected results

d.) Feasibility of medical device system integration and testing

e.) Effectiveness of modifications made to correct anomalies

f.) Maintainability


Page 15 of 30

5.1.8 Software Validation Section Conformity Requirements Result Documentation Status Assessor Notes and Observations

5.1.8.1 Establish software validation plan

Required

5.1.8.1 Specify in plan test type and activities that will be used to validate software

5.1.8.1 Include in plan OTS software components

5.1.8.2 Prepared test requirements,

-Test cases,

-Test specifications

5.1.8.3 Test requirements, test cases, test specifications to reflect requirements for specific intended use,

-Known types of intended users in their intended environment,

-Range of intended platforms

5.1.8.4 Software validation tests documented

Required

5.1.8.4 Documented tests identify software or hardware version tested,

Required


Page 16 of 30

-Test tools,

-Revision level or date of test cases,

-Date tested,


5.1.9 Software Release Section Conformity Requirements Result Documentation Status Assessor Notes and Observations

5.1.9.1 Ensure software testing has been completed and results evaluated

5.1.9.2 Document versions of software and documentation being released

Required

5.1.9.3 Document procedure and environment used to created released software

Required

5.1.9.4 Archive and maintain master copies of software,

-Source code

-Documentation for the life of device

5.1.9.5 Handle, protect, store, label, package and deliver software in accordance with established procedures

Required


Page 17 of 30

5.2 Maintenance Process

5.2.1 Process Implementation Section Conformity Requirements Result Documentation Status Assessor Notes and Observations

5.2.1.1 Establish plans conducting activities and tasks of the Maintenance Process

Required

5.2.1.2 Establish procedures for receiving,

Required

-Recording,

-Tracking problem reports and modification requests

5.2.1.2 Record problems and enter into the Problem Resolution Process (6.5)

5.2.1.3 Establish procedures to evaluate and implement upgrades,

Required

-Bug fixes,

-Patched for OTS software

5.2.1.4 Implement the Configuration Management Process (6.3) for managing modification to existing system


Page 18 of 30

5.2.2 Problem and Modification Analysis Section Conformity Requirements Result Documentation Status Assessor Notes and Observations

5.2.2.1 Analyze the problem report for its impact on organization, existing supported systems, interfacing systems for the following:

Required

a.) Type

b.) Scope

c.) Criticality

5.2.2.2 Verify reported problems

5.2.2.3 conduct analysis and determine which documentation need to be modified or deleted and if any new items will be produced

5.2.2.3 New, modified and deleted items documented

Required

5.2.2.4 Consider options for implementing modification

5.2.2.5 Modification option reviewed and approved

5.2.2.6 Document problem/modification request,

Required

-Analysis request,


Page 19 of 30

-Implementation options,

-Selected and approved option

5.2.3 Modification Implementation Section Conformity Requirements Result Documentation Status Assessor Notes and Observations

5.2.3.1 Utilize Development Process (5.1) or an established maintenance process to implement modifications Included in the requirements process:

Required

a.) Document test and evaluation criteria for testing new and modified items of the system

Required

b.) Document rationale for amount of regression testing necessary to ensure unmodified items of system were not affected

Required

5.2.3.2 Complete implementation of new and modified requirements ensured

5.2.3.2 Original, unmodified requirements were not affected and hazard mitigations not compromised.

Required


Page 20 of 30

6 Support Processes

6.1 Software Hazard Management Process


6.1.1 Establish plan to conduct activities of software hazard management process

Required

6.1.1 OTS software included

6.1.2 Software Hazard Analysis Section Conformity Requirements Result Documentation Status Assessor Notes and Observations

6.1.2.1 Identify software functionality whose failure could result in hazards identified in medical device risk analysis activity of ISO 14971 (4.2.1)

6.1.2.1 Include hazards that could be result of software failure or for which software implements risk control measure

6.1.2.2 Identify potential causes for failure of software functionality identified above

6.1.2.2 Consider the following potential causes:

a.) Software defects in identified functionality

b.) Failure or unexpected results from OTS software


Page 21 of 30

c.) Hardware failures or other software defects that could result in unpredictable operation

6.1.3 Mitigation of Hazards Section Conformity Requirements Result Documentation Status Assessor Notes and Observations

6.1.3 Define, Document and review hardware,

Required

-Software,

-Working environment,

-User instruction risk control measures

6.1.4 Verification of Mitigations Section Conformity Requirements Result Documentation Status Assessor Notes and Observations

6.1.4.1 Mitigations verified and documented

Required

6.1.4.2 Documented traceability of software hazards as follows:

Required

a.) From hazards to software functionality

b.) From software functionality to specific software causes of failure

c.) From software cause of failure to mitigation


Page 22 of 30

d.) From mitigation to verification of mitigation

6.1.5 Hazard Management of Software Changes Section Conformity Requirements Result Documentation Status Assessor Notes and Observations

6.1.5.1 Changes to medical device analyzed to determine if:

a.) Any new software mitigations for hazards are required

b.) Any new hazards that could be caused by software added

c.) Any new software causes for existing hazards introduced

6.1.5.2 Changes to software and OTS software analyzed to determine if software modification could interfere with risk control measures

6.1.5.3 Relevant hazard management activities defined in (6.1.2), (6.1.3), and (6.1.4) performed


Page 23 of 30

6.2 Documentation Process


6.2.1 Establish a plan that identifies documents to be produced during life cycle of software product

Required

6.2.1 For each identified document address:

a.) Title or name

b.) Purpose

c.) Intended audience

d.) Documentation standards applicable for document

Required

e.) Procedures for development, review, approval, modification and configuration management

6.2.2 Design and Development Section Conformity Requirements Result Documentation Status Assessor Notes and Observations

6.2.2.1 Design and develop each identified document in accordance with applicable documentation standards defined in plan

6.2.2.2 Prepared documents reviewed against their documentation standards and approved as defined


Page 24 of 30

6.2.3 Maintenance Section Conformity Requirements Result Documentation Status Assessor Notes and Observations

6.2.3.1 Establish Change Control procedures in accordance with Configuration Management Process (6.3)

Required

6.2.3.2 Required tasks to be performed when documentation is modified performed (5.2)

6.3 Configuration Management Process


6.3.1.1 Establish configuration management plan

Required

6.3.1.1 Plan describes:

-Items to be controlled

-Configuration management activities

-Procedures and schedule for performing activities

-Organization(s) responsible for performing configuration management activities


Page 25 of 30

-Their relationship with other organizations

6.3.1.2 Establish procedures to control receipt,

Required

-Installation

-Acceptance of OTS software component

6.3.2 Configuration Identification Section Conformity Requirements Result Documentation Status Assessor Notes and Observations

6.3.2 Scheme established for identification of software configuration and versions

Required

6.3.2 OTS software components included in scheme

6.3.2 For software configuration item and its version, documentation that defines baseline version references identified

6.3.3 Configuration Control Section Conformity Requirements Result Documentation Status Assessor Notes and Observations

6.3.3.1 Perform the following:

-Identification and recording of change requests,


Page 26 of 30

-Analysis and evaluation of changes,

-Approval or disapproval of request,

-Implementation, verification and release of modified software item

6.3.3.1 Evaluate and document each upgrade and bug fix or patch for OTS software

Required

6.3.3.2 Audit trail exists and can trace each modification,

-Reason for modification,

-Authorization of modification

6.3.4 Configuration Status Accounting Section Conformity Requirements Result Documentation Status Assessor Notes and Observations

6.3.4.Records of history of controlled items including software baseline retrievable

6.4 Verification Process


Formal verification process established Required


Page 27 of 30

Qualified group or individual assigned with verification responsibility - and independence and authority

Activities and deliverables of each phase requiring verification identified

Required

Method and tools for verification of each deliverable identified

Required

The remaining sub-sections of 6.4.2 correspond to each life cycle process. There should be some verification for each process and its deliverables.

6.5 Problem Resolution Process


6.5.1. Problem resolution process established for handling problems detected in software products

Required

6.5.1 Procedures specify each stage of life cycle aspects of problem resolution process that will be formal and documented

Required

6.5.1 Process complies with following requirements:

a.) Closed-loop process ensuring that:

-All detected problems reported and entered into Problem Resolution Process,

-Action initiated on them,


Page 28 of 30

-Relevant parties advised of existence of problem,

-Causes identified, analyzed and eliminated,

-Resolution and disposition achieved,

-Status tracked and reported,

-Records of problems maintained

b.) Include in process evaluation of possible relevance to safety as specified in ISO 14971

c.) Analysis performed to detect trends in reported problems

d.) Problem resolutions and dispositions verified to determine if:

-Problems resolved,

-Adverse trends reversed,

-Changes correctly implemented in appropriate software products and activities,

-Determine whether additional problems introduced


Page 29 of 30

6.5.2 Problem Resolution Section Conformity Requirements Result Documentation Status Assessor Notes and Observations

6.5.2 When problems detected in software product or activity, [problem report prepared

6.5.2 Problem report used as part of closed-loop process described above:

-From detection of problem,-Through investigation,

-Analysis and resolution of problem and cause,

- trend detection across problems

END OF CHECKLIST

Page 30 of 30

®


Winchester, MA 01890 781-721-2921

Copyright © Copyright 2002 Crisis Prevention and Recovery, LLC. (CPRLLC), all rights reserved. SoftwareCPR is a division of Crisis Prevention and Recovery, LLC and the SoftwareCPR logo is a registered trademark. SoftwareCPR authorizes its clients and SoftwareCPR.com subscribers use of this document for internal review and training. Any other use or dissemination of this document is expressly prohibited without the written authorization of SoftwareCPR. Legal Disclaimer The training document example that follows should only be applied in the appropriate context with oversight by regulatory and software professionals with direct knowledge and experience with the topics presented. The document should not be used as a cookbook since it is not adequate for some situations and may be excessive for others. While SoftwareCPR attempts to assure the accuracy of information presented no guarantees are made since regulatory interpretations and enforcement practices are constantly changing, and are not entirely uniform in their application. Disclaimer of Warranties: The information is provided AS IS, without warranties of any kind. CPRLLC does not represent or warrant that any information or data provided herein is suitable for a particular purpose. CPRLLC hereby disclaims and negates any and all warranties, whether express or implied, relating to such information and data, including the warranties of merchantability and fitness for a particular purpose.



Winchester, MA 01890 781-721-2921

Copyright © Copyright 2005 Crisis Prevention and Recovery, LLC. (CPRLLC), all rights reserved. SoftwareCPR is a division of Crisis Prevention and Recovery, LLC and the SoftwareCPR® logo is a registered trademark. SoftwareCPR® authorizes its clients and SoftwareCPR®.com subscribers use of this document for internal review and training. Any other use or dissemination of this document is expressly prohibited without the written authorization of SoftwareCPR. Individual original FDA documents in their original form without SoftwareCPR® annotations are public information and may be shared without restriction. Legal Disclaimer The training document that follows should only be applied in the appropriate context with oversight by regulatory and software professionals with direct knowledge and experience with the topics presented. The document should not be used as a cookbook since it is not adequate for some situations and may be excessive for others. While SoftwareCPR® attempts to ensure the accuracy of information presented, no guarantees are made since regulatory interpretations and enforcement practices are constantly changing, and are not entirely uniform in their application. Disclaimer of Warranties: The information is provided AS IS, without warranties of any kind. CPRLLC does not represent or warrant that any information or data provided herein is suitable for a particular purpose. CPRLLC hereby disclaims and negates any and all warranties, whether express or implied, relating to such information and data, including the warranties of merchantability and fitness for a particular purpose.