1

DOE O 414.1C, Quality Assurance “Improving Safety Software Quality”

Frank Russo

Deputy Assistant Secretary

Corporate Performance Assessment (EH-3)

2

Welcome

3

Video Conference Ground Rules Mute remote locations Silence cell phones Hold questions until the end Allow us to defer your question if it will be

covered during the FAQs

4

DOE O 414.1C General Information Video Conference Agenda Welcome (F. Russo)

DOE Expectations for Safety and Quality (R. Shearer) Background - Order and Guide Development (F. Russo) Safety Software Quality Responsibilities (F. Russo) Safety Software Requirements (B. Danielson) Field Experiences (C. Lagdon) PSO Perspective (L. Dever, E. Schmidt, R. Singh, J. Poppiti,

M. Janaskie) Path Forward (R. Loesch) Q&A (D. Sparkman, C. Lagdon, C. Thayer)

5

DOE Expectations for Safety and Quality

Russell Shearer

Principle Deputy Assistant Secretary

Environment, Safety and Health

6

Background

7

Background

Importance of Software Quality Assurance as part of a Quality Assurance Program

New and revised Quality Assurance requirements in DOE directives

Majority of new requirements are for software quality assurance

First phase of a Department-wide rollout

8

Order and Guide Development DOE EH-31 responsible for QA Order requirements

Surveyed applicable industry standards and guidance Consulted DOE organizations, field staff, and SMEs

Working groups established for development of Guides Wide spectrum of individuals Selected based upon DOE organization, site location,

and SME knowledge Members authored sections in Guides Members were SME reviewers for all sections

9

Informal and Formal Review Processes Informal Reviews

SME Reviewers Defense Nuclear Facility Safety Board

Formal Process RevCom Hundreds of valuable comments

10

Thank You DOE G 414.1-2A Quality Assurance Writing Team

Bud Danielson Team LeaderDOE Office of Quality Assurance Programs

Ron FarchminWest Valley Nuclear Services Company, Inc.

Dennis K. DreyfusBechtel National, Inc.

Charles H. Moseley Writing Team Coordinator BWXT/Y-12

Roy LebelBrookhaven National Laboratory

Ron Schrotke Pacific Northwest National Laboratory

Geoffrey L. BeausoleilDOE Idaho Operations Office

David ShugarsWashington Group International-WTP

Amy EcclesineLos Alamos National Laboratory

David StroupBechtel National, Inc

11

Thank You DOE G 414.1-4 Safety Software Quality Assurance

Writing TeamDebra SparkmanTeam Leader, DOEOffice of QA Programs

Pranab Guha

DOE Office of Quality Assurance Programs

Ron Schrotke

Pacific Northwest National Laboratory

Toni Austin

Bechtel, Hanford Waste Treatment Plant

Scott Mathews

Los Alamos National Laboratory

Subir Sen

DOE Office of Quality Assurance Programs

Dwight Brayton

Fluor Government Group, Hanford

Keith Morrell Westinghouse Savannah River Company

John Shultz, DOE Office of Safeguards and Security Policy

Bud Danielson

DOE Office of Quality Assurance Programs

Kevin O’Kula Washington Safety Management Solutions

Bob StevensDOE Office of Quality Assurance Programs

12

Roles and Responsibilities

13

Order Implementation Roles and Responsibilities EH roles and responsibilities

DOE’s independent element responsible for safety of the public, worker and environment

Develops and maintains QA policy requirements, guides, and standards for all DOE work

Provides advise and assistance to DOE elements and contractors implementing DOE QA policy

Identifies and proposes resolutions for crosscutting QA issues

Manages DOE Safety Software Central Registry

14

Order Implementation Roles and Responsibilities continued PSO roles and responsibilities

Ensure HQ, field elements, and contractors implement the requirements in the QA Order

Set priorities and schedule for compliance Provide direction and resources, including

prioritization, for implementing the QA requirements

Review and approve QAPs or other documents that include implementation approaches for the QA requirements

15

Safety Software Requirements

Gustave E. (Bud) Danielson, Jr.

Quality Policy Manager

Office of Quality Assurance Programs

16

Significant Changes to DOE O 414.1C Addition of SQA for safety software

Cancels DOE N 411.1, SQA Functions, Responsibilities and Authorities

Reflect existing requirements in 10 CFR 830 Order references 1 updated and 2 new Guides

DOE G 414.1-2A, QA Management System DOE G 414.1-3, Suspect/Counterfeit Items DOE G 414.1-4, Safety Software

Inclusion of aviation safety into Corrective Action Management Program (CAMP)

17

Significant Changes to DOE G 414.1-2A Quality Management Strengthened the use of a single management

system or work process for similar requirements Incorporated review guidance for use by DOE in

evaluating contractor quality management systems

Discussed the use of third party management system validation

New generic QAP assessment plan

18

Significant Changes to DOE G 414.1-2A Quality Management continued Updated information pertaining to the identification and

control of suspect/counterfeit items (S/CIs) and links to expanded guidance for S/CIs

Expanded information on the grading process to include programmatic and mission‑critical considerations and a description of the steps in implementing the grading process

Expanded the description of identification, tracking, and resolution of quality problems

Clarified the guidance on procurement processes for nuclear safety applications

19

New Guidance with DOE G 414.1-3 Suspect & Counterfeit Items Issued November 3, 2004

Incorporates the new complex-wide S/CI Process

Updates S/CI information and resources

Supersedes DOE G 440.1-6, Suspect Counterfeit

Items Guide for use with DOE O 440.1, Worker

Protection Management; 10 CFR 830.120; and DOE

O 5700.6C, Quality Assurance

20

Safety Software Changes to DOE O 414.1C Scope of 10 CFR 830 and DOE O 414.1B

Included software related to nuclear (including

radiological) facilities

Establishes specific category of software,

SAFETY SOFTWARE

Identification of Safety Software definitions,

responsibilities and requirements

21

Safety Software Definitions Safety System Software: Software for a nuclear

facility that performs a safety function as part of a

structure, system, or component and is cited in

either (a) a DOE approved documented safety

analysis or (b) an approved hazard analysis per

DOE P 450.4, Safety Management System Policy,

dated 10‑15‑96, and the DEAR clause.

22

Safety Software Definitions continuedSafety and Hazard Analysis Software and Design

Software: Software that is used to classify,

design, or analyze nuclear facilities.  This

software is not part of a structure, system, or

component (SSC) but helps to ensure the proper

accident or hazards analysis of nuclear facilities

or an SSC that performs a safety function.

23

Safety Software Definitions continued Safety Management and Administrative

Controls Software: Software that performs a hazard control function in support of nuclear facility or radiological safety management programs or technical safety requirements or other software that performs a control function necessary to provide adequate protection from nuclear facility or radiological hazards. This software supports eliminating, limiting, or mitigating nuclear hazards to workers, the public, or the environment as addressed in 10 CFR 830, 10 CFR 835, and the DEAR ISMS clause.

24

Significant Changes to DOE O 414.1C continued Invokes ASME NQA-1-2000,supplemented by other

consensus standards or other consensus standards that provide an equivalent level of SQA requirements

Sites define grading levels and consensus standards in DOE approved QAP or other appropriate DOE approved document

Using the grading levels, select and implement the applicable software quality assurance work activities defined in the Order

Identification, documentation, and maintenance of safety software inventory

25

Significant Changes to DOE O 414.1C continued Identifies 10 SQA work activities

Software project management Software risk management Software configuration management Procurement and supplier management Software requirements identification and management Software design and implementation Software safety Verification and validation Problem reporting and corrective action Training of personnel in the design, development, use and

evaluation of safety software

26

New Guidance in DOE G 414.1-4 Safety Software Facilitates implementation of SQA

responsibilities and requirements Detail guidance for each of the10 SQA work

activities for safety software Procedures for updating, adding and removing

tool box codes from Central Registry Cross references from ASME NQA-1-2000 to

10 CFR 830 and DOE O 414.1C Sample Criteria Review and Approach

Document for DOE O 414.1C

27

Impact Update and maintain an inventory of safety software

NA and EM have initial lists Grade safety software

Many sites institutional SQA programs include graded approach

Implement 10 SQA work activities Basic SQA practices Consistent with consensus standards Map very closely to most sites institutional SQA

program practices

28

Field Experiences

Chip Lagdon

Chief of Nuclear Safety (Acting)Energy, Science and Environment

29

Lessons Learned Software Requirement Specification (SRS) and

Software Design Document (SDD) are essential for developing quality software and life cycle maintenance.

Majority of software projects did not have SRSs and SDDs Sites using the SRSs and SDDs have clear understanding of

what was needed to develop and maintain software quality. The sites without SRSs and SDDs appeared to be relying

heavily on the available experts to ensure software is developed or procured to meet the project needs.

Related SQA Work ActivitiesSoftware requirements identification and management (#5)Software design and implementation (#6)Verification and validation (#8)

30

Lessons Learned continued Software procurement specifications should

specify details of software requirements, not just catalog data. Sites procuring PLC’s for process systems only specified

the vendors’ catalog model information as procurement specifications

Supporting documentation for the suitability and applicability of the technical requirements not included

Related SQA Work ActivitiesSoftware requirements identification and management

(#5)

31

Lessons Learned continued Formal procedures for software problem reporting

and corrective actions for software errors and failures need to be maintained and rigorously implemented. Many sites resolve software errors and corrective actions

at the project level and maintain informal coordination with vendors or other effected entities.

Related SQA Work ActivitiesProcurement and supplier management (#4)

Problem reporting and corrective action (#9)

32

Lessons Learned continued Appropriate qualifications and training on

software use is essential for proper use of safety software. Very sophisticated and complex software are

being used without appropriate training in their use.

Related SQA Work ActivitiesTraining of personnel in design, development,

use and evaluation of safety software (#10)

33

Lessons Learned continued Appropriate software control and configuration

management are essential for safe use of the software. Lack of proper control has resulted in multiple versions

being available at the same time and even some with known errors.

Deficiencies have been noted with configuration control in terms of software version and documentation.

Related SQA Work ActivitiesSoftware configuration management (#3)

34

SC Perspective

Leah Dever

Associate Director for Laboratory Operations

Office of Science

35

NNSA Weapons Perspective

Ed Schmidt

Director

Office of Nuclear Weapon Surety and Quality

36

NNSA Perspective

Rabi Singh

NNSA QA Roadmap Leader

National Nuclear Security Administration

37

EM Perspective

James Poppiti

Environmental Management

38

NE Perspective

Mark Janaskie

Office of Integrated Safety and Project Management

Nuclear Energy, Science and Technology

39

Path Forward

Robert Loesch

Director (Acting)

Office of Quality Assurance (EH-31)

40

Commitment and Accountability The Department is committed to implementing SQA

requirements as part of its overall Quality Assurance Program

We have worked extensively with NNSA (NA-121 & NA-124) and EM

Directives have been revised to reflect the SQA requirements along with roles and responsibilities

Central Registry established for “toolbox” codes http://www.eh.doe.gov/sqa/central_registry.htm

QA and SQA web sites along with SQA List Server established to share information and solicit feedback

41

Upcoming Activities

Toolbox code upgrades Other directive revisions

Minor changes to include reference to revised QA Order and new SQA Guide

QA and SQA Qualification Standards Update Order Roll Out

Regional training and support Site visits upon request Newsletter series on 10 work activities Safety software assessment support

42

Resources EH QA Web Site

http://www.eh.doe.gov/qa/

EH SQA Knowledge Portal http://www.eh.doe.gov/sqa/

SQA List Server CENTREG@VM1.HQADMIN.DOE.GOV

EH S/CI Web Site http://www.eh.doe.gov/sci/

43

Resources continued

EH Staff QA - Bud Danielson 301-903-2954

bud.danielson@eh.doe.gov SQA - Debra Sparkman 301-903-6888

debra.sparkman@eh.doe.gov S/CI - Rick Green 301-903-7709

rick.green@eh.doe.gov

44

FAQs

Debra Sparkman,Software Quality EngineerGustave (Bud) Danielson, Jr.Quality Policy ManagerOffice of Quality Assurance Programs

45

FAQs1. Why does the safety software definition include

references to 10 CFR 835, DOE P 450.4, and

the DEAR ISMS clause?

2. How were the 10 work activities determined?

3. Our site currently does not use NQA-1-2000.

Will this be a big change for our programs?

46

FAQs continued4.Our site’s SQA program is based on 10-CFR-

830 , ASME NQA-1 2000, QC1, RW0333P, and DOE Orders. Our SQA / QA program and implementing procedures cover all software. Can we continue to use our  grading levels if they are different from those suggested in the Guide?

5.Facility design software used by a DOE contractor may be graded differently than the same software used by a supplier of design services to the DOE contractor. Why does DOE G 414.1-4 recommend different grading of the work activities?

47

FAQs continued6.The Order is silent on software quality

requirements for "non-safety software".  What software quality standards are required for "non-safety software"?

7.How do the safety software requirements in DOE O 414,1C apply to DOE weapons related work?

8.How do the safety software requirements in DOE O 414.1C differ from those in QC-1?

48

FAQs continued9. Are there any changes in the way software

users will be contacted on software bugs or major issues, especially with respect to software used by many contractors?

10. Is there a centralize list of safety analysis software used by DOE contractors?

11. How does the graded approach apply to safety software? Can you provide

examples?

49

FAQs continued

12. Can a developer or contractor submit

software to DOE to be considered a toolbox

code and included in the Central Registry?

13. When will the contractor be required to

comply with DOE O 414.1C?