1

NASA Quality Policy

What’s Working?

What’s New?

Goddard Space Flight Center

- Supply Chain Conference 2008 -

Brian Hughitt

NASA Office of Safety & Mission Assurance

2

- NASA Quality Roadmap -

Establish Requirements

Assure Compliance with Requirements

Organization Government Performing Work Contracting Agent

Quality System Requirements Contract Quality Program Elements

Prevention Detection/ Prevention Detection/ Correction Correction

3

NASA Quality Roadmap

Quality Sys te m Requireme nts for Activities Performing Work

AS9100 AS9003 FAR

Inspection Clause(s)

ISO 9001 Quality Clauses

(ARP 9009)

Critical and Complex Work

X X

Critical / Not Complex Work

X

X

Non-Critical Work

X

X

Engineering and Quality Assurance Support Services

X

4

Federal Acquisition Regulations (FAR) Part 46, Quality Assurance

Contract quality requirements: Requirements in the contract relating to the quality of the product and those contract clauses prescribing inspection, and other quality controls incumbent on the contractor, to assure that the product conforms to the contractual requirements.

The extent of quality assurance is based upon the complexity and criticality of the contract item.

Government contract quality assurance: The various functions, including inspection, performed by the Government to determine whether a contractor has fulfilled the contract obligations.

The Government is required to perform “all actions necessary” to verify whether supplies conform to contract quality requirements.

5

“ Work that is both critical and complex shall be performed in accordance with the quality system requirements of AS9100.” NPD 8730.5, NASA Quality Assurance

Policy

Critical work is any hardware task that, if performed incorrectly or in violation of prescribed requirements, could result in loss of human life, serious injury, loss of mission, or loss of a significant mission resource.

Complex work involves either: a) the design, manufacture, fabrication, assembly, testing, integration, maintenance, or repair of machinery, equipment, subsystems, systems, or platforms; or b) the manufacture/fabrication of parts or assemblies which have quality characteristics not wholly visible in the end item.

6

AS9003 Inspection and Test Quality System

- Management Responsibility- Quality System- Contract Review- Design Control- Document & Data Control- Purchasing - Process Control- Training- Servicing- Corrective Action- Control of Customer Supplied Product- Product Identification and Traceability- Inspection and Testing- Control of Nonconforming Product- Handling, Storage, Packaging, Preservation and Delivery- Control of Quality Records- Internal Quality Assessment- Statistical Techniques

Noncomplex Product

A hardware item whose conformance of key characteristics can be wholly established through measurement, inspection, and/or test of the end item

7

US Federal Acquisition Regulations (FAR) Inspection Clause

“The Contractor shall maintain an inspection system acceptable to the Government covering supplies under this contract and shall tender to the Government for acceptance only supplies that have been inspected in accordance with the inspection system and have been found by the Contractor to be in conformity with contract requirements.”

8

ARP9009Aerospace Contract Clauses

• Quality System• Material Identification, Damage, Count• Right of Access• Flow Down Requirements• Certificate of Compliance (C of C)

• Certificate of Compliance Raw Material• Certificate of Compliance - Calibration• Special Process Certification• Calibration System• Configuration Management System• Change Authority• Critical Processes• Government Source Inspection (GSI)• Contractor Source Inspection (CSI)• First Article Inspection• Nondestructive Inspection (NDI) / Nondestructive Test (NDT) Certification

• 100% Attribute Clauses• Limited Operating Life Items• Limited Life and Age Control (Shelf Life) • Packaging Requirements• Packaging Handling & Labeling• Shipping Documents• Nonconformance Reporting• GIDEP• Record Retention• Electrical Wire and Cable Test Report• EEE parts Date of Manufacture• EEE Single Lot / Date Code• Electrostatic Discharge (ESD) Protection Program• High-Strength Fasteners• Pressure Vessels• Solvent Containers

9

Contract Quality Clause Example

Certificate of compliance- Raw materials

“Organization will include with each shipment the raw material manufacturer's test report (e.g., mill test report) that states that the lot of material furnished has been tested, inspected, and found to be in compliance with the applicable material specifications. The test report will list the specifications, including revision numbers or letters, to which the material has been tested and/or inspected and the identification of the material lot to which it applies. When the material specification requires quantitative limits for chemical, mechanical, or physical properties, the test report will contain the actual test and/or inspection values obtained.

10

CIRCULAR NO. A-119

MEMORANDUM FOR HEADS OF EXECUTIVE DEPARTMENTS AND AGENCIES

SUBJECT:  Federal Participation in the Development and Use of Voluntary Consensus Standards and in Conformity Assessment Activities

Your agency must use voluntary consensus standards in its regulatory and procurement activities in lieu of government-unique standards unless use of such standards would be inconsistent with applicable law or otherwise impractical.

(1) "Use" means incorporation of a standard in whole, in part... (2) "Impractical" includes circumstances in which such use would fail to serve

the agency's program needs; would be infeasible; would be inadequate, ineffectual, inefficient, or inconsistent with agency mission; or would impose more burdens, or would be less useful.

11

Voluntary Consensus vs NASA Standards

- NASA Administrator Comments -

From: Lloyd, James D. (HQ-GA000) Sent: Tuesday, June 26, 2007 8:05 AMTo:Hughitt, Brian (HQ-GD000); Oconnor, Bryan (HQ-GA000)Cc:Stamatelatos, Michael G. (HQ-GD000); Malone, Roy W. (MSFC-QD01)Subject: RE: CxP Workmanship Standards Status

Brian and Bryan,

I went to a CxP brief to the Administrator on Friday … and Mike clearly and repeatedly indicated that NASA does not have the market cornered on how to do everything related to space. In general, in some cases he said we are far behind the free market! He wants the preference to be national consensus, government consensus (looking closely at DOD specs and standards), NASA standards and, lastly, center-based unique standards. His premise is that being unique without reason and when not warranted is costing money. The risks need to be fully understood for complying or not complying with this order of precedence.

Jim

12

Document Stewardship

Widespread & Consistent Usage

Transitioning to Voluntary Consensus Standards

Gap analysis to ensure technical adequacy

Standards Lifecycle

ANSI/ESD S20.20 (Electrostatic discharge): Implementation Plan for Cross-Agency Use

J-STD-001 Space Addendum (Soldering):

NASA Proposal in Industry Coordination. CxP requirements carried in parallel.

IPC-A-620 Space Addendum (Harness Mfrg):

Preparing “shall” list in preparation for gap analysis of forthcoming draft.

NASA Chairs IPC Working Group.

ANSI/ESD S20.20: Currently examining requirements for tools & gloves

Address gaps: a) Additional requirements b) Modification to VCS

13

Voluntary Consensus Standards Adopted by NASA

• Soldering (Workmanship) – J-STD-001D / DS

• Non-Destructive Evaluation – NAS/ASTM/SAE (various)

• Metrology / Calibration – ANSI/NCSL Z540.3

• Electrostatic Discharge Protection – ANSI/ESD S20.20

• First Article Inspection – AS9102

• QMS for Maintenance Organizations – AS9110

• QMS for Distributors – AS9120

• Inspection Sampling Procedures – ANSI/ASQC Z1.4/Z1.9

• Variation Management – AS9103

• Contract Quality Clauses – ARP9009

14

Risk

15

The quality system provides assurance that all requirements are complied with- those that matter a lot and those that don’t matter as much. For highly complex and critical operations, MANY requirements that matter a lot.

NASA = HIGHLY CRITICAL AND COMPLEX

Don’t waste your time assuring requirements that don’t matter much. Be deliberate, be watchful, be smart. Careful planning and measurement of risk is the inviolable first step before quality program execution.

16

It is NASA policy to mitigate risks associated with noncompliance.

NPD 8730.5

Determination of risk considers:

• the likelihood of noncompliance • the consequences associated with noncompliance• the maturity, complexity, criticality, importance & the value of work performed• past performance

17

Separate the vital few from the trivial many

Joseph Juran

18

19

20

21

Joint Audit Planning Committee

One NASA

22

Sharing Quality Data

Provides:

_ Quality Leading Indicators _ Delivery Ratings_ Source Selection Tool

Accessed by:

_ NASA Program Managers_ NASA Quality Assurance_ NASA Procurement_ Prime Contractors _ Other Govm’t Agencies

JAPC AuditsPDREP

NADCAP

Supplier Assessment System

DCMA

GIDEP

23

Joint Audit Planning Committee

Authority

The United States Code of Federal Regulations (CFR) Title 15, Part 287, Guidance on Federal Conformity Assessment

Responsibilities of Federal Agencies:

– Coordinate its quality assurance activities with those of other appropriate Government agencies and with those of the private sector (Sec 287.1(a)).

– Participate in efforts designed to improve coordination among governmental and private sector conformity assessment activities (Sec 287.4(g)).

– Conduct joint supplier audits and share conformity assessment information among agencies (Sec 287.4(h)).

– Use the results of private sector or other governmental conformity assessment activities to schedule audits more effectively (Sec 287.4 (e))

24

Legal Propriety

- Supplier agrees in advance that data generated from audits will be shared among JAPC member organizations

- Supplier validates that JAPC audit data is factually accurate and that it does not contain trade secrets, confidential commercial or financial data, or export controlled information

- JAPC member organizations are not permitted to audit suppliers from which they routinely or specifically compete with for the same or similar work

- JAPC member organizations are required to protect supplier information received

25

JAPC Member Organizations

ATK Launch SystemsBall Aerospace and Technologies CorporationBoeing Space ExplorationCalifornia Institute of Technology Jet Propulsion Laboratory (JPL)The Johns Hopkins University/Applied Physics Laboratory (JHU-APL)Lockheed Martin Space SystemsNorthrop Grumman Space TechnologyOrbital Sciences CorporationPratt & Whitney RocketdyneRaytheon Missile SystemsUnited Space AllianceNASA HeadquartersNASA CentersDCMAMissile Defense Agency(National Reconnaissance Office)

26

Identification of Prospective JAPC Suppliers

A supplier for which an organization desires a higher level of assurance than that provided by the QMS certification process. Considerations for identification of JAPC suppliers include:

– importance of product– complexity of product– maturity of product/process– new supplier– history of quality problems– special processes not covered by Nadcap

27

28

JAPC Suppliers(Audited or Scheduled)

Aerojet Corporation Honeywell Defense & Space Electronics Systems Honeywell Space Systems Division L-3 Communications Cincinnati Electronics Merrimac Industries Moog Inc., Systems Group Starsys, Inc. Vacco Industries Hamilton Sundstrand – Rockford Hamilton Sundstrand – Windsor Locks Emcore AMPAC Reinhold Industries Parker Hannifin Aeroflex Laboratories Microsemi

29

Supplier Calendar (proposed)

Lead Org Mar Apr May June July Aug Sep Oct Nov Dec Sppt Orgs

Orbital L3 Ball, MDA, LM

JPL Moog Orb, USA, Ball, LM

Orbital Swales Ball, APL

JPL Spectrolab

BAE Ball, APL, LM

JPL Ball, LM, Orb

Raytheon Aerojet Sacrmt

JPL, Orb, LM

Lockheed M Honey

Well Clr

Orb, PWR, NG

Lockheed M Starsys Ball, JPL, NASA

NASA Adole LM, APL,JPL

NASA Aerojet Redmnd

JPL,USA, LM, Ball

Northrop G Glenair APL, Ray, LM, PWR,USA,Ball

Northrop G Merimac APL

Orbital Arde JPL, LM

Raytheon Conx JPL, Orb, LM

USA Vacco Boeing, JPL, APL,

USA Honeywell Glen

Ball, LM

Boeing General

Dynamic

Ball, LM, NG

30

Recalibrating the JAPC

Bumps along the road:

1. Audit scope

2. Competitors

3. Thanks, but no thanks…

4. Feedback loop.

31

Audit Scope- High Risk QMS Elements -

QUALITY MANAGEMENT SYSTEM• Control of outsourced processes 4.1• Configuration Management 4.3 / 7.5.3PRODUCT REALIZATION• Identification and Inspection of Key Characteristics 7.3.3.9/ 7.5.1/ 8.1• Supplier Quality Management (Purchasing) 7.4• Process control 7.5.1

- Documentation of processes- Control of key characteristics - In process verification points

• Criteria for workmanship• Validation, Monitoring, and Measurement of Special Processes 7.5.2 / 8.2.3• Identification of Monitoring/Measurement Status 7.5.3• Traceability of Product to Records of Objective Quality Evidence7.5.3• Preservation of Product / Work Environment 7.5.5 / 6.4• Electrostatic Discharge• Foreign Object Debris/Damage 7.5.1.i• - CleanlinessMEASUREMENT, ANALYSIS AND IMPROVEMENT• Internal Audit 8.2.2• Monitoring, Measurement, and Control of Key Characteristics 8.2.4• Control of Nonconforming Product 8.3• Corrective Action 8.5.2

32

Competitors

“Audit results are withheld from JAPC member organizations with whom the supplier routinely or specifically competes for the same or similar work, whether such work is awarded by a governmental organization or in commercial trade.

(note: this access limitation does not apply to JAPC member organizations that have a current open contract with the audited supplier where such contract includes an access provision for the conduct of quality assurance surveillance activities).”

JAPC Supplier Invitation Letter

33

Thanks, but no thanks…

“Suppliers declining JAPC membership (or failing to respond) shall be notified by the lead JAPC organization of their intention to perform a quality system audit, and shall be provided a second invitation for the supplier to become a JAPC supplier.”

JAPC MOU, Rev A

34

JAPC Feedback Loop

“Providing continuous improvement feedback to third party, Industry Controlled Other Party (ICOP), and Government programs which serve to assess the conformance and/or effectiveness of supplier quality management systems or quality system processes.”

JAPC MOU, Rev

A

35

Knowledge Café

•

36

Quality Taxonomy

36

2006 Cases Documentation - AS9100 Para 4.0Management Responsibility - AS9100 Para 5.0Resource Management - AS9100 Para 6.0Planning/Customer Interface - AS9100 Para 7.1, 7.2 Design - AS9100 Para 7.3Supply Chain - AS9100 Para 7.4 Production/ Process - AS9100 Para 7.5Calibration/Metrology - AS9100 Para 7.6General Quality Assurance - AS9100 Para 8.1Monitoring/Audit - AS9100 Para 8.2Control of NCM - AS9100 Para 8.3Analysis of Data - AS9100 Para 8.4Corrective and Preventive Action - AS9100 Para 8.5QA SCOREDeath on the Steppes - The Nedelin Rocket Disaster X X X X X X X X X 10SUBSAFE - USS Thresher X X X X X X 7

2007 Cases

Almost Perfect - X-31 X X X X X X 7Supercritical - SL-1 Nuclear Reactor X X X X X X X X X 10Derailed - The Eschede Train Disaster X X X X X 6Innovation Pushed Too Far Too Fast - R-101 Dirigible X X X X X X X X X 10Rocky Mountain Death Trap: The Mann Gulch Fire X X X 4

Lewis Spins Out of Control - Lewis Space Craft X X X X X X X X X X 11

2008 CasesForrestal in Flames - US Aircraft Carrier Forrestal X X X X X X X 8BP Texas City Refinery Accident X X X X X X X X X X X X 13

Fire in the Cockpit - The Apollo 1 Tragedy X X X X X 6Powerless - 2003 North American Blackout X X X X X X X 8

Hanging by a Thread - Hyatt Walkway Collapse X X X X X X X X X X X 12

Totals: 7 9 10 9 12 6 10 4 8 7 4 7 6

Proximate CausesUnderlying Causes

The marked boxes indicate system elements that would be subject to audit Findings if the audits had been conducted

effectively.

37

Marking indicates an Op Amp from ADI…

… but contains die for a Voltage Reference from PMI

Evidence of prior marking for a part with inferior performance …

… accompanied by bogus test report

Device lead condition shows parts were used

Part number indicates a CLCC package, but this package is a CDP…

Counterfeiting

Retopping Remarking

GIDEP Counterfeit Report Submissions

0

10

20

30

40

50

1976 1978 19801982 1984 1986 1988 1990 1992 1994 1996 1998 2000 20022004 2006 2008

Year

Reports

38

39

40

Quality is never an accident; it is always the result of high intention, sincere effort, intelligent direction and skillful execution; it represents the wise choice of many alternatives.

William A. Foster

41