Federal Aviation Administration SL-1 Federal Aviation Administration Concepts and Implementation for the Pilot Project Participant Part II - Implementation

Federal AviationAdministration

SL-1


Concepts and Implementation for the Pilot Project Participant

Part II - Implementation

MSMS Team Member

FAA Aircraft Certification Service


SL-2

Agenda

Related SMS ActivitiesSMS Fundamentals SMS Pilot Project Implementation Guidance & Tools Preliminary Gap Analysis meetingWorking together during the Pilot ProjectQuestions


SL-3

Pilot Project Objectives

Provide recommendations and analysis to support of development of FAA regulations and associated policies

Validate or refine the D&M SMS Framework Validate or refine guidance materials Propose the appropriate applicability and

scalability of SMS Refine assessment and oversight policies


SL-4

MSMS Organizational Structure

AIR-150

MSMSTeam Lead

CommunicationLead

MSMSPilot Project

Lead Data Collection &Analysis Focal

Share point Focal

FAA ParticipantManagement Teams (PMT)

PMT Includes:• MSMS Team Member as FAA PMT Lead• Manufacturing representatives (Field)• Engineering representatives (Field)

Pilot Pilot ProjectProject

GAP Analysis & Framework

Team

ONETeam

FAA ParticipantManagement Teams (PMT)FAA Participant

Management Teams (PMT)FAA ParticipantManagement Teams (PMT)FAA Participant

Management Teams (PMT)


SL-5

?

??

MSMS Team Lead

Pilot Project Lead(PPL)

Participant OrganizationPoint of Contact

(POC)

Participant ManagementTeam Lead

(PMTL)

Participant ManagementTeam Members

(PMT)

Oth

er

Re

so

urc

es

AS

I

AS

E

AS

I

AS

E

MANUF

$$

ENG

HRQA

ParticipantExecutive

/ Corporate


SL-6

Roles & Responsibilities

D&M SMS Pilot Project Lead

Serves as the point of contact for the D&M Pilot Project

Coordinates and manages the FAA Participant Management Teams

Provides final acceptance to exit a Level

Pilot Project Lead: [email protected]


SL-7

Roles & Responsibilities (Continued)

FAA Participant Management Team (PMT) Consists of members of the MSMS Team & FAA field

personnel from MIDOs and ACOs

Participate in meetings / telecons and site visits

Participate in procedure and process assessments

Provide status/updates to the oversight organization

throughout the Pilot Project

Company PMT:

• MSMS Team: Kurt Krumlauf (lead) and Steve Slagle

• LAACO: Cheyenne DelCarmen and Joe Costa

• Phoenix MIDO: Don Henchar


SL-8


FAA PMT LeadLeads the Participant Management TeamProvides direction, guidance and assistance to PMT

and participant Coordinates formal interaction with the participant, such

as meetings, telecons, output deliverables, reviews of exit criteria

PMT Lead for Company: Kurt Krumlauf [email protected] (562) 627-5285


SL-9


Participant Point of Contact (POC)

Company SMS Project Manager (or similar)

Serve as POC for SMS activities

Serve as primary contact to the FAA


SL-10

SMS will be implemented in 4 progressive Levels

This approach enables an organization to establish the basic building blocks (levels 1 and 2) before implementing the more complex activities of SRM and SA (levels 3 and 4)

Development and implementation of the SMS is more manageable

AIR SMS Implementation Strategy


SL-11

Level 1 focuses on an agreement with the FAA for a completed gap analysis and development of an implementation plan.

Levels 2 & 3 will be subject to an assessment process by the FAA’s PMT

Level 4 is for the organization to verify the performance and effectiveness of their SMS management practices and operational processes

AIR SMS Implementation Strategy


SL-12

Defining the Scope of Your Project

For an SMS to be fully effective it must be implemented organization-wide

However, for the pilot project you may want to scale down the project Consider the company’s products, processes, and

departments that you would like to include in the pilot project

The scope should be of appropriate magnitude with the goal of exiting Level 1 within six months.


SL-13

2ReactiveProcesses

3ProactiveProcesses

4ContinuousImprovement

1Planning &Organization

Implementation LevelsImplementation Levels


SL-14

D&M SMS Pilot Project Process (Level 1)

PMT Lead

2. Initial Pilot Project Participant and Local FAA Office Telecon

StartMSMS Team

1. Organize the FAA (Participant

Management Team)

Pilot Project Participant

5.a Company Conducts Detailed Gap Analysis Assessment prior to formal Gap meeting with PMT

Detailed Gap Analysis


5.b Present Detailed Gap Analysis with PMT

PMT

5.c Provide feedback to the Participant on

Detailed Gap Analysis Assessment

Not Ready


7.a Develop Implementation Plan, Safety Commitment approach & Training

Plan

Implementation Plan


7.b Present Implementation Plan,

Commitment, and Training Plan and Review with PMT

PMT

7.c Provide feedback to the Participant on

Imp. Plan, Commitment approach, and Training

Plan

6.Participant

ready to proceed with

Implementation Plan?

8.Participant

ready to proceed to Level

2?

Not Ready

Participant Proceeds with

Implementation (Level 2)

Yes

Yes

PMTL

4.c Conduct Preliminary Gap

Analysis


3.a Send Statement of Commitment and POC

Info to PMTL


3.b. Begin developing Initial System

Description prior to the preliminary gap

analysis

PMT

4.b Conduct Pilot Project Participant

Orientation

Orientation and Preliminary Gap Analysis Meeting

PMT & Participant

4.a Conduct PMT team meeting

1

Planning &Organization


SL-15

Planning &Organization

Level 1 Output Level 1 Output DocumentsDocuments

Management Commitment definition (Letter)

Organization Safety Policy

Comprehensive SMS Implementation Plan for entire organization through level 4

SMS Training Plan for all employees

1


SL-16

Planning &OrganizationLevel 1 Exit CriteriaLevel 1 Exit Criteria

Evidence of Accountable executive’s commitment to implement SMS, define safety policy and convey safety expectations and objectives to its employees

Evidence of adequate resources to implement SMS

Designate a management representative responsible for SMS development

Define safety-related positions for those who will participate in SMS development and implementation

Obtain agreement with FAA on your Detailed Gap Analysis and Implementation Plan

Identify safety competencies and develop a training plan

Complete a exit level one meeting with the FAA

1


SL-17

Level 2 Output DocumentsLevel 2 Output Documents ReactiveProcesses

1. Objective evidence that SRM processes and procedures have been applied to at least one existing hazard and the mitigation process has been initiated

2. Updated comprehensive SMS implementation plan for all elements to take the participant through Level 4

3. Updated SMS training plan for all employees

2


SL-18

Level 2 Exit CriteriaLevel 2 Exit Criteria2 Reactive

Processes

1.Processes & procedures documented for operating the SMS to the level of reactive analysis, assessment and mitigating actions

2.Develop documentation relevant to SMS implementation plan and SRM components (reactive processes)

3. Initiate and document voluntary non-punitive employee reporting and feedback program

4.Conduct SMS training for staff directly involved in the SMS process to at least the level necessary for the SMS reactive processes


SL-19

Level 2 Exit Criteria Level 2 Exit Criteria (Continued)(Continued)ReactiveProcesses

5.Apply SRM processes and procedures to at least one known (existing) hazard and initiate the mitigation process to control/mitigate the risk associated with the hazard

6.Update the Detailed Gap Analysis on the entire organization (pending scope) for all elements of the SMS Framework

7.Update the comprehensive SMS Implementation Plan for all elements to take the organization through Level 4

2


SL-20

Level 3 Output DocumentsLevel 3 Output Documents3 Proactive

Processes

1. Objective evidence that SRM processes and procedures have been applied to all Element 2.1 (1) Hazard Identification and Analysis items

2. Objective evidence that SRM processes and procedures have been applied to at least one existing hazard and that the mitigation process has been initiated

3. Updated comprehensive SMS implementation plan for all elements

4. Updated SMS Training Plan for all employees


SL-21

ProactiveProcesses

Objective evidence that Hazard Identification and Analysis has been accomplished

Objective evidence that SRM processes and procedures have been applied to at least one existing hazard and mitigation process has been initiated

Updated comprehensive SMS implementation plan for all elements

Updated SMS training plan for all employees

3Level 3 Exit Criteria Level 3 Exit Criteria (Continued)(Continued)


SL-22

ProactiveProcesses

6. Objective evidence of compliance with Element 4.1 Competencies and Training

7. All applicable SMS processes and procedures have been applied to at least one existing hazard and mitigation process have been initiated

8. Complete SMS Training for Staff directly involved in SMS process to the level of accomplishing all SMS processes

9. Complete employee training commensurate with this level of implementation maturity



SL-23

ProactiveProcesses

Objective evidence of compliance with Competencies & Training

All applicable SRM processes and procedures have been applied to at least one existing hazard and that the mitigation process has been initiated

Complete SMS training for the staff directly involved in the SMS process to the level of accomplishing all SMS processes

Complete employee training commensurate with this level of implementation



SL-24

Level 4 ExpectationsLevel 4 Expectations 4 ContinuousImprovements

1. The complete SA process, continuous monitoring & other SRM processes are functioning

2. Verify the performance and effectiveness of the SMS management practices and operational processes


SL-25

Agenda

IntroductionsRelated SMS ActivitiesSMS FundamentalsSMS Pilot Project ImplementationGuidance & Tools Preliminary Gap Analysis meetingWorking together during the Pilot ProjectQuestions


SL-26

Guidance and Tools

D&M SMS Pilot Project Guide, referred to as “The Guide”

Developmental Guidance (DG) for D&M SMS Framework

Tools Gap Analysis Tool Assessment Tool Implementation Planning Tool

AIR SMS Website (SharePoint)


SL-27

D&M SMS Pilot Project Guide

SMS Background and Introduction

Roles, Responsibilities and Relationships

Implementation Processes Pilot project process for implementation of level 1 Pilot project process for implementation of level 2 & 3 FAA Assessment of the SMS levels

D&M SMS Framework

Information on SRM & SA integration

System Description and Hazard Identification

Definitions and Acronyms


SL-28

System Description

Assists an organization to:

Define its system and identifying hazards

Establish the scope and boundaries of the company to which SMS applies

Appendix D of “The Guide”

Currently it provides a summary of a system description

A more comprehensive system description is under development and will be included

Data and information received from the Pilot will be used to improve “The Guide”


SL-29

Developmental Guidance (DG) for D&M SMS Framework

Purpose

Provides assistance to an organization in developing their SMS using the D&M SMS Framework

Clarifies the intent and explains the expectations in the D&M SMS Framework


SL-30

Developmental Guidance (Continued)

Overview

Provide examples and ways an organization may choose to develop its SMS

Formatted to follow the Framework in “The Guide”

Comparable to an Advisory Circular (imbeds Framework expectations and guidance)

Focuses on the areas that may not be self-explanatory to the participant

Designed to be a living document


SL-31

Developmental Guidance (Continued)


SL-32

Gap Analysis Tool

Purpose

Assists participants in documenting their gaps and status toward meeting the D&M SMS expectations

Provides a mechanism to learn more about the expectations

Provides a basis for a participant’s Implementation Plan

Gap tool


SL-33

Gap Analysis Tool (Continued)

Structure

Excel Spreadsheet with multiple tabs

Combines Preliminary and Detailed Gap Analysis tools in one file

Follows the structure of the D&M SMS Framework

Provides step-by-step instructions on how to use

Provides an Executive Summary

Automatically populates system segment ratings

Records Lessons Learned and comments for each Element and Sub-Element Gap tool


SL-34

Assessment Tool

Structure

Assessment tab is within the Gap tool

Assessment tab will be used by PMT to assess policies and procedures

Tracks and documents the PMT's assessment of an organization's SMS

Documents an organization's evidence for achieving specified ratings for each D&M SMS Framework Expectation

Gap tool


SL-35

Assessment Tool (Continued)

Structure

Provides a single repository for PMT assessment outputs

Provides a Lessons Learned mechanism to collect feedback on the assessment process

Gap tool


SL-36

Implementation Plan Tool Structure

Implementation plan tab is in the Gap tool

Identifies tasks and activities that are needed in order to close the gaps

Identifies the anticipated timeframe and point of contacts for the planned tasks and activities

Provide a high-level view of the timeline

Provides a narrative that describes how the organization intends to close all expectation gaps and achieve a fully functioning SMS.

Gap tool


SL-37


Project information / data will be shared with the FAA on the MSMS SharePoint site

Each participant places deliverable documents in their own file location to share with FAA

Files are protected from viewing from other participants

Only selected FAA users will have access to view participant information

Data will not be kept beyond PMT assessment


SL-38


Other Information to be shared with all participants

FAA SMS guidance documents

Gap analysis and assessment tools

Frequently Asked Question’s (FAQ’s) (non-proprietary)

Generic (non-proprietary) Lessons Learned information

Best Practices

Other SMS information


SL-39

Pilot Project - Data collection

Specific data, lessons learned, feedback will be collected to assist the MSMS team in the development of the SMS framework, policy and processes

Data and information will be general (i.e. procedures) and not technical (i.e. Design) in nature

If proprietary data is received, it will not be preserved or shared outside the MSMS team (data will be deleted / returned when FAA assessment is complete)

Data collection and analysis tools will be used


SL-40

Pilot Project - Data Collection

Specific data and information to be collected is being developed by MSMS team

Examples of data and information that will be requested are: Lessons learned data and feedback on the SMS

framework and pilot project processes Number of days to complete gap analysis and to exit

each level Number of procedures and processes (new and existing) Types and number of hazards identified SMS processes and procedures that are also part of

quality system


SL-41

Protection of Participant Information

The FAA Recognizes the importance of your participation in the pilot project and protection of data

FAA will exercise due diligence to not share data outside FAAFAA will limit the types of data it retains Limit data within the FAA

Please note: We may share de-identified informationSMS information that is not specifically

protected under Part 193 and may be subject to FOIA


SL-42

Agenda

IntroductionsRelated SMS ActivitiesSMS FundamentalsSMS Pilot Project ImplementationAIR SMS Guidance & ToolsPreliminary Gap Analysis meetingWorking together during the Pilot ProjectQuestions


SL-43

Over the next two days we will work together on a Preliminary Gap Analysis

The Gap analysis tool will be used

The meeting will be facilitated by the POC and PMT

Preliminary Gap Analysis


SL-44

Purpose of the preliminary gap analysis

Ensure the organization and FAA have a common understanding of the Gap Analysis Process and framework

Develop a common understanding of the D&M SMS Framework Expectations in the context of the organization and its operations

Preliminary Gap Analysis Tool assists the organization in conducting an initial high-level “first look” / “best guess” in assessing compliance with the SMS

Provides a starting place for the organization to conduct a detailed evaluation of the gaps


SL-45

Agenda

IntroductionsRelated SMS ActivitiesSMS FundamentalsSMS Pilot Project ImplementationAIR SMS Guidance & ToolsPreliminary Gap Analysis meetingWorking together during the Pilot ProjectQuestions


SL-46

Working together during the Pilot Project

Pilot project should not interfere with the FAA’s Certificate Management (CM) responsibilities and Type Certification activities (e.g. ACSEP, DOIP, Designee Mgt)

Changes to a PAH’s FAA approved quality system will continue to be approved per Part 21 and FAA policy

The PMT will work with the participant for the duration of the pilot

Communication process is outlined in “The Guide”

Meetings / telecons are anticipated Formal communications / decisions will be between the POC and

PMTL Etc.


SL-47

Deviations, clarifications (interpretation), improvements concerning the framework, guidance material, tools will be coordinated per instructions in “The Guide”

Feedback and lessons learned will be collected throughout the pilot project

Changes to pilot project guidance will be posted on SharePoint

Working together during the Pilot Project (continued)


SL-48

After this meeting:

After the preliminary gap analysis the PMT will continue working with you

Within the next few weeks the POC and PMT Lead will schedule ongoing meetings / telecons

• The PMT will work closely together in initial discussions and activities

Participants should complete a Detailed Gap Analysis and have an Implementation Plan within six months

Schedule an Exit Level 1 validation meeting with PMT

Working together during the Pilot Project (continued)


SL-49

What happens After the Pilot Project?

Company should continue developing their SMS to get to Level 4

Sustaining support by the FAA will continue per instructions in “The Guide”

Local FAA Offices will continue to work with company until SMS rule and FAA policy is finalized

POCs should continue working with the local FAA offices


SL-50