Upload
hoangngoc
View
216
Download
0
Embed Size (px)
Citation preview
Presented to: AAAE/MITRE/ACI-NA Safety Management Systems for Airports Conference
By: FAA Airport Safety and Standards
Date: Thursday, October 30, 2008
Federal AviationAdministrationAirport SMS
Pilot Project IFindings
2 2Federal AviationAdministration
SMS Pilot Project
Objectives• History/Purpose• What is SMS?• Why SMS?• Study Methodology.• Findings• Recommendations.
Stakeholders
Effectiveness Efficiency
Safety
Management
Policy Safety Risk Management
Promotion Assurance
3 3Federal AviationAdministration
SMS Pilot Project
“If compliance guaranteed safety, then we would only need one rule: Don’t crash. Obviously, it takes a lot more than
that!”
William R. Voss, President FSF
4 4Federal AviationAdministration
SMS Pilot Project
Consider for a Moment!-- The elimination of accidents is The elimination of accidents is
impossible. impossible. -- Failures will occur, in spite of the Failures will occur, in spite of the
most accomplished prevention most accomplished prevention efforts.efforts.
-- No human endeavour or humanNo human endeavour or human--made system can be free from made system can be free from risk and error. risk and error.
-- Controlled risk and error are Controlled risk and error are acceptable in an inherently safe acceptable in an inherently safe system.system.
5 5Federal AviationAdministration
SMS Pilot Project
“Safety is like fighting a guerilla war in which you are never able to declare total victory.”
James Reason
6 6Federal AviationAdministration
SMS Pilot Project
Why SMS? Because imperfection Why SMS? Because imperfection in a system is normal!in a system is normal!
Operational performance
Operational performance
OperationalOperationaldeploymentdeployment
System System designdesign
Baseline performanceBaseline performance
“Practicaldrift”
7 7Federal AviationAdministration
SMS Pilot Project
What exactly is this SMS thing that I have heard so much about? And how is it going to help me?
8 8Federal AviationAdministration
SMS Pilot Project
SMS DefinedICAO Document 9859, Safety Management
Manual“A systematic approach to managing safety,
including the necessary organizational structures, accountabilities, policies and procedures.”
9 9Federal AviationAdministration
SMS Pilot Project
SMS Defined
FAA AC 150/5200-37, Introduction to SMS for Airport Operators.
“The formal, business like approachformal, business like approach to managing safety risk. It includes systematic procedures, practices, and policies for the management of safety (including safety policy, safety risk management, safety assurance, and safety promotion).”
10 10Federal AviationAdministration
SMS Pilot Project
ICAO SMS Programme RequirementsAnnex 14, Volume I
SMS at a minimum:- Identifies safety hazards- Ensures remedial action is taken- Provides for continuous monitoring
and assessments.- Aims for continuous improvement.- Establishes clearly defined roles and
responsibilities.
11 11Federal AviationAdministration
SMS Pilot Project
FAA AC SMS Guidance (AC150/5200-37)• Safety Policy and Objectives.
– Management commitment– Written safety policy, targets and goals.– Established roles and responsibilities.
• Safety Risk Management Process– Identify hazards and associated risks– Identify appropriate mitigation strategies– Plan, do, check, act (PDCA)
• Safety Assurance System– Validation of performance indicators and targets.– Non-punitive reporting system– Internal and external audits.
• Safety Promotion Program.– Training– Safety communications
12 12Federal AviationAdministration
SMS Pilot Project
SMS Provides
– Tools to enhance safety level
– A structured proactive approach to managing the safety effort.
– Generation of an awareness of safety leading to an improved safety culture.
– Efficient application of safety resources
– Continuous improvement
13 13Federal AviationAdministration
SMS Pilot Project
The Good News• A good SMS is tailored to the
specific conditions, requirements, and resources of the aerodrome operator…so it meets your specific needs.
• SMS is not an exercise in futility.• There is a payoff at the end of the
trail. • Most of what is needed for a
functioning SMS…you already have.
14 14Federal AviationAdministration
SMS Pilot Project
SMS Regulatory Requirements
• FAA regulatory requirement for airports not yet established.
• AC150/5200-37 published.• ICAO requirement established.
Standardized across Annexes 6/11/14• Annex 14 requires a functioning SMS.• SMS is a mandatory USOAP audit item.
15 15Federal AviationAdministration
SMS Pilot Project
OFFICE OF AIRPORT SAFETY AND STANDARDSAIRPORT CERTIFICATION PROGRAM
SAFETY MANAGEMENT SYSTEMPILOT STUDY
PARTICIPANT GUIDEMARCH 1, 2007
16 16Federal AviationAdministration
SMS Pilot Project
Study Methodology• 22 airports participated.• Goals:
– Identify how well 14CFR part139 meets or exceeds the requirements of SMS
– Review existing programs/processes for functioning elements of SMS.
– Identify and quantify costs associated with implementation.
– Develop a Safety Program Manual– Implement full SMS functions in at least one
department.
17 17Federal AviationAdministration
SMS Pilot Project
Study Methodology• The ‘four pillars’ approach was the
dominant format.• Wide variations in conduct of ‘gap analysis’
and in format of SPM.• Inherent SMS bias evident in results.• Attempts made to standardize responses so
that program submissions/answers could be categorized and counted.
18 18Federal AviationAdministration
SMS Pilot Project
Small Airport Operators
• Findings indicated a lack of information related to ‘smaller’airports (Class IV).
• Recommendation made regarding second pilot project.
• Pilot Project II initiated July 2008…8 Class IV airports participating.
Talladega, AL (ASN)
19 19Federal AviationAdministration
SMS Pilot Project
Gap Analysis Review-Goal/Purpose: Systematic effort to determine:
-Which components/elements of SMS may already exist
-Which existing SMS program elements may need to be modified.
-Which normally accepted elements of SMS are missing.
-Different approaches taken in completion of Gap Analysis.
20 20Federal AviationAdministration
SMS Pilot Project
Gap Analysis Review-Attempt to categorize responses in standardized manner:
- Policy 10 categorical questions
- SRM 7 categorical questions
- Assurance 5 categorical questions
- Promotion 7 categorical questions
- Miscellaneous 2 categorical questions
-Data analyzed in terms of these 31 categorical responses.
- If response could not be determined then airport was not counted in that category (yes or no).
21 21Federal AviationAdministration
SMS Pilot Project
Gap Analysis Responses
• Does the airport have an existing SMS?– 17 answers / 1 yes / 16 no 5.9 % yes5.9 % yes
• Is management committed to the goal of safety enhancement through application of SMS?– 12 answers / 11 yes / 1 no 8.3 % yes8.3 % yes
• Does the airport have written safety policy that would meet the intent of SMS?– 19 answers / 9 yes / 10 no 47.3 % yes47.3 % yes
22 22Federal AviationAdministration
SMS Pilot Project
Gap Analysis Responses
• Is the written safety policy effectively communicated to airport employees?– 13 answers / 1 yes / 12 no 7.7 % yes7.7 % yes
• Does the airport have an existing organizational structure to manage safety?– 12 answers / 10 yes / 2 no 83.3 % yes83.3 % yes
• Does the airport have a functioning safety risk management system in place?– 19 answers / 0 yes / 19 no 0 % yes0 % yes
23 23Federal AviationAdministration
SMS Pilot Project
Gap Analysis Responses
• Is there an existing system in place for hazard reporting, identification and analysis? Is it sufficient and does it meet the needs of a functioning SMS?– 18 answers / 3 yes / 15 no 16.7 % yes16.7 % yes
• Has the airport established appropriate safety targets, goals, and/or indicators that would be used to guide the safety effort of the organization?– 18 answers / 2 yes / 16 no 11.1 % yes11.1 % yes
24 24Federal AviationAdministration
SMS Pilot Project
Gap Analysis Responses• Does the airport have, and utilize, an internal
safety audit system that meets the intent of SMS?– 18 answers / 1 yes / 17 no 5.5 % yes5.5 % yes
• Is the existing safety culture within the organization adequate to support the goals and objectives of SMS?– 13 answers / 7 yes / 6 no 53.8 % yes53.8 % yes
25 25Federal AviationAdministration
SMS Pilot Project
Gap Analysis Responses
• Does the airport have an established non-punitive reporting system?– 17 answers / 0 yes / 17 no 0 % yes0 % yes
• Does management support the idea of non-punitive reporting?– 11 answers / 10 yes / 1 no 90.9 % yes90.9 % yes
26 26Federal AviationAdministration
SMS Pilot Project
Gap Analysis Responses
• Does the airport have an existing SMS training program?– 17 answers / 1 yes / 16 no 5.9 % yes5.9 % yes
• Has the airport identified training requirements associated with safety and SMS?– 17 answers / 10 yes / 7 no 58.8 % yes58.8 % yes
• Is the current training program adequate to meet the demands of a functioning SMS?– 15 answers / 5 yes / 10 no 33.3 % yes33.3 % yes
27 27Federal AviationAdministration
SMS Pilot Project
Gap Analysis Summary• Support of SMS• Management
commitment• Support of non-
punitive reporting• Training needs
identified.• Existing safety culture
could support SMS implementation.
• Existing SMS• Safety policies
communicated• Existing SRM process• Internal/external safety
audit• Goals, targets,
indicators• Non-punitive reporting• SMS training• Data collection
inadequate
28 28Federal AviationAdministration
SMS Pilot Project
Plan Review‘Format’
•Best examples followed a structured approach
•Best examples followed the ‘four-pillars’structure to develop their program and their SPM.
•All attempted to structure format to fit organizational needs.
•If FAA moves ahead with regulatory requirement there may be some value to a more standardized SPM format.
29 29Federal AviationAdministration
SMS Pilot Project
Plan Review‘SMS vs. 139’
•Apparent that 139 is not SMS in and of itself…
•Evidence that SMS is something larger, more comprehensive, than is currently found in the act of complying with 139 requirements.
•Clear that compliance with Federal Regulations does not ensure a safe system…nor does regulatory compliance establish a functioning SMS
•Evidence that 139 compliance may eventually become part of airport SMS program...contrary to the idea that SMS would eventually become part of 139.
30 30Federal AviationAdministration
SMS Pilot Project
Plan Review‘Program Integration’
•Most identified need to address 139 as part of the SMS.
•Reference made to the idea that SMS could be used to ensure 139 compliance…but that 139 by itself could not ensure that SMS was functioning.
•Several attempted to dissect 139 to assign each part to a corresponding SMS functional area.
•Some indication that it will take effort to ensure congruency between SMS and 139.
•Some attempted to integrate tenant activities.
•Reference made to integration through infusion of SMS
31 31Federal AviationAdministration
SMS Pilot Project
Plan Review‘Policy Statement’
•Most provided written policy statement in their plans.
•Some were very simple…others (most) more complicated.
•Best statements:•Written & signed at the top level of the organization
•Expressed commitment to continuous improvement
•Integrated all parties in organization to the same goal(s)
•Promised open, continuous safety communications
•Promised implementation of all elements of SMS
•Dedicated airport to create environment that minimized hazards.
•Many addressed ALARP…one shooting for zero accidents
32 32Federal AviationAdministration
SMS Pilot Project
Plan Review‘Assignment of Responsibilities’
•All plans addressed assignment of responsibilities
•A few tried to define responsibilities and commitment at levels above immediate airport organization.
•Size of organization dramatically affected responsibilities.
•All identified the one person in control of SMS.
•Not all addressed the level of financial/HR control of that person.
•Accountable Executive vs. other titles.
33 33Federal AviationAdministration
SMS Pilot Project
Plan Review‘Acknowledgement of
Accountabilities’•Not all plans addressed accountability for safety performance and actions.
•Plans that properly addressed accountabilities tended to be broader in scope and somewhat more detailed.
•Some evidence that later stages of implementation will more likely see greater emphasis on accountability.
34 34Federal AviationAdministration
SMS Pilot Project
Plan Review‘SMS/Safety Organization’
•All plans established a formal safety organization.
•Wide variances in organization and titles used.
•Small airports tended to assign responsibilities as collateral duty.
•Larger airports tended to assign/develop new positions.
•Some established ‘steering committees’ or similar groups.
•Many established (or restarted) safety committees.
•Best examples provided detailed descriptions of responsibilities, yet provided sufficient latitude and flexibility for safety response.
35 35Federal AviationAdministration
SMS Pilot Project
Plan Review‘Support for SMS, Non-Punitive Reporting,
Continuous Improvement…’
•Wide variance in documented support.
•Documented support for continuous improvement the most elusive.
•One excellent example plan referred to support for continuous improvement as a “moral obligation”.
•Several addressed NPR system in detail leaving little doubt about their level of support for NPR.
36 36Federal AviationAdministration
SMS Pilot Project
Plan Review‘SRM Process & Application’
•All addressed safety risk management (SRM).
•Some continuity in responses (IAW AC150/5200-37).
•Differences found in level of authority responsible for SRM process.
•Best plans:•Developed guidance as to when to perform SRM.
•Implemented continuous, self-monitoring processes to ID hazards.
•Consolidated responsibility for SRM (at least in initial phases).
•Defined administrative process for SRM.
•Discussed management responsibility for tracking, mitigation andownership of risks and hazards.
•Addressed use and maintenance of safety data.
37 37Federal AviationAdministration
SMS Pilot Project
Plan Review‘Goals, Targets and Objectives…’
•Wide variance found here
•Most tied initial goals/targets/objectives to implementation.
•Evidence that initially a systems structure would need to be established, then a safety baseline defined…before goals/targets/objectives could be set.
•One airport established ‘8’ primary goals, each with corresponding performance objectives, all tied to implementation.
38 38Federal AviationAdministration
SMS Pilot Project
Plan Review‘Auditing’
•Variance in that a few established continuous planned audit programs…while at least one felt that audits should be the “exception rather than the rule”.
•Most tied auditing to continuous improvement.
•Most at least tried to define both internal and external audit requirements.
•Some evidence that the idea of auditing is still misunderstood (i.e. overemphasis and overreliance of 139 program elements).
39 39Federal AviationAdministration
SMS Pilot Project
Plan Review‘Auditing’
• Best ‘Audit Program’ examples provided:– Defined safety manager responsibilities for auditing.– Defined audit plan.– Defined audit checklist(s), database(s).– Conduct of internal audit.– Arrangement for external audit.– Use of checklist for all audits.– Communication of findings.– Population of database.– Performance of , and responsibility for, trend analysis and
‘data-mining’.
40 40Federal AviationAdministration
SMS Pilot Project
Plan Review‘Safety Data’
•As SMS data does not yet exist…many proposals for database systems were theoretical in nature.
•Many indicated an initial use of paper records for SMS efforts.
•Some developed special forms and records to use.
•Several discussed implementation of dedicated safety databases.
•One discussed third party involvement for data collection, maintenance, and analysis.
41 41Federal AviationAdministration
SMS Pilot Project
Plan Review‘Indoctrination & Training’
• Addressed by all participants…some concentrated on existing ‘139’training programs.
•All acknowledged SMS requirements and many tried to integrate programs.
•One good example addressed:
• Policy and objectives
•Training development, documentation, measurement, feedback and revision.
•Indoctrination and job specific training requirements.
•Specific SMS training requirements…including SMS skills training for managers.
•Documentation requirements and procedures to measure training effectiveness.
42 42Federal AviationAdministration
SMS Pilot Project
Plan Review‘Safety Communications’
•Well developed proposals…topic/requirement adequately addressed by most.
•Some consistency between participants.
•Training programs integral to communications process.
•Centralized communications effort by most.
•Communications conduit assigned by most.
43 43Federal AviationAdministration
SMS Pilot Project
Plan Review‘Safety Communications’
Proposed Program Elements:
•Suggestion program
•Safety bulletins
•Committee minutes
•Newsletters
•Safety reports
•Safety Summaries
•Ramp campaign
•Solicitation of feedback
•Safety contests
•Safety database
•Safety literature
•Website
•Rewards/recognition program
•Safety courses
•Promotional campaigns
44 44Federal AviationAdministration
SMS Pilot Project
Plan Review‘Non-Punitive Reporting’
•Addressed by all participants.
•A few addressed NPR as a ‘future need’…a few established NPR reporting systems and written guidance.
•All acknowledged confidentiality…most addressed issues associated with anonymity.
•‘Best Practice’ program elements:•Confidentiality & protection.
•Reporting instructions & process.
•Limitations addressed.
•Tied to ‘just’ safety culture.
•Reactive and proactive use.
•Review process.
•Feedback ensured.
•Connection to safety database.
45 45Federal AviationAdministration
SMS Pilot Project
Plan Review‘Miscellaneous’
•Establishment of safety committee: 87%87%
•Establishment of additional committees/groups: 13%13%
•Airports that used ‘Accountable Executive’title: 33%33%
•Additional duty positions associated with SMS: 20%20%
•Identification of SMS lead position: 73%73%
•New forms, tables, data tools etc.: 66%66%
46 46Federal AviationAdministration
SMS Pilot Project
Plan Review‘Miscellaneous’
•Internal audit procedures: 100%100%
•External audit procedures: 33%33%
•Detailed guidance regarding when to perform SRM: 40%40%
•Identification of goals, targets, indicators: 33%33%
•Basic SMS indoctrination: 100%100%
•SMS training for SMS responsible parties: 93%93%
•Management SMS training: 53%53%
47 47Federal AviationAdministration
SMS Pilot Project
Implementation Schedule
•Scant data provided
•Years…not months
•Phased implementation seemed universally accepted
•Program suggested by most seemed revolutionary and evolutionary…thus significant time would be required to fully implement.
•Considerable evidence that implementation schedule will vary by complexity and size of airport.
48 48Federal AviationAdministration
SMS Pilot Project
Implementation Costs•Scant data provided.
•Some hinted at significant costs associated with program implementation and development. One cost figure provided…$968,960.00
•Others hinted that most program elements already exist ‘in-house’ and that program could be developed with minimal impact to financial system. One cost figure indicated that only additional costs would be for database procurement and installation.
•One indicated inability to quantify costs associated with SMS effort by existing airport staff.
49 49Federal AviationAdministration
SMS Pilot Project
Recommendations
• Need for SMS
•Integration of SMS with 14 CFR part 139
•SMS by complexity and size of organization
•Program recommendations
•NPRM recommendation
•Advisory Circular guidance
50 50Federal AviationAdministration
SMS Pilot Project
What’s Next?• Growth in acceptance of the concept• FAA ARP internal SMS initiative• Pilot Project II• NPRM (part 139)
51 51Federal AviationAdministration
SMS Pilot Project
Review
• Why SMS
•Gap Analysis findings
•Report findings
•Recommendations
•Where do we go next…“It is the consulting team’s view that the airport’s ACM is supportive of SMS but in reality there are only portions of the ACM that can be directly adapted to SMS.”