Operational Safety Assessment Interim Report

Operational Safety Assessment Interim Report 23-07-2010 PASS/WA4/WP5/130/D Version 1.2

EUROCONTROL HQ ATC Domain – DSNA, Deep Blue, Egis Avia & QinetiQ – PASS Project Page 1/43

Work Area 4 / Work Package 5 : Operational Safety Assessment Interim Report

Performance and safety Aspects

of Short-term Conflict Alert – full Study

PASS Project

Drafted by: Anne Damidau – Luca Save

Authorised by: Béatrice Raynaud on 23/07/10

ADDRESSEES: EUROCONTROL Participants, DSNA Participants, Deep Blue Participants, QinetiQ Participants, Egis Avia Participants.

COPY TO: -



RECORD OF CHANGES

Issue Date Detail of changes

0.1 05-11-2009 Document initialisation

0.2 07-12-2009 Document update

0.3 16-12-2009 Document update to account for EUROCONTROL comments

1.0 21-01-2010 Revision of the document including comments from PASS phone conference n°20

1.1 24-02-2010 Final document update accounting for EUROCONTROL comments

1.2 23-07-2010 Editorial corrections

IMPORTANT NOTE: ANY NEW VERSION SUPERSEDES THE PRECEDING VERSION, WHICH MUST BE

DESTROYED OR CLEARLY MARKED ON THE FRONT PAGE WITH THE MENTION

OBSOLETE VERSION



TABLE OF CONTENTS

ACRONYMS ............................................................................................................................4

GLOSSARY .............................................................................................................................6

EXECUTIVE SUMMARY .........................................................................................................7

1. INTRODUCTION ......................................................................................................9

1.1. PASS PROJECT..........................................................................................................9 1.2. SAFETY CONTEXT.......................................................................................................9 1.3. OPERATIONAL SAFETY ASSESSMENT OBJECTIVE.......................................................10 1.4. OPERATIONAL SAFETY ASSESSMENT SCOPE.............................................................11 1.6. DOCUMENT OVERVIEW..............................................................................................12

2. PRELIMINARY QUALITATIVE SAFETY ANALYSIS............................................13

2.1. IDENTIFICATION OF BASIC TYPES OF HAZARDS ..........................................................13 2.2. SEVERITY ASSESSMENT............................................................................................13

3. PRELIMINARY QUANTITATIVE (EVENT TREE) ANALYSIS...............................16

3.3. SCOPE REDEFINITION ...............................................................................................16 3.4. SAFETY METHODOLOGY............................................................................................18 3.5. PRELIMINARY SAFETY OBJECTIVES DERIVATION RESULTS .........................................19

4. COMPARISON OF BOTH SAFETY ANALYSES ..................................................21

5. FUTURE STEPS.....................................................................................................25

6. REFERENCES .......................................................................................................27

7. ANNEX A : DESCRIPTION OF BASIC TYPES OF HAZARDS – QUALITATIVE SAFETY ANALYSIS ..............................................................................................................29

7.1. STCA VS CONFLICT..................................................................................................29 7.2. ATC VS STCA..........................................................................................................29 7.3. ATC VS TCAS..........................................................................................................31 7.4. CREW VS ATC AVOIDING INSTRUCTION .....................................................................31 7.5. CREW VS TCAS .......................................................................................................32 7.6. CREW VS RA REPORT ..............................................................................................32 7.7. STCA VS TCAS .......................................................................................................32

8. ANNEX B : EXAMPLE OF SEVERITY ASSESSMENT – QUALITATIVE SAFETY ANALYSIS .............................................................................................................................34

9. ANNEX C: PROCESS FOR APPORTIONNING THE ATM SAFETY TARGETS..35

10. ANNEX D : EXAMPLE OF AN EVENT TREE ANALYSIS (QUANTITATIVE SAFETY ANALYSIS).............................................................................................................38

10.1.1. Detailed Hazard Effects Analysis ..................................................................38 10.1.10. Barriers Summary .........................................................................................40 10.1.13. Safety Objectives Derivation .........................................................................42



ACRONYMS

ACAS Airborne Collision Avoidance System

ACC Area Control Centre

ANSP Air Navigation Service Provider

ATC Air Traffic Control

ATCO Air Traffic Controller

ATCU Air Traffic Control Unit

ATM Air Traffic Management

ATMSP ATM Service Provider

CFL Cleared Flight Level

DSNA Direction des Services de la Navigation Aérienne

EHQ EUROCONTROL Head Quarters

ET Event Tree

LOS Loss of Separation

N/A Not Applicable

OE Operational Effect

PASS Performance and safety Aspects of Short-term Conflict Alert – full Study

Pe Probability of Effects

RA Resolution Advisory

RCS Risk Classification Scheme

RFG Requirement Focus Group

SAM Safety Assessment Methodology

SC Severity Class

SO Safety Objective

SPR Safety and Performance Requirements

ST Safety Target



STCA Short Term Conflict Alert

SPIN Safety nets Performance Improvement Network

TCAS Traffic Alert and Collision Avoidance System

WA Work Area

WP Work Package



GLOSSARY

Air Traffic Control Unit

A generic term meaning variously, area control centre, approach control unit or aerodrome control tower (PANS ATM 4444)

Barrier Barriers of hazards represented in the event trees are mitigation means that help in detecting and recovering from a hazard, once the hazard has occurred

Conflict A conflict is a converging of aircraft in space and time which constitutes a predicted violation of a given set of separation minima, as per EUROCONTROL Specification for Short Term Conflict Alert [9].

Event Tree An Event Tree is a graphical representation of the logic model that identifies and quantifies all possible outcomes following an initiating event, i.e. the hazard.

Failure The inability of any element of the Air Traffic Management System to perform its intended function or to perform it correctly within specified limits. (ESARR4 [2])

Failure condition

A condition having an effect on the aircraft and/or its occupants, either directly or indirectly through loss of separation, which is caused or contributed to by one or more failures, or errors, considering flight phase and relevant adverse operational (density of air traffic, TMA etc…) or environmental conditions. (ESARR4 [2])

Ground-based Safety net

A ground-based safety net is functionality within the ATM system that is assigned by the ANSP with the sole purpose of monitoring the environment of operations in order to provide timely alerts of an increased risk to flight safety which may include resolution advice [9].

Hazard Any condition, event, or circumstance which could induce an accident (ESARR4 [2])

In the qualitative analysis, hazard is more generically defined as a failure condition which could induce an incident or an accident ([4]).

In the quantitative analysis a hazard is, in addition to the above definition, defined at system boundary.

Near Mid-Air Collision

It is defined as an encounter during which at some time the horizontal separation of the two aircraft is less than 500ft and simultaneously the vertical separation of the aircraft is less than 100ft (cf. ASARP Project [8])

Safety Objective

A safety objective is a qualitative or quantitative statement that defines the maximum frequency or probability at which a hazard can be accepted to occur. (ESARR4 [2])

Probability of Effects (Pe)

Probability that a hazard could generate a given effect. This probability can be obtained through event trees through the quantification of the failure/success of identified barriers.

Short-term conflict

A potential infringement of separation minima that will occur in the short-term (e.g. within less than 2 minutes).



EXECUTIVE SUMMARY

In the frame of PASS project, a safety assessment has been conducted to address the safety aspects of joint STCA and TCAS operations and includes a hazard identification, risk assessment and mitigation as required by ESARR 4. The work is planned to be performed in two steps:

- Step 1: Preliminary operational safety assessment (Phase 2). This step was realised in 2009.

- Step 2: Consolidated operational safety assessment and requirement determination (Phase 3). This step will be conducted in 2010.

Step 1 of this safety assessment was conducted in three stages, which are detailed below:

- A preliminary hazard identification (qualitative analysis): the objectives of this study were to identify a preliminary list of basic types of hazards based on the analysis of real ATC incidents studied in WA1 of PASS Project and integrated with inputs from other studies, such as those promoted by the SPIN Task Force. Then for each of these basic types of hazards, operational scenarios were defined and were discussed in a workshop with operational experts, in order to assess the severity of their operational consequences and to identify possible further basic types of hazards with relevance for the study of the STCA-TCAS overall concept.

- A preliminary event tree analysis (quantitative analysis): the main objective of this study was to derive preliminary safety objectives based on the results of previous qualitative analysis as required by ESARR4 [2]. A safety objective is a qualitative or quantitative statement that defines the maximum frequency or probability at which a hazard can be accepted to occur. The method consisted in developing event trees, which are graphical means that enable to identify and quantify all possible outcomes of the hazards.

- A summary and comparison of the qualitative and quantitative analyses: the objectives are to provide the results of both studies and to compare them. The identification of future steps to be performed in Phase 3 of PASS Project, including the improvement fields, is also addressed. This synthesis is the objective of the present document.

The comparison has been made between the severity class determined for the hazards in the quantitative analysis based on the event trees, and the severity assessed for each basic type of hazard in the qualitative analysis based on the operational expertise. It was found that in general, both analyses have similar severity classes for most of the hazards/basic types of hazards. Meanwhile discrepancies for some hazards/basic types of hazards have been identified and can be explained by the limitation of both analyses and by the need for refinement of the preliminary work done in Phase 2 of PASS Project.

The following improvement fields / activities to be undertaken in Phase 3 of PASS Project have been identified and concern the consolidation of Phase 2 material:

- Harmonize the severity results in both analyses. This would also imply to revisit the definition of some hazards.

- Validate / refine the barriers identified for each hazard in the quantitative safety analysis by operational experts or simulations and the assigned severity classes.



- Validate the probabilities of success/failures of each barrier by operational experts or by the fast time simulations results made in the frame of PASS.

Then, the main activity of Phase 3 willl consist in determining how the system architecture (encompassing people, procedures, equipment) could be made safe, and as such in deriving safety requirements/recommendations. That could be done in developing fault trees based on a generic high-level architecture in order to identify the causes and combination of causes leading to each OH. After the identification of the hazards causes, the next step will consist in allocating the safety objectives and in deriving the corresponding (probabilistic and / or qualitative) safety requirements / recommendations to elements of the system.



1. Introduction

1.1. PASS project

1.1.1. PASS (Performance and Safety Aspects of STCA – Full Study) is a EUROCONTROL project with the objective to study performance and safety aspects of Short Term Conflict Alert (STCA), including human performance aspects and consideration of interactions between operational use of STCA and Airborne Collision Avoidance System (ACAS) [1].

1.1.2. The fourth Work Area (WA4) of the PASS project specifically addresses the safety aspects of STCA. Both qualitative and quantitative safety analyses are performed with a specific focus on the identification and assessment of operational factors, in addition to the environmental and technical factors, which may influence the safety of joint STCA and ACAS operations.

1.1.3. The work is planned to be conducted in two steps:

- Step 1: Preliminary operational safety assessment (Phase 2); and

- Step 2: Consolidated operational safety assessment and requirement determination (Phase 3).

1.2. Safety Context

1.2.1. As the name suggests, the sole aim of a ground-based safety net is to positively contribute to the safety of the ATM system. In the EUROCONTROL STCA specification document [9], the STCA is given the following definition: “It is intended to assist the controller in preventing collision between aircraft by generating, in a timely manner, an alert of a potential or actual infringement of separation minima”. It is also recalled that its “presence is ignored when calculating capacity” as well as efficiency.

1.2.2. As per SRC28.06 policy [3], ground-based safety nets are now confirmed being part of the ATM system and they are, as such, subject to hazard identification, risk assessment and mitigation as required by ESARR 4. Indeed the real-time use of ground based safety nets can have unintended negative effects. They can induce new hazards, or degrade effects of existing hazards.

1.2.3. In particular, this policy indicates that the risk assessment and mitigation process for ground based safety nets should consider interaction between ground-based safety nets and similar airborne functions.

1.2.4. This safety assessment is focusing on the potential negative effects of STCA with TCAS. On the other hand, the mitigation by airborne safety nets of STCA-related hazards effects are not to be accounted for.



1.3. Operational Safety Assessment Objective

1.3.1. The operational safety assessment seeks to identify the errors and malfunctions of the ATM system which are related to the functioning of STCA or to the interoperability aspects of STCA and TCAS analyzed as an overall concept. The identified hazards are then used to derive safety objectives, as a first step for the identification of Safety Requirements and Recommendations in PASS Phase 3.

1.3.2. The EUROCONTROL ANS Safety Assessment Methodology [28] (SAM) identifies the need for “success” approach and “failure” approach to safety assessment1. The “success” case covers the reduction in risk of accident to air traffic that would otherwise exist, afforded by the desired functional and performance properties of the ATM system. On the other hand, the “failure” case covers the mitigation of anomalous behaviour of the ATM system that could induce a risk that would otherwise not have arisen.

1.3.3. The objective of the operational safety assessment in WA4 is to address the “failure” approach. The “success” approach (prevention of accidents) will be covered during the PASS fast-time simulations that will take place in Phase 3, as it is intended: “to compare the “initial” severity of the encounter without the effect of the controller’s instruction prompted by STCA and the “final” severity resulting from the pilot’s response to the controller’s instruction” (see [29]).

1.3.4. Step 1 of this operational safety assessment in Phase 2 of the PASS Project was conducted in three stages:

- Preliminary hazard identification (qualitative analysis): the objectives of this study were to identify a preliminary list of basic types of hazards based on the analysis of real ATC incidents studied in WA1 of PASS Project and integrated with inputs from other studies (see [23] and [24]), such as those promoted by the SPIN Task Force. Then for each of these basic types of hazards, operational scenarios were defined and were discussed in a workshop with operational experts, in order to assess the severity of their operational consequences and to identify possible further basic types of hazards with relevance for the study of the STCA-TCAS overall concept. The detailed analysis is provided in a separate document [4].

- Preliminary event tree analysis (quantitative analysis): the main objective of this study was to derive preliminary safety objectives based on the results of previous qualitative analysis as required by ESARR4 [2]. A safety objective is a qualitative or quantitative statement that defines the maximum frequency or probability at which a hazard can be accepted to occur. The method consisted in developing event trees, which are graphical means that enable to identify and quantify all possible outcomes of the hazards. The detailed analysis is provided in a separate document [5].

1 However, SAM currently does not provide much guidance on what the success case applies.



- Summary and comparison of the qualitative and quantitative analyses: the objectives are to provide the results of both studies and to compare them. The identification of future steps to be performed in Phase 3 of PASS Project, including the improvement fields, is also addressed. The summary and comparison are provided in the present document.

1.3.5. Step 2 of this operational safety assessment has not yet been initiated and will be conducted in Phase 3 of PASS Project.

1.4. Operational Safety Assessment Scope

1.4.1. The preliminary hazard identification (qualitative analysis) [4] pointed out the errors and malfunctions of the ATM system which are related to the functioning of STCA or to the interoperability aspects of STCA and TCAS analyzed as an overall concept. The study did not investigate the hazards related only to TCAS. These are considered out of the scope of the PASS project and have been already covered by other safety studies, such as those made in the context of the IAPA project.

1.4.2. The diagram in Figure 1 represents the area of interest of the analysis delimited by the light grey area, which excludes the part of the nominal sequence only referred to TCAS. It is a combination of the STCA and TCAS loops, and is aimed at depicting the nominal sequence of events in case of activation of a TCAS RA.

Figure 1: The transition between STCA and TCAS control loops in case of TCAS RA

1.4.3. In the preliminary event tree analysis (quantitative analysis) [5], the part of the ATM system under assessment comprises the controller(s) assisted by STCA to prevent collision between aircraft. The difference with the qualitative analysis is that in the quantitative study, pilots are considered external to the system under assessment. When controllers’ avoiding instructions are issued, the presence or

STCA AlertReceived

Resolving ActionDetermined

InstructionIssued

Read-backReceived

ProgressObserved

ImplementationVerified

SituationObserved

InstructionReceived

Instruction AcceptedAnd Acknowledged

Aircraft Reconfigured(auto/manual)


NavigationContinued

Ground Air

STCA AlertReceived


InstructionIssued

Read -backReceived

ProgressObserved


SituationObserved

InstructionReceived




NavigationContinued

Ground Air

TCAS TAReceived

TCAS RAReceived


RA reported(if deviating

from clearance)

Implementation Verified

Clear ofConflict Adv.

Received

ResumeClearance

Report

RA ReportReceived

Resume ReportReceived

Resume ReportAcknowledged

Ra Report Acknowledged

XXXX

XXXX

STCA AlertReceived


InstructionIssued

Read-backReceived

ProgressObserved


SituationObserved

InstructionReceived




NavigationContinued

Ground Air

STCA AlertReceived


InstructionIssued

Read -backReceived

ProgressObserved


SituationObservedSituationObserved

InstructionReceived




NavigationContinued

Ground Air

TCAS TAReceived

TCAS RAReceived


RA reported(if deviating

from clearance)

Implementation Verified

Clear ofConflict Adv.

Received

ResumeClearance

Report

RA ReportReceived

Resume ReportReceived

Resume ReportAcknowledged

Ra Report Acknowledged

XXXX

XXXX



not of STCA in assisting controllers is assumed not to influence pilots’ manoeuvers. Meanwhile the pilots’ behaviour to controllers’ instructions (either by implementing or not the ATC instructions) is considered in the event trees during the effects analysis of the hazards.

1.5. Next figure illustrates the system under assessment considered in the quantitative analysis [5] (STCA and controllers’ actions triggered by STCA in the grey boxes) interacting with other external components in a given operating environment (En-route or TMA). External components encompass pilots’ related actions after receiving an avoiding instruction. Note that technical aspects related to STCA, i.e. the components providing information to STCA to generate alerts such as the Surveillance Data Processing, Environment Data Processing and Flight Data Processing, have not been illustrated in this figure.

Figure 2: Boundaries of the System under Assessment

1.6. Document overview

1.6.1. Section 1 of this document presents the objectives and the scope of both preliminary safety analyses realised in Phase 2 of PASS Project.

1.6.2. Section 2 includes the results of the qualitative analysis: the preliminary identification of basic types of hazards along with the preliminary severity assignment.

1.6.3. The results of the quantitative safety analysis are provided in section 3, which includes the scope redefinition of the hazards and the various steps followed to finally derive preliminary safety objectives to hazards.

1.6.4. Then section 4 compares and comments the safety results of both analyses.

1.6.5. Finally section 5 identifies the proposals for future activities to be undertaken in Phase 3 of PASS Project, including some areas for improvement.

STCA Alert

SituationAssessment

by ATCo

Pilot actions(- maneuversaccordingto ATC instruction)- Manœuvresaccord ingRA receivedon board- RA report to ATC - Visuallydetectsa conflict)

ATCoinstruction

System under assessment

ENVIRONMENT:-TMA vs. En -route;-Traffic density …

Short-termconflict

STCA Alert

SituationAssessment

by ATCo


ATCoinstruction



Short-termconflict

STCA Alert

SituationAssessment

by ATCo


ATCoinstruction



Short-termconflict



2. Preliminary Qualitative Safety Analysis

2.1. Identification of Basic Types of Hazards

2.1.1. Based on the analysis of ATC incidents extracted from the “Analysis of real ATC Incidents” (WA1-WP4, [14]), the study has first identified 20 basic types of hazards, classified into the following 7 main categories:

STCA vs Conflict

ATC vs STCA

ATC vs TCAS

Crew vs ATC Avoiding Instruction

Crew vs TCAS

Crew vs RA Report

STCA vs TCAS.

2.1.2. The list of these basic types of hazards is provided in Table 1 and the detailed definition of each basic type of hazard is provided in annex A.

2.1.3. The study has then elaborated a list of 57 detailed scenarios, based on a combination of same basic hazards with different contributing factors.

2.1.4. In this study, contributing factors are elements that may cause or be frequently associated to the basic type of hazard identified. For example for the basic type of hazard ‘lack of STCA genuine alert”, the main contributor factors or causes identified were the following ones: Level bust and Low vertical threshold in STCA parameters for TMA, STCA processing failure, Radar Processor Failure, Erroneous implementation of STCA Rule Set, SSR code/flight ID erroneously inserted in suppression list, Error in implementation of STCA Parameter Region, Transponder Failure.

2.2. Severity Assessment

2.2.1. A preliminary assessment of the severity of each scenario has been conducted, based on individual expert judgment and on a brainstorming with three operational experts in a dedicated workshop. An example of the severity analysis for the category “STCA versus Conflict” is provided in annex B and the results at the level of the basic types of hazards are summarized in Table 1. Indeed the effects on safety of the different scenarios identified for the same basic hazard are generally the same.

2.2.2. The classification of the severity has been made using as reference the Severity Classification Scheme recommended in the SAM Methodology guidance material (SAM v2.1, SAF.ET1.ST03.1000-MAN-01-01-03-D), which is compliant with the Severity Classification Scheme provided in ESARR4 [2], and as criterion the evaluation of the worst credible effect.



Category Basic Type of Hazard Severity

Class

Lack of genuine STCA alert 3

Late STCA alert (i.e. below 20s before loss of separation) 3

Delayed STCA alert (between 20s and 40s before a loss of separation)

4

False STCA alert 3/4

STCA vs. Conflict

Nuisance STCA alert 3/4

Lack of ATC reaction to genuine STCA alert 3

Late ATC reaction to genuine STCA alert 4

Insufficient ATC reaction to genuine STCA alert 3/4

ATC vs. STCA

Incorrect ATC reaction to genuine STCA alert 3/4

TCAS RA and compatible ATC instruction simultaneously received by crew

5

TCAS RA and contrary ATC instruction simultaneously received by crew

4

Avoiding instruction by ATC contrary to subsequent TCAS RA 4

Avoiding instruction by ATC contrary to ongoing TCAS RA 3/4

Compatible instruction issued to a/c after TCAS RA report by crew

5

ATC vs. TCAS

Contrary instruction issued to a/c after TCAS RA report by crew

3/4

Lack of TCAS RA report by crew 4/5 Crew vs. RA

report Late TCAS RA report by crew (40 seconds after TCAS RA) (after the clear of conflict)

5

Time overlap between STCA alert and TCAS RA (less than 15s before the TCAS RA activation)

4

STCA alert after TCAS RA 3 STCA vs.

TCAS

STCA alert after CoC2 advice by TCAS 4

Table 1: Severity Assessment based on Qualitative Analysis

2 Clear of Conflict



2.2.3. Some basic types of hazards have been assigned two severity classes. According to the operational experts who participated in the Safety Workshop, the choice between one severity class to another was difficult to make because it depends on some influencing factors that may vary.

2.2.4. For example, in the case of “false alert” severity 3 or 4 depends on the following variables:

False alert rate (e.g. less/more than 4 false alerts per hour per sector)

Extension of the problem (e.g. less/more than 10% of the concerned ACC sectors

Traffic density (e.g. traffic near to nominal sector capacity versus low traffic density



3. Preliminary Quantitative (Event Tree) Analysis

3.1. The aim of this analysis is to determine how safe the system needs to be, and as such to define Safety Objectives.

3.2. STCA-related failures will be analysed during the Fault Tree Analysis performed in Phase 3 of PASS (see more details of this step in section 5), as the hazards have been defined at the boundary of the system and STCA is part of the system under assessment. Mitigation means external to STCA have been identified during the event tree analysis as explained in the following sub-sections:

3.3. Scope Redefinition

3.3.1. The basic types of hazards defined in the previous qualitative analysis [4], have been identified at different levels. They map onto the system as shown in the Figure 3 below.

3.3.2. According to Eurocontrol SAM (Safety Assessment Methodology, FHA – Chapter 3) they should be defined at the boundary of the ATM system under assessment with the environment / external systems in order to be as close as possible to the operations.

STCA Alert

Situation Assessment

by ATCo

Pilot actions(- maneuvers according to ATC instruction)- Manœuvres according RA received on board- RA report to ATC - Visually detects a conflict)

-Lack of genuine STCA alert-Late STCA alert-False STCA alert-Nuisance STCA alertTime overlap between STCA alert and TCAS RA (less than 15s before the TCAS RA activation)STCA alert after TCAS RASTCA alert after CoC advice by TCAS

-Lack of ATC reaction to genuineSTCA alert-Late ATC reaction to genuine STCA Alert-Insufficient ATC reaction to STCA-Incorrect ATC reaction to STCA

ATCoinstruction


ENVIRONMENT:-TMA vs. En-route;-Traffic density …Short-term conflict

Hazards defined atsystem boundary

-TCAS RA and compatible/ contrary instruction simultaneously received by crew-Avoiding instruction by ATC contrary to subsequent TCAS RA-Avoiding instruction by ATC contrary to ongoing TCAS RA-Compatible/Contrary instruction issued to a/c after TCAS RA report by crew

Figure 3 : Failures and Hazard Identification Level

3.3.3. Hence, the following hazards have been defined at the level of “ATCO instruction”:



Hazard Title Hazard Definition

OH1: Lack of ATCO instruction to solve a short-term conflict

The ATCO does not issue any avoiding instruction, although there is a real short-term conflict with Loss of Separation (LOS) (See qualitative safety analysis [4], §3.4.4).

OH2: Late ATCO instruction to solve a short-term conflict - no interaction with TCAS RA

The ATCO issues an avoiding instruction with delay such that he/she cannot prevent LOS but with time prior to a potential TCAS RA.

OH3: Avoiding instruction by ATCO received shortly before TCAS RA

The ATCO, unaware of the TCAS RA that the crew is going to receive, issues an avoiding instruction (See qualitative safety analysis [4], §3.4.9)

OH4: TCAS RA and ATCO instruction simultaneously received by crew

The ATCO -unaware of the imminent TCAS RA- issues an avoiding instruction which is received by the crew at about the same time as the TCAS RA (See qualitative safety analysis [4], §3.4.9).

OH5: Avoiding instruction by ATCO received after an ongoing TCAS RA

The ATCO issues an avoiding instruction while the flight crew has already received a TCAS RA– (See qualitative safety analysis [4], §3.4.9).

OH6: Insufficient ATCO instruction to solve a short-term conflict

The ATCO issues an avoiding instruction to solve a short-term conflict, which does not allow to maintain or restore separation (See qualitative safety analysis [4], §3.4.4).

OH7: Incorrect ATCO instruction to solve a short-term conflict

The ATCO issues a corrective instruction to solve a short-term conflict that leads to a reduction of the safety margins instead of an increase (See qualitative safety analysis [4], §3.4.4).

Table 2: List of Hazards for the Event Tree Analysis

3.3.4. The interaction between the prevention of collision by ATCOs assisted by STCA and the airborne safety net, i.e. TCAS, is explored through the analysis of OH3 to OH5 effects and related Safety Objectives Derivation.

3.3.5. The detailed traceability between the basic types of hazards of the qualitative analysis and the hazards defined in the quantitative analysis is provided in annex E of [5].

3.3.6. Delayed STCA alert (between 20s and 40s before a loss of separation to occur) was not considered as a hazard in the quantitative analysis, as in the case of nominal sequencing (ATC=> pilot => aircraft manoeuvres), the conflict can be resolved before a LOS to occur.



3.4. Safety Methodology

3.4.1. The following steps have been defined to derive safety objectives:

- ATM Safety Targets (corresponding to the maximum tolerable frequency of occurrence of effects directly caused by ATM for the severity classes 1 to 4 according to ESARR4) have been apportioned to the system under assessment and the retained Safety Targets are provided in the following table. The detailed process developed to derive these Safety Targets is explained in annex C of this document.

Severity Class

Safety Target

[flight hour]

1 3.0E-09

2 3.0E-06

3 3.0E-05

4 3.0E-03

Table 3: Apportionment of ATM Safety Targets to be considered

- Barriers of hazards, which are mitigation means that help in detecting and recovering from a hazard, have been identified. It is recalled that as recommended by ED125 [6] barriers that have to be excluded from the risk mitigation strategy include the following ones,:

o Airborne Safety Nets;

o Mitigation means that have already been taken into account as internal to the system under assessment.

- Afterwards, the efficiency of each barrier has been determined. It corresponds to a quantitative probability of success and failure of the barrier. These probabilities have been obtained from other projects, such as ASARP [8], FARADS, from safety engineering judgement, from WA1 documents (Monitoring of ATC incidents, using results from “Analysis of SNET performance based on monitoring data” PASS document (doc 64) [13], “Descriptive analysis of observed SNET performance” PASS document (doc 90) [7], etc) and some others from initial values from WA2.

- An event tree diagram has been used to formalise the process and the results of the safety analysis. In the Event Tree, each branch level describes a barrier. The trees are typically built on a binary accounting for the “success” or “failure” of the barrier or mitigation means in becoming effective. At the end of each branch, the effects of the hazards have been assessed along with the corresponding Severity Class as per ESARR4 [2], and the probability that the hazard generates that effect is calculated based on the success/failure probability of each concerned barrier. An example of Event Tree is provided in annex D.



3.4.2. Finally Safety Objectives have been calculated based on the Safety Targets (ST) provided in Table 3, the probability of effects (Pe) and the number of hazards identified for this severity class using the following formula:

NiPei

iSTSOi

* , where i corresponds to each Severity Class (i from 1 to 4), Pei is

the probability for the hazard to have a severity SCi and Ni to the number of hazards having credible effects in that severity class i.

3.4.3. As each hazard can have as many safety objectives as the operational effects identified in the event tree, only one Safety Objective is retained for each hazard corresponding to the most stringent frequency of occurrence of the hazard. This ensures that all safety objectives are satisfied for that hazard.

3.4.4. One may argue that an event tree is a static representation of events relationship. As for static fault trees, event trees use traditional Boolean logic gates (AND; sometimes OR) to represent the combinations of events (i.e. success/failures of barriers) in an inductive way (rather than deductive way for fault trees). Meanwhile the temporal relationship of events (i.e. the relationship between STCA alert, Controllers avoiding instruction and TCAS RA) has been assessed, as far as practical, through the definition of hazards (see OH2 to OH5, where different timeframes have been defined for the reception of ATC avoiding instruction on board the aircraft compared to the TCAS RA triggering time) and through the definition of the barriers in the event trees, which try to be time sequential. Note that barriers represented in the event trees developed in [5], are ANDed, and that the change of the barriers sequence may not modify, in principle, the main outcomes, i.e. the probability of effects (Pe) corresponding to each path of the event trees.

3.5. Preliminary Safety Objectives Derivation Results

3.5.1. This table below provides a summary of the severity (SC), the probability of effects (Pe values) and Preliminary Safety Objective for each hazard.

3.5.2. The severity assigned to each hazard in the following table corresponds to the severity from which the most stringent Safety Objective has been derived.

3.5.3. Safety Objectives are initially expressed per flight.hour but for a better understanding they have also been converted into Safety Objectives per year. Two examples have been selected to do this unit conversion and are provided in the following table, as this conversion depends on the volume of air traffic (number of flight hours) that a given Air Traffic Control Unit (ATCU) handles in a year:

- ATCU-1 has a volume of 100 000 flight hours per year (corresponding to, e.g., one area control centre in France) and

- ATCU-2 has a volume of 500 000 flight hours per year, representing a more complex ATC centre (corresponding to an area control centre equivalent to the size of Maastricht ACC).



3.5.4. Safety objectives have been computed based on the formula presented in paragraph 3.4.2, using a conservative approach where the Safety Targets per severity class have been uniformly distributed towards the 7 hazards (and as such N equals to 7 in that formula).

Hazard SC Pe

Value

Safety Objective

[Per flight.hour]

Safety Objective ATCU -1 100 000

flight hours per year

Safety Objective ATCU -2 500 000

flight hours per year

OH1: Lack of ATCO instruction to solve a short-term conflict - no interaction with TCAS RA

2 6.0E-02 7.14E-06 0.7 event

every year/ATCU

3.5 events every

year/ATCU OH2: Late ATCO instruction to solve a short-term conflict - no interaction with TCAS RA

3 1.1E-02 3.91E-04 39 events

every year/ATCU

196 events every

year/ATCU OH3: Avoiding instruction by ATCO received shortly before TCAS RA

3 9.9E-01 4.31E-06 0.4 event

every year/ATCU

2 events every

year/ATCU OH4: TCAS RA and ATCO instruction simultaneously received by crew

3 9.9E-01 4.31E-06 0.4 event

every year/ATCU

2 events every

year/ATCU OH5: Avoiding instruction by ATCO received after an ongoing TCAS RA

3 9.9E-01 4.31E-06 0.4 event

every year/ATCU

2 events every

year/ATCU OH6: Insufficient ATCO instruction to solve a short-term conflict

3 1.1E-02 3.91E-04 39 events

every year/ATCU

196 events every

year/ATCU

OH7: Incorrect ATCO instruction to solve a short-term conflict

3 9.2E-01 4.67E-06 0.5 event

every year/ATCU

3 events every

year/ATCU

Table 4: Preliminary Safety Objectives

3.5.5. These Safety Objectives may be quite demanding for the following main reasons:

- In the safety assessment, it is assumed that a short-term conflict will end up with a loss of separation if no avoiding action is been issued or followed by the contacted flight crew

- The likelihood of occurrence of short-term conflicts is included in the likelihood of occurrence of each hazard, e.g. there is a need to combine the probability of having a conflict between two aircraft with the probability that the ATC controller does not react, for OH1 to occur (either ATC controller does not intervene to a genuine STCA Alert or lack of genuine STCA alert and ATC controller does not detect there is a short-term conflict).

- Moreover these Safety Objectives are derived from the risk mitigation strategy applied, which excludes the airborne safety net from the barriers that can mitigate the effects of the hazards.



4. Comparison of Both Safety Analyses

4.1. The objective of this section is to compare the results obtained from both quantitative and qualitative analyses.

4.2. The comparison is made between the severity class determined for the hazards in the quantitative analysis based on the event trees, and the severity assessed for each basic type of hazard in the qualitative analysis based on the operational expertise.

4.3. In the analysis made on the traceability between the basic types of hazards of the qualitative analysis and the hazards defined in the quantitative analysis, it was found that a given basic type of hazard could be a cause to several hazards. On the other hand, the basic types of hazards of “ATC versus TCAS” category are effects to hazards OH2 to OH5 as the distinction between compatible versus contrary instruction to TCAS RA has been made during the event tree analysis. The various relationships are detailed in the following table.

4.4. The conclusion of this comparison is made in the last column of the following table and consists in providing the commonalities and the differences (in bold text) for each hazard.



Quantitative Analysis (Event Tree Analysis) Qualitative Analysis (Worst Credible Case) Comparative Analysis

Hazard SC Basic Type of Hazard SC Comment(s)

Lack of genuine STCA alert 3

Lack of ATC reaction to genuine STCA alert 3 False STCA alert3 3/4

OH1: Lack of ATCO instruction to solve a short-term conflict

2

Nuisance STCA alert3 3/4

- The quantitative analysis maybe too conservative/pessimistic - The qualitative analysis maybe too optimistic (large separation reduction seems to be a valid worse credible case)

Late STCA alert 3 Consistency between both analyses

Late ATC reaction to genuine STCA alert 4 Possible inconsistency between the definition of “late STCA alert” and “late ATC reaction” in the qualitative analysis

False STCA alert3 3/4

OH2: Late ATCO instruction to solve a short-term conflict - no interaction with TCAS RA

3

Nuisance STCA alert3 3/4 Consistency between both analyses

Avoiding instruction by ATC contrary to subsequent TCAS RA

4

Late ATC reaction to genuine STCA alert 4 The qualitative analysis maybe to optimistic

Late STCA alert 3 False STCA alert3 3/4

OH3: Avoiding instruction by ATCO received prior to a subsequent TCAS RA

3


3 If too many false/nuisance alerts, the controller may disregard a genuine STCA alert leading to “late ATC instruction to solve a short-term conflict (OH2 to OH5) or even to OH1 “lack of ATC instruction to solve a short-term conflict. On the other hand the supervisor might also decide to switch off the STCA in the ATSU if the number of false alerts is very high.



Quantitative Analysis (Event Tree Analysis) Qualitative Analysis (Worst Credible Case) Comparative Analysis

Hazard SC Basic Type of Hazard SC Comment(s)

TCAS RA and compatible ATC instruction simultaneously received by crew

5

TCAS RA and contrary ATC instruction Simultaneously received by crew 4

Late ATC reaction to genuine STCA alert 4

The qualitative analysis maybe too optimistic

Late STCA alert 3 False STCA alert3 3/4

OH4: TCAS RA and ATCO instruction simultaneously received by crew

3


Avoiding instruction by ATC contrary to ongoing TCAS RA

3/4 Consistency between both analyses


5 - The qualitative analysis maybe too optimistic - The quantitative analysis maybe too conservative



Late ATC reaction to genuine STCA alert 4 The qualitative analysis maybe too optimistic Late STCA alert 3 False STCA alert3 3/4

OH5: Avoiding instruction by ATCO received after an ongoing TCAS RA

3


OH6: Insufficient ATCO instruction to solve a short-term conflict

3 Insufficient ATC reaction to genuine STCA alert


Incorrect ATC reaction to genuine STCA alert

3/4

Late STCA alert 3 Consistency between both analyses OH7: Incorrect ATCO instruction to solve a

short-term conflict 3

Late ATC reaction to genuine STCA alert 4 The qualitative analysis maybe too optimistic

Table 5: Comparison between both Safety Analyses



4.5. In general, both analyses have found similar severity classes between the hazards and the basic types of hazards.

4.6. Meanwhile discrepancies for some hazards/basic types of hazards have been identified and can be explained by the limitation of both analyses and by the need for refinement of the preliminary work done in Phase 2 of PASS Project. The improvement fields to be undertaken in next phase, Phase 3, are developed in section 5.

4.7. For OH1 (Lack of ATCO instruction to solve a short-term conflict) the quantitative analysis concluded that a large separation reduction with no control by ATCO nor pilot (severity class SC2) was the worst credible effect based on the quantification of the hazard outcomes and the safety objective derivation. On the other hand the qualitative analysis considered that a lack of ATCO reaction to a genuine STCA alert for a conflict leading to any kind of loss of separation (including minor LOS) can be a considerably common event when dealing with some kind of vertical geometries. In such cases STCA often alerts only shortly before LOS, especially in case of a minor LOS, but will very rarely lead to serious incidents, in which both the crew and the ATCO are unable to control and recover from the situation. So it was argued that a lack of ATC reaction alone will not be sufficient to cause a serious incident, without incorporating other possible factors. A SC3 was then assigned, taking into account that a SC2 would have implied a too ambitious safety objective. OH1 assessement is thus to be reviewed in both analyses.

4.8. For OH2 (Late ATCO instruction to solve a short-term conflict - no interaction with TCAS RA) the quantitative analysis concluded that a large separation reduction under control by ATCO or pilot was the worst credible effect (SC3). However in the qualitative analysis, a SC4 was assigned to reflect the need to distinguish the severity of this case from the severity of “Lack of ATC reaction to STCA”. OH2 assessement is thus to be reviewed in both analyses.

4.9. In the quantitative analysis OH3, OH4 and OH5 hazards encompass the conflicting situation, in which a TCAS RA is being issued meaning that in most of the cases, the situation is already critical and a SC3 was thus assigned in these three cases. On the contrary, in the qualitative analysis, the severity was assigned taking into account the different operational impact of an ATCO’s instruction received by crew before or after a TCAS RA, with the first case being considerably frequent in case of TCAS activation (TCAS is intended to be a last resource, after ATC has failed in mantaining separation).. An harmonization of these hazards definition and scope is thus to be made between both analyses.



5. Future Steps

5.1. The following improvement fields / activities to be undertaken in Phase 3 of PASS Project have been identified and concern the consolidation of Phase 2 material:

- Harmonize the severity results in both analyses, especially for OH1 and OH2. This would also imply to revisit the definition of the hazards (especially for OH3 to OH5).

- Validate/refine the barriers identified for each hazard in the quantitative safety analysis by operational experts or fast-time simulations and the assigned severity classes.

- Ensure that TCAS has not been accounted for as a mitigation means for the hazards in the event trees and exclude TCAS benefit in the calculation of the safety objectives.

- Validate the probabilities of success/failures of each barrier by operational experts or by the fast time simulations results made in the frame of PASS. In particular:

Refine the assumptions (see ASSUMP-18, 19 and 20 in [5]) that have been made on the probabilities that a pilot follows an RA while ATC avoiding instruction is received just before, simultaneously or just after a TCAS RA, and respectively addressed in hazards OH3, OH4 and OH5. Indeed in the preliminary analysis the same probability has been used for the three different situations. Meanwhile there is a feeling that the most critical situations are when the pilot simultaneously receives the TCAS RA and the ATCO avoiding instruction or when the pilot receives the ATCO avoiding instruction and shortly after a TCAS RA, as he/she might be misled and might manoeuvre according to TCAS RA with some delay. Assumptions made on the Pilots’ behaviours with respect to TCAS RA (e.g. rate of non compliance) could be consolidated with data from SIRE + Project [26], which collected and anaylsed 56 RA encounters during a three month period in Paris TMA.

Try to identify any difference with respect to the barriers effectiveness between TMA and En-Route airspaces. Indeed during the 27th October 2009 RA Downlink Workshop [27], it was stated that pilot compliance with RAs decreases in lower airspace [26].

Refine the assumptions (see ASSUMP-3 and 4 in [5]) that have been made on the probability that a short-term conflict, may lead to minor loss of separation or to a large reduction in separation using the results from the fast time simulations results made in the frame of PASS

- Try to refine the hazard definition with more precise timeframe especially for OH3.



5.2. Then, the main activity of Phase 3 willl consist in determining how the system architecture (encompassing people, procedures, equipment) could be made safe, and as such in deriving appropriate safety requirements / recommendations. That could be done in developing fault trees based on a generic high-level architecture in order to identify the causes and combination of causes leading to each OH. After the identification of the hazards causes, the next step will consist in allocating the Safety Objectives and in deriving the corresponding (probabilistic and / or qualitative) safety requirements / recommendations to elements of the system.

5.3. It is also proposed to explore the possibility to contribute to the current Outline Safety Case of STCA [30] on specific issues related to the interaction of STCA with TCAS.



6. References

[1] ‘PASS – Project Management Plan’, EUROCONTROL, PASS/WA0/WP1/01/D version 1.1, 28th January 2008.

[2] EUROCONTROL Safety Regulatory Requirement - ESARR4 Risk assessment and Mitigation in ATM (2001)

[3] SRC28.06, 15/03/07 ITEM 6.3

[4] ‘PASS - Preliminary Identification of STCA-TCAS Related Hazards’, EUROCONTROL, WA4/WP1/91W, version 3.0, 17-04-2009

[5] ‘PASS - Preliminary Event Tree Analysis’, EUROCONTROL, WA4/WP3/102/W, version 1.1, 20-10-2009

[6] Process for specifying risk classification scheme and deriving safety objectives in ATM, pending document (ED-125)

[7] ‘PASS - Descriptive analysis of observed SNET performance’, PASS, PASS/WA1/WP4/90/W, version 1.0, 20/01/09

[8] Final Report on post-RVSM ACAS full-system safety study, ASARP/WP6/58/D, version 0.2, 15/02/06

[9] EUROCONTROL Specification for Short Term Conflict Alert, edition 1.1, 19/05/09

[10] ESARR4 Advisory Material on Safety Nets, edition 0.5, 15/04/09, working draft (under review)

[11] DOT/FAA/TC-07/22, Human Factors Analysis of Safety Alerts in Air Traffic Control

[12] ’PASS - European STCA environment’, EUROCONTROL, PASS_WA2_WP1_09W version 1.1, 24th April 2008

[13] ’PASS - Analysis of SNET performance based on monitoring data’, EUROCONTROL, PASS_WA1_WP5_64D, version 1.3, 29th April 2009

[14] ‘PASS - Consolidated analysis of a set of events of interest’, EUROCONTROL, PASS_WA1_WP4_42W, version 1.1, 2nd December 2008

[15] ICAO, PANS-OPS Volume I, Part III

[16] Update of contingency tree event probabilities, ACAS Safety Analysis post-RVSM, ASARP/WP6/44/W, version 1.3 2006/02/12

[17] ACAS RA Downlink, Combined FHA & PSSA Report, version 1.3, 31/05/2007

[18] “Limitations of the See-an-Avoid Principle”, research report, Australian Transport Safety Bureau, November 2004



[19] EUROCONTROL Safety Regulatory Requirement - ESARR2 Reporting and Assessment of Safety Occurrences in ATM, edition 2.0, 03-11-2000

[20] ’PASS Modified radar encounters with probable ATCO intervention’, EUROCONTROL, PASS_WA2_WP4_111W, version 1.1, 09th June 2009

[21] ’PASS Encounter Model for Safety-Net Related Occurrences – Specification, EUROCONTROL, PASS_WA2_WP5_75W, version 1.2, 07th September 2009

[22] ‘Main report for the: 2005/2012 integrated risk picture for Air Traffic Management in Europe’, EEC Note No. 05/06, issued: April 2006

[23] EUROCONTROL Guidance Material for Short Term Conflict Alert. Appendix D-2: Functional Hazard Assessment of STCA for ATCC Semmerzake

[24] EUROCONTROL Guidance Material for Short Term Conflict Alert. Appendix B-3: Outline Safety Case for STCA System

[25] EUROCONTROL, EAM 4 / GUI 6, Explanatory Material on Ground Based Safety Nets, Edition 0.2, 26th October 2009

[26] TCAS II performance in European TMAs - Part 1: Analysis Safety Issue Rectification Extension 2006-2008 Project - SIRE+ Project, Edition 1.1, 03rd February 2009

[27] Airborne Collision Avoidance System (ACAS) Resolution Advisory (RA) Downlink Workshop Report, 20 November 2009, Released version

[28] EUROCONTROL, 2006, Air Navigation System Safety Assessment Methodology, Ed 2.1

[29] ‘PASS – Performance metrics definition’, EUROCONTROL, PASS_WA2_WP8_120D, version 1.0, 15th July 2009

[30] EUROCONTROL Guidance Material for Short Term Conflict Alert, Appendix B-3 Outline Safety Case for STCA System, Edition 2.0, 19th May 2009



7. Annex A : Description of Basic types of Hazards – Qualitative Safety Analysis

This section presents and defines the basic types of hazards identified in the qualitative safety assessment [4] grouped in the 7 categories listed above. Although only a subset of these have been further addressed in the Detailed Hazard Analysis (Chapter 4 of [4]), a definition is provided for all the types wich are considered relevant. The titles of the basic types of hazards that have not been further analyzed are marked in a grey colour, also providing a methodological motivation for their exclusion.

7.1. STCA vs Conflict

Title Definition

Lack of genuine STCA alert STCA does not trigger although the conflict will results in a Loss of Separation (LOS) in the absence of ATC intervention

Late STCA alert STCA triggers less than 25s before LOS in En-

route and less than 20s in TMA4

Delayed STCA alert STCA triggers less than 40s before LOS but more than 25 before LOS in En-route and 20s before LOS in TMA

False STCA alert (Not derived from PASS WA1 Study)

STCA triggers an alert in a condition in which no LOS is predicted due to a technical malfunction of the STCA itself or of other systems connected with STCA.

Nuisance STCA alert (Not derived from PASS WA1 Study)

STCA triggers an unnecessary alert which has the potential of distracting the controller an/or fostering cry-wolf syndrome in the medium long-term

7.1.1. All the basic types of hazards in this category have been assessed. Nevertheless the possible technical failures originating these basic types of hazards –particularly “Lack of genuine STCA alert” and “False STCA alert” –will only be analyzed at a high level.

7.2. ATC vs STCA

4 The time thresholds identified here have been used as heuristic criteria to identify hazards from the analysis of ATC incidents in WA1 and to distinguish between “late” and “delayed” occurrences, based on operational considerations. The indicative value of 20s as minimum threshold to label an STCA alert as “late” is an approximation derived from the FAA human factors study on safety nets by Dino Piccione [Piccione 07]. All the presented time values, however, are not intended here as indications or requirements for the performance of STCA. Requirements concerning the performance of STCA will be indicated only as part of the final results of the project.



Title Definition

Lack of ATC reaction to genuine STCA alert

The ATCO does not perform any avoiding action, although the STCA alert corresponds to a real threat with LOS

Late ATC reaction to genuine STCA alert

The ATCO issues a corrective instruction more than 30s after a genuine STCA alert

Insufficient ATC reaction to STCA

The ATCO issues a corrective instruction in response to STCA which does not allow to maintain or restore separation

Incorrect ATC reaction to STCA

The ATCO issues a corrective instruction in response to STCA that leads to a reduction of the safety margins instead of an increase

Misuse of STCA by ATC (Not derived from WA1 Study)

The ATCO uses the STCA as a controller tool in order to decide how to provide separation between aircraft

Mistrust in STCA by ATC (Not derived from WA1 Study)

The ATCO tends to ignore all STCA alerts as its indications are considered unreliable

7.2.1. This category of basic types of hazards should encompass also the cases of misuse of STCA or controller mistrust with respect to the STCA.

7.2.2. An example of misuse is the consideration of STCA as a control tool rather than as safety nets. E.g. the controller uses the CFL feature of STCA as a “what-if” function, to detected the effects of a continued climb/descent and consequently decide on the kind of instructions to give to the flight crew.

7.2.3. An example of mistrust is the deliberate decision to ignore the STCA alerts as the ATCO does not rely on its performance, based on previous experiences (it can actually be a causal factors for a “Lack of ATC reaction” or “Late ATC reaction”).

7.2.4. Both misuse of STCA and mistrust are a potential threats for the safety of ATM. Nevertheless, for the sake of simplicity, the present study does not investigate further on them, assuming a correct behaviour of ATCOs and an active role of the ANSP in preventing mistrust phenomena by mean of adequate training.



7.3. ATC vs TCAS

Title Definition

TCAS RA and compatible ATC instruction simultaneously received by crew (Not derived from WA1 Study)

The ATCO -unaware of the imminent TCAS RA- issues a compatible avoiding instruction which is received by the crew at about the same time as the TCAS RA

TCAS RA and contrary ATC instruction simultaneously received by crew (Not derived from WA1 Study)

The ATCO -unaware of the imminent TCAS RA- issues a contrary avoiding instruction which is received by the crew at about the same time as the TCAS RA

Avoiding instruction by ATC contrary to subsequent TCAS RA (Not derived from WA1 Study)

The ATCO, unaware of the TCAS RA that the crew is going to receive, issues an avoiding instruction in opposite direction.

Avoiding instruction by ATC contrary to ongoing TCAS RA

The ATCO –unaware that the flight crew has just received a TCAS RA– issues an avoiding instruction in opposite direction.


The ATCO issues an avoiding instruction compatible with the TCAS RA, although s/he has just received a TCAS RA report by the flight crew .


The ATCO issues an avoiding instruction contrary to the TCAS RA, although s/he has just received a TCAS RA report by the flight crew .

7.4. Crew vs ATC Avoiding Instruction

Title Definition

ATC instruction not followed by crew The flight crew does not follow an ATC instruction issued in response to STCA.

Insufficient reaction of crew to ATC instruction

The flight crew does not react sufficiently to an ATC instruction issued in response to STCA.

Incorrect reaction of crew to ATC instruction

The flight crew does not react correctly to an ATC instruction issued in response to STCA (wrong manoeuvre).

Delayed crew reaction to ATC instruction

The flight crew reacts too late to an ATC instruction issued in response to STCA.

7.4.1. Although the concerned sequences of actions are part of the STCA and TCAS control loops, the basic types of hazards in this category have been excluded from the Detailed Hazard Analysis in [4]. Indeed it is not always easy to distinguish an ATC instruction in reaction to STCA, from the same kind of action simply performed as part of the normal controller activity. Further the behaviour of the crew in response to an ATC instruction triggered by STCA is considered to complex for being modelled in the context of PASS.



7.5. Crew vs TCAS

Title Definition

TCAS RA not followed by crew The flight crew does not follow the TCAS RA.

Insufficient reaction of crew to TCAS RA

The flight crew performs a manoeuvre in the direction required by the TCAS RA, but in a delayed manner or with insufficient ROD/ROC.

Incorrect reaction of crew to TCAS RA

The flight crew performs a manoeuvre in direction opposite to the one required by the TCAS RA.

Excessive reaction of crew to TCAS RA

The flight crew performs a manoeuvre in the direction required by the TCAS RA, but with an excessive ROD/ROC or continuing it considerably after the CoC indication by TCAS..

7.5.1. Also the basic types of hazards in this category have been excluded from the detailed analysis. Indeed, the safety analysis of this part of the TCAS loop has been already covered in other studies [OHA AIPA] and it is therefore considered out of the PASS scope.

7.6. Crew vs RA Report

Title Definition

Lack of TCAS RA report by crew The pilot does not report to the ATC that s/he has received a TCAS RA requiring a deviation from the current ATC clearance or instruction

Late TCAS RA report by crew The pilot issues a TCAS RA report to the ATC more than 40s after having received it.

7.7. STCA vs TCAS

Title Definition

Time overlap between STCA alert and TCAS RA

The STCA triggers less than 15s before the TCAS RA activation in one of the a/c concerned by the conflict

STCA alert after TCAS RA The STCA triggers later than the TCAS RA activated in one of the a/c concerned by the conflict

STCA alert after CoC advice by TCAS (Not derived from WA1 Study)

The STCA triggers later than the Clear of Conflict (CoC) annunciation of TCAS in one of the a/c concerned in the conflict

7.7.1. The basic types of hazards in this category might appear similar to the basic types of hazards in the category ATC vs TCAS, as controllers and pilots’ communications are currently the only existing connections between the functioning of TCAS on the airborne side and STCA on the ground side. The very



few cases in which the downlink of aircraft parameters to the CWP has been implemented represent an exception. In this context however, the situation of proximity between STCA and TCAS triggering time is considered as such an hazardous condition (even if the two system are technically independent) as it may foster other basic types of hazards, like the one identified in the category ATC vs TCAS.

7.7.2. The hazard “STCA alert after CoC advice by TCAS” represents an “extreme situation” which can occur in reduced radar coverage environments, with low radar updated, such as areas over the sea (unlike to be in ‘Core Area’).



8. Annex B : Example of severity Assessment – Qualitative Safety Analysis

8.1. The table below provides an extract of the severity assessment realised on the frame of the qualitative safety analysis, for “Lack of Genuine STCA Alert” basic type of hazard.

Unpredicted GOC

ID Hazard Description Main contributing factors Vert. Horiz. Effect on ATM Severity Ex.

Event

1.1 STCA does not alert genuine conflict caused by level bust, due to low vertical threshold parameter set for STCA in TMA

- Level bust - Low vertical threshold in STCA parameters for TMA

The controller may not become aware of the conflict to resolve it before a collision scenario develops.

3 G - 12

1.2 STCA does not alert genuine conflict due to STCA processor failure

- STCA Processor Failure - - - -


3 - -

1.3 STCA does not alert genuine conflict due to radar processor failure

- Radar Processor Failure - - - -


3 - -

1.4 STCA does not alert genuine conflict due to implementation error in STCA rule set

- Erroneous implementation of STCA Rule Set - - - -


3 - -

1.5 STCA does not alert genuine conflict because SSR code/ID of concerned flight was erroneously inserted in suppression list

- SSR code/flight ID erroneously inserted in suppression list - - - -


3 - -

1.6 STCA does not alert genuine conflict due to STCA parameter region erroneously implemented

- Error in implementation of STCA Parameter Region - - - -


3 - -

1.7 STCA does not alert genuine conflict due to transponder failure in a/c concerned by the conflict

- Transponder Failure - - - -


3 - -



9. Annex C: Process for apportionning the ATM Safety Targets

9.1. ESARR4 [2] has established a Target level of Safety for accidents (Severity Class SC1), 1.55E-08 flight.hour, which corresponds to the overall maximum frequency of accident whatever the type of accidents (e.g. mid air collision, controlled flight into terrain), directly caused by ATM for the ECAC area.

9.2. The European ED125 pending Document [6] proposes a maximum tolerable frequency for the other severity classes. The Risk Classification Scheme (RCS) that sets the maximum ATM Safety Targets to be used as input to the project is presented in Table 6.

9.3. Note that this safety assessment is focusing on current operations and not on future operations. Therefore no ambition factor has been considered to account for traffic growth.

Severity Class

ATM Safety Target

[flight hour]

1 1.0E-08

2 1.0E-05

3 1.0E-04

4 1.0E-02

Table 6: Safety Targets Based on ESARR4 Approach (ED125)

9.4. The main accidents categories where ATM can make a significant contribution either in causing or preventing accident are:

Mid-air collision

Controlled flight into terrain (CFIT)

Wake turbulence accident

Runway collision

Taxiway collision.

9.5. For the other accidents categories (loss of control in flight, take-off, landing, structural accident or fire/explosion), direct contribution of ATM is negligible.

9.6. The above categories are extracted from a Eurocontrol document [22] and are consistent with ESARR2 “Reporting and Assessment of Safety Occurrences in ATM” [19] accident categories with the exception of the “wake turbulence” one, but this latter was considered to be of potential importance for ATM in that document [22].

9.7. Two accident categories are applicable in the context of the PASS study. Indeed, the system under assessment (controllers assisted by STCA to prevent collision



between aircraft) plays an important role in the prevention of mid-air and wake turbulence accidents.

9.8. In this Eurocontrol document [22], an analysis of historical accident rates was performed for each of these above accident categories. It is also assumed that ATC is the only element of ATM that contributes to the direct causes of accidents.

9.9. The contribution of ATC direct causes to each accident category that involved commercial flights in 2005 in the ECAC region is provided in the following table and is directly extracted from that report. It appears that these two accident categories, mid-air collision and wake turbulence accident, contribute to 33.52% of total accidents directly caused by ATC.

Fatal accident frequency per flight in 2005

ATC Direct causes

Frequency of Fatal accident directly caused by ATC

% of fatal accidents directly

caused by ATC

Mid-air collision 5.40E-09 64.50% 3.48E-09 31.46%

Runway collision 3.30E-08 18.90% 6.24E-09 56.34%

Taxiway collision 3.40E-09 9.20% 3.13E-10 2.83%

CFIT 5.40E-08 1.50% 8.10E-10 7.32% Wake turbulence

accident 3.30E-09 6.90% 2.28E-10 2.06%

Loss of control in flight*

1.30E-07 -

Loss of control in take-off*

4.80E-08 -

Loss of control in landing*

6.40E-08 -

Structural accident 1.60E-08 -

Fire/explosion 2.20E-08 -

TOTAL 3.79E-07 - 1.11E-08 100.00%

Table 7: Contributions of ATC to Direct Causes of Fatal Aircraft Accidents for Commercial Flights within ECAC Region in 2005

9.10. *Potential ATM contributions to these accidents categories have not yet been estimated.

9.11. In this report it is assumed an average flight duration of 1.5 hour.



9.12. It is proposed to retain a more conservative value, 30%, in order to keep safety margins due to the uncertainties of statistical data handling, and to apply it to the total ATM TLS for all severity classes. The final TLSs corresponding to the retained risk budget that is used to derive safety objectives to “mid-air collision prevention” task by ATC assisted by STCA, are provided in the following table:

Severity Class

Safety Target

[flight hour]

1 3.0E-9

2 3.0E-06

3 3.0E-05

4 3.0E-03

Table 8: Apportioned Safety Targets



10. Annex D : Example of an Event Tree Analysis (Quantitative Safety Analysis)

10.1. This annex provides the event tree of hazard OH3: “Avoiding instruction by ATCO received prior to a subsequent TCAS RA”.

10.1.1. Detailed Hazard Effects Analysis

10.1.2. The layout of the event tree is the following:

- The left-hand column shows the initiating event, i.e. the hazard in question.

- The second column from the right provides the consequences and the corresponding severity class of the various possible outcomes of the hazards.

- The intervening columns show the barriers that were identified as providing potential mitigation of the consequences of the hazard. The Q values show the probability that the related barrier would not be successful. The term “null” signifies that a barrier is not applicable in a given path.

- It is usual to show the mitigations in the order in which they are most likely to occur in order to aid understanding and help ensure that the logic is complete and correct.



Failure:Q=1.00

Success:Q=9.00e-1

Failure:Q=1.00e-1

Null:Q=1

Success:Q=8.00e-1

Failure:Q=2.00e-1

Null:Q=1

Null:Q=1

Success:Q=7.00e-1

Failure:Q=3.00e-1

Null:Q=1

Null:Q=1

Null:Q=1

Success:Q=9.92e-1

Failure:Q=8.00e-3

Null:Q=1:scenario ILarge loss of separation under control by ATC or pilot: SC3

9.00e-1

Null:Q=1:scenario IILarge loss of separation under control by ATC or pilot: SC3

8.00e-2

Null:Q=1:scenario IIILarge loss of separation under control by ATC or pilot: SC3

1.40e-2

Null:Q=1:scenario IVLarge reduction in separation with no control by ATCO nor pilot: SC2

5.95e-3

Success:Q=9.00e-1:scenario V

Large reduction in separation with no control by ATCO nor pilot: SC2

4.32e-5

Failure:Q=1.00e-1:scenarioVI

Accident: SC1 4.80e-6

Avoiding instruction byATCo received prior toa subsequent TCAS

RAw=1.00

Pilot maneuversaccording to RA -(ATC

intervention prior toRA)

Q=1.00e-1

ATC compatible tocoordinated RA

Q=2.00e-1

'See and avoid'principle

Q=3.00e-1

Conflict leads to largereduction in separation

rather than NMAC

Q=8.00e-3

NMAC instead ofcollision

Q=1.00e-1

Consequence Frequency

1.00

Figure 4: Event Tree “Avoiding instruction by ATCO received prior to a subsequent TCAS RA”.



10.1.3. Six possible scenarios have been identified when “Avoiding instruction by ATCO received prior to a subsequent TCAS RA” hazard occurs. It is assumed in that scenario that aircraft are in a collision course because there is an imminent TCAS RA.

10.1.4. If an ATC avoiding instruction is received before an imminent TCAS RA, it might confuse the pilot, who could decide to choose between them. Because ATCO/STCA and TCAS are two independent loops then an ATCO avoiding action and a TCAS RA can be contradictory. Normally, in the event of an RA, pilots shall follow the RA even if there is a conflict between the RA and an ATC instruction to manoeuvre.

10.1.5. If the pilot follows the TCAS RA a severity 3 is assigned to this scenario as it foreseen that aircraft will be close to each other. It can also lead, if combined with crew error (e.g. due to stressful situation) to “Excessive reaction of crew to TCAS RA” hazard. This is not addressed in the Event Tree.

10.1.6. But experience shows that pilots do not always do this (see §5.3.7 of [8]). The case where a small proportion of pilots may give precedence to a controller instruction over an RA is considered in the event tree. Then if the ATCO instruction is compatible to a coordinated RA (e.g. heading change), recorded as the second barrier in the event tree, aircraft trajectories will diverge as it is assumed that the other aircraft will follow the RA. A severity 3 is then assigned to that scenario II.

10.1.7. Manoeuvres opposite to the sense of an RA (e.g. an instruction to expedite climb when the RA requires a level-off) might result in a reduction in vertical separation with the threat aircraft. It might be ultimately mitigated by visual aircraft acquisition as the pilot is aware of the threat and manoeuvre accordingly (see scenario III), fourth barrier in the event tree. A severity 3 is also assigned to that scenario.

10.1.8. If the see and avoid” principle barrier fails and aircraft are close by, it will not always lead to a collision. Indeed, some conflict configurations exist where the minimum distance between aircraft is more than 500 ft but less than 50% of separation minima. In such a case, the hazard effect is a large reduction in separation without crew and ATC controlling the situation. Therefore a severity of 2 is assigned to this scenario (cf. scenario IV in the event tree above).

10.1.9. Finally it is generally reckoned that 10% of the cases, an NMAC could be a collision [8] (severity 2, see scenario V in case of NMAc and severity 1, see scenario VI for the accident sequence - mid-air-collision or wake vortex accident).

10.1.10. Barriers Summary

10.1.11. The following table provides a summary of the barriers that have been identified as mitigating the effects of the hazard “Avoiding instruction by ATCO received prior to a subsequent TCAS RA”.

10.1.12. The last column provides the probability of success of each barrier.



Barrier Description Probability of success

Pilot manoeuvres according to RA (ATC intervention prior to imminent RA)

In the event of a RA, pilots shall follow the RA even if there is a conflict between the RA and an air traffic control (ATC) instruction to manoeuvre

0.90

ATCO avoiding action is consistent with coordinated RA

ATCO avoiding action to solve a conflict can be given on the horizontal plane (heading change) or on a vertical plane ((expedite) climb, descent, stop climbing/descending…).

Because ATCO/STCA and TCAS are two independent loops then an ATCO avoiding action and a TCAS RA can be consistent or on the contrary contradictory.

0.80

”See and Avoid” principle

If visibility and conflict geometry permits, flight crew visually acquires the traffic aircraft proximity and thereafter takes appropriate action to avoid an NMAC or collision.

0.70

Conflict leads to large reduction in separation

This barrier deals with the conflict geometry. Given the environment, if there is a conflict that leads to larger reduction in separation it will not always lead to a near-mid air collision.

It means that the closest distance between aircraft will less than 50% of the separation minima but more than 500 ft.

0.992

NMAC is not a collision

It is generally reckoned that 10% of the cases, an NMAC could be a collision [8].

0.1

Table 9: Barriers of “Avoiding instruction by ATCO received prior to a subsequent TCAS RA” Hazard



10.1.13. Safety Objectives Derivation

10.1.14. The safety objectives for each severity class are provided in next table, along with the Pe.

Severity ST per hazard

[per f.h] Pe SO

[per f.h] SC1 4.29E-10 4.8E-06 8.93E-05 SC2 4.29E-07 6.4E-04 6.72E-04 SC3 4.29E-06 9.9E-01 4.31E-06

SC4 4.29E-04 -

Table 10: Safety objectives for “Avoiding instruction by ATCO received prior to a subsequent TCAS RA” hazard

10.1.15. The most stringent safety objective is obtained from severity class 3.



*** END OF DOCUMENT ***

Documents

Operational Safety Assessment Interim Report