Report of the Third ASAS Technical Interchange Meeting · Report of the Third ASAS Technical Interchange Meeting Action Plan 1 ... was held at NASA Ames Research Center near San Francisco,

Report of the Third ASAS Technical Interchange Meeting Action Plan 1

FAA/EUROCONTROL/Action Plan 1 - Version 1.2 - January 20, 2002 page 1

ACTION PLAN 1

FAA/EUROCONTROL COOPERATIVE R&D

Report of the Third ASAS TechnicalInterchange Meeting

Version: 1.2 Date: 20 January 2003

Executive summary

The Third Technical Interchange Meeting of the FAA/EUROCONTROL R&D Committee, AP-1 (AirborneSeparation Assistance Systems) was held at NASA Ames Research Center near San Francisco, Californiafrom October 21 - 23, 2002. The meeting was jointly sponsored by the FAA, NASA and EUROCONTROL,and attracted over 60 participants from Europe and the United States. Details of the Technical InterchangeMeeting, including copies of all the presentations and a list of participants, can be found athttp://asas.arc.nasa.gov/ .

The purpose of this workshop was to provide an opportunity for the participants to share experimental results,experience, and viewpoints in the areas of ASAS research, development, evaluation, and implementation.Participants were expected to contribute to clarifying technical and operational issues, and developing anapproach for resolution.

After an introductory presentation on the meeting objectives, followed by descriptions of ASAS applicationsbeing evaluated in Europe and the U.S., four technical sessions were held:

§ Session 1: ASAS Applications

§ Session 2: Safety in ASAS Applications

§ Session 3: Validation of ASAS Applications

§ Session 4: Systems and Architecture (Airborne & Ground)

Under the leadership of session co-chairs, the session participants were given presentations on related topics.All the TIM participants attended Session 1, and then the remaining three technical sessions were heldconcurrently. On the final day, the session co-chairs briefed a summary of the discussions and outcome oftheir session, and a final wrap up session was held.

Action Plan 1 Report of the Third ASAS Technical Interchange Meeting

Page 2 FAA/EUROCONTROL/Action Plan 1 - Version 1.2 - January 20, 2002

ASAS application session

The session covered ASAS applications from the four categories defined in PO-ASAS ranging from situationawareness to self-separation for which recent experimental results are available. All ASAS applicationspresented were explicitly linked to PO-ASAS categories – providing an indication of common vocabulary andreference being shared.

Topics of interest included representative US and European operational needs to be addressed, actual results,lessons learned and their significance in terms of operational procedures, human factors or benefits. Actualmethodologies used in terms of validation techniques or safety assessments were specifically addressed insubsequent sessions.

All ASAS applications presented were addressing capacity-constrained airspace and the associated benefits– showing that the same operational issues were addressed between the US and Europe. Encouraging resultsand positive trends were displayed in all presentations, although a large number of validation issues wereidentified, followed by safety issues, and architecture issues. It should be kept in mind that due to the verynature of the TIM, the briefings were explicitly selected for their relevance to R&D, thus allowing for theidentification of issues still to be addressed.

Safety session

The session sought to present and share experience related to the evaluation of the safety of ASASapplications, and the development of methods to prove that ASAS applications are safe and to develop safeASAS applications.

Topics invited included the definition of safety, safety analyses for different categories of ASAS applications(both long and short term), safety analyses using different methodologies, the derivation of system functionaland performance requirements from the safety analyses, and the consideration of human behaviour in theseanalyses.

The group discussed the definition of a safe ASAS application, the metrics to be used to determine that it issafe, and the need for feedback between the various methods of study. The studies presented coveredanalyses based on requiring the risk of collision and the risk of wave vortex encounters be acceptably low,analyses of the required surveillance performances fed into the risk analyses, and investigations of the abilityof humans to contribute to the safety of ASAS applications.

The discussions and outcome of the session should prove extremely valuable in the drafting of the document‘Safety and ASAS applications’ under development within AP1.

Validation session

The session sought to provide an opportunity to discuss different approaches for evaluating expected benefitsand operational issues related to the use of ASAS, as well as assessing the impact of ASAS applications onthe ATM system in various areas, except safety for which there was a parallel session.

Topics of interest included the description and lessons learned from recent experimental design, performanceand results, including description of experimental conditions, hypotheses and metrics used. The presentationscovered many validation techniques (e.g., human in the loop experiments, fast-time studies, field/flight tests)



applied to ASAS applications, and the role of each of them was discussed. There were also many discussionsregarding the relationship between design and validation, as well as the role of validation with respect tocertification.

In general, the group advocated a more systematic and integrated approach in validation, which would provideconsolidation of the various assessments performed during the development process. A common validationframework between US and Europe was also considered essential for comparability of results and forguidance regarding R&D methods. The need for end-to-end validation, approval and certification wasacknowledged to fully test the concepts and gain user acceptance.

Architecture session

The purpose of the session was to present and to share recent developments in System Architectures tosupport ASAS applications, to exchange and consolidate views based on those developments, to evaluatecurrent co-ordination between airborne and ground research activities, and to promote harmonisation betweenthe United States and Europe.

Specific topics for which presentations were solicited included current aircraft equipage and ground systemsand assessments of their future evolution; information on current research and flight trials; cockpit decisionsupport systems; anticipated impact on current avionics, for example FMS (Flight Management System) andATM systems; ASAS integration with other data link applications, and mixed-environment services such asTIS-B.

Due to the numerous presentations for the session, there was no general discussion apart from thosefollowing the presentation themselves. There was an attempt to briefly formulate issues andrecommendations linked to architecture work. It should be kept in mind that due to the very nature of theTIM, these conclusions should not be considered as definitive ones.

TIM conclusions

It was clear at the end of the TIM that a wide range of ASAS applications are currently considered.o Some applications are close to maturity in terms of R&D. The associated issues are related to

demonstrating sufficient benefits, involving the users, the development of systems, certification andimplementation solutions.

o Other applications are longer term. They require R&D efforts and their feasibility is still to bedemonstrated.

The sessions were very productive and there were little time left during the wrap-up session to reallysummarise and identify the most important points.

To get the most valuable feedback from this Third ASAS Technical Interchange Meeting, thereader is invited to review the presentations and detailed discussions reported herein.



TABLE OF CONTENTSTable of Contents............................................................................................................................4

1. Introduction ...........................................................................................................................71.1. TIM objectives .........................................................................................................................................................................71.2. TIM organisation .....................................................................................................................................................................71.3. Briefings by the presenters.....................................................................................................................................................7

2. Review of the Introduction Session Briefings..............................................................82.1. Vic Lebacq (NASA/Ames)......................................................................................................................................................82.2. Rose Ashford (NASA/Ames)................................................................................................................................................82.3. Gene Wong (FAA)...................................................................................................................................................................82.4. Francis Casaux (CENA on behalf of EUROCONTROL) .....................................................................................................82.5. Dan Hawkes (JAA – UK CAA).............................................................................................................................................8

ASAS APPLICATION SESSION....................................................................................................9

1. Introduction ...........................................................................................................................91.1. Session objectives ...................................................................................................................................................................91.2. Session organisation ...............................................................................................................................................................91.3. Briefings by the presenters.....................................................................................................................................................9

2. Review of the Application Session Briefings............................................................. 102.1. Introduction ............................................................................................................................................................................102.2. Everett Palmer (NASA/Ames)..............................................................................................................................................102.3. Karim Zeghal (EEC)................................................................................................................................................................112.4. Randall Bone (MITRE/CAASD) ..........................................................................................................................................122.5. Gary W. Lohr (NASA/Langley)...........................................................................................................................................122.6. Gary W. Lohr (NASA/Langley) and Vern Battiste (NASA/Ames)................................................................................132.7. David Wing (NASA/Langley)..............................................................................................................................................132.8. Andy Barff (EEC)....................................................................................................................................................................14

3. Conclusions ....................................................................................................................... 15

SAFETY SESSION........................................................................................................................ 16

1. Introduction ........................................................................................................................ 161.1. Session objectives .................................................................................................................................................................161.2. Session organisation .............................................................................................................................................................161.3. Briefings by the presenters...................................................................................................................................................16

2. Review of the Safety Session Briefings ...................................................................... 172.1. Introduction and objectives of the session - Ken Carpenter (QinetiQ).........................................................................172.2. Overview of SAF-ASAS - Andy Zeitlin (MITRE/CAASD).............................................................................................172.3. Safety analysis of IMC Final Approach Spacing - Jonathan Hammer (MITRE/CAASD) ..........................................172.4. On safe separation of an ASAS based operation - Bart Klein-Obbink (NLR) ..............................................................192.5. ADS-B accuracy, integrity and continuity - Stan Jones (MITRE/CAASD)..................................................................192.6. Piloted simulation of ASAS scenarios - David Wing and Richard Barhydt (NASA/Langley)..................................20

3. Discussion during the Safety Session ........................................................................ 213.1. General.....................................................................................................................................................................................213.2. What does it mean to say an ASAS application is safe?.................................................................................................213.3. What metrics for TLS?...........................................................................................................................................................21



3.4. Methods available and used ................................................................................................................................................223.5. Lessons learnt.........................................................................................................................................................................223.6. Safety of applications drives system requirements ..........................................................................................................223.7. Humans....................................................................................................................................................................................22

4. Conclusions ....................................................................................................................... 23

VALIDATION SESSION ............................................................................................................... 24


2. Review of the Validation Session Briefings ............................................................... 252.1. Introduction ............................................................................................................................................................................252.2. Professor Kevin Corker (San Jose State University)........................................................................................................262.3. Dr. Karl Bilimoria (NASA/Ames).........................................................................................................................................262.4. Ulrich Borkenhagen (EUROCONTROL) & Béatrice Raynaud (CENA)..........................................................................272.5. Mark Watson (NATS)...........................................................................................................................................................282.6. Chad Jennings and Professor J. David Powell (Stanford University)............................................................................28

3. Discussion during the Validation Session ................................................................. 293.1. General.....................................................................................................................................................................................293.2. Relationship between design and validation.....................................................................................................................293.3. Role of different validation techniques...............................................................................................................................293.4. Role of validation versus certification ................................................................................................................................30

4. Conclusions ....................................................................................................................... 31

ARCHITECTURE SESSION ....................................................................................................... 33


2. Review of the Architecture Session Briefings........................................................... 342.1. Richard Barhydt (NASA/Langley)......................................................................................................................................342.2. Ed Johnson (NASA/Langley)..............................................................................................................................................352.3. Daniel Ferro (Airbus Industrie)............................................................................................................................................352.4. Chris Nehls (Honeywell International)................................................................................................................................362.5. Rich Kochanski (John Hopkins University).......................................................................................................................362.6. Alessandro Prister (EUROCONTROL)................................................................................................................................372.7. AP4 Workshop Report - Alessandro Prister (EUROCONTROL)...................................................................................38

3. Discussion during the Architecture Session............................................................. 38

4. Conclusions ....................................................................................................................... 39

TIM WRAP-UP DISCUSSIONS and CONCLUSION SESSION ......................................... 40

TIM Agenda.................................................................................................................................... 41



TIM Participants ............................................................................................................................ 44



1. Introduction

The Third Technical Interchange Meeting of the FAA/EUROCONTROL R&D Committee, AP-1(Airborne Separation Assistance Systems) was held at NASA Ames Research Center near SanFrancisco, California from October 21 - 23, 2002. The meeting was jointly sponsored by the FAA,NASA and EUROCONTROL, and attracted over 60 participants from Europe and the United States.

Details of the Technical Interchange Meeting, including copies of all the presentations and a list ofparticipants, can be found at http://asas.arc.nasa.gov/ .

1.1. TIM objectives

The purpose of this workshop was to provide an opportunity for the participants to share experimentalresults, experience, and viewpoints in the areas of ASAS research, development, evaluation, andimplementation. Participants were expected to contribute to clarifying technical and operationalissues, and developing an approach for resolution.

Another important objective for Action Plan 1 was to encourage collaboration and commonality ofASAS research.

1.2. TIM organisation

After an introductory presentation on the meeting objectives, followed by descriptions of ASASapplications being evaluated in Europe and the U.S., four technical sessions were held.

o Session 1: ASAS Applications

o Session 2: Safety in ASAS Applications

o Session 3: Validation of ASAS Applications

o Session 4: Systems and Architecture (Airborne & Ground)

All participants attended Session 1, and then the remaining three technical sessions were heldconcurrently. On the final day, the session chairpersons briefed a summary of the outcome of theirsession, and a final wrap up session was held.

1.3. Briefings by the presenters

The following presentations were made during the introductory Session

Briefings Presenters

Welcome Dr. Vic Lebacqz, NASA AmesResearch Center

Organisation and Objectives of the TIM Rose Ashford, NASA AmesResearch Center

US ASAS applications Gene Wong, FAA

European ASAS applications Francis Casaux, CENA on behalfof EUROCONTROL

JAA/CNS ATM steering group report Daniel Hawkes, JAA – UK CAA



2. Review of the Introduction Session Briefings

2.1. Vic Lebacqz (NASA/Ames)

Dr. Lebacqz welcomed the meeting participants to NASA Ames Research Center, and spoke brieflyabout the importance of ASAS research, and about the value of collaboration between the US andEurope in furthering ASAS implementation.

2.2. Rose Ashford (NASA/Ames)

Rose Ashford outlined the meeting objectives, organisation and logistics. This was followed by anoverview of the four application categories in the “Principles of Operation of Airborne SeparationAssurance Systems”, which was prepared by Action Plan 1 and approved by theFAA/EUROCONTROL R&D Committee in June 2001. She described the current work in preparing afollow-on document “Safety and ASAS Applications”.

2.3. Gene Wong (FAA)

Gene described the accomplishments of the FAA’s Safe Flight 21 programme, which includes manyindustry participants. There are 14 near term ASAS applications under development within SF21. Hedescribed the RTCA R&D decision process, with its three phases, concept definition/viability,concept refinement/ validation, and initial low-rate implementation. Some applications are beingdeveloped jointly with EUROCAE; others are unique to either RTCA or EUROCAE. After anintroduction to some of the applications, Gene described the Capstone work in Alaska. Challenges toimplementation of these ASAS applications include acceptance of changing roles andresponsibilities, solving HF issues (mixed equipage operations), achieving a satisfactory cost benefitratio and global interoperability.

2.4. Francis Casaux (CENA on behalf of EUROCONTROL)

Francis reviewed the European Joint Co-ordination Board, which oversees the European ASASprojects and applications being studied. He described the current projects including NUP II, MFF,MA-AFAS and EVP, as well as other European ADS projects. He reviewed “Package 1”, the nearterm European ASAS applications, and the future packages.

2.5. Dan Hawkes (JAA – UK CAA)

Dan gave a brief update on the work of the JAA/CNS ATM steering group. A position paper wasdistributed at the meeting.



ASAS APPLICATION SESSION1. Introduction

The Applications session (session 1) was planned from Day 1, afternoon till Day 2 mid morning, prior tothe Validation, Safety session and Architecture sessions. This session was co-chaired by Mark Ballinfrom NASA Langley and Eric Hoffman from the EUROCONTROL Experimental Centre. Finally, a reportof Applications session occurred Day 3 in plenary.

1.1. Session objectives

The purpose of this session was to present detailed descriptions of a number of ASAS applications interms of operational requirements for which recent experimental results are available.

Topics of interest included the operational needs to be addressed, actual results, lessons learnedand their significance in terms of operational procedures, human factors or benefits. Actualmethodologies used in terms of validation techniques or safety assessments were rather addressedspecifically in subsequent sessions.

The session covered applications from the four categories defined in PO-ASAS ranging from situationawareness to self-separation, and encompassing a representative spread of US and Europeanoperational needs.

The goal was to achieve a shared understanding of these applications in terms of operationalrequirements – similarities and differences –, expected benefits, overall maturity and potentialimplementation timeframe.

1.2. Session organisation

The application session was organised so as to allow for discussion between the participants:

o Following each briefing presented in the session (30 minutes presentation and 15 minutesleft for discussion), and

Finally, a session report and synthesis was prepared by the co-chairs and was used for discussion inplenary on Day 3.


The following briefings were presented in the Applications session during the TIM:


DAG-TM Air-Ground Integration Experiment, September 2002 Everett Palmer, NASA AmesResearch Center

Limited delegation with arrival streams: more insight on its impacton controller activity

Karim Zeghal, EUROCONTROLExperimental Centre

CDTI Enhanced Flight Rules: Concept and Initial SimulationResults

Randy Bone, MITRE/CAASD

Evaluation of a Time-Based Airborne Inter-arrival Spacing Tool Gary Lohr, NASA LangleyResearch Center

Airborne Information for Lateral Spacing: A Concept forIndependent, Closely Spaced Parallel Approaches

Gary Lohr, NASA Langley;Vernol Battiste, NASA AmesResearch Center




Use of Traffic Intent Information by Autonomous Aircraft inConstrained Operations

David Wing, NASA LangleyResearch Center

Mediterranean Free Flight Andy Barff, EUROCONTROLExperimental Centre

2. Review of the Application Session Briefings

2.1. Introduction

While the Day 1 morning plenary session emphasis was very much on PO-ASAS category 1applications, the Application session briefings put more emphasis on category 2 to 4. The content ofthe session was very much a reflection of the topic and PO-ASAS categories on which R&D iscurrently being performed.

The overall objectives of the Applications session were to get an understanding of the operationalrequirements addressed by the various applications, of whether the same “words” were indeeddescribing similar applications, of the maturity of the applications and finally and utmost of theresearch issues that should be forwarded to the validation, safety and architecture sessions.

Note that issues are listed after the briefing in which they were first identified. For the sake of brevity,they were not repeated if raised again in subsequent briefings. This explains why the first briefingshave potentially more issues associated with them.

2.2. Everett Palmer (NASA/Ames)

2.2.1. Brief description

Everett Palmer reported on the DAG-TM Air-ground Integration Experiment. It investigates acombination of the CE-6 (Trajectory negotiation), CE-5 (route free manoeuvring) and CE-11 (self-spacing in TRACON) DAG-TM Concept Elements. The programme includes a set of periodicdemonstrations with pilots and controllers. Autonomous aircraft (“totally” equipped) and managedaircraft (everything except conflict detection) share the same airspace. The CTAS tool set (trafficmanagement arrival) is used to provide time at the corner fix. The baseline scenario consisted oftoday’s operations with the addition of ADS-B and intent. The trajectory negotiation scenario relied onCTAS, speed advisory tools. Winds and STA were up-linked. CD&R functions were assumed to beavailable on board the aircraft. No radar vectoring was allowed: FMS route was to be used. In theautonomous aircraft scenario, RTAs were imposed on aircraft. The controller had the option to cancelfree-flight (pilot unable to comply with RTA). Rules of the road were used to handle priority issues inconflicts. Finally, self-spacing was performed in the TRACON area.

2.2.2. Issues

In the experimental environment design, the controller had and needed to maintain the “big picture”.The controller’s current heuristics were in opposition with the FF scenario: ‘see a problem, solve aproblem’ versus wait & see, positive control versus FF.

è Validation: Issue of maintaining current roles and working methods in a radically differentenvironment. That can be linked either to the concept definition (as for example the same workingmethods are assumed in the new environment) or to the validation subjects or techniques (testsubjects have difficulties projecting their experiences in the proposed environment).

It was reported that test subjects/controllers were often confused about who was responsible forwhat.

è Validation: what is the required knowledge of the situation for each actor (flight crew andcontrollers) to be able to perform its tasks? What is meant by “shared” understanding of this



situation and how is this achievable?

The experiment was undoubtedly of exploratory nature:

è Validation: how does this type of exploratory experiment fit in an overall concept development andvalidation cycle?

Three different Concept Elements, in two different types of environment (TRACON and En route) wereinvestigated simultaneously.

è Validation: what is the proper balance between large-scale multi-objective integrated experimentsand smaller more focused and controlled experiments? Should this be linked to the level ofconcept and technology maturity?

è Validation: what is the required training for such a novel complex? Is there a metric to determinethat the proper level of training has been achieved?

RTA and spacing constraints appear to be potentially complementary.

è Validation: how should that be demonstrated/ further investigated? Who is planning to do so?

Perfect “intent” was assumed – no limitation in terms of look ahead horizon, range or validity

è Architecture/Validation: how to achieve common understanding of requirements and technicalfeasibility of exchange of intent information?

è Architecture: Is there a default set of realistic assumptions for aircraft equipage levels so thatresearch can be co-ordinated?

Right of way rules, implicit vs explicit co-ordination: redundancy is often used as an argument for oragainst

è Safety: what are the requirements in terms of co-ordination from a safety perspective?

2.3. Karim Zeghal (EEC)


The motivation of the study presented by Karim Zeghal was to increase controller availability toimprove safety first, then efficiency and capacity. Both air and ground perspectives were investigatedin a set of 4 experiments from June 1999 onwards. The presentation focussed on the latest November2001 experiment results, covering the different metrics used, the airspace simulated (Paris arrivalsectors). The results obtained in previous experiments were confirmed through the use of point ofgaze measurements, suggesting also an increase of availability.

2.3.2. Issues

Are abnormal conditions and equipment failures being studied?

è Safety: At what stage of the development/validation process should we start investigatingabnormal and failure conditions? What are the appropriate techniques to do this?

è Safety: How do we ensure that once a new concept becomes operational, the actors will not usea new procedure beyond its intended design?

è Safety: For new concepts, how to trade off workload benefits that occur under nominal conditionswith workload increases that may occur under abnormal conditions?

Initial results suggest that controllers can handle very high traffic. How do we determine system-widebenefits?

è Validation: What is the appropriate approach to bridging the gap between initial positive results ina local environment and system-wide results? Series of simulations? Are new simulationcapabilities needed? How do we capture human behaviour, especially in fast-time simulations?

è Validation: At what point are the results complete enough to consider the research finished?



2.4. Randall Bone (MITRE/CAASD)


Randy Bone presented the CEFR CDTI Enhanced Flight Rules as part of Safe Flight 21 Programme.This application was designed to enhance visual approach procedures by extending the domain inwhich the visual approaches can be used. It is still in the concept definition phase: aspects likewhether a specific spacing value will be required or what is the maximum cloud thickness aircraftwould be allowed to traverse using CEFR are still to be clarified. The feedback from experimentsshow that pilots were ready to accept separation responsibility and the workload appeared to beacceptable.

2.4.2. Issues

Technology does not appear to be an issue, but acceptance by pilots and controllers must also beconsidered.

è Validation: Are end users’ needs accounted for in the development and validation process?

How are aircraft not equipped for CEFR integrated into the operations?

è Validation: How does mixed equipage impact benefits of the concept?

Is a cockpit-based spacing alert required?

è Safety: What is the level of automation required?

What is the relationship between ASAS and wake turbulence avoidance?

è Safety: Is wake turbulence avoidance hindered if out-the-window visual cues are lost (i.e., onlyusing CDTI)?

è Safety: Will a hard limit for wake turbulence need to be prescribed?

Can CTDI be used in lieu of visual sighting?

è Safety: Would the use of CDTI under these conditions require a new set of separation minima? Ifso, how would these minima compare to current existing ground-based separation minima?

What is a safe ASAS Application?

è Safety: How do we compare new concepts and procedures against established operations?Should we revisit the safety of established operations? Are there alternative approaches?

è Safety: Can safety be addressed without considering capacity issues? Should we develop a co-ordinated approach to better understand and report about the capacity/safety trade-offs?

2.5. Gary W. Lohr (NASA/Langley)


Gary Lohr described the Approach spacing – time-based (CE-11 – Terminal Arrival: self-separation formerging and in-trail Operations) developments and experiments run at NASA Langley. The conceptcalled for time spacing between aircraft. No fixed routes were required and the operations werecompatible with radar vectoring. Benefits were in terms of increased runway throughput, reduction ofvoice communication. On the flight deck, algorithms were implemented with minimal architecturechange. A simulation was run in February 2002 to compare 3 conditions on the flight deck: referencewith no automation, manual Throttles with guidance cues, MCP Speed with guidance cues, ATAASalgorithms fully coupled. A flight evaluation was conducted 17-21 September 2002 involving NASAB757, Rockwell Collins Sabreliner, and Piper Chieftain Flight Inspection aircraft. Out of the 28 runswith the three aircraft, the mean time error at the runway was under 1 second and standard deviation8 seconds.



2.5.2. Issues

This specific ASAS application was found to be robust to wind uncertainty.

è Validation: How do we validate an applications’ robustness to wind uncertainty?

è Validation: What are the requirements for, and associated benefits of, more accurate windinformation?

What are the system architecture issues associated with making use of history information?

è Architecture/Safety: Are there failure modes that may cause history information to be lost orcorrupted? What are the impacts of the lost information?

Is it safe for an aircraft’s’ guidance or crew actions to be coupled to another aircraft’s actions ornavigation system?

è Architecture/Safety: What are relative RNP requirements?

è Safety: Can trailing aircraft always assume that the leading aircraft’s navigation is safe? Is there acomplacency issue for aircraft in trail? (For example, following a leading aircraft into a mountain.)

è Safety: How do impacts of unplanned actions by leading aircraft compare between the ASASapplication and current operations?

What is the impact of a spacing alert and who is responsible for action?

è Safety: Does an abnormal event induce a new shift of task or responsibility between the flightcrew and the controller?

2.6. Gary W. Lohr (NASA/Langley) and Vern Battiste (NASA/Ames)


Gary Lohr and Verne Battiste gave presentations on 2 complementary aspects of AILS - Closelyspaced parallel approaches research performed at NASA in the past years. The Langley experimentsinvolved the NASA-B757 in co-operation with Honeywell while Ames focussed rather on B747 pilotedsimulations. Results suggested that AILS appeared to be better than Precision Runway Monitoringfrom a pilot perspective. Concerns were expressed about wake turbulence aspects, altitude bustsobserved during escape manoeuvres as well as to the response time to AILS alerts.

2.6.2. Issues

Can or should TCAS be used as a mitigation factor?

è Safety: What is the relationship between ASAS and ACAS?

Many altitude busts were observed during AILS escape manoeuvres.

è Validation: What ASAS applications require escape procedures? How should they be designedand validated? Is there an airspace design impact?

It doesn’t take very many unequipped aircraft before the benefits are lost.

è Validation: What are controller workload impacts of identifying equipage levels?

2.7. David Wing (NASA/Langley)


The focus of the briefing by David Wing was the use of traffic Intent information by autonomousaircraft in constrained operations. The DAG concept follows RTCA free flight select committee. DAGstudies are co-ordinated between the various sites of NASA. NASA/Ames (ground aspects)NASA/Glen (expertise in CNS infrastructure), NASA/Langley (airborne aspects) – Joint efforts tointegrate the various components. The research presented was specifically on the DAG-TM ConceptElement CE-5 – En-route free manoeuvring. With a goal to check feasibility, autonomous aircraft and



managed aircraft were sharing the same airspace. Aircraft equipped for free manoeuvring were givenpriority over managed aircraft as a matter of redundancy and incentive. Priority rules were usedbetween autonomous aircraft. The tactical mode was using state information while the strategic modewas using intent information. A prototype Autonomous Operations planner was implemented tosupport the flight deck. Experiment results showed that the strategic mode was preferred to thetactical mode. A need for an intelligent filtering of the aircraft to be displayed was expressed. Therewas no difference in the induced conflicts similar between the two modes.

2.7.2. Issues

Who is responsible for resolving conflicts between managed and autonomous aircraft?

è Validation: Should user incentives to equip be addressed for ASAS applications?

Is mixed autonomous and managed equipage in the same airspace feasible and viable?

è Validation: What is the division of responsibility between flight crews (for autonomous aircraft)and controllers (for managed aircraft)?

è Validation: What is the ratio of autonomous to managed aircraft required to achieve positivebenefits?

What are benefits of strategic vs. tactical modes?

è Validation: Is there a valid range of aircraft capabilities that can coexist, thereby providing a trade-off of cost-to-equip vs. benefits received for the user?

Would flight crew workload over an extended period of time be an issue?

è Validation: What can we conclude from short-duration part-task experiment scenarios? What arethe limitations of workload and situational awareness measurements under these conditions?

2.8. Andy Barff (EEC)


Andy Barff gave an overview of the Mediterranean Free Flight (MFF) project. Free route operations andASAS applications were simultaneously investigated. The project foresaw model-based simulation,real time simulations and flight trials. Initial model based ASAS spacing simulation results werepresented as well as results concerning self separation suggesting that it required a re-design of theairspace and would lead to shorter flight time. The results of RTS1, the first real time human in theloop simulation run in May 2002 as part of a series of 3 with the subsequent ones planned inFebruary and November 2003, were detailed. Traffic considered was not found dense enough (RVSMwas implemented) to make for the ASAS spacing application envisaged very useful.

2.8.2. Issues

Controllers were concerned about the monitoring workload for heading and merge manoeuvres. DuringCo Space, controllers were given more extensive and different training, which produced differentresults.

è Validation: How to achieve a shared understanding of an ASAS application while it is underdevelopment, and how to avoid the resulting difficulties in its evaluation?

Differences are anticipated between TCP information and ground flight plan information.

è Architecture: During a modernisation transition, how can aircraft intent information be integratedin the ground flight plan processing system?



3. Conclusions

All ASAS applications presented in this session:

§ were explicitly linked to PO-ASAS categories – providing an indication of common vocabularyand reference being shared

§ were addressing capacity-constrained airspace and the associated benefits – showing that thesame operational issues were addressed between the US and Europe.

Encouraging results and positive trends were displayed in all presentations

A large number of validation issues were identified, followed by safety issues, and architectureissues, noting that benefits issues were considered a part of validation. The number of issuesidentified could be an indication of low maturity of most ASAS applications provided, although itshould be kept in mind that due to the very nature of the TIM, the briefings were explicitly selected ontheir relevance to R&D.

US and European applications under study are very similar despite the fact that different paths led tothem. There are few or no issues introduced from one side of Atlantic Ocean that are not alreadyunderstood and considered by those on the other side.

However, we can note differences in and the complementary nature of the focus of research betweenthe US and Europe:

§ Current US research focus regarding spacing is on TRACON applications, while currentEuropean focus is spacing in En Route arrival transition airspace.

§ Initial European focus is on ground surveillance; much of the US focus on airborne surveillance.

§ FAA Safe Flight 21 activity is investigating many PO-ASAS Category 1 applications.

§ EC projects focus is to support Package 1 (All applications are within PO-ASAS categories 1and 2.)

§ Current NASA focus is primarily long term.



SAFETY SESSION1. Introduction

Session 2, centred on the Safety of ASAS, of the ASAS TIM ran from 10:00 on Day 2 to about thesame time on Day 3, in parallel with the Validation and Architecture sessions. This session was co-chaired by Andy Zeitlin from Mitre Corporation, and Ken Carpenter from QinetiQ. Five papers werepresented and discussed. A report was drafted over-night, and reviewed and modified beforepresentation to Plenary by Andy Zeitlin on Day 3.


The initial purpose of the session was to present and share experience related to the evaluation of thesafety of ASAS applications, and the development of methods to prove that ASAS applications aresafe and to develop safe ASAS.

The objectives were originally listed as follows.

§ To reach consensus on what it means to say that an ASAS application is safe.

§ To agree metrics and values for the metrics that can be used as target levels of safety (TLS).

§ To agree which metrics and values are appropriate in particular circumstances.

§ To note analysis methods currently available to find whether particular applications meet theappropriate TLS.

§ To note particular safety analyses and lessons learnt.

§ To agree how the analysis of the requirement that ASAS applications be safe is to beinterpreted and used to derive functional and performance requirements on ASAS as a system.

Topics invited included the definition of safety, safety analyses for different categories of applications(both long and short term), safety analyses using different methodologies, and the derivation ofsystem functional and performance requirements from the safety analyses. The need to consider thefull range of human behaviour in these analyses was noted. Should the safety analysis for theapplication allow for perverse and unusual behaviour as a condition for operational approval? To whatextent should the derivation of system requirements allow for human variability and fallibility?


The Safety session was organised so as to allow for discussion between the participants:

§ Following each briefing presented in the session (30 minutes presentation and 15 minutes leftfor discussion), and

§ After all the briefings, during a wrap-up session that concluded the session on Day 2.

Finally, the session report prepared by the co-chaired (in the evening of Day 2) was reviewed anddiscussed by the session participants on Day 3, before presentation in plenary.


The following briefing were delivered:


The AP1 paper 'Safety and ASAS applications' Andy Zeitlin, MITRE CAASD

Safety analysis of an IMC Final Approach Spacing application Jonathan Hammer, MITRE CAASD

On safe separation of an ASAS based operation Bart Klein-Obbink, NLR




Contribution of ADS-B accuracy, integrity and continuity tosurveillance separation service

Stan Jones, MITRE CAASD

Piloted simulation of airborne separation assurance scenariosthat pose a potential safety risk: preliminary results

David Wing and Richard Barhydt,et al., NASA Langley

2. Review of the Safety Session Briefings

2.1. Introduction and objectives of the session - Ken Carpenter (QinetiQ)

Ken outlined various activities that required input on safety questions, and then rephrased theobjectives of the session as follows:

§ What does it mean to say an ASAS application is safe?

§ What metrics and values can be used for the Target Level of Safety (TLS)?

§ And in particular cases, are some applications different?

§ Note methods available and already used in particular cases

§ Lessons learnt from studies to date.

§ How does safety of applications drive system requirements?

§ How and when do we consider perverse humans?

§ Should system requirements allow for human fallibility?

This report addresses these questions.

2.2. Overview of SAF-ASAS - Andy Zeitlin (MITRE/CAASD)

Key words of the briefing: safety issues; methodologies; guidelines; human factors; hazards.

The FAA/EUROCONTROL R&D Committee Action Plan 1 has been tasked to prepare a paper on‘Safety and ASAS applications’ (SAF-ASAS), a successor to the ‘Principles of Operation of ASAS’(PO-ASAS). A first draft is expected in May 2003, for comment from as wide a community aspossible. The purpose of SAF-ASAS is to provide guidance for the developers of ASAS applicationsand ASAS.

Andy outlined the contents of the document:

§ Safety issues relating to ASAS applications.

§ Current safety methodologies.

§ Guidance for developers of ASAS applications.

The issues are discussed by application category, type of issue and topic.

The paper was presented as background information. It was expected that this session would provideinput for the document.

2.3. Safety analysis of IMC Final Approach Spacing - Jonathan Hammer(MITRE/CAASD)

Key words of the briefing: analysis; methodology; process diagram; hazard; consequence; fault tree.

2.3.1. Presentation

The briefing used a specific application to illustrate the methodology used by SC186 to analyse thesafety of ASAS applications. The purpose of the analysis is to derive ASAS system functional andperformance requirements from the higher requirement that ASAS application be safe. The application



used as an example was IMC Final Approach Spacing, an airborne separation application.

Jonathan pointed out that it is also necessary to analyse the application for intended function, and inthis connection he reported that Monte Carlo simulations indicate a potential 10% increase in runwaythroughput. This talk concentrated on the safety analysis, in particular on the risk of a wake vortexencounter.

The steps in the method are:

§ Operational description

§ Phase diagrams

§ Process diagrams

§ Hazards analysis, severity classification – avoidances and mitigations

§ Fault tree analysis for the allocation of risk

The final step, the fault tree analysis, requires a TLS, in this case a TLS for collision risk and asecond TLS for wake vortex encounters. Collisions are catastrophic, and a TLS of < 10-9 per flighthour is being used. Wake vortex upsets are severe-major, and a TLS of < 10-7 per flight hour is beingused.

For this application, the Monte Carlo simulations were also used to examine the extent of thehazards, in particular the penetration of the wake vortex zone that could occur before the pilot wouldbe alerted. The purpose here was to derive requirements on accuracy, update rate and latency.

2.3.2. Discussion

The group discussed the roles of the controller and the flight crew in this application, noting theconstrained freedom of action for the flight crew. The application is a ‘probe’ application, that is, it isbeing used as an example of a stressful application with demanding requirements, and is notintended for early implementation. Nevertheless, the discussion emphasised the need for clearprocedures, and clear presentation of the consequences of implementing any ASAS application.

The analysis presented is required for the ASA MASPS, so only those issues affecting the systemare being analysed at this stage. Nevertheless, the group noted the large size of the job. It has to becarried out for each application, and it has to be carried out for implementation as well as for derivingsystem requirements. Fortunately, many elements of the analysis are reusable, for example betweenapplications.

It was observed that the approach attempts to implement the RTCA DO264/EUROCAE ED-78Amethodology, although some modifications have been found to be required. In this connection, it wasnoted that the environment for the application was not being considered. Consideration of theenvironment was not considered to affect the conclusions required for the MASPS, and it was alsofelt that the method could be extended to include the environment.

The group considered how this talk related to the objectives of the session.

§ The application was judged safe if it met ‘the’ TLS for each undesired consequence.

§ The TLS used in this analysis are those normally used for equipment certification. It this case,it is collision risk and wake vortex risk that must meet the TLS.

§ ‘How does safety drive the system requirements?’ – that was the point of the talk. MASPSrequirements were derived against the TLS.

§ The analysis assumes reasonable, ‘nominal’ human behaviour. This is discussed further insection 3.7.

§ It was noted that the numbers used relating to human factors require validation. Indeed, all thenumbers require validation.



2.4. On safe separation of an ASAS based operation - Bart Klein-Obbink (NLR)

Key words of the briefing: feedback; risk assessment; TOPAZ; bias; uncertainty; model.

2.4.1. Presentation

TOPAZ is a highly developed methodology and supporting tool set based on stochastic dynamicmodelling and incorporating a range of models for various aspects of ATM operations. Thepresentation described the use of TOPAZ to find the safe spacing between two anti-parallel tracks, inthe absence of controllers (and TCAS). The flight crew have a variety of tools to support the provisionof separation (of 5 NM in this case), including conflict detection and resolution algorithms.

The study quantified the risk of collision as a function of the spacing between the tracks. It found thatthere is a spacing (about 7NM) at which the current ICAO TLS (5×10-9 collisions per flight hour perdimension) is met, indicating that the operation appears feasible. The risk of collision is dominated bythe risk of collision between an aircraft that is nominally on track and an aircraft in a non-nominalsituation that has deviated from the opposing track.

It is possible to study the sensitivity of the results to the assumptions made (122 of them), and toestimate the bias and uncertainty in the model. The upper and lower bounds of the collision riskdiffered by nearly two orders of magnitude for the optimum spacing, but this seemed to correspond toonly about a factor of two in the safe spacing. Not all the assumptions mattered. For example, RNP1was assumed, but improved RNP resulted in negligible reduction in the safe spacing.

2.4.2. Discussion

The talk emphasised the need for iterative feedback between the ATM design for the application andthe safety assessment. The method appeared excellent for proof of concept and the development ofapplications.

The group considered how this talk related to the objectives of the session.

§ Safety can be a matter of perception, of functional dependability or of calculated risk. It wasconsidered that calculated risk is paramount amongst these.

§ The study used ICAO’s En Route TLS of 5×10-9 collisions per flight hour per dimension.

§ The method can be used to derive system requirements from high level requirements, but inthis case it was used to derive a parameter of the operational procedure – the spacing betweenthe tracks.

§ The tool used models realistic human behaviour, with particular attention to cognitiveperformance simulation and situational awareness error evolution.

2.5. ADS-B accuracy, integrity and continuity - Stan Jones (MITRE/CAASD)

Key words of the briefing: accuracy; integrity; continuity; monitoring; GPS; surveillance; separation.

2.5.1. Presentation

The talk discussed the ADS-B parameters NAC (positional error), NIC (integrity containment radius)and SIL (integrity level) in relationship to GPS integrity and its monitoring. It went on to review theICAO method of surveillance assessment for the provision of separation and the Close ApproachProbability (CAP) model. It proposed a way to combine with the CAP model to assess ADS-B basedsurveillance.

The talk brought out the need to trade off factors such as integrity false alarms and missed detectionof integrity failure against performance and safety requirements. GPS-based ADS-B and radar-basedTIS-B were discussed in the context of the link between surveillance quality and separation.

The talk recommended that:

§ GPS and RNP performance parameters should be examined for their applicability tosurveillance;



§ The relationship between the usage of parameters in the ADS-B MASPS (NAC, NIC and SIL)and integrity monitoring and service continuity should be clarified;

§ A better understanding is required of the operational requirements on integrity monitoring andthe definition of failure;

§ The proposed linkage between the ADS-B performance monitoring measures and the CAPmodel used by ICAO needs to be co-ordinated.

2.5.2. Discussion

The group was slightly overwhelmed! However, it noted that this sort of analysis needs to bediscussed in the ICAO SAS Panel. It is clearly essential to sound and coherent system design.

2.6. Piloted simulation of ASAS scenarios - David Wing and Richard Barhydt(NASA/Langley)

Key words of the briefing: free flight; airborne self-separation; pop-up conflict; separation standard;strategic and tactical conflict management.

2.6.1. Presentation

The talk described human-in the loop simulations of a free flight operational scenario. Some aircraftwere under control, others, including the subject aircraft, were self-separating. The scenarios includedproximate special-use airspace, reduced separation standards, pop-up conflicts, and an over-constrained conflict where it was impossible to resolve the conflict and meet the required time ofarrival at a fix.

The overall purpose is to prove the Distributed Air-Ground Traffic Management concept, in this caseby investigating safety issues. Particular issues investigated were:

§ Sensitivity to hazard proximity (hazards including traffic and restricted airspace);

§ The ability to regain separation after pop-up conflicts; and

§ The ability and means to cope with over-constrained conflicts.

The simulations included pilot use of integrated strategic and tactical conflict resolution tools.Strategic resolution uses intent data and projects up to 10 minutes ahead (in this work). Tacticalresolution considers only state data and projects 2-5 minutes ahead. The value of priority rules wasinvestigated. The metrics were mainly operational, but included loss of separation.

Analysis is not complete and only preliminary conclusions were available. The indications, but theyare subject to review, are that priority rules may show promise in over-constrained conflict situationsprovided that correct intent information is broadcast. The reported losses of separation occurred partlyas a consequence of autoflight system behaviour unexpected by the pilots. The intent datatransmitted in the simulations unfortunately included constraint information rather than estimatedactual performance. The predicted conflict therefore was not altered even though the precedingconflict resolution manoeuvre by the burdened aircraft caused a change in the command trajectorythat would in fact resolve the conflict. The result was occasions of unnecessary manoeuvring by theprivileged aircraft.

2.6.2. Discussion

The group concluded that one should not lie! In more technical terms, the experimental dataillustrated a pitfall of broadcasting intent to which the aircraft cannot adhere, which in this case wasthe assigned arrival time over the fix.

The group discussed the need to provide feedback in both directions between these real timesimulation work and other more analytical approaches. All parties are encouraged to talk to eachother, and to listen. An additional proposed solution was to increase compatibility between human-in-the-loop simulations and batch simulations such that similar assumptions and models are used.



3. Discussion during the Safety Session

3.1. General

Discussions following each of the presentations have been covered above.

Two particular issues were raised during the general discussion session at the end of day two.

§ There was a plea for a further explanation of the distinction between airborne spacing andairborne separation. This is addressed in section 3.2, in the context of what it means for anASAS application to be safe.

§ There was an expression of concern at the profile adopted by some in the ASAS community.

The concern is that there is too much enthusiasm, particularly for some potential ASAS application,and not enough realism concerning the end state. ASAS potentially involves a radical change in theATM paradigm, yet we have an ATM system that we believe to be safe. There is confusion betweenthe roles of navigation and surveillance, with the potential introduction of new common failure modes.(It was observed during the report back that the existence of common failure modes betweennavigation and surveillance is not entirely new.) It was asserted that far more clarity is requiredconcerning roles, and it is necessary to provide a map back from the proposals to the present ATMparadigm.

The group accepted these as valid concerns, although not all necessarily shared the degree ofscepticism implied. It was agreed readily that the burden of proof is on those who wish to changecurrent practice.

The original objectives for the session, described in 2.1 above, were discussed and reduced to sixslides overnight. These were discussed and modified by the group before presentation in plenarymeeting.

3.2. What does it mean to say an ASAS application is safe?

ASAS applications were considered safe when they are conducted in accordance with well-definedrules that have been determined to be acceptably safe. All hazards must be controlled to acceptablelevels.

In the presentations, the group had heard about analyses based on requiring that the risk of collisionand the risk of wake vortex encounters be acceptably low. It had also heard a discussion of issuessurrounding required surveillance performance, which would feed into the risk analyses, and aboutinvestigations of the ability of humans to play the roles that might be anticipated.

The question of the distinction between airborne spacing and airborne separation was addressed inthis context. A central requirement for safe ASAS applications is that the spacing between aircraftexceeds established separation minima.

§ In airborne spacing applications, the flight crew delivers a spacing that exceeds the separationminimum. The value of the spacing would be specified by the controller, or be standardised andknown. The controller would monitor the actual spacing delivered, and remains responsible forensuring that the spacing exceeds the separation minimum.

§ In airborne separation applications, the flight crew ensures that the spacing exceeds astandardised airborne separation minimum. The controller would have delegated to the flightcrew the responsibility for this specific separation provision task. The controller is notresponsible for ensuring that the spacing exceeds any separation minimum.

3.3. What metrics for TLS?

The metrics used to determine that an application is acceptably safe have to be suited to theapplication. For example, a metric related to wake vortices is required for approach spacingapplications. Collision risk is always a metric. Similarly, the units have to be suited to the application,e.g. collisions per operation for approaches, or collisions per flight hour for en-route operations.



Jonathan Hammer’s talk included the usual set of values used for equipment certification for TLS forcatastrophic, hazardous and major adverse events. The group asked:

‘Do we know that these are adequate to meet reasonable TLS for ATM operations?’

Bart Klein-Obbink used a different TLS value when deriving the spacing required between tracks, buthis value was equally familiar from work on separation standards. The distinction lies in the allocationof risk budget to all the elements in the process of providing safety, and the question asked by thegroup implies that we did not know whether this allocation has been carried out.

A TLS relates to an undesirable operational consequence, and the group asked whether there is,anywhere, a standard list of ‘operational consequences’. Such a list would be useful.

When the ASAS application involves a very small change from existing practice, the proof of safetymight be by direct comparison with the current operation. In this case, the comparison has to besimple, direct and convincing. In this case, the TLS could be expressed in terms of frequency of lossof separation, and the TLS for more serious consequences (e.g. collision risk) would be implicit.

3.4. Methods available and used

The group noted:

§ The methods being used by RTCA SC186, and that they are derived from the RTCA DO-264 /EUROCAE ED-78A OSA methodology;

§ TOPAZ;

§ The use of human-in-the-loop simulations; and

§ Monte Carlo and fast time simulations.

There was a request for help and advice to those carrying out the work. It was noted that SAF-ASASwill record the available methods of analysis.

3.5. Lessons learnt

Lessons were learnt from each of the presentations and are recorded above.

The group considered that there is a need for feedback between the various methods of study. Theonly suggestion they could make was the use of mutual presentations – talk to each other, andlisten.

The group would like to see the promotion of compatible methods and analyses. An example wouldbe a simulation facility that is capable of both human-in-the-loop operation and batch / Monte Carlooperation. (Such a facility is under development by NASA Langley for DAG-TM research.)

3.6. Safety of applications drives system requirements

The group noted, from Jonathan Hammer’s presentation of the RTCA MASPS work, how therequirement of safety does indeed result in functional and performance requirements for ASAS, whichcan be derived on a logical basis. The group noted that this was also the intention of the presentEUROCAE approach to the Package 1 of applications for Europe.

3.7. Humans

The group noted that the RTCA analysis presented by Jonathan Hammer assumes reasonable,‘nominal’ human behaviour – behaviour that approximates that required. Aberrations are not considered.

The group considered this approach acceptable for this analysis. However, it considered that it wouldbe inadequate for proving the safety of the application, and it hoped also for Flight Operations approval.



4. Conclusions

The group noted that it had covered multiple levels in its (relatively small number of) talks.

Two talks covered the safety analysis of ASAS applications, one from the point of view of derivingsystem requirements, the other to derive safe spacing values in a particular application. Both used adeveloped methodology, and in both cases there is a feedback process between the safety analysisand the operational description for the application. (For the development of MASPS this has occurred,but might be relatively weak. Consistently with the OSA methodology, it should be more in evidencewhen the question is whether the applications are safe.)

The third talk also concerned the design of safe systems, but from the point of view of ensuring thatthe ADS-B and TIS-B parameters fields provide a consistent and logical basis for the ASASsurveillance function, and thus for separation.

The last talk concerned the design of safe procedures and the consideration of human behaviour froma safety point of view. Its purpose is to demonstrate that the advanced ASAS applications studied arecapable of being carried out safely by flight crew.

Discussion included the following points.

§ The task of carrying out the analyses required even just for the relatively small number ofASAS applications envisaged for implementation in the short term will be huge.

§ Applications are safe when they are conducted in accordance with well-defined rules, and allhazards are controlled to acceptable levels.

§ The metrics for TLS have to be related to the application, and should cover all the undesirableoperational consequences.

§ It was asked whether there is a standard list of operational consequences.

§ The ASAS requirements are being derived on the basis of the usual TLS values for equipmentcertification. However, it is not known that these are adequate to meet a reasonable TLS forATM operations when the global risk budget is allocated to all the elements in the process ofproviding safety.

§ There were pleas for help and advice, between those working on different approaches. There isa need for feedback between the various methods of study.

§ The group considered it acceptable to assume reasonable, ‘nominal’ human behaviour whenderiving system requirements from high-level requirements such as safety. However, theyconsidered this would not be acceptable when proving that the application is safe. It wasobserved that different numerical values for the TLS would be used for these two processes.

This session should prove extremely valuable in the drafting of the AP1 document ‘Safety and ASASapplications’.



VALIDATION SESSION1. Introduction

The validation session (session 3), co-chaired by Sandy Lozito and Béatrice Raynaud, ran from midmorning of Day 2 to mid morning of Day 3, in parallel with Safety session and Architecture sessions.Five papers were presented and discussed. Finally, Sandy reported the discussions and conclusionsfrom the Validation session on Day 3 in plenary.

The individuals who attended the Validation session were representatives of various governmentagencies, academia, and industry who were interested in sharing ideas about validation of ASASconcepts.

The various briefings of the session, as well as main discussion items raised during thepresentations, are reported respectively in section 11 and section 12 of this report. Finally, section 13summarises the general conclusions drawn as a group with regards to questions related to validationof ASAS applications.


The initial purpose of this session was to present and to share experience related to the validation ofASAS applications with the objective of gaining confidence in the ability of ASAS applications tooperate against a predefined level of functionality, operability and performance.

The session was to provide an opportunity to discuss different approaches for evaluating expectedbenefits and operational issues related to the use of ASAS, as well as assessing the impact ofASAS applications on the ATM system, in various areas, except safety, for which there was aparallel session. It was intended to address the broad scope of validation activities applied along thedevelopment of ASAS applications, with particular attention to the concept definition, feasibility andacceptability assessment phases.

Topics of interest included the description and lessons learned from recent experimental design,performance and results, including description of experimental conditions, hypotheses and metricsused to assess:

§ Benefits of concept options for the operational use of ASAS;

§ Human performance and acceptance of ASAS applications

§ Possible impact of ASAS applications on ATM operations efficiency, flexibility andpredictability.


The Validation session was organised so as to allow for discussion between the participants:

§ Following each briefing presented in the session (30 minutes presentation and 15 minutes leftfor discussion), and

§ After all the briefings, during a wrap-up session that concluded the session on Day 2.

Finally, the session report prepared by the co-chaired (in the evening of Day 2) was reviewed anddiscussed by the session participants on Day 3, before presentation in plenary.




The following briefings were initially scheduled in the Validation session, five of which were givenduring the TIM:


Free Flight: Context of Control Dr Kevin Corker, San Jose University

Fast-time Simulation Studies of Airborne Self-Separation Dr. Karl Bilimoria, NASA Ames

Comparison of EACAC and AGIE Metrics Ulrich Borkenhagen, EUROCONTROLand Béatrice Raynaud, CENA

CARE-ASAS Validation Framework, Guidelines and CaseStudies

Mark Watson, NATS

Use of Master Air Traffic Management European ValidationPlan (MAEVA) Guidelines in the INTENT Project

Not presented

Threat Displays for Closely Spaced Parallel Approach Chad Jennings and J. David Powell,Stanford University

The various validation methods and topics addressed through the briefings included: Real-time (RT)simulations, Fast-time (FT) simulations, Field tests, combination of these methods, and frameworkfor use of these methods.

The ASAS applications under validation in the presented studies included: airborne self-separationapplications related to DAG-TM, ASAS spacing applications and the Closely Spaced ParallelApproaches application.

2. Review of the Validation Session Briefings

2.1. Introduction

Prior to the briefings, Sandy Lozito presented the audience with the objectives of the session, whichwere to present and to share experience related to the validation of ASAS applications throughbriefings related to recent US and Europe experiments and studies.

To support elaboration of general conclusions from the session, the participants were also invited todiscuss (and, as far as possible, to conclude) on the followings items:

§ What are the validation issues specific to ASAS applications?

§ Is there a need to distinguish between the four ASAS categories?

§ Which areas of work related to ASAS validation could be done in AP1?

§ What is relationship with other validation work (e.g. FAA/EUROCONTROL AP5)?

The various briefings, as well as the main discussion items raised during the presentations, arereported hereafter.



2.2. Professor Kevin Corker (San Jose State University)

Key words of the briefing: Shift separation responsibility, human in the loop experiment, and humanperformance models.

2.2.1. Briefing

Dr. Corker’s presentation discussed data from a human in the loop simulation, and how those datamight be used in conjunction with fast-time human performance modelling.

The goal of his initial real-time simulation was to characterise the impact of shifts in separationauthority on controller performance in Center operations. The impact of authority shift and trafficdensity upon controller workload and procedures were examined. Four conditions were investigatedincluding direct-routing and mixed autonomous/managed aircraft environment, with almost no supporttools for the controller.

The findings from the human-in-the-loop experiment indicate that the simple proceduralimplementation of an aircraft self-separation authority in a free flight paradigm correlates with anincrease in ATC workload in some significant dimensions. As an example, correlation betweencontroller (subjective) workload and objective controller/pilot communications load was observed. Inaddition, human performance models (MIDAS) appear to replicate the essential patterns of controllerbehaviour in response to the simulation of free flight.

2.2.2. Discussion

Discussions after Dr. Corker’s presentation included the value of real-time simulations and how dataderived from them can be used to assist in fast-time model development and validation.

The audience also discussed the appropriate application of human performance modelling. It was feltthat when it is necessary to identify the effects of shifts in operational roles of the operators upon theaviation system, human performance modelling can be beneficial.

The drivers for the use of human performance models in place of real-time simulations or field tests(apart from economics) were discussed. It was agreed that the set of parameters to be consideredand balanced include the amount of human training required, the level of fidelity, the level of changesin cognitive process, and the maturity of the concept.

The issue of how to define a set of experiments that would build up from each other to support design(and validation) of the end state environment was mentioned, but not really addressed.

In conclusion, lessons learnt from this experience would support the combined use of differentvalidation activities (simulation, model, field tests, ...), although it was not obvious which one validateswhat.

2.3. Dr. Karl Bilimoria (NASA/Ames)

Key words of the briefing: DAG-TM Free-manoeuvring feasibility, CD&R algorithms, fast-timesimulations.

2.3.1. Briefing

The presentation provided by Dr. Karl Bilimoria discussed the use of fast-time models to investigatethe system-level performance characteristics of airborne self-separation in free flight.

Three studies were reported which used the Future ATM Concepts Evaluation Tool (FACET), asimulation tool developed at NASA Ames Research Center for exploring advanced ATM concepts. Inthe research, FACET was used to assess different Conflict Detection and Resolution (CD&R)algorithms and different conflicts and resolution procedures.

More precisely, the three studies reported aimed at investigating:

§ Two distinct CD&R algorithms with regard to conflict resolution effectiveness and flightefficiency;



§ How often conflicts requiring multiple decision-makers occur, using a pilot agent modelallowing for negotiation.

§ The impact of route network (free or structured) on conflicts occurring in the airspace.

Some results from Dr. Bilimoria’s investigations reveal that conflicts can be resolved without centralco-ordination, and that free routing reduces the number and complexity of en-route conflicts. Inaddition, the results support the feasibility of airborne self-separation in a free manoeuvringenvironment (as envisaged in DAG-TM Concept Element 5).

2.3.2. Discussion

The use of fast-time modelling was discussed after Dr. Bilimoria’s presentation. Limitations of real-time simulation, having to do with resources and the inability to represent large airspace areas, wereacknowledged. In addition, it was felt that the use of fast-time modelling techniques to assess broadsystem impacts was very beneficial.

The participants also acknowledged that fast-time modelling is only one piece of other requiredstudies for validation, including human factors studies through RT simulations.

Through the detailed question of the ‘conflict’ definition commonly used in fast-time simulation (whichactually corresponds to loss of separation), the presentation also raised the issue of commonterminology in the various validation areas.

2.4. Ulrich Borkenhagen (EUROCONTROL) & Béatrice Raynaud (CENA)

Key words of the briefing: Air-Ground Integration Experiment (AGIE), EACAC’2000 experiments,comparison of US and Europe metrics, INTEGRA metrics,

2.4.1. Briefing

Ulrich Borkenhagen first introduced the context of this metrics comparison performed within theframework of FAA/EUROCONTROL Action Plan 5. Then, Béatrice Raynaud presented the results ofthis comparative analysis of metrics based on case studies, i.e. the AGIE performed jointly byNASA/FAA and the EACAC’2000 experiments at EEC.

Although investigating rather different ASAS-related operational concepts, the human in the loopexperiments had some similarities in terms of scope of the assessment (with a majority of human-related performance metrics), together with some major differences in the validation approachsustaining the experiments. The study was also an opportunity to discuss the relevance of metricsrelated to human performance (and acceptance), efficiency (either control or flight perspective) andsafety.

Consolidation (and lessons learnt) from this study were reported which include the need for:

§ Clear identification of validation objectives (of experiment i.e. ways to assess expected benefitsfrom concept), and a way to conclude on these objectives;

§ Use of models at various levels, including a model of human performance within the new ATMsystem, to support consolidation of experiments design and results;

§ Further refine the set of relevant performance areas and associated metrics measurablethrough human-in-the-loop experiments.

2.4.2. Discussion

Following the presentation, the issue of models that would allow combining and correlating metricswas discussed. It was agreed that objective metrics that relate to expected outputs of ASASapplications (like safe aircraft separation or control efficiency) should be linked to more high-levelobjectives (like capacity) through models that more generally apply to ATM.

On the other hand, the model that would relate the human performance factors (like workload,confidence or skill) to the expected outputs from the human missions in ASAS concepts wasconsidered an open issue with regard to the current state of the art.



The discussion also included comments on the value of collecting more subjective metrics, such aworkload. There was a concern about the use of measures like workload, since often a correlation isnot made between that measure and other objective measures. It was mentioned that more effortcould be made to link objective measures and subjective measures to validate concepts andtechnologies.

2.5. Mark Watson (NATS)

Key words of the briefing: CARE-ASAS Validation framework, template for ASAS operationalscenarios, ATM system and human metrics.

2.5.1. Briefing

The presentation provided by Mark Watson reported the recent work performed within CARE-ASAS,which aims at establishing a common Validation Framework (VF), for ASAS applications in Europe.

The CARE/ASAS VF closely aligns with MAEVA Validation Guidelines Handbook. As such, itconsists in a step-by-step route map for the creation of validation exercises for any ASASapplication. It also includes a scenario template that supports the creation of a validation scenario forASAS applications. Finally, the project has identified a set of system and human performancemetrics to be used for validation purposes.

A case study example dealing with a time-based sequencing application was presented to illustratehow the framework applies. Finally, Mark Watson concluded that the VF presented would encourageuniformity of ASAS validations.

2.5.2. Discussion

During the discussions following the Mark’s presentation, the participants agreed on the necessity tohave a common validation framework.

In this respect, it was mentioned that both the scenario templates and the metrics that were part ofthe CARE-ASAS validation framework will soon be included in the Validation Data Repository (VDR)currently under construction in Europe.

The need to trace back from the metrics to the validation objectives was re-enforced. In this respect,it was pointed out that the usability of the system metrics identified during the project would need tobe validated against the various validation techniques.

2.6. Chad Jennings and Professor J. David Powell (Stanford University)

Key words of the briefing: Closely Spaced Parallel Approach, display design, flight tests, Monte Carlotechniques.

2.6.1. Briefing

Chad Jennings and Professor J. David Powell gave a presentation discussing the development anduse of threat displays for closely spaced parallel approach (CSPA) applications.

The purpose of their research was to design and evaluate a system of sensors and displays thatreduces the necessary spacing for CSPA runways without sacrificing safety. Their methods includedthe use of simulations, flight tests, and Monte-Carlo techniques. The findings from their studiesindicated that synthetic vision displays improved pilots situational awareness while using CSPA, andthat it may be possible to land aircraft in poor visual conditions with runway separations of less than2500 feet.

2.6.2. Discussion

Following the presentation by Mr. Jennings and Dr. Powell, the relative importance of simulation andfield studies was discussed. The authors felt that they learned of particular benefits and issues of thedisplay in the flight tests that they did not learn in simulation. Others stated that flight tests helpprovide user acceptance and political advantages from the actual application in the field.



3. Discussion during the Validation Session

3.1. General

Through the discussions after each briefing, as well as the wrap-up discussions that concluded theValidation session, the following general topics were addressed:

§ Relationship between design and validation;

§ Role of the different validation techniques (e.g., human in the loop experiments, fast-timestudies, field/flight tests);

§ Role of validation with respect to certification.

There was no controversial topic from the audience. The main ideas shared by the participants areprovided hereafter for each of these topics.

3.2. Relationship between design and validation

There were many discussions regarding the relationship between design and validation. The groupfelt that there is a need for a more systematic approach in validation. Examples of this would be thecreation of a validation framework, such as those discussed by Mark Watson and Béatrice Raynaud.

It was also agreed that research efforts need to be guided by the level of maturity of the operationalconcept. High fidelity and costly studies are not appropriate for the testing of a concept that is notyet well developed. A validation framework would help provide guidance concerning the appropriateresearch methods. It would also assist in focusing the experiment methods relative to the objectives.If applied, the framework would allow for the comparison of results across a variety of studies.

Furthermore, it was pointed out that a master validation plan (rather than a validation strategy) forASAS would help to provide common guidelines for efforts in this research area. It would alsosupport cost/benefits analysis necessary for ASAS applications. This plan would reflect acombination of techniques and activities to address different issues and goals. This plan wouldsupport the set-up of several experiments along the development life cycle of ASAS applications.

The question whether such validation framework and validation plan would be specific to ASAS orwould almost derive from general guidelines applicable to ATM was raised. Although not fullyaddressed, it was acknowledged that ASAS applications were not sufficiently different to other ATMoperational concepts to warrant a different overall validation approach. It was also acknowledged thatthe various ASAS applications under investigation were associated with different issues depending onthe shift in separation responsibilities from the air to the ground, and that some specific guidelineswould also help.

3.3. Role of different validation techniques

The presentations covered many validation techniques applied to ASAS applications. Three wereselected for further discussion during the wrap-up session: human-in-the-loop (HITL), fast-timemodelling, and field or flight tests.

3.3.1. Human-in-the-loop

Many presentations discussed HITL efforts as an important technique for assessing humanperformance issues. This approach helps instill confidence in the operational concepts. It alsoserves to identify operational issues that may not have been determined with less mature studymethods.

The group also agreed that concept maturity should be fairly high before the use of HITL techniqueswith a level of fidelity. The cost and resources are indeed fairly demanding, and this is particularlytrue for ASAS applications since they require means for distributed air/ground simulation. It was alsonoted that this method does not require large-scale simulations; smaller scale simulations can alsobe very valuable without requiring the large amount of experiment and financial resources. ActionPlan 5 hosted a meeting in which the application of HITL efforts was summarised (see web sitehttp://www.eurocontrol.int/faa-euro/AP-group-meetings/AP5/ap5-tim.htm).



HITL studies also can be helpful in the development and testing of fast-time models. These studiescan provide data to fast-time models, and allow for the validation of the models within the real-timeenvironment. These efforts were discussed in the presentations by Kevin Corker and ChadJennings/David Powell.

The issue of training of the HITL participants in new concepts (typically those related to majorchanges in roles and responsibility for aircraft separation) was raised during the discussions. It wasmentioned that the issue would also apply to real pilots before ASAS operations are implemented. Inthis regards, it was agreed that the ASAS applications under development should not transform pilotsinto air traffic controllers.

3.3.2. Fast-time studies

Karl Bilimoria and Kevin Corker presented work concerning their efforts in fast-time modelling. Ourgroup felt that fast-time studies played a very valuable role in ASAS concept evaluation. Fast-timemodelling allows for the examination of system-wide effects for a particular change within the ATMenvironment. Additionally, it can provide for many iterations and allow for the evaluation of severalmodifications within that system. Obviously, different levels of system evaluation could be performed.For example, some work presented by Karl Bilimoria’s was addressing CD&R systems, whereas thesystem metrics presented by Mark Watson were related to the overall ATM system from theperspective of different players.

Fast-time techniques require much less resources than a human-in-the-loop experiment, although thedevelopment of the model(s) is often very difficult. Although concepts do not have to be fullydeveloped to use fast-time modelling, the more mature the concepts the more accurate your resultswill be. The group felt that model validation is critical to increase the confidence in the data for themodel, and that models can be provided with more realistic data if some input data are derived fromHITL methods.

Kevin Corker’s work specifically discussed the use of human performance models. This techniqueoffers the advantages of fast-time models applied to human behaviour and cognition. He alsospecifically discussed the linking of HITL methods and model development and validation. It wasagain stressed that this validation of the models is critical for data integrity and user acceptance.

3.3.3. Field studies/ flight tests

We felt that field studies or flight tests play a very unique role in the validation approach for ASASapplications. Chad Jennings and J. David Powell discussed the use of flight tests (in conjunction withother validation techniques) in their presentation.

It was agreed that these methods allow for obtaining user feedback and acceptance of differentprocedures and/or tools. The ability to apply the new operational concept, and evaluate designchoices, in a realistic setting also leads to the discovery of unexpected issues and benefits. Theusers are able to use the new technologies and procedures in ways that are likely not anticipated.Also, field studies/flight tests can help to promote the operational concepts by providing exposure toit for a large number of people, including users and other stakeholders.

One drawback to field studies/flight tests is that they are by nature limited in their scope. It is difficultto represent a variety of procedure or tool changes due to the expense of the test and preparation,and it is often dangerous to represent non-normal events. Thus, it is very hard to conduct thoroughanalyses or to extrapolate the data much beyond the test itself.

In summary, all the participants acknowledged the need to set-up a close loop between the variousvalidation techniques. Nevertheless, the way to do it was considered an open issue. In addition, itwas also recognised that such feedback (for instance, from HITL towards fast-time modelling, andvice-versa) might not have to be done in a first step.

3.4. Role of validation versus certification

Our group discussed the relationship between the validation process and the certification process.Although the processes vary between Europe and the US, as well as flight deck certification andground system certification, it was felt that a stronger connection between validation and certification



would be beneficial.

In general, it was felt that a more integrated validation of ASAS procedures and tools would provideassistance to the certification of the changes that ASAS may require in these areas. End-to-endASAS validation, while being very difficult, would help individuals who are responsible for certificationin the various domains.

It was also considered essential to balance our concerns about certification with our concerns aboutASAS validation design and methods during R&D studies. Many ASAS concepts may involvesignificant modifications to current procedures and/or technologies, and the certification process maynot have the ability to anticipate those changes within the methods that currently are applied. Forexample, an innovative display method may not be able to be certified for use in today’s ATMsystem, but may prove to be very valuable for a new and beneficial ASAS application in a long-termfuture. It would be unproductive to limit ourselves to today’s assumptions about ATM when weconsider the validation of fundamental shifts within our system that ASAS may require.

However, a general view of certification constraints may be possible, and a closer link betweenindividuals within the ASAS community and those within certification can only increase our chancesof creating more useful ATM procedures and technologies.

4. Conclusions

The Validation Session was intended to consider some major questions related to ASAS validationmethods. The conclusions within our group reflect our attempt to address those questions based onthe presentations and discussions during the meeting.

• Which validation methods for ASAS concepts?

The first question concerned the appropriate validation methods for ASAS concepts of use. Wefelt that the techniques related to fast-time models, HITL, and field study or flight tests are allrequired at various stages or concept development and maturity. We also felt that specialemphasis needs to be placed upon the roles of the human operators and how those roles mayshift (sometimes dynamically). In addition, a common validation framework is critical forcomparability of results and for guidance regarding research methods. Finally, we stated thatend-to-end validation, approval, and certification would ultimately be necessary to fully test theconcepts and to gain user acceptance.

• Is there a need to distinguish between the four ASAS categories?

The second question dealt with the necessity to distinguish between the four ASAS categorieswhen considering validation methods. The group felt that the distinction is critical, because theroles of the human operators are different within the four categories. More specifically, controllerand pilot procedures differ, and the automation requirements also vary within the categories. Thiscreates the need for different research questions and techniques to address them. A particularconcern was expressed that we should be cautious about procedures that require the pilot toassume too many controller tasks. These tasks are not currently a part of pilot selection ortraining, and we should be careful about changes that reflect that shift.

• What is relationship with other validation work?

The third question was about the relationship between our session and other validation work. Theworkshop was an opportunity to find a variety of sources that can provide us more informationabout ASAS validation and other concept validation. The Validation Data Repository (VDR) withinAction Plan 5 is a source of data and method guidelines for researchers in the ATM field. Anumber of research professionals will have access to this tool. We can both provide data to theVDR and obtain data from it. Work within CARE/ASAS (Mark Watson’s presentation) will help tosupply and maintain a validation framework using the approach that has been developed. Thepresentation by Béatrice Raynaud also discussed a case study approach in which validationtechniques were compared and assessed. This can allow for thorough examinations of ASASmethods as they are applied and can help us determine the value of similar techniques. Finally,the Virtual Airspace Modelling and Simulation (VAMS) project within NASA is developing andassessing methods and metrics for testing new operational concepts. The VAMS project will



provide valuable data regarding concept evaluation that can be applied to ASAS concepts andapplications.

In summary, there was a general agreement inside the group that much validation work is beingaddressed in many different ATM research areas. The application of the work within Action Plan 1,coupled with the use of research activities and resources from other organisations, should allow for astrong foundation for the validation of ASAS concepts and applications.



ARCHITECTURE SESSION1. Introduction

The Systems and Architecture Session (session 4) was conducted in parallel with sessions onSafety and Validation. The session was chaired by Frank Mackowick, Johns Hopkins University –Applied Physics Laboratory, and Daniel Ferro, Airbus. The session commenced at 10:00 on Day 2and concluded with a discussion of issues and recommendations the morning of Day 3. Sevenreports were presented and discussed. A summary Report of the Systems and Architecture Sessionwas presented the afternoon of Day 3 in plenary.


The goal of this session was to present and to share recent developments in System Architectures tosupport ASAS applications, and to promote harmonisation between the United States and Europe.Specific topics for which presentations were solicited included current aircraft equipage and groundsystems and assessments of their future evolution; information on current research and flight trials;cockpit decision support systems; anticipated impact on current avionics, for example FMS (FlightManagement System) and ATM systems; ASAS integration with other data link applications, andmixed-environment services such as TIS-B. Specific objectives established at the start of theSession were:

§ to present results of research about implementation of ASAS applications on ground &airborne systems;

§ to exchange and consolidate views based on those results; e.g., estimate what is needed toimplement the future applications;

§ to analyse current co-ordination between ground and airborne related research activities; can itbe improved ?

§ to provide guidance and recommendations for future work.


The Architecture session was organised so as to allow for discussion between the participants:

§ On Day 2: Following each briefing presented in the session (30 minutes presentation and 15minutes left for discussion), and

§ On Day 3: The session participants again met to formulate the issues and recommendationsresulting from the session. Finally, the session report prepared by the co-chaired waspresented in plenary.


The following briefings were provided by the listed presenters and authors:


"Newly Enacted Intent Changes to ADS-B MASPS:Emphasis on Operations, Compatibility, and Integrity"

Richard Barhydt(NASA Langley Research Center)

Coauthor: Tony Warren(Boeing Air Traffic Management)

"Flight Deck Decision Support System for AirborneAutonomous Operations"

Ed Johnson(NASA Langley Research Center)

"The Airframer Point of View - Airbus" Daniel Ferro (Airbus)




"ASAS Surveillance Architectures"Chris Nehls (Honeywell International)

Coauthor: Daryal Kuntman(Honeywell International)

"TIS-B Error Budget Analysis: Status and SelectedResults"

Richard Kochanski

(Johns Hopkins University AppliedPhysics Laboratory)

"EUROCONTROL ADS Programme - Systems andArchitecture Activities Supporting ASAS Applications"

Alessandro Prister (EUROCONTROL)

Author: Giles Caligaris(EUROCONTROL)

“AP4 Workshop Report”Alessandro Prister (EUROCONTROL)

2. Review of the Architecture Session Briefings

2.1. Richard Barhydt (NASA/Langley)

Title: Newly Enacted Intent Changes to ADS-B MASPS: Emphasis on Operations, Compatibility, andIntegrity

Key words of the briefing: DO-242A, Intent, Target State, DAG-TM

2.1.1. Presentation

The briefing presented a detailed review of the evolution of ADS-B intent information as defined in theRTCA SC-186 ADS-B MASPS; and showed examples of how it can be used to broadcast both FMS-and autopilot-derived flight profile information (Trajectory Change and Target State Reportsrespectively). Included in the briefing was an overview of the rationale behind the recent expansion ofintent information in the latest version of the ADS-B MASPS (DO-242A) to include additional dataelements for each trajectory change point; and a new Target State Report providing currentinformation on the active flight segment included commanded heading and altitude and an indicationof the integrity of the heading and altitude sources. The commanded trajectory appears as being themost unambiguous information available. The briefing concluded with an introduction to NASA’sDistributed Air/Ground Traffic Management (DAG-TM) program that will explore ways in which ADS-Bintent information can be used to support the development of ASAS applications such as conflictdetection and resolution. (This was a lead-in to the following presentation by Ed Johnson ofNASA/Langley.)

2.1.2. Discussion

The meeting felt the material to be a very useful review of many of the fine points of the new intentinformation that were not readily apparent from the MASPS material. Several posited whether itmight be possible to use ADS-B intent information as a surrogate for some or all of the Mode-S DAPparameters. This was not explored during the discussion although it was generally recognised thatvalidation of this new ADS-B data content has not yet been completed.

At least one attendee was curious whether the planned DAG-TM research would explore the use ofdifferent update rates for Trajectory Change vs. Target State reports. Richard noted that their currentDAG-TM analysis is concentrating on long term strategic planning and, as a result, their simulatedupdate rates are being set to serve that environment. Further research is needed, as the updaterates remain unspecified in the ADS-B MASPS. Similar work is to be done on the required number ofTrajectory Changes and required time horizon at any rate, the impact on data link loading would alsohave to be taken into account.



One commenter noted that fundamental avionics system interactions are, in general, not wellunderstood by the industry. MCP/FCU override of FMS commanded path will be an issue as willvarying abilities of various product lines to be modified to provide new intent bus information.

The session also highlighted important differences in behaviour of (for instance) Airbus and BoeingFMS, as well as pointed out differences of vocabulary between the attendants (in-trail spacing havingquite different senses, from approach spacing to sequencing).

2.2. Ed Johnson (NASA/Langley)

Title: Flight Deck Decision Support System for Airborne Autonomous Operations

Key words of the briefing: DAG-TM, Decision Support System, Airborne Autonomous Operations,FMS

2.2.1. Presentation

Picking up where Richard Barhydt left off in the preceding presentation, this presentation presentedthe ongoing development by NASA Langley Research Center of an experimental AutonomousOperations Planner (AOP) system under the DAG-TM program. The research to date has producedan operating AOP prototype that interacts with an FMS to provide the crew with a decision supportsystem that proposes resolutions to conflicts detected via ADS-B. Basically, AOP performsstrategic optimisation (taking into account airline policy and some crew preferences), while FMSperforms trajectory optimisation. Solutions proposed by the AOP algorithm also take account of own-ship performance limitations and airspace restrictions including areas of hazardous weather.Resolutions can be state-based (i.e., based on present aircraft state vector) or intent-based (i.e.,based on own-ship and/or intruder trajectories). AOP has intended applications beyond ASAS and itwas shown that the system can also formulate an optimum route modification on an ad hoc basis inresponse to new constraints such as a revised RTA or new special use airspace.

The flight crew interacts with the AOP system via the navigation display (ND) and a dedicated MCDUpage. Using experimental symbology, the ND overlays a depiction of the projected conflict and anoptimised resolution based on iterative interactions with the FMS. Also briefed were information onthe internal system architecture of the AOP system and ways in which an operational implementationcould be apportioned to elements of the Arinc 660A CNS/ATM avionics functional architecture.

2.2.2. Discussion

The work was very well received by the meeting. One discussion point noted that the AOP’sresolution strategy for seeking a solution to a new RTA can look beyond the time-based look-aheadhorizon for conflict detection and resolution (which is itself configurable for any look-ahead interval.

It was also noted that horizontal and vertical resolutions are calculated independently. To facilitatethis process, AOP iterates a problem through the FMS until it receives a calculated profile that it canaccept. In addition, the flight crew are always able to revise an AOP trajectory at any time afteractivation thereby forcing AOP to treat the revised trajectory as a new constraint should furtheroptimisation be requested.

The current focus of the AOP research is limited primarily to en route algorithm development. Thislimitation is due to the capabilities of the FMS currently available to the team. Work on AOPfunctionality for descent and terminal operations is hoped to begin in about one year depending onthe availability of an updated FMS to include a VNAV capability.

Also, it would be interesting to study airborne back-up functions, since the one presented would besupported by a system with a dependability close to that of the FMS (relatively low).

2.3. Daniel Ferro (Airbus Industrie)

Title: The Airframer Point of View

Key words of the briefing: HMI, flight deck integration, data correlation



2.3.1. Presentation

The briefing presented Airbus’ view on the integration of ASAS functionality into their flight deck HMI.Experimental ND depictions of situational awareness and spacing applications were shown includingthe concurrent depiction of ADS-B and TCAS target information. Also presented were examples of afunctional architecture to implement spacing applications on the Airbus flight deck based on NUP Ilessons-learned including automation issues. It was also noted that CPDLC is likely a powerfulenabler for spacing applications. The briefing stressed the need for a top/down approach based onthe ED78A methodology to derive validated requirements.

2.3.2. Discussion

There was some discussion about the implementation of in-trail spacing applications including someconcern about data accuracy / integrity and potential mitigations. It was noted that the crew wouldhave to apply basic procedures as a component of the mitigations. Additional assistance tools maybe needed. In addition, there may be a need to add target airspeed to the ADS-B Target State report.One attendant recommended planning for the evolution of the flight data recorders to cope with futureASAS applications. Finally, it was agreed that ACAS functions must remain independent from ASASones.

2.4. Chris Nehls (Honeywell International)

Title: ASAS Surveillance Architectures

Key words of the briefing: product roadmap, ACAS/ASAS co-ordination, avionics architecture

2.4.1. Presentation

The presentation reviewed the first principles of ASAS and tied them to the evolving certificationstandards (MASPS, MOPS). High-level architectural options were presented for adding ASAS andADS-B data link functionality incrementally to different types of avionics architectures spanningcurrent and first generation digital avionics systems and light GA applications. Also explored was abroad market roadmap perspective of the likely implementation of ASAS applications based on ADS-B and ACAS functionality across various classes of aircraft.

It was noted that the U.S. dual link decision could create ASAS transition issues that could causedelays in operational capability. The primary risk cited in this connection was that a dual linkenvironment could complicate aircraft equipage and would place additional ground infrastructuredevelopment in the critical path of ASAS deployment.

It was posited that the TCAS computer system, evolved into an Airborne Traffic SurveillanceProcessor (ATSP) could serve as a host providing a partitioned execution environment for ASAprocessing as is expected to be defined in the eventual ASSAP MOPS. However, having said this itwas felt that ACAS and ASAS functions must be both independent and co-ordinated based on thespecific procedural environment for each application.

2.4.2. Discussion

There was some curiosity as to which applications or ASAS categories are being addressed inHoneywell’s architecture planning. Chris indicated that as the development of MASPS for theapplications was not yet mature, it would be premature to assess the suitability of their anticipatedequipment architecture for specific applications. Honeywell felt their high level planning work to dateto be generic enough to not exclude any application. Their primary goal at this point is to minimisethe number of new LRUs and antennas needed. Also, the unavailability of some NIC (NavigationIntegrity Category), NAC (Navigation Accuracy Category) and SIL (Surveillance Integrity Level)parameters on simple GPS receivers of some GA aircraft could be an issue.

2.5. Rich Kochanski (John Hopkins University)

Title: TIS-B Error Budget Analysis: Status and Selected Results



Key words of the briefing: TIS-B, radar error analysis, track accuracy, covariances

2.5.1. Presentation

The presentation reviewed ongoing work under the FAA Safe Flight 21 program to characterise FAAradars in order to support the development of a TIS-B error budget. Simulation results for the ARSR-4long-range radar were presented showing bearing and range error statistics at various ranges from theantenna. The resultant error models were used to drive subsequent simulations that depict the likelysingle-sensor error contours for a TIS-B target as perceived by an own-ship CDTI when viewing anintruder aircraft.

The briefing also illustrated how the use of covariance data (rather than NIC/NAC) to define a TIS-Btarget’s uncertainty bubble can have a beneficial effect. This is because covariance data associatedwith a TIS-B track will define a relatively narrow error ellipse while the use of NIC/NAC forces the useof a circular area of uncertainty which can be quite large if the true error ellipse is to becircumscribed. However, it was recognised that transmitting covariance information might not bepractical due to the adverse impact on data link loading.

Also noted was the notion that the coasting of TIS-B tracks between radar updates must beaddressed in the design of the TIS-B system. Decisions are needed on whether TIS-B tracks are tobe coasted on the ground and/or in the ASAS application using TIS-B data. In that connection,validated ASAS performance requirements are needed.

2.5.2. Discussion

There was general agreement that the TIS-B error analysis work will be enhanced once agreedperformance requirements for ASAS applications are available for use in assessing the degree towhich TIS-B data can be used in an ADS-B environment.

The concept of using covariance data (which is generated in a radar tracking function) was new tomost attendees. Although the concept seemed straightforward, it was felt that sending covarianceterms over the data link could be a non-starter due to excessive load. However, this very fact mightseem to argue for ground-coasted TIS-B data with updates sent over the data link at an update rateakin to that of ADS-B emitters. The meeting generally felt that the TIS-B ground system shouldcalculate NAC (and NIC) and let the aircraft decide if a TIS-B Report is useful for the desiredoperation.

2.6. Alessandro Prister (EUROCONTROL)

Title: EUROCONTROL ADS Programme - Systems and Architecture Activities Supporting ASASApplications

Key words of the briefing: functional architecture, physical architecture, ARTAS, TIS-B, surveillancedata processing.

2.6.1. Presentation

The meeting was briefed on EUROCONTROL’s ongoing activities to develop and validate an ADS-B/TIS-B system architecture and three packages of airborne and ground surveillance applicationsenvisaged to be deployed in a phased approach through 2015. Plans for the initial phase include fiveground surveillance applications including ATC surveillance and airport surface applications and, forairborne surveillance, seven applications such as enhanced situational awareness, enhanced visualapproach, sequencing/merging/crossing applications, and oceanic in-trail procedures.

EUROCONTROL anticipate that the functional and physical architecture definitions for their groundADS-B/TIS-B infrastructure will be essentially complete by the end of 2002 so that development of aninitial TIS-B server to support application validation activities can conclude by mid-2003. This server,along with recent and in-process upgrades to the ARTAS network, will be used to support validationof the ADS-B applications planned for their first-phase deployment.

Concurrent with work on ground infrastructure development, EUROCONTROL are also sponsoringvarious systems analysis work packages concentrating on areas such as airborne surveillance data



processing applications, ACAS/ASAS relationship, and end-to-end functional architecture definition.This work, expected to begin soon, is closely aligned with airframers, avionics manufacturers, relatedorganisations with relevant experience who have made proposals for various work packages.

2.6.2. Discussion

Due to schedule constraints, only a very brief and general discussion of the material was possible.

2.7. AP4 Workshop Report - Alessandro Prister (EUROCONTROL)

Title: AP4 Workshop Report

Key words of the briefing: AP4

2.7.1. Presentation

A short briefing was provided on the June 2002 workshop held by Action Plan 4 (AP4) on the impactof ATM/CNS evolution on avionics and ground systems architectures. Several findings of note fromthe meeting report include the following:

§ Manufacturers and operators are reluctant to invest in new systems until standards, validatedconcepts of operation, and realistic implementation plans have been completed.

§ TIS-B is viewed as a critical enabler for ASAS due to the likely difficulty of universal ADS-Bequipage. In addition, TIS-B can serve as a useful integrity check on ADS-B data.

§ Transition issues for ASAS implementation need to be addressed.

§ There is a need for a global standard for the generation and exchange of 4D trajectories.

The briefing also noted several recommendations made by AP4 for consideration by AP1 during thecourse of this meeting:

§ Identify possible user applications at an early stage.

§ Consider architecture-level benefits in addition to those specific to individual concepts andsystems.

§ Adopt a total system approach to safety assessments.

2.7.2. Discussion

As this presentation was a late addition to the session agenda and was offered primarily forinformation, lack of time remaining precluded further discussion of the material.

3. Discussion during the Architecture Session

Due to the rather full agenda of presentations for the session, there was no general discussion apartfrom the post-presentation discussions summarised immediately above. On the morning of Day 3, thesession reconvened briefly in order to formulate issues and recommendations to be presented duringthe session report presented in Plenary later in the day. Briefly, these were as follows:

• Issue 1: The community at-large is using ASAS terminology without due attention tothe operational implications. This puts at-risk user acceptance of ASAS applicationsif expectations become unreasonable.

• Issue 2: Manufacturer business cases for ASAS-enabling products remain uncertaindue to the tenuous nature of envisaged operational benefits.

• Recommendation 1: The research area needs to support standards development bydelivering data products that define the functional, performance, and safetyrequirements of validated applications for a particular operational environment (enroute, terminal, surface).



• Recommendation 2: The integration of ASAS applications on the flight deck mustensure a harmonised philosophy of display and alerting functions without invalidatingthe independence of protection systems (e.g., ACAS/TCAS, TAWS)

• Recommendation 3: We need a high-level ASAS functional architecture document,akin to PO-ASAS, that can be used as a common baseline for study anddevelopment.

• Recommendation 4: The development of ATC monitoring and alerting functions mustbe co-ordinated with the development of equivalent aircraft functions.

4. Conclusions

In the opinion of the co-chairmen, the architecture work is mainly (but not only) the conclusion of along design process. This process starts with the definition of the ASAS applications, and, evenbefore, with the operational improvements they intend to address.

It is quite obvious that the architecture work, while useful anyway to close the design loop of thoseASAS applications, indeed requires more maturity in:

§ The definition of the applications themselves; and

§ The definition of high-level operational requirements addressed to each domain of the totalsystem (e.g. airborne, ATC, ground vehicles, TIS-B provider, communication network provider).

As long as the definition of the ASAS applications and of their consequent operational requirementsis not stable enough, the definition of the ground and airborne architectures will remain difficult andrisky.



TIM WRAP-UP DISCUSSIONSAND

CONCLUSION SESSION

Starting at 10 am on Wednesday, October 23, a summary of each session was given by one of thesession co-chairs to the plenary meeting. The main points of each summary are covered in theconclusion section of each session above.

There was lively discussion on how to move towards implementation of the early ASAS applications.Enhanced see-and-avoid and Enhanced situational awareness-airborne applications are alreadycertified for use by UPS aircraft. Enhanced visual approaches will hopefully be certified next year.There are differences between the US and European approaches, but there was some consensus onthe difficulties associated with certification, and demonstrating sufficient benefits with the near termapplications to warrant their implementation. The R&D community should focus on all the applicationcategories, although some organisations will be more near term, and some, like NASA, will be morefar term. In Europe, it is easier to develop operational scenarios for far term applications because theircommunity can discuss issues with air traffic controller organisations. In US this is harder becausethe focus of SF21 activities is solely near term applications.

There was discussion of acceptance of ASAS by pilots and controllers, who might feel theirtraditional roles were threatened. The value of involving end users in the development of applicationswas debated, because practitioners often find it difficult to distance themselves from current practice.It might be more appropriate to involve them more in near term applications.

At the end of the plenary meeting, the TIM organisers and session co-chairs were thanked for theirefforts in organising and running a successful meeting. Some participants were able to stay for ademonstration, including the Ames Boeing 747-400 full motion simulator, of the Airborne Spacingwork described by Dr. Everett Palmer during the Applications Session.



TIM AGENDAMonday, October 21, 2002

Introduction

8:00 Registration and Coffee

9:00 Welcome AddressDr Victor Lebacqz, NASA, Ames Research Center

9:15 Organization & Objectives of Technical Interchange MeetingRose Ashford, NASA, Ames Research Center

9:45 US ASAS ApplicationsGene Wong, FAA

11:00 European ASAS ApplicationsFrancis Casaux, CENA

Session 1: ASAS Applications

Co-Chairs: Eric Hoffman, EUROCONTROL Experimental Centre Mark Ballin, NASA Langley Research Center

13:00 Objectives of the Applications SessionEric Hoffman and Mark Ballin

13:15 DAG-TM Air-Ground Integration Experiment, September 2002Everett Palmer, NASA Ames Research Center

14:00 Limited delegation with arrival streams: more insight on its impact on controller activityKarim Zeghal, EUROCONTROL Experimental Centre

15:00 CDTI Enhanced Flight Rules: Concept and Initial Simulation ResultsRandy Bone, MITRE/CASSD

15:45 Evaluation of a Time-Based Airborne Inter-arrival Spacing ToolGary Lohr, NASA Langley Research Center

16:30 Airborne Information for Lateral Spacing: A Concept for Independent, Closely SpacedParallel ApproachesGary Lohr, NASA Langley; Vernol Battiste, NASA Ames

Tuesday, October 22, 2002

Session 1: ASAS Applications, continued

8:00 Use of Traffic Intent Information by Autonomous Aircraft in Constrained OperationsDavid Wing, NASA Langley Research Center

8:45 Mediterranean Free FlightAndy Barff, EUROCONTROL Experimental Centre

Session 2: Safety in ASAS Applications

Co-Chairs: Ken Carpenter and Andrew Zeitlin, MITRE/CAASD



10:00 Objectives of the Safety SessionKen Carpenter

10:30 Overview of the AP1 Paper “Safety and ASAS Applications”Andrew Zeitlin

11:00 Safety Analysis of An IMC Final Approach Spacing ApplicationJonathan Hammer, MITRE/CAASD

13:00 On Safe Separation of an ASAS-based OperationBart Klein-Obbink, NLR

13:40 Contribution of ADS-B Accuracy, Integrity and Continuity to Surveillance SeparationServiceStan Jones, MITRE/CAASD

14:20 Piloted Simulation of Airborne Separation Assurance Scenarios that Pose a PotentialSafety Risk: Preliminary ResultsDavid Wing, Richard Barhydt, Ed Johnson, Mike Palmer, NASA Langley

15:15 Safety of ASAS operations – Discussion

Session 3: Validation of ASAS Applications

Co-Chairs: Béatrice Raynaud, CENA and Sandy Lozito, NASA Ames Research Center

10:00 Introduction and Objectives of SessionSandy Lozito and Béatrice Raynaud

10:15 Free Flight: Context of ControlProfessor Kevin Corker, San Jose State University

11:00 Fast-time Simulation Studies of Airborne Self-SeparationDr. Karl Bilimoria, NASA Ames Research Center

13:00 Comparison of EACAC and AGIE MetricsUlrich Borkenhagen, EUROCONTROL and Béatrice Raynaud, CENA

13:45 CARE-ASAS Validation Framework, Guidelines and Case StudiesMark Watson, NATS

14:30 Use of Master Air Traffic Management European Validation Plan (MAEVA) Guidelines inthe INTENT Project – not presented

15:00 Threat Displays for Closely Spaced Parallel ApproachChad Jennings and J. David Powell, Aero/Astro Dept., Stanford University

15:45 Validation of ASAS Applications – Discussion

Session 4: Architecture (Airborne and Ground)

Co-Chairs: Frank Mackowick, Johns Hopkins University Applied Physics Laboratory and Daniel Ferro, Airbus Industrie

10:00 Opening RemarksFrank Mackowick and Daniel Ferro

10:15 Newly Enacted Intent Changes to ADS-B MASPS: Emphasis on Operations,Compatibility, and IntegrityRichard Barhydt, NASA Langley Research Center and Tony Warren, Boeing Air TrafficManagement



11:00 Flight Deck Decision Support System for Airborne Autonomous OperationsEd Johnson, NASA Langley Research Center

13:00 Recap and Introduction to the Afternoon SessionFrank Mackowick and Daniel Ferro

13:15 The Airframer Point of View – AirbusDaniel Ferro, Airbus

14:00 ASAS Surveillance ArchitecturesChris Nehls and Daryal Kuntman, Honeywell International

15:15 TIS-B Error Budget Analysis: Status and Selected ResultsRichard Kochanski, Johns Hopkins University Applied Physics Laboratory

16:00 EUROCONTROL ADS Programme - Systems and Architecture Activities SupportingASAS ApplicationsAlessandro Prister and Gilles Caligaris, EUROCONTROL

Wednesday, October 23, 2003

Session 4: Architecture, continued

8:00 Presentation from Thales ATM

Session Reports

10:00 Report of Session 1, ASAS ApplicationsEric Hoffman, EUROCONTROL Experimental Center

10:30 Report of Session 2, Safety of ASAS ApplicationsAndrew Zeitlin, MITRE/CAASD

11:00 Report of Session 3, Validation of ASAS ApplicationsSandy Lozito, NASA, Ames Research Center

11:30 Report of Session 4, Systems and ArchitectureFrank Mackowick, Johns Hopkins University Applied Physics Laboratory

Concluding Session

13:15 DiscussionFrancis Casaux, CENA

Demonstration

15:15 Trajectory negotiation demonstration at the Ames Air/ground Integration LaboratoryEverett Palmer, NASA, Ames Research Center



TIM PARTICIPANTSName Affiliation EmailAshford, Rose NASA ARC [email protected], Brian FAA [email protected], Mark NASA LaRC [email protected], James (Andy) EUROCONTROL [email protected], Richard NASA LaRC [email protected], Vern NASA ARC [email protected], Karl NASA ARC [email protected], Randy The MITRE Corp [email protected], Ulrich EUROCONTROL [email protected], Steve FAA [email protected], John A. Boeing [email protected], Kenneth M. QinetiQ [email protected], Francis CENA/SOFREAVIA [email protected], Soren EUROCONTROL [email protected], Daniel Airbus France [email protected], Tom Trios Inc. [email protected], Jonathan The MITRE Corp [email protected], Bernard CENA/SOFREAVIA [email protected], Daniel Civil Aviation Authority [email protected], Bob UPS [email protected], Doug FAA [email protected], Eric EUROCONTROL [email protected], David NASA ARC SJSU [email protected], Edward NASA LaRC [email protected], Kenneth NASA LaRC [email protected], Stan The MITRE Corp [email protected], Richard Johns Hopkins University [email protected], Steve Rockwell Collins Inc [email protected], John NASA LaRC [email protected], Parimal NASA ARC [email protected], Richard NASA ARC [email protected], Diana FAA [email protected], Gary NASA LaRC [email protected], Sandy NASA ARC [email protected], Frank Johns Hopkins University [email protected], Lynne NASA ARC SJSU [email protected], Colin EUROCONTROL [email protected], K. Tysen Seagull TechnologiesNehls, E. Chris HoneywellObbink, Bart Klein NLR [email protected], Ev NASA ARC [email protected], Robert FAA [email protected], J. David Stanford University [email protected], Carmine FAA [email protected], Alessandro EUROCONTROL [email protected], Béatrice CENA/SOFREAVIA [email protected], Jean-Claude Thales Avionics [email protected], James (Jim) FAA [email protected], Mike NATCA [email protected]



TIM Participants, continued

Name Affiliation EmailSimons, Elliott The MITRE Corp [email protected], Nancy NASA ARC [email protected], John Seagull Technologies [email protected] Gool, Mick EUROCONTROL [email protected], Savvy NASA ARC SJSU [email protected], Ganghuai The MITRE Corp [email protected], Mark National Air Traffic Services Ltd [email protected], Joel Rockwell Collins Inc [email protected], Keith Smiths Aerospace [email protected], David NASA LaRC [email protected], Gene FAA [email protected], Karim EUROCONTROL [email protected], Andrew The MITRE Corp [email protected], Robert NASA ARC [email protected]

Documents

Report of the Third ASAS Technical Interchange Meeting · Report of the Third ASAS Technical Interchange Meeting Action Plan 1 ... was held at NASA Ames Research Center near San Francisco,