EUROCONTROL EXPERIMENTAL CENTRE · Commission and support by the Swedish (LFV) and Danish (SLV) Civil Aviation Authorities. ... up through a programme of concept validations to address

EUROPEAN ORGANISATIONFOR THE SAFETY OF AIR NAVIGATION

EUROCONTROL EXPERIMENTAL CENTRE

European ATC Pre-Operational Validation & Experimental Trials Platform(PROVE)

PROVE/SYSCO TRIAL B REPORT

EEC Report No. 354

Project STS-U-00

Issued: October 2000

The information contained in this document is the property of the EUROCONTROL Agency and no part shouldbe reproduced in any form without the Agency’s permission.

The views expressed herein do not necessarily reflect the official views or policy of the Agency.

EUROCONTROL

REPORT DOCUMENTATION PAGE

Reference:EEC Report No. 354

Security Classification:Unclassified

Originator:ATMDC - NATSNational Air Traffic Services

Originator (Corporate Author) Name/Location:National Air Traffic Services Ltd.,Air Traffic Management Development Centre,Bournemouth International Airport, Christchurch,Dorset BH23 6DFUnited KingdomTelephone: +44 1202 472340

Sponsor:EURCONTROL Experimental CentreChris Chicken

Sponsor (Contract Authority) Name/Location:EUROCONTROL AgencyRue de la Fusée, 96B –1130 BRUXELLESTelephone : +32 2 729 9011

TITLE:

PROVE (Pre-Operational Validation & Experimental Trials Platform)

SYSCO TRIAL B REPORT

AuthorSusan Russell

Peter McKinnon

Date10/2000

Pagesxii + 61

Figures12

Tables1

Appendix2

References10

-Project

STS-U-00Task No. Sponsor Period

April – May 2000

Distribution Statement:(a) Controlled by: Head of ERIS(b) Special Limitations: None(c) Copy to NTIS: NO

Descriptors (keywords):

OLDI – PROVE - SYSCO – Electronic Co-ordination – Shadow-Mode Trials – Validation – ESCAPE –EATCHIP – DSI

Abstract:

This report has been produced for the EUROCONTROL Experimental Centre, Brétigny by National AirTraffic Services Ltd, and reports the results from the second trial conducted simultaneously on thePROVE platforms at Malmö and Copenhagen in April/May 2000

This document has been collated by mechanical means. Should there be missing pages, please report to:

EUROCONTROL Experimental CentrePublications Office

Centre des Bois des BordesB.P. 15

91222 - BRETIGNY-SUR-ORGE CEDEXFrance

PROVE/SYSCO Trial B Report

v

FOREWORD FOR EUROPEAN ATC PRE-OPERATIONAL VALIDATION &EXPERIMENTAL TRIALS PLATFORM (PROVE) SYSCO TRIAL B REPORT

This report presents the results of the Live Shadow Mode Trial of SYSCO betweenCopenhagen’s and Malmo’s ACC performed in the context of the project PROVE.

PROVE was sponsored by EUROCONTROL with co-funding provided by TEN-T of the EuropeanCommission and support by the Swedish (LFV) and Danish (SLV) Civil Aviation Authorities.

Numerous research and simulation studies are performed in Europe both by industry andresearch establishments every year. However, the time elapsed between these studies and theirdeployment in operational systems is often of the order of ten to twenty years. Many reports,including EATMS Validation Strategy Document, version 1.1 and EUROCONTROL’s IndustrialSuppliers Relationship, version 1.0, have suggested that there are two principal contributingfactors to this phenomenon:

1. Industry will not invest in new concepts that have not been validated, hence there is a lack ofa dedicated infrastructure to validate concepts in a "live" pre-operational environment,

1. The varied and non standard requirements placed on industry by the national administrations,especially in the domain of ODS European ATM projects.

PROVE (European Pre-Operational Validation & Experimental Trials Platform) is organised inseveral phases to address these issues; initially to provide a pre-operational validationinfrastructure and to perform an initial trial to validate SYSCO. This will subsequently be followedup through a programme of concept validations to address the issue of non standard ODS andFDPS specifications.

PROVE took EUROCONTROL’s Simulation Capability and Platform for Experimentation(ESCAPE) and interfaced it to the ATM operational systems in Malmo and Copenhagen andestablished a dedicated data link between the two centres. This platform ran in parallel (shadowmode) to the real systems, being fed with the same live operational data (radar and flight planinformation) and was manned by certified controllers.

This platform supported a trial that was executed during April and May 2000 with the objective ofvalidating OLDI 2.2 (SYSCO) under shadow mode conditions.

Chris ChickenEUROCONTROL Project Manager


vi

This page intentionally blank


vii

SUMMARY

The European ATC Pre-Operational Validation and Experimental Trials Platform(PROVE) is a project jointly funded by EUROCONTROL and the European Commission(Ref. 1). The project is being supported by Denmark and Sweden together with othermember states. This report has been produced for the EUROCONTROL ExperimentalCentre, Brétigny, by National Air Traffic Services Ltd, and reports the results from thesecond trial conducted on the PROVE platform at Malmö (Sweden) and Copenhagen(Denmark) in April/May 2000. This trial built upon the results of the first trial which tookplace in Malmö in December 1999 (Ref. 2).System-Supported Co-ordination (SYSCO) is a development of the on-line datainterchange (OLDI) standard and provides support to electronic co-ordination betweensectors and between ATC centres. The system employed for the trials was adevelopment of the HMI developed under the Denmark-Sweden Interface (DSI) projectand EUROCONTROL’s ESCAPE platform, evolved to interface with live radar tracksand flight plan sources. The aim of this trial was to determine the fitness for purpose ofthe SYSCO message set using DSI at the Malmö-Copenhagen boundary.The PROVE platforms were constructed at the Area Control Centres (ACCs) in Malmöand Copenhagen. The trial was conducted in ‘shadow-mode’, that is the prototype wasfed live data from radar and other systems, but not actually used to control or influencelive traffic.Fourteen controllers participated throughout the 14 day trial, eight of whom were fromLuftfartsverket (LFV), Sweden, and six were from Statens Luftfartsvaesen (SLV),Denmark. The potential value of the trial was reduced by the incorrect or incompletefunctionality of some features, in particular the medium term conflict detection (MTCD),and the need to use a number of work-arounds. However, the controllers were able tocarry out an initial evaluation of the SYSCO features in the context of real traffic, and anumber of requirements and improvements were identified.The controllers felt that SYSCO worked well for basic co-ordinations. However, withincurrent day procedures the system was not sufficiently flexible. The system was unableto take into account changes to the sector sequence following direct route co-ordinationsand unexpected climb profiles. Thus, there was a fundamental need to override thesystem’s prediction of the next sector. In addition, SYSCO as implemented within DSIwas not able to support simultaneous co-ordination with both an upstream anddownstream sector.

Level co-ordinations were more successful than direct route co-ordinations, howeverconcern was raised amongst the controllers that the implementation of SYSCO withinDSI does not display coordination messages for 'standard' conditions, principally whenchanges were made from one 'standard' level to another.

In shadow-mode, the controllers were unable to fully evaluate the impact of SYSCO oncontroller workload. The controllers felt constrained by not having responsibility for thetraffic, and having to react to situations rather than being in control and able to planahead. However, the controllers suggested that SYSCO has the potential to reduceworkload in an en-route environment through the reduction in telephone calls. It was feltthat further consideration is required to evaluate its appropriateness to approach control.


viii

The general attitude of the controllers towards the use of SYSCO in the future wasoptimistic. However they recognised that further development is required before it isacceptable for operational use.

The PROVE/SYSCO trials were, in the end, insufficient to provide a thorough validationof SYSCO, mostly for engineering reasons. However, they provided the operationalclimax to an intense engineering effort, moving software out of the experimental centreinto the operations room. They also deepened the understanding of how and when toconduct shadow-mode trials in that they should not be used to measure workload, andcan only validate systems within the constraints of current day airspace, procedures andtraffic volumes. Shadow mode does however provide an ideal platform for betterunderstanding operational issues, and exposing new concepts to a wider audience ofvalidated controllers.


ix

TABLE OF CONTENTS

LIST OF FIGURES .................................................................................................................................. XREFERENCES ........................................................................................................................................ XABBREVIATIONS .................................................................................................................................. XI1. INTRODUCTION ...........................................................................................................................1

1.1 General ..........................................................................................................................................11.2 Background....................................................................................................................................11.3 Trial Aim .........................................................................................................................................11.4 Report Structure.............................................................................................................................2

2. SYSCO...........................................................................................................................................32.1 Notification Phase ..........................................................................................................................32.2 Co-ordination Phase ......................................................................................................................32.3 Transfer Phase...............................................................................................................................4

3. THE TRIAL SYSTEM ....................................................................................................................63.1 Introduction ....................................................................................................................................63.2 Denmark-Sweden Interface (DSI)..................................................................................................73.3 Aircraft States.................................................................................................................................73.4 Co-ordination Window....................................................................................................................83.5 Sector Inbound List ........................................................................................................................93.6 Callsign Menu ................................................................................................................................93.7 Instruction Menus.........................................................................................................................10

4. TRIAL DESIGN............................................................................................................................114.1 Introduction ..................................................................................................................................114.2 Airspace and Traffic .....................................................................................................................114.3 Trial Timing ..................................................................................................................................134.4 Participants and Training .............................................................................................................134.5 Method of Operations ..................................................................................................................134.6 Measurements and Analysis........................................................................................................134.7 Usage and Usability .....................................................................................................................134.8 Workload......................................................................................................................................144.9 Safety144.10 Quality of Service....................................................................................................................14

5. RESULTS AND DISCUSSION....................................................................................................155.1 Trial Account ................................................................................................................................155.2 Usage and Usability .....................................................................................................................165.3 Workload......................................................................................................................................235.4 Safety245.5 Quality of Service.........................................................................................................................255.6 Shadow Mode Trials ....................................................................................................................25

6. CONCLUSIONS...........................................................................................................................277. RECOMMENDATIONS ...............................................................................................................29

APPENDIX A - Aircraft Label States and Associated Colours (For Standard Events)..........................29APPENDIX B – Completed Trials Questionnaires.................................................................................31

FRENCH TRANSLATION......................................................................................................................57Green pages:French translation of the foreword, summary, introduction, conclusions and recommendations.Pages vertes:Traduction en langue française de l’avant-propos, du résumé, de l’introduction, des conclusions et

recommandations.


x

LIST OF FIGURESFigure 1 – Examples of Co-ordination Phase Messages ..............................................4Figure 2 – Examples of Transfer Phase Messages ......................................................5Figure 3 – PROVE Platform in Malmö ..........................................................................6Figure 4 – PROVE Platform in Copenhagen.................................................................7Figure 5 – Co-ordination Window (Sector X/B/I) ...........................................................8Figure 6 – Examples of Callsign Menus .......................................................................9Figure 7 – Flight Level Menu ......................................................................................10Figure 8 – Airspace ....................................................................................................11Figure 9 – Simplified Cross Section of Airspace .........................................................12Figure 10 – Traffic Volumes .......................................................................................15Figure 11 – Analysis of Exit Times .............................................................................21Figure 12 – SUMI Results ..........................................................................................23

Table 1 – Examples of ‘Standard’ Levels....................................................................12Table 2 – Traffic Volumes...........................................................................................15

REFERENCES1. http://www.eurocontrol.fr/projects/prove.2. S Russell, PROVE/SYSCO Trial A Report, EEC Report No 353, September 2000.3. Air Traffic Control System Supported Co-ordination, Concept and Functional Description,

Eurocontrol, version 1.1, January 1997.4. DSI Enroute and Approach HMI Specifications, Chapter 10: System Supported Co-ordination,

EEC Task ISS3E1, Version 1.1, April 1999.5. The Eurocontrol Standard Document for On Line Data Interchange, Edition 2.2, September

1998.6. D Marsh, Validation Plan for the PROVE/SYSCO Trial, NATS, August 1999.7. DSI Enroute and Approach HMI Specifications, Chapter 7: ATC Tools, EEC Task ISS3E1,

Version 1.1, February 1999.8. SweDen 98 Real-time Simulation, EEC Report Number 335, February 1999.9. EATCHIP III Evaluation and Demonstration, Phase 2 Experiment 2: SYSCO Level 1, Final

Report, EEC Report 319, September 1997.10. D Marsh et al: A Methodology for the Validation of Air Traffic Tools through Shadow-Mode

Trials: NATS R&D Report 9886, December 1998.


xi

ABBREVIATIONS

Abbreviation De-CodeABI advanced boundary information messageacceptingunit/sector (AS)

the ATC unit/sector that is to take control (or has taken control) of aflight when the transfer from one unit to the next is to (or has taken)place.

ACC area control centreACP accept messageACT activate messageATS air traffic systemCDN co-ordination messageCIW co-ordination in windowCFL cleared flight levelCFMU Central Flow Management UnitCOF change of frequency messageCOP co-ordination pointCOW co-ordination out windowCRD conflict risk displayDATMAS Danish next-generation ATC systemDFL dynamic flight legDSI Denmark-Sweden interfaceEATCHIP European Air Traffic Control Harmonisation and Integration

ProgrammeESCAPE Eurocontrol’s Simulation Capability and Platform for ExperimentationETO expected time over (the COP)FIR flight information regionFL flight levelGL ground levelHMI human-machine interfaceHOP handover proposal messageISA instantaneous self assessment (of workload)LFV Luftfartsverket (Civil Aviation Authority, Sweden)LoA letters of agreementMAESTRO Moyen d’Aide à l’écoulement Sequence du Trafic avec Recherche

d’OptimisationMSL mean sea levelMTCD medium term conflict detectionNM nautical milesOLDI on-line data interchange standardPROVE European ATC Pre-Operational Validation and Experimental Trials

PlatformR/T radio telephonyRAP referred activate proposal messagereceiving unit the ATC unit to which a message is sentREV revision messageRFL requested flight levelRJC reject messageROF request on frequency messageRRV referred revision proposal message


xii

sending unit the ATC unit which is sending a messageSIL sector inbound listSLV Statens Luftfartsvaesen (Civil Aviation Authority, Denmark)SSR secondary surveillance radarSUMI software usability measurement inventorySYSCO system-supported co-ordination conceptsystem the refined DSI together with the new SYSCO serverSystem 2000 Swedish next-generation ATC systemtransferringunit/sector (TS)

the ATC unit/sector responsible for providing a service to a flightbefore the boundary, and which initiates the co-ordination phase withthe next unit

VAW vertical aid window


1

1. INTRODUCTION1.1 GENERAL

The European ATC Pre-Operational Validation and Experimental Trials Platform (PROVE)is a project jointly funded by Eurocontrol and the European Commission (Ref. 1). Theproject is being supported by Denmark and Sweden together with other member states.This report has been produced for the Eurocontrol Experimental Centre, Brétigny, byNational Air Traffic Services Ltd, and reports the results from the second trial (Trial B)conducted simultaneously on the PROVE platforms at Malmö (Sweden) and Copenhagen(Denmark) in April/May 2000. Trial A took place at Malmö in December 1999 (Ref. 2).Engineering lessons from the trial are being reported separately.

1.2 BACKGROUNDThe aim of the PROVE project is to provide a pre-operational validation infrastructure.The first trials to take place on the platform are of the System-Supported Co-ordination(SYSCO) concept (Refs. 3 and 4). Subsequent phases will address, for example,trajectory prediction and medium term conflict detection (MTCD). SYSCO is adevelopment of the on-line data interchange (OLDI) standard (Ref. 5). It defines aprocedure for using an enhanced set of messages (OLDI 2.2) for passing electronicinformation related to the co-ordination and transfer of aircraft between ATC centres.

It is proposed that both Sweden’s System 2000 and the Danish planned system,DATMAS, will be made compliant with some version of SYSCO. The results from the trialwill be used to provide guidance on how SYSCO and the respective systems will need tochange before the combination of both can be made effective and brought into operation.

1.3 TRIAL AIMThe aim of these trials was to validate SYSCO within the Denmark-Sweden Interface(DSI) in the context of real traffic at the Denmark-Sweden boundary. Here ‘validate’ isused to mean a process by which a number of interested groups can increase theirconfidence in the fitness for purpose (i.e. the fitness to control air traffic) of the SYSCOmessage set and of the means of using it that DSI provides.

Detailed objectives of the trial are defined in the validation plan (Ref. 6). In outline, thesewere to:

• assess the usability of SYSCO, that is the extent to which it was effective, intuitiveand satisfying to use;

• assess the impact of SYSCO on:

(i) controller workload;

(ii) safety;

(iii) quality of service provided to airlines.


2

1.4 REPORT STRUCTUREThe structure of this report is as follows:

• Section 2 briefly describes SYSCO;

• Section 3 describes the PROVE platform and the DSI interface;

• Section 4 presents the trial design;

• Section 5 provides a summary and discussion of the results of the trial;

• Section 6 presents a summary of the principal conclusions;

• Section 7 provides recommendations for further development.


3

2. SYSCOSYSCO provides support to electronic co-ordination between flight information regions(FIRs). SYSCO defines three basic phases for a flight; the notification phase, the co-ordination phase and the transfer phase. The related OLDI messages are divided intocategories supporting these phases.

2.1 NOTIFICATION PHASEDuring the notification phase the transferring unit transmits data to update the system atthe accepting unit in preparation for the co-ordination phase. An advanced boundaryinformation (ABI) message provides possible acquisition of missing flight plan data andearly update of basic flight plan data. The ABI is automatically transmitted a presetparameter time before the estimated time at the co-ordination point (COP); for this trial theABI was sent up to 30 minutes before the COP. An entry will appear in the sector load list(Ref. 7) for aircraft for which the receiving sector has received an ABI message. Theentry contains information such as aircraft identification, departure aerodrome, COP,estimated time over (ETO) the COP, and any other flight plan data.

2.2 CO-ORDINATION PHASEDuring the co-ordination phase the transferring unit and the accepting unit agree theconditions under which a flight will pass from the control of one to the other. The co-ordination phase commences following the automatic transmission of an activate (ACT)message for 'standard' conditions (i.e. in accordance with the Letters of Agreement(LoAs)) or a RAP (referred activate proposal) message for non-standard conditions (i.e.not in accordance with the LoAs). However, RAP messages were not implemented duringthis trial. Both messages facilitate the automatic exchange of estimate data, and aretransmitted at a preset time before the COP. The preset time is an adjustable systemparameter; for this trial 7 minutes was always used. The ACT message containsinformation such as aircraft identification, SSR mode and code, ETO COP, and other flightplan data as specified for each individual co-ordination partner. For departures, aPreliminary Activate Message (PAC) would be used in place of an ACT, however PACmessages were not implemented during this trial.

SYSCO provides the controller with the ability to transfer co-ordination information suchas requests for level changes and direct routes (but not heading). If the offering sectorproposes changes to the exit conditions which are 'standard', then a revision (REV)message is sent to the receiving sector, and the aircraft information is updated. If theproposed conditions are 'non-standard', a referred revision message (RRV) is sent whichthe receiving sector is able to accept, reject or counter propose (see Figure 1, case 1).Should the receiving controller accept the proposed co-ordination, an ACP message isgenerated and is sent from the receiving controller to the offering controller. If thereceiving controller rejects or counter proposes the co-ordination, then a reject (RJC) orco-ordination (CDN) message is sent to the offering sector. (Counter proposals did notwork consistently during the trial.)


4

RRV

ACP

transferringsector

receivingsector

RJC CDN

ACP RJC

Case 1Case 2

or

or

time

Figure 1 – Examples of Co-ordination Phase Messages

A CDN message is also generated when the receiving sector proposes changes to theentry conditions (see Figure 1, case 2).

2.3 TRANSFER PHASEThe transfer phase, during which the transfer of control is executed, follows the co-ordination phase. SYSCO provides the ability for the transferring controller to transfer,release, and handover aircraft, and provides the receiving controller with the ability torequest aircraft on frequency. The transfer phase is initiated by a change of frequency(COF) message, a request on frequency (ROF) message, or a handover proposal (HOP)message. The transfer phase can also be system initiated by a Transfer InitiationMessage (TIM) message at a preset time (another system parameter) from the boundary.A TIM message is also generated automatically following a ROF message. (TIMmessages were not implemented in this trial.)

A COF message is sent by the transferring unit to the receiving unit to indicate that theflight has been instructed to contact the receiving controller. The message containsinformation such as release indication, frequency, and if available, cleared flight level,assigned heading and assigned speed. A manual assumption of communicationsmessage (MAS) is sent from the receiving controller to the transferring controller inresponse to a COF to indicate that two way radio contact has been established (seeFigure 2, case 1).


5

HOP

ROF

COF

MAS

transferringsector

receivingsector

Case 1Case 2

Case 3

time

Figure 2 – Examples of Transfer Phase Messages

A ROF message is sent by the receiving unit to the transferring unit to request that thetransferring controller instruct the aircraft to change frequency, perhaps to request theearly transfer of a flight. On receipt of a ROF, the transferring controller will initiate a COF(see Figure 2, case 2).

A HOP message is sent from the transferring controller to the receiving controller topropose a flight for handover if, for example, the aircraft is under speed restriction or onan assigned heading. The proposed conditions are highlighted to the receiving controllerin the aircraft label in ‘co-ordination’ colour. A ROF message is initiated by the receivingcontroller on acceptance of the HOP (see Figure 2, case 3).

Supplementary Data Messages (SDM) are used to transmit control data from thetransferring sector to the accepting sector, where the changes do not need to be agreed.

System messages which are generated in order to indicate receipt of the message at thereceiving sector, such as the Logical Acknowledgement Message (LAM) and Standby(SBY) message, were not implemented during this trial.


6

3. THE TRIAL SYSTEM3.1 INTRODUCTION

The system employed for the trials was a development of the HMI built under theDenmark-Sweden Interface (DSI) project and Eurocontrol’s ESCAPE platform, evolved tointerface with live radar tracks and flight plan sources. Previous trials have beenconducted at Brétigny (Ref. 8) using DSI, and the European Air Traffic ControlHarmonisation and Integration Programme (EATCHIP) interface from which DSI wasdeveloped (Ref. 9). The DSI trial did not restrict itself to the messages in the SYSCOstandard, and used a SYSCO-like co-ordination process in the system.

The PROVE platforms were constructed at the Area Control Centres (ACCs) in Malmö(Erreur! Source du renvoi introuvable.) and Copenhagen (Erreur! Source du renvoiintrouvable.). The trial was conducted in ‘shadow-mode’, that is the prototype was fedlive data from radar, and flightplan information, but was not actually used to control orinfluence live traffic (Ref. 10). Each position had a receive-only feed of the operationalR/T and phones, and phone connections were put in place between the shadow positions.

Figure 3 – PROVE Platform in Malmö


7

Figure 4 – PROVE Platform in Copenhagen

3.2 DENMARK-SWEDEN INTERFACE (DSI)DSI provides a number of tools that enable controllers to interact with the system, either toobtain information (most notably about interactions between aircraft) or to make inputs (ofcontrol instructions, for example). The four principal tools are the co-ordination window,the conflict risk display (CRD), the vertical aid window (VAW) and the dynamic flight leg(DFL). Due to unreliable MTCD information, neither the CRD nor the VAW were usedduring this trial. A brief description of the co-ordination window is given here, along with adescription of the sector inbound lists (SILs), and the callsign and instruction menus.References 7 and 8 provide more detailed information on the DSI tools. The Danish andSwedish versions of DSI differ in a number of ways, however these differences are notrelevant to the assessment of SYSCO.

3.3 AIRCRAFT STATESThe colour of the aircraft label indicates the aircraft state. The four main states areadvanced, assumed, concerned and unconcerned (see appendix A for associated coloursfor the Danish and Swedish interface). When an ACT is received for a flight, and hencethe co-ordination phase commences, the aircraft label is displayed to the receivingcontroller in ‘advanced’ colour and on the transferring sector the ‘next sector’ field isdisplayed in line one of the aircraft label.

When the aircraft is transferred to the receiving sector and once radio contact has beenestablished, the aircraft is assumed by the receiving controller and the aircraft labelchanges to ‘assume’ colour. On the transferring sector, the aircraft label turns to‘concerned’ colours until the aircraft crosses the boundary, at which time it becomes‘unconcerned’.


8

The colour of the callsign changes in relation to that of the label to indicate that an aircrafthas been transferred or released from the offering sector but has still to be assumed bythe receiving sector.

3.4 CO-ORDINATION WINDOWCo-ordination is carried out using two windows; the co-ordination in window (CIW) and theco-ordination out window (COW) (see Figure 5). They are used to support the co-ordination process and provide the controller with the ability to offer, accept, counterpropose or reject a co-ordination proposal, and to acknowledge rejection of a proposedco-ordination.

CO-ORDINATION-IN

DKAPE12:24:48 SAS245 CDN PEL 120 to 140CO-ORDINATION-OUT

DKA12:24:20 KLM6231

290 to 310 ACKREJ RRV XFLDKAPE12:24:36 SAS316 RRV NWP TOBIS to MAXEL REJ

upstream coordinationlevel revision

coordinationproposal rejected by

sector A

downstreamdirect route

request

proposedlevel/route

time messagereceived

offering sector(APP-O)

receiving sector(sector A)

offering sector(APP-O)

callsign

acknowledge/reject

Figure 5 – Co-ordination Window (Sector X/B/I)

These co-ordination messages are generated as a result of inputs made to aircraft labels,lists, or any other supportive tool (such as the VAW), and are also connected to certainoperational events. A co-ordination message is removed once the co-ordination has beenaccepted, rejected, counter proposed or timed out. The controllers had the option todisplay the co-ordination windows only when a message had been sent or received. Inthe co-ordination windows, callsign interactions initiate the same dialogues as those forthe aircraft label or anywhere the callsign is presented.

Co-ordination messages are displayed in the co-ordination windows if the conditions arenon-standard (RRVs) (i.e. not in agreement with the LoAs), but are not displayed to thecontroller for ‘standard’ conditions (REVs). For example, a change to the XFL of anaircraft during the co-ordination phase to a non-standard level generates an RRVmessage in the offering controllers COW, and in the receiving controllers CIW. Thereceiving controller can accept the co-ordination by selecting the proposed level(displayed in ‘co-ordination colour’). This will in turn generate an ACP message, whichupdates the aircraft labels on the displays of both the offering and receiving controllers.The receiving controller can reject the co-ordination proposal by selecting the REJ field inthe CIW, or counter propose (CDN) a level by selecting the flight level menu (see Figure7) from the proposed level field. Should the receiving controller reject the co-ordination, aREJ message will be displayed in the offering controllers CIW, which the offeringcontroller is required to acknowledge via the ACK field.


The proposed levels are also displayed in ‘co-ordination colour’ in the aircraft label, andcan be accepted or counter proposed from there. CDN messages are displayed forupstream co-ordinations with non-standard conditions.

During this trial, if the receiving controller failed to accept, reject or counter propose theco-ordination within 45 seconds, the message would time out and the system would returnto the previous conditions.

3.5 SECTOR INBOUND LISTThe sector inbound lists (SILs) display information concerning aircraft planned to enter asector. The SILs can be arranged geographically and data are presented in whicheverSIL is appropriate to the aircraft’s crossing point into the sector. The advance informationis presented in the relevant SIL, until the aircraft is assumed, following receipt of an ACTor RAP message. The controller can select which optional information fields to display ineach SIL.

3.6 CALLSIGN MENUThe controller uses the callsign menu to advise the system and other sectors of thecontrol responsibility of an aircraft. Only options available for action are displayed,depending on the state of the flight (see 3.3). For an aircraft in the assumed state forwhich the ACT has been sent to the next sector, the options available to the offeringcontroller are to release, transfer or handover the aircraft (see Figure 6(i)). Selecting therelease or transfer options in the callsign menu (or clicking on the next sector field in theaircraft label) will generate a COF message. Selecting the HAND option will generate aHOP message, which is displayed to both the receiving and offering controllers in theSYSCO field (line 4) on the aircraft label.

For an aircraft in the advanced state, which has also been transferred to the receivingsector, the options available to the receiving controller are mark and assume (Figure 6(ii)).The assume function generates a MAS message, and this can also be initiated by clickingon the next sector field in the aircraft label.

A ROF message is initiated via the ROF option (not shown) in the callsign menu, and isavailable to the receiving controller when an aircraft is in advanced state, but has yet to betransferred. A ROF is also automatically generated by acceptance of a HOP message.

REL

CIM128C/S

MARK

HANDTRANS

(i)

Figure 6 – Examples of

menurelease

handover

transfer

aircraftcallsign

9

CIM128C/S

ASSUMEMARK

(ii)

Callsign Menus

type


10

3.7 INSTRUCTION MENUSClearances and other flight data are entered via menus invoked from the aircraft labels,the DSI tools (see 3.2), co-ordination windows or lists. For example, the flight level menu(see Figure 7) is invoked by selecting the cleared flight level (CFL) field in the aircraftlabel. A new cleared flight level can then be entered by selecting a value. During the co-ordination phase, if the offering controller selects a new non-standard level from the XFLmenu (invoked via the XFL field in the label), a co-ordination proposal will be sent to thereceiving sector. Levels in accordance with the LoAs are shown in green.

CIM128

350

CFL�

330310

290280

270

�

260

Figure 7 – Flight Level Menu

scroll buttons

cleared flight level

menu typeaircraftcallsign


4. TRIAL DESIGN4.1 INTRODUCTION

The aim of the trial was to validate SYSCO, within the DSI interface in the context of realtraffic at the Denmark-Sweden boundary. Here ‘validate’ is used to mean a process bywhich a number of interested groups can increase their confidence in the fitness forpurpose (i.e. the fitness to control air traffic) of the SYSCO message set and of the meansof using it that DSI provides.

SYSCO was trialled in shadow-mode as this provides a wide range of real trafficexamples with which to test this fitness for purpose. Within this, a wide range of co-ordinations can be exercised. The participants are also well placed to make judgementsof the difference that SYSCO makes to a task with which they are very familiar.

4.2 AIRSPACE AND TRAFFICThree Swedish sectors and three Danish sectors were shadowed simultaneously (seeFigure 8 and Figure 9).

SecSec

Sector ASector X/B/I

Sector S/M

APP-O

Figure 8 – Airspace

Low-level sectors S and M combined (S/M) and high-level secshadowed in Malmö. Sector S/M handled traffic into and out of CopSturup airports. Sectors 8 and 9 handled overflights as well as tCopenhagen and Malmö Sturup airports to and from sector S/M.

In Copenhagen, approach sector APP-O, sectors X, B and I combinA were shadowed. APP-O handled traffic into and out of CopenhagX/B/I. Sector A handled overflights and traffic into and out of Copenfrom APP-O.

vertical cross sectionas shown in Figure 9

11

tor 9tor 8

tors 8 and 9 wereenhagen and Malmö

raffic into and out of

ed (X/B/I), and sectoren airport, and sectorhagen airport, to and


12

Two manned feed sectors, Copenhagen Feed and Malmö Feed, fed the traffic to themeasured sectors. The primary task of the feed controllers was to maintain the sequenceof aircraft states by assuming aircraft leaving the shadowing sectors. In addition, the feedcontrollers assumed aircraft entering the 'play area' before transferring them to theshadowing sectors. The feed controllers were also able to coordinate both verbally andelectronically with the shadowing sectors, thus providing increased realism.

Sector S/M

Sector 8

Sector 9Sector A

Sector X/B/IAPP-O

FL 245

FL 340

FL 285

FL 195

FL 460FL 460

EKCHGIMRU ELVIXESMS

Figure 9 – Simplified Cross Section of Airspace

Table 1 gives examples of LoAs (i.e. ‘standard’ levels) between the Danish and SwedishFIRs. Note that although intra-centre standing agreements are generally defined assingle levels, multiple levels are defined under the inter-centre LoAs. However, it wasdecided that for trial B there would be no difference in the functionality of SYSCO betweenintra-centre and inter-centre co-ordination.

ATS Route COP Flight Level

(U)N851 MOSIN All FLs

(U)N975 NISLO Odd FLs

(U)M611 MALIV Odd FLs

(U)P605 MALIV Odd FLs

(U)L983 KOSMO Odd FLs

Table 1 – Examples of ‘Standard’ Levels

Being in shadow-mode, the traffic in the trial was determined purely by when the trialexercises took place. The exercises were planned to look at a range of different types ofpeak and off-peak traffic, and took place either during morning sessions (0600-1400 UTC)or afternoon sessions (1200-2000 UTC), weekday or weekend. The off peak times wereplanned so that the shadow controller had time to look for opportunities to improve thequality of service using the SYSCO system.


13

4.3 TRIAL TIMINGTrial B ran for 14 days between the 14th April and 5th May 2000. For planning purposes,the trial was divided into two ‘parts’; part one taking place between 14th and 19th April, andpart two between 26th April and 5th May.

4.4 PARTICIPANTS AND TRAININGEight controllers from Luftfartsverket (LFV), Sweden, and six controllers from StatensLuftfartsvaesen (SLV), Denmark, participated in the trial. Four of the Swedish controllersparticipated in part one of the trial and further four participated in part two of the trial. Forboth parts, two of the Swedish controllers were valid on sectors S and M, and two valid onsectors 8 and 9. In Copenhagen, two controllers participated for the full duration of thetrial, with two sets of ‘on the job training’ (OJT) controllers manning two of the sectors ineach of part one and two. Four of the eight Swedish controllers and two of the Danishcontrollers had previous experience of the DSI interface.

The intent was to spend the first day of each phase of the trial on training. After an initialbriefing on the aims and measurements of the trial, and a brief introduction to DSI, thecontrollers had hands-on training sessions with trials staff on hand to guide them throughthe use of the system. In practice, due to system problems, the training extended into daythree of both phases of the trial.

4.5 METHOD OF OPERATIONSFollowing the recommendations from trial A, a single controller performing the roles ofboth the planner and executive controllers staffed each sector. To evaluate any benefitsthat SYSCO provides over the current operations, two methods of operations for each rolewere devised. In a ‘passive’ role, the shadow controllers monitored and implemented thedecisions of the operational controller using SYSCO and the DSI interface. This was todetermine whether SYSCO could support current operations, and if so, whether workloadwas reduced by the use of the DSI interface and electronic co-ordination. In an ‘active’role, the controllers looked for opportunities to give an improved service to traffic, forexample, direct routings and earlier requested flight levels (RFLs). The objective herewas to determine whether SYSCO gave opportunities for greater flexibility and todetermine if more flexible transfers, that SYSCO allowed through transfer phase electronicco-ordination, would result.

SYSCO is evaluated here within the constraints of the current operations and procedures.However, it should be noted that in order to support the use of SYSCO, operations andprocedures may need to be modified.

4.6 MEASUREMENTS AND ANALYSISSome general details about the measurements and analysis method are given in thissection. The measurements are grouped under the same categories as the objectives forthe trial: usage and usability, workload, safety, and quality of service. The ‘baseline’ forthe trial was the real operational situation developing on the shadowed sector. Thereforeany differences identified were with respect to the current day system.

4.7 USAGE AND USABILITYThe usability assessment involved the systematic collection of information about thespecific ways in which the DSI-SYSCO system was an efficient, intuitive and satisfyingmeans to meet the operational goals. How usable the controllers found SYSCO wasassessed by means of questionnaires, close observation by a controller or analyst of the


14

participants during the exercises, debriefs and the Software Usability MeasurementInventory (SUMI) questionnaires. SUMI is a generic questionnaire that can be applied toassess the usability of any software. The overall usability is composed of five factors thatare combined to give a global usability score. The five factors are efficiency, affect (or‘likeability’), helpfulness, control (reflecting the responsiveness of the system), andlearnability.

During the trial two main debriefs took place; one with the Malmö participants followingpart one of the trial, and a combined debrief on the last day of the trial with participatingcontrollers from Malmö and Copenhagen.

To understand how controllers used the prototype, the usage of the OLDI message set -for example which messages get used, when, by whom and with what content - was alsorecorded. In addition, a number of ‘talk-through’ exercises were planned to address eachof the trial objectives.

In talk-through, a video recording is taken looking over a participant’s shoulder at theirdisplay during an exercise. The participant is asked to ‘think aloud’ during an exerciseabout the tasks that they are conducting: what they are doing, why they are doing it in thatmanner, what (information or part of the interface) are they looking for, etc. An observernotes down the times of comments from the controller of particular interest. These arethen used to pick out interesting periods in the exercise for discussion with the participantin a debrief using the video replay immediately after the exercise.

4.8 WORKLOADIn addition to measuring workload subjectively through debrief and observations, it wasplanned to measure workload through recorded data. Objective measurements includedthe number of telephone calls, time spent planning each aircraft, and time spent intransfer phase planning for each aircraft.

Additionally at the end of each measured exercise, the controllers were to complete a taskload index (TLX) questionnaire. With TLX, the controller is asked to think back over theexercise and rate their workload on a 20 point scale for five separate factors: mentaldemand, physical demand, time pressure, effort expended, and frustration experienced.The questionnaire also included a pairwise comparison between the factors, where thecontrollers would circle the more workload intensive factor from each pair.

4.9 SAFETYThis was addressed subjectively by gathering participants’ opinions in questionnaires anddebriefs.

4.10 QUALITY OF SERVICEResults in this section were to include a summary of how often SYSCO co-ordinationsdiffered from the operational agreements, of how they differed, how achievable thischange was and whether it made much difference to the aircraft.

Quantitative measurements include the number of ‘non-standard’ co-ordinations agreed,for example direct routes and climbing or descending co-ordinations. Some of thesemeasurements will be of interest both in terms of quality of service (i.e. impact on aircraft)and usage (i.e. usefulness to the controller).


5. RESULTS AND DISCUSSIONThis section provides an account of the trial and a summary and discussion of the results.The principal points from the qualitative data are summarised for each objective.Controller comments, questionnaire responses and observations are included throughoutwhenever these are relevant to the result in question, whether supporting these results ornot. Direct quotations from controllers are shown in italics. The questionnaire responsesare presented in full in Appendix B.

5.1 TRIAL ACCOUNTOver the 14 days, 49 exercises (including training exercises) of between 25 minutes andone hour1 duration took place, during both morning and afternoon sessions. Thisamounts to approximately 300 controller hours of exposure to the system. However, thefirst nine days of the trial were dedicated to resolving system and HMI problems, andairspace definitions which resulted in incorrect aircraft states and sector sequences.Nineteen exercises took place during the final five days.

Figure 10 shows the distribution of hourly traffic flows, over all exercises2.

num

ber o

f airc

raft

80

60

40

20

0

The majoritywork-aroundSYSCO areencountered

1 Measured from theminutes after the sys2 As the majority of areport excludes data

median

outlier

maximum(excluding outliers)

aq

uppernd loweruartiles

15

sector

8/9S/MXBIAPP-OA

Figure 10 – Traffic Volumes

of airspace problems were resolved by day 4 of the trial, and a number ofs were established over the course of the trial. Those of most relevance to detailed below, along with an outline of some of the system problems.

time when the majority of tracks (80-90%) had been correlated (up to 30

tem had been initiated).irspace problems were not resolved until day 4, the objective analysis in thisfrom days 1-3.

minimum


16

• Ultimately, the controllers were limited to only those co-ordinations that they couldexercise; namely direct route and flight level co-ordinations which did not change thesector sequence.

• To co-ordinate an aircraft direct to a point already on the flightplan, the controllerswould invoke the waypoint menu from the waypoint field and select the desiredwaypoint. The waypoint menu displays the next five waypoints on the route.However, the controllers reported that because the menu incorporated a number ofminor waypoints, the waypoints they required were often not displayed. Toovercome this problem, the list was extended to incorporate the next 10 waypoints.

• To send an aircraft to a point not on the original flightplan, as is often the case, it wasnecessary for the controllers to first use the elastic vector function to update thetrajectory, thus adding the desired waypoint to the waypoint menu. The controllercould then invoke the waypoint menu to make the co-ordination as before. Thisincreased the controllers workload considerably, and was especially evident forinbounds to Copenhagen. Each inbound was co-ordinated direct to the ‘10 milesfinals’ waypoint, and because the waypoint is specific to the runway in use, these arenot detailed in the flightplan. In addition, to give a route direct to a point off theradar, the controllers had to first zoom out and locate the waypoint, adding moreworkload and frustration.

• The countdown in the label intended to indicate to controllers that the ACT wasabout to be sent to the receiving sector did not work consistently, therefore thecontrollers had no warning of when the co-ordination phase would commence.

• For the first part of the trial, the co-ordination messages were often inconsistentbetween the offering and receiving sectors, especially for upstream co-ordinationsand counter proposals, and hence the content of the messages was unreliable. Thisimproved during the later part of the trial.

• From day 9 of the trial, sectors 8 and 9 were combined to overcome a systemproblem whereby duplicate messages were presented for a single co-ordination.

• During each exercise, a number of aircraft remained ‘uncorrelated’, that is thesystem could not match a flight plan to the radar track, and the controllers could notinteract with the aircraft. (Note that DATMAS for example will support the manualcreation and modification of flight plans, addressing this problem.)

• As the traffic volumes were fairly low on most sectors, the talkthrough debriefs werecarried out during the exercises, rather than following them.

Due to the unrealistic workload imposed on the controllers as a result of the systemproblems and work-arounds detailed above, use of the TLX questionnaire wasdiscontinued. Controller comments from all trial days are included in the results; however,the selection is weighted towards those made towards the end of the trial, since that iswhen most of the debriefs and questionnaires were conducted.

5.2 USAGE AND USABILITYThe following section presents the results under the headings of each of the SYSCOdefined phases. It incorporates an evaluation of the use of the SYSCO message set, theuse of SYSCO within the DSI interface, and of the impact of shadow-mode on such anevaluation. The results of the SUMI questionnaires are also presented.


17

Notification PhaseThe ABI messages were typically sent to the receiving controller when the aircraft hadentered the ‘play area’, which was generally later than 30 minutes before the boundary.At this time, an entry for the aircraft appeared in the sector load list, however thecontrollers reported that they did not generally use this list. Hence there was insufficientawareness of the notification phase and, as a result, it was not possible to draw anyconclusions about the phase from the trial.

Co-ordination PhaseSector sequencing – trajectory predictionThe system prediction of the sector sequence of a flight was based on the filed flight plan.If there were any changes to the route of the aircraft, the system was unable to take intoaccount any resulting changes to the sector sequence.

While the system used trajectory prediction, the planned trajectory used was not able totake account of different climb performances of the same type of aircraft with differentloads and different company procedures. Often occasions arose where the actual climbprofile of an aircraft did not match the predicted climb profile, and as a result the actualsector sequence differed from that expected by the system. On such occasions thecontrollers were unable to electronically co-ordinate with the correct sector.

For example, sector B has an agreement with Maastricht that Copenhagen Kastrupdepartures are cleared to FL330 (i.e. into sector A). Depending on the expected aircraftperformance for that aircraft type, the system would often predict sector A to be the nextsector. However, the predicted climb rate was often over estimated, and the actual climbprofile of the aircraft would not take it out of sector B.

Sector sequencing – flexibilityA number of occasions were also observed where the current LoAs required the controllerto verbally co-ordinate with a sector which an aircraft would pass close to, but not passthrough, in addition to co-ordinating with the next sector. Following on from the examplegiven above, if the actual climb profile of the aircraft was such that the aircraft did notenter sector A as anticipated, current procedures still require the controllers to verballycoordinate with sector A.

The controllers felt that coordinating with a sector where the aircraft may not actually enterthat sector, was something that should be considered for implementation electronically.Such traffic should also be shown as ‘concerned’ to the sector it would pass by. ‘We doneed to have the ability to co-ordinate with more than one [sector]’. ‘There has to be theflexibility to make a sector sequence’.

Similarly, if Copenhagen Kastrup departures were routed to the south, perhaps to avoidconflict with Malmö outbounds to the south west, before turning for SALLO, they might cutthe corner of sector B. On such occasions the system would send the ACT to sector B;however under current procedures the traffic would be transferred direct to sector S/Mfollowing a verbal co-ordination with sector B.

Thus the controllers identified that there were two occasions when they needed tocoordinate with a sector other than that which the aircraft would be transferred to; whenthe aircraft passed close to another sector, or entered a sector only for a short time.


18

‘Fuzzy boundaries’ were considered to be a good idea, whereby if system detects anaircraft approaching a sector which it will be in for less that a preset parameter time orvertical distance, then an 'information only' coordination is affected with that sector. Anynecessary coordination would be carried out with the next logical sector to which theaircraft would be transferred.

Direct route co-ordinations - operational issues

Electronic co-ordination was successfully carried out for direct route co-ordinations to apoint in the next downstream sector where there was no change to the sector sequence.However, overflights are often given a direct route to a waypoint in a sector that is not thenext sector, but a few sectors downstream.

As SYSCO has been specifically developed for coordination over the FIR boundary, it hasnot been designed to support coordination more than one sector upstream ordownstream. Intra-centre coordination with more than one sector up or down streamwould be being supported by an intra-centre coordination system. One of the limitationsof choosing to treat intra-centre and inter-centre coordination the same was that it was notpossible for the controllers to coordinate direct routes more than one sector upstream ordownstream even within a centre. ‘You can only co-ordinate with one sector. Direct co-ordination sometimes involves two sectors’.

However, it became evident during the trial that the controllers would often want to carryout coordination more than one sector upstream or downstream over the FIR boundary.For example a route direct from KOTAM to ALS would involve sector 9, sector A andsector C (sector to the west of sector A). Under current operational procedures the sector9 controller would verbally co-ordinate with sector A, who would verbally co-ordinate withsector C and then advise the sector 9 controller if the proposed route was approved.Using SYSCO, as implemented, co-ordinations concerning more than one sectorupstream or downstream required the use of the telephone, with the system beingupdated for each sector only once the ACT has been received in that sector. In the aboveexample the sector A controller was unable to co-ordinate electronically with sector C untilco-ordination had been completed with sector 9, and the aircraft was assumed insector A.

The controllers felt that this would discourage them from giving direct routings to pointswell outside their own sector, and subsequently would reduce the quality of service whichthey would be able to provide. There was no evidence from the trial to suggest that it isnot sensible to use then same functionality for both inter-centre and intra-centrecoordination. From the controllers point of view it would make sense to have completetransparency.

Direct route co-ordinations – flexibilityAlthough the controllers felt that SYSCO ‘works well’ for basic co-ordinations, asimplemented it was considered to lack the necessary flexibility. Currently the controllerswill often carry out a single verbal co-ordination for all traffic inbound to a point over thenext 45 minutes, for example to send all aircraft inbound to Copenhagen direct to the ‘10miles finals’ waypoint LAMOX. Although the controllers were able to co-ordinate this overthe phone, they still had to carry out the electronic co-ordination for each individual aircraftin order to keep the system up to date. The controllers suggested that ‘it would be good ifyou could select not to send a message in those cases’.


19

Thus an operational need to be able to choose not refer non-standard conditions wasidentified (i.e. to send an ACT or REV in place of a RAP or RRV). This could take place atthe HMI level, where the controllers would coordinate their intentions over the telephoneand then input temporary LoAs so that non-standard conditions are not referred.However, such changes would need to be agreed between both parties, andconsideration should be given as to whether future development of SYSCO shouldsupport this 'meta-coordination' electronically.

Direct route co-ordinations – re-routeA manual means of inputting a new route, or new waypoints not in the flightplanned route,is necessary as greater flexibility is required for inputting waypoints appropriate to STARSfor arrivals, and waypoints outside screen coverage for long direct routes. ‘When usingassigned heading I often had to re-arrange my range to see the routing point requested’.

It is planned that if the desired waypoint is not shown in the menu, selection of the ‘FPL’button at the bottom of the waypoint menu will give access to the whole flightplan. Thecontroller will then be able to enter a re-route via the keyboard.

The controllers suggested that if the system was aware of which runway was currently inuse, then the appropriate waypoint could perhaps be automatically added to the waypointmenu. If more than one runway was in use, as sometimes happens, the phone wouldhave to be used, and then the 'elastic vector' used to define a new waypoint. Only thencould the electronic coordination be completed.

Direct route co-ordinations - reversionOnce a direct co-ordination has been agreed, all the intermediate points are removedfrom the waypoint menus. Controllers gave a number of examples of where they hadmade inputs that they subsequently needed to change but were unable to do so. Forexample, the Copenhagen approach sector co-ordinated a direct route to the finalwaypoint for one runway, but subsequently there was a runway change and it was notthen possible to overwrite the planning. On another occasion, following an emergency, anaircraft had to return to Copenhagen. The waypoint menu only showed downstreamwaypoints and hence the controller was unable to update the system.

Flight level co-ordinations - ‘standard’ conditionsLevel co-ordinations generally generated fewer problems than route changes, howeverconcern was raised amongst the controllers that the implementation of SYSCO within DSIdoes not display coordination messages for 'standard' conditions. For example, nomessages are displayed for co-ordinations at ‘standard’ levels. This was particularly aproblem for inter-centre co-ordinations where many levels are standard (see Table 1). Ifthe offering controller subsequently changed the XFL from one ‘standard’ level for whichan ACT had been sent to a different ‘standard’ level, then the receiving controller wouldnot be notified of this change. Neither would a message be displayed if an aircraft hadpreviously been co-ordinated at a ‘non-standard’ level that was then subsequentlychanged to a ‘standard’ level. The offering controller was also not able to ‘refer’ therevision. This was less of an issue for intra-centre co-ordinations where generally only asingle level is considered to be standard. The controllers were all in agreement that amessage should be displayed if the proposed level is different to the expected level,whether the conditions were standard or non-standard.


20

Timeouts

As mentioned in section 3.4, if a co-ordination was not accepted, rejected or counterproposed within 45 seconds of receipt of the message, the message ‘timed out’ and wascleared from the co-ordination in window.

The controllers felt that if there was sufficient indication that a message was awaiting aresponse, then a time out facility would not be necessary. As implemented, the controllersfelt that there was insufficient indication that a co-ordination had timed out. ‘What I wouldlike is an indication at the sending sector that you tried to co-ordinate something but ittimed out…you can be in doubt about what happened at the other side’. ‘I need to have aconfirmation from the system, whether a co-ordination is noticed or not’. However, thisissue was compounded by system problems where co-ordinations had been accepted butwere not reflected in the offering controller’s label, and subsequently the offering controllerthought that the message had timed out. It should be noted that as the controllers werenot responsible for executive decisions during the trial, they were able to respond quicklyto new co-ordination messages. Hence, operationally it could take longer to respond to aco-ordination message and a time out parameter of 45 seconds may not be appropriate.Further, the controllers suggested that an indication of an imminent time out may be ofuse, and also that a ‘repeat message’ facility would be of use for the sender.

The controllers suggested that in order to let the offering sector know that an offer wasbeing considered, perhaps to allow them time to co-ordinate further downstream or tocomplete other tasks, a ‘standby’ facility would be of use, which would perhaps also delaya time out.

ACT parametersAn ACT message is transmitted when an aircraft is a system parameter time (7 minutes)before the sector boundary. During the trial it was noticed that when an ACT was sentbefore the aircraft had been assumed by the transferring sector, a number of problemsoccurred. In particular, as a consequence of the HMI design the system did not knowwhether to send co-ordination messages to the upstream or the downstream sector. As awork around, the system was programmed to only send an ACT after the aircraft hadbeen assumed. As a result of this the ACT was ‘delayed’ for 66%3 of flights.

The result of this was that the ACT was often sent very late, too late for co-ordination tobe effective, ‘I tried to force the ACT but I don’t think it worked every time’. To illustratethis, Figure 11 shows the distribution of the time to the boundary that both standard ACTs,and delayed ACTs (i.e. those only sent once the aircraft had been assumed in theprevious sector) were sent, for the measured sectors only.

3This figure does not include ACTs sent before the start of the exercise or whose correspondingsector exit time was after the end of the exercise.


21

>10

8 to

10

6 to

8

4 to

6

2 to

4

0 to

2

-2 to

0

-4 to

-2

-6 to

-4

-8 to

-6

-10

to -8

-12

to -1

0

-14

to -1

2

-140

50

100

150

200

250

300

350

time to boundary (mins)

frequ

ency

0%

10%

20%30%

40%

50%

60%

70%80%

90%

100%

cum

ulat

ive

frequ

ency

delayed ACTACTdelayed ACT, cumulativeACT, cumulative

after boundary (negative)

before boundary (positive)

boun

dary

Figure 11 – Analysis of Exit Times

As expected there is a peak in the number of ACTs sent at 6 to 8 minutes before theboundary. Some of the observed variation on this may be attributed to aircraft that arriveat the boundary later or earlier than predicted by the TP (perhaps because the aircraft hadbeen delayed or re-routed). This also meant that a number of the ‘delayed’ ACTs weresent at a suitable time; however 40% of the ‘delayed’ ACTs were actually sent after theaircraft had entered the receiving sector (less than 0 minutes to boundary). A largeproportion of these were departures from Copenhagen (Kastrup) and Malmö (Sturup)airports entering APP-O or sector S/M where a PAC message would normally be sent.

The controllers noted that there were occasions where the downstream sector requiredthe co-ordination phase to begin early, for example for inbounds to Malmö from the north.They suggested that it would be useful to have a facility for the downstream sector torequest the ACT. ‘Sometimes you must wait for a label to get advanced to be able to co-ordinate via SYSCO, perhaps you would like to co-ordinate earlier’. However this mayhave been as a result of the problems noted above, and may be solved throughrefinement of ACT parameters for each COP. ‘It is essential that the parameter for theACT-sending is appropriate’.

HMIThe controllers reported on a number of occasions that they had accidentally made thewrong selection from the menus when making a co-ordination proposal. ‘It is easy to clickthe mouse twice by accident and thereby request a wrong point’. On such occasions, anincorrect co-ordination proposal would still be sent to the receiving sector. ‘If mistake isdone on input, the co-ordination is done at once. So, to correct it you would need to do anew co-ordination’. Although it was felt that the errors could reduce with training, it isunlikely that they could be eliminated.

As the controllers were focused on the traffic, co-ordination proposals were generallynoticed in the aircraft label before the co-ordination in window. However, most controllersresponded to proposals via the co-ordination window, as overlapping aircraft labels wereoften a problem.

The controllers reported that they did not systematically use the SILs, but the approachsectors used the SILs as a guide to their expected traffic load.


22

Division of Controller ResponsibilitiesThe participants felt that they had insufficient exposure to a working system to determinethe method of operations that should be adopted in terms of planning and executive roles.‘Too short time to have an opinion. It feels like a planner is needed’. ‘Depending on theamount of traffic, low traffic, you [executive] can do the planner role. More traffic, you mayneed a planner to assist you, the problem is to divide the duties’.

The controllers felt that many of the current approach tasks would become significantlymore difficult with SYSCO, ‘the system is better for en-route’. Several heading, altitudeand speed changes may be passed to each aircraft in a short time and even with currenttraffic levels, it would be very difficult to electronically input each instruction individually tokeep the system up to date. ‘High level traffic is easy to work. Low level you have somany inputs to do, so you easily get behind’. Currently when busy, instructions arewritten on the strips in a form of shorthand that is recognised by all controllers.

Transfer PhaseThe controllers were unable to fully explore the HAND function (see section 2.3), as dueto system problems the assigned heading and assigned speed fields were only displayedto the controller once the aircraft had been assumed, rather than being displayed in thelabel as part of the handover proposal. However, as the current version of OLDI does notsupport the co-ordination of aircraft on headings, the controllers were using the HANDfunction to refer the proposal to the receiving sector, well before wanting to transfercommunication. The associated ROF indication in the SYSCO field of the label becamedistracting to both controllers. From this, it was the controllers’ opinion that it wasfundamental that OLDI be modified to provide support for co-ordination on a heading, andto a lesser extent co-ordinate speed restrictions. It was also felt that, under the intendeduse of the handover function, an acceptance of a handover proposal should notautomatically generate a request on frequency message (see Figure 2). However, thecontrollers reported that the ROF function was useful for requesting the aircraft onfrequency early and for reminding the upstream controller to transfer the aircraft.

There were inconsistencies in the display of aircraft states in the transfer phase. Aircraftwere displayed to the offering controller as being ‘unconcerned’ whilst still in the offeringsector, and others were shown as ‘concerned’ even when they were well clear of theoffering sector boundary. This problem was not consistent for all aircraft on the sameroute, and was perhaps due to errors in the trajectory prediction.

The controllers found it useful to be able to distinguish between a release and a transfer,although it was not used often during the trial. The controllers suggested that ‘it may beuseful to input REL after TRANS so manoeuvres can start before the agreed LoA releaseline’.

Software Usability Measurement Inventory (SUMI)The Software Usability Measurement Inventory (SUMI) measures how usable a system(SYSCO and the means of using it that DSI provides) is, according to the perceptions andattitudes of the users. Eight controllers each completed one SUMI questionnaire. The‘global’ score gives an overall view of the controllers’ perception of the usability of thesystem. Given the low number of controllers who completed the questionnaires, it is alsothe most reliable of the six SUMI indicators. The scales are arranged so that a globalscore 50 is standard for commercial software. Software rated below this requires furtherdevelopment and the subscales point to where this development might best be focused.


23

Efficiency Affect Helpfulness Control LearnabilityGlobal0

10

20

30

40

50

60

70S

UM

I sco

res

Figure 12 – SUMI Results

From Figure 12 we can see that DSI/SYSCO achieved a global score of 48 which isrelatively high for prototype software. The lowest scores are in Efficiency and Control.When both of these scores are low it indicates that the basic functionality is poor in somerespects. The controllers commented that ‘sector sequence or rather change of, is notpossible’ and that ‘if you need to co-ordinate something through more than one sector it isimpossible’.

A low score in control can also mean that system feedback is poor. When discussing co-ordination messages, one controller commented that ‘sometimes you wouldn’t know if itwas a time out or a rejection’. There was also some indication that managing tasks wasnot as well supported with SYSCO as with paper strips. ‘It is impossible to take a glanceover the screen to see if everything is updated’, ‘I could imagine it would be difficult to givea clearance to a departure out of a small airport. You would have to remember whichlevels you had given, or check each label’.

The highest SUMI score was in Learnability. This reflects the controllers’ view that thesystem was intuitive to learn, ‘it shouldn’t be too hard to learn since it’s based on windowslike an ordinary PC’.

There were some differences in the responses between the Danish and Swedishcontrollers, with the Danish controllers scoring slightly lower. This may in part beattributed to the problems associated with direct route co-ordinations, which were moreprevalent within the Danish airspace.

5.3 WORKLOADThe workload of the controllers was increased in part by the work-arounds introduced(see 5.1). In addition, the controllers were not supported by MTCD during this trial, whichseverely affected the controllers’ ability to carry out conflict detection. Hence workloadmeasures taken during the trial are not reported. This section presents subjective viewson the impact of shadow-mode trials and the effect of SYSCO on controller workload.


24

By not having responsibility for separation and control decisions the controllers foundworking in shadow-mode very artificial and, as such, the controllers were unable to fullyevaluate the impact that SYSCO might have on controller workload. ‘It is very difficult toknow if you really reduce the workload of the controller when you’re not responsible’.‘Since we did not speak with the aircraft we could use all time for ‘clicking’’. With theabsence of MTCD in a stripless environment, the controllers were not able to fulfil any ofthe planning or conflict detection tasks.

The controllers felt that in shadowing the R/T, additional workload was incurred by firsthaving to locate the aircraft concerned before being able to input the tactical instruction.‘When you are working with the traffic yourself before you give the instructions to theaircraft you might actually choose the label in order to make the input, do it at the sametime. Here we can only do it when we’ve actually heard the controllers saying it’. Hadthey been controlling the traffic, they would have been thinking ahead of their actions andwould have selected the aircraft label, and be poised to put in the tactical instructions priorto the transmission.

Controllers generally felt that the use of SYSCO showed the potential to decreasecontroller workload when compared with using current methods of operation, particularlyfor overflights, by virtue of reducing the number of telephone calls required for co-ordination. However, ‘I am still not convinced that low level sectors, where you have to doa lot of inputs, gains you any capacity from SYSCO’.

The controller was able to continue with other tasks while the receiving controllerconsidered the co-ordination, rather than waiting for a phone call to be answered and thenfor the co-ordination considered. ‘What reduced the workload is that you could send themessage…you did not have to wait on the phone until someone answered’.

However the controllers felt that during busy periods the cumulative effect of co-ordinatingaircraft on direct routes could result in an increase in workload, and it was not possible toevaluate the extent to which MTCD would support the controller under suchcircumstances.

The unanimous view from the controllers was that in order to measure the workloadbenefits provided by SYSCO, a simulation, building up to predicted traffic loadings, wouldbe more appropriate. ‘If you’re just talking about evaluating the technical inputs andoutputs to see if they are wrong or right the this is the perfect thing to do of course, but ifyou evaluate the situation as the whole situation then its another issue’.

5.4 SAFETYThere was little opportunity during the trial to explore safety issues. However, thecontrollers did feel that one of the advantages with SYSCO was that there was noambiguity as to what had been co-ordinated, and misheard co-ordinations wereeliminated, for example mistaking 290 for 390. ‘It‘s inherently safer because you shouldnot be in doubt about what you have co-ordinated’. Similarly, due to the reduction in thenumber of telephone transmissions, there would be less chance of missing transmissionsthrough simultaneous telephone and radio traffic. The controllers also reported that withSYSCO as they are less likely to be distracted by telephone calls they can maintain theirfocus on the radar screen. ‘This system allows you to focus on the radar screen andthat’s really the big advantage’.


25

5.5 QUALITY OF SERVICEDue to the limitations of the trial system, it was not possible for the controllers to fullyexplore the effect of SYSCO on quality of service, therefore no analysis of the objectivedata was carried out. Quality of service was only briefly addressed during the debriefs.

The controllers felt that unless the system could support changes to the sector sequenceand co-ordinations to a sector further up or downstream then, in terms of routings, thequality of service which could be provided to the airlines would be reduced.

It was also thought that any gain in capacity, and corresponding increase in the number ofaircraft, would ultimately reduce the quality of service as direct routings would need to belimited.

5.6 SHADOW MODE TRIALSA number of lessons were learnt about how and when shadow mode trials should beconducted. New systems and concepts may be designed in such a way that changes willbe required to the current procedures and airspace. Shadow-mode trials are limited tocurrent procedures and airspace. In addition, it is not possible to expose controllers totraffic volumes above those of current day.

As discussed in section 5.3, the controllers felt that, regardless of the workarounds inplace, their workload was not realistic under shadow mode as they were not responsiblefor separation and control decisions. Any evaluation of workload should be addressedthrough real-time simulation, and the issue of being 'one step behind' the operationalcontroller and the traffic could have other effects on the evaluation, and so must beconsidered carefully.

However, shadow mode trials allows the designers to put the concepts into the context ofreal traffic, airspace and local procedures, thus giving them a better understanding ofoperational issues. In addition, by conducting the trials alongside the operationalsystems, it exposes the concept to a wider audience of validated controllers, thushopefully encouraging ownership of the system.

The range of issues raised elsewhere in this section point to the value that shadow-modetrials can bring to pre-operational validation.


26



27

6. CONCLUSIONSSYSCO trial B gave controllers in the two ACCs around 300 hours of experience of usingSYSCO. As a result the general attitude of the controllers towards the use of SYSCO inthe future was optimistic. ‘It is very fast with standard co-ordination and you are hardlydistracted’. ‘There are advantages, but the system is not very flexible’. The potentialvalue of the trial was reduced by the incorrect, or missing functionality of some features, inparticular the MTCD, and the resulting need to use a number of work-arounds. However,a number of requirements for improvements to the system were identified:

• The system was not designed to take into account any changes to the sectorsequence, but there is a clear operational need to have the flexibility to override thesystem’s prediction of the next sector.

• Within the current procedures, co-ordination may involve more than one sectorfurther upstream/downstream than the next, particularly for direct routes. Undersuch circumstances it is necessary to be able to co-ordinate both upstream anddownstream simultaneously. SYSCO as implemented did not support this, andduring the trial the controllers were prevented from co-ordinating downstream prior toassuming the aircraft.

• Concern was raised amongst the controllers that, within DSI, co-ordinationmessages were not displayed for ‘standard’ conditions (see section 3.4). Thecontrollers were all in agreement that a message should be displayed for bothstandard and non-standard conditions if different to that expected by the controller.

• There was insufficient indication given to controllers that a message had timed out.The controllers suggested that, if it is necessary for co-ordination messages to timeout, there might be a count-down warning to allow them to re-prioritise tasks, and a'standby' facility to allow them time to consider the proposal further or complete othertasks.

Working in shadow mode, controllers felt constrained by not having responsibility for thetraffic, and having to react to situations rather than being in control and able to planahead. The controllers were therefore unable to fully evaluate the impact of SYSCO oncontroller workload. The controllers suggested that SYSCO has the potential to reduceworkload through the reduction in telephone calls. However, it was felt that while SYSCOcould perhaps reduce controller workload within an en-route environment, furtherconsideration is required to evaluate its appropriateness to approach control.

A single controller was able to fulfil the roles of both executive and planner. However, dueto the artificial workload and the lack of MTCD information, it was not possible to fullyevaluate this, and further investigation is required.

Although one of the advantages with SYSCO was that there was no ambiguity as to whathad been co-ordinated, this was at the expense of the increasing the likelihood ofaccidentally sending an incorrect message.

The PROVE/SYSCO trials were, in the end, insufficient to provide a thorough validation ofSYSCO, mostly for engineering reasons. However, there were three areas in which thetrials provided significant achievements:


28

• They deepened the understanding of how and when to conduct shadow-mode trials.

• They provided the operational climax to an intense engineering effort, movingsoftware out of the experimental centre into the operations room.

• They provided evidence that, while SYSCO has advantages, the price of these, asimplemented, is a considerable reduction in flexibility.


29

7. RECOMMENDATIONSFurther development and validation of SYSCO within the context of the DSIimplementation needs to:

• improve the trajectory prediction and medium term conflict detection;

• implement a means by which the controller has the flexibility to override the system’sprediction of the next sector;

• consider the extent to which SYSCO needs to support current procedures tocoordinate with another sector in addition to the next, or modify procedures toaccommodate SYSCO;

• implement a means whereby the controller can effect inter-centre coordination withmore than one sector up or downstream;

• provide controllers with the ability to co-ordinate aircraft on a heading and underspeed restrictions;

• consider displaying coordination messages in DSI for changes to standardconditions;

• include an indication to controllers that a message is about to time out, or removethe time out;

• consider incorporating a 'standby' facility, to allow controllers time to further considerthe proposal or complete other tasks before responding;

• improve the indication of when the ACT will go the next sector, and refining the ACTparameter to suit different routes;

• use real-time simulation to evaluate the impact of SYSCO on controller workload;

• consider how the use of SYSCO could be tailored for use in an approachenvironment;

• explore the method of operations and the roles of a planner and executive controller.

In addition, it is recommended that future shadow-mode trials should:

• take the form of a series of short trials, with the aim of providing an operational ATCfocus to what is mostly an engineering process;

• be used to increase awareness of developments and to encourage subjectivecomment;

• not be used to take quantitative workload measurements.


30



31

APPENDIX A: AIRCRAFT LABEL STATES AND ASSOCIATED COLOURS

A.1 SWEDEN

Sector N STATUS Sector N-1

Sector N

SectorN+1

TIMING

ABI[UNCONCERNED]

Black Grey Grey ABI sent 30 mins. beforesector N boundary.

ACT[ADVANCED]

Black Blue Grey ACT sent 7 mins. beforesector N boundary.

Sector N-1TRANSFER/RELEASE

BrownBlack

BlackBlue

Grey After ACT, beforesector N boundary.

[ASSUMED] Brown Black Grey

Sector N+1ABI

Grey Black Grey Exits sector N-1.ABI sent 30 mins. beforesector N+1 boundary.

Sector N+1ACT

Black Blue ACT sent 7 mins. beforesector N+1 boundary.

TRANSFER/RELEASE

BrownBlack

BlackBlue

Sector N+1ASSUMES[CONCERNED]

Brown Black

[UNCONCERNED] Grey Black Exits sector N.

KEY:

Blue Indicates the colour of the text in the aircraft label.

BlackBlue

First line indicates the colour of the callsign field (if different), and second line indicatescolour of other fields.


32

A.2 DENMARK

Sector N STATUS Sector N-1

Sector N

SectorN+1

TIMING

ABI[UNCONCERNED]

White Grey Grey ABI sent 30 mins. beforesector N boundary.

ACT[ADVANCED]

White Pink Grey ACT sent 7 mins. beforesector N boundary.

Sector N-1TRANSFER/RELEASE

MustardWhite

WhitePink

Grey After ACT, beforesector N boundary.

[ASSUMED] Mustard White Grey

Sector N+1ABI

Grey White Grey Exits sector N-1.ABI sent 30 mins. beforesector N+1 boundary.

Sector N+1ACT

White Pink ACT sent 7 mins. beforesector N+1 boundary.

TRANSFER/RELEASE

MustardWhite

WhitePink

Sector N+1ASSUMES[CONCERNED]

Mustard White

[UNCONCERNED] Grey White Exits sector N.

KEY:

Pink Indicates the colour of the text in the aircraft label.

WhitePink

First line indicates the colour of the callsign field (if different), and second line indicatescolour of other fields.


33

APPENDIX B: COMPLETED TRIALS QUESTIONNAIRES

B.1 INTRODUCTION

The following appendix contains the controllers' responses to the questionnaires.Twelve controllers completed the questionnaires, however there were occasions wherea response was not given. The tick boxes were numbered from 1 to 5 in thequestionnaires completed by the controllers, but are not included here to improveclarity.


34

1. OPERATIONSStrongly

AgreeStronglyDisagree

2 81.1 As an Executive controller, with SYSCO Iam able to fulfil the planner role alongsidemy own.

Please comment.• N/A for APP (could be useful when working as APPCO).

• No planner in APP.

• Depending on the amount of traffic. Low traffic – you can do the planner role. Moretraffic – you may need a planner to assist you, the problem is to divide the duties.

• I think it’s a bit harder to find conflicts early than with today’s system. On the other hand“ the green line” you get when you click in the label is a very good help.

• Too short time to have an opinion. It feels like a planner is needed.

• I have little experience as a Planner.

• Of course other equipment is needed to be able to do the job. Don’t understand theexact intention of this question.

• Yes, up to a certain point where the traffic load demands a Planner controller as well.

• Long term planning is more difficult without paper strips.

• Not enough traffic!


35

StronglyAgree

StronglyDisagree

1 6 1 21.2 SYSCO forces me to change the way I thinkabout co-ordination.

Please comment.• In some cases we lose the flexibility.

• It has some limitations: 1. You can only co-ordinate with one sector. Direct co-ordination sometimes involves 2 sectors.

• It seems less flexible than expected.

• Sometimes you must wait for a label to change to blue to be able to co-ordinate viaSYSCO, perhaps you would have liked to co-ordinate earlier.

• Action taken will be different but the way I think, probably not.

• In some ways it is a different way of co-ordination, but not really any fundamentallydifferent way of thinking.

• I need to adapt to the response time of the system and I also need to have aconfirmation from the system, whether a co-ordination is noticed or not. I can’t do a co-ordination whenever I want – it’s only possible within the limits of the system.

• I have to consider what ‘phase’ the aircraft is in before I make the co-ordination. Adisadvantage is that it’s not possible to ‘back co-ordinate’ e.g. co-ordination with theprevious sector.

• Today, my co-ordinations are often based on a mutual understanding of the trafficsituation.


36

StronglyAgree

StronglyDisagree

2 4 61.3 With SYSCO, pilots will notice a betterservice.

Please comment.• In most cases it will be a question of getting a direct track 2 –3 minutes before. You can

easily get behind in updating labels etc., so you lose the flow of the traffic, your handlingget “stiff”.

• High level traffic is easy to work. With low level traffic you have so many inputs to do,so you easily get behind.

• Not for APP traffic. (Too many inputs in too short a time) - see also 1.6.

• The system should provide better opportunities for direct tracks etc.

• Minor changes.

• Possibly the pilots will get direct routes earlier than today, because you don’t have towait until the aircraft is on your frequency to issue it, and if you see that the previoussector is busy you often don’t disturb them with a direct route if you don’t think it helpsthe previous sector in some way.

• Thinking of today and trying to compare, I don’t think so.

• Hard to say, but probably not. There are both advantages and disadvantages.

• I don’t think they notice a difference at all.


37

StronglyAgree

StronglyDisagree

4 6 11.4 I prefer speaking to a person on the phonerather than co-ordinating electronically.

Please comment.• On the phone you work out a solution which is acceptable for both parties. With

electronic co-ordination it is all or nothing.

• Yes and no. It is easy to do standard co-ordinations, but sometimes you need morespecific co-ordination, which is better to get on the phone.

• Talking to a person could make counterproposals faster.

• I think the co-ordination is important … not how it is done.

• Normally not, but when speaking to a controller, you can better understand when it’surgent, emergency etc.

• Sometimes it’s much easier to not speak to a person, especially when it is to othercountries.

• Misunderstanding might decrease, I can also answer when I decide to.

• Electronic co-ordination could take place when one controller is unoccupied. Speakingon phone needs both.

• The most common co-ordination is probably more efficient with SYSCO than with verbalco-ordinations (level revisions, etc.).

• Being so familiar with today’s system, it’s only natural to have a ‘human’ response. Themore I get used to the new way of working, my acceptance will improve.

• Not generally, but there are co-ordinations that are not possible to do electronically.

• Today, you co-ordinate when it’s mostly a complex situation. I feel there is a need totalk to the other controller to be sure he/she gets the situation right. Other things (likedirect routing) are covered by standard agreements between sectors.


38

StronglyAgree

StronglyDisagree

5 5 1 11.5 Using SYSCO is as safe as the current co-ordination process.

Please comment.• When the label is correctly updated there are no misunderstandings. You could think of

cases where you are “forced” to accept something you normally wouldn’t (direct tracks,levels) and create a conflict for yourself.

• The system update gives a clear picture of what has been co-ordinated.

• Providing the systems works correctly it is safe (could be safer than phone co-ord.), butit is also less flexible.

• Less chance of misunderstanding due to language etc.

• There is a risk to force next sector to accept too many proposals that will createproblems when the sector is busy, when it is so easy to co-ordinate.

• Difficult to say - but I think it’s safe. I don’t think it’s less safe. Today you can “hearwrong” and you can forget a revision, but with SYSCO you get reminded if current andexit level are not the same.

• Not so many misunderstandings.

• Better indication when the request/revision is not accepted (when co-ordination is notresponded/timed out) is needed.

• Should be, once I am experienced enough. It must be.

• I don’t know, but I have a fear of ‘clicking’ on the e.g., wrong level, point and not noticingit.


39

StronglyAgree

StronglyDisagree

5 4 2 11.6 Using SYSCO, I could not start co-ordination soon enough.

Please comment.• When using direct tracks to solve conflicts, you need to turn the aircraft early. As it was,

here you couldn’t ask for direct. Before, it was close to the boundary, maybe even whenyou normally would ask the aircraft to change frequency.

• If the sector you are handling is very small, it is a problem.

• As APP controller you would like to be able to co-ordinate earlier both for departuresand arrivals.

• You had to wait for the aircraft to be in the right “state” which was often too late.

• Today we sometimes make co-ordination with more than next sector. This means thenext sector has to co-ordinate with the next one.

• Too late sometimes (to a previous controller). To the next controller there’s no problemas you can make a forced ACT.

• It was at some times impossible to force the act when co-ordination was to be sent.

• Sometimes I want to do co-ordination affecting more than one sector downstream.Sometimes I want to start co-ordinations when the traffic enters my sector.

• In some cases this might be true. To be able to do the co-ordination, you have to forcethe ACT in order to co-ordinate, which probably would be an extra workload to theplanner/controller.

• I need to know earlier than today (this exercise), the amount of inbound traffic.

• Sometimes I need to make a co-ordination early to be able to plan the traffic long beforeit is in my sector.

StronglyAgree

StronglyDisagree

2 2 2 11.7 Using SYSCO, I sometimes revised co-ordinations which I should have leftunchanged.

Please comment.• I would like to have an “up” button.

• In the APP set up, we tested so many different co-ordinations in real life, you wouldn’tdo that, so it’s a bit hard to say.

• Might be because of the fact I have not worked with the system for a long enough time.

• No opinion.


40

StronglyAgree

StronglyDisagree

4 2 2 2 11.8 As a combined Executive and Planner, Ifound that I never confused the co-ordination phase and the transfer phase4.

Please comment.• It takes a little time to get used to the new features, but once you have tried it you

sometimes rush a little and get confused.

• Normally you would not work as both in APP. Possible confusion could be due to lackof knowledge of DSI which would not be a problem when used to the system.

• Mainly due to wrong aircraft “states” and late change of phase - probably wouldn’t be aproblem if the system “worked”.

• It was very clear. You can see in the label when the ACT is sent and to which sector.

• In the beginning I did.

• Only worked as combined in this test.

• What’s the difference, the both named phases never change.

• No problem.

4 As there were no TIM messages, this question is not so relevant.


41

1.9 How useful is it to be able to co-ordinate electronically:

Veryuseful

Not at alluseful

a) direct routings within the FIR 7 3 1 1b) on headings within the FIR 5 5 1 1c) speed within the FIR 6 3 1 2d) direct routings between the FIRs 7 4 1e) on headings between the FIRs 5 4 1 1f) speed between the FIRs 5 4 1 1 1

Please comment.• Standard co-ordination – it is a perfect tool but if you need to co-ordinate something out

of the ordinary – or the receiving sector have problems with that solution, you might aswell pick up the phone and work it out together.

• It is essential that the parameter for the ACT-sending is appropriate.

• Basically a good idea, but what if next sector dislikes your idea. A common solution byphone is more flexible, I think.

• Heading and speed – co-ordinations are not highlighted (why?) in SYSCO!

• The speed is not so useful because we have an agreement with most surrounding FIRsthat we can ask the pilot to report a speed restriction to the next sector whenestablishing contact.

• The speed and heading function did not highlight to next sector, though informationfollowed the label with the transfer.

• No differences outside/inside FIR.

• We have every option today on letters of agreement to use direct routes.

• The ‘bugs’ in the system caused problems, but otherwise it looks promising.

• Most direct routings are covered by agreements.


42

StronglyAgree

StronglyDisagree

4 2 41.10 There are benefits to using SYSCOcompared with the current method ofoperations.

Please comment.• It is very fast with standard co-ordination and you are hardly distracted.

• Standard co-ordination is faster using SYSCO. Using the phone could givemisunderstandings.

• There can be no doubt as to what has been co-ordinated.

• It would save you the telephone co-ordination - but again very different to say, due tothe present problems with the system.

• Much faster than an ordinary co-ordination and less disturbing to get. You can still co-ordinate if the communication system is out of service.

• There is a benefit in that you answer a co-ordination when you have the time to do so.There have been many bugs in the system though.

• Less phone co-ordinations. Updated flight information sector to sector. No flight strips.

• There are advantages! But the system is not very flexible.

• Once you’re trained enough, yes.

• Faster co-ordinations, less interruption in the controller’s main job e.g. controllingaircraft.

• I can see benefits when working in the new system, when you have to make inputs oneverything.


43

StronglyAgree

StronglyDisagree

2 4 41.11 I can always co-ordinate with the person Ineed to.

Please comment.• No, if you need to co-ordinate something through more than one sector it is impossible.

• No!!! Direct routes.

• Sector sequence or rather change of, is not possible. Also a problem if you need to co-ordinate with more than one sector.

• You can only co-ordinate with one sector . That is not always enough.

• It’s hard to see the effect of the fact that you can only co-ordinate with the sector next toyour own. Rules have to be written taking care of this fact.

• Co-ordinations with more than two sectors involved are impossible.

• There is a need to be able to co-ordinate back to previous sector even after traffic hasbeen transferred.

• Again, see 1.2. Co-ordinations in the ‘far future’ have to be made the ‘old way’.

• I’m not able to make a co-ordination go to a specific sector.

StronglyAgree

StronglyDisagree

2 5 41.12 With SYSCO, I felt I was helping themachine do its job, rather than it helpingme.

Please comment.• When the traffic was high you got behind in updating and didn’t have time to check

whether the co-ordination went through or timed out.

• Maybe due to many problems with the platform.

• Again due to the “bugs”.

• Sometimes that was a fact because of all the bugs. It can also be that I am not toogood at using the system, due to little practice.

• I have to feed the machine, but the machine helps me with a lot of things.

• SYSCO would be a useful tool, but it needs to be user flexible and improved features.(For example, it’s only possible to input a direct route from present position and not e.g.ROE – VES. And it is not possible to co-ordinate different routings).

• It takes time to get adapted to the way the machine works.


44

StronglyAgree

StronglyDisagree

2 5 2 11.13 I needed to know more about whichSYSCO messages were being sent andwhat data was in them.

Please comment.• It is important that you get the correct understanding of SYSCO before training. You

can easily get confused.

• If I could have worked in a well functioning system before the trial test, I might havebeen better in my performance.

• I don’t think it’s useful.

• The more I knew from the start, the better.

• Incoming messages were easily missed because I simply didn’t watch the windowenough. Maybe an audio signal or a flashing message is needed to become aware ofan incoming message.

• I feel there was a need for a brief instruction of how the system works and how tooperate it. I would also have needed a list of all the ‘shortfall’, e.g. the text in the label,sector inbound list etc.

StronglyAgree

StronglyDisagree

8 2 1 11.14 I was always aware of which aircraft wereunder my control.

Please comment.• With the different colours we have chosen, there was no question about which aircraft

was controlled by whom.

• The flight states displayed by colour – coding are easy to understand and give you ageneral view of the traffic situation, but sometimes we had aircraft in incorrect states(white/grey). If the system had been more stable (correct), I would always have knownwhich aircraft were under my control.

• Too many problems with the system.

• A hard one as there were no aircraft under my control. Due to some bugs in thesystem, some labels stayed grey even when it was in my sector and some didn’tcorrelate at all. But I guess that would be fixed if it were real life.

• The system was not aware of this all the time.

• No problem.

• The colour coding was easily understood. The aircraft outside my sector are too hard tosee. The light grey colour is not good. They should be in more contrast to thebackground.


45

2. WORKLOADStrongly


1 5 4 12.1 Using SYSCO was hard work.

Please comment.• In some situations e.g. if you propose 2 aircraft on parallel headings and the accepting

unit would like them to come at different levels, this co-ordination would be hard workand take too much time.

• Not really, but I had to make more inputs than I had expected.

• A lot of “mouse work”.

• Easy to learn. You just had to understand the difference between cleared flight leveland exit flight level etc. Of course it would be easier to have it explained in Swedish.

• Since we did not speak with the aircraft we could use all the time for “clicking”. Allfunctions were not at hand.

• Not having in mind the amount of traffic, which was low. With more traffic from the verybeginning, I think I would have to fight the system more.

• Again, system bugs made this question hard to answer. The system wasn’t tested in ahigh load environment, because all traffic wasn’t correlated.

• I don’t know, because the amount of traffic was too low.

StronglyAgree

StronglyDisagree

3 3 42.2 Using SYSCO my workload was alwaysacceptable.

Please comment.• When controlling a holding you couldn’t keep up in updating the labels. With high level

traffic it was very helpful.

• If I was working as executive ATCO the above mentioned applies. As planner (APPCO)the answer would be more like 1 & 2.

• Will you be occupied by system (co-ordination) problems (when appearing) and forgetthe traffic problems?

• It wasn’t always acceptable to be both planner and executive. When it was really busyyou could need help from a planner. But in case you have that, it would be acceptable.

• But frustrating when the system didn’t cope with reality.

• Impossible to have an opinion.

• Not possible to answer - not much traffic during exercise hours.


46

StronglyAgree

StronglyDisagree

9 12.3 With SYSCO I could safely control moreaircraft than with the current system.

Please comment.• Yes, when it came to high level overflights it was very helpful. The updating of labels

with in- and out-bound took a lot of capacity.

• For high level transit traffic SYSCO is a plus, I am still not convinced that low levelsectors, where you have to do a lot of inputs, gains you any capacity from SYSCO.

• As 2.2.

• I think I need more training on the system to answer correctly.

• Too early to say. Probably yes.

• I don’t think so. But it’s hard to say. If you get really used to it perhaps you could.

• Too little experience.

• Very difficult to compare ‘real life’ with the situation during this test.

• Probably no difference at all. However, in previous DSI experience, I noticed a quiteannoying problem to get the correct object/label highlighted/under my control, in peakhours (due to overlapping labels).

• Not applicable.

• Maybe, since the co-ordinations are made faster.

• Refer to 2.1.

StronglyAgree

StronglyDisagree

1 1 82.4 I have more spare time when working withSYSCO.

Please comment.• You have no chance of taking your eyes off the screen, you might miss some co-

ordination which will time out.

• Transit traffic: Yes. Climbing, descending, holding, spacing: No.

• As planner (APPCO): Yes. As executive: no.

• It’s nice to work without strips!

• The co-ordination is faster.

• See 2.3.

• Unable to compare after this test.

• Possibly true, with an improved SYSCO system.

• See 2.3.


47

StronglyAgree

StronglyDisagree

1 3 1 12.5 My work was affected by the absence ofMAESTRO.

Please comment.• Because we were just shadowing, I am not able to answer.

• I work at sector 8 and 9 and we don’t use MAESTRO.

• I could never compare this because of the layout of the exercise.

• No.

• MAESTRO is not used in the sectors I work.

• Not applicable, working sector 8/9.

• Long term planning is necessary when MAESTRO is in use.

3. HMIStrongly


2 5 1 2 13.1 I found it easy to make mistakes enteringco-ordination conditions into DSI/SYSCO.

Please comment.• No. The chance of making mistakes is very small, but the co-ordinations lose some

flexibility due to the “all or nothing” answers.

• Could be due to the lack of training or unexpected system response, due to systemimmaturity.

• It is easy to click the mouse twice by accident and thereby request a wrong point.

• How do you take away co-ordination mistakes?

• Only a matter of getting used to the system I think. The only thing I still find a bitconfusing is that the route is both on line 2 and 3, and when you should use which one.But I’m sure time would solve that (or maybe I’m stupid).

• Definitely in the beginning. Better after some practice.

• It needs some training, but no problem.

• Yes, lack of training.

• Too easy to inadvertently reset the range. Sometimes, delays in slowing scroll lists etc.made me do the wrong input.

• Entering the wrong point/level.


48

StronglyAgree

StronglyDisagree

1 3 5 23.2 I was able to identify my input mistakesand correct them before sending.

Please comment.• Once the input was made, the messages were sent.

• See 3.1. However, it should be possible to change/modify routes/tracks when the traffic(aircraft) is assumed, without it affecting the co-ordination.

• I was able to identify my mistakes, but not able to correct them (no “reset” button).

• That’s not possible when the ACT is gone?

• Sometimes I make a co-ordination by mistake, so it became a question of direct routeinstead of an input that the a/c was going direct to that point. But as I wrote in 3.1. I’msure it’s only a matter of getting used to the system.

• I could identify them, but sometimes not correct them. Could be too little practice.

• If mistake in done on input, the co-ordination is done at once. So, to correct it youwould need to do a new co-ordination.

• Not at all in the beginning, but I improved after a while.

• SYSCO doesn’t allow me to change input mistakes. If you mean before sending themessage to the aircraft, the answer is yes.

StronglyAgree

StronglyDisagree

1 4 3 33.3 Sometimes I did not notice that a messagehad arrived.

Please comment.• If your traffic is moderate, you have your eyes on the aircraft and don’t notice the Co-

ordination Window. I suppose your planner in these cases would notice it.

• The message ‘in’ and ‘out’ windows, to notice the co-ordinations, should be placed onthe lower half of the screen.

• If I was working a complex problem, I did not always notice the message (3.9).

• It happens. It could be frustrating for the sector trying to make the co-ordination.

• Mostly in the Feed Sector when I was busy clicking labels far from sector 8/9 and S/M.

• You have to be used to the system. I don’t think that it will be a problem.

• Not in these exercises. However, I know (due to prior experience) that this may happenin peak hour traffic.

• Perhaps there is a need for a stronger colour or an audio input.

• Yes, see 1.13.


49

StronglyAgree

StronglyDisagree

2 2 2 1 43.4 Sometimes I did not notice that I couldbegin co-ordination with the receivingsector.

Please comment.• Most of the time I was waiting for the ACT to go to be able to co-ordinate.

• See 3.3.

• The blue colour is very obvious. It’s not that easy to see when the next controller hasgot the ACT. Perhaps another colour (maybe green) would be better.

• I had a hard time seeing the difference in the label as to whether it was possible or not.

• If it’s really necessary to co-ordinate, you pay attention, otherwise you could miss whenyou’re able to. Human factor as always.

• As I understand, I can only expand the label (put the cursor over it) before I can see ifthe ACT has been sent. This must be shown more clearly.

• You couldn’t see it in the label. You have to ‘hook’ the callsign.

StronglyAgree

StronglyDisagree

5 3 43.5 SYSCO/DSI let me co-ordinate in all theways I wanted to.

Please comment.• Only with standard co-ordination, like level changes and direct tracks.

• Sometimes you need to do co-ordinations with two sectors at the same time. You arenot able to “discuss” solutions.

• I found the system far too rigid (inflexible) - a sort of manual override may be thesolution. A fair guess is that 90 – 95% of the co-ordinations could be done via SYSCO.

• You need to be able to co-ordinate with more than one sector at the same time.

• Route – very good, Level – good, difficult to get used to, with 3x level-information,Heading – not possible, Speed – not possible.

• Headings are difficult to co-ordinate. Speed is not possible to co-ordinate. I don’t knowif it’s possible to make a co-ordination that you don’t want the aircraft on your frequency,but directly to the next one.

• The flight plans had too few possible co-ordination points.

• Early co-ordination (before co-ordination phase) is a problem.

• The elastic vector must be able to use more than one point. This comment may be inthe wrong place.

• Again, as an approach controller, some discussions have to be made in the planning ofarriving aircraft, which is not possible in an electronic way.


50

StronglyAgree

StronglyDisagree

1 3 2 4 23.6 I always knew when the actions I had takenon the HMI had been completedsuccessfully.

Please comment.• You had to check each label to see whether the message had timed out or was

accepted.

• If the system had been more stable the answer would had been ‘1’.

• Most of the time.

• The only uncertain thing was, if you have made a co-ordination out and didn’t get ‘OK’,you don’t know if the other person didn’t have time to answer or didn’t see it (or didn’treject it).

• We had to confirm via telephone many times. The colours of the labels did notcorrespond between ESMM/EKDK.

• For example – when revisions have “timed out” and therefore not been accepted byreceiving sector, you might not see it until you’re transferring it.

• No. I didn’t know if a message was not responded to, or not being noticed.

• Not in the beginning. It took some time before I understood the ‘system language’.

StronglyAgree

StronglyDisagree

4 3 53.7 Co-ordination with SYSCO feels lessflexible than the current process.

Please comment.• It is all or nothing required. If you would like to co-ordinate something a little out of the

ordinary, it would be easier to phone � more flexible.

• Level change and direct route co-ordination is OK with SYSCO.

• This is most likely the key problem! SYSCO may indeed be able to help you with most(90-95%) of the standard co-ordination, but you would certainly need a manualinput/override possibility.

• The system works well with “standard” co-ordinations. More complex situations will stillneed telephone co-ordination.

• Especially when you give direct routes. There must (or should) be a possibility to selectmore points.

• Too many bugs and missing correlation to have an opinion - myself not being welltrained.

• Not so flexible when it became a dialog. Faster when the answer is a simple ‘yes’ or‘no’.

• To co-ordinate waypoints not in ‘X’ point list was not flexible.

• Sometimes. See previous answers.


51

StronglyAgree

StronglyDisagree

2 6 1 13.8 It was always obvious what I had to donext.

Please comment.• If the system works, and you know how to use it, (not that difficult), you should not be in

doubt about what you have to do.

• Most of the time this was correct.

• With some training, yes.

• Yes, the system is easy to understand.

StronglyAgree

StronglyDisagree

5 5 1 13.9 I often found myself waiting for a responseto a co-ordination message I had sent.

Please comment.• And when the ‘co-ordination out’ window was cleared, you had to check the label.

• Sometimes, it could be that the controller is busy, or has not seen the co-ordination.

• We often saw the problem because of system immaturity, but if being an executivecontroller, I was working a complex problem, the same thing could happen (3.3).

• To be fair, you also have to wait with the present system.

• It’s probably because we’re not used to the system yet and we had a Feed that at sometimes was quite busy.

• Probably due to EKDK and ESMM having different correlation.

• No! I do other things meanwhile.

• You tend to focus on the co-ordination, and if you don’t get an answer relatively soon,you get annoyed and pick up the phone. This way you can lose concentration on actualtraffic.

• Could be the system bugs.


52

StronglyAgree

StronglyDisagree

2 5 1 33.10 I am able to see all I need to on the radar.

Please comment.• Except for the unconcerned traffic. In case of the e.g. a slow climber, it would be nice to

be able to extend the label to see the cleared level.

• Sometimes I could be afraid that the amount of info. (“SIL”/ “VAW”/”Co-ord in/out” etc.)could saturate the CWP to a degree where the traffic becomes less important becauseof the “cluttering” of the screen . The reason for the ‘4’ is therefore mostly because ofanxiety for not seeing the traffic on the screen due to the amount of other information.

• The labels and menus tend to cover too much of the screen.

• Do we need SIL (Sector Inbound List)? Missing: Information about ATS-Routes fromthe flight plan.

• Easy to pre-select a couple of different ranges to get a quick overview.

• When using AHD I often had to re-arrange my range to see the routing point requested.

• Given the correct parameters, yes.

• See 1.17 and 3.4. Important: the actual aircraft symbol is not shown clearly, or bigenough. You could find yourself controlling the labels instead. Historical trail dots arenot shown well enough.

• I’m not able to see traffic in the surrounding sectors.

3.11 Could the method of working be made more efficient in any way?

• If the system could calculate a new sector sequence in case of re-routing, returning ordirect tracks.

• We only worked part of the HMI/DSI in this simulation.

• Fewer mouse inputs, manual inputs (what do I/you exactly want). Co-ordination with morethan one sector for a given input.

• After co-ordination – when clearing an aircraft direct to a point outside your radar range,you’ve got problems. This method is not so good.

• When the ACT has been sent, the system shows the next controller. Sometimes that isnot the right one (due to level change or re-routing etc.) There should be an option tochange to another one.

• I have too little experience to give a good answer.

• As written earlier, SYSCO could/would increase efficiency if improved and a bit moreflexible.

• I definitely need a higher workload to give an answer.

• When re-routing an aircraft, I’m limited to the points that show up on the list. This list mustinclude more points, or I should have access to some kind of database via the keyboard.


53

3.12 What requirements for additional system feedback following an input orinteraction did you identify whilst using the system?

• I would like to be informed (e.g. with a colour change) when a co-ordination is acceptedso I don’t have to check each label for direct track, level change etc.

• Co-ordination messages for all “ASH”, “SPD” and level changes. Counter proposals forall “ASH” as well as level changes (and speeds).

• Highlight for Heading and Speed.

• If you make a co-ordination on the phone and then make the proper inputs in the label,you could initiate a co-ordination out that you don’t want. It would be good if you couldselect to not send a co-ordination in those cases.

• See above question.

• See 3.11. I would also like to be able to input several points into the routing of anaircraft.

3.13 In which circumstances, with SYSCO, would you use the phone?

• If I wanted to accept on more than one thing. If I wanted to solve a problem so that ithelps both sides. If I wanted something done now e.g. a direct track to solve a conflict.

• If more than one sector is involved. If more than one aircraft is involved.

• If I need an immediate response. If I need to co-ordinate with more than one sector. If Ihave more than one possible solution to a problem.

• When it is urgent, complex situations that need to be explained.

• Very important, urgent co-ordination. Emergency.

• To co-ordinate speed. To co-ordinate an aircraft that wasn’t supposed to enter anothersector but for some reason did. To co-ordinate a point that it is not possible to select inSYSCO. To tell previous controllers to send the aircraft to the next controller instead ofyou. To co-ordinate that an aircraft will be climbing into the next sector.

• When in need of a quick response to an action taken shortly before a transfer.

• Level revisions, direct routes, speed restrictions and similar co-ordinations with SYSCO.More complex co-ordinations with the phone.

• If a situation is really complicated/complex, or when I can’t interpret an answer.

• See 3.5. Also, complicated situations where e.g. a question for a direct route is acceptedIF you proceed behind this or that aircraft.

• In co-ordinations which involve more than one aircraft. When you need a confirmationthat the other controller understands the traffic situation/problem. When you need to do adirect request early to plan ahead.


54

4. COMMENTS4.1 The best thing about SYSCO is:

• It makes standard co-ordination easy and fast.

• The immediate update of the label so that you don’t forget or misunderstand a co-ordination done by SYSCO.

• The safety. Provided the system works as expected you would never be in doubt (evenbetter, you would always know) what has been co-ordinated.

• The absence of “crossed transmissions” between RT and telephone. Much lesschance of misunderstandings between Denmark and Sweden.

• Reducing verbal co-ordination. No strips. The possibility to co-ordinate REL (release).

• Fast. Easy to use. You save time. Better service to pilots.

• It shouldn’t be too hard to learn since it’s based on windows like an ordinary PC. Theinformation is given in different positions, but you handle the inputs in the same way inthe different positions.

• Less phone calls. No strips.

• Co-ordinations are quicker with SYSCO, compared to phone co-ordinations.

• It would definitely work in an area sector.

4.2 The worst thing about SYSCO is:

• I found it was lacking flexibility. The various windows take up a lot of room on myscreen. All is done via the mouse.

• SYSCO’s limitations.

• The inflexibility.

• The lack of flexibility. No human contact, and therefore no way to stress a co-ordination.

• Technical problems with the system. Risk of receiving many co-ordinations that I haveto reject.

• Getting used to not having strips (perhaps that’s good!).

• With adjustments here and there, I don’t think there is any worst thing.

• Not flexible.


55

4.3 Any other comments?

• It is impossible to take a glance over the screen to see if everything is updated. You hadto check each aircraft every time. I could imagine it would be difficult to give a clearanceto a departure out of a small airport. You would have to remember which levels you hadgiven, or check each label.

• It was annoying that the system- function was so bad for the first 14 days. Weconcentrated on bug finding, rather than on SYSCO, which was the purpose of the trial.However, we managed to get some understanding of the system, advantages anddisadvantages. For 90% of co-ordinations done by controllers, SYSCO is easy andquick to use, for the last 10 % you have to co-ordinate by phone. SYSCO tools do nothave the same flexibility as the current procedures, but as a combination I do believeSYSCO will be accepted by controllers as a as a useful tool.

• It has been very interesting working on the PROVE trial and even though many of thecomments may be somewhat negative. I still believe that SYSCO can perform around90-95% of the necessary co-ordination (provided it has proved itself operationally). Thesystem as it is today however, is far too rigid (the controller is not in the “loop”, whichcould easily give unnecessary frustration. To avoid this, more flexibility must be built intoSYSCO (i.e. manual input possibilities).

• Even better: If the system could use real time radar updates (for trajectories), manyproblems could be overcome.

• Therefore, let’s have the system matured over a couple of years and then have anothertrial, (possibly using the same set-up), but with better phones (so that we better candiscuss what’s going on).

• I don’t know how it’s supposed to work, if you work two persons in the same sector asexecutive and planner. But I think both persons would need a SYSCO position each tobe able to make inputs. If you change an exit level after the ACT has been sent, itinitiates co-ordination to the next controller, who has to approve it, otherwise it will turnyellow when you transfer it. It would be great if that only happened when it’s less than 5minutes to the border of the next sector, as that is the normal procedure today. If it’smore than 5 minutes, you just to make a revision.

• You have to know more about the system than I did before participation in a trial. Ofcourse, some of my opinions can be useful but it is very hard to answer questions whenyou really do not know anything about how the system should work in the perfectsituation. I think it’s hard comparing when you do not speak to the aircraft and since youdo not feel the actual responsibility. We did not use all available options, due to, Isuppose, technical reasons. This is the first time ever that I’ve been confronted with astripless system. I do not fear the future in this respect. I do not think that you can workalone, without a Planner, with this system. You need assistance just as much as in the‘old’ system.

• SYSCO would be a good and useful tool (especially if the system is improved) as acomplement to phone co-ordinations. During these exercises/simulations the simulatordid not work properly, so about 80% of the time, these 5 days was pretty much just awaste of time. For future simulations, SYSCO studies or other studies/simulations ofinterest, it might be a good idea to validate the simulator first – and then validatewhatever should be validated.

• Personally, I think this test would have been of more value made completely as asimulation. The overall workload has been too low, thus not giving me the opportunity toreally benefit from the system. It gives me proper training in advance to get to know howeverything works, that is! A big credit to everyone involved in this test! Thanks!

EUROCONTROLRapport relatif aux essais B du système PROVE/SYSCO

56


Rapport relatif aux essais B du système PROVE/SYSCO EUROCONTROL

57

FRENCH TRANSLATION

AVANT-PROPOS DU RAPPORT RELATIF AUX ESSAIS ‘B’ DU SYSTEMEPROVE/SYSCO

Le présent rapport expose les résultats des essais en mode miroir du système SYSCOeffectués en conditions réelles entre les CCR de Copenhague et de Malmö, dans lecadre du projet PROVE.

Le projet PROVE, lancé à l’initiative d’EUROCONTROL, est cofinancé au titre desTEN-T de la Commission européenne et soutenu par les Administrations de l’Aviationcivile de la Suède (LFV) et du Danemark (SLV).

De nombreuses recherches et études de simulation sont effectuées chaque année enEurope, à la fois par l’industrie et des centres de recherche. Il n’empêche qu’il s’écoulesouvent entre dix à vingt ans avant que ces études soient appliquées au sein desystèmes opérationnels. De nombreux rapports, dont le Document « EATMS ValidationStrategy Document » version 1.1 et le Document « EUROCONTROL’s IndustrialSuppliers Relationship » version 1.0, mettent en évidence deux facteurs principauxconcourant à ce phénomène :

1. L’industrie n’investira pas dans des nouveaux concepts non validés, d’où l’absenced’infrastructure spécialisée pour valider des concepts dans un environnement réelpréopérationnel ;

2. Diverses exigences non normalisées sont imposées à l’industrie par lesadministrations nationales, tout particulièrement dans les projets ATM européensODS.

PROVE (Plate-forme européenne d’expérimentation et de validation préopérationnelle)s’articule en plusieurs phases pour aborder ces thèmes ; dans un premier temps, pourfournir une infrastructure de validation préopérationnelle et servir de baseexpérimentale destinée à valider le système SYSCO. Cette étape sera suivie d’unprogramme de validation de concepts visant à traiter la question des spécificationsODS et FDPS non normalisées.

PROVE a repris la plate-forme ESCAPE (plate-forme de simulation etd’expérimentation d’EUROCONTROL), l’a interfacée avec les systèmes ATMopérationnels de Malmö et Copenhague, et a établi une liaison de données spécialiséeentre les deux centres. Cette plate-forme a fonctionné parallèlement (en miroir) auxsystèmes réels, avec les mêmes données opérationnelles (informations radar et plansde vol) et le concours de contrôleurs certifiés.

C’est sur cette plate-forme qu’a été effectuée, en avril et mai 2000, une expérimentationdont l’objectif était de valider OLDI 2.2 (SYSCO) en mode miroir.

Chris ChickenResponsable du Projet – EUROCONTROL


58

RESUME

La Plate-forme d’expérimentation et de validation préopérationnelle de l’ATC européen(PROVE) est un projet cofinancé par Eurocontrol et la Commission européenne (Réf.1),avec le soutien du Danemark et de la Suède, ainsi que celui d’autres états membres.Le présent rapport a été élaboré, pour le compte du Centre expérimental d’Eurocontrolà Brétigny, par les National Air Traffic Services Ltd, et présente les résultats de ladeuxième phase d’essais menée sur la plate-forme PROVE à Malmö (Suède) etCopenhague (Danemark) en avril/mai 2000, compte tenu des résultats de la premièrephase qui s’était déroulée à Malmö en décembre 1999 (Réf.2).Le système de Coordination Automatisée (SYSCO) est basé sur le standard OLDI(échange de données en ligne) ; il fournit un support à la coordination électroniqueentre secteurs et entre centres ATC. Le système utilisé durant les expérimentations secompose de l’interface HMI mise au point dans le cadre du projet DSI (InterfaceDanemark/Suède) et de la plate-forme ESCAPE d’EUROCONTROL, adaptée pourrecevoir des pistes radar réelles et des données plan de vol du centre. L’objectif étaitde déterminer si l’ensemble des messages SYSCO correspondaient à l’usage qu’il estprévu d’en faire, en prenant l’interface DSI à la frontière Malmö-Copenhague.Les plates-formes PROVE ont été assemblées aux Centres de contrôle régional (CCR)de Malmö et Copenhague. Les essais ont été menés en mode miroir, c’est-à-dire que leprototype a été alimenté en données réelles émanant des radars et d’autres systèmes,mais n’a pas été utilisé pour contrôler ou influencer le trafic aérien réel.Les 14 jours qu’ont duré les essais ont mobilisé quatorze contrôleurs, dont huit del’Administration de l’Aviation civile de la Suède (LFV) et six de celle du Danemark(SLV). Les essais n’ont pas permis d’obtenir tous les résultats escomptés en raison dudysfonctionnement de certains dispositifs, notamment de la détection des conflits àmoyen terme (MTCD), et de la nécessité de recourir à un certain nombre de palliatifs.Les contrôleurs ont été malgré tout en mesure d’effectuer une première évaluation desfonctionnalités du système SYSCO en conditions de trafic réelles et de recenser uncertain nombre d’impératifs et d’améliorations nécessaires.De l’avis des contrôleurs, SYSCO fonctionne bien pour les coordinations simples. Iln’est cependant pas suffisamment flexible dans le cadre des procédures actuelles. Iln’a pas été en mesure de prendre en compte les modifications de séquence desecteurs faisant suite aux coordinations de routes directes et de profils de montéeimprévus. Les contrôleurs ont donc été obligés de passer outre les prévisions dusystème relatives au secteur suivant. De plus, le système SYSCO, tel qu’il a été mis enœuvre dans le cadre de l’interface DSI, s’est révélé incapable d’accepter lacoordination simultanée avec un secteur en amont et un secteur en aval.

Les coordinations de niveaux se sont révélées plus probantes que les coordinations deroutes directes, même si les contrôleurs se déclarent préoccupés par le fait que lesystème SYSCO, tel qu'il se présente avec l’interface DSI, n’affiche pas les messagesde coordination applicables aux conditions « standard », principalement lorsqu’il s’agitde modifications d’un niveau « standard » à un autre.


59

En mode miroir, les contrôleurs n’ont pas pu évaluer intégralement l’incidence deSYSCO sur leur charge de travail. Ils ont eu le sentiment d’être privés de leurresponsabilité sur le trafic et d’être contraints de réagir à des situations au lieu de lescontrôler et de les planifier. Il ont cependant laissé entendre que SYSCO offre lapossibilité d’alléger la charge de travail dans un environnement en route grâce à ladiminution du nombre d’appels téléphoniques. Ils ont ajouté que ce système doit fairel’objet d’un examen approfondi pour déterminer s’il peut être utilisé en contrôled’approche.

D’une façon générale, les contrôleurs se sont déclarés optimistes quant à l’utilisationfuture de SYSCO. Ils ont cependant estimé que le système doit faire l'objet d'une miseau point avant de pouvoir être exploité.

Les essais PROVE/SYSCO se sont finalement révélés insuffisants pour une validationapprofondie de SYSCO, en partie pour des raisons techniques. Néanmoins, ils sontvenus couronner des recherches techniques intenses, le logiciel du Centreexpérimental ayant pu être transféré vers une salle d’exploitation. Ils ont égalementpermis de mieux connaître la manière et le moment d’effectuer des essais en miroir,lesquels ne doivent pas servir à mesurer la charge de travail et ne peuvent valider lessystèmes que dans les limites des contraintes de l’espace aérien, des procédures etdes volumes de trafic actuels. Le mode miroir fournit néanmoins une plate-forme idéalepour mieux cerner les problèmes opérationnels et soumettre les nouveaux concepts àune plus vaste palette de contrôleurs certifiés.


60

1. INTRODUCTION1.1 GENERALITES

La Plate-forme ATC européenne d’expérimentation et de validation préopérationnelle(PROVE) est un projet cofinancé par Eurocontrol et la Commission européenne (Réf.1),avec le soutien du Danemark et de la Suède, ainsi que celui d’autres états membres.Le présent rapport a été élaboré pour le compte du Centre expérimental d’Eurocontrol àBrétigny, par les National Air Traffic Services Ltd : il présente les résultats de ladeuxième phase d’essais (Essais B) menée simultanément sur les plates-formesPROVE à Malmö (Suède) et Copenhague (Danemark) en avril/mai 2000. Les Essais Aavaient été effectués à Malmö en décembre 1999 (Réf.2).Les enseignements techniques tirés de ces essais font l’objet d’un rapport séparé.

1.2 CONTEXTELe projet PROVE vise à fournir une infrastructure de validation préopérationnelle. Lespremiers essais effectués sur la plate-forme portent sur le concept de Coordinationautomatisée (SYSCO) (Réfs. 3 et 4). Les phases suivantes traiteront notamment de laprévision des trajectoires et de la détection des conflits à moyen terme (MTCD).SYSCO est une application mettant en oeuvre le standard OLDI (échange de donnéesen ligne) (Réf.5) ; elle définit une procédure pour l’utilisation d’un ensemble renforcé demessages (OLDI 2.2) dans le but de transmettre des informations électroniquesrelatives à la coordination et au transfert d’aéronefs entre centres ATC.

Il est prévu que les systèmes « System 2000 » suédois et « DATMAS » seront mis enconformité avec une certaine version de SYSCO. Le bilan de l’essai orientera lamanière dont SYSCO et les deux systèmes susvisés devront être modifiés afin que leurcombinaison puisse fonctionner et être exploitée.

1.3 BUT DE L’EXPERIMENTATIONLe but de ces essais était de valider SYSCO dans le cadre de l’interface DSI, dans unenvironnement de trafic réel à la frontière entre le Danemark et la Suède. Ici, le terme‘valider’ désigne le processus qui doit permettre de démontrer les avantages deSYSCO (et les moyens mis en oeuvre dans DSI pour son exploitation) aux controleurs.

Les objectifs détaillés de l’expérimentation sont définis dans le plan de validation(Réf.6). En synthèse il s’agissait :

• d’évaluer les possibilités d’utilisation de SYSCO, c’est-à-dire la mesure danslaquelle l’utilisation de ce système se révèle efficace, intuitive et satisfaisante ;

• d’évaluer l’incidence de SYSCO sur :

(iv) (i) la charge de travail des contrôleurs ;

(v) (ii) la sécurité ;

(vi) (iii) la qualité du service fourni aux compagnies aériennes.


61

2. CONCLUSIONSLes Essais ‘B’ de SYSCO ont permis aux contrôleurs des deux CCR d’utiliser SYSCOpendant environ 300 heures, à l’issue desquelles ils se sont déclarés globalementoptimistes quant à l’utilisation future de SYSCO. « Le système est très rapide pour lacoordination standard et favorise la concentration ». « Il présente des avantages, maismanque de souplesse ». Les essais n’ont cependant pas permis d’obtenir tous lesrésultats escomptés en raison du dysfonctionnement de certains dispositifs, notammentla détection des conflits à moyen terme (MTCD), et de la nécessité de recourir à despalliatifs. Les contrôleurs ont cependant pu recenser un certain nombre d’impératifs etd’améliorations nécessaires :

• Le système n’est pas conçu pour prendre en compte d’éventuelles modificationsdans la séquence des secteurs, mais il est à l’évidence nécessaire, sur le planopérationnel, de disposer de la faculté de passer outre les prévisions du systèmerelatives au secteur suivant.

• Dans le cadre des procédures actuelles, la coordination peut faire intervenirplusieurs secteurs en amont/aval, en plus du secteur suivant, particulièrementpour les routes directes. Dans ces circonstances, il est nécessaire de pouvoircoordonner en amont et en aval simultanément. Tel qu’il a été mis en œuvre,SYSCO n’offre pas cette possibilité et, durant les essais, les contrôleurs ont étéempêchés de coordonner en aval avant de prendre en charge un aéronef.

• Les contrôleurs se sont déclarés préoccupés par le fait que, dans l’interface DSI,les messages de coordination ne s’affichaient pas pour des conditions« standard » (voir section 3.4). Tous les contrôleurs se sont accordés à dire qu’unmessage devrait être affiché tant pour les conditions standard que pour les autres,lorsqu’elles diffèrent des conditions attendues par le contrôleur.

• Les contrôleurs n’ont pas été suffisamment informés de l’expiration du délai devalidité d’un message. Ils ont suggéré que, s’il est nécessaire d’appliquer unetemporisation aux messages de coordination, un échéancier devrait leur permettrede redéfinir la priorité de leurs tâches, et un dispositif « d’attente » devrait leurlaisser suffisamment de temps pour examiner plus avant la proposition ou acheverd’autres tâches.

En mode miroir, les contrôleurs ont eu le sentiment d’être privés de leur responsabilitésur le trafic et d’être contraints de réagir à des situations au lieu de les contrôler et deles planifier. Ils n’ont donc pas pu évaluer intégralement l’incidence de SYSCO sur leurcharge de travail. Ils ont cependant laissé entendre que SYSCO offre la possibilité del’alléger, grâce à la diminution du nombre d’appels téléphoniques, mais ont ajouté quesi le système permet peut-être de réduire la charge de travail dans un environnementen route, un examen approfondi doit être effectué pour évaluer s’il peut être utilisé encontrôle d’approche.

Un contrôleur unique a été capable de remplir les rôles du contrôleur exécutif et duplanificateur. Toutefois, en raison du caractère artificiel de la charge de travail et del’absence d’informations MTCD, il n’a pas été possible d’évaluer à fond cette capacité,qui appelle donc un examen plus poussé.

Si l’un des avantages de SYSCO est l’élimination de toute ambiguïté sur ce qui a étécoordonné, c’est au risque d’accroître la probabilité d’envoi accidentel d’un messageincorrect.


62

Les essais PROVE/SYSCO se sont en fin de compte révélés insuffisants pour unevalidation complète de SYSCO, essentiellement pour des raisons techniques. Ils ontcependant permis de tirer des enseignements précieux sur trois aspects :

• Ils ont permis de mieux connaître la manière et le moment d’effectuer des essaisen miroir.

• Ils ont apporté un couronnement opérationnel à des recherches techniquesintenses, en transférant le logiciel du Centre expérimental vers une salled’exploitation.

• Ils ont démontré que si le système SYSCO présente des avantages, c’est, dansles conditions actuelles de mise en œuvre, au prix d’une forte réduction de laflexibilité.


63

3. RECOMMANDATIONSLa poursuite de la mise au point et de la validation de SYSCO dans le cadre de la miseen œuvre de l’interface DSI doit :

• améliorer la prévision des trajectoires et la détection des conflits à moyen terme ;

• mettre en œuvre un moyen offrant au contrôleur la possibilité de passer outre lesprévisions du système relatives au secteur suivant ;

• évaluer la mesure dans laquelle SYSCO doit prendre en compte les procéduresactuelles, afin de permettre la coordination avec un autre secteur en plus dusecteur suivant, ou les procédures être modifiées pour s’adapter à SYSCO ;

• mettre en œuvre un moyen permettant au contrôleur d’effectuer une coordinationinter-centres avec plusieurs secteurs, en amont ou en aval ;

• offrir aux contrôleurs la capacité de coordonner un aéronef sur un cap et avec deslimitations de vitesse ;

• envisager l’affichage des messages de coordination sur l’interface DSI en cas demodification des conditions standard ;

• prévoir d’indiquer aux contrôleurs que le délai de validité d’un message va êtredépassé, ou supprimer la temporisation ;

• envisager l’introduction d’un dispositif « d’attente » offrant aux contrôleurssuffisamment de temps pour examiner plus avant une proposition ou acheverd’autres tâches avant de répondre ;

• améliorer l’indication du moment où l’ACT sera transmis au secteur suivant, etaffiner le paramètre ACT pour l’adapter à différentes routes ;

• recourir à une simulation en temps réel pour évaluer l’incidence de SYSCO sur lacharge de travail des contrôleurs ;

• examiner comment SYSCO pourrait être adapté à un environnement d’approche ;

• étudier le mode d'exploitation et les rôles d’un planificateur et d’un contrôleurexécutif.

De plus, il est recommandé que les futurs essais en mode miroir :

• prennent la forme d’une série de brefs essais, afin de conférer un caractère ATCopérationnel à ce qui est essentiellement un processus technique ;

• permettent de mieux faire connaître les développements et suscitent descommentaires subjectifs ;

• ne servent pas à évaluer quantitativement la charge de travail.

Documents

EUROCONTROL EXPERIMENTAL CENTRE · Commission and support by the Swedish (LFV) and Danish (SLV) Civil Aviation Authorities. ... up through a programme of concept validations to address