ADVANCED HUMAN-COMPUTER Dependent Surveillance (ADS ... · aircraft, direct the traffic, and have the final word in all decisions. Therefore, how computer systems present information

1

American Institute of Aeronautics and Astronautics

ADVANCED HUMAN-COMPUTERINTERFACES FOR AIR TRAFFIC

MANAGEMENT ANDSIMULATION

Ronald Azuma and Mike DailyHughes Research Laboratories

3011 Malibu Canyon Blvd. MS RL96Malibu, CA 90265

Jimmy KrozelSeagull Technology

21771 Stevens Creek Blvd.Cupertino, CA 95014-1175

Abstract

New technologies will significantly change Air TrafficControl over the next 15 years. These changes willrequire improved human-computer interfaces for theless regulated, more complex future environment. Thispaper describes a highly interactive, real timedemonstration of 3-D visualization and interfaceconcepts for the air traffic domain, including FreeFlight. This demonstration offers a 3-D, stereoscopicview from both the controller's and pilot's perspectives,featuring representations of projected flight paths, 3-Dgraphical and audio proximity warnings, and 3-D textlabels that automatically reorient themselves. Feedbackfrom domain experts is described. In Free Flight, pilotsand airlines will set their own courses and resolveconflicts autonomously when possible. Thisdemonstration also shows visualizations of theProtected Airspace Zone and Tactical Alert Zone safetyregions around Free Flight aircraft, which are mosteasily understood through the use of 3-D graphics.Future versions of this demonstration will acquire morerealistic data, improve the interaction techniques andintegrate the visualization more closely with conflictdetection and resolution algorithms.

Motivation

During the next 15 years, Air Traffic Control (ATC)systems will undergo significant changes due to newtechnologies16. Such changes are required to meet theexpected growth in air traffic. New technologies suchas the Global Positioning System (GPS), Automatic Copyright © 1996 by the American Institute ofAeronautics and Astronautics, Inc. All rightsreserved.

Dependent Surveillance (ADS) communications, andmore sophisticated ATC software will provide some ofthese required improvements. These technologies willestablish an "Internet in the sky," providing aninformation-rich environment for distributing manytypes of data amongst pilots and controllers. However,these technologies by themselves will not satisfy all theneeds of future ATC systems. Another vital butsometimes overlooked component is the human-computer interface: how controllers and pilots willinteract with the information provided by these newtechnologies. Despite increased automation and moresophisticated computers and sensors, humans willalways be "in the loop." People, not computers, fly theaircraft, direct the traffic, and have the final word in alldecisions. Therefore, how computer systems presentinformation to the human controllers and pilots willplay a crucial role in the effectiveness of future ATCsystems.

While existing human-computer interfaces may besufficient for today's ATC environment, they will notbe in the future when air traffic patterns will be morecomplicated and less ordered. Existing controllerinterfaces use flat, 2-D displays with trackball andbutton-based controls. Pilots may have no trafficdisplays at all, relying on charts and voice commandsfrom controllers. However, the advent of Free Flightwill change the current situation dramatically. Undercertain conditions, Free Flight allows pilots to set theirown course and resolve conflicts by themselves,without involving controllers except when needed.This distributes the task of air traffic control over allaircraft, reducing the air traffic controllers’ workloads.However, instead of the orderly well-regulated trafficpatterns that exist today, there will be the potential formore complex and variable traffic patterns. Predictingand avoiding conflicts in a such an environment will bemore difficult than it is today. More importantly,airlines will take an active role in air trafficmanagement. Therefore, pilots and Airline OperationCenters (AOCs) must also have displays that show theATC situation and potential conflicts and solutions in away that is easy to quickly understand, without drawingconcentration away from the task of flying the aircraft.Existing interfaces will not meet these needs for eitherpilots or controllers.

We believe the best course for meeting these futureneeds is to combine automated conflict detection andresolution mechanisms with advanced human-computerinterfaces that use intuitive and natural three-dimensional displays and controls, such as those beingdeveloped in Virtual Reality systems. Such displays

2


offer the potential to reduce the cognitive workload onboth pilots and controllers, resulting in fasterrecognition and improved situational awareness of theair traffic situation and potential conflicts. Thesedisplays will be multimodal, engaging the visual,auditory and vocal channels to provide an interface thatfits more naturally to the way humans normally operate,rather than forcing the human to meet the demands ofthe computer. The air traffic management domainprovides a rich environment to study the potential ofthese multimodal interface techniques. This domainrequires the simulation and display of a multiple aircraftairspace, intent data, proximity information for aircraftconflict detection, and conflict resolution options. Inthis paper, we present multimodal display techniquesaiding situation understanding of the pilot and air trafficcontroller.

Contribution

Several investigators have applied 3-D visualizationand virtual reality techniques to the air trafficmanagement problem. Several recent studies5, 8, 9, 11, 12

have investigated the potential benefit of 3-D orperspective displays to aid specific ATC tasks. Onestudy17 showed that when a display does not show thethird spatial dimension as readily apparent as the othertwo, pilots tend to solve conflict avoidance problems inthe displayed two directions more often than in thethree dimensions. For conflict resolution, this impliesthat a plan view display may bias conflict resolutionsolutions to right and left turning maneuvers. A follow-on study8 provided a perspective display to pilots fortraffic avoidance and found the pilots were more likelyto choose a solution with a vertical component. At leasttwo groups have used virtual reality interfaces toexamine ATC issues3, 4, 20. Pruyn15 explored anddemonstrated several concepts for applying 3-Dvisualization techniques to the ATC environment.

The contribution of this paper lies in a highlyinteractive, real-time demonstration of some conceptsthat we have not seen previously demonstrated in theATC environment and in the lessons learned aboutthese ideas. This is especially true of our concepts asapplied to the Free Flight domain, which we have notseen discussed in any previous visualization effort. Weintroduced some of our concepts in a previous paper7;the difference here is the actual implementation,demonstration and evaluation of those concepts. Ourdemonstration code is based on an original version fromthe MIT Media Lab18, which was greatly modified and

extended by the authors to demonstrate our ideas.

Our approach differs from some previous works in thatour goal is to create and demonstrate visualizationconcepts for the ATC domain, receive rapid feedbackfrom domain experts, and then repeat the cycle. Wehave not been running user studies on incrementalimprovements from traditional 2-D ATC displays. Webelieve rapid prototyping followed by rapid evaluationis a better approach to find the more advancedvisualization concepts that seem to be beneficial, whichcan then be formally evaluated by user studies.Furthermore, we have restricted our use of specializedequipment to the minimum required for demonstratingour concepts. While other virtual reality efforts useHead-Mounted Displays (HMDs) and fully-immersivedisplays, we have avoided that in favor of a stereodisplay on a low-end graphics workstation, even thoughour laboratory is equipped to run HMDs on expensivegraphics engines. In this way, we have reduced the"culture shock" of exposing our ideas to controllers andpilots who are used to flat, 2-D display panels andincreased the likelihood of eventual implementation ofsome of our concepts, due to lower cost requirements.

Initial Demonstration

The initial demonstration system is shown in Figure 1.The graphics engine is a low-cost Silicon Graphics(SGI) Indy. The user views the monitor and sees stereoimages through a pair of Crystal Eyes stereo glasses,made by Stereographics. The user controls the demothrough a mouse-based interface. 3-D sound isgenerated by two Alphatron sound boards, made byCrystal River Engineering. These boards run in a '486PC, connected to the Indy through a serial line. Thesynthesized 3-D sound is routed to an audio amplifierand broadcast to a pair of 900 MHz wireless radioheadphones.

Wireless stereoheadphones

Stereoglasses

SGI Indy

'486 PCwith 2 soundboards

Amp

Xmitter

Monitor

serial lineFigure 1: System diagram.

3


Figure 2: A sample view of what the user sees.

We first demonstrated this system on the exhibit floorof the 1995 Air Traffic Control AssociationConference1. Figure 2 shows a typical scene from thedemonstration. This system demonstrated thefollowing features:

• 3-D Perspective Display: Standard ATC displaysshow the situation in a top-down, plan-view 2-Dformat. This system provides that mode but also allowsthe user to navigate and view the scene in 3-D. Theview can be exocentric, looking at the entire situationfrom a remote perspective, or egocentric, following anindividual aircraft to see the pilot’s perspective. Theuser controls the point of view with a mouse, using a"virtual trackball" metaphor to change the pitch andyaw of the gaze direction. Simply clicking on anaircraft or other object in the environment causes thesystem to focus the user's view on that object. Objectsthat can be clicked are indicated by pulsating 2-D iconsthat appear over the 3-D object when the cursor ismoved over that object. Zooming the viewpointtowards or away from an object is done by pushing theappropriate buttons. The visuals can be displayed instereo, further enhancing the 3-D effect. To provideadditional depth cues, altitude lines and shadowsindicate the position of each aircraft projected onto theground. The entire database represents an area aroundBoston's Logan airport. The display runs at interactive

rates, varying from 5-20 Hz depending on whichvisualization features are activated.

Certain rendering techniques are used to achieveinteractive rates on a low-cost platform. Hidden-surface removal is performed primarily by using apainter's algorithm. Z-buffering is only applied to thesmall areas covered by the 3-D aircraft models.Textures are not supported in hardware, so grid linesare used instead to provide context for aircraft motionrelative to the ground. The total polygon and linecounts are kept low, and simplified models are used torender distant objects in a "level of detail" scheme.

• Spatial and Temporal Continuity: When the userchanges viewpoints, transitions are handled smoothly.Instead of immediately changing from the currentviewpoint to the selected new viewpoint, the systemautomatically interpolates between the old and newviewpoints. This transition takes 1-2 seconds and usesthe "slow in, slow out" metaphor that slows down therate of transition as the user leaves the initial viewpointand arrives at the new viewpoint. Smooth viewpointtransitions aid spatial awareness by helping the userretain context with respect to the overall situation.

• Object-Oriented Text: In existing ATC displays, 2-Dtext labels are attached to aircraft and other objects.

4


This program also supports that mode bysuperimposing 2-D labels on top of their associated 3-Dobjects. However, another way of displaying the sameinformation is to draw the text as 3-D objects near thelabelled object. Users may perceive 3-D text labels tobe more closely associated with their objects than with2-D text labels. To be readable, these 3-D labelsautomatically reorient and reposition themselves basedupon the user's viewpoint with respect to the object.That way, the user never has to read text that isbackwards or upside down. This is done by computingthe orientation of the object with respect to the currentviewpoint and adjusting the label orientation andposition appropriately. Figures 3 and 4 show a 3-Dlabel (the aircraft's call sign) attached to an aircraft asviewed from two different viewpoints, demonstratinghow the text automatically reorients itself to remainlegible. Figure 5 shows 3-D runway labels on themodel of Boston Logan.

Figure 3: 3-D text label attached to aircraft.

Figure 4: Reoriented 3-D text at a new viewpoint.

Figure 5: 3-D runway labels on Boston Logan.

One potential drawback to written information in 3-Dis reduced precision in reading values along any oneparticular axis19. With object-oriented text, theorientation of the text is modified based on the viewingangle, reducing the effect of this potential problem.However, another issue that must be addressed iswhether the text should remain at a fixed size or vary insize with respect to the distance from the viewpoint tothe text. In this demo, the text remains at a constantsize, but it could be changed to remain roughly constantin apparent screen area by scaling the 3-D font size.This prevents nearby text from occluding nearbyobjects and distant text from becoming illegible.Scaling the font size may also have a reinforcing effectwith other motion cues.

• Visualization of Flight Paths: In the future, aircraftequipped with GPS antennas and ADS communicationswill be able to broadcast their position and intendedpath to control towers and other aircraft. This leads tothe possibility of visualizing this information to helpforesee future conflicts. This demonstration displayspast and future flight paths through the use of a "ghostplane" metaphor. Ghost planes are projected future orpast versions of aircraft, drawn as wireframe objects toavoid confusion with the actual aircraft (see Figure 6).Users can adjust how far into the future or past theghost planes are projected. Since the flight paths for allaircraft are predetermined in this demo, both past andfuture projections use the actual paths. In reality, futureprojections would combine extrapolated trajectorieswith intent data. A wireframe line between the ghostplane and actual aircraft helps associate the two and theprojected course of the aircraft (Figure 7). The ghostplane information is drawn in four different modes:static, shooting, pulsing, and airspace. Static modesimply puts the ghost plane at the projected location.Shooting mode repeatedly "shoots" the ghost planefrom the actual aircraft to the projected position,following the projected path. Pulsing mode is similar,except that the ghost planes move forward andbackward in time in a sinusoidal pulsing motion. By

5


adding temporal motion, these two modes help userssee how groups of aircraft will move in space and time.Finally, airspace mode represents a "highway in thesky" by drawing a set of linked wireframe rings inspace that extend out as far as the projected ghost planeposition (Figure 8). The result is a tunnel in the skythat the aircraft appears to fly towards (or away from),with the tunnel constantly staying ahead (or behind) theaircraft. This is useful in conflict detection by showingwhether or not the airspace of two aircraft will intersectin the future, indicating a potential conflict.

Figure 7: Several ghost plane future paths.

Figure 8: Ghost plane airspace mode.

Figure 6: Ghost plane in front of actual aircraft.

• 3-D Spatial Audio Cues: The use of 3-D soundgeneration hardware allows the system to attach soundsto specific objects in space. This is more than juststereo. The sounds appear to emanate from their 3-Dlocations, giving an additional cue for the locations ofcertain objects. By offering multimodal displays, userscan keep track of the rough location of an object evenwhen it leaves the visual field of view. Since we do notperform head tracking in this system, the 3-D soundcomputations assume the user looks straight at theworkstation screen at all times. 3-D sound is used inthree separate modes in this demonstration.

First, it is used as an audio alert to identify nearbyaircraft. The program has a mode called "greenspace"that draws color-coded lines from the current aircraftbeing gazed at to other nearby aircraft: green for faraway, yellow for medium distance, and red for tooclose. 3-D audio alerts supplement these graphicwarnings, where the audio alerts are attached in spaceto the other nearby aircraft. 3-D audio alerts are anatural way of presenting warnings. An automobiledriver is usually first aware of the presence of anambulance by the wail of its siren and roughlyestimates the location of the ambulance using soundcues. Similarly, in experiments with TCAS II collisionavoidance logic, 3-D spatialized sound simulated withpilot headphones allowed pilots to acquire conflictaircraft targets approximately 2.2 seconds faster thancrew members who used monotone one-earpieceheadsets2.

The second 3-D sound mode attaches an audio sourceto the end of one of the runways at Boston Logan.Pilots can use this as an audio beacon for keeping trackof the location of a critical object in space (e.g. anairport location, a military no-fly zone, etc.) withouthaving to look at it.

The third and last 3-D sound mode demonstrates theability of 3-D sound to help users sort out confusingaudio situations. While the user looks out at the scenefrom inside the control tower at Boston Logan, four

6


conversations are played simultaneously. First, theconversations are played in mono so they all overlapeach other. Listeners find it extremely difficult to focuson any of the conversations. Then we spatialize thesounds, placing each of the four conversations on oneof the four main windows in the control tower (so theyare separated by 90 degrees from their neighbors). Asthe user rotates his view within the tower, theconversations appear to rotate around his head. Thistakes advantage of the "cocktail party" effect: theability to focus your hearing on a conversation thatcomes from a particular direction in space. Listenersfind it much easier to select one conversation and focuson that with spatialized sound.

Feedback and Lessons Learned

We have not run user studies to measure theeffectiveness of our ideas. Instead, we sought feedbackfrom domain experts (controllers and pilots) whoexperienced the demo1. We learned several lessons:

• Pilots and controllers provided different feedback:Most controllers liked the ability to choose a static 3-Dperspective view to watch the traffic from, but they didnot seem to care about the interactive navigationtechniques or most of the other features in the demo. Afew were adamant that the 3-D displays were bad ideasand too confusing. One controller did suggest the useof 3-D sound for directing ground traffic. By making apilot's voice appear to come from the direction of theaircraft's position on the runway, a controller issuingcommands would gain an additional spatial cue to helphim keep track of where the aircraft were on therunways, reducing the chance of issuing the wrong setof orders. Controllers suggested using these features intraining new controllers, to help them build a "mentalmodel" of the 3-D nature of the airspace. A fewsuggested it might speed up transition time whenchanging controllers at a station, where the incomingcontroller has to spend some time watching displays tounderstand the 3-D situation before he can relieve thecontroller on duty.

Pilots, on the other hand, liked most of the features,especially the visualization of future flight paths using"ghost planes" and the spatial cues for indicatingneighboring aircraft. Reaction to the 3-D sound wasmixed: some thought it would be useful, but othersfound it irritating. Military personnel thought 3-Ddisplays could be useful for communicating informationabout enemy aircraft, targets on the ground, etc. thatcame from other sensors, such as an AWACS aircraft.Pilots suggested using this type of display in "Free

Flight," when pilots will have more authority to choosetheir own flight path and resolve potential conflictswithout the direct intervention of controllers.

These different reactions are probably due to thedifferent requirements and biases in the twoprofessions. Controllers train on and use 2-D displayswith fixed viewpoints (no zoom capabilities) that helpthem quickly and consistently estimate criticaldistances, looking at the environment from anexocentric or "god's eye" view. This may cause a biasagainst 3-D displays with changing viewpoints. Pilotsare used to egocentric viewpoints that see theenvironment from the point of view of a specificaircraft, so they were more accepting of those displaymodes and the attempts to aid situational awarenessfrom the pilot's perspective.

• Weather and terrain features: This demo does notmodel weather or any terrain features. Both are factorssignificantly affecting air traffic and need to beconsidered for future visualization efforts.

• Motion is vital for 3-D sound: An automobile driverwho hears an approaching ambulance will rotate hishead to locate the siren’s origin. Without motion,people have a more difficult time locating the source ofa sound. Having controls that let the user rapidlychange the orientation of his viewpoint was importantfor making 3-D sound work. Ideally, head trackingshould also cue sound changes as the user turns hishead.

• Overload: Displays and interfaces must be designedto avoid information overload. While users can turn thevarious display features on and off individually, it isstill easy in this demo for the display to becomeconfusing. After listening to 3-D audio alerts for awhile, some users found them fatiguing and distracting.Audio alerts must be used more sparingly. Once thedriver finds the ambulance, he doesn't need the sirenwailing anymore. This demo has seven aircraft overall,four of which are assumed to be equipped with ADS.Even with these few, the display can become clutteredin some visualization modes. A more realistic scenariomight put 40-50 aircraft in the skies around the airport.Future versions of the demo need to be more sensitiveto the potential for information overload.

Free Flight Demonstration

Based on the direction of other ATC work we areperforming and the feedback from the domain experts,we are focusing on building visualization aids for Free

7


Flight. In March 1996, the FAA decided to begin theprocess of shifting the current air traffic control systemto a new policy of air traffic management called FreeFlight13. Free Flight shifts the emphasis from active topassive control, with a policy of intervention byexception14. Under certain circumstances, aircraft willbe allowed to select their own routes to theirdestinations, provided that they do not conflict withother aircraft. This freedom to set courses could resultin significant time and financial savings for commercialairlines. When potential conflicts occur, Free Flightpolicies emphasize having aircraft attempt to resolveconflicts by themselves, with air traffic managers (whooversee dynamically-defined airspace sectors)intervening only when required.

What constitutes a conflict? To ensure safety, noaircraft should penetrate another aircraft's ProtectedAirspace Zone (PAZ): a cylindrical region around eachaircraft with a 5 nautical mile radius and total height of2000 feet (Figure 9). To avoid penetrating a PAZ,aircraft can change velocity, heading, or altitude. TheTactical Alert Zone (TAZ) is a region around an aircraftthat defines the space where it is no longer possible toexecute a maneuver to avoid penetrating the PAZ. Forexample, see Figure 10. Aircraft B is inside aircraft A'sVelocity TAZ. That means even if both aircraft A andB cooperate and optimally change their velocities,penetrating the PAZ is inevitable due to limitations onaircraft performance. However, aircraft B is stilloutside aircraft A's Heading TAZ, so it is possible tochange headings and avoid penetrating the PAZ. In theworst case, if it is impossible to avoid penetrating thePAZ, then any available control (velocity, heading,altitude) must be used to maximize the miss distance.

Figure 9: Protected Airspace Zone (PAZ).

Figure 10: Tactical Alert Zones (TAZs).

The TAZ are complex, constantly changing shapes thatare not easily explained or communicated without theuse of 3-D graphics. Computing the shape of the TAZrequires knowledge of performance limits (maximumturn rate, maximum climb rate, etc.) and controllabilityand reachability constraints derived from the relativeequations of motion. These computations are describedin another paper10. The PAZ and TAZ are importantsafety regions that both pilots and air traffic managersmust clearly understand to perform conflict detectionand resolution in Free Flight. Both the PAZ and TAZare centered on a particular aircraft and move with theaircraft. While the PAZ does not change shape withtime, the TAZ will change depending on the relativepositions, headings, speeds and conditions of the twoaircraft. The TAZ cannot be easily generalized in termsof a heuristic that a pilot or air traffic manager can usein a non-visual mode. The best way to make use of thetheoretical and algorithmic backgrounds behindcomputing the TAZ is to display the TAZ regionsaround the aircraft in a visual manner.

To illustrate TAZ regions for heading controlmanuevering, we have created a three-aircraft tacticalconflict detection and resolution example. Threeaircraft, labelled A, B, and C, are headed on coursesthat will cause them to penetrate each other's PAZregions. These aircraft are too close to initiate speedcontrol manuevers and are currently being told to staywithin the same flight level by air traffic management.Figures 11-13 show what happens if the aircraft do notexecute any heading changes. Figures 14-19 show howthings change if the aircraft cooperate and adjust their

8


headings to avoid penetrating each other's safety zones.In this case, the conflict resolution maneuver performedby aircraft A and B leads to a secondary conflict andsubsequent resolution between A and C, allowing forthe safe passage of all three aircraft. In all these figureswe see the TAZ regions around aircraft B or C withrespect to possible penetration by aircraft A. Ghostplanes extended 30 seconds into the future show theintended paths of each aircraft. By displaying what willhappen with no course change, these visual displaysprovide advance warning of potential conflicts andallow the pilots and the air traffic managers tounderstand the control advice provided by the TAZregions. This three-aircraft scenario is a particularlydifficult multi-aircraft conflict that might occur underFree Flight. Investigating such scenarios provides away to research multiple aircraft conflict detection andresolution algorithms and visualization techniques toaid situational awareness and the presentation ofconflict resolution advisory information.

Figure 11: 3 aircraft scenario, no maneuvering.Time = 10 seconds.

Figure 12: 3 aircraft scenario, no maneuvering.Time = 38 seconds. Pilot A's view of impending

contact with aircraft B's Heading TAZ.

Figure 13: 3 aircraft scenario, no maneuvering.Time = 55 seconds. Aircraft A has penetrated

aircraft B's Heading TAZ.

9


Figure 14: 3 aircraft scenario, with maneuvering.Time = 33 seconds. Aircraft A and B both start

turning to avoid penetration of B's TAZ.

Figure 15: 3 aircraft scenario, with maneuvering.Time = 46 seconds. Aircraft A is almost finished

clearing aircraft B's TAZ.

Figure 16: 3 aircraft scenario, with maneuvering.Time = 80 seconds. Aircraft A has cleared aircraftB but is now approaching aircraft C. Aircraft A and

C begin cooperative turn to avoid penetration ofC's TAZ.

Figure 17: 3 aircraft scenario, with maneuvering.Time = 103 seconds. Aircraft A has just finished

clearing aircraft C's TAZ.

Notes on Figures 11-18: All three aircraft stay at34000 feet and are labelled A, B, and C on thediagrams. Altitude lines extend from the threeaircraft positions to the ground. Shadows for theTAZ regions and the “ghost plane” lines are drawnon the ground. The lines extending from theaircraft positions and terminating in an arrow arethe ghost plane lines. These visualizations areeasier to see on the workstation monitor becausethe user can interactively change his viewpoint inreal time and can see them as 3-D stereo images.

10


Figure 18: 3 aircraft scenario, with maneuvering.Time = 53 seconds. Pilot A's view just after

clearing aircraft B's TAZ.

Future Work

We have the following plans to continue developingvisualization and interface techniques for aiding FreeFlight:

• New hardware and software infrastructure: Theexisting demonstration is written in the C languageusing the GL graphics library. Consequently, addingnew features requires changing low-level code, which istime consuming. We will switch to a higher-levelsoftware toolkit that is built for 3-D display andinteraction, such as Sense 8's WorldToolKit orDivision's dVISE library. This should result in fasterchanges and more rapid prototyping. We alsoanticipate moving to a more capable hardware platform,such as an SGI Infinite Reality, to avoid restricting ourvisualization ideas to the performance limitations of ourSGI Indy. We will also build a version of the code thatshows images on a wide field-of-view immersive stereodisplay. Hughes Research Labs has a 160 degree field-of-view toroidal screen driven by three projectors.While not appropriate for pilots, the air traffic manageror AOC personnel of the future might stand in front ofsuch a display and take advantage of the large visualdisplay area provided. Such displays are sometimesreferred to as CAVEs6.

• Improved interaction: The existing mouse-basedinterface may be satisfactory for a workstation-baseddemo, but as we move to large-screen displays thisinterface will no longer be appropriate. Selectingobjects and changing modes should be allowed byissuing voice commands, rather than clicking objects onthe display. Speech recognition has progressed to thepoint where small vocabulary speaker-independentrecognizers, suitable for this task, are available on PCs.Head tracking will be required to correctly render the

scene and allow user gaze to select objects. We willalso track hand locations or handheld objects that theuser manipulates to allow gesture-based controlmechanisms.

• More realistic data: The current demonstration hasonly seven aircraft, flying routes that were not modelledaccording to existing flight paths or physicalsimulation. We will acquire more realistic databases ofan airspace with flight routes from real commercialaircraft. We must populate the airspace with moreaircraft, around 40 or 50, to more realistically simulatethe congestion that can exist around busy airports.

• Integration with conflict detection and resolution:Airspace congestion can lead to operator overloadproblems. To avoid this overload, filters will be usedso that the user will only see the information that iscritical. A pilot in Free Flight does not need to keeptrack of all neighboring aircraft, but rather only the onesthat threaten to result in a conflict. Conflict detectionand resolution algorithms can provide that informationwhile improved visualization reduces operatoroverload.

• More visualization modes: More work needs to bedone in exploring additional ways of representing theTAZ regions. What are the best ways to show the TAZfor heading, velocity, and altitude changes? Thevisualizations might be different for each type of zone.

References

[1] 40th Annual Air Traffic Control AssociationConference. (Las Vegas, NV, 10-14 September1995).

[2] Begault, D. R. Head-Up Auditory Displays forTraffic Collision Avoidance System Advisories: APreliminary Investigation. Human Factors 35 , 4(December 1993), 707-717.

[3] Brown, Mark A. On the Evaluation of 3D DisplayTechnologies for Air Traffic Control. QueenMary & Westfield College Dept. of ComputerScience Technical Report No. 682 (16 August1994).

[4] Brown, Mark A. 3D Display Technologies forAir Traffic Control: A Pilot Study. Queen Mary& Westfield College Dept. of Computer ScienceTechnical Report No. 690 (12 December 1994).

[5] Burnett, Meridyth Svensa and Woodrow Barfield.Perspective versus plan view air traffic controldisplays: Survey and empirical results.Proceedings of Human Factors Society 35thAnnual Meeting (1991), 87-91.

11


[6] Cruz-Neira, Carolina, Daniel J. Sandin, andThomas A. DeFanti. Surround-Screen Projection-Based Virtual Reality: The Design andImplementation of the CAVE. Proceedings ofSIGGRAPH '93 (Anaheim, CA, 1-6 August 1993),135-142.

[7] Daily, Michael J. and Jimmy A. Krozel. Human-Centered Interfaces for Air Traffic Control.Proceedings of the 40th Annual Air TrafficControl Assoication (ATCA) Conference (LasVegas, NV, 10-14 September 1995), 215-219.

[8] Ellis, Stephen R. and Michael W. McGreevy.Perspective Traffic Display Format and AirlinePilot Traffic Avoidance. Human Factors 29, 2(August 1987), 371-382.

[9] Haskell, I.D. and Christopher D. Wickens. Two-and Three-Dimensional Displays for Aviation: ATheoretical and Empirical Comparison.International Journal of Aviation Psychology 3, 2(1993), 87-109.

[10] Krozel, Jimmy, T. Mueller and G. Hunter. FreeFlight Conflict Detection and ResolutionAnalysis. To be published in Proceedings ofAIAA Guidance, Navigation and Controlconference (San Diego, CA, 29-31 July 1996).

[11] May, Patricia A., Margaret Campbell andChristopher D. Wickens. Perspective Displays forAir Traffic Control: Display of Terrain andWeather. Air Traffic Control Quarterly 3, 1(1995), 1-17.

[12] Moller, H. and G. Sachs. Synthetic Vision forEnhancing Poor Visibility Flight Operations.IEEE AES Systems Magazine (March 1994), 27-33.

[13] Nomani, Asra Q. FAA to Let Pilots ChangeFlight Paths. The Wall Street Journal (15 March1996), A3.

[14] Nordwall, Bruce D. Free Flight: ATC Model forthe Next 50 Years. Aviation Week & SpaceTechnology (31 July 1995), 38-39.

[15] Pruyn, Peter W. and Donald P. Greenberg.Exploring 3D Computer Graphics in CockpitAvionics. IEEE Computer Graphics andApplications (May 1993), 28-35.

[16] Scardina, John A., Theodore R. Simpson, andMichael J. Ball. ATM: The only constant ischange. Aerospace America (March 1996), 20-23and 40.

[17] Smith, J.D., Stephen R. Ellis and Edward Lee.Avoidance Manuevers Selected While ViewingCockpit Traffic Displays. NASA Ames ResearchCenter technical report TM-84269 (October1982), 34 pages.

[18] Ventrella, Jeffrey and Robert McNeil. MITMedia Laboratory, personal communication(1994).

[19] Wickens, C. D., S. Todd and K. Seidler. Three-Dimens iona l D i sp l ays : Pe rcep t ion ,Implementation, and Applications. University ofIllinois at Urbana-Champaign Aviation ResearchLaboratory technical report ARL-89-11 (1989).

[20] Zeltzer, David and Steven Drucker. A VirtualEnvironment System for Mission Planning.Proceedings of the IMAGE VI conference(Scottsdale, AZ, 14-17 July 1992), 125-134.

Documents

ADVANCED HUMAN-COMPUTER Dependent Surveillance (ADS ... · aircraft, direct the traffic, and have the final word in all decisions. Therefore, how computer systems present information