Tactical Tomahawk Weapon Control System

User Interface Analysis and Design

A Thesis

Presented to

The Faculty of the School of Engineering and Applied Science

University of Virginia

In Partial Fulfillment

Of the Requirements for the Degree of

Master of Science in Systems Engineering

Submitted by

Robert A. Willis

May 2001

Approval Sheet

This thesis is submitted in partial fulfillment of the requirements for the degree of Master

of Science in Systems Engineering.

Robert A. Willis

This thesis has been read and approved by the examining committee:

William T. Scherer, Chair

Stephanie A. Guerlain, Advisor

Thomas E. Hutchinson, Co-advisor

Wayne L. Harman, NSWC

Bruce R. Copeland (Ph.D), NSWC

Accepted for the School of Engineering and Applied Science:

Dean, School of Engineering

and Applied Science

i

Acknowledgements

Many thanks go to my principal advisor, Dr. Stephanie Guerlain, as well as Dr. Tom

Hutchinson and Dr. Christina Mastrangelo. I have certainly learned that the design

process thrives in a team environment, and I must thank my capstone team members,

Julie Besselman, Meredith Logsdon, Brian Whisnant, and Tim Yewcic. I would like to

thank the guys at Dahlgren: Wayne Harman, Bruce Copeland, Alan Thomas, and Bob

Athay for their mentoring, reality checks, and perhaps most of all, for their money. I

must thank Tom and Adam at Creative Perspectives, Inc. Indeed, the relationship

between our designer team and their programmer team provided some of the most

valuable lessons in the project, as well as a prototype at well below any reasonable cost.

Finally, I must thank Melissa, whose love, support, and encouragement I have relied

upon greatly over the course of the project.

ii

Executive Summary

Military personnel are continually faced with new and more advanced forms of

automated technology. Of interest is the design of command and control (C2) systems

that allow personnel to remotely monitor not only an emerging battlespace, but also the

numerous semi-autonomous vehicles and weapons currently being developed. The U.S.

Navy is currently developing its next-generation cruise missile, the Tactical Tomahawk,

which improves upon current versions by its ability to be retargeted in flight against

emergent time-critical targets. These targets include mobile surface-to-air missile

systems, surface-to-surface ballistic missile launchers (Scud), and other high-value

relocatable targets. Based partly on experience during Operation Allied Force (1999), the

military services have established a goal of ten minutes to service these types of targets

once identified. Tactical Tomahawk expands the list of weapons available for this

purpose, and adds to the Navy�s arsenal of Land Attack Warfare systems for integration

into the revolutionary design of the DD 21 destroyer. Further, the Navy's Director of

Surface Warfare recently focused weapon system development on the operator, stating,

�...our ability to successfully and effectively employ Land Attack Warfare systems will

directly reflect our commitment to Human Centered Design...[and]...Human Systems

Integration� (Mullen, 2000).

In this study, we developed an operator interface prototype for the Tactical Tomahawk

Weapon Control System (TTWCS) and empirically tested the effect of mission

complexity on the ability of operators to maintain situational awareness in various

operational scenarios. The first phase of research involved a domain analysis of three

iii

primary domains: the weapon system; time-critical decisionmaking; and principles of

interface design. The second phase was a concurrent Cognitive Work/Task Analysis

(CW/TA) and Display Development effort. For the CWA, we first profiled the user

population and then analyzed the tasks, using scenarios to decompose the system into

objects & properties, a task flowchart, and a formal list of functional requirements.

Major functional requirement headings include the following: 1.) Monitor and

communicate status, 2.) Conduct queries to the system, 3.) Develop and modify plans,

and 4.) Facilitate retargeting decisions. Next, tasks and system properties were

synthesized into prototypes of individual display features as well as a system operator

interface. We iterated on this development process to produce three prototypes of

increasing fidelity. The final version of the system interface is dynamic and interactive,

and incorporates varying degrees of automation for different tasks. As a separate but

relevant project, Appendix G proposes a methodology and program for rapidly analyzing

terrestrial constraints on missile terminal dive angle. The program is implemented in

ArcView GIS (geographical information system), and could be used by targeting cell

personnel to instantly assess whether a target is suitable for Tomahawk attack with

respect to required attack heading and dive angle.

In the final phase, we trained and tested twenty graduate students on the dynamic and

interactive prototype, based on hypotheses pertaining to both monitoring and retargeting

tasks. Statistical results support two primary conclusions. First, operators can maintain

adequate situational awareness when monitoring eight missiles and twelve targets

simultaneously. Second, results support the use of the missile timebar feature in the

interface to compare events. Subjective results indicate the requirement for a robust

iv

decision support tool to facilitate rapid retargeting decisions. The tested prototype was

subsequently expanded to include such a tool. These and other results form the basis for

recommendations to the Naval Surface Warfare Center, Dahlgren Division about how to

most effectively allocate personnel resources in the designing of a command and control

watchstation for the Tactical Tomahawk cruise missile system.

v

Table of Contents

1 Problem Statement.................................................................................................... 1

2 General Approach..................................................................................................... 5

3 Domain Study............................................................................................................ 7

3.1. Cruise Missile Weapons ..................................................................................... 7

3.1.1. ATWCS 3.1.2. TTWCS

3.2. Time Critical Targeting..................................................................................... 11 3.3. Principles of Interface Design........................................................................... 14

3.3.1. System Complexity 3.3.2. Situational Awareness 3.3.3. Automation and Decision Support

4 Interface Design ...................................................................................................... 27

4.1. Introduction....................................................................................................... 27 4.2. User Analysis .................................................................................................... 28 4.3. Task Analysis.................................................................................................... 30

4.3.1. Object Definition 4.3.2. Retargeting Task Flowchart 4.3.3. Scenarios 4.3.4. Interface Functional Requirements

4.4. Prototype Development .................................................................................... 43

4.4.1. Initial Low-Fidelity Design & Testing (Version 1) 4.4.2. Component Prototypes 4.4.3. System Prototypes

5 Subject Testing & Analysis .................................................................................... 86

5.1. Experimental plan ............................................................................................. 86

5.1.1. Hypotheses 5.1.2. Experimental Design 5.1.3. Method 5.1.4. Online Measures 5.1.5. Scenario Description 5.1.6. Participants

vi

5.2. Results............................................................................................................... 94

5.2.1. Objective Results 5.2.2. Subjective Results

6 Conclusions............................................................................................................ 101

6.1. Design Iteration............................................................................................... 101 6.2. Operator Workload ......................................................................................... 101 6.3. Level of automation ........................................................................................ 102 6.4. Location of Interface....................................................................................... 103 6.5. Application to Other Fields............................................................................. 104

References...................................................................................................................... 106

Appendix A Glossary

Appendix B Object Definition

Appendix C Scenarios

Appendix D Functional Requirements

Appendix E Emergent Target Decision Support Tool Algorithm

Appendix F Storyboard

Appendix G Terrestrial and Tactical Constraints on Missile Dive Angle

Appendix H ANOVA Results

vii

List of Tables Table 3-1 Levels of Automation��������������������.23

Table 4-1 Critical Target Property Definitions���.�����������. 33

Table 4-2 Critical Missile Property Definitions �������������� 33

Table 4-3 Object-Property Database������������������.. 34

Table 4-4 Rigid Retargeting Constraints����������������� 37

Table 4-5 Retargeting Minimization Objectives���������.����... 37

Table 4-6 Mission Essential Functional Headings�������������. 42

Table 4-7 Prototype Versions and Testing��������������..�.. 44

Table 4-8 Component Prototypes�������������������... 47

Table 4-9 Target Symbol Fill Color��������������� ���...49

Table 4-10 Target Symbol Graphic Modifiers�����������.. ...���50

Table 4-11 Missile Symbol Frame Style������� ����������..52

Table 4-12 Missile Symbol Fill Color�����������.�������. 53

Table 4-13 Missile Symbol Fill Color Conflict���������������54

Table 4-14 Missile Symbol Graphic Modifiers��������������� 55

Table 4-15 Properties Incorporated into Timebar Component��������� 61

Table 4-16 Message Color Convention������������������ 65

Table 4-17 Coverage Zone Time Factors�����������.������ 66

Table 4-18 Default Weights in Retargeting Scheme������������. ...70

Table 4-19 Prototype Fidelity����������������.�����..74

Table 5-1 Experimental Design��������������� �����..87

Table 5-2 Control Variable Combination��������������� �...88

Table 5-3 Group Testing Sequence��������������� ����89

Table 5-4 Accuracy Scale����������������������...91

viii

List of Figures Figure 2-1 Phases of Research��������������������.. 6

Figure 3-1 Tactical Tomahawk Missions����������������.. 9

Figure 3-2 JEFX 2000 Time-Critical Targeting (TCT) Process������.�... 13

Figure 3-3 Levels of Situational Awareness��������������.�. 19

Figure 4-1 Design Approach������������.��������� 27

Figure 4-2 Object-Property-Value Thread���������������� 31

Figure 4-3 Properties of Target and Missile Objects������������ 32

Figure 4-4 Retargeting Task Flowchart�����������������. 39

Figure 4-5 Sample Retargeting Scenario���������������.�.. 41

Figure 4-6 Steps in TTWCS System Prototype Development��������.. 43

Figure 4-7 System Prototypes (Version 1)����������������.. 45

Figure 4-8 Target Text Details��������������������.. 51

Figure 4-9 Target Symbol Examples������������������. 51

Figure 4-10 Missile Text Details��������������������. 56

Figure 4-11 Missile Symbol Examples������������������ 56

Figure 4-12 Missile Routes and Health & Status Point�����������... 58

Figure 4-13 Missile-Target Assignments����������������� 59

Figure 4-14 Missile Timebar Component����������������� 62

Figure 4-15 Timebar Section (4 Missiles)����������������� 62

Figure 4-16 Emergent Target Time Attributes, Two Candidate Missiles����� 64

Figure 4-17 Coverage Zones at 9-Minute Time Factor�����������... 67

Figure 4-18 Emergent Target Decision Support Tool������������. 71

Figure 4-19 Initially Proposed Workstation����������������. 73

Figure 4-20 Interactive Prototype (Version 2)��������������� 75

ix

Figure 4-21 Use of Storyboarding in Prototype Development��������� 76

Figure 4-22 Interactive, Dynamic User-Testing Prototype (Version 3)�����. 79

Figure 4-23 Flex Targeting Event�������������������� 81

Figure 4-24 Retarget Mode Popup Window���������������� 82

Figure 4-25 Emergent Targeting Event (Prototype Version 4)�..�������.. 83

x

List of Acronyms

APS Afloat Planning System ATWCS Advanced Tomahawk Weapons

Control System AUR All-Up-Round AWACS Airborne Warning and Control

System AWC Air Warfare Coordinator BB Blended Body (subsonic missile) BDI Battle Damage Indication BDII BDI Imagery BFTT Battle Force Tactical Training CAOC Combined Air Operations Center C2 Command and Control CCS Combat Control System CLAWS Common Land Attack Warfare

System CMSA Cruise Missile Support

Activities CNO Chief of Naval Operations COE Common Operating

Environment COI Critical Operational Issue COP Common Operating Picture CTF Coverage Time Factor CTA Cognitive Task Analysis CWA Cognitive Work Analysis DII Defense Information

Infrastructure DoD Department of Defense DPPDB Digital Point Positional Data

Base DSMAC Digital Scene Matching Area

Correlation DTED Digital Terrain Elevation Data

Base ETDST Emergent Target Decision

Support Tool FDC Fire Direction Center FPPWP First Pre-planned Waypoint GPS Global Positioning System H&S Health and Status HAM Horizontal Attack Maneuver HCI Human-Computer Interaction IADS Integrated Air Defense Systems ICE Integrated Command

Environment

ILSP Integrated Logistics Support Plans

IMMM Inflight Missile Modification Message

INS Inertial Navigation System IOC Initial Operational Capability JMPS Joint Mission Planning System JSCP Joint Strategic Capabilities Plan LAM Land Attack Missile LARIAT Loiter And Router Interactive

Analysis Tool LOC Loiter opportunity cost LPA Littoral Penetration Area LPMP Launch Platform Mission

Planning LPZ Littoral Penetration Zone LWC Land Warfare Coordinator MDS Mission Distribution System METOC Meteorology and Oceanography MML Master Mission Library MSC Military Sealift Command MNS Mission Need Statement MPT Manpower, Personnel, and

Training MRL Mobile Rocket Launcher MTBOMF Mean Time Between Operational

Mission Failures NBCC Nuclear, Biological, and

Chemical Contamination NFCS Naval Fires Control System NSFS Naval Surface Fire Support NTSP Navy Training System Plan ORD Operational Requirements

Document PEO(W) Program Executive Office for

Strike Weapons and Unmanned Aviation

POC Priority Opportunity Cost PPDB Point Positional Data Bases PTAN Precision Terrain Aided

Navigation QSP Quick Strike Planner QST Quick Strike Tomahawk RDS Rapid Deployment Suites RODS Rapid Ordnance Deployment

System

xi

SAASM Selective Availability Anti-Spoofing Module

SAM Surface-to-Air Missile TCT Time Critical Targeting TC2S Tomahawk Command and

Control System TCOMMS Tomahawk Communications

System TERCOM Terrain Contour Matching TIIR Time of Impact if Retargeted TFTA Tomahawk Fleet Training Aid TLAM Tomahawk Land Attack Missile TMPC Theater Mission Planning Center

TRB Tomahawk Record Book TRPPM Training Planning Process

Methodology TT Tactical Tomahawk TTWCS Tactical Tomahawk Weapon

Control System TWS Tomahawk Weapon System UAV Unmanned Aerial Vehicle UCAV Uninhabited Combat Air Vehicle ULSS Users� Logistics Support

Summary VLS Vertical Launch System WMD Weapons of Mass Destruction

1

1 Problem Statement

Time-critical decisionmaking in complex, dynamic systems is among the most elusive

and most studied areas in the military today. The notion of decisionmaking, issuing

orders and guidance, and supervising falls under the broad field of command and control

(C2). U.S. Air Force General John P. Jumper, Air Combat Command chief, stated during

a recent military experiment that, �Command and control is our number one priority to

explore, and dealing with time-critical situations is a key element of C2� (Scott, 2000).

All of the U.S. military services are wrestling with the need to find, identify, and destroy

�time-critical targets� in an air campaign or coordinated air/ground war, with a goal of

less than ten minutes to complete this process (Fulgham & Wall, 2000a, 2000b, 2001a).

These increasingly complex high-value and high-priority targets include the most

dangerous mobile air defense systems (SA-10, SA-12), ballistic missile launchers,

armored forces, and logistics columns. Efforts by the U.S. Air Force to detect, locate,

and strike these targets have shown that a new command and control �concept and

process are ready for fielding, but the [weapon] systems are another issue� (Scott, 2000).

Conventional systems employed against time-critical targets include manned high-

performance aircraft such as F-16, F-15E, and F-117, as well as surface-to-surface

weapons such as the Army Tactical Missile System (ATACMS). Recent experiments

and research show a move toward uninhabited weapon platforms: The Predator

unmanned aerial vehicle (UAV), which is used predominantly for reconnaissance,

successfully launched laser-guided missiles against ground targets in February, 2001

2

(Baker, 2001). The primary mission of the Air Force�s and Navy�s uninhabited combat

air vehicles (UCAV) will be offensive air-to-ground and air-to-air attack (Fulgham 2000;

Fulgham, 2001). Until now, however, cruise missiles have not been considered a viable

option to service time-critical targets. The U.S. Navy�s Tactical Tomahawk system

merges missile and data link technologies to create a formidable on-call weapon, further

expanding the arsenal suitable for time-critical targets. It also introduces command and

control challenges not yet faced in the domain of cruise missile warfare. Development of

a command and control architecture for these missiles is ongoing (LoPresto & Rice,

2000). Of the many components in the Tactical Tomahawk Weapon Control System

(TTWCS, pronounced �tee-twicks�), the human-system interface is currently being

developed by Lockheed-Martin (Lockheed-Martin, n.d.).

In his Surface Combatant Land Attack Warfare Guidance Document, the Navy�s Director

of Surface Warfare plainly stated that, �techniques that reduce the time delays inherent in

the kill chain should be foremost in our minds� (Mullen, 2000). He focused weapon

system development on their operators, noting that the �ability to successfully and

effectively employ Land Attack Warfare systems [which include TTWCS] will directly

reflect our commitment to Human Centered Design [and] Human Systems Integration�

(ibid.). Only proper design of decision support displays for the TTWCS operator will

bring the system to full maturity, enabling it as a time-critical targeting weapon. For the

TTWCS operator, time-critical decisionmaking will excel based on his/her ability to

access and interpret information, and to execute missions in a minimum amount of time.

In research into optimum manning considerations for the Navy�s Zumwalt-class 21st

Century Destroyer, (designated �DD 21�), Campbell, Pharmer, and Hildebrand (2000)

3

state that �improved control and display interfaces have the potential to free the operator

of rote and mundane tasks, and provide more time for the higher level tasks of planning,

predicting, and critical thinking.�

The client for the project is Department K, Naval Surface Warfare Center � Dahlgren

Division (NSWCDD). Their central questions included the following:

• How many missiles can one TTWCS operator reliably control?

• What is the appropriate level of automation for the system?

• Where should the interface be located?

o Launch vessels

o Battlegroup command vessel

• At what level of responsibility should a retargeting decision be made?

Requirements in the research included:

• Develop TTWCS display concepts with a goal of minimizing mission

execution time while maximizing operator situational awareness.

• Test the interface as a decision support system enabling rapid (single-digit

minutes) missile-to-target matching and mission execution.

• Provide display recommendations to the client for use in future contracted

TTWCS user-interface development.

It is important to note that while a team of four undergraduate systems engineering

students augmented the study, the author either personally conducted or supervised each

step of the work presented within this document. It is equally important to point out that

4

concurrent work by Lockheed-Martin involves integrating TTWCS into existing Naval

vessels. At the time of publication of this document, Lockheed-Martin�s operator

interface had recently been completed as part of its Phase 1 program development. This

interface was never reviewed by the UVa design team, and there was no communication

between the two groups regarding interface design. Any similarities in appearance and

functional requirements between the two designs should not be interpreted as

collaboration. While the results of the study presented here could be integrated into

Lockheed-Martin�s Phase 2 (which adds increased automation and expansion to the

Navy�s Common Land Attack Warfare System (CLAWS) for initial operating capability

not earlier than 2003 (Operational Requirements Document, 1998)), the findings may be

more significant in the context of the development of the DD 21 combat information

center.

As a separate but relevant project, Appendix G proposes a methodology and program for

rapidly analyzing terrestrial constraints on missile terminal dive angle. The program is

implemented in ArcView GIS (geographical information system), and could be used by

targeting cell personnel to instantly assess whether a target is suitable for Tomahawk

attack with respect to required attack heading and dive angle.

5

2 General Approach

The following Thesis Statement introduces the research approach taken to address the

problem.

The application of a human-centered systems approach to the task analysis

of the operator and the functional requirements and design of the system

will produce an operator interface enabling efficient accomplishment of

time-critical tasks.

The approach conforms to the Systems Analysis method outlined by Gibson (1991). The

final recommendations represent a transition to the �normative scenario� (the desired

future state (ibid.)) in which the TTWCS interface is effective in the time-critical

decisionmaking process. This broad method includes setting goals, developing and

comparing alternatives, and iterating. While the overall design and testing process was

iterated twice during the 13-month study, an iterative approach was also used in �nested

loops� within the overall project, including the stages of Object Definition, Scenario

development, Component Prototype development, System Prototype development, and

Subject Testing. We generalized the problem to include a core of math quantification,

human factors considerations, and a policy component. We focused our efforts around

the human factors of the most immediate stakeholder in such a system � the Land

Warfare Coordinator in the CIC.

6

The research was conducted in four phases (Figure 2 -1). The first phase was a Domain

Analysis of three primary areas: cruise missile weapons, time-critical targeting, and

principles of interface design. Second, potential scenarios were developed to identify

functional requirements. Next, interface components were developed and synthesized

into a prototype. Finally, twenty subjects were trained and tested on representative tasks

in a set of scenarios. We used objective and subjective measures in the testing, and

analyzed the results for significance.

Figure 2-1 Phases of Research

7

3 Domain Study

3.1. Cruise Missile Weapons

The Tomahawk cruise missile system has given our military forces a formidable tool for

use against many types of stationary enemy targets. Previously, targets deep in enemy

territory would have to be attacked by manned aircraft or special operations personnel.

With the addition of the Tomahawk missile system to the arsenal in the 1980�s, these

targets can be attacked with precision from safe distance (up to a thousand miles) thereby

reducing the threat to friendly servicemen. The capabilities and advantages of this system

were greatly exploited in Operation Desert Storm (1991), during the strikes against

Afghanistan and Sudan in 1998, and in Operation Allied Force in Serbia (1999) (Voth,

2000). As with many weapon systems, the Tomahawk has seen several variants since its

initial operating capability. The currently deployed Advanced Tomahawk Weapon

Control System (ATWCS) uses the Block III Tomahawk missile variant and has been

deployed since 1998. The Tactical Tomahawk Weapon System (TTWCS) adds unique

functionality to the weapon system, and is expected to upgrade ATWCS beginning in

2003 (Operational Requirements Document, 1998).

3.1.1. ATWCS

The U.S. Navy�s current cruise missile (the Block III Tomahawk Land Attack Missile) is

launched from either a ship or a submarine, and flies a programmed route to a

predetermined target. Target and route planning is a complex process conducted in

Hawaii and Virginia, with some limited capability on aircraft carriers (Pike, 1997).

8

Target and route data is downloaded to the missile on the vessel prior to launch, and the

operator of the Advanced Tomahawk Weapon Control System (ATWCS) cannot modify

or abort a mission once the missile is enroute. The system is best used against �high-

value fixed targets such as electricity generating facilities and command & control nodes,

weapons assembly and storage facilities� (Operational Requirements Document, 1998).

The ATWCS operator works in a Windows computer environment with a mouse, instead

of the green, push-button display inherent in prior versions.

3.1.2. TTWCS

Because of the huge success of early and current Tomahawk versions, the Navy initiated

the Tactical Tomahawk Weapons Control System (TTWCS) program in December, 1997

for Phase 1 initial operating capability in 2003 (Operational Requirements Document,

1998). Its broad operational requirements include the following:

• Enable rapid tactical mission planning on board the launching vessel.

• Enable the missile to �loiter� near the default target area to decrease the

time required to attack preplanned alternate (flex) targets as well as newly

identified emergent targets.

• Enable in-flight reprogramming of the missile (to change route and/or

target).

• Facilitate the monitoring of missile health and location on the Common

Operating Picture (COP).

9

The schematic in Figure 3-1 depicts the three types of Tactical Tomahawk missions (the

figure is not a proposed operator display). The class of the target to which the missile is

destined defines the missions; the mission type may change during flight. Target classes

(and thus mission types) include default, flex, and emergent (or d-, f-, and e-targets). A

missile is launched along a path initially toward its d-target. While enroute, the operator

can command the missile to abort its default mission, and execute one of several pre-

programmed flex missions. The operator can also respond to the appearance of an e-

target by rapidly programming and transmitting to an inflight missile a completely new

route and aimpoint. The emergent target class is of most interest due to the flexibility

and time-critical decisionmaking required to service it.

The Tactical version of the missile offers �improved lethality against a wider target set,�

and can meet the potential requirement to �provide support for ground forces�

(Operational Requirements Document, 1998), a mission never considered for cruise

missiles. The U.S. Marine Corps has a 2.5-minute response time standard for Naval

Figure 3-1 Tactical Tomahawk Missions (revised from LoPresto and Rice, 2000)

10

Surface Fire Support (NSFS) (Mullen, 2000), and the TTWCS could meet that goal if the

command and control and human-systems integration (HSI) challenges are solved.

The Operational Requirements Document (1998) describes additional functions of

TTWCS over ATWCS including a two-way satellite data link (SDL) to transmit battle

damage imagery, communicate missile health and status, and to transmit inflight missile

modification messages (IMMM). The IMMM results in a missile�s changing of mission

to either flex or emergent. The requirements document states that the system will �be

able to simultaneously control up to 4 ownship-launched missiles, and be able to retarget

each at least once∗ .�

One of the major challenges introduced with the TTWCS concerns the operator�s ability

to maintain situational awareness when multiple missiles are being controlled, and to

rapidly make retargeting decisions given specified criteria. Some of the criteria, such as

the opportunity cost of diverting a missile to one of its flex targets, are somewhat

subjective. Others, such as missile fuel remaining and target priority can be logically

compared, suggesting a decision process that is to some degree automated. Regardless of

the level of automation, the ability to rapidly select missiles for retargeting while

maximizing the coverage for future e-targets and minimizing opportunity costs will hinge

largely on the effectiveness of the displays (Endsley & Kiris, 1995).

∗ Because the number of missiles that a controller can effectively monitor and control was a primary

question from the client, the prototype and testing scenarios were not limited to four missiles. However, to

simplify prototype development as well as subject training, the prototype allows for the retargeting of a

missile not more than once.

11

3.2. Time-Critical Targeting

The military�s joint publication for the targeting process defines a time-critical target

(TCT) as �a lucrative, fleeting target of such high priority to friendly forces�as requiring

immediate response� (FM 90-36, 1997). Time-critical targets include mobile rocket

launchers (MRL), mobile high-threat surface-to-air missiles (SAM), theater ballistic

missiles, mobile weapons of mass destruction (WMD), and mobile command and control

vehicles. The services� 10-minute goal is derived primarily from the time it takes an

enemy mobile missile launcher to escape after firing. In this thesis, the terms �time-

critical target� and �emergent� target are synonymous. Servicing these fleeting but

extremely critical targets requires rapid and precise coordination between numerous

airborne and ground-based agents. The chain of events between target identification and

attack is significant enough that it may offer the TTWCS operator and/or system only

seconds to decide which missile to retask for time-critical missions (Athay, 2000). The

time constraint and information requirements call for an automated or semi-automated

decision support system. To identify specific requirements for such a tool, it is important

to first define the TTWCS �subsystem� within the boundaries of the overall air campaign

command and control process that facilitates time-critical targeting.

In coordinated air campaigns such as those carried out in Operation Desert Storm (1991),

Operation Deliberate Force (1995), and Operation Allied Force (1999), cruise missiles

share the battlespace with a range of joint (sister-service) and combined (allied country)

assets. These include low-flying attack and rescue helicopters, air-to-ground and air-to-

air fighters, unmanned aerial vehicles, surface-to-surface rockets, command and control

12

aircraft, medium and high altitude reconnaissance aircraft, and orbiting satellites (Owens,

1997). As a retargetable system, TTWCS brings the cruise missile into the realm of on-

call weapons available to an air campaign or ground commander for prosecution of

emergent and relocatable targets (Scott, 2000). Interface development must therefore

consider from whom an attack order is received, and in what format.

A U.S. Air Force exercise recently surfaced relevant command and control issues within

the larger context in which the TTWCS (and other land attack systems) will operate. The

September 2000 Joint Expeditionary Force Experiment (JEFX 2000) demonstrated a

targeting command and control system that can �consistently detect, geolocate, and strike

time-critical targets�in a matter of minutes� (Scott, 2000). Results indicate that

�feeding information through a time-critical targeting cell and matching on-call weapon

systems to specific targets greatly reduces the time needed to eliminate �pop-up� targets�

(Scott, 2000).

13

As depicted in Figure 3�2 , the Time-Critical Targeting cell performs the following

functions, concluding with the issuing of an attack assignment to a weapon operator:

• Detect � manage and exploit target collection efforts

• Decide � prioritize time-critical targets; generate essential target data

• Deliver � assign on-call weapon systems to attack selected targets

The TTWCS operator will thus receive retarget missions from the TCT cell. Current Air

Force plans involve locating the TCT cell with the next-generation combined air

operations center (CAOC-X) at Langley Air Force Base, Virginia, regardless of the

location of the actual area of operations (Fulgham, 2001). The Navy has initiated a

program named Hairy Buffalo which attempts to focus the same capabilities of the TCT

Figure 3-2 JEFX 2000 Time-Critical Targeting (TCT) Process (prepared from explanation provided in Scott, 2000)

14

cell into a single aircraft in the vicinity of the area of operations (Fulgham & Wall,

2001a).

Understanding this targeting process is important in the development of the TTWCS

interface. The communication between the targeting cell and the TTWCS launch vessels

(ship or submarine) initially appears one-way: the targeters issue an attack order to the

vessel with a �data block� that includes target location, description, priority, and various

time attributes describing when the target can or should be attacked. The Tomahawk

operator then carries out the mission by either selecting one of its inflight missiles to

retarget, or in some cases by rapidly planning and executing a mission with a missile not

yet launched. Another scenario includes the possibility that the targeting cell or air

operations center might �ask� if a vessel could accomplish a certain mission. The

displays must therefore facilitate rapid analysis of �what-if� scenarios to determine the

acceptance/rejection of a possible time-critical mission.

3.3. Principles of Interface Design

There are numerous references in the area of interface design. The following sections are

a synopsis of design principles relevant to the development of the TTWCS interface

prototype. The System Complexity section lists common sources of complexity in

generic socio-technical systems, and discusses their relevance in the context of the

TTWCS interface. The Situational Awareness section summarizes issues pertaining to an

operator�s ability to perceive and predict events in the working environment. Finally, the

Automation and Decision Support section discusses techniques to improve situational

15

awareness and performance in complex systems, and sets the stage for the development

of several central components and functions of the interface prototype.

3.3.1. System Complexity

Vicente (1999) offers a list of interrelated characteristics which combine to create the

multidimensional concept of complexity. This section provides a description of each of

these sources of complexity with respect to the Tactical Tomahawk system. The

explanations address both the immediately relevant Tactical Tomahawk system, as well

as the broader system of time-critical targeting as described in Section 3.2.

3.3.1.1. Large Problem Space

Command and control of the TTWCS system is made complex partially by the large

number of variables that must be considered by the operator to make retargeting

decisions. Given the location of an emergent target and the order to service it, the

decision of which candidate missile to dispatch may appear to depend solely on the

distance from the individual candidate missiles to the target. A complete analysis,

however, should consider the locations and loiter coverage areas of all other candidates,

the fuel/flight time remaining per missile, the priority of both the default and flex targets

that would be given up for each candidate missile, locations of no-fly or threat areas, and

the probability of appearance of emergent targets across the area of responsibility.

16

3.3.1.2. Heterogeneous Perspectives

Many people and organizations are involved in the targeting process, and each may have

different objectives. For example, there may be a difference in opinion on the priority of

targets between an Army unit, which has personnel on the ground near potential flex or

emergent targets, and the Air Force, whose near-term mission is to suppress enemy air

defenses. The assignment of priority to targets is external to the TTWCS system, and

only the final decision should filter to its operator. However, the existence of possible

conflicting objectives is relevant when encompassing the targeting cell within the

boundary of the TTWCS system as discussed in Section 3.2.

3.3.1.3. Distributed Decision

A geographical displacement between stakeholders in the time-critical targeting process

adds to the complexity of inflight retargeting. The combined air operations center for

Operation Allied Force was located in northern Italy, while vessels were spread through

the Adriatic and Aegean Seas and air operations were conducted over Serbia. As with the

heterogeneous perspectives of different stakeholders mentioned in Section 3.3.1.2., the

complexity resulting from a distributed decision process must be ameliorated by efficient

merging of information in the larger boundary of the system.

17

3.3.1.4. Dynamic Workspace

The operator�s actions have a distributed effect in time. For example, the assignment of a

missile to an emergent target will decrease the coverage available over the remaining

target space. Thus, selecting a missile for retargeting against a relatively low priority

emergent target could preclude the servicing of a higher priority emergent target a short

time later. Further, decisions which may be possible at one moment in time (such as

selecting missile X to attack an emergent target) may not be optimal or even feasible

several seconds later. If a decision is not made quickly, a candidate missile could travel

out of range of the emergent target, or the target may move or disappear from

surveillance. This effect could be lessened by �discretizing� the dynamic emergent

targeting decision into short, static decisions. Each default route could include �ghost�

points at two-minute intervals (regardless of turning waypoints) to be used as emergent

route branch points. Emergent routes and corresponding times of impact for candidate

missiles would be calculated from the next branch point instead of the constantly

changing current locations. Though the data would change incrementally as a missile

passed these ghost points, a rapid decision could be based on such a static approximation.

3.3.1.5. Potentially High Hazards

As evidenced by the unintended bombing of the Chinese embassy in Belgrade during

Operation Allied Force (Department of Defense, 2000), the effect of a mistake can be

unintentional loss of life. The operator must be clear about the priorities, be capable of

making appropriate decisions, and possess the capability to reverse or abort decisions in

real-time.

18

3.3.1.6. Coupled Systems

The Tactical Tomahawk system is one asset among many in an air campaign. Other

assets include multi-service deep strike assets, Special Forces, intelligence agencies,

command and control nodes, and an airspace management framework. All of these

elements are tightly interrelated and interdependent. Changes in the plans or operations

of these other systems affects the actions of the Tactical Tomahawk operator, and vice

versa.

3.3.1.7. Automated System

The potential difficulty an automated system brings to the decisionmaking in the system

is inherent in a �black box� concept � the operator may not understand how the system

works and may therefore not utilize it correctly. Section 3.3.3 expands on the critical

importance of determining how automation can enhance the control of the system.

3.3.1.8. Uncertain Data

Several uncertain parameters are present in the TTWCS and time-critical targeting

systems. An important property that is inherently uncertain is the time a relocatable

emergent target is predicted to move. While this value should be determined

conservatively by intelligence personnel using a high confidence value (95% perhaps), it

is possible for an emergent target to move prior to the impact of a properly retargeted

missile. There is also some uncertainty in a missile's status, as it is not continuously

updated. Its status is assumed to be that which was transmitted from the last Health and

Status point (see Section 4.4.2.3.1). Without a current AWACS (airborne warning and

19

control system) picture, current missile locations are similarly assumed based on the

planned route, elapsed time of flight, and location at last status transmission.

3.3.2. Situational Awareness

3.3.2.1. Definition

A buzzword in the field of commercial and military piloting, situational awareness (SA)

is an inherent cognitive function in any dynamic system where reality is represented by

visual or auditory displays. Endsley (1998) defines situational awareness as �perception

of the elements of the environment,�the comprehension of their meaning, and the

projection of their status in the near future�. SA has been shown to be important in the

context of a variety of dynamic systems including air traffic control, refineries, nuclear

power plants, and military command and control (Endsley, 1995b). Pilots have described

�having good SA� as akin to �being ahead of the aircraft�, meaning their perception and

comprehension of the aircraft state allows them to anticipate events in the near future.

This effect is descriptive of Endsley�s (1995a) expanded definition specifying three levels

of situational awareness (Figure 3-3).

Figure 3-3 Levels of Situational Awareness

20

Relevant literature offers numerous subjective and objective techniques to measure SA.

Many situational awareness testing procedures (Endsley & Kiris, 1995; Campbell,

Freeman, & Hildebrand, 2000) include time to make an acceptable decision as a primary

objective metric. Subjective metrics include an operator�s rating of confidence in their

decisions, assessments of SA on a numbered scale by subjects as well as observers, and

assessment of workload by the measurement of the number or complexity of tasks. In

testing on a Multimodal Watchstation (MMWS) for the next generation combat

information center, Campbell, Pharmer, & Hildebrand (2000) employed several

categories for measuring situational awareness. Some categories and their relevance to

TTWCS include:

• Task performance � indicated by measuring time and accuracy of many

TTWCS operator tasks.

• Multitasking � indicated by observing/measuring the effect on

performance of increasing the number of missiles and/or targets, or

requiring an operator to simultaneously perform both monitoring and

retargeting tasks.

• Critical thinking � indicated by subjects talking aloud as they conduct a

course of action analysis during complex retargeting tasks.

3.3.2.2. Consequence in TTWCS Design

The TTWCS operator must maintain situational awareness during two basic types of

tasks: monitoring and retargeting. Monitoring tasks include keeping track of missiles,

targets to which they are assigned, times of impact, missile health statuses, and target

priorities. When monitoring, the operator does not issue active commands; he/she is a

21

recipient and processor of information. Maintaining Level 1 and 2 situational awareness

during monitoring tasks is critical for successful retargeting. �SA [during monitor tasks]

provides the basis for subsequent decisionmaking [in retargeting tasks] and performance

in the operation of complex, dynamic systems� (Endsley, 1995a).

Maintaining SA during retargeting tasks is more complex. Given an unplanned, time-

critical event (the appearance of an emergent target), the operator must be able to filter

courses of action from the entire solution space of all inflight missiles, and then compare

the feasible courses of action. With multiple courses of action, the comparison involves

the integration of three potentially conflicting goals (time of impact if retargeted, priority

opportunity cost, and loiter opportunity cost; described in Section 4.3.2) to predict the

best outcome. Campbell, Freeman, & Hildebrand (2000) suggest that this �ability of

watchstanders to filter voluminous data, seek key information, plan, and take appropriate

action hinges on the accuracy of their assessments.�

Common in all military weapon systems, the TTWCS will be monitored and controlled

by formally trained operators. When compared with novices and advanced beginners,

competent performers and experts benefit from an enhanced recognition of situations,

context-dependent thinking (Federico, 1995), and a reliance on efficient rule-based

behavior. In contrast with software such as Microsoft Word or Excel, which may

sacrifice advanced functionality for the benefit of �user-friendliness� to novices and

advanced beginners, the TTWCS should afford the best situational awareness for

competent performers and experts.

22

3.3.3. Automation and Decision Support

3.3.3.1. Definition

The use of automation in complex systems is becoming commonplace, with advanced

automation being touted as a means to reduce manning, standardize processes, increase

throughput, and reduce costs (Guerlain, Brown, & Lambert, 2001). However, humans

must interact with automation at some level, particularly in dynamic environments where

the automation cannot handle all situations. As automation is introduced, operators may

be delegated to the role of supervisory controller (Sheridan, 1987). Their level of

situational awareness must expand to include awareness of the automation. While

automation can replace or minimize certain manual activities, new activities are

introduced, including monitoring the automation, directing the automation�s activities,

and diagnosing automation failures. Successful system performance relies not only on

the effectiveness of automated features, but also on the compatibility between human

skills and automated aids.

Improved operator SA in complex systems can result from the integration and

presentation of information by automated decision aids. Decision aids, decision support

systems, and problem solving aids can perform repetitive, labor-intensive tasks while

permitting system operators to focus on bigger-picture goals. They can be best

implemented in systems constrained by cognitive limitations and bias, complexity

pitfalls, and where high complacency and low vigilance may exist (DISA, 1996). Tasks

best performed by automation include processing quantitative data in prespecified ways,

23

while human skills are better employed to draw on experience and adapt decisions to a

situation (Schneidermann, 1997).

Hopkin (1995) suggests three basic levels of problem solving aids: the simplest points

out the existence of a problem; the next level supports an aspect of the problem solving

process; the most complex type of aid incorporates computer assistance that offers a

solution. Endsley & Kiris (1995) expand on these ideas, defining five levels of

automated decision aids varying from completely unautomated (Level 1) to fully

automated (Level 5) ( Table 3-1).

Automation Level Type System Function Operator Tasks

1 Manual No automation support or information Decide, act

2 Decision support

Automated system provides information but does not execute actions

View recommendation, decide, act

3 Consensual automation

Automated system highlights recommended choice; user makes any selection

Concur or select other option

4 Monitored automation

Automated system executes the choice. User has veto power for some fixed period of time, but system defaults to its choice if the user does not act.

Veto if nonconcur

5 Full automation

System acts and provides no veto authority to the user.

Observe full automation

Table 3-1 Levels of Automation

24

3.3.3.2. Benefits of Automation

Among the most significant advantages of automating a system is �to allow for additional

operator responsibilities while decreasing the manning requirements� (Operational

Requirements Document, 1998). The U.S. Navy�s feverish research and development

efforts into the next-generation destroyer (DD 21) aim to reduce the crewing requirement

by seventy percent, from approximately 320 to 95 (Danzig, 2000). Reduced manning on

this scale can only be achieved through innovative automation methods in numerous

integrated systems. The combat information center (CIC) on today�s Aegis destroyer

houses up to 25 officers and enlisted sailors performing missions ranging from engine

and electrical system management, damage control, air track management, and land

attack management (SHAI, 2001). Many of these personnel currently operate standalone,

or �stovepipe� systems. Researchers at the Naval Surface Warfare Center have

developed and tested a prototype MultiModal Watchstation (MMWS) in an Integrated

Command Environment (ICE) laboratory (Campbell, Pharmer, & Hildebrand, 2000).

MMWS integrates information and employs automation at many levels to reduce the

manning requirement in the CIC to seven watchstanders. MMWS removes the stovepipe

structure by combining the tasks of a separate Tomahawk operator, Harpoon missile

operator, and Naval gunfire coordinator into a single Land Warfare Coordinator (LWC)

who can monitor and control all land attack tasks from one workstation (NSWC, 2000).

3.3.3.3. Drawbacks of Automation

Automating decision support systems may involve certain drawbacks. The potential for

operator mistrust of automation may enter into the design decision of what role the

25

human plays. Critical decisions left to a Level 5 automated system (where the operator is

completely �out-of-the-loop�) may lack the ability to be analyzed, and rely on decision

criteria, weights, and algorithms that may not take into account specifics of a current

situation. In some situations, high levels of automation can result in optimum

performance, but the tasks of the operator are reduced to encompass mostly monitoring

functions. Over-reliance and over-confidence can render the automated system

susceptible to �automation-induced complacency�, where the operator fails to maintain

vigilant attention and is slow in intervening when necessary (Parasuraman, Molloy, &

Singh, 1993.) It has been further suggested that an out-of-the-loop performance

decrement can accompany automation, and that this can lead to a reduction or lack of

situational awareness (Carmody & Gluckman, 1993; Endsley, 1987; Wickens, 1992).

3.3.3.4. Consequence in Uninhabited Vehicle Control

The primary application of automated decision support into the provided TTWCS

prototype (Section 4.4.2.9) pertains to retargeting tasks. The ten-minute goal to service

time-critical targets allows little time for the TTWCS operator to compare alternatives

and execute a retarget mission. However, it is assumed that retargeting events follow

periods where the operator merely monitors default missions. The operator�s level of

alertness during monitor tasks is thus of interest. In some campaigns, emergent target

situations may be infrequent, and the operator may monitor default missions for hours

without being given a retargeting mission. Regardless, actions following the receipt of a

retarget mission must be rapid and accurate. Any lack of vigilance must be overcome in

the interface by an effective decision support system.

26

Functional requirements of other uninhabited vehicles support the application of

�different levels of autonomy for different parts of the mission� (Wall, 2001c; Kandebo,

2000b). While it automates enroute tasks, Boeing�s X-45 uninhabited combat air vehicle

(UCAV) retains a human in the control loop and requires operator clearance for weapons

release (Kandebo, 2000b). The definition of �uninhabited� implies there will always be a

man in the loop because there are lethal capabilities involved (Fulgham, 2000). Where

UAVs (unmanned aerial vehicles) can be employed completely autonomously, UCAV

will likely retain more human interaction. The proposed TTWCS interface (Section

4.4.3) suggests Level 5 automation for portions of the mission, while retaining Level 3

for critical, high-consequence decisions.

27

4 Interface Design

4.1. Introduction

The major development effort for this thesis was to design, build, and test a dynamic and

interactive prototype to explore the monitoring and retargeting tasks in the Tactical

Tomahawk system. In addition to the author, four undergraduate systems engineering

students were involved in the effort, making up a design team. We followed a systems

approach to design and develop several prototypes of varying fidelity. The process began

with a User and Task Analysis of the problem, in which the system and tasks were

Figure 4-1 Design Approach

28

decomposed into four products: a definition of objects, properties, and values; a

retargeting task flowchart; a list of scenarios; and a list of system functional

requirements. Next, a synthesis process was iteratively conducted to create successively

higher fidelity interface prototypes. As indicated in Figure 4-1, individual objects and

properties were combined with functional requirements to generate component features

of the prototype. These components were then synthesized with broad task groups (such

as retargeting) to generate prototypes of the system interface.

4.2. User Analysis

It has been suggested that to develop an interface for an existing system, designers should

begin by observing users in action in a �field� environment. Interviews with users in a

conference environment fail to demonstrate time aspects of tasks, and are subject to

estimates by the users which may be inaccurate. Further, managers, lab researchers, and

expert users may unknowingly misrepresent the typical user who will operate a system

for majority of the time (Hackos & Redish, 1998). On-site visits are thus the preferred

first step in the interface design of an existing system.

Although many TTWCS functions stem from the legacy ATWCS system, this study has

focused on the unique set of tasks involving inflight retargeting. The design process was

therefore framed as designing an entirely new product. While the team was unable to

observe the operation of a typical ATWCS operator on a vessel or in a simulator, we did

evaluate the system in a lab setting during a site visit to the client. We essentially

observed the old, and studied the requirements of the new.

29

In understanding the users for a system, Hackos & Redish (1998) describe four stages of

use pertaining to software/hardware systems:

• Novice

• Advanced beginner

• Competent performer

• Expert

In the case of common software products such as Microsoft Word, 80% of users never

move beyond the advanced beginner stage of use (Hackos & Redish, 1998). For these

products, interface functions and rules must be salient to a casual user; there is a high

value in user-friendliness. In the case of advanced systems including weapons, aircraft,

or power plant controls, we assume that the operators will be highly trained (beyond the

level of the novice or advanced beginner) on all the features and rules in the interface.

First-tour operators may be at the �competent performer� level, while seasoned crew may

perform at an expert level.

Logistical constraints prevented our ability to observe experts at work, as well as to reap

the benefits of an expert on the design team for checks during development and testing

away from the operating environment. Kellmeyer & Osga (2000) noted during MMWS

testing that �obtaining skilled Naval operators [for design consultation and subject

testing]�is a non-trivial task�∗ , and stressed the importance of recruiting a core set of

∗ Skilled Army operators, on the other hand, are fairly abundant.

30

expert users as an integral part of the design team. Thus, the absence of a subject matter

expert in the field of cruise missile operation or even air traffic control potentially had a

negative impact on the development efforts. The subjects used for formal testing (as well

as the design team members themselves) should be considered novices and/or advanced

beginners with the system. However, we did reference the TTWCS requirements

document extensively, and consulted with engineers who are developing aspects of the

TTWCS retargeting capability.

4.3. Task Analysis

For the task analysis, we maintained a focus on the user�s overall goals. Although users

may vary the steps taken to achieve goals depending on features of specific designs,

overall goals are inherent in the system. Four primary products resulted from the goal-

oriented decomposition of the system into objects, tasks, and decisions. First, we defined

relevant Objects (Section 4.3.1) and inter-object relationships in the TTWCS system to

maintain common terminology between team members. We then developed a

Retargeting Task Flowchart (Section 4.3.2) to decompose the identification and selection

of courses of action in generic retargeting situations. Next, we developed a set of

Scenarios (Section 4.3.3). By analyzing the scenarios, we identified and formalized a

detailed list of 253 Functional Requirements (Section 4.3.4). Each of the products was

iterated in design meetings and/or formal testing.

31

4.3.1. Object Definition

The following seven objects stand out as relevant in the system:

• Target

• Missile

• Vessel

• Constraint (airspace control measures and threats)

• Mission

• BattleGroup

• Campaign

Each object is defined by its properties which take on specific values or types of values.

An example of a single object-property-value thread is shown in Figure 4-2.

Figure 4-2 Object-Property-Value Thread

32

Most of the effort on developing the operator interface addressed the Target and Missile

objects. Properties of these objects are shown in Figure 4-3. While the definition of

properties of the Constraint object were iterated in the domain study, airspace control

measures and threats were not included in the final dynamic, interactive prototype due to

coding complexity. Tables 4�1 and 4�2 list definitions and explanations critical to the

comprehension of the remainder of the thesis. Many properties are defined (along with

their graphical representations) in Section 4.4.2. Table 4�3 lists all the properties of all

seven objects. The entire list of object definitions is reserved for Appendix B.

Figure 4-3 Properties of Target and Missile Objects

33

Table 4-2 Critical Missile Property Definitions

Table 4-1 Critical Target Property Definitions

34

Table 4-3 Object – Property Dat

abase

35

4.3.2. Retargeting Task Flowchart

Missiles can be retargetd to either a pre-planned flex target or to an emergent, time-

critical target. While both retargeting methods require a command from the operator to a

missile, the proposed triggers and operator decisions are different. This section describes

the inherent differences; suggested methods for implementation in the interface are

described in Section 4.4.

The proposed primary consideration in flex targeting is target priority. In addition to a

default target, individual missiles may have multiple pre-planned, pre-loaded flex

missions. It is feasible that two or more missiles could have flex missions to the same

flex target. These missions include a branch point from the default route, waypoints and

altitudes for the flex route, and target information including the dive angle required by an

assigned missile. It is assumed that at the time of missile launch, a missile�s default

target outprioritizes all of its flex targets; else, the highest priority flex target would be

the default target. The trigger to command a missile to abort its default mission to

execute one of its flex missions is based on a real-time increase in priority of the flex

target to a value above the default�s. The decision to increase an assigned priority is

assumed to occur at the air operations center, and is based on a real-time change in the

tactical value of a target on the changing battlefield. It is assumed that the TTWCS

receives the increase in priority via the common operating picture (Operational

Requirements Document, 1998; National Research Council, 1999).

36

The emergent targeting process is more complex at both the targeting cell and the

TTWCS operator levels. Recalling from Figure 3�2, the targeting cell performs the

following functions:

• Identification of time-critical targets

• Analysis of targets

• Assignment of targets to an on-call weapon system

Upon receipt of an e-target mission, the TTWCS operator and/or system must:

• Identify and compare courses of action (COA)

• Select best course of action

• Transmit inflight missile modification message (IMMM)

(Operational Requirements Document, 1998)

The missile then flies the new route and destroys/suppresses the target.

The military services have collectively set a 10-minute goal for the servicing of time-

critical targets (from target identification to destruction). This requirement essentially

�renders decisionmaking as critical as weapons design� (Fulgham & Wall, 2000a). It

leaves the TTWCS operator little time (perhaps less than 30 seconds) to perform the steps

of course of action comparison, selection, and IMMM transmission. We developed a task

flowchart (Hackos & Redish, 1998; Campbell & Cannon-Bowers, n.d.) outlining the

course of action identification and comparison steps for the retargeting task (see Figure

4�4, page 39). Identifying courses of action involves eliminating missiles from

consideration that fail to meet rigid constraints specified in Table 4�4.

37

Table 4-4 Rigid Retargeting Constraints

The reduction of the solution space by rigid constraints results in the identification of

zero, one, or greater than one missile candidates. The operator obviously need not make

any decision if zero or one candidate remains. Assuming at least two candidates were

identified through the application of rigid constraints, the operator must next compare the

courses of action. Three criteria are proposed for comparison (Table 4�5).

Table 4-5 Retargeting Minimization Objectives

Comparison criteria were selected and defined based on the analysis of seven scenarios

(see Section 4.3.3 and Appendix C). Time of impact if retargeted (TIIR) is estimated by

calculating the distance along the most direct path from each missile to the e-target,

compensating for required missile turn-around maneuvering, vectoring around restrictive

airspace control measures and threats, and maneuvering for target attack heading

∗ Imposed for simplicity in prototype design; Operational Requirements Document (1998) specifies that the

system should be capable of retargeting each missile �at least once�.

COA Identification – Rigid Constraints Warhead-target match

Fuel available to reach target

Proximity to reach e-target prior to expected movetime

Missile not already retargeted∗

COA Comparison – Minimization Objectives Time of impact if retargeted (TIIR)

Priority opportunity cost (POC)

Loiter opportunity cost (LOC)

38

constraints as suggested in Appendix G. Priority opportunity cost is calculated by

summing the priority values of the targets that would be given up if a missile were

selected for e-targeting. Loiter opportunity cost represents the loss of a candidate

missile�s coverage zone (see Section 4.4.2.8), and is directly correlated to the amount of

fuel remaining. The operator wishes to minimize each criterion. Unless one candidate

dominates in all three criteria, the minimization will result in conflicting objectives and

suggests a rate-and-weight comparison. Once a candidate missile is selected for e-

targeting, the operator must compose and transmit to the missile the new mission

consisting of waypoints, route segment altitudes, and the e-target coordinates.

39

Figure 4-4 Retargeting Task Flowchart

40

4.3.3. Scenarios

The design team developed seven scenarios for analysis in the task analysis. In keeping

with a user- and goal-oriented approach, scenarios use the notion of storytelling to gain

�insight into the attributes to accommodate in the design, what users value, and what they

see as aids and obstacles to accomplishing their goals� (Hackos & Redish, 1998). In a

separate contracted program involving the uninhabited combat air vehicle (UCAV), Navy

officials �gave industry teams specific mission scenarios that the UCAV and its operator

will have to perform� (Wall, 2001a). As mentioned earlier (Section 4.2), skilled users

were not available to provide input and feedback to the TTWCS design team. This made

careful scenario development even more crucial.

The design team implemented the vignette type of scenario � a brief narrative with a

visual, high-level, broad view of the environment. The seven scenarios facilitated the

identification of operator information requirements and interface functional requirements.

A sample scenario (Scenario #4) is depicted in Figure 4�5. All of the scenarios are

presented in Appendix C.

41

4.3.4. Interface Functional Requirements

The first iteration of analysis of Scenario #4 identified the following functional

requirements:

• Receive retargeting mission by voice or data

• Display e-target time properties

• Display ACM/threat time properties

• Display threat risk value

Figure 4-5 Sample Retargeting Scenario

42

• Aid in decisionmaking for candidate missile selection

• Analyze opportunity cost of options

• Highlight routes through threat areas

• Enable rapid decisionmaking for simultaneous, multiple flex and

emergent targets

First-iteration functional requirements that were drawn from other scenario analyses

included:

• Display straightline times of impact

• Forecast and display waypoint crossing times

• View modified routes

• Compare missile events

• Display current missile-to-target assignments

The requirements were formally organized into a master list of functional requirements

which branch from four broad mission-essential requirements:

Table 4-6 Mission Essential Functional Headings

An iterative analysis of all of the scenarios lead to the specification of 253 lines of

Functional Requirements, listed in Appendix D.

Major Functional Requirements Headers Monitor and Communicate Status

Conduct Queries to the System

Develop and Modify Plans

Facilitate Retargeting Decisions

43

4.4. Prototype Development

The iterative process of prototyping was split into two categories. Component prototypes

were designed to meet individual or small sets of functional requirements. As indicated

in Figure 4-6, components were created by a process combining functional requirements

with objects. The design team performed most of the iterations at weekly design

meetings using cognitive walkthroughs - group reviews of a user�s interaction with a

component (Kellmeyer & Osga, 2000). Two of the three system prototypes were iterated

via formal subject testing. At the time of publication of this thesis, work continues

(hopefully into another year of funding) on a third full iteration of component and system

prototypes based on results of the second series of subject testing.

Figure 4-6 Steps in TTWCS System Prototype Development

44

The process of iterating out poor components and functions (especially concepts to which

team members were personally attached) proved difficult at times. Kellmeyer & Osga

(2000) note that �iteration implies a willingness to discard design ideas and code,� some

of which were long in development. However, sticking to the proposed iterative

approach resulted in dramatic changes being factored into new low-fidelity prototypes.

The remainder of the Prototype Development section is organized as indicated in Table

4�7, and represents the sequence in which the interface was developed. For

simplification in the remainder of this document, system prototypes will be referenced by

the version number indicated below.

Table 4-7 Prototype Versions and Testing

Description System

prototype version

Formal subject testing

Initial low-fidelity design v.1 ●

Component prototypes

Interactive prototype v.2

Design document and storyboarding

User testing prototype v.3 ●

Post user testing prototype v.4

45

1 2 3 5 Next launch

Endurance

4

51

2

3

415:07 14:32 14:10 14:57 13:54 13:44

Default Timeon Target

13:40:01Current

00:14:01 00:03:59Countdown 00:29:59 01:16:5901:26:59 00:51:59

Launch Time

Countdown

1 2 3 5 Next launch

Endurance

4

51

2

3

415:07 14:32 14:10 14:57 13:54 13:44

Default Timeon Target

13:40:01Current

00:14:01 00:03:59Countdown 00:29:59 01:16:5901:26:59 00:51:59

Launch Time

Countdown

4.4.1. Initial Low-Fidelity Design & Testing (Version 1)

Sanders & Willis (2000) developed and tested three low-fidelity, static system prototype

alternatives after conducting an informal task analysis and functional requirements study.

The testing was not analyzed for statistical significance, but incorporated the University

of Virginia�s Eye Gaze Response Interactive Computer Aid (ERICA) to assist in the

analysis of subject visual attention in response to verbal questions posed by an

administrator. The simple prototypes are shown in Figures 4�7 a, b, and c.

The low fidelity prototypes proved useful for exchanging ideas, but do not sufficiently

test key dynamic qualities such as time and workload. Results and recommendations

from the initial testing included the following:

Figure 4-7a

System Prototype Version 1.1

46

4

51

2

3

Missile #4 StatusDS955652FL200 TGT AB1242TOT 14:57Remaining: 01:16:59

14:32 15:07

14:10

(cursor)

13:54

14:57

4

51

2

3

Missile #4 StatusDS955652FL200 TGT AB1242TOT 14:57Remaining: 01:16:59

14:32 15:07

14:10

(cursor)

13:54

14:57

Figure 4

System Version

Figure 4

System Version

• Minimize the use of text and numbers. Subjects

tendency to focus on digital readouts when present, ev

mere comparison was required and discernable from a

coverage zone, or other graphical representation.

• Layer the workspace format. Many properties and

features must be quickly accessible while minimizing cl

• Map domain characteristics into graphical elements.

key domain characteristics, and a critical criteria in

missile for an emergent target, is the combined missil

6 3

4

1

2

5

Coverage TimeFactor

Lifetime

60 min

20 min

10 min

(specify)

Dwell countup 00:09:32

EEmergent TargetEmergent TargetEmergent TargetEmergent TargetDS45562961DS45562961DS45562961DS45562961Alt +00634 Alt +00634 Alt +00634 Alt +00634 TGT E242TGT E242TGT E242TGT E242Appeared 13:37:25Appeared 13:37:25Appeared 13:37:25Appeared 13:37:25Dwell 00:09:32Dwell 00:09:32Dwell 00:09:32Dwell 00:09:32

6 3

4

1

2

5

Coverage TimeFactor

Lifetime

60 min

20 min

10 min

(specify)

Dwell countup 00:09:32

EEmergent TargetEmergent TargetEmergent TargetEmergent TargetDS45562961DS45562961DS45562961DS45562961Alt +00634 Alt +00634 Alt +00634 Alt +00634 TGT E242TGT E242TGT E242TGT E242Appeared 13:37:25Appeared 13:37:25Appeared 13:37:25Appeared 13:37:25Dwell 00:09:32Dwell 00:09:32Dwell 00:09:32Dwell 00:09:32

-7b

Prototype 1.2

-7c

Prototype 1.3

showed a

en when a

bar chart,

graphical

utter.

One of the

selecting a

e coverage

47

area. The �blob-like� coverage zone features in Version 1.3

seemed to most effectively facilitate subjects in their selecting a

missile for an emergent target.

• Ensure graphical representations and metaphors are obvious. The

use of line thickness of future missile routes to represent endurance

proved confusing to several subjects.

Each of these recommendations are highlighted in the appropriate sections to follow.

4.4.2. Component Prototypes

Seven types of component prototypes were iterated via a cognitive walkthrough process

during design meetings. Table 4�8 lists the main components.

Table 4-8 Component Prototypes

4.4.2.1. Icons and Symbology

The development of symbols for the TTWCS interface is in accordance with the

Department of Defense Common Warfighting Symbols Interface Standard (MIL-STD-

2525B, 1999) to the greatest degree feasible. While the intent of adhering to such a

standard is to promote interoperability at the information level, the rules are provided to

Component Prototypes Icons and Symbology

Routes

Missile-Target Assignments

Map

Missile Timebars

Message Section

Coverage Zones

E-targeting Decision Support Tool

48

allow symbology to be tailored to operational requirements and system capabilities. In

two cases (the �frame� of a missile symbol and the �fill color� of missile and target

symbols), we tailored a symbol in such a way that could be argued as an outright

violation of the standard. Symbols are composed of a frame, fill color, icon, text

modifiers, and graphic modifiers. Symbol development and appropriate rationale for

deviations from the MIL-STD-2525B standard are described in the following sections.

49

4.4.2.1.1. Targets

4.4.2.1.1.1. Target Symbol Frame

All target symbols (representing hostile units, installations, and equipment) are diamond-

shaped. Hostile affiliation is redundantly indicated by the red line color of the frame

border. MIL-STD-2525B does not address the use of frame color to indicate any symbol

properties.

4.4.2.1.1.2. Target Symbol Fill Color

Fill color represents the class of target as indicated in Table 4-9. Contrastingly, MIL-

STD-2525B utilizes fill color to indicate affiliation (friend, hostile). All targets on the

proposed TTWCS display are assumed hostile. Section 4.4.2.1.2.2 addresses the related

conflict in fill color conventions for missiles that would likely be found between the

TTWCS display and an air picture display.

Table 4-9 Target Symbol Fill Color

50

4.4.2.1.1.3. Target Symbol Graphic Modifiers

The graphic modifiers shown in Table 4-10 represent three target properties which are

central to the Tomahawk system; they are not specifically addressed in MIL-STD-2525B.

4.4.2.1.1.4. Target Symbol Text Modifiers

Target text modifiers appear adjacent to the target symbol. Sanders & Willis (2000)

described subjects becoming overwhelmed when viewing text-dense displays, while the

DoD HCI Style Guide (1996) states that designers should �minimize text information

density by presenting only information essential to the user at a given time.� Text

modifiers are thus available in three levels: none, ID label, and Full Details (see Figure 4-

8). ID labels and Full Details for all the targets can be turned on and off with radio

buttons on the interface. Full Details can also be viewed for selected targets during

Table 4-10 Target Symbol Graphic Modifiers

51

mouse rollover or when an individual target symbol is selected via double-click. The

target identification numbering follows the following convention.

“T(target number)-(class)(priority)-(warhead required)”

Figure 4-9 Target Symbol Examples

Figure 4-8 Target Text Details

52

4.4.2.1.2. Missiles

4.4.2.1.2.1. Missile Symbol Frame and Icon

MIL-STD-2525B requires air weapon tracks (missiles) to be bullet-shaped, with an open

bottom to indicate the �air� dimension. The proposed border line color (as opposed to fill

color as specified in MIL-STD-2525B) is cyan blue to indicate friendly affiliation. Also

contrasting from the standard is the line style of the border, which is proposed to indicate

the health status, as opposed to the current/planned location status of the object specified

in the symbology standard. See Section 4.4.2.3.1 for explanation of Health and Status

transmissions. The missile-shaped icon distinguishes the vehicle as an air track weapon,

as opposed to a manned aircraft or unmanned aerial vehicle.

Nominal Solid border: missile transmitted nominal health at last

scheduled Health & Status point.

Caution

Dashed border: one or more health/status parameters is in a caution state, or the missile failed to transmit health status at a single Health & Status point.

Alert

Flashing border: one or more health/status parameters is in an alert state, or the missile failed to transmit status at a two consecutive Health & Status points.

Table 4-11 Missile Symbol Frame Style

53

4.4.2.1.2.2. Missile Symbol Fill Color

Fill color of a missile symbol is proposed to indicate the current mission of that missile.

As mentioned, the degree of tailorability with the use of fill color has been taken to the

point of �violating the standard�. By MIL-STD-2525B, fill color is to be used as a

redundant indication of the symbol�s affiliation (unknown, friend, neutral, and hostile).

Fill color in the proposed TTWCS symbols represents the �default�, �flex�, or

�emergent� status of a missile, while affiliation is represented by the frame color (outline)

of the symbol.

Default missile Flex missile E-target candidate missile Emergent missile

No fill indicates a default missile (d-missile), attacking a default target. All missiles are launched as d-missiles. Yellow fill indicates a flex missile (f-missile), a missile that the operator has selected to service a flex target. Pink fill indicates that the missile is a candidate for striking an emergent target, but the operator has not yet selected it to do so. Red fill indicates an emergent missile (e-missile), a missile that the operator has selected to service an emergent target. Other pink candidate missiles return to �no fill.�

Table 4-12 Missile Symbol Fill Color

The DoD HCI Style Guide (1996) recommends the standardization of color coding

schemes for operational applications, but acknowledges that �flexibility in schemes may

be desirable for a terminal dedicated to a single user.� This is a non-trivial issue in the

context of a common operating picture (COP) (National Research Council, 1999),

54

especially when considering the likelihood of an integrated combat information center

(CIC) such as that tested in Multimodal Watchstation (MMWS) studies (Campbell,

Freeman, & Hildebrand, 2000; Campbell, Pharmer, & Hildebrand, 2000; Kellmeyer &

Osga, 2000). Two primary stakeholders, the land warfare coordinator (LWC) and air

warfare coordinator (AWC), are both concerned with inflight missiles but their attention

is focused on different properties of the missiles. The LWC is concerned primarily with

missile warhead type, target assignment, and mission (d-, f-, or e-), as well as target

properties. Every air track on the land warfare display should be of friendly affiliation.

The AWC has little (if any) concern about ground targets. Of the missiles (which are air

tracks), his primary concern is affiliation (friend/hostile), as well as heading, altitude,

airspeed, and current location (NSWC, 2000). Table 4�13 illustrates the conflict that the

missile fill colors on the two officers� displays would not agree using the proposed

convention. While the usability may be enhanced for the LWC (especially for comparing

55

e-target candidate missiles, a time-critical task), this conflict may serve to degrade the

interoperability for which a common operating picture (COP) exists.

With the proposed color scheme, the same missile is represented on two officers� displays

by different colored symbols. A potential solution would be to incorporate a �Standard

Symbology� and �Enhanced Symbology� radio button on the land warfare display. In

violating the icon color scheme, the operator may get additional information from the

�delta� between the standard and the proposed selectable color schemes.

4.4.2.1.2.3. Missile Symbol Graphic Modifiers

Soft Warhead

A proposed modifier to indicate a missile carrying a soft warhead, and only usable against soft targets. A missile lacking this symbol has a hard warhead and can attack hard targets.

Heading (Direction) Indicator

A line extending from the center of the missile icon, indicating the current direction of travel (MIL-STD-2525B).

Table 4-14 Missile Symbol Graphic Modifiers

56

4.4.2.1.2.4. Missile Symbol Text Modifiers

Text modifiers for missiles are displayed and selectable with radio buttons similar to

targets (Figure 4-10). The missile identification number naming convention is

“M(missile number)-(mission)(priority)-(warhead)”

4.4.2.2.

Figure 4-10 Missile Text Details

Figure 4-11 Missile Symbol Example

57

4.4.2.3. Routes

Routes define the actual path over the ground that a missile is scheduled to follow to

reach a target. The type of route is defined by the target at the end: default routes, flex

routes, and emergent routes lead to respective targets. Flex (and emergent) routes

represent potential paths for a missile currently on a default route. Figure 4-12 depicts

missile Routes, which are represented in the interface as follows:

• Defined by a set of waypoints (turning points)

• Displayed when selected by the operator

• Currently scheduled routes are represented with heavy line weight.

• Potential routes are represented with light line weight.

4.4.2.3.1. Health & Status Points

Health & Status (H & S) points are preplanned waypoints on a route where a missile is

scheduled to transmit to the operator a data message pertaining to the status of the

mission (Operational Requirements Document, 1998.) Critical missile health parameters

should include functionality of on-board systems, fuel remaining, and speed. Status

parameters pertain to mission data such as current missile location, target assignment,

time at last waypoint, and time of impact at scheduled target. Health and Status points

are depicted in the interface with small triangles on top of missile routes as shown in

Figure 4-12. When a missile transmits a nominal message at a H & S point, the triangle

58

disappears. For Caution or Alert transmisisons, or when a missile fails to transmit at a H

& S point, the triangle remains visible.

• A green triangle indicates a future H & S point.

• A yellow triangle indicates a passed H & S point where a missile

failed to transmit, or transmitted a caution status. The missile

frame turns dashed in accordance with Table 4-11, and the

previously green triangle turns yellow and remains visible.

• A red triangle indicates a passed H & S point where a missile

failed to transmit a second consecutive time, or transmitted an alert

status. The missile frame turns dashed and flashes in accordance

with Table 4-11, and the previously green or yellow triangle turns

red and remains visible.

Figure 4-12 Missile Routes and Health & Status Point

59

4.4.2.4. Missile-Target Assignments

The task analysis showed that during monitoring and retargeting tasks, the operator is

frequently not concerned with the route over the ground that a missile will fly. Rather,

the operator wants to identify the target to which a missile is currently bound, as well as

any flex or emergent targets to which a missile could be retargeted. The Assignments

tool (Figure 4-13) maps this domain characteristic and information requirement (missile-

target matching) into a useful graphical component (Norman, 1990; Williams, 1994.)

Assignments can reduce display clutter caused by viewing all missile Routes, and are

represented in the interface as follows.

• Displayed when selected by

the operator

• Current assignments (lines

between a missile and the

target it is currently targeted

against) are shown as a heavy

straight line

• Potential assignments (lines

between a missile and any

targets to which it can be

commanded to fly toward) are

shown as light curved lines Figure 4-13 Missile-Target Assignments

60

4.4.2.5. Map

Regardless of the country/region of the area of operations, the background map display is

made subtle (see System Prototype section). Provided with processed intelligence data,

the operator may have no immediate interest in regional or country borders, or of where

land meets water. The DoD HCI Style Guide (1996) recommends that �the level of map

detail should be consistent with the operational need.� Soft colors are used (light blue for

water, light gray for land, darker gray for borders).

4.4.2.6. Missile Timebars

The domain study revealed that many of the properties from several objects pertain to

time. Consequently, many functional requirements require careful mapping of time

properties into graphical elements (Norman, 1990; Williams, 1994). Several iterations

were undertaken in combining these properties with functional and cognitive

requirements to develop the timebar display component. Each timebar corresponds to

one missile, along with its default, flex, and emergent targets. The timebar section of the

interface incorporates additional time properties from emergent targets (when present), as

well as a current system time. The relevant properties are listed in Table 4-15.

61

Missile Target BattleGroup time_actual_launch time_appearance current_time

time_fuel_burnout expected_movetime

time_of_impact

time_to_impact

straightline_time_of_impact

current_speed

Table 4-15 Properties Incorporated into Timebar Component

Having identified the time-related objects and properties, we next incorporated the

functional and cognitive requirements from the scenarios to design the timebar display

component. Many scenarios revealed the need to quickly see the time of impact for a

missile�s current target, as well as for all of its potential targets (flex and emergent).

Additionally, Scenario 1 lead to the requirement to depict the time that a missile would

impact its target if it were commanded to abandon its programmed route and fly to its

target via a straight-line, or the most direct route compensated for restrictive airspace

control measures, threats, and required turn-around and terminal maneuvering. An

emergent feature (Norman, 1990b) in the timebar is the loiter, which is the difference

between the straightline (or most direct) route and the circuitous path of a loiter route or

pattern. These basic requirements lead to the timebar shown in Figure 4�14 (next page).

Once critical time properties to represent were defined, valuable iterations in the timebar

design occurred following the construction of an initial �visualization�, or �minimalistic

prototype�allowing team members to see a literal representation of what they�ve been

talking about� (Wagner, 1990).

62

Figure 4�15 illustrates the timebar section for four default missiles.

Figure 4-14 Missile Timebar Component

Figure 4-15 Timebar Section (4 missiles)

63

A current goal of precision weapon development, and indeed the most notable feature of

the Tactical Tomahawk system, is the prosecution of time-critical emerging and

relocatable targets (Operational Requirements Document, 1998; Mullen, 2000; Scott,

2000b). Many of the scenarios involved the appearance of one or more emergent targets,

and required the operator to compare candidate missiles as courses of action. Critical to

this comparison is the time element associated with the relocatable target. As mentioned

earlier, the command to service an emergent target is assumed to come from a targeting

cell, where intelligence analysts have generated a data block to send to an appropriate

weapon system. With this data must come the time the target was first confirmed by

some collection source (�time_appearance�), and the time that the analyst predicts the

target will move. This �expected_movetime� is thus stochastic, and is likely based off a

probability distribution whose parameters depend on the type of target and other theater-

specific characteristics. The interface uses a vertical red band (Figure 4�16, next page)

through the timebar section to represent these two time attributes. During a cognitive

walkthrough design evaluation (Kellmeyer & Osgo, 2000), the misleading nature of

representing the movetime with a deterministic closed rectangle became apparent. A

simple color gradient through the expected movetime better represents the probabilistic

nature of the predicted movetime.

64

4.4.2.7. Message Section

The interface message section enables the operator to communicate in two modes:

• Missile-operator mode

• Chain of command mode

Data communication with inflight missiles (missile-operator mode) pertains to both

�unsafe, unusual, and problematic� conditions (such as a failed transmission at a Health

& Status point), as well as �conditions related to selected processes that do not represent

problematic� situations (such as scheduled launches, impacts, and good Health & Status

transmissions) (OMI CMG, 2001). Many of the missile mode communications are

augmented with automated graphical functions in the interface. For example, a missed

Health & Status point event results in a message in the message window as well as a

Figure 4-16 Emergent Target Time Attributes, Two Candidate Missiles

65

highlighting in the tactical picture section of the response, a change in missile symbol

frame line style (solid to dashed), and the corresponding H & S point in the tactical

picture turning yellow and remaining visible.

The system should also facilitate message traffic between vessels, higher headquarters, a

combined air operations center, and/or a targeting cell (chain of command mode).

Critical messages from the targeting headquarters would command the operator to service

specified flex and emergent targets. These messages would include the required data

block for each target. This two-way message system facilitates mission-critical

communications including:

• Acknowledgement of message receipt

• Acceptance of mission

• Transfer of critical target data

• Notification of e-target opportunity costs

• Notification of inability to execute a mission

• Notification of mission execution and completion

The interface incorporates a message section to accomplish these functions. Incoming

messages have a priority stamp, which corresponds to a text color of the title of incoming

messages. The color convention and basic examples are described in Table 4�16.

Alarm Warning Notification

Receipt of emergent mission Alert H & S condition

Receipt of flex mission Caution H & S condition

Launch notification Impact notification Normal H & S message

Table 4-16 Message Color Convention

66

4.4.2.8. Coverage Zones

A recurring concept which surfaced in design meetings pertains to the amount of land in

the area of operations which multiple missiles have the ability to reach. We defined this

concept as �combined missile coverage area�. An individual missile�s coverage zone

depicts the area of possible action of the missile for different time factors. Recalling that

missiles are assumed to travel at a constant speed, the coverage time factor (CTF) defines

the radius of the coverage zone. The operator can select time factors as indicated in

Table 4-17.

CTF Selection Definition Lifetime Until fuel burnout

Specified Clock Until a specified clock time

30 minutes In next 30 minutes

20 minutes In next 20 minutes

10 minutes In next 10 minutes

Specified Duration In next specified duration

Table 4-17 Coverage Zone Time Factors

When the time factor is selected for a fixed time period, such as �10 minutes�, the

interface displays a constant-radius circle which surrounds and translates with the moving

missile symbols, representing where they could reach in that amount of time. With a

time factor such as �Lifetime� or a specified clock time, the radius of the coverage zone

gradually decreases. Regardless of the operator�s selection, the smaller of either the

lifetime or the selected CTF is used. (Indeed, if a missile has 12 minutes of fuel

remaining, a 30-minute CTF is meaningless).

67

An example illustrates the cognitive utility of the coverage zone component. A recurring

type of time-critical target (such as a mobile SA-6 surface-to-air missile launcher) may

have a average dwell time of 18 minutes. Half of this time could be assumed to be used

by the targeting cell to generate the attack order, while the other half is available for

TTWCS retargeting tasks and the flight time to the target. The TTWCS operator could

use Specify Duration to see the combined missile coverage area reachable by all inflight

missiles. Figure 4�17 illustrates a CTF duration of 9 minutes, which clearly identifies a

significant gap across the center of the area of responsibility. The operator or automated

loiter planner (B.R. Copeland, personal communication, Oct. 23, 2000) could then edit

the loiter routes to optimize the overall coverage.

Figure 4-17 Coverage Zones at 9-Minute Time Factor

68

Coverage zones are also used by an intermediate-level automated function to illustrate

applicable constraints (Hopkin, 1995) in e-targeting situations. Upon receipt of an e-

target mission and data block, candidate missile coverage zones appear automatically

with a Specified Clock CTF equal to the expected movetime of the e-target.

Initial subject testing using static Powerpoint prototypes (Sanders & Willis, 2000)

indicated that the coverage zone feature improved subjective understanding of the

system, and facilitated faster response times to e-targeting related questions. The

coverage zone component remained a central feature of continued prototyping.

4.4.2.9. E-targeting Decision Support Tool (ETDST)

This section describes the decision support technique and component designed for the

course of action comparison task specified in the retargeting task flow diagram (Figure

4�4). The entire retargeting process and events in the system interface are described in

Section 4.4.3.

As mentioned in Section 4.3.2, the operator wishes to do the following when presented

with more than one candidate missile for an emergent target:

• Minimize time of impact if retargeted (TIIR)

• Minimize priority opportunity cost (POC)

• Minimize loiter opportunity cost (LOC).

As these objectives are likely to conflict with each other, the final decision is based on a

rate-and-weight decision method that relies on and is sensitive to the quantitatively-

expressed relative importance of each criteria.

69

The complexity of evaluating multiple missile candidates with respect to the e-targeting

criteria lends itself to the application of decision aids. The DoD HCI Style Guide (1996)

recommends the use of decision aids in systems where complexity pitfalls and cognitive

limitations exist which may require a user to combine multiple criteria and rely on short-

term memory. Difficulties in both of these areas were observed during TTWCS subject

testing without the use of a decision aid. While all of the desired information was present

on the operator display, the process of collecting, remembering, and comparing values for

the three criteria was error prone, time consuming, and subject to limitations in operator

short-term memory. Hopkin (1995) discusses the use of intermediate-level problem

solving aids in air traffic control to indicate benefits or penalties associated with

alternative solutions. More complex aids relieve ATC controllers from the burden of

gathering relevant information, and can ultimately offer a solution. Based on results from

subject testing, the Emergent Targeting Decision Support Tool (ETDST) integrated these

concepts to improve the speed and situational awareness with which the operator can

assess and decide during e-targeting tasks.

Upon receipt of an e-target data block, the ETDST automatically calculates and

normalizes values for the three e-targeting criteria for each candidate missile. The

current version calculates an estimate of TIIR based on the straight-line distance from a

candidate missile to the e-target. The straight-line simplification assumes negligible

missile turn-around maneuver time (180-degree turn in less than 30 seconds) and an

absence of restrictive airspace control measures and threats. As mentioned earlier,

priority opportunity cost is calculated by summing the priority values of the targets that

would be given up if a missile were selected for e-targeting; loiter opportunity cost is

70

calculated from the amount of fuel remaining. Once normalized values are available, the

recommendation hinges on the weighting scheme used for the criteria. The least

desirable sequel following the assignment of a candidate to an e-target is premature

movement of the time-critical target. It is thus argued that the TIIR criteria should carry

the most weight. It could also be argued that the cost of sacrificing default and flex

targets (POC) outweighs the loss of loiter coverage (LOC, which is the ability to service

future emergent targets). The default weighting scheme currently used by ETDST is

listed in Table 4�18:

Criteria Default Weight Time of impact if retargeted (TIIR) 3

Priority opportunity cost (POC) 2

Loiter opportunity cost (LOC) 1

Table 4-18 Default Weights in Retargeting Scheme

The weighted recommendation, as well as the normalized criteria data, are presented by

the ETDST using a bar graph (Figure 4�18, next page). Using an eye-gaze system in

low-fidelity Powerpoint interface screenshot testing, Sanders & Willis (2000) found a

trend among subjects in fixating on tabular data, even when graphical features were

present to convey the required information. The DoD HCI Style Guide (1996)

recommends the use of graphs over tabular data for their advantage in summarizing

complex relationships among variables and facilitating information understanding.

Although the ETDST has not been tested on subjects, cognitive walkthroughs (Kellmeyer

& Osgo, 2000) with the graphs indicated a difficulty in rapidly identifying the smallest

value in a set of bars. Indeed, a dominating candidate missile will minimize all three

71

criteria. The input data was therfore inverted to represent a �more is better� convention,

and problem assessment was faster.

Presented with a recommendation based on the default weights, the operator may choose

to concur and execute the e-targeting mission. However, a rapidly changing battlespace

may present circumstances not anticipated or accommodated by the automated system. A

critical question is, �What weighting scheme would change the recommendation?� The

DoD HCI Style Guide (1996) recommends �setting defaults for fields�, and �reducing

user�s data entry requirements as much as possible.� The ETDST incorporates these

recommendations into a graphical sensitivity tool to explore alternative weighting

schemes.

Located in the upper right of the ETDST, the sensitivity tool is a triangular feature whose

vertices represent the three e-targeting criteria. The weighting scheme is normalized by a

single point inside (or on the edges) of the triangle. A point on a vertex represents a

Figure 4-18 Emergent Target Decision Support Tool

72

scheme of (1, 0, 0), while a point at the midpoint of an edge is (0.5, 0.5, 0). Interior

points represent non-zero weights which sum to 1. Non-normalized numerical values of

the resulting weights are shown in the �Weight Ratio� row to the left of the tool, with the

lowest value equal to 1 for ease of comparison. The operator can �click and hold� the

mouse button over the point, and move around the triangle to see the �Weighted Avg� bar

move dynamically, and quickly discern any sensitivity to weights. In addition, the color

of the dot and the triangle change to indicate a change in recommended missile candidate.

The ETDST currently supports the comparison of up to four candidates. It could be

expanded to include more than four candidates, as well as more than three criteria.

While the system prototype (Section 4.4.3) employs a fully automated (Level 5) process

for COA Identification, the ETDST employs Level 2/3 automation for COA Comparison.

The operator may concur with the recommendation (Level 3), or may view the

recommendation, investigate alternatives, and act (Level 2) (Endsley & Kiris, 1995).

Regardless, the use of this decision support tool �overcomes basic cognitive limitations

while retaining the role of human judgment� (DoD HCI Style Guide, 1996).

4.4.3. System Prototypes

When component development was approximately half completed, the team began

compilation of all functional requirements, components, and system concepts into a

design document for the system prototype. The workstation setup described in the

document incorporates two monitors for the operator. Similar to Boeing�s UCAV

interface, which incorporates one display for �aircraft tactical surroundings, while the

other is reconfigurable and indicates aircraft data such as subsystem status [and] target

73

lists� (Kandebo, 2000a), the initially proposed workstation included an upper Data

Screen and a lower Situation Display (Figure 4�19). Due to budget and time constraints,

the Data Screen and Situation Display were merged in Versions 2, 3 and 4.

Though the presentations of components and system prototypes are sequential in this

report, iteration on the seven components continued during integration with system

prototypes versions 2 through 4. Kellmeyer & Osga (2000) allude to this requirement

stating, �part-task design concepts that appear simple on paper appear much more

Figure 4-19 Initially Proposed Workstation

74

complicated when placed within the larger scope of the interface features with dynamic

time and task pressures.� Wagner (1990) recommends incorporating an increasing

degree of fidelity in the prototypes, beginning with visualizations, through the interactive

prototype, and finally a user testing prototype. The fidelity of each version is

summarized in Table 4�19. Version 1 design and testing results were discussed in

Section 4.4.1. The remainder of this section details the development and functions of

Versions 2 through 4.

Description System

prototype version

Formal subject testing

Interactive Dynamic Medium

Initial low-fidelity design (visualization) v.1 ● Powerpoint

Component prototypes Powerpoint, Flash 5

Interactive prototype v.2 ● Powerpoint

Storyboard Powerpoint

User testing prototype v.3 ● ● ● Macromedia Director 8

Post user testing prototype v.4 ● ● Macromedia Director 8

Table 4-19 Prototype Fidelity

4.4.3.1. Interactive Prototype (Version 2)

The Version 2 interactive prototype (Figure 4�20) proved easy to produce and was

extremely useful in design meetings and subject training. Recommendations from low-

fidelity testing (Sanders & Willis, 2000) pertaining to layering and information accessing

were implemented in the single Powerpoint slide prototype using command buttons and

simple macros. This interaction allowed team members to �try out an interface and get

75

their hands dirty�[and]�to demonstrate how a chunk of an interface will work�

(Wagner, 1990).

Version 2 displays the merged Data Screen and Situation Displays from the design

document. The three sections in the singe monitor, which remained for Versions 3 and 4,

bring together the components:

The team continued to iterate the interactive prototype through cognitive walkthroughs.

Kellmeyer and Osga (2000) highlight the value of this method to identify �interaction

methods,�coding [requirements], or portions of the interface that were confusing to

operators.� Walkthroughs with the prototype facilitated improvement of symbology

sizing and design, addition of missile-target assignments, and experimentation with

coverage zones. The interactive prototype was valuable to programmers in the

Figure 4-20 Interactive Prototype (Version 2)

Message section

Timebar section

Tactical picture section

76

development of Versions 3 and 4, and was used in the training of subjects for usability

testing.

4.4.3.2. Storyboarding

The eventual weapon system and interface deployed to the Navy will interface with

numerous other systems on and off the vessel. Events in the interface will correspond to

actual decisions and events in the real world. In contrast, the development of a user

testing prototype requires one or more scenarios with a finite length and scripted events.

The dynamic prototype is thus an interactive movie seeded with events to elicit behaviors

that are of interest to usability testers. The team used a storyboard to design the testing

movies.

Sequences depicted in the storyboard (and the ensuing development of the user testing

prototype) followed the requirements of the user testing design. The experimental design

(Section 5.1.2) dictated the events in the user testing prototype (Figure 4-21).

Figure 4-21 Use of Storyboarding in Prototype Development

77

Storyboards are commonly used in the context of a complete use scenario to illustrate

how the new design will fit into the user�s world (Hackos & Redish, 1998; Wagner,

1990). As mentioned earlier, scenarios indicated that the user�s total tasks fall under two

broad categories: monitoring tasks and retargeting tasks. The storyboard �script� thus

walks through two scenes. First, a monitoring scene involves normal operation of default

missiles, a missed Health & Status point, and a flex targeting situation. Scene two

presents the subject with an emergent target and requires the operator to conduct

retargeting tasks.

Kellmeyer & Osga (2000) used low fidelity prototypes to guide the development of a

scripted storyboard for implementation in Macromedia Director. Similarly, the TTWCS

design team used Powerpoint to create a storyboard which was made dynamic and

interactive in Director. This set of slides is included as Appendix F. The storyboard

facilitated a significantly improved understanding between the designers and

programmers.

4.4.3.3. User Testing Prototypes (Versions 3 & 4)

The design document and storyboard together state the requirements for a medium-to-

high fidelity user testing prototype. Wagner (1990) confesses, �when I reach the stage of

user testing today, I must enlist an experienced programmer to write code to�add

minimal functionality.� Indeed, the functionality specified in the design document

proved too complex for the intermediate-level object oriented programmers on the design

team. The design document and a request for estimate were submitted to four graphic

design firms for development of the user testing prototype. We selected Creative

78

Perspectives, Inc. (name used with permission) of Charlottesville, Virginia due to their

demonstrated expertise, enthusiasm for the project, and cost.

The statement of work required the Version 3 prototype to be convincing and functional

enough to conduct user testing with the desired storyboard, while offering an affordable

man-hour requirement. The resulting coding allows the program to significantly �fake�

interaction concepts before they are translated beyond Director and into the product using

�real code� (Wagner, 1990; Kellmeyer & Osga, 2000). The Version 4 prototype is an

expansion of Version 3, and incorporates statistically-based as well as subjective

recommendations from the analysis of subject testing. The remainder of this section

describes current functions of Version 4 (the delivered product), followed by functions

recommended for future testing.

79

4.4.3.3.1. Current Functions

Screenshots of Versions 3 and 4 appear similar to the interactive prototype (Version 2)

constructed using Powerpoint. Many of the components need no explanation beyond that

found in Section 4.4.2. This section describes interface functions involving interacting

components.

4.4.3.3.1.1. Monitoring

Figure 4�22 depicts the Version 3 interface during nominal monitoring tasks.

Automation is minimal during these tasks, and the operator is not required to interact to

any specified degree. The use of layering and text minimization has been widely

recommended (DISA, 1996; Sanders & Willis, 2000). System automation is minimal

Figure 4-22 Interactive, Dynamic User Testing Prototype (Version 3)

80

during monitoring tasks. Versions 3 and 4 use radio buttons to automatically switch

between the following components:

• Routes or assignments

• Text details or ID labels

• Coverage zones (on or off)

• Coverage time factor

In addition, the operator can display details on a particular object (missile/target) by

mousing over the symbol, or by clicking the respective missile timebar.

For the receipt of messages, the system automatically highlights relevant objects in the

tactical picture with a colored box. For instance, a missed Health & Status point results

in a message, the actual point highlighted in yellow and remaining visible, and a yellow

box around the H & S point and missile symbol until the operator acknowledges the

message.

4.4.3.3.1.2. Retargeting

The TTWCS operator will be required to perform two types of retargeting tasks: flex and

emergent. Flex targeting involves commanding an inflight missile to service one of its

preplanned alternate targets, executed by branching off of its default route at a planned

waypoint, and flying a preplanned specified route. As explained in Section 4.3.2, flex

targeting decisions are triggered by an increase in the priority of a flex target beyond that

of the missile�s default target. This event is presented to the operator by:

81

• Notification in the message window of the priority increase

• Command in the message window to service the flex target

• Highlighting of relevant graphics in tactical picture

• Selection button adjacent to each possible missile (if more than

one)

A flex targeting event is depicted in Figure 4 � 23.

Similar to the Boeing UCAV system, where operator interaction varies over different

mission events, TTWCS Version 4 offers different levels of autonomy for different parts

of the mission (Wall, 2001c). The most automation, including decision support, is

provided for emergent retargeting tasks. On receipt of an emergent target message and

Figure 4-23 Flex Targeting Event

82

target symbol, the operator is presented with a pop-up window as shown in Figure 4�24.

Hopkin (1995) states that in air traffic control, �few problems are so urgent that the

controller should be interrupted immediately under all circumstances.� The system does

not execute the automation and decision support steps until instructed by the operator via

the �Enter Retarget Mode� button on the popup window. This feature allows the operator

to quickly complete any unfinished tasks, but remains visible and movable while

displaying a countup clock to convey urgency.

Once Retarget Mode is entered, the reduction of the solution space from all inflight

missiles to e-target candidate missiles involves the evaluation of the four rigid constraints

(Section 4.3.2). As each of the questions in the COA Identification section of the task

flow diagram (Figure 4�4) are Boolean criteria, the reduction of the solution space to

candidate missiles is performed instantly by a fully automated (Level 5) process (Endsley

& Kiris, 1995). This results in the following events in the interface (Figure 4�25):

• Candidates� symbol fill color turns pink

• E-target window (red vertical band) appears in timebar section

• E-target time of impact (TOI) graphics appear on candidate

timebars

Figure 4-24 Retarget Mode Popup Window

83

• Red rectangle appears around all relevant e-targeting graphics

• Non-candidate symbols and timebars �gray out�

• Candidate missile assignments are selected

• Non-candidate missile routes/assignments remove from visibility

Additionally, the Emergent Target Decision Support Tool (Section 4.4.2.9) is displayed

to facilitate COA Comparison. This Level 2 automated function (view recommendation,

query, act) presents processed information, and allows the operator to investigate a

recommendation (Endsley & Kiris, 1995).

Figure 4-25 Emergent Targeting Event (Prototype Version 4)

84

While the automated system processes the quantitative data and applies the default

weighting scheme, the human operator retains the freedom to draw on experience, and

adapt the decision to the situation (Schneidermann, 1997). This function is similar to

other uninhabited vehicles, including Boeing�s UCAV, where the operator can set new

rules of engagement during a mission (Kandebo, 2000b).

4.4.3.3.2. Future Functions

Future prototype development should include the following features.

• Incorporate a second monitor as indicated in Section 4.4.3, tailored to

expanded data presentation.

• Incorporate the ability to pan, zoom, and represent map scale in the

tactical picture. Add a constantly visible display of coordinates associated

with the cursor in coordinate units (DISA, 1996).

• Integrate airspace control measures (ACM). As ACMs have both altitude

and time attributes, new features and additional integration into the

timebar section is necessary. ACM start and stop times could be indicated

similar missile launch and burnout times: via an individual timebar for

each ACM. Altitude properties for the tactical picture could be queried

with a vertical altitude slider bar on the far right of the tactical picture

section.

85

• Integrate threat objects. Threats to inflight missiles include �electronic

attack directed at missile data links�basic AAA and SAM, and integrated

air defenses� (Operational Requirements Document, 1998). Threat and

ACM information should be layered into the interface similarly. The

interface should support the addition of ACM and threat objects manually

(via menu) as well as through the common operating picture (COP).

• To support �what-if� querying, the interface should enable the operator to

view the effects of a decision in a fast-forward mode prior to selection of a

course of action.

• Expand the decision support tools to accommodate multiple simultaneous

emergent targets, as well as common candidates for the multiple targets.

86

5 Subject Testing & Analysis

While early versions of the system and its components were iterated through informal

cognitive walkthroughs during design meetings, a primary step in the design process was

an iteration based on formal, statistically valid subject testing.

5.1. Experimental Plan

This section describes the planning and conduct of subject testing by addressing

hypotheses, experimental design, method, measures, testing scenarios, and the

participants.

5.1.1. Hypotheses

The study tests the following five hypotheses pertaining the two major types of

operator/system tasks.

During monitoring tasks:

• An operator�s overall situational awareness will decrease as the number of

total icons (missiles plus targets) increases.

• Operators using the �coverage zone� display feature will show poorer

performance and accuracy due to the added screen clutter than those

operators not using the feature.

87

During retargeting tasks:

• Operator situational awareness will decrease as the number of candidate

missiles increases.

• Operators using the �coverage zone� display feature will benefit from its

utility by showing better situational awareness than those operators not

using the feature.

We also expected to observe an interaction effect between the number of e-target

candidate missiles and the use of the coverage zone features.

5.1.2. Experimental Design

The split-plot experimental design facilitates separate experiments for each task. Control

variables were selected based on the hypotheses, and were manipulated as indicated in

Table 5�1.

Table 5-1 Experimental Design

The client indicated in the initial statement of work that a central area of interest was the

number of missiles an operator could reliably monitor and control. It was postulated that

operator workload in monitoring tasks depends on the sum of the number of targets and

Control Variable Values Variance

Monitor Experiment Number of objects 10 20 Within subjects

Display type CZ No CZ Within subjects

Retarget Experiment Number of candidates 2 4 Within subjects

Display type CZ No CZ Within subjects

88

missiles (objects) at a given time; �number of objects� was thus defined. Scenarios with

10 objects included 4 missiles and 6 targets, while those with 20 objects included 8

missiles with 12 targets. Display type restricted or required the use of coverage zones.

For experiments using CZs, subjects could change the time factor using the appropriate

interface controls, but could not disable the feature. Number of candidates for the

retargeting experiment pertained to the number of courses of action remaining after the

automated system reduced the solution space by applying the rigid constraints discussed

in Section 4.3.2.

To minimize both the number of scenarios required as well as testing time, each subject

did not see every combination of the control variables. We assumed a homogeneous

subject population, and that all subjects had the same cognitive processing abilities

following classroom training. The split-plot design allowed for the use of four testing

�movies�. Each consisted of an 8-minute monitor scene followed by a 5-minute

retargeting scene. The movies (annotated with circled numbers) included control variable

combinations as indicated in Table 5�2. Table 5�3 depicts the sequence of movies

administered to the randomized groups of subjects.

Group Monitor 1 Retarget 1 Monitor 2 Retarget 2

1 10 No CZ 2 No CZ 20 CZ 4 CZ

2 10 CZ 2 CZ 20 No CZ 4 No CZ

3 20 No CZ 2 No CZ 10 CZ 4 CZ

4 20 CZ 2 CZ 10 No CZ 4 No CZ

Table 5-2 Control Variable Combination

2

1 3

4

89

Movie Number Sequence

Training Testing

First movie Second movie Group 1 4 (No CZ) 1 (No CZ) 3 (CZ)

Group 2 4 (CZ) 1 (CZ) 3 (No CZ)

Group 3 1 (No CZ) 2 (No CZ) 4 (CZ)

Group 4 1 (CZ) 2 (CZ) 4 (No CZ)

Table 5-3 Group Testing Sequence

5.1.3. Method

The testing approach was modeled from successful testing of the Multi-Modal

Watchstation (MMWS) (Campbell, Freeman, & Hildebrand, 2000). To evaluate specific

interface functions, we presented scenarios that afforded the opportunity for relevant

behaviors to emerge, and measured those behaviors using objective and subjective

instruments. Major steps included the following:

• Pre-Test

o Two 1-hour classroom training sessions

o In-class written quiz

o Survey of age, gender, level of experience with graphical user

interfaces

o Individual review with subject

o One training movie

90

• On-line (during test) questions

o Measurement of response times

o Measurement of response accuracy

o Measurement of subject situational awareness

• Post-Test

o Interface evaluation survey

During monitoring scenes, subjects were required to maintain situational awareness

pertaining to combined coverage area, as well as answer questions pertaining to times of

impact and missile/target properties. Retargeting scenes facilitated the assessment of the

time-critical decisionmaking using the interface.

5.1.4. Online Measures

While the automatic time stamping of subject actions could have been employed within

the Director movies, test administrators measured subject response times using a

stopwatch. Most questions had a well defined �correct answer�, but accuracy was

measured on a scale of 1 to 4 (Table 5�4, next page). This method allowed for the

recording of �slips� by the subject, where they change their answer in a �wait-a-second�

fashion. The intent with this scale is to identify possible idiosyncrasies in the design of

the interface that could lead to operator errors.

91

Accuracy of Response Accuracy Rating Incorrect 1

Right to Wrong 2

Wrong to Right 3

Correct 4

Table 5-4 Accuracy Scale

Testing administrators used the subjective technique of �online observer rating� to

measure situational awareness for several questions (Endsley, 1995a). A scale of 1 to 7

was used.

5.1.5. Scenario Description

Kellmeyer & Osga (2000) recommend testing durations of not more than one hour. As

the lifetime of a single missile exceeds this significantly, there is a trade-off between

duration of the testing movies and fidelity of the scenarios. As mentioned, each testing

movie consists of an 8-minute monitoring scene followed by a 5-minute retargeting

scene. The monitor scene begins with the launch of the first missile, and plays at 30-

times speed until all missiles have been launched and flown to their initial waypoints.

The movie then assumes normal one-to-one time. Endsley (1995a) states that �though

SA consists of an operator�s knowledge of the state of the environment at any point in

time, this knowledge includes temporal aspects relating to the past.� The accelerated

time method was used to foster better situational awareness at the start of the scenario.

Questions were posed to the subjects via the Message Section of the interface to

minimize distractions by the observers, and to evaluate the usability of the message

92

window component. Each developed question supported at most two hypotheses. The

following questions were posed during the monitoring scene of each testing movie.

5.1.5.1. Monitoring Scene Questions

1. �Say aloud the Missile ID number of the missile that would reach its default target

first if all missiles were commanded to go directly to their default target.�

2. �Say aloud the Missile ID numbers of all the missiles that will impact before

12:30. �

3. �In the timebar section, point to the time bar associated with the southern most

missile. �

4. �Say aloud the number of targets that require or prefer more than one missile. �

5. �Say aloud the Missile ID numbers of soft warhead missiles servicing targets with

a priority of 6 or more. �

6. �How many missiles are you monitoring right now? �

7. �Describe or show the general areas in which you could NOT service an e-target

whose expected move time is 12 minutes from now. �

Testing movies continue without interruption from the monitor scene to the retargeting

scene. The retargeting scene begins with a flex mission, where the priority of a flex

target common to two missiles increases to a value greater than both missiles� default

93

targets. Finally, an e-targeting mission is initiated with the appearance of a single

emergent target. The following messages and/or situations are presented to the subject:

5.1.5.2. Retargeting Scene Questions

1. Retarget a flex missile to a flex target given that its priority has increased to 9.

2. �Point to the candidate missiles that, if retargeted, will sacrifice both their default

target as well as a flex target. �

3. �In the tactical picture, point to the candidate missile that can reach the e-target

first. �

4. �With respect to target priority, time to get there, and fuel remaining, verbally

compare the candidates. Is there a dominating solution? �

5. �If the system were designed to automatically select the best missile for an e-

target, why do you think it would select missile number [X] in this case? �

5.1.6. Participants

Kellmeyer & Osga (2000) discuss the importance and difficulty of obtaining skilled

Naval operators for usability testing of Navy systems. For the TTWCS testing, a limited

budget and geographical displacement from any military base exacerbated this issue.

Thus, the subject pool consisted of twenty graduate students in the Systems Engineering

Department at the University of Virginia. All were students in a Cognitive Systems

Engineering course, and were asked to participate in the study because of its relevance to

the course content of the class.

94

The preparation for subject testing was thorough, and generally followed

recommendations found in Gomoll (1990). Pre-test preparations included completion of

institutional review board requirements, two hours of classroom training, an in-class

written quiz, a pre-testing refresher briefing and subject backbrief, and a practice

iteration. The team established a goal to train the subjects to the level of an operator with

baseline training and experience who had been on shift for one hour. Subjects began

testing only after demonstrating a firm grasp of their goals and tasks, as well as the

operation of the interface. Once subjects showed a thorough grasp of the rules and

techniques of the system during the backbrief, they were administered the two testing

iterations, and finally a post-test questionnaire.

5.2. Results

5.2.1. Objective Results

Almost all of the subject responses scored a �4� (correct) on the accuracy scale. Thus,

accuracy results fail to show a dependence on any of the experimental variables.

Statistical evaluation of response times, however, indicate that the number of objects

present on the screen and the absence or presence of coverage zones were significant

factors for many questions. Not surprisingly, the number of objects appears to affect

response times for questions pertaining to the sequential evaluation of individual objects.

These questions included:

�Say aloud the number of targets that require or prefer more than one

missile.� (R2 = 0.2914, p-value = 0.0023)

95

�Say aloud the missile ID number of soft warhead missiles servicing

targets with a priority of six or more.� (R2 = 0.1947, p-value = 0.008)

Contrastingly, questions requiring a comparison between objects, especially if the

comparison was made using the timebar section of the interface, did not show a

significant dependence on the number of objects. These questions included:

�Say aloud the Missile ID number of the missile that would reach its

default target first if all missiles were commanded to go directly to their

default target.� (R2 = 0.0024, p-value = Insignificant)

�Say aloud the Missile ID numbers of all the missiles that will impact

before 12:30.� (R2 = 0.0305, p-value = Insignificant)

This finding lends credibility to the design of the timebar component in the interface.

Using the feature, the comparison of missile-time attributes does not grow more difficult

(as measured by response time) with an increase in the number of missiles and targets (at

least up to 8 missiles and 12 targets, the maximum tested.)

The absence or presence of coverage zones during testing scenarios was found to

significantly affect the response time as well as the rated situational awareness score for

one question:

�Describe or show the general areas in which you could NOT service an e-

target whose expected movetime is 12 minutes from now.� (Response

time R2 = 0.1518, p-value = .0377) (Situational awareness R2 = 0.3800, p-

value < .0001)

This finding indicates that subjects can more effectively (as measured by SA and

response time) monitor the combined loiter coverage over an entire area of operations by

96

using the coverage zone feature. Results from the analysis of variance are tabulated for

the monitor and retarget experiments in Appendix H.

5.2.2. Subjective Results

5.2.2.1. Automation

To explore the effects of potential automation in retargeting tasks, subjects were asked

two similar questions requiring different mental processing. The first required a step-by-

step analysis of the retargeting criteria without the aid of decision support:

�With respect to target priority, time to get there, and fuel remaining,

verbally compare the candidates. Is there a dominating solution?�

The second question required the subject to verbalize an understanding of the reasoning

behind a Level 5 automated selection:

�If the system were to automatically execute this e-targeting mission with

Missile 010, why do you think it would do so?�

There was no significant measured difference in SA between manual decisionmaking and

a subject�s verbalized understanding of the automated selection. However, the

unautomated case required subjects to develop and maintain a mental picture, while the

potentially automated (Level 4 or 5) situation required them to �catch up� to the situation

when asked the question. This potential loss of SA could be attributable to the

�difference between active and passive processing of information� (Endsley & Kiris,

1995).

97

The most prominent interface difficulty involved correlating a missile symbol in the

tactical picture section with its timebar, especially when 8 missiles were present . The

lack of proximity (Norman, 1990) between the two features can be addressed in a variety

of ways, including color, but should not violate the color standards in MIL-STD-2525B

or the guidelines in the DoD HCI Style guide.

5.2.2.2. Workload

Many subjects grasped the interface methods and rules quickly, and benefited during low

workload periods with relatively high SA. Other subjects, though, demonstrated

evidence of both �low SA, low workload�, where vigilance and motivation issues seemed

to become relevant factors. For example, when asked the question pertaining to

combined missile coverage area, some subjects appeared rushed and perplexed, even

though they were not performing any other tasks. Testing also revealed cases of �low

SA, high workload�, where the subjects appeared overwhelmed (Endsley, 1995b). This

was most common during the retargeting analysis, where subjects sometimes talked in

circles about which candidate to select. Had testing been conducted using subject matter

experts as participants, it is reasonable to expect a more homogeneous and higher SA

group.

5.2.2.3. Number of Objects

Many subjects noted in the post-test survey that 20 objects was too many to reliably

monitor; some believed they could handle up to 20 objects only with additional practice.

Very few subjects predicted that they would be able to handle more than 20 objects

98

simultaneously. As mentioned, accuracy did not suffer when the number of missiles and

targets increased; response time did suffer for certain tasks.

5.2.2.4. Optional Features

Kellmeyer & Osga (2000) observed that subjects placed in the dynamic MMWS

environment tended to use only small portions of the interface features, while unused

features cluttered the display. Similarly, most subjects answering questions concerning

missile coverage of the area of responsibility indicated that the coverage zones

component was useful. However, many felt that the coverage zones cluttered the screen

and were distracting during tasks not requiring an assessment of coverage area. Nearly

every subject wanted the ability to turn the coverage zones on and off (a feature not

included in the test scenarios).

5.2.2.5. Expert-Novice Effects

After arguably thorough training and verification of subject understanding of goals and

tools prior to testing, many subjects confirmed that �learning both a new technology and

a new program at the same time can be overwhelming� (Hackos & Redish, 1998). Some

grew impatient with learning rather than performing, showed only a theoretical

understanding of the system, and used interface functions randomly. Very few showed a

comprehensive and consistent mental model of the functionality of the interface

indicative of expert performance.

The limited access to subject matter expertise in the field of cruise missile operation or

even air traffic control potentially had a negative impact on the development efforts. The

99

overall process can be described as �design for the expert, test on the novice.� However,

the participants were adequate for this level of testing, especially given this new weapon

technology. Future testing using more robust prototypes should be conducted with Navy

personnel.

Cultural and motivational issues may have further tainted the results. While the

observers noted a �level of motivation� and �level of understanding� score for each

subject following the refresher briefing, nationality was not recorded. Some of the

subjects, while seeming to grasp the intent during training, suffered from a lack of

command of the English language. Gomoll (1990) describes the importance of finding

�people who have the same experience level as the typical user�. In retrospect, we

should have only tested subjects with native English-speaking capability, as this may be a

more representative subject population for a system intended for use by U.S. Navy

personnel.

5.2.2.6. SA Measurement

While the measurement of situational awareness provided significant results, the use of

an online observer rating to obtain the raw data has the drawback that the observer can

only see �operator actions and imbedded or elicited verbalizations� (Endsley, 1995a).

The technique does not provide a complete representation of unverbalized internal

information. There is almost certainly experimental bias in the manner of scoring. For

example, test administrators fully expected subjects to better understand the concept of

combined missile coverage area when they used the coverage zone features. The

administrators could have been more likely to rate subjects higher on the SA scale during

100

tests using the coverage zones. Future testing methods should employ a more robust

method of SA measurement than online observer ratings.

101

6 Conclusions

6.1. Design Iteration

During the pre-test briefing, test administrators made clear to each subject that any

difficulties they had during the testing were attributable to faulty interface design, and not

to the participants themselves (Gomoll, 1990). As a group, the subjects performed well

on the individual reviews prior to testing, which lends confidence to this notion. In the

weeks following subject testing, the next iteration of designer-programmer modification

ensued to enhance the user-testing interface (Version 3) based on the results from the

testing. The most notable development was the addition of the Emergent Target Decision

Support Tool described in Section 4.4.2.9. Use of the ETDST supports the general intent

of decision aids to �present alternatives, supporting evidence, and assist the user in

evaluating the alternatives� (DISA, 1996). Future testing will reveal the true worth of

this component in the TTWCS interface.

6.2. Operator Workload

By current requirements, both TTWCS and UCAV operators would be able to control

about 4 vehicles simultaneously during a mission (Operational Requirements Document,

1998; Kandebo, 2000b). The issue is that the number of missiles that an operator can

handle will partially depend on the effectiveness of the interface in combating vigilance

during monitoring tasks, and the robustness of the decision support structure during

retargeting tasks. Though this study fails to offer a correlation of execution time to

number of missiles, it is clear that the vigilance required to maintain SA when monitoring

102

many missiles poses a moderate challenge. Though not tested, retargeting tasks

involving more than four candidates for a single target may be overwhelming even with

the use of a Level 3 automated decision aid. Decisionmaking in the face of multiple e-

candidates is even more complex and time-critical. Indeed, the appearance of two or

more e-targets having common candidates will almost certainly require an automated

decision aid.

6.3. Level of Automation

Hopkin (1995) states that in the domain of air traffic control, �the idea that machines

should implement decisions without the knowledge of the controller or pilot�appears to

be receding.� This argues against the notion of a fully automated (Level 5) retargeting

decision scheme. The ETDST uses Level 3 automation, where the operator is presented a

recommendation along with quickly interpretable supporting data, and may either concur

with the recommendation to execute the mission, or may examine supporting data and

execute a non-recommended course of action. The ETDST provides to the operator both

individual criteria �scores� for each candidate, as well the ability to rapidly explore

sensitivity to other weighting schemes.

In what cases would the operator select a course of action deemed inferior by the semi-

automated system? Given an accept/reject recommendation, what are permissible

grounds for overriding the automated solution? The operator would require �possession

of knowledge that invalidates the computer decision, and which the computer could not

have used in its formulation (Hopkin, 1995).� Further, this �implies that the controller

knows the entire basis for the automated decision, including criteria and relative weights�

103

(ibid.). For example, a recommended candidate missile and its emergent route may be in

close proximity to an area in which the Air Force frequently and with little notice

activates a restricted fly zone to conduct attacks with manned aircraft. Though the

airspace is currently clear, the probability of a restrictive airspace control measure

appearing between the recommended missile and its emergent target might be enough to

warrant selection of the �second-place� candidate.

6.4. Location of Interface

The Operational Requirements Document (1998) states that �TTWCS will not control

missiles that have been launched by other platforms.� However, the notion of a land

warfare coordinator controlling all Naval land attack assets in a given area of

responsibility suggests the placement of the interface on a battlegroup command vessel

monitoring and perhaps controlling missiles launched by all firing vessels. Alternatively,

it could be feasible to transfer control of missiles between operators. The network

control architecture employed in Boeing�s UCAV system enables seamless hand-off of a

vehicle between operators (Kandebo, 2000b).

Kellmeyer & Osga (2000) point out that �most current Naval systems were designed

around a one system, one console, one operator paradigm�. The move away from this

�stovepipe� approach to system integration will lead to the service�s desired goals of

reduced manning, consoles, and training. Wherever the TTWCS interface is located, it

should itself be a component in a common land attack watchstation with a robust human

interface.

104

With the freedom to make retargeting decisions comes significant responsibility. Does

the land warfare coordinator have all the information relevant and necessary to make the

decision? The provided Emergent Targeting Decision Support Tool takes into account

only criteria pertaining to the TTWCS. It is arguable that a fully informed decision must

be made keeping in mind the goals of other Naval land attack systems as well as other air

assets in the area of responsibility. A continually updated common operating picture can

facilitate these information requirements, but a broader encompassing decision support

system would likely be necessary.

As the unique feature of the Tactical Tomahawk involves loitering to extend the ability to

service emergent targets, the optimization of the loiter routes and thus coverage areas

over the area of responsibility is critical. Any selection of a candidate missile for an

emergent target leaves a gap in the previously optimized combined coverage patterns.

Re-optimization could be accomplished by two approaches. The first could use

technology similar to Northrop Grumman�s UCAV, which incorporates �an inflight data

link [for the air vehicles] to communicate with each other� (Fulgham & Wall, 2001b). If

the loitering missiles continually communicated current loiter areas to each other, re-

optimization could be done onboard the vehicle. A second approach could involve a re-

optimization scheme calculated at the operator interface and transmitted to all inflight

missiles upon selection of a candidate.

6.5. Application to Other Fields

Research and findings from this study are applicable to the control of other semi-

autonomous vehicle systems, as well as in analagous domains. The monitoring and

105

inflight mission modification techniques presented are certainly relevant to similar tasks

in the UCAV system, where �an operator [will] monitor vehicle conditions and be

involved in altering a flight or mission path�, as well as deciding �which UCAV in a

multiship flight should prosecute [an] attack� (Kandebo, 2000b). The methodology

proposed in Appendix G could be used by personnel in the time-critical targeting cell to

rapidly assess whether a target is suitable for Tomahawk attack with respect to required

attack heading and dive angle. The timebar component has been suggested for use in

passenger train monitoring (M. Evans, personal communication, April 9, 2001). The

concepts of viewing and optimizing coverage areas could be applied to police patrol car

or taxi routing, where the appearance of an �emergent activity� (crime scene, new

passenger) is analogous to the emergent target in the TTWCS. Finally, the Emergent

Target Decision Support Tool suggested could easily be adapted to any type of

decisionmaking scheme where an operator wishes to be presented with a

recommendation based on a set of predefined criteria at variable weights.

106

References Athay, R.N. (2000). Loiter and optimal route planning for unmanned aircraft. A dissertation

proposal to the Faculty of the University of Virginia School of Engineering and Applied Science.

Baker, S. (2001, February 28). Predator hellfire missile tests �totally successful�.[WWW document]. URL http://www.afmc.wpafb.af.mil/HQ-AFMC/PA/news/archive/2001/feb/predator_hellfire_test.htm. Retrieved March 5, 2001.

Campbell, G.E. & Cannon-Bowers, J.A. (n.d.). The application of human performance models in the design and operation of complex systems[WWW document]. URL http://www.manningaffordability.com/S&tweb/. Retrieved December 4, 2000.

Campbell, G.E., Freeman, J.T., & Hildebrand, G. (2000). Measuring the impact of advanced technologies and reorganization on human performance in a combat information center. In the Proceedings of the Human Factors and Ergonomics Society 44th Annual Meeting, San Diego, California.

Campbell, G.E., Pharmer, J.A., & Hildebrand, G. (2000). Measuring the impact of human-centric design on human performance in a combat information center[WWW document]. URL http://www.manningaffordability.com/S&tweb/. Retrieved January 15, 2001.

Carmody, M.A. & Gluckman, J.P. (1993). Task-specific effects of automation and automation failure on performance, workload and situational awareness. In the Proceedings of the Seventh International Symposium on Aviation Psychology (pp.167-171). Columbus, OH: The Ohio State University, Department of Aviation.

Danzig, R. (2000, January 6). Department of Defense briefing: Application of new technologies to Navy destroyers[WWW document]. URL http://www.prop1.org/nucnews/2000nn/0001nn/000106nn.htm. Washington, D.C. Retrieved April 19, 2001.

Defense Information Systems Agency (1996). Technical Architecture Framework for Information Management, Volume 8: Department of Defense (DoD) Human-Computer Interaction (HCI) Style Guide, Version 3.0.

Department of Defense (2000). Report to Congress: Kosovo/Operation Allied Force after action report[WWW document]. http://www.defenselink.mil/pubs/kaar02072000.pdf. Retrieved April 19, 2001.

Endsley, M. (1987). SAGAT: A methodology for the measurement of situation awareness (NOR DOC 87-83) Hawthorne, CA: Northrop Corp.

Endsley, M. (1988). Situation awareness global assessment technique (SAGAT). In Proceedings of the National Aerospace and Electronics Conference (NAECON) (pp.789-795). New York: IEEE.

107

Endsley, M. (1995a, March). Measurement of situational awareness in dynamic systems. Human Factors, 37 (1), 65-84.

Endsley, M. (1995b, March). Toward a theory of situational awareness in dynamic systems. Human Factors, 37 (1), 32-64.

Endsley, M., & Kiris, E. (1995). The out-of-the-loop performance problem and level of control in automation. Human Factors, 37 (2), 381-394.

Erickson, T.D. (1990). Working with interface metaphors. In B. Laurel (Ed.). The art of human-computer interface design (pp.65-73). Reading, MA: Addison-Wesley.

Owens, R.C. (1997). The Balkans air campaign study.[WWW document]. URL http://www.fas.org/man/dod-101/ops/docs/apj--fall97-owen.html. Retrieved January 15, 2001.

Federico, P.A. (1995, March). Expert and novice recognition of similar situations. Human Factors, 37 (1), 105-122.

FM 90-36 (1997). The joint targeting process and procedures for targeting time-critical targets. Department of Defense.

Fulgham, D.A. & Wall, R. (2000a, September 25). Hypersonic Weapons Attack Time Problem. Aviation Week and Space Technology, 153 (13) 88-89.

Fulgham, D.A. & Wall, R. (2000b, September 25). New munitions mandate: more focused firepower. Aviation Week and Space Technology, 153 (13) 78-79.

Fulgham, D.A. & Wall, R. (2001a, February 19). Navy�s hairy buffalo aims for quick kill. Aviation Week & Space Technology, 154 (8), 56-57.

Fulgham, D.A. & Wall, R. (2001b, February 19). New demonstrator spurs Navy UCAV development. Aviation Week & Space Technology, 154 (8), 52-54.

Fulgham, D.A. (2000, October 9). US Navy eyes full range of unmanned aircraft. Aviation Week & Space Technology, 153 (19), 59-60.

Fulgham, D.A. (2001, February 26). USAF plans rapid, all-stealth task force. Aviation Week & Space Technology, 154 (9), 24-25.

Gibson, J. (1991). How to do systems analysis. Charlottesville, VA: PS Publishing.

Gomoll, K. (1990). Some techniques for observing users. In B. Laurel (Ed.). The art of human-computer interface design (pp.85-90). Reading, MA: Addison-Wesley.

Guerlain, S., Brown, D., & Lambert, J. (2001). Information fusion for decision teams. Grant application submitted to the U.S. Air Force Office of Sponsored Research.

108

Hackos, J.T. & Redish, J.C. (1998). User and task analysis for interface design. New York: Wiley & Sons.

Hopkin, V.D. (1995). Human factors in air traffic control. Bristol, PA: Taylor & Francis.

Kandebo, S.W. (2000a, October 2). Boeing premieres UCAV demonstrator. Aviation Week & Space Technology, 153 (14), 30-33.

Kandebo, S.W. (2000b, October 2). Suppression of enemy air defense, other capabilities planned for UCAVs. Aviation Week & Space Technology, 153 (14), 30-31.

Kellmeyer, D. & Osga, G.A. (2000). Usability testing & analysis of advanced MultiModal watchstation functions. In the Proceedings of the Human Factors and Ergonomics Society 44th Annual Meeting, San Diego, California.

Lockheed-Martin (n.d.). Tactical Tomahawk Weapons Control System[WWW document] URL http://www.lockheedmartin.com/factsheets/product174.html. Retrieved March 6, 2001.

LoPresto, M. & Rice, K. (2000). Tomahawk Strike Network[Microsoft Powerpoint presentation]. Johns Hopkins University Applied Physics Lab.

MIL-STD 2525B (1999). Common warfighting symbology. Department of Defense.

Morrocco, J.D. (2000a, October 9). Dassault flight tests stealthy UAV design. Aviation Week & Space Technology, 153 (15), 33-34.

Morrocco, J.D. (2000b, October 9). Europe joining forces on future air systems. Aviation Week & Space Technology, 153 (15), 32-33.

Mountford, S.J. (1990) Tools and techniques for creative design. In B. Laurel (Ed.). The art of human-computer interface design (pp.17-30). Reading, MA: Addison-Wesley

Mullen, M.G. (2000, September 11). Surface Combatant Land Attack Warfare Guidance Document. Department of the Navy Memorandum.

National Research Council, Committee to Review DOD C4I Plans and Programs. (1999) Realizing the potential of C4I. Washington, D.C.: National Academy Press.

Naval Surface Warfare Center (NSWC) (2000). Integrated command environment demonstration[brochure], Dahlgren, VA.

Norman, D. (1990a). Why interfaces don�t work. In B. Laurel (Ed.). The art of human-computer interface design (pp.209-219). Reading, MA: Addison-Wesley.

Norman, D. A. (1990b). The psychology of everyday things. New York: Basic Books, Inc.

Operational Requirements Document (ORD) for Tomahawk Weapon System (TWS) Baseline IV, Revision 2, January, 1998, UNCLASSIFIED.

109

Operator-Machine Interface Commonality Working Group (OMI CMG) (2001). Undersea warfare (USW) control and display standards and conventions, Version 1.1.

Parasuraman, R., Molloy, R., Singh, I.L. (1993). Performance consequences of automation-induced complacency. The International Journal of Aviation Psychology, 3 (1), 1-23.

Pike, J. (1997, November 11). Cruise missile support activities (CMSA)[WWW document]. URL http://www.fas.org/irp/agency/dod/uspacom/cmsa/. Retrieved January 15, 2001.

Sanders, D. & Willis, R. (2000). Tactical Tomahawk Missile User Interface Analysis. Department of Systems Engineering, University of Virginia.

Schneidermann, B. (1997). Designing the user interface: strategies of effective human-computer interaction (3rd ed.). Reading, MA: Addison-Wesley.

Scott, W.B. (2000, October 2). Experimental center nails time-critical targets. Aviation Week & Space Technology, 153 (14), 70-73.

SHAI (Stottler Henke Associates, Inc.). Automating real-time sensor equipment operation and sensor data collection[WWW document]. URL http://www.shai.com/projects/sensors.htm. Retrieved March 24, 2001.

Sheridan, T.B. (1987). Supervisory control. In G. Salvendy (Ed.) Handbook of Human Factors (pp. 1243-1267). New York: Wiley & Sons, Inc.

Smith, K. & Hancock, P.A. (1995, March). Situational awareness is adaptive, externally directed consciousness. Human Factors 37, (1) 137-148.

Tufte, E. (1990). Envisioning information. Cheshire, CT: Graphics Press.

Vicente, K. (1999). Cognitive work analysis: toward safe, productive, and healthy computer-based work. Mahwah, New Jersey: Lawrence Erlbaum Associates.

Voth, P. (2000, December 22). Military analysis network: BGM-109 Tomahawk[WWW document] URL http://www.fas.org/man/dod-101/sys/smart/bgm-109.htm. Retrieved January 15, 2001.

Wagner, A. (1990). Prototyping: a day in the life of an interface designer. In B. Laurel (Ed.) The art of human-computer interface design (pp.79-84). Reading, MA: Addison-Wesley.

Wall, R. (2001a, January 1). Naval UCAV concepts starting to crystallize. Aviation Week & Space Technology, 154 (1), 58.

Wall, R. (2001b, January 22). Hyperspectral recce piques USAF interest. Aviation Week & Space Technology, 154 (4), 55.

Wall, R. (2001c, January 29). USAF updates plans for future air wars. Aviation Week & Space Technology, 154 (5), 61-62.

110

Wickens, C.D. (1992). Workload and situation awareness: An analogy of history and implications. Insight: The Visual Performance Technical Group Newsletter, 14 (4), 1-3.

Williams, R. (1994). The non-designer�s design book. Berkeley, CA: Peachpit Press.

A - 1

Appendix A – Glossary Attack Angles A pair of angles (0-360 degrees) representing the directions from

which incoming missiles must attack the target due to tactical considerations. Represented in the tactical picture as a pair of brackets extending from a target symbol. Lack of brackets indicates no attack angle restriction.

Branch Point A predetermined point on a missile route where a missile can change its course in order to attack a preplanned flex target.

Candidate Missile A missile that satisfies all the criteria needed to attack an emergent target (including fuel remaining, coverage area, and warhead type)

Coverage Zone The area that a missile can reach in a specified time, which is selectable by the operator. See �coverage time factor�.

Coverage Time Factor The length of time that defines the size/extent of a missile�s coverage zone. Selectable with radio buttons on the tactical picture. �Lifetime� indicates the set of points that a missile can reach before fuel burnout. Other time options are 30, 20, and 10 minutes, as well as a specified real clock time.

Default Missile

A missile currently on a route to its default target. All missiles are launched as d-missiles.

Default Target

A preplanned primary target. A missile will strike its default target if the operator does not command it to retarget to a flex or emergent target.

Emergent Missile

A missile that has been selected by the operator to abandon its default route and target, and execute an automatically planned route to an emergent target from the missile�s current position.

Emergent Target A high priority, time-critical target that is issued to the operator/interface by voice/data from a higher headquarters targeting center. Usually exists for a short duration; comes with a predicted move time. No preplanned routes or missile assignment.

A - 2

Expected movetime The time that an emergent target is predicted to move. Based on a probability distribution that is unique to the target description and theater of operations. Given by a higher headquarters targeting center. The probability that a target�s location will not change by the expected movetime is greater than 90%.

Flex Missile A missile that an operator has selected to abandon its default route to execute a preplanned route to a flex target of higher priority. The preplanned route branches from the default route at a branch point, not the missile�s current position.

Flex Route A preplanned route branching from a missile�s default route at a branch point; leads to a flex target.

Flex Target A preplanned alternate target. Has a preplanned route branching from a default route. May have preplanned routes from multiple d-missiles. Selected for attack if its priority increases (above that of default targets).

Flight Level Altitude above sea level (hundreds of feet) (FL240 = 24000 feet)

Hard Target A target requiring a direct impact with high explosive for the desired effect. Examples are concrete bridges, bunkers, buildings. Compare with soft target.

Hard Warhead Detonates on impact with a target; high explosive.

Heading Indicator A line extending from the center of the missile icon, indicating the current direction of travel.

Health/Status Point A preplanned point along a missile�s route where the missile is expected to transmit location and health status to the operator.

Loiter Pattern An area or set of waypoints for a missile to fly to delay striking its default target for the opportunity to strike a higher priority flex or emergent target.

Missile Health Nominal, caution, or alert conditions transmitted by the missile to the operator. Caution and alert conditions are also deduced from the lack of a transmission at a planned Health/Status point.

Missile ID # Missile ID # numbers are in the form M###-Z#-Y

(M=missile, ### = missile number, Z = D/F/E for default, flex, or emergent missile, # = priority of the target currently being serviced, Y = H/S for hard or soft warhead on the missile)

A - 3

Number of Missiles Required

Property of a target. The number of missiles required to create the desired effects on the target.

Number of missiles flexibility

Two values: �if possible� indicates that the target should still be serviced with fewer missiles than that required if additional missiles are not available. �All or none� indicates that the target should be serviced only if the number of missiles required are available.

Priority A rating of target importance on a scale of 1 � 10 (10 being highest). Used to determine the order in which targets should be attacked.

Soft Target A target requiring attack over a large or poorly defined area. Examples are enemy vehicle convoys, large troop formations or assembly areas.

Soft Warhead Detonates high above the target, and disperses hundreds of �cluster munitions� over a large area around the target.

Tactical Picture Real-time, overhead-view map of the theater of operations with icons representing the current position and status of all missiles, targets, vessels, routes, etc. The primary source of spatial information for the operator.

Target ID Number Target ID numbers are in the form T###-Z#-Y

(T = target, ### = target number, Z = D/E/F for default, flex or emergent target, # = target priority, Y = H/S for hard or soft target).

Time to Impact If Retargeted (TIIF)

Property of a candidate missile. The time that the missile would reach an e-target if the operator selected it to service the e-target.

Vessel ID # Two digit number associated with a launching vessel

B - 1

Appendix B - Object definition

Targets

ID number

Coded number

Location

Latitude-longitude grid coordinate of aimpoint

Altitude

Feet above mean sea level (MSL) of aimpoint

Warhead required

Hard - steel-concrete/hardened structures including bridges, buildings, bunkers, etc.

Soft - area target e.g. vehicles, tactical command post, SAM battery

Priority - measure of importance of target, taking into account timeliness

Low - target to be attacked at a convenient time if missile supply allows

Medium - valuable target to be attacked any time in next 24-48 hours

High - high value target that must be serviced at a planned time (including immediately) to support other tactical operations

Attack heading constraints � a pair of headings (min and max) describing the allowable impact headings (given by intelligence, may be due to surrounding terrain (terrestrial) or nature of construction of target or nearby collateral objects

Entities in the system are defined by:

Objects Properties Values

Major types of objects: Targets

Missiles Constraints Launch vessels Missions Battlegroup Campaign

B - 2

(tactical)). For example, (030, 125) indicates the target must be attacked from between 030 and 125 degrees (the attacking missile must have a heading between 210 and 305 at impact)

Scheduled attack heading - planned final course of missile at impact

0 to 360 degrees, discretized to 36 10-degree increments

Dive angle constraints - required angle between level ground and missile trajectory at impact; varies depending on construction of target

Shallow - 0 to 15 degrees

Medium - 16 to 45 degrees

Steep - 46 to 90 degrees

Dive angle - planned final dive angle of missile at impact

0 to 90 degrees

Time of appearance - What time did it get there?

Clock time

Predicted duration - How long do these types of targets usually stick around?

Random time variable - sampled from a theoretical distribution

Expected movetime - What time does intelligence think (with 95% confidence) it will move?

Clock time

Countdown to expected time to move - How long until it is expected to move?

Stopwatch time

B - 3

Missiles

ID number

Coded number

Time of scheduled launch

Clock time

Time of actual launch

Clock time

Fuel remaining - until burnout, calculated at constant burn rate for the program (updated during missile health queries in flight)

Hours/minutes countdown

Time of fuel burnout

Clock time

Time of impact � time of missile impact with currently targeted target

Clock time

Time to impact � with currently targeted target

Hours/minutes countdown

Straightline time of impact � time that missile would impact if it were commanded to fly direct from its next waypoint to the selected target (compensated for ACM/threat)

Clock time

Current location

Lat-long grid coordinate

Current altitude

Feet above MSL

Current speed - assumed constant (500 knots/550 mph)

B - 4

Warhead/fuse combination - the type of explosion that will happen, and the manner in which the explosion is initiated; depends on Target Description

Hard warhead

Soft warhead

Launch vessel ID number - identification of vessel from which the missile was launched

Text (e.g., DD21, DD18)

Launch vessel location at launch - location of the vessel when it launched this missile

Lat-long grid coordinate

Launch vessel current location

Lat-long grid coordinate

Historical path - defined by its launch location and the series of turns (does not represent altitude)

Series of grid coordinates

Scheduled future path - defined by current location, scheduled turns, and current target location

Series of grid coordinates

Coverage area - defined by the corners of the polygon (2-D top view) representing the limits that a missile can reach under specified constraints

Constrained by: - political or country boundaries - airspace control measures (3-D) - other missile coverage areas

Coverage time factor

Lifetime (until fuel burnout), 60, 45, 30, 20, 10 minutes

B - 5

Constraints

Geographic boundaries (object) –

A polygon which approximately covers the area of operations (usually a country's borders) and is defined by a reasonable number of corner points

Corner point #1

Lat-long coordinates

Corner point #2�

Lat-long coordinates

Airspace control measures (object) -

ACM ID number

Coded number

Type � either airspace control measure or threat

ACM - blocks of airspace (3-D) designated by the command and control structure through which the Tomahawk is prohibited from flying due to risk of interference with other friendly operations

Threat – type of airspace control measure; a block of airspace through which penetrating missiles will incur varying degrees of risk of interception or destruction

Risk if penetrated � measure of consequence if missile penetrates ACM

Low Medium High (default value for ACM)

Location of corners

Lat-long coordinates

Location of center (if applicable)

Lat-long coordinates

Radius (if applicable)

Miles

B - 6

Floor - effective min altitude

Feet MSL

Ceiling - effective max altitude

Feet MSL

Start/stop times

Clock time

Launch vessels

ID number

Text (e.g., DD21, DD18)

Name

Text (e.g., USS Arleigh Burke)

Type vessel Destroyer Submarine Other

Current location

Lat-long grid coordinate

Location at launch of missile #1(missile ID number)

Lat-long grid coordinate

Location at launch of missile #2(missile ID number)�

Lat-long grid coordinate

Time at launch of missile #1(missile ID number)

Lat-long grid coordinate

Time at launch of missile #2(missile ID number)

Lat-long grid coordinate

B - 7

Missile max capacity

Maximum number of missile the vessel can hold

Initial inventory soft � # of soft warhead missiles with which a vessel began a campaign

Integer

Inflight soft �

Integer

Impacted soft �

Integer

Lost soft �

Integer

Fired soft � sum of three above

Integer

Available soft � initial inventory minus number fired

Integer

Initial inventory hard � # of soft warhead missiles with which a vessel began a campaign

Integer

Inflight hard �

Integer

Impacted hard �

Integer

Lost hard �

Integer

Fired hard � sum of three above

Integer

Available hard � initial inventory minus number fired

B - 8

Integer

Initial inventory missiles � total # of missiles with which a vessel began a campaign

Integer

Inflight missiles �

Integer

Impacted missiles �

Integer

Lost missiles �

Integer

Fired missiles � sum of three above

Integer

Available missiles � initial inventory minus number fired

Integer

Missions

Mission (object)

Mission ID number

Launch vessel ID number

Target ID number

Mission type

Default (D-mission)

Retarget (E-mission)

Target location

Target priority

B - 9

Warhead/fuse required

Missile ID number

Time of impact

Battle Group

Current time

Clock time

Initial inventory soft � # of soft warhead missiles with which a group began a campaign

Integer

Inflight soft �

Integer

Impacted soft �

Integer

Lost soft �

Integer

Fired soft � sum of three above

Integer

Available soft � initial inventory minus number fired

Integer

Initial inventory hard � # of soft warhead missiles with which a group began a campaign

Integer

Inflight hard �

Integer

Impacted hard �

B - 10

Integer

Lost hard �

Integer

Fired hard � sum of three above

Integer

Available hard � initial inventory minus number fired

Integer

Initial inventory missiles � total # of missiles with which a group began a campaign

Integer

Inflight missiles �

Integer

Impacted missiles �

Integer

Lost missiles �

Integer

Fired missiles � sum of three above

Integer

Available missiles � initial inventory minus number fired

Integer

C - 1

Appendix C – Scenarios

C - 2

C - 3

C - 4

E - 1

Appendix E - Emergent Target Decision Support Tool Algorithm

Minimize TTG (time to get there)

• Calculate the time to get there (TTG) for each candidate (use number of minutes, not a clock time). (TTG1, TTG2) (based on straightline flight, which we already have from the pink TOIs).

• Read the expected movetime (EMT) from the message. Calculate the �time until move� (TUM, in number of minutes) between current time (receipt time of the message) and expected movetime indicated in the message.

• For each candidate, normalize the TTG values by dividing the TTG by the TUM. These normalized values will be between zero and 1, with lower being better.

Minimize TOC (target opportunity cost)

• Calculate opp cost for each candidate = (default priority) + Sum(flex priorities).

o Ex. A missile with only a default target whose priority is 6 has a opp cost of 6. A missile with a default (priority 4), flex targets (priority 2 and 3) has an opp cost of 4 + (2 + 3) = 9.

• Normalize each opp cost by dividing the opp cost from step b by the sum of the opportunity costs for all the candidates.

o Ex. Using candidate missiles from b. above, the normalized value for a single d-target of priority 6 is: opp cost (norm) = 6 / (6+9) = 0.4. The norm value for the second candidate is opp cost (norm) = 9 / (6+9) = 0.6.

Minimize LOC (loiter opportunity cost)

• For each candidate, calculate the fuel remaining in minutes (from the timebar).

• Normalize the value by dividing by the lifetime of a missile in minutes (we think this will be about 1 hour 20 minutes, or 80 minutes).

o Ex. A missile with 20 minutes remaining normalizes to 20/80 = 0.25.

Figure out weights for each criterion (TTG, TOC, LOC)

• With a weighting scheme of 3-2-1 (the default), we need to normalize the numbers to sum to �1� so they will work with the triangle sensitivity tool.

• Divide each weight by the sum of the weights.

o Ex. The default scheme of 3-2-1 normalizes to 0.5 � 0.33 � 0.17.

E - 2

• For each candidate, multiply the normalized weight times the appropriate normalized criterion, then sum to get a normalized sum which you can compare between the missiles for a recommendation.

F - 1

Appendix F - Storyboard

F - 2