Reviews of Human Factors and Ergonomics - … Reviews of Human Factors and Ergonomics, Volume 3 Figure 1.1. A cognitive analysis requires consideration of two perspectives: examination

httprevsagepubcomErgonomics

Reviews of Human Factors and

httprevsagepubcomcontent311The online version of this article can be found at

DOI 101518155723408X299825

2007 3 1Reviews of Human Factors and ErgonomicsAnn Bisantz and Emilie Roth

Analysis of Cognitive Work

Published by

httpwwwsagepublicationscom

On behalf of

Human Factors and Ergonomics Society

found at can beReviews of Human Factors and ErgonomicsAdditional services and information for

httprevsagepubcomcgialertsEmail Alerts

httprevsagepubcomsubscriptionsSubscriptions

httpwwwsagepubcomjournalsReprintsnavReprints

httpwwwsagepubcomjournalsPermissionsnavPermissions

httprevsagepubcomcontent311refshtmlCitations

at University at Buffalo Libraries on February 18 2014revsagepubcomDownloaded from at University at Buffalo Libraries on February 18 2014revsagepubcomDownloaded from

What is This

- Nov 1 2007Version of Record gtgt


CHAPTER 1


By Ann Bisantz amp Emilie Roth

Cognitive task and work analyses are approaches to the analysis and support of cognitivework (rather than primarily physical or perceptual activities) Although a variety of meth-ods exist for performing cognitive task and work analyses they share a common goal of pro-viding information about two mutually reinforcing perspectives One perspective focuses onthe fundamental characteristics of the work domain and the cognitive demands they imposeThe other focuses on how current practitioners respond to the demands of the domain Thisincludes a description of the knowledge and skills practitioners have developed to operateeffectively as well as any limitations in knowledge and strategies that contribute to perfor-mance problems This chapter provides a broad survey of cognitive task analysis and cog-nitive work analysis methods Some of the methods highlight techniques for knowledgegathering whereas others focus on aspects of analysis and representation Still other tech-niques emphasize process outputs such as formal (computational) models of cognitive activ-ities or design artifacts and associated rationales In this chapter we review specific cognitivetask and work analysis methods and describe through illustration how these methods canbe adapted to meet specific project objectives and pragmatic constraints

Cognitive task and work analyses are approaches to the analysis and support of cogni-tive work (rather than primarily physical or perceptual activities) Although types of cog-nitive task and work analysis span a variety of perspectives and methodologies they sharethe goal of providing information about two mutually reinforcing perspectives which willbe emphasized throughout this chapter One perspective focuses on the fundamental char-acteristics of the work domain and the cognitive demands they impose The other focuseson how current practitioners respond to the demands of the domain This includes a char-acterization of the knowledge and strategies that domain practitioners have developed thatallow them to function at an expert level as well as limitations in knowledge and strate-gies that contribute to performance problems

In this chapter we summarize and highlight aspects of numerous cognitive engineer-ing methods that share the goals of cognitive task and work analyses The chapter pro-vides a broad survey of alternative often complementary methods and using illustrativecases demonstrates how these methods can be adapted and combined to meet the goalsand pragmatic constraints of real-world projects

Historical Context and the Changing Nature of Work

In the last quarter of the 20th century high-profile system failures (eg Three Mile Islandnumerous aviation accidents and military incidents such as the July 1988 accidental

1

Copyright 2008 by Human Factors and Ergonomics Society Inc All rights reserved DOI 101518155723408X299825 at University at Buffalo Libraries on February 18 2014revsagepubcomDownloaded from

shooting down of Iran Air Flight 655 by the USS Vincennes) provided evidence regardingthe need for specific attention to the cognitive activities associated with complex systemcontrol as well as the impetus for research and methodological developments in theseareas Since that time numerous researchers and practitioners have put forth methodolo-gies intended to explicitly identify the requirements of cognitive work so as to be able toanticipate contributors to performance problems (eg sources of high workload contrib-utors to error) and specify ways to improve individual and team performance be it throughnew forms of training user interfaces or decision aids

These methodologies stem from and extend a century of research and applied method-ologies that have focused on the improvement of human work through systematic analy-sis This tradition can be traced back to early studies in areas of scientific managementthat put forward the notion that work could be decomposed into fundamental repeat-able components (Taylor 1911) Additional advances in work measurement identifiedfundamental motions in work (eg grasp reach) as well as unnecessary or inefficientmotions and developed innovative methodologies for work analysis (eg using motionpictures Gilbreth amp Gilbreth 1919)

The focus of these early methods on observable physical work elements was well suitedto the extensively manual work of the day Refinements and applications of time-and-motion study such as the development of predetermined time systems (Sellie 1992) con-tinued through much of the 20th century providing a framework for task analysis methodsthat allowed the physical perceptual and cognitive demands of task components to becompared against human capabilities

Methods for examining cognitive work emerged as an adaptation and extension ofthese techniques in response to fundamental shifts in work that were driven by advancesin automation and computerization from primarily manual observable activities (or rou-tinized interactions with technology) to complex (and more hidden) cognitive activitiessuch as monitoring planning problem solving and deciding (Schraagen Chipman ampShalin 2000)

Analysis and Support of Cognitive Work

Analyses of cognitive work have variously been referred to as cognitive task analyses (CTAs)or cognitive work analyses depending on their focus and scope Although we are sensitiveto these distinctions we have chosen here to focus on an eclectic and purposefully broadset of methods that share the goal of analysis and support of cognitively complex workTherefore our use of the terms task analysis and work analysis should be interpretedthroughout this chapter in a general and somewhat interchangeable sense

CTAs typically produce descriptions of domain characteristics that shape and constraincognitive and collaborative performance as well as descriptions of the knowledge andstrategies that underlie the performance of individuals operating in that domain BecauseCTAs are generally conducted with an applied purpose in mind they also typically includedesign recommendations regarding systems facets such as information displays strategiesfor adaptive and dynamic deployment of automation andor recommendations for train-ing Cognitive analyses have also been used to guide other aspects of complex systemanalysis and design (eg personnel selection manning and function allocation decisions)

2 Reviews of Human Factors and Ergonomics Volume 3

at University at Buffalo Libraries on February 18 2014revsagepubcomDownloaded from

or as input to workload analysis and human reliability modelingPerforming a cognitive analysis of complex human-system interaction necessarily en-

compasses knowledge-gathering activities to learn about the system and complexities inquestion and the practitionersrsquo knowledge and skill that allow them to cope with systemcomplexity It also requires analysis activities to synthesize and draw conclusions consis-tent with project goals The output of the analysis can take various representational formssuch as text descriptions summary tables diagrams and computational models

Although all CTA methods necessarily involve knowledge gathering analysis and rep-resentation of results some CTA methods highlight techniques for knowledge gatheringwhereas others focus on aspects of analysis and representation Still other techniques em-phasize process outputs such as formal (computational) models of cognitive activities ordesign artifacts and associated rationales

Chapter Organization

In this chapter we provide an overview of the kinds of information that CTA methods areintended to extract and represent and a survey of specific methods available for knowl-edge acquisition and representation The next section introduces two mutually informingperspectives that are important to keep in mind when performing a CTA the need to ana-lyze domain characteristics that serve to shape and constrain cognitive performance andthe need to analyze the knowledge skills and strategies of domain practitioners We reviewCTA methods and applications that are representative of each of these two perspectivesUltimately both types of information are required to gain a full understanding of thefactors that influence practitioner performance and to identify opportunities for moreeffective support

Next we survey knowledge acquisition analysis and representation methods used inperforming CTAs We provide both an overview of knowledge acquisition techniques anda description of ways of representing and communicating the output of CTA analyses

We then review methods that are closely related to and sometimes integrated withCTA This includes task-analytic approaches as well as computational models of cogni-tive task performance

We next return to the theme that CTA is fundamentally about uncovering the demandsof the domain and the knowledge and strategies that practitioners have developed in re-sponse We show through illustration that specific CTA methods can be ldquomixed andmatchedrdquo and modified to meet the objectives and pragmatic constraints of particularprojects

We end with a discussion of ongoing and future research directions regarding CTAmethodologies including macroergonomic approaches software support and the inte-gration of CTA methods within the larger systems design process

MUTUALLY REINFORCING CTA PERSPECTIVES

Two mutually reinforcing perspectives are needed to fully understand the factors thatcontribute to cognitive performance and opportunities for improving performance (see

Analysis of Cognitive Work 3


Figure 11) One perspective involves analysis of the characteristics of a domain thatimpose cognitive demands This includes examination of the physical environment socio-organizational context technical system (or systems) and task situations that domainpractitioners confront The second perspective examines the goals motivations knowl-edge skills and strategies that are used by domain practitioners when confronting tasksituations

Analysis of domain characteristics provides the framework for understanding the goalsand constraints in the domain the task situations and complexities that domain practi-tioners are likely to encounter the cognitive demands that arise and the opportunitiesthat might be available to facilitate cognitive and collaborative performance For instanceanalysis can identify interacting goals in the domain that can complicate practitionerdecision making what information is available to practitioners and whether key neededinformation is missing or unreliable and more generally inherent performance lim-itations that are attributable to characteristics of the task or current technologiesDocumenting domain characteristics also defines the requirements for effective perfor-


Figure 11 A cognitive analysis requires consideration of two perspectives examination ofdomain characteristics and constraints that impose cognitive demands on domain practi-tioners which include components of the task technical system social and organizationalstructure and physical environment and examination of the goals knowledge skills andstrategies that domain practitioners utilize in response at University at Buffalo Libraries on February 18 2014revsagepubcomDownloaded from

mance and support including the information that needs to be sensed to allow operatorcontrol constraints and interactions that should be displayed and contexts in which auto-mation or other aids could be effectively deployed

The second complementary perspective examines the goals motivations knowledgeskills and strategies of domain practitioners This perspective provides insight into theknowledge skills and strategies that enable domain practitioners to operate at an expertlevel as well as the cognitive factors that limit the performance of less experienced indi-viduals (eg incomplete or inaccurate mental models) The results can be used to iden-tify opportunities to improve performance either through training (eg to bring lessexperienced personnel to the level of experts) or through the introduction of systems thatmore effectively support cognitive performance (eg eliminating the need for the expertstrategies that compensate for poor designs)

CTA researchers and practitioners have typically emphasized one perspective or theother some tend to emphasize the need to uncover the knowledge and skills underlyingperformance (eg Klein 1998) and others emphasize the need to analyze characteristicsof the domain that serve to shape cognitive and collaborative performance (Rasmussen1986 Rasmussen Pejtersen amp Goodstein 1994 Sanderson 2003 Vicente 1999) In thefollowing two sections we provide an overview of work that is representative of each ofthese perspectives It needs to be stressed that the two perspectives are clearly mutuallyinforming Importantly the demands of the tasks interact with practitioner expertise em-bedded work practices and environmental supports to make aspects of system controlmore or less challenging To effectively support system design and performance-aidingefforts CTAs must reveal these complex interdependencies Ultimately therefore bothperspectives need to be taken into account for a full picture of the factors that influencepractitioner performance and the opportunities available to more effectively support per-formance (Hoffman amp Lintern 2006 Potter Roth Woods amp Elm 2000)

Understanding Domain Characteristics and Constraints

In order to aid complex cognitive work one must understand the performance-shapingfactors of the domain within which that work is performed Human activity can be under-stood not only in terms of tasks procedures or decisions but also in terms of the con-straints that restrict and the goals that provide direction to action

Vicente (1990) provided a convincing argument regarding the degree to which an in-depth understanding of the environment in which humans operate is not only helpful butnecessary to make sense of and support performance in complex unpredictable environ-ments Vicente quoted an example from Simon (1981) regarding an ant traveling acrossa beach Although the path taken by the ant is irregular the complexity is a function ofthe beachrsquos irregular surface not of the ant One can observe a similar example when fly-ing at night Whereas the city boundaries are visible from the patterns of lights the rea-sons for their complexity are revealed only when one can see the underlying geography ofmountains valleys lakes and rivers Vicente (1990) described three factors that influencethe actions that an ant (or a person) will take to reach the same goal state the state of thesystem at the time the goal-directed activity begins external unpredictable disturbancesfor which the operator must compensate and individual differences in strategy Thus a



successful task analysis methodology must provide a description of the work domain aswell as tasks and strategies

Cognitive engineering methodologies have been developed to provide such a descrip-tion Roth and Woods (1988) for instance provided a description of a competence modelnecessary for the successful operation of a nuclear power plant A competence modelcharacterizes essential complexities in the domain (eg competing goals and nonlinearsystem dynamics) that experts have to manage and the strategies that are needed for ac-complishing tasks in the face of these difficulties Woods and Hollnagel (1987) provideda more formal representation of the goals of a nuclear power plant and the functionalmeans available in the system to accomplish them

Rasmussenrsquos abstraction hierarchy (Rasmussen 1986 Rasmussen et al 1994 Vicente1999) is a commonly adopted framework for representing a complex system at multiplelevels of abstractionmdashfrom the physical form and objects in the system to processesfunctions constraints or abstract laws to the highest-level purposes for which the systemwas designed (see Figure 12 for an example) Key aspects of this representation includethe fact that levels differ in the manner in which they represent the system (goals vs ob-jects) rather than the level of detail and that the links between nodes represent means-ends relationships Lower-level nodes provide the means by which higher-level goals areaccomplished and the higher-level nodes are the reasons for the existence of lower-levelnodes Importantly therefore nodes are decomposed not into the activities or human ac-tions that are deployed to accomplish a goal or function but rather into the functionsprocesses and objects that are part of the system The abstraction hierarchy has been usedin performing work domain analyses as part of a more comprehensive cognitive engineer-ing methodology called cognitive work analysis (Vicente 1999)

Ecological interface design (EID Burns amp Hajdukiewicz 2004 Vicente 2002 Vicenteamp Rasmussen 1992) is a framework based on work domain analysis as well as other aspectsof cognitive work analysis that support the development of human-computer interfacesfor complex systems Here a work domain analysis (typically using an abstraction hier-archy representation) identifies information requirements (associated with all levels ofthe hierarchy) necessary to allow effective control under different circumstances Addi-tionally the EID approach focuses on allowing operators to act whenever possible at lesseffortful skill- and rule-based levels while still providing information necessary forknowledge-based reasoning when required Importantly identifying information require-ments associated with a systemrsquos purposes functions and physical objects compared withrequirements associated with specific tasks and activity sequences makes it possible foroperators to reason about the system in unexpected circumstances (Vicente 2002) EIDhas been applied in a number of domains such as nuclear power (Itoh Sakuma amp Monta1995) and computer network management (Burns Kuo amp Ng 2003) Sanderson and Watson(2005) applied EID principles to the design of auditory alerts in a medical environment

Lintern (2006) applied work domain analysis to describe the goals functions andphysical resources of an insurgency operation in order to aid intelligence analysts Heaugmented nodes in the abstraction hierarchy with activity descriptions derived from ascenario narrative provided by a subject matter expert This analysis was used to developa prototype computer workspace to support insurgency analysis in which information



Ca

su

alty M

an

age

me

nt

Ba

lan

ce

of A

uth

ority

Ris

k-B

enefit B

ala

nce

Re

sou

rce

Ba

lance

Eco

nom

ic B

ala

nce

Resp

on

se P

lan

sO

pe

ratio

ns P

roce

sse

s

Co

ord

ina

tin

g a

nd I

mp

lem

en

ting

Me

dic

al O

pe

ratio

ns

Co

ord

ina

tin

g a

nd I

mp

lem

en

ting

She

lte

r O

pera

tio

ns

Fix

ed L

oca

tion

Re

so

urc

es

(Ho

sp

ita

ls 9

11

Ca

ll C

ente

rs

Am

bu

lan

ce D

isp

atc

h F

ire

Sta

tion

s P

olic

e S

tatio

ns)

Mo

bile

Re

so

urc

es

(Am

bu

lan

ce

s

Fire

Re

scu

e

Fir

e E

ngin

es P

olic

eV

eh

icle

s)

Hu

ma

n R

eso

urc

es

(Po

lice

E

MT

F

ire

)

Sta

te o

f b

uild

ing

s o

pe

ratin

gcon

ditio

ns lo

ca

tio

n

ca

pa

city ca

pacity v

s lo

ad

(eg

fo

r a

ho

spita

l)

Nu

mb

er

co

nd

itio

n

op

era

tion

al sta

te

locatio

nof m

obile

reso

urc

es

pers

on

ne

l

Fu

nctio

na

l P

urp

ose

Ge

ne

ral P

roce

sse

s

Ab

str

act F

un

ction

Ph

ysic

al F

un

ctio

n

Ph

ysic

al F

orm

Sys

tem

Sub

-Sys

tem

Fig

ure

12

Por

tion

of a

n ab

stra

ctio

n hi

erar

chy

wor

k do

mai

n m

odel

for

an

emer

genc

y re

spon

sesy

stem

B

ased

on

wor

k fr

om B

isan

tz

Littl

e a

nd R

ogov

a (2

004)

7


from levels and nodes from the work domain model were instantiated in information pan-els on a large-format display

Work domain analysis has application beyond interface design Naikar (2006) describedthe use of work domain analysis for identifying training needs and training system re-quirements for a fighter aircraft for comparing and evaluating competing design propos-als for a new military system for designing team structures and for developing trainingstrategies that manage human error For instance values and priority functions (abstractfunctions) identified for a fighter aircraft such as minimizing collateral damage sug-gested the need to both explicitly train on and measure these variables For design eval-uations the work domain analysis framework enabled the technical assessment of physicalobjects (a typical step in evaluation) to be additionally evaluated against higher-level pro-cesses functions and goals Therefore system components were evaluated not just interms of the degree to which they met technical performance criteria but also in terms oftheir significance to the overall sociotechnical system

Bisantz et al (2003) incorporated work domain analysis as part of the initial designphase for a new naval vessel Among other things the analysis revealed that the sameplanned weapon system was to be used to support multiple potentially conflicting goalsAn implication of the analysis was that either the physical system needed to be redesignedto eliminate the potential goal conflict or that procedures would need to be put in placereflecting how the use of that resource would be prioritized in case of goal conflict sit-uations Similarly the analysis revealed how the operational processes of moving the shipand emitting signals from sensor systems were both means associated with the functionof sensing but that their use at a particular point in time could differentially affect thedefensive and offensive purposes of the ship This revealed a need for mutual awarenessand close communication among operators involved in the two functions

Applied cognitive work analysis (Elm Potter Gualtieri Easter amp Roth 2003) is a com-prehensive design methodology that also integrates an explicit representation of the workdomain Here the domain analysis results in a functional abstraction network (FAN) whichrepresents goals along with associated processes and system components This network islinked to and provides the basis for additional stages of analysis including informationrequirements and representation design Potter Gualtieri and Elm (2003) described anapplication of this methodology to military command and control in which the FAN wasused to represent abstract concepts such as ldquocombat powerrdquo as well as high-level goals ofcomplying with military law and sociopolitical constraints Subsequent stages of analysissupported the development of innovative displays that visually represented levels of com-bat power to support commander decision making

Uncovering Practitioner Knowledge and Strategies

The complementary goal of CTA is to understand and represent the knowledge of domainpractitioners and the problem-solving and decision-making strategies that they use to per-form tasks This tradition has its roots in cognitive psychology and cognitive science inwhich there was an attempt to understand the nature of expertise (Chase amp Simon 1973Chi Glaser amp Farr 1988 Hoffman 1987)

One of the classic strategies for uncovering the basis of expert performance is to



compare the performance of experts with that of less experienced individuals Early stud-ies examined the performance of experts under controlled laboratory conditions to under-stand what made experts different from novices For example Chi Feltovich and Glaser(1981) conducted laboratory studies comparing the performance of individuals with dif-ferent levels of expertise in physics on simple tasks such as sorting and classifying differ-ent physics problems They were able to show differences between experts and novices intheir organizational structure of knowledge

Comparing the performance of individuals of different levels of expertise continues tobe a powerful technique for uncovering the basis of expertise For example Dominguez(2001) used this approach to reveal differences between staff surgeons and residents in theirawareness of boundary conditions for safe operation in laparoscopic surgery

Collecting think-aloud protocols is another classic strategy for understanding the natureof expertise that has its roots in cognitive science (Ericsson amp Simon 1993) Individualsare asked to ldquothink aloudrdquo as they attempt to solve problems Protocol analyses of theirutterances and actions can be used to map the detailed knowledge and reasoning that indi-viduals use in solving the problems The results can be used to inform the design of train-ing or support systems (Hall Gott amp Pokorny 1995 Means amp Gott 1988)

CTA methods have also been used to understand and model the process of decisionmaking under real-world conditions which has come to be referred to as naturalisticdecision making (Klein 1998) Klein and his colleagues developed a variety of structuredinterview techniques to uncover how experts make decisions in high-risk uncertain andtime-pressured domains such as firefighting and clinical nursing (Klein 1998) The researchhas led to recognition-primed models of expert decision making These models stress theimportance of situation recognition processes that rely on subtle cues and mental simu-lation processes that enable experts to make effective decisions in dynamic time-pressuredsituations

Cognitive work can be examined at different ldquograinsrdquo of analysis For some purposesit is appropriate to model the elemental mental processes that underlie performance (egvisual scanning retrieval of information from long-term memory short-term memorystorage of information specific mental computations attention shift) Gray and Boehm-Davis (2000) demonstrated that analyses of mental processes at the millisecond level couldinform the design of improved user interfaces Techniques suited for microlevel analysesinclude think-aloud protocols keystroke capture and eye movement data because theycapture detailed mental and physical activity Examples of studies that have used this ap-proach include an investigation by Seagull and Xiao (2001) who used eye-tracking videodata to examine the detailed visual sampling strategies of medical staff performing trache-al intubations and a study by Luke Brook-Carter Parkes Grimes and Mills (2006) whoexamined visual strategies of train drivers

Generally cognitive work has been analyzed at a more ldquomacrograinrdquo level of analysissometimes referred to as macrocognition in which the focus is on describing information-gathering decision-making and collaborative strategies rather than the elemental cog-nitive processes (Klein Ross Moon Klein amp Hollnagel 2003) Examples include a studythat examined the strategies by which railroad dispatchers managed the multiple demandsplaced on track usage to maintain efficiency and safety (Roth Malsch Multer amp Coplen1999) a study that examined the strategies used by experienced hackers to attack a



computer network (Stanard et al 2004) and research that examined how emergency am-bulance dispatchers keep track of ambulances and make ambulance allocation decisions(Chow amp Vicente 2002)

CTA methods have also been used to reveal sources of vulnerability and contributorsto error For example Patterson Roth and Woods (2001) examined the information searchstrategies of intelligence analysts under simulated data overload conditions Figure 13shows the typical analysis process that the intelligence analysts used to search for and in-tegrate information Patterson et al (2001) were able to identify a number of suboptimalstrategies such as premature closure that created the potential for incomplete or inaccu-rate analysis Figure 14 illustrates how participant search strategies often caused analyststo fail to locate and exploit ldquohigh-profitrdquo documents that contained more complete andaccurate information

Analysis of domain practitioner strategies can provide the basis for defining new sys-tem design requirements (eg Bisantz et al 2003) For example CTA methods can uncoverwork-around strategies that experienced practitioners have developed to compensate forsystem limitations (eg Mumaw Roth Vicente amp Burns 2000 Roth amp Woods 1988) Theexamination of these strategies can provide the basis for establishing cognitive supportrequirements to guide new system design Similarly CTA methods can provide insightinto critical features in the current environment that are exploited by experienced domainpractitioners and that should be preserved or otherwise reproduced as new technology isintroduced (eg Roth Multer amp Raslear 2006 Roth amp Patterson 2005)

Although the discussion thus far has focused on empirical analyses aimed at provid-ing descriptive models of the knowledge and strategies of domain practitioners in the cur-rent environment cognitive analyses can also be performed to develop formative modelsthat specify the cognitive requirements for effective task performance without referenceto the actual performance of domain practitioners (Vicente 1999) This approach is rele-vant when trying to analyze the cognitive demands that are likely to be imposed by asystem design that does not yet exist or to compare the impact in terms of cognitive per-formance requirements of alternative envisioned designs

Decision ladders provide one formalism for representing the knowledge and information-processing activities necessary to make a decision or achieve a goal (Rasmussen 1983)Nehme Scott Cummings and Furusho (2006) used this approach to develop informationand display requirements for futuristic unmanned systems for which no current imple-mentations exist They used a decision ladder formalism to map out the monitoringplanning and decision-making activities that would be required of operators of thesesystems Callouts were then used to specify information and display requirements in orderto support the corresponding cognitive tasks

Empirical techniques have also been used to explore how changes in technology andtraining are likely to affect practitioner skills strategies and performance vulnerabilities(Woods amp Dekker 2000) Techniques include using concrete scenarios or simulationsof the cognitive demands that are likely to be confronted Woods and Hollnagel (2006)referred to these methods as staged-world techniques One example is a study that useda high-fidelity training simulator to explore how new computerized procedures andadvanced alarms were likely to affect the strategies used by nuclear power plant crews tocoordinate activities and maintain shared situation awareness (Roth amp Patterson 2005)


(text continues on page 13) at University at Buffalo Libraries on February 18 2014revsagepubcomDownloaded from

11

Qu

ery

(k

ey

wo

rds

refi

nem

en

t)

Sele

ct ldquo

key

rdquo d

ocu

men

ts

Co

rro

bo

rate

in

form

ati

on

(o

r

reso

lve d

iscr

ep

an

cies

in

info

rmati

on

) an

d f

ill

in g

ap

s

wit

h s

up

po

rt d

ocu

men

ts

Sy

nth

esi

ze i

nfo

rmati

on

to

con

stru

ct a

co

here

nt

sto

ry

Bro

wse

by

tit

le a

nd

o

r d

ate

19

960

80

8

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

19

960

50

0

AR

IAN

E 5

A

N A

LL

NE

W L

AU

NC

HE

R F

OR

TH

E G

LO

BA

L M

AR

KE

T

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

19

960

61

0

AR

IAN

E 5

EX

PL

OD

ES

IN

TH

E S

KY

OV

ER

KO

UR

OU

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

eu

rop

e 1

996

(eu

rop

e 1

996)

amp (

lau

nch

fail

ure

)

(eu

rop

e 1

996)

amp (

(la

un

ch f

ail

ure

)

2)

378

140

22

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

Da

te

Au

gu

st 8

1996

So

urc

e

FB

IS r

ep

ort

AC

CO

RD

ING

T

O T

HE

R

EP

OR

T R

EC

EN

TL

Y S

UB

MIT

TE

D B

Y T

HE

IND

EP

EN

DE

NT

IN

QU

IRY

B

OA

RD

S

ET

U

P

AF

TE

R T

HE

AR

IAN

E 5

ltlt

LA

UN

CH

gtgt

ltlt

FA

ILU

RE

gtgt

ON

JU

NE

4 (

RE

FT

EL

A

) T

HE

EX

PLO

SIO

N W

AS

DU

E T

O S

PE

CIF

ICA

TIO

N A

ND

DE

SIG

NE

RR

OR

S I

N

TH

E S

OF

TW

AR

E

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M T

HE

RE

PO

RT

FU

RT

HE

R S

TR

ES

SE

D T

HE

F

AC

T T

HA

T

TH

E

ALIG

NM

EN

T F

UN

CT

ION

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M W

HIC

H

SE

RV

ED

A

P

UR

PO

SE

O

NL

YB

EF

OR

E L

IFT

OF

F (B

UT

R

EM

AIN

ED

O

PE

RA

TIV

E

AF

TE

RW

AR

DS

) W

AS

N

OT

TA

KE

N I

NT

O A

CC

OU

NT

IN

T

HE

S

IMU

LA

TIO

NS

AN

D T

HA

T T

HE

EQ

UIP

ME

NT

AN

DS

YS

TE

M T

ES

TS

WE

RE

N

OT

S

UF

FIC

IEN

TL

Y R

EP

RE

SE

NT

AT

IVE

T

HE

SY

ST

EM

AR

CH

ITE

CT

UR

E W

AS

NO

T IM

PLIC

AT

ED

T

HE

R

EP

OR

T

RE

CO

MM

EN

DE

D E

XT

EN

SIV

ER

EV

IEW

OF

TH

E S

OF

TW

AR

E D

ES

IGN

AN

D T

ES

TIN

G P

RO

CE

DU

RE

S

TH

E C

OS

T O

F C

OR

RE

CT

IVE

M

EA

SU

RE

S I

S E

ST

IMA

TE

D A

T 2

TO

4P

ER

CE

NT

OF

TH

E G

LO

BA

L

CO

ST

O

F T

HE

P

RO

JE

CT

T

HA

T IS

U

SD

150 T

O 3

00

MIL

LIO

N

CO

RR

EC

TIV

E A

CT

ION

IS

E

XP

EC

TE

D T

O P

OS

TP

ON

E T

HE

NE

XT

LA

UN

CH

TO

MID

SE

ME

ST

ER

1997

E

SA

DO

ES

NO

T E

XC

LU

DE

TH

E P

OS

SIB

ILIT

Y T

HA

TT

HE

TH

IRD

AR

IAN

E 5

LA

UN

CH

IN

ITIA

LLY

P

LA

NN

ED

A

S A

C

OM

ME

RC

IAL L

AU

NC

H

MA

Y E

VE

NT

UA

LLY

B

E T

RE

AT

ED

A

S A

Q

UA

LIF

ICA

TIO

N LA

UN

CH

ltlt

Eu

rop

egt

gts

he

avy-

lift la

un

che

r pro

gra

mm

e w

ill c

on

tin

ue

de

spite

th

eJu

ne

4se

tba

ck

bu

t th

e m

arg

in fo

r err

or

ha

s n

ow

be

en

re

du

ce

d t

o z

ero

Th

e lo

ss

of th

e fir

st A

ria

ne

5 o

n it

s in

itia

l q

ualif

ica

tio

n flig

ht

on

Jun

e 4

ma

rks a

se

ve

re b

ut

by

no

me

an

s fa

tal s

etb

ack

fo

r lt

ltE

uro

pe

gtgt

sn

ew

he

avy-

lift

laun

che

r a

nd

for

Ari

an

esp

ace

s a

mb

itio

ns t

o c

on

solid

ate

its

do

min

an

t p

osi

tio

n o

n t

he

glo

bal co

mm

erc

ial la

un

ch m

ark

et

It

will

take

mo

re t

ha

n a

sin

gle

ltlt

lau

nch

gtgt

ltlt

failu

regt

gt t

o d

era

il th

e 1

5-y

ea

r $

82

bill

ion

pro

gra

mm

e

bu

t th

e m

arg

in f

or

err

or

ha

s h

en

ce

fort

h b

een

red

uce

d to

zero

Th

e 4 J

un

e 1

996

la

un

ch o

f th

e fi

rst

Eu

rop

ean

Sp

ace

Ag

en

cy (

ES

A)

Ari

an

e 5

sp

ace

lau

nch

veh

icle

carr

yin

g t

he f

ou

r E

uro

pean

ldquoC

lust

errdquo

sci

en

tifi

c sa

tell

ites

en

ded

in

fail

ure

ju

st

ov

er

36 s

eco

nd

s aft

er l

ift-

off

du

e t

o i

nh

eri

t so

ftw

are

des

ign

fla

w i

n t

he g

uid

an

ce s

yst

em

Th

e im

bed

ded

so

ftw

are

in

bo

th o

f th

e p

rim

ary

an

d b

ack

-up

id

enti

call

y-d

esi

gn

ed

inert

ial

gu

idan

ce p

latf

orm

s c

urr

en

tly

pro

ven

an

d u

sed

on

bo

ard

th

e A

rian

e 4

w

as

ill-

desi

gn

ed

fo

r th

e f

lig

ht

pro

file

of

the

Ari

an

e 5

I

nsu

ffic

ien

t re

qu

irem

en

ts a

nd

test

ing

of

the

Ari

an

e 5 a

vio

nic

s sy

stem

du

rin

g d

ev

elo

pm

en

t d

id n

ot

un

cov

er t

he d

esi

gn

fla

w

Th

e acc

elera

tio

n o

f th

e A

rian

e 5

is

mu

ch g

reate

r th

an

th

e A

rian

e 4

aft

er l

ift-

off

T

his

cau

sed

th

e i

ner

tial

gu

idan

ce p

latf

orm

em

bed

ded

so

ftw

are

w

hic

h w

as

des

ign

ed

aro

un

d

the

slo

wer

Ari

an

e 4

acc

ele

rati

on

pro

file

to

pro

vid

e n

um

eri

cal

va

lues

bey

on

d t

he

pro

gra

mm

ed

lim

its

of

the

flig

ht

com

pu

ter

wh

ich

th

en

sh

utd

ow

n b

oth

in

erti

al

pla

tfo

rms

Th

e p

latf

orm

s in

itia

ted

a d

iag

no

stic

ldquore

setrdquo

mo

de

tha

t fe

d i

nco

rrec

t v

alu

es t

o t

he

flig

ht

com

pu

ter

wh

ich

th

en c

om

man

ded

an

ex

cess

ive

pit

ch-o

ver

by

gim

ball

ing

th

e tw

o s

trap

-

on

bo

ost

er

no

zzle

s an

d t

hen

fir

st-s

tag

e m

ain

en

gin

e n

ozz

le T

his

rap

id p

itch

in

to a

hig

h

an

gle

of

att

ack

cau

sed

th

e f

air

ing

to

sep

ara

te f

rom

th

e v

eh

icle

du

e to

th

e h

igh

aero

dy

nam

ic l

oad

ing

T

his

cau

sed

th

e o

n-b

oard

self

-des

tru

ct s

yst

em

to

act

iva

te w

hic

h

was

late

r fo

llo

wed

by

a r

an

ge s

afe

ty-i

ssu

ed d

est

ruct

co

mm

an

d

Th

e f

lig

ht

com

pu

ter

pro

gra

m s

hu

tdo

wn

th

e in

ert

ial

pla

tfo

rms

as

pa

rt o

f a g

uid

an

ce r

e-a

lig

nm

ent

rou

tin

e

Th

is r

ou

tin

e i

s p

art

of

the

Ari

an

e 4

reali

gn

men

t ro

uti

ne d

esi

gn

ed

to

all

ow

qu

ick

up

da

tes

aft

er a

ho

ld l

ate

in

th

e c

ou

nt-

do

wn

T

his

ro

uti

ne i

s a

llo

wed

to

op

era

te u

p t

o 4

0 s

eco

nd

s

aft

er l

ift-

off

D

uri

ng

th

is p

eri

od

th

e A

ria

ne

5 t

rav

els

fart

her

do

wn

ra

ng

e th

an

th

e

Ari

an

e 4

an

d t

her

efo

re p

rod

uce

s a

la

rger

ho

rizo

nta

l v

elo

city

co

mp

on

en

t v

alu

e W

hen

this

valu

e r

each

ed

th

e fl

igh

t co

mp

ute

r p

rog

ram

lim

it (

ie n

um

eri

cal

ov

erfl

ow

) w

hic

h i

s

no

t re

ach

ed

by

th

e A

rian

e 4

in

th

e sa

me t

imefr

am

e t

he f

lig

ht

com

pu

ter

shu

tdo

wn

re

set

the

inert

ial

pla

tfo

rms

Du

rin

g d

evelo

pm

en

t o

f th

e av

ion

ics

for

the

Ari

an

e 5

th

e S

exta

nt

Av

ion

iqu

e r

ing

la

ser

gy

ro i

ner

tail

pla

tfo

rms

were

no

t te

sted

in

th

e l

oo

p a

s p

art

of

an

en

d-t

o-e

nd

sim

ula

tio

n

sin

ce t

hey

had

pre

vio

usl

y b

een

pro

ven

to

wo

rk o

n t

he A

rian

e 4

I

nst

ead

th

eir

in

pu

t

was

on

ly s

imu

late

d A

co

mp

lete

hard

ware

-in

-th

e-l

oo

p t

est

wo

uld

lik

ely

hav

e

un

cov

ere

d t

he

syst

em

ic f

ail

ure

of

thes

e A

rian

e-4

deri

ved

in

ert

ial

pla

tfo

rms

in t

he

Ari

an

e 5

fli

gh

t en

vir

on

men

t

As

a r

esu

lt o

f th

is f

ail

ure

E

SA

lo

st t

he

US

$50

0 m

illi

on

Clu

ster

sate

llit

e p

rog

ram

an

d

est

ima

tes

a 2

-4

in

crease

in

co

st o

f th

e U

S $

8 b

illi

on

Ari

an

e 5

pro

gra

m alo

ng

wit

h a

n

ap

pro

xim

ate

on

e-y

ear

dela

y i

n t

he A

rian

e 5

pro

gra

m i

n o

rder

to

per

form

mo

re

hard

wa

re s

imu

lati

on

test

s an

d d

esi

gn

a p

rov

e a

new

gu

idan

ce p

rog

ram

Fig

ure

13

Typ

ical

ana

lysi

s pr

oces

s us

ed b

y in

telli

genc

e an

alys

ts t

o se

arch

a d

ocum

ent

data

base

and

synt

hesi

ze r

esul

ts to

form

ulat

e a

resp

onse

to a

n an

alys

is q

uery

Rep

rinte

d fr

om P

atte

rson

E S

R

oth

E M

amp

Woo

ds D

D P

redi

ctin

g vu

lner

abili

ty in

com

pute

r-su

ppor

ted

infe

rent

ial a

naly

sis

unde

rda

ta o

verlo

ad

Cog

nitio

n T

echn

olog

y amp

Wor

k 3

22

4ndash23

7 C

opyr

ight

200

1 w

ith k

ind

perm

issi

on o

fS

prin

ger

Sci

ence

and

Bus

ines

s M

edia


12

Key documents Key documents that are high profitHigh profit documents

copy1999 Patterson

S2 73 minutesesa amp ariane(esa amp ariane) amp failure

161

29

S3 24 minuteseurope 1996(europe 1996) amp (launch failure)(europe 1996) amp ((launch failure)2)

22

5

S4 68 minutes(european space agency)3 amp ariane amp failure amp (launcher | rocket))

169

15

S5 96 minutesESA | (european amp space amp agency)(ESA | (european amp space amp agency)) gt (19960601) Infodate

419

28

S6 32 minutes1996 amp Ariane(1996 amp Ariane) amp (destr | explo)(1996 amp Ariane) amp (destr | explo) amp (fail)

7

184

S7 73 minutessoftware amp guidance

66

14

S8 27 minutesesa amp arianeariane amp 5(ariane amp 5)2((ariane amp 5)2) amp (launch amp failure)

12

194

S9 44 minutes1996 amp European Space Agency amp satellite1996 amp European Space Agency amp lost1996 amp European Space Agency amp lost amp rocket

4

29

Figure 14 Information-sampling process employed by intelligence analysts in the Pattersonet al (2001) study Largest circle represents articles in the database Internal circles repre-sent articles returned from database search queries The thick-circumference circle representsarticles that were read The filled-in circles represent which of the documents were ldquohigh prof-itrdquo in the sense of containing extensive accurate information which were ldquokeyrdquo in the sense ofbeing relied upon heavily by participant and which of the key documents were also highprofit Reprinted from Patterson E S Roth E M amp Woods D D Predicting vulnerability incomputer-supported inferential analysis under data overload Cognition Technology amp Work3 224ndash237 Copyright 2001 with kind permission of Springer Science and Business Media at University at Buffalo Libraries on February 18 2014revsagepubcomDownloaded from

Another example is a study by Dekker and Woods (1999) that used a future incident tech-nique to explore the potential impact of contemplated future air traffic managementarchitectures on the cognitive demands placed on domain practitioners Controllers pilotsand dispatchers were presented with a series of future incidents to jointly resolve Exam-ination of their problem solving and decision making revealed dilemmas trade-offs andpoints of vulnerability associated with the contemplated architectures this enabled prac-titioners and developers to think critically about the requirements for effective perfor-mance for these envisioned systems

Cognitive analyses that capture the knowledge and strategies of domain practitionershave application beyond design For example they have been used to support developmentof training (Johnson et al 2006 OrsquoHare Wiggins Williams amp Wong 1998 SchaafstalSchraagen amp van Berlo 2000 Seamster Redding amp Kaempf 1997) as well as specifica-tion of proficiency evaluation requirements (Cameron et al 2000) More recently CTAshave been used as a means to capture domain expertise for archival purposes For exam-ple government and private sector organizations have found a need to capture expertknowledge from individuals who are about to retire so as to preserve and transmit thecorporate knowledge (Hoffman amp Hanes 2003 Klein 1992)

CTA METHODS

CTA methods provide knowledge acquisition techniques for collecting data about theknowledge and strategies that underlie performance as well as methods for analyzing andrepresenting the results Schraagen et al (2000) provided a broad survey of different CTAapproaches Crandall Klein and Hoffman (2006) produced an excellent ldquohow-tordquo hand-book with detailed practical guidance on how to perform a CTA In this section we describesome of the most widely used knowledge acquisition and representation methods high-lighting some of the key factors that distinguish among methods

Knowledge Acquisition Methods

Effective knowledge acquisition depends on an understanding of the factors that enabledomain practitioners to more easily access and describe their own knowledge and thoughtprocesses Extensive psychological research suggests that self-reports of memory and deci-sion processes can often be inaccurate (eg Banaji amp Crowder 1989 Nisbett amp Wilson1977) The key is to understand the conditions under which self-reports are likely to bereliable Reviews of relevant factors that contribute to accurate self-reports can be foundin Ericsson and Simon (1993) Leplat (1986) and Roth and Woods (1989)

Roth and Woods (1989) highlighted three dimensions that affect the quality of theinformation obtained One important factor is the specificity of the information beingelicited Domain practitioners are likely to give a more accurate and complete descriptionof their reasoning process and the factors that influence their thinking when asked todescribe a specific example than when asked a general question such as ldquoHow do yougenerally approach a problemrdquo or ldquoCan you describe your typical procedurerdquo (Hoffmanamp Lintern 2006) A second important factor is how similar the conditions under which



knowledge acquisition is conducted are to the actual ldquofieldrdquo conditions in which the do-main practitioners operate The more the acquisition context allows the domain prac-titioner to display his or her expertise rather than reflect on it the more valid the resultswill be Thus domain practitioners can more easily demonstrate how they perform a taskin the actual work context than describe the task outside the work context

The third important factor relates to the interval between when the information wasexperienced or attended to by the domain practitioner and the time he or she is askedabout it Think-aloud protocols conducted while a person is engaged in performing a taskare the most effective Retrospective reports in which a person is asked to describe tasksor events that occurred in the distant past are less likely to be reliable When retrospec-tive reports must be used they can be improved by providing effective retrieval cues Forexample Hoc and Leplat (1983) demonstrated that a cued retrospective methodology inwhich people are asked to describe how they went about solving a problem while watch-ing a videotape of their own performance improved the quality of the information theyprovided

A variety of specific techniques for knowledge acquisition have been developed thatdraw on basic principles and methods of cognitive psychology (Cooke 1994 Ericssonamp Simon 1993 Hoffman 1987) Although there are many specific knowledge acquisitionmethods fundamentally they can be classified into methods that primarily involve inter-viewing domain practitioners and those that primarily involve observing domain practi-tioners engaged in domain-relevant tasks The next two sections describe methods thatfall into each of these classes

Interview approaches Interviews are among the most common knowledge acquisitionmethods Unstructured interviews are free-form interviews of domain practitioners inwhich neither the content nor the sequence of the interview topics is predetermined(Cooke 1994) Unstructured interviews are most appropriate early in the knowledge ac-quisition process when the analyst is attempting to gain a broad overview of the domainwhile building rapport with the domain practitioners More typically CTA analysts willuse a semistructured interview approach in which a list of topics and candidate questionsis generated ahead of time but the specific topics and the order in which they are coveredis guided by the responses obtained (eg Mumaw et al 2000 Roth et al 1999)

Structured interview techniques utilize a specific set of questions in a specific order Anumber of structured and semistructured interview techniques for CTA have been devel-oped One of the most widely used structured CTA interview techniques is the criticaldecision method (CDM) developed by Klein and his colleagues (Hoffman Crandall ampShadbolt 1998 Klein amp Armstrong 2005 Klein Calderwood amp MacGregor 1989) TheCDM is a structured approach for analyzing actual challenging cases that the domainpractitioner has experienced It is a variant of the critical incident technique developedby Flanagan (1954) for analyzing critical cases that have occurred in the past Analysis ofactual past cases provides a valuable window for examining the cognitive demands inher-ent in a domain The incidents can be analyzed to understand what made them challeng-ing and why the individuals who confronted the situation succeeded or failed (Dekker2002 Flanagan 1954)

A CDM session includes four interview phases or ldquosweepsrdquo that examine a past



incident in successively greater detail The first sweep identifies a complex incident thathas the potential to uncover cognitive and collaborative demands of the domain and thebasis of domain expertise In the second sweep a detailed incident timeline is developedthat shows the sequence of events The third sweep examines key decision points moredeeply by using a set of probe questions (eg ldquoWhat were you noticing at that pointrdquoldquoWhat was it about the situation that let you know what was going to happenrdquo ldquoWhatwere your overriding concerns at that pointrdquo) Finally the fourth sweep uses ldquowhat ifrdquoqueries to explore the space of possibilities more broadly For example ldquowhat ifrdquo questionsare used to probe for potential expertnovice differences (eg whether someone else per-haps with less experience might have responded differently) The output is a descriptionof the subtle cues knowledge goals expectancies and expert strategies that domain expertsuse to handle cognitively challenging situations It has been successfully employed to ana-lyze the basis of expertise in a variety of domains such as firefighting neonatal caregiv-ing and intelligence analysis (Baxter Monk Tan Dear amp Newell 2005 Hutchins Pirolliamp Card 2003 Klein 1998)

Concept mapping is another structured interview technique that is widely used to un-cover and document the knowledge and strategies that underlie expertise (Crandall et al2006) In concept mapping knowledge elicitation the CTA analyst helps domain practi-tioners build up a representation of their domain knowledge using concept maps Conceptmaps are directed graphs made up of concept nodes connected by labeled links They areused to capture the content and structure of domain knowledge that experts employ insolving problems and making decisions

Whereas many structured interview techniques are conducted with a single domainpractitioner as interviewee concept mapping is typically conducted in group sessions thatinclude multiple domain practitioners (eg three to five) and two facilitators One facil-itator provides support in the form of suggestions and probe questions and the secondfacilitator creates the concept map based on the participantsrsquo comments for all to reviewand modify The output is a graphic representation of expert domain knowledge that canbe used as input to the design of training or decision aids See Figure 15 for an exampleof a concept map that depicts the knowledge of cold fronts in Gulf Coast weather of anexpert in meteorology (Hoffman Coffey Ford amp Novak 2006) It was created using asoftware suite called CmapTools (Institute for Human and Machine Cognition 2006)Icons below the nodes provide hyperlinks to other resources (eg other Cmaps and dig-ital images of radar and satellite pictures digital videos of experts)

Other CTA methods that rely on interviews include the applied cognitive task analy-sis method (ACTA Militello amp Hutton 1998) and the goal-directed task analysis method(Endsley Bolte amp Jones 2003) ACTA was designed specifically to guide less experiencedcognitive analysts in performing a CTA The goal-directed task analysis method providesanother example of a CTA method that is based on semistructured interviews Its focusis on deriving information requirements to support the design of displays and decisionaids intended to foster situation awareness

Observational methods A second common method of data collection to support cog-nitive task and work analyses is the observation of domain practitioners as they performdomain tasks Observational methods used in cognitive engineering research are informed



by a number of traditions including the case study and ethnographic approaches usedin social science (Blomberg Giacomi Mosher amp Swenton-Wall 1993 Hammersley ampAtkinson 1983 Lincoln amp Guba 1985 Yin 1989) as well as industrial engineering tech-niques of work analysis (Salvendy 2001)

Bisantz and Drury (2005) noted that the use of observation methods can vary along anumber of key dimensions many of which are relevant to the use of these methods forCTA These choices include the setting for observations whether they are drawn from real-life or videotaped sessions and the use of other forms of data that are collected and com-bined with observations

Observations to support CTA can occur in a variety of settings including actual work


Figure 15 An example of a concept map This concept map represents the knowledge ofan expert in meteorology regarding Gulf Coast weather Figure courtesy of R R HoffmanInstitute for Human and Machine Cognition


environments high-fidelity simulations of work environments (eg cockpit simulators)and laboratories Additionally observations can occur during actual work during train-ing exercises or while operators are performing analyst-provided work tasks The recordedoutput of observations made to support CTAs can vary from unstructured opportunis-tic field notes (informed by the analystsrsquo expertise and goals) to more structured obser-vations based on predetermined categories

Observations are often made in real time as work activities are unfolding (eg Rothet al 2004) Roth and Patterson (2005) emphasized that naturalistic observations takenin real settings allow analysts to understand the full complexity of the work environmentThis includes understanding the complexities and cognitive demands faced by domainpractitioners and the strategies developed by domain practitioners to cope with demands

Observational studies are particularly useful for identifying mismatches between howwork is depicted in formal processes and procedures and how it is actually performedoften revealing ldquohome-grownrdquo tools and work-arounds that domain practitioners gen-erate to cope with aspects of task complexity that are not well supported (eg Roth Scottet al 2006) Divergence between so-called canonical descriptions of work and actual workpractice can reveal opportunities to improve performance through more effective support

Real-time observations in actual work settings are often combined with informal inter-views conducted as the task progresses In some cases participant observation methodsare employed in which analysts participate in the work performance (often in an appren-ticeship capacity) In most cases additional forms of data are collected (eg objectiverecords of unfolding events) and combined with the observations that are made (eitherin real time or from recordings) to create a rich protocol or process trace that captures theunfolding events and task activities thus allowing the activities of operators to be under-stood within the context of the task itself (Woods 1993)

As noted by Roth and Patterson (2005) naturalistic observational studies do not relyon the experimental design logic of controlled laboratory studies in which situational vari-ables are explicitly varied or controlled for Instead methodological rigor required forgeneralization is achieved by (a) sampling broadly including observing multiple domainpractitioners who vary in level of expertise and observing different work conditions (egshifts phases of operation) (b) triangulation using a variety of data collection and analy-sis methods in addition to observations and (c) employing multiple observersanalystswith differing perspectives (when possible) As with other qualitative analysis techniquesan important method for ensuring the validity of the observational components of a CTAis to check the findings with domain practitioners and experts themselves

CTAs demonstrate a rich variety of approaches in their use of observational method-ologies For instance Patterson and Woods (2001) conducted observations that focusedon space shuttle mission control shift change and handovers during an actual space shut-tle mission They combined observations with handwritten logs and spontaneous ver-balizations of the controllers (captured via audiotape) along with flight plans to identifyhandover activities that were related to fault management replanning and maintainingcommon communicational ground

Mumaw et al (2000) used observational methods to study operator monitoring strate-gies in nuclear power plant control rooms They conducted observational studies at mul-tiple sites to uncover the variety of information sources and strategies that are used by



power plant operators to support monitoring performance Initial observations com-bined with feedback from an operator who reviewed the preliminary findings were lever-aged to define more targeted observational goals for subsequent observations (eg to noteoperator use of the interface to support monitoring to identify ways monitoring couldbecome difficult) as well as to generate specific probe questions to ask operators as theobservations were taking place (eg asking about reasons for monitoring or about reg-ular monitoring patterns)

Baxter et al (2005) used targeted observations to log alarm events and caregiver inter-actions with equipment in a neonatal intensive care unit Observations were used alongwith interviews based on the CDM along with analyses showing communication patternsand written document use to make recommendations for the design of a decision aidintended to support the selection of ventilator settings

Observations can also be performed under more controlled conditions For exampleindividuals may be instructed to think aloud as they perform the task to provide an ongo-ing verbal protocol of the task (see Bainbridge amp Sanderson 1995 for extensive details onthe collection and analysis of verbal protocol data) Observations to support CTA can alsooccur in the laboratory For instance Gorman et al (2004) observed functionally blindusers performing specified Internet search tasks in conjunction with a screen reader Userswere asked to think aloud during the task in a laboratory environment and decision mod-els were developed to describe their activities

Video and audio recordings can be used to capture observational data for later analy-sis Video recordings of activities can be employed to support different types of analysesincluding qualitative analysis of activities (Miles amp Huberman 1984 Sanderson amp Fisher1994) For example Kirschenbaum (2004) observed groups of weather forecasters in eithertheir everyday work setting as they performed normal forecasting duties or in a simulatedshipboard forecasting center as they worked on a provided scenario Team activities alongwith think-aloud protocols were captured via videotape to allow for the detailed quali-tative data analysis of cognitive activities related to weather forecasting

Seagull and Xiao (2001) used video recordings on which eye-tracking data had beensuperimposed to study a surgical procedure The recordings were made from the per-spective of the physician performing the procedure (wearing mobile recording and eye-tracking equipment) The eye-tracking data indicated where (in the operating room) thephysician looked throughout the procedure The tapes were reviewed by the physician andother subject matter experts to determine what the physician had to look at to accom-plish the task what that information would indicate and why it was sought by the physi-cian at that point in the taskmdashin essence to identify information cues and their purposeduring the task Seagull and Xiao (2001) found that this technique provided informationregarding task strategies that they had not uncovered through other analyses Furthercomparing the eye-tracking recordings with previously completed task analyses led tothe discovery of nonvisual information use strategies (instances in which the task analy-sis indicated the need for information but the cue was absent from the eye-trackingrecording)

Video records make it possible to collect cued retrospective explanations of task per-formance by the individuals who participated in the task (Hoc amp Leplat 1983) They canalso be leveraged to elicit additional knowledge from other subject matter experts J E



Miller Patterson and Woods (2006) described a critiquing process for performing a CTAthat relies on video- and audio-recorded data of a novice performing a task The resultsare used to create a script of the novicersquos performance that can then be critiqued by sub-ject matter experts The researchers recorded a novice completing a complex (militaryintelligence analysis) task during which the novice was asked to think aloud Six expertintelligence analysts were asked to read a transcript of the novicersquos verbalizations whilebeing shown additional material (eg screen shots captured documents accessed andhandwritten notes generated by the novice during the task) Experts were asked to com-ment on the novicesrsquo performance as the script was presented Audio and video record-ings along with handwritten notes of the critiquing process were used to generate aprotocol which was then analyzed to provide insight into how experts approach this task

Observational data are amenable to both qualitative and quantitative analysis depend-ing on the type of data collected Typically however observation-based CTAs lean towarda more qualitative thematic analysis approach to identify the work complexities associ-ated cognitive demands and practitioner strategies that are the focus of a CTA In somecases previous research or theories are used to structure the analysis

For example Patterson Roth Woods Chow and Gomes (2004) applied a structuredapproach to a meta-analysis of four previously conducted observational studies of ambu-lance dispatch space shuttle control railroad dispatch and nuclear power control Ob-servational data from the original studies were coded according to a set of predefinedcategories related to shift change and hand-off strategies to identify common strategiesacross the multiple domains Ultimately analyses are guided by the theoretical stance andassociated representational forms adopted by the analyst as described in the next section

Methods for Representing the Results of CTA

The representations used to synthesize and communicate the results of a CTA play an im-portant (if somewhat underappreciated) role in the success or failure of any particularanalysis Forms of information representation can shape the process of cognitive task andwork analyses as well as provide a means for communicating analysis results Informationthat is gathered through means such as observations interviews or document analysismust be processed and structured in a way that reveals the complexities of the task andwork domain These representations are useful not only in supporting analysis but alsoin eliciting additional information from domain experts because the current understand-ing can be inspected and improved (Bisantz et al 2003)

The variety of representational forms used to synthesize and summarize CTA resultsis large representative of the background and inclinations of analysts performing thework and we will not survey them in detail What can be said however is that the repre-sentations used during analysis and for the presentation of results range from themati-cally organized narrative outputs to highly structured graphic representations

Narrative presentations of results are often associated with ethnographic observationmethods and are most suitable for presenting themes and conclusions that emerge fromdata collection and reflection (often a data-driven process) These can include providingsegments of think-aloud protocols transcriptions of dialogue or summary descriptionsof strategies that illustrate a theme (eg Mumaw et al 2000 Pfautz amp Roth 2006 Roth



Multer amp Raslear 2006 Roth amp Woods 1988 Watts et al 1996 Weir et al 2007) Table11 provides an example of a narrative representation that lists power plant operator strate-gies for extracting information about plant state

More structured representations are often used to provide summary depictions ofselected aspects of highly complex data sets such as timeline representations that mapthe evolution of events and decisions over time (eg Figure 16 which shows the highworkload and interruptions that nurses must cope with during medication administra-tion) and link analysis graphs that show operator movements within a workplace or com-munication events between individuals (eg Figure 17 and Baxter et al 2005)

In some cases theoretically motivated representations impose structure on the kindsof information that will be the focus of the analysis For instance the decision ladderstructure (see Figure 18 page 23 for an example) utilized in the suite of cognitive workanalysis methods focuses the analyst on identifying (human or automated) information-processing stages states of knowledge and shortcuts across those stages The abstractionhierarchy (means-end) formalism (see Figure 12) focuses analystsrsquo attention on intentionaland structural properties of the work domain These representations are often presentedin graphical form as sets of interlinked nodes (Bisantz et al 2003 Bisantz amp Vicente 1994Lintern 2006 Naikar 2006 Vicente 1999) but they can also be represented in tabular for-mat to support additional annotation (eg see Vicente 1999 pp 199ndash200)

Other forms of representation have a more bottom-up focus For example conceptmaps (see Figure 15) enable analysts to represent knowledge about a domain (eg gath-ered through interviews with domain practitioners) in a way that is structured by thepractitionersrsquo conceptualization of objects and relationships in a domain rather than bypredefined categories specified by a theoretical framework

In many cases the structure of information representation is intimately linked with theknowledge acquisition or analysis methodology itself Outputs from the cognitive workanalysis methodology such as the abstraction hierarchy and decision ladders noted pre-viously are examples Another example is the decision requirements table that is associ-ated with the CDM The decision requirements table documents key decisions cues and


Table 11 Example Narrative Representation

Strategies That Maximize Information Extraction From Available Data

Operators have developed strategies that can be used to maximize the information theyextract from the plant state data available to them

Reduce noise Operators displayed a variety of alarm management activities designedto remove noise so that meaningful changes could be more readily observed The followingare examples of these activities

(a) Clear alarm printer At shift turnover operators clear the printer of all alarms generatedon the previous shifthellip

(b) Cursor alarms (ie delete the alarm message from the screen before the alarmactually clears but do not disable it) when they are considered to be unimportant

Enhance signal This action increases the salience of visibility of an indicator

Note Excerpt taken with permission from Mumaw R J Roth E M Vicente K J and BurnsC M (2000) pp 47ndash48 at University at Buffalo Libraries on February 18 2014revsagepubcomDownloaded from

21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion


strategies used in making the decision specific challenges that complicate the decision-making process and potential pitfalls and errors to which less experienced practitionersare prone (Crandall et al 2006) The applied cognitive work analysis methodology (Elmet al 2003) produces multiple linked information representations (graphical and tabu-lar) that connect information-gathering activities to display design in order to provide cleardesign traceability The information representations include a functional abstraction net-work representing goals and associated processes and system components cognitive workrequirements stemming from the goals and processes associated information require-ments and finally requirements for information representation


Figure 17 Link analysis showing hypothetical communication links among personnel in ahospital emergency department Large circles represent individual caregivers (different physi-cians and nurses) and small circles represent groups of caregivers of a particular type withwhich the individuals hypothetically communicated Labels indicate types of caregivers (ieATTG = attending physician R1R2 = first- or second-year resident Tx = transporter) Thethickness of the links represents the frequency (in terms of percentage of communicationevents) of communication between a particular caregiver and other caregiver types For arelated study see Fairbanks Bisantz and Sunm (2007)

Communication partner of events


In other cases analysts have developed or adopted ad hoc representations to suit theirparticular project Examples include the graphical representations of intelligence analystsearch strategies developed by Patterson et al (2001 see Figures 13 and 14) abstractionhierarchy representations annotated with activity elements (Lintern 2006) a schematicshowing relative physical locations and types of communication links among NASA mis-sion controllers (Watts et al 1996) and cross-referenced functional matrices utilized byBisantz et al (2003 see Figure 19) to link system function decompositions to associatedhigher-level cognitive activities and to display areas that would support those functionsand activities

From a practical standpoint analysts will typically use a variety of representations inan opportunistic way choosing complementary capabilities to focus on and highlight keyaspects of the analysis


Figure 18 Example decision ladder model showing part of a task of detecting and identifyingsubmarines based on sensor data Small nodes represent information-processing stages thatare not part of this task Reprinted from the International Journal of Human-Computer Studies58 A M Bisantz E M Roth B Brickman L Gosbee L Hettinger and J McKinney Inte-grating cognitive analyses in a large-scale system design process 177ndash 206 Copyright 2003with permission from Elsevier

Assessing movement with activesonar Does it move ping-to-ping Is movement consistent withDoppler information Is signalshape consistent with movement

Continued observa-tion over time to gather informa-tion regarding movement allow evidence to build


RELATED APPROACHES

As noted in the beginning of the chapter methods for analyzing complex cognitive workhave historical roots in a number of disciplines these methods are also informed by andmay be performed in concert with a number of related analysis and modeling techniquesHere we review three such approaches that are closely related to CTAs We describe howtask-analytic methods that focus primarily on documenting observable or better-definedwork activities may be usefully combined with CTA techniques particularly in large proj-ects how cognitive modeling techniques may be used to represent what is learned fromCTA knowledge acquisition activities in a form that can be used to generate specific pre-dictions about human performance and finally how participatory approaches fromhuman-computer interaction may complement CTA methods


Figure 19 Cross-linked functional matrices showing links from ship functions to cognitivefunctions to display requirements Reprinted from the International Journal of Human-Computer Studies 58 A M Bisantz E M Roth B Brickman L Gosbee L Hettinger andJ McKinney Integrating cognitive analyses in a large-scale system design process 177ndash206Copyright 2003 with permission from Elsevier


Task-Analytic Approaches

Other task analysis methodologies may be useful in the analysis of cognitive work althoughthey do not traditionally focus on expertise and task and environmental complexities asdo the methods described previously

Hierarchical task analysis (HTA) is a well-known and often-utilized task analysis tech-nique that represents tasks through a goal-subgoal-activity decomposition (Annett 2003Kirwan amp Ainsworth 1992 Shepherd amp Stammers 2005) As with the CTA techniquesdescribed thus far information to support an HTA can be drawn from a number ofsources including interviews with subject matter experts document analysis and obser-vation (Stanton 2001) Tasks are described in terms of the operations (activities) thatachieve the task goals and in terms of the plans that indicate the order and preconditionsnecessary for executing the activities For instance a plan may specify operations that needto be performed iteratively until a stopping condition is met or may indicate that someoperations are optional based on a condition

The level of detail is flexible depending on the analystrsquos needs For instance Kirwanand Ainsworth (1992 p 11) noted that when one is analyzing how people interact withor control systems the level of analysis must capture details of the interaction (eg readinformation from screen enter a control action) however for applications such as train-ing support the level of detail of the analysis should be guided by the likelihood that anerror would be made combined with the cost of such an error The descriptive compo-nent of an HTA (the goal decomposition with related plans) is typically represented inan annotated tree structure (see Figure 110) and can be augmented with an analysis ofpotential failure modes (and thus the information knowledge andor skills required toalleviate these) associated with activities or plans (Annett 2003)

HTA has been used in a variety of applications such as specifying training requirements(Annett Cunningham amp Mathias-Jones 2000 Shepherd amp Kontogiannis 1998) identi-fying error potential (Shryane Westerman Crawshaw Hockey amp Sauer 1998) analyzingthe fit between emergency medical technician tasks and a portable computer designed toaid those tasks (Tang Zhang Johnson Bernstam amp Tindall 2004) and modeling tasksas input to the iterative design of user interfaces (Tselios amp Avouris 2003)

Another goal decomposition approach operator function modeling (OFM) has beenused to model human interaction with complex systems (Mitchell 1987 Mitchell amp Miller1986) In this technique system goals subgoals and operator activities are representedas a set of interconnected nodes Each node (corresponding to a goal or activity) has anassociated state space and next-state transition diagram showing how nodes change statesin response to external inputs or the states of higher-level goals For instance a node couldrepresent the activity of ldquocontrol cell phone ring modalityrdquo with states corresponding toldquoaudible signal onrdquo and ldquovibrate signal onrdquo One would transition among these states basedon higher-level goals (eg work uninterrupted by noise) or the situational context andassociated external demands (eg a concert begins) Operator function models have beeninstantiated to support human performance in a number of contexts including trainingsupport using intelligent tutoring in satellite ground control (Chu Mitchell amp Jones 1995)and display design for information retrieval (Narayanan Walchli Reddy amp Balachan-dran 1997)



Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26


Although not typically considered CTA techniques methods such as HTA and OFMcan provide a framework that allows the identification of areas for CTA analysis may allowsome aspects of tasks identified through a CTA to be specified in more detail or may pro-vide information that is complementary to that derived from other forms of analysisC A Miller and Vicente (2001) for example demonstrated how performing an HTA inaddition to a work domain analysis of a thermal-hydraulic microworld provided infor-mation to support display design that complemented the results from the work domainanalysis (specifically information related to executing task procedures) Shepherd andStammers (2005) noted that it is important to recognize that techniques such as HTA arenot in opposition to those labeled CTA and that the choice is not exclusive rather meth-ods should be chosen and applied in a way that accomplishes the necessary analysis

These methods could be used as a framework to represent an overall task and to iden-tify task aspects such as planning decision making or fault diagnosis which can then beexplored using CTA analyses For instance Chrenka Hutton Klinger and Anastasi (2001)described a tool in which an operator function model of a complex system is used as anorganizing framework against which cognitively challenging components can be identifiedand categorized for additional focus using extensive CTA methods (such as those describedpreviously) Tang et al (2004) used an HTA to describe emergency medical technician tasksand then performed a GOMS (goals operators methods selection rules) analysis of someof the cognitively demanding tasks

Lee and Sanquist (2000) described a CTA method that augments an operator functionmodel by specifying the cognitive activities information-processing demands task inputsand outputs and task and environmental demands associated with the functions and ac-tivities specified in an OFM For example for a target identification function they iden-tified cognitive activities such as identification task inputs such as a potential threat seenon a radar screen information-processing requirements such as perception and long-termmemory a task output of a restricted set of objects to monitor and external demands suchas the number of targets and their rate of change Raby McGehee Lee and Nourse (2000)applied this method to aid in display design for snowplow operators

Cognitive Modeling Approaches

Another technique that can complement or augment CTAs as described previously is todevelop a formal (often computer-based) model that represents the knowledge and infor-mation processes that are presumed to be required for cognitive task performance (CardMoran amp Newell 1983 Gray amp Altmann 2001 Ritter amp Young 2001) Cognitive modelsprovide a means to represent what is learned from CTA knowledge acquisition activitiesin a form that can be used to generate specific predictions about the performance of hu-mans when confronted with different situations (eg when using different displays orsupport systems to perform the same cognitive tasks)

There are a variety of approaches to cognitive modeling Some types of cognitive mod-els such as the GOMS family of models (John amp Kieras 1996) utilize a cognitively ori-ented goal decomposition approach that falls under the broad class of task-analyticmethods Other types of cognitive models are computational models that simulate the cog-nitive processes that are hypothesized to underlie task performance We summarize some



of the most prominent approaches Comprehensive reviews can be found in Pew andMavor (1998) Ritter et al (2003) Chipman and Kieras (2004) and Gray (2007)

The GOMS family of models represents one of the most accessible and widely used cog-nitive modeling approaches (John amp Kieras 1996) GOMS models provide a formalismfor decomposing and representing tasks in terms of the personrsquos goals elemental mentaland physical operators that combine to achieve goals (eg pressing a key retrieving a pieceof information from memory) available methods which are sequences of operators thatcan be used to accomplish the goals and selection rules that specify which methods to usein different situations

GOMS models are particularly suited for modeling well-understood routine tasks (Grayamp Altmann 2001) They provide an analytic means to predict task performance timeslearning times and workload (Gray amp Boehm-Davis 2000 Kieras 1998) They have beensuccessfully used to evaluate the adequacy of a user interface design as well as to comparealternative designs (Kieras 1998) Design aspects that can be checked with a GOMS modelinclude whether methods are available for all user goals that have been specified whetherthere are efficient methods for common user goals and whether there are ways to recoverfrom errors (Chipman amp Kieras 2004)

Network models are another common approach for cognitive modeling A prominentexample is the family of IMPRINT (IMproved Performance Research INTegration Tool)models that are used by the US Army to predict the performance of military systems(Booher 2003) Network models decompose tasks into elemental subtasks that are com-bined in a network representation to predict performance Typically each elemental sub-task has associated with it an estimated performance time and probability of successparameter (typically represented as a distribution) Monte Carlo simulations are per-formed to generate statistical distribution predictions for overall task performance timeslearning times andor workload measures

There are also computer-based models that attempt to simulate the actual mental pro-cesses (sensory perceptual cognitive and motor activities) that are presumed to underliehuman cognitive performance Examples include COGNET (COGnition as a NETworkof Tasks Zachary Ryder Ross amp Weiland 1992) MIDAS (Man-Machine IntegrationDesign and Analysis System Laughery amp Corker 1997) and OMAR (Operator ModelArchitecture Deutsch 1998) Included in this class are models built using cognitive archi-tectures that embody psychological theories of human cognitive performance Cognitivearchitectures with an extensive research base include SOAR (Laird Newell amp Rosenbloom1987) ACT-R (Adaptive Control of ThoughtmdashRational Anderson et al 2004 Andersonamp Lebiere 1998) and EPIC (Executive-ProcessInteractive Control Kieras amp Meyer 1997Kieras Woods amp Meyer 1997)

Cognitive models have been successfully used to develop fine-grained models of rou-tine task performance as a way to explore the impact of different interface designs (Grayamp Boehm-Davis 2000 Kieras 2003) Examples include evaluation of telephone informa-tion operator workstations (Gray John amp Atwood 1993) the efficiency of alternative cellphone menu structures (St Amant Horton amp Ritter 2007) and the design of commercialcomputer-aided design systems (Gong amp Kieras 1994) In each case the models success-fully predicted substantial differences in performance times as a function of system design

More recently cognitive models particularly cognitive simulations built on cognitive



architectures have been applied to more complex cognitive tasks and tasks that involvemultiperson communication and coordination For example the NASA Aviation Safetyand Security program had five teams develop cognitive simulation models of pilots per-forming taxi operations and runway instrument approaches with and without advanceddisplays (Foyle Goodman amp Hooey 2003 Foyle et al 2005) The models utilized differ-ent cognitive architectures to illuminate different aspects of pilot cognitive performanceand contributors to error

For example Byrne and Kirlik (2005) used ACT-R to model pilotsrsquo scanning behaviorand to explore the impact of the structure of the environment on errors Deutsch and Pew(2004) utilized D-OMAR to examine the impact of expectations and habits on potentialfor error Lebiere et al (2002) employed a hybrid model that integrates IMPRINT withACT-R to explore the impact of differences in individual cognitive perceptual and motorabilities as well as changes in the environment on performance and error

Boehm-Davis Holt Chong and Hansberger (2004) as part of a separate project uti-lized ACT-R to examine crew interaction during the descent phase of flight They createdseparate models for each of two pilots (one flying and one not) which they ran jointlyunder different conditions They manipulated the level of expertise and task load andshowed an impact on performance and error including differences in situation awarenessbetween the two pilots and crew miscommunications Other examples of cognitive mod-eling for complex dynamic domains include a cognitive simulation model of nuclearpower plant operator performance during emergencies (Roth Woods amp Pople 1992) andcognitive modeling of submarine officers (Gray amp Kirschenbaum 2000)

Cognitive models provide an effective means of establishing the adequacy of a cogni-tive analysis A cognitive model can be used to establish that the knowledge and process-ing assumed to underlie human performance in a particular task are in fact sufficient togenerate the observed behavior For example Roth et al (1992) developed a cognitivesimulation of dynamic fault management in nuclear power plant emergencies The cog-nitive simulation provided an objective means for establishing some of the cognitive ac-tivities required to handle the emergency event successfully As such it provided a tool forvalidating and extending the CTA that was performed based on discussions with instruc-tors review of procedures and observations of crews in simulated emergencies

Cognitive models not only provide a formal means for representing the results of a CTAthey can also generate new insights into the cognitive contributors to performance Forexample Byrne and Bovair (1997) developed a cognitive model that embodied a theoryof memory activation to explain a common type of human error called a postcompletionerror It has often been observed that task steps that need to occur after a personrsquos maingoal has been achieved are prone to omission errors (eg people regularly forget to takethe original sheet out of the copier or to take their bank card out of the automatic tellermachine) Byrne and Bovair (1997) built a computer model based on a theory of mem-ory that exhibited that behavior This model served both to strengthen the validity of thetheory and to illuminate the reason for the error

Another example of using a model to illuminate the psychological basis of an ob-served phenomenon was provided by Kieras and Meyer (2000) It has been repeatedlyobserved that when people have to suddenly take over a function from an automated sys-tem performance is initially degraded This is referred to as automation deficit Kieras and



Meyer (2000) developed a cognitive model based on psychological theory that exhibitedsimilar behavior thus providing a theoretically grounded account of the phenomenon

These two examples illustrate the use of cognitive models as a way to build and testcognitive theories to explain observed performance The ultimate aim is to build cogni-tive models that have sufficient theoretical grounding that they can be generalized acrossapplications and domains Although they are not examples of CTAs aimed at specific ap-plication they illustrate ways to illuminate the cognitive contributors to performance

Related Approaches From Human-Computer Interaction

As with CTA methods within the human-computer interaction and software design com-munities have been developed that focus the requirements-gathering development anddesign processes on users in the context of their work and tasks Those who employ par-ticipatory analysis and design techniques take the view that for software systems to be suc-cessful the ultimate users of the systems need to be directly involved in all phases of thedesign process and empowered to make design decisions (Bodker Kensing amp Simonsen2004 Clement amp Van den Besselaar 1993 Greenbaum amp Kyng 1991 Mueller Hasl-wanter amp Dayton 1997 Schuler amp Namioka 1993) Participatory design includes a vari-ety of hands-on techniques and methods that tend to involve small groups of designersand users performing activities such as paper prototyping and brainstorming (For anextensive set of examples see Mueller et al 1997)

Contextual inquiry (Beyer amp Holtzblatt 1998) a comprehensive analysis and designprocess involving users encompasses a number of activities that in some ways correspondto those conducted during a CTA Information about a work domain is gathered throughobservations and interviews and specific models are generated that allow work processescommunication patterns task steps workplace objects and layout cultural practices orga-nizational factors and workplace artifacts to be documented shared and used as inputto a design process For example work flow models though not emphasizing the cogni-tive activities or domain complexities typically identified in a CTA provide a means ofrepresenting people along with the types of communication and coordination activitiesthat occur between them

Scenario-based design (Carroll 1995 2000) emphasizes the development and analysisof user interaction scenarios that describe work activities Scenarios are ldquoconcrete narra-tive descriptions of activity that the user engages in when performing a specific task adescription sufficiently detailed so that design implications can be inferred and reasonedaboutrdquo (Carroll 1997 p 396) Scenarios can support a number of functions during adesign process For example they can facilitate discussion among designers and usersregarding current activities and how new technology could be used (during requirementsgathering) they can provide the basis for tasks during testing and evaluation and theycan be used in training to demonstrate system functionality to users in a meaningful way(Carroll 1997) In similar ways scenarios are often integrated into CTA methodologies(eg to guide discussion with subject matter experts to generate concrete tasks for think-aloud protocols)



ADAPTING METHODS TO PROJECT OBJECTIVES AND CONSTRAINTS

The foregoing review of CTA and related methods makes clear that a large toolkit of meth-ods is available to an analyst attempting to characterize cognitive work in a particular set-ting As we have tried to emphasize what is fundamentally important in performing a CTAis to capture (a) the domain characteristics and constraints that define the cognitive re-quirements and challenges and (b) the knowledge skills and strategies that underlie bothexpert performance and the error-vulnerable performance of domain practitioners

The selection and timing of particular CTA methods will depend on the goals and prag-matic constraints of the specific situation What kind of information and level of detailare needed How much time is available What kind of access is available to domainexperts What is the nature of the work and does it lend itself to observation

The choice of CTA method (or methods) will be strongly guided by analysis objectivesIf the goal of the analysis is to identify ldquoleverage pointsrdquo where new technology could havesignificant positive impact then techniques that provide a broad-brush overview of cog-nitive and collaborative requirements and challenges in a domain such as field obser-vations and structured interviews can be very effective If the goal is to develop trainingprograms or to produce assessment protocols to establish practitioner proficiency (egfor accreditation purposes) then methods that capture the detailed knowledge and skills(eg mental models declarative and procedural knowledge) that distinguish practition-ers at different levels of proficiency (eg the CDM and process trace approaches) can beparticularly useful On the other hand if the goal is to develop a computer model thatsimulates the detailed mental processes involved in performing a task then techniquessuch as think-aloud verbal protocol methods may be most appropriate

The particular set of techniques selected will also be strongly determined by the prag-matics of the specific local conditions For example access to domain practitioners is oftenlimited In those cases other sources of domain knowledge (eg written documents)should be leveraged to maximize productive use of time with domain experts In somecases observing domain experts in actual work practice (eg using ethnographic meth-ods or simulator studies) may be impractical in those cases structured interview tech-niques (eg concept mapping) and critical incident analysis may be the most practicalmethods available In other cases domain experts may not be accessible at all (eg in highlyclassified government applications) in which case it may be necessary to look for sur-rogate experts (eg individuals who have performed the task in the past) or analogousdomains to examine

Several CTA studies serve to illustrate the impact of analysis goals and local pragmat-ics on the selection of CTA methods For example researchers interested in uncoveringmismatches between the prescribed approach to task performance and actual work prac-tice tend to use field observations because they provide a direct window on actual practice(eg Patterson Cook amp Render 2002) However field observations are not always a prac-tical option Field observations are impractical when studying work that happens pri-vately or over a long span of time (eg planning or design tasks that can span multipledays and involve solitary work that is not externally observable) Field observations arealso inefficient for the study of rare events the occurrence of which cannot be reliably



predicted (eg response to emergencies) In those situations other CTA approaches arerequired

The CDM was developed partly in response to the need to study expertise in situationsin which field observation was not a practical option (Klein Calderwood amp Clinton-Cirocco 1986) As described in Crandall et al (2006) Klein and his associates initiallyattempted to study the decision making of firefighters by ldquoshadowingrdquo them and gettingthem to think aloud (Klein et al 1986) However they quickly discovered that fires arerelatively rare occurrences and that asking individuals to think aloud is not a practical re-quest in high-stress time-pressured conditions such as firefighting Thus the CDM grewout of a need to tailor methods to the demands of the knowledge acquisition conditions

Another example of the need to adapt methods to deal with local pragmatics arisesin domains such as intelligence analysis and information assurance analysis in whichsecurity concerns prevent analysts from discussing actual past cases Researchers have hadto come up with ingenious new methods to enable domain practitioners to express theirexpertise Patterson et al (2001) dealt with the challenge by having intelligence analystswork on analogous unclassified information search and integration tasks This enabledthem to uncover analystsrsquo search strategies in the face of data overload conditions DrsquoAmicoWhitley Tesone OrsquoBrien and Roth (2005) faced similar hurdles in trying to study howinformation assurance analysts detect and pursue network attacks They overcame thesecurity concern issues by asking the analysts to create hypothetical scenarios that sharedcritical characteristics with actual cases they encountered This enabled the research teamto uncover critical challenges that arise in the domain and the strategies that expert infor-mation assurance analysts have developed to handle them without needing to analyzeactual cases

Although we have focused on specific CTA methods it should be emphasized that CTAis fundamentally an opportunistic bootstrap process (Potter et al 2000) In most casesmultiple converging CTA methods are employed The selection and timing of specificCTA methods depend on local constraints The key is to develop an understanding of boththe characteristics of the domain that influence cognitive and collaborative performanceand the knowledge and strategies that domain practitioners possess

Typically the cognitive analyst might start by reading available documents that pro-vide background on the field of practice (eg training manuals or policy and procedureguides) This background knowledge will raise questions that can then be pursued throughfield observations andor interviews with domain practitioners In turn these may pointto complicating factors in the domain that place heavy cognitive demands on the user andcreate opportunities for user error It may also highlight discrepancies between how workis ldquosupposed to be donerdquo and how it ldquoactually gets donerdquo These in turn can point to oppor-tunities to improve performance and reduce the disconnect between proscriptions and ac-tual practice through improved training or support systems Further observations andorinterviews perhaps with different domain practitioners at different locations can then beconducted to build on and test the generality of initial tentative insights

When the results of using multiple methods domain practitioners and sites reinforceeach other confidence in the adequacy of understanding is increased If differences arefound it signals the need for analysis revision The research logic employed is similar tothe rationale that underpins grounded theory (Glaser amp Strauss 1967)



RESEARCH FRONTIERS

CTA research is continuing on several fronts Some of these fronts have been described inearlier sections and include the development of new knowledge acquisition methods andvariants the development of new computational modeling tools and the expansion ofpsychological theory on the cognitive and collaborative processes of individuals and teamsHere we focus on three research trends that are particularly salient (a) CTAs as appliedto multiperson teams and organizations (ie macroergonomic and macrocognition ap-plications) (b) development of software tools to support the CTA endeavors and (c)integration of CTA results into the systems engineering process particularly to supporthuman-system integration issues that arise as part of large first-of-a-kind design efforts

Macrocognition and Macroergonomic Applications of CTA

Over the past few years there has been growing interest in applying CTA methods to mul-tiperson units (Klein 2000 Klein et al 2003) This includes understanding the cognitiveand collaborative processes that underlie small team performance as well as the distrib-uted cognitive processes that span organizational- and managerial-level boundaries Theterm macrocognition was coined to capture the need to study this higher-level distributedaspect of cognition (Klein et al 2003) This move has coincided with growing interest inanalyzing and supporting the design of large complex sociotechnical systems and systemsof systems (eg military command and control systems railroad operations health caresystems) that fall under the umbrella of macroergonomics (Hendrick 2007 [chapter 2 thisvolume] Hendrick amp Kleiner 2001)

Examples of cognitive activities that underlie multiperson performance include com-munication patterns that foster shared situation awareness shared mental models andproblem-solving and decision-making strategies that lead to resilient team performance(or the converse brittle performance subject to error) Team CTA methods are relevant tothe analysis and design of team and organizational structures (eg Naikar Pearce Drummamp Sanderson 2003) the development of support systems for distributed multiperson per-formance (eg OrsquoHara amp Roth 2005) and the development of team and organizationaltraining (eg Salas amp Priest 2005)

Generally CTA studies of distributed cognitive processes have used variants of stan-dard CTA interview and observation techniques For example Klein Armstrong WoodsGokulachandra and Klein (2000) employed the CDM to examine the role of commonground in supporting coordination and replanning in distributed military teams RothMulter et al (2006) employed a combination of field observation and semistructured in-terviews to examine informal cooperative strategies developed by railroad workers (includ-ing train crews roadway workers and dispatchers) They documented a variety of informalcommunication strategies that served to foster shared situation awareness across the dis-tributed organization which contributed to efficiency safety and resilience to error of rail-road operations

A Miller and Xiao (2006) used semistructured interviews to examine resource alloca-tion strategies employed across organizational levels in a trauma hospital to cope with highpatient demand pressures They interviewed individuals at different managerial levels



(surgical unit medical director anesthesia staff and nursing staff schedulers and chargenurses) to understand scheduling and decision-making strategies at different levels of thework organization and how they combine in a nested fashion to achieve organizationalresilience in the face of variable-tempo resource demands

New CTA methods have also emerged that are specifically intended to analyze theknowledge and strategies that underlie multiperson performance These include methodsto analyze team knowledge (Cooke 2005) to measure shared situation awareness (MacMil-lan Paley Entin amp Entin 2005) to elicit and represent communication and coordinationpatterns (Harder amp Higley 2004 Jentsch amp Bowers 2005) and to understand the dis-tributed decision-making strategies and information requirements (Klinger amp Hahn2003 2005)

Software Tools to Support CTA Capture and DisseminationCurrently there is a paucity of software tools specifically tailored to the capture and dis-semination of CTA results Generally cognitive analysts rely on standard text-processingand drawing tools to document CTA results However these tools are limited in their abil-ity to support knowledge maintenance update and reuse This is a particular drawbackin the case of large projects that span multiple years and that involve collection across mul-tiple domain practitioners and sites and multiple cognitive analysts

Some efforts have been made to develop software tools to support cognitive analystsin capturing integrating and disseminating CTA results These include the Work DomainAnalysis Workbench developed by Skilton Cameron and Sanderson (1998) the CmapToolssoftware suite created at the Institute for Human and Machine Cognition (2006) and theCognitive Systems Engineering Tool for Analysis (CSET-A Cognitive Systems EngineeringCenter 2004) However most systems to date have been developed as part of research anddevelopment efforts and are limited in robustness

Integrating Cognitive Requirements Into the SystemsEngineering ProcessAnother important research frontier is the development of methods and tools for moreeffectively integrating cognitive and domain analyses into large-scale system design proj-ects (eg next-generation ships or process control plants) Human-system integration spansa wide range of activities throughout a system life cycle (Booher 2003) It includes initialconcept development hardware and software specification function allocation staffing andorganization design procedures and training development and testing activities AlthoughCTA methods are clearly applicable there has been growing recognition of the need todevelop more systematic methods and tools for integrating the results of CTA into thesystems development process (Osga 2003 Pew amp Mavor 2007)

A number of cognitive engineering methods have emerged that incorporate cognitiveand work domain analyses as core activities These include decision-centered design (HuttonMiller amp Thordsen 2003) cognitive work analysis (Vicente 1999) applied cognitivework analysis (Elm et al 2003) situation awarenessndashoriented design (Endsley Bolte ampJones 2003) use-centered design (Flach amp Dominguez 1995) and work-centered design(Eggleston 2003)



There are also a number of successful examples of the application of cognitive and workdomain analysis in systems development These include a redesign of the weapons direc-tor station in an advanced surveillance and command aircraft (Klinger amp Gomes 1993)design of crew composition for a new air defense platform (Naikar et al 2003) design ofnext-generation navy ships (Bisantz et al 2003 Burns Bisantz amp Roth 2004) and designof integrated visualizations to support dynamic mission monitoring and replanning foran airlift service organization (Roth Stilson et al 2006) There is a need for further workin developing ways to better integrate cognitive and domain analyses into large-scale sys-tems engineering as well as for more examples of successful integration efforts

CONCLUSIONS

Rather than representing a single technique or procedure CTA comprises a wide range oftheoretical perspectives data collection methods and analysis and representational choicesThis rich diversity of approaches is held together by a common goal of understanding andsupporting complex cognitive work CTAs necessarily involve examination of both thecharacteristics and demands of the work domain as well as the knowledge and strategiesthat domain practitioners have developed in response to domain demands The surveyof CTA cognitive work analysis and related methods presented in this chapter demon-strates the wide diversity of available methods and how they can be combined and adaptedto meet the goals and pragmatic constraints of real-world projects

REFERENCES

Anderson J R Bothell D Byrne M D Douglass S Lebiere C amp Quin Y (2004) An integrated theory ofthe mind Psychological Review 111 1036ndash1060

Anderson J R amp Lebiere C (1998) The atomic components of thought Mahwah NJ ErlbaumAnnett J (2003) Hierarchical task analysis In E Hollnagel (Ed) Handbook of cognitive task design (pp 17ndash35)

Mahwah NJ ErlbaumAnnett J Cunningham D amp Mathias-Jones P (2000) A method for measuring team skills Ergonomics 43

1076ndash1094Bainbridge L amp Sanderson P (1995) Verbal protocol analysis In J R Wilson amp E N Corlett (Eds) Evaluation

of human work (pp 169ndash201) London Taylor amp FrancisBanaji M amp Crowder R (1989) The bankruptcy of everyday memory American Psychologist 44 1185ndash1193Baxter G D Monk A F Tan K Dear P R F amp Newell S J (2005) Using cognitive task analysis to facilitate

the integration of decision support systems into the neonatal intensive care unit Artificial Intelligence inMedicine 35 243ndash257

Beyer H amp Holtzblatt K (1998) Contextual design Defining customer-centered systems New York MorganKaufmann

Bisantz A M amp Drury C G (2005) Applications of archival and observational data In J R Wilson amp NCorlett (Eds) Evaluation of human work (3rd ed pp 61ndash82) Boca Raton FL Taylor amp Francis

Bisantz A M Little E amp Rogova G (2004) On the integration of cognitive work analysis within a multi-source information fusion development methodology In Proceedings of the Human Factors and ErgonomicsSociety 48th Annual Meeting (pp 494ndash498) Santa Monica CA Human Factors and Ergonomics Society

Bisantz A M Roth E M Brickman B Gosbee L Hettinger L amp McKinney J (2003) Integrating cogni-tive analyses in a large-scale system design process International Journal of Human-Computer Studies 58177ndash206



Bisantz A M amp Vicente K J (1994) Making the abstraction hierarchy concrete International Journal ofHuman-Computer Studies 40 83ndash117

Blomberg J Giacomi J Mosher A amp Swenton-Wall P (1993) Ethnographic field methods and their relation-ship to design In D Shiler amp A Namioka (Eds) Participatory design Principles and practices (pp 123ndash157)Mahwah NJ Erlbaum

Bodker K Kensing F amp Simonsen J (2004) Participatory IT design Designing for business and workplace real-ities Cambridge MA MIT Press

Boehm-Davis D A Holt R W Chong R amp Hansberger J T (2004) Using cognitive modeling to understandcrew behavior In Proceedings of the Human Factors and Ergonomics Society 48th Annual Meeting (pp 99ndash103)Santa Monica CA Human Factors and Ergonomics Society

Booher H R (2003) Handbook of human system integration Hoboken NJ WileyBurns C M Bisantz A M amp Roth E M (2004) Lessons from a comparison of work domain models Repre-

sentational choices and their implications Human Factors 46 711ndash727Burns C M amp Hajdukiewicz J (2004) Ecological interface design Boca Raton FL CRC PressBurns C M Kuo J amp Ng S (2003) Ecological interface design A new approach for visualizing network man-

agement Computer Networks The International Journal of Computer and Telecommunications Networking43(3) 369ndash388

Byrne M D amp Bovair S (1997) A working memory model of a common procedural error Cognitive Science21 31ndash61

Byrne M D amp Kirlik A (2005) Using computational cognitive modeling to diagnose possible sources of avi-ation error International Journal of Aviation Psychology 15 135ndash155

Cameron C A Beemsterboer P L Johnson L A Mislevy R J Steinberg L S amp Breyer F J (2000) Acognitive task analysis for dental hygiene Journal of Dental Education 64(5) 333ndash351

Card S Moran T amp Newell A (1983) The psychology of human-computer interaction Hillsdale NJ ErlbaumCarroll J M (1995) Scenario-based design Envisioning work and technology in system development New York

WileyCarroll J M (1997) Scenario-based design In M Helander T K Landauer amp P V Prabhu (Eds) Handbook

of human-computer interaction (pp 383ndash405) Amsterdam Elsevier ScienceCarroll J M (2000) Making use Scenario-based design of human-computer interactions Cambridge MA

MIT PressChase W G amp Simon H A (1973) Perception in chess Cognitive Psychology 4 55ndash81Chi M T H Feltovich P J amp Glaser R (1981) Categorization and representations of physics problems by

experts and novices Cognitive Science 5 121ndash152Chi M T H Glaser R amp Farr M L (Eds) (1988) The nature of expertise Mahwah NJ ErlbaumChipman S F amp Kieras D E (2004) Operator centered design of ship systems Presented at the Total Ship

Symposium Gaithersburg MD Available httpwwwcscmuedu~bejCognitiveModelingForUIDesignChipmanKieras04pdf

Chow R amp Vicente K J (2002) A field study of emergency ambulance dispatching Implications for decisionsupport In Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting (pp 313ndash317)Santa Monica CA Human Factors and Ergonomics Society

Chrenka J Hutton R J B Klinger D W amp Anastasi D (2001) The cognimeter Focusing cognitive taskanalysis in the cognitive function model In Proceedings of the Human Factors and Ergonomics Society 45thAnnual Meeting (pp 1738ndash1742) Santa Monica CA Human Factors and Ergonomics Society

Chu R W Mitchell C M amp Jones P M (1995) Using the operator function model and OFMspert as thebases for an intelligent tutoring system Towards a tutoraid paradigm for operators of supervisory controlsystems IEEE Transactions on Systems Man and Cybernetics 25 1054ndash1075

Clement A amp Van den Besselaar P (1993) A retrospective look at participatory design projects Communicationsof the ACM 36(4) 29ndash37

Cognitive Systems Engineering Center (2004) Cognitive Systems Engineering Tool for Analysis (Version 10)[Computer software] Pittsburgh PA ManTech International

Cooke N J (1994) Varieties of knowledge elicitation techniques International Journal of Human-ComputerStudies 41 801ndash849

Cooke N J (2005) Measuring team knowledge In N Stanton A Hedge K Brookhuis E Salas amp H Hendrick(Eds) Handbook of human factors and ergonomics methods (pp 5251ndash5255) Boca Raton FL CRC Press



Crandall B Klein G A amp Hoffman R R (2006) Working minds A practitionerrsquos guide to cognitive task analy-sis Cambridge MA MIT Press

DrsquoAmico A Whitley K Tesone D OrsquoBrien B amp Roth E (2005) Achieving cyber defense situational aware-ness A cognitive task analysis of information assurance analysts In Proceedings of the Human Factors andErgonomics Society 49th Annual Meeting (pp 229ndash233) Santa Monica CA Human Factors and ErgonomicsSociety

Dekker S W A (2002) The field guide to human error investigation London AshgateDekker S W A amp Woods D D (1999) To intervene or not to intervene The dilemma of management by excep-

tion Cognition Technology amp Work 1 86ndash96Deutsch S (1998) Interdisciplinary foundations for multiple-task human performance modeling in OMAR In

Proceedings of the 20th Annual Meeting of the Cognitive Science Society (pp 303ndash308) Mahwah NJ ErlbaumDeutsch S amp Pew R (2004) Examining new flight deck technology using human performance modeling In

Proceedings of the Human Factors and Ergonomics Society 48th Annual Meeting (pp 108ndash112) Santa MonicaCA Human Factors and Ergonomics Society

Dominguez C (2001) Expertise and metacognition in laparoscopic surgery A field study In Proceedings of theHuman Factors and Ergonomics Society 45th Annual Meeting (pp 1298ndash1302) Santa Monica CA HumanFactors and Ergonomics Society

Eggleston R G (2003) Work-centered design A cognitive engineering approach to system design In Proceed-ings of the Human Factors and Ergonomics Society 47th Annual Meeting (pp 263ndash267) Santa Monica CAHuman Factors and Ergonomics Society

Elm W C Potter S S Gualtieri J W Easter J R amp Roth E M (2003) Applied cognitive work analysis Apragmatic methodology for designing revolutionary cognitive affordances In E Hollnagel (Ed) Handbookof cognitive task design (pp 357ndash382) Mahwah NJ Erlbaum

Endsley M Bolte B Jones D G (2003) Designing for situation awareness An approach to user-centered designNew York Taylor and Francis

Ericsson K A amp Simon H A (1993) Protocol analysis Verbal reports as data Cambridge MA MIT PressFairbanks R J Bisantz A M amp Sunm M (2007) Emergency department communication links and patterns

Annals of Emergency Medicine 50 396ndash406Flach J M amp Dominguez C (1995) Use-centered design Integrating the user instrument and goal Ergonom-

ics in Design 3(3) 19ndash24Flanagan J C (1954) The critical incident technique Psychological Bulletin 51 327ndash358Foyle D C Goodman A amp Hooey B L (2003) An overview of the NASA aviation safety program (AvSP)

system-wide accident prevention (SWAP) human performance modeling (HPM) element In D C FoyleA Goodman amp B L Hooey (Eds) Proceedings of the 2003 Conference on Human Performance Modelingof Approach and Landing with Augmented Displays (pp 1ndash13) Moffett Field CA NASA

Foyle D C Hooey B L Byrne M D Corker K M Deutsch S Lebiere C et al (2005) Human perfor-mance models of pilot behavior In Proceedings of the Human Factors and Ergonomics Society 49th AnnualMeeting (pp 1109ndash1113) Santa Monica CA Human Factors and Ergonomics Society

Gilbreth F amp Gilbreth L (1919) Applied motion study London Sturgis and WaltonGlaser R amp Strauss A (1967) The discovery of grounded theory Strategies for qualitative research New York

AldineGong R amp Kieras D E (1994) A validation of the GOMS model methodology in the development of a spe-

cialized commercial software application In Proceedings of CHI (pp 351ndash357) New York Association forComputing Machinery

Gorman M E Militello L G Swierenga S J amp Walker J L (2004) Internet searching by ear Decision flowdiagrams for sightless Internet users In Proceedings of the Human Factors and Ergonomics Society 48th AnnualMeeting (pp 243ndash247) Santa Monica CA Human Factors and Ergonomics Society

Gray W D (Ed) (2007) Integrated models of cognitive systems New York Oxford University PressGray W D amp Altmann E M (2001) Cognitive modeling and human-computer interaction In W Karwowski

(Ed) International encyclopedia of ergonomics and human factors (Vol 1 pp 387ndash391) New York Tayloramp Francis

Gray W D amp Boehm-Davis D A (2000) Milliseconds matter An introduction to microstrategies and totheir use in describing and predicting interactive behavior Journal of Experimental Psychology Applied 6322ndash335



Gray W D John D E amp Atwood M E (1993) Project Ernestine Validating a GOMS analysis for predictingand explaining real-world task performance Human-Computer Interaction 8 237ndash309

Gray W D amp Kirschenbaum S S (2000) Analyzing a novel expertise An unmarked road In J M C SchraagenS F Chipman amp V L Shalin (Eds) Cognitive task analysis (pp 275ndash290) Mahwah NJ Erlbaum

Greenbaum J amp Kyng M (1991) Design at work Cooperative design of computer systems Mahwah NJ ErlbaumHall E P Gott S P amp Pokorny R A (1995) A procedural guide to cognitive task analysis The PARI methodol-

ogy Brooks Air Force Base TX Air Force Materiel CommandHammersley M amp Atkinson P (1983) Ethnography principles in practice London TavistockHarder R amp Higley H (2004) Application of thinklets to team cognitive task analysis In Proceedings of the

37th Hawaii International Conference on System Science (pp 1ndash9) New York Institute of Electrical andElectronics Engineers

Hendrick H W (2007) Macroergonomics The analysis and design of work systems In D Boehm-Davis (Ed)Reviews of human factors and ergonomics (Vol 3 pp 44ndash78) Santa Monica CA Human Factors and Ergo-nomics Society

Hendrick H W amp Kleiner B M (2001) Macroergonomics An introduction to work system design Santa MonicaCA Human Factors and Ergonomics Society

Hoc J M amp Leplat J (1983) Evaluation of different modalities of verbalization in a sorting task InternationalJournal of Man-Machine Studies 18 283ndash306

Hoffman R (1987 Summer) The problem of extracting the knowledge of experts from the perspective ofexperimental psychology AI Magazine 8 53ndash67

Hoffman R Coffey J W Ford K M amp Novak J D (2006) A method for eliciting preserving and sharingthe knowledge of forecasters Weather and Forecasting 21 416ndash428

Hoffman R Crandall B amp Shadbolt N (1998) Use of the critical decision method to elicit expert knowledgeA case study in the methodology of cognitive task analysis Human Factors 40 254ndash276

Hoffman R amp Hanes L F (2003) The boiled frog problem IEEE Intelligent Systems 68ndash71Hoffman R amp Lintern G (2006) Eliciting and representing the knowledge of experts In K A Ericsson N

Charness P Feltovich amp R Hoffman (Eds) Cambridge handbook of expertise and expert performance (pp203ndash222) New York Cambridge University Press

Hutchins S G Pirolli P amp Card S (2003) Use of critical analysis method to conduct a cognitive task analy-sis of intelligence analysts In Proceedings of the Human Factors and Ergonomics Society 47th Annual Meeting(pp 478ndash482) Santa Monica CA Human Factors and Ergonomics Society

Hutton R J B Miller T E amp Thordsen M L (2003) Decision-centered design Leveraging cognitive task analy-sis in design In E Hollnagel (Ed) Handbook of cognitive task design (pp 383ndash416) Mahwah NJ Erlbaum

Institute for Human and Machine Cognition (2006) CmapTools knowledge modeling kit Retrieved November6 2006 from httpcmapihmcus

Itoh J Sakuma A amp Monta K (1995) An ecological interface for supervisory control of BWR nuclear powerplants Control Engineering Practice 3 231ndash259

Jentsch F amp Bowers C (2005) Team communication analysis In N Stanton A Hedge K Brookhuis E Salasamp H Hendrick (Eds) Handbook of human factors and ergonomics methods (pp 5051ndash5055) Boca RatonFL CRC Press

John B E amp Kieras D E (1996) Using GOMS for user interface design and evaluation Which techniqueACM Transactions on Computer-Human Interaction 3 287ndash319

Johnson S Healey A Evans J Murphy M Crawshaw M amp Gould D (2006) Physical and cognitive taskanalysis in interventional radiology Clinical Radiology 61(1) 97ndash103

Kieras D E (1998) A guide to GOMS model usability evaluation using NGOMSL In M Helander (Ed)Handbook of human-computer interaction (pp 391ndash438) Amsterdam Elsevier

Kieras D E (2003) Model-based evaluation In J Jacko amp A Sears (Eds) Handbook for human-computer inter-action (pp 1139ndash1151) Mahwah NJ Erlbaum

Kieras D E amp Meyer D E (1997) An overview of the EPIC architecture for cognition and performance withapplication to human-computer interaction Human-Computer Interaction 12 391ndash438

Kieras D E amp Meyer D E (2000) The role of cognitive task analysis in the application of predictive modelsof human performance In J M C Schraagen S F Chipman amp V L Shalin (Eds) Cognitive task analysis(pp 237ndash260) Mahwah NJ Erlbaum



Kieras D E Woods D amp Meyer D E (1997) Predictive engineering models based on the EPIC architecturefor a multimodal high-performance human-computer interaction task Transactions on Computer-HumanInteraction 4 230ndash275

Kirschenbaum S S (2004) The role of comparison in weather forecasting Evidence from two hemispheres InProceedings of the Human Factors and Ergonomics Society 48th Annual Meeting (pp 306ndash310) Santa MonicaCA Human Factors and Ergonomics Society

Kirwan B amp Ainsworth L K (1992) A guide to task analysis London Taylor amp FrancisKlein G A (1992) Using knowledge elicitation to preserve corporate memory In R R Hoffman (Ed) The

psychology of expertise Cognitive research and empirical AI (pp 170ndash190) Mahwah NJ ErlbaumKlein G A (1998) Sources of power How people make decisions Cambridge MA MIT PressKlein G A (2000) Cognitive task analysis of teams In J M Schraagen S F Chipman amp V L Shalin (Eds)

Cognitive task analysis (pp 417ndash429) Mahwah NJ ErlbaumKlein G A amp Armstrong A A (2005) Critical decision method In N Stanton A Hedge K Brookhuis E

Salas amp H Hendrick (Eds) Handbook of human factors and ergonomics methods (pp 3531ndash3538) BocaRaton FL CRC Press

Klein G A Armstrong A A Woods D D Gokulachandra M amp Klein H (2000) Cognitive wavelength Therole of common ground in distributed replanning (Tech Rep AFRL-HE-WP-TR-2001-0029) Wright-PattersonAir Force Base OH US Air Force Research Laboratory

Klein G A Calderwood R amp Clinton-Cirocco A (1986) Rapid decision making on the fire-ground In Pro-ceedings of the Human Factors and Ergonomics Society 30th Annual Meeting (pp 576ndash580) Santa MonicaCA Human Factors and Ergonomics Society

Klein G A Calderwood R amp MacGregor D (1989) Critical decision method for eliciting knowledge IEEETransactions on Systems Man and Cybernetics 19 462ndash472

Klein G A Orasanu J Calderwood R amp Zsambok C E (Eds) (1993) Decision making in action Modelsand methods Norwood NJ Ablex

Klein G A Ross K G Moon B M Klein D E amp Hollnagel E (2003 MayndashJune) Macrocognition IEEEIntelligent Systems 18 81ndash85

Klinger D W amp Gomes M G (1993) A cognitive systems engineering application for interface design InProceedings of the Human Factors and Ergonomics Society 37th Annual Meeting (pp 16ndash20) Santa MonicaCA Human Factors and Ergonomics Society

Klinger D W amp Hahn B B (2003) Handbook of team CTA Fairborn OH Klein AssociatesKlinger D W amp Hahn B B (2005) Team decision requirement exercise Making team decision requirements

explicit In N Stanton A Hedge K Brookhuis E Salas amp H Hendrick (Eds) Handbook of human factorsand ergonomics methods (pp 5251ndash5255) Boca Raton FL CRC Press

Laird J E Newell A amp Rosenbloom P S (1987) SOAR An architecture for general intelligence ArtificialIntelligence 33(1) 1ndash64

Laughery K R amp Corker K M (1997) Computer modeling and simulation of humansystem performanceIn G Salvendy (Ed) Handbook of human factors (2nd ed pp 1375ndash1408) New York Wiley

Lebiere C Biefeld E Archer R Archer S Allender L amp Kelley T D (2002) IMPRINTACT-R Integrationof a task network modeling architecture with a cognitive architecture and its application to human errormodeling In M J Chinni (Ed) 2002 Military government and aerospace simulation (Vol 34 pp 13ndash19)San Diego CA Society for Modeling and Simulation International

Lee J D amp Sanquist T F (2000) Augmenting the operator function model with cognitive operations Assess-ing the cognitive demands of technological innovation in ship navigation IEEE Transactions on SystemsMan and CyberneticsmdashPart A Systems and Humans 30 273ndash285

Leplat J (1986) The elicitation of expert knowledge In E Hollnagel G Mancini amp D D Woods (Eds)Intelligent decision support (pp 107ndash122) New York Springer-Verlag

Lincoln Y S amp Guba E G (1985) Naturalistic inquiry Newbury Park CA SageLintern G (2006) A functional workspace for military analysis of insurgent operations International Journal

of Industrial Ergonomics 36 409ndash422Luke T Brook-Carter N Parkes A M Grimes E amp Mills A (2006) An investigation of train driver visual

strategies Cognition Technology amp Work 8 15ndash29



MacMillan J Paley M J Entin E B amp Entin E E (2005) Questionnaires for distributed assessment of teammutual awareness In N Stanton A Hedge K Brookhuis E Salas amp H Hendrick (Eds) Handbook ofhuman factors and ergonomics methods (pp 5151ndash5159) Boca Raton FL CRC Press

Means B amp Gott S P (1988) Cognitive task analysis as a basis for tutor development Articulating abstractknowledge representations In L D M J Psotka amp S A Mutter (Eds) Intelligent tutoring systems Lessonslearned (pp 35ndash57) Mahwah NJ Erlbaum

Miles M B amp Huberman A M (1984) Qualitative data analysis Newbury Park CA SageMilitello L G amp Hutton R J B (1998) Applied cognitive task analysis (ACTA) A practitionerrsquos toolkit for

understanding cognitive task demands Ergonomics 41 1618ndash1641Miller A amp Xiao Y (2006) Multi-level strategies to achieve resilience for an organisation operating at capac-

ity A case study at a trauma centre Cognition Technology amp Work 9 51ndash66Miller C A amp Vicente K J (2001) Comparison of display requirements generated via hierarchical task and

abstraction-decomposition space analysis techniques International Journal of Cognitive Ergonomics 5335ndash356

Miller J E Patterson E S amp Woods D D (2006) Elicitation by critiquing as a cognitive task analysis method-ology Cognition Technology amp Work 8 90ndash102

Mitchell C M (1987) GT-MSOCC A domain for research on human-computer interaction and decision aid-ing in supervisory control systems IEEE Transactions on Systems Man and Cybernetics 17 553ndash572

Mitchell C M amp Miller R A (1986) A discrete control model of operator function A methodology for infor-mation display design IEEE Transactions on Systems Man and Cybernetics SMC-16 343ndash357

Mueller M J Haslwanter J amp Dayton T (1997) Participatory practices in the software lifecycle In MHelander T K Landauer amp P V Prabhu (Eds) Handbook of human-computer interaction (pp 256ndash296)Amsterdam Elsevier ScienceNorth Holland

Mumaw R J Roth E M Vicente K J amp Burns C M (2000) There is more to monitoring a nuclear powerplant than meets the eye Human Factors 42 36ndash55

Naikar N (2006) Beyond interface design Further applications of cognitive work analysis International Journalof Industrial Ergonomics 36 423ndash438

Naikar N Pearce B Drumm D amp Sanderson P (2003) Designing teams for first-of-a-kind complex systemsusing the initial phases of cognitive work analysis A case study Human Factors 42 202ndash217

Narayanan S Walchli S E Reddy N amp Balachandran R (1997) Model-based design of an information re-trieval system for a university library International Journal of Cognitive Ergonomics 1 149ndash167

Nehme C E Scott S D Cummings M L amp Furusho C Y (2006) Generating requirements for futuristicheterogeneous unmanned systems In Proceedings of the Human Factors and Ergonomics Society 50th AnnualMeeting (pp 235ndash239) Santa Monica CA Human Factors and Ergonomics Society

Nisbett R E amp Wilson T D (1977) Telling more than we can know Verbal reports on mental processesPsychological Review 84 231ndash259

OrsquoHara J M amp Roth E M (2005) Operational concepts teamwork and technology in commercial nuclearpower stations In C Bowers E Salas amp F Jentsch (Eds) Creating high-tech teams Practical guidance on workperformance and technology (pp 139ndash159) Washington DC American Psychological Association

OrsquoHare D Wiggins M Williams A amp Wong W (1998) Cognitive task analyses for decision-centereddesign and training Ergonomics 41 1698ndash1718

Osga G A (2003) Human-centered shipboard systems and operations In H R Booher (Ed) Handbook ofhuman systems integration (pp 743ndash793) Hoboken NJ Wiley

Patterson E S Cook R I amp Render M L (2002) Improving patient safety by identifying side effects fromintroducing bar coding in medication administration Journal of the American Informatics Association 9540ndash553

Patterson E S Roth E M amp Woods D D (2001) Predicting vulnerability in computer-supported inferentialanalysis under data overload Cognition Technology amp Work 3 224ndash237

Patterson E S Roth E M Woods D D Chow R amp Gomes J O (2004) Handoff strategies in settings withhigh consequences for failure Lessons for health care operations International Journal for Quality in HealthCare 16 125ndash132

Patterson E S amp Woods D D (2001) Shift changes updates and the on-call architecture in space shuttle mis-sion control Computer-Supported Cooperative Work 10 317ndash346



Pew R amp Mavor A S (Eds) (1998) Modeling human and organizational behavior Application to militarysimulations Washington DC National Academy Press

Pew R W and Mavor A S (Eds) (2007) Human-system integration in the system development process A newlook (Committee on Human-System Design Support for Changing Technology) Washington DC Com-mittee on Human Factors Division of Behavioral and Social Sciences and Education National ResearchCouncil National Academies Press

Pfautz J amp Roth E M (2006) Using cognitive engineering for system design and evaluation A visualizationaid for stability and support operations International Journal of Industrial Ergonomics 36(5) 389ndash407

Potter S S Gualtieri J W amp Elm W C (2003) Case studies Applied cognitive work analysis in the design ofinnovative decision support In E Hollnagel (Ed) Handbook of cognitive task design (pp 653ndash678) MahwahNJ Erlbaum

Potter S S Roth E M Woods D amp Elm W C (2000) Bootstrapping multiple converging cognitive task analy-sis techniques for system design In J M Schraagen S F Chipman amp V L Shalin (Eds) Cognitive taskanalysis (pp 317ndash340) Mahwah NJ Erlbaum

Raby M McGehee D V Lee J D amp Nourse G E (2000) Defining the interface for a snowplow lane-trackingdevice using a systems-based approach In Proceedings of the XIVth Triennial Congress of the InternationalErgonomics Association and 44th Annual Meeting of the Human Factors and Ergonomics Society (pp3369ndash3372) Santa Monica CA Human Factors and Ergonomics Society

Rasmussen J (1983) Skills rules and knowledge Signals signs and symbols and other distractions in humanperformance models IEEE Transactions on Systems Man and Cybernetics SMC-13 257ndash266

Rasmussen J (1986) Information processing and human-machine interaction An approach to cognitive engineer-ing New York North-Holland

Rasmussen J Pejtersen A M amp Goodstein L P (1994) Cognitive systems engineering New York WileyRitter F E Shadbolt N R Elliman D Young R M Gobet F amp Baxter G D (2003) Techniques for model-

ing human performance in synthetic environments A supplementary review Wright-Patterson Air Force BaseOH Human Systems Information Analysis Center

Ritter F E amp Young R M (2001) Embodied models as simulated users Introduction to this special issue onusing cognitive models to improve interface design International Journal of Human-Computer Studies 551ndash14

Roth E M Christian C K Gustafson M L Sheridan T Dwyer K Gandhi T K et al (2004) Using fieldobservations as a tool for discovery Analyzing cognitive and collaborative demands in the operating roomCognition Technology amp Work 6 148ndash157

Roth E M Malsch N Multer J amp Coplen M (1999) Understanding how train dispatchers manage andcontrol trains A cognitive task analysis of a distributed team planning task In Proceedings of the HumanFactors and Ergonomics Society 43rd Annual Meeting (pp 218ndash222) Santa Monica CA Human Factorsand Ergonomics Society

Roth E M amp Multer J (2005) Fostering shared situation awareness and on-track safety across distributedteams in railroad operations In Proceedings of the Human Factors and Ergonomics Society 49th AnnualMeeting (pp 529ndash533) Santa Monica CA Human Factors and Ergonomics Society

Roth E M Multer J amp Raslear T (2006) Shared situation awareness as a contributor to high reliability per-formance in railroad operations Organization Studies 27 967ndash987

Roth E M amp Patterson E S (2005) Using observational study as a tool for discovery Uncovering cogni-tive and collaborative demands and adaptive strategies In H Montgomery R Lipshitz amp B Brehmer(Eds) How professionals make decisions (pp 379ndash393) Mahwah NJ Erlbaum

Roth E M Scott R Deutsch S Kuper S Schmidt V Stilson M et al (2006) Evolvable work-centered sup-port systems for command and control Creating systems users can adapt to meet changing demands Ergo-nomics 49 688ndash705

Roth E M Stilson M Scott R Whitaker R Kazmierczak T Thomas-Meyers G et al (2006) Work-centered design and evaluation of a C2 visualization aid In Proceedings of the Human Factors and ErgonomicsSociety 49th Annual Meeting (pp 332ndash336) Santa Monica CA Human Factors and Ergonomics Society

Roth E M amp Woods D D (1988) Aiding human performance 1 Cognitive analysis Le Travail Humain 4139ndash64



Roth E M amp Woods D D (1989) Cognitive task analysis An approach to knowledge acquisition for intelli-gent system design In G Guida amp C Tasso (Eds) Topics in expert system design (pp 233ndash264) AmsterdamElsevier Science

Roth E M Woods D D amp Pople H E (1992) Cognitive simulation as a tool for cognitive task analysisErgonomics 36 1163ndash1198

Salas E amp Priest H (2005) Team training In N Stanton A Hedge K Brookhuis E Salas amp H Hendrick(Eds) Handbook of human factors and ergonomics (pp 4441ndash4447) Boca Raton FL CRC Press

Salvendy G (2001) Handbook of industrial engineering Technology and operations management New YorkWiley

Sanderson P M (2003) Cognitive work analysis In J M Carroll (Ed) HCI models theories and frameworksToward a multi-disciplinary science (pp 225ndash264) San Francisco Morgan Kaufmann

Sanderson P M amp Fisher C (1994) Exploratory sequential data analysis Human-Computer Interaction 9251ndash317

Sanderson P M amp Watson M O (2005) From information content to auditory display with ecological inter-face design Prospects and challenges In Proceedings of the Human Factors and Ergonomics Society 49thAnnual Meeting (pp 259ndash263) Santa Monica CA Human Factors and Ergonomics Society

Schaafstal A Schraagen J M amp van Berlo M (2000) Cognitive task analysis and innovation of trainingThe case of structured troubleshooting Human Factors 42 75ndash86

Schraagen J M Chipman S F amp Shalin V L (Eds) (2000) Cognitive task analysis Mahwah NJ ErlbaumSchuler D amp Namioka A (1993) Participatory design Mahwah NJ ErlbaumSeagull F J amp Xiao Y (2001) Using eye-tracking video data to augment knowledge elicitation in cognitive task

analysis In Proceedings of the Human Factors and Ergonomics Society 45th Annual Meeting (pp 400ndash403)Santa Monica CA Human Factors and Ergonomics Society

Seamster T L Redding R E amp Kaempf G L (1997) Applied cognitive task analysis in aviation BurlingtonVT Ashgate

Sellie C N (1992) Predetermined motion-time systems and the development and use of standard data In GSalvendy (Ed) Handbook of industrial engineering (2nd ed pp 1639ndash1698) New York Wiley

Shepherd A amp Kontogiannis T (1998) Strategic task performance A model to facilitate the design of instruc-tion International Journal of Cognitive Ergonomics 2 349ndash372

Shepherd A amp Stammers R B (2005) Task analysis In J Wilson amp E N Corlett (Eds) Evaluation of humanwork (3rd ed pp 129ndash158) Boca Raton FL CRC Press

Shryane N M Westerman S J Crawshaw C M Hockey G R J amp Sauer J (1998) Task analysis for theinvestigation of human error in safety-critical software design A convergent methods approach Ergonomics41 1719ndash1736

Simon H A (1981) The sciences of the artificial (2nd ed) Cambridge MA Cambridge University PressSkilton W Cameron S amp Sanderson P M (1998) Supporting cognitive work analysis with the Work Domain

Analysis Workbench (WDAW) In Proceedings of the Computer Interaction Conference Australasian (pp260ndash267) Los Alamitos CA IEEE Computer Society

St Amant R Horton T E amp Ritter F E (2007) Model-based evaluation of expert cell phone menu interac-tion ACM Transactions on Human-Computer Interaction 14 1ndash24

Stanard T Lewis W R Cox D A Malek D A Klein J amp Matz R (2004) An exploratory qualitative studyof computer network attacker cognition In Proceedings of the Human Factors and Ergonomics Society 48thAnnual Meeting (pp 401ndash405) Santa Monica CA Human Factors and Ergonomics Society

Stanton N A (2001) Hierarchical task analysis In W Karwowski (Ed) International encyclopedia of ergonom-ics and human factors (pp 3183ndash3190) Boca Raton FL CRC Press

Tang Z Zhang J Johnson T R Bernstam E amp Tindall D (2004) Integrating task analysis in software usabil-ity evaluation A case study In Proceedings of the Human Factors and Ergonomics Society 48th Annual Meeting(pp 1741ndash1745) Santa Monica CA Human Factors and Ergonomics Society

Taylor F W (1911) The principles of scientific management New York NortonTselios N K amp Avouris N M (2003) Cognitive task modelling for system design and evaluation in non-

routine task domains In E Hollnagel (Ed) Handbook of cognitive task design (pp 303ndash330) MahwahNJ Erlbaum

Vicente K J (1990) A few implications of an ecological approach to human factors Human Factors SocietyBulletin 33(11) 1ndash4



Vicente K J (1999) Cognitive work analysis Mahwah NJ ErlbaumVicente K J (2002) Ecological interface design Progress and challenges Human Factors 44 62ndash78Vicente K J amp Rasmussen J (1992) Ecological interface design Theoretical foundations IEEE Transactions

on Systems Man and Cybernetics SMC-22 589ndash606Watts J C Woods D D Corban J M Patterson E S Kerr R L amp LaDessa C H (1996) Voice loops as

cooperative aids in space shuttle mission control Proceedings of the 1996 ACM Conference on ComputerSupported Cooperative Work (pp 48ndash56) New York ACM

Weir C R Nebeker J J R Hicken B L Campo R Drews F amp LeBar B (2007) A cognitive task analysisof information management strategies in a computerized provider order entry environment Journal ofthe American Medical Informatics Association 14(1) 65ndash75

Woods D D (1993) Process tracing methods for the study of cognition outside of the experimental psychol-ogy laboratory In G A Klein J Orasanu R Calderwood amp C E Zsambok (Eds) Decision-making inaction Models and methods (pp 228ndash251) Norwood NJ Ablex

Woods D D amp Dekker S W A (2000) Anticipating the effects of technological change A new era of dynam-ics for human factors Theoretical Issues in Ergonomics Science 1 272ndash282

Woods D D amp Hollnagel E (1987) Mapping cognitive demands in complex problem-solving worlds Inter-national Journal of Man-Machine Studies 26 257ndash275

Woods D D amp Hollnagel E (2006) Joint cognitive systems Patterns in cognitive systems engineering Boca RatonFL CRC Press

Yin R (1989) Case study research Design and methods Newbury Park CA SageZachary W Ryder J Ross L amp Weiland M (1992) Intelligent human-computer interaction in real time multi-

tasking process control and monitoring systems In M Helander amp M Nagamachi (Eds) Human factors indesign for manufacturability (pp 377ndash402) New York Taylor amp Francis


What is This

- Nov 1 2007Version of Record gtgt


CHAPTER 1








1











































Ca

su

alty M

an

age

me

nt

Ba

lan

ce

of A

uth

ority

Ris

k-B

enefit B

ala

nce

Re

sou

rce

Ba

lance

Eco

nom

ic B

ala

nce

Resp

on

se P

lan

sO

pe

ratio

ns P

roce

sse

s

Co

ord

ina

tin

g a

nd I

mp

lem

en

ting

Me

dic

al O

pe

ratio

ns

Co

ord

ina

tin

g a

nd I

mp

lem

en

ting

She

lte

r O

pera

tio

ns

Fix

ed L

oca

tion

Re

so

urc

es

(Ho

sp

ita

ls 9

11

Ca

ll C

ente

rs

Am

bu

lan

ce D

isp

atc

h F

ire

Sta

tion

s P

olic

e S

tatio

ns)

Mo

bile

Re

so

urc

es

(Am

bu

lan

ce

s

Fire

Re

scu

e

Fir

e E

ngin

es P

olic

eV

eh

icle

s)

Hu

ma

n R

eso

urc

es

(Po

lice

E

MT

F

ire

)

Sta

te o

f b

uild

ing

s o

pe

ratin

gcon

ditio

ns lo

ca

tio

n

ca

pa

city ca

pacity v

s lo

ad

(eg

fo

r a

ho

spita

l)

Nu

mb

er

co

nd

itio

n

op

era

tion

al sta

te

locatio

nof m

obile

reso

urc

es

pers

on

ne

l

Fu

nctio

na

l P

urp

ose

Ge

ne

ral P

roce

sse

s

Ab

str

act F

un

ction

Ph

ysic

al F

un

ctio

n

Ph

ysic

al F

orm

Sys

tem

Sub

-Sys

tem

Fig

ure

12

Por

tion

of a

n ab

stra

ctio

n hi

erar

chy

wor

k do

mai

n m

odel

for

an

emer

genc

y re

spon

sesy

stem

B

ased

on

wor

k fr

om B

isan

tz

Littl

e a

nd R

ogov

a (2

004)

7



























11

Qu

ery

(k

ey

wo

rds

refi

nem

en

t)

Sele

ct ldquo

key

rdquo d

ocu

men

ts

Co

rro

bo

rate

in

form

ati

on

(o

r

reso

lve d

iscr

ep

an

cies

in

info

rmati

on

) an

d f

ill

in g

ap

s

wit

h s

up

po

rt d

ocu

men

ts

Sy

nth

esi

ze i

nfo

rmati

on

to

con

stru

ct a

co

here

nt

sto

ry

Bro

wse

by

tit

le a

nd

o

r d

ate

19

960

80

8

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

19

960

50

0

AR

IAN

E 5

A

N A

LL

NE

W L

AU

NC

HE

R F

OR

TH

E G

LO

BA

L M

AR

KE

T

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

19

960

61

0

AR

IAN

E 5

EX

PL

OD

ES

IN

TH

E S

KY

OV

ER

KO

UR

OU

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

eu

rop

e 1

996

(eu

rop

e 1

996)

amp (

lau

nch

fail

ure

)

(eu

rop

e 1

996)

amp (

(la

un

ch f

ail

ure

)

2)

378

140

22

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

Da

te

Au

gu

st 8

1996

So

urc

e

FB

IS r

ep

ort

AC

CO

RD

ING

T

O T

HE

R

EP

OR

T R

EC

EN

TL

Y S

UB

MIT

TE

D B

Y T

HE

IND

EP

EN

DE

NT

IN

QU

IRY

B

OA

RD

S

ET

U

P

AF

TE

R T

HE

AR

IAN

E 5

ltlt

LA

UN

CH

gtgt

ltlt

FA

ILU

RE

gtgt

ON

JU

NE

4 (

RE

FT

EL

A

) T

HE

EX

PLO

SIO

N W

AS

DU

E T

O S

PE

CIF

ICA

TIO

N A

ND

DE

SIG

NE

RR

OR

S I

N

TH

E S

OF

TW

AR

E

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M T

HE

RE

PO

RT

FU

RT

HE

R S

TR

ES

SE

D T

HE

F

AC

T T

HA

T

TH

E

ALIG

NM

EN

T F

UN

CT

ION

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M W

HIC

H

SE

RV

ED

A

P

UR

PO

SE

O

NL

YB

EF

OR

E L

IFT

OF

F (B

UT

R

EM

AIN

ED

O

PE

RA

TIV

E

AF

TE

RW

AR

DS

) W

AS

N

OT

TA

KE

N I

NT

O A

CC

OU

NT

IN

T

HE

S

IMU

LA

TIO

NS

AN

D T

HA

T T

HE

EQ

UIP

ME

NT

AN

DS

YS

TE

M T

ES

TS

WE

RE

N

OT

S

UF

FIC

IEN

TL

Y R

EP

RE

SE

NT

AT

IVE

T

HE

SY

ST

EM

AR

CH

ITE

CT

UR

E W

AS

NO

T IM

PLIC

AT

ED

T

HE

R

EP

OR

T

RE

CO

MM

EN

DE

D E

XT

EN

SIV

ER

EV

IEW

OF

TH

E S

OF

TW

AR

E D

ES

IGN

AN

D T

ES

TIN

G P

RO

CE

DU

RE

S

TH

E C

OS

T O

F C

OR

RE

CT

IVE

M

EA

SU

RE

S I

S E

ST

IMA

TE

D A

T 2

TO

4P

ER

CE

NT

OF

TH

E G

LO

BA

L

CO

ST

O

F T

HE

P

RO

JE

CT

T

HA

T IS

U

SD

150 T

O 3

00

MIL

LIO

N

CO

RR

EC

TIV

E A

CT

ION

IS

E

XP

EC

TE

D T

O P

OS

TP

ON

E T

HE

NE

XT

LA

UN

CH

TO

MID

SE

ME

ST

ER

1997

E

SA

DO

ES

NO

T E

XC

LU

DE

TH

E P

OS

SIB

ILIT

Y T

HA

TT

HE

TH

IRD

AR

IAN

E 5

LA

UN

CH

IN

ITIA

LLY

P

LA

NN

ED

A

S A

C

OM

ME

RC

IAL L

AU

NC

H

MA

Y E

VE

NT

UA

LLY

B

E T

RE

AT

ED

A

S A

Q

UA

LIF

ICA

TIO

N LA

UN

CH

ltlt

Eu

rop

egt

gts

he

avy-

lift la

un

che

r pro

gra

mm

e w

ill c

on

tin

ue

de

spite

th

eJu

ne

4se

tba

ck

bu

t th

e m

arg

in fo

r err

or

ha

s n

ow

be

en

re

du

ce

d t

o z

ero

Th

e lo

ss

of th

e fir

st A

ria

ne

5 o

n it

s in

itia

l q

ualif

ica

tio

n flig

ht

on

Jun

e 4

ma

rks a

se

ve

re b

ut

by

no

me

an

s fa

tal s

etb

ack

fo

r lt

ltE

uro

pe

gtgt

sn

ew

he

avy-

lift

laun

che

r a

nd

for

Ari

an

esp

ace

s a

mb

itio

ns t

o c

on

solid

ate

its

do

min

an

t p

osi

tio

n o

n t

he

glo

bal co

mm

erc

ial la

un

ch m

ark

et

It

will

take

mo

re t

ha

n a

sin

gle

ltlt

lau

nch

gtgt

ltlt

failu

regt

gt t

o d

era

il th

e 1

5-y

ea

r $

82

bill

ion

pro

gra

mm

e

bu

t th

e m

arg

in f

or

err

or

ha

s h

en

ce

fort

h b

een

red

uce

d to

zero

Th

e 4 J

un

e 1

996

la

un

ch o

f th

e fi

rst

Eu

rop

ean

Sp

ace

Ag

en

cy (

ES

A)

Ari

an

e 5

sp

ace

lau

nch

veh

icle

carr

yin

g t

he f

ou

r E

uro

pean

ldquoC

lust

errdquo

sci

en

tifi

c sa

tell

ites

en

ded

in

fail

ure

ju

st

ov

er

36 s

eco

nd

s aft

er l

ift-

off

du

e t

o i

nh

eri

t so

ftw

are

des

ign

fla

w i

n t

he g

uid

an

ce s

yst

em

Th

e im

bed

ded

so

ftw

are

in

bo

th o

f th

e p

rim

ary

an

d b

ack

-up

id

enti

call

y-d

esi

gn

ed

inert

ial

gu

idan

ce p

latf

orm

s c

urr

en

tly

pro

ven

an

d u

sed

on

bo

ard

th

e A

rian

e 4

w

as

ill-

desi

gn

ed

fo

r th

e f

lig

ht

pro

file

of

the

Ari

an

e 5

I

nsu

ffic

ien

t re

qu

irem

en

ts a

nd

test

ing

of

the

Ari

an

e 5 a

vio

nic

s sy

stem

du

rin

g d

ev

elo

pm

en

t d

id n

ot

un

cov

er t

he d

esi

gn

fla

w

Th

e acc

elera

tio

n o

f th

e A

rian

e 5

is

mu

ch g

reate

r th

an

th

e A

rian

e 4

aft

er l

ift-

off

T

his

cau

sed

th

e i

ner

tial

gu

idan

ce p

latf

orm

em

bed

ded

so

ftw

are

w

hic

h w

as

des

ign

ed

aro

un

d

the

slo

wer

Ari

an

e 4

acc

ele

rati

on

pro

file

to

pro

vid

e n

um

eri

cal

va

lues

bey

on

d t

he

pro

gra

mm

ed

lim

its

of

the

flig

ht

com

pu

ter

wh

ich

th

en

sh

utd

ow

n b

oth

in

erti

al

pla

tfo

rms

Th

e p

latf

orm

s in

itia

ted

a d

iag

no

stic

ldquore

setrdquo

mo

de

tha

t fe

d i

nco

rrec

t v

alu

es t

o t

he

flig

ht

com

pu

ter

wh

ich

th

en c

om

man

ded

an

ex

cess

ive

pit

ch-o

ver

by

gim

ball

ing

th

e tw

o s

trap

-

on

bo

ost

er

no

zzle

s an

d t

hen

fir

st-s

tag

e m

ain

en

gin

e n

ozz

le T

his

rap

id p

itch

in

to a

hig

h

an

gle

of

att

ack

cau

sed

th

e f

air

ing

to

sep

ara

te f

rom

th

e v

eh

icle

du

e to

th

e h

igh

aero

dy

nam

ic l

oad

ing

T

his

cau

sed

th

e o

n-b

oard

self

-des

tru

ct s

yst

em

to

act

iva

te w

hic

h

was

late

r fo

llo

wed

by

a r

an

ge s

afe

ty-i

ssu

ed d

est

ruct

co

mm

an

d

Th

e f

lig

ht

com

pu

ter

pro

gra

m s

hu

tdo

wn

th

e in

ert

ial

pla

tfo

rms

as

pa

rt o

f a g

uid

an

ce r

e-a

lig

nm

ent

rou

tin

e

Th

is r

ou

tin

e i

s p

art

of

the

Ari

an

e 4

reali

gn

men

t ro

uti

ne d

esi

gn

ed

to

all

ow

qu

ick

up

da

tes

aft

er a

ho

ld l

ate

in

th

e c

ou

nt-

do

wn

T

his

ro

uti

ne i

s a

llo

wed

to

op

era

te u

p t

o 4

0 s

eco

nd

s

aft

er l

ift-

off

D

uri

ng

th

is p

eri

od

th

e A

ria

ne

5 t

rav

els

fart

her

do

wn

ra

ng

e th

an

th

e

Ari

an

e 4

an

d t

her

efo

re p

rod

uce

s a

la

rger

ho

rizo

nta

l v

elo

city

co

mp

on

en

t v

alu

e W

hen

this

valu

e r

each

ed

th

e fl

igh

t co

mp

ute

r p

rog

ram

lim

it (

ie n

um

eri

cal

ov

erfl

ow

) w

hic

h i

s

no

t re

ach

ed

by

th

e A

rian

e 4

in

th

e sa

me t

imefr

am

e t

he f

lig

ht

com

pu

ter

shu

tdo

wn

re

set

the

inert

ial

pla

tfo

rms

Du

rin

g d

evelo

pm

en

t o

f th

e av

ion

ics

for

the

Ari

an

e 5

th

e S

exta

nt

Av

ion

iqu

e r

ing

la

ser

gy

ro i

ner

tail

pla

tfo

rms

were

no

t te

sted

in

th

e l

oo

p a

s p

art

of

an

en

d-t

o-e

nd

sim

ula

tio

n

sin

ce t

hey

had

pre

vio

usl

y b

een

pro

ven

to

wo

rk o

n t

he A

rian

e 4

I

nst

ead

th

eir

in

pu

t

was

on

ly s

imu

late

d A

co

mp

lete

hard

ware

-in

-th

e-l

oo

p t

est

wo

uld

lik

ely

hav

e

un

cov

ere

d t

he

syst

em

ic f

ail

ure

of

thes

e A

rian

e-4

deri

ved

in

ert

ial

pla

tfo

rms

in t

he

Ari

an

e 5

fli

gh

t en

vir

on

men

t

As

a r

esu

lt o

f th

is f

ail

ure

E

SA

lo

st t

he

US

$50

0 m

illi

on

Clu

ster

sate

llit

e p

rog

ram

an

d

est

ima

tes

a 2

-4

in

crease

in

co

st o

f th

e U

S $

8 b

illi

on

Ari

an

e 5

pro

gra

m alo

ng

wit

h a

n

ap

pro

xim

ate

on

e-y

ear

dela

y i

n t

he A

rian

e 5

pro

gra

m i

n o

rder

to

per

form

mo

re

hard

wa

re s

imu

lati

on

test

s an

d d

esi

gn

a p

rov

e a

new

gu

idan

ce p

rog

ram

Fig

ure

13

Typ

ical

ana

lysi

s pr

oces

s us

ed b

y in

telli

genc

e an

alys

ts t

o se

arch

a d

ocum

ent

data

base

and

synt

hesi

ze r

esul

ts to

form

ulat

e a

resp

onse

to a

n an

alys

is q

uery

Rep

rinte

d fr

om P

atte

rson

E S

R

oth

E M

amp

Woo

ds D

D P

redi

ctin

g vu

lner

abili

ty in

com

pute

r-su

ppor

ted

infe

rent

ial a

naly

sis

unde

rda

ta o

verlo

ad

Cog

nitio

n T

echn

olog

y amp

Wor

k 3

22

4ndash23

7 C

opyr

ight

200

1 w

ith k

ind

perm

issi

on o

fS

prin

ger

Sci

ence

and

Bus

ines

s M

edia


12


copy1999 Patterson


161

29


22

5


169

15


419

28


7

184


66

14


12

194


4

29




CTA METHODS




































































21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion














RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES



































































































































































































CHAPTER 1








1











































Ca

su

alty M

an

age

me

nt

Ba

lan

ce

of A

uth

ority

Ris

k-B

enefit B

ala

nce

Re

sou

rce

Ba

lance

Eco

nom

ic B

ala

nce

Resp

on

se P

lan

sO

pe

ratio

ns P

roce

sse

s

Co

ord

ina

tin

g a

nd I

mp

lem

en

ting

Me

dic

al O

pe

ratio

ns

Co

ord

ina

tin

g a

nd I

mp

lem

en

ting

She

lte

r O

pera

tio

ns

Fix

ed L

oca

tion

Re

so

urc

es

(Ho

sp

ita

ls 9

11

Ca

ll C

ente

rs

Am

bu

lan

ce D

isp

atc

h F

ire

Sta

tion

s P

olic

e S

tatio

ns)

Mo

bile

Re

so

urc

es

(Am

bu

lan

ce

s

Fire

Re

scu

e

Fir

e E

ngin

es P

olic

eV

eh

icle

s)

Hu

ma

n R

eso

urc

es

(Po

lice

E

MT

F

ire

)

Sta

te o

f b

uild

ing

s o

pe

ratin

gcon

ditio

ns lo

ca

tio

n

ca

pa

city ca

pacity v

s lo

ad

(eg

fo

r a

ho

spita

l)

Nu

mb

er

co

nd

itio

n

op

era

tion

al sta

te

locatio

nof m

obile

reso

urc

es

pers

on

ne

l

Fu

nctio

na

l P

urp

ose

Ge

ne

ral P

roce

sse

s

Ab

str

act F

un

ction

Ph

ysic

al F

un

ctio

n

Ph

ysic

al F

orm

Sys

tem

Sub

-Sys

tem

Fig

ure

12

Por

tion

of a

n ab

stra

ctio

n hi

erar

chy

wor

k do

mai

n m

odel

for

an

emer

genc

y re

spon

sesy

stem

B

ased

on

wor

k fr

om B

isan

tz

Littl

e a

nd R

ogov

a (2

004)

7



























11

Qu

ery

(k

ey

wo

rds

refi

nem

en

t)

Sele

ct ldquo

key

rdquo d

ocu

men

ts

Co

rro

bo

rate

in

form

ati

on

(o

r

reso

lve d

iscr

ep

an

cies

in

info

rmati

on

) an

d f

ill

in g

ap

s

wit

h s

up

po

rt d

ocu

men

ts

Sy

nth

esi

ze i

nfo

rmati

on

to

con

stru

ct a

co

here

nt

sto

ry

Bro

wse

by

tit

le a

nd

o

r d

ate

19

960

80

8

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

19

960

50

0

AR

IAN

E 5

A

N A

LL

NE

W L

AU

NC

HE

R F

OR

TH

E G

LO

BA

L M

AR

KE

T

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

19

960

61

0

AR

IAN

E 5

EX

PL

OD

ES

IN

TH

E S

KY

OV

ER

KO

UR

OU

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

eu

rop

e 1

996

(eu

rop

e 1

996)

amp (

lau

nch

fail

ure

)

(eu

rop

e 1

996)

amp (

(la

un

ch f

ail

ure

)

2)

378

140

22

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

Da

te

Au

gu

st 8

1996

So

urc

e

FB

IS r

ep

ort

AC

CO

RD

ING

T

O T

HE

R

EP

OR

T R

EC

EN

TL

Y S

UB

MIT

TE

D B

Y T

HE

IND

EP

EN

DE

NT

IN

QU

IRY

B

OA

RD

S

ET

U

P

AF

TE

R T

HE

AR

IAN

E 5

ltlt

LA

UN

CH

gtgt

ltlt

FA

ILU

RE

gtgt

ON

JU

NE

4 (

RE

FT

EL

A

) T

HE

EX

PLO

SIO

N W

AS

DU

E T

O S

PE

CIF

ICA

TIO

N A

ND

DE

SIG

NE

RR

OR

S I

N

TH

E S

OF

TW

AR

E

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M T

HE

RE

PO

RT

FU

RT

HE

R S

TR

ES

SE

D T

HE

F

AC

T T

HA

T

TH

E

ALIG

NM

EN

T F

UN

CT

ION

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M W

HIC

H

SE

RV

ED

A

P

UR

PO

SE

O

NL

YB

EF

OR

E L

IFT

OF

F (B

UT

R

EM

AIN

ED

O

PE

RA

TIV

E

AF

TE

RW

AR

DS

) W

AS

N

OT

TA

KE

N I

NT

O A

CC

OU

NT

IN

T

HE

S

IMU

LA

TIO

NS

AN

D T

HA

T T

HE

EQ

UIP

ME

NT

AN

DS

YS

TE

M T

ES

TS

WE

RE

N

OT

S

UF

FIC

IEN

TL

Y R

EP

RE

SE

NT

AT

IVE

T

HE

SY

ST

EM

AR

CH

ITE

CT

UR

E W

AS

NO

T IM

PLIC

AT

ED

T

HE

R

EP

OR

T

RE

CO

MM

EN

DE

D E

XT

EN

SIV

ER

EV

IEW

OF

TH

E S

OF

TW

AR

E D

ES

IGN

AN

D T

ES

TIN

G P

RO

CE

DU

RE

S

TH

E C

OS

T O

F C

OR

RE

CT

IVE

M

EA

SU

RE

S I

S E

ST

IMA

TE

D A

T 2

TO

4P

ER

CE

NT

OF

TH

E G

LO

BA

L

CO

ST

O

F T

HE

P

RO

JE

CT

T

HA

T IS

U

SD

150 T

O 3

00

MIL

LIO

N

CO

RR

EC

TIV

E A

CT

ION

IS

E

XP

EC

TE

D T

O P

OS

TP

ON

E T

HE

NE

XT

LA

UN

CH

TO

MID

SE

ME

ST

ER

1997

E

SA

DO

ES

NO

T E

XC

LU

DE

TH

E P

OS

SIB

ILIT

Y T

HA

TT

HE

TH

IRD

AR

IAN

E 5

LA

UN

CH

IN

ITIA

LLY

P

LA

NN

ED

A

S A

C

OM

ME

RC

IAL L

AU

NC

H

MA

Y E

VE

NT

UA

LLY

B

E T

RE

AT

ED

A

S A

Q

UA

LIF

ICA

TIO

N LA

UN

CH

ltlt

Eu

rop

egt

gts

he

avy-

lift la

un

che

r pro

gra

mm

e w

ill c

on

tin

ue

de

spite

th

eJu

ne

4se

tba

ck

bu

t th

e m

arg

in fo

r err

or

ha

s n

ow

be

en

re

du

ce

d t

o z

ero

Th

e lo

ss

of th

e fir

st A

ria

ne

5 o

n it

s in

itia

l q

ualif

ica

tio

n flig

ht

on

Jun

e 4

ma

rks a

se

ve

re b

ut

by

no

me

an

s fa

tal s

etb

ack

fo

r lt

ltE

uro

pe

gtgt

sn

ew

he

avy-

lift

laun

che

r a

nd

for

Ari

an

esp

ace

s a

mb

itio

ns t

o c

on

solid

ate

its

do

min

an

t p

osi

tio

n o

n t

he

glo

bal co

mm

erc

ial la

un

ch m

ark

et

It

will

take

mo

re t

ha

n a

sin

gle

ltlt

lau

nch

gtgt

ltlt

failu

regt

gt t

o d

era

il th

e 1

5-y

ea

r $

82

bill

ion

pro

gra

mm

e

bu

t th

e m

arg

in f

or

err

or

ha

s h

en

ce

fort

h b

een

red

uce

d to

zero

Th

e 4 J

un

e 1

996

la

un

ch o

f th

e fi

rst

Eu

rop

ean

Sp

ace

Ag

en

cy (

ES

A)

Ari

an

e 5

sp

ace

lau

nch

veh

icle

carr

yin

g t

he f

ou

r E

uro

pean

ldquoC

lust

errdquo

sci

en

tifi

c sa

tell

ites

en

ded

in

fail

ure

ju

st

ov

er

36 s

eco

nd

s aft

er l

ift-

off

du

e t

o i

nh

eri

t so

ftw

are

des

ign

fla

w i

n t

he g

uid

an

ce s

yst

em

Th

e im

bed

ded

so

ftw

are

in

bo

th o

f th

e p

rim

ary

an

d b

ack

-up

id

enti

call

y-d

esi

gn

ed

inert

ial

gu

idan

ce p

latf

orm

s c

urr

en

tly

pro

ven

an

d u

sed

on

bo

ard

th

e A

rian

e 4

w

as

ill-

desi

gn

ed

fo

r th

e f

lig

ht

pro

file

of

the

Ari

an

e 5

I

nsu

ffic

ien

t re

qu

irem

en

ts a

nd

test

ing

of

the

Ari

an

e 5 a

vio

nic

s sy

stem

du

rin

g d

ev

elo

pm

en

t d

id n

ot

un

cov

er t

he d

esi

gn

fla

w

Th

e acc

elera

tio

n o

f th

e A

rian

e 5

is

mu

ch g

reate

r th

an

th

e A

rian

e 4

aft

er l

ift-

off

T

his

cau

sed

th

e i

ner

tial

gu

idan

ce p

latf

orm

em

bed

ded

so

ftw

are

w

hic

h w

as

des

ign

ed

aro

un

d

the

slo

wer

Ari

an

e 4

acc

ele

rati

on

pro

file

to

pro

vid

e n

um

eri

cal

va

lues

bey

on

d t

he

pro

gra

mm

ed

lim

its

of

the

flig

ht

com

pu

ter

wh

ich

th

en

sh

utd

ow

n b

oth

in

erti

al

pla

tfo

rms

Th

e p

latf

orm

s in

itia

ted

a d

iag

no

stic

ldquore

setrdquo

mo

de

tha

t fe

d i

nco

rrec

t v

alu

es t

o t

he

flig

ht

com

pu

ter

wh

ich

th

en c

om

man

ded

an

ex

cess

ive

pit

ch-o

ver

by

gim

ball

ing

th

e tw

o s

trap

-

on

bo

ost

er

no

zzle

s an

d t

hen

fir

st-s

tag

e m

ain

en

gin

e n

ozz

le T

his

rap

id p

itch

in

to a

hig

h

an

gle

of

att

ack

cau

sed

th

e f

air

ing

to

sep

ara

te f

rom

th

e v

eh

icle

du

e to

th

e h

igh

aero

dy

nam

ic l

oad

ing

T

his

cau

sed

th

e o

n-b

oard

self

-des

tru

ct s

yst

em

to

act

iva

te w

hic

h

was

late

r fo

llo

wed

by

a r

an

ge s

afe

ty-i

ssu

ed d

est

ruct

co

mm

an

d

Th

e f

lig

ht

com

pu

ter

pro

gra

m s

hu

tdo

wn

th

e in

ert

ial

pla

tfo

rms

as

pa

rt o

f a g

uid

an

ce r

e-a

lig

nm

ent

rou

tin

e

Th

is r

ou

tin

e i

s p

art

of

the

Ari

an

e 4

reali

gn

men

t ro

uti

ne d

esi

gn

ed

to

all

ow

qu

ick

up

da

tes

aft

er a

ho

ld l

ate

in

th

e c

ou

nt-

do

wn

T

his

ro

uti

ne i

s a

llo

wed

to

op

era

te u

p t

o 4

0 s

eco

nd

s

aft

er l

ift-

off

D

uri

ng

th

is p

eri

od

th

e A

ria

ne

5 t

rav

els

fart

her

do

wn

ra

ng

e th

an

th

e

Ari

an

e 4

an

d t

her

efo

re p

rod

uce

s a

la

rger

ho

rizo

nta

l v

elo

city

co

mp

on

en

t v

alu

e W

hen

this

valu

e r

each

ed

th

e fl

igh

t co

mp

ute

r p

rog

ram

lim

it (

ie n

um

eri

cal

ov

erfl

ow

) w

hic

h i

s

no

t re

ach

ed

by

th

e A

rian

e 4

in

th

e sa

me t

imefr

am

e t

he f

lig

ht

com

pu

ter

shu

tdo

wn

re

set

the

inert

ial

pla

tfo

rms

Du

rin

g d

evelo

pm

en

t o

f th

e av

ion

ics

for

the

Ari

an

e 5

th

e S

exta

nt

Av

ion

iqu

e r

ing

la

ser

gy

ro i

ner

tail

pla

tfo

rms

were

no

t te

sted

in

th

e l

oo

p a

s p

art

of

an

en

d-t

o-e

nd

sim

ula

tio

n

sin

ce t

hey

had

pre

vio

usl

y b

een

pro

ven

to

wo

rk o

n t

he A

rian

e 4

I

nst

ead

th

eir

in

pu

t

was

on

ly s

imu

late

d A

co

mp

lete

hard

ware

-in

-th

e-l

oo

p t

est

wo

uld

lik

ely

hav

e

un

cov

ere

d t

he

syst

em

ic f

ail

ure

of

thes

e A

rian

e-4

deri

ved

in

ert

ial

pla

tfo

rms

in t

he

Ari

an

e 5

fli

gh

t en

vir

on

men

t

As

a r

esu

lt o

f th

is f

ail

ure

E

SA

lo

st t

he

US

$50

0 m

illi

on

Clu

ster

sate

llit

e p

rog

ram

an

d

est

ima

tes

a 2

-4

in

crease

in

co

st o

f th

e U

S $

8 b

illi

on

Ari

an

e 5

pro

gra

m alo

ng

wit

h a

n

ap

pro

xim

ate

on

e-y

ear

dela

y i

n t

he A

rian

e 5

pro

gra

m i

n o

rder

to

per

form

mo

re

hard

wa

re s

imu

lati

on

test

s an

d d

esi

gn

a p

rov

e a

new

gu

idan

ce p

rog

ram

Fig

ure

13

Typ

ical

ana

lysi

s pr

oces

s us

ed b

y in

telli

genc

e an

alys

ts t

o se

arch

a d

ocum

ent

data

base

and

synt

hesi

ze r

esul

ts to

form

ulat

e a

resp

onse

to a

n an

alys

is q

uery

Rep

rinte

d fr

om P

atte

rson

E S

R

oth

E M

amp

Woo

ds D

D P

redi

ctin

g vu

lner

abili

ty in

com

pute

r-su

ppor

ted

infe

rent

ial a

naly

sis

unde

rda

ta o

verlo

ad

Cog

nitio

n T

echn

olog

y amp

Wor

k 3

22

4ndash23

7 C

opyr

ight

200

1 w

ith k

ind

perm

issi

on o

fS

prin

ger

Sci

ence

and

Bus

ines

s M

edia


12


copy1999 Patterson


161

29


22

5


169

15


419

28


7

184


66

14


12

194


4

29




CTA METHODS




































































21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion














RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES












































































































































































































































Ca

su

alty M

an

age

me

nt

Ba

lan

ce

of A

uth

ority

Ris

k-B

enefit B

ala

nce

Re

sou

rce

Ba

lance

Eco

nom

ic B

ala

nce

Resp

on

se P

lan

sO

pe

ratio

ns P

roce

sse

s

Co

ord

ina

tin

g a

nd I

mp

lem

en

ting

Me

dic

al O

pe

ratio

ns

Co

ord

ina

tin

g a

nd I

mp

lem

en

ting

She

lte

r O

pera

tio

ns

Fix

ed L

oca

tion

Re

so

urc

es

(Ho

sp

ita

ls 9

11

Ca

ll C

ente

rs

Am

bu

lan

ce D

isp

atc

h F

ire

Sta

tion

s P

olic

e S

tatio

ns)

Mo

bile

Re

so

urc

es

(Am

bu

lan

ce

s

Fire

Re

scu

e

Fir

e E

ngin

es P

olic

eV

eh

icle

s)

Hu

ma

n R

eso

urc

es

(Po

lice

E

MT

F

ire

)

Sta

te o

f b

uild

ing

s o

pe

ratin

gcon

ditio

ns lo

ca

tio

n

ca

pa

city ca

pacity v

s lo

ad

(eg

fo

r a

ho

spita

l)

Nu

mb

er

co

nd

itio

n

op

era

tion

al sta

te

locatio

nof m

obile

reso

urc

es

pers

on

ne

l

Fu

nctio

na

l P

urp

ose

Ge

ne

ral P

roce

sse

s

Ab

str

act F

un

ction

Ph

ysic

al F

un

ctio

n

Ph

ysic

al F

orm

Sys

tem

Sub

-Sys

tem

Fig

ure

12

Por

tion

of a

n ab

stra

ctio

n hi

erar

chy

wor

k do

mai

n m

odel

for

an

emer

genc

y re

spon

sesy

stem

B

ased

on

wor

k fr

om B

isan

tz

Littl

e a

nd R

ogov

a (2

004)

7



























11

Qu

ery

(k

ey

wo

rds

refi

nem

en

t)

Sele

ct ldquo

key

rdquo d

ocu

men

ts

Co

rro

bo

rate

in

form

ati

on

(o

r

reso

lve d

iscr

ep

an

cies

in

info

rmati

on

) an

d f

ill

in g

ap

s

wit

h s

up

po

rt d

ocu

men

ts

Sy

nth

esi

ze i

nfo

rmati

on

to

con

stru

ct a

co

here

nt

sto

ry

Bro

wse

by

tit

le a

nd

o

r d

ate

19

960

80

8

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

19

960

50

0

AR

IAN

E 5

A

N A

LL

NE

W L

AU

NC

HE

R F

OR

TH

E G

LO

BA

L M

AR

KE

T

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

19

960

61

0

AR

IAN

E 5

EX

PL

OD

ES

IN

TH

E S

KY

OV

ER

KO

UR

OU

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

eu

rop

e 1

996

(eu

rop

e 1

996)

amp (

lau

nch

fail

ure

)

(eu

rop

e 1

996)

amp (

(la

un

ch f

ail

ure

)

2)

378

140

22

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

Da

te

Au

gu

st 8

1996

So

urc

e

FB

IS r

ep

ort

AC

CO

RD

ING

T

O T

HE

R

EP

OR

T R

EC

EN

TL

Y S

UB

MIT

TE

D B

Y T

HE

IND

EP

EN

DE

NT

IN

QU

IRY

B

OA

RD

S

ET

U

P

AF

TE

R T

HE

AR

IAN

E 5

ltlt

LA

UN

CH

gtgt

ltlt

FA

ILU

RE

gtgt

ON

JU

NE

4 (

RE

FT

EL

A

) T

HE

EX

PLO

SIO

N W

AS

DU

E T

O S

PE

CIF

ICA

TIO

N A

ND

DE

SIG

NE

RR

OR

S I

N

TH

E S

OF

TW

AR

E

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M T

HE

RE

PO

RT

FU

RT

HE

R S

TR

ES

SE

D T

HE

F

AC

T T

HA

T

TH

E

ALIG

NM

EN

T F

UN

CT

ION

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M W

HIC

H

SE

RV

ED

A

P

UR

PO

SE

O

NL

YB

EF

OR

E L

IFT

OF

F (B

UT

R

EM

AIN

ED

O

PE

RA

TIV

E

AF

TE

RW

AR

DS

) W

AS

N

OT

TA

KE

N I

NT

O A

CC

OU

NT

IN

T

HE

S

IMU

LA

TIO

NS

AN

D T

HA

T T

HE

EQ

UIP

ME

NT

AN

DS

YS

TE

M T

ES

TS

WE

RE

N

OT

S

UF

FIC

IEN

TL

Y R

EP

RE

SE

NT

AT

IVE

T

HE

SY

ST

EM

AR

CH

ITE

CT

UR

E W

AS

NO

T IM

PLIC

AT

ED

T

HE

R

EP

OR

T

RE

CO

MM

EN

DE

D E

XT

EN

SIV

ER

EV

IEW

OF

TH

E S

OF

TW

AR

E D

ES

IGN

AN

D T

ES

TIN

G P

RO

CE

DU

RE

S

TH

E C

OS

T O

F C

OR

RE

CT

IVE

M

EA

SU

RE

S I

S E

ST

IMA

TE

D A

T 2

TO

4P

ER

CE

NT

OF

TH

E G

LO

BA

L

CO

ST

O

F T

HE

P

RO

JE

CT

T

HA

T IS

U

SD

150 T

O 3

00

MIL

LIO

N

CO

RR

EC

TIV

E A

CT

ION

IS

E

XP

EC

TE

D T

O P

OS

TP

ON

E T

HE

NE

XT

LA

UN

CH

TO

MID

SE

ME

ST

ER

1997

E

SA

DO

ES

NO

T E

XC

LU

DE

TH

E P

OS

SIB

ILIT

Y T

HA

TT

HE

TH

IRD

AR

IAN

E 5

LA

UN

CH

IN

ITIA

LLY

P

LA

NN

ED

A

S A

C

OM

ME

RC

IAL L

AU

NC

H

MA

Y E

VE

NT

UA

LLY

B

E T

RE

AT

ED

A

S A

Q

UA

LIF

ICA

TIO

N LA

UN

CH

ltlt

Eu

rop

egt

gts

he

avy-

lift la

un

che

r pro

gra

mm

e w

ill c

on

tin

ue

de

spite

th

eJu

ne

4se

tba

ck

bu

t th

e m

arg

in fo

r err

or

ha

s n

ow

be

en

re

du

ce

d t

o z

ero

Th

e lo

ss

of th

e fir

st A

ria

ne

5 o

n it

s in

itia

l q

ualif

ica

tio

n flig

ht

on

Jun

e 4

ma

rks a

se

ve

re b

ut

by

no

me

an

s fa

tal s

etb

ack

fo

r lt

ltE

uro

pe

gtgt

sn

ew

he

avy-

lift

laun

che

r a

nd

for

Ari

an

esp

ace

s a

mb

itio

ns t

o c

on

solid

ate

its

do

min

an

t p

osi

tio

n o

n t

he

glo

bal co

mm

erc

ial la

un

ch m

ark

et

It

will

take

mo

re t

ha

n a

sin

gle

ltlt

lau

nch

gtgt

ltlt

failu

regt

gt t

o d

era

il th

e 1

5-y

ea

r $

82

bill

ion

pro

gra

mm

e

bu

t th

e m

arg

in f

or

err

or

ha

s h

en

ce

fort

h b

een

red

uce

d to

zero

Th

e 4 J

un

e 1

996

la

un

ch o

f th

e fi

rst

Eu

rop

ean

Sp

ace

Ag

en

cy (

ES

A)

Ari

an

e 5

sp

ace

lau

nch

veh

icle

carr

yin

g t

he f

ou

r E

uro

pean

ldquoC

lust

errdquo

sci

en

tifi

c sa

tell

ites

en

ded

in

fail

ure

ju

st

ov

er

36 s

eco

nd

s aft

er l

ift-

off

du

e t

o i

nh

eri

t so

ftw

are

des

ign

fla

w i

n t

he g

uid

an

ce s

yst

em

Th

e im

bed

ded

so

ftw

are

in

bo

th o

f th

e p

rim

ary

an

d b

ack

-up

id

enti

call

y-d

esi

gn

ed

inert

ial

gu

idan

ce p

latf

orm

s c

urr

en

tly

pro

ven

an

d u

sed

on

bo

ard

th

e A

rian

e 4

w

as

ill-

desi

gn

ed

fo

r th

e f

lig

ht

pro

file

of

the

Ari

an

e 5

I

nsu

ffic

ien

t re

qu

irem

en

ts a

nd

test

ing

of

the

Ari

an

e 5 a

vio

nic

s sy

stem

du

rin

g d

ev

elo

pm

en

t d

id n

ot

un

cov

er t

he d

esi

gn

fla

w

Th

e acc

elera

tio

n o

f th

e A

rian

e 5

is

mu

ch g

reate

r th

an

th

e A

rian

e 4

aft

er l

ift-

off

T

his

cau

sed

th

e i

ner

tial

gu

idan

ce p

latf

orm

em

bed

ded

so

ftw

are

w

hic

h w

as

des

ign

ed

aro

un

d

the

slo

wer

Ari

an

e 4

acc

ele

rati

on

pro

file

to

pro

vid

e n

um

eri

cal

va

lues

bey

on

d t

he

pro

gra

mm

ed

lim

its

of

the

flig

ht

com

pu

ter

wh

ich

th

en

sh

utd

ow

n b

oth

in

erti

al

pla

tfo

rms

Th

e p

latf

orm

s in

itia

ted

a d

iag

no

stic

ldquore

setrdquo

mo

de

tha

t fe

d i

nco

rrec

t v

alu

es t

o t

he

flig

ht

com

pu

ter

wh

ich

th

en c

om

man

ded

an

ex

cess

ive

pit

ch-o

ver

by

gim

ball

ing

th

e tw

o s

trap

-

on

bo

ost

er

no

zzle

s an

d t

hen

fir

st-s

tag

e m

ain

en

gin

e n

ozz

le T

his

rap

id p

itch

in

to a

hig

h

an

gle

of

att

ack

cau

sed

th

e f

air

ing

to

sep

ara

te f

rom

th

e v

eh

icle

du

e to

th

e h

igh

aero

dy

nam

ic l

oad

ing

T

his

cau

sed

th

e o

n-b

oard

self

-des

tru

ct s

yst

em

to

act

iva

te w

hic

h

was

late

r fo

llo

wed

by

a r

an

ge s

afe

ty-i

ssu

ed d

est

ruct

co

mm

an

d

Th

e f

lig

ht

com

pu

ter

pro

gra

m s

hu

tdo

wn

th

e in

ert

ial

pla

tfo

rms

as

pa

rt o

f a g

uid

an

ce r

e-a

lig

nm

ent

rou

tin

e

Th

is r

ou

tin

e i

s p

art

of

the

Ari

an

e 4

reali

gn

men

t ro

uti

ne d

esi

gn

ed

to

all

ow

qu

ick

up

da

tes

aft

er a

ho

ld l

ate

in

th

e c

ou

nt-

do

wn

T

his

ro

uti

ne i

s a

llo

wed

to

op

era

te u

p t

o 4

0 s

eco

nd

s

aft

er l

ift-

off

D

uri

ng

th

is p

eri

od

th

e A

ria

ne

5 t

rav

els

fart

her

do

wn

ra

ng

e th

an

th

e

Ari

an

e 4

an

d t

her

efo

re p

rod

uce

s a

la

rger

ho

rizo

nta

l v

elo

city

co

mp

on

en

t v

alu

e W

hen

this

valu

e r

each

ed

th

e fl

igh

t co

mp

ute

r p

rog

ram

lim

it (

ie n

um

eri

cal

ov

erfl

ow

) w

hic

h i

s

no

t re

ach

ed

by

th

e A

rian

e 4

in

th

e sa

me t

imefr

am

e t

he f

lig

ht

com

pu

ter

shu

tdo

wn

re

set

the

inert

ial

pla

tfo

rms

Du

rin

g d

evelo

pm

en

t o

f th

e av

ion

ics

for

the

Ari

an

e 5

th

e S

exta

nt

Av

ion

iqu

e r

ing

la

ser

gy

ro i

ner

tail

pla

tfo

rms

were

no

t te

sted

in

th

e l

oo

p a

s p

art

of

an

en

d-t

o-e

nd

sim

ula

tio

n

sin

ce t

hey

had

pre

vio

usl

y b

een

pro

ven

to

wo

rk o

n t

he A

rian

e 4

I

nst

ead

th

eir

in

pu

t

was

on

ly s

imu

late

d A

co

mp

lete

hard

ware

-in

-th

e-l

oo

p t

est

wo

uld

lik

ely

hav

e

un

cov

ere

d t

he

syst

em

ic f

ail

ure

of

thes

e A

rian

e-4

deri

ved

in

ert

ial

pla

tfo

rms

in t

he

Ari

an

e 5

fli

gh

t en

vir

on

men

t

As

a r

esu

lt o

f th

is f

ail

ure

E

SA

lo

st t

he

US

$50

0 m

illi

on

Clu

ster

sate

llit

e p

rog

ram

an

d

est

ima

tes

a 2

-4

in

crease

in

co

st o

f th

e U

S $

8 b

illi

on

Ari

an

e 5

pro

gra

m alo

ng

wit

h a

n

ap

pro

xim

ate

on

e-y

ear

dela

y i

n t

he A

rian

e 5

pro

gra

m i

n o

rder

to

per

form

mo

re

hard

wa

re s

imu

lati

on

test

s an

d d

esi

gn

a p

rov

e a

new

gu

idan

ce p

rog

ram

Fig

ure

13

Typ

ical

ana

lysi

s pr

oces

s us

ed b

y in

telli

genc

e an

alys

ts t

o se

arch

a d

ocum

ent

data

base

and

synt

hesi

ze r

esul

ts to

form

ulat

e a

resp

onse

to a

n an

alys

is q

uery

Rep

rinte

d fr

om P

atte

rson

E S

R

oth

E M

amp

Woo

ds D

D P

redi

ctin

g vu

lner

abili

ty in

com

pute

r-su

ppor

ted

infe

rent

ial a

naly

sis

unde

rda

ta o

verlo

ad

Cog

nitio

n T

echn

olog

y amp

Wor

k 3

22

4ndash23

7 C

opyr

ight

200

1 w

ith k

ind

perm

issi

on o

fS

prin

ger

Sci

ence

and

Bus

ines

s M

edia


12


copy1999 Patterson


161

29


22

5


169

15


419

28


7

184


66

14


12

194


4

29




CTA METHODS




































































21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion














RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES



































































































































































































































Ca

su

alty M

an

age

me

nt

Ba

lan

ce

of A

uth

ority

Ris

k-B

enefit B

ala

nce

Re

sou

rce

Ba

lance

Eco

nom

ic B

ala

nce

Resp

on

se P

lan

sO

pe

ratio

ns P

roce

sse

s

Co

ord

ina

tin

g a

nd I

mp

lem

en

ting

Me

dic

al O

pe

ratio

ns

Co

ord

ina

tin

g a

nd I

mp

lem

en

ting

She

lte

r O

pera

tio

ns

Fix

ed L

oca

tion

Re

so

urc

es

(Ho

sp

ita

ls 9

11

Ca

ll C

ente

rs

Am

bu

lan

ce D

isp

atc

h F

ire

Sta

tion

s P

olic

e S

tatio

ns)

Mo

bile

Re

so

urc

es

(Am

bu

lan

ce

s

Fire

Re

scu

e

Fir

e E

ngin

es P

olic

eV

eh

icle

s)

Hu

ma

n R

eso

urc

es

(Po

lice

E

MT

F

ire

)

Sta

te o

f b

uild

ing

s o

pe

ratin

gcon

ditio

ns lo

ca

tio

n

ca

pa

city ca

pacity v

s lo

ad

(eg

fo

r a

ho

spita

l)

Nu

mb

er

co

nd

itio

n

op

era

tion

al sta

te

locatio

nof m

obile

reso

urc

es

pers

on

ne

l

Fu

nctio

na

l P

urp

ose

Ge

ne

ral P

roce

sse

s

Ab

str

act F

un

ction

Ph

ysic

al F

un

ctio

n

Ph

ysic

al F

orm

Sys

tem

Sub

-Sys

tem

Fig

ure

12

Por

tion

of a

n ab

stra

ctio

n hi

erar

chy

wor

k do

mai

n m

odel

for

an

emer

genc

y re

spon

sesy

stem

B

ased

on

wor

k fr

om B

isan

tz

Littl

e a

nd R

ogov

a (2

004)

7



























11

Qu

ery

(k

ey

wo

rds

refi

nem

en

t)

Sele

ct ldquo

key

rdquo d

ocu

men

ts

Co

rro

bo

rate

in

form

ati

on

(o

r

reso

lve d

iscr

ep

an

cies

in

info

rmati

on

) an

d f

ill

in g

ap

s

wit

h s

up

po

rt d

ocu

men

ts

Sy

nth

esi

ze i

nfo

rmati

on

to

con

stru

ct a

co

here

nt

sto

ry

Bro

wse

by

tit

le a

nd

o

r d

ate

19

960

80

8

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

19

960

50

0

AR

IAN

E 5

A

N A

LL

NE

W L

AU

NC

HE

R F

OR

TH

E G

LO

BA

L M

AR

KE

T

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

19

960

61

0

AR

IAN

E 5

EX

PL

OD

ES

IN

TH

E S

KY

OV

ER

KO

UR

OU

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

eu

rop

e 1

996

(eu

rop

e 1

996)

amp (

lau

nch

fail

ure

)

(eu

rop

e 1

996)

amp (

(la

un

ch f

ail

ure

)

2)

378

140

22

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

Da

te

Au

gu

st 8

1996

So

urc

e

FB

IS r

ep

ort

AC

CO

RD

ING

T

O T

HE

R

EP

OR

T R

EC

EN

TL

Y S

UB

MIT

TE

D B

Y T

HE

IND

EP

EN

DE

NT

IN

QU

IRY

B

OA

RD

S

ET

U

P

AF

TE

R T

HE

AR

IAN

E 5

ltlt

LA

UN

CH

gtgt

ltlt

FA

ILU

RE

gtgt

ON

JU

NE

4 (

RE

FT

EL

A

) T

HE

EX

PLO

SIO

N W

AS

DU

E T

O S

PE

CIF

ICA

TIO

N A

ND

DE

SIG

NE

RR

OR

S I

N

TH

E S

OF

TW

AR

E

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M T

HE

RE

PO

RT

FU

RT

HE

R S

TR

ES

SE

D T

HE

F

AC

T T

HA

T

TH

E

ALIG

NM

EN

T F

UN

CT

ION

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M W

HIC

H

SE

RV

ED

A

P

UR

PO

SE

O

NL

YB

EF

OR

E L

IFT

OF

F (B

UT

R

EM

AIN

ED

O

PE

RA

TIV

E

AF

TE

RW

AR

DS

) W

AS

N

OT

TA

KE

N I

NT

O A

CC

OU

NT

IN

T

HE

S

IMU

LA

TIO

NS

AN

D T

HA

T T

HE

EQ

UIP

ME

NT

AN

DS

YS

TE

M T

ES

TS

WE

RE

N

OT

S

UF

FIC

IEN

TL

Y R

EP

RE

SE

NT

AT

IVE

T

HE

SY

ST

EM

AR

CH

ITE

CT

UR

E W

AS

NO

T IM

PLIC

AT

ED

T

HE

R

EP

OR

T

RE

CO

MM

EN

DE

D E

XT

EN

SIV

ER

EV

IEW

OF

TH

E S

OF

TW

AR

E D

ES

IGN

AN

D T

ES

TIN

G P

RO

CE

DU

RE

S

TH

E C

OS

T O

F C

OR

RE

CT

IVE

M

EA

SU

RE

S I

S E

ST

IMA

TE

D A

T 2

TO

4P

ER

CE

NT

OF

TH

E G

LO

BA

L

CO

ST

O

F T

HE

P

RO

JE

CT

T

HA

T IS

U

SD

150 T

O 3

00

MIL

LIO

N

CO

RR

EC

TIV

E A

CT

ION

IS

E

XP

EC

TE

D T

O P

OS

TP

ON

E T

HE

NE

XT

LA

UN

CH

TO

MID

SE

ME

ST

ER

1997

E

SA

DO

ES

NO

T E

XC

LU

DE

TH

E P

OS

SIB

ILIT

Y T

HA

TT

HE

TH

IRD

AR

IAN

E 5

LA

UN

CH

IN

ITIA

LLY

P

LA

NN

ED

A

S A

C

OM

ME

RC

IAL L

AU

NC

H

MA

Y E

VE

NT

UA

LLY

B

E T

RE

AT

ED

A

S A

Q

UA

LIF

ICA

TIO

N LA

UN

CH

ltlt

Eu

rop

egt

gts

he

avy-

lift la

un

che

r pro

gra

mm

e w

ill c

on

tin

ue

de

spite

th

eJu

ne

4se

tba

ck

bu

t th

e m

arg

in fo

r err

or

ha

s n

ow

be

en

re

du

ce

d t

o z

ero

Th

e lo

ss

of th

e fir

st A

ria

ne

5 o

n it

s in

itia

l q

ualif

ica

tio

n flig

ht

on

Jun

e 4

ma

rks a

se

ve

re b

ut

by

no

me

an

s fa

tal s

etb

ack

fo

r lt

ltE

uro

pe

gtgt

sn

ew

he

avy-

lift

laun

che

r a

nd

for

Ari

an

esp

ace

s a

mb

itio

ns t

o c

on

solid

ate

its

do

min

an

t p

osi

tio

n o

n t

he

glo

bal co

mm

erc

ial la

un

ch m

ark

et

It

will

take

mo

re t

ha

n a

sin

gle

ltlt

lau

nch

gtgt

ltlt

failu

regt

gt t

o d

era

il th

e 1

5-y

ea

r $

82

bill

ion

pro

gra

mm

e

bu

t th

e m

arg

in f

or

err

or

ha

s h

en

ce

fort

h b

een

red

uce

d to

zero

Th

e 4 J

un

e 1

996

la

un

ch o

f th

e fi

rst

Eu

rop

ean

Sp

ace

Ag

en

cy (

ES

A)

Ari

an

e 5

sp

ace

lau

nch

veh

icle

carr

yin

g t

he f

ou

r E

uro

pean

ldquoC

lust

errdquo

sci

en

tifi

c sa

tell

ites

en

ded

in

fail

ure

ju

st

ov

er

36 s

eco

nd

s aft

er l

ift-

off

du

e t

o i

nh

eri

t so

ftw

are

des

ign

fla

w i

n t

he g

uid

an

ce s

yst

em

Th

e im

bed

ded

so

ftw

are

in

bo

th o

f th

e p

rim

ary

an

d b

ack

-up

id

enti

call

y-d

esi

gn

ed

inert

ial

gu

idan

ce p

latf

orm

s c

urr

en

tly

pro

ven

an

d u

sed

on

bo

ard

th

e A

rian

e 4

w

as

ill-

desi

gn

ed

fo

r th

e f

lig

ht

pro

file

of

the

Ari

an

e 5

I

nsu

ffic

ien

t re

qu

irem

en

ts a

nd

test

ing

of

the

Ari

an

e 5 a

vio

nic

s sy

stem

du

rin

g d

ev

elo

pm

en

t d

id n

ot

un

cov

er t

he d

esi

gn

fla

w

Th

e acc

elera

tio

n o

f th

e A

rian

e 5

is

mu

ch g

reate

r th

an

th

e A

rian

e 4

aft

er l

ift-

off

T

his

cau

sed

th

e i

ner

tial

gu

idan

ce p

latf

orm

em

bed

ded

so

ftw

are

w

hic

h w

as

des

ign

ed

aro

un

d

the

slo

wer

Ari

an

e 4

acc

ele

rati

on

pro

file

to

pro

vid

e n

um

eri

cal

va

lues

bey

on

d t

he

pro

gra

mm

ed

lim

its

of

the

flig

ht

com

pu

ter

wh

ich

th

en

sh

utd

ow

n b

oth

in

erti

al

pla

tfo

rms

Th

e p

latf

orm

s in

itia

ted

a d

iag

no

stic

ldquore

setrdquo

mo

de

tha

t fe

d i

nco

rrec

t v

alu

es t

o t

he

flig

ht

com

pu

ter

wh

ich

th

en c

om

man

ded

an

ex

cess

ive

pit

ch-o

ver

by

gim

ball

ing

th

e tw

o s

trap

-

on

bo

ost

er

no

zzle

s an

d t

hen

fir

st-s

tag

e m

ain

en

gin

e n

ozz

le T

his

rap

id p

itch

in

to a

hig

h

an

gle

of

att

ack

cau

sed

th

e f

air

ing

to

sep

ara

te f

rom

th

e v

eh

icle

du

e to

th

e h

igh

aero

dy

nam

ic l

oad

ing

T

his

cau

sed

th

e o

n-b

oard

self

-des

tru

ct s

yst

em

to

act

iva

te w

hic

h

was

late

r fo

llo

wed

by

a r

an

ge s

afe

ty-i

ssu

ed d

est

ruct

co

mm

an

d

Th

e f

lig

ht

com

pu

ter

pro

gra

m s

hu

tdo

wn

th

e in

ert

ial

pla

tfo

rms

as

pa

rt o

f a g

uid

an

ce r

e-a

lig

nm

ent

rou

tin

e

Th

is r

ou

tin

e i

s p

art

of

the

Ari

an

e 4

reali

gn

men

t ro

uti

ne d

esi

gn

ed

to

all

ow

qu

ick

up

da

tes

aft

er a

ho

ld l

ate

in

th

e c

ou

nt-

do

wn

T

his

ro

uti

ne i

s a

llo

wed

to

op

era

te u

p t

o 4

0 s

eco

nd

s

aft

er l

ift-

off

D

uri

ng

th

is p

eri

od

th

e A

ria

ne

5 t

rav

els

fart

her

do

wn

ra

ng

e th

an

th

e

Ari

an

e 4

an

d t

her

efo

re p

rod

uce

s a

la

rger

ho

rizo

nta

l v

elo

city

co

mp

on

en

t v

alu

e W

hen

this

valu

e r

each

ed

th

e fl

igh

t co

mp

ute

r p

rog

ram

lim

it (

ie n

um

eri

cal

ov

erfl

ow

) w

hic

h i

s

no

t re

ach

ed

by

th

e A

rian

e 4

in

th

e sa

me t

imefr

am

e t

he f

lig

ht

com

pu

ter

shu

tdo

wn

re

set

the

inert

ial

pla

tfo

rms

Du

rin

g d

evelo

pm

en

t o

f th

e av

ion

ics

for

the

Ari

an

e 5

th

e S

exta

nt

Av

ion

iqu

e r

ing

la

ser

gy

ro i

ner

tail

pla

tfo

rms

were

no

t te

sted

in

th

e l

oo

p a

s p

art

of

an

en

d-t

o-e

nd

sim

ula

tio

n

sin

ce t

hey

had

pre

vio

usl

y b

een

pro

ven

to

wo

rk o

n t

he A

rian

e 4

I

nst

ead

th

eir

in

pu

t

was

on

ly s

imu

late

d A

co

mp

lete

hard

ware

-in

-th

e-l

oo

p t

est

wo

uld

lik

ely

hav

e

un

cov

ere

d t

he

syst

em

ic f

ail

ure

of

thes

e A

rian

e-4

deri

ved

in

ert

ial

pla

tfo

rms

in t

he

Ari

an

e 5

fli

gh

t en

vir

on

men

t

As

a r

esu

lt o

f th

is f

ail

ure

E

SA

lo

st t

he

US

$50

0 m

illi

on

Clu

ster

sate

llit

e p

rog

ram

an

d

est

ima

tes

a 2

-4

in

crease

in

co

st o

f th

e U

S $

8 b

illi

on

Ari

an

e 5

pro

gra

m alo

ng

wit

h a

n

ap

pro

xim

ate

on

e-y

ear

dela

y i

n t

he A

rian

e 5

pro

gra

m i

n o

rder

to

per

form

mo

re

hard

wa

re s

imu

lati

on

test

s an

d d

esi

gn

a p

rov

e a

new

gu

idan

ce p

rog

ram

Fig

ure

13

Typ

ical

ana

lysi

s pr

oces

s us

ed b

y in

telli

genc

e an

alys

ts t

o se

arch

a d

ocum

ent

data

base

and

synt

hesi

ze r

esul

ts to

form

ulat

e a

resp

onse

to a

n an

alys

is q

uery

Rep

rinte

d fr

om P

atte

rson

E S

R

oth

E M

amp

Woo

ds D

D P

redi

ctin

g vu

lner

abili

ty in

com

pute

r-su

ppor

ted

infe

rent

ial a

naly

sis

unde

rda

ta o

verlo

ad

Cog

nitio

n T

echn

olog

y amp

Wor

k 3

22

4ndash23

7 C

opyr

ight

200

1 w

ith k

ind

perm

issi

on o

fS

prin

ger

Sci

ence

and

Bus

ines

s M

edia


12


copy1999 Patterson


161

29


22

5


169

15


419

28


7

184


66

14


12

194


4

29




CTA METHODS




































































21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion














RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES






















































































































































































































Ca

su

alty M

an

age

me

nt

Ba

lan

ce

of A

uth

ority

Ris

k-B

enefit B

ala

nce

Re

sou

rce

Ba

lance

Eco

nom

ic B

ala

nce

Resp

on

se P

lan

sO

pe

ratio

ns P

roce

sse

s

Co

ord

ina

tin

g a

nd I

mp

lem

en

ting

Me

dic

al O

pe

ratio

ns

Co

ord

ina

tin

g a

nd I

mp

lem

en

ting

She

lte

r O

pera

tio

ns

Fix

ed L

oca

tion

Re

so

urc

es

(Ho

sp

ita

ls 9

11

Ca

ll C

ente

rs

Am

bu

lan

ce D

isp

atc

h F

ire

Sta

tion

s P

olic

e S

tatio

ns)

Mo

bile

Re

so

urc

es

(Am

bu

lan

ce

s

Fire

Re

scu

e

Fir

e E

ngin

es P

olic

eV

eh

icle

s)

Hu

ma

n R

eso

urc

es

(Po

lice

E

MT

F

ire

)

Sta

te o

f b

uild

ing

s o

pe

ratin

gcon

ditio

ns lo

ca

tio

n

ca

pa

city ca

pacity v

s lo

ad

(eg

fo

r a

ho

spita

l)

Nu

mb

er

co

nd

itio

n

op

era

tion

al sta

te

locatio

nof m

obile

reso

urc

es

pers

on

ne

l

Fu

nctio

na

l P

urp

ose

Ge

ne

ral P

roce

sse

s

Ab

str

act F

un

ction

Ph

ysic

al F

un

ctio

n

Ph

ysic

al F

orm

Sys

tem

Sub

-Sys

tem

Fig

ure

12

Por

tion

of a

n ab

stra

ctio

n hi

erar

chy

wor

k do

mai

n m

odel

for

an

emer

genc

y re

spon

sesy

stem

B

ased

on

wor

k fr

om B

isan

tz

Littl

e a

nd R

ogov

a (2

004)

7



























11

Qu

ery

(k

ey

wo

rds

refi

nem

en

t)

Sele

ct ldquo

key

rdquo d

ocu

men

ts

Co

rro

bo

rate

in

form

ati

on

(o

r

reso

lve d

iscr

ep

an

cies

in

info

rmati

on

) an

d f

ill

in g

ap

s

wit

h s

up

po

rt d

ocu

men

ts

Sy

nth

esi

ze i

nfo

rmati

on

to

con

stru

ct a

co

here

nt

sto

ry

Bro

wse

by

tit

le a

nd

o

r d

ate

19

960

80

8

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

19

960

50

0

AR

IAN

E 5

A

N A

LL

NE

W L

AU

NC

HE

R F

OR

TH

E G

LO

BA

L M

AR

KE

T

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

19

960

61

0

AR

IAN

E 5

EX

PL

OD

ES

IN

TH

E S

KY

OV

ER

KO

UR

OU

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

eu

rop

e 1

996

(eu

rop

e 1

996)

amp (

lau

nch

fail

ure

)

(eu

rop

e 1

996)

amp (

(la

un

ch f

ail

ure

)

2)

378

140

22

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

Da

te

Au

gu

st 8

1996

So

urc

e

FB

IS r

ep

ort

AC

CO

RD

ING

T

O T

HE

R

EP

OR

T R

EC

EN

TL

Y S

UB

MIT

TE

D B

Y T

HE

IND

EP

EN

DE

NT

IN

QU

IRY

B

OA

RD

S

ET

U

P

AF

TE

R T

HE

AR

IAN

E 5

ltlt

LA

UN

CH

gtgt

ltlt

FA

ILU

RE

gtgt

ON

JU

NE

4 (

RE

FT

EL

A

) T

HE

EX

PLO

SIO

N W

AS

DU

E T

O S

PE

CIF

ICA

TIO

N A

ND

DE

SIG

NE

RR

OR

S I

N

TH

E S

OF

TW

AR

E

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M T

HE

RE

PO

RT

FU

RT

HE

R S

TR

ES

SE

D T

HE

F

AC

T T

HA

T

TH

E

ALIG

NM

EN

T F

UN

CT

ION

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M W

HIC

H

SE

RV

ED

A

P

UR

PO

SE

O

NL

YB

EF

OR

E L

IFT

OF

F (B

UT

R

EM

AIN

ED

O

PE

RA

TIV

E

AF

TE

RW

AR

DS

) W

AS

N

OT

TA

KE

N I

NT

O A

CC

OU

NT

IN

T

HE

S

IMU

LA

TIO

NS

AN

D T

HA

T T

HE

EQ

UIP

ME

NT

AN

DS

YS

TE

M T

ES

TS

WE

RE

N

OT

S

UF

FIC

IEN

TL

Y R

EP

RE

SE

NT

AT

IVE

T

HE

SY

ST

EM

AR

CH

ITE

CT

UR

E W

AS

NO

T IM

PLIC

AT

ED

T

HE

R

EP

OR

T

RE

CO

MM

EN

DE

D E

XT

EN

SIV

ER

EV

IEW

OF

TH

E S

OF

TW

AR

E D

ES

IGN

AN

D T

ES

TIN

G P

RO

CE

DU

RE

S

TH

E C

OS

T O

F C

OR

RE

CT

IVE

M

EA

SU

RE

S I

S E

ST

IMA

TE

D A

T 2

TO

4P

ER

CE

NT

OF

TH

E G

LO

BA

L

CO

ST

O

F T

HE

P

RO

JE

CT

T

HA

T IS

U

SD

150 T

O 3

00

MIL

LIO

N

CO

RR

EC

TIV

E A

CT

ION

IS

E

XP

EC

TE

D T

O P

OS

TP

ON

E T

HE

NE

XT

LA

UN

CH

TO

MID

SE

ME

ST

ER

1997

E

SA

DO

ES

NO

T E

XC

LU

DE

TH

E P

OS

SIB

ILIT

Y T

HA

TT

HE

TH

IRD

AR

IAN

E 5

LA

UN

CH

IN

ITIA

LLY

P

LA

NN

ED

A

S A

C

OM

ME

RC

IAL L

AU

NC

H

MA

Y E

VE

NT

UA

LLY

B

E T

RE

AT

ED

A

S A

Q

UA

LIF

ICA

TIO

N LA

UN

CH

ltlt

Eu

rop

egt

gts

he

avy-

lift la

un

che

r pro

gra

mm

e w

ill c

on

tin

ue

de

spite

th

eJu

ne

4se

tba

ck

bu

t th

e m

arg

in fo

r err

or

ha

s n

ow

be

en

re

du

ce

d t

o z

ero

Th

e lo

ss

of th

e fir

st A

ria

ne

5 o

n it

s in

itia

l q

ualif

ica

tio

n flig

ht

on

Jun

e 4

ma

rks a

se

ve

re b

ut

by

no

me

an

s fa

tal s

etb

ack

fo

r lt

ltE

uro

pe

gtgt

sn

ew

he

avy-

lift

laun

che

r a

nd

for

Ari

an

esp

ace

s a

mb

itio

ns t

o c

on

solid

ate

its

do

min

an

t p

osi

tio

n o

n t

he

glo

bal co

mm

erc

ial la

un

ch m

ark

et

It

will

take

mo

re t

ha

n a

sin

gle

ltlt

lau

nch

gtgt

ltlt

failu

regt

gt t

o d

era

il th

e 1

5-y

ea

r $

82

bill

ion

pro

gra

mm

e

bu

t th

e m

arg

in f

or

err

or

ha

s h

en

ce

fort

h b

een

red

uce

d to

zero

Th

e 4 J

un

e 1

996

la

un

ch o

f th

e fi

rst

Eu

rop

ean

Sp

ace

Ag

en

cy (

ES

A)

Ari

an

e 5

sp

ace

lau

nch

veh

icle

carr

yin

g t

he f

ou

r E

uro

pean

ldquoC

lust

errdquo

sci

en

tifi

c sa

tell

ites

en

ded

in

fail

ure

ju

st

ov

er

36 s

eco

nd

s aft

er l

ift-

off

du

e t

o i

nh

eri

t so

ftw

are

des

ign

fla

w i

n t

he g

uid

an

ce s

yst

em

Th

e im

bed

ded

so

ftw

are

in

bo

th o

f th

e p

rim

ary

an

d b

ack

-up

id

enti

call

y-d

esi

gn

ed

inert

ial

gu

idan

ce p

latf

orm

s c

urr

en

tly

pro

ven

an

d u

sed

on

bo

ard

th

e A

rian

e 4

w

as

ill-

desi

gn

ed

fo

r th

e f

lig

ht

pro

file

of

the

Ari

an

e 5

I

nsu

ffic

ien

t re

qu

irem

en

ts a

nd

test

ing

of

the

Ari

an

e 5 a

vio

nic

s sy

stem

du

rin

g d

ev

elo

pm

en

t d

id n

ot

un

cov

er t

he d

esi

gn

fla

w

Th

e acc

elera

tio

n o

f th

e A

rian

e 5

is

mu

ch g

reate

r th

an

th

e A

rian

e 4

aft

er l

ift-

off

T

his

cau

sed

th

e i

ner

tial

gu

idan

ce p

latf

orm

em

bed

ded

so

ftw

are

w

hic

h w

as

des

ign

ed

aro

un

d

the

slo

wer

Ari

an

e 4

acc

ele

rati

on

pro

file

to

pro

vid

e n

um

eri

cal

va

lues

bey

on

d t

he

pro

gra

mm

ed

lim

its

of

the

flig

ht

com

pu

ter

wh

ich

th

en

sh

utd

ow

n b

oth

in

erti

al

pla

tfo

rms

Th

e p

latf

orm

s in

itia

ted

a d

iag

no

stic

ldquore

setrdquo

mo

de

tha

t fe

d i

nco

rrec

t v

alu

es t

o t

he

flig

ht

com

pu

ter

wh

ich

th

en c

om

man

ded

an

ex

cess

ive

pit

ch-o

ver

by

gim

ball

ing

th

e tw

o s

trap

-

on

bo

ost

er

no

zzle

s an

d t

hen

fir

st-s

tag

e m

ain

en

gin

e n

ozz

le T

his

rap

id p

itch

in

to a

hig

h

an

gle

of

att

ack

cau

sed

th

e f

air

ing

to

sep

ara

te f

rom

th

e v

eh

icle

du

e to

th

e h

igh

aero

dy

nam

ic l

oad

ing

T

his

cau

sed

th

e o

n-b

oard

self

-des

tru

ct s

yst

em

to

act

iva

te w

hic

h

was

late

r fo

llo

wed

by

a r

an

ge s

afe

ty-i

ssu

ed d

est

ruct

co

mm

an

d

Th

e f

lig

ht

com

pu

ter

pro

gra

m s

hu

tdo

wn

th

e in

ert

ial

pla

tfo

rms

as

pa

rt o

f a g

uid

an

ce r

e-a

lig

nm

ent

rou

tin

e

Th

is r

ou

tin

e i

s p

art

of

the

Ari

an

e 4

reali

gn

men

t ro

uti

ne d

esi

gn

ed

to

all

ow

qu

ick

up

da

tes

aft

er a

ho

ld l

ate

in

th

e c

ou

nt-

do

wn

T

his

ro

uti

ne i

s a

llo

wed

to

op

era

te u

p t

o 4

0 s

eco

nd

s

aft

er l

ift-

off

D

uri

ng

th

is p

eri

od

th

e A

ria

ne

5 t

rav

els

fart

her

do

wn

ra

ng

e th

an

th

e

Ari

an

e 4

an

d t

her

efo

re p

rod

uce

s a

la

rger

ho

rizo

nta

l v

elo

city

co

mp

on

en

t v

alu

e W

hen

this

valu

e r

each

ed

th

e fl

igh

t co

mp

ute

r p

rog

ram

lim

it (

ie n

um

eri

cal

ov

erfl

ow

) w

hic

h i

s

no

t re

ach

ed

by

th

e A

rian

e 4

in

th

e sa

me t

imefr

am

e t

he f

lig

ht

com

pu

ter

shu

tdo

wn

re

set

the

inert

ial

pla

tfo

rms

Du

rin

g d

evelo

pm

en

t o

f th

e av

ion

ics

for

the

Ari

an

e 5

th

e S

exta

nt

Av

ion

iqu

e r

ing

la

ser

gy

ro i

ner

tail

pla

tfo

rms

were

no

t te

sted

in

th

e l

oo

p a

s p

art

of

an

en

d-t

o-e

nd

sim

ula

tio

n

sin

ce t

hey

had

pre

vio

usl

y b

een

pro

ven

to

wo

rk o

n t

he A

rian

e 4

I

nst

ead

th

eir

in

pu

t

was

on

ly s

imu

late

d A

co

mp

lete

hard

ware

-in

-th

e-l

oo

p t

est

wo

uld

lik

ely

hav

e

un

cov

ere

d t

he

syst

em

ic f

ail

ure

of

thes

e A

rian

e-4

deri

ved

in

ert

ial

pla

tfo

rms

in t

he

Ari

an

e 5

fli

gh

t en

vir

on

men

t

As

a r

esu

lt o

f th

is f

ail

ure

E

SA

lo

st t

he

US

$50

0 m

illi

on

Clu

ster

sate

llit

e p

rog

ram

an

d

est

ima

tes

a 2

-4

in

crease

in

co

st o

f th

e U

S $

8 b

illi

on

Ari

an

e 5

pro

gra

m alo

ng

wit

h a

n

ap

pro

xim

ate

on

e-y

ear

dela

y i

n t

he A

rian

e 5

pro

gra

m i

n o

rder

to

per

form

mo

re

hard

wa

re s

imu

lati

on

test

s an

d d

esi

gn

a p

rov

e a

new

gu

idan

ce p

rog

ram

Fig

ure

13

Typ

ical

ana

lysi

s pr

oces

s us

ed b

y in

telli

genc

e an

alys

ts t

o se

arch

a d

ocum

ent

data

base

and

synt

hesi

ze r

esul

ts to

form

ulat

e a

resp

onse

to a

n an

alys

is q

uery

Rep

rinte

d fr

om P

atte

rson

E S

R

oth

E M

amp

Woo

ds D

D P

redi

ctin

g vu

lner

abili

ty in

com

pute

r-su

ppor

ted

infe

rent

ial a

naly

sis

unde

rda

ta o

verlo

ad

Cog

nitio

n T

echn

olog

y amp

Wor

k 3

22

4ndash23

7 C

opyr

ight

200

1 w

ith k

ind

perm

issi

on o

fS

prin

ger

Sci

ence

and

Bus

ines

s M

edia


12


copy1999 Patterson


161

29


22

5


169

15


419

28


7

184


66

14


12

194


4

29




CTA METHODS




































































21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion














RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES


















































































































































































































Ca

su

alty M

an

age

me

nt

Ba

lan

ce

of A

uth

ority

Ris

k-B

enefit B

ala

nce

Re

sou

rce

Ba

lance

Eco

nom

ic B

ala

nce

Resp

on

se P

lan

sO

pe

ratio

ns P

roce

sse

s

Co

ord

ina

tin

g a

nd I

mp

lem

en

ting

Me

dic

al O

pe

ratio

ns

Co

ord

ina

tin

g a

nd I

mp

lem

en

ting

She

lte

r O

pera

tio

ns

Fix

ed L

oca

tion

Re

so

urc

es

(Ho

sp

ita

ls 9

11

Ca

ll C

ente

rs

Am

bu

lan

ce D

isp

atc

h F

ire

Sta

tion

s P

olic

e S

tatio

ns)

Mo

bile

Re

so

urc

es

(Am

bu

lan

ce

s

Fire

Re

scu

e

Fir

e E

ngin

es P

olic

eV

eh

icle

s)

Hu

ma

n R

eso

urc

es

(Po

lice

E

MT

F

ire

)

Sta

te o

f b

uild

ing

s o

pe

ratin

gcon

ditio

ns lo

ca

tio

n

ca

pa

city ca

pacity v

s lo

ad

(eg

fo

r a

ho

spita

l)

Nu

mb

er

co

nd

itio

n

op

era

tion

al sta

te

locatio

nof m

obile

reso

urc

es

pers

on

ne

l

Fu

nctio

na

l P

urp

ose

Ge

ne

ral P

roce

sse

s

Ab

str

act F

un

ction

Ph

ysic

al F

un

ctio

n

Ph

ysic

al F

orm

Sys

tem

Sub

-Sys

tem

Fig

ure

12

Por

tion

of a

n ab

stra

ctio

n hi

erar

chy

wor

k do

mai

n m

odel

for

an

emer

genc

y re

spon

sesy

stem

B

ased

on

wor

k fr

om B

isan

tz

Littl

e a

nd R

ogov

a (2

004)

7



























11

Qu

ery

(k

ey

wo

rds

refi

nem

en

t)

Sele

ct ldquo

key

rdquo d

ocu

men

ts

Co

rro

bo

rate

in

form

ati

on

(o

r

reso

lve d

iscr

ep

an

cies

in

info

rmati

on

) an

d f

ill

in g

ap

s

wit

h s

up

po

rt d

ocu

men

ts

Sy

nth

esi

ze i

nfo

rmati

on

to

con

stru

ct a

co

here

nt

sto

ry

Bro

wse

by

tit

le a

nd

o

r d

ate

19

960

80

8

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

19

960

50

0

AR

IAN

E 5

A

N A

LL

NE

W L

AU

NC

HE

R F

OR

TH

E G

LO

BA

L M

AR

KE

T

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

19

960

61

0

AR

IAN

E 5

EX

PL

OD

ES

IN

TH

E S

KY

OV

ER

KO

UR

OU

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

eu

rop

e 1

996

(eu

rop

e 1

996)

amp (

lau

nch

fail

ure

)

(eu

rop

e 1

996)

amp (

(la

un

ch f

ail

ure

)

2)

378

140

22

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

Da

te

Au

gu

st 8

1996

So

urc

e

FB

IS r

ep

ort

AC

CO

RD

ING

T

O T

HE

R

EP

OR

T R

EC

EN

TL

Y S

UB

MIT

TE

D B

Y T

HE

IND

EP

EN

DE

NT

IN

QU

IRY

B

OA

RD

S

ET

U

P

AF

TE

R T

HE

AR

IAN

E 5

ltlt

LA

UN

CH

gtgt

ltlt

FA

ILU

RE

gtgt

ON

JU

NE

4 (

RE

FT

EL

A

) T

HE

EX

PLO

SIO

N W

AS

DU

E T

O S

PE

CIF

ICA

TIO

N A

ND

DE

SIG

NE

RR

OR

S I

N

TH

E S

OF

TW

AR

E

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M T

HE

RE

PO

RT

FU

RT

HE

R S

TR

ES

SE

D T

HE

F

AC

T T

HA

T

TH

E

ALIG

NM

EN

T F

UN

CT

ION

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M W

HIC

H

SE

RV

ED

A

P

UR

PO

SE

O

NL

YB

EF

OR

E L

IFT

OF

F (B

UT

R

EM

AIN

ED

O

PE

RA

TIV

E

AF

TE

RW

AR

DS

) W

AS

N

OT

TA

KE

N I

NT

O A

CC

OU

NT

IN

T

HE

S

IMU

LA

TIO

NS

AN

D T

HA

T T

HE

EQ

UIP

ME

NT

AN

DS

YS

TE

M T

ES

TS

WE

RE

N

OT

S

UF

FIC

IEN

TL

Y R

EP

RE

SE

NT

AT

IVE

T

HE

SY

ST

EM

AR

CH

ITE

CT

UR

E W

AS

NO

T IM

PLIC

AT

ED

T

HE

R

EP

OR

T

RE

CO

MM

EN

DE

D E

XT

EN

SIV

ER

EV

IEW

OF

TH

E S

OF

TW

AR

E D

ES

IGN

AN

D T

ES

TIN

G P

RO

CE

DU

RE

S

TH

E C

OS

T O

F C

OR

RE

CT

IVE

M

EA

SU

RE

S I

S E

ST

IMA

TE

D A

T 2

TO

4P

ER

CE

NT

OF

TH

E G

LO

BA

L

CO

ST

O

F T

HE

P

RO

JE

CT

T

HA

T IS

U

SD

150 T

O 3

00

MIL

LIO

N

CO

RR

EC

TIV

E A

CT

ION

IS

E

XP

EC

TE

D T

O P

OS

TP

ON

E T

HE

NE

XT

LA

UN

CH

TO

MID

SE

ME

ST

ER

1997

E

SA

DO

ES

NO

T E

XC

LU

DE

TH

E P

OS

SIB

ILIT

Y T

HA

TT

HE

TH

IRD

AR

IAN

E 5

LA

UN

CH

IN

ITIA

LLY

P

LA

NN

ED

A

S A

C

OM

ME

RC

IAL L

AU

NC

H

MA

Y E

VE

NT

UA

LLY

B

E T

RE

AT

ED

A

S A

Q

UA

LIF

ICA

TIO

N LA

UN

CH

ltlt

Eu

rop

egt

gts

he

avy-

lift la

un

che

r pro

gra

mm

e w

ill c

on

tin

ue

de

spite

th

eJu

ne

4se

tba

ck

bu

t th

e m

arg

in fo

r err

or

ha

s n

ow

be

en

re

du

ce

d t

o z

ero

Th

e lo

ss

of th

e fir

st A

ria

ne

5 o

n it

s in

itia

l q

ualif

ica

tio

n flig

ht

on

Jun

e 4

ma

rks a

se

ve

re b

ut

by

no

me

an

s fa

tal s

etb

ack

fo

r lt

ltE

uro

pe

gtgt

sn

ew

he

avy-

lift

laun

che

r a

nd

for

Ari

an

esp

ace

s a

mb

itio

ns t

o c

on

solid

ate

its

do

min

an

t p

osi

tio

n o

n t

he

glo

bal co

mm

erc

ial la

un

ch m

ark

et

It

will

take

mo

re t

ha

n a

sin

gle

ltlt

lau

nch

gtgt

ltlt

failu

regt

gt t

o d

era

il th

e 1

5-y

ea

r $

82

bill

ion

pro

gra

mm

e

bu

t th

e m

arg

in f

or

err

or

ha

s h

en

ce

fort

h b

een

red

uce

d to

zero

Th

e 4 J

un

e 1

996

la

un

ch o

f th

e fi

rst

Eu

rop

ean

Sp

ace

Ag

en

cy (

ES

A)

Ari

an

e 5

sp

ace

lau

nch

veh

icle

carr

yin

g t

he f

ou

r E

uro

pean

ldquoC

lust

errdquo

sci

en

tifi

c sa

tell

ites

en

ded

in

fail

ure

ju

st

ov

er

36 s

eco

nd

s aft

er l

ift-

off

du

e t

o i

nh

eri

t so

ftw

are

des

ign

fla

w i

n t

he g

uid

an

ce s

yst

em

Th

e im

bed

ded

so

ftw

are

in

bo

th o

f th

e p

rim

ary

an

d b

ack

-up

id

enti

call

y-d

esi

gn

ed

inert

ial

gu

idan

ce p

latf

orm

s c

urr

en

tly

pro

ven

an

d u

sed

on

bo

ard

th

e A

rian

e 4

w

as

ill-

desi

gn

ed

fo

r th

e f

lig

ht

pro

file

of

the

Ari

an

e 5

I

nsu

ffic

ien

t re

qu

irem

en

ts a

nd

test

ing

of

the

Ari

an

e 5 a

vio

nic

s sy

stem

du

rin

g d

ev

elo

pm

en

t d

id n

ot

un

cov

er t

he d

esi

gn

fla

w

Th

e acc

elera

tio

n o

f th

e A

rian

e 5

is

mu

ch g

reate

r th

an

th

e A

rian

e 4

aft

er l

ift-

off

T

his

cau

sed

th

e i

ner

tial

gu

idan

ce p

latf

orm

em

bed

ded

so

ftw

are

w

hic

h w

as

des

ign

ed

aro

un

d

the

slo

wer

Ari

an

e 4

acc

ele

rati

on

pro

file

to

pro

vid

e n

um

eri

cal

va

lues

bey

on

d t

he

pro

gra

mm

ed

lim

its

of

the

flig

ht

com

pu

ter

wh

ich

th

en

sh

utd

ow

n b

oth

in

erti

al

pla

tfo

rms

Th

e p

latf

orm

s in

itia

ted

a d

iag

no

stic

ldquore

setrdquo

mo

de

tha

t fe

d i

nco

rrec

t v

alu

es t

o t

he

flig

ht

com

pu

ter

wh

ich

th

en c

om

man

ded

an

ex

cess

ive

pit

ch-o

ver

by

gim

ball

ing

th

e tw

o s

trap

-

on

bo

ost

er

no

zzle

s an

d t

hen

fir

st-s

tag

e m

ain

en

gin

e n

ozz

le T

his

rap

id p

itch

in

to a

hig

h

an

gle

of

att

ack

cau

sed

th

e f

air

ing

to

sep

ara

te f

rom

th

e v

eh

icle

du

e to

th

e h

igh

aero

dy

nam

ic l

oad

ing

T

his

cau

sed

th

e o

n-b

oard

self

-des

tru

ct s

yst

em

to

act

iva

te w

hic

h

was

late

r fo

llo

wed

by

a r

an

ge s

afe

ty-i

ssu

ed d

est

ruct

co

mm

an

d

Th

e f

lig

ht

com

pu

ter

pro

gra

m s

hu

tdo

wn

th

e in

ert

ial

pla

tfo

rms

as

pa

rt o

f a g

uid

an

ce r

e-a

lig

nm

ent

rou

tin

e

Th

is r

ou

tin

e i

s p

art

of

the

Ari

an

e 4

reali

gn

men

t ro

uti

ne d

esi

gn

ed

to

all

ow

qu

ick

up

da

tes

aft

er a

ho

ld l

ate

in

th

e c

ou

nt-

do

wn

T

his

ro

uti

ne i

s a

llo

wed

to

op

era

te u

p t

o 4

0 s

eco

nd

s

aft

er l

ift-

off

D

uri

ng

th

is p

eri

od

th

e A

ria

ne

5 t

rav

els

fart

her

do

wn

ra

ng

e th

an

th

e

Ari

an

e 4

an

d t

her

efo

re p

rod

uce

s a

la

rger

ho

rizo

nta

l v

elo

city

co

mp

on

en

t v

alu

e W

hen

this

valu

e r

each

ed

th

e fl

igh

t co

mp

ute

r p

rog

ram

lim

it (

ie n

um

eri

cal

ov

erfl

ow

) w

hic

h i

s

no

t re

ach

ed

by

th

e A

rian

e 4

in

th

e sa

me t

imefr

am

e t

he f

lig

ht

com

pu

ter

shu

tdo

wn

re

set

the

inert

ial

pla

tfo

rms

Du

rin

g d

evelo

pm

en

t o

f th

e av

ion

ics

for

the

Ari

an

e 5

th

e S

exta

nt

Av

ion

iqu

e r

ing

la

ser

gy

ro i

ner

tail

pla

tfo

rms

were

no

t te

sted

in

th

e l

oo

p a

s p

art

of

an

en

d-t

o-e

nd

sim

ula

tio

n

sin

ce t

hey

had

pre

vio

usl

y b

een

pro

ven

to

wo

rk o

n t

he A

rian

e 4

I

nst

ead

th

eir

in

pu

t

was

on

ly s

imu

late

d A

co

mp

lete

hard

ware

-in

-th

e-l

oo

p t

est

wo

uld

lik

ely

hav

e

un

cov

ere

d t

he

syst

em

ic f

ail

ure

of

thes

e A

rian

e-4

deri

ved

in

ert

ial

pla

tfo

rms

in t

he

Ari

an

e 5

fli

gh

t en

vir

on

men

t

As

a r

esu

lt o

f th

is f

ail

ure

E

SA

lo

st t

he

US

$50

0 m

illi

on

Clu

ster

sate

llit

e p

rog

ram

an

d

est

ima

tes

a 2

-4

in

crease

in

co

st o

f th

e U

S $

8 b

illi

on

Ari

an

e 5

pro

gra

m alo

ng

wit

h a

n

ap

pro

xim

ate

on

e-y

ear

dela

y i

n t

he A

rian

e 5

pro

gra

m i

n o

rder

to

per

form

mo

re

hard

wa

re s

imu

lati

on

test

s an

d d

esi

gn

a p

rov

e a

new

gu

idan

ce p

rog

ram

Fig

ure

13

Typ

ical

ana

lysi

s pr

oces

s us

ed b

y in

telli

genc

e an

alys

ts t

o se

arch

a d

ocum

ent

data

base

and

synt

hesi

ze r

esul

ts to

form

ulat

e a

resp

onse

to a

n an

alys

is q

uery

Rep

rinte

d fr

om P

atte

rson

E S

R

oth

E M

amp

Woo

ds D

D P

redi

ctin

g vu

lner

abili

ty in

com

pute

r-su

ppor

ted

infe

rent

ial a

naly

sis

unde

rda

ta o

verlo

ad

Cog

nitio

n T

echn

olog

y amp

Wor

k 3

22

4ndash23

7 C

opyr

ight

200

1 w

ith k

ind

perm

issi

on o

fS

prin

ger

Sci

ence

and

Bus

ines

s M

edia


12


copy1999 Patterson


161

29


22

5


169

15


419

28


7

184


66

14


12

194


4

29




CTA METHODS




































































21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion














RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES










































































































































































































Ca

su

alty M

an

age

me

nt

Ba

lan

ce

of A

uth

ority

Ris

k-B

enefit B

ala

nce

Re

sou

rce

Ba

lance

Eco

nom

ic B

ala

nce

Resp

on

se P

lan

sO

pe

ratio

ns P

roce

sse

s

Co

ord

ina

tin

g a

nd I

mp

lem

en

ting

Me

dic

al O

pe

ratio

ns

Co

ord

ina

tin

g a

nd I

mp

lem

en

ting

She

lte

r O

pera

tio

ns

Fix

ed L

oca

tion

Re

so

urc

es

(Ho

sp

ita

ls 9

11

Ca

ll C

ente

rs

Am

bu

lan

ce D

isp

atc

h F

ire

Sta

tion

s P

olic

e S

tatio

ns)

Mo

bile

Re

so

urc

es

(Am

bu

lan

ce

s

Fire

Re

scu

e

Fir

e E

ngin

es P

olic

eV

eh

icle

s)

Hu

ma

n R

eso

urc

es

(Po

lice

E

MT

F

ire

)

Sta

te o

f b

uild

ing

s o

pe

ratin

gcon

ditio

ns lo

ca

tio

n

ca

pa

city ca

pacity v

s lo

ad

(eg

fo

r a

ho

spita

l)

Nu

mb

er

co

nd

itio

n

op

era

tion

al sta

te

locatio

nof m

obile

reso

urc

es

pers

on

ne

l

Fu

nctio

na

l P

urp

ose

Ge

ne

ral P

roce

sse

s

Ab

str

act F

un

ction

Ph

ysic

al F

un

ctio

n

Ph

ysic

al F

orm

Sys

tem

Sub

-Sys

tem

Fig

ure

12

Por

tion

of a

n ab

stra

ctio

n hi

erar

chy

wor

k do

mai

n m

odel

for

an

emer

genc

y re

spon

sesy

stem

B

ased

on

wor

k fr

om B

isan

tz

Littl

e a

nd R

ogov

a (2

004)

7



























11

Qu

ery

(k

ey

wo

rds

refi

nem

en

t)

Sele

ct ldquo

key

rdquo d

ocu

men

ts

Co

rro

bo

rate

in

form

ati

on

(o

r

reso

lve d

iscr

ep

an

cies

in

info

rmati

on

) an

d f

ill

in g

ap

s

wit

h s

up

po

rt d

ocu

men

ts

Sy

nth

esi

ze i

nfo

rmati

on

to

con

stru

ct a

co

here

nt

sto

ry

Bro

wse

by

tit

le a

nd

o

r d

ate

19

960

80

8

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

19

960

50

0

AR

IAN

E 5

A

N A

LL

NE

W L

AU

NC

HE

R F

OR

TH

E G

LO

BA

L M

AR

KE

T

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

19

960

61

0

AR

IAN

E 5

EX

PL

OD

ES

IN

TH

E S

KY

OV

ER

KO

UR

OU

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

eu

rop

e 1

996

(eu

rop

e 1

996)

amp (

lau

nch

fail

ure

)

(eu

rop

e 1

996)

amp (

(la

un

ch f

ail

ure

)

2)

378

140

22

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

Da

te

Au

gu

st 8

1996

So

urc

e

FB

IS r

ep

ort

AC

CO

RD

ING

T

O T

HE

R

EP

OR

T R

EC

EN

TL

Y S

UB

MIT

TE

D B

Y T

HE

IND

EP

EN

DE

NT

IN

QU

IRY

B

OA

RD

S

ET

U

P

AF

TE

R T

HE

AR

IAN

E 5

ltlt

LA

UN

CH

gtgt

ltlt

FA

ILU

RE

gtgt

ON

JU

NE

4 (

RE

FT

EL

A

) T

HE

EX

PLO

SIO

N W

AS

DU

E T

O S

PE

CIF

ICA

TIO

N A

ND

DE

SIG

NE

RR

OR

S I

N

TH

E S

OF

TW

AR

E

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M T

HE

RE

PO

RT

FU

RT

HE

R S

TR

ES

SE

D T

HE

F

AC

T T

HA

T

TH

E

ALIG

NM

EN

T F

UN

CT

ION

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M W

HIC

H

SE

RV

ED

A

P

UR

PO

SE

O

NL

YB

EF

OR

E L

IFT

OF

F (B

UT

R

EM

AIN

ED

O

PE

RA

TIV

E

AF

TE

RW

AR

DS

) W

AS

N

OT

TA

KE

N I

NT

O A

CC

OU

NT

IN

T

HE

S

IMU

LA

TIO

NS

AN

D T

HA

T T

HE

EQ

UIP

ME

NT

AN

DS

YS

TE

M T

ES

TS

WE

RE

N

OT

S

UF

FIC

IEN

TL

Y R

EP

RE

SE

NT

AT

IVE

T

HE

SY

ST

EM

AR

CH

ITE

CT

UR

E W

AS

NO

T IM

PLIC

AT

ED

T

HE

R

EP

OR

T

RE

CO

MM

EN

DE

D E

XT

EN

SIV

ER

EV

IEW

OF

TH

E S

OF

TW

AR

E D

ES

IGN

AN

D T

ES

TIN

G P

RO

CE

DU

RE

S

TH

E C

OS

T O

F C

OR

RE

CT

IVE

M

EA

SU

RE

S I

S E

ST

IMA

TE

D A

T 2

TO

4P

ER

CE

NT

OF

TH

E G

LO

BA

L

CO

ST

O

F T

HE

P

RO

JE

CT

T

HA

T IS

U

SD

150 T

O 3

00

MIL

LIO

N

CO

RR

EC

TIV

E A

CT

ION

IS

E

XP

EC

TE

D T

O P

OS

TP

ON

E T

HE

NE

XT

LA

UN

CH

TO

MID

SE

ME

ST

ER

1997

E

SA

DO

ES

NO

T E

XC

LU

DE

TH

E P

OS

SIB

ILIT

Y T

HA

TT

HE

TH

IRD

AR

IAN

E 5

LA

UN

CH

IN

ITIA

LLY

P

LA

NN

ED

A

S A

C

OM

ME

RC

IAL L

AU

NC

H

MA

Y E

VE

NT

UA

LLY

B

E T

RE

AT

ED

A

S A

Q

UA

LIF

ICA

TIO

N LA

UN

CH

ltlt

Eu

rop

egt

gts

he

avy-

lift la

un

che

r pro

gra

mm

e w

ill c

on

tin

ue

de

spite

th

eJu

ne

4se

tba

ck

bu

t th

e m

arg

in fo

r err

or

ha

s n

ow

be

en

re

du

ce

d t

o z

ero

Th

e lo

ss

of th

e fir

st A

ria

ne

5 o

n it

s in

itia

l q

ualif

ica

tio

n flig

ht

on

Jun

e 4

ma

rks a

se

ve

re b

ut

by

no

me

an

s fa

tal s

etb

ack

fo

r lt

ltE

uro

pe

gtgt

sn

ew

he

avy-

lift

laun

che

r a

nd

for

Ari

an

esp

ace

s a

mb

itio

ns t

o c

on

solid

ate

its

do

min

an

t p

osi

tio

n o

n t

he

glo

bal co

mm

erc

ial la

un

ch m

ark

et

It

will

take

mo

re t

ha

n a

sin

gle

ltlt

lau

nch

gtgt

ltlt

failu

regt

gt t

o d

era

il th

e 1

5-y

ea

r $

82

bill

ion

pro

gra

mm

e

bu

t th

e m

arg

in f

or

err

or

ha

s h

en

ce

fort

h b

een

red

uce

d to

zero

Th

e 4 J

un

e 1

996

la

un

ch o

f th

e fi

rst

Eu

rop

ean

Sp

ace

Ag

en

cy (

ES

A)

Ari

an

e 5

sp

ace

lau

nch

veh

icle

carr

yin

g t

he f

ou

r E

uro

pean

ldquoC

lust

errdquo

sci

en

tifi

c sa

tell

ites

en

ded

in

fail

ure

ju

st

ov

er

36 s

eco

nd

s aft

er l

ift-

off

du

e t

o i

nh

eri

t so

ftw

are

des

ign

fla

w i

n t

he g

uid

an

ce s

yst

em

Th

e im

bed

ded

so

ftw

are

in

bo

th o

f th

e p

rim

ary

an

d b

ack

-up

id

enti

call

y-d

esi

gn

ed

inert

ial

gu

idan

ce p

latf

orm

s c

urr

en

tly

pro

ven

an

d u

sed

on

bo

ard

th

e A

rian

e 4

w

as

ill-

desi

gn

ed

fo

r th

e f

lig

ht

pro

file

of

the

Ari

an

e 5

I

nsu

ffic

ien

t re

qu

irem

en

ts a

nd

test

ing

of

the

Ari

an

e 5 a

vio

nic

s sy

stem

du

rin

g d

ev

elo

pm

en

t d

id n

ot

un

cov

er t

he d

esi

gn

fla

w

Th

e acc

elera

tio

n o

f th

e A

rian

e 5

is

mu

ch g

reate

r th

an

th

e A

rian

e 4

aft

er l

ift-

off

T

his

cau

sed

th

e i

ner

tial

gu

idan

ce p

latf

orm

em

bed

ded

so

ftw

are

w

hic

h w

as

des

ign

ed

aro

un

d

the

slo

wer

Ari

an

e 4

acc

ele

rati

on

pro

file

to

pro

vid

e n

um

eri

cal

va

lues

bey

on

d t

he

pro

gra

mm

ed

lim

its

of

the

flig

ht

com

pu

ter

wh

ich

th

en

sh

utd

ow

n b

oth

in

erti

al

pla

tfo

rms

Th

e p

latf

orm

s in

itia

ted

a d

iag

no

stic

ldquore

setrdquo

mo

de

tha

t fe

d i

nco

rrec

t v

alu

es t

o t

he

flig

ht

com

pu

ter

wh

ich

th

en c

om

man

ded

an

ex

cess

ive

pit

ch-o

ver

by

gim

ball

ing

th

e tw

o s

trap

-

on

bo

ost

er

no

zzle

s an

d t

hen

fir

st-s

tag

e m

ain

en

gin

e n

ozz

le T

his

rap

id p

itch

in

to a

hig

h

an

gle

of

att

ack

cau

sed

th

e f

air

ing

to

sep

ara

te f

rom

th

e v

eh

icle

du

e to

th

e h

igh

aero

dy

nam

ic l

oad

ing

T

his

cau

sed

th

e o

n-b

oard

self

-des

tru

ct s

yst

em

to

act

iva

te w

hic

h

was

late

r fo

llo

wed

by

a r

an

ge s

afe

ty-i

ssu

ed d

est

ruct

co

mm

an

d

Th

e f

lig

ht

com

pu

ter

pro

gra

m s

hu

tdo

wn

th

e in

ert

ial

pla

tfo

rms

as

pa

rt o

f a g

uid

an

ce r

e-a

lig

nm

ent

rou

tin

e

Th

is r

ou

tin

e i

s p

art

of

the

Ari

an

e 4

reali

gn

men

t ro

uti

ne d

esi

gn

ed

to

all

ow

qu

ick

up

da

tes

aft

er a

ho

ld l

ate

in

th

e c

ou

nt-

do

wn

T

his

ro

uti

ne i

s a

llo

wed

to

op

era

te u

p t

o 4

0 s

eco

nd

s

aft

er l

ift-

off

D

uri

ng

th

is p

eri

od

th

e A

ria

ne

5 t

rav

els

fart

her

do

wn

ra

ng

e th

an

th

e

Ari

an

e 4

an

d t

her

efo

re p

rod

uce

s a

la

rger

ho

rizo

nta

l v

elo

city

co

mp

on

en

t v

alu

e W

hen

this

valu

e r

each

ed

th

e fl

igh

t co

mp

ute

r p

rog

ram

lim

it (

ie n

um

eri

cal

ov

erfl

ow

) w

hic

h i

s

no

t re

ach

ed

by

th

e A

rian

e 4

in

th

e sa

me t

imefr

am

e t

he f

lig

ht

com

pu

ter

shu

tdo

wn

re

set

the

inert

ial

pla

tfo

rms

Du

rin

g d

evelo

pm

en

t o

f th

e av

ion

ics

for

the

Ari

an

e 5

th

e S

exta

nt

Av

ion

iqu

e r

ing

la

ser

gy

ro i

ner

tail

pla

tfo

rms

were

no

t te

sted

in

th

e l

oo

p a

s p

art

of

an

en

d-t

o-e

nd

sim

ula

tio

n

sin

ce t

hey

had

pre

vio

usl

y b

een

pro

ven

to

wo

rk o

n t

he A

rian

e 4

I

nst

ead

th

eir

in

pu

t

was

on

ly s

imu

late

d A

co

mp

lete

hard

ware

-in

-th

e-l

oo

p t

est

wo

uld

lik

ely

hav

e

un

cov

ere

d t

he

syst

em

ic f

ail

ure

of

thes

e A

rian

e-4

deri

ved

in

ert

ial

pla

tfo

rms

in t

he

Ari

an

e 5

fli

gh

t en

vir

on

men

t

As

a r

esu

lt o

f th

is f

ail

ure

E

SA

lo

st t

he

US

$50

0 m

illi

on

Clu

ster

sate

llit

e p

rog

ram

an

d

est

ima

tes

a 2

-4

in

crease

in

co

st o

f th

e U

S $

8 b

illi

on

Ari

an

e 5

pro

gra

m alo

ng

wit

h a

n

ap

pro

xim

ate

on

e-y

ear

dela

y i

n t

he A

rian

e 5

pro

gra

m i

n o

rder

to

per

form

mo

re

hard

wa

re s

imu

lati

on

test

s an

d d

esi

gn

a p

rov

e a

new

gu

idan

ce p

rog

ram

Fig

ure

13

Typ

ical

ana

lysi

s pr

oces

s us

ed b

y in

telli

genc

e an

alys

ts t

o se

arch

a d

ocum

ent

data

base

and

synt

hesi

ze r

esul

ts to

form

ulat

e a

resp

onse

to a

n an

alys

is q

uery

Rep

rinte

d fr

om P

atte

rson

E S

R

oth

E M

amp

Woo

ds D

D P

redi

ctin

g vu

lner

abili

ty in

com

pute

r-su

ppor

ted

infe

rent

ial a

naly

sis

unde

rda

ta o

verlo

ad

Cog

nitio

n T

echn

olog

y amp

Wor

k 3

22

4ndash23

7 C

opyr

ight

200

1 w

ith k

ind

perm

issi

on o

fS

prin

ger

Sci

ence

and

Bus

ines

s M

edia


12


copy1999 Patterson


161

29


22

5


169

15


419

28


7

184


66

14


12

194


4

29




CTA METHODS




































































21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion














RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES



































































































































































































Ca

su

alty M

an

age

me

nt

Ba

lan

ce

of A

uth

ority

Ris

k-B

enefit B

ala

nce

Re

sou

rce

Ba

lance

Eco

nom

ic B

ala

nce

Resp

on

se P

lan

sO

pe

ratio

ns P

roce

sse

s

Co

ord

ina

tin

g a

nd I

mp

lem

en

ting

Me

dic

al O

pe

ratio

ns

Co

ord

ina

tin

g a

nd I

mp

lem

en

ting

She

lte

r O

pera

tio

ns

Fix

ed L

oca

tion

Re

so

urc

es

(Ho

sp

ita

ls 9

11

Ca

ll C

ente

rs

Am

bu

lan

ce D

isp

atc

h F

ire

Sta

tion

s P

olic

e S

tatio

ns)

Mo

bile

Re

so

urc

es

(Am

bu

lan

ce

s

Fire

Re

scu

e

Fir

e E

ngin

es P

olic

eV

eh

icle

s)

Hu

ma

n R

eso

urc

es

(Po

lice

E

MT

F

ire

)

Sta

te o

f b

uild

ing

s o

pe

ratin

gcon

ditio

ns lo

ca

tio

n

ca

pa

city ca

pacity v

s lo

ad

(eg

fo

r a

ho

spita

l)

Nu

mb

er

co

nd

itio

n

op

era

tion

al sta

te

locatio

nof m

obile

reso

urc

es

pers

on

ne

l

Fu

nctio

na

l P

urp

ose

Ge

ne

ral P

roce

sse

s

Ab

str

act F

un

ction

Ph

ysic

al F

un

ctio

n

Ph

ysic

al F

orm

Sys

tem

Sub

-Sys

tem

Fig

ure

12

Por

tion

of a

n ab

stra

ctio

n hi

erar

chy

wor

k do

mai

n m

odel

for

an

emer

genc

y re

spon

sesy

stem

B

ased

on

wor

k fr

om B

isan

tz

Littl

e a

nd R

ogov

a (2

004)

7



























11

Qu

ery

(k

ey

wo

rds

refi

nem

en

t)

Sele

ct ldquo

key

rdquo d

ocu

men

ts

Co

rro

bo

rate

in

form

ati

on

(o

r

reso

lve d

iscr

ep

an

cies

in

info

rmati

on

) an

d f

ill

in g

ap

s

wit

h s

up

po

rt d

ocu

men

ts

Sy

nth

esi

ze i

nfo

rmati

on

to

con

stru

ct a

co

here

nt

sto

ry

Bro

wse

by

tit

le a

nd

o

r d

ate

19

960

80

8

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

19

960

50

0

AR

IAN

E 5

A

N A

LL

NE

W L

AU

NC

HE

R F

OR

TH

E G

LO

BA

L M

AR

KE

T

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

19

960

61

0

AR

IAN

E 5

EX

PL

OD

ES

IN

TH

E S

KY

OV

ER

KO

UR

OU

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

eu

rop

e 1

996

(eu

rop

e 1

996)

amp (

lau

nch

fail

ure

)

(eu

rop

e 1

996)

amp (

(la

un

ch f

ail

ure

)

2)

378

140

22

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

Da

te

Au

gu

st 8

1996

So

urc

e

FB

IS r

ep

ort

AC

CO

RD

ING

T

O T

HE

R

EP

OR

T R

EC

EN

TL

Y S

UB

MIT

TE

D B

Y T

HE

IND

EP

EN

DE

NT

IN

QU

IRY

B

OA

RD

S

ET

U

P

AF

TE

R T

HE

AR

IAN

E 5

ltlt

LA

UN

CH

gtgt

ltlt

FA

ILU

RE

gtgt

ON

JU

NE

4 (

RE

FT

EL

A

) T

HE

EX

PLO

SIO

N W

AS

DU

E T

O S

PE

CIF

ICA

TIO

N A

ND

DE

SIG

NE

RR

OR

S I

N

TH

E S

OF

TW

AR

E

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M T

HE

RE

PO

RT

FU

RT

HE

R S

TR

ES

SE

D T

HE

F

AC

T T

HA

T

TH

E

ALIG

NM

EN

T F

UN

CT

ION

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M W

HIC

H

SE

RV

ED

A

P

UR

PO

SE

O

NL

YB

EF

OR

E L

IFT

OF

F (B

UT

R

EM

AIN

ED

O

PE

RA

TIV

E

AF

TE

RW

AR

DS

) W

AS

N

OT

TA

KE

N I

NT

O A

CC

OU

NT

IN

T

HE

S

IMU

LA

TIO

NS

AN

D T

HA

T T

HE

EQ

UIP

ME

NT

AN

DS

YS

TE

M T

ES

TS

WE

RE

N

OT

S

UF

FIC

IEN

TL

Y R

EP

RE

SE

NT

AT

IVE

T

HE

SY

ST

EM

AR

CH

ITE

CT

UR

E W

AS

NO

T IM

PLIC

AT

ED

T

HE

R

EP

OR

T

RE

CO

MM

EN

DE

D E

XT

EN

SIV

ER

EV

IEW

OF

TH

E S

OF

TW

AR

E D

ES

IGN

AN

D T

ES

TIN

G P

RO

CE

DU

RE

S

TH

E C

OS

T O

F C

OR

RE

CT

IVE

M

EA

SU

RE

S I

S E

ST

IMA

TE

D A

T 2

TO

4P

ER

CE

NT

OF

TH

E G

LO

BA

L

CO

ST

O

F T

HE

P

RO

JE

CT

T

HA

T IS

U

SD

150 T

O 3

00

MIL

LIO

N

CO

RR

EC

TIV

E A

CT

ION

IS

E

XP

EC

TE

D T

O P

OS

TP

ON

E T

HE

NE

XT

LA

UN

CH

TO

MID

SE

ME

ST

ER

1997

E

SA

DO

ES

NO

T E

XC

LU

DE

TH

E P

OS

SIB

ILIT

Y T

HA

TT

HE

TH

IRD

AR

IAN

E 5

LA

UN

CH

IN

ITIA

LLY

P

LA

NN

ED

A

S A

C

OM

ME

RC

IAL L

AU

NC

H

MA

Y E

VE

NT

UA

LLY

B

E T

RE

AT

ED

A

S A

Q

UA

LIF

ICA

TIO

N LA

UN

CH

ltlt

Eu

rop

egt

gts

he

avy-

lift la

un

che

r pro

gra

mm

e w

ill c

on

tin

ue

de

spite

th

eJu

ne

4se

tba

ck

bu

t th

e m

arg

in fo

r err

or

ha

s n

ow

be

en

re

du

ce

d t

o z

ero

Th

e lo

ss

of th

e fir

st A

ria

ne

5 o

n it

s in

itia

l q

ualif

ica

tio

n flig

ht

on

Jun

e 4

ma

rks a

se

ve

re b

ut

by

no

me

an

s fa

tal s

etb

ack

fo

r lt

ltE

uro

pe

gtgt

sn

ew

he

avy-

lift

laun

che

r a

nd

for

Ari

an

esp

ace

s a

mb

itio

ns t

o c

on

solid

ate

its

do

min

an

t p

osi

tio

n o

n t

he

glo

bal co

mm

erc

ial la

un

ch m

ark

et

It

will

take

mo

re t

ha

n a

sin

gle

ltlt

lau

nch

gtgt

ltlt

failu

regt

gt t

o d

era

il th

e 1

5-y

ea

r $

82

bill

ion

pro

gra

mm

e

bu

t th

e m

arg

in f

or

err

or

ha

s h

en

ce

fort

h b

een

red

uce

d to

zero

Th

e 4 J

un

e 1

996

la

un

ch o

f th

e fi

rst

Eu

rop

ean

Sp

ace

Ag

en

cy (

ES

A)

Ari

an

e 5

sp

ace

lau

nch

veh

icle

carr

yin

g t

he f

ou

r E

uro

pean

ldquoC

lust

errdquo

sci

en

tifi

c sa

tell

ites

en

ded

in

fail

ure

ju

st

ov

er

36 s

eco

nd

s aft

er l

ift-

off

du

e t

o i

nh

eri

t so

ftw

are

des

ign

fla

w i

n t

he g

uid

an

ce s

yst

em

Th

e im

bed

ded

so

ftw

are

in

bo

th o

f th

e p

rim

ary

an

d b

ack

-up

id

enti

call

y-d

esi

gn

ed

inert

ial

gu

idan

ce p

latf

orm

s c

urr

en

tly

pro

ven

an

d u

sed

on

bo

ard

th

e A

rian

e 4

w

as

ill-

desi

gn

ed

fo

r th

e f

lig

ht

pro

file

of

the

Ari

an

e 5

I

nsu

ffic

ien

t re

qu

irem

en

ts a

nd

test

ing

of

the

Ari

an

e 5 a

vio

nic

s sy

stem

du

rin

g d

ev

elo

pm

en

t d

id n

ot

un

cov

er t

he d

esi

gn

fla

w

Th

e acc

elera

tio

n o

f th

e A

rian

e 5

is

mu

ch g

reate

r th

an

th

e A

rian

e 4

aft

er l

ift-

off

T

his

cau

sed

th

e i

ner

tial

gu

idan

ce p

latf

orm

em

bed

ded

so

ftw

are

w

hic

h w

as

des

ign

ed

aro

un

d

the

slo

wer

Ari

an

e 4

acc

ele

rati

on

pro

file

to

pro

vid

e n

um

eri

cal

va

lues

bey

on

d t

he

pro

gra

mm

ed

lim

its

of

the

flig

ht

com

pu

ter

wh

ich

th

en

sh

utd

ow

n b

oth

in

erti

al

pla

tfo

rms

Th

e p

latf

orm

s in

itia

ted

a d

iag

no

stic

ldquore

setrdquo

mo

de

tha

t fe

d i

nco

rrec

t v

alu

es t

o t

he

flig

ht

com

pu

ter

wh

ich

th

en c

om

man

ded

an

ex

cess

ive

pit

ch-o

ver

by

gim

ball

ing

th

e tw

o s

trap

-

on

bo

ost

er

no

zzle

s an

d t

hen

fir

st-s

tag

e m

ain

en

gin

e n

ozz

le T

his

rap

id p

itch

in

to a

hig

h

an

gle

of

att

ack

cau

sed

th

e f

air

ing

to

sep

ara

te f

rom

th

e v

eh

icle

du

e to

th

e h

igh

aero

dy

nam

ic l

oad

ing

T

his

cau

sed

th

e o

n-b

oard

self

-des

tru

ct s

yst

em

to

act

iva

te w

hic

h

was

late

r fo

llo

wed

by

a r

an

ge s

afe

ty-i

ssu

ed d

est

ruct

co

mm

an

d

Th

e f

lig

ht

com

pu

ter

pro

gra

m s

hu

tdo

wn

th

e in

ert

ial

pla

tfo

rms

as

pa

rt o

f a g

uid

an

ce r

e-a

lig

nm

ent

rou

tin

e

Th

is r

ou

tin

e i

s p

art

of

the

Ari

an

e 4

reali

gn

men

t ro

uti

ne d

esi

gn

ed

to

all

ow

qu

ick

up

da

tes

aft

er a

ho

ld l

ate

in

th

e c

ou

nt-

do

wn

T

his

ro

uti

ne i

s a

llo

wed

to

op

era

te u

p t

o 4

0 s

eco

nd

s

aft

er l

ift-

off

D

uri

ng

th

is p

eri

od

th

e A

ria

ne

5 t

rav

els

fart

her

do

wn

ra

ng

e th

an

th

e

Ari

an

e 4

an

d t

her

efo

re p

rod

uce

s a

la

rger

ho

rizo

nta

l v

elo

city

co

mp

on

en

t v

alu

e W

hen

this

valu

e r

each

ed

th

e fl

igh

t co

mp

ute

r p

rog

ram

lim

it (

ie n

um

eri

cal

ov

erfl

ow

) w

hic

h i

s

no

t re

ach

ed

by

th

e A

rian

e 4

in

th

e sa

me t

imefr

am

e t

he f

lig

ht

com

pu

ter

shu

tdo

wn

re

set

the

inert

ial

pla

tfo

rms

Du

rin

g d

evelo

pm

en

t o

f th

e av

ion

ics

for

the

Ari

an

e 5

th

e S

exta

nt

Av

ion

iqu

e r

ing

la

ser

gy

ro i

ner

tail

pla

tfo

rms

were

no

t te

sted

in

th

e l

oo

p a

s p

art

of

an

en

d-t

o-e

nd

sim

ula

tio

n

sin

ce t

hey

had

pre

vio

usl

y b

een

pro

ven

to

wo

rk o

n t

he A

rian

e 4

I

nst

ead

th

eir

in

pu

t

was

on

ly s

imu

late

d A

co

mp

lete

hard

ware

-in

-th

e-l

oo

p t

est

wo

uld

lik

ely

hav

e

un

cov

ere

d t

he

syst

em

ic f

ail

ure

of

thes

e A

rian

e-4

deri

ved

in

ert

ial

pla

tfo

rms

in t

he

Ari

an

e 5

fli

gh

t en

vir

on

men

t

As

a r

esu

lt o

f th

is f

ail

ure

E

SA

lo

st t

he

US

$50

0 m

illi

on

Clu

ster

sate

llit

e p

rog

ram

an

d

est

ima

tes

a 2

-4

in

crease

in

co

st o

f th

e U

S $

8 b

illi

on

Ari

an

e 5

pro

gra

m alo

ng

wit

h a

n

ap

pro

xim

ate

on

e-y

ear

dela

y i

n t

he A

rian

e 5

pro

gra

m i

n o

rder

to

per

form

mo

re

hard

wa

re s

imu

lati

on

test

s an

d d

esi

gn

a p

rov

e a

new

gu

idan

ce p

rog

ram

Fig

ure

13

Typ

ical

ana

lysi

s pr

oces

s us

ed b

y in

telli

genc

e an

alys

ts t

o se

arch

a d

ocum

ent

data

base

and

synt

hesi

ze r

esul

ts to

form

ulat

e a

resp

onse

to a

n an

alys

is q

uery

Rep

rinte

d fr

om P

atte

rson

E S

R

oth

E M

amp

Woo

ds D

D P

redi

ctin

g vu

lner

abili

ty in

com

pute

r-su

ppor

ted

infe

rent

ial a

naly

sis

unde

rda

ta o

verlo

ad

Cog

nitio

n T

echn

olog

y amp

Wor

k 3

22

4ndash23

7 C

opyr

ight

200

1 w

ith k

ind

perm

issi

on o

fS

prin

ger

Sci

ence

and

Bus

ines

s M

edia


12


copy1999 Patterson


161

29


22

5


169

15


419

28


7

184


66

14


12

194


4

29




CTA METHODS




































































21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion














RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES




























































































































































































































11

Qu

ery

(k

ey

wo

rds

refi

nem

en

t)

Sele

ct ldquo

key

rdquo d

ocu

men

ts

Co

rro

bo

rate

in

form

ati

on

(o

r

reso

lve d

iscr

ep

an

cies

in

info

rmati

on

) an

d f

ill

in g

ap

s

wit

h s

up

po

rt d

ocu

men

ts

Sy

nth

esi

ze i

nfo

rmati

on

to

con

stru

ct a

co

here

nt

sto

ry

Bro

wse

by

tit

le a

nd

o

r d

ate

19

960

80

8

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

19

960

50

0

AR

IAN

E 5

A

N A

LL

NE

W L

AU

NC

HE

R F

OR

TH

E G

LO

BA

L M

AR

KE

T

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

19

960

61

0

AR

IAN

E 5

EX

PL

OD

ES

IN

TH

E S

KY

OV

ER

KO

UR

OU

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

eu

rop

e 1

996

(eu

rop

e 1

996)

amp (

lau

nch

fail

ure

)

(eu

rop

e 1

996)

amp (

(la

un

ch f

ail

ure

)

2)

378

140

22

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

Da

te

Au

gu

st 8

1996

So

urc

e

FB

IS r

ep

ort

AC

CO

RD

ING

T

O T

HE

R

EP

OR

T R

EC

EN

TL

Y S

UB

MIT

TE

D B

Y T

HE

IND

EP

EN

DE

NT

IN

QU

IRY

B

OA

RD

S

ET

U

P

AF

TE

R T

HE

AR

IAN

E 5

ltlt

LA

UN

CH

gtgt

ltlt

FA

ILU

RE

gtgt

ON

JU

NE

4 (

RE

FT

EL

A

) T

HE

EX

PLO

SIO

N W

AS

DU

E T

O S

PE

CIF

ICA

TIO

N A

ND

DE

SIG

NE

RR

OR

S I

N

TH

E S

OF

TW

AR

E

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M T

HE

RE

PO

RT

FU

RT

HE

R S

TR

ES

SE

D T

HE

F

AC

T T

HA

T

TH

E

ALIG

NM

EN

T F

UN

CT

ION

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M W

HIC

H

SE

RV

ED

A

P

UR

PO

SE

O

NL

YB

EF

OR

E L

IFT

OF

F (B

UT

R

EM

AIN

ED

O

PE

RA

TIV

E

AF

TE

RW

AR

DS

) W

AS

N

OT

TA

KE

N I

NT

O A

CC

OU

NT

IN

T

HE

S

IMU

LA

TIO

NS

AN

D T

HA

T T

HE

EQ

UIP

ME

NT

AN

DS

YS

TE

M T

ES

TS

WE

RE

N

OT

S

UF

FIC

IEN

TL

Y R

EP

RE

SE

NT

AT

IVE

T

HE

SY

ST

EM

AR

CH

ITE

CT

UR

E W

AS

NO

T IM

PLIC

AT

ED

T

HE

R

EP

OR

T

RE

CO

MM

EN

DE

D E

XT

EN

SIV

ER

EV

IEW

OF

TH

E S

OF

TW

AR

E D

ES

IGN

AN

D T

ES

TIN

G P

RO

CE

DU

RE

S

TH

E C

OS

T O

F C

OR

RE

CT

IVE

M

EA

SU

RE

S I

S E

ST

IMA

TE

D A

T 2

TO

4P

ER

CE

NT

OF

TH

E G

LO

BA

L

CO

ST

O

F T

HE

P

RO

JE

CT

T

HA

T IS

U

SD

150 T

O 3

00

MIL

LIO

N

CO

RR

EC

TIV

E A

CT

ION

IS

E

XP

EC

TE

D T

O P

OS

TP

ON

E T

HE

NE

XT

LA

UN

CH

TO

MID

SE

ME

ST

ER

1997

E

SA

DO

ES

NO

T E

XC

LU

DE

TH

E P

OS

SIB

ILIT

Y T

HA

TT

HE

TH

IRD

AR

IAN

E 5

LA

UN

CH

IN

ITIA

LLY

P

LA

NN

ED

A

S A

C

OM

ME

RC

IAL L

AU

NC

H

MA

Y E

VE

NT

UA

LLY

B

E T

RE

AT

ED

A

S A

Q

UA

LIF

ICA

TIO

N LA

UN

CH

ltlt

Eu

rop

egt

gts

he

avy-

lift la

un

che

r pro

gra

mm

e w

ill c

on

tin

ue

de

spite

th

eJu

ne

4se

tba

ck

bu

t th

e m

arg

in fo

r err

or

ha

s n

ow

be

en

re

du

ce

d t

o z

ero

Th

e lo

ss

of th

e fir

st A

ria

ne

5 o

n it

s in

itia

l q

ualif

ica

tio

n flig

ht

on

Jun

e 4

ma

rks a

se

ve

re b

ut

by

no

me

an

s fa

tal s

etb

ack

fo

r lt

ltE

uro

pe

gtgt

sn

ew

he

avy-

lift

laun

che

r a

nd

for

Ari

an

esp

ace

s a

mb

itio

ns t

o c

on

solid

ate

its

do

min

an

t p

osi

tio

n o

n t

he

glo

bal co

mm

erc

ial la

un

ch m

ark

et

It

will

take

mo

re t

ha

n a

sin

gle

ltlt

lau

nch

gtgt

ltlt

failu

regt

gt t

o d

era

il th

e 1

5-y

ea

r $

82

bill

ion

pro

gra

mm

e

bu

t th

e m

arg

in f

or

err

or

ha

s h

en

ce

fort

h b

een

red

uce

d to

zero

Th

e 4 J

un

e 1

996

la

un

ch o

f th

e fi

rst

Eu

rop

ean

Sp

ace

Ag

en

cy (

ES

A)

Ari

an

e 5

sp

ace

lau

nch

veh

icle

carr

yin

g t

he f

ou

r E

uro

pean

ldquoC

lust

errdquo

sci

en

tifi

c sa

tell

ites

en

ded

in

fail

ure

ju

st

ov

er

36 s

eco

nd

s aft

er l

ift-

off

du

e t

o i

nh

eri

t so

ftw

are

des

ign

fla

w i

n t

he g

uid

an

ce s

yst

em

Th

e im

bed

ded

so

ftw

are

in

bo

th o

f th

e p

rim

ary

an

d b

ack

-up

id

enti

call

y-d

esi

gn

ed

inert

ial

gu

idan

ce p

latf

orm

s c

urr

en

tly

pro

ven

an

d u

sed

on

bo

ard

th

e A

rian

e 4

w

as

ill-

desi

gn

ed

fo

r th

e f

lig

ht

pro

file

of

the

Ari

an

e 5

I

nsu

ffic

ien

t re

qu

irem

en

ts a

nd

test

ing

of

the

Ari

an

e 5 a

vio

nic

s sy

stem

du

rin

g d

ev

elo

pm

en

t d

id n

ot

un

cov

er t

he d

esi

gn

fla

w

Th

e acc

elera

tio

n o

f th

e A

rian

e 5

is

mu

ch g

reate

r th

an

th

e A

rian

e 4

aft

er l

ift-

off

T

his

cau

sed

th

e i

ner

tial

gu

idan

ce p

latf

orm

em

bed

ded

so

ftw

are

w

hic

h w

as

des

ign

ed

aro

un

d

the

slo

wer

Ari

an

e 4

acc

ele

rati

on

pro

file

to

pro

vid

e n

um

eri

cal

va

lues

bey

on

d t

he

pro

gra

mm

ed

lim

its

of

the

flig

ht

com

pu

ter

wh

ich

th

en

sh

utd

ow

n b

oth

in

erti

al

pla

tfo

rms

Th

e p

latf

orm

s in

itia

ted

a d

iag

no

stic

ldquore

setrdquo

mo

de

tha

t fe

d i

nco

rrec

t v

alu

es t

o t

he

flig

ht

com

pu

ter

wh

ich

th

en c

om

man

ded

an

ex

cess

ive

pit

ch-o

ver

by

gim

ball

ing

th

e tw

o s

trap

-

on

bo

ost

er

no

zzle

s an

d t

hen

fir

st-s

tag

e m

ain

en

gin

e n

ozz

le T

his

rap

id p

itch

in

to a

hig

h

an

gle

of

att

ack

cau

sed

th

e f

air

ing

to

sep

ara

te f

rom

th

e v

eh

icle

du

e to

th

e h

igh

aero

dy

nam

ic l

oad

ing

T

his

cau

sed

th

e o

n-b

oard

self

-des

tru

ct s

yst

em

to

act

iva

te w

hic

h

was

late

r fo

llo

wed

by

a r

an

ge s

afe

ty-i

ssu

ed d

est

ruct

co

mm

an

d

Th

e f

lig

ht

com

pu

ter

pro

gra

m s

hu

tdo

wn

th

e in

ert

ial

pla

tfo

rms

as

pa

rt o

f a g

uid

an

ce r

e-a

lig

nm

ent

rou

tin

e

Th

is r

ou

tin

e i

s p

art

of

the

Ari

an

e 4

reali

gn

men

t ro

uti

ne d

esi

gn

ed

to

all

ow

qu

ick

up

da

tes

aft

er a

ho

ld l

ate

in

th

e c

ou

nt-

do

wn

T

his

ro

uti

ne i

s a

llo

wed

to

op

era

te u

p t

o 4

0 s

eco

nd

s

aft

er l

ift-

off

D

uri

ng

th

is p

eri

od

th

e A

ria

ne

5 t

rav

els

fart

her

do

wn

ra

ng

e th

an

th

e

Ari

an

e 4

an

d t

her

efo

re p

rod

uce

s a

la

rger

ho

rizo

nta

l v

elo

city

co

mp

on

en

t v

alu

e W

hen

this

valu

e r

each

ed

th

e fl

igh

t co

mp

ute

r p

rog

ram

lim

it (

ie n

um

eri

cal

ov

erfl

ow

) w

hic

h i

s

no

t re

ach

ed

by

th

e A

rian

e 4

in

th

e sa

me t

imefr

am

e t

he f

lig

ht

com

pu

ter

shu

tdo

wn

re

set

the

inert

ial

pla

tfo

rms

Du

rin

g d

evelo

pm

en

t o

f th

e av

ion

ics

for

the

Ari

an

e 5

th

e S

exta

nt

Av

ion

iqu

e r

ing

la

ser

gy

ro i

ner

tail

pla

tfo

rms

were

no

t te

sted

in

th

e l

oo

p a

s p

art

of

an

en

d-t

o-e

nd

sim

ula

tio

n

sin

ce t

hey

had

pre

vio

usl

y b

een

pro

ven

to

wo

rk o

n t

he A

rian

e 4

I

nst

ead

th

eir

in

pu

t

was

on

ly s

imu

late

d A

co

mp

lete

hard

ware

-in

-th

e-l

oo

p t

est

wo

uld

lik

ely

hav

e

un

cov

ere

d t

he

syst

em

ic f

ail

ure

of

thes

e A

rian

e-4

deri

ved

in

ert

ial

pla

tfo

rms

in t

he

Ari

an

e 5

fli

gh

t en

vir

on

men

t

As

a r

esu

lt o

f th

is f

ail

ure

E

SA

lo

st t

he

US

$50

0 m

illi

on

Clu

ster

sate

llit

e p

rog

ram

an

d

est

ima

tes

a 2

-4

in

crease

in

co

st o

f th

e U

S $

8 b

illi

on

Ari

an

e 5

pro

gra

m alo

ng

wit

h a

n

ap

pro

xim

ate

on

e-y

ear

dela

y i

n t

he A

rian

e 5

pro

gra

m i

n o

rder

to

per

form

mo

re

hard

wa

re s

imu

lati

on

test

s an

d d

esi

gn

a p

rov

e a

new

gu

idan

ce p

rog

ram

Fig

ure

13

Typ

ical

ana

lysi

s pr

oces

s us

ed b

y in

telli

genc

e an

alys

ts t

o se

arch

a d

ocum

ent

data

base

and

synt

hesi

ze r

esul

ts to

form

ulat

e a

resp

onse

to a

n an

alys

is q

uery

Rep

rinte

d fr

om P

atte

rson

E S

R

oth

E M

amp

Woo

ds D

D P

redi

ctin

g vu

lner

abili

ty in

com

pute

r-su

ppor

ted

infe

rent

ial a

naly

sis

unde

rda

ta o

verlo

ad

Cog

nitio

n T

echn

olog

y amp

Wor

k 3

22

4ndash23

7 C

opyr

ight

200

1 w

ith k

ind

perm

issi

on o

fS

prin

ger

Sci

ence

and

Bus

ines

s M

edia


12


copy1999 Patterson


161

29


22

5


169

15


419

28


7

184


66

14


12

194


4

29




CTA METHODS




































































21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion














RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES



















































































































































































































11

Qu

ery

(k

ey

wo

rds

refi

nem

en

t)

Sele

ct ldquo

key

rdquo d

ocu

men

ts

Co

rro

bo

rate

in

form

ati

on

(o

r

reso

lve d

iscr

ep

an

cies

in

info

rmati

on

) an

d f

ill

in g

ap

s

wit

h s

up

po

rt d

ocu

men

ts

Sy

nth

esi

ze i

nfo

rmati

on

to

con

stru

ct a

co

here

nt

sto

ry

Bro

wse

by

tit

le a

nd

o

r d

ate

19

960

80

8

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

19

960

50

0

AR

IAN

E 5

A

N A

LL

NE

W L

AU

NC

HE

R F

OR

TH

E G

LO

BA

L M

AR

KE

T

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

19

960

61

0

AR

IAN

E 5

EX

PL

OD

ES

IN

TH

E S

KY

OV

ER

KO

UR

OU

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

eu

rop

e 1

996

(eu

rop

e 1

996)

amp (

lau

nch

fail

ure

)

(eu

rop

e 1

996)

amp (

(la

un

ch f

ail

ure

)

2)

378

140

22

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

Da

te

Au

gu

st 8

1996

So

urc

e

FB

IS r

ep

ort

AC

CO

RD

ING

T

O T

HE

R

EP

OR

T R

EC

EN

TL

Y S

UB

MIT

TE

D B

Y T

HE

IND

EP

EN

DE

NT

IN

QU

IRY

B

OA

RD

S

ET

U

P

AF

TE

R T

HE

AR

IAN

E 5

ltlt

LA

UN

CH

gtgt

ltlt

FA

ILU

RE

gtgt

ON

JU

NE

4 (

RE

FT

EL

A

) T

HE

EX

PLO

SIO

N W

AS

DU

E T

O S

PE

CIF

ICA

TIO

N A

ND

DE

SIG

NE

RR

OR

S I

N

TH

E S

OF

TW

AR

E

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M T

HE

RE

PO

RT

FU

RT

HE

R S

TR

ES

SE

D T

HE

F

AC

T T

HA

T

TH

E

ALIG

NM

EN

T F

UN

CT

ION

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M W

HIC

H

SE

RV

ED

A

P

UR

PO

SE

O

NL

YB

EF

OR

E L

IFT

OF

F (B

UT

R

EM

AIN

ED

O

PE

RA

TIV

E

AF

TE

RW

AR

DS

) W

AS

N

OT

TA

KE

N I

NT

O A

CC

OU

NT

IN

T

HE

S

IMU

LA

TIO

NS

AN

D T

HA

T T

HE

EQ

UIP

ME

NT

AN

DS

YS

TE

M T

ES

TS

WE

RE

N

OT

S

UF

FIC

IEN

TL

Y R

EP

RE

SE

NT

AT

IVE

T

HE

SY

ST

EM

AR

CH

ITE

CT

UR

E W

AS

NO

T IM

PLIC

AT

ED

T

HE

R

EP

OR

T

RE

CO

MM

EN

DE

D E

XT

EN

SIV

ER

EV

IEW

OF

TH

E S

OF

TW

AR

E D

ES

IGN

AN

D T

ES

TIN

G P

RO

CE

DU

RE

S

TH

E C

OS

T O

F C

OR

RE

CT

IVE

M

EA

SU

RE

S I

S E

ST

IMA

TE

D A

T 2

TO

4P

ER

CE

NT

OF

TH

E G

LO

BA

L

CO

ST

O

F T

HE

P

RO

JE

CT

T

HA

T IS

U

SD

150 T

O 3

00

MIL

LIO

N

CO

RR

EC

TIV

E A

CT

ION

IS

E

XP

EC

TE

D T

O P

OS

TP

ON

E T

HE

NE

XT

LA

UN

CH

TO

MID

SE

ME

ST

ER

1997

E

SA

DO

ES

NO

T E

XC

LU

DE

TH

E P

OS

SIB

ILIT

Y T

HA

TT

HE

TH

IRD

AR

IAN

E 5

LA

UN

CH

IN

ITIA

LLY

P

LA

NN

ED

A

S A

C

OM

ME

RC

IAL L

AU

NC

H

MA

Y E

VE

NT

UA

LLY

B

E T

RE

AT

ED

A

S A

Q

UA

LIF

ICA

TIO

N LA

UN

CH

ltlt

Eu

rop

egt

gts

he

avy-

lift la

un

che

r pro

gra

mm

e w

ill c

on

tin

ue

de

spite

th

eJu

ne

4se

tba

ck

bu

t th

e m

arg

in fo

r err

or

ha

s n

ow

be

en

re

du

ce

d t

o z

ero

Th

e lo

ss

of th

e fir

st A

ria

ne

5 o

n it

s in

itia

l q

ualif

ica

tio

n flig

ht

on

Jun

e 4

ma

rks a

se

ve

re b

ut

by

no

me

an

s fa

tal s

etb

ack

fo

r lt

ltE

uro

pe

gtgt

sn

ew

he

avy-

lift

laun

che

r a

nd

for

Ari

an

esp

ace

s a

mb

itio

ns t

o c

on

solid

ate

its

do

min

an

t p

osi

tio

n o

n t

he

glo

bal co

mm

erc

ial la

un

ch m

ark

et

It

will

take

mo

re t

ha

n a

sin

gle

ltlt

lau

nch

gtgt

ltlt

failu

regt

gt t

o d

era

il th

e 1

5-y

ea

r $

82

bill

ion

pro

gra

mm

e

bu

t th

e m

arg

in f

or

err

or

ha

s h

en

ce

fort

h b

een

red

uce

d to

zero

Th

e 4 J

un

e 1

996

la

un

ch o

f th

e fi

rst

Eu

rop

ean

Sp

ace

Ag

en

cy (

ES

A)

Ari

an

e 5

sp

ace

lau

nch

veh

icle

carr

yin

g t

he f

ou

r E

uro

pean

ldquoC

lust

errdquo

sci

en

tifi

c sa

tell

ites

en

ded

in

fail

ure

ju

st

ov

er

36 s

eco

nd

s aft

er l

ift-

off

du

e t

o i

nh

eri

t so

ftw

are

des

ign

fla

w i

n t

he g

uid

an

ce s

yst

em

Th

e im

bed

ded

so

ftw

are

in

bo

th o

f th

e p

rim

ary

an

d b

ack

-up

id

enti

call

y-d

esi

gn

ed

inert

ial

gu

idan

ce p

latf

orm

s c

urr

en

tly

pro

ven

an

d u

sed

on

bo

ard

th

e A

rian

e 4

w

as

ill-

desi

gn

ed

fo

r th

e f

lig

ht

pro

file

of

the

Ari

an

e 5

I

nsu

ffic

ien

t re

qu

irem

en

ts a

nd

test

ing

of

the

Ari

an

e 5 a

vio

nic

s sy

stem

du

rin

g d

ev

elo

pm

en

t d

id n

ot

un

cov

er t

he d

esi

gn

fla

w

Th

e acc

elera

tio

n o

f th

e A

rian

e 5

is

mu

ch g

reate

r th

an

th

e A

rian

e 4

aft

er l

ift-

off

T

his

cau

sed

th

e i

ner

tial

gu

idan

ce p

latf

orm

em

bed

ded

so

ftw

are

w

hic

h w

as

des

ign

ed

aro

un

d

the

slo

wer

Ari

an

e 4

acc

ele

rati

on

pro

file

to

pro

vid

e n

um

eri

cal

va

lues

bey

on

d t

he

pro

gra

mm

ed

lim

its

of

the

flig

ht

com

pu

ter

wh

ich

th

en

sh

utd

ow

n b

oth

in

erti

al

pla

tfo

rms

Th

e p

latf

orm

s in

itia

ted

a d

iag

no

stic

ldquore

setrdquo

mo

de

tha

t fe

d i

nco

rrec

t v

alu

es t

o t

he

flig

ht

com

pu

ter

wh

ich

th

en c

om

man

ded

an

ex

cess

ive

pit

ch-o

ver

by

gim

ball

ing

th

e tw

o s

trap

-

on

bo

ost

er

no

zzle

s an

d t

hen

fir

st-s

tag

e m

ain

en

gin

e n

ozz

le T

his

rap

id p

itch

in

to a

hig

h

an

gle

of

att

ack

cau

sed

th

e f

air

ing

to

sep

ara

te f

rom

th

e v

eh

icle

du

e to

th

e h

igh

aero

dy

nam

ic l

oad

ing

T

his

cau

sed

th

e o

n-b

oard

self

-des

tru

ct s

yst

em

to

act

iva

te w

hic

h

was

late

r fo

llo

wed

by

a r

an

ge s

afe

ty-i

ssu

ed d

est

ruct

co

mm

an

d

Th

e f

lig

ht

com

pu

ter

pro

gra

m s

hu

tdo

wn

th

e in

ert

ial

pla

tfo

rms

as

pa

rt o

f a g

uid

an

ce r

e-a

lig

nm

ent

rou

tin

e

Th

is r

ou

tin

e i

s p

art

of

the

Ari

an

e 4

reali

gn

men

t ro

uti

ne d

esi

gn

ed

to

all

ow

qu

ick

up

da

tes

aft

er a

ho

ld l

ate

in

th

e c

ou

nt-

do

wn

T

his

ro

uti

ne i

s a

llo

wed

to

op

era

te u

p t

o 4

0 s

eco

nd

s

aft

er l

ift-

off

D

uri

ng

th

is p

eri

od

th

e A

ria

ne

5 t

rav

els

fart

her

do

wn

ra

ng

e th

an

th

e

Ari

an

e 4

an

d t

her

efo

re p

rod

uce

s a

la

rger

ho

rizo

nta

l v

elo

city

co

mp

on

en

t v

alu

e W

hen

this

valu

e r

each

ed

th

e fl

igh

t co

mp

ute

r p

rog

ram

lim

it (

ie n

um

eri

cal

ov

erfl

ow

) w

hic

h i

s

no

t re

ach

ed

by

th

e A

rian

e 4

in

th

e sa

me t

imefr

am

e t

he f

lig

ht

com

pu

ter

shu

tdo

wn

re

set

the

inert

ial

pla

tfo

rms

Du

rin

g d

evelo

pm

en

t o

f th

e av

ion

ics

for

the

Ari

an

e 5

th

e S

exta

nt

Av

ion

iqu

e r

ing

la

ser

gy

ro i

ner

tail

pla

tfo

rms

were

no

t te

sted

in

th

e l

oo

p a

s p

art

of

an

en

d-t

o-e

nd

sim

ula

tio

n

sin

ce t

hey

had

pre

vio

usl

y b

een

pro

ven

to

wo

rk o

n t

he A

rian

e 4

I

nst

ead

th

eir

in

pu

t

was

on

ly s

imu

late

d A

co

mp

lete

hard

ware

-in

-th

e-l

oo

p t

est

wo

uld

lik

ely

hav

e

un

cov

ere

d t

he

syst

em

ic f

ail

ure

of

thes

e A

rian

e-4

deri

ved

in

ert

ial

pla

tfo

rms

in t

he

Ari

an

e 5

fli

gh

t en

vir

on

men

t

As

a r

esu

lt o

f th

is f

ail

ure

E

SA

lo

st t

he

US

$50

0 m

illi

on

Clu

ster

sate

llit

e p

rog

ram

an

d

est

ima

tes

a 2

-4

in

crease

in

co

st o

f th

e U

S $

8 b

illi

on

Ari

an

e 5

pro

gra

m alo

ng

wit

h a

n

ap

pro

xim

ate

on

e-y

ear

dela

y i

n t

he A

rian

e 5

pro

gra

m i

n o

rder

to

per

form

mo

re

hard

wa

re s

imu

lati

on

test

s an

d d

esi

gn

a p

rov

e a

new

gu

idan

ce p

rog

ram

Fig

ure

13

Typ

ical

ana

lysi

s pr

oces

s us

ed b

y in

telli

genc

e an

alys

ts t

o se

arch

a d

ocum

ent

data

base

and

synt

hesi

ze r

esul

ts to

form

ulat

e a

resp

onse

to a

n an

alys

is q

uery

Rep

rinte

d fr

om P

atte

rson

E S

R

oth

E M

amp

Woo

ds D

D P

redi

ctin

g vu

lner

abili

ty in

com

pute

r-su

ppor

ted

infe

rent

ial a

naly

sis

unde

rda

ta o

verlo

ad

Cog

nitio

n T

echn

olog

y amp

Wor

k 3

22

4ndash23

7 C

opyr

ight

200

1 w

ith k

ind

perm

issi

on o

fS

prin

ger

Sci

ence

and

Bus

ines

s M

edia


12


copy1999 Patterson


161

29


22

5


169

15


419

28


7

184


66

14


12

194


4

29




CTA METHODS




































































21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion














RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES











































































































































































































11

Qu

ery

(k

ey

wo

rds

refi

nem

en

t)

Sele

ct ldquo

key

rdquo d

ocu

men

ts

Co

rro

bo

rate

in

form

ati

on

(o

r

reso

lve d

iscr

ep

an

cies

in

info

rmati

on

) an

d f

ill

in g

ap

s

wit

h s

up

po

rt d

ocu

men

ts

Sy

nth

esi

ze i

nfo

rmati

on

to

con

stru

ct a

co

here

nt

sto

ry

Bro

wse

by

tit

le a

nd

o

r d

ate

19

960

80

8

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

19

960

50

0

AR

IAN

E 5

A

N A

LL

NE

W L

AU

NC

HE

R F

OR

TH

E G

LO

BA

L M

AR

KE

T

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

19

960

61

0

AR

IAN

E 5

EX

PL

OD

ES

IN

TH

E S

KY

OV

ER

KO

UR

OU

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

eu

rop

e 1

996

(eu

rop

e 1

996)

amp (

lau

nch

fail

ure

)

(eu

rop

e 1

996)

amp (

(la

un

ch f

ail

ure

)

2)

378

140

22

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

Da

te

Au

gu

st 8

1996

So

urc

e

FB

IS r

ep

ort

AC

CO

RD

ING

T

O T

HE

R

EP

OR

T R

EC

EN

TL

Y S

UB

MIT

TE

D B

Y T

HE

IND

EP

EN

DE

NT

IN

QU

IRY

B

OA

RD

S

ET

U

P

AF

TE

R T

HE

AR

IAN

E 5

ltlt

LA

UN

CH

gtgt

ltlt

FA

ILU

RE

gtgt

ON

JU

NE

4 (

RE

FT

EL

A

) T

HE

EX

PLO

SIO

N W

AS

DU

E T

O S

PE

CIF

ICA

TIO

N A

ND

DE

SIG

NE

RR

OR

S I

N

TH

E S

OF

TW

AR

E

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M T

HE

RE

PO

RT

FU

RT

HE

R S

TR

ES

SE

D T

HE

F

AC

T T

HA

T

TH

E

ALIG

NM

EN

T F

UN

CT

ION

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M W

HIC

H

SE

RV

ED

A

P

UR

PO

SE

O

NL

YB

EF

OR

E L

IFT

OF

F (B

UT

R

EM

AIN

ED

O

PE

RA

TIV

E

AF

TE

RW

AR

DS

) W

AS

N

OT

TA

KE

N I

NT

O A

CC

OU

NT

IN

T

HE

S

IMU

LA

TIO

NS

AN

D T

HA

T T

HE

EQ

UIP

ME

NT

AN

DS

YS

TE

M T

ES

TS

WE

RE

N

OT

S

UF

FIC

IEN

TL

Y R

EP

RE

SE

NT

AT

IVE

T

HE

SY

ST

EM

AR

CH

ITE

CT

UR

E W

AS

NO

T IM

PLIC

AT

ED

T

HE

R

EP

OR

T

RE

CO

MM

EN

DE

D E

XT

EN

SIV

ER

EV

IEW

OF

TH

E S

OF

TW

AR

E D

ES

IGN

AN

D T

ES

TIN

G P

RO

CE

DU

RE

S

TH

E C

OS

T O

F C

OR

RE

CT

IVE

M

EA

SU

RE

S I

S E

ST

IMA

TE

D A

T 2

TO

4P

ER

CE

NT

OF

TH

E G

LO

BA

L

CO

ST

O

F T

HE

P

RO

JE

CT

T

HA

T IS

U

SD

150 T

O 3

00

MIL

LIO

N

CO

RR

EC

TIV

E A

CT

ION

IS

E

XP

EC

TE

D T

O P

OS

TP

ON

E T

HE

NE

XT

LA

UN

CH

TO

MID

SE

ME

ST

ER

1997

E

SA

DO

ES

NO

T E

XC

LU

DE

TH

E P

OS

SIB

ILIT

Y T

HA

TT

HE

TH

IRD

AR

IAN

E 5

LA

UN

CH

IN

ITIA

LLY

P

LA

NN

ED

A

S A

C

OM

ME

RC

IAL L

AU

NC

H

MA

Y E

VE

NT

UA

LLY

B

E T

RE

AT

ED

A

S A

Q

UA

LIF

ICA

TIO

N LA

UN

CH

ltlt

Eu

rop

egt

gts

he

avy-

lift la

un

che

r pro

gra

mm

e w

ill c

on

tin

ue

de

spite

th

eJu

ne

4se

tba

ck

bu

t th

e m

arg

in fo

r err

or

ha

s n

ow

be

en

re

du

ce

d t

o z

ero

Th

e lo

ss

of th

e fir

st A

ria

ne

5 o

n it

s in

itia

l q

ualif

ica

tio

n flig

ht

on

Jun

e 4

ma

rks a

se

ve

re b

ut

by

no

me

an

s fa

tal s

etb

ack

fo

r lt

ltE

uro

pe

gtgt

sn

ew

he

avy-

lift

laun

che

r a

nd

for

Ari

an

esp

ace

s a

mb

itio

ns t

o c

on

solid

ate

its

do

min

an

t p

osi

tio

n o

n t

he

glo

bal co

mm

erc

ial la

un

ch m

ark

et

It

will

take

mo

re t

ha

n a

sin

gle

ltlt

lau

nch

gtgt

ltlt

failu

regt

gt t

o d

era

il th

e 1

5-y

ea

r $

82

bill

ion

pro

gra

mm

e

bu

t th

e m

arg

in f

or

err

or

ha

s h

en

ce

fort

h b

een

red

uce

d to

zero

Th

e 4 J

un

e 1

996

la

un

ch o

f th

e fi

rst

Eu

rop

ean

Sp

ace

Ag

en

cy (

ES

A)

Ari

an

e 5

sp

ace

lau

nch

veh

icle

carr

yin

g t

he f

ou

r E

uro

pean

ldquoC

lust

errdquo

sci

en

tifi

c sa

tell

ites

en

ded

in

fail

ure

ju

st

ov

er

36 s

eco

nd

s aft

er l

ift-

off

du

e t

o i

nh

eri

t so

ftw

are

des

ign

fla

w i

n t

he g

uid

an

ce s

yst

em

Th

e im

bed

ded

so

ftw

are

in

bo

th o

f th

e p

rim

ary

an

d b

ack

-up

id

enti

call

y-d

esi

gn

ed

inert

ial

gu

idan

ce p

latf

orm

s c

urr

en

tly

pro

ven

an

d u

sed

on

bo

ard

th

e A

rian

e 4

w

as

ill-

desi

gn

ed

fo

r th

e f

lig

ht

pro

file

of

the

Ari

an

e 5

I

nsu

ffic

ien

t re

qu

irem

en

ts a

nd

test

ing

of

the

Ari

an

e 5 a

vio

nic

s sy

stem

du

rin

g d

ev

elo

pm

en

t d

id n

ot

un

cov

er t

he d

esi

gn

fla

w

Th

e acc

elera

tio

n o

f th

e A

rian

e 5

is

mu

ch g

reate

r th

an

th

e A

rian

e 4

aft

er l

ift-

off

T

his

cau

sed

th

e i

ner

tial

gu

idan

ce p

latf

orm

em

bed

ded

so

ftw

are

w

hic

h w

as

des

ign

ed

aro

un

d

the

slo

wer

Ari

an

e 4

acc

ele

rati

on

pro

file

to

pro

vid

e n

um

eri

cal

va

lues

bey

on

d t

he

pro

gra

mm

ed

lim

its

of

the

flig

ht

com

pu

ter

wh

ich

th

en

sh

utd

ow

n b

oth

in

erti

al

pla

tfo

rms

Th

e p

latf

orm

s in

itia

ted

a d

iag

no

stic

ldquore

setrdquo

mo

de

tha

t fe

d i

nco

rrec

t v

alu

es t

o t

he

flig

ht

com

pu

ter

wh

ich

th

en c

om

man

ded

an

ex

cess

ive

pit

ch-o

ver

by

gim

ball

ing

th

e tw

o s

trap

-

on

bo

ost

er

no

zzle

s an

d t

hen

fir

st-s

tag

e m

ain

en

gin

e n

ozz

le T

his

rap

id p

itch

in

to a

hig

h

an

gle

of

att

ack

cau

sed

th

e f

air

ing

to

sep

ara

te f

rom

th

e v

eh

icle

du

e to

th

e h

igh

aero

dy

nam

ic l

oad

ing

T

his

cau

sed

th

e o

n-b

oard

self

-des

tru

ct s

yst

em

to

act

iva

te w

hic

h

was

late

r fo

llo

wed

by

a r

an

ge s

afe

ty-i

ssu

ed d

est

ruct

co

mm

an

d

Th

e f

lig

ht

com

pu

ter

pro

gra

m s

hu

tdo

wn

th

e in

ert

ial

pla

tfo

rms

as

pa

rt o

f a g

uid

an

ce r

e-a

lig

nm

ent

rou

tin

e

Th

is r

ou

tin

e i

s p

art

of

the

Ari

an

e 4

reali

gn

men

t ro

uti

ne d

esi

gn

ed

to

all

ow

qu

ick

up

da

tes

aft

er a

ho

ld l

ate

in

th

e c

ou

nt-

do

wn

T

his

ro

uti

ne i

s a

llo

wed

to

op

era

te u

p t

o 4

0 s

eco

nd

s

aft

er l

ift-

off

D

uri

ng

th

is p

eri

od

th

e A

ria

ne

5 t

rav

els

fart

her

do

wn

ra

ng

e th

an

th

e

Ari

an

e 4

an

d t

her

efo

re p

rod

uce

s a

la

rger

ho

rizo

nta

l v

elo

city

co

mp

on

en

t v

alu

e W

hen

this

valu

e r

each

ed

th

e fl

igh

t co

mp

ute

r p

rog

ram

lim

it (

ie n

um

eri

cal

ov

erfl

ow

) w

hic

h i

s

no

t re

ach

ed

by

th

e A

rian

e 4

in

th

e sa

me t

imefr

am

e t

he f

lig

ht

com

pu

ter

shu

tdo

wn

re

set

the

inert

ial

pla

tfo

rms

Du

rin

g d

evelo

pm

en

t o

f th

e av

ion

ics

for

the

Ari

an

e 5

th

e S

exta

nt

Av

ion

iqu

e r

ing

la

ser

gy

ro i

ner

tail

pla

tfo

rms

were

no

t te

sted

in

th

e l

oo

p a

s p

art

of

an

en

d-t

o-e

nd

sim

ula

tio

n

sin

ce t

hey

had

pre

vio

usl

y b

een

pro

ven

to

wo

rk o

n t

he A

rian

e 4

I

nst

ead

th

eir

in

pu

t

was

on

ly s

imu

late

d A

co

mp

lete

hard

ware

-in

-th

e-l

oo

p t

est

wo

uld

lik

ely

hav

e

un

cov

ere

d t

he

syst

em

ic f

ail

ure

of

thes

e A

rian

e-4

deri

ved

in

ert

ial

pla

tfo

rms

in t

he

Ari

an

e 5

fli

gh

t en

vir

on

men

t

As

a r

esu

lt o

f th

is f

ail

ure

E

SA

lo

st t

he

US

$50

0 m

illi

on

Clu

ster

sate

llit

e p

rog

ram

an

d

est

ima

tes

a 2

-4

in

crease

in

co

st o

f th

e U

S $

8 b

illi

on

Ari

an

e 5

pro

gra

m alo

ng

wit

h a

n

ap

pro

xim

ate

on

e-y

ear

dela

y i

n t

he A

rian

e 5

pro

gra

m i

n o

rder

to

per

form

mo

re

hard

wa

re s

imu

lati

on

test

s an

d d

esi

gn

a p

rov

e a

new

gu

idan

ce p

rog

ram

Fig

ure

13

Typ

ical

ana

lysi

s pr

oces

s us

ed b

y in

telli

genc

e an

alys

ts t

o se

arch

a d

ocum

ent

data

base

and

synt

hesi

ze r

esul

ts to

form

ulat

e a

resp

onse

to a

n an

alys

is q

uery

Rep

rinte

d fr

om P

atte

rson

E S

R

oth

E M

amp

Woo

ds D

D P

redi

ctin

g vu

lner

abili

ty in

com

pute

r-su

ppor

ted

infe

rent

ial a

naly

sis

unde

rda

ta o

verlo

ad

Cog

nitio

n T

echn

olog

y amp

Wor

k 3

22

4ndash23

7 C

opyr

ight

200

1 w

ith k

ind

perm

issi

on o

fS

prin

ger

Sci

ence

and

Bus

ines

s M

edia


12


copy1999 Patterson


161

29


22

5


169

15


419

28


7

184


66

14


12

194


4

29




CTA METHODS




































































21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion














RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES



































































































































































































11

Qu

ery

(k

ey

wo

rds

refi

nem

en

t)

Sele

ct ldquo

key

rdquo d

ocu

men

ts

Co

rro

bo

rate

in

form

ati

on

(o

r

reso

lve d

iscr

ep

an

cies

in

info

rmati

on

) an

d f

ill

in g

ap

s

wit

h s

up

po

rt d

ocu

men

ts

Sy

nth

esi

ze i

nfo

rmati

on

to

con

stru

ct a

co

here

nt

sto

ry

Bro

wse

by

tit

le a

nd

o

r d

ate

19

960

80

8

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

19

960

50

0

AR

IAN

E 5

A

N A

LL

NE

W L

AU

NC

HE

R F

OR

TH

E G

LO

BA

L M

AR

KE

T

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

19

960

61

0

AR

IAN

E 5

EX

PL

OD

ES

IN

TH

E S

KY

OV

ER

KO

UR

OU

19

960

60

0

AR

IAN

E 5

FA

ILU

RE

LE

AV

ES

EU

RO

PE

NO

MA

RG

IN F

OR

ER

RO

R

eu

rop

e 1

996

(eu

rop

e 1

996)

amp (

lau

nch

fail

ure

)

(eu

rop

e 1

996)

amp (

(la

un

ch f

ail

ure

)

2)

378

140

22

AR

IAN

E 5

FA

ILU

RE

IN

QU

IRY

BO

AR

D F

IND

ING

S

Da

te

Au

gu

st 8

1996

So

urc

e

FB

IS r

ep

ort

AC

CO

RD

ING

T

O T

HE

R

EP

OR

T R

EC

EN

TL

Y S

UB

MIT

TE

D B

Y T

HE

IND

EP

EN

DE

NT

IN

QU

IRY

B

OA

RD

S

ET

U

P

AF

TE

R T

HE

AR

IAN

E 5

ltlt

LA

UN

CH

gtgt

ltlt

FA

ILU

RE

gtgt

ON

JU

NE

4 (

RE

FT

EL

A

) T

HE

EX

PLO

SIO

N W

AS

DU

E T

O S

PE

CIF

ICA

TIO

N A

ND

DE

SIG

NE

RR

OR

S I

N

TH

E S

OF

TW

AR

E

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M T

HE

RE

PO

RT

FU

RT

HE

R S

TR

ES

SE

D T

HE

F

AC

T T

HA

T

TH

E

ALIG

NM

EN

T F

UN

CT

ION

OF

T

HE

IN

ER

TIA

L

RE

FE

RE

NC

E S

YS

TE

M W

HIC

H

SE

RV

ED

A

P

UR

PO

SE

O

NL

YB

EF

OR

E L

IFT

OF

F (B

UT

R

EM

AIN

ED

O

PE

RA

TIV

E

AF

TE

RW

AR

DS

) W

AS

N

OT

TA

KE

N I

NT

O A

CC

OU

NT

IN

T

HE

S

IMU

LA

TIO

NS

AN

D T

HA

T T

HE

EQ

UIP

ME

NT

AN

DS

YS

TE

M T

ES

TS

WE

RE

N

OT

S

UF

FIC

IEN

TL

Y R

EP

RE

SE

NT

AT

IVE

T

HE

SY

ST

EM

AR

CH

ITE

CT

UR

E W

AS

NO

T IM

PLIC

AT

ED

T

HE

R

EP

OR

T

RE

CO

MM

EN

DE

D E

XT

EN

SIV

ER

EV

IEW

OF

TH

E S

OF

TW

AR

E D

ES

IGN

AN

D T

ES

TIN

G P

RO

CE

DU

RE

S

TH

E C

OS

T O

F C

OR

RE

CT

IVE

M

EA

SU

RE

S I

S E

ST

IMA

TE

D A

T 2

TO

4P

ER

CE

NT

OF

TH

E G

LO

BA

L

CO

ST

O

F T

HE

P

RO

JE

CT

T

HA

T IS

U

SD

150 T

O 3

00

MIL

LIO

N

CO

RR

EC

TIV

E A

CT

ION

IS

E

XP

EC

TE

D T

O P

OS

TP

ON

E T

HE

NE

XT

LA

UN

CH

TO

MID

SE

ME

ST

ER

1997

E

SA

DO

ES

NO

T E

XC

LU

DE

TH

E P

OS

SIB

ILIT

Y T

HA

TT

HE

TH

IRD

AR

IAN

E 5

LA

UN

CH

IN

ITIA

LLY

P

LA

NN

ED

A

S A

C

OM

ME

RC

IAL L

AU

NC

H

MA

Y E

VE

NT

UA

LLY

B

E T

RE

AT

ED

A

S A

Q

UA

LIF

ICA

TIO

N LA

UN

CH

ltlt

Eu

rop

egt

gts

he

avy-

lift la

un

che

r pro

gra

mm

e w

ill c

on

tin

ue

de

spite

th

eJu

ne

4se

tba

ck

bu

t th

e m

arg

in fo

r err

or

ha

s n

ow

be

en

re

du

ce

d t

o z

ero

Th

e lo

ss

of th

e fir

st A

ria

ne

5 o

n it

s in

itia

l q

ualif

ica

tio

n flig

ht

on

Jun

e 4

ma

rks a

se

ve

re b

ut

by

no

me

an

s fa

tal s

etb

ack

fo

r lt

ltE

uro

pe

gtgt

sn

ew

he

avy-

lift

laun

che

r a

nd

for

Ari

an

esp

ace

s a

mb

itio

ns t

o c

on

solid

ate

its

do

min

an

t p

osi

tio

n o

n t

he

glo

bal co

mm

erc

ial la

un

ch m

ark

et

It

will

take

mo

re t

ha

n a

sin

gle

ltlt

lau

nch

gtgt

ltlt

failu

regt

gt t

o d

era

il th

e 1

5-y

ea

r $

82

bill

ion

pro

gra

mm

e

bu

t th

e m

arg

in f

or

err

or

ha

s h

en

ce

fort

h b

een

red

uce

d to

zero

Th

e 4 J

un

e 1

996

la

un

ch o

f th

e fi

rst

Eu

rop

ean

Sp

ace

Ag

en

cy (

ES

A)

Ari

an

e 5

sp

ace

lau

nch

veh

icle

carr

yin

g t

he f

ou

r E

uro

pean

ldquoC

lust

errdquo

sci

en

tifi

c sa

tell

ites

en

ded

in

fail

ure

ju

st

ov

er

36 s

eco

nd

s aft

er l

ift-

off

du

e t

o i

nh

eri

t so

ftw

are

des

ign

fla

w i

n t

he g

uid

an

ce s

yst

em

Th

e im

bed

ded

so

ftw

are

in

bo

th o

f th

e p

rim

ary

an

d b

ack

-up

id

enti

call

y-d

esi

gn

ed

inert

ial

gu

idan

ce p

latf

orm

s c

urr

en

tly

pro

ven

an

d u

sed

on

bo

ard

th

e A

rian

e 4

w

as

ill-

desi

gn

ed

fo

r th

e f

lig

ht

pro

file

of

the

Ari

an

e 5

I

nsu

ffic

ien

t re

qu

irem

en

ts a

nd

test

ing

of

the

Ari

an

e 5 a

vio

nic

s sy

stem

du

rin

g d

ev

elo

pm

en

t d

id n

ot

un

cov

er t

he d

esi

gn

fla

w

Th

e acc

elera

tio

n o

f th

e A

rian

e 5

is

mu

ch g

reate

r th

an

th

e A

rian

e 4

aft

er l

ift-

off

T

his

cau

sed

th

e i

ner

tial

gu

idan

ce p

latf

orm

em

bed

ded

so

ftw

are

w

hic

h w

as

des

ign

ed

aro

un

d

the

slo

wer

Ari

an

e 4

acc

ele

rati

on

pro

file

to

pro

vid

e n

um

eri

cal

va

lues

bey

on

d t

he

pro

gra

mm

ed

lim

its

of

the

flig

ht

com

pu

ter

wh

ich

th

en

sh

utd

ow

n b

oth

in

erti

al

pla

tfo

rms

Th

e p

latf

orm

s in

itia

ted

a d

iag

no

stic

ldquore

setrdquo

mo

de

tha

t fe

d i

nco

rrec

t v

alu

es t

o t

he

flig

ht

com

pu

ter

wh

ich

th

en c

om

man

ded

an

ex

cess

ive

pit

ch-o

ver

by

gim

ball

ing

th

e tw

o s

trap

-

on

bo

ost

er

no

zzle

s an

d t

hen

fir

st-s

tag

e m

ain

en

gin

e n

ozz

le T

his

rap

id p

itch

in

to a

hig

h

an

gle

of

att

ack

cau

sed

th

e f

air

ing

to

sep

ara

te f

rom

th

e v

eh

icle

du

e to

th

e h

igh

aero

dy

nam

ic l

oad

ing

T

his

cau

sed

th

e o

n-b

oard

self

-des

tru

ct s

yst

em

to

act

iva

te w

hic

h

was

late

r fo

llo

wed

by

a r

an

ge s

afe

ty-i

ssu

ed d

est

ruct

co

mm

an

d

Th

e f

lig

ht

com

pu

ter

pro

gra

m s

hu

tdo

wn

th

e in

ert

ial

pla

tfo

rms

as

pa

rt o

f a g

uid

an

ce r

e-a

lig

nm

ent

rou

tin

e

Th

is r

ou

tin

e i

s p

art

of

the

Ari

an

e 4

reali

gn

men

t ro

uti

ne d

esi

gn

ed

to

all

ow

qu

ick

up

da

tes

aft

er a

ho

ld l

ate

in

th

e c

ou

nt-

do

wn

T

his

ro

uti

ne i

s a

llo

wed

to

op

era

te u

p t

o 4

0 s

eco

nd

s

aft

er l

ift-

off

D

uri

ng

th

is p

eri

od

th

e A

ria

ne

5 t

rav

els

fart

her

do

wn

ra

ng

e th

an

th

e

Ari

an

e 4

an

d t

her

efo

re p

rod

uce

s a

la

rger

ho

rizo

nta

l v

elo

city

co

mp

on

en

t v

alu

e W

hen

this

valu

e r

each

ed

th

e fl

igh

t co

mp

ute

r p

rog

ram

lim

it (

ie n

um

eri

cal

ov

erfl

ow

) w

hic

h i

s

no

t re

ach

ed

by

th

e A

rian

e 4

in

th

e sa

me t

imefr

am

e t

he f

lig

ht

com

pu

ter

shu

tdo

wn

re

set

the

inert

ial

pla

tfo

rms

Du

rin

g d

evelo

pm

en

t o

f th

e av

ion

ics

for

the

Ari

an

e 5

th

e S

exta

nt

Av

ion

iqu

e r

ing

la

ser

gy

ro i

ner

tail

pla

tfo

rms

were

no

t te

sted

in

th

e l

oo

p a

s p

art

of

an

en

d-t

o-e

nd

sim

ula

tio

n

sin

ce t

hey

had

pre

vio

usl

y b

een

pro

ven

to

wo

rk o

n t

he A

rian

e 4

I

nst

ead

th

eir

in

pu

t

was

on

ly s

imu

late

d A

co

mp

lete

hard

ware

-in

-th

e-l

oo

p t

est

wo

uld

lik

ely

hav

e

un

cov

ere

d t

he

syst

em

ic f

ail

ure

of

thes

e A

rian

e-4

deri

ved

in

ert

ial

pla

tfo

rms

in t

he

Ari

an

e 5

fli

gh

t en

vir

on

men

t

As

a r

esu

lt o

f th

is f

ail

ure

E

SA

lo

st t

he

US

$50

0 m

illi

on

Clu

ster

sate

llit

e p

rog

ram

an

d

est

ima

tes

a 2

-4

in

crease

in

co

st o

f th

e U

S $

8 b

illi

on

Ari

an

e 5

pro

gra

m alo

ng

wit

h a

n

ap

pro

xim

ate

on

e-y

ear

dela

y i

n t

he A

rian

e 5

pro

gra

m i

n o

rder

to

per

form

mo

re

hard

wa

re s

imu

lati

on

test

s an

d d

esi

gn

a p

rov

e a

new

gu

idan

ce p

rog

ram

Fig

ure

13

Typ

ical

ana

lysi

s pr

oces

s us

ed b

y in

telli

genc

e an

alys

ts t

o se

arch

a d

ocum

ent

data

base

and

synt

hesi

ze r

esul

ts to

form

ulat

e a

resp

onse

to a

n an

alys

is q

uery

Rep

rinte

d fr

om P

atte

rson

E S

R

oth

E M

amp

Woo

ds D

D P

redi

ctin

g vu

lner

abili

ty in

com

pute

r-su

ppor

ted

infe

rent

ial a

naly

sis

unde

rda

ta o

verlo

ad

Cog

nitio

n T

echn

olog

y amp

Wor

k 3

22

4ndash23

7 C

opyr

ight

200

1 w

ith k

ind

perm

issi

on o

fS

prin

ger

Sci

ence

and

Bus

ines

s M

edia


12


copy1999 Patterson


161

29


22

5


169

15


419

28


7

184


66

14


12

194


4

29




CTA METHODS




































































21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion














RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES



































































































































































































12


copy1999 Patterson


161

29


22

5


169

15


419

28


7

184


66

14


12

194


4

29




CTA METHODS




































































21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion














RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES





































































































































































































CTA METHODS




































































21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion














RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES
































































































































































































































































21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion














RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES
























































































































































































































































21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion














RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES

















































































































































































































































21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion














RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES











































































































































































































































21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion














RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES


































































































































































































































21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion














RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES


























































































































































































































21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion














RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES

















































































































































































































21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion














RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES



































































































































































































21

Fig

ure

16

Atim

elin

e re

pres

enta

tion

of n

urse

act

iviti

es il

lust

ratin

g th

e hi

gh n

umbe

r of

inte

rrup

tions

(indi

cate

d by

arr

ows)

tha

t nu

rses

mus

t co

pe w

ith d

urin

g m

edic

atio

n ad

min

istr

atio

n (P

atte

rson

C

ook

amp R

ende

r 20

02)

Rep

rinte

d w

ith p

erm

issi

on f

rom

the

Am

eric

an M

edic

al I

nfor

mat

ics

Ass

ocia

tion














RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES















































































































































































































RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES










































































































































































































RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES



































































































































































































RELATED APPROACHES













Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES











































































































































































































Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES



































































































































































































Fig

ure

11

0E

xam

ple

of a

hie

rarc

hica

l tas

k an

alys

is f

or t

he t

ask

of fi

ndin

g pa

pers

usi

ng a

nel

ectr

onic

dat

abas

e

26



















































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES




















































































































































































































































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES












































































































































































































































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES



































































































































































































































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES



























































































































































































































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES



















































































































































































































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES











































































































































































































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES



































































































































































































RESEARCH FRONTIERS


















CONCLUSIONS


REFERENCES












































































































































































































CONCLUSIONS


REFERENCES




































































































































































































CONCLUSIONS


REFERENCES



































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































Documents

Reviews of Human Factors and Ergonomics - … Reviews of Human Factors and Ergonomics, Volume 3 Figure 1.1. A cognitive analysis requires consideration of two perspectives: examination