Looking for errors of omission and commission or The Hunting of the Snark revisited

Looking for errors of omission and commission orThe Hunting of theSnarkrevisited

E. Hollnagel*

Department of Computer Science and Informatics, University of Linko¨ping, S-581 83 Linko¨ping, Sweden

Received 2 March 1999; accepted 11 January 2000

Abstract

Since the early 1990s, considerable effort has been spent to understand what is meant by an “error of commission” (EOC), to complementthe traditional notion of an “error of omission” (EOO). This paper argues that the EOO–EOC dyad, as an artefact of the PSA event tree, isinsufficient for human reliability analysis (HRA) for several reasons: (1) EOO–EOC fail to distinguish between manifestation and cause; (2)EOO–EOC refer to classes of incorrect actions rather than to specific instances; (3) there is no unique way of classifying an event usingEOO–EOC; (4) the set of error modes that cannot reasonably be classified as EOO is too diverse to fit into any single category of its own.Since the use of EOO–EOC leads to serious problems for HRA, an alternative is required. This can be found in the concept of error modes,which has a long history in risk analysis. A specific system for error mode prediction was tested in a simulator experiment. The analysis of theresults showed that error modes could be qualitatively predicted with sufficient accuracy (68% correct) to propose this method as a way todetermine how operator actions can fail inPSA-cum-HRA. Although this still leaves the thorny issue of quantification, a consistent predictionof error modes provides a better starting point for determining probabilities than the EOO–EOC dyad. It also opens a possibility forquantification methods where the influence of the common performance conditions is prior to and more important than individual failurerates.q 2000 Published by Elsevier Science Ltd. All rights reserved.

Keywords: Error of omission; Error of commission; Error mode; Performance prediction; Human reliability analysis; Common performance conditions

They sought it with thimbles, they sought it with care;They pursued it with forks and hope;They threatened its life with a railway-share;They charmed it with smiles and soap.

Lewis Carroll (1832–1898) “The Hunting of theSnark”

1. The modelling problem

The noble goal of Human Reliability Analysis (HRA)is to determine how probable it is that operatorsperform incorrectly one or more of the actions requiredin response to an event. That need, in turn, derives fromthe structure of the PSA event tree, which is thecommon way of representing the event or scenario.HRA can therefore be seen as a way of expanding thenodes representing human actions, corresponding to thefault tree expansion of technologically based events. I shall

not on this occasion go into arguments about the appropri-ateness of the PSA event tree representation, or whether thewholePSA-cum-HRAperspective is sensible or reasonable.For the sake of this discussion I will simply accept the PSAevent tree and thePSA-cum-HRAapproach as given, and asconstituting the conditions under which HRA is performed.Furthermore, throughout this paper the term HRA does notrefer to any specific method, but rather to the endeavour tofind, by whatever means possible, the probability that theaction or task corresponding to a node of the event tree willfail. The situation to be discussed can therefore be repre-sented as in Fig. 1, where Step3 of the stylised event treerepresents an operator action.

The history of HRA is marked with a substantial numberof attempts to deliver the coveted “human error” probabil-ity, and a few of these are named in Fig. 1. These attemptsrepresent widely different ways of looking at the problem, ofwhich the following are typical:

• Fundamental calculations based on rudimentarypsychological assumptions about the nature of operatorresponses, such as the TRC [1] and THERP [2].

• Direct estimation of the probability by subject matterexperts, such as SLIM [3].

Reliability Engineering and System Safety 68 (2000) 135–145

0951-8320/00/$ - see front matterq 2000 Published by Elsevier Science Ltd. All rights reserved.PII: S0951-8320(00)00004-1

www.elsevier.com/locate/ress

* Fax: 1 81-774-38-4406.E-mail address:[email protected] (E. Hollnagel).

• More elaborate models or classifications, such as HCR[4] and HEART [5].

• More recently, very detailed and explicit theories andmodels, typical of the second generation HRA methods,such as ATHEANA [6], CREAM [7] or MERMOS [8].

These attempts differ in many ways, but for the currentdiscussion it is especially interesting to consider (1)howincorrectly performed actions are classified and described,and (2)how the probability estimates are produced. In theend, since everybody more or less openly agrees that prob-abilities are not nearly as precise as they should be, itmatters more what they are about and how they are derived.The qualitative outcome of the human reliability analysis istherefore more important than the quantitative.

1.1. Categories of “actions gone wrong”

It is clearly important to have meaningful categories todescribe how actions can go wrong, which means howoperators can fail to carry them out correctly. It may evenbe seen as more important than the methods used to generatethe probabilities, since the methods to a considerable extentdepend on the categories. The methods produce a number,or a range, but the categories determine and justify themeaning of the number, i.e. what the probability represents[9]. In this respect, the representation of possible actionoutcomes provided by the PSA event tree is a liability,because the binary nature of the event tree dictates thatthere are only two possible outcomes: something can eitherbe done correctly (a success) or done incorrectly (a failure).

This is clearly reflected by the early approaches to HRA,such as the time-reliability correlation (TRC), whichaddress the probability ofnot performing an action withina certain time interval. Although the event tree does not initself imply that the failure branch from a node is an “errorof omission”, it is all too easy to interpret it as such, hence toconsider the binary outcomes as representing either acorrectly performed action or an omission of the same.

This tendency is reinforced by the inherent meaningful-ness of an “error of omission” as a phenomenologicalcategory. Classical human factors approaches to “humanerror” and HRA have, for instance, put great emphasis ona categorisation of the external manifestations of incorrectactions, hence invariably included the category of an “errorof omission”. It does makes immediate sense that a personcan either do something or not do it. This is a phenomenonwe have the opportunity to observe every day, if not inourselves then at least in others (who may say the same,of course). We may forget to do things for any number ofreasons, and as a result find ourselves in situations andconditions that we did not want and did not anticipate [10].

Practically all proposals for categorisations of humanactions, whether for the purpose of “error analysis” orHRA, use multiple categories of incorrectly performedactions—sometimes even rather many. At the very least, adistinction is made between whether a person respondsappropriately, inappropriately, or not at all. The problemwith the traditional form of the event tree is that it onlyrepresents binary outcomes, hence can only distinguishbetween two types of actions—correctly performed andincorrectly performed. An omission is certainly an exampleof an incorrectly performed action, but is by no means theonly one. In addition to looking at the probability of notperforming an action, HRA has a need to consider the prob-ability of performing the wrong type of action. Although thehistory of the term is unclear, it seems plausible that notionof an “error of commission” came about as a response to thisneed. The simple binary distinction between an “error ofomission” (EOO) and “error of commission” (EOC) canin the same vein be seen as an artefact of the event treerepresentation, cf. Fig. 2. The first two nodes, “actionperformed?” and “correct action?”, are sufficient to providethe main categories of EOO and EOC. The third node,“correct execution?”, introduces the distinctionbetween a correct and incorrect execution of a correctaction.

The need of PSA is to get a single number, or a range, thatrepresents the probability of a failed action as a node in theevent tree. The crucial question is what the number repre-sents. The above introduction to the EOC as a category hasalready shown that an EOC is not a single or uniform desig-nation. In this paper I will try to argue that, indeed, neitherEOO nor EOC are meaningful categories within theframework ofPSA-cum-HRA[11], and that their use there-fore should be avoided. The consequences of following thisrecommendation are very limited, since perfectly good

E. Hollnagel / Reliability Engineering and System Safety 68 (2000) 135–145136

Fig. 1. The role of HRA in PSA.

Fig. 2. A pseudo-event tree for EOO–EOC.

alternatives are available. The argument receives supportfrom by the fact that no HRA methods use EOO and EOCas the only categories of performance failures. It seems thatthe EOO–EOC dyad is used more to talk about HRA, andthe event tree, than to do it.

2. Omission and commission as “error” categories

Whereas the notion of an EOO was present from the verybeginning of HRA, it appears that the notion of an EOC didnot gain popularity until the 1990s. One reason for thechange was undoubtedly Ed Dougherty’s influential reviewof the then current HRA methods, which listed a number ofinadequacies and shortcomings [12]. Another reason wasthe growing realisation that operators could not be describedsimply as if they were well-intending information pro-cessing mechanism where component processes now andthen might fail. This led to a need to account for the oper-ators’ cognitive functions and the role of cognition in humanerroneous actions [13].

When the EOO and EOC are considered as categories oferroneous actions, they are normally defined as follows:

• An (error of) omission is the failure to carry out some ofthe actions necessary to achieve a desired goal.

• An (error of) commission is carrying out an unrelatedaction, which prevents the achievement of the goal.

These definitions, however, lead to a number of problems,which are discussed in more detail in the following.

2.1. Distinguishing between causes and manifestations

The first problem is that the very notion of an errorimplies the existence of a cause. (In addition, the notionof an error is fundamentally ambiguous, since the term isused to refer to either the cause, the event, or the outcome.)Specifically, the naming of EOO and EOC seems to implythat they are causes, for instance, when it is said that anoperator has made (or can make) an “error of commission”.As argued elsewhere [14,15], a cause is an attribution afterthe fact or a judgement in hindsight, i.e. a cause is a socialjudgement rather than an objective fact. The very sameaction may either have the intended effects or lead tounwanted consequences, depending on such diverse factorsas the latent system conditions, the current working envir-onment, the common performance conditions, the variabil-ity of human performance, the occurrence of unexpectedexternal events, etc. It is, however, only when the unwantedconsequences occur that we begin to look for an “error” thatcan serve as a cause. In other words, the cause only becomesimportant when the effect has been recognised.

While an EOO in hindsight may be seen as the cause of anoutcome, it is by itself a manifestation—or rather, thelackof an observable event. According to the general principle ofcausality, any manifestation, including the failure to dosomething or the omission of an action, must have a

cause. The EOO is therefore a manifestation as well as acause, and should properly be considered as a phenotyperather than a genotype [14]. The same, of course, goes forthe notion of an EOC, except that this represents a morecomplex case. An EOC is, according to the definition, anaction unrelated to the current goal, hence clearly aphenotype or manifestation. It is therefore conceptuallymisleading to call either an EOO or an EOC for an “errorof…” since neither are errors in the sense of being causes.The situation is sometimes made worse by confoundingEOC and “cognitive error”, where the latter in itself is aterm of dubious reputation [7]. As a practical technique,HRA is unquestionably more interested in manifestationsthan in causes, since the concern is with events that mayhappen in the future. The very fact that the approach has aprobabilistic rather than a deterministic basis also meansthat causality is less of an issue than phenomenology. Yetthe notions of EOO and EOC do not semantically distin-guish between the concepts of cause and effect, and their useshould for that reason alone be avoided.

The distinction between causes and manifestationsreflects a parallel distinction between error types anderror modes. Anerror type is a category that is basedon and derives its meaning from an underlying model ofhuman action—usually of human information proces-sing or “cognition in the mind”. Well-known examplesof model-defined error types range from EOO–EOC toskill-based, rule-based, and knowledge-based lapses andmistakes. Error types are linked to a specific model ofthe processes underlying human action, and how theseprocesses mail fail. In contrast to that, anerror mode—or human failure mode—refers to a description ofobservable manifestations. Error modes may even belogically defined by referring to the small number ofphysically possible failures [16], cf. Table 1. As anexample, “action too late” is an error mode that refersto the aspect of timing of an action. It is an overtmanifestation and does not in itself make any assump-tions of what lay behind it. Yet performing an actiontoo late may also be described in the language of errortypes, i.e. referring to what the underlying cause areassumed to be. Depending on the theoretical stance ofthe analyst, it may be described as an error of commis-sion or as a rule-based mistake. For a retrospectiveaccident analysis it may be important to construct anacceptable explanation for the conditions and causes ofan accident, hence to focus on error types. For a performanceprediction, such as an HRA, it is more important to identify thekinds of incorrect actions that can occur, regardlessof what thecauses may be, hence to focus on error modes.

2.2. Classes and instances

Even when the terms EOO–EOC are used to denotemanifestations (phenotypes) rather than causes (genotypes),the terms provide a wholesale rather than a specific

E. Hollnagel / Reliability Engineering and System Safety 68 (2000) 135–145 137

description. Both EOO and EOC are names of classes or setsof distinct and characteristic error modes, which constitutethe instances. Since the instances describe different types ofaction failures with different consequences, it follows thatpredictions should be about the instances rather than aboutthe class. For instance, entering the wrong set-point valuefor regulator A and entering the correct set-point value forregulator B (instead of regulator A) may both be classifiedas an EOC. (Additional examples of instances of EOO and

EOC will be given in the following.) Both actions will beincorrect relative to what was required, but they may havesignificantly different consequences. Predicting that an EOCmay occur is of little value, unless it can also be predictedwhich type of EOC it will be. In other words, the predictionshould be about the specific instances rather than about theclass. The consequence of this is that since both EOO andEOC are names of classes rather than instances, neithershould be used as the target for performance predictionand HRA. Furthermore, since it is more important toknow about the instances than the class, the predictionshould be qualitative rather than quantitative.

2.3. Omissions and time windows

Following the definition given above, an omission is thefailure to carry out some of the actions necessary to achievea desired goal. This failure must, however, be relative to atime window (or window of opportunity) as recognisedalready by the TRC [1]. An omission should therefore bedefined as the failure of carrying out an action during thetime window when it was required. In normal cases, thebeginning and the end of the time window (earliest startingtime and latest finishing time [17], are determined by theprocess in question, whether it is landing an aeroplane orestablishing emergency core cooling). As shown in Fig. 3,this means that an omission could be an action that wascarried out too early, an action that was carried out toolate, or an action that was not carried out at all. SomeHRA approaches propose a distinction between an“omission [of an action] at large” and an “omission [of anaction] in time”. The difference between these is that theformer does not define a time window, but rather impliesthat it is the duration of the event. Phenomenologically, thetwo forms of EOO are not different at all, unless an“omission in time” implies that the action is carried tooearly or too later. In that case it would, however, be moreappropriate to apply either the error mode of “action donetoo early” or “action done too late”.

In most HRA methods, an EOO is considered only as anaction done too late, i.e. as the third of the four categories


Fig. 3. EOO and time windows.

Table 1Basic human error modes

Error mode Specific effects Definition/explanation

Timing Too early An action started too early, before asignal was given or the requiredconditions had been established(premature action)

Too late An action started too late (delayedaction)

Omission An action that was not done at all(within the time interval allowed)

Duration Too long An action that continued beyond thepoint when it should have stopped

Too short An action that was stopped before itshould have been

Force Too little Insufficient forceToo much Surplus force, too much effort

Distance/ Too far A movement taken too farMagnitude Too short A movement not taken far enoughSpeed Too fast Action performed too quickly, with

too much speed or finished too earlyToo slow Action performed too slowly, with

too little speed or finished too lateDirection Wrong direction Movement in the wrong direction,

e.g. forwards instead of backwardsor left instead of right

Wrong movement The wrong kind of movement, suchas pulling a knob instead of turningit

Wrongobject

Neighbour An object that is in physicalproximity to the object that shouldhave been used

Similar object An object that is similar inappearance to the object that shouldhave been used

Unrelated object An object that was used by mistake,even though it had no obviousrelation to the object that shouldhave been used

Sequence Omission An action that was not carried out.This includes in particular theomission of the last action(s) of aseries (interruption)

Jump forward One or more actions in a sequencewere skipped

Jump backwards One or more actions that have beencarried out, are carried out again

Repetition The previous action is repeatedReversal The order of two neighbouring

actions is reversedWrong action An extraneous or irrelevant action is

carried out (action inertia)

shown in Fig. 3. This is quite consistent with the principle ofthe TRC, which considers the probability of not performingan action within a given time interval relative to the onset ofthe initiating condition, such as an alarm. In practise, there-fore, the EOO depends on whether the action is done beforea given time period has elapsed. An action done after thelatest finishing time, whether done 10 min later (“omissionin time”) or not at all (“omission at large”), is an action donetoo late. An “action too late”, as well as an “action tooearly”, will effectively constitute an EOC relative to othertime windows, as well as an EOO relative to the current timewindow. This leads to a terminological conundrum, which isdiscussed in the following section.

2.4. The dependency between EOO and EOC

On closer inspection the classes of EOO and EOC turnout to be mutually dependent. Consider, for instance, thefollowing situation (Fig. 4), which describes the expectedoccurrence of two task steps Tl and Tm.

In situation (A) both task steps, Tl and Tm, are carried outduring the time window where they are required and further-more in the right sequence. In situation (B), the task step Tl

is carried out during the time window where the task step Tm

was expected. If we consider the preceding time windowwhere task step Tl was expected, then nothing happened.This is therefore clearly a case of omission (of Tl). If,however, we consider the time window where task stepTm was expected, then Tl occurred instead. Relative to thefirst time window, Tl is carried out too late, corresponding toa case of omission or delay. Relative to the second timewindow and Tm, there are several ways of describing thesituation:

• If the delayed execution of Tl prevents the achievementof the goal then, by definition, it is an EOC. There are,however, also two cases of EOO, because neither Tl norTm were carried out during the time windows when theywere required.

• Even if the delayed execution of Tl does not in itselfprevent the achievement of the goal, then the failure tocarry out Tm during the appropriate time window may do

so, i.e. there is an omission of Tm. The reason for thisEOO is the delayed execution of Tl, which corresponds toan EOC of Tl.

This example shows that it is an oversimplification tocategorise a single task step in isolation as either an EOOor an EOC. By using the class names as if they weremutually exclusive, important information is neglectedand the analysis is incomplete. An instance of the classEOC is always, by definition, an instance of the classEOO. Even a cursory analysis shows that the class EOCincludes: (1) correct actions executed at the wrong time;(2) incorrect actions executed instead of correct actionsbut at the right time; and (3) incorrect actions carried outat the wrong time. To refer to the class of EOC instead of theinstances is helpful neither for accident analysis nor forperformance prediction.

2.5. The multiplicity of “errors of commission”

The last problem considered here is that the EOO–EOCdyad is inadequate to account for all the possible manifes-tations of incorrectly performed actions. As discussed else-where [7,16], these manifestations can be described indifferent ways such as simple error modes, logical pheno-types, or cognitive functions. Only the first of these will beused here as an illustration.

Consider, for instance, the systematic description ofhuman error modes, i.e. the ways in which incorrect actionscan manifest themselves or the ways in which they can beobserved. The eight basic error modes (Table 1) provide thenecessary and sufficient basis for classifyinganyincorrectlyperformed action, and each can be described in furtherdetail. For example, “incorrect timing” can be specifiedfurther as “action too early” or “action too late”—whichboth can be seen as examples of an omission. Most of theother categories are equally simple to expand, since theyrefer to physical or temporal characteristics of actions.One of the more complex error modes relates to actionscarried out in the wrong sequence, where the followingcases can be distinguished:

• An omission of one or more steps. This is a special caseof a jump forward in the sequence of actions.

• A repetition of a previous action. This is a special case ofa jump backwards in the sequence of actions, which alsomay be considered as a restart.

• A reversal of two steps or sub-sequences of actions.• Carrying out a wrong or unrelated action (an intrusion).• Action inertia, i.e. the actions continue beyond the last

step of a sequence.• Interruption, where a sequence of actions is prematurely

ended. The special case is that of omitting the last actionof a sequence.

In terms of manifestations, and in terms of the consequencesof the manifestations, the error mode of “sequence” isclearly so complex that the EOO–EOC dyad is inadequate.


Fig. 4. Dependencies between EOO and EOC.

In the control of a process, or in driving around in a city, thereversal of two actions may lead to a completely differentoutcome from repeating an action, omitting an action, etc.They should therefore be referred to as specific instancesrather than by the class EOC.

In relation to the error modes, an EOO can either be anaction that is out of sequence or an action that is carried outat the wrong time. The two descriptions are logically depen-dent, but not completely synonymous. More importantly,the two instances of the EOO class usually have completelydifferent consequences. The same type of argument can berepeated for EOC, but with even more force. The class ofEOC apparently serves as a receptacle for a number ofdiverse and potentially incompatible instances, whichhave in common only that they arenot EOOs. Even if theterminological confusion could be written off as anacademic dispute, the fact remains that different manifes-tations may have very different consequences—e.g. activat-ing something for too short a duration versus pressing thewrong button. Since the differences in outcome are essentialfor the PSA, it follows that the manifestations—i.e. thedifferent error modes or the different instances of EOCs—should also be treated differently. This can only be achievedby abandoning the classes EOO and EOC and instead usethe specific instances to more precisely distinguish betweenthe different error modes.

2.6. Summing up

The discussion above has presented five different argu-ments why the EOO–EOC dyad is insufficient for HRA.Firstly, the notions of “error of omission” and “error ofomission” suffer from the common weakness of manyerror classification schemes, by failing to distinguish clearlybetween manifestation and cause. The term “human error”is itself a good—or rather, bad—example of that [7].Secondly, the terms EOO and EOC refer to classes of incor-rect actions; performance prediction should address theinstances of the classes, rather than the classes themselves.Thirdly, the notion of an omission should be used in a rela-tive rather than an absolute sense, i.e. with reference to timewindows. It was demonstrated how the class EOO containsat least three phenotypes: “action too early”, “action toolate”, and “omission”. Fourthly, the classes EOO andEOC are mutually dependent, in the sense that an instanceof the class EOC always implies an instance of the classEOO. There is therefore no single or unique way of classi-fying an event, using the class names themselves. Finally,the set of error modes that are not instances of the class EOO(even accepting the imprecision of that category) is toodiverse to fit into any single class of its own. The classEOC, as being the complement of the class EOO, is mainlya container for a number of diverse action types. It is,however, quite misleading to use a single term for themall, since it hides the fact that different (incorrect) actionsmay have widely different consequences—something,

which clearly is important for PSA. Since both EOO andEOC lead to serious problems for HRA, and since there is nogood reason why the terms should be used in the first place,the inescapable conclusion is that it makes little sense to goon using them.

3. Action error modes

If the above conclusions are accepted, then HRA is inserious need of an alternative way to describe incorrectlyperformed actions. Fortunately, the search for an alternativedoes not have to look very far, but can end almost the momentit starts. The solution is to return to the PSA event tree andconsider what HRA is really about. As noted in the introduc-tion (with reference to Fig. 1), HRA is the endeavour to deter-mine, by whatever means possible, the probability that theaction or task corresponding to a node of the event tree willfail. This can be stated more precisely as being composed oftwo steps. The first is to determinewhich typesof incorrectactions can occur and the second is to calculatehow probablethey are. As I haveargued above,using the EOO–EOC dyad toclassify incorrect actions is a gross oversimplification.The question is therefore which classification shouldbe used instead?

If the attention for a while is turned away from PSA torisk analysis in a wider sense, practitioners of that noble arthave for many years referred to concepts such as failuremodes and error modes. Indeed, one of the major analysistechniques is called Failure Mode and Effects Analysis(FMEA; or FMECA) [18]. This method is used to determinethe probability that a system will function without failure ina given time interval. The analysis begins by listing all thecomponents of the system, the ways in which they can fail(the failure modes) and how the failures may manifest them-selves. For each failure mode, the consequences of a failurefor other components and for the system as a whole aredetermined, following which the seriousness (and probabil-ity) of the failure modes are assessed. Other well-knownapproaches to risk analysis that use the notion of afailure mode are fault trees and cause-consequencetrees [19].

Since all these techniques have been developed in thecontext of risk analysis and reliability engineering, theirmain focus is on the technological system components.There is, however, no reason why one should not applythe same approach to consider the possible failure modesof humans—without, of course, going so far as consideringthe human as just another system component. In a recentsurvey of “human error” classification schemes it was foundto be the norm, rather than the exception, to use a richvocabulary of failures modes [7]. Practically no one, fromAltman [20] and onwards, have found the simple omission–commission classification sufficient for accident analysis. Ittherefore seems rather inexplicable that it was ever consid-ered appropriate for HRA.


3.1. Phenotypes and genotypes

The concept of human error modes provides precisely theinformation that is needed by aPSA-cum-HRA, i.e. a way ofdescribing which types of incorrect actions that may occurat the specific node of the event tree. The error modeconcept unfortunately does not solve the irksome problemof quantification, but does at least make it clearwhat it isthat needs to be quantified—regardless of the methods usedto do it. The concept of an error mode also avoids thesemantic pitfall in using terms such as EOO and EOC,since an error mode clearly is an incorrectly performedaction rather than a cause [15].

An error mode is closely related to the concept of aphenotype [14], which is defined as the overt and observablemanifestations of an incorrectly performed action. Thephenotype is seen in contrast to the genotype, which refersto the set of causes that is deemed sufficient to explain afailed action or accident. The importance of distinguishingbetween the two categories is that the phenotypes describeswhat can be observed, regardless of what the possible causesmay be, whereas the genotype describes the likely set ofcauses that in the given situation are necessary and sufficientto account for the phenotype. The purpose of an accidentanalysis is to search for the genotype, i.e. to construct inhindsight the most plausible explanation for the observedevent. In HRA, which is concerned with performanceprediction, the genotype is of less importance. The pheno-type, however, is precisely what is needed since the first stepof the HRA is to determine which types of incorrect actionscan occur.

As far as finding a set of error modes goes, the task ismade simple by the fact that there only is a limited numberof ways in which something can be done incorrectly(surprising as this may seem to some). The eight basicerror modes were described in Table 1. Having establishedthe basic classification of error modes, the question iswhether it is possible in practice to use this to expand thenodes of the PSA event tree that represent human action,and thereby provide a proper basis for the quantification.One part of the question is whether the error modes arereasonable as categories of description. Another is whetherit is possible to predict the error modes with a reasonable

degree of certainty. Fortunately, a positive answer can begiven to both parts.

3.2. Error mode prediction in the human error analysisproject

The OECD Halden Reactor Project (HRP) in Norway hasin the years 1994–1999 been engaged in a long term effortto study human erroneous actions (the Human Error Analy-sis Project, hereafter called HEAP [21–23]. The purposes ofthis project was to provide a better understanding andexplicit modelling of how and why erroneous actionsoccur, and to provide improved design guidance for thedevelopment of human–machine systems that can avoidor compensate for erroneous actions.

In 1996, an experiment was performed to evaluate aspecific method for performance prediction, and specificallyto look at error mode prediction. The overall purpose ofperformance prediction is to describe how a scenario maypossibly develop, given the existing working conditions. Inmany cases the representation of a scenario only providesthe basic structure of the events but leaves out the detailedconditions that may influence how an event develops. Inorder to make the prediction, the scenario descriptionmust therefore be supplemented by information about theconditions or factors that can influence the propagation ofevents. One of these is the variability of human perfor-mance, which in itself depends on the general performanceconditions—including the previous developments (cf. Fig.5). Performance prediction must therefore describe thelikely context before it goes on to consider the actionsthat may occur.

The method used in HEAP was based on CREAM, theCognitive Reliability and Error Analysis Method [7]. Themethod enables the analysts to achieve the following:

• identify the types of incorrect performance (error modes)that are possible for the given task or scenario;

• qualitatively rank or rate the likelihood of the possibleerror modes, to identify those that are the more likely tohappen.

In the present case the purpose of the performance predic-tion was to identify the kinds of performance failures that


Fig. 5. Basic dependencies in performance prediction.

could occur, expressed in terms of specific error modes.Since the error modes basically are deviations fromexpected performance, the starting point for making aprediction of this type must be a description of expectedperformance, provided by, e.g. task analysis, operatingprocedures, ideal paths or performance time-lines, orevent trees. The basic steps of the prediction method canbe summarised as follows:

• Event sequences were constructed using a task analysis,which decomposed the main goal into subgoals and iden-tified the associated tasks.

• The Common Performance Conditions were describedfor each scenario and for their major segments or sub-parts. The latter was necessary as it was expected thatthere would be significant differences betweensegments—e.g. early in an accident and later. Thesegmentation of the scenarios was done with the supportof process matter experts, who also ensured that thedescriptions were on the same level of detail for allsegments.

• The actions within a performance segment weredescribed using a set of standard categories, called acognitive activity list. The list is derived from accumu-lated experience from operator performance studies,hence has an empirical rather than an analytical basis.Each action reported by the task analysis was charac-terised in terms of the corresponding cognitive activity,using a table of generic cognitive activities [7]. As anexample, the detailed cognitive activities for the “Oil incompressed air (TP) system” scenario are shown incolumn 2 of Table 3.

• The next step was to identify the likely error modes.Rather than use the pure phenotypes [14], a set of errormodes was developed that corresponds to four represen-tative cognitive functions called execution, interpreta-tion, observation, and planning [24]. (The reader shouldnote that the concept of cognitive functions does notimply a sequential information processing model.) Foreach cognitive activity, the corresponding cognitivefunctions were determined using a predefined mappingof the four functions. For cognitive activities thatinvolved more than one cognitive function, the choiceof the most important one was based on the descriptionof the likely performance conditions. The error modeswere found using a table of the possible cognitive func-tion failures or error modes for each of the basic cogni-tive functions. The list, shown in Table 2, was limited to


Table 2Generic (cognitive) error modes

Cognitive function Potential (cognitive) error mode

Execution E1 Execution of wrong type performed, withregard to force, distance, speed ordirection

E2 Action performed at wrong time, eithertoo early or too late

E3 Action on wrong object (neighbour,similar or unrelated)

E4 Action performed out of sequence, suchas repetitions, jumps, and reversals

E5 Action missed, not performed (i.e.,omission), including the omission of thelast actions in a series (“undershoot”)

Interpretation I1 Faulty diagnosis, either a wrongdiagnosis or an incomplete diagnosis

I2 Decision error, either not making adecision or making a wrong orincomplete decision

I3 Delayed interpretation, i.e., not made intime

Observation O1 Observation of wrong object. A responseis given to the wrong stimulus or event

O2 Wrong identification made, due to e.g. amistaken cue or partial identification

O3 Observation not made (i.e., omission),overlooking a signal or a measurement

Planning P1 Priority error, as in selecting the wronggoal (intention)

P2 Inadequate plan formulated, when theplan is either incomplete or directlywrong

Table 3Possible error modes for scenario #1, Segment A

Action (OPAS) Cognitive activity Likely (cognitive) error mode

Registration of alarm Verify O2—wrong identification madeCheck air system for otherproblems as oil in the system

Evaluate I1—faulty diagnosis

Checking faulty RV10S05 valve Evaluate I1—faulty diagnosisBypassing of RH10 upperheaters

Execute E1—execution of wrong type

Send FO to check faultyinstrumentation

Co-ordinate P2—inadequate plan formulated

TO asks FO if filter automaticsare working

Communicate E5—action missed, notperformed

Send another FO to checkcompressors

Co-ordinate P2—inadequate plan formulated

Checking of FP-heater! FO Evaluate I2—decision error

the main error modes. A more extensive list can be foundelsewhere [7].

Since several possible error modes are defined for eachcognitive function, the analyst must select the one that bestmatches the description of the scenario and the performanceconditions. This combination requires a thorough consider-ation of the nature of the scenario, together with appropriateknowledge of the method. The result of determining theerror modes is illustrated by column 3 of Table 3, using asegment of one of the experimental scenarios.

3.3. The experiment

The main experiment was carried out in the HAldenMan–Machine LABoratory (HAMMLAB) as part of alarger study of alarm systems, which aimed to investigatedifferent types of alarm display and different alarm proces-sing levels. The experimental conditions of the study werederived from a combination of display types and alarmprocessing levels. The experiment used eight scenarios,designed and rated by subject matter experts and assignedto either of two groups of high, respectively low complexity.The process model is a full scope simulation of a pressurisedwater reactor plant with two parallel feedwater trains,turbines and generators, similar to the plant model used in

the training simulator at the Loviisa nuclear power station inFinland. The participants in this study were 12 licensedcommercial power plant operators from Loviisa. Sixcrews of operators participated with two operators per crew.

3.4. Outcome of performance prediction

For the purpose of the experiment it was decided to selectthe conditions that were most likely to produce a differencein performance with regard to the differences in alarmdisplay. These were: (1) a scenario with no alarm processingand mixed display, (2) a scenario with nuisance alarmsremoved and integrated display, and finally (3) a scenariowith nuisance alarms removed and tile display as a baselinefor purposes of comparison.

To evaluate the prediction quality, time windows for eachscenario were developed by a process expert, and eachaction was described in terms of its ideal time and criticaltime for solution, referring to a plant safety criterion. Thescoring provided information about whether the operatorperformed the action, when the action was performed,whether the operator followed the correct sequences ornot, etc. However, the data did not provide information onplanning activities, nor for some of the observation andexecution activities. These potential error modes weretherefore analysed using other sources of information.

To validate the classification system of the error modepredictions, the scenarios were scored independently by twoanalysts. The agreement was high, with a mean of about72%, ranging from 53 to 88% agreement. All predictionswere analysed, and a match percentage for each scenariowas calculated. The match percentage ranged from 42 to100%, with a mean of 67.8%. The histogram (Fig. 6)shows the match percentage distribution of the differentscenarios.

Fig. 6 shows that the match between predicted observederror modes in four scenarios was between 41 and 50%, inseven scenarios between 51 and 60%, in ten scenarios


Fig. 6. Percentage match of scenario predictions.

Table 4Detailed analyses of match between predicted and observed error modes

Error mode Category Percentage of matches Number of matches Total predicted

Execution of wrong typeperformed

E1 5 1 19

Action performed at wrong time E2 64 88 137Action on wrong object E3 Not observed Not observed 1Action performed out ofsequence

E4 0 0 3

Action missed, not performed E5 58 15 26Faulty diagnosis I1 14 2 14Decision error I2 53 24 45Delayed interpretation I3 85 74 87Observation of wrong object O1 Not observed Not observed 0Wrong identification made O2 Not observed Not observed 0Observation not made (in time) O3 100 66 66Priority error P1 Not observed Not observed 0Inadequate plan formulated P2 Not observed Not observed 1

between 61 and 70%, in eight scenarios between 71 and80%, in five scenarios between 81 and 90% and in twoscenarios between 91 and 100%. The average matchpercentage at 68.6 was taken as an acceptable result of theprediction.

The data were further analysed to find the match of thepredicted error modes for each main category. This showedthat the overall match between predicted and observedobservation error modes was 100%, interpretation errormodes had a 68% match, planning error modes were notobserved, and execution error modes had a 56% match.Table 4 shows the amount and percentage of correctlypredicted error modes in more detail.

The predictions also included an evaluation of what themost likely error mode would be for each scenario. This wasbased on the scorer judgements together with the character-isation made by the common performance conditions of thescenario. The error mode judged to be the most likely foreach scenario actually occurred in 72% of the cases.

3.5. Evaluation of prediction method

The lack of observations of certain error modes (O1, O2,P1, P2, and E4) could be related to the way the data werecollected. The task analysis only described the actions andobservations the operator should carry out, and did notinclude any planning activities, nor any possible mistakesor alternative ways to reach the goal.

When looking at Table 4, the number of predictions forthe remaining error modes was quite high, except for errormodes E1 and E4. These error modes can be said to be quitesimilar to the error modes that were not observed. Other datasources could be used to resolve ambiguities in the obser-vations, but could not be used to provide the missing predic-tions. If a more complete range of prediction is required, thebasic performance description must therefore be refinedbeyond the current task analysis categories. This refinementshould include an evaluation of the appropriateness of thedifferent error modes. It might also be considered whetherthe predictions should be made in relation to different stagesor segments of the scenario, rather than on the level ofspecific activities.

3.6. Discussion

The results from the data analysis indicated that themethod used to predict performance failures was reasonablyprecise, with an average match of 68.6%. The method wasdeliberately based on rather simple assumptions about theoperator’s cognitive functions, and used a consistent classi-fication of error modes. The experience from using themethod is that it was easy to learn and efficient in use, asthe analysis of a single scenario could be accomplished inabout half a day. This is important for possible future use,since a cumbersome method is unlikely to be applied inpractice.

The needs ofPSA-cum-HRAare firstly to determine how

operator actions can fail and secondly to derive the prob-ability that the failure will happen. The work reported herehas shown that the first need can be met in a relativelysimple manner, yet one that honours the requirements tocomprehensiveness and consistency. The second need isstill in search of an answer, and here anybody’s guess willdo. It must, however, be considered a good starting point tohave a clear description of the types of failures for whichprobabilities must be found, instead of the oversimplifiedEOO–EOC dyad. The error modes are furthermore deter-mined on the basis of a description of the common perfor-mance conditions, i.e. they are related to and depend on thespecific context rather than disconnected from it. This opensa possibility for the development of quantification methodswhere the influence of the common performance conditionsis prior to and more important than individual variations.

3.7. The Hunting of the Snark

In 1874, Lewis Carroll wrote “The Hunting of the Snark”,described as “An Agony in Eight Fits”. It describes how amixed party (a Bellman, a Boots, a Barrister, a Broker, aBilliard maker, a Banker, a Beaver, and a Baker) sets out tohunt for the Snark. They sailed across the seas, and finallylanded in a place where the Snark could be found. The partyspread out on the island, to search for the Snark each in theirown way. The Baker, finally, met with the Snark and theothers heard his cries. But when they went out to bring himback to the camp, they found nothing.

They hunted till darkness came on, but they foundNot a button, or feather, or mark,

By which they could tell that they stood on the groundWhere the Baker had met with the Snark.

In the midst of the word he was trying to say,In the midst of his laughter and glee,

He had softly and suddenly vanished away–For the Snarkwasa Boojum, you see.

The reason was that the Snark was not really a Snark, buta Boojum. In the search for the EOO–EOC, which unfortu-nately is not nearly as entertaining as the hunting of theSnark, one may hope that it is, indeed, the EOO–EOCthat will vanish once they have been exposed for whatthey are. Instead, we can begin to struggle with the realbeast, the Boojum. If only Lewis Carroll were there tohelp us!

References

[1] Hall RE, Fragola J, Wreathall J. Post event human decision errors:operator action tree/time reliability correlation (NUREG/CR-3010).Washington, DC: US Nuclear Regulatory Commission, 1982.

[2] Swain AD, Guttman HE. Handbook of human reliability analysis withemphasis on nuclear power plant applications (NUREG CR-1278).Washington, DC: Nuclear Regulatory Commission, 1983.

[3] Embrey DE, Humphreys P, Rosa EA, Kirwan B, Rea K. SLIM-MAUD. An approach to assessing human error probabilities using


structured expert judgement (NUREG/CR-3518). Washington, DC:US Nuclear Regulatory Commission, 1984.

[4] Hannaman GW, Spurgin AJ, Lukic YD. Human cognitive reliabilitymodel for PRA analysis (NUS-4531). Palo Alto, CA: Electric PowerResearch Institute, 1984.

[5] Williams JCA. A data-based method for assessing and reducinghuman error to improve operational performance. Proceedings ofthe Fourth IEEE Conference on Human factors in Power Plants,Monterey, CA, 6–9 June 1988.

[6] Cooper SE, Ramey-Smith AM, Wreathall J, Parry GW, Bley DC,Luckas WJ, Taylor JH, Barriere MT. A technique for human erroranalysis (ATHEANA) (NUREG/CR-6350). Washington, DC: USNuclear Regulatory Commission, 1996.

[7] Hollnagel E. Cognitive reliability and error analysis method. London:Elsevier, 1998.

[8] Bieder C, Le Bot P, Desmares E, Cara F, Bonnet J-L. MERMOS:EdF’s new advanced HRA method. In: Mosleh A, Bari RA, editors.Probabilistic safety assessment and management (PSAM 4), London:Springer, 1998. p. 129–34.

[9] Hollnagel E. What is a man that he can be expressed by a number?. In:Apostolakis G, editor. Probabilistic safety assessment and manage-ment, New York: Elsevier, 1991.

[10] Reason JT, Mycielska K. Absent-minded? The psychology of mentallapses and everyday errors. Englewood Cliffs, NJ: Prentice-Hall, 1982.

[11] Hollnagel E, Wreathall J. HRA at the turning point?. In: Cacciabue C,Papazoglou I, editors. Probabilistic safety assessment and manage-ment ‘96, Berlin: Springer, 1996.

[12] Dougherty Jr. EM. Human reliability analysis—where shouldst thouturn?. Reliability Engineering and System Safety 1990;29(3):283–99.

[13] Dougherty Jr. EM. Human errors of commission revisited: an evalua-tion of the ATHEANA approach. Reliability Engineering and SystemSafety 1998;60(1):71–82.

[14] Hollnagel E. The phenotype of erroneous actions. International Jour-nal of Man–Machine Studies 1993;39:1–32.

[15] Woods DD, Johannesen LJ, Cook RI, Sarter NB. Behind human error:cognitive systems, computers and hindsight. Columbus, OH: CSER-IAC, 1994.

[16] Hollnagel E. Human reliability analysis: context and control. London:Academic Press, 1993.

[17] Allen J. Maintaining knowledge about temporal intervals. Commu-nication of the ACM 1983;26:832–43.

[18] Vesely WE, Goldberg FF, Roberts NH, Haasl DF. Fault tree hand-book (Technical report NUREG-0492). Washington, DC: US NuclearRegulatory Commission, 1980.

[19] Roland HE, Moriarty B. System safety: engineering and management.New York: Wiley, 1983.

[20] Altman JW. Improvements needed in a central store of human perfor-mance data. Human Factors 1964;6:681–6.

[21] Follesø K, Kaarstad M, Drøivoldsmo A, Kirwan B. Relations betweentask complexity, diagnostic strategies and performance in diagnosingprocess disturbances. In Norros L, editor. Proceedings of the FifthEuropean Conference on Cognitive Science Approaches to ProcessControl, Espoo, Finland, 30 August–1 September 1995. Espoo,Finland: VTT.

[22] Kaarstad M, Follesø K, Collier S, Hauland G, Kirwan B. Humanerror—the second pilot study (HWR-421). Halden, Norway: OECDHalden Reactor Project, 1995.

[23] Kaarstad M, Kirwan B, Follesø K, Endestad T, Torralba B. Humanerror—the first pilot study (HWR-417). Halden, Norway: OECDHalden Reactor Project, 1994.

[24] Hollnagel E, Cacciabue PC. Cognitive modelling in system simula-tion. Proceedings of the Third European Conference on CognitiveScience Approaches to Process Control, Cardiff, 2–6 September1991.


Documents

Looking for errors of omission and commission or The Hunting of the Snark revisited