Author's personal copy - Pennsylvania State University · Author's personal copy The nature of this question to any particular person is deterministic, not random. Fuzzy set generalizes

This article was published in an Elsevier journal. The attached copyis furnished to the author for non-commercial research and

education use, including for instruction at the author’s institution,sharing with colleagues and providing to institution administration.

Other uses, including reproduction and distribution, or selling orlicensing copies, or posting to personal, institutional or third party

websites are prohibited.

In most cases authors are permitted to post their version of thearticle (e.g. in Word or Tex form) to their personal website orinstitutional repository. Authors requiring further information

regarding Elsevier’s archiving and manuscript policies areencouraged to visit:

http://www.elsevier.com/copyright

http://www.elsevier.com/copyright

Autho

r's

pers

onal

co

py

A fuzzy logic-based computational recognition-primeddecision model

Yanqing Ji a, R. Michael Massanari b, Joel Ager c, John Yen d,Richard E. Miller e, Hao Ying a,*

a Department of Electrical and Computer Engineering, Wayne State University, Room 3144, Engineering Building,

5050 Anthony Wayne Drive, Detroit, MI 48202, USAb Department of Internal Medicine and Center for Healthcare Effectiveness Research, Wayne State University, Detroit, MI, USA

c Department of Family Medicine and Public Health Sciences and Division of Biostatistics and Epidemiology,

Wayne State University, Detroit, MI, USAd College of Information Sciences and Technology, The Pennsylvania State University, University Park, PA, USA

e Veterans Affairs Medical Center, Detroit, MI, USA

Received 9 November 2006; received in revised form 17 February 2007; accepted 23 February 2007

Abstract

The recognition-primed decision (RPD) model is a primary naturalistic decision-making approach which seeks toexplicitly recognize how human decision makers handle complex tasks and environment based on their experience. Moti-vated by the need for quantitative computer modeling and simulation of human decision processes in various applicationdomains, including medicine, we have developed a general-purpose computational fuzzy RPD model that utilizes fuzzysets, fuzzy rules, and fuzzy reasoning to represent, interpret, and compute imprecise and subjective information in everyaspect of the model. Experiences acquired by solicitation with experts are stored in experience knowledge bases. New localand global similarity measures have been developed to identify the experience that is most applicable to the currentsituation in a specific decision-making context. Furthermore, an action evaluation strategy has been developed to selectthe workable course of action. The proposed fuzzy RPD model has been preliminarily validated by using it to calculatethe extent of causality between a drug (Cisapride, withdrawn by the FDA from the market in 2000) and some of its adverseeffects for 100 hypothetical patients. The simulated patients were created based on the profiles of over 1000 actual patientstreated with the drug at our medical center before its withdrawal. The model validity was demonstrated by comparing thedecisions made by the proposed model and those by two independent internists. The levels of agreement were establishedby the weighted Kappa statistic and the results suggested good to excellent agreement.� 2007 Elsevier Inc. All rights reserved.

Keywords: Medical decision-making; Naturalistic decision-making; Recognition-primed decision model; Computational recognition-primed decision model; Experience-based reasoning; Adverse drug reactions; Fuzzy logic; Similarity measure

0020-0255/$ - see front matter � 2007 Elsevier Inc. All rights reserved.

doi:10.1016/j.ins.2007.02.026

* Corresponding author. Tel.: +1 313 577 3738; fax: +1 313 578 5833.E-mail address: [email protected] (H. Ying).

Information Sciences 177 (2007) 4338–4353

www.elsevier.com/locate/ins

Autho

r's

pers

onal

co

py

1. Introduction

Classical decision-making strategies largely use various axiomatic models where the optimal choice is basedon a specific criterion or evaluative standard, usually the maximization of expected utility [3,27,38]. Studieshave shown that sometimes this class of models cannot adequately describe real-world decision-making[28]. The naturalistic decision-making (NDM) paradigm is an alternative, which investigates the cognitionprocess of human decision makers in a more realistic setting. NDM researchers argue that experts extensivelyrely on situation assessment rather than generating a set of alternatives first and then weighing their proba-bilities and selecting the best one among them. Novices, on the other hand, are inclined to utilize a more delib-erative decision-making approach. Image Theory [2] is one of the well-developed NDM approaches, wherethree different images are used to organize experts’ thinking about decisions. It was recently used, among otherapplications, to assist clinicians in understanding medical decisions and developing ways to overcome the dis-parity between principle and clinical practice [9]. Another method is called Task Analysis where human deci-sion processes are explored by analyzing a specific task in the natural environment of interest [35]. Some(healthcare) applications have been attempted [13,33].

Klein and his colleagues studied how fire commanders made decisions and found that the decision-makingof the expert commanders was largely based on careful observation, situation recognition and past experience.After studying hundreds of experienced decision makers, Klein found that about 50–80% of all the decisionswere made in this way [15]. They proposed a qualitative recognition-primed decision (RPD) model to charac-terize the decisions in naturalistic settings [19]. Instead of trying to find the best (i.e., optimal) solution, theRPD identifies the first workable option based on previous decision experiences (a decision strategy called sat-isfying) [16]. The reason is that real-world decision tasks are often characterized by ill-structured problems,dynamic environments, ill-defined or competing goals, etc., which make it difficult to find the optimum solu-tion. The RPD model is more efficient and suitable for modeling experts. For a variety of applications, it canrepresent human decision behavior more reasonably than the optimizing approach, and thus is widelyaccepted by decision makers. For instance, a study showed that the RPD was followed for 95% of all the deci-sions made by the naval officers on a cruiser [14] The classical RPD model provides important implications fordecision support system design, decision skill training, and personnel selection for critical incident managers[18].

Motivated by the need for quantitative computer modeling and simulation of human decision process aswell as computerized assistance to enhance this process, researchers recently attempted to make the classicalRPD model quantifiable. By quantifiable, we mean a quantitative and computable RPD model that is readilyimplementable by computer. A long-term memory structure was proposed to represent experience [40], result-ing in a ‘‘decision-specific’’ architecture (other aspects of cognition such as cue abstraction, action evaluation,etc. were ignored). Liang et al. studied the simple match of RPD and employed a neural network to formalizean experience [22]. Robichaud further extended the neural network model by using fuzzy techniques to inter-pret the external environment [31]. There are also several studies in which the computational RPD model wasdeveloped and integrated with agent technologies. Norling et al. employed the computational RPD in theBelief-Desire-Intention agent framework as a more realistic way for simulating human societies [25]. Morerecently, Yen et al. developed an agent architecture (R-CAST) that includes a computational RPD model thatextends the RPD model in order to support human agent collaboration and the sharing of relevant informa-tion within a team [10,42]. In R-CAST, experiences are organized in a hierarchical structure. Both the cuematching and the expectancy monitoring components of RPD are implemented as collaborative activities thatinvolve agents reasoning about information that may be indirectly linked to cues and expectancy throughinference rules. Fuzzy logic was used in [34] to incorporate a subjective and imprecise interpretation of thecue values, but the use of fuzzy logic was rather limited since environmental variables were still representedas crisp values, not as fuzzy numbers/sets. The fuzziness in the process of cue abstraction and feature matchingwere not dealt with, either.

Fuzzy set theory is an effective paradigm to quantitatively represent and process deterministic imprecision,uncertainty, or subjectivity, which are frequently encountered in real-world applications [4,24,37]. The basis ofthe theory is a fuzzy set [12,23,45]. As an example, a person’s age is numerically precise. However, relating aparticular age to ‘‘young’’ can be difficult, confusing, and uncertain. What age is young and what age is not?

Y. Ji et al. / Information Sciences 177 (2007) 4338–4353 4339

Autho

r's

pers

onal

co

py

The nature of this question to any particular person is deterministic, not random. Fuzzy set generalizes 0 and 1membership values of a classical set to a membership function of a fuzzy set ranging from 0 to 1; 0 means noassociation, 1 indicates complete association, and any number in between means partial association. Fuzzy settheory lays a foundation for computing with words. To illustrate this, suppose that 30 years is ‘‘young’’ with amembership value of 0.8 and is ‘‘old’’ (a fuzzy set) with a membership value of 0.3. Then, the membershipvalue for the age of 30 being ‘‘young and old’’ (a fuzzy set) is 0.24 if the algebraic product fuzzy AND operatoris used. The membership value for the age being ‘‘young or old’’ (a fuzzy set) is 0.8 if Zadeh fuzzy OR operatoris utilized. They are partial memberships because the memberships being ANDed and ORed are partial, notfull. The membership values of ‘‘not young’’ and ‘‘not old’’ can be computed as 0.2 and 0.7, respectively.

Another important feature of the theory is fuzzy if–then rules (e.g., if x is Small and y is Fast then z is High,where Small, Fast, and High are fuzzy sets). Such rules have been widely used in fuzzy modeling and control asa powerful tool for quantitatively representing knowledge and experience [29,30,36]. Conclusions can beinferred from fuzzy rules for given partially matching inputs [44,46].

In this paper, we develop, in a systematic manner, a fuzzy logic-based general-purpose computational fuzzyRPD model. Compared with the literature, our novelties include the following. First of all, fuzzy sets areemployed to formalize the representation of imprecise cues, and fuzzy reasoning is used to abstract higher levelcues from lower level elementary data. Second, to search for the experience that is most applicable to the sit-uation of interest, similarity measures are developed to evaluate the degree of matching between the situationand a prior experience because dealing with partial matching is critical for many real-world problems. Thesesimilarity measures can handle different types of cues involving nominal values, quantitative data and fuzzynumbers/sets. Finally, a quantitative action evaluation strategy empowered by fuzzy logic is developed toexamine whether a course of action is workable.

To better establish the validity of the proposed model and demonstrate its practical utility, we applied it toa pilot study of assessing the causality between a drug (we chose Cisapride) and its adverse effects in the con-text of post-marketing surveillance. One hundred hypothetical patients were involved in the study whose char-acteristics were created based upon the profiles of more than 1000 patients treated with the drug at ourVeterans Affairs Medical Center before it was withdrawn from the market in late 1990s due to the severeadverse events. Decisions made by the model were compared to those made by two independent physicians.The extent of the agreement was evaluated by the weighted Kappa statistic and results suggested good toexcellent agreement between the model and the physicians.

To the best of our knowledge, there is no report in the literature on medical application of the RPD mod-eling approach, conventional/classical or computational. Therefore, the evaluation itself is novel with respectto application.

The remainder of this paper is organized as follows. Section 2 briefly introduces the classical RPD model.Section 3 describes in detail the proposed computational fuzzy RPD model. To better present the theoreticaldevelopment and also show the practice relevance of the model, we use detection of (unknown) drug adverseevents as an illustrative example throughout this section. In Section 4, the model is preliminarily validatedusing 100 hypothetical patients. The statistic results of comparing the decisions made by the proposed modeland two physicians are shown. We wrap up with conclusions in Section 5.

2. Brief background on classical recognition-primed decision model

Fig. 1 shows the classical RPD model that includes two processes: (1) assessing the current situation to rec-ognize which course of action makes sense, and (2) evaluating the course of action by mental simulation[17,42]. It assumes that experts employ ‘‘situation-experience matching’’ decision rules in which they matchthe current situation with prior experiences. Once they are matched, the decisions (or actions) in that experi-ence will be utilized to solve the current problem. Being more than simple matching, the RPD model providesa well-structured process in which decisions are part of a decision-action cycle rather than a single judgment.This process enables decision makers to adapt and refine a decision according to feedback and changingsituation.

Four by-products are generated in the first process (Fig. 1): relevant cues, expectancies, plausible goals, andactions. Cues represent the higher-level information (synthesized from elementary or environmental data) that

4340 Y. Ji et al. / Information Sciences 177 (2007) 4338–4353

Autho

r's

pers

onal

co

pya decision maker must pay attention to. Expectancies describe what will happen next as the current situationcontinues to evolve in a changing context. If one of them is found to be ‘‘anomaly’’ (i.e. what the decisionmaker expects conflicts with the new observed facts), it indicates that the current situation is misinterpretedand adjustments may be required. Goals represent an end state that the decision maker is trying to achieve.Actions represent a set of potential decisions that the decision maker can take in the current situation.

In the second process illustrated in Fig. 1, the course of action will be singly evaluated by imaging how aparticular action will evolve. The decision maker may either modify the course of action, or reject it and lookfor another option.

3. Proposed computational recognition-primed decision model

Our development of a computational decision-making model was motivated by a real medical applicationof computer assisted post-marketing surveillance of unknown severe adverse drug reactions (ADRs), whichrepresents a significant public health problem. In the context of the present paper, an ADR refers to the unan-ticipated drug-associated adverse incident(s) that follows the administration of a drug when it is used properlyand at an appropriate dosage [8]. ADRs can be severe with serious long-term effects and even cause death [20].

To illustrate our model more clearly, we will utilize some of the cognitive processes that a physician wouldemploy when making decisions regarding causality assessment between drug and adverse effect. The descrip-tion and formalization of this application do not, and cannot, cover all aspects of the decision making relatedto the ADR detection problem. We focus on how a physician assesses the evidence for the causality in an indi-vidual patient case. Our intention here is to demonstrate the applicability of the proposed model in the medical

Fig. 1. The classical RPD model [17].


Autho

r's

pers

onal

co

py

domain. We stress that the proposed model is general and useful for many application domains besidesmedicine.

To develop such a model, the primary difficulties are: (1) how to quantitatively represent an experience, (2)how to quantitatively perceive, comprehend, and assess current situation, (3) how to quantitatively decidewhich past experience is most similar to the current situation, and (4) how to quantitatively evaluate whetherthe course of action is workable or not.

Our fuzzy RPD model deals with these issues (Fig. 2). The box ‘‘Elementary data gathering from environ-ment’’ is the input of the model and the ‘‘implementation’’ of the chosen action is the output. Compared withthe classical RPD model shown in Fig. 1, two boxes, ‘‘Fuzzy abstraction’’ and ‘‘Experience representation’’,are added in order to help the reader better understand the computational model. In the classical model, thesetwo processes are assumed to be internal activities of human mind and are not explicitly represented in Fig. 1.We now explain the proposed fuzzy RPD model component by component.

3.1. Situation awareness

A decision task is often characterized by a pattern of higher level cues. In the phase of situation awareness,each cue is abstracted or synthesized from some external environmental variables whose values are observable.

Fig. 2. The proposed fuzzy logic-based computational fuzzy RPD model. The differences between Figs. 1 and 2 outline the proposed fuzzyRPD framework with respect to the classical qualitative RPD model.


Autho

r's

pers

onal

co

py

Feature-matching is then used to diagnose the current situation. That is, these synthesized cues are comparedwith the pre-specified domain-dependent features. If they match, the current situation is recognized as typicaland four by-products are achieved (i.e., the types of goals, important cues, next expectancies, and a course ofaction that likely will succeed). If this fails due to lack of information on experience, more information will becollected and a new feature-matching process will be performed.

3.1.1. Cue types

Cue is a key concept in the RPD methodology since both the situation awareness and action evaluationprocesses are centered on cues. Cues are usually abstracted or fused from elementary data and their typescould be quantitative, nominal or fuzzy in the proposed computational fuzzy RPD model. A quantitativecue refers to a variable whose values are described by a certain order. It can be either continuous or discrete.While a quantitative continuous cue uses real values (e.g., blood pressure), a quantitative discrete cue usesinteger values (e.g., number of children). A nominal or symbolic cue is a discrete cue whose values are notnecessarily in any order (e.g., patient gender). The addition of fuzzy cues to our model is novel relative tothe literature and is inspired by the need for representing information encountered in the real world that isimprecise in nature. Fuzzy information may be due to the imprecision of real data or arise from subjectivejudgments. For instance, the representation of causal relationship between an adverse event and a specific drugmay be qualitative using such words as probable, possible, unlikely, rather than precise values. Sometimeseven though a variable has precise values one may only care about categories (e.g., young, old). Such impreciseclasses play an important role in human thinking. Therefore, the cues in an experience are more likely to bevague due to the imprecise nature of abstract thoughts and concepts.

In the case of ADR assessment, the cues employed to evaluate the causality are abstracted from the descrip-tion in Edwards’ paper [8] and summarized in Table 1. The degree of causality is categorized as ‘‘very likely’’,‘‘probable’’, ‘‘possible’’, ‘‘unlikely’’, and the evidence to determine each term is defined by a pattern of cuevalues. Among these cues, temporal association is the most important one. It refers to the temporal relation-ship between taking the drug and occurrence of the adverse event. Other explanations denote alternative expla-nations by concurrent disease or other drugs. Dechallenge is defined as the relationship between withdrawal ofthe drug and abatement of the adverse effect. Contrary to dechallenge, rechallenge describes the relationshipbetween re-introduction of the drug and recurrence of the adverse event. The weights for these cues are designparameters and are assigned by domain experts.

The cues temporal association, dechallenge and rechallenge are all fuzzy variables which can be representedby fuzzy sets or derived through fuzzy reasoning. Interestingly, the linguistic terms like ‘‘possible’’ or ‘‘unli-kely’’ employed to represent these cues are frequently used in the literature [8], which indicates the vaguenessof both these cues and the problem. This also suggests fuzzy set theory is a natural way to formalize thisproblem.

3.1.2. Fuzzy abstraction

In this subsection, we demonstrate how to use fuzzy reasoning to abstract high level cues from elementarydata in the proposed computational fuzzy RPD model. Fuzzy sets and fuzzy if–then rules are widely used torepresent imprecise information and model human expertise in a variety of domains [1,39,43]. Fuzzy reasoningprovides an inference procedure that derives conclusions from known facts and a set of fuzzy if–then rules.These theories suggest a systematic strategy to comprehend observable environmental variables and abstractor synthesize higher level cues.

Table 1Cues for drug causality assessment

Cues Cue type Examples of cue values Abstraction method Significance weight

Temporal association Fuzzy Likely, possible, unlikely Fuzzy reasoning 1Other explanations Nominal Yes, no Crisp reasoning 0.6Dechallenge Fuzzy Likely, probable, unlikely Fuzzy reasoning 0.7Rechallenge Fuzzy Likely, possible, unlikely Fuzzy reasoning 0.8


Autho

r's

pers

onal

co

py

For the cues listed in Table 1, we chose temporal association as an example to show how fuzzy reasoning isemployed to achieve this process. The cue value of temporal association can be inferred from the time durationbetween taking the drug and appearance of an adverse effect (td). It should be noted that in the case of a sus-pected ADR, exposure to the causal agent (drug) should always precede the effect (ADR) of interest. This dis-tinction is important because the clinical manifestations of an ADR might result from entirely different causes(e.g., underlying diseases). Therefore we define the following fuzzy reasoning rules to link cause (drug) to effect(ADR):

Both td and temporal association are fuzzy variables which are characterized by the triangular fuzzy sets(Fig. 3). Other fuzzy rules that we used cover the adverse effect of interest (e.g., ventricular tachycardia, syn-cope) as shown below:

The triangular, trapezoidal, and Gaussian fuzzy sets are arguably the most widely used fuzzy set types inthe literature. Note that the triangular type is a special case of the trapezoidal type. The trapezoidal type isconsidered, to some extent, similar to the Gaussian type. There does not exist a general theory to guide theuser to select the best type for a given application. Section is case by case and in most application is basedon the trial and error approach. The fuzzy values for the variables as well as the fuzzy sets were determinedafter intensive consultation with an experienced internal medicine physician on the research team.

For a particular ADR, td = 1 day, then only the first two rules are activated. If the algebraic product fuzzyAND operator and Zadeh fuzzy OR operator are used, we can compute the fuzzy set for the correspondingtemporal association:

lctemporal associationðxÞ ¼

215

x; 0 6 x < 0:52

225ð28x2 þ 153x� 76Þ; 0:5 6 x 6 1

(ð1Þ

Fig. 3. Fuzzy sets for time duration and temporal association.

If td is short, then temporal association is likely.If td is medium, then temporal association is possible.If td is long, then temporal association is unlikely.

If td is short and ventricular tachycardia is found, then temporal association is likely.If (td is medium and ventricular tachycardia is found) or (td is short and only syncope is found), then tem-poral association is possible.If td is long, then temporal association is unlikely.


Autho

r's

pers

onal

co

py

This membership function represents the fuzzy cue value abstracted from time duration through fuzzyreasoning. Its value will be compared with the corresponding cue values stored in the experience knowledgebase.

The following shows relevant rules that are utilized to extract the fuzzy values of dechallenge.

To extract the fuzzy values of rechallenge, we use p1 to represent the first drug-ADR pair and p2 the seconddrug-ADR pair on the same patient. The following rules are defined:

Comparing relevant cue values with the cue values stored in prior experiences requires similarity measures.We will introduce new measures for the proposed model. But first, let us discuss how to quantitatively repre-sent an experience and define an experience knowledge base.

3.1.3. Experience representation

The proposed model is grounded on domain experts’ experiences. As such, prior experiences need to beacquired from human experts and stored in an experience knowledge base for future use. Below, we first givea sample ADR experience and then abstract its formal representation.

Our knowledge base for the task of assessing suspected ADR is achieved by intensive discussion with anexperienced internist as well as by careful analysis of relevant papers in the literature. According to the clas-sification scheme in [8], a particular pattern of cue values characterizes a specific type of causality which mayrequire certain courses of action to handle the ADR. Therefore, we can define various experiences, each ofwhich is associated with a type of causality (e.g. very likely, probable, and possible). These experiences forman experience knowledge base. The following is a sample experience which is illustrated in natural language forthe easier understanding:

If stop date is available during the time of admission and the patient is discharged alive, then dechallengeis likely.If stop date is not available during the time of admission and the patient is discharged alive, then dechal-lenge is possible.If temporal association is unlikely, then dechallenge is unlikely.

If p1 has likely temporal association and p2 has likely temporal association, then rechallenge is likely.If p1 has likely temporal association and p2 has possible temporal association, then rechallenge islikely.If p1 has likely temporal association and p2 has unlikely temporal association, then rechallenge is possible.If p1 has possible temporal association and p2 has likely temporal association, then rechallenge islikely.If p1 has possible temporal association and p2 has possible temporal association, then rechallenge ispossible.If p1 has possible temporal association and p2 has unlikely temporal association, then rechallenge ispossible.If p1 has unlikely temporal association and p2 has likely temporal association, then rechallenge is possible.If p1 has unlikely temporal association and p2 has possible temporal association, then rechallenge ispossible.If p1 has unlikely temporal association and p2 has unlikelytemporal association, then rechallenge isunlikely.


Autho

r's

pers

onal

co

py

Note that the definition of an experience knowledge base is always goal-oriented. From physician’s perspec-tive, another critical goal in the ADR problem would be how to manage the patient who most likely has theADR. In this case, more relevant cues may be required, and the action would be to reduce the dosage of thedrug or discontinue the drug and initiate alternative therapy.

To make our work systematic, we now provide a general representation. An experience can be representedby a quintuple:

ei ¼ ðxi;Ci;Ei;Gi;AiÞ

in which xi is the name of the ith experience in an experience knowledge base. Ci is a collection of high-levelcues that are abstracted from environment variables. For a medical problem, environment variables could beany lower level information that may affect a decision-making, such as physician’s observation (e.g., symp-toms), patient’s medical history, and laboratory tests. Ei and Gi denote the set of expectancies and goals ofthe experience, respectively. Ai is a set of courses of action associated with the experience, each of which isa sequence of lower level actions. In Sections 3.1.5 and 3.2, we will describe how a fuzzy logic-based represen-tation of cues and courses of action is used in the proposed fuzzy RPD model for realizing cue matching andevaluating courses of action.

3.1.4. Similarity measures

In the proposed computational fuzzy RPD model, similarity measures are employed to assess the degree oflikeness between the current situation and past experience so as to find out whether the past experience couldbe used to solve the current problem. Other computational RPD models use different ways (e.g. crisp matching[25], neural network [22,31]) to achieve the matching between the current situation and past experience, butthey either cannot capture partial matching or assume that all cues are crisp values. Our approach can over-come these limitations.

We assume that (1) the cues in current situation and the ones in the past experience have the same set offeatures or attributes, and (2) the values assigned to them characterize the current situation and past experi-ences. Suppose that V and V 0, two vectors, denote the set of cue values in the current situation and a priorexperience, respectively:

V ¼ ðc1; c2; . . . ; cj; . . . ; cnÞ;V 0 ¼ ðc01; c02; . . . ; c0j; . . . ; c0nÞ;

where cues cj and c0j can be nominal, quantitative or fuzzy values. The similarity measure between V and V 0,called global similarity, is needed and such measure should be computed using the local similarities calculatedfor the cue value pairs cj and c0j. There are many similarity measures in the literature [7,41]. However, they arenot capable of handling the different types of cues in our model. Furthermore, they do not provide a means toaggregate local similarities for pattern matching when the required information is not satisfied. Therefore, wehad to develop new ones ourselves.

Experience_N: The causality between the drug and a particular ADR is very likely.Cues: (1) The occurrence of ADR has a likely temporal association with the drug.

(2) There is no other explanation.(3) The ADR has likely dechallenge relationship with the drug.(4) The ADR has likely rechallenge relationship with the drug.

Goal: Find the strength of the causality between a drug and a particular ADR.Actions: (1) Suggest for further analytical studies.

(2) Suggest filing an ADR report online.Expectancy: (1) More similar cases are available.

(2) Relevant information for filing an ADR report is available.


Autho

r's

pers

onal

co

py

We propose a local similarity measure that can handle heterogeneous types of cues. After a set of local sim-ilarities for known cue value pairs are obtained, they are amalgamated into a global similarity measure. Wedefine the following heterogeneous similarity measure to calculate the local similarity between cj and c0j:

SLðcj; c0jÞ ¼

0; if cj or c0j is unknown

overlapðcj; c0jÞ; if cue j is nominal

1� normalized diff ðcj; c0jÞ; if cue j is quantitative

fuzzy simðcj; c0jÞ; if cue j is fuzzy value

8>>><>>>:

ð2Þ

Decision makers often have to make decisions under the condition of incomplete information. In the abovedefinition, the cue similarity is set 0 if either of the cue values is unknown. The function overlap() is definedas:

overlapðcj; c0jÞ ¼1; if cj ¼ c0j0; otherwise

�ð3Þ

That is, for nominal cue values, the similarity is 1 if the cue values are equal; otherwise it is 0. Computing thesimilarity for quantitative cue values involves distance measure normalized_diff:

normalized diff ðcj; c0jÞ ¼jcj � c0jj

Djð4Þ

where Dj = aj � bj is employed to normalize the cue difference, and aj and bj are the maximum and minimumvalues for cue j, respectively.

The last expression in the above formula, fuzzy simðcj; c0jÞ, deals with cues represented by fuzzy sets. Let cuej be defined on the universe of discourse X and x be an element of X. In this case, fuzzy cue values cj and c0j aretwo fuzzy sets whose membership functions are defined in terms of x. Their similarity could be defined on thebasis of possibility measures on the ½0; 1� interval [5]:

fuzzy simðcj; c0jÞ ¼possðcj; c0jÞ if possð�cj; c0jÞ < 0:5

ð1:5� possð�cj; c0jÞÞ � possðcj; c0jÞ otherwise

(ð5Þ

where

possðcj; c0jÞ ¼ maxxðminðlcj

ðxÞ; lc0jðxÞÞÞ 8x 2 X ð6Þ

Here, lcjðxÞ and lc0j

ðxÞ are the membership functions of fuzzy set cj and c0j, respectively. The min( ) and max( )are standard fuzzy operators, and hence maxxðminðlcj

ðxÞ; lc0jðxÞÞÞ computes the maximum of all the minimums

between pairs lcjðxÞ and lc0j

ðxÞ for all the elements x. �cj is the complement of cj (i.e., lcjðxÞ ¼ 1� lcj

ðxÞ).Eq. (5) is one of the most commonly used similarity functions for fuzzy pattern matching [6,7], and further-

more, it has been incorporated into some fuzzy inference engines (e.g. software FuzzyJess [26]) for general use.What we have discussed so far involves only the local similarity for a single cue value pair cj and c0j. For our

proposed model, a global similarity measure is also needed to combine the local similarities. Let us continue touse the ADR problem as an example to illustrate the development. We now calculate all the local similaritiesfirst and then combine them.

Recall that the cue value for temporal association has been abstracted in Section 3.1.2. It is represented as afuzzy set whose membership function is given by Eq. (1). Its cue value stored in Experience_N is ‘‘likely’’ withthe membership function being:

lc0temporal association

ðxÞ ¼0; 0 6 x 6 0:5

2x� 1; 0:5 6 x 6 1

�ð7Þ

To calculate the local similarity between these two cue values, we apply Eq. (5) and get

SLðctemporal association; c0temporal associationÞ ¼ 0:92

where ctemporal_association and c0temporal association represent the cue values of temporal association in the current sit-uation and the chosen experience in the past, respectively.


Autho

r's

pers

onal

co

py

For cues dechallenge and rechallenge, we can employ the same procedure to abstract their actual cue valuesand find out their similarities with corresponding cue values store in Experience_N. Here, without loss ofgenerality, we simply assume that SLðcdechallenge; c0dechallengeÞ ¼ 0:86 and SLðcrechallenge; c0rechallengeÞ ¼ 0:79. For thecue other explanations, we regard it as a nominal variable since its values ‘‘yes’’ and ‘‘no’’ are not fuzzy.We assume that the value of this cue is ‘‘no’’ for the current situation. Therefore, we can computeSLðcother explanations; c0other explanationsÞ ¼ 1 by applying Eq. (3).

After obtaining the similarities for each of the cue value pairs, our next step is to integrate them into a glo-bal similarity SG(V,V 0) using the following formula, where V and V 0 represents two sets of cue values:

SGðV ; V 0Þ ¼0; if there exists j 2 ð1; nÞ; for which wj ¼ 1 and SLðcj; c0jÞ < dPn

j¼1wjSLðcj;c0jÞPn

j¼1wj

; otherwise

8><>: ð8Þ

where d 2 [0, 1] is a threshold design parameter determined by the user. wj 2 [0, 1] is the weight for cue j, whichrepresents the relative significance of the cue and is also assigned by the user. To reuse the actions in a pastexperience, some important cue or cues are often required to be satisfied. That is, if the value of a required cuein the current situation is not close to the value of the same cue in a past experience, this experience cannot beutilized to solve the current problem no matter how similar the other cues are. For example, temporal associ-

ation is a required cue to assess the causal relationship between a drug and an adverse event. If there is noreasonable temporal relationship between the time of taking the drug and the time that development of anadverse event occurs, it is almost certain that this adverse event is not caused by the drug even if all the otherfactors (e.g., dechallenge) match well. In (8), we deal with this issue by assigning 0 to SG(V,V 0) when one of theimportant cues whose weight is 1 is not similar.

Outside of this exception, global similarity is computed by the second half of the formula, which is the nor-malized weighted local similarities, making the global similarity lie in [0, 1].

Continue our ADR example. Using Eq. (8), we can calculate the global similarity value between the currentsituation and Experience_N as:

SGðV ; V 0Þ ¼ ð0:92� 1þ 1� 0:6þ 0:86� 0:7þ 0:79� 0:8Þ=ð1þ 0:6þ 0:7þ 0:8Þ ¼ 0:89

Here, V refers to the set of observed cue values and V 0 represents the set of cue values stored in Experience_N.

3.1.5. Feature matching

The feature matching process is carried out through the global similarity measure. We assume that there aredifferent types of experience knowledge bases, and each base can deal with a specific decision type which couldbe inferred from the information associated with the decision task [42]. Once the decision type is determined,the experience knowledge base (and thus the collection of cues) to be matched is fixed. After that, the currentsituation is compared with the past experiences in the selected experience knowledge base. Suppose that V is aset of observed cues in the current situation and V 0i ði ¼ 1; 2; . . . ;M , where M is the number of experiences inthe experience knowledge base) is a set of cues considered in a past experience. The matching is performedthrough computing the global similarity between V and V 0i coordinate-wise using Eq. (8) in the feature space.The current situation is said to be matched with the past experience to degree a, if SGðV ; V 0iÞP a wherea 2 [0,1] is a similarity threshold chosen by the user. If more than one experience is matched with the currentsituation, the one with the highest global similarity is chosen as the matching result.

3.2. Action evaluation

While situation awareness is to diagnose a problem, action evaluation is a process of selecting a workablecourse of action to solve the problem. When applying the RPD methodology to medical decision making, acourse of action may refer to a medical procedure, a treatment plan, etc. For human decision makers, actionevaluation can be achieved through mental simulation: people image how to assemble a course of action, howthe actions may evolve and whether the relevant goal can be fulfilled. For the computational fuzzy RPDmodel, we assume that each course of action has an initial state, a terminal state and several actions between


Autho

r's

pers

onal

co

pythem. Each action has three parts in sequence: precondition, execution and effect (Fig. 4). While an initial stateis usually the trigger for a series of actions, the terminal state often stands for our final expectation to bereached after these actions. For instance, the diagnosis result (e.g., a disease) for a patient could be an initialstate to prompt a therapy plan, and in this case the terminal state could be the cure of that disease. The pre-

condition of an action is a set of cues that serve as the prerequisite of execution. For example, a patient’s age,physical conditions, and medical history may be important factors that must be considered when his/her phy-sician makes a plausible treatment action. The effect is our expectation for the execution of an action. Notethat the effect is not the actual execution results which may be different from our expectation. Usually, twoactions in sequence have causal relationship, that is, the effect of one action is one of the preconditions forthe subsequent action. Intuitively, the initial state is one of the preconditions for the first action, and the ter-minal state is the effect of the last action.

To evaluate a course of action, we first compare the initial state (often along with a few other cues) againstthe precondition of action 1 using the similarity measures introduced earlier. If they well match (i.e., their glo-bal similarity measure is greater than a predetermined threshold), the effect of action 1 and other cues will beemployed to compare with the precondition of action 2. We continue this process until we reach the terminalstate which is one of the effects of the last action. If we use SGðV 1; V 01Þ; SGðV 2; V 02Þ; . . . ; SGðV n; V 0nÞ to representthe similarities between the current state and the preconditions of action 1 until action n. Consequently thedegree of our confidence, represented as confidence factor (CF), with this course of action is defined as:

CF ¼ SGðV ; V 0iÞ � ½SGðV 1; V 01Þ � SGðV 2; V 02Þ � � � SGðV n; V 0nÞ� ð9Þwhere � denotes a fuzzy conjunction operator (e.g. algebraic product, Zadeh min operator). V and V 0i repre-sent the sets of cue values in the current situation and in the prior experience that is selected through the fea-ture matching process, respectively. SGðV ; V 0iÞ is added to the formula because it reflects the degree ofappropriateness to use the chosen experience. As such, its value should affect our total confidence with theselected course of action. CF provides a quantitative way to measure the extent to which the initial statecan be transferred to the terminal state through a course of action. The larger the CF value is, the more con-fident we are.

To show the action evaluation process, we take the action ‘‘Suggest for further analytical studies’’ in Expe-

rience_N as an example. This action is the simplest in that it cannot be further divided into lower level actions.The precondition and effect of this action would be ‘‘the strength of signal pair (i.e. a drug and a particularADR associated with this drug) in the current patient is evaluated as ‘very likely’’’ and ‘‘this patient is selectedfor further analytical studies’’, respectively. Calculating the similarity between the current situation and theprecondition for the action using Eqs. (2) and (8) yields SGðV 1; V 01Þ ¼ 1, if we assume that the strength of signalpair is ‘‘very likely’’ after evaluating the relevant cues, Now we can calculate the confidence factor:CF = 0.89 · 1 = 0.89 using the algebraic product operator for the conjunction. If this CF is satisfactoryaccording to a predefined user criterion, the course of action will be implemented or adjusted by human inter-vention in order to handle the current situation. Otherwise, a different course of action will be evaluated usingthe same procedure. If a satisfactory course of action is not achieved after all the actions have been evaluated,more relevant information needs to be collected. A new evaluation procedure starts or a novel course of actionis added through human intervention.

The process of selecting a workable course of action is straightforward in the above example. In general,this is usually the case because it is an important feature of the RPD methodology, which is based on the idea

Fig. 4. A course of actions of the computational fuzzy RPD model.


Autho

r's

pers

onal

co

py

that experts spend most of their time and energy on understanding the situation in naturalistic environments.Once the situation is recognized, properly selecting a course of action almost always automatically follows.The course of action is usually the one that was successful in the previous, similar situation [18,25].

4. Preliminary evaluation of the model

To establish the proposed model more firmly and also to demonstrate its practical utility, we carried out apreliminary evaluation experiment. The experiment was related to the ADR detection for post-marketing sur-veillance described above. More specifically, we sought to assess the model’s ability to quantitatively identifythe strength of the causal relationship between a drug and an adverse effect. We compared the model’s deci-sions with physicians’ and used the weighted Kappa statistics to establish the extent of their agreements.

4.1. Evaluation design

We targeted the drug Cisapride in this study. The drug was introduced into the marketplace in 1993 uponapproval by the Federal Drug Administration (FDA) for the treatment of gastro-esophageal reflux. It pro-vided symptomatic relief for this painful, but non-fatal condition. Spontaneous reports linking Cisapride witha sometimes fatal cardiac ventricular arrhythmia began to appear in the FDA and the medical literature.Approximately seven years later the drug was removed from the marketplace because of idiosyncratic,high-risk, adverse reactions.

With approval from the Human Investigations Committee, we undertook a descriptive study of all patientstreated with Cisapride at our Veterans Affairs Medical Center between 1993 and 1999. The sources for dataincluded the standard hospital discharge abstract database and the pharmacy database. Patient specific datawere linked by patient identifiers, e.g. name, birth date. A statistically de-identified data set was provided to usfor analysis by a hospital data analyst. 1,015 patients were identified that had received Cisapride on one ormore encounters in the institution. Among this cohort, 21 patients were diagnosed with the cardiac arrhythmiaof interest and 303 patients died during the period covered by this study. A group of 10,326 control patientsthat never received Cisapride were also identified by randomly selecting controls for every case that werematched by hospital admission date (+/�30 days). These patients were used to identify the data fields thatwould provide useful information for weighting potential causal linkages among signal pairs. After identifyingrelevant fields in the data set of the patients who were exposed to Cisapride, we then created a set of 100 hypo-thetical patients. Each case contained information regarding signal pairs including the administration of thedrug and the adverse event under review as a potential toxic side-effect of the drug administration. Cases var-ied in the strength of the possible causal association between the drug and an event based on (1) the chrono-logical association between the drug start date and onset of the adverse event; (2) evidence for dechallenge; (3)evidence for rechallenge; and (4) presence or absence of an alternative explanation for the adverse event.Hence, the signal pairs for the hypothetical patients represented a spectrum of causal weights from a ‘‘verylikely’’ to ‘‘unlikely’’ causal link. The data set of the 100 hypothetical patients was comparable to real datasets of patients who received Cisapride. We created a Microsoft Access database for storing the hypotheticaldata that would be readily accessible for continuous scrutiny by a computer program.

The strength of the causal relationship between the drug and an adverse effect is categorized as ‘‘verylikely’’, ‘‘probable’’, ‘‘possible’’ and ‘‘unlikely’’. Based on the assignment of the likelihood of associationbetween signal pairs as determined by our proposed fuzzy RPD model, patients would be selected for furtherstudies (i.e., actions of the proposed model) if the likelihood is high or for no further action if the likelihood islow. More specifically,

• Signal pairs assigned to ‘‘very likely’’ association or ‘‘probable’’ association will be included in a cohort ofexposed subjects for further analytical studies.

• Signal pairs assigned to ‘‘unlikely’’ association will be excluded from further analytical studies.• Signal pairs assigned to ‘‘possible’’ association will be excluded from further analytical studies. It is conceiv-

able, however, that if the analytical study design adjusts for the weight of the association this group mightbe included in further studies.


Autho

r's

pers

onal

co

py

The further actions would include the following steps: (1) more information regarding the drug exposureand adverse outcomes will be sought from the medical records of the selected signal pairs; (2) an appropriategroup of control subjects will be identified; and, (3) an analytical study will be completed. These tasks, how-ever, are beyond the scope of the present paper and will be addressed in our future work.

These 100 hypothetical patients were used to validate the assignment of causal link strengths by the pro-posed computational fuzzy RPD model and physicians. The model was used to identify and assign likelihoodsof causal associations between the drug (cause) and adverse event (effect) for the 100 hypothetical patients. Toprovide preliminary validation for the capacity of the proposed model, two physicians, both trained in internalmedicine, participated in the experiment. They were provided a general overview of the objectives of the studyand were asked to independently review each of the 100 scenarios and make a judgment regarding the likeli-hood of a significant causal association between the drug and adverse events. Physicians assigned a numericalscore between 1 and 4 based on the strength of the perceived causality, where 4 means ‘‘very likely causal rela-tionship’’, 3 is ‘‘probable casual relationship’’, 2 indicates ‘‘possible causal relationship’’, and 1 stands for‘‘unlikely causal relationship’’.

We examined agreement between the scores generated by the proposed fuzzy RPD model and those by eachof the two physicians. Because scoring of the causal association was based on ordinal data, we utilized theweighted Kappa statistic to estimate the levels of agreement [11,32]. The Kappa coefficient is an estimate ofthe agreement between two raters after chance agreement is controlled. Kappa scores range between 1 (com-plete agreement) and 0. Because there are no ‘‘absolute’’ interpretations of the Kappa coefficients, experts haveoffered opinions regarding interpretations of agreement. Landis and Koch [21] suggest that for values ofKappa greater than 0.75, there is excellent agreement. For values less than 0.4, there is poor agreement,and for values between 0.40 and 0.75, there is fair to good agreement.

Table 2Drug causality assessment results generated by the proposed computational fuzzy RPD model and two independent physicians (P1 and P2mean physician #1 and physician #2, respectively)

PatientID

FuzzyRPDmodel

P1 P2 PatientID

FuzzyRPDmodel

P1 P2 PatientID

FuzzyRPDmodel

P1 P2 PatientID

FuzzyRPDmodel

P1 P2

1 1 1 1 26 2 2 3 51 2 2 2 76 3 3 32 3 3 3 27 3 3 4 52 2 2 2 77 2 2 23 2 2 3 28 4 4 4 53 3 3 3 78 1 1 14 4 4 4 29 3 3 4 54 2 2 2 79 4 4 45 4 4 3 30 3 3 3 55 3 3 4 80 4 4 46 4 4 4 31 2 2 3 56 3 3 4 81 2 2 27 1 1 2 32 2 2 2 57 2 2 2 82 3 3 48 4 3 4 33 4 3 4 58 4 4 4 83 3 3 49 2 2 3 34 3 3 4 59 2 2 2 84 2 2 2

10 4 4 4 35 2 2 2 60 3 3 3 85 2 2 211 3 3 3 36 3 3 3 61 4 4 4 86 2 2 212 1 1 1 37 1 1 1 62 3 3 3 87 2 2 213 2 2 2 38 3 3 3 63 2 2 2 88 4 4 414 2 2 2 39 3 4 4 64 4 4 4 89 2 2 215 1 1 1 40 2 2 3 65 2 2 2 90 3 3 416 2 2 2 41 2 2 3 66 1 1 1 91 4 4 317 4 3 4 42 3 3 4 67 3 3 4 92 1 1 118 3 3 3 43 3 3 3 68 2 2 2 93 2 2 219 2 2 2 44 4 4 4 69 3 3 4 94 2 2 220 1 1 1 45 4 3 4 70 3 3 3 95 4 4 321 3 3 3 46 4 3 4 71 3 3 4 96 3 3 422 2 2 3 47 2 2 2 72 4 4 3 97 2 2 223 3 3 3 48 3 3 4 73 4 4 4 98 2 2 224 2 2 3 49 2 2 2 74 2 2 2 99 4 4 325 3 3 4 50 3 3 4 75 2 2 2 100 3 3 4


Autho

r's

pers

onal

co

py

4.2. Evaluation results

Table 2 summarizes the assignment of scores for the strength of the causal association provided by the pro-posed computational fuzzy RPD model and the two physician reviewers. The estimate of agreements is as fol-lows: Kappa = 0.939 for physician 1 and the model; Kappa = 0.700 for physician 2 and the model;Kappa = 0.657 for physician 1 and physician 2. These coefficients suggest good to excellent agreementbetween the proposed model and the physicians. We computed the confidence intervals using both the asymp-totic formula and the jackknife method (leave one out method). The results of these two methods agreed to thesecond decimal as shown in Table 3.

5. Conclusion

We have developed a novel general-purpose computational fuzzy RPD model using fuzzy logic technology.Our approach has several desirable features. First, fuzzy sets and fuzzy reasoning are employed to quantita-tively represent and interpret imprecise information, and handle the uncertainty in the decision-making pro-cess. Second, local and global similarity measures are created to evaluate the degree of feature matching. Thesesimilarity measures are very flexible since they can not only handle different types of information (e.g. quan-titative, nominal, and fuzzy) but also incorporate various conditions including the calculation of similarity atpresence of missing information and the aggregation of similarity values even when the required information isnot satisfied. Finally, we have developed a more realistic action evaluation strategy, an issue that has not beenwell addressed in the literature. Based on the hospital patient data, we have designed and implemented a pre-liminary validation experiment for the proposed model in the context of drug ADR detection. The resultingKappa statistics indicate excellent agreement between the model and the two physicians.

Acknowledgements

The authors would like to thank Dr. Alan Baptist in the Department of Internal Medicine at Wayne StateUniversity for his participation in the experiment. We are grateful to Ellison Floyd, Joanthan Small, CharlesGrace, and Robert Johnson of the Veterans Affairs Medical Center for their help in retrieving the patientinformation for the experiment study.

This work was supported in part by a Wayne State University Research Enhancement Program grant.

References

[1] S. Barro, R. Marin, Fuzzy Logic in Medicine, first ed., Physica-Verlag, Heidelberg, 2002.[2] L.R. Beach, Image Theory: Decision Making in Personal and Organizational Contexts, Wiley, New York, 1990.[3] L.R. Beach, R. Lipshitz, Decision Making in Action: Models and Methods, Ablex Publishing Corporation, Norwood, 1993.[4] H. Bustince, M. Pagolaa, E. Barrenecheaa, Construction of fuzzy indices from fuzzy DI-subsethood measures: application to the

global comparison of images, Information Sciences 177 (3) (2007) 906–929.[5] M. Cayrol, H. Farreny, H. Prade, Fuzzy pattern-matching, Kybernetes 11 (2) (1982) 103–116.[6] D. Dubois, J.-L. Koning, A decision engine based on rational aggregation of heuristic knowledge, Decision Support Systems 11 (4)

(1994) 337–361.[7] D. Dubois, H. Prade, C. Testemale, Weighted fuzzy pattern matching, Fuzzy Sets and Systems (28) (1988) 313–331.[8] I.R. Edwards, J.K. Aronson, Adverse drug reactions: definitions, diagnosis, and management, Lancet 356 (9237) (2000) 1255–1259.[9] P.R. Falzer, Cognitive schema and naturalistic decision making in evidence-based practices, Journal of Biomedical Informatics 37 (2)

(2004) 86–98.

Table 395% confidence intervals for weighted Kappa coefficients from Asymptotic formula and Jackknife method

Kappa coefficient Asymptotic 95% confidence interval Jackknife 95% confidence interval

Physician 1 vs. fuzzy RPD model 0.939 (0.891, 0.986) (0.891, 0.988)Physician 2 vs. fuzzy RPD model 0.700 (0.605, 0.792) (0.606, 0.798)Physician 1 vs. Physician 2 0.657 (0.560, 0.754) (0.561, 0.760)


Autho

r's

pers

onal

co

py

[10] X. Fan, S. Sun, M. McNeese, J. Yen, Extending recognition-primed decision model for human–agent collaboration, in: Proceedingsof The Fourth International Joint Conference on Autonomous Agents and Multi Agent Systems (AAMAS 2005), Netherlands, 2005,pp. 945–952.

[11] J.L. Fleiss, Statistical Methods for Rates and Proportions, John Wiley & Sons, New York, 1981.[12] W. Homenda, Balanced fuzzy sets, Information Sciences 176 (17) (2006) 2467–2506.[13] S.K. Howard, D.M. Gaba, B.E. Smith, M.B. Weinger, C. Herndon, S. Keshavacharya, M.R. Rosekind, Simulation study of rested

versus sleep-deprived anesthesiologists, Anesthesiology 98 (6) (2003) 1345–1355, discussion 1345A.[14] G.L. Kaempf, G. Klein, M.L. Thorsden, S. Wolf, Decision making in complex naval command-and-control environments, Human

Factors 38 (2) (1996) 220–231.[15] G. Klein, Strategies of decision making, Military Review 69 (5) (1989) 56–64.[16] G.A. Klein, A recognition-primed decision (RPD) model of rapid decision making, in: G.A. Klein, J. Orasanu, R. Calderwood, C.E.

Zsambok (Eds.), Decision Making in Action: Models and Methods, Ablex, Norwood, 1993, pp. 138–147.[17] G.A. Klein, The recognition-primed decision (rpd) model: looking back, looking forward, Naturalistic Decision Making (1997) 285–

292.[18] G.A. Klein, Sources of Power, The MIT Press, 1998.[19] G.A. Klein, R. Calderwood, Decision models: some lessons from the field, IEEE Transactions on Systems Man and Cybernetics 21 (5)

(1991) 1018–1026.[20] L.T. Kohn, J.M. Corrigan, M.S. Donaldson, To Error is Human: Building a Safer Health System, National Academy Press,

Washington, DC, 1999.[21] J.R. Landis, G.G. Koch, The measurement of observer agreement for categorical data, Biometrics 33 (1) (1977) 159–174.[22] Y. Liang, F. Robichaud, B.J. Fugere, K.N. Ackles, Implementing a naturalistic command agent design, in: Proceedings of the Tenth

Conference on Computer Generated Forces, Norfolk, VA, 2001.[23] J.M. Mendel, Advances in type-2 fuzzy sets and systems, Information Sciences 177 (1) (2007) 84–110.[24] Y. Nakamori, M. Ryoke, Treating fuzziness in subjective evaluation data, Information Sciences 176 (24) (2006) 3610–3644.[25] E. Norling, L. Sonenberg, R. Rnnquist, Enhancing multi-agent based simulation with human-like decision making strategies, in: S.

Moss, P. Davidsson (Eds.), Proceedings of the Second International workshop on Multi-Agent Based Simulation, 2000, pp. 214–228.[26] R. Orchard, Fuzzy reasoning in Jess: the FuzzyJ Toolkit and FuzzyJess, in: Third International Conference on Enterprise

Information Systems, Setubal, Portugal, 2001, pp. 533–542.[27] A. Pankowska, M. Wygralak, General IF-sets with triangular norms and their applications to group decision making, Information

Sciences 176 (18) (2006) 2713–2754.[28] V.L. Patel, D.R. Kaufman, J.F. Arocha, Emerging paradigms of cognition in medical decision-making, Journal of Biomedical

Informatics 35 (2002) 52–75.[29] Z. Pei, G. Resconi, A.J.V.D. Wal, K. Qin, Y. Xu, Interpreting and extracting fuzzy decision rules from fuzzy information systems and

their inference, Information Sciences 176 (13) (2006) 1869–1897.[30] D. Pelta, A. Sancho-Royo, C. Cruz, J.L. Verdegay, Using memory and fuzzy rules in a co-operative multi-thread strategy for

optimization, Information Sciences 176 (13) (2006) 1849–1868.[31] F. Robichaud, Implementing a naturalistic command agent design, Workshop on Computerized Representation of RPD, Boulder,

CO, 2002.[32] S. Siegel N.J.C. Jr., Nonparametric Statistics for The Behavioral Sciences, second ed., McGraw-Hill Humanities/Social Sciences/

Languages, 1988.[33] J. Slagle, M.B. Weinger, M.T. Dinh, V.V. Brumer, K. Williams, Assessment of the intrarater and interrater reliability of an

established clinical task analysis methodology, Anesthesiology 96 (5) (2002) 1129–1139.[34] J.A. Sokolowski, Enhanced decision modeling using multiagent system simulation, in: Proceedings of the Twelfth Conference on

Behavior Representation in Modeling and Simulation, Scottsdale, AZ, 2003, pp. 179–186.[35] T.R. Stewart, P.J. Roebber, L.F. Bosart, The importance of the task in analyzing expert judgment, Organizational Behavior and

Human Decision Processes 69 (3) (1997) 205–219.[36] E. Straszecka, Combining uncertainty and imprecision in models of medical diagnosis, Information Sciences 176 (20) (2006) 3026–

3059.[37] E. Trillas, On the use of words and fuzzy sets, Information Sciences 176 (11) (2006) 1463–1487.[38] E. Tsiporkova, V. Boeva, Multi-step ranking of alternatives in a multi-criteria and multi-expert decision making environment,

Information Sciences 176 (18) (2006) 2673–2697.[39] S. Tsumoto, Mining diagnostic rules from clinical databases using rough sets and medical diagnostic model, Information Sciences 162

(2) (2004) 65–80.[40] W. Warwick, R. Hutton, Developing computational models of recognition-primed decision making, in: Proceedings of the Tenth

Conference on Computer Generated Forces, Norfolk, VA, 2001, pp. 232–331.[41] D.R. Wilson, T.R. Martinez, Improved heterogeneous distance functions, Journal of Artificial Intelligence Research 6 (1997).[42] J. Yen, X. Fan, S. Sun, M. McNeese, D. Hall, Supporting anti-terrorist analyst teams using agents with shared RPD process, in: IEEE

International Conference on Computational Intelligence for Homeland Security and Personal Safety, Venice, Italy, July 2004.[43] J. Yen, R. Langari, L.A. Zadeh, Industrial Applications of Fuzzy Logic and Intelligent Systems, IEEE Press, 1995.[44] H. Ying, Fuzzy Control and Modeling: Analytical Foundations and Applications, IEEE Press, 2000.[45] L.A. Zadeh, Fuzzy sets, Information and Control 8 (1965) 338–353.[46] L.A. Zadeh, Toward a generalized theory of uncertainty (GTU) – an outline, Information Sciences 172 (1-2) (2005) 1–40.


Documents

Author's personal copy - Pennsylvania State University · Author's personal copy The nature of this question to any particular person is deterministic, not random. Fuzzy set generalizes