Journal of Behavioral Decision Making

J. Behav. Dec. Making, 22: 455–474 (2009)

Published online 12 March 2009 in Wiley InterScience

(www.interscience.wiley.com) DOI: 10.1002/bdm.642

*Correspondence to: Shahar Ayal, DE-mail: s.ayal@duke.eduyBoth authors contributed equally to

Copyright # 2009 John Wiley &

Ignorance or Integration: The CognitiveProcesses Underlying Choice Behavior

SHAHAR AYAL1�,y and GUY HOCHMAN2y

1Center for Behavioral Economics, Fuqua School of Business, Duke University, USA2MaxWertheimerMinerva Center for Cognitive Studies, Faculty of IndustrialEngineering andManagement,TheTechnion�Israel Institute of Technology, Haifa,Israel

ABSTRACT

The fast-and-frugal heuristic framework assumes noncompensatory tools for humanpreferences (i.e., priority heuristic) and inferences (i.e., take the best heuristic).According to this framework, these heuristics predict choice behavior as well as modelthe cognitive processes underlying such behavior. The current paper presents twostudies that juxtapose predictions derived from these two simple heuristics withalternative predictions derived from compensatory principles. Dependent measuresthat included reaction time, choice pattern, confidence level, and accuracy were betterpredicted by compensatory indices than by noncompensatory indices. These findingssuggest that people do not rely on limited arguments only, but tend to integrate allacquired information into their choice processes. This tendency was replicated evenwhen the experimental task facilitated the use of noncompensatory principles. We arguethat the fast and frugal heuristics can predict the final outcome only under certainconditions, but even in these particular situations they are not applicable to theprocesses underlying choice behavior. An integrative model for choice behavior isproposed that better represents the data. Copyright # 2009 John Wiley & Sons, Ltd.

key words compensatory models; noncompensatory models; take the best heuristic;

priority heuristic; choice behavior

INTRODUCTION

Decision-making models have been traditionally classified into compensatory versus noncompensatory

theoretical frameworks (e.g., Einhorn, 1971; Elrod, Johnson, & White, 2004; Ford, Schmitt, Schechtman,

Hults, & Doherty, 1989; Payne, Bettman, & Johnson, 1993). Generally speaking, compensatory models (e.g.,

Expected Utility Theory, by von Neumann &Morgenstern, 1944; Prospect Theory by Kahneman & Tversky,

1979; Tversky & Kahneman, 1992) assume that choices are governed by tradeoffs between conflicting

uke University, Fuqua School of Business, 1 Towerview Drive, Durham, NC 27708, USA.

the article.

Sons, Ltd.

456 Journal of Behavioral Decision Making

attributes (e.g., reasons, cues). For example, when decision makers have to choose between two stochastic

alternatives, they use weighting and summing processes of some sort, and choose the alternative with the

highest subjective expected utility. Thus, in a compensatory process, high values on some attributes can

compensate for low values on others, and the magnitude of the difference between the expected values of the

given alternatives is a crucial element in the decision process (cf. Busemeyer & Townsend, 1993).

In contrast, noncompensatory models (e.g., Elimination by Aspects, by Tversky, 1972; Lexicographic

Theory by Fishburn, 1974) rely on the assumption that different attributes of the alternatives are used in the

order of their validity or importance. If the most valid attribute (i.e., the ‘‘best’’) differentiates between

alternatives, then the alternative it supports is chosen, and the information search is terminated. Otherwise,

the next best argument is chosen in an argument-wise search pattern. Thus, according to noncompensatory

models, there is no tradeoff between conflicting attribute values. The results of these processes are twofold.

First, decisions can be based on one argument while ignoring all others. Second, these processes circumvent

the need for more complex processes of information integration, such as weighting and summing

(Brandstatter, Gigerenzer, & Hertwig, 2006).

Recently, Gigerenzer and colleagues (e.g., Brandstatter et al., 2006; Gigerenzer, Todd, & the ABC

research group, 1999) put forward the fast-and-frugal heuristics as noncompensatory process models of

human preferences (e.g., the priority heuristic, by Brandstatter et al., 2006) and inferences (e.g., the take the

best heuristic, by Gigerenzer & Goldstein, 1996). As Brandstatter et al. (2006) argue, ‘‘The priority heuristic

is intended to model both choice and process: It not only predicts the outcome but also specifies the order of

priority, a stopping rule, and a decision rule’’ (p. 427).

While many studies confirmed that under some conditions fast-and-frugal heuristics predict choice

behavior (e.g., Broder, 2000, 2003; Broder & Schiffer, 2003a; Rieskamp& Hoffrage, 1999, 2005), there is an

ongoing debate as to whether these heuristics can also serve as process models (e.g., Broder, 2002; Glockner,

2007; Glockner & Betsch, 2008a; Johnson, Schulte-Mecklenbeck, &Willemsen, 2008). In the current paper,

we intend to demonstrate that process models of individuals’ choice behavior cannot be accounted for by

simple noncompensatory principles, but rather have to integrate compensatory and noncompensatory

components. To do so, we derive predictions about indirect measures of process from two prototypical fast-

and-frugal heuristics, and juxtapose them with contradicting predictions from simple compensatory

principles.

The process of the priority heuristicThe priority heuristic (henceforth PH) (Brandstatter et al., 2006) is a simple lexicographic model that

describes human preferences. According to this fast-and-frugal heuristic, the choice between alternatives that

yield different monetary prospects (e.g., the choice between ‘‘Alternative A—gain $3 with a probability of

0.25, otherwise nothing’’ and ‘‘Alternative B—gain $4 with a probability of 0.2, otherwise nothing’’) is

governed by a hierarchy of three rules:

Rule 1: If the difference between the minimum payoffs is larger than 10% of the maximum payoff (i.e., the

aspiration level), choose the gamble with the higher minimum payoff.

Rule 2: Otherwise, if the difference between probabilities of the minimum payoff is larger than 0.1, choose

the gamble with the smaller probability of the minimum payoff.

Rule 3: Otherwise, choose the gamble with the highest maximum payoff.

According to Brandstatter et al. (2006), this fast-and-frugal heuristic can account for a number of

empirical phenomena, such as the Allais paradox (Allais, 1953), the certainty and the possibility effects

(Kahneman & Tversky, 1979), as well as for the majority of choices documented in previous research in the

field (e.g., the data sets from Erev, Roth, Slonim, & Barron, 2002 and from Lopes &Oden, 1999). In addition,

Copyright # 2009 John Wiley & Sons, Ltd. Journal of Behavioral Decision Making, 22, 455–474 (2009)

DOI: 10.1002/bdm

S. Ayal and G. Hochman The Cognitive Processes Underlying Choice Behavior 457

according to Brandstatter et al. (2006), the PH has the highest predictive value relative to other frugal

heuristics (e.g., the minimax heuristic) or even relative to complex modifications of expected utility (e.g.,

Cumulative Prospect Theory by Tversky & Kahneman, 1992).

In order to support the PH as a process model, Brandstatter et al. (2006) used two indirect measures:

Reaction time (RT) and choice proportion (proportion of choices predicted by the model). Their results were

in line with the PH predictions. First, in an original set of 40 problems, RTwas faster for problems in which

only one PH rule was required to make a choice as compared to problems in which three PH rules were

required. Second, in a reanalysis of two other sets of problems (Kahneman & Tversky, 1979; Lopes & Oden,

1999), Brandstatter et al. found that choice proportion was correlated with the number of PH rules required to

make a choice in each problem. Namely, problems that required examination of only one rule had the highest

choice proportion, two-rule problems had lower proportion, and three-rule problems had the lowest

proportion.

Two recent studies on the choice between two risky alternatives challenged the PH as a process model.

Using a mouse-lab task in addition to RT and choice proportion measures, Johnson et al. (2008) showed that

the search for information does not adhere to the fast-and-frugal principles. Instead, decision makers

exhibited more evidence for integrating data (e.g., probability-payoff transitions within alternative) than for

relying on limited data and ignoring the rest (e.g., comparison of outcomes across the two alternatives).

Glockner and Betsch (2008a) replicated these results and strengthened their validity by using an open mouse-

lab methodology (for a detailed discussion of mouse-lab methods, see Glockner & Betsch, 2007).

In addition, Glockner and Betsch (2008a) show that the RTand choice proportion measures support the PH

predictions only in those cases in which these predictions correspond to the predictions of Cumulative

Prospect Theory (Tversky & Kahneman, 1992). These results suggest that the original support for the PH as a

process model might stem from an overlap between predictions of noncompensatory and compensatory

models. It is thus essential to disentangle this confound by focusing on situations in which the predictions of

these two models are contrasting.

The process of the take the best heuristicThe take the best (henceforth TTB) heuristic (Gigerenzer & Goldstein, 1999; Gigerenzer et al., 1999) is a

simple noncompensatory tool for probabilistic inferences. According to the fast-and-frugal framework, TTB

is implemented when people are required to choose the correct answer for a given question and they are

uncertain about the answer (e.g., which city has more residents? A. Phoenix; B. Dallas). According to

Gigerenzer et al. (1999), in these cases people tend to base their decision on a hierarchy of probabilistic cues.

First, cues will be ordered according to the probability that the cue will identify the correct answer. This

hierarchy determines the lexicographic order in which information (i.e., cues) will be searched. Second, TTB

works by activating three hierarchical rules:

Rule 1: Take the cue with the highest validity, and look up the cue values for the two alternatives.

Rule 2: If the cue is diagnostic (that is, if the cue is positive for one alternative and negative for the other),

stop the search and go to Step 3. Otherwise, go back to Step 1 and search for the next cue.

Rule 3: Choose the alternative with the positive cue value for the decision criterion.

According to Gigerenzer and Goldstein (1996), in some environments this fast-and-frugal heuristic

outperforms more complex compensatory strategies, such as the linear weighted additive (henceforth

WADD) strategy. The WADD strategy computes the sum of all cue values multiplied by the validity of the

cue, and favors the alternative with the largest sum. Indeed, several studies show that when information

acquisition is costly (Broder, 2000), under time pressure (Rieskamp & Hoffrage, 1999, 2005), or when

Copyright # 2009 John Wiley & Sons, Ltd. Journal of Behavioral Decision Making, 22, 455–474 (2009)

DOI: 10.1002/bdm

458 Journal of Behavioral Decision Making

information must be retrieved from memory (Broder & Schiffer, 2003a), the TTB model predicts people’s

inferences quite accurately. In addition, it has been shown that when the cost of information acquisition is

high, the pattern of information search fits the TTB predictions (Newell & Shanks, 2003; Newell, Weston, &

Shanks, 2003). However, the WADD model was found to be more suitable than the TTB in predicting

people’s inferences when information acquisition costs are low, when there is no time pressure, and when

information is provided simultaneously (cf. Rieskamp&Otto, 2006). In addition, Glockner (2007) shows that

even though simple noncompensatory strategies such as the TTBmight sometimes lead to accurate decisions,

the majority of people do not tend to adopt them.

Note, however, that focusing solely on predicting the outcomes, the TTB can be considered a submodel of

the WADD (i.e., considering different transformation functions for cue weights, the WADD can always

produce the same choice as TTB), so choice data that support the TTB cannot rule out the possibility that a

WADD strategywas used (Glockner, in press; Lee&Cummins, 2004). Thus, as in the case of the PH, support for

the TTB as a process model might stem from the overlap between predictions of noncompensatory and

compensatory models. Therefore, it is important to focus on situations in which the predictions of these two

models contrast each other rather than to merely rely on their overall predictive power (cf. Broder, 2002).

Indirect measures for processPrevious research shows that people employ both compensatory and noncompensatory strategies, depending

on task demands and environmental factors (Ford et al., 1989; Payne et al., 1993). Since the choice

predictions of different models overlap, it is essential to use multiple means of investigation simultaneously

in the attempt to identify the nature of a specific cognitive process. In the current study, several indirect

measures are offered for the differentiation between the compensatory and the noncompensatory strategies

for choice behavior, including reaction time, choice proportion, and confidence level.

Reaction time (RT)

Fast-and-frugal heuristics assume a sequential limited search of arguments (i.e., information), with an

explicit stopping rule according towhich search is terminated as soon as a discriminative piece of information

is acquired. Therefore, noncompensatory models predict that the fewer arguments (e.g., cues, rules) are

required to make a choice, the faster it will take people to respond. Alternatively, compensatory principles

assume that decision makers integrate relevant arguments, outcomes, and probabilities. Thus, compensatory

models predict that RT will be faster the less integration is required (Payne et al., 1993), or when the

integration of arguments is summed up by larger differences between the expected utilities of the alternatives

(Bergert & Nosofsky, 2007).

Choice proportion

Choice proportion describes the proportion of individuals’ choices that are in line with the prediction of a

specific decision strategy. If the predictions of these two models are sufficiently different, then choice

proportions in each problem can be used to efficiently identify which decision strategy has been adopted

(Broder & Schiffer, 2003b). However, when the model predicts majority of choices, such as in the case of the

PH, choice proportion cannot serve as a means by which to assess the model (Brandstatter et al., 2006). In

these cases a rank order of choice proportion could be used to evaluate models with the assumption that

people make processing errors in each step of their chosen strategy. According to the noncompensatory

principles, this assumption implies that the fewer arguments (e.g., cues, reasons) required to make a choice

the higher the choice proportion will be (for an elaborate explanation, see Brandstatter et al., 2006, p. 428).

Copyright # 2009 John Wiley & Sons, Ltd. Journal of Behavioral Decision Making, 22, 455–474 (2009)

DOI: 10.1002/bdm

S. Ayal and G. Hochman The Cognitive Processes Underlying Choice Behavior 459

Alternatively, according to compensatory principles, choice proportion depends on the magnitude of the

perceived distance between the expected utilities of the two alternatives. That is, a higher choice proportion

will be obtained when the perceived difference between the two alternatives is larger.

Confidence level

Noncompensatory tools postulate a lexicographic search pattern of information according to a natural

validity hierarchy, so that one-reason (rule) decision-making will be based on the most valid diagnostic

argument. This implies that basing the decision on arguments that have a higher natural priority or importance

will lead to the highest level of confidence in any particular judgment. Take for example a situation in which

the capital city cue (i.e., whether the city is the capital? yes/no) is a better predictor of the city size than the

soccer team cue (i.e., whether the city has a team in the first league? yes/no). In this situation, relying on the

capital city cue is supposed to lead to greater confidence in determining that City A is bigger than City B, as

compared to relying on the soccer team cue. Importantly, this predicted correlation between the priority of

information (e.g., cues, rules) and the level of confidence in a choice has not been explicitly stated for the PH.

However, it seems reasonable to predict such a correlation on the basis of previous assumptions postulated

within the fast-and-frugal heuristics framework. Specifically, the linear relationship between priorities of

arguments (e.g., cues) and level of confidence was one of the building blocks of the probabilistic mental

model and the TTB (Gigerenzer, Hoffrage, & Kleinbolting, 1991; Gigerenzer & Hug, 1992). According to

these noncompensatory models, a confidence rating is determined by the subjective validity of the cue on

which a choice is based (Broder, 2000; Gigerenzer et al., 1991).

Compensatory principles, on the other hand, suggest that an adaptive level of confidence should rely on all

acquired information as well as on its integration. Thus, compensatory models predict that confidence level

will be higher the larger the difference perceived between the alternatives (cf. Glockner & Betsch, 2008b).

Accordingly, participants can base their confidence level either on the magnitude of distance between the

expected utilities of the alternatives, or on the magnitude of difference between the overall weighted sums of

cues supporting each alternative (Budescu, 2006; Erev, Wallsten, & Budescu, 1994; Winkler & Poses, 1993).

STUDY 1: EXAMINING THE PROCESSES UNDERLYING RISKY CHOICES

In this study we examine preferences in two datasets. The first set (called here the Priority Replication set)

replicates the two-outcome choice problems used by Brandstatter et al. (2006) (see Appendix 1). The second

set is based on previous data from the collapse condition of Wakker, Erev, and Weber (1994) (called here the

Wakker et al. set, see Appendix 2). As in the Brandstatter et al. (2006) experiment, the two sets were classified

into a 2 (one rule or three rules required)� 2 (gambles of similar or dissimilar expected value) mixed-

factorial design. The level of similarity in the two datasets was defined according to Brandstatter et al.’s

(2006) classification. That is, any choice problem in which the difference between the expected values (EVs)

of the gambles did not exceed 1% of the maximal EV was classified as similar. All the other choice problems

were classified as dissimilar.

As can be seen in Table 1, this classification allows the formulation of two competing predictions for each

of the aforementioned measures—one that fits the PH and another that follows the compensatory principle of

expected utility maximization.

MethodParticipants

The participants of the priority replication set were 48 undergraduate students from the Faculty of Industrial

Engineering and Management at the Technion—Israel Institute of Technology. The sample included

29 women and 19 men, with a mean age of 23.54 years (SD¼ 1.90).

Copyright # 2009 John Wiley & Sons, Ltd. Journal of Behavioral Decision Making, 22, 455–474 (2009)

DOI: 10.1002/bdm

Table 1. Competing predictions of the PH and COMP models for each of the indirect measures (Study 1)

Indirect measure PH prediction COMP prediction

Reaction time (RT) RT will be faster in the one-rule-requiredchoice problems than in the three-rules-required choice problems, independentlyof the level of similarity between theexpected values

RT will be faster in the dissimilar-expected-value choice problems thanin the similar-expected-value choiceproblems, independently of thenumber of rules required

Choice proportion Choice proportion aligned with the PHwill be higher in the one-rule-requiredchoice problems than in the three-rules-required choice problems, independentlyof the level of similarity between theexpected values

Choice proportion aligned with thecompensatory model (EV maximization)will be higher in the dissimilar-expected-value choice problems than in the similar-expected-value choice problems,independently of the number of rulesrequired

�Confidence level Confidence-level will be higher in theone-rule-required choice problems thanin the three-rules-required choice problems,independently of the level of similaritybetween the expected values

Confidence-level will be higher in thedissimilar-expected-value choice problemsthan in the similar-expected-value choiceproblems, independently of the numberof rules required

PH¼ priority heuristic; COMP¼ compensatory model. Choice problems are classified according to (a) the number of PH rules requiredto reach a decision (one vs. three) (b) the level of similarity between the expected values (similar vs. dissimilar).�Note: As already mentioned, it has been postulated that there is a direct relationship between the level of confidence in a choice and thecue validity hierarchy (Broder, 2000; Gigerenzer et al., 1991). According to the fast-and-frugal framework, including the PH, the searchfor information is terminated when a certain aspiration level is met (Brandstatter et al., 2006; Gigerenzer, 1997; Gigerenzer & Goldstein,1996; Hoffrage, 2005). This implies that we can generalize the proposed relationship between confidence and validity of cues to a similarrelationship between confidence and priority of reason.

460 Journal of Behavioral Decision Making

The Wakker et al. (1994) dataset consisted of data from 22 psychology students who had participated in

the study as part of their Introductory Psychology course at the University of North Carolina at Chapel Hill.

Design and procedure

The priority replication set. The experiment was programmed in Microsoft Visual Basic 6 and run on a

desktop PC. Participants were presented with a series of 20 two-alternative forced-choice decisions between

gambles (e.g., choose between Alternative A—gain $2000 with probability 0.6, otherwise $500; and

Alternative B—gain $2000 with probability 0.4, otherwise $1000). All gambles were from the gain domain

(i.e., all outcomes are positive or zero) and they were presented in a random order. The participants were

asked to choose the most attractive alternative as fast as possible. RTwas measured by the software from the

time the choice problem appeared on the screen to the time of decision. At the end of the RT section,

participants were asked to rate how confident they were about each choice they had made, on a confidence

scale ranging from 50 to 100% in steps of 10%.

The Wakker et al. set. Participants’ task was to choose the preferred option in each pair of gambles. For a full

description of the design and procedure, see Wakker et al. (1994). We reanalyzed the original data, which

consisted of RTand choice proportion measures. Our analysis included all the gain gambles that should have

been solved according to 1 or 3 priority rules, thus excluding two of the original choice problems.

Results and discussionAnalysis of RT

The RT results of the priority replication set are shown in the left panel of Figure 1. When the two options had

similar EVs, the mean RT for the one-rule-required choice problems was 12.31 seconds (SD¼ 8.34), and

when the two options had dissimilar EVs the mean RT decreased to 10.03 (SD¼ 5.40). Similarly, for the

Copyright # 2009 John Wiley & Sons, Ltd. Journal of Behavioral Decision Making, 22, 455–474 (2009)

DOI: 10.1002/bdm

Figure 1. Mean reaction time (RT) as a function of the similarity between the expected values (EVs) and the number ofreasons examined. Error bars represent standard errors

S. Ayal and G. Hochman The Cognitive Processes Underlying Choice Behavior 461

three-rules-required choice problems the mean RT for similar EVs was 10.9 (SD¼ 6.18) and the mean for the

dissimilar EV choices was 8.78 (SD¼ 4.34). A 2� 2 repeated measures analysis of variance (ANOVA) was

conducted to test the effects of the number of rules required and the level of similarity between the expected

values on RT as the dependent variable. This analysis revealed a significant main effect both for the level of

similarity between the EVs (F (1, 47)¼ 11.16, p< 0.001) and for the number of rules required (F (1,

47)¼ 4.18, p< 0.05). No significant interaction between the two variables was found.

The RT results of theWakker et al. set are shown in the right-hand panel of Figure 1. When the two options

had similar EVs, the mean RT for the one-rule-required choice problems was 13.12 (SD¼ 7.24) and when the

two options had dissimilar EVs the mean RT decreased to 10.2 (SD¼ 3.89). Similarly, for the three-rules-

required choice problem the mean RT for similar EVs was 14.4 (SD¼ 7.04) and the mean for dissimilar EV

choices was 12.52 (SD¼ 4.72). A 2� 2 ANOVA was conducted to test the effects of the number of rules

required and the level of similarity between the expected values on RT as the dependent variable. This

analysis revealed a significant main effect of the level of similarity between the expected values (F (1,

17)¼ 9.64, p< 0.01). Neither a main effect of the number of rules required (F (1, 17)¼ 1.00, ns) nor an

interaction between the two variables was found.

These results strongly support the RT prediction of the compensatory model. In both datasets RT was

significantly faster for the choice problems with dissimilar EVs than for problems with similar EVs,

regardless of the number of rules required. On the other hand, the RT prediction of the PH was not confirmed,

as the relevant pattern of results was inconsistent across datasets. In the first set, there was a significant effect

for the number of rules but it was in the opposite direction to the one predicted by the PH and in contrast to the

results reported by Brandstatter et al. (2006) with the same set of choice problems. However, in the second

set, the number of rules did not have any effect on RT. One possible explanation for the inconsistency between

the current results and the results of Brandstatter et al. (2006) could be based on a sampling bias. Namely, in

the current study, participants were engineering students while in the Brandstatter et al. study participants

were psychology students. Since engineering students are more mathematically oriented, it is possible that

they are more familiar with the concept of EV and can thus utilize it more easily, whereas the effect of the

number of rules became marginal and reflected random errors.

Taken together, the RT results suggest that participants were much more sensitive to the magnitude of

difference between the EVs of the alternatives than to the numbers of rules required according to the PH

(which yielded inconsistent results). Thus, under the current settings, the RT data indicated that participants

made tradeoffs between probabilities and outcomes, and employed complex compensatory processes for

calculating and comparing EVs.

Copyright # 2009 John Wiley & Sons, Ltd. Journal of Behavioral Decision Making, 22, 455–474 (2009)

DOI: 10.1002/bdm

Figure 2. Left-hand panel. Mean choice proportion (CP) as a function of the similarity between the expected values(EVs) and the number of reasons examined. Right-hand panel. Choice proportion (CP) predicted by the PH and the

COMP as a function the equality between the predictions of the models. Error bars represent standard errors

462 Journal of Behavioral Decision Making

Analysis of choice proportion

The choice proportion results of the Priority Replication set are shown in the left panel of Figure 2. Across

participants, the mean proportion of choices that were in line with PH for the one-rule-required choice

problems was 0.55 (SD¼ 0.22) when the two options had similar EVs and this mean increased to 0.85

(SD¼ 0.19) for dissimilar EV problems. For the three-rules-required choice problem the mean proportion of

choices was 0.6 (SD¼ 0.23) for similar EVs and 0.89 (SD¼ 0.18) for dissimilar EV problems. A 2� 2

within-participants repeated measures ANOVA was conducted to test the effects of the number of rules

required and the level of similarity between the expected values on choice proportion as the dependent

variable. This analysis revealed a significant main effect of the level of similarity between the expected values

(F (1, 47)¼ 119.91, p< 0.001). Neither a main effect of the number of rules required (F (1, 47)¼ 2.71, ns),

nor an interaction between the two variables was found.

A closer examination of the priority replication set revealed that in all the dissimilar EV choice problems the

PH predictions coincided with EV maximization. In contrast, in the Wakker et al. set, there were not only

14 dissimilar problems in which this was the case, but also 6 choice problems in which the PH predictions

contradicted the compensatory predictions (see Appendix 2). An analysis of the dissimilar problems only and a

comparison of choice proportion under the equal and the non-equal prediction choice problems allowed for a

more thorough examination of the two competing models. The results of this analysis (presented in the right-

hand panel of Figure 2) show that when the two models share the same predictions, the mean proportion of

choices that aligns with this prediction is 0.697 (SD¼ 0.16). However, when the two models had contradicting

predictions, choice proportion according to the PH prediction decreased to 0.52 (SD¼ 0.18) and was not

significantly above chance level (t (21)¼ 0.51, ns), whereas choice proportion according to the compensatory

prediction remained robust at 0.61 (0.19), and above chance level (t (21)¼ 2.7, p< 0.03). A paired-samples t-test

analysis revealed a significant decrease in choice proportion according to the PH prediction (t (21)¼ 2.93,

p< 0.005), and no decrease in choice proportion according to the compensatory predictions (t (21)¼ 1.35, ns). In

addition, when the two models had contradicting predictions, choice proportion according to the compensatory

prediction was significantly higher than choice proportion according to the PH (t (21)¼�1.91, p< 0.05).

The choice proportion results confirm the compensatory prediction for choice proportion, and not the PH

prediction. The pattern of results in the first set converges with the pattern of results for the RT measure.

Choice proportion depended on the level of similarity between EVs, with higher proportions obtained for

choice problems with dissimilar EVs as compared to choice problems with similar EVs. In addition, the

choice proportion was not sensitive to the number of rules required. Moreover, the choice proportion results

of the second set suggest that when the two models share the same choice predictions, both PH and

Copyright # 2009 John Wiley & Sons, Ltd. Journal of Behavioral Decision Making, 22, 455–474 (2009)

DOI: 10.1002/bdm

S. Ayal and G. Hochman The Cognitive Processes Underlying Choice Behavior 463

compensatory models have high predictive power. These cases cannot discriminate between the two models

and thus do not contribute to our understanding of the underlying process (cf. Glockner and Betsch, 2008a).

When focusing on diagnostic cases in which the two models have contradicting predictions, it turns out that

only the compensatory model retained its predictive power and could correctly predict choice behavior in

those situations, while the PH predictive power was at chance level.

Finally, it is important to note that the choice proportion was not at chance level in the similar EV choice

problems in which the compensatory model predicted indifference (for example, in the priority replication set

the aggregated mean of choice proportion across the two conditions with similar EVs was 0.57, t (47)¼ 3.48,

p< 0.001). It could thus be argued that when integration does not prevail, participants employ additional

strategies, possibly noncompensatory in nature, in order to reach to a decision.

Analysis of confidence level

This variable was measured only in the Priority Replication set, since it was not a part of the Wakker et al.

design. The confidence level results are shown in Figure 3. For the one-rule-required choice problems the

mean confidence level was 85.73% (SD¼ 11.29) when the two options had similar EVs and this mean

increased to 91.82% (SD¼ 8.07) for the dissimilar EV choice problems. For the three-rules-required choice

problems the mean confidence level was 85.02% (SD¼ 10.42) for similar EVs and 90.3% (SD¼ 8.75) for

the dissimilar EV choices. A 2� 2 within participants repeated measures ANOVAwas conducted to test the

effects of the number of rules required and the level of similarity between the expected values on confidence

level as the dependent variable. This analysis revealed a significant main effect for the level of similarity

between the expected values (F (1, 47)¼ 27.84, p< 0.001). Neither a main effect for the number of rules

required (F (1, 47)¼ 2.57, ns), nor an interaction between the two variables was found.

These results support the compensatory prediction for confidence-level, not the PH prediction. Participants

tended to exhibit higher levels of confidence in their preferences for choice problemswith dissimilar EVs relative

to choice problems with similar EVs, regardless of the number of rules required. The confidence for one-rule-

required problems was slightly higher than the confidence for the three-rules-required problems, but this

difference was not significant. Similar to the RT and the choice proportion results, the confidence level results

suggest that participants tend to integrate outcome and probability information in a compensatory manner.

Figure 3. Mean level of confidence (LOC) as a function of the similarity between the expected values (EVs) and thenumber of reasons examined. Error bars represent standard errors

Copyright # 2009 John Wiley & Sons, Ltd. Journal of Behavioral Decision Making, 22, 455–474 (2009)

DOI: 10.1002/bdm

464 Journal of Behavioral Decision Making

In summary, the findings of the three different measures in Experiment 1 support the idea that people tend

to integrate information regarding both outcome and probabilities when facing preference decision tasks,

rather than rely on limited information in a noncompensatory manner. Furthermore, these results demonstrate

that the predictive power of noncompensatory models cannot be generalized to diagnostic cases, when the

two models have contradicting predictions.

Note, however, that these results provide evidence for the cognitive processes underlying choice behavior

when all reasons are displayed simultaneously, and all the required information is available and fully

displayed at no cost (i.e., description based decisions; Hertwig, Barron, Weber, & Erev, 2004). In contrast, in

situations in which information needs to be searched (either in memory or in information stores), and is

costly, it is more likely that the cognitive processes will be more noncompensatory and lexicographic in

nature (e.g., Gigerenzer et al., 1999). Study 2 was designed to test this possibility.

STUDY 2: EXAMINING THE PROCESSES UNDERLYING PROBABILISTIC INFERENCES

In this study we explore the cognitive processes underlying the inferences that people makewhen trying to select

the correct answer in two-alternative forced-choice questions (e.g., Which team won the 2007 European-Cup

final? A. Milan; B. Liverpool). In order to examine the processes underlying these situations, we constructed a

binary choice task in which information was provided in a cue-wise manner, and cue acquisition was costly. This

paradigmwas adapted from previous studies (e.g., Broder, 2000; Newell & Shanks, 2003; Rieskamp&Hoffrage,

1999), in an attempt to create an optimal environment for using the TTB heuristic (cf. Rieskamp & Otto, 2006).

Individuals’ choice strategies were analyzed using a Maximum Likelihood strategy classification method

(also called Bayesian strategy classification, Broder & Schiffer, 2003b; Broder, in press). Participants are

thereby classified according to their choices as TTB orWADD users, using maximum likelihood estimation.1

In addition, participants’ level of confidence in their choice was documented. These classifications made it

possible to formulate two competing predictions for each of the aforementioned measures, one for the TTB

and one for the WADD strategy (Table 2).

MethodParticipants

Sixty undergraduate students from the faculty of Industrial Engineering and Management at the Technion—

Israel Institute of Technology participated in the experiment. The sample included 28 women and 32 men,

with a mean age of 23.9 years (SD¼ 2.10).

1Maximum likelihood estimation was calculated with the formula developed by Broder and Schiffer (2003b, p. 201). The conditionallikelihood Lk for a data set (individual set of choices) under the condition that a certain decision strategy k was applied with a constanterror rate ek is given by

Lk ¼ pðnjk

��k; "kÞ ¼YJ

j¼1

nj

njk

� �ð1� "kÞnjk"k

ðnj�njkÞ;

with J being the total number of categories of decision tasks, nj being the number of choices in category j, and njk being the number ofchoices in line with the prediction of strategy k in category j. The error rate ek for each decision strategy is estimated by

"k ¼XJ

j¼1

ðnj � njkÞ" #

�XJ

j¼1

nj

" #:

Copyright # 2009 John Wiley & Sons, Ltd. Journal of Behavioral Decision Making, 22, 455–474 (2009)

DOI: 10.1002/bdm

Table 2. Competing predictions of the TTB and the COMP models for each of the indirect measures (Study 2)

Indirect measures TTB prediction COMP prediction

Choice proportion The majority of participants will usethe TTB strategy

The majority of participants will usethe WADD strategy

Confidence level Confidence level will depend on thecue precedence, independently of thedifference in weighted cue values

Confidence level will depend on thedifference in weighted cue values forthe options, independently of cueprecedence

TTB¼Take The Best model;WADD¼Weighted Additive model; COMP¼Compensatory model. Cue precedence is determined by thelocation of the cue within the validity hierarchy. Weighted cue values are the sum of gathered cues multiplied by their validity (i.e.,weight).

S. Ayal and G. Hochman The Cognitive Processes Underlying Choice Behavior 465

Design and stimulus

The experiment was programmed in Visual Basic 6 and was run on a desktop PC. Participants were presented

with a series of 40 two-alternative forced-choice questions regarding two soccer teams playing a match.

Matches were sampled from the inventory of actual games played in the Italian soccer league in 1998–1999

that did not end in a tie. However, to ensure that answers are based solely on gathered information, a randomly

generated fictitious namewas assigned to each team to prevent familiarity and identification. In addition, each

team appeared in the sample no more than four times on average, and the questions were presented in a

random order between participants. An example of a question could be: ‘‘Which of the two teams won the

game played on February 14, 1999? (A) Fedele; (B) Cocci.’’ Three binary cues provided information in the

form of a yes/no question concerning the teams in question. The three cues were:

(1) T

Copy

he previous game cue—Did the team win the previous match between the two teams in question?

(2) T

he top team cue—Was the team considered a top team at the time (i.e., one of the six leading teams)?

(3) T

he big city cue—Does the team come from a big city (i.e., a city with more than 250 000 residents)?

The three cues differed in their objective validity, with the previous-game cue being the most valid

(p¼ 0.78), the top-team cue the second most valid (p¼ 0.72), and the big-city cue the least valid (p¼ 0.65).

Validity was defined as the conditional probability that the cue would correctly identify the winning team for

all possible comparisons of two teams that differed on this cue. Following each forced choice question,

participants were asked to state how confident they were about their choice, on a confidence level scale

(ranging from 50 to 100% in steps of 10%) presented separately for each question.

Procedure

Participants were told that on each trial they would be asked to determine which of two teams that played in

the Italian soccer league in the years 1998–1999 had won the match. To help them make these choices, they

were encouraged to buy up to three cues for each specific game. To create optimal conditions for TTB

adoption, each cue was priced 1/2 NIS (approximately 0.15 USD). Cues could be bought online and

participants were given the value (YES or NO) of each cue they bought. In order to ensure interest and

credibility, an additional monetary incentive of 50 NIS (approximately 15 USD) was offered for at least

19 correct answers, and 30 NIS (approximately 9 USD) for 16–18 correct answers. The cost of the cues was

subtracted from the potential reward, to ensure that the cost would not exceed the gain. Otherwise, 15 NIS

(approximately 4.5 USD) were paid as a show-up fee.

Participants were randomly assigned to one of two conditions. Under the Subjective condition,

participants were first instructed to determine the cue validity hierarchy by rating the three cues according to

perceived validity. Then, cues were provided according to this order and each participant had the opportunity

right # 2009 John Wiley & Sons, Ltd. Journal of Behavioral Decision Making, 22, 455–474 (2009)

DOI: 10.1002/bdm

466 Journal of Behavioral Decision Making

to gather information in a lexicographical order according to his/her subjective hierarchy. Under the

Objective condition, participants were directly informed that information would be provided according to the

accuracy probability of the cues. Then, participants were given the opportunity to gather information through

these three cues according to this objective order (i.e., first the previous-game cue, then the top-team cue, and

finally the big-city cue).

Participants were instructed to provide their answers as quickly and as accurately as possible. After

providing each answer, participants were asked to rate how confident they were about their choice.

Strategy classification and model prediction calculations

For each decision, questions were first classified into three categories: (a) non-differentiating cases—in

which both strategies make the same predictions; (b) differentiating cases—in which different predictions

are made; and (c) random choice cases—in which at least one strategy does not lead to a decision and

predicts random choices. Error probabilities in applying the respective strategy (i.e., answering against the

model’s predictions) were determined for each individual across all decisions (e¼wrong predictions divided

by number of decisions with prediction). The conditional likelihood of each person’s choice pattern was

estimated given the application of a certain strategy with a constant error rate.2 Participants were classified as

users of the strategy for which the highest likelihood was found. A Bayes-ratio was calculated as a measure of

the classification certainty by dividing the likelihood for the application of the identified strategy by the

likelihood of the other strategy (Broder, in press).

The predictions for the WADD strategy were determined by comparing WADD scores for both options

(weighted sum of cue validity factor wcue and cue value; negative cue values were coded as �1, positive as

þ1). Cue validity factors were derived by correcting for the fact that cues with a 0.50 validity are

uninformative (i.e., wcue¼ pcue—0.50) (cf. Glockner & Betsch, 2008b). In addition, for each problem, we

determined the prediction of the TTB and the number of cues that were necessary to apply a TTB strategy.

Results and discussionAs in Study 1, the focus here was on situations that could distinguish between the noncompensatory and the

compensatory predictions. To provide a fair evaluation, only trials in which information was acquired were

considered. Thus, 17% of the trials (24% of the trials in the objective condition and 11% of the trials in the

subjective condition) were excluded from the analyses because they were based on pure guessing without

acquiring any piece of information.

Manipulation and paradigm check

To check whether the manipulation yielded different information search patterns, we examined the chosen

hierarchies of the subjective condition. As seen in Table 3, 12 of the 30 participants (40%) intuitively adopted

the objective validity hierarchy (i.e., previous game-top team-big city), whereas 18 participants (60%)

adopted a different hierarchy. Of these 18, 15 participants (50%) adopted a subjective hierarchy in which the top-

team cuewas considered themost valid cue and the big city cuewas considered the least valid cue (i.e., top team-

previous game-big city). The remaining 3 participants (10%) adopted different cue validity hierarchies.

2The equal weighted model (EQW) (Dawes, 1979) was not tested separately, since in the majority of cases it includes similarcompensatory processes and thus makes the same predictions as the WADD. It should be noted, however, that the WADD classificationsmight therefore partially contain some EQW adherents.

Copyright # 2009 John Wiley & Sons, Ltd. Journal of Behavioral Decision Making, 22, 455–474 (2009)

DOI: 10.1002/bdm

Table 3. Proportion of correct responses (hit rate) as a function of cue hierarchy (Study 2)

Experimental condition Cue hierarchy N Hit rate

Objective PG—TT—BC 30 0.725 (0.097)Subjective PG—TT—BC 12 0.720 (0.121)

TT—PG—BC 15 0.710 (0.077)BC—PG—TT 2 0.718 (0.056)PG—BC—TT 1 0.692

N¼ number of participants employing the specific cue hierarchy (order of search pattern); PG¼ previous game; TT¼ top team; BC¼ bigcity. The right column contains proportions of hit rate across trials. Standard deviations are in brackets.

S. Ayal and G. Hochman The Cognitive Processes Underlying Choice Behavior 467

To provide additional evidence for the information search pattern and determine whether our experimental

paradigm replicated previous findings in the field, we evaluated the number of TTB violations, both in the

subjective and in the objective condition. Calculation of TTB violations relied on the parameters used by

Newell, Weston, and Shanks (2003). Our results appear to correspond to previous empirical examinations of

the TTB (e.g., Broder, 2000; Newell & Shanks, 2003; Newell et al., 2003), and support what was termed by

Broder (2000) the weak hypothesis, according to which only some but not all participants acquired

information and decided according to the prediction of the TTB strategy (for full information about

individual TTB violations in each of the experimental condition, see http://tx.technion.ac.il/�hochmang/

Online.htm).

Analysis of choice strategy classification

The analysis of individual choice patterns revealed a clear dominance of the WADD strategy. Namely, the

maximum likelihood estimations showed that 72% of the participants were classified as users of the WADD

strategy, whereas only 18% used the TTB. Choice strategy could not be determined for the remaining 10% of

the sample. In addition, we analyzed Bayes-ratios to determine the quality of the strategy classification. The

median of the Bayes ratio was 11305.84 for the WADD classifications and 8963.58 for the TTB

classification.3 Thus, in both cases the classification seems to be very robust.

These results confirm the compensatory prediction for choice proportion, not the TTB prediction.

Examining choice patterns that differentiated between compensatory and noncompensatory predictions,4 we

found that the majority of participants based their decisions on integration of information rather than on

limited information.

Analysis of confidence level

To examine the relative importance of the predictors of confidence level, we applied a dominance analysis

(Azen & Budescu, 2003; Budescu, 1993). Dominance analysis is a general method for comparing the relative

importance of predictors in a multiple regression, by comparing the R2 values from all possible subset

models. An evaluation of the relative contribution of each predictor to the R2 can determine the contribution

of each predictor in explaining the dependent variable (e.g., confidence level in our case).

The results of this analysis suggest that the WADD index completely dominated the precedence of cue, as

the additional contribution of the WADD index was greater than the contribution of the cue precedence

3Bayes-ratios reflect how likely it is that participants whowere classified as users of a certain choice strategy in fact used this strategy andnot the other one. Thus, these results suggest, for example, that it is 11306 times more likely that a participant who was classified as aWADD user was in fact a WADD user and not a TTB user (Broder, in press).4The average number of cases in which the predictions of the choice pattern differed between the two models across participants was22.05 (SD¼ 12.27) out of 40. This average is dependent upon actual behavior of participants (e.g., number of cues acquired).

Copyright # 2009 John Wiley & Sons, Ltd. Journal of Behavioral Decision Making, 22, 455–474 (2009)

DOI: 10.1002/bdm

Table 4. Dominance analysis of TTB and WADD as predictors of confidence-level (Study 2)

Subset model (X) r2X�LOC

Additional contribution

TTB WADD

All sampleTTB 0.03 0.13WADD 0.16 0.03

TTB adherentsTTB 0.038 0.036WADD 0.074 0.006

The column labeled r2X�LOC represents the variance in confidence-level explained by the model appearing in the corresponding row.Columns labeled TTB andWADD contain the additional contributions to the explained variance gained by adding the column variable tothe row model. Blank cells indicate that data are not applicable.

468 Journal of Behavioral Decision Making

variable (see the upper panel in Table 4). A regression analysis in which theWADD index and the precedence

of cue predicted confidence level together was significant (R2¼ 0.16, F (1, 405)¼ 76.46, p< 0.0001).

However, in this regression theWADD index was significant (b¼ 18.13, t (405)¼ 8.74, p< 0.0001), whereas

the precedence of cue was not significant (t (611)¼�1.53, p¼ 0.13). These results suggest that the

confidence level could be better explained by compensatory principles than by noncompensatory principles.

Note that it could be argued that the confidence level results are a byproduct of the low percentage of TTB

users in the current dataset. To examine this possibility, we conducted the same analysis only for those cases

in which the choice pattern was predicted by the TTB (i.e., the search was stopped for the first discriminating

cue and decision was based on this cue alone). The lower panel of Table 4 shows that this analysis yielded a

similar pattern of results. Again, the WADD index was found to dominate completely the precedence of cue.

A regression analysis revealed a significant overall prediction (R2¼ 0.071, F (1261)¼ 20.02, p< 0.0001).

Yet, while the WADD index was significant (b¼ 14.86, t (261)¼ 4.48, p< 0.0001), the precedence of cue

was not significant (t (261)¼�1.64, p¼ 0.10). These results suggest that even when the choice pattern is

captured by a noncompensatory strategy, there is evidence for underlying compensatory processes.

One-reason decision making or integration

Lastly, although the results of both the strategy classification and the confidence level analysis clearly

demonstrate that people tend to gather more information than predicted by noncompensatory models, one can

argue that individuals’ final decision is based on one reason, rather than on the integration of all the available

information. To examine this possibility, we analyzed the proportions of correct responses (hit rate).

Traditionally, the hit rate is used to assess the objective accuracy of the model. However, due to our cue

validity hierarchy manipulation, along with the environment that facilitated TTB use, the hit rate could

further indicate which process was used. More specifically, if individuals were basing their decision on the

most valid cue, while ignoring all other information (as implied by the TTB), the hit rate should be higher

when only few cues are acquired. Of course, this trend is expected only under the objective cue hierarchy but

not under the subjective cue hierarchy. On the other hand, if people integrate all the information they gather in

a compensatory manner, the hit rate should increase rather than decrease the more cues are acquired.

The results of the hit rate analysis showed that across choice problems the hit rate increased as a function

of the number of cues, regardless of the cue validity hierarchy. A 3� 2 ANOVA was conducted to test the

effect of the number of cues (one, two, or three) and the cue validity hierarchy (objective, subjective) on hit

rate as the dependent variable. This analysis revealed a significant main effect of the number of cues (F (2,

74)¼ 6.213, p< 0.005). Neither a main effect of the type of cue validity hierarchy was found, nor an

interaction between the two variables.

Copyright # 2009 John Wiley & Sons, Ltd. Journal of Behavioral Decision Making, 22, 455–474 (2009)

DOI: 10.1002/bdm

S. Ayal and G. Hochman The Cognitive Processes Underlying Choice Behavior 469

This pattern of results converges with the strategy classification and the confidence level results. The fact

that under both conditions more cues increased the hit rate rather than decreased, it suggests that people tend

to integrate acquired information in a compensatory manner.

In summary, since not all available information was acquired (in several cases no information was

acquired at all), the results of Study 2 imply that decisionmakers tend to save resources (e.g., processing time,

costs) by neglecting some of the available information, as suggested by the fast-and-frugal framework (e.g.,

Gigerenzer et al., 1999). At the same time, these results indicate that people do not neglect information that

has already been gathered, as seen by the fact that such information affects their choice patterns, their

confidence judgments, and their hit rates.

GENERAL DISCUSSION

The theoretical appeal of noncompensatory models lies in their parsimony and in their potentially impressive

predictive power under certain situations. Traditionally, these models were designed to predict choice

behavior (i.e., paramorphic models), and they reflected, to some extent, a tradeoff between simplicity and

descriptive accuracy. For example, in an early noncompensatory model proposed by Tversky (i.e.,

Elimination by Aspects, 1972) the author stated that ‘‘. . .there may be many contexts in which it (the model)

provides a good approximation to much more complicated compensatory models and could thus serve as a

useful simplification procedure. . .’’ (p. 298). However, the fast-and-frugal framework undermines this

traditional convention, suggesting that a new characterization of the mechanism underlying choice behavior

is in order, and that this new framework should be strictly noncompensatory in nature, and free of the classic

compensatory principles of integration.

The results of the current study indicate that the fast-and-frugal framework might be too simplistic, as it

cannot capture the full essence and complexity of the processes underlying choice behavior. Five hypotheses

concerning indirect measures of processes were derived from the PH and the TTB models and were

juxtaposed with contradicting compensatory hypotheses to conduct an empirical examination of the

mechanisms underlying choice behavior. The pattern of results suggests that the compensatory model is

better at predicting decision behavior than is the noncompensatory model.

Specifically, when information was provided simultaneously, and at no cost (i.e., in the preference task in

Study 1), RT, choice proportion, and confidence level in one’s judgment were all better explained by

compensatory principles, such as summing and weighting. These results converge with recent findings (e.g.,

Glockner & Betsch, 2008a; Johnson et al., 2008) according to which people exhibit much more evidence for

integrating data in risky choice than for relying on limited data and ignoring the rest. Moreover, the choice

pattern analysis suggests not only that noncompensatory models cannot capture the underlying processes of

human preferences, but also that they are poorly qualified as as-if models (cf. Birnbaum, 2008; Glockner &

Betsch, 2008a; Rieger &Wang, 2008). The same pattern of results was replicated in an environment in which

information was provided sequentially and at high costs (i.e., the inference task in Study 2), an environment

traditionally considered to encourage the employment of noncompensatory choice strategies. As in Study 1,

the process measures of choice proportion and level of confidence, as well as a measure of accuracy, were

better explained by compensatory principles. It could be argued, however, that the manipulation in Study 2

provided no opportunity to learn the structure of the environment. Thus, participants failed to learn that the

TTB is an appropriate strategy (cf. Rieskamp & Otto, 2006). This account can explain why the TTB choice

proportions were low. In addition, instead of arguing that compensatory processes govern these tasks, it could

simply be argued that in unfamiliar environments people might tend to employ compensatory strategies (e.g.,

WADD) (see for example Broder, 2003). It is important to point out, however, that the level of confidencewas

predicted by the compensatory index alone even in those cases in which participants were classified as TTB

adherents exclusively (i.e., when the TTB accurately predicted choice pattern). Namely, even when the TTB

predicted choice behavior, it could not account for the underlying processes.

Copyright # 2009 John Wiley & Sons, Ltd. Journal of Behavioral Decision Making, 22, 455–474 (2009)

DOI: 10.1002/bdm

470 Journal of Behavioral Decision Making

In summary, the results from different indirect measures (i.e., RT, choice proportion, confidence level, and

hit rate) that were tested in diverse environments converge and point to the conclusion that compensatory

processes of integration dominate choice behavior. Although the use of noncompensatory tools in certain

situations could not be ruled out, it seems that people do not forsake more complex compensatory processes

of integration, and prefer to attend to previously gathered information.

Note that Brandstatter et al. (2006) were familiar with the conclusion that people might employ

compensatory tools, and suggested that the PH model might have been stronger had it assumed that people

computed EVs, took their ratio into account, and chose the gamble with the highest EV when the ratio

exceeded 2. The results of the current study support this conclusion, and posit that the coexistence of

compensatory and noncompensatory principles may lead to the formation of better decision-making models

(see also Payne et al., 1992). Importantly, Brandstatter et al. (2006) suggested that measures of reaction time

and choice proportions could examine the process underlying choice. However, this line of research has led to

inconsistent conclusions in the past (i.e., support of noncompensatory principles in Brandstatter et al. and of

compensatory principles in Glockner & Betsch, 2008a). The current study sheds light on this puzzle and

suggests that the predictive accuracy of the noncompensatory models primarily depends on their agreement

with compensatory predictions, and thus it is reduced dramatically in diagnostic situations (cf. Birnbaum,

2008; Glockner & Betsch, 2008a).

The use of confidence level in the comparison of compensatory and noncompensatory hypotheses is quite

novel (but see also Glockner, 2006; Glockner & Betsch, 2008b). The importance of measuring subjective

levels of confidence in a choice stems not only from the direct connection between this confidence level and

basic assumptions of noncompensatory models, but also from the fact that confidence level may reflect a

subjective evaluation of the processes that led to a certain decision. Indeed, the fact that evidence for

compensatory processes was obtained both through a meta-cognitive measure (confidence level) and through

different controlled and uncontrolled measures (i.e., RT, choice proportion, and hit rate) strengthens the

validity of the present results. In addition, these results highlight the importance of usingmultiple measures to

examine and understand the processes underlying choice behavior, and to distinguish between competing

theories.

The current results lead us to suggest a model according towhich choice behavior integrates compensatory

and noncompensatory principles. Under such a model, decision makers start a choice task with a preliminary

intuitive or automated compensatory process of integrating all (or most of) the available information. For

example, this integration is used to evaluate the expected value of the alternatives, or to evaluate the validity

of the available cues. If the results of this preliminary process are satisfactory (e.g., the difference between the

EVs is large enough) (cf. Decision Field Theory of Busemeyer & Townsend, 1993), the decision process is

terminated and a decision is made. In all other cases, decision makers use a rational compensatory selection

among the many tools available at their disposal. Presumably, most of these tools are noncompensatory in

nature.

Our model explains why previous research reported high predictive power for noncompensatory models

(e.g., Brandstatter et al., 2006; Gigerenzer & Goldstein, 1996; Gigerenzer et al., 1999) but failed to support

process predictions (e.g., Glockner & Betsch, 2008a; Johnson et al., 2008). The noncompensatory models’

predictive power is evident in two specific situations. In the first situation, the preliminary compensatory

model governs choice but the final outcome also fits with noncompensatory predictions (i.e., non diagnostic

situations). In the second situation, the preliminary compensatory screening does not provide a clear answer

(e.g., choice between two alternatives with similar expected utility), and thus the final decision is based on a

noncompensatory heuristic. In any case, the model assumes that the default initial process underlying choice

behavior is compensatory.

Thus, contrary to the conclusions made by Gigerenzer and colleagues (e.g., Brandstatter et al., 2006;

Gigerenzer et al., 1999), the claim that decision makers have noncompensatory heuristics in their toolbox

highlights the importance of the compensatory processes, rather than makes them dispensable.

Copyright # 2009 John Wiley & Sons, Ltd. Journal of Behavioral Decision Making, 22, 455–474 (2009)

DOI: 10.1002/bdm

APPENDIX 1: THE CHOICE PROBLEMS OF THE PRIORITY REPLICATION SET (STUDY 1)

One reason examined Three reason examined

EV similar EV dissimilar EV similar EV dissimilar

(2000,0.60; 500, 0.40) (3000,0.60; 1500, 0.40) (2000,0.10; 500, 0.90) (5000,0.10; 500, 0.90)(2000, 0.40; 1000, 0.60) (2000, 0.40; 1000, 0.60) (2500, 0.05; 550, 0.95) (2500, 0.05; 550, 0.95)

(5000, 0.20; 2000, 0.80) (6000, 0.20; 3000, 0.80) (4000, 0.25; 3000, 0.75) (7000, 0.25; 3000, 0.75)(4000, 0.50; 1200, 0.50) (4000, 0.50; 1200, 0.50) (5000, 0.20; 2800, 0.80) (5000, 0.20; 2800, 0.80)

(4000, 0.20; 2000, 0.80) (5000, 0.20; 3000, 0.80) (6000, 0.30; 2500, 0.70) (9000, 0.30; 2500, 0.70)(3000, 0.70; 1000, 0.30) (3000, 0.70; 1000, 0.30) (8200, 0.25; 2000, 0.75) (8200, 0.25; 2000, 0.75)

(900, 0.40; 500, 0.60) (1900, 0.40; 1500, 0.60) (3000, 0.40; 2000, 0.60) (6000, 0.40; 2000, 0.60)(2500, 0.20; 200, 0.80) (2500, 0.20; 200, 0.80) (3600, 0.35; 1750, 0.65) (3600, 0.35; 1750, 0.65)

(1000, 0.50; 0, 0.50) (2000, 0.50; 1000, 0.50) (2500, 0.33; 0, 0.67) (5500, 0.33; 0, 0.67)(500, 1.00) (500, 1.00) (2400, 0.34; 0, 0.66) (2400, 0.34; 0, 0.66)

All the choice problems were taken from the problems used in Brandstatter et al. (2006) original priority heuristic article.The choice problems are classified here according to (a) the number of priority heuristic rules that are required to be examined (one vs.three), and (b) the level of similarity between the excepted values (similar vs. dissimilar).

APPENDIX 2: THE CHOICE PROBLEMS OF THE WAKKER ET AL. SET (STUDY 1)

One reason examined Three reason examined

EV similar EV dissimilar EV similar EV dissimilar

(4.5, 0.25; 9, 0.20; 6.5, 0.55) (3, 0.40; 2, 0.50; 0, 0.10) (4.5, 0.25; 9, 0.20; .5, 0.55) (0.5, 0.40; 7.5, 0.20; 1.5, 0.40)(6, 0.25; 7, 0.20; 6.5, 0.55) (2, 1.00) (6, 0.25; 7, 0.20; 0.5, 0.55) (0.5, 0.40; 6, 0.20; 2.5, 0.40)

(4.5, .25; 9, 0.20; 9.5, 0.55) (3, 0.40; 0, 0.60) (4.5, 0.25; 9, 0.20; 3.5, 0.55) (3, 0.40; 7.5, 0.20; 1.5, 0.40)(6, .25; 7, 0.20; 9.5, 0.55) (2.5, 0.50; 1.5, 0.50) (6, 0.25; 7, 0.20; 3.5, 0.55) (3, 0.40; 6, 0.20; 2.5, 0.40)

(3.5, 0.10; 6, 0.70; 12.5, 0.20) (6, 0.15; 0.5, 0.65; 3, 0.20) (5.5, 0.40; 7.5, 0.20; 1.5, 0.40)(5.5, 0.10; 6, 0.70; 10.5, 0.20) (5.5, 0.15; 0.5, 0.65; 3.5, 0.20) (5.5, 0.40; 6, 0.20; 2.5, 0.40)

(3.5, 0.10; 9.5, 0.70; 12.5, 0.20) (6, 0.15; 2.5, 0.65; 3, 0.20) (8, 0.40; 7.5, 0.20; 1.5, 0.40)(5.5, 0.10; 9.5, 0.70; 10.5, 0.20) (5.5, 0.15; 2.5, 0.65; 3.5, 0.20) (8, 0.40; 6, 0.20; 2.5, 0.40)

(3.5, 0.10; 13, 0.70; 12.5, 0.20) (6, 0.15; 4.5, 0.65; 3, 0.20) (3.5, 0.10; 2.5, 0.70; 12.5, 0.20)(5.5, 0.10; 13, 0.70; 10.5, 0.20) (5.5, 0.15; 4.5, 0.65; 3.5, 0.20) (5.5, 0.10; 2.5, 0.70; 10.5, 0.20)

(3, 0.40; 0, 0.10; 2, 0.50) (6, 0.15; 6.5, 0.65; 3, 0.20) (3, 0.40; 0, 0.60)(2, 1.00) (5.5, 0.15; 6.5, 0.65; 3.5, 0.20) (2, 0.50; 0, 0.50)

(3, 0.40; 0, 0.10; 4, 0.50) (2, 0.20; 0, 0.80) (2, 0.60; 5, 0.40)(2, 0.50; 4, 0.50) (1.5, 0.25; 0, 0.75) (2, 0.50; 4, 0.50)

(3, 0.40; 0, 0.10; 6, 0.50)(2, 0.50; 6, 0.50)

(4, 0.50; 2, 0.10; 5, 0.40)(4, 1.00)

(6, 0.50; 2, 0.10; 5, 0.40)(6, 0.50; 4, 0.50)

(8, 0.50; 2, 0.10; 5, 0.40)(8, 0.50; 4, 0.50)

(4, 0.10; 0, 0.80; 1, 0.10)(1, 0.20; 2, 0.80)

(2, 0.80; 0, 0.20)(1.5, 1.00)

All these choice problems were presented in the original article ofWakker et al. (1994). The choice problems are classified here accordingto (a) the number of priority heuristic rules that are required to be examined (one vs. three), and (b) the level of similarity between theexcepted values (similar vs. dissimilar).

Copyright # 2009 John Wiley & Sons, Ltd. Journal of Behavioral Decision Making, 22, 455–474 (2009)

DOI: 10.1002/bdm

S. Ayal and G. Hochman The Cognitive Processes Underlying Choice Behavior 471

472 Journal of Behavioral Decision Making

REFERENCES

Allais, M. (1953/1979). La psychologie de l’home rationnel devant le risque: Critique des postulats et axiomes de l’ecoleAmericaine. JT Econometrica, 21, 503–546. Translated and reprinted in Allais & Hagen, 1979 (in French).

Azen, R., & Budescu, D. V. (2003). The dominance analysis approach for comparing predictors in multiple regression.Psychological Methods, 8, 129–148.

Bergert, F. B., & Nosofsky, R. M. (2007). A response-time approach to comparing generalized rational and take-the-bestmodels of decision making. Journal of Experimental Psychology: Learning, Memory, and Cognition, 33, 107–129.

Birnbaum, M. H. (2008). Evaluation of the priority heuristic as a descriptive model of risky decision making: Commenton Brandstatter, Gigerenzer, and Hertwig (2006). Psychological Review, 115, 253–262.

Brandstatter, E., Gigerenzer, G., & Hertwig, R. (2006). The priority heuristic: Making choices without trade-offs.Psychological Review, 113, 409–432.

Broder, A. (2000). Assessing the empirical validity of the ‘‘take-the-best’’ heuristic as a model of human probabilisticinference. Journal of Experimental Psychology: Learning, Memory, and Cognition, 26, 1332–1346.

Broder, A. (2002). Take the best, Dawes’ rule, and compensatory decision strategies: A regression-based classificationmethod. Quality and Quantity, 36, 219–238.

Broder, A. (2003). Decision making with the ‘‘adaptive toolbox’’: Influence of environmental structure, intelligence,and working memory load. Journal of Experimental Psychology: Learning, Memory, and Cognition, 29, 611–625.

Broder, A., & Schiffer, S. (2003a). ‘‘Take the best’’ versus simultaneous feature matching: Probabilistic inferences frommemory and effects of representation format. Journal of Experimental Psychology: General, 132, 277–293.

Broder, A., & Schiffer, S. (2003b). Bayesian strategy assessment in multi-attribute decision making. Journal ofBehavioral Decision Making, 16, 193–213.

Broder, A. (in press). Outcome-based strategy classification. In A. Glockner , & C. L. M. Witteman (Eds.), Tracingintuition: Recent methods in measuring intuitive and deliberate processes in decision making. London: PsychologyPress & Routledge.

Budescu, D. V. (1993). Dominance analysis: A new approach to the problem of relative importance of predictors inmultiple regression. Psychological Bulletin, 114, 542–551.

Budescu, D. V. (2006). Confidence in aggregation of opinions from multiple sources. In K. Fiedler , & P. Juslin (Eds.),Information sampling and adaptive cognition (pp. 327–352). New York, NY; Cambridge University Press.

Busemeyer, J. R., & Townsend, J. T. (1993). Decision field theory: A dynamic cognition approach to decision making.Psychological Review, 100, 432–459.

Dawes, R. M. (1979). The robust beauty of improper linear models in decision making. American Psychologist, 34, 571–582.

Einhorn, H. J. (1971). Use of nonlinear, noncompensatory models as a function of task and amount of information.Organizational Behavior and Human Performance, 6, 1–27.

Elrod, T., Johnson, R. D., & White, J. (2004). A new integrated model of noncompensatory and compensatory decisionstrategies. Organizational Behavior and Human Decision Processes, 95, 1–19.

Erev, I., Roth, A. E., Slonim, R. L., & Barron, G. (2002). Combining a theoretical prediction with experimental evidenceto yield a new prediction: An experimental design with a random sample of tasks. Unpublished manuscript, ColumbiaUniversity and Faculty of Industrial Engineering and Management, The Technion, Haifa, Israel.

Erev, I., Wallsten, T. S., & Budescu, D. V. (1994). Simultaneous over- and underconfidence: The role of error in judgmentprocesses. Psychological Review, 101, 519–527.

Fishburn, P. C. (1974). Lexicographic order, utilities and decision rules. Management Science, 20, 1442–1471.Ford, J. K., Schmitt, N., Schechtman, S. L., Hults, B. M., & Doherty, M. L. (1989). Process tracing methods:

Contributions, problems and neglected research questions. Organizational Behavior and Human Decision Processes,43, 75–117.

Gigerenzer, G. (1997). Bounded rationality: Models of fast and frugal inference. Swiss Journal of Economics andStatistics, 133, 201–218.

Gigerenzer, G., & Goldstein, D. G. (1996). Reasoning the fast and frugal way: Models of bounded rationality.Psychological Review, 103, 650–669.

Gigerenzer, G., & Goldstein, D. G. (1999). Betting on one good reason: The take the best heuristic. In G. Gigerenzer ,P. M. Todd , & the ABC Research Group (Eds.), Simple heuristics that make us smart (pp. 75–95). Oxford, England:Oxford University Press.

Gigerenzer, G., & Hug, K. (1992). Domain-specific reasoning: Social contracts, cheating, and perspective change.Cognition, 43, 127–171.

Copyright # 2009 John Wiley & Sons, Ltd. Journal of Behavioral Decision Making, 22, 455–474 (2009)

DOI: 10.1002/bdm

S. Ayal and G. Hochman The Cognitive Processes Underlying Choice Behavior 473

Gigerenzer, G., Hoffrage, U., & Kleinbolting, H. (1991). Probabilistic mental model: A Brunswikian theory ofconfidence. Psychological Review, 98, 506–528.

Gigerenzer, G., Todd, P. M. & the ABC Research Group. (1999). Simple heuristics that make us smart. Oxford, England:Oxford University Press.

Glockner, A. (2006). Automatische Prozesse bei Entscheidungen (Automatic processes in decision making). Hamburg:Kovac. (in German).

Glockner, A. (2007). Does intuition beat fast and frugal heuristics? A systematic empirical analysis. In H. Plessner , C.Betsch , & T. Betsch (Eds.), Intuition in judgment and decision making (pp. 309–326). New York: Lawrence ErlbaumAssociates.

Glockner, A. (in press). Multiple measure strategy classification: Outcomes, decision times and confidence ratings. In A.Glockner , & C. L. M. Witteman (Eds.), Tracing intuition: Recent methods in measuring intuitive and deliberateprocesses in decision making. London: Psychology Press & Routledge.

Glockner, A., & Betsch, T. (2007). Evidence for multiple-reason decision making under time limits. Manuscript submittedfor publication.

Glockner, A., & Betsch, T. (2008a). Do people make decisions under risk based on ignorance? An empirical test of thepriority heuristic against cumulative prospect theory. Organizational Behavior and Human Decision Processes, 107,75–95.

Glockner, A., & Betsch, T. (2008b). Multiple-reason decision making based on automatic processing. Journal ofExperimental Psychology: Learning, Memory, and Cognition, 34, 1055–1075.

Hertwig, R., Barron, G., Weber, E. U., & Erev, I. (2004). Decision from experience and the effect of rare events.Psychological Science, 15, 534–539.

Hoffrage, U. (2005). Heuristics: Fast and frugal. In B. C. Everitt , & D. C. Howell (Eds.), Encyclopedia of statistics inbehavioral science (Vol. 2, pp. 795–799). Chichester, UK: Wiley.

Johnson, E. J., Schulte-Mecklenbeck, M., &Willemsen, M. C. (2008). Process models deserve process data: A commenton Brandstatter, Gigerenzer, and Hertwig (2006. Psychological Review, 115, 263–273.

Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47, 263–291.

Lee, M. D., & Cummins, T. D. R. (2004). Evidence accumulation in decision making: Unifying the ‘take the best’ and‘rational’ models. Psychonomic Bulletin and Review, 11, 343–352.

Lopes, L. L., & Oden, G. C. (1999). The role of aspiration level in risky choice: A comparison of cumulative prospecttheory and SP/A theory. Journal of Mathematical Psychology, 43, 286–313.

Newell, B. R., & Shanks, D. R. (2003). Take the best or look at the rest? Factors influencing ‘‘one-reason’’ decisionmaking. Journal of Experimental Psychology: Learning, Memory, and Cognition, 29, 53–65.

Newell, B. R., Weston, N. J., & Shanks, D. R. (2003). Empirical tests of a fast and frugal heuristic: Not everyone ‘‘takes-the-best.’’ Organizational Behavior and Human Decision Processes, 91, 82–96.

Payne, J. W., Bettman, J. R., Coupey, E., & Johnson, E. J. (1992). A constructive process view of decision making:Multiple strategies in judgment and choice. Acta Psychologica, 80, 107–141.

Payne, J. W., Bettman, J. R., & Johnson, E. J. (1993). The adaptive decision maker. New York: Cambridge UniversityPress.

Rieger, M. O., & Wang, M. (2008). What is behind the priority heuristic? A mathematical analysis and comment onBrandstatter, Gigerenzer, and Hertwig (2006. Psychological Review, 115, 274–280.

Rieskamp, J., & Hoffrage, U. (1999). When do people use simple heuristics and how can we tell? In G. Gigerenzer , &P. M. Todd , & the ABC Research Group (Eds.), Simple heuristics that make us smart (pp. 141–167). Oxford, England:Oxford University Press.

Rieskamp, J., & Hoffrage, U. (2005). The use of simple heuristics: Inferences and preferences under time pressure.Manuscript submitted for publication.

Rieskamp, J., & Otto, P. E. (2006). SSL: A theory of how people learn to select strategies. Journal of ExperimentalPsychology: General, 135, 207–236.

Tversky, A. (1972). Elimination by aspects: A theory of choice. Psychological Review, 79, 281–299.Tversky, A., & Kahneman, D. (1992). Advances in prospect theory: Cumulative representation of uncertainty. Journal of

Risk and Uncertainty, 5, 297–323.von Neumann, J., & Morgenstern, O. (1944). Theory of game and economic behavior. Princeton: Princeton UniversityPress.

Wakker, P., Erev, I., & Weber, E. U. (1994). Comonotonic independence: The critical test between classical and rank-dependent utility theories. Journal of Risk and Uncertainty, 9, 195–230.

Winkler, R. L., & Poses, R. M. (1993). Evaluating and combining physicians’ probabilities of survival in an intensive careunit. Management Science, 39, 1526–1543.

Copyright # 2009 John Wiley & Sons, Ltd. Journal of Behavioral Decision Making, 22, 455–474 (2009)

DOI: 10.1002/bdm

474 Journal of Behavioral Decision Making

Authors’ biographies:

Shahar Ayal is a postdoctoral fellow at Fuqua School of Business, Duke University. His research interests are heuristicsand biases, risk perception, unethical behavior, and the effect of inter-group relations on decision making.

Guy Hochman is a psychology doctoral student at the Technion—Israel Institute of Technology. His research interestsare heuristics and biases, process models, and physiological measures and their relation to decision-making processes.

Authors’ addresses:

Shahar Ayal, Duke University, The Fuqua School of Business, 1 Towerview Drive, Durham, NC 27708, USA

Guy Hochman, William Davidson Faculty of Industrial Engineering and Management. Technion - Israel Institute ofTechnology, Technion City, Haifa 32000, Israel.

Copyright # 2009 John Wiley & Sons, Ltd. Journal of Behavioral Decision Making, 22, 455–474 (2009)

DOI: 10.1002/bdm