Testing Transitivity with Individual Data

Michael H. Birnbaum and Jeffrey P. Bahra

California State University, Fullerton

Transitivity of Preference

• If A > B and B > C then A > C.• Weak Stochastic Transitivity: • If P(A, B) > 1/2 and P(B, C) > 1/2

then P(A, C) > 1/2.

Tversky (1969)

• Tversky (1969) reported that selected subjects showed a pattern of intransitive data consistent with a lexicographic semi-order.

• Tversky tested Weak Stochastic Transitivity.

Issues

• Iverson & Falmagne (1985) argued that Tversky’s statistical analysis was incorrect of WST.

• Tversky went on to publish transitive theories of preference (e.g., CPT).

Renewed Interest in Intransitive Preference

• New analytical methods for analysis of transitivity (Iverson, Myung, & Karabatsos; Regenwetter & Stober, et al); Error models (Sopher & Gigliotti, ‘93; Birnbaum, ‘04; others).

• Priority Heuristic (Brandstaetter, et al., 2006); stochastic difference model (González-Vallejo,

2002; similarity judgments, Leland, 1994; majority rule, Zhang, Hsee, Xiao, 2006). Renewed interest in Fishburn, as well as in Regret Theory.

Lexicographic Semi-order• G = (x, p; y, 1 - p). F = (x’, q; y’, 1 -

q).

• If y - y’ ≥ L choose G

• If y’ - y ≥ L choose F

• If p - q ≥ P choose G

• If q - p ≥ P choose F

• If x > x’ choose G; if x’ > x choose F;• Otherwise, choose randomly.

Priority Heuristic• 10% of largest prize, rounded to nearest

prominent number(In this study, L = $10).

• Compare gambles by lowest consequences. If difference exceeds the aspiration level, choose by lowest consequence.

• If not, compare probabilities; choose by probability if difference ≥ 0.1 (P = 0.1).

• Compare largest consequences; choose by largest consequences.

Birnbaum & Gutierez (OBHDP, in press)

• Four studies used Tversky’s 5 gambles, formatted with tickets or with pie charts.

• Failed to find evidence that more than a very small percentage of participants (~ 6%) were intransitive.

• Other tests refuted lexicographic semiorder and priority heuristic.

Pie Chart Format

Response to Birnbaum-Gutierrez

• Perhaps the intransitivity only develops in longer studies. Tversky used 20 replications of each choice.

• Perhaps consequences of Tversky’s gambles diminished since 1969 due to inflation. Perhaps those prizes are now too small.

Birnbaum & Bahra• Collected up to 40 choices/pair per

person. (20 reps). 2 Sessions, 1.5 hrs, 1 week apart.

• Cash prizes up to $100. • 51 participants, of whom 10 to win

the prize of one of their chosen gambles.

• 3 5 x 5 Designs to test transitivity vs. Priority heuristic predictions

Choice Format

Notation-Two-branch Gambles

• G = (x, p; y, 1 - p); x > y ≥ 0• L = Lower Consequence• P = Probability to win higher prize• H = Higher consequence

LH (Lower & Higher Consequences) Design

• A = ($84, .50; $24)• B = ($88, .50; $20)• C = ($92, .50; $16)• D = ($96, .50; $12)• E = ($100, .50; $8)

LP (Lower & Probability) Design

• F = ($100, .50; $24)• G = ($100, .54; $20)• H = ($100, .58; $16)• I = ($100, .62; $12)• J = ($100, .66; $8)

PH (Probability & Higher) Design

• K = ($100, .50; $0)• L = ($96, .54; $0)• M = ($92, .58; $0)• N = ($88, .62; $0)• O = ($84, .66; $0)

Priority Heuristic implies: Intransitive and Consistent

LH design Second GambleFirst A B C D E

A=(84, 24) 2 2 1 1B= (88, 20) 1 2 2 1C= (92, 16) 1 1 2 2D= (96, 12) 2 1 1 2E= (100, 8) 2 2 1 1

Transitive & Consistent1 = Chose First; 2 = Chose

SecondSecond Gamble

First A B C D EA 1 1 1 1B 2 1 1 1C 2 2 1 1D 2 2 2 1E 2 2 2 2

Within-Rep Consistency

• Count the number of consistent choices in a replicate of 20 choices (10 x 2).

• If a person always chose the same button, consistency = 0.

• If a person were perfectly consistent within replicate, consistency = 10.

• Randomly choosing between responses produces expected consistency of 5.

Within-Replicate Consistency

• The average rate of agreement was 8.63 (86% self-agreement).

• 46.4% of all replicates were scored 10.

• An additional 19.9% were scored 9.

LH Design: Overall Proportions Choosing Second Gamble

Second GambleFirst A B C D E

A= (84, 24) 0.41 0.38 0.34 0.27

B= (88, 20) 0.58 0.40 0.36 0.30

C= (92, 16) 0.61 0.59 0.44 0.32

D= (96, 12) 0.64 0.61 0.55 0.33

E= (100, 8) 0.70 0.69 0.66 0.66

LP Design: Overall Proportions Choosing Second Gamble

LP Second GambleFirst F G H I J

F 0.44 0.43 0.42 0.36G 0.54 0.42 0.42 0.38H 0.54 0.55 0.45 0.40I 0.56 0.56 0.53 0.41J 0.60 0.59 0.57 0.56

PH Design: Overall Proportions Choosing Second Gamble

PH Second GambleFirst K L M N O

K 0.61 0.64 0.64 0.64L 0.37 0.61 0.63 0.65M 0.34 0.37 0.64 0.64N 0.34 0.35 0.33 0.63O 0.34 0.33 0.35 0.34

Averaged Data Fit WST

• LH Design A > B > C > D > E• LP Design F > G > H > I > J• PH Design O > N > M > L > K• Consistent with special TAX with its

“prior” parameters.• This analysis obscures individual

diffs

Individual Data

• Choice proportions calculated for each individual in each design.

• These were further examined within each person replication by replication.

S# 8328 C = 9.6 Reps = 20

S# 8328 C = 9.8 Reps = 20

LP Second GambleF G H I J

F 0.05 0.02 0.00 0.00G 0.00 0.00 0.00H 0.05 0.02I 0.00J

S# 8328 C = 9.9 Reps = 20

PH Second GambleK L M N O

K 1.00 1.00 1.00 0.98L 1.00 1.00 1.00M 0.95 1.00N 0.95O

S# 6176 C = 9.8 Rep = 20; started with this pattern, then switched to perfectly consistent

with the opposite pattern for 4 replicates at the end of the first day; back to this pattern for 10

reps on day 2.

PH Second GambleK L M N O

K 0.28 0.20 0.23 0.20L 0.25 0.20 0.20M 0.20 0.20N 0.20O

Data Summary

• With n = 51, there are 153 matrices. Of these, 87% were perfectly consistent with WST: P(A,B) ≥ 1/2 & P(B,C) ≥ 1/2 then P(A,C) ≥ 1/2.

• 32 people (63%) had all three arrays fitting WST; no one fit priority heuristic nor did anyone have all three intransitive arrays.

• Those arrays that were not perfect fits to WST were either close to perfect, from “noisy” participants, or from people who changed orders.

Within-Person Changes in Preference Pattern

• Criterion: Person show perfect consistency (10 out of 10) to one pattern in one replication, and perfect consistency to different pattern on another replication.

• 16 Such arrays were found (~10% of 153) involving 12 participants.

• This result is troubling to the assumption that errors arise independently, but consistent with idea that people have changing parameters that drift rather than random changes.

Summary

• Recent studies fail to confirm systematic violations of transitivity predicted by lexicographic semiorders, including priority heuristic.

• No individual’s data agreed with predictions of priority heuristic. These data add to growing case against this model as a description.

• Individual data are mostly transitive.