Subjectivity in Decision Analysis David L. Olson James & H.K. Stuart Professor in MIS University of Nebraska Lincoln Human Centered Processes - 2008

Subjectivity in Decision Analysis

David L. OlsonJames & H.K. Stuart Professor in MIS

University of Nebraska Lincoln

Human Centered Processes - 2008

Rational Choice TheoryK. Morrell, Journal of Business Ethics [2004]

• Dominant model for business decision making

• Compared with Image Theory– Utilitarian ethics

• both consistent

– Kantian ethics• Image theory yes, RCT no

– Virtue-based ethics• Image theory yes, RCT no


Ideal

• Objective measuresMax Weber

• Accurate preference input

• “Rational” decision makerAccounting: Jensen - Agency Theory

Economics: Williamson - Transaction Cost Analysis


Basic Preference Model

Can use multiplicative model for interactions

ijs

iw

K

j

i

swValue

ij

i

K

iijij

criterion on ealternativ of score

criterion ofweight

criteria ofnumber

index ealternativ

indexcriterion 1


Objective Measures

• Objective preferred– can measure

• past profit, after tax

• Subjective– know conceptually, but can’t accurately

measure• response to advertising


Problems with Objective Approach

von Neumann & Morgenstern [1944] Theory of Games– utility is measurable

Georgescu-Roegen [1954] The Quarterly Journal of Economics – requires to many assumptions of rationality

Lindblom [1965] Public Administration Review– muddling through

Morgenstern [1972] Journal of Economic Literature – 13 critical points

• uncertainty

• ambiguity

• disagreement in groupsHuman Centered Processes - 2008

EMPIRICAL EVIDENCE

contrary to rational choice models

Braybrooke & Lindblom [1969]; Simon [1985] Payne, et al. [1993]

• Some problems never reach decision maker

• decision makers often have simple maps of real problems

• all alternatives not known, so decision makers do not have full, relevant information

• individual altruism

Tversky [1969]

• systematic & predictable economic intransitivities

Kahneman, Slovic & Tversky [1982]

• people use heuristics rather than follow rational model


James G. March

Bell Journal of Economics [1978]

• Rational choice involves guesses:– About future consequences of current actions– About future preferences of those consequences

Administrative Science Quarterly [1996]

• Alternatives & their consequences aren’t given, but need to be discovered & estimated

• Bases of action aren’t reality, but perceptions of reality• Supplemental exchange theories emphasize the role of

institutions in defining terms of rationality


Overview

• Inputs to preference models involve subjectivity– Weights are function of individual

– Scores also valued from perspective of individual

• Subjective assessment MAY be more accurate• Purpose of analysis should be to design better

alternatives


Means to Cope

Payne, Bettman & Johnson [1993]

• strategy will differ by number of alternatives– few - focus on all relevant information

– many - noncompensatory simplifying heuristics

• $/lives tradeoff varies by context

• Hogarth: many find explicit tradeoffs uncomfortable

• PROSPECT THEORY: initial analysis simple, weed out; for selected alternatives, more detailed

• As people learn more about problem structure, construct choice strategies


Objective/Subjective

• OBJECTIVE: what is convenient to model– ideal - eliminate bias, arbitrary judgment– extreme: cost/benefit analysis spanning years of

measuring the unmeasurable

• SUBJECTIVE: what people do to cope– value is subjective after all anyway– value is what MAUT, MCDA seeks to measure


ACCURATE PREFERENCE INPUT

• incomplete information• uncertain measures

• uncertain preferences

• group participation

• risk• time pressure: Edwards - how can you calculate

expected utility in available time?• change competition complexity


RELATIONSHIP TO MCDA

• We shouldn’t expect so much theoretical purity– the world has shifted away from appreciation of

numerical analysis

• Just because assumptions are not met doesn’t mean pure approach better


MCDA Methods

• Multiattribute utility theory

• Analytic hierarchy process

• Outranking– ELECTRE, PROMETHEE

• Fuzzy, DEA, Verbal Decision Analysis

• Image Theory


Spectrum

MAUT with strictly objective measures

MAUT with constructed measures

Likert scales

SMART - swing weighting rather than lottery tradeoffs

AHP - ratios of subjective scale


PROMETHEE Spectrum

Class I:ordinal

Class II: step advantage

Class III: proportional advantage (in range)

Class IV: three step

Class V: proportional with indifference

range

Class VI: normal distributionHuman Centered Processes - 2008

MAUT Hierarchy

Overall


Cost$billions

Lives lostExpected value

RiskProbability of major catastrophe

Civic improvement Families with upgraded housing

Objective Measures

Cost Lives Lost Risk Civic improvement

Nome AK 39.548 billion 61 0.0165 312 upgrades

Newark NJ 98.467 billion 143 0.0002 68,472 upgrades

Rock Springs WY

58.930 billion 41 0.0036 4,138 upgrades

Duquesne PA 60.156 billion 39 0.0069 20,653 upgrades

Gary IN 69.693 billion 86 0.0027 56,847 upgrades


Swing Weighting

Attribute On BEST On WORST

W on Best W on Worst

Final

Lives lost 100 290 0.541 0.569 0.556

Risk 60 175 0.324 0.343 0.333

Cost 20 35 0.108 0.069 0.089

Civic 5 10 0.027 0.020 0.022

185 510


SMART with swing weighting

Cost Lives lost Risk Civic imp RESULT

Weights 0.089 0.556 0.333 0.022

Nome AK 0.991 0.564 0.175 0.003 0.468 (4)

Newark NJ 0.026 0.007 0.990 0.685 0.351 (5)

Rock Springs WY

0.685 0.707 0.820 0.041 0.736 (1)

Duquesne PA

0.664 0.721 0.655 0.207 0.675 (2)

Gary IN 0.505 0.386 0.865 0.568 0.552 (3)


Logical Decision

• Hierarchy of criteria• Single-attribute Utility Functions

– Worst imaginable utility = 0

– Best imaginable utility = 1

– Assess 0.5 level of either value or utility

• Tradeoffs– Pairwise comparisons

– Select preferred extreme

– Improve other until equalHuman Centered Processes - 2008

SUF for Lives Lost

PROBLEM: sensitive to limits set – may warp values more than intended


Weight TradeoffsPROBLEM: weights reflect both choices, scale – again hard to control

• Cost < Lives Lost[0,1000]>[500,0] [0,1000]=[5,1000]

• Cost < Risk[0,1]>[500,0] [0,0.35]=[500,0]

• Cost > Civic Improvement[0,100000]>[500,0] [0,100000]=[350,0]

• Weights (including scale):– Risk 5.029– Cost 1.577– Lives Lost 29.337– Civic Improvement 64.058

Tradeoff: Cost vs. Civic Improvement


Result


Contributions by Criteria


Rock Springs vs. Newark


Subjective Ratings

Cost Lives Lost Risk Civic improvement

Nome AK Moderate Low Very high Low

Newark NJ Very high Very high Very low Very high

Rock Springs WY

High Very low Low High

Duquesne PA High Very low Medium Medium

Gary IN Higher High Low Very high


Subjective SMART

Cost Lives Lost Risk Civic Imp RESULT

weights 0.089 0.556 0.333 0.022

Nome AK 0.8 0.7 0 0.2 0.465 (3)

Newark NJ 0 0 1 1 0.355 (5)

Rock Springs WY

0.4 0.9 0.8 0.8 0.820 (1)

Duquesne PA

0.4 0.9 0.5 0.6 0.716 (2)

Gary IN 0.2 0.2 0.8 1 0.417 (4)


Output Comparisons

MAUT SMART SMART subjective

Nome AK 0.629 (5) 0.468 (4) 0.465 (3)

Newark NJ 0.794 (3) 0.351 (5) 0.355 (5)

Rock Springs WY 0.723 (4) 0.736 (1) 0.820 (1)

Duquesne PA 0.839 (1) 0.675 (2) 0.716 (2)

Gary IN 0.830 (2) 0.552 (3) 0.417 (4)


Comparison with SMART

• Simpler allows decision maker to see exactly what ratings are

• MAUT– Distrusts human – masks tradeoffs in effort to make “objective”

– “Objective” here means have no idea

– Theoretically, preferences will be identical

– Does allow for nonlinear interaction, but severe impact

• My contention:– DIRECT IS BETTER THAN MACHINE


Image Theory

INTEREST GROUP Criteria reflecting concerns

Government Cost, lives lost, risk, civic improvement

Nuclear Industry Permanent storage of nuclear wasteNuclear power demand

Local Citizens EquityCultural artifactsEmployment

General Population Nuclear power generation safetyTransportation riskLow-cost electricity


Image Theory process

• Frame decision– Desired states

– Actions needed to attain desired states

– MORE CRITERIA

• Helpful to MCDA in structuring– Context

• Elicit participation of as many views as possible

– Identify alternatives• Design an ideal rather than settle for existing

• MORE ALTERNATIVES


Verbal Decision Analysis

• Controlled pairwise comparisons of tradeoffs• Focus on critical criteria

– Don’t use falsely precise measures• Fuzzify – categorical ratings

• Screen alternatives– Preemptive

• Focus on critical tradeoffs


VDA Process

• Eliminate very high lives lost– Newark eliminated

• Eliminated risk high or worse– Nome eliminated

• Rock Springs now dominates Duquesne

• FOCUS ON– Rock Springs

– Gary

• TRADEOFFS– Rock Springs – a little lower cost, improved lives lost

– Gary – civic improvement slightly better


Inferences

• Objective can’t capture all the complexity of real decisions– OBJECTIVES ARE ALWAYS LEFT OUT

– Conventional wisdom – at most 7 matter

– BUT THERE IS NO PARETO OPTIMAL unless all considered

• When Groups are involved, THERE IS NO ONE BEST DECISION– Ward Edwards – never saw a group pick an option that was first

choice of one subgroup

– NEGOTIATION


Conclusion

• Measures of alternative future performance, preference for that performance both subjective

• Objective measures not always better

• Focus should be on:– Learning (changing preference)– Design of better alternatives (Image Theory)

Documents

Subjectivity in Decision Analysis David L. Olson James & H.K. Stuart Professor in MIS University of Nebraska Lincoln Human Centered Processes - 2008