Embed Size (px)
Citation preview
Making Smart Decisions (and
Avoiding Decision Traps) in
Municipal Energy Management
Plans
Basil Stumborg Decision Analysis Expert
Finance Group, BC Hydro
Introduction
• Prioritizing Energy Management Projects is
hard!
Introduction
• At BC Hydro
– Treated as a formal decision problem
• Susceptible to Decision Traps
• Business Case Requirements
– Use Structured Decision Making (SDM) format
• Roadmap for today
– Walk through a fictitious prioritization problem
• Lots of interaction
– Identify potential decision traps (and solutions)
– Show how SDM can help avoid these traps
• Leave you with three simple, practical tools to use
Exercise #1 – The Dollar Auction
• A simple auction with simple rules
– Bids are in increments of $0.05, starting at $0.05
– Highest bidder wins (and has to pay their bid)
– Second highest bidder wins (and has to pay their bid)
Exercise #1 – The Dollar Auction
• What happened?
– A decision trap
– Chasing sunk costs
• Additional elements
– Time pressure
– “Gut feelings”, no time for analysis
• HVAC Upgrade (item #5)
– Elements of desire to justify sunk costs
– A potential decision trap – beware!
Exercise #2 – The Bat & Ball Sale
• A bat and a ball cost $1.10 in total. The bat
costs $1 more than the ball.
• Write down the first answer that comes into your
head…
– How much does the ball cost?
The answer is: ball costs 5 cents…
If you said 10 cents…
So did 50% of Princeton students (and me) (Fredrick, cited in Kahnemann 2002 )
Exercise #2 – The Bat & Ball Sale
• What happened?
– Another decision trap
– Decision short cut (heuristic)
• “Gut feeling” type response overrode analytical
approach
• Additional elements
– Time pressure
– Pattern recognition
Structured Decision Making at
BC Hydro
• Recognizes role of
Decision Analysis in
business cases.
• 5 steps to structure
– Thinking
– Data collection
– Options analysis
– Reporting
Structured Decision Making at
BC Hydro
• Rest of today, focus on:
– Decision objectives
– Measures
• Including uncertainty
– Consequence Table
– Tradeoff analysis
• Lots of time for interaction
Decision Objectives
• Decision objectives are “what matter” when
comparing options
– Linked to “fundamental needs”
– Underpin “positions” or “solutions”
• But are not to be confused with these!
Decision Objectives
Minimize Costs
Maximize Green
Leadership Visibility
Maximize Energy Conservation
Minimize Cost Risk
Maximize Public Safety
Maximize Worker Safety
Decision Objectives
Minimize Costs
Maximize Green
Leadership Visibility
Maximize Energy Conservation
Minimize Cost Risk
Maximize Public Safety
Maximize Worker Safety
Measures
• Key step where SDM can add value to the
whole decision process
• Does my measure tell me – ‘what am I getting
and what am I giving up when choosing A over
B?’
– A trade-off question
• The enemy of good trade-off analysis
– The generic 1 – 5 scale.
– The “hi, medium, low” scale.
Measures
• Some measures will be “natural”
– Costs are measured by dollars (NPV, lifecycle)
• Some measures may be “proxies”
– Creative proxy measures may be useful
• E.g. public safety impact of street lighting may be
measured by expected sq metres of unlit parking lot
– Just ensure proxy measures properly inform trade-off
considerations
• Some measures may need to be constructed
Measures
• Some measures may need to be “constructed”
– E.g. Green Leadership Visibility
3 = national recognition (national award, one or more
national media pieces)
2 = provincial recognition (provincial award, one or more
provincial media pieces)
1 = local recognition (local media coverage)
0 = no change in recognition for conservation efforts
• Constructed scales take some work
– But are useful to inform on specific, tough to measure
impacts for trade-off consideration
Exercise #3 – Words or numbers?
• Measuring uncertainty is difficult
• People often skirt around it by using words
– Very likely
– Probable
– Fairly unlikely
• Beside each description above, write the
probability you think best applies
Exercise #3 – Words or numbers?
• What happened?
• Probability is a difficult
topic
• Linguistic fuzziness can
add to uncertainty!
• Tip – try to use underlying
numbers, strive for clarity
and common understanding
18
Exercise #4 – How confident is too
confident?
• For each of the following ten items, provide a low
and high guess such that you are 90 percent sure
the correct answer falls between the two.
• Your challenge is to be neither too narrow (i.e.,
overconfident) nor too wide (i.e., underconfident).
• If you successfully meet the challenge you should
have 10 percent misses—that is, exactly one miss
19
Exercise #4 – How confident is too
confident?
90% confidence range Low High
1. Martin Luther King’s age at death
2. Length of Nile River (in km)
3. Percentage of African-Americans in the U.S.
4. Number of books in the Old Testament
5. Diameter of the moon (in km)
6. Weight of an empty Boeing 747 (in kg)
7. Current population of California
8. Year in which Wolfgang Amadeus Mozart was born
9. Air distance from London to Tokyo (in km)
10. Deepest known point in the ocean (in m)
Source: Adapted from J.E. Russo and P.J.H. Schoemaker, Decision Traps: Ten Barriers to Brilliant Decision Making and How to Overcome Them (New York: Simon & Schuster, 1989), p. 71
20
Exercise #4 – How confident is too
confident?
1. Martin Luther King’s age at death 39 years
2. Length of Nile River (in km) 6,738 km
3. Percentage of African-Americans in the U.S. 13.4 % (2004);
4. Number of books in the Old Testament 39 books
5. Diameter of the moon (in km) 3,476 km
6. Weight of an empty Boeing 747 (in kg) 176,901 kg
7. Current population of California 36.5 million (2006)
8. Year in which Wolfgang Amadeus Mozart was born 1756
9. Air distance from London to Tokyo (in km) 9,590 km
10. Deepest known point in the ocean (in m) 11,033 m
Source: Adapted from J.E. Russo and P.J.H. Schoemaker, Decision Traps: Ten Barriers to Brilliant Decision Making and How to Overcome Them (New York: Simon & Schuster, 1989), p. 71
21
Exercise #4 – How confident is too
confident?
• What happened?
• Decision Trap – people have a tendency to be
overconfident – Surprised by the future too often
• Tip – use this knowledge when dealing with your
“experts” – Remind them of this bias (use this exercise!)
– Push them to think about ways in which the future might surprise
them
– Their job isn’t to be exactly right. Their job is mostly to protect us
from negative surprises
22
The Consequence Table
• Key milestone in SDM process
• Powerful when developed with stakeholders
• Far more useful than list of pros and cons
• Decision Trap solved – notice how sunk costs fall away from decision
• So, which are the best options?
23
The Consequence Table
• Decision trap – too much information for human brain to use – Unconsciously filter out info using heuristics (short cuts)
• Next step is to simplify – Focus on key options, key decision drivers
– But how?
24
The Consequence Table
• Interactive, coloured consequence table
– Shows performance of options relative to a base comparison
• Red is significantly worse than, green is significantly better than, no colour is
roughly tied.
– One use of this is to simplify info down to key elements
25
Swing Weighting – assessing relative
importance of objectives
• Not sensible to talk about “which objective is most important” in general
• Does make sense to ask that question in context of problem at hand
– Think of “swing” in scores from best to worst
26
Swing Weighting – assessing relative
importance of objectives
• Imagine Alt X scored very poorly on all measures…
• Which measure would you want to “swing” from lowest to highest first?
– Give that a rank of “1”
• Which measure would you want to “swing” next? – Give that a rank of “2” and so on…
1
3
4
2
5
27
Swing Weighting – assessing relative
importance of objectives
• Now for the “swing” you ranked #1 – give that a weight of 100
• For the “swing” you ranked #2 – how important is that swing versus the
one you ranked #1.
– Enter a weight that roughly approximates this relative difference.
• By the same method, compare #3 to #1, #4 to #1 and #5 to #1.
1
3
2
4
5
100
.002
50
40
10
28
Swing Weighting – assessing relative
importance of objectives
• Notice how this changes the question of “relative importance”
• Not a global question of “cost” vs “safety” – Rather, posed in a very specific context
– Bounded by what can be impacted by these options
– Can be framed as tradeoffs (if this is useful)
• Very different conversation flows from this
29
Swing Weighting – assessing relative
importance of objectives
• So for each person, weights can be normalized – Useful for comparing weight values when group consensus is required
30
Swing Weighting – assessing relative
importance of objectives
• So for each person, weights can be cross multiplied into consequence
table
– Gives a weighted score for each individual, for each alternative
31
Swing Weighting – assessing relative
importance of objectives
• These scores can then be applied for each member of the group
– Shows alternatives rankings for each member of the group
– Despite wide divergence of value, group would agree on top two, bottom
three projects
• Useful for selecting a portfolio of projects
32
Concluding Remarks
• Prioritizing investment options is difficult
– Multiple objectives
– Multiple alternatives
– Hard to assess impacts
– Multiple view points / values
• All of the above makes decisions hard – Subject to decision traps
• SDM – builds advances in decision analysis into process – A few basic steps
– Some tips and tricks to follow
– Doing just a little bit is useful!
• In particular ….
33
Concluding Remarks
• 3 tools to take away from today
– Use the PROACT framework
– Assume that you (or your experts) are overconfident
• And plan accordingly
– Use consequence tables
• And never, ever use a list of pros/cons again.
34
Bibliography
• Keeney, Ralph - Smart Choices
• Gregory, Robin et al – Structured Decision Making,
A Practical Guide to Environmental Management
• Khanmen, Daniel – Thinking Fast, Thinking Slow
• Russo and Shoemaker – Decision Traps: Ten
Barriers to Brilliant Decision Making and How to
Avoid Them
• Hubbard, Douglas - How to measure anything
