Games We Play Payoffs & Chaos MonkeysAllison Miller @selenakyle

Overview

• What is Game Theory

• Scenarios & Games

• Playing the in real world

• Secrets of irrational economic agents

• Applying economic tools to risk control design

Let’s Play Our Game(s)1. Pick a whole number between 1 and a 100. The

winner of this game is the person who gets closest to two-thirds times the average number.

2. You and I are playing Rock Paper Scissors. It’s your turn: what do you pick, and what’s your p(win)?

3. You are a football (soccer) player kicking a penalty kick. You are stronger kicking left but the goalie knows it. Which direction do you kick, R or L?

4. Write down the last 2 digits of your phone #

5. I have an extra iPhone how much would you pay for one? (in Euros)

Game theory• Branch of applied mathematics

• Studies decisions made by players interacting (or competing)- Scenarios have rules and pay-offs

- Costs & benefits dependent on decisions of other players

• Used as a framework in economics, comp sci, biology, & philosophy- Also business, negotiation, and military strategy

Economics applied to security

• Utility theory

• Externalities

• Information Asymmetries

• Signaling

• Marginal cost

Discussing Games

• Mechanics of a payoff matrix

Player 2

A B

Player 1

A A1, A2 A1, B2

B B1, A2 B1, B2

Discussing Games

• Mechanics of decision trees

UP

DOWN

CIRCLE

RED

BLUE

MARIO

LUIGI

KIRBY

GIZMO

10, 3

2, 10

2, 5

-3, 3A

B

B

A

A

A

Discussing strategies & outcomes• Cooperation / Defection

• Dominant strategies

• Equilibrium Players in a game have selected a strategy

Neither side can change it’s strategy independently & improve position

Optimal solution in games with limited outcomes

Typical game theory "games" • Tragedy of the Commons

- Share and share alike (cumulative effect of cheating)

• Volunteer’s Dilemma- For the greater good

• Chicken / Brinkmanship- Push it to the edge

• Prisoner’s Dilemma

Discussing Games

• Prisoner’s Dilemma

Player 2

Keep quiet Confess

Player 1

Keep quiet -1, -1Mutual cooperation

-10, 0Individual defection

Confess 0, -10Individual defection

-3, -3Mutual punishment

Setting up risk problems as games

• Identify players in the game

• Clarify the “rules”

• Show me your moves

• Describe payoffs

• Single move or repeated game

How games are won

• Clarify dominant strategies

• Find equilibrium

• Pursue equilibrium or change the payoffs

Wait, wait, there’s more• But everyone’s out to get me• But we’re all in this together• But it never ends…• But wait, what game are we playing?

Tragedy of the Defender• MaxMin – When other players looking to

minimize your payoff (i.e. cause you harm)- Maximise the “worst case” payoff

- Maxmin value = minimum payoff guaranteed by MaxMin strategy- ARG MAX si MINs-i ui(s1, s2)

• MinMax – Attacker- Trying to minimize –i’s best case payoff

- ZSG (zero sum game)

Team dynamics• Coalitional Game Theory- Basic modeling unit = team

- Coordination may be possible

- What is the distribution of the surplus (payoff)- Shapley value: payoffs proportional to each

agent’s marginal contribution- “The Core”: more stable solution, agents don’t

have incentive to deviate

The Never Ending Story• Repeated games• Learning reduces uncertainty• Tit for tat• Payoffs modeled as a limit, net back to

present value using discount factor (like cash flows)

Moves like Bayesians• Uncertainty

- Army 1 is weak or strong

- Army 2 is always weak

• Payoffs - Island = M- Cost = s if strong,

w if weak, no cost if other doesn’t attack

1=Weak Army 2

p Right Left

1Right -w, -w M, 0

Left 0, M 0, 0

1=strong Army 2

1-p Right Left

1Right M-s, -w M, 0

Left 0, M 0, 0

Real World• While many competitive or risky

scenarios can be interpreted as games, two things the math hinges on:

• Rationality of Actors

• Payoff values

Let’s Get Real

• 2/3 game• Hands in the air• Iterated removal of dominated

strategies

1 30 44 66 100

Stratagems LiveAdhering scenarios

Deviating scenarios

Rock Paper Scissors• What did you

pick?• What did you

estimate was your p(win)?

• What’s the dominant strategy?

You

Rock Paper Scissors

Me

Rock

0, 0 -1, 1 1, -1

Paper 1, -1 0, 0 -1, 1

Scissors -1, 1 1, -1 0, 0

Chaos monkeys to the rescue

You

Rock Paper Scissors

Me

Rock

0, 0 -1, 1 1, -1

Paper 1, -1 0, 0 -1, 1

Scissors -1, 1 1, -1 0, 0

Le Foot• It pays to be unpredictable• Rational agents employ mixed

strategies to improve their payoffs in games where “pure” strategies don’t maximize value

Le Foot

• You are a football (soccer) player kicking a penalty kick.

• Simple set-up: Kick goes in when G goes to wrong side

Le Foot

• You are a football (soccer) player kicking a penalty kick.

• Simple set-up: Kick goes in when G goes to wrong side They will randomize @ 1/2

Le Foot

• Now let’s add in skill

• You’re better on the left; the goalie knows this

• Which way do you kick? They need to be random, if either side demonstrates a

propensity, their opponent will take advantage

Le Foot

• Now let’s add in skill

• You’re better on the left; the goalie knows this

• Which way do you kick? They need to be random, if either side demonstrates a

propensity, their opponent will take advantage

Le Foot

• Now let’s add in skill

• You’re better on the left; the goalie knows this

• Which way do you kick? Kicker becomes indifferent based on their payoffs for going R vs

L, which depends on goalie behavior

Le Foot

p(0) + (1-p)(1)

p(.75) + (1-p)(0)

(1-p) = .75(p)

p = 4/7q(1) + (1-q)(.25) q(0) + (1-q)(1)

2q - .25q = .75

q = 3/7

Le Foot• How does

theory map to reality?

• Actual win-rates in payoff matrix

• Kicker indifferent @! p(.58)+(1-p)(.95) = p(.93)+(1-p)(.70) ! p=.42

• Goalie indifferent @! q(.42)+(1-q)(.07) = q(.05)+(1-q)(.30) ! q=.38

Le Foot

Goalie Left Goalie Right Kicker Left Kicker Right

Reality .42 .58 .38 .62

Nash .42 .58 .40 .60

• Ignacio Palacios-Heurta (2003) “Professionals Play Minimax”. Review of Economic Studies, Volume 70, pp 395-415.- 1417 penalty kicks from FIFA games

• Jackson, Leyton-Brown & Shoham. Game Theory. (Stanford University and University of British Columbia: Coursera), http://www.coursera.org, Accessed 2013.

• Polak, Ben. Game Theory (Yale University: Open Yale Courses), http://oyc.yale.edu, Accessed 2012. License: Creative Commons BY-NC-SA

Self-interested Agents

• Attacker mindsets: APT/Mercenary, Hacktivist, Vuln Researcher

• Defender mindsets: Risk-based defense, Practice-based defense, Compliance!

• But are we rational?

Example of rational irrationality

• Ultimatum Game- Player A given $1000

Player A needs to split the $ with Player BPlayer A gets to choose the split

- Player B receives offerIf B accepts, both get $If B rejects, both get 0

Still playing? • Economic/mathematical models

depend on rational participants

• Free will doesn’t imply rationality

• Economics studies what should happen, behavioral economics studies what does happen

Take it or leave itOutcomes- Player A’s usually offer ~50%

- Player B’s often reject if offered <30%

- This behavior occurs across cultures, levels of wealth

Emotions matter- Heightened brain activity in

Bilateral antierior insula (disgust) w/low offersDorsolateral prefrontal cortext (cognitive decision making) w/high offers

- Fairness, Fear, Punishing the mean

One last game• How much are you willing to pay?

• Purely rational economic agents make choices based on utility, preferences

• Not arbitrary external factors

• However…

Decision Illusions• Choice architecture/defaults

• Framing

• Anchoring

• Loss aversion

Pesky Payoffs• Opportunity cost

• Relativity

• Mental accounting

• Pain of paying

For the win• Making choices that maximize payoffs

- Select strategies in a chaotic world- Utility not so straightforward

Wait, what?• Game theory

- Awesome framework for understanding competitive (risky) decisions

• Behavioral economics- People are prone to bias (irrational)

• Q: How do I use these tools to control risk?- Fuse frameworks with actual decisions

- Risk Management is dead- Long live risk management

Managing Risk• Game Theory is a framework for studying decisions

- What should happen - Payoffs depend on other players’ choices, moves are risky

(uncertain)

- Players play based on their risk appetite & preferences

• Defenders design control systems that make decisions- What does happen - Risks manifest in observable behavior

- Controls are moves/counter-moves depending on context and expectations of an actor’s identity or intent- Effective controls change payoffs (add friction or remove benefit)

• Risk management = decision management

Applying Decisions

• Risk management is decision management

ACTOR ATTEMPTS ACTION

SuSUBMIT

WHAT IS THE REQUEST

HOW TO HONOR THE REQUEST

SHOULD WE HONOR?

RESULTACTIONOCCURS

Decisions, Decisions

Authorize Block

Good false positive

Bad false negative

RESPONSE

POPULATION

• Incorrect decisions have a cost • Correct decisions are free (usually)

Good Action Gets Blocked

Bad Action Gets Through

Downstream Impacts

Therefore: Winning strategies depend on understanding behavior • Attackers and defenders • Both self-interested• Both prone to decision bias

• Leverage conceptual models, incorporate actual experiences & data• Dynamics can’t always be mapped cleanly, lookout for cognitive

illusions• Costs/payoffs have both concrete and hidden elements

• Game strategy = decision strategy, risk controls:- Change friction (cost), or

- Change expected value of pay-off

Prediction is very difficult, especially about the future

Niels Bohr

Allison Miller @selenakyle

Some referencesAriely, Dan. Predictably Irrational: The Hidden Forces That Shape Our Decisions. New York, NY: HarperCollins, 2008.

Axelrod, Robert M. The Evolution of Cooperation. New York: Basic, 1984.

Dixit, Avinash and Nalebuff, Barry. The Art of Strategy: A Game Theorist’s Guide to Success in Business and in Life.

Fisher, Len. Rock, Paper, Scissors: Game Theory in Everyday Life. New York: Basic, 2008.

Gibbons, Robert. Game Theory for Applied Economists. Princeton, NJ: Princeton UP, 1992.

Gintis, Herbert. Game Theory Evolving: A Problem-centered Introduction to Modeling Strategic Behavior. Princeton, NJ: Princeton UP, 2000.

Ignacio Palacios-Heurta (2003) “Professionals Play Minimax”. Review of Economic Studies, Volume 70, pp 395-415.

Jackson, Leyton-Brown & Shoham. Game Theory. (Stanford University and University of British Columbia: Coursera), http://www.coursera.org, Accessed 2013.

Kahneman, Daniel. Thinking, Fast and Slow. New York: Farrar, Straus and Giroux, 2011.

Leyton-Brown, Kevin, and Yoav Shoham. Essentials of Game Theory: A Concise, Multidisciplinary Introduction. [San Rafael, Calif.]: Morgan & Claypool, 2008.

Meadows, Donella. Thinking in Systems: A Primer.

Polak, Ben. Game Theory (Yale University: Open Yale Courses), http://oyc.yale.edu, Accessed 2012. License: Creative Commons BY-NC-SA

Thomas, L. C. Games, Theory, and Applications. Chichester: E. Horwood, 1984.

Wikipedia’s sections on Game Theory, Economics, & Probability.