Risk Neutral Equilibria of Noncooperative Games

Robert NauDuke UniversityApril 12, 2013

References• “Coherent behavior in noncooperative games”

(with K. McCardle, JET 1990)

• “Coherent decision analysis with inseparable probabilities and utilities” (JRU, 1995)

• “On the geometry of Nash and correlated equilibria” (with S. G. Canovas & P. Hansen, IJGT 2003)

• “Risk neutral equilibria of noncooperative games” (Theory and Decision, forthcoming)

• “Efficient correlated equilibria in games of coordination” (working paper)

• Arbitrage and Rational Choice (manuscript in progress)

Nash vs. Correlated Equilibrium

• What is the most fundamental solution concept for noncooperative games?

– Nash equilibrium (or a refinement thereof)?

– Rationalizability?

– No, correlated equilibrium.

• A correlated equilibrium is a generalization of Nash equilibrium in which randomized strategies are permitted to be correlated.

• More to the point: Nash equilibrium is special case of correlated equilibrium in which randomization, if any, is required to be performed independently.

A canonical example: Battle-of-the-Sexes• Correlation of randomized strategies is most useful in

games where some coordination of strategies is beneficial or equitable

• Canonical example: the Battle-of-Sexes game

• The obvious solution: flip a coin!

Left RightTop 2, 1 0, 0

Bottom 0, 0 1, 2

Nash vs. Correlated Equilibrium

• A Nash equilibrium satisfies a fixed-point condition analogous to that of a Walrasian equilibrium

• A correlated equilibrium satisfies a no-arbitrage condition that is a strategic generalization of de Finetti’s fundamental theorem of probability

Properties of Nash equilibria

• The set of Nash equilibria may be nonconvex and/or disconnected

• May require randomization with probabilities that are irrational numbers, even when game payoffs are rational

• Existence proof is based on a fixed point theorem

• Computation can be hard in games with more than a few players and/or strategies

Properties of correlated equilibria

• The set of correlated equilibria is a convex polytope, like sets of probabilities that arise elsewhere in decision theory (incomplete preferences, incomplete markets….)

• Extreme points of the polytope have rational coordinates if the game payoffs do

• Existence proof only depends on the existence of a stationary distribution of a Markov chain

• Computation is easy in games of any size: solutions can be found by linear programming

• The number of extreme points of the polytope can be huge, but usually only a small number are efficient.

Fundamental theorems of rational choice

• Subjective probability (de Finetti)

• Expected utility (vNM)

• Subjective expected utility (Savage)

• Decision analysis (Nau)

• Asset pricing (Black-Scholes/Merton/Ross….)

• (2nd theorem of) Welfare economics (Arrow)

• Utilitarianism (Harsanyi)

All of these theorems are duality theorems, provable by separating-hyperplane arguments, in which the primal rationality condition is no-arbitrage and the dual condition is the existence of a probability distribution and/or utility function and/or prices and/or weights with respect to which an action or allocation is optimal.

• Correlated equilibrium, rather than Nash, fits on this list.

The duality theorem for games

• Primal definition of common knowledge:

– The players should be willing to put their money where their mouth is, i.e., publicly accept bets that reveal the differences in payoff profiles between their strategies

• Primal definition of strategic rationality:

– The outcome of the game should be jointly coherent, i.e., not allow an arbitrage profit to an observer, when the rules are revealed in this way

• THEOREM (Nau & McCardle 1990): The outcomes of the game that are jointly coherent are the ones that have positive support in some correlated equilibrium

• Consider a generic 2x2 game whose “real” rules are:

• The “revealed” rules (all that can be commonly known under noncooperative conditions) are encoded in the matrix whose rows are the payoff vectors of 4 bets on the game’s outcome that might be offered to an observer:

Left RightTop a, a b, b

Bottom c, c d, d

TL TR BL BR1TB a – c b – d 1BT c – a d – b2LR a – b c – d 2RL b – a d – c

How this argument works

=

G =

in units of money, with risk neutral players

• “1TB” is the payoff vector of a bet that player 1 should be willing to accept if she chooses Top in preference to Bottom

• In that case she gets the payoff profile (a, c) rather than (b, d) as a function of player 2’s choice among (L, R)

• She should choose Top if (a,c) yields a greater expected payoff than (b,d) at the instant of her move

• In this event, a bet with payoff profile (a-c, b-d) evidently has non-negative expected value for her.

• She can reveal information about her payoff function by offering to accept this bet conditional on Top being played

TL TR BL BR1TB a – c b – d 1BT c – a d – b2LR a – b c – d 2RL b – a d – c

The real game vs. the revealed game G

• The contents of G (rather than in ) suffice to determine all the noncooperative equilibria of the game.

• Definition: a correlated equilibrium of the game is a distribution that satisfies G 0, i.e., that assigns non-negative expected value to each of the rows of G

– If used by a mediator to generate recommended strategies, it would be incentive-compatible for all players to comply

– It is determined merely by linear inequalities

• Definition: a Nash equilibrium is a correlated equilibrium in which randomization, if any, is independent

But… why require independence? Correlation may be beneficial and/or it could describe the rational beliefs of an observer who doesn’t know which equilibrium has been selected

The real game vs. the revealed game G

• What’s in that is not in G? Information about externalities: how one player benefits from a change in another’s strategy, holding her own strategy fixed.

• The missing information is what may give rise to dilemmas in which strategically rational behavior is not socially rational

• It also determines which side of the correlated equilibrium polytope is the efficient frontier.

Geometry of Nash & Correlated Equilibria• THEOREM (Nau et al. 2003): The Nash equilibria of a game

are all located on the surface of the correlated equilibrium polytope, i.e., on supporting hyperplanes to it.

• Example: Battle-of-the-sexes

• The set of correlated equilibria is a hexadron, with 5 vertices and 6 faces.

– 2 of the vertices are pure Nash equilibria

– 1 is a mixed Nash equilibrium

– 2 are extremal non-Nash correlated equilibria

The correlated equilibrium polytope of Battle-of-the-Sexes

• The polytope is determined by a system of linear inequalities: incentive constraints for not deviating from the strategy recommended to you by a possibly-correlated randomization device

• The Nash equilibria are points where the polytope touches the “saddle” of independent distribuitions

Left RightTop 2, 1 0, 0

Bottom 0, 0 1, 2

The pure Nash equilibria (as seen from 2 angles)

The mixed Nash equilibrium

In general Nash equilibria do not need to be vertices of the polytope: they can fall in the middle of edges or lie on curves within faces of it

The extremal non-Nash correlated equilibria

The “obvious” coin-flip solution of battle-of-the-sexes is the midpoint of the edge connecting TL and BR, which is neither a Nash equilibrium nor a vertex of the polytope

• THEOREM: In a 2x2 game with a 3-dimensional correlated equilibrium polytope (like BOS), the efficient frontier may only consist of one of the following:

i. One of the two pure Nash equilibra

ii. Both of the pure Nash equilibria, and their convex combinations

iii. Both of the pure Nash equilibria and one of the two extremal non-Nash correlated equilibria, and their convex combinations

• The mixed strategy Nash equilibrium is never efficient, no matter what the externalities are.

• Completely mixed Nash equilibria are generally inefficient in games with multiple equilibria, because they satisfy the incentive constraints with equality and hence they are also equilibria of the game with the opposite payoffs.

• Conjecture: a completely mixed Nash equilibrium can never be efficient in any game whose correlated equilibrium polytope is of maximal dimension (i.e., dimension n-1 in a game with n outcomes), which is a generalized game of coordination.

• Seems to be empirically true based on numerical experiments, but a general proof is (so far) elusive

• Example of a game with a unique Nash equilibrium in completely mixed strategies that is not a vertex of the correlated equilibrium polytope.

• Similar to a “poker” game devised by Shapley and discussed by Nash (1951)

• The polytope is 7-dimensional (i.e., full dimension) with 33 vertices

• The Nash equilibrium lies in the middle of an edge and has irrational coordinates.

• Example: a game with a continuum of inefficient Nash equilibria lying in a face of the polytope:

• The CE polytope is has 7 vertices (full dimension).• By manipulating externalities, any of the vertices can be

placed on the efficient frontier.• The Nash equilibria lie in a face of the polytope along

an open curve connecting two of the vertices.• The face containing the Nash equilibria can be placed

on the inefficient frontier, but not the efficient frontier.

• The solution concept of correlated equilibrium provides a tight link between subjective probability theory (á la de Finetti), game theory (á la Aumann), and finance theory (á la Black-Scholes)

• In all of the fundamental theorems, the subjective parameters of belief are revealed through betting, and the rationality criterion is that of no-arbitrage.

• But there’s a catch: the interpretation of the probabilities as measures of pure belief is complicated by the presence of risk aversion.

• What if utility for money is state-dependent due to risk aversion and prior stakes in events?

A unified theory of individual, strategy, and competitive rationality?

Risk neutral probabilities• In financial markets, the probability distributions that

rationalize asset prices are not measures of pure belief.

• They are “risk neutral probabilities” that characterize a “risk neutral representative agent” but which do NOT reflect the beliefs of risk averse real agents

• Risk neutral probabilities are interpretable as products of true probabilities and state-dependent marginal utilities

• The average real agent is risk averse with higher marginal utilities for money in states where asset prices are low, so her risk neutral distribution is typically left-shifted compared to her true probability distribution.

• The mean of the risk neutral distribution of asset returns must equal the risk free rate of return

Risk neutral equilibria

• How does the presence of risk aversion and large stakes change the solution of the game whose parameters are made commonly known through betting?

– The parameters of equilibria become risk neutral probabilities!

– It also changes the geometry of the set of equilibria

– The polytope of equilibria is distorted in a systematic way, analogous to the way the risk neutral distribution of an asset prices is left-shifted.

– The polytope “blows up” to a larger size.

Example: Matching pennies

• This is a zero-sum game, not a coordination game like Battle-of-the-Sexes or Chicken

• The correlated equilibrium polytope consists of a single point—a mixed Nash equilibrium in which both players use independent 50/50 randomization when payoffs are in units of money and both players are risk neutral

Left RightTop 1, 1 1, 1

Bottom 1, 1 1, 1

…and the unique Nash/correlated equilibrium the point in the middle of the saddle of independent distributions (the uniform distribution)

TL TR BL BR1TB 1 -1 0 01BT 0 0 -1 12LR 1 0 -1 02RL 0 -1 0 1

The rules-of-the-game matrix G looks like this:

Risk averse players

• Suppose players in such a game are significantly risk averse (i.e., have small risk tolerances compared to game payoffs)?

• The game is then non-zero-sum in units of utility

• The very idea of a game that is zero-sum when utility for money is not linear does not really make sense.

• In general, strategic incentives measured in units of utility cannot be made common knowledge through conversations in terms of acceptable bets.

• Consider a generic 2x2 game whose monetary payoffs are

• Let u1(.) and u2(.) denote the players’ utility functions• The rows of the (unobservable) “true” rules-of-the-game

matrix would be bets with payoffs in units of utility:

… because this is the necessary condition for them to have zero expected utility w.r.t. the players’ true probabilities

Left RightTop a, a b, b

Bottom c, c d, d

TL TR BL BR1TB u1(a) – u1(c) u1(b) – u1(d) 1BT u1(c) – u1(a) u1(b) – u1(d)2LR u1(a) – u1(b) u1(c) – u1(d) 2RL u1(b) –

u1(a) u1(d) – u1(c)

G* =

• Let v1(.) and v2(.) denote the players’ marginal utility functions (i.e., vi is the derivative of ui)

• Then the payoff vectors of acceptable monetary bets that reveal their strategic incentives (denoted “G*”) looks like this:

• The payoffs of the bets are in units of money, and they are distorted compared to what their payoffs would have been under linear utility for money

• The monetary payoff of a bet (e.g., (u1(a) – u1(c))/v1(a)) is the true-game utility difference divided by the marginal utility that prevails in that outcome of the game (e.g., v1(a))

TL TR BL BR1TB (u1(a) – u1(c))/v1(a) (u1(c) – u1(a))/v1(a) 1BT (u1(b) – u1(d))/v1(c) (u1(b) – u1(d))/v1(c)

2LR (u2(a) – u1(b))/v2(a) (u2(c) – u1(d))/v2(c)

2RL (u2(b) – u1(a))/v2(b) (u2(d) – u1(c))/v2(c)

Risk neutral equilibria defined• The appropriate generalization of a correlated equilibrium

under conditions where players are risk averse is a risk neutral equilibrium

• Definition: a risk neutral equilibrium of the game is a distribution that satisfies G* 0, i.e., that assigns non-negative expected value to each of the rows of G*

• Implementation would require a mediator to use a randomization device whose inputs include events about which players might have different true probabilities but would must have identical risk neutral probabilities, as if they have already reached equilibrium in a contingent claims market with assets pegged to those events.

Left Right Top 1, 1 1, 1

Bottom 1, 1 1, 1

Left Right Top a, b b, a

Bottom b, a a, b

where a = 1 ½ 0.293 and b = 1 2 0.414. This is a negative sum game in units of utility.

The local marginal utilities are u’(a) = 0.245 and u’(b) = 0.490

Let u(x) = 1 exp(x) for both players, with risk aversion = LN(2). Then the rules of the game in units of utility are:

Example: matching pennies with monetary payoffs of 1:

TL TR BL BR 1TB 1 1/2 0 0 1BT 0 0 1/2 1 2LR 1/2 0 1 0 2RL 0 1 0 1/2

TL TR BL BR EV>0? Vertex 1 2/15 4/15 1/15 8/15 1BT Vertex 2 8/15 1/15 4/15 2/15 1TB Vertex 3 4/15 8/15 2/15 1/15 2RL Vertex 4 1/15 2/15 8/15 4/15 2LR

The real rules of the game matrix, G*, whose rows are the acceptable monetary bets (scaled to maximum of +1):

The risk-neutral-equilibrium polytope, determined by the inequalities G* 0, has 4 vertices, each of which assigns strictly positive expected value to exactly one of the bets:

Main result• THEOREM: The set of correlated equilibria of a nontrivial

game with monetary payoffs played by strictly risk neutral players is a strict subset of the set of risk neutral equilibria of the same game played by risk averse players.

• The set of risk neutral equilibria of Matching Pennies is a tetrahedron with points on both sides of the saddle of independent distributions.

• The unique Nash/correlated equilibrium is in its interior.

• From an observer’s perspective, the players’ reciprocal beliefs are less precisely determined than in the risk neutral case.

Conclusions• The game-theoretic assumption of common knowledge of

payoff functions can be made precise in terms of the language of betting.

• The assumption of common knowledge of rationality can be made precise in terms of no-arbitrage.

• When players are risk neutral the solution concept to which this leads is correlated equilibrium, not Nash equilibrium.

• A Nash equilibrium is a special case of a correlated equilibrium, not the other way around.

• The “solution” of a game is generally a convex set of correlated equilibria.

• The constraints that determine equilibria ignore information about externalities, which is why strategic equilibria are not necessarily efficient.

Conclusions• When the players in a game are risk averse, the assumption

that they have common knowledge of each others’ utilities is very problematic.

• There is no “market” in which assets are priced in units of utility, and prices that agents are willing to pay for monetary assets do not reveal their utility functions.

• This paper proposes a method that theoretically could address this problem, and it leads to the conclusion that in games with risk averse players:– The parameters of strategic equilibria are risk neutral

probabilities, as in financial markets with risk averse investors

– They are less-precisely determined (even for zero-sum games) than if players were risk averse.