2003-05-05 CS634 1

Beyond Nash Equilibrium

Correlated Equilibrium and Evolutionary Equilibrium

Jie Bao2003-05-05

Iowa State University

2003-05-05 CS634 2

Equilibrium in Games Pure strategy Nash equilibrium Mixed strategy Nash equilibrium Correlated equilibrium Evolutionary equilibrium Bayesian Nash equilibrium …

NE is too strict on what is “ration”…

2003-05-05 CS634 3

Correlated Equilibrium Aumann 1974 A generalization of “rational” solution In a CE

the action played by any player is a best response (in the expected payoff sense) to the conditional distribution over the other players given that action,

and thus no player has a unilateral incentive to deviate from playing their role in the CE.

Example: Traffic Signal- a single bit of shares information allows a fair split of waiting times.

“running a light” can’t bring greater expected payoff

The actions of players are “correlated”

2003-05-05 CS634 4

Example : BoS

Bach or Stravinsky Game (or Battle of Sex) NE: strategy profile -> payoff profile

(Bach, Stravinsky) ->(2 ,1) (Stravinsky, Bach) ->(1 ,2) (1/3 Bach, 1/3 Bach) -> (2/3, 2/3)

Another equilibrium: the player observe the outcome of a public coin toss, which determines which of the two pure strategy Nash equilibria they play.->(3/2,3/2)

Bach Stravinsky

Bach 2,1 0,1

Stravinsky 1,0 1,2

?

2003-05-05 CS634 5

BoS: Mixed Strategy

(b) Agent 2’s expected utility, blue line is it’s best response to given p (agent 1’s strategy)

(d) Look from the top, (1/3,1/3) is the only mixed NE. pure NEs include (0,1) and (1,0)

(a) Agent 1’s expected utility, red line is it’s best response to given q (agent 2’s strategy)

(c) Overlap of a and b

2003-05-05 CS634 6

BoS: CE - Toss Coin Equ. <N,(Ai),(ui)>= <{1,2} ,{Bach , Stravinsky},

payoff matrix U> Probability space (Ω,π)

Ω = {x,y} π(x) = π(y) = ½

information partition of each agent P1 = P2 = {{x},{y}}

For each player {1,2} action function σi : Ω->Ai, σ1 (x) = σ2 (x) = Bach σ1 (y) = σ2 (y) = Stravinsky

Payoff profile: ½(B,B), ½(S,S) -> (3/2, 3/2) Compared with (for agent 1):

½(S,B), ½(S,S) ->1/2

½(B,B), ½(B,S) ->1

½(S,B), ½(B,S) ->0

2003-05-05 CS634 7

CE: formal definition A strategic game <N,(Ai),(ui)> A finite probability space (Ω,π)

Ωis a set of states and π is a probability measure on π For each player i∈N, a partition Pi of Ω(player i’s information

partition) For each player i∈N, a function σi : Ω->Ai, with σi (w)= σi (w’)

whenever w∈Pi, and w’∈Pi for some P∈Pi, (σi is player i’s strategy) such that for every I∈N and every function τi : Ω->Ai for which

τi(w)=τi(w’) whenever w∈Pi, and w’∈Pi for some P∈Pi (i.e. for every strategy of player i) we have

Note that we assume the players share a common belief about the probabilities with which the states occur.

))(),(()())(),(()( iiiiii uu

2003-05-05 CS634 8

CE contains Mixed NE For every mixed strategy Nash equilibriu

mαof a finite game <N,(Ai),(ui)>, there is a correlated equilibrium <(Ω,π), Pi ,σi > in which for each player i∈N, the distribution on Ai induced by σi is αi.

Construct CE from Mixed NE Ω= A=(X j ∈NAj) – strategy profiles π(α)=Πj ∈N αj(aj) – prob. Of the strategy pr

ofiles Pi(bi)={a ∈A: ai= bi}, Pi consist of the |Ai| se

ts Pi(bi) σi(a)=ai

CE is a more general concept than Mixed NE and Pure NE

NE

CE

Mixed NE

2003-05-05 CS634 9

Convex Combination of CE->CE Let G=<N,(Ai),(ui)> be a strategic

game. Any convex combination of correlated equilibrium payoff of G is a correlated equilibrium payoff of G

Interpret: first a public random device determines which of the K correlated equilibria is to played, and then the random variable corresponding to the kth correlated equilibrium is realized.

•CE: <(Ωk,πk), Pik ,σi

k >

•CE payoff profiles: U1 ,.. Uk

•c1,…ck , all ci>=0, and Σ ci=1

•Construct a new CE• Ω = union of all Ωk

• π(w) = ck πk(w) , if w in Ωk

• Pi=union (on k) of Pik

•σi (w) = σi k

(w)

• Payoff profiles Σ ckUk

2003-05-05 CS634 10

Example

Pure NE payoff profile: (7,2) (2,7) Mixed NE payoff profile: (42/3 ,42/3) CE: Ω={x,y,z}, π(x)= π(y)=

π(z)=1/3, P1={{x},{y,z}}, P2={{x,y},{z}}, σ1(x)=B, σ1(y)= σ1(z)=T, σ2(x)= σ2(y)=L, σ2(z)=R,

-> (5,5) The CE is outside the convex hull of

Pure / Mixed NE payoff profiles

1 2 L R

T 6,6 2,7

B 7,2 0,0

1 2 L R

T y z

B x -

2003-05-05 CS634 11

States and outcomes in CE Let G=<N,(Ai),(ui)> be a strategic game. Every proba

bility distribution over outcomes that can be obtained in a CE of G can be obtained in a CE’ in which

the set of state Ω’ is A and for each i∈N, player i’s information partition Pi’ consis

ts of all sets of the form {a∈Ai: ai=bi} for some action bi∈Ai [π’(a) = πk({w ∈ Ω: σ(w)=a}) σi’ (a) = σi

k (a i)]

This theorem allows us to confine attention to equilibria in which the set states is the set of outcomes

2003-05-05 CS634 12

Notes about CE If players hold different beliefs, additional

equilibrium payoff profiles are possible. Nash equilibrium is a special case of CE in

which we demand that πbe a product distribution for some distribution πi, so every player acts independently of all others

It’s possible to compute CE via linear programming in polynomial time, while NE is exponentially complex!

2003-05-05 CS634 13

Evolutionary Equilibrium ESS: Maynard Smith &

Price, 1972 A steady state in which

all organism take this action and not mutant can invade the population.

Example: the sex ratio in bee population is 1(male):3(female)

Selfish

Selfish

Selfish

SelflessSelfless Selfless

Selfless

Selfless

Selfless

Selfish

2003-05-05 CS634 14

ESS: Definition:

Let G=<{1,2},(B,B),(ui)>be a symmetric strategic game, where u1(a,b)=u2(b,a)=u(a,b) for some function u.

An evolutionarily stable strategy(ESS) of G is an action b*∈B for which (b*,b*) is a NE of G and u(b,b)<u(b*,b) for every best

response b∈B to b* with b≠b*.

2003-05-05 CS634 15

Example: Hawk--Dove

Choose to be Hawk or dove? Pure NE: (D,H) & (H,D) Mixed NE: (0.5D/0.5H, 0.5D/05H) If the players have the freedom to

choose to be hawk or dove in a repeated game, and utility will be used to reproduce their offspring, what’ll be the optimal strategy?

D H

D ½, ½ 0,1

H 1,0 (1-c/)2, (1-c/)2

2003-05-05 CS634 16

HD Game – mixed strategy

(a) Agent 1’s expected utility, red line is it’s best response to given q (agent 2’s strategy)

(b) Agent 2’s expected utility, blue line is it’s best response to given p (agent 1’s strategy)

(c) Overlap of a and b

(d) Look from the top, (0.5,0.5) is the only mixed NE. pure NEs include (0,1) and (1,0)

With c = 2, Action = {'Hawk' , 'Dove'} ;

2003-05-05 CS634 17

HD Game – ESS invading

In all those games, ESS (half-dove-half-hawk mixed strategy (0.5,0.5)) starts from a percentage of 0.1 in the population

Population: 200, Game Round = 1000 Reproduce: proportional to total utility of each type Note that Dove is not completely eliminated

ESS vs. Dove ESS vs. Hawk ESS vs. 1/4 Hawk ESS vs. 1/4 Dove

2003-05-05 CS634 18

HD Game – ESS being invaded

In all those games, ESS (half-dove-half-hawk mixed strategy (0.5,0.5)) starts from a percentage of 0.9 in the population

Setting is same to the last slide ESS can successfully defend the invasion of mutants, although it may

not be a completely expelling The experiment shows that ESS can be taken to be the set of mixed

strategy over some finite set of actions

ESS vs. Dove ESS vs. Hawk ESS vs. 1/4 Hawk ESS vs. 1/4 Dove

2003-05-05 CS634 19

Not all NE are ESS A strict equilibrium b* is an ESS

(b*,b*) if a symmetric NE and no strategy other than b* is a best r

esponse to b* A nonstrict equilibrium may not be an

ESS Mixed NE (1/3, 1/3, 1/3) expected payof

f t/3Can be invaded by any pure strategy

Receives expected utility t/3 when it encounters MixedNE

Receives expected t when it encounters another pure strategy

t,t 1,-1 -1,1

-1,1 t,t 1,-1

1,-1 -1,1 t,t

Example:

ESS

Mixed NE

2003-05-05 CS634 20

More about ESS Widely used in sociobiology

See Dawkins<selfish gene>, chapter 6 Wilson, < sociobiology – New

Synthesis>, chapter 5 And in politic science and sociology

See <the evolution of cooperation>, where tit-for-tat is a ESS in Evolutionary Pensioner Dilemma Game