1

Economics & EvolutionNumber 2

2

GENERAL:

1. R. DAWKINS: The Selfish Gene, OUP 1976 2. J. MAYNARD SMITH: Evolution and the Theory of Games, CUP 1982 3. T. SCHELLING: Micromotive and Macrobehavior, W.W. Norton 1978

LEARNING PROCESSES: BEST RESPONSE DYNAMICS (COURNOT), FICTITIOUS PLAY, SOCIAL LEARNING

4. K. BINMORE: Fun and Games, Heath 1992 5. D. FUDENBERG, D. K. LEVINE: The Theory of Learning in Games, MIT Press

1998 6. D. MONDERER, D. SAMET, AND A. SELA: “Belief Affirming in Learning

Processes”, JET 1997

REPLICATOR DYNAMICS, EVOLUTIONARILY STABLE STRATEGY (E.S.S.) 7. J. Weibull: Evolutionary Game Theory, MIT 1995

LOCAL INTERACTION 8. ESHEL I., L. SAMUELSON AND A. SHAKED: “ Altruists Egoists and Hooligans

in a Local Interaction Model,” American Economic review, 1998.

Reading List

3

FURTHER READING: 1. J. HOFBAUER, K. SIGMUND: The Theory of Evolution and Dynamical Systems, CUP 1988 2. K. SIGMUND: Games of Life, Explorations in Ecology, Evolution and Behavior, Penguin 1993 3. M. Ridley: The Red Queen: Sex and the Evolution of Human Nature, Perennial, 2003. 4. R. Baker: Sperm Wars, Pan 2000 5. R. Wright: The Moral Animal, Abacus 2004 6. Kandori, Mailath, Rob: Learning, Mutation, and Long Run Equilibria in Games, Econ., 1993 7. T. BERGSTROM: “Storage for Good Times and Bad: Of Rats and Men”, Santa Barbara, mimeo,

1997 (www.econ.ucsb.edu/~tedb/Evolution/store.pdf) 8. T. BORGERS, R. SARIN: “Naive Reinforcement Learning with Endogeneous Aspirations”, JET

1998 COOPERATION: 9. R. AXELROD: The Evolution of Cooperation, Basic Books 1985 10. L. SAMUELSON, K. BINMORE: “Evolutionary Stability in Repeated Games Played by Finite

Automata”, JET 1992 11. ESHEL I., E. SANSONE AND A.SHAKED: “ The Emergence of Kinship Behavior in Structured

Populations of Unrelated Individuals,” International Journal of Game Theory, 1999. 12. ESHEL I., D. HERREINER, L. SAMUELSON, E. SANSONE AND A. SHAKED: “Cooperation, Mimesis

and Local Interaction” Sociological Methods and Research, 2000. 13. ESHEL I. AND A. SHAKED: “Partnership” Journal of Theoretical Biology, 2002.

4

A B

A 0 , 2 3 , 0B 2 , 1 1 , 3

0

1

1

An ExampleAn Example

(B , A )(B , B )

(A , B )

(A , A )

time is spent in each quadranttime is spent in each quadrant

0 0

1t , p ,

2

1 1

1t , ,q

2,2 2

1t , p

2 =

ap t 1 -t =

bq tt

00

=ap t 1 -t

00a = t 1 - p

0

0 =bq tt

0tb =2

0 0=

t 1 - pp t 1 -

t 0=

tq t

2t

5

A B

A 0 , 2 3 , 0B 2 , 1 1 , 3

0

1

1

An ExampleAn Example

(B , A )(B , B )

(A , B )

(A , A )

time is spent in each quadranttime is spent in each quadrant

0 0

1t , p ,

2

1 1

1t , ,q

2,2 2

1t , p

2 0 0=

t 1 - pp t 1 -

t 0=

tq t

2t

0

11

0= =

t 1 - p1p t 1 -2 t

1 0 0t = 2t 1 - p

01 0 0- 1 - 2pt t = t

6

A B

A 0 , 2 3 , 0B 2 , 1 1 , 3

0

1

1

An ExampleAn Example

(B , A )(B , B )

(A , B )

(A , A )

0 0

1t , p ,

2

1 1

1t , ,q

2,2 2

1t , p

2

1 0 0 0-t t = 2 1 - p - 1 t

time spent in the time spent in the firstfirst quadrantquadrant

0 0=

t 1 - pp t 1 -

t 0=

tq t

2t

1 1=

t 1 - pp t 1 -

t 1 1t 1 - q

q t = 1 -t

analogously:

7

A B

A 0 , 2 3 , 0B 2 , 1 1 , 3

0

1

1

An ExampleAn Example

(B , A )(B , B )

(A , B )

(A , A )

0 0

1t , p ,

2

1 1

1t , ,q

2,2 2

1t , p

2

time spent in the time spent in the secondsecond quadrantquadrant

1 1=

t 1 - pp t 1 -

t 1 1t 1 - q

q t = 1 -t

1

1=

1t 1 -2 t

p t 1 - = 1 -t 2t

1 1t 1 - qq t = 1 -

t

And at t2 : 2

1 1t 1 - q1 = 1-2 t

2 1 12t 2t 1 - q=

8

A B

A 0 , 2 3 , 0B 2 , 1 1 , 3

0

1

1

An ExampleAn Example

(B , A )(B , B )

(A , B )

(A , A )

0 0

1t , p ,

2

1 1

1t , ,q

2,2 2

1t , p

2

time spent in the time spent in the secondsecond quadrantquadrant

2

1 1t 1 - q1 = 1-2 t

2 1 1t 2t 1 - q=

12 1 1- tt t 1 - 2q= 0 0 12t 1 - p 1 - 2q= (slide 2)

0

11 0

=t 1q =

2t 4 1 - p(slide 4) 2 1 0 0-t t t 1 - 2p=

2 1 1 0- -t t t t=Spending the same time in each quadrant

9

Fictitious Play: Failure to ConvergeFictitious Play: Failure to ConvergeAn example by L. ShapleyAn example by L. Shapley

L C R

T 0 , 0 0 , 1 1 , 0

M 1 , 0 0 , 0 0 , 1

B 0 , 1 1 , 0 0 , 0

The only Nash equilibrium is (⅓, ⅓, ⅓)

10

Fictitious Play: Failure to ConvergeFictitious Play: Failure to ConvergeAn example by L. ShapleyAn example by L. Shapley

L C R

T 0 , 0 0 , 1 1 , 0

M 1 , 0 0 , 0 0 , 1

B 0 , 1 1 , 0 0 , 0

Tp1=1

Mp2=1

Bp3=1

LC R

Lq1=1

Cq2=1

Rq3=1

TM B

History of player 1(in player’s 2 mind)

History of player 2(in player’s 1 mind)

Best Response of 2 Best Response of 1

The process does not converge, (spends longer periods in any part)

11

Fictitious Play: Failure to ConvergeFictitious Play: Failure to ConvergeAn example by L. ShapleyAn example by L. Shapley

L C R

T 0 , 0 0 , 1 1 , 0

M 1 , 0 0 , 0 0 , 1

B 0 , 1 1 , 0 0 , 0

Tp1=1

Mp2=1

Bp3=1

LC R

Lq1=1

Cq2=1

Rq3=1

TM B

History of player 1(in player’s 2 mind)

History of player 2(in player’s 1 mind)

Best Response of 2 Best Response of 1

They (almost) always play either (0,1) or (1,0)

12

Fictitious Play: Failure to ConvergeFictitious Play: Failure to ConvergeAn example by L. ShapleyAn example by L. Shapley

L C R

T 0 , 0 0 , 1 1 , 0

M 1 , 0 0 , 0 0 , 1

B 0 , 1 1 , 0 0 , 0

Tp1=1

Mp2=1

Bp3=1

LC R

Lq1=1

Cq2=1

Rq3=1

TM B

History of player 1(in player’s 2 mind)

History of player 2(in player’s 1 mind)

Best Response of 2 Best Response of 1

Exercise: What happens when they start at the following points ???

13

A proof of the non convergence of Fictitious Play in the Shapley game.

Monderer, Samet, Sela: ‘Belief Affirming in Learning Processes’JET, vol 73, April 1997

player belief at time about player

the other player's action at time

it

i it ti

-it

-i it -1 t -1i

t

i's t j

t

γ

σ = BR γ

s

s + t - 1γ =

t

γ

ˆ = i i i it t t tiU γ π σ ,γ

i -i i ii t -1 t -1 i t -1 t -1i i i i

t t ti i t -1

π σ ,s + t - 1 π σ ,γπ σ ,γ π σ ,γ

t

14

i -i i ii t -1 t -1 i t -1 t -1i i i i

t t ti i t -1

π σ ,s + t - 1 π σ ,γπ σ ,γ π σ ,γ

t

i -i i ii t -2 t -2 i t -2 t -2i i i i

i t -1 t -1 i t -t - 12

π σ ,s + t - 2 π σ ,γπ σ ,γ π σ ,γ

t - 1

i -i i -i i ii t -1 t -1 i t -2 t -2 i t -2 t -2i i

t ti

π σ ,s + π σ ,s + t - 2 π σ ,γπ σ ,γ

t

etc. etc.

i -i i ii -i i -ii t -1 t -1 i t -2 t -2 i 1 1i i

t tii 1 1+π σ ,s + π σ , πs + ...+ π σ ,s

π σ ,γ,γ

t

σ

ex ante ex post

ˆ i i it t tU γ U ˆ -i -i -i

t t tU γ U ˆ i i

1 1U γ

15

ˆ ˆi i -i -i i -it t t t t tU γ +U γ U +U

If the process converges it must be that:

ˆ ˆ i -i i i -i -it t t t t t

1= = =

3γ γ U γ U γ

= 2

3

Impossible !!!!

So the process cannot converge.

16

A Social Interpretation to Fictitious play dynamics

• Two populations of size N, meet at random and play a

2X2 game.

• At time t, p(t) of the the first population play the second

strategy [q(t) for the second population]

• Players die and are replaced by new ones.• The newly born learn to play the best response against

the other population at the time of their birth.

17

A B

A 0 , 2 3 , 0B 2 , 1 1 , 3

The game: (same as in the previous fictitious game example)

18

A B

A 0 , 2 3 , 0B 2 , 1 1 , 3 q

0

1

(A)

(B)

(A) p 1(B)

As long as ( p(t) , q(t) ) is in the first quadrant,

the best responses are: ( B , A ).

19

In a short interval

boys die and the newly born learn to play strategy

girls die and the newly born learn to play strategy .

2

τ, λτNλτN 1

Boys playing strategy

=2 :

Np t + τ N - λτN p t + λτN

=p t + τ - p t λτ 1 - p t

=•p λ 1 - p t

•p+ λp t = λ λteX -λtp t = 1 -ae

For girls:

=Nq t + τ N - λτN q t

=-•q λq t -λtq t = be

20

-λtp t = 1 -ae -λtq t = be

1- p t abq t

=

,Compared with the fictitious game:

the social process is faster.

a bp t = 1 - q t =t t

21

The Replicator Dynamics and

Evolutionarily Stable StrategiesE.S.S.

• Replicator: gene, phenotype• It replicates, according to how well it did.• It determines the behaviour, the strategy.• The replicators play, replicate and then die so that the

population remains of a fixed size.

22

D H

D 1 ,1 0 , 2

H 2 , 0 -1 , -1

Chicken game, or Dove & Hawk

The population plays: 1-p strategy D, and p strategy H.

The fitness of a player (the no. of his offsprings):

D

H

f p = U + 1- p

f p = U + 2 1- p - p

In a short time interval τ: the total number of D,H players after replication:

D DN 1- p + N 1- p τf p N 1- p 1+ τf= p

HNp 1+ τf pH players:

23

The new proportion of H players:

H

H D

Np 1+ τf p

Np 1+ τf p N 1- p 1+p + τ =

τft

p

H

H D

Np 1+ τf p

N + Nτ pf p 1 - p fp

pt + τ =

HNp 1+ τf p

Np t + τ =

+ Nτf

H D

HNp 1+ τf p

N + Nτ pf p 1 - p fp

pt + τ =

H=

p 1+ τf p

1+ τf

H=p p f p - f

H D= p 1- p f p - f p

24

H=p p f p - f

H D= p 1- p f p - f p

We have not used the particular numbers of the Chicken game.The above equation holds for all 2x2 games,

And it is independent of U.

The Replicator Dynamics

Now apply it to Chicken: D H

D 1 ,1 0 , 2

H 2 , 0 -1 , -1

D

H

f = 1 1 - p + 0p

f = 2 1 - p + -1 p

= 1 - p

= 2 - 3pH Df - f = 2 - 3p

25

H=p p f p - f

H D= p 1- p f p - f p

D H

D 1 ,1 0 , 2

H 2 , 0 -1 , -1

H Df - f = 2 - 3p

=p p 1- p 1 - 2p

t

1/2

D

H

p < 1/2

p

26

C D

C 2 ,2 0 , 3

D 3 , 0 1 , 1

D Cf - f = 1

tC

D

The Prisoners’ Dilemma

C

D

f = 2 1 - p + 0p = 2 - 2p

f = 3 1 - p + 1p = 3 - 2p

D C=p p 1- p f p - f p

p

=p p 1- p

27

L R

L 1 ,1 0 , 0

R 0 , 0 2 , 2

R Lf - f = 3p - 1

tL

R

p < 1/3

Coordination Game

L

R

f = 1 - p

f = 2p

R L=p p 1- p f p - f p

p

=p p 1- p 3p - 1

1/3

p > 1/3

Spontaneous Order, (no one maximizes)Spontaneous Order, (no one maximizes)