Repeated Games - London School of Economicsdarp.lse.ac.uk/presentations/MP2Book/OUP/RepeatedGames.pdf · Repeated Games Repeated games The alternative approach • take a series of

Frank Cowell: Repeated Games

REPEATED GAMESMICROECONOMICSPrinciples and AnalysisFrank Cowell

April 2018 1

Almost essential Game Theory: Dynamic

Prerequisites


Overview

Basic structure

Equilibrium issues

Applications

Repeated Games

Embedding the game in context

April 2018 2


IntroductionAnother examination of the role of timeDynamic analysis can be difficult

• more than a few stages• can lead to complicated analysis of equilibrium

We need an alternative approach• one that preserves basic insights of dynamic games• for example, subgame-perfect equilibrium

Build on the idea of dynamic games• introduce a jump • move from the case of comparatively few stages• to the case of arbitrarily many

April 2018 3


Repeated games The alternative approach

• take a series of the same game• embed it within a time-line structure

Basic idea is simple• connect multiple instances of an atemporal game • model a repeated encounter between the players in the same situation of

economic conflict

Raises important questions• how does this structure differ from an atemporal model?• how does the repetition of a game differ from a single play?• how does it differ from a collection of unrelated games of identical

structure with identical players?

April 2018 4


HistoryWhy is the time-line different from a collection of unrelated games? The key is history

• consider history at any point on the timeline • contains information about actual play• information accumulated up to that point

History can affect the nature of the game• at any stage all players can know all the accumulated information• strategies can be conditioned on this information

History can play a role in the equilibrium• some interesting outcomes aren’t equilibria in a single encounter• these may be equilibrium outcomes in the repeated game• the game’s history is used to support such outcomes

April 2018 5


Repeated games: Structure The stage game

• take an instant in time• specify a simultaneous-move game• payoffs completely specified by actions within the game

Repeat the stage game indefinitely• there’s an instance of the stage game at time 0,1,2,…,t,…• the possible payoffs are also repeated for each t• payoffs at t depends on actions in stage game at t

A modified strategic environment• all previous actions assumed as common knowledge• so agents’ strategies can be conditioned on this information

Modifies equilibrium behaviour and outcome?

April 2018 6


Equilibrium Simplified structure has potential advantages

• whether significant depends on nature of stage game• concern nature of equilibrium

Possibilities for equilibrium • new strategy combinations supportable as equilibria?• long-term cooperative outcomes • absent from a myopic analysis of a simple game

Refinements of subgame perfection simplify the analysis:• can rule out empty threats • and incredible promises• disregard irrelevant “might-have-beens”

April 2018 7


Overview

Basic structure

Equilibrium issues

Applications

Repeated Games

Developing the basic concepts

April 2018 8


Equilibrium: an approach Focus on key question in repeated games:

• how can rational players use the information from history?• need to address this to characterise equilibrium

Illustrate a method in an argument by example• outline for the Prisoner's Dilemma game• same players face same outcomes from their actions that they may

choose in periods 1, 2, …, t, …

Prisoner's Dilemma particularly instructive given: • its importance in microeconomics • pessimistic outcome of an isolated round of the game

April 2018 9


[RIG

HT]

1,13,0

0,32,2

[LEFT]Alf

Bill[left] [right]

Prisoner’s dilemma: Reminder Payoffs in stage gameIf Alf plays [RIGHT] Bill’s best response is [right]If Bill plays [right] Alf’s best response is [RIGHT]Nash EquilibriumOutcome that Pareto dominates NE

The highlighted NE is inefficient

Could the Pareto-efficient outcome be an equilibrium in the repeated game?

Look at the structure

April 2018 10

* detail on slide can only be seen if you run the slideshow


Repeated Prisoner's dilemma

Bill

Alf

[LEFT] [RIGHT]

[left] [right] [left] [right]

(2,2) (0,3) (3,0) (1,1)

Bill

Alf

[LEFT] [RIGHT]


(2,2) (0,3) (3,0) (1,1)

2

1Stage game between (t=1)Stage game (t=2) follows hereor here

Bill

Alf

[LEFT] [RIGHT]


(2,2) (0,3) (3,0) (1,1)

2

or here

Bill

Alf

[LEFT] [RIGHT]


(2,2) (0,3) (3,0) (1,1)

2

or here

Repeat this structure indefinitely…?

April 2018 11

Bill

Alf

[LEFT] [RIGHT]


(2,2) (0,3) (3,0) (1,1)

2

* detail on slide can only be seen if you run the slideshow


Repeated Prisoner's dilemma

… … …

Bill

Alf

[LEFT] [RIGHT]


(2,2) (0,3) (3,0) (1,1)

… … …

Bill

Alf

[LEFT] [RIGHT]


(2,2) (0,3) (3,0) (1,1)

t

1The stage game

repeated though time

Let's look at the detail

April 2018 12


Repeated PD: payoffs To represent possibilities in long run:

• first consider payoffs available in the stage game• then those available through mixtures

In the one-shot game payoffs simply represented• it was enough to denote them as 0,…,3• purely ordinal• arbitrary monotonic changes of the payoffs have no effect

Now we need a generalised notation• cardinal values of utility matter• we need to sum utilities, compare utility differences

Evaluation of a payoff stream:• suppose payoff to agent h in period t is υh(t)• value of (υh(1), υh(2),…, υh(t)…) is given by

∞[1−δ] ∑ δt−1υh(t)

t=1

• where δ is a discount factor 0 < δ < 1

April 2018 13


PD: stage gameA generalised notation for the stage game

• consider actions and payoffs• in each of four fundamental cases

Both socially irresponsible: • they play [RIGHT], [right] • get ( υa, υb) where υa > 0, υb > 0

Both socially responsible: • they play [LEFT],[left] • get (υ*a, υ*b) where υ*a > υa, υ*b > υb

Only Alf socially responsible: • they play [LEFT], [right] • get ( 0,υb) where υb > υ*b

Only Bill socially responsible: • they play [RIGHT], [left] • get (υa, 0) where υa > υ*a

A diagrammatic view

April 2018 14


Repeated Prisoner’s dilemma payoffs

𝕌𝕌*

υa

υb

0

( υa, υb ) •

( υ*a, υ*b ) •

υb_

υa_

Space of utility payoffsPayoffs for Prisoner's Dilemma

•

•

Nash-Equilibrium payoffs

Payoffs available through mixingFeasible, superior points"Efficient" outcomes

Payoffs Pareto-superior to NE

April 2018 15


Choosing a strategy: setting Long-run advantage in the Pareto-efficient outcome

• payoffs (υ*a, υ*b) in each period • clearly better than ( υa, υb) in each period

Suppose the agents recognise the advantage• what actions would guarantee them this?• clearly they need to play [LEFT], [left] every period

The problem is lack of trust: • they cannot trust each other • nor indeed themselves: • Alf tempted to be antisocial and get payoffυa by playing [RIGHT]• Bill has a similar temptation

April 2018 16


Choosing a strategy: formulationWill a dominated outcome still be inevitable? Suppose each player adopts a strategy that

1. rewards the other party's responsible behaviour by responding with the action [left]

2. punishes antisocial behaviour with the action [right], thus generating the minimax payoffs (υa, υb)

Known as a trigger strategyWhy the strategy is powerful

• punishment applies to every period after the one where the antisocial action occurred

• if punishment invoked offender is “minimaxed for ever”

Look at it in detail

April 2018 17


[RIGHT]Anything else

Bill’s action in 0,…,t Alf’s action at t+1

Repeated PD: trigger strategies

Take situation at t

First type of historyResponse of other player to continue this history

Second type of historyPunishment response

[LEFT][left][left],…,[left]

[right]Anything else

Alf’s action in 0,…,t Bill’s action at t+1

[left][LEFT][LEFT],…,[LEFT]Will it work?

sTa

sTb

Trigger strategies [sTa, sT

b]

April 2018 18


Will the trigger strategy “work”?Utility gain from “misbehaving” at t: υa − υ*a

What is value at t of punishment from t + 1 onwards?• Difference in utility per period: υ*a − υa

• Discounted value of this in period t + 1: V := [υ*a − υa]/[1 −δ ]• Value of this in period t: δV = δ[υ*a − υa]/[1 −δ ]

So agent chooses not to misbehave if • υa − υ*a ≤ δ[υ*a − υa ]/[1 −δ ]

But this is only going to work for specific parameters• value of δ• relative to υa, υa and υ*a

What values of discount factor will allow an equilibrium?

April 2018 19


Discounting and equilibrium For an equilibrium condition must be satisfied for both a and b Consider the situation of a Rearranging the condition from the previous slide:

• δ[υ*a − υa ] ≥ [1 −δ] [υa − υ*a ]• δ[υa − υa ] ≥ [υa − υ*a ]

Simplifying this the condition must be • δ ≥ δa

• where δa := [υa − υ*a ] / [υa − υa ] A similar result must also apply to agent b Therefore we must have the condition:

• δ ≥ δ• where δ := max {δa , δb}

April 2018 20


Repeated PD: SPNE Assuming δ ≥ δ, take the strategies [sT

a, sTb] prescribed by the Table

If there were antisocial behaviour at t consider subgame that would start at t + 1• Alf could not increase his payoff by switching from [RIGHT] to [LEFT], given that

Bill is playing [left] • a similar remark applies to Bill• so strategies imply a NE for this subgame• likewise for any subgame starting after t + 1

But if [LEFT],[left] has been played in every period up till t:• Alf would not wish to switch to [RIGHT]• a similar remark applies to Bill• again we have a NE

So, if δ is large enough, [sTa, sT

b] is a Subgame-Perfect Equilibrium• yields the payoffs (υ*a, υ*b) in every period

April 2018 21


Folk Theorem The outcome of the repeated PD is instructive

• illustrates an important result• the Folk Theorem

Strictly speaking a class of results• finite/infinite games• different types of equilibrium concepts

A standard version of the Theorem:• for a two-person infinitely repeated game:• suppose discount factor is sufficiently close to 1• any combination of actions observed in any finite number of stages • this is the outcome of a subgame-perfect equilibrium

April 2018 22


Assessment The Folk Theorem central to repeated games

• perhaps better described as Folk Theorems • a class of results

Clearly has considerable attraction Put its significance in context

• makes relatively modest claims • gives a possibility result

Only seen one example of the Folk Theorem• let’s apply it• to well known oligopoly examples

April 2018 23


Overview

Basic structure

Equilibrium issues

Applications

Repeated Games

Some well-known examples

April 2018 24


Cournot competition: repeated Start by reinterpreting PD as Cournot duopoly

• two identical firms• firms can each choose one of two levels of output – [high] or [low]• can firms sustain a low-output (i.e. high-profit) equilibrium?

Possible actions and outcomes in the stage game:• [HIGH], [high]: both firms get Cournot-Nash payoff ΠC > 0• [LOW], [low]: both firms get joint-profit maximising payoff ΠJ > ΠC• [HIGH], [low]: payoffs are (Π, 0) where Π > ΠJ

Folk theorem: get SPNE with payoffs (ΠJ, ΠJ) if δ is large enough• Critical value for the discount factor δ is

Π − ΠJδ = ──────

Π − ΠC

But we should say more• Let’s review the standard Cournot diagram

April 2018 25


q1

q2

χ2(·)

χ1(·)

0

(qC, qC)1 2

(qJ, qJ)1 2

Cournot stage gameFirm 2’s Iso-profit curves

Firm 2’s reaction functionCournot-Nash equilibrium

Firm 1’s Iso-profit curves

Firm 1’s reaction function

Outputs with higher profits for both firmsJoint profit-maximising solutionOutput that forces other firm’s profit to 0

q1

q2

April 2018 26


Repeated Cournot game: Punishment Standard Cournot model is richer than simple PD:

• action space for PD stage game just has the two output levels • continuum of output levels introduces further possibilities

Minimax profit level for firm 1 in a Cournot duopoly• is zero, not the NE outcome ΠC

• arises where firm 2 sets output to q2 such that 1 makes no profit

Imagine a deviation by firm 1 at time t• raises q1 above joint profit-max level

Would minimax be used as punishment from t + 1 to ∞?• clearly (0,q2) is not on firm 2's reaction function• so cannot be best response by firm 2 to an action by firm 1• so it cannot belong to the NE of the subgame• everlasting minimax punishment is not credible in this case

April 2018 27


Repeated Cournot game: Payoffs

Π1

Π2

0

(ΠC,ΠC)

Π•

•Π

Space of profits for the two firmsCournot-Nash outcomeJoint-profit maximisation

(ΠJ,ΠJ)

Minimax outcomesPayoffs available in repeated game

Now review Bertrand competition

April 2018 28


p2

c

c

pM

pM

p1

Bertrand stage game

Firm 1’s reaction functionFirm 2’s reaction function

Marginal cost pricingMonopoly pricing

Nash equilibrium

April 2018 29


Bertrand competition: repeatedNE of the stage game:

• set price equal to marginal cost c• results in zero profits

NE outcome is the minimax outcome• minimax outcome is implementable as a Nash equilibrium• in all the subgames following a defection from cooperation

In repeated Bertrand competition• firms set pM if acting “cooperatively”• split profits between them• if one firm deviates from this• others then set price to c

Repeated Bertrand: result• can enforce joint profit maximisation through trigger strategy• provided discount factor is large enough

April 2018 30


Repeated Bertrand game: Payoffs

Π1

Π2

0ΠM

•

•ΠM

Space of profits for the two firmsBertrand-Nash outcomeFirm 1 as a monopolyFirm 2 as a monopolyPayoffs available in repeated game

April 2018 31


Repeated games: summaryNew concepts:

• Stage game• History• The Folk Theorem• Trigger strategy

What next?• Games under uncertainty

April 2018 32

Documents

Repeated Games - London School of Economicsdarp.lse.ac.uk/presentations/MP2Book/OUP/RepeatedGames.pdf · Repeated Games Repeated games The alternative approach • take a series of