Game Theory -- Lecture 5 - EURECOMloiseau/GameTheory/slides/Lecture5.pdf · Backward Induction • Fundamental concept in game theory • Idea: players that move early on in the game

GameTheory--

Lecture5

PatrickLoiseauEURECOMFall2016

1

Lecture3-4recap• DefinedmixedstrategyNashequilibrium• ProvedexistenceofmixedstrategyNashequilibriuminfinitegames

• DiscussedcomputationandinterpretationofmixedstrategiesNashequilibrium

• Definedanotherconceptofequilibriumfromevolutionarygametheory

àToday:introduceothersolutionconceptsforsimultaneousmovesgames

àIntroducesolutionsforsequentialmovesgames

2

Outline

• Othersolutionconceptsforsimultaneousmoves– Stabilityofequilibrium• Trembling-handperfectequilibrium

– Correlatedequilibrium–Minimax theoremandzero-sumgames– ε-Nashequilibrium

• Thelenderandborrowergame:introductionandconceptsfromsequentialmoves

3

Outline




4

TheLocationModel• Assumewehave2N playersinthisgame(e.g.,N=70)

– Playershavetwotypes:tallandshort– ThereareN tallplayersandN shortplayers

• Playersarepeoplewhoneedtodecideinwhichtowntolive• Therearetwotowns:EasttownandWesttown

– EachtowncanhostnomorethanN players

• Assume:– Ifthenumberofpeoplechoosingaparticulartownislargerthanthe

towncapacity,thesurpluswillberedistributedrandomly

• Game:– Players:2N people– Strategies:EastorWesttown– Payoffs 5

TheLocationModel:payoffs

0

1

1/2

7035#ofyourtypeinyourtown

Utilityforplayeri • Theideais:– Ifyouareasmall

minority inyourtownyougetapayoffofzero

– Ifyouareinlargemajority inyourtownyougetapayoffof½

– Ifyouarewellintegrated yougetapayoffof1

• Peoplewouldliketoliveinmixedtowns,butiftheycannot,thentheyprefertoliveinthemajoritytown

6

Initialstate

• Assumetheinitialpictureisthisone

• Whatwillplayersdo?

7

TallplayerShortplayer

WestTown

EastTown

Firstiteration• Fortallplayers• There’saminorityof

easttown“giants”tobeginwith

à switchtoWesttown

• Forshortplayers• There’saminorityof

westtown“dwarfs”tobeginwith

àswitchtoEasttown

8


WestTown

EastTown

Seconditeration

• Sametrend

• Stillafewplayerswhodidnotunderstand–Whatistheirpayoff?

9


WestTown

EastTown

Lastiteration

• Peoplegotsegregated

• Buttheywouldhavepreferredintegratedtowns!–Why?Whathappened?– Peoplethatstartedinaminority(eventhoughnota“bad”minority)hadincentivestodeviate

10


WestTown

EastTown

TheLocationModel:Nashequilibria

• TwosegregatedNE:– Short,E;Tall,W– Short,W;Tall,E

• IsthereanyotherNE?

11

Stabilityofequilibria• Theintegratedequilibriumisnotstable

– Ifwemoveawayfromthe50%ratio,evenalittlebit,playershaveanincentivetodeviateevenmore

– Weendupinoneofthesegregatedequilibrium• Thesegregatedequilibria arestable

– Introduceasmallperturbation:playerscomebacktosegregationquickly

• NotionofstabilityinPhysics:ifyouintroduceasmallperturbation,youcomebacktotheinitialstate

• Tippingpoint:– IntroducedbyGrodzins (WhiteflightsinAmerica)– ExtendedbyShelling(Nobelprizein2005)

12

Trembling-handperfectequilibrium

• Fully-mixedstrategy:positiveprobabilityoneachaction

• Informally:aplayer’sactionsi mustbeBRnotonlytoopponentsequilibriumstrategiess-i butalsotosmallperturbationsofthoses(k)-i. 13

Definition: Trembling-handperfectequilibriumA (mixed)strategyprofilesisatrembling-handperfectequilibriumifthereexistsasequences(0),s(1),…offullymixedstrategyprofilesthatconvergestowardssandsuchthatforallkandallplayeri,si isabestresponsetos(k)-i.

TheLocationModel

• Thesegregatedequilibria aretrembling-handperfect

• Theintegratedequilibriumisnottrembling-handperfect

14

Outline




15

Example:battleofthesexes

• NE:(O,O),(S,S)and((1/3,2/3),(2/3,1/3))– Themixedequilibriumhaspayoff2/3each

• Supposetheplayerscanobservetheoutcomeofafairtosscoinandconditiontheirstrategiesonthisoutcome– Newstrategiespossible:Oifhead,Siftails– Payoff1.5each

• Thefaircoinactsasacorrelatingdevice

2,1 0,00,0 1,2

Opera

Soccer

Opera

Player1

Player2Soccer

16

Correlatedequilibrium:generalcase

• Inthepreviousexample:bothplayersobservetheexactsamesignal(outcomeofthecointossrandomvariable)

• Generalcase:eachplayerreceivesasignalwhichcanbecorrelatedtotherandomvariable(cointoss)andtotheotherplayerssignal

• Model:– nrandomvariables(oneperplayer)– AjointdistributionoverthenRVs– NaturechoosesaccordingtothejointdistributionandrevealstoeachplayeronlyhisRV

à AgentcanconditionhisactiontohisRV(hissignal)

17

Correlatedequilibrium:definition

18

Definition: CorrelatedequilibriumAcorrelatedequilibriumofthegame(N,(Ai),(ui))isatuple(v,π,σ)where• v=(v1,…,vn)isatupleofrandomvariableswith

domains(D1,…,Dn)• πisajointdistributionoverv• σ=(σ1,…,σn)isavectorofmappingsσi:DiàAisuchthatforalli andanymappingσi’:DiàAi,

π (d)u(σ1(d1),,σ i (di ),,σ n (dn )) ≥d∈D1××Dn

∑ π (d)u(σ1(d1),, %σ i (di ),,σ n (dn ))d∈D1××Dn

∑

• Thesetofcorrelatedequilibria containsthesetofNashequilibria

• Proof:constructitwithDi=Ai,independentsignals(π(d)=σ*1(d1)x…xσ*n(dn))andidentitymappingsσi

Correlatedvs Nashequilibrium

19

Theorem:ForeveryNashequilibriumσ*,thereexistsacorrelatedequilibrium(v,π,σ)suchthatforeachplayeri,thedistributioninducedonAiisσi*.

Correlatedvs Nashequilibrium(2)

20

• Notallcorrelatedequilibria correspondtoaNashequilibrium

• Example,thecorrelatedequilibriuminthebattle-of-sexgame

à CorrelatedequilibriumisastrictlyweakernotionthanNE

Outline




21

Maxmin strategy

• Maximize“worst-casepayoff”

• Example– Attacker:Notattack– Defender:Defend

• ThisisnotaNashequilibrium!22

-2,1 2,-20,-1 0,0

Attack

Notatt

Defend

attacker

defenderNotdef

Definition:Maxmin strategyThemaxmin strategyforplayeri is argmax

simins− iui (si, s−i )

Maxmin strategy:intuition

• Playeri commitstostrategysi (possiblymixed)• Player–i observesi andchooses-i tominimizei’spayoff

• Playeri guaranteespayoffatleastequaltothemaxmin value

23

maxsimins− iui (si, s−i )

Twoplayerszero-sumgames

• Definition:a2-playerszero-sumgameisagamewhereu1(s)=-u2(s)forallstrategyprofiles– Sumofpayoffsconstantequalto0

• Example:Matchingpennies• Defineu(s)=u1(s)– Player1:maximizer– Player2:minimizer

24

heads tails

heads

tails

1,-1 -1,1

1,-1-1,1

Player1

Player2

Minimax theorem

• Thisquantityiscalledthevalue ofthegame– correspondstothepayoffofplayer1atNE

• Maxmin strategiesó NEstrategies• Canbecomputedinpolynomialtime(throughlinearprogramming) 25

Theorem:Minimax theorem(VonNeumann1928)Foranytwo-playerzero-sumgamewithfiniteactionspace: max

s1mins2u(s1, s2 ) =mins2

maxs1u(s1, s2 )

Outline




26

ε-Nashequilibrium

• ItisanapproximateNashequilibrium– Agentsindifferenttosmallgains(couldnotgainmorethanε byunilateraldeviation)

• ANashequilibriumisanε-Nashequilibriumforallε!

27

Definition: ε-NashequilibriumForε>0,astrategyprofile(s1*,s2*,…,sN*)isanε-Nashequilibriumif,foreachplayeri,

ui(si*,s-i*)≥ui(si,s-i*)- ε forallsi ≠si*

Outline




28

“CashinaHat”game(1)

• Twoplayers,1and2• Player1strategies:put$0,$1or$3inahat

• Then,thehatispassedtoplayer2

• Player2strategies:either“match”(i.e.,addthesameamountofmoneyinthehat)ortakethecash

29

“CashinaHat”game(2)

Payoffs:

• Player1:

• Player2:

$0à $0$1à ifmatchnetprofit$1,-$1ifnot$3à ifmatchnetprofit$3,-$3ifnot

Match$1à Netprofit$1.5Match$3à Netprofit$2Takethecashà $inthehat

30

Lender&Borrowergame

• The“cashinahat”gameisatoyversionofthemoregeneral“lenderandborrower”game:– Lenders:Banks,VCFirms,…– Borrowers:entrepreneurswithprojectideas

• Thelenderhastodecidehowmuchmoneytoinvest intheproject

• Afterthemoneyhasbeeninvested,theborrowercould– Goforwardwiththeprojectandworkhard– Shirk,andruntoMexicowiththemoney

31

Simultaneousvs.SequentialMoves

• Whatisdifferentaboutthisgamewrt gamesstudieduntilnow?

• Itisasequentialmovegame– Playerchoosesfirst,thenplayer2

• Timingisnotthekey– ThekeyisthatP2observesP1’schoicebeforechoosing

– AndP1knowsthatthisisgoingtobethecase

32

Extensiveformgames

• Ausefulrepresentationofsuchgamesisgametrees alsoknownastheextensiveform– Eachinternalnodeofthetreewillrepresenttheabilityofaplayertomakechoicesatacertainstage,andtheyarecalleddecisionnodes

– Leafsofthetreearecalledendnodes andrepresentpayoffstobothplayers

• Normalformgamesàmatrices• Extensiveformgamesà trees

33

“Cashinahat”representation

1

2

2

2

(0,0)

(1,1.5)

(-1,1)

(3,2)

(-3,3)

$0

$1

$3

$1

- $1

$3

- $3

Howtoanalyzesuchgame?34

BackwardInduction• Fundamentalconceptingametheory

• Idea:playersthatmoveearlyoninthegameshouldputthemselvesintheshoesofotherplayersplayinglater

à anticipation

• Lookattheendofthetreeandworkbacktowardstheroot– Startwiththelastplayerandchosethestrategiesyielding

higherpayoff– Thissimplifiesthetree– Continuewiththebefore-lastplayeranddothesamething– Repeatuntilyougettotheroot

35

BackwardInductioninpractice(1)

1

2

2

2

(0,0)

(1,1.5)

(-1,1)

(3,2)

(-3,3)

$0

$1

$3

$1

- $1

$3

- $3

36


1

2

2

2

(0,0)

(1,1.5)

(-3,3)

$0

$1

$3

37


1

2

2

2

(0,0)

(1,1.5)

(-1,1)

(3,2)

(-3,3)

$0

$1

$3

$1

- $1

$3

- $3

Outcome:Player1choosestoinvest$1,Player2matches38

Theproblemwiththe“lendersandborrowers”game

• Itisnotadisaster:– Thelenderdoubledhermoney– Theborrowerwasabletogoaheadwithasmallscaleproject

andmakesomemoney

• But,wewouldhavelikedtoendupinanotherbranch:– Largerprojectfundedwith$3andanoutcomebetterforboth

thelenderandtheborrower

• Verysimilartoprisoner’sdilemna

• Whatpreventsusfromgettingtothislattergoodoutcome?

39

MoralHazard• Oneplayer(theborrower)hasincentivestodothingsthatarenot

intheinterestsoftheotherplayer(thelender)– Bygivingatoobigloan,theincentivesfortheborrowerwillbesuch

thattheywillnotbealignedwiththeincentivesonthelender– Noticethatmoralhazardhasalsodisadvantagesfortheborrower

• Example:Insurancecompaniesoffers“full-risk”policies– Peoplesubscribingforthispoliciesmayhavenoincentivestotake

care!– Inpractice,insurancecompaniesforcemetobearsomedeductible

costs(“franchise”)

• Onepartyhasincentivetotakeariskbecausethecostisfeltbyanotherparty

• HowcanwesolvetheMoralHazardproblem?40

Solution(1):Introducelaws

• Todaywehavesuchlaws:bankruptcylaws

• But,therearelimitstothedegreetowhichborrowerscanbepunished– Theborrowercansay:Ican’trepay,I’mbankrupt– Andhe/she’smoreorlessallowedtohaveafreshstart

41

Solution(2):Limits/restrictionsonmoney

• Asktheborrowersaconcreteplan(businessplan)onhowhe/shewillspendthemoney

• Thisboilsdowntochangingtheorderofplay!

• Alsofacessomeissues:– Lackofflexibility,whichisthemotivationtobeanentrepreneurinthefirstplace!

– Problemoftiming:itissometimeshardtopredictup-frontalltheexpensesofaproject

42

Solution(3):Breaktheloanup

• Lettheloancomeinsmallinstallments• Ifaborrowerdoeswellonthefirstinstallment,thelenderwillgiveabiggerinstallmentnexttime

• Itissimilartotakingthisone-shotgameandturnitintoarepeatedgame

43

Solution(4):Changecontracttoavoidshirk-- Incentives

• Theborrowercouldre-designthepayoffsofthegameincasetheprojectissuccessful

• Profitdoesn’tmatchinvestmentbuttheoutcomeisbetter– Sometimesasmallershareofalargerpiecanbe

biggerthanalargershareofasmallerpie

1

2

2

2

(0,0)

(1,1.5)

(-1,1)

(1.9,3.1)

(-3,3)

$0

$1

$3

$1

- $1

$3

- $3

1

2

2

2

(0,0)

(1,1.5)

(-1,1)

(3,2)

(-3,3)

$0

$1

$3

$1

- $1

$3

- $3

44

Absolutepayoffvs ROI

• Previousexample:largerabsolutepayoffinthenewgameontheright,butsmallerreturnoninvestment(ROI)

• Whichmetric(absolutepayofforROI)shouldaninvestmentbanklookat?

45

Solution(5):Beyondincentives,collaterals

• Theborrowercouldre-designthepayoffsofthegameincasetheprojectissuccessful– Example:subtracthousefromrunawaypayoffs

– Lowersthepayoffstoborroweratsometreepoints,yetmakestheborrowerbetteroff!

1

2

2

2

(0,0)

(1,1.5)

(-1,1- HOUSE)

(3,2)

(-3,3- HOUSE)

$0

$1

$3

$1

- $1

$3

- $3

46

Collaterals

• Theydohurtaplayerenoughtochangehis/herbehavior

èLoweringthepayoffsatcertainpointsofthegame,doesnotmeanthataplayerwillbeworseoff!!

• Collateralsarepartofalargerbranchcalledcommitmentstrategies– Next,anexampleofcommitmentstrategies

47

NormanArmyvs.SaxonArmyGame

• Collateralsarepartofalargerbranchcalledcommitmentstrategies

• Backin1066,WilliamtheConquerorleadaninvasionfromNormandyontheSussexbeaches

• We’retalkingaboutmilitarystrategy• Sobasicallywehavetwoplayers(thearmies)andthestrategiesavailabletotheplayersarewhetherto“fight”or“run”

48


N S

N

N

(0,0)

(1,2)

(2,1)

(1,2)

invade

fight

run

fight

fight

run

run

Let’sanalyzethegamewithBackwardInduction

49


N S

N

N

(0,0)

(1,2)

(2,1)

(1,2)

invade

fight

run

fight

fight

run

run

50


N S

N

N

(1,2)

(2,1)

invade

fight

run

51


N S

N

N

(0,0)

(1,2)

(2,1)

(1,2)

invade

fight

run

fight

fight

run

run

BackwardInductiontellsus:• Saxonswillfight• Normanswillrunaway

WhatdidWilliamtheConquerordo?

52


N

S

N

N

(0,0)

(1,2)

(2,1)

(1,2)

fight

run

fight

fight

run

run

S

Notburnboats

Burnboats

fight

run

N

N

fight

fight

(0,0)

(2,1)

53


N

S

N

N

(1,2)

(2,1)

fight

run fight

run

S

Notburnboats

Burnboats

fight

run

N

N

fight

fight

(0,0)

(2,1)

54


N

S(1,2)

S

Notburnboats

Burnboats

(2,1)

55


N

S

N

N

(0,0)

(1,2)

(2,1)

(1,2)

fight

run

fight

fight

run

run

S

Notburnboats

Burnboats

fight

run

N

N

fight

fight

(0,0)

(2,1)

56

Commitment

• Sometimes,gettingridofchoicescanmakemebetteroff!

• Commitment:– Feweroptionschangethebehaviorofothers

• Theotherplayersmustknow aboutyourcommitments– Example:Dr.Strangelovemovie

57

Documents

Game Theory -- Lecture 5 - EURECOMloiseau/GameTheory/slides/Lecture5.pdf · Backward Induction • Fundamental concept in game theory • Idea: players that move early on in the game