December 13 Tue., 2016, 15:10-15:30, Invited Session Game-Theoretic Control And Incentive Design, Starvine 5, Tub05.6 @ 802

Parameterization  of  Equilibrium  Assessment  in  Bayesian  Game  with  Its  Application  to  Belief  Computation

Kiminao KOGISO and Takashi SUZUKIThe University of Electro-Communications

Tokyo, Japan

The 55th Conference on Decision and ControlARIA Resort & Casino, Las Vegas, USA

December 12 to 14, 2016

Supported by JSPS Grant-in-Aid for Challenging Exploratory Research

2014 to 2016

Outline

2

Introduction  Problem  Formulation  Equilibrium  Assessment  Main  Result:  Belief  Computation  Numerical  Example  Conclusion

Introduction

3

Strategic game enables to consider uncertainties in player’s decisions. player: a reasonable decision maker

action: what a player chooses

utility: player’s preference over the actions

type: a label of player’s private valuation (e.g. normal, malicious[2])

belief: a probability distribution over the type (e.g. degree of normal or malicious)

Bayesian game[1]

[1] Harsanyi, 1967. [2] Alpcan and Basar, et al., 2010, 2013. [3] Roy, et al., 2010. [4] Liu, et al., 2006. [5] Akkarajitsakul, et al., 2011. [6] Sedjelmachi, et al., 2014, 2015.

The Bayesian game is recently used in engineering problems to analyze a Bayesian Nash equilibrium (BNE) and to design a game mechanism. network security[2,3], intrusion detection[4,5,6], electricity pricing[7,8], mechanism design[9]

belief learning[10]

[7] Li, et al., 2011, 2014. [8] Yang, et al., 2013. [9] Tao, et al., 2015. [10] Nachbar, 2008.

Introduction

4

BNE and beliefThe BNE plays key roles in analysis and design. equilibrium analysis: for given belief, find BNEs.

mechanism design: for given utility, find a mechanism to truthfully report type.

belief learning: for given BNE, find a corresponding belief.

Objective of this studyPropose a belief computation method based on parameterization of BNE.

formulate a discrete-time nonlinear dynamical system of the BNE and belief[12],

show parameterization of the BNE and belief as a theorem, and

confirm that the corresponding belief can be computed from a given BNE.

[11] Powell, 2011. [12] Kogiso, 2015.

However, there are a few theoretical-guaranteed methods to compute a belief that corresponds to a BNE.[11]

Bayesian game

a player set

an action set

a type set

utility

a mixed strategy

a belief profile

Problem Formulation

5

Two-player two-action Bayesian game w/ two types G(N ,A,⇥, u, µ, S)

N := {1, 2}

A := A1 ⇥A2

⇥ := ⇥1 ⇥⇥2

u := (u1, u2)

µ := (µ1, µ2)

S := (S1, S2)

ai 2 Ai := {a, a} 8i 2 N

✓i 2 ⇥i := {✓, ✓} 8i 2 N

µi 2 ⇧(⇥i) 8i 2 N

Si : ⇥i ! ⇧(Ai) 8i 2 Nsi 2 Si(⇥i) 8i 2 N

⇧(X) : a probability distribution over a finite set X

Ui(✓i, ✓�i) :=

ui(a, a, ✓i, ✓�i) ui(a, a, ✓i, ✓�i)ui(a, a, ✓i, ✓�i) ui(a, a, ✓i, ✓�i)

�: utility matrix8i 2 N , 8✓ 2 ⇥

ui : A⇥⇥ ! < 8i 2 N

i 2 N

Problem Formulation

6

Service in tennis

2, 2 0, 1

1, 21, 1

flat

spin

flat spin

0, 1 1, 2

0, 11, 2

flat

spin

flat spin

side

line 1, 0 1, 1

2, 00, 1

flat

spin

flat spin

1, 3 1, 2

0, 32, 2

flat

spin

flat spin

cent

er li

ne s1(a|✓)

s1(a|✓)

s1(a|✓)

s1(a|✓)

s2(a|✓) s2(a|✓)s2(a|✓)s2(a|✓)center line ✓ side line ✓

✓✓

a

a a

a

a a

aaa

a a

a

µ1(✓)

µ1(✓)

µ2(✓)µ2(✓)

a

a a

a

typebelief

The game can model where each player is unsure of other players’ preferences.

Problem Formulation

7

Bayesian Nash equilibrium[13]

using a best response to opponent strategy:

[13] Y. Shoham and K. Leyton-Brown, Multiagent Systems, Cambridge University Press, 2009.

Problem: Belief computationFor a given (desired or measured) BNE, compute the corresponding belief.

This is an inverse problem of computing BNEs.

Expected utility of player :i

EUi(si, s�i) =X

✓i2⇥i

X

✓�i2⇥�i

µi(✓i)µ�i(✓�i)si(✓i)TUi(✓i, ✓�i)s�i(✓i)

expected utilities of the normal-form games

probabilities of choosing a game

Definition:

EUi(si, s�i) � EUi(s0, s�i) 8s0i 2 Si, s0i 6= si

is a Bayesian Nash equilibrium (BNE).

Given a probability of choosing a game , for any , the strategy satisfyingi 2 N sµ

Computation of BNEs: For a given belief, find BNEs.

Equilibrium Assessment

8

Our policyA pair of is a key variable of the Bayesian game.

a pair of the belief and BNE is named equilibrium assessment (EA).

(µ, s)

equilibrium analysis[10]: find a BNE .

[14] Fudenberg and Tirole, 1991.

assessment

⇥EA

(µ, s)

assessment

⇥EA

(µ+ �µ, s+ �s)

Use a nonlinear map from EA at step to EA at step .k k + 1

A pair of the belief and strategy is called assessment[14], and

Given initial EA, if there exists such that the game satisfies the following condition regarding utility matrices: ,

Equilibrium Assessment

9

Autonomous nonlinear systemTheorem 1[12]

⇥1 �1

⇤Ui(✓i, ✓)

�1

1� �1

�= 0

⇥1 �1

⇤Ui(✓i, ✓)

�2

1� �2

�= 0

8✓i 2 ⇥i8i 2 N

ci(k) :=µi(✓i, k + 1)

µi(✓i, k) : row stochastic matrices, and .Ai 2 <2⇥2 8i 2 N

� = [�1 �2]T 2 <2

then a nonlinear autonomous system in equilibrium assessment:

transfers from EA to EA , where (µ(k), s(k)) (µ(k + 1), s(k + 1))

ci(k) ! 1

A�(1) = I

ci(k) :=µi(✓i, k + 1)

µi(✓i, k)µ(k + 1) = diag(A1, A2)µ(k) s(k + 1) = A�(ci(k))si(k)µ(k)

µ(k + 1)stable linear system: time-varying system:· s(k)

µ(k)

[12] Kogiso, 2015.

µ(k + 1) = diag(A1, A2)µ(k) s(k + 1) = A�(ci(k))si(k),

Given initial EA, if there exists such that the game satisfies the following condition regarding utility matrices: ,

Main Result

10

Relation between BNE and beliefTheorem 2

⇥1 �1

⇤Ui(✓i, ✓)

�1

1� �1

�= 0

⇥1 �1

⇤Ui(✓i, ✓)

�2

1� �2

�= 0

8✓i 2 ⇥i8i 2 N� = [�1 �2]

T 2 <2

then a nonlinear autonomous system in equilibrium assessment:

satisfies the following fractional forms (parameterizations):

where

µ(k + 1) = diag(A1, A2)µ(k) s(k + 1) = A�(ci(k))si(k),

.

,

D(si(✓, k)) = �2si(a|✓, k)� (1� �2)si(a|✓, k) 6= 0

D(si(✓, k)) = (1� �1)si(a|✓, k)� �1si(a|✓, k) 6= 0

8k 2 {0, 1, 2, · · · ,1},constant constant

↵i =µi(✓, k)

D(si(✓, k))and ↵i =

µi(✓, k)

D(si(✓, k))8i

Main Result

11

Belief computation methodCorollary 1Suppose that one EA of is known. If a strategy belongs to an allowable strategy set, then a belief that corresponds to the strategy can be computed as follows:

G sµ

Procedure of belief computation:

1. consider the game ,

2. obtain one EA and the ratios,

3. set a desired or measured BNE, and

4. compute a corresponding belief using the equations above.

G

where the ratios and are given by the EA, and the obtained pair is EA.↵i ↵i

µi(✓) = ↵iD(si(✓)) 8i 2 N ,µi(✓) = ↵iD(si(✓))

Numerical Example

12

U1(✓, ✓) =

2 01 1

�, U1(✓, ✓) =

0 11 0

�,

U1(✓, ✓) =

1 10 2

�, U1(✓, ✓) =

1 12 0

�,

U2(✓, ✓) =

2 11 2

�, U2(✓, ✓) =

1 22 1

�,

U2(✓, ✓) =

0 11 1

�, U2(✓, ✓) =

3 22 3

�,

Belief computationBayesian game with utility matrices:

s2(0) =⇥0.2583 0.7417 0.1905 0.8095

⇤T.

Initial EA:s1(0) =

⇥0.6792 0.3208 0.8737 0.1263

⇤T,

µ(0) =⇥0.6759 0.3241 0.5614 0.4386

⇤T,

The ratios: ↵1 = ↵1 = 1.809 ↵2 = ↵2 = �1.814.

Desired BNE: s1 =⇥0.9231 0.0769 0.6298 0.3702

⇤T,

s2 =⇥0.1435 0.8565 0.3503 0.6947

⇤T.

Computed belief: µ =⇥0.2348 0.7652 0.3532 0.6468

⇤T.

Numerical Example

13

Verification: Trajectories of EA[13]

belief strategy

0

0.2

0.4

0.6

0.8

1.0

the number of iterations

pro

bab

ility

on

5 10 15 20 25 0

0.2

0.4

0.6

0.8

1.0

the number of iterations

pro

bab

ility

on

5 10 15 20 25

Desired BNE: s1 =⇥0.9231 0.0769 0.6298 0.3702

⇤T,

s2 =⇥0.1435 0.8565 0.3503 0.6947

⇤T.

Computed belief: µ =⇥0.2348 0.7652 0.3532 0.6468

⇤T.

0.2348

0.3532

0.6468

0.7652

[13] Suzuki and Kogiso, 2016.

Conclusion

14

Introduction Problem Formulation belief computation for two-player two-action Bayesian game with two types.

Equilibrium Assessment use of the discrete-time nonlinear dynamical system.

Main Result fractional forms of the EA that provide constant ratios, belief computation procedure.

Numerical Example confirmation of the belief computation.

Future work incentive design (how to update utility matrices) to achieve a desired BNE.