Introduction to Game Theory application to networks

Joy Ghosh

CSE 716, CSE@UB

25th April, 2003

What is Game Theory ?

Study of problems of conflict and cooperation amongst independent decision makers

Formal way of analyzing interactions among a group of rational agents who behave strategically

Games of Strategy rather than Games of Chance! Ingredients:

Players / decision makers Choices / feasible actions / pure strategies Payoffs / benefits / utilities Preferences to payoffs

Some basic concepts

Group – Any game consisting of more than one player with single player the game becomes a decision problem!

Interaction – Actions of one affects the other else it would become simple sequence of independent decisions

Strategic – Players account for interdependence Rationality – Players consistently opt for best choices Common Knowledge

All players know that all players are rational

Equilibrium – a point of best shared interest for all

Classification of Problems

Static vs. Dynamic In Dynamic problems the sequence of choices are relevant

Cooperative vs. Non-cooperative In non-cooperative games players watch out for their own interests. In

cooperative games, players form coalitions with shared objectives.

Decision Theory under Certainty

Decision problem (A, ≤) Finite set of outcomes A = {a1, a2, …. an} Preference relation ‘≤’ : a ≤ b => ‘b’ is at least as good as ‘a’

Completeness – for all a, b in A, either a≤b, or b≤a Transitivity – if a≤b and b≤c, then a≤c

Utility function u: A R (consist with preference relation) For all a,b in A, u(a) <= u(b) iff a ≤ b

Rational decision maker tries to maximize utility Choose outcome a* in A s.t. for all a in A, a ≤ a*

Decision Theory under Uncertainty

Lottery L = {(a1, p1), (a2, p2), …… (an, pn)} Σiεn pi = 1, 0 ≤ pi ≤ 1

Outcome ai occurs with probability pi

Infinitely many different possible lotteries Large number of lottery comparisons Preference relation unobservable

“Under additional restrictions on preferences over lotteries there exists a utility function over outcomes such that the expected utility of a lottery provides a consistent ranking of all lotteries” - John von Neumann and Oscar Morgenstern

u(L) = Σiεn u(ai).pi

Allais Paradox

Lottery A A1 (sure win of 3000) vs. A2 (80% chance to win 4000)

A1 strictly preferred to A2

Lottery B B1 (90% chance to win 3000) vs. B2 (70% chance to win 4000)

B1 might still be preferred to B2

Lottery C C1 (25% chance to win 3000) vs. C2 (20% chance to win 4000)

Most people start preferring C2 over C1 even though these two lotteries are variations of the 1st pair in Lottery A

Game Theory – multi agent decision problem

A normal (strategic) form game G consists of: A finite set of agents D = {1, 2, ….. N} Strategy sets S1, S2, ... SN = set of feasible actions for agents

Strategy profile S = S1 x S2 x ... x SN

Payoff function ui : S R (i = 1, 2, …. N)

NOTE: The preference an agent has is to the outcome and not to the individual action

Some standard games in normal form

Matching Pennies

Row: gains if pennies matchCol: gains if there is no match

Tough vs. Chicken

A game of head-on collision

Iterated Deletion of Dominated Strategies

Common Knowledge assumptions about other people’s rational behavior

Some more definitions: S-i = S1 x S2 ....x S(i-1) x S(i+1) ....x SN (strategy sets of others) utility function of player i for a given pure strategy:

ui (s S) = ui (si, s-i) Belief i of agent i = probability distribution over S-i

for pure strategies the probability distribution is a point distribution

Player i is rational with beliefs i if: si arg max s-iS-i

ui (s’i, s-i). i (s-i) for all s’i Si

Note: as i gets fixed, player i faces a simple decision problem

Dominated Strategies

Strongly Dominated si Si is strictly dominated if:

s’i Si s.t. ui(s’i, s-i) > ui(si, s-i) for all s-i S-i

Weakly Dominated if the inequality is weak () for all s-i S-i, and strong (>) for at

least one

Rational players do not play strongly dominated strategy

Iterated Dominance (deletion)

•With common knowledge about rationality of players U,L is the outcome

• M is strictly dominated by L. Rational column player ignores M

• If row player knows column player is rational, he will ignore D

• If column player knows the above, then he will choose L

Iterated Dominance – Formal Definition

The game is solvable by pure strategy iterated strict dominance only if S contains a single strategy profile

Does the order of elimination matter?

In games that are solvable by iterated dominance, the speed and order or elimination doesn’t matter.

This is however not true for weakly dominated strategies.

Deletion Sequence #1: T, L

- (2,1) is the playoff

Deletion Sequence #2: B, R

- (1,1) is the playoff

Nash Equilibrium for pure strategy

No incentive for a player to deviate from his best response to his/her belief about other player’s strategy U,L was the NE in the example of strongly dominated strategies

Definitions: A strategy profile s* is a pure strategy Nash equilibrium of G iff

ui (si*, s-i*) ≥ ui (si, s-i*) for all players i and for all si Si

A pure strategy NE is strict if the inequality is strict

There can be multiple Nash equilibria for a particular G Two people trying to meet at one out of 2 places (NY Game!)

Do pure strategies always work?

Most games are not solvable by dominance Coordination game, zero-sum game

Penny matching Game Whatever pure strategy one player chooses, the other

can win by choosing a better strategy

Players have to consider mixed strategies

Mixed strategies - definitions

Mixed strategy i for player i is a probability distribution over his strategy space Si

i : Si R+ s.t. siSi i(si) = 1

i is the set of probability distributions on S i

= 1 x 2 x … x N

Player i’s expected payoff with mixed strategies

ui (i, -i) = si, s-i ui(si, s-i) i(si) -i(s-i)

Mixed strategies – more definitions

Mixed strategy NE of G is a * such that: ui (i*, -i*) ≥ ui (i, -i*) for all i and for all i i

In a finite game, support of a mixed strategy i: supp (i) = { si Si | i (si) > 0 }

Proposition if i* is a mixed strategy NE and si’, si’’ supp (i*),

then ui (si’, -i*) = ui (si’’, -i*)

Proof of previous proposition

A mixed strategy example game There is no pure strategy NE Row plays U with probability Column plays L with probability Players need to be indifferent to their

choice of strategies: u1 (U, 2*) = u1 (D, 2*)

= 2 (1 - ) u2 (L, 1*) = u2 (R, 1*)

+ 2 (1 - ) = 4 + (1 - ) = 1/4 ; = 2/3

Unique mixed NE 1* = 1/4 U + 3/4 D

2* = 2/3 L + 1/3 R

Two People Zero Sum Games – Pure Strategy

One player’s winnings is another player’s loss!

Each player does the following: For each of his/her strategies, compute the maximum

of losses that he could incur. Choose the strategy with the minimum max loss

Example 2 people 0 sum Game

Row is player 1; Column is player 2 If aij > 0, player 1 wins, else player 2 Player 1:

i* = arg maxi (minj (aij))

V(A) = minj a i*j is the gain-floor for the game A In this case, V(A) = -2, with i* {2, 3}

Player 2: j* = arg minj (maxi (aij))

(A) = maxi a ij* is the loss-ceiling for the game A

In this case, (A) = 0, with j* = 3

Two People Zero Sum Game – Mixed Strategy

If (A) = V(A) then A has a point of equilibrium Else we need to develop mixed strategy Consider the following game:

For player 1, we have V(A) = 0, with i* = 2 For player 2, we have (A) = 1, with j* = 2 No saddle point or equilibrium Let players 1, 2 play strategy i with probability xi, yi

Best Choice Analysis

Optimization Problem

In a nutshell, the players are solving the following pair of dual linear programming problems

Player 1

Player 2

Application to networks

Formulation for n users competing for fixed resources Generic non-cooperative game Each user has access control /parameter n

Each user receives certain amount n() of network resources

(1, 2, …. n)

n [0, nmax] for some n

max > 0

n () is a non-decreasing function of n

n (.) is continuous in n=1N [0, n

max] and is differentiable with respect to n

If n = 0, n () =0 for all n () maybe interpreted as the QOS received by the nth user

Formulation (contd.)

Let network charges be fixed at M / unit resources Each user tries to maximize his/her net utility

Un(n()) – M.n() Un is non-decreasing and Un(0) = 0; U’n is non-increasing, i.e. Un is concave

Maximum net benefit of nth user yn = arg max (Un(n()) – M.n()) = (U’n)-1.(M)

Action of nth user Modify n to make received QOS n() equal to desired yn

User iterations and equilibrium

After the jth iteration/step, access parameter of user n: n

j+1 = min (G (yn, n(j), nj), n

max) G (y, , )

, if = y > , if < y < , if > y

Nash equilibria A fixed or equilibrium point of this iteration is any * [0, n

max] n* = min (G (yn, n(*), n*), n

max)

By Brouwer’s fixed point theorem there exists at least one such fixed point.

Non-cooperative game for circuit switched network

N users compete for K circuits nth user’s connection setup request is Poisson with

intensity n and arbitrary holding time distribution with mean 1/ n

Total traffic intensity: .1/ Aggregate arrival rate n=1 N n

Mean holding time over all connections 1/ = n=1N 1/n n/

Hence, = n=1N n/n

Per user connection blocking probability (Erlang’s form) K() (K/K!) / (k=0

K k/k!)

Formulation leading to equilibrium

Net arrival rate of nth user: n(1 - K())

Mean number of occupied circuits for the nth user: n() 1/n n(1 - K(()))

Thus, n and depend on all arrival rates

Iteration using multiplicative increase and decrease n

j+1 = min {yn/n . n, nmax}

or, nj+1 = min {yn.n / (1 - K((j))) , n

max}

By previous formulation we can find an equilibrium!

References

Game Theory .NET - college lecture notes (http://www.gametheory.net) IE675: Game Theory - Dr. Wayne Bialas, Dept. of IE, SUNY Buffalo,

(http://www.acsu.buffalo.edu/~bialas/IE675.html ) “Computational Finance: Game and Information Theoretic Approach” –

Dr. B. Mishra, Dept. of CS, NYU (http://www.cs.nyu.edu/mishra/COURSES/GAME/game.html)

Introduction to Game Theory – Markus Mobius, Dept. of Economics, Harvard (http://icg.fas.harvard.edu/~ec1052/lecture/index.html)

Infocom 2003 - “Nash equilibria of a generic networking game with applications to circuit-switched networks” - Youngmi Jin and George Kesidis, Dept. of EE & CS, Pennsylvania State University.