Game and Evolutionary Game in Communication Networks Yuedong Xu 2013.12.04

Game and Evolutionary Game in Communication Networks

Yuedong Xu

2013.12.04

Outline

• Game Theory: A Premier

• Evolutionary Game

• Applications to Networks

• Potential Research Fields

Using as less math as possible !

Game Theory: A Premier• What is “game” about?

• Game of Chicken– driver who swerves away looses

• What should drivers do?– To swerve or to stay?

2

2

Game Theory: A Premier• What is “game” about?

• Game of Chicken– driver who swerves away looses

2

2

swerve stay

swerve

0, 0 -1, 5

stay 5, -1-10, -

10

Driver 1

Driver 2Drivers want to do opposite of one another

Game Theory: A Premier

• A Game consists of– at least two players – a set of strategies for each player– a payoff for each strategy profile

• Basic assumption (rationality of players)• Nash Equilibrium

– no player can improve its payoff by unilaterally changing its strategy

• Pareto optimality, price of anarchy


• Non-Cooperative (Competitive) Games– individualized play

• Cooperative Games– play as a group

• Repeated, Stochastic and Evolutionary Games– not one shot

Classification 1:


Classification 2:

Non-cooperative Cooperative

Static Dynamic (repeated)

Strategic-form Extensive-form

Perfect information Imperfect information

Complete information

Incomplete information


Internet Application

Selfish Routing game

s

v

w

t

C(x) = 1

C(x) = xC(x) = 1

C(x) = x

C(x) = 0



P2P Networks: Bittorrent, Xunlei, Pplive, PPStream, QQLive …



Internet Ecosystem (Business Models)



Cloud Computing game



Online Social Networks



Network Security Game


Wireless Application

802.11 multiple access game



3G/4G Power Control Game



Packet forwarding game

?

?

Blue Green



Cognitive radio network game



Wireless jamming and eavesdrop games

E

Outline





Recap

• Classical game theory (CGT) – Outcome depends on strong rationality

assumption– Each individual uses a strategy that is the "best

response" to other players’ choice

– Question: what is the meaning of a symmetric NE , , given a large number of players ?

Follow the crowd!

Evolutionary game theory

• Evolutionary game theory (EGT)

– refinement of CGT– game in a population– dynamics of strategy adoption– mutual learning among players

Evolutionary game theory differs from classical game theory by focusing more on the dynamics of strategy change as influenced not solely by the quality of various competing strategies, but by the effect of frequency with which the various competing strategies are found in the population.


• Evolutionary game theory (EGT)– Usually two types of game: games against the field

and games with pairwise contests

A game against the field is one in which there is no specific “opponent”for a given individual - their payoff depends on what everyone in thepopulation is doing. Ex: Choice of Gender

A pairwise contest game describes a situation in which a givenindividual plays against an opponent that has been randomly selected(by nature) from the population and the payoff depends just on whatboth individual do. Ex: Hawk-Dove Game


• A profile of evolutionary game

• Payoff (fitness)

Given a set of pure strategy S. A population profile is a vector x that gives a probability x(s) with which each strategy s S is played in the population.

Consider a particular individual in the population with profile x. If that individual uses a profile σ={, }, the individual’s payoff is denoted as . The payoff of this strategy for a pair-wise game is


• Evolutionary stable strategy (ESS)

• Theorem (ESS)

An evolutionarily stable strategy is a strategy which, if adopted by a population of players, cannot be invaded (or replaced) by any alternative strategy that is initially rare.


• Example (Hawk-Dove Game)– H: aggressive; D: mild– Population strategy – Mixed strategy (H,D) of an individual– Payoff matrix (v<c):

– Suppose the existence of an ESS


• Example (Hawk-Dove Game) ‘cont– In the population, the payoff of a mutant is



• ESS– no statement of dynamics– monomorphic / polymorphic

• Replicator Dynamics– individuals, called replicator, exist in several different types

(e.g Hawk and Dove)– each type of individual uses a pre-programmed strategy and

pass it to its descendants– individuals only use PURE strategy in a finite set– the population state is , where is fraction of individuals using

strategy


• Replicator Dynamics

– Fixed point: – Stability of fixed point:

– Stability proof:

�̇�𝑖=𝑥 𝑖(𝜋 ¿¿ 𝑖 (𝑠𝑖 , 𝒙 )−𝜋 (𝑡 ))¿

Lyapunov stability vs asymptotic stability

Lyapunov function and Engenvalue approach


• ESS vs NE in associated two-player game

– An ESS is a (mixed) NE – A NE might not be an ESS

• Asymmetric NE in monomorphic population• Unstable NE


• Replicator dynamics and NE– In a two-strategy game

• Any NE is a fixed point of replicator dynamics• Not every fixed point corresponds to a NE

• Replicator dynamics and ESS– ESS is an asymptotically stable fixed point– Two strategy pair-wise contest

– More than two strategies

ESS Asymp. Stable f.p. sym. NE f.p.

ESS Asymp. Stable f.p. sym. NE f.p.

Outline





Peer-to-peer file sharingWireless networks

Peer-to-peer file sharing

• File Piece (e.g. chunk, block)– A content is split in pieces– Each piece can be independently downloaded

• Leecher– A peer that is client and server– In the context of content delivery

• Has a partial copy of the content

• Seed– A peer that is only server– In the context of content delivery

• Has a full copy of the content

• Great improvement over customer-server mode

• Ideal system: single chunk, fully cooperative

• Big System: many peers, many chunks, stochastic system

time

t=0

t=T

t=2T

Seed

34


Which peers shall I serve in each time slot?


• If no good incentive strategy– Slow service– Even overwhelmed by requests

• Incentive model– A strategy is the behavior (providing/rejecting a

service) of a peer against other peers– A policy is the set of rules of for incentivization– A point is a utility measure of peers– A system is robust : convergence and cooperationQ. Zhao, J. Lui, D. Chiu“A Mathematical Framework for Analyzing Adaptive Incentive Protocols in P2P Networks”, IEEE/ACM Trans. Networking, 2012


• Incentive model (’cont)– Strategy = type of peer– Finite strategies

• {cooperator, defector, reciprocator}

Always serve

Always reject

Serve cooperators and reciprocators with certain probabilities,reject defectors


• Incentive model (’cont)– System description:

– Incentive scheme (esp. for reciprocators)

At the beginning of each time slot, each peer randomly selects another peer to request for service. The selected peer chooses to serve the request based on his current strategy. A peer obtains α points if its request is served and loses β (=1) points if it provides service to others.

- Prob. that a type i peer provides service to a type j peer


• Utility model– After a long way, the points gained by a type-i peer

• We can now study– equilibrium state (given G)– is the equilibrium stable?– how to reach this equilibrium?– how good is the incentive scheme

Type-i payoffNetwork payoff

Is this enough?


• Learning model in P2P networks– Current best learning model

At the end of each slot, a peer chooses to switch to another strategy s’ with certain prob. To decide which strategy to choose, the peer learns from other peers.

𝑥𝑖 (𝑡+1 )=𝑥 𝑖 (𝑡 )−𝛾 𝑥𝑖 (𝑡 ) (𝑃h (𝑡 )− 𝑃 𝑖 (𝑡 ) ) , 𝑖≠h ,

𝑥h (𝑡+1 )=𝑥h (𝑡 )+𝛾 ∑𝑖=1 , ≠h

3

𝑥 𝑖 (𝑡 )(𝑃h (𝑡 )−𝑃𝑖 (𝑡 ))

Needs to compute the gains of all other peers !


• Learning model in P2P networks– Opportunistic learning model

At the end of each slot, each peer chooses another peer as its teacher with certain prob. If the teacher is of a different type and performs better, this peer adapts to the teacher’s strategy with another prob.

𝑥 𝑗 (𝑡+1 )=𝑥 𝑗 (𝑡 )+𝛾 𝑥 𝑗 (𝑡 )(𝑃 𝑗 (𝑡 )− 𝑃 (𝑡 ))

Simpler !


• Now we can study– Robustness of incentive scheme

• Mirror incentive policy– reciprocators are tit-for-tat

• Proportional incentive policy– A reciprocators always serves any other reciprocator

• Linear incentive policy

Each scheme generates a different matrix G !

Prob. That reciprocators serve other types of peers!



• In relation to EG– pair-wise contest population game– peers players; chunk exchange2 players games

�̇�𝑖 (𝑡 )=𝛾 𝑥 𝑖 (𝑡 )(𝑃 𝑖 (𝑡 )−𝑃 (𝑡 ))Opp. Learning

𝜋 𝑖(𝑠𝑖 , 𝒙)=𝑃 𝑗 (𝑡 ) After some efforts

�̇�𝑖=𝑥 𝑖(𝜋 ¿¿ 𝑖 (𝑠𝑖 , 𝒙 )−𝜋 (𝑡 ))¿ Replicator dynamics

�̇�𝑖 (𝑡 )=𝛾 𝑥 𝑖 (𝑡 ) (𝑃h (𝑡 )−𝑃 𝑖 (𝑡 ) ) ,𝑖≠h𝑥h (𝑡 )=𝛾 (𝑃h (𝑡 )− 𝑃 (𝑡 ) )

Curr. Best Learning

Large-scale wireless networks

• Random multiple access (slotted ALOHA)– A node transmits with prob. p in each slot– Simultaneous transmission collisions


• Power control game (signal to noise interference ratio, SINR)

– Large power better throughput– Large power more interference to other

receivers



• Sad facts:– Selfishness is unsuccessful– Optimal cooperation is hard in a large distributed

networks (bargaining, Shapley value)

• Evolutionary game kicks in!

H. Tembine, E. Altman, “Evolutionary Games in Wireless Networks”, IEEE Trans. Syst. Man Cyber. B, 2010

What if wireless nodes learn from each other in local interactions?


• Challenges– Standard EGT: a player interacts with all other

players (or average population)– Large-scale wireless networks:

• no longer strategic pair-wise competition• random number of local players• non-reciprocal interactions

– Finite strategies of a player {transmit, stay quiet} in multiple access game {high power, low power} in power control game

Non standard EGT Standard EGT


• WCDMA power control game– SINR with distance r between transmitter and

receiver of node i is given by

PH

PLPH

Pi : the strategy of node i (i.e., PH or PL)x : the proportion of the population choosing PHg : channel gain, r0 is the radius-of-reception circle of receiverα : the attenuation order with value between 3 and 6, σ : the noise power, and β : the inverse of processing gainI(x) : total interference from all nodes to the receiver of node i


• WCDMA power control game– Payoff of node i is as follows:

R : transmission rangewp : cost weight due to adopting power Pi (e.g. energy consumption)ζ(r) : probability density function given the density of receiver


• WCDMA power control game– Existence of uniqueness of ESS

• Replicator dynamics

This function is continuous and strictly monotonic, which is required for the proof of stability based on sufficient condition


• Some other related works– Extensions to EGT

– Applications

P. Coucheney, C. Touati. “Fair and Efficient User-Network Association Algorithm for Multi-Technology Wireless Networks”, IEEE Infocom 2009 (mini)S. Shakkottai, E. Altman. “The Case for Non-cooperative Multihoming of Users to Access Points in IEEE 802.11 WLANs”, IEEE Infocom 2006C. Jiang, K. Liu, “Distributed Adaptive Networks: A Graphical Evolutionary Game-Theoretic View”, IEEE Trans. Signal Processing, 2013

E. Altman, Y. Hayel. “Markov Decision Evolutionary Games”, IEEE Trans. Auto. Ctrl. 2010X. Luo and H. Tembine. “Evolutionary Coalitional Games for Random Access Control”, IEEE Infocom 2013 (mini)


• Summary– P2P : practical problem EG theory– WCDMA: EG theory practical problem

– Common Challenges:• difficult to find important problem• difficult to have theoretical contributions to EGT

Two different styles !

Thank you!

Q & A

Documents

Game and Evolutionary Game in Communication Networks Yuedong Xu 2013.12.04