View
216
Download
1
Category
Preview:
Citation preview
An Architectural View of Game Theoretic Control
Raga Gopalakrishnan and Adam WiermanCalifornia Institute of Technology
Jason R. MardenUniversity of Colorado at Boulder
6/18/2010 Hotmetrics 2010
Distributed Resource AllocationSensor Coverage Wireless Access Point Selection
Wireless Channel Selection Power Control (sensor networks)
Resource Allocation Problem – A Simple Model
• Set of (distributed) agents, N = {1, 2, . . ., n}• Set of resources, R• Action sets, Ai µ 2R for agents i 2 N– Set of action profiles, A = A1 £ A2 £ . . . £ An
– Set of agents choosing resource r in action profile a, {a}r
• Objective function, W : A! R– Linearly separable, i.e., W(a) = r2R Wr ( {a}r )
Goal: Find an allocation a 2 A that maximizes W(a)
Distributed Approaches
Distributed Optimization
Lyapunov-based Control
Physics-inspired Control
Game-theoretic Control
Distributed Approaches
Distributed Optimization
Lyapunov-based Control
Physics-inspired Control
Game-theoretic Control
Promising new approach Model the agents as “self-interested”
players in a non-cooperative game
Still being explored The solution to the problem emerges
as the equilibrium of the game
Modeling the problem as a game
• Set of players, N = {1, 2, . . ., n}
• Action sets, Ai µ 2R for players i 2 N– Set of action profiles, A = A1 £ A2 £ £ An
– Set of players choosing resource r in action profile a, {a}r
• Utility functions, Ui : A! R for players i 2 N– Linearly separable, i.e., Ui(a) = r2R fr ( i,
{a}r )
• Welfare function W : A! R– Linearly separable, i.e., W(a) = r2R Wr
( {a}r )
Resource Allocation Problem Resource Allocation Game
• Set of agents, N = {1, 2, . . ., n}• Set of resources, R• Action sets, Ai µ 2R for agents i 2 N
– Set of action profiles, A = A1 £ A2 £ . . . £ An
– Set of agents choosing resource r in action profile a, {a}r
• Objective function, W : A! R– Linearly separable, i.e., W(a) = r2R Wr
( {a}r )
Game Theoretic Control (GTC)
Setup the game1
Design the players2
decision makers/players action sets
utility functions
agent decision rules(learning rules)
Desirable globalbehavior emergesas equilibrium ofthe game
Goal:
• A Nash equilibrium is an action profile a*2 A such that for each player i,
• Measures of efficiency for Nash equilibrium:
Game Theoretic Control (GTC)
Setup the game1
Design the players2
decision makers/players action sets
utility functions
agent decision rules(learning rules)
Desirable properties Existence of an equil. Efficiency of an equil. Tractability Locality of information Budget balance …
Desirable properties Locality of information Fast convergence Equilibrium selection Robust convergence …
Learning Design
Utility Design
Inherited
DesignedDesigned
Many other applications: [Akella et al. 2002, Kaumann et al. 2007, Marden et al. 2007, 2008, Mhatre et al. 2007, Komali and MacKenzie 2007, Zou and
Chakrabarty 2004, Campos-Nanez 2008, Marden & Effros 2009]
[Marden, Wierman 2008]
[Campos-Nanez, Garcia, Li 2008]
Applications of GTC
Utility Design
Learning Design
Sensor Coverage
Power Control (sensor networks)Is there a way to view Game Theoretic Control from an application-independent
perspective?
Architectural View for GTC
Utility Design
Learning Design
Class of Games
“Virtualization”layer
IP
NetworkApps
Networkhardware
OS
software
hardware
• Potential Games are games for which there exists a potential function F : A! R such that ∀ i 2 N, ∀a–i 2 A–i , ∀ ai, ai’ 2 Ai , it holds that
F (ai , a–i) – F (ai’ , a–i) = Ui (ai , a–i) – Ui (ai’ , a–i)
• Key Property: Local maxima of F are Nash equilibria
Potential Games-based Architecture
Utility Design
Learning Design
Potential Games
Unifying view of several existing designs:
[Akella et al. 2002][Kaumann et al. 2007]
[Marden et al. 2007, 2008][Mhatre et al. 2007]
[Komali and MacKenzie 2007][Zou and Chakrabarty 2004]
[Campos-Nanez 2008][Marden & W 2008]
[Marden & Effros 2009]and many others…
Utility Design (examples)
• Wonderful Life Utility (WLU) [Wolpert et al. 1999]
– Potential game with © = W (hence, price of stability = 1)– Price of anarchy = ½ for sub-modular games
• Shapley Value Utility (SVU) [Shapley 1953]– Potential game– Price of anarchy = Price of stability = ½ for sub-modular games
• Weighted SVU [Shapley 1953]– Similar properties as SVU
Adapted from cost-sharing literature in economic theory [Marden, Wierman]
Learning Design (examples)• Gradient Play [Ermoliev et al. 1997, Shamma et al. 2005]– Convergence to a Nash equilibrium
• Joint Strategy Fictitious Play (JSFP) [Marden et al. 2009]– Convergence to a Nash equilibrium
• Log-Linear Learning [Blume 1993, Marden et al.]– Convergence to the best Nash equilibrium
• Many others . . . [Ozdaglar et al. 2009, Shah et al. 2010]
Potential Games-based Architecture
Utility Design
Learning Design
Potential Games
SVU WonderfulLife
WSVU
GradientPlay
Log-Linear
Learning JSFP
+ Modularity / Decoupling
+ Flexibility
? Relationships to other approaches
? Limitations
+ Modularity / Decoupling
+ Flexibility
? Relationships to other approaches
? Limitations
Distributed Approaches
Distributed Optimization
Lyapunov-based Control
Physics-inspired Control
Potential Games
UtilityDesign
LearningDesign
Relationships to Other Approaches
Game-theoretic Control
• Distributed Constraint Optimization Problem (DCOP)
– Utility Design: WLU– Learning Design: Variety
Chapman, Rogers, Jennings – Benchmarking hybrid algorithms for distributed constraint optimization games [OptMAS ‘08]
Potential Games
WLU
Variety
Distributed Optimization
Distributed Approaches
Distributed Optimization
Lyapunov-based Control
Physics-inspired Control
Potential Games
UtilityDesign
LearningDesign
Game-theoretic Control
Relationships to Other Approaches
• Gibbs-sampler-based control―Utility Design: WLU―Learning Design: Log-Linear Learning
Access Point Selection Channel Selection
Kauffmann, Baccelli, Chaintreau, Mhatre, Papagiannaki, Diot – Measurement-based self organization of interfering 802.11 wireless access networks [INFOCOM ‘07]
Potential Games
WLU
Log-Linear Learning
Physics-inspired Control
We prove that
Distributed Approaches
Distributed Optimization
Lyapunov-based Control
Physics-inspired Control
Potential Games
UtilityDesign
LearningDesign
Game-theoretic Control
Relationships to Other Approaches
Distributed Approaches
Distributed Optimization
Lyapunov-based Control
Physics-inspired Control
Potential Games
UtilityDesign
LearningDesign
Game-theoretic Control
Relationships to Other Approaches
Potential Games-based Architecture
Utility Design
Learning Design
Potential Games
SVU WonderfulLife
WSVU
GradientPlay
Log-Linear
Learning JSFP
+ Modularity / Decoupling
+ Flexibility Relationships to
other approaches
? Limitations
+ Modularity / Decoupling
+ Flexibility Relationships to
other approaches
? Limitations
Do Potential Games Suffice?No utility design with all the
desirable properties
Utility Design
Learning Design
POTENTIAL GAMES
Desirable properties Existence of an equil. Efficiency of an equil.Budget balanceTractability Locality of information …
Not always!
Open Question: What other limitations are there?
Any linearly separable, budget-balanced utility design that guarantees equilibrium existence has PoS · ½
[Marden, Wierman 2009]
Summary
Utility Design
Learning Design
Potential Games
SVU WonderfulLife
WSVU
GradientPlay
Log-Linear
Learning JSFP
+ Modularity / Decoupling
+ Flexibility Relationships to
other approaches― Not all desirable
properties can be achieved
+ Modularity / Decoupling
+ Flexibility Relationships to
other approaches― Not all desirable
properties can be achieved
? Beyond Potential Games
Conclusion
Utility Design
Learning Design
Potential Games
SVU WonderfulLife
WSVU
GradientPlay
Log-Linear
Learning JSFP
+ Modularity / Decoupling
+ Flexibility Relationships to
other approaches― Not all desirable
properties can be achieved
?
Other choices for virtualization layer
[MW’09,AJWG’09,Sv’09]Strengths and Limitations
A library of architectures
Thank You
Recommended