Upload
velma
View
251
Download
1
Tags:
Embed Size (px)
DESCRIPTION
COLOR TEST COLOR TEST COLOR TEST COLOR TEST. Dueling Algorithms. Nicole Immorlica , Northwestern University with A. Tauman Kalai , B. Lucier , A. Moitra , A. Postlewaite , and M. Tennenholtz. Social Contexts. Normal-form games : - PowerPoint PPT Presentation
Citation preview
COLOR TESTCOLOR TESTCOLOR TESTCOLOR TEST
Dueling Algorithms
NICOLE IMMORLICA, NORTHWESTERN UNIVERSITY
WITH A. TAUMAN KALAI, B. LUCIER, A. MOITRA, A. POSTLEWAITE, AND M. TENNENHOLTZ
Social Contexts
Normal-form games:Players choose strategies to maximize expected von Neumann-Morgenstern utility.
Social context games [AKT’08]:Players choose strategies to achieve particular social status among peers.
Social Contexts
Ranking games [BFHS’08]:Players choose strategies to achieve particular payoff rank among peers.
Two-Player Ranking Games
GAlice
Bob
RG payoff of Alice:
1 Alice beats Bob in G
Alice and Bob play game:
½ Alice ties Bob in G
0 Alice loses to Bob in G
Implicit Representations
Succinct games [FIKU’08]:Payoff matrix represented by boolean circuit. NE hard to solve or approximate.
Blotto games [B’21, GW’50, R’06, H’08]:Distribute armies to battlefields.
Implicit Representations
Optimization duels [this work]:Underlying game is optimization problem. Goal is to optimize better than opponent.
Ranking Duel
A search engine is an algorithm that inputs• set Ω = {1, 2, …, n} of items• probabilities p1 + … + pn = 1 of eachand outputs a permutation π of Ω.
Monopolist objective: minimize Ei~p[π(i)].
Ranking Duel
Competitive objective: Let the expected score of a ranking π versus a ranking π’ be
Pri~p[ π(i) < π’(i) ] + (½) Pri~p[ π(i) = π’(i) ].
Then objective is to output a π that maximizes expected score given algorithm of opponent.
Optimizing a Search Engine
?
User searches for object drawn according to known probability dist.
0.19 0.16 0.27 0.07 0.22 0.09
Search: pretty shape
1. (27%)
2. (22%)
3. (19%)
4. (16%)
5. (09%)
6. (07%)
Greedy is optimal.
Choosing a Search Engine
1. Search for “pretty shape”.2. See which search engine ranks
my favorite shape higher.3. Thereafter, use that one.
0.19 0.16 0.27 0.07 0.22 0.09
Search: pretty shape
1. (27%)
2. (22%)
3. (19%)
4. (16%)
5. (09%)
6. (07%)
Search: pretty shape
6. (27%)
1. (22%)
2. (19%)
3. (16%)
4. (09%)
5. (07%)
Questions
Can we efficiently compute an equilibrium of a ranking duel?
How poorly does greedy perform in a competitive setting?
What consequences does the duel have for the searcher?
Optimization Problems as DuelsRanking Binary Search Routing
ParkingCompressionHiring
Start
Finish
?
??
??
?
?
Duel Framework
Finite feasible set X of strategies.Prob. distribution p over states of nature Ω.Objective cost c: Ω × X R.
Monopolist: choose x to minimize Eω~p[cω(x)].
Duel Framework
1. Players select strategies x, x’ from X.2. Nature selects state ω from Ω according to p.3. Payoffs v(x,x’), (1-v(x,x’)) are realized.
1 if cω(x) < cω(x’)
0 if cω(x) > cω(x’)
½ if cω(x) = cω(x’)
v(x,x’) = Eω~p
Results: Computation
An LP-based technique to compute exact equilibria, A low-regret learning technique to compute approximate equilibria,
… and a demonstration of these techniques in our sample settings
Computing Exact Equilibria
Formulate game as bilinear duel:1. Efficiently map strategies to points X in Rn.2. Define constraints describing K=convex-hull(X).3. Define payoff matrix M that computes values.4. Maps points in K back to strategies in original
setting.
Bilinear Duels
If feasible strategies X are points in Rn, and payoff v(x, x’) is xtMx’ for some M in Rnxn, then
maxv,x v s.t. xtMx’ ≥ v for all x’ in X x is in K (=convex-hull(X))
Exponential, but equivalent poly-sized LP.
Ranking Duel
Formulate game as bilinear duel:1. Efficiently map strategies to points X in Rn.
X = set of permutation matrices(entry xij indicates item i placed in position j)
2. Define constraints describing K=convex-hull(X).K = set of doubly stochastic matrices(entry yij = prob. item i placed in position j)
Ranking Duel
Formulate game as bilinear duel:4. Design “rounding alg.” that maps points in K back
to strategies in original setting.Birkhoff–von Neumann Theorem: Can efficiently construct permutation basis for doubly stochastic matrix (e.g., via matching).
Ranking Duel
Formulate game as bilinear duel:3. Define payoff matrix M that computes values.
Ep,y,y’[v(x,x’)] = ∑i p(i) ( ½ Pry,y’ [x i = x’i ] + Pry,y’ [x i > x’i ])= ∑i p(i) (∑i yij ( ½ y’ij + ∑k>j y’ik ))
which is bilinear in y,y’ and so can be written ytMy’.
Ranking Duel
Result: Can reduce computation time to poly(n) versus poly(n!) with standard LP approach.
Technique also applies to hiring duel and binary search duel.
Compression Duel
data
Goal: smaller compression (i.e., lower depth in tree).
(each with prob. p(.))
Classical Algorithm
Huffman coding:Repeatedly pair nodes with
lowest probability.
Compression Duel
Formulate game as bilinear duel:1. Efficiently map strategies to points X in Rn.
X = subset of zero-one matrices*(entry xij indicates item i placed at depth j)
2. Define constraints describing K=convex-hull(X).K = subset of row-stochastic matrices*(entry yij = prob. item i placed at depth j)
* Must correspond to depth profile of some binary tree!
Compression Duel
Formulate game as bilinear duel:3. Define payoff matrix M that computes values.
Ep,y,y’[v(x,x’)] = ∑i p(i) (∑i yij ( ½ y’ij + ∑k>j y’ik ))
which is bilinear in y,y’ and so can be written ytMy’.
Compression Duel
Bilinear Form:maxv,x v s.t. xtMx’ ≥ v for all x’ in X x is in K (=convex-hull(X))
Problems:1. How to round points in K back to a random binary tree with right depth profile?2. How to succinctly express constraints describing K?
Approximate Minimax
Defn. For any ε > 0, an approximate minimax strategy guarantees payoff not worse than best possible value minus ε.
Defn. For any ε > 0, an approximate best response has payoff not worse than payoff of best response minus ε.
Best-Response Oracle
Idea. Use approximate best-response oracle to get approximate minimax strategies.
1. Low-regret learning: if x1,…,xT and x’1,…,x’T have low regret, then ave. is approx minimax.2. Follow expected leader: on round t+1, play best-response to x1,…,xt to get low-regret.
Compression Best-Response
Given lists of items with values and weights, pick one from each list with max total value and total weight at most one.
Multiple-choice Knapsack:
Compression Best-Response
Depth: 1 2 3 4
Compression Best-Response
(each with prob. p(.)) x’ in K
For j from 1..n, list of depth j:
v( ) = Pr[win at depth j | x’ ]w( ) = 2-j
… Kraft inequality
Other Duels
1. Hiring duel: constraints defining Euclidean subspace correspond to hiring probabilities.
2. Binary search duel: similar to hiring duel, but constraints defining Euclidean subspace more complex (must correspond to search trees).
3. Racing duel: seems computationally hard, even though single-player problem easy.
Conclusion
• Every optimization problem has a duel.• Classic solutions (and all deterministic
algorithms) can usually be badly beaten.• Duel can be easier or harder to solve, and can
lead to inefficiencies.
OPEN QUESTION: effect of duel on the solution to the optimization problem?