Although this may seem a paradox,
all exact science is dominated by
the idea of approximation.
Bertrand Russell (1872-1970)
Exact algorithms have been studied
intensively for over four decades,
and yet basic insights are still being obtained.
Since polynomial time solvability is the exception
rather than the rule, it is only reasonable
to expect the theory of approximation algorithms
to grow considerably over the years.
Beyond the list …
Unique Games Conjecture
Simpler proof of PCP Theorem
Online algorithms for AdWords problem
Beyond the list …
Unique Games Conjecture
Simpler proof of PCP Theorem
Online algorithms for AdWords problem
Integrality gaps vs approximability
Raghevendra, 2008: Assuming UGC,
for every constrained satisfaction problem:
Can achieve approximation factor
= integrality gap of “standard SDP”
NP-hard to approximate better.
Future Directions
Status of UGC
Raghavendra-type results for LP-relaxations
Randomized dual growth in
primal-dual algorithms
Approximability: sharp thresholds
For a natural problem:
Can achieve approximation factor in P.
If we can achieve in P
=> complexity-theoretic disaster
α(n)
α(n) − ∈(n)
Conjecture
There is a natural problem
having sharp thresholds
w.r.t. time classes
α1(n) > α 2 (n) > ... > α k (n)
P=T1(n) ⊂ T2 (n) ⊂ ... Tk(n)
Group Steiner Tree Problem
Chekuri & Pal, 2005:
Halperin & Krauthgamer, 2003:
log2−∈n factor algorithm in time
2^ (2^ ( log nO(∈)))
time = 2^(2^( log no(∈)))⇒ subexponential algorithm for 3SAT
Combinatorial optimization
Central problems have LP-relaxations
that always have integer optimal solutions!
ILP: Integral LP
Combinatorial optimization
Central problems have LP-relaxations
that always have integer optimal solutions!
ILP: Integral LP
i.e., it “behaves” like an IP!
Cornerstone problems in P
Matching (general graph) Network flow Shortest paths Minimum spanning tree Minimum branching
Is combinatorial optimizationrelevant today?
Why design combinatorial algorithms,
especially today that LP-solvers are so fast?
Combinatorial algorithms
Very rich theory
Gave field of algorithms some of its formative
and fundamental notions, e.g. P
Preferable in applications, since efficient
and malleable.
Rational convex program
A nonlinear convex program that
always has a rational solution (if feasible),
using polynomially many bits,
if all parameters are rational.
Rational convex program
Always has a rational solution (if feasible)
using polynomially many bits,
if all parameters are rational.
i.e., it “behaves” like an LP!
Rational convex program
Always has a rational solution (if feasible)
using polynomially many bits,
if all parameters are rational.
i.e., it “behaves” like an LP!
Do they exist??
KKT optimality conditions
−∇ f0 (x) = yi fii
∑ '(x) + z jj
∑ a j
yi ≥ 0 for 1≤ i ≤ m
yi > 0 ⇒ fi (x) = 0 for 1≤ i ≤ m
fi (x) ≤ 0 for 1≤ i ≤ m
a jT x ≤ b j for 1≤ j ≤ p
Two opportunities for RCPs:
Program A: Combinatorial, polynomial time
(strongly poly.) algorithm
Program B: Polynomial time (strongly poly.)
algorithm, given LP-oracle.
Helgason, Kennington & Lall, 1980Single constraint
Minoux, 1984Minimum quadratic cost flow
Frank & Karzanov, 1992Closest point from origin to bipartite perfect
matching polytope.
Karzanov & McCormick, 1997Any totally unimodular matrix.
Combinatorial Algorithms
Ben-Tal & Nemirovski, 1999
Polyhedral approximation of second-order cone
Main technique: Solves any quadratic RCP
in polynomial time, given an LP-oracle.
Ben-Tal & Nemirovski, 1999
Polyhedral approximation of second-order cone
Main technique: Solves any quadratic RCP
in polynomial time, given an LP-oracle.
Strongly polynomial algorithm?
Logarithmic RCPs
Rationality is the exception to the rule,
and needs to be established piece-meal.
f0 (x) = − mii∑ log(ui (x))
where mi > 0 and ui (x) is linear in x.
Logarithmic RCPs
Optimal solutions to such RCPs capture
equilibria for various market models!
f0 (x) = − mii∑ log(ui (x))
where mi > 0 and ui (x) is linear in x.
Arrow-Debreu Theorem, 1954
Celebrated theorem in Mathematical Economics
Established existence of market equilibrium under very general conditions using a deep theorem from topology - Kakutani fixed point theorem.
Arrow-Debreu Theorem, 1954
Celebrated theorem in Mathematical Economics
Established existence of market equilibrium under very general conditions using a theorem from topology - Kakutani fixed point theorem.
Highly non-constructive!
Arrow-Debreu Theorem, 1954
Celebrated theorem in Mathematical Economics
Established existence of market equilibrium under very general conditions using a theorem from topology - Kakutani fixed point theorem.
Continuous, quasiconcave,
satisfying non-satiation.
Complexity-theoretic question
For “reasonable” utility fns.,
can market equilibrium be computed in P?
If not, what is its complexity?
Short summary
So far, all markets
whose equilibria can be computed efficiently
admit convex or quasiconvex programs,
many of which are RCPs!
Combinatorial Algorithm for Linear Case of Fisher’s Model
Devanur, Papadimitriou, Saberi & V., 2002
By extending primal-dual paradigm to setting of convex programs & KKT conditions
Eisenberg-Gale Program, 1959
max log
. .
:
: 1
: 0
i ii
i ij ijj
iji
ij
m v
s t
i v
j
ij
u xx
x
prices pj
KKT conditions
1). ∀j : pj ≥0
2). ∀j : pj > 0 ⇒ xij =1i∑
3). ∀i, j :uij
pj
≤vi
m(i)
4). ∀i, j : xij > 0 ⇒uij
pj
=vi
m(i)=
uijxijj∑m(i)
Proof of rationality
Guess positive allocation variables (say k).
Substitute 1/pj by a new variable.
LP with (k + g) equations and
non-negativity constraint for each variable.
Auction for Google’s TV ads
N. Nisan et. al, 2009:
Used market equilibrium based approach.
Combinatorial algorithms for linear case
provided “inspiration”.
Long-standing open problem
Complexity of finding an equilibrium for
Fisher and Arrow-Debreu models under
separable, piecewise-linear, concave utilities?
Long-standing open problem
Complexity of finding an equilibrium for
Fisher and Arrow-Debreu models under
separable, piecewise-linear, concave utilities?
Equilibrium is rational!
Markets with separable, plc utilitiesare PPAD-complete
Chen, Dai, Du, Teng, 2009
Chen & Teng, 2009
V. & Yannakakis, 2009
Markets with separable, plc utilitiesare PPAD-complete
Chen, Dai, Du, Teng, 2009
Chen & Teng, 2009
V. & Yannakakis, 2009
(Building on combinatorial insights from DPSV)
Theorem (V., 2002): Generalized linear Fisher market to Spending constraint utilities. Polynomial time algorithm for computing equilibrium.
Is there a convex program for this model?
“We believe the answer to this question should be ‘yes’. In our experience, non-trivial polynomial time algorithms for problems are rare and happen for a good reason – a deep mathematical structure intimately connected to the problem.”
EG convex program = Devanur’s program
Price disc. Market
Goel & V.
Spending constraint marketV., 2005
Nash BargainingV., 2008
Eisenberg-Gale MarketsJain & V., 2007
EG[2] MarketsChakrabarty, Devanur & V.
2008
V., 2010: Assuming perfect price
discrimination, can handle:
Continuously differentiable, quasiconcave
(non-separable) utilities, satisfying non-satiation.
V., 2010:
Continuously differentiable, quasiconcave
(non-separable) utilities, satisfying non-satiation.
Compare with Arrow-Debreu utilities!!
continuous, quasiconcave, satisfying non-satiation.
A new development
Orlin, 2009: Strongly polynomial algorithm
for Fisher’s linear case, using scaling.
Open: For rest
Sturmfels & Uhler, 2009:
S f =0 n×n, sample covariance matrix
G=([n],E) chordal graph
Then the following is an RCP:min log det Σ
s.t. Σij =Sij ∀(i, j)∈E or i = j
EG convex program = Devanur’s program
Price disc. Market
Goel & V.
Spending constraint marketV., 2005
Nash BargainingV., 2008
Eisenberg-Gale MarketsJain & V., 2007
EG[2] MarketsChakrabarty, Devanur & V.
2008
Building on
Karzanov & McCormick, 1997:
Combinatorial algorithm for min cost flow
under concave cost functions on edges.
EG convex program = Devanur’s program
Price disc. Market
Goel & V.
Spending constraint marketV., 2005
Nash BargainingV., 2008
Eisenberg-Gale MarketsJain & V., 2007
EG[2] MarketsChakrabarty, Devanur & V.
2008
EG convex program = Devanur’s program
Price disc. Market
Goel & V.
Spending constraint marketV., 2005
Nash BargainingV., 2008
Eisenberg-Gale MarketsJain & V., 2007
EG[2] MarketsChakrabarty, Devanur & V.
2008