Approximability& Sums of Squares

Ryan O’DonnellCarnegie Mellon

Basic Optimization Problems

Minimum-Balanced-Separator:

Given G=(V,E), partition V into 2 parts,each of size at least n/3,

minimize # of edges crossing partition.

Basic Optimization Problems

Minimum-Balanced-Separator:

Minimum-Vertex-Cover:

Given G=(V,E), partition V into 2 parts,each of size at least n/3,

minimize # of edges crossing partition.

Given G=(V,E), choose the smallestsubset S ⊆ V such that each edge touches S.

Both are NP-hard

n-vbl 3SATformula F

O(n)-vtxgraph G

poly(n) time

, β

F satisfiable

F unsatisfiable

⇒

⇒

Min-BS(G) = β

Min-BS(G) > β

Distinguishing requires* at least

2Ω(n) time.⇒

Distinguishing requires* at least

2Ω(n) time.

Approximate Optimization

“C-approximation algorithm”

Guaranteed to find a solution with value at most C times the minimum.

“C-certification algorithm”

• Output form: “I certify the minimum is ≥ α”.• Must always be correct.• Guaranteed that α ≥ (true minimum) / C.

Stronger

Minimum Balanced-Separator

Is there a 1.01-approximationalgorithm running in O(n) time?

Is there a 10000-certification

algorithm running in 2n.99 time?

DON’TKNOW

DON’TKNOW

[AMS’11]: Cannot* 1.0000000000000001-certify in poly(n) time.

Minimum Vertex-Cover

Can 2-approximate in linear time.

Cannot* 1.17-certify even in 2n.99999 time.

Cannot* 1.36-certify even in 2n.000001 time.

Can you 1.5-certify in polynomial time?

DON’TKNOW

How could you show that you can’t 1.5-certify Min-VC in poly time?

n-vbl 3SATformula F

O(n)-vtxgraph G

poly(n) time

, β

F satisfiable

F unsatisfiable

⇒⇒

Min-VC(G) ≤ β

Min-VC(G) > 1.5 β

This would show 1.5-certifying Min-VCrequires* superpolynomial time.

DON’TKNOWHOW

How could you show that you can’t 1.5-certify Min-VC in poly time?

give evidence that

Show that known powerful poly-timeoptimization techniques fail to do it.

Prehistory: Linear programming can’t 1.999999-certify Min-VC.

[ABL’02]: Lovász-Schrijverd Super-LP

can’t 1.999999-certify Min-VC.

[GK’95]: Semidefinite programming can’t 1.999999-certify Min-VC.

[GMPT’07]: Lovász-Schrijverd Super-SDP

can’t 1.999999-certify Min-VC.

[BCGM’10]: Sherali-Adamsd Super-Duper-SDP

can’t 1.999999-certify Min-VC.

+++

nO(d) time

[KS’09], [RS’09]: Sherali-Adamsd Super-Duper-SDP

can’t 10000-certify Min-Bal-Sep.

For Min-Balanced-Separator, a similar situation:

Prehistory: Linear programming can’t 1.999999-certify Min-VC.

I.e., there are graphs G on n vertices such that:

• Min-VC(G) ≥ .999999 n• LP(G) = “I certify Min-VC(G) ≥ .500001 n”

α = minimize: ∑v∈V Xv

subject to: Xv ∈ {0,1} for all v∈V

Xu + Xv ≥ 1 for all (u,v)∈E

[0,1]

LP certif. alg. for Min-VC outputs α, where

I.e., there are graphs G on n vertices such that:

• Min-VC(G) ≥ .999999 n

• SAd(G) = “I certify Min-VC(G) ≥ .500001 n”

[BCGM’10]: Sherali-Adamsd Super-Duper-SDP

can’t 1.999999-certify Min-VC.

Specifically, this is true for “Frankl-Rödl graphs” [FR’87]:

V = {0,1}m, E = {(x,y) : ∆(x,y)=.999 m}

I.e., there are graphs G on n vertices such that:

• Min-BS(G) ≥ β

• SAk(G) = “I certify Min-BS(G) ≥ ”.

Specifically, this is true for “Khot-Vishnoi graphs” [KV’05].

[KS’09], [RS’09]: Sherali-Adamsd Super-Duper-SDP

can’t 10000-certify Min-Bal-Sep.

These are tough instances.

We, the mathematicians, can analyze their opt.

But our strongest poly-time algorithms cannot.

Actually…

There is one more algorithm…

It’s even stronger, but hard to analyze…

The “Lasserred Super-Duper-Ultra-SDP”…

Also known as…

SOSd

nO(d) time

Our Results

[OZ’13]: SOS4 is a C-certification algorithm

(for some small C, maybe 5)for Min-BS on Khot-Vishnoi graphs.

SOSd is also pretty good for Max-Cut

on Khot-Vishnoi graphs.

[KOTZ’13]: SOSd is essentially a 1-certif.

alg. for Min-VC on all but the‘hardest’ Frankl-Rödl graphs.

So your whole result is that

one particular algorithm

does well on one particular

instance?

An Old Joke

Q: Why did the complexity theorist work on algorithms?

A: To get lower bounds on his lower bounds.

SOSd is a dozen years old, but hard to analyze.

The Dream: it’s great certification alg. not justfor these known hard graphs, but for all graphs.

Our Inspiration:

STOC’12 paper of Barak, Brandão, Harrow, Kelner, Steurer, and Zhou.

• Showed SOS4 is good certification alg.

on known hard instances of “Unique-Games”.

• Somewhat demystified analysis of SOSd.

So what is SOSd?

“Min-Balanced-Separator(G) > α”

⇔

has no real solutions”

“

infeasibility certificate:

identity −1 = Q0 + Q1P1 + Q2P2+ ••• +QmPm

where each Qi is a “sum of squares”:

Qi = Ri12 + ••• + Rik

2

Positivstellensatz

Subject to some mild technical conditions,every infeasible system has such a certificate.

Caveat: Qi’s might need to have high degree.

SOSd algorithm: [Shor’87,Lasserre’00,Parrilo’00]

If there exists an infeasibility certificate

where all the Qi’s have degree ≤ d,

finds it in time nO(d).

E.g.: SOSd for Min-VC(G)

“Min-VC(G) > α” ⇔

Xv2 = Xv for all v∈V,

Xu+Xv ≥ 1 for all (u,v)∈E,

∑v Xv ≤ α

infeasible

−1 = Q0 + Q1 (α−∑ Xv) + ∑ Quv (Xu+Xv−1) + •••

existence of sum-of-squares Q’s such that

Find largest α such that degree-d Q’s exist.

⇐

Our Results

[OZ’13]: SOS4 is a C-certification algorithm

(for some small C, maybe 5)for Min-BS on Khot-Vishnoi graphs.

I.e., for Khot-Vishnoi graphs G, there are degree-4 SOS Q’s certifying

“Min-Bal-Sep(G) > α” for some α > (true Min-Bal-Sep) / C.

One Slide How-ToThm: Min-VC in this graph is ≥ .999nProof: … vertex isoperimetry…

… inductive argument…

Thm: Min-BS in this graph is ≥ blahProof: … hypercontractivity…

“Check out these polynomials.”

“Check out these polynomials.”

Tiny Taste

A bit of the analysis for Max-Cut:

Lemma: Let a,b,c ∈ {−1,1}. If a ≠ c then either a ≠ b or b ≠ c.

Formalization with polynomials:

SOS Proof:

Open Problems

Can you give an SOS proof of…

• Vertex Isoperimetric Theorem in {0,1}n:

If A, B ⊆ {0,1}n, |A|,|B| ≥ .1·2n,

then ∃x∈A,y∈B with ∆(x,y) ≤

• Central Limit Theorem

Thanks!

+