On the Unique Games Conjecture Subhash Khot Georgia Inst. Of Technology. At FOCS 2005

On the Unique Games Conjecture

Subhash Khot Georgia Inst. Of Technology.

At FOCS 2005

NP-hard Problems

• Vertex Cover• MAX-3SAT• Bin-Packing • Set Cover • Clique • MAX-CUT • ……………..• ……………..

Approximability : Algorithms

A C-approximation algorithm computes (C > 1), for problem instance I , solution A(I) s.t.

Minimization problems :

A(I) C OPT(I)

Maximization problems :

A(I) OPT(I) / C

Some Known Approximation Algorithms

• Vertex Cover 2 - approx.

• MAX-3SAT 8/7 - approx. Random assignment. • Packing/Scheduling (1+) – approx. > 0

(PTAS)

• Set Cover ln n approx.

• Clique n/log n [Boppana Halldorsson’92] • Many more , ref. [Vazirani’01]

PCP Theorem

[B’85, GMR’89, BFL’91, LFKN’92, S’92,……] [PY’91] [FGLSS’91, AS’92 ALMSS’92]

Theorem : It is NP-hard to tell whether a MAX-3SAT instance is * satisfiable (i.e. OPT = 1) or * no assignment satisfies more than 99%

clauses (i.e. OPT 0.99).

i.e. MAX-3SAT is 1/0.99 = 1.01 hard to approximate.

i.e. MAX-3SAT and MAX-SNP-complete problems [PY’91] have no PTAS.

Approximability : Towards Tight Hardness Results

• [Hastad’96] Clique n1-

• [Hastad’97] MAX-3SAT 8/7 -

• [Feige’98] Set Cover (1- ) ln n

[Dinur’05] Combinatorial Proof of PCP Theorem !

Open Problems in Approximability

– Vertex Cover (1.36 vs. 2) [DinurSafra’02]

– Coloring 3-colorable graphs (5 vs. n3/14) [KhannaLinialSafra’93, BlumKarger’97]– Sparsest Cut (1 vs. (logn)1/2) [AroraRaoVazirani’04]– Max Cut (17/16 vs 1/0.878… )

[Håstad’97, GoemansWilliamson’94] ………………………..

Unique Games Conjecture [Khot’02]

Implies these hardness results : • Vertex Cover 2- [KR’03]

• Coloring 3-colorable (1) [DMR’05]

graphs (variant of UGC)

• MAX-CUT 1/0.878.. - [KKMO’04]

• Sparsest Cut, Multi-cut [KV’05,

(1) CKKRS’04]

Min-2SAT-Deletion [K’02, CKKRS’04]

Unique Games Conjecture

Led to …

[MOO’05] Majority Is Stablest Theorem

[KV’05] “Negative type” metrics do not embed into L1 with O(1) “distortion”.

Optimal “integrality gap” for MAX-CUT

SDP with “Triangle Inequality”.

Integrality Gap : Definition Given : Maximization Problem + Specific SDP relaxation.

• For every problem instance G,

SDP(G) OPT(G)

• Integrality Gap = Max G SDP(G) / OPT(G)

• Constructing gap instance = negative result.

Overview of the talk

• The UGC • Hardness of Approximation Results• I hope UGC is true • Attempts to Disprove : Algorithms

Connections/applications : • Fourier Analysis • Integrality Gaps• Metric Embeddings


• A maximization problem called “Unique Game” is hard to approximate.

• “Gap-preserving” reductions from Unique Game Hardness results for Vertex Cover,

MAX-CUT, Graph-Coloring, …..

Example of Unique GameOPT = max fraction of equations that can be satisfied by any assignment. x1 + x3 = 2 (mod k)

3 x5 - x2 = -1 (mod k)

x2 + 5 x1 = 0 (mod k)

UGC For large k, it is NP-hard to tell whether OPT 99% or OPT 1%

2-Prover-1-Round Game (Constraint Satisfaction

Problem )

variables

constraints


Problem )

variables

k labelsHere k=4

constraints


Problem )

variables

k labelsHere k=4

Constraints = Bipartite graphsor Relations [k] [k]


Problem )

variables

k labelsHere k=4

OPT(G) = 7/7

Find a labeling that satisfies max # constraints

Hardness of Finding OPT(G)

• Given a 2P1R game G, how hard is it to find OPT(G) ? • PCP Theorem + Raz’s Parallel Repetition Theorem

:

For every , there is integer k(), s.t. it is NP-hard to tell whether a 2P1R game with k = k() labels has OPT = 1 or OPT

In fact k = 1/poly()

Reductions from 2P1R Game

• Almost all known hardness results (e.g. Clique, MAX-3SAT, Set Cover, SVP, …. ) are reductions from 2P1R games.

• Many special cases of 2P1R games are known to be hard, e.g. Multipartite graphs,

Expander graphs, Smoothness property, ….

What about unique games ?

Unique Game = 2P1R Game with

Permutationsvariable

k labelsHere k=4

Unique Game = 2P1R Game with Permutations

variable

k labelsHere k=4

Permutations or matchings : [k] [k]

OPT(G) = 6/7

Find a labeling that satisfies max # constraints

Unique Game = 2P1R Game with Permutations

Unique Games Considered before …… [Feige Lovasz’92] Parallel Repetition of UG reduces OPT(G).

How hard is approximating OPT(G) for a unique game G ?

Observation : Easy to decide whether OPT(G) = 1.

MAX-CUT is Special Case of Unique Game

• Vertices : Binary variables x, y, z, w, …….

• Edges : Equations x + y = 1 (mod 2)

• [Hastad’97] NP-hard to tell whether OPT(MAX-CUT) 17/21 or OPT(MAX-CUT) 16/21


For any , , there is integer k(, ), s.t. it is NP-hard to tell whether a UniqueGame with k = k(, ) labels has OPT 1- or OPT

i.e. Gap-Unique Game (1- , ) is NP-hard.




Case Study : MAX-CUT

• Given a graph, find a cut that maximizes fraction of edges cut.

• Random cut : 2-approximation.

• [GW’94] SDP-relaxation and rounding. min 0 < < 1 / (arccos (1-2) / ) = 1/0.878 … approximation.

• [KKMO’04] Assuming UGC, MAX-CUT is 1/0.878… - hard to approximate.

Reduction to MAX-CUT Unique Game Graph H

• Completeness : OPT(UG) > 1-o(1) - o(1) cut.

• Soundness : OPT(UG) < o(1) No cut with size arccos (1-2) / + o(1)

• Hardness factor = / (arccos (1-2) / ) - o(1)

• Choose best to get 1/0.878 … (= [GW’94])

Reduction from Unique Game

Gadget constructed via Fourier theorem + Connecting gadgets via Unique Game instance

[DMR’05] “UGC reduces the analysis of the entire construction to the analysis of the gadget”.

Gadget = Basic gadget ---> Bipartite gadget ---> Bipartite gadget with permutation

Basic Gadget

A graph on {0,1} k with specific properties

(e.g. cuts, vertex covers, colorability)

{0,1} k

k = # labelsx = 011

Y = 110

Basic Gadget : MAX-CUT Weighted graph, total edge weight = 1. Picking random edge : x R {0,1} k

y <-- flip every co-ordinate of x with

probability ( 0.8)

x

{0,1} k

y

MAX-CUT Gadget : Co-ordinate Cut Along Dimension i

Fraction of edges cut = Pr(x,y) [xi yi ]

=

Observation : These are the maximum cuts.

xi = 0 xi = 1

Bipartite Gadget

A graph on {0,1} k {0,1} k (double cover of basic

gadget)x = 011

y’ = 110

Cuts in Bipartite Gadget {0,1} k {0,1} k

Matching co-ordinate cuts have size =

Bipartite Gadget with Permutation : [k] -> [k] Co-ordinates in second hypercube permuted via

.

x = 011

Y ’ = 110

(y’) = 011

Example : = reversal of co-ordinates.

Reduction from Unique Game

Variables

k labels

OPT 1 – o(1)or OPT o(1)

Permutations : [k] [k]

Instance H of MAX-CUT

{0,1} k

Vertices

Edges

Bipartite Gadgetvia

Proving Completeness

Unique Game Graph H

(Completeness) : OPT(UG) > 1-o(1) H has - o(1) cut.

Completeness : OPT(UG) 1-o(1)

label = 2

label = 1

label = 3

label = 1label = 1

label = 3

label = 2Labels = [1,2,3]

Completeness : OPT(UG) 1-o(1)

{0,1} k

Vertices

Edges

Hypercubes are cut along dimensions = labels.

MAX-CUT - o(1)

Proving Soundness

Unique Game Graph H

(Soundness) : OPT(UG) < o(1) H has no cut of size arccos (1-2) / + o(1)

MAX-CUT Gadget

Cuts = Boolean functions f : {0,1} k {0,1}

Compare boolean functions * that depend only on single co-ordinate

vs * where every co-ordinate has negligible “influence” (i.e. “non-junta” functions)

{0,1} k

x

y

f(x1 x2 …….. xk) = xi

f(x1 x2 …….. xk) = MAJORITY Influence (i, f) = Prx [ f(x) f(x+ei) ]

Gadget : “Non-junta” Cuts

How large can non-junta cuts be ? i.e. cuts with all influences negligible ? Random Cut : ½ Majority Cut : arccos (1-2) / > ½

• [MOO’05] Majority Is Stablest (Best) Any cut slightly better than Majority Cut must have “influential” co-ordinate.

Non-junta Cuts in Bipartite Gadget

[MOO’05] Any “special” cut with value arccos (1-2) / + must define a matching pair of influential co-ordinates.

{0,1} k {0,1} k

Non-junta Cuts in Bipartite Gadget

{0,1} k {0,1} k

f : {0,1} k --> {0, 1}

g : {0,1} k --> {0, 1}

i Infl (i, f), Infl (i, g) > (1)

cut > arccos (1-2) / +

Instance H of MAX-CUT

{0,1} k

Vertices

Edges

Bipartite Gadgetvia

Proving Soundness

• Assume arccos (1-2) / + cut exists.

• On /2 fraction of constraints, the bipartite gadget has arccos (1-2) / + /2 cut.

matching pair of labels on this constraint.

This is impossible since OPT(UG) = o(1).

Done !

Other Hardness Results• Vertex Cover Friedgut’s Theorem Every boolean function with low “average sensitivity” is a junta.

• Sparsest Cut, Min-2SAT Deletion KahnKalaiLinial Every balanced boolean function has a

co-ordinate with influence log n/n.

Bourgain’s Theorem (inspired by Hastad-Sudan’s 2-bit Long Code test)

Every boolean function with low “noise sensitivity” is a junta.

• Coloring 3-Colorable [MOO’05] inspired. Graphs

Basic Paradigm by [BGS’95,

Hastad’97] Hardness results for Clique, MAX-3SAT, ……. • Instead of Unique Games, use reduction from general 2P1R Games (PCP Theorem + Raz).

• Hypercube = Bits in the Long Code [Bellare

Goldreich Sudan’95]

• PCPs with 3 or more queries (testing Long Code).

• Not enough to construct 2-query PCPs.

Why UGC and not 2P1R Games?

Power in simplicity. “Obvious” way of encoding a

permutation constraint. Basic Gadget ----> Bipartite Gadget with permutation.




I Hope UGC is True• Implies all the “right” hardness results in a unifying way.

• Neat applications of Fourier theorems [Bourgain’02, KKL’88, Friedgut’98, MOO’05]

• Surprising application to theory of metric embeddings and SDP-relaxations [KV’05].

• Mere coincidence ?

Supporting Evidence

[Feige Reichman’04] Gap-Unique Game (C, ) is NP-hard.

i.e. For every constant C, there is s.t. it is NP-hard to tell if a UG has OPT > C or OPT < .

However C --> 0 as --> 0.

Supporting Evidence

[Khot Vishnoi’05]

SDP relaxation for Unique Game

has integrality gap (1- , ).




Disproving UGC means ..

For small enough (constant) , given a UG with optimum 1- , algorithm that finds a labeling

satisfying (say) 50% constraints.

Algorithmic Results

Algorithm that finds a labeling satisfying f(, k, n) fraction of constraints.

[Khot’02] 1- 1/5 k2 [Trevisan’05] 1- 1/3 log1/3 n [Gupta Talwar’05] 1- log n [CMM’05] 1/k , 1- 1/2 log1/2 k

None of these disproves UGC.

Quadratic Integer Program For Unique Game [Feige

Lovasz’92] variable

k labels

: [k] [k]

u1 , u2 , … , uk {0,1}

v1 , v2 , … , vk {0,1}

u

v

vi = 1 if Label(v) = i = 0 otherwise

Quadratic Program for Unique Games

Constraints on edge-set E.

• Maximize ui vπ(i)

(u, v) E i=1,2,..,k

• u i [k], ui {0,1}

• u ui2 = 1

i

• u i ≠ j , ui uj = 0

SDP Relaxation for Unique Games

• Maximize ui, vπ(i)

(u, v) E i=1,2,..,k

• u i [k], ui is a vector.

• u || ui ||2 = 1 i=1,2,..,k

• u i≠j [k], ui, uj = 0

[Feige Lovasz’92]

• OPT(G) SDP(G) 1.

• If OPT(G) < 1, then SDP(G) < 1.

• SDP(Gm) = (SDP(G))m

• Parallel Repetition Theorem for UG : OPT(G) < 1 OPT(Gm) 0

[Khot’02] Rounding Algorithm

u1

uk

u2

vk

v2

v1

r r

Label(u) = 2, Label(v) = 2

Pr [ Label(u) = Label(v) ] > 1 - 1/5 k2

Labeling satisfies 1 - 1/5 k2 fraction of constraints in expected sense.

Random ru v

[CMM’05] Algorithm• Labeling that satisfies 1/k fraction

of constraints. (Optimal [KV’05]) vk

v2

v1

r

u1

uk

u2

r

All i s.t. ui is “close” to r are taken as candidate labels to u.

Pick one of them at random.

[Trevisan’05] Algorithm

• Given a unique game with optimum 1- 1/log n, algorithm finds a labeling that satisfies 50% of constraints.

• Limit on hardness factors achievable via UGC (e.g. loglog n for Sparsest

Cut).


[Leighton Rao’88] Delete a few constraints and

remaining graph has connected

components of low diameter.

Variables and constraints


A good algorithm for graphs with low

diameter.




Already Covered

Let’s move on ….




[KV’05] Integrality Gaps for

SDP-relaxations • MAX-CUT • Sparsest Cut • Unique Game

Gaps hold for SDPs with “Triangle Inequality”.

Integer Program for MAX-CUT

Given G(V,E)

• Maximize ¼ |vi - vj |2

(i, j) E

• i, vi {-1,1}

• Triangle Inequality (Optional) : i, j , k, |vi - vj |2

+ |vj - vk |2 |vi - v k|2

Goemans-Williamson’s SDP Relaxation for MAX-CUT

• Maximize ¼ || vi - vj ||2

(i, j) E

• i, vi Rn, || vi || = 1

• Triangle Inequality (Optional) : i, j , k, || vi - vj ||2

+ || vj - vk ||2 || vi - v k||2

Integrality Gap for MAX-CUT• [Goemans Williamson’94]

Integrality gap 1/0.878..

• [Karloff’99] [Feige Schetchman ’01]

Integrality gap 1/0.878.. -

SDP solution does not satisfy Triangle Inequality.

Does Triangle Inequality make the SDP tighter ? NO if Unique Games Conj. is true !

Integrality Gap for Unique Games SDP

Unique Game G with

OPT(G) = o(1)

SDP(G) = 1-o(1)

OrthonormalBases for Rk

u1 , u2 ,

… , uk

v1 , v2 ,

… , vkvariables

k labels

Matchings [k] [k]u

v

Integrality Gap for MAX-CUT with

Triangle Inequality

{-1,1}k

u1 , u2 ,

… , uk

u1 u2 u3 ……… uk-1

uk

PCP Reduction

OPT(G) = o(1)

No large cut

Good SDP solution




Metrics and Embeddings

• Metric is a distance function on [n] such that

d(i, j) + d(j, k) d(i, k).

• Metric d embeds into metric with distortion 1 if i, j d(i, j) (i, j) d(i, j).

Negative Type Metrics

Given a set of vectors satisfying Triangle Inequality : i, j , k, || vi - vj ||2

+ || vj - vk ||2 || vi - v k||2

d(i, j) = || vi - vj ||2 defines a metric.

These are called “negative type metrics”.

L1 NEG METRICS

NEG vs L1 Question [Goemans, Linial’ 95] Conjecture : NEG metrics embed into

L1

with O(1) distortion.

Sparsest Cut

O(1) Integrality Gap O(1) Approximation

[Linial London Rabinovich’94][Aumann Rabani’98]


[Chawla Krauthgamer Kumar Rabani Sivakumar ’05][KV’05]

(1) hardness result

NEG vs L1 Lower Bound

(loglog n) integrality gap for Sparsest

Cut SDP. [KhotVishnoi’05, KrauthgamerRabani’05]

A negative type metric that needs distortion (loglog n) to embed into L1.

Open Problems

• (Dis)Prove Unique Games Conjecture.

• Prove hardness results bypassing UGC.

• NEG vs L1 , Close the gap.

(log log n) vs (log n loglog n) [Arora Lee Naor’04]

Open Problems

• Prove hardness of Min-Deletion version of Unique Games. (log n approx. [GT’05])

• Integrality gaps with “k-gonal” inequalities.

• Is hypercube (Long Code) necessary ?

Open Problems More hardness results, integrality gaps, embedding lower bounds, Fourier Analysis,

……

[Samorodnitsky Trevisan’05] “Gowers Uniformity, Influence of Variables, and PCPs”. UGC Boolean k-CSP is hard to approximate within 2k- log k

Independent Set on degree D graphs is hard to approximate within D/poly(log D).

Open Problems in Approximability Traveling Salesperson

Steiner Tree Max Acyclic Subgraph, Feedback Arc Set Bin-packing (additive approximation) ……………………

Recent progress on Edge Disjoint Paths Network Congestion Shortest Vector Problem Asymmetric k-center (log* n) Group Steiner Tree (log2 n) Hypergraph Vertex Cover ………………

Linear Unique Games System of linear equations mod k. x1 + x3 = 2

3 x5 - x2 = -1

x2 + 5 x1 = 0

[KKMO’04] UGC UGC in the special case of linear equations mod k.

Variations of Conjecture• 2-to-1 Conjecture [K’02]

-Conjecture [DMR’05]

NP-hard to color 3-colorable graphs with O(1) colors.

[k] [k]

[k] [k]

Documents

On the Unique Games Conjecture Subhash Khot Georgia Inst. Of Technology. At FOCS 2005