Sarah R. Allen Ryan O’Donnell David Witmer Carnegie Mellon University

How to refute a random CSP

Sarah R. Allen Ryan O’Donnell David WitmerCarnegie Mellon University

Average-case complexity of CSPs.

E.g., random 3SAT.

How to refute a random CSP

Random 3SAT

n variables, m = m(n) constraints, each an OR of 3 uniformly random literals

m

4.267n

satisfiable (w.v.h.p.) unsatisfiable (w.v.h.p.)

Algorithmic task:

find a satisfyingassignment

Algorithmic task:

find a refutation(proof of unsatisfiability)

What is this “refutation” task?

1. Algorithm should (w.h.p.) output a proof, encoded in some formal language like ZFC, that the CSP instance is unsatisfiable.

2. Algorithm should output “unsatisfiable” or “no comment”. It should never be wrong, and it should output “unsatisfiable” w.h.p.

Potential example: Algorithm solves an LP/SDP relaxation,

outputs “unsatisfiable” iff its value < m.

Random 3SAT

n variables, m = m(n) constraints, each an OR of 3 uniformly random literals

m

4.267n

find a sat. assignment find a refutation

3.52n

provablydoable

[HS03,KKL03]

Random 3SAT

n variables, m = m(n) constraints, each an OR of 3 uniformly random literals

m

4.267n

find a sat. assignment find a refutation

3.52n

provablydoable

[HS03,KKL03]

heuristicallydoable (?)[MPR15]

provablydoable

[FGK01]

???

Perhaps when, say, m = n1.4,

there’s no poly-time refutation algorithm.

Cool! An efficient way to randomly generatesimple, hard-to-solve algorithmic tasks!

Potential application 1: Cryptography. [Gol00, ABW09, Applebaum…]

Potential application 2: Hardness of learning.

[Daniely–Linial–Shalev-Shwartz13+]

For applications, investigating random CSP(P)for different predicates P matters.

E.g., for certain kinds of k-ary predicates P,

if refuting random CSP(P) is hard when m = nC…

App. 1: Gives evidence for PRGs in NC0

with stretch n ↦ nC

App. 2: Shows hardness of PAC-learning concept classes “related to” P,

assuming C → ∞ as k → ∞

Refuting random CSPs is

heavily studied for kSAT,

less studied for other predicates.

Our main theorem:

A poly-time algorithm that, for any k-arypredicate P, refutes (whp) uniformly random

CSP(P) instances, provided m ≫ n cmplx(P) / 2.

Our main theorem:

A poly-time algorithm that, for any k-arypredicate P, refutes (whp) uniformly random

CSP(P) instances, provided m ≫ n cmplx(P) / 2.

The “complexity” of P, an integer we’ll define shortly.It’s always at most k, and “often” small, like 2, 3, 4.

Our main theorem:

A poly-time algorithm that, for any k-arypredicate P, refutes (whp) uniformly random

CSP(P) instances, provided m ≫ n cmplx(P) / 2.

There’s some evidence that this is the “right” answer.We’ll say more at the end of the talk.

Our main theorem:

A poly-time algorithm that, for any k-arypredicate P, refutes (whp) uniformly random

CSP(P) instances, provided m ≫ n cmplx(P) / 2.

Ignoring polylog(n) factors.

Our main theorem:

A poly-time algorithm that, for any k-arypredicate P, refutes (whp) uniformly random

CSP(P) instances, provided m ≫ n cmplx(P) / 2.

Recovers the known n1.5 result for 3SAT;

cmplx(OR3) = 3.

Our main theorem:

A poly-time algorithm that, for any k-arypredicate P, refutes (whp) uniformly random

CSP(P) instances, provided m ≫ n cmplx(P) / 2.

New result for kSAT, when k ≥ 5 is odd:

we refute provided m ≫ nk/2;

previous best was m ≫ n⌈k/2⌉.

Our main theorem:

A poly-time algorithm that, for any k-arypredicate P, refutes (whp) uniformly random

CSP(P) instances, provided m ≫ n cmplx(P) / 2.

Our results also hold for non-Boolean predicates.

Our main theorem:

A poly-time algorithm that, for any k-arypredicate P, refutes (whp) uniformly random

CSP(P) instances, provided m ≫ n cmplx(P) / 2.

Running time is nO(k).

The algorithm can be“k rounds of the SOS SDP hierarchy”.

Our main theorem:

A poly-time algorithm that, for any k-arypredicate P, refutes (whp) uniformly random

CSP(P) instances, provided m ≫ n cmplx(P) / 2.

We even provide a δ-refutation,i.e., a proof that OPT ≤ (1−δ)m,for some constant δ = δ(P) > 0.

Our main theorem:

A poly-time algorithm that, for any k-arypredicate P, refutes (whp) uniformly random

CSP(P) instances, provided m ≫ n cmplx(P) / 2.

If you take m ≫ nk/2, we provide “strong refutation”:

i.e., proof that OPT ≤ E[P]·m + o(m).E.g., that OPT ≤ ⅞·m + o(m) for 3SAT, OPT ≤ ½·m + o(m) for kXOR.

cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution.

3SAT example: P = OR3 =

001010100011101110111

Does it support a 2-wiseuniform distribution? Yes.

cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution.

3SAT example: P = OR3 =

001010100011101110111

Does it support a 2-wiseuniform distribution? Yes. Uniform distrib.

on these 4 strings.

cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution.

3SAT example: P = OR3 =

001010100011101110111

Does it support a 2-wiseuniform distribution? Yes.

Does it support a 3-wiseuniform distribution? No.

∴ cmplx(OR3) = 3.

We can refute oncem ≫ n1.5

cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution.

2-out-of-4-SAT: P =

001101010110100110101100

Does it support a 2-wise uniform distribution? No.

(If it did, by symmetrization that distribution could be the uniform distribution over weight-2, length-4 strings,

but that’s not 2-wise uniform.)

cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution.

2-out-of-4-SAT: P =

001101010110100110101100

Does it support a 2-wise uniform distribution? No.

∴ cmplx(P) = 2.We can refute random instances once m ≫ n.

cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution.

Does it support a 2-wise uniform distribution? No.

∴ cmplx(P) = 2.We can refute random instances once m ≫ n.

The exact same thing holds for any j-out-of-k-SAT.

(Refutation fact was already known [BB02].)

cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution.

[DLS14]: A very strong conjecture implying that for many k-ary predicates P, random

CSP(P) instances with nωk→∞(1) constraints

are not refutable in polynomial time.

Instantiated with 3 families (Pk),

derived 3 hardness-of-learning results.

Unfortunately, we show all these

predicates have cmplx(Pk) ≤ 4.

cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution.

[DLS14]: A very strong conjecture implying that for many k-ary predicates P, random

CSP(P) instances with nωk→∞(1) constraints

are not refutable in polynomial time.

Instantiated with 3 families (Pk),

derived 3 hardness-of-learning results.

Fortunately, [DS14,Dan15] mostly recovered the results based on more plausible assumptions.

A condition similar to m ≫ n cmplx(P) / 2 also arises

in work on finding satisfying assignmentsin random CSPs with a planted solution [FPV14].

Warning / Remark

Nevertheless, I assure you:solving planted CSPs & refuting random CSPs

are quite distinct algorithmic tasks.

A poly-time algorithm that, for any k-arypredicate P, refutes (whp) uniformly random

CSP(P) instances, provided m ≫ n cmplx(P) / 2.

Plan for the rest of the talk

Part 1: Strong refutation for kXOR

Part 2: Strong refutation for any k-CSP, provided m ≫ nk/2

Part 3: Proof of the main theorem

Part 1: Strong refutation for kXOR

Given a random kXOR CSP with m ≫ nk/2,

want to certify “OPT ≤ ½·m +o(m)” (whp).

• [CCF10] strongly refute any CSP with m ≫ n⌈k/2⌉.

• For kXOR specifically, m ≫ n⌈k/2⌉ follows pretty

easily from approximation algs literature.

• For 3SAT, strong refutation provided m ≫ n1.5

achieved by [CGL07]. Messy, but if you stare at it long enough, it’s clear it generalizes to 3XOR.

• This, as well as the generalization to kXOR, done independently by [BM15].

Given a random kXOR CSP with m ≫ nk/2,

want to certify “OPT ≤ ½·m +o(m)” (whp).

• The weaker bound m ≫ n⌈k/2⌉ follows from the

k = 2 case, and a very trivial trick.

• The m ≫ nk/2 bound requires adding in a very

clever trick (from [FGK01,CGL07]).

• Sadly, only time for a sketch of the k = 2 case.

Given a random 2XOR CSP with m ≫ n,want to certify “OPT ≤ ½·m +o(m)” (whp).

n variables, x1, …, xn ∈ {0,1}

m constraints, each like

Switch to ±1 notation.

Given a random 2XOR CSP with m ≫ n,want to certify “OPT ≤ ½·m +o(m)” (whp).

n variables, x1, …, xn ∈ {−1,+1}

m constraints, each like

Let OPT denote (# sat − # unsat) instead;i.e., sum of right-hand sides achieved.

Given a random 2XOR CSP with m ≫ n,want to certify “OPT ≤ o(m)” (whp).

n variables, x1, …, xn ∈ {−1,+1}

m constraints, each like

Let OPT denote (# sat − # unsat) instead;i.e., sum of right-hand sides achieved.

Given a random 2XOR CSP with m ≫ n,want to certify “|OPT| ≤ o(m)” (whp).

n variables, x1, …, xn ∈ {−1,+1}

m constraints, each like

Let OPT denote (# sat − # unsat) instead;i.e., sum of right-hand sides achieved.

Given a random 2XOR CSP with m ≫ n,want to certify “|OPT| ≤ o(m)” (whp).

n variables, x1, …, xn ∈ {−1,+1}

m constraints, each like

Let and instead include each

possible constraint independently with prob. p.

Given a random 2XOR CSP with p ≫ 1/n,

want to certify “|OPT| ≤ o(pn2)” (whp).

n variables, x1, …, xn ∈ {−1,+1}

m constraints, each like ~pn2

Let A be the random n×n symmetric matrix with

entries {−1,0,+1} depending on the xi,xj constr.

Given assignment x ∈ {−1,+1}n, it has

(# sat − # unsat) = xTAx ≤ n||A||

Given a random 2XOR CSP with p ≫ 1/n,

want to certify “|OPT| ≤ o(pn2)” (whp).

Let A be the random n×n symmetric matrix with

entries {−1,0,+1} depending on the xi,xj constr.

Given assignment x ∈ {−1,+1}n, it has

|# sat − # unsat| = |xTAx| ≤ n||A||

Done if ||A|| ≤ o(pn) whp. (Can efficiently certify.)

A good old-fashioned random matrix fact.||A|| ≤ whp, by the Trace Method [FK81].

Part 1: Strong refutation for kXOR

Part 2: Strong refutation for any k-CSP, provided m ≫ nk/2

✔

m

k

Def: Given an assignment x∈{0,1}n,

m

k

Def: Given an assignment x∈{0,1}n,

the table distribution Dx is the probability

distribution on {0,1}k given by choosing a

uniformly random row from the above table.

m

k

Def: Given an assignment x∈{0,1}n,

the table distribution Dx is the probability

distribution on {0,1}k given by choosing a

uniformly random row from the above table.

Note: Has nothing to do with the predicate P.

Def: Given an assignment x∈{0,1}n,

the table distribution Dx is the probability

distribution on {0,1}k given by choosing a

uniformly random row from the above table.

Def: An instance I is quasirandom if

for all assignments x∈{0,1}n,

the table distribution Dx is o(1)-close

to the uniform distribution on {0,1}k.

Fact: If a CSP(P) instance I is quasirandom, thenOPT(I) ≤ E[P]·m + o(m).

Given a random k-ary CSP with m ≫ nk/2,

want to certify it’s quasirandom (whp).

By “Vazirani XOR Lemma”, a table distribution

Dx is o(1)-close to uniform if and only if

Dx has o(1)-correlation with XOR of S coordinates

for all ∅ ≠ S ⊆ [k].

∴ can certify this using the |S|-XOR refutation

algorithm from Part 1, for all 2k−1 subsets S.

Part 1: Strong refutation for kXOR

Part 2: Strong refutation for any k-CSP, provided m ≫ nk/2

✔

✔

Part 3: Main theorem, refuting random CSP(P) provided m ≫ n

cmplx(P) / 2

cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution.

Suppose P does not supporta t-wise uniform distribution.

Suppose P does not supporta t-wise uniform distribution.

⇒ every t-wise uniform distribution is δ-far from being supported on P (for some constant δ = δ(P) > 0, by basic LP theory).

Given CSP(P) instance, instead of certifying quasirandomness by strongly refuting S-XOR for all S ⊆ [k]…

Suppose P does not supporta t-wise uniform distribution.

⇒ every t-wise uniform distribution is δ-far from being supported on P (for some constant δ = δ(P) > 0, by basic LP theory).

Given CSP(P) instance, strongly refute S-XOR for all |S| ≤ t.

This is doable (whp) provided m ≫ nt/2.

This certifies that all table distributions Dx

are o(1)-close to t-wise uniform, and hence (δ−o(1))-far from being supported on P.

Suppose P does not supporta t-wise uniform distribution.

⇒ every t-wise uniform distribution is δ-far from being supported on P (for some constant δ = δ(P) > 0, by basic LP theory).

Given CSP(P) instance, strongly refute S-XOR for all |S| ≤ t.

This is doable (whp) provided m ≫ nt/2.

This certifies that all table distributions Dx

are o(1)-close to uniform, and hence (δ−o(1))-far from being supported on P.

I.e., it certifies that OPT ≤ (1−δ+o(1))m.

Part 1: Strong refutation for kXOR

Part 2: Strong refutation for any k-CSP, provided m ≫ nk/2

✔

✔

Part 3: Main theorem, refuting random CSP(P) provided m ≫ n

cmplx(P) / 2 ✔

Open directions

Give evidence that n cmplx(P) / 2 is optimal.

I.e., if P supports a (t−1)-wise uniform distribution,try to show various efficient refutation systems

fail on random instances with m ≪ nt/2 constraints.

[BW99]: Resolution fails on kXOR and kSAT.[Sch08]: SOS fails on kXOR and kSAT.[BGMT12,TW13,OW14,MWW15]: Sherali–Adams+ fails.[FPV15]: Certain “statistical algorithms” fail.[BCK15]: For t=3 (pairwise unif.), “pruned” random

instances with m = O(n), SOS fails.

Thanks!