Tradeoffs in Backdoors:Inconsistency Detection, DynamicSimplification, and Preprocessing

Bistra Dilkina, Carla Gomes, Ashish SabharwalCornell University

ISAIM 2008Fort Lauderdale, FL

Note: Part of this work was presented at CP-07

ISAIM-08 2

Context

Constraint Satisfaction Problems (CSPs) In particular, Boolean Satisfiability or SAT :

Given a Boolean formula F in conjunctive normal forme.g. F = (a or b) and (a or c or d) and (b or c)determine whether F is satisfiable

NP-complete widely used in practice, e.g. in hardware & software

verification, design automation, AI planning, …

ISAIM-08 3

From annual SAT competitions:

Large industrial benchmarks (10K+ vars) are solved within seconds by state-of-the-art complete SAT solvers

Good scaling behavior seems to defy “NP-completeness” !

How can we explain this gap between theory and practice?

Real-world problems have tractable sub-structure

“Backdoors” explain how solvers can get “smart” and solve very large instances

SAT: Gap between theory & practice

ISAIM-08 4

Backdoors to tractability

Informally [Gomes et al ’03]:

A backdoor is a critical set of variables for a givenproblem such that, once assigned values, theremaining instance simplifies to a “tractable” class

How is tractability captured formallyin the notion of backdoors?

sub-solver capturing poly-time mechanisms(not necessarily syntactically defined)

e.g. Unit Propagation, Arc Consistency, LP.

Note: concept of backdoor applicable to CSPs, MIP, …

ISAIM-08 5

Tractability: sub-solver

Sub-solver S: An algorithm that, given formula F, satisfies four properties:

1. Efficiency: S runs in polynomial time(often linear or quadratic time)

2. Trivial solvability: S can determine whether F is trivially True (has no clauses) or trivially False (has an empty clause, or empty domain)

3. Trichotomy: reject, declare sat, declare unsat4. Self-reducibility: also solves F[v/x]

ISAIM-08 6

Strong Backdoors

A subset B of variables is a strong backdoor fora formula F w.r.t. a sub-solver S if:

for every truth assignment to BS solves the “simplified” formula F[/B]

Note: have provided powerful insights, leading to techniques like randomization, restarts, and algorithm portfolios for SAT

Two key featuresA. Dynamic simplification based on truth values and constraint semantics (e.g. unit propagation)B. Trivial solvability / inconsistency detection

ISAIM-08 7

Our Results

1. Relaxing these two key features leads to exponentially larger backdoors

2. Finding strong backdoors can become harder* than NP just by adding inconsistency detection to tractable class(“static” notions trivially within NP, thus “weaker”)

3. State-of-the-art solvers do find very small strong backdoors (with inconsistency detection)

4. Preprocessing: small & mixed effect on backdoor size

5. Satisfiable instances: backdoors smaller w.r.t. PL than UP

ISAIM-08 8

Our Results

1. Relaxing these two key features leads to exponentially larger backdoors

2. Finding strong backdoors can become harder* than NP just by adding inconsistency detection to tractable class(“static” notions trivially within NP, thus “weaker”)

3. State-of-the-art solvers do find very small strong backdoors (with inconsistency detection)

4. Preprocessing: small & mixed effect on backdoor size

5. Satisfiable instances: backdoors smaller w.r.t. PL than UP

From algorithmic backdoors to syntactically defined tractable classes

ISAIM-08 10

Horn/2CNF backdoors

Horn formula – every clause has at most one positive literal 2CNF formula – every clause is binary

“Strong” Backdoors w.r.t. Horn/2CNF a subset B of variables such that for every value assignment to B, the simplified sub-formula is Horn/2CNF (not including trivial inconsistency)

[Nishimura, Ragde, Szeider ’04] : “Strong” backdoors for Horn/2CNF equivalent to deletion backdoors

ISAIM-08 11

Horn/2CNF backdoors

Horn formula – every clause has at most one positive literal 2CNF formula – every clause is binary

“Strong” Backdoors w.r.t. Horn/2CNF a subset B of variables such that for every truth assignment to B, the simplified sub-formula is Horn/2CNF (not including trivial inconsistency)

[Nishimura, Ragde, Szeider ’04] : “Strong” backdoors for Horn/2CNF equivalent to deletion backdoors

Aside: [Dechter ’92] Cycle cutset is a deletion backdoor w.r.t. the tractable class of acyclic CSPs

Deletion

once deleted from F,

[Chandru, Hooker ’92]

ISAIM-08 12

Deletion backdoors for Horn/2CNF

Deciding whether a formula has Horn/2CNF deletion backdoor of size k is fixed-parameter tractable (FPT) O(f(k)nc) – possibly exponential in k,

but polynomial in n independent of k

Why do we care about deletion backdoors?In general

Often easier to reason about and characterize (“syntactic”) Deletion backdoors for a class are also “strong” backdoors

BUT, are deletion backdoors always as small as “strong” backdoors?

(for Horn/2CNF: yes; what about richer classes?)

[Nishimura, Ragde, Szeider ’04]

ISAIM-08 13

Not The Case in General!

Renamable Horn (RHorn) formulas The formula is Horn up to renaming of the variables Rename(v) : flip the sign of all literal occurrences of v

Theorem. “Strong” backdoors w.r.t. RHorn can be exponentially smaller than deletion backdoors w.r.t. RHorn

Proof idea: given a strong backdoor set B, for each assignment to B, the simplified formula F[/B] is RHorn However, different require different and mutually incompatible

renamings to convert to Horn

ISAIM-08 14

Deletion backdoors ignore the interplay between semantics of the constraints and value assignments Thus, they capture only static tractable sub-structure

Dynamic structure analysis also much more powerful in other contexts, e.g. dynamic caching for constraint reasoning beyond SAT

[Bacchus CP-07 invited talk]

Lesson?

Ignoring truth value assignmentsmay not always be a good idea

ISAIM-08 15

How about inconsistency detection?

What if the simplified formula is not Horn/2CNF butcontains an empty clause?

Clearly unsatisfiable due to x0

But smallest Horn backdoor has size n !!

Missing trivial solvability – empty clause detection

)...)()(( 21000 nxxxxxx

ISAIM-08 16

Empty clause detection

C{} : class of formulas containing an empty clause Inconsistency detection is a key ingredient of backtrack

solvers (distinguishes DPLL from naïve search) size of C{} backdoors correlated with search effort by

zChaff [Lynce et al ’04]

Proposition: Strong backdoors w.r.t. C{} can bearbitrarily smaller than strong backdoors w.r.t.pure Horn/2CNF

[from the example on previous slide]

ISAIM-08 17

Our Results

1. Relaxing these two key features leads to exponentially larger backdoors

2. Finding strong backdoors can become harder* than NP just by adding inconsistency detection to tractable class(“static” notions trivially within NP, thus “weaker”)

3. State-of-the-art solvers do find very small strong backdoors (with inconsistency detection)

4. Preprocessing: small & mixed effect on backdoor size

5. Satisfiable instances: backdoors smaller w.r.t. PL than UP

ISAIM-08 18

Backdoors w.r.t. Horn/2CNF C{}

Horn/2CNF C{} sub-solver Deletion backdoors Strong backdoors

Deciding the existence of strong backdoor of size k w.r.t. pure Horn/2CNF is NP-complete [Nishimura et al ’04]

Intuitively speaking, not surprising since harder-to-find objects often capture richer structure more succinctly

Theorem. Deciding the existence of a strong backdoor of size k w.r.t. Horn/2CNF C{} is both NP-hard and coNP-hard.

ISAIM-08 19

Our Results

1. Relaxing these two key features leads to exponentially larger backdoors

2. Finding strong backdoors can become harder* than NP just by adding inconsistency detection to tractable class(“static” notions trivially within NP, thus “weaker”)

3. State-of-the-art solvers do find very small strong backdoors (with inconsistency detection)

4. Preprocessing: small & mixed effect on backdoor size

5. Satisfiable instances: backdoors smaller w.r.t. PL than UP

ISAIM-08 20

What matters in practice

Backdoors characterize hidden structure in interesting combinatorial problems

It is a key notion to understand the behavior of state-of-the-art solvers and their sophisticated propagation mechanisms

Backdoors provide powerful insights motivating new strategies for algorithm design (e.g., randomization, restarts, algorithm portfolios)

Do problems exhibit small backdoors? Do SAT solvers exploit them?

An Experimental Study of Backdoors

Finer grained insights into backdoors

ISAIM-08 22

Experimental study

Syntactic classes: Horn, Rhorn

Algorithmic sub-solvers:UP (unit propagation), PL (pure literal elimination), UP+PL,UP+PL+probing (“branch-free Satz”, failed lits)

Preprocessors:2-Simplify, HyPre, SatELite, 3-Resolution

Various problem domains:Graph coloring, logistics planning,car configuration, game theory (finding pure Nash eq.)

Kilby et al. ’05: Satz-backdoor sizecorrelated with problem hardness

ISAIM-08 23

Finding minimum deletion backdoors

Decision version within NP (unlike strong backdoors) For Horn (same as “strong” backdoors) and RHorn

Use Integer Programming:

E.g for Horn, one {0,1} variable yi for each variable i in F

Minimize size:

Every clause should be Horn:

ci i Fcy ,1

)var()1(min

Fi iy

ISAIM-08 24

Finding small strong backdoors

Not within NP rely on indirect methods for finding a strong backdoor Gives upper bound on the minimum strong backdoor size

Randomized version of Satz [Li and Anbulagan ’97] a complete backtrack search solver that employs UP, PL, probing can be easily configured to use selective propagation (e.g. only PL)

Using Satz to find (several) strong backdoors Run Satz-Rand multiple times without restarts Record the set B of vars not fixed by propagation mechanism S Can verify: B is a strong backdoor w.r.t. S [for unsat instances]

ISAIM-08 25

Results: graph coloring

3 binary variables per graph node Planted clique = backdoor of size 4x3=12 w.r.t. C{}

12/n

UP+PL+probing

optimal upper bounds

|B|/n

ISAIM-08 26

Results: graph coloring

3 binary variables per graph node Planted clique = backdoor of size 4x3=12 w.r.t. C{}

Propagation sub-solvers find even smaller backdoors Horn and RHorn backdoors ignore constraint semantics

backdoor size linear in number of variables

optimal upper bounds

ISAIM-08 27

Results: logistics planning

SATZ backdoors of size 0 !! UP+PL+probing solves these problems without branching

UP and UP+PL also excellent at finding the core inconsistencies

Horn and RHorn backdoors quite large 48%

ISAIM-08 28

Results: car configuration

RHorn deletion backdoors are also very small The original formulas are almost Horn

Again SATZ is excellent at finding small backdoor sets

ISAIM-08 29

Results: game theory

Games on random graphs G(n,p) with avg. degree 3

SATZ backdoors again almost all of size 0 UP backdoors of size 6 to 8 critical players

Horn and RHorn backdoors quite large 66%

ISAIM-08 30

Our Results

1. Relaxing these two key features leads to exponentially larger backdoors

2. Finding strong backdoors can become harder* than NP just by adding inconsistency detection to tractable class(“static” notions trivially within NP, thus “weaker”)

3. State-of-the-art solvers do find very small strong backdoors (with inconsistency detection)

4. Preprocessing: small & mixed effect on backdoor size

5. Satisfiable instances: backdoors smaller w.r.t. PL than UP

ISAIM-08 31

Effect of Preprocessors

How does preprocessing affect backdoor size?

[Refer to paper for details] Considered 2-Simplify, HyPre, SatELite, 3-Resolution Compared to backdoor size without preprocessing

Key observations Backdoor size in the same order of magnitude

(e.g. 156-219 vars for UP+PL on an instance) Mixed effect overall: often decreases but can increase SatELite usually gave the smallest backdoors

Preprocessing often simplifies formula but sometimesobfuscates hidden structure

ISAIM-08 32

Backdoors for Satisfiable Instances

How do strong backdoors behave for satisfiable instances?

Satisfiable formulas mostly studied w.r.t. weak backdoors, although strong backdoors are also applicable

[Refer to paper for details] Considered satisfiable car configuration instances Also tested with various preprocessing techniques

Key observations: PL yields smaller strong backdoors than UP

Perhaps because pure literal elimination is a “streamliner” Preprocessing does not have any significant effect

ISAIM-08 33

Summary

The notion of backdoor sets takes into account two key features of state-of-the-art constraint solvers:

1. interplay between variable assignments and constraint semantics2. inconsistency detection

These features are critical for small backdoors

In the worst-case, backdoor detection is not in NP(unless PH collapses to NP)

Despite this, in practice, state-of-the art solvers do find surprisingly small backdoors

ISAIM-08 34

Future Directions

Study of backdoors in CSPs, MIP, …

Semantics of backdoors

Backdoors in the context of learning/caching,and restarts

Finer theoretical characterization of the complexity of backdoor detection