Unifying SAT-based and Graph-based Planning

Unifying SAT-based and Graph-based Planning

Henry KautzAT&T Labs

Bart SelmanCornell University

IJCAI-99

2

SATPLAN (Kautz & Selman 1996)SATPLAN (Kautz & Selman 1996)

axiomschemas instantiated

propositionalclauses

satisfyingmodelplan

mapping

length

problemdescription

SATengine(s)

instantiate

interpret

3

SAT AlgorithmsSAT Algorithms

Systematic Search• DP (Davis Putnam Logemann Loveland)

backtrack search + unit propagation

• satz (Chu Min Li) - variable selection by forward checking: max unit props

• relsat (Bayardo) - dependency directed backtracking: add new clauses at dead-ends

Local Search• Walksat (Selman, Kautz & Cohen)

local search + noise to escape minima

4

Critically-constrained Logistics Planning Problems

Critically-constrained Logistics Planning Problems

0.01

0.1

1

10

100

1000

10000

rocket.a

rocket.b

log.b

log.a

log.c

log.d

log

so

luti

on

tim

e

Graphplan

DP

DP/Satz

Walksat

5

Tradeoffs of SAT ApproachTradeoffs of SAT Approach

Advantages

• Can trade space for time by avoiding variable binding during search

• Domain modeling can substitute for algorithm development

• New high powered SAT algorithms can take advantage of implicit structure of encoded problems

Disadvantages

• Instantiated formulas huge, much redundancy

• Good domain models can be hard to develop - automatic STRIPS translations disappointing

• No way to explicitly leverage structure

6

SATPLAN & Graphplan: Disjunctive Planners

SATPLAN & Graphplan: Disjunctive Planners

Graphplan (Blum & Furst 1995)

Set new paradigm for planning

Like SATPLAN...

• Two phases: instantiation of propositional structure, followed by search

Unlike SATPLAN...

• Interleaves instantiation and pruning of plan graph

• Employs specialized search engine

Neither approach best for all domains or all instances

• Graphplan - better instantiation

• SATPLAN - better search

IJCAI Challenge in Bridging Plan Synthesis Paradigms (Kambhampati 1997)

7

BlackboxBlackbox

STRIPSPlan Graph

Reachability Analysis

CNF

GeneralStochastic / Systematic SAT engines

Solution

SimplifierTranslator

CNF

8

Staged InferenceStaged Inference

Domain specific model

Polytime domain specific inference

General language encoding

Full general inference(NP complete)

Solution

Polytime general inference

Abstract problem specification

Encoding scheme

Combinatorial CORE

9

IntuitionIntuition

Many real-world problems not tractable, but are nearly so

• polytime inference takes advance of special kinds of structure

• structure may be visible at the level of a domain specific representation, or only after the problem is encoded

• small number of practical methods for combinatorial core

10

Component 1: Reachability AnalysisComponent 1: Reachability Analysis

Graphplan instantiates in a forward direction, pruning unreachable nodes • conflicting actions are mutex

• if all actions that add two facts are mutex, the facts are mutex

• if the preconditions for an action are mutex, the action is unreachable

Reachability analysis in unfolded Petri Nets(K. McMillian 1992)

11

The Plan GraphThe Plan Graph

Facts FactsActions

... ...

Facts FactsActions

... ...

preconditions add effects

mutually exclusive

delete effects

12

Component 2: TranslationComponent 2: Translation

Fact Act1 Act2

Act1 Pre1 Pre2

¬Act1 ¬Act2

Act1

Act2

Fact

Pre1

Pre2

Backward-chaining axioms force groundedness

Prevents underconstrained variables from taking on arbitrary values

13

Mutex Algorithm as ResolutionMutex Algorithm as Resolution

Each mutex computation equivalent to a series of resolutions

• one resolvant always negative binary clause

K actions add P (1 clause)

K actions add Q (1 clause)

all P adders mutex Q adders (K2 clauses)

Inferring (~P v ~Q) requires 4K2 resolutions

14

Improved EncodingsImproved Encodings

Translations of Logistics.a:

STRIPS Axiom Schemas SAT(Medic system, Weld et. al 1997)

• 3,510 variables, 16,168 clauses

• 24 hours to solve

STRIPS Plan Graph SAT

• 2,709 variables, 27,522 clauses

• 5 seconds to solve!

15

Component 3: SimplificationComponent 3: Simplification

Generated wff can be further simplified by consistency propagation techniques

• unit propagation: is Wff inconsistant by resolution against unit clauses?

O(n)

• failed literal rule: is Wff + { P } inconsistant by unit propagation?

O(n2)

• binary failed literal rule: is Wff + { P V Q } inconsistant by unit propagation?

O(n3)

General limited inference complements domain specific limited inference (mutex)

Reveals hidden local structure

16

General Limited InferenceGeneral Limited Inference

Percent vars set byProblem Varsunitprop

failedlit

binaryfailed

bw.a 2452 10% 100% 100%bw.b 6358 5% 43% 99%bw.c 19158 2% 33% 99%log.a 2709 2% 36% 45%log.b 3287 2% 24% 30%log.c 4197 2% 23% 27%log.d 6151 1% 25% 33%

17

Component 4: Improved Systematic SAT Solvers

Component 4: Improved Systematic SAT Solvers

Systematic search generally best for wffs derived from STRIPS operators

• Wffs not as “flat” - long chains of unit propagations

Problem:

Solution time for backtrack search highly variable as problem instance varied

• “easier” problems may take orders of magnitude longer to solve than “harder” ones!

18

Unpredictability of Systematic Search

Unpredictability of Systematic Search

0.01

0.1

1

10

100

1000

10000

rocket.a

rocket.b

log.a

log.b

log.c

log.d

log

so

luti

on

tim

e

Satz

19

Randomized RestartsRandomized Restarts

Heavy tailed distribution of running times

Solution: randomize the systematic solver

• Add noise to the heuristic branching (variable choice) function

• Cutoff and restart search after a fixed number of backtracks

In practice: rapid restarts with low cutoff can dramatically improve performance

(Gomes 1996, Gomes, Kautz, and Selman 1997, 1998)

20

Increased PredictabilityIncreased Predictability

0.01

0.1

1

10

100

1000

10000

rocket.a

rocket.b

log.a

log.b

log.c

log.d

log

so

luti

on

tim

e

Satz

Satz/Rand

21

Summary of ResultsSummary of Results

Blackbox /satz-rand

Graphplan /IPP

SATPLANmake (walk) satz-rand

rocket.b 5 sec 55 sec 41 (1) 1 sec

log.a 5 sec 31 min 72 (2) 4 sec

log.b 7 sec 13 min 78 (3) 7 sec

log.c 9 sec * 102 (2) 1 sec

log.d 28 sec * 210 (7) 96 sec

22

ObservationsObservations

SAT engines can outperform direct search of plan graph

• when problems critically constrained

• bottleneck is extraction (not reachability)

• when graphplan/IPP heuristics for non-optimal planning (e.g. RIFO) not applicable

Solution time using best randomized systematic SAT algorithm virtually identical for BlackBox and SATPLAN wffs

• although SATPLAN wffs included much extra explicit domain knowledge - invariants, etc.

Scaling of BlackBox/satz-rand closely matches scaling of SATPLAN/walksat (~ 4x)

23

ApplicabilityApplicability

When is the BlackBox approach not a good idea?

• when domain too large for propositional planning approaches

• when long sequential plans are needed

• when solution time dominated by reachability analysis (plan-graph generation), not extraction

• when optimal or near optimal planning not necessary

24

Efficiency of Translation ApproachEfficiency of Translation Approach

Translation usually not a bottleneck• wff grows linearly in size of plan graph

• modified translation reduces explicit mutex clauses by 75%

• new compact representations of plan graph will challenge this approach!

(Koehler, Fox & Long, Smith & Weld...)

Loss of cached information acceptable on hardest problems

• Graphplan caches info when searching “too short” graphs, use to speed up search of expanded graph

• For critically constrained problems, nearly all effort goes into searching last (or next to last) size problem

25

Next Steps...Next Steps...

1. Domain-specific Control Knowledge• Encode state invariants & heuristics axiomatically

– Trucks always in one location

– Don’t move a package from a destination location

Dramatic speedup possible (Kautz & Selman 1998)

• For non-admissible control knowledge, tradeoff between speed / solution quality (Huang, Selman, Kautz AAAI-99)

– Temporal logic specification used to generate axioms and/or prune plan graph

– Using control knowledge from TLPlan (Bacchus 1996), can find better parallel plans

• Current work: inductive learning of control knowledge

26

Comparison between Blackbox and TLPlan(Parallel Plan Length)

0

5

10

15

20

25

30

35

log-c log-d log-e log-1 log-2

Par

alle

l P

lan

Len

gth

TLPlan Blackbox

27

Next Steps...Next Steps...

2. Beyond SAT: Planning with Resources and Optimization Criteria

• SAT special case of 0/1 integer linear programming

• ILPPlan (Kautz & Walser AAAI-99)Model extended STRIPS in AMPL, solve with

– Branch and bound

– Local search WSAT(OIP)

• Current work: IP translator for BlackBox(Nau et al 1999) - better encodings for B&B solvers

(Weld et al 1999) - new SAT+LP engine

28

Next Steps...Next Steps...

3. Planning with Incomplete & Uncertain Information

• The “Holy Grail”

• SAT-encoding approaches

– Contingent planning via QBF (Rintanen 1999)

– C-MAXPLAN, ZANDER (Littman & Majercik 1999)Probabilistic planning via stochastic SAT

state of the art performance on (small, hard) POMDP problems

• Extensions to Graphplan

– contingent plans (Weld, Anderson, Smith 1998)

– probabilistic plans (Blum & Langford 1998)

• GOAL: a universal BlackBox

29

Big PictureBig Picture

Domain specific model

Polytime domain specific inference

General language encoding

Full general inference(NP complete)

Solution

Polytime general inference

Abstract problem specification

Encoding scheme

Combinatorial CORE