Planning as Satisfiability

CSE 574 April 8, 2003Dan Weld

Schedule• BASICS

  Intro  Graphplan  SATplan  State-space  Refinement

• SPEEDUP TECHNIQUES• SIMULTANEOUS + DURATIVE ACTIONS• INCOMPLETE INFORMATION• INTERLEAVED PLANNING + EXECUTION• DECISION-THEORETIC PLANNING

The Idea

• Suppose a plan of length n exists• Encode this hypothesis in SAT

  Init state true at t0

  Goal true at Tn

  Actions imply effects, etc

• Look for satisfying assignment• Decode into plan

RISC: The Revolutionary Excitement

History• Green IJCAI-69• STRIPS AIJ-71• Decades of work on “specialized theorem provers”• Kautz+Selman ECAI-92• Rapid progress on SAT solving• Kautz+Selman AAAI-96

  Electrifying results (on hand coded formulae)

• Kautz, McAllester & Selman KR-96  Variety of encodings (but no compiler)

• CSE 573 => Ernst et al. IJCAI-97

Blackbox• Blackbox solves planning problems by

converting them into SAT.   Very fast  Tried different solvers

• Local search (GSAT)• Systematic search with EBL (RelSAT)• In 2000, GP-CSP could beat Blackbox

– But in 2001, a newer “SUPER-DUPER” SAT solver called CHAFF was developed, that significantly speeds up Graphplan

» By folks in CAD community (EE folks—see getting other people to do our work)

– Currently, CSP people are trying to copy over the ideas from CHAFF to CSP.

Action Representation

Regular fully-instantiated action

Paint-A-Red,Paint-A-Blue,Move-A-Table

Representation One PropositionalVariable per

Example

Simply-split fully-instantiated action’s argument

Paint-Arg1-A Paint-Arg2-Red

Overloaded-split fully-instantiated argument

Act-Paint Arg1-A Arg2-Red

morevars

moreclses

Bitwise Binary encodings of actions

Bit1 ~Bit2 Bit3

Paint-A-Red = 5

Small # Variables, Literals

• Variables for actions at t  Regular = n  Bitwise = log(n)

Main Ideas

• Clear taxonomy• Utility of

  Explanatory frame axioms (most things don’t change)

  Parallelism & conflict exclusion  Type inference  Domain axioms

• Surprising  Effectiveness of regular action encodings

Comparison Among Encodings

• Explanatory Frames are superior to classical- few actions affect each fluent- explanatory frames aid simplifications

• Parallelism is a major factor- fewer mutual exclusion clauses- fewer time steps

• Regular actions representation is smallest!- exploits full parallelism- aids simplification

• Overloaded, bitwise reps. are infeasible- prohibitively many clauses- sharing hinders simplification

Optimization 1: Factored Splitting

- use partially-instantiated actions

HasColor-A-Blue-(t-1) ^ Paint-Arg1-B-t ^Paint-Arg2-Red-t HasColor-A-Blue-(t+1)

.46 .69

.30 .50

.20 .38

Simple Overloaded

Variables

Clauses

Literals

factoredunfactored

Explanatory Frames

Optimization 2: Types

A type is a fluent which no actions affects.• type interference• prune impossible operator

instantiations• type elimination

.27 .39 .34 .30

.10 .97 .67 .74

Classical

Explanatory

Type optsNo type opts

Regular Simple Overloaded

BitwiseLiterals

Domain-Specific Axioms

Adding domain-specific axioms decreases solve time dramatically.

.86 1.53 .26

.88 1.84 .38

.86 2.24 <.05

Vars Clauses Time

a

b

c

domain infono domain info

bw-large

Weaknesses

• Too much info, poor organization• Graphs – lots of data, little insight• Characterize best/worst domain /

encoding• No experiments on effects of type opt

Future Work

• Negation, disjunctive preconds, • Domain axioms

t clear(x, t) y on(y, x, t)

Future Work

• Automatically choose best encoding  Might do this for frame axioms

• Automating domain axioms (invariants)• Use bounds on plan-length

  Rather than linear or binary search

• Resource Planning• Compilation to …?

Domain Axioms• Domain knowledge

  Synchronic vs. Diachronic constraints• Speedup knowledge

  Action conflicts (=> by action schemata alone)

  Domain invariants (=> by initial state+schemata)

  Optimality heuristics

  Simplifying assumptions

Sample Metric Planning Problem

Ship RedVines from one location to another  Fuel used by moving  Maximum fuel level in truck  Maximum load for truck  Limited amount of available RedVines

Sample Metric Planning Domain

Action: MOVE-TRUCK(load) preconditions: load 45 ; max. avail. RedVines fuel 7 + load / 2 ; min. required fuel fuel 15 ; fuel capacity load 30 ; load capacity

effects: (deliver) if (load = 45) then ; good-trip moves (good-trip) ; all the RedVines

Steve Wolfman:

I will use this as a running example

Steve Wolfman:

I will use this as a running example

LPSAT Architecture

planningcompiler

LPSATsolver

planningdecoder

input planning language

LCNF intermediate language

value assignment

solution plan

Systematic SAT Solver+

Incremental LP Solver

LCNF Language

CNF formula with triggered constraints  Boolean variable triggers one (or no)

constraint  Constraint triggered iff its trigger

variable’s truth assignment is true  Truth/real-value assignment is a

solution iff •CNF formula is satisfied•All triggered constraints are satisfied

Sample Problem in LCNF

good-trip1

deliver1

move-truck0 max-load

move-truck0 max-fuel

move-truck0 min-fuel

move-truck0 deliver1

(move-truck0 all-loaded)

good-trip1

...

max-load:

load 30max-fuel:

fuel 15min-fuel:

fuel 7 + load/2all-loaded:

load = 45

...

CNF formula Constraint triggers

Variable names and, in particular, trigger names added for clarity

Variable names and, in particular, trigger names added for clarity

That’s the front end of the system; the back end simply reads off which actions are assigned true and what the constraint vars values are

That’s the front end of the system; the back end simply reads off which actions are assigned true and what the constraint vars values are

Set aside for later use!Set aside for later use!

LPSAT

• Inputs LCNF subset  Supports operators +, -, *, and   Restricts constraints to linear

(in)equalities

• Outputs truth/real-value assignment or reports failure  Sound  Complete  Satisficing

As opposed to optimizing

As opposed to optimizing

SOLVER Architecture

SatisfiabilitySolver

Linear Programming

System

new/revoked constraints

consistency info,real variable values

input problem

solution

LPSAT DesignSAT controller calls LP black box• SAT solver based on RelSAT [Bayardo&Schrag]

  Systematic solver  Sound and complete  Gradual changes to constraint set  Learning/backjumping based on conflict sets

• LP system is Cassowary [Badros&Borning]  supports linear constraints over real variables  fast updates to constraint set

Explores all possible truth assignments without repeating any

Explores all possible truth assignments without repeating any

LPSAT Algorithm

Procedure LPSAT(φ: LCNF problem)If φ is satisfied, return YESElse if φ is inconsistent, return NOElse if there is a unit clause {} or pure literal

in φ, return LPSAT(φ|)Else choose a variable in φ. If LPSAT(φ|),

return YESElse, return LPSAT(φ|)

Put on other projector

Put on other projector

Experimental Results

• Tested against Zeno planner  Many times faster than Zeno  Solved problems Zeno could not solve

• Tested conflict set discovery techniques  Without learning/backjumping (slow)  With global conflict sets (faster)  With minimal conflict sets (fastest)

Under resource bounds (memory)

Under resource bounds (memory)

Learning Speedup

0.01

0.1

1

10

100

1000

10000

easy-1 easy-2 easy-3 easy-4 log-a log-b log-c

Metric Logistics Problems

tim

e (s

)

No learning Global Conflict Sets Minimal Conflict Sets

Conflict Set Discovery ResultsThe domain is a metric version of Kautz and Selman’s logistics domain. Log-c is difficult in the original domain even for modern planners

The domain is a metric version of Kautz and Selman’s logistics domain. Log-c is difficult in the original domain even for modern planners

Alternative encodings..

• The problem of finding a valid plan from the planning graph can be encoded on any combinatorial substrate

• Alternatives:  CSP [GP-CSP]  SAT [Blackbox; SATPLAN]  IP [Vossen et al.]   BDDs [Cimatti ….]

SAT is CSP w

ith B

oolean

Variables

Compilation to CSP[Do & Kambhampati, 2000]

Variables: Propositions (In-A-1, In-B-1, ..At-R-E-0 …)Domains: Actions supporting that proposition in the plan In-A-1 : { Load-A-1, #} At-R-E-1: {P-At-R-E-1, #} Constraints: Mutual exclusion ~[ ( In-A-1 = Load-A-1) & (At-R-M-1 = Fly-R-1)] ; etc..

Activation In-A-1 != # & In-B-1 != # (Goals must have action assignments) In-A-1 = Load-A-1 => At-R-E-0 != # , At-A-E-0 != # (subgoal activation constraints)

[Corr

esp

onds

to a

regre

ssio

n-b

ase

d p

roof]

At(R,E)

At(A,E)

At(B,E)

1: Load(A)

2 : Load(B)

3 : Fly(R)

P-At(R,E)

P-At(A,E)

P-At(B,E)

In(A)

In(B)

At(R,M)

At(R,E)

At(A,E)

At(B,E)

Goals: In(A),In(B)

CSP: Given a set of discrete variables,the domains of the variables, and constraints on the specific values a set of variables can take in combination, FIND an assignment of values toall the variables which respects all constraints

CSP Encodings can be more compact: GP-CSP

Graphplan Satz Relsat GP-CSP

Problem time (s) mem time(s) mem time (s) mem time (s) mem

bw-12steps 0.42 1 M 8.17 64 M 3.06 70 M 1.96 3M

bw-large-a 1.39 3 M 47.63 88 M 29.87 87 M 1.2 11M

rocket-a 68 61 M 8.88 70 M 8.98 73 M 4.01 3M

rocket-b 130 95 M 11.74 70 M 17.86 71 M 6.19 4 M

log-a 1771 177 M 7.05 72 M 4.40 76 M 3.34 4M

log-b 787 80 M 16.13 79 M 46.24 80 M 110 4.5M

hsp-bw-02 0.86 1 M 7.15 68 M 2.47 66 M .89 4.5 M

hsp-bw-03 5.06 24 M > 8 hs - 194 121 M 4.47 13 M

hsp-bw-04 19.26 83 M > 8 hs - 1682 154 M 39.57 64 M

Do & Kambhampati, 2000

Compilation to LP

MotivationsAbility to handle numeric quantities, and optimizeHeuristic value of the LP relaxation of ILP problems

Conversion Convert a SAT/CSP encoding to ILP inequalities

E.g. X v ~Y v Z => x + (1 - y) + z >= 1

Explicitly set tighter ILP inequalities (Cutting constraints)If X,Y,Z are pairwise mutex, we can write x+y+z <= 1 (instead of x+y <=1 ; y+z <=1 ; z +x <= 1)

[ Walser &

Kautz;

Vossen et. al;

Bockmayr & Dimopolous]

Integer Linear Programming: Given a set of real valued variables, a linear objective function on the variables, a set of linear inequalities on the variables, and a set of integrality restrictions on the variables, Find the values of the feasible variables for which the objective function attains the maximum value -- 0/1 integer programming corresponds closely to SAT problem

Vossen/Ball/Lotem/Nau work  Vossen et. Al. [IJCAI 99] tried two different IP encodings

1. Simple conversion of PG SAT encoding to IP (last slide)2. Two new ideas

  Compile away proposition variables.• The change of truth value of a proposition is equated to the

occurrence of one of the actions that can give that proposition. • This is basically a known transformation—see the

McAllester/Kautz/Selman paper. (see next slide)  Reformulate the constraints to make better use of

expressiveness provided by IP• E.g. IP is particularly good at compactly encoding n-ary mutex

constraints—even for n>2• To say only one of X1..Xn can be true, we just say X1+X2+…Xn <= 1 • In SAT, we have to write n constraints of the form

– Xi => ~X1,~X2…~Xi-1,~Xi+1,….~Xn• (IP, on the other hand, will have harder time representing “All-

different” constraints)

Performance of IP Encodings

• As seen in the previous slide, IP encodings are not competitive with SAT/CSP encodings for classical planning  Aren’t exercizing the “optimizing” properties of IP,   Aren’t exercizing its ability to encode numerical variables

• IP may well do better if we are interested in finding “cost-optimal” plans of a given makespan (length)  No one has confirmed this as yet…

• IP encodings may well be more useful when we have numerical quantities such as actions consuming metric resources (preconditions and effects involve numbers)  See the work by Kautz and Walser(AAAI-99) or   LPSat (Wolfman/Weld—IJCAI 99).

Tradeoffs as f(compilation substrates)

• CSP encodings support implicit representations  More compact encodings [Do & Kambhampati, 2000]  Easier integration with Scheduling techniques

• ILP encodings support numeric quantities   Seamless integration of numeric resource constraints [Walser & Kautz, 1999]  Not competitive with CSP/SAT for problems without numeric constraints

• SAT encodings support axioms in propositional logic form  May be more natural to add (for whom ;-)  Very active research on finding faster solvers..

• Davis Putnam ( Satz; RelSat; Chaff)• Gsat

Direct vs. compiled solution extraction

Need to adapt CSP/SAT techniques

Can exploit approaches for compacting the plan

Can make the search incremental across iterations

Compilation stage can be time consuming, leads to memory blow-up

Makes it harder to exploit search from previous iterations

Can exploit the latest advances in SAT/CSP solvers

Makes it easier to add declarative control knowledge

DIRECT Compiled

Resource Planning

Challenges in SAT