View
226
Download
1
Category
Tags:
Preview:
Citation preview
Classical Planning
2
What is Planning?
generate sequences of actions to perform tasks and achieve objectives involving states, actions & goals
a search for a solution over an abstract space of plans the assumptions for classical planning problems
fully observable we see everything that matters
deterministic the effects of actions are known exactly
static no changes to environment other than those caused by agent
actions discrete
changes in time and space occur in quantum amounts single agent
no competition or cooperation to account for
Classical Planning
3
Background: Planning
real systems help humans in practical applications design and manufacturing environments military operations, games, space exploration
notes: the methods of Ch 10 (Classical Planning)
assume the classical environment assumptions in Ch 11 (Planning & Acting in the Real World)
we'll introduce methods to handle real-world situations where the classical assumptions may not hold
Classical Planning
4
Planning: Language
What is a good language? expressive enough to describe a wide variety of problems restrictive enough for efficient algorithms to operate on it planning algorithm should be able to take advantage
of the logical structure of the problem
historical AI planning languages STRIPS was used in classical planners
Stanford Research Institute Problem Solver ADL addresses expressive limitations of STRIPS
Action Description Language adds features not in STRIPS
negative literals, quantified variables, conditional effects, equality
we'll use one version of the now de facto standard PDDL
Classical Planning
5
Planning language
there's now wide agreement on some planning language basics
one key is adopting of a factored representation for states each state is represented as a collection of variables this contrasts with
the atomic representation typical of our previous search algorithms (recall the Romanian driving problem: state = city)
the language we'll use is a variant of PDDL Planning Domain Definition Language
to see its expressive power, recall propositional agent in the Wumpus World, which requires 4Tn2 actions to describe a movement of 1 square
PDDL captures this with a single Action Schema
Classical Planning
6
State & Action Representations
each state is represented as a conjunction of fluents these are ground, functionless atoms in addition, we'll use Database semantics 1. the Close World Assumption
fluents not explicitly mentioned are false 2. the Unique Names Assumption
different ground terms are different objects: Plane1, Plane2 this state representation allows alternative algorithms
it can be manipulated either by logical inference techniques or by set operations
actions are defined by a set of action schemas these implicitly define the ACTIONS(s) & RESULT(s, a)
functions required to apply search techniques
Classical Planning
7
Action Schemas
PDDL & the Frame Problem recall the representational issue of capturing what stays the
same given some action in PDDL we specify what changes, and if something is not
mentioned, it stays the same Action Schemas
are a lifted representation (recall Generalized Modus Ponens) lifts from propositional logic to a restricted subset of FOL each one stands for a set of variable-free actions each includes the schema name, list of variables used,
preconditions & effects we consider variables as universally quantified, choose values
to instantiate them PRECOND: defines states in which an action can be executed EFFECT: defines the result of executing the action
Classical Planning
8
Action Schemas
each represents a set of variable-free actions form: Action Schema = predicate + preconditions + effects example:
Action(Fly(p, from, to),PRECOND: At(p, from) Plane(p) Airport(from) Airport(to)EFFECT: ¬AT(p, from) At(p, to))
an action schema in which (p, from, to) need to be instantiated the action name and its parameter list its preconditions
a conjunction of function-free literals its effects
a conjunction of function-free literals
Classical Planning
9
Applying Action Schemas
action a is applicable in state s if its preconditions are satisfied in s
its preconditions are entailed by s given variables, there can be multiple applicable
instantiations for v variables in a domain with k unique object names, worst
case time to find applicable ground actions is O(vk) leads to 1 approach for solving PDDL planning problems
propositionalize by replacing action schemas with sets of ground actions & applying a propositional solver like SATPlan
impractical for large v & k
result of executing a in s is s' s' is derived from s & action a's EFFECTs remove -ve literal fluents (the delete list, DEL(a)) add +ve literals in EFFECTs (the add list, ADD(a)) RESULT(s, a) = (s - DEL(a)) ADD(a)
Classical Planning
10
Action Schemas
PDDL schemas 1. variables & ground terms
variables in effects must also be in precondition so matching to state s yields results with all variables bound i.e. that contain only ground terms
2. handling of time no explicit time terms, unlike axioms we saw for logical agents instead time is implicitly represented in PDDL schemas preconditions always refer to time: t effects always refer to time: t + 1
3. a set of schemas defines a planning domain a specific problem adds initial & goal states
Classical Planning
11
Initial States, Goals, Solutions
initial state conjunction of ground terms
goal conjunction of +ve & -ve literals both ground terms & those containing variables variables are treated as existentially quantified
solution a sequence of actions ending in s that entails the goal example: Plane(P1) At (P1, SFO) entails At(p, SFO) Plane (p)
defines planning as a search problem
Classical Planning
12
Some Language Semantics
How do actions affect states? an action is applicable in any state that satisfies its
precondition(s) applicability involves a substitution for the variables in the
PRECOND example
State:At(P1,JFK) At(P2,SFO) Plane(P1) Plane(P2) Airport(JFK)
Airport(SFO)
Satisfies PRECOND of Fly action:At(p, from) Plane(p) Airport(from) Airport(to)
With substitution ={p/P1, from/JFK, to/SFO}Thus the action is applicable.
Classical Planning
13
Language Semantics
result of executing action a in state s is the state s’ s’ is same as s except
any positive literal P in the effect of a is added to s’ any negative literal ¬P is removed from s’ example
Fly(p, from, to)EFFECT: ¬AT(p, from) At(p, to), ={p/P1, from/JFK, to/SFO}
s: At(P1,JFK) At(P2,SFO) Plane(P1) Plane(P2) Airport(JFK) Airport(SFO)s': At(P1,SFO) At(P2,SFO) Plane(P1) Plane(P2) Airport(JFK)
Airport(SFO)
Classical Planning
14
Example: air cargo transport
simple actions to illustrate moving cargo between New York(JFK) & San Francisco(SFO)
Init(At(C1, SFO) At(C2,JFK) At(P1,SFO) At(P2,JFK) Cargo(C1) Cargo(C2) Plane(P1) Plane(P2) Airport(JFK) Airport(SFO))
Goal(At(C1,JFK) At(C2,SFO))
Action(Load(c, p, a)PRECOND: At(c, a) At(p, a) Cargo(c) Plane(p) Airport(a)EFFECT: ¬At(c, a) In(c, p))
Action(Unload(c, p, a)PRECOND: In(c, p) At(p, a) Cargo(c) Plane(p) Airport(a)EFFECT: At(c, a) ¬In(c, p))
Action(Fly(p, from, to)PRECOND: At(p, from) Plane(p) Airport(from) Airport(to)EFFECT: ¬ At(p, from) At(p, to))
Classical Planning
15
Air Cargo Example
this planning domain involves 3 actions and 2 affects predicates, In(c, p) & At(x, a) properly maintaining At predicates could be an issue
where is cargo that is in a plane? without quantifiers, problem of expressing what happens to cargo
that is in a plane solve by treating At predicate as applying only to cargo after it is
unloaded, in effect not At anywhere when it is in a plane another issue is that this representation allows "empty"
actions that produce contradictory effects Fly(P1, JFK, JFK) yields At(P1, JFK) ¬At(P1, JFK) can ignore issues like this, rarely produce incorrect plans, or add inequality precondition: from to
Classical Planning
16
Air Cargo Example
the solution is pretty obviousInit(At(C1, SFO) At(C2,JFK) At(P1,SFO) At(P2,JFK) Cargo(C1) Cargo(C2)
Plane(P1) Plane(P2) Airport(JFK) Airport(SFO))Goal(At(C1,JFK) At(C2,SFO))
Action(Load(c, p, a)PRECOND: At(c, a) At(p, a) Cargo(c) Plane(p) Airport(a)EFFECT: ¬At(c, a) In(c, p))
Action(Unload(c, p, a)PRECOND: In(c, p) At(p, a) Cargo(c) Plane(p) Airport(a)EFFECT: At(c, a) ¬In(c, p))
Action(Fly(p, from, to)PRECOND: At(p, from) Plane(p) Airport(from) Airport(to)EFFECT: ¬ At(p, from) At(p, to))
[Load(C1, P1, SFO), Fly(P1, SFO, JFK), Unload (C1, P1, JFK), Load(C2, P2, JFK), Fly(P2, JFK, SFO), Unload(C2, P2, SFO)]
Classical Planning
17
Example: flat tire problem
simple actions to illustrate changing a flat tire (in a bad neighbourhood)
Init(At(Flat, Axle) At(Spare, Trunk))Goal(At(Spare, Axle))
Action(Remove(Spare, Trunk)PRECOND: At(Spare, Trunk)EFFECT: ¬At(Spare, Trunk) At(Spare, Ground))
Action(Remove(Flat, Axle)PRECOND: At(Flat, Axle)EFFECT: ¬At(Flat, Axle) At(Flat, Ground))
Action(PutOn(Spare, Axle)PRECOND: At(Spare, Ground) ¬At(Flat, Axle)EFFECT: At(Spare, Axle) ¬At(Spare, Ground))
Action(LeaveOvernightPRECOND:EFFECT: ¬ At(Spare, Ground) ¬ At(Spare, Axle) ¬ At(Spare, Trunk) ¬ At(Flat, Ground) ¬ At(Flat, Axle) )
Classical Planning
18
Flat Tire example
highly simplified, abstracted version of the problem of constructing a plan to fix a flat tire 4 actions note capturing of "bad neighbourhood" as tires "disappearing"
if the car is left overnight the simple solution
[Remove (Flat, Axle), Remove(Spare, Trunk), PutOn(Spare, Axle)]
Classical Planning
19
Example: Blocks World
simple actions to illustrate building a tower in Blocks World
Init(On(A, Table) On(B, Table) On(C,A) Block(A) Block(B) Block(C) Clear(B) Clear(C))
Goal(On(A,B) On(B,C))
Action(Move(b, x, y)PRECOND: On(b, x) Clear(b) Clear(y) Block(b) (b x) (b y) (x y) EFFECT: On(b, y) Clear(x) ¬ On(b, x) ¬ Clear(y))
Action(MoveToTable(b, x)PRECOND: On(b, x) Clear(b) Block(b) (b x) EFFECT: On(b, Table) Clear(x) ¬ On(b, x))
simple plan given the initial state note PRECOND of Move uses inequality to prevent empty moves Move(B,C,C)
Classical Planning
21
Blocks World example
problem domain: stack cube-shaped blocks that are on a table
Notes: only 1 block can fit on top of another moves can be onto another block or onto the table FOL would use quantifiers to express
there's no block on top of some other block without quantifiers, PDDL requires Clear(x) predicate
Move action schema must be complemented by MoveToTable to avoid errors with the Clear predicate
since Clear(Table) is always true residual problem if bind y to Table in Move(b, x, y)
search space grows, though answers still correct fix with a Block(m) predicate & adding Block(b) Block(y) to the
preconditions for Move
Classical Planning
22
Complexity & Classical Planning
first, we distinguish 2 versions of the planning problems PlanSAT:
is there a plan that solves the problem? Bounded PlanSAT:
is there a solution of k or fewer steps? this can be used to find optimal plans
so long as we don't allow functions the number of states is finite & both categories are decidable
if we allow functions, PlanSAT becomes semi-decidable (may not terminate on unsolvable problems) though Bounded PlanSAT remains decidable even with functions
23
Complexity & Classical Planning
sadly, in general, both problems are NP-hard putting restrictions on the expressiveness may help for example, disallowing -ve preconditions lets PlanSAT reduce
to P fortunately for planners, many useful problems are easier than
the worst case for blocks world & air cargo domains, Bounded PlanSAT is NP-
Complete, while PlanSAT is in P! so, optimal planning is hard, sometimes sub-optimal planning is
easy reminder of complexity category relationships
24
Planning with state-space search
given the problem formulation we can use search algorithms to generate plans
systematic heuristic search local search (while recording trail of states)
search can be in either the forward or the backward direction
recall the bi-directional example from search topic problems related to non-invertible successor functions
Progression planners do a forward state-space search consider the effects of all possible actions in a given state
Regression planners do a backward state-space search to achieve a goal
consider what must have been true in the previous state from effects through actions to preconditions
Classical Planning
26
Progression algorithm
formulation as a state-space search problem Initial state = initial state of the planning problem
literals not appearing are false Actions = apply those whose preconditions are satisfied
add positive effects, delete negative Goal test = does the state satisfy the goal Step cost = for simplicity, each action costs 1 recall, functions are not permitted in the representation
any graph search that is complete is a complete planning algorithm
for example, A*
Classical Planning
27
Progression algorithm
Progression approach while complete is inefficient
any applicable action is a possible next step typically its problems are related to branching factor
(1) irrelevant actions just because the preconditions of an action are satisfied does not mean that it should be done
(2) requirement for a good heuristic in order for improved efficiency of search
Classical Planning
28
Progression: issues
illustrate with the air cargo example assume 10 airports, 5 planes & 20 pieces of cargo at each problem: move all cargo at airport A to airport B solution is pretty simple, but uninformed search has serious
complexity difficulties just considering planes: 50 x 9 = 450 possible flights suppose all planes & packages were at 1 airport: 10450 possible
actions (200 x 50 + 50 x 9) if 2000 actions available on average, 200041 nodes in search
graph fortunately, good (domain-independent) heuristics are
possible, & can be derived automatically
Classical Planning
29
Regression algorithm
what about Regression planners issues:
How do we determine predecessors? as we saw for bi-directional search, the problem may not allow
predecessor determination (n-Queens) fortunately PDDL formulation facilitates it terminology: predecessor is g', action a, goal g g' = (g - Add(a)) Precond(a), since any effects added by a might
not have been true previously & preconditions must have been met or could not execute a
this process requires partially uninstantiated actions & states use standardized variable names (avoid clashes, retain generality
of variables) implicitly quantify over these so 1 description summarizes
possibility of using any appropriate ground term
Classical Planning
30
Regression: relevant actions
which action? forward uses applicable actions (their preconditions are met) backward uses relevant actions (they could be the last step
leading to current state)
relevant: at least 1 effect (+ve or -ve) must unify with a term in the (current) goal & there can't be any effect that negates a term in the goal
process: use the action formed by substituting the most general unifier into the standardized action schema
more formal definition of relevant actions if goal g contains literal gi & action schema A (standardized to
produce A') has effect literal ej' where Unify (gi, ej') = & a' = SUBST(, A') & there's no effect in a' that is the negation of a literal in g, then a' is a relevant action toward g
terminates when a predecessor is satisfied by the initial state
Classical Planning
31
Progression & Regression
status of the regression approach 1. there's a lower branching factor in the backward direction,
the result of only considering relevant actions 2. but each state is really a set of states
the state defined by true or false for the ground fluents, and the states not mentioned (see mid p 374)
this representation of stages in the backward search as sets of states complicates the process of developing heuristics
progression versus regression the greater availability of accurate heuristics for forward
search has resulted in it being used in many more planning systems
Classical Planning
32
Heuristics for planning
the complexity of planning problems means that we need good heuristics below: 2 general approaches to finding admissible heuristics
recall that an admissible h(n) (i.e. one that never overestimates cost) allows A* search for optimal solutions
recall from our (informed search) discussion of finding heuristics
as the optimal solution to a relaxed version of the problem from problem decomposition
if we can make the subgoal independence assumption we can approximate the cost of solving a conjunction of subgoals as the sum of the costs of solving the corresponding subproblems
(1) relaxation: search problem viewed as a graph allows (a) adding edges to make a path to a solution easier to find (b) merging states in an abstraction of the original problem so
searching is in a space with fewer nodes
Classical Planning
33
Heuristics by problem relaxation
(a) adding edges to the search space (1) ignore all preconditions
assume all actions are applicable in all states, so any goal fluent takes just a single step
apparently #steps ≈ #unsatisfied goals, but note: some actions may achieve > 1 goal, & others may undo goals ignoring the latter, h(n) = min # actions for which the union of their
effects satisfies the goal this is a version of the "set-cover problem", known to be NP-hard! a greedy algorithm yields set covering within log n of the true min (n
is # literals in goal), but is not guaranteed admissible
(2) ignore selected PRECONDs of actions as in the N-puzzle heuristics: misplaced tiles, manhattan distance the factored representation in action schemata for planning
problems allows automated derivation of such heuristics (3) ignore delete lists
if all goals & preconditions have only +ve literals (true for many problems anyway & others can be converted to this form)
Classical Planning
34
Heuristics by problem relaxation
(a) adding edges to search space (3) ignore delete lists
with only +ve literals in all goals & preconditions then remove -ve literals from effects now steps are monotonic progress towards the goal w/o undoing but, it's still NP-hard to find an optimal solution, though it can be
approximated in polynomial time with hill climbing summary
adding edges relaxes a planning problem to yield heuristics but they're still expensive to calculate
(b) reduce nodes by abstraction to merge states return to the air cargo example
10 airports, 50 planes, 200 packages means the general case has 5010 x 20050+10 or 10155 states
a specific problem: say all pkgs are at 5 airports & all pkgs at any airport have 1 destination
35
Heuristics by problem relaxation
(b) reduce nodes by abstraction to merge states in the air cargo example, in general, 10155 states
a specific problem has all pkgs at 5 airports & pkgs at any airport have 1 destination
an abstraction of this version omits At fluents except those re: 1 plane & 1 pkg at each of 5 airports, reducing # states to 510 x 55+10 or 1017 states
so a solution is shorter & an admissible h(n) & is extensible to the original problem by adding more Load & Unload actions
problem decomposition approaches for h(n)'s the relevant pattern is:
subdivide problem solve subproblems combine solutions the subgoal independence assumption
is useful in developing a heuristic as a sum of subgoal costs is too optimistic if there are -ve interactions between subplans is too pessimistic (inadmissible) when there are redundant actions
in subplans
Classical Planning
36
Heuristics by decomposition
problem decomposition approaches for h(n)'s notation: represent the goal as a set of fluents G, where
disjoint subsets of G are G1, …Gn, for which we find plans P1, … Pn & then use their costs to estimate cost for G
Cost(Pi) is an estimate, a possible heuristic
maxi Cost(Pi) is admissible, overly optimistic
sumi Cost(Pi) is not generally admissible unless all Gi & Gj are independent (Pi does not affect preconds or goals of Pj) in which case the sum is admissible & more accurate
37
Heuristics by decomposition
the challenge of abstraction is to find one that has a significantly lower total cost (defining the
abstraction + doing the abstracted search + mapping back to original problem) than the original problem
the pattern database techniques may be useful recall them from informed search discussion of heuristics amortize the cost of building the database over multiple problem
solution instances
the FF (FASTFORWARD) planning system is a hybrid it uses heuristics from a planning graph (our next discussion) it uses local search (storing plan steps) at a plateau or local maximum it does iterative deepening
systematic search for a better state or gives up & restarts
Classical Planning
38
Planning Graphs
Planning Graphs are an alternative intermediate representation/data structure
they are polynomial complexity approximations of full (exponential) trees of all states & actions
they can't answer the question: is G reachable? though they are correct when they say it's not reachable they provide an estimate of the number of steps to G any such estimate is optimistic, so is an admissible heuristic
planning graphs provide a possible basis for better search heuristics and they can also be used directly, for extracting a solution to a
planning problem, by applying the GRAPHPLAN algorithm
Classical Planning
39
Planning Graphs
a planning graph is a directed graph organized in levels the levels of nodes correspond to time steps in a plan
they consist of alternating S levels (in which the nodes are fluents that hold) & A levels (in which the nodes are actions that might be applicable)
level S0 is the initial state each level is a set of literals Si or a set of actions Ai
Si: all the literals that could be true at that time step depending on the actions executed at the previous steps
Ai: all actions that could have PRECONDs satisfied at that step depending on which of the literals actually hold
Classical Planning
40
Planning Graphs
explanation regarding could on the previous slide a planning graph only captures a restricted subset of the
possible -ve interactions so a literal might appear at a level earlier than it actually would
(if it would at all), though it never appears too late despite this error, the level j of the first appearance is a good
estimate of how difficult it is to achieve from the initial state refers to the approximate nature of the state/action lists in part, this allows efficient construction by recording a restricted
subset of possible -ve interactions among actions
note: planning graphs apply only for propositional problems action schemas for problems with variables can be
propositionalized their advantages may make this worthwhile, despite the
increased size of the problem description
41
Planning Graphs
our simple example: "have your cake & eat it too" the problem description
Init(Have(Cake))Goal(Have(Cake) Eaten(Cake))Action(Eat(Cake)
PRECOND: Have(Cake)EFFECT: ¬Have(Cake) Eaten(Cake))
Action(Bake(Cake)PRECOND: ¬ Have(Cake)EFFECT: Have(Cake))
the corresponding planning graph
42Planning Graph Cake Example
start at level S0, determine action level A0 & next level S1
A0: all actions whose preconditions are satisfied in the previous level
(initial state) actions are shown in rectangular boxes lines connect PRECONDs at S0 to EFFECTs at S1
also, for each literal in Si, there's a persistence action (square box) & line
to it in the next level Si+1
level A0 contains the actions that could occur conflicts between actions are represented by arcs: mutual exclusion or
mutex links
43Planning Graph Cake Example
level S1 contains all the literals that could result from picking any subset of actions in A0
so S1 is a belief state consisting of the set of all possible states each is a subset of literals with no mutex links between members
conflicts between literals that cannot occur together are represented by the mutex links.
the level generation process is repeated eventually consecutive levels are identical: leveling off
Classical Planning
44Planning Graph Cake Example
mutex relation holds between 2 actions at a level when 1. inconsistent effects
one action negates the effect of another 2. interference
an effect of one action negates a precondition of the other 3. competing needs
a precondition of one action is mutex with a precondition of the other
Classical Planning
45Planning Graph Cake Example
mutex relation holds between 2 literals at a level when 1. one is the negation of the other 2. if each possible action pair that could achieve the literals is
mutex (Have(Cake) & Eaten(Cake) at S1) this is termed inconsistent support
Classical Planning
46
Planning Graphs & Heuristics
Planning Graphs construction has complexity polynomial in the size of the
planning problem given l literals, a actions, & a PG of n levels: O(n(a + l)2)
the completed PG provides information about the problem & candidate heuristics 1. a goal literal g that does not appear in the final level cannot be
achieved by any plan 2. the level cost, the level at which a goal literal first appears, is
useful as a cost estimate of achieving that goal literal note that level cost is admissible, though possibly inaccurate
since it counts levels, not actions we could find a better alternative level cost by using a serial
planning graph variation, restricted to one action per level mutex links between every pair of actions except persistence actions
Classical Planning
47
Planning Graphs & Heuristics
Planning Graph provides possible heuristics for the cost of a conjunction of goals 1. max-level: highest level of any conjunct in the goal
admissible, possibly not accurate 2. level sum: the sum of level costs of conjuncts in the goal
incorporates the subgoal independence assumption so may be inadmissible to degree the assumption does not hold
3. set-level: level where all goal conjuncts are present without mutex links
this one dominates the max level heuristic & is good where there are interactions among subplans
Classical Planning
48
Planning Graphs & Heuristics
a Planning Graph is a relaxed version of the problem if a goal literal g does not appear, no plan can achieve it, but
if it does appear, is not guaranteed to be achievable why?
the PG only captures pairwise conflicts & there could be higher order conflicts
likely not worth the computational expense of checking for them similar to Constraint Satisfaction Problems where arc consistency
was a valuable pruning tool 3-consistency or even higher order consistency would have made
finding solutions easier but was not worth the additional work an example where PG fails to indicate unsolvable problem
blocks world problem with goal of A on B, B on C, C on A any pair of subgoals are achievable, so no mutexes problem only fails at stage of searching the PG
Classical Planning
49
The GRAPHPLAN Algorithm
the GRAPHPLAN algorithm generates the Planning Graph & extracts a solution from it
function GRAPHPLAN(problem) return solution or failuregraph INITIAL-PLANNING-GRAPH(problem)goals CONJUNCTS(problem. GOAL)nogoods an empty hash tablefor tl = 0 to do
if goals all non-mutex in St of graph then
solution EXTRACT-SOLUTION(graph, goals,NUMLEVELS(graph), nogoods)
if solution failure then return solutionif graph and nogoods have both leveled off then return failuregraph EXPAND-GRAPH(graph, problem)
Classical Planning
50
Example: Spare Tire Problem
recall the spare tire problemInit(At(Flat, Axle) At(Spare, Trunk))
Goal(At(Spare, Axle))
Action(Remove(Spare, Trunk)PRECOND: At(Spare, Trunk)EFFECT: ¬At(Spare, Trunk) At(Spare, Ground))
Action(Remove(Flat, Axle)PRECOND: At(Flat, Axle)EFFECT: ¬At(Flat, Axle) At(Flat, Ground))
Action(PutOn(Spare, Axle)PRECOND: At(Spare, Ground) ¬At(Flat, Axle)EFFECT: At(Spare, Axle) ¬At(Spare, Ground))
Action(LeaveOvernightPRECOND:EFFECT: ¬ At(Spare, Ground) ¬ At(Spare, Axle) ¬ At(Spare, Trunk)
¬ At(Flat, Ground) ¬ At(Flat, Axle) )
Classical Planning
51GRAPHPLAN Spare Tire Example
Notes: this figure shows the complete Planning Graph for the problem arcs show mutex relations
but arcs between literals are omitted to avoid clutter it also omits unchanging +ve literals (for example, Tire(Spare))
& irrelevant -ve literals bold boxes & links indicate the solution plan
Classical Planning
52GRAPHPLAN Spare Tire Example
S0 is initialized to 5 literals from the problem initial state and the CWA literals
no goal literal in S0 so EXPAND-GRAPH add actions those with preconditions satisfied in S0
also adds persistence actions for literals in S0
adds the effects at level S1, analyzes & adds mutex relations repeat until the goal is in level Si or failure
53GRAPHPLAN Spare Tire Example
EXPAND-GRAPH adds constraints: mutex relations inconsistent effects (action x vs action y)
Remove(Spare, Trunk) & LeaveOvernight: At(Spare, Ground) & ¬At(Spare, Ground)
interference (effect negates a precondition) Remove(Flat, Axle) & LeaveOvernight: At(Flat, Axle) as PRECOND & ¬At(Flat,
Axle) as EFFECT competing needs (mutex preconditions)
PutOn(Spare, Axle) & Remove(Flat, Axle): At(Flat, Axle) & ¬At(Flat, Axle) inconsistent support (actions to produce literals are mutex)
in S2, At(Spare, Axle) & At(Flat, Axle)
Classical Planning
54
Reminder: GRAPHPLAN Algorithm
the GRAPHPLAN algorithm both generates the Planning Graph & extracts a solution from it
function GRAPHPLAN(problem) return solution or failuregraph INITIAL-PLANNING-GRAPH(problem)goals CONJUNCTS(problem. GOAL)nogoods an empty hash tablefor tl = 0 to do
if goals all non-mutex in St of graph then
solution EXTRACT-SOLUTION(graph, goals,NUMLEVELS(graph), nogoods)
if solution failure then return solutionif graph and nogoods have both leveled off then return failuregraph EXPAND-GRAPH(graph, problem)
55GRAPHPLAN Spare Tire Example
in S2, the goal literals exist, & are not mutex just 1 goal literal so obviously not mutex with any other goal
since a solution may exist, EXTRACT-SOLUTION tries to find it
EXTRACT-SOLUTION may search backwards for a solution initial state: last level of the PG + goals of the planning problem actions from Si
select a set of conflict-free actions in Ai-1 with effects covering the goals conflict free = no 2 actions are mutex & no pair of preconditions are mutex
goal: reach a state at level S0 such that all goals are satisfied cost: 1 for each action
56
GRAPHPLAN Solutions
if EXTRACT-SOLUTION fails at that point it records (level, goals) as a "no-good" subsequent calls can fail immediately if they require the same
goals at that level
we already know planning problems are computationally hard require good heuristics
greedy search with level cost of literals as a heuristic works well
1. pick literal with highest level cost 2. to achieve it, pick actions with easier preconds
action with smallest sum (or max) of level costs for its preconds
Classical Planning
57
GRAPHPLAN Solutions
alternative to backward search for a solution EXTRACT-SOLUTION could formulate a Boolean CSP
variables are actions at each level values are Boolean: an action is either in or out of the plan constraints are mutex relations & the need to satisfy each goal &
precondition
58
GRAPHPLAN Termination
does GRAPHPLAN terminate? does it stop & return failure if there's no solution the general rationale is presented here for some proof details see the textbook & its Ghallab reference
background recall that levelled off means consecutive PG levels are
identical now note that a graph may level off before a solution can be
found, on a problem for which there is a solution example of 2 airports, 1 plane & n pieces of cargo at one of
them, plan required to move all n to the other airport but only 1 piece at a time fits in the plane
graph levels off at level 4, from which full solution can't be extracted (that would require 4n - 1 steps)
we need to take account of the no-goods (goals that were not achievable) as well
these may decline even after the graph levels off
59
GRAPHPLAN Termination
does GRAPHPLAN terminate? given that no-goods might continue to decrease after the graph
levels off we now require a proof that both the graph & no-goods will
level off literals increase monotonically
once a literal appears, its persistence action causes it to stay actions increase monotonically
once a literal that is a precondition appears, the action stays mutexes decrease monotonically
though the graph simplifying conventions may not show it if 2 actions are mutex at Ai, they are also mutex at all previous levels
where they appear
60
GRAPHPLAN Termination
does GRAPHPLAN terminate? no-goods decrease monotonically
if a set of goals is not achievable at level i, they are not achievable at any previous level
so literals & actions increase monotonically given a finite number of actions and literals, they will level off
and mutexes & no-goods decrease monotonically since there cannot be fewer than 0, they must level off
if the graph & no-goods have levelled off & a goal is missing or is mutex with another goal, the algorithm can stop & return failure
Classical Planning
61
Alternative Planning Approaches
the following slides outline brief discussions of 4 alternative approaches to solving planning problems 1. convert the PDDL description into a form solvable by a
propositional logic satisfiability algorithm: SATPlan 2. convert into a situational calculus representation solvable
by First-Order Logic inference techniques 3. represent & solve as a Constraint Satisfaction Problem 4. use a "plan refinement" approach to searching in a space of
partially ordered plans
Classical Planning
62
Planning with Propositional Logic
1. form a propositional logic satisfiability problem for submission to SATPlan iteratively tries plans of increasing length (increasing number
time steps) up to the Tmax parameter value, so finds the shortest plan, if one exists
the SATPlan algorithm finds models for a (very long) PL sentence that includes initial
state, goal, successor-state axioms, precondition axioms, & action-exclusion axioms
assigns true to the actions that are part of the correct plan & false to the others
if the planning is unsolvable the sentence will be unsatisfiable any model satisfying the sentence will be a valid plan an assignment that corresponds to an incorrect plan will not be a
model because of inconsistency with the assertion that the goal is true
Classical Planning
63
Planning with Propositional Logic
translation from PDDL to PL form for SATPlan the process involves 6 steps
step 1: propositionalize actions, replacing each schema with a set of ground actions (constants substituted for variables)
step 2: define an initial state asserting F0 for all fluents in the initial state, ¬F0 for all fluents not in the initial state
step 3: propositionalize the goal - for each variable replace literals containing it with a disjunction over constants
step 4: include successor-state axioms for each fluentFt+1 ActionCausesFt (Ft ¬ActionCausesNotFt), where
ActionCausesF is disjunction of all ground actions with F in their add List
ActionCausesNotF is disjunction of all ground actions with F in their delete list
air cargo example: axioms are required for each plane, airport and time step!
Classical Planning
64
Planning with Propositional Logic
the 6 steps from PDDL to SATPlan format step 5: add precondition axioms (for every ground action A,
add At PRE(A)t
step 6: add action exclusion axioms to say every action is distinct from every other action
submit to SATPlan
Classical Planning
65
The SATPLAN Algorithm
the algorithm note that in AIMA 3e, this is first presented in the earlier
section on propositional logic agents
function SATPLAN(init, transition, goal, Tmax) returns solution or failure
inputs: init, transition, goal form a description of the planning problem Tmax, an upper limit to the plan length
for t = 0 to Tmax do
cnf TRANSLATE-TO-SAT(init, transition, goal, t )model SAT-SOLVER(cnf)if model is not null then return EXTRACT-SOLUTION(model)
return failure
Classical Planning
66
Planning with Propositional Logic
some explanatory notes distinct propositions for assertions about each time step
superscripts denote the time step: At(P1,SFO)0 At(P2,JFK)0
PDDL includes the Closed World Assumption, so conversion involves the need to specify which propositions are not true
¬At(P1,JFK)0 ¬At(P2,SFO)0
unknown propositions are left unspecified goal is associated with some particular time-step: which one?
Classical Planning
67
Planning with Propositional Logic
some explanatory notes we can't know in advance how many steps are required to
achieve a goal so, to determine the time step T where the goal will be reached, the algorithm loops until success or some arbitrary time step limit
repeat start at T = 0
Assert At(P1,JFK)0 At(P2,SFO)0
on failure: try T = 1 Assert At(P1,JFK)1 At(P2,SFO)1
add terms for next time step
until a solution is found or some max path length Tmax is exceeded: setting Tmax assures termination
Classical Planning
68
Models
the pre-condition axioms avoid models with illegal actions
e.g. for T = 1 Fly(P1, SFO, JFK)0 Fly(P1, JFK, SFO)0 Fly(P2, JFK, SFO)0
the second action is infeasible, precluded if we include the appropriate pre-condition axiom: Fly(P1,JFK,SFO)0 At(P1,JFK)0
since At(P1, JFK)0 false in initial state, Fly(P1, JFK, SFO)0 false in any model
Classical Planning
69
Models
without action-exclusion axioms a model might include actions, for example having 1 plane fly
to two destinations in a time step Fly(P1, SFO, JFK)0 Fly(P2, JFK, SFO)0 Fly(P2, JFK, LAX)0
though the second P2 action is infeasible, it requires the added action-exclusion axioms to avoid such solutions
¬(Fly(P2,JFK,SFO)0 Fly(P2,JFK,LAX)0)
Classical Planning
70
Planning in First-Order Logic
alternative 2: convert to the Situation Calculus format of FOL & apply FOL inferencing recall Situation Calculus as a topic in the 352 Knowledge
Representation discussion of time & action PDDL omits quantifiers which limits expressiveness but helps
control complexity of algorithms applied to it PL representation also has limitations like the requirement
that a "timestamp" becomes part of each fluent Situation Calculus replaces explicit timestamps by using
situations that correspond to a sequence or history of actions 1. situations
initial state is a situation if s is a situation & a an action, RESULT(s, a) is also a situation situations correspond to sequences or history of actions 2 situations are the same only if their start & actions are the
same
71
Planning in First-Order Logic
convert to FOL Situation Calculus 2. fluents are functions or relations that vary over situations
syntax: situation is conventionally the last argument At(x, l, s) is a relational fluent, true when object x is at location l in
situation s & Location(x, s) is a functional fluent such that Location(x, s) = l in the same situation
3. a possibility axiom describes preconditions of each action (s) Poss(a, s) indicates when an action can be taken is a formula giving preconditions Alive(Agent, s) Have(Agent, Arrow, s) Poss(Shoot, s)
4. successor-state axioms for each fluent indicate what happens, depending on the action taken
here are the general form & an example from Wumpus WorldAction is possible (Fluent is true in result state Action's effect made it true VIt was true before & the action left it unchanged)Poss(a, s) (At(Agent, y, Result(a, s)) a = Go(x, y) V (At(Agent, y, s) Λ a Go(y, z)))
72
Planning in First-Order Logic
express problem using FOL Situation Calculus 5. unique action axioms required to allow deductions such as
a Go(y, z) for each pair of action names Ai & Aj there's an axiom to say they
are different: Ai(x, …) Aj(y, …)
plus, for each action name Aj an axiom indicates 2 uses are only equal if all arguments are equal
Ai (x1, …, xn) = Ai(y1, …, yn) x1 = y1 … xn = yn
6. the solution is a situation (action sequence) that satisfies the goal
unfortunately, no major planning systems use situation calculus basic efficiency issues of FOL inference compounded by lack of good heuristics for planning with SC
Classical Planning
73
Planning: Constraint Satisfaction
we noticed in discussions in 352 the similarities between Constraint Satisfaction Problems &
boolean satisfiability conversion to a CSP is similar to that of conversion to a
propositional logic satisfiability problem one difference is that it only requires a single variable Actiont
at each time step: its domain is the set of all possible actions thus we also don't need the action exclusion axioms
Classical Planning
74
Partial-order planning
plans may be developed through refinement of partially ordered plans approaches describe so far, including both Progression and
Regression planning, are totally ordered forms of plan search they cannot take advantage of problem decomposition decisions must be made on how to sequence the actions for all the
subproblems though some are independent (in the air cargo problem, loading cargo at 2 different airports)
a least commitment strategy would, where possible, delay choices during search
partially ordered planning implements this adds actions to plans without committing to an absolute time/step,
instead dealing mainly with relative constraints action A must precede action B
partially ordered plans are generated by a search in the space of plans rather than through the state space
so the planning operators are different from the real-world actions
Classical Planning
75
A plan to put your shoes on
an illustration from 2e notes
Goal(RightShoeOn LeftShoeOn)Init()
Action(RightShoe,PRECOND: RightSockOnEFFECT: RightShoeOn)
Action(RightSock,PRECOND: EFFECT: RightSockOn)
Action(LeftShoe,PRECOND: LeftSockOnEFFECT: LeftShoeOn)
Action(LeftSock,PRECOND: EFFECT: LeftSockOn)
Classical Planning
76
A plan to put your shoes on
a partially-ordered planner can place two actions into a plan without specifying which comes first sample POP and corresponding totally ordered plans
77
POP as a search problem
states are (mostly unfinished) plans the empty plan contains only the special start & finish actions
each state (plan) is a partially ordered structure it consists of a set of actions (the steps of the plan) and a set of
ordering constraints of the form Before(ai, aj) meaning that one action occurs before another
graphically, actions are shown as boxes, ordering constraints as arrows
the starting point is an empty plan consisting of the initial state & goal, with no actions between them
the search process adds an action to correct a flaw in the plan or if it can't, it backtracks
flaws keep a partial plan from being a solution as an example, the next slide shows POPs corresponding to the
"flat tire" planning problem, both the empty plan and a solution
78
POP for the flat tire problem
at the top the empty plan & below it, the solution plan Remove(spare, trunk), Remove(flat, axle) can be done in either
order, so long as both are done before PutOn(spare, axle)
Classical Planning
79
POP for the flat tire problem
search in plan space seeks to add to the plan to correct a flaw flaw: not achieving At(spare, axle) in the flat tire empty plan adding PutOn(spare, axle) fixes that, but introduces new flaws
in the form of its (unachieved) preconditions process repeats, backtracking as necessary each step makes the least commitment (ordering constraints
& variable bindings) in order to fix a flaw Remove(spare, trunk) must be before PutOn(spare, axle) but
no other ordering requirements
Classical Planning
80
Partial-Order Planning
was previously popular because of explicitly representing independent subproblems but it lacks an explicit representation of states in the state-
transition model & was superseded by progression planners with heuristics that allowed discovery of independent subproblems
nevertheless it's useful for tasks like operations scheduling, enhanced with domain specific heuristics
also appropriate where humans must understand the resulting plans
plan refinement approach is easier to understand & verify example: spacecraft/rover plans are generated by this approach
Classical Planning
81
Solving POP
POP in action the initial plan contains
only the Start & Finish steps Start step/action has the initial state description as its effect Finish step/action has the goal description as its precondition
the single ordering constraint Before (Start, Finish) there's a set of open preconditions: all those in Finish
the successor function picks an open precondition p on an action B and generates a
successor plan using an action A that achieves p then does goal test
any remaining open preconditions?
Classical Planning
82
Process summary
operators on partial plans may add link from an existing action to an open precondition add a step (a new action) to fulfill an open condition order one step wrt another to remove possible conflicts
plan generation involves gradually moving from incomplete/vague plans to complete/correct plans
plan construction may require backtracking if an open condition is unachievable or conflict is
unresolvable
83
Classical Planning Summary
approaches rely on techniques of search or logic search to assemble a plan or to constructively prove one exists
handling the inherent complexity of these problems is a key identifying independent subproblems provides potential for
exponential speedup, but full decomposability is inconsistent with -ve interactions between actions
heuristically guided search can gain efficiency even if subproblems are not completely independent
some problems allow efficient solution by ruling out -ve interactions serializable subgoals = there's a subgoal ordering for a solution
that avoids having to undo any subgoals tower problems in Blocks World are serializable if subgoals are
ordered bottom to top
Classical Planning
84
Classical Planning Summary
serializable subgoals tower problems in Blocks World are serializable if subgoals are
ordered bottom to top NASA spacecraft control commands are serializable
(unsurprising since the control systems are designed for easy control)
when problem is serializable, a planner can seriously prune the planning search
what's next? where does further progress come from? probably not simply factored representations & propositional
approaches combine efficient heuristics + first-order, hierarchical
representations
Classical Planning
85
Classical Planning Summary
PDDL (Planning Domain Definition Language) initial & goal states as conjunctions of literals actions as lists of preconditions & effects
search in state space may be forward (Progression) or backward (Regression) derive heuristics from subgoal independence assumption &
relaxed versions of the problems Planning Graphs
build from the initial state to the goal with alternating incremental sets of literals or actions representing possibilities for each time step, & encoding mutexes between inconsistent pairs of literals or actions
layers are supersets of literals/actions possible at that time step the basis of good heuristics for state space searches or can
yield a plan by application of the GRAPHPLAN algorithm
Classical Planning
86
Classical Planning Summary
possible alternative approaches FOL inference on a situation calculus knowledge base conversion to boolean satisfiability problem for SATPlan encoding as a Constraint Satisfaction Problem search backwards over a space of Partially Ordered Plans
what's best? there's no agreement on a single best representation or
algorithm we saw one hybrid that performs well a particular domain may provide a best fit to one particular
technique
Recommended