View
19
Download
0
Category
Tags:
Preview:
DESCRIPTION
Learning Control Knowledge for Planning. Yi-Cheng Huang. Outline. I.Brief overview of planning II.Planning with Control knowledge III.Learning control knowledge IV.Conclusion. I. Overview of Planning. Planning - a very general framework for many applications: Robot control; - PowerPoint PPT Presentation
Citation preview
Learning Control Knowledge for Planning
Yi-Cheng Huang
Outline
I. Brief overview of planning
II. Planning with Control knowledge
III. Learning control knowledge
IV. Conclusion
I. Overview of Planning
Planning - a very general framework for many applications:Robot control;Airline scheduling;Hubble space telescope control.
Planning – find a sequence of actions that leads from an initial state to a goal state.
Planning Is Difficult –Abundance of Negative Complexity Results
Domain-independent planning: PSPACE-complete or worse (Chapman 1987; Bylander 1991; Backstrom 1993).
Domain-dependent planning: NP-complete or worse (Chenoweth 1991; Gupta and Nau 1992).
Approximate planning: NP-complete or worse (Selman 1994).
Recent State-of-the-art Planners
Constraint-based Planners – Graphplan, Blackbox.
Heuristic Search Planners – HSP, FF.
Both kinds of planners can solve problems in seconds or minutes that traditional planners take hours or days.
Graphplan (Blum & Furst, 1995)
Facts
... ...
FactsActions
...
Search on planning graph to find plan
Time i Time i+1
Blackbox (Kautz & Selman, 1999)
z)yu)(xstd)(cb)(a(
Satisfiability Tester ( Chaff ,WalkSat, Satz, RelSat, ...)
plan
problem
Heuristic Search Based Planning (Bonet & Geffner, ‘9
7)
Use various heuristic functions to approximate the distance from the current state to the goal state based on the planning graph.
Use Best-First Search or A* search to find plans.
II. Planning With Control
General focus on planning: avoid search as much as possible.
Many real-world applications are tailored and simplified by domain specific knowledge.
TLPlan is an efficient planner when using control knowledge to guild a forward-chaining search planner (Bacchus & Kabanza 2000) .
TLPlan
Temporal Logic Control Formula
A Simple Control Rule Example
Goal
Do NOT move an object at the goal location
(goal (at (obj loc)) at (obj loc))
Temporal logic operator: “always” “next”
Question:
Whether the same level of control can be effectively incorporated into constraint-based planner?
I. Rules involves only static information.
II. Rules depends on the current state.
III. Rules depends on the current state and
require dynamic user-defined predicates.
Control Rules Categories
Category I Control Rules(only depends on goal; toy example)
Do NOT unload an package from an airplane if the current location is not in the package’s goal
Goal
L
a
a
a
p)) in(a l) goal(al) at(pp) (in(a
Pruning the Planning GraphCategory I Rules
Facts FactsActions
... ...
Effect of Graph Pruning
0
2000
4000
6000
8000
10000
log-a log-b log-c log-d
nu
mb
er o
f n
od
es
Original Pruned
Category II Control Rules
L
Do NOT move an airplane if there is an object in the airplane that needs to be unloaded at that location.
a
l))) at(p l) at(pp) (in(a l) goal(a:la,(
Control by Adding Constraints
Temporal Logic Control Rules
Planning Formula Constraints Clauses
)yyx( 1iii
l))) at(p l) at(pp) (in(a l) goal(a:la,(
Rules Without Compact Encoding
NYC
SFO
ORL
Do NOT move a vehicle unless(a) there is an object that needs to be picked up(b) there is an object in the vehicle that needs to be unloaded
Goal
DC
a
a
b
b
Complex Encoding for Category III Rules
Need to define extra predicates:
need_to_move_by_airplane; need_to_unload_by_airplane Introduce extra literals and clauses.
O(mn) ground literals; O(mn+km^2) clauses at each time step.
m: #cities, n: #objects, k: #airports
No easy encoding for category III rules. However, it appears category I & II rules do m
ost of work.
Blackbox with Control Knowledge(Logistics domain with hand-coded rules)
1
10
100
1000
10000
log-a log-b log-c log-d log-e
tim
e (s
ec)
blackbox blackbox(I) blackbox(II) blackbox(I&II)
Note: Logarithmic time scale
Comparison of Blackbox and TLPlan (Run Time)
0
20
40
60
80
log-a log-b log-c log-d log-e
Tim
e (
se
c)
TLPlan Blackbox(I&II)
Comparison of Blackbox and TLPlan(parallel plan length; “plan quality”)
0
5
10
15
20
25
30
35
log-c log-d log-e log-1 log-2
Par
alle
l P
lan
Len
gth
TLPlan TLPlan-R Blackbox
Summary Adding Control Knowledge
We have shown how to add declarative control knowledge to a constraint-based planners by using temporal logic statements.
Adding such knowledge gives significant speedups (up to two orders of magnitude).
Pure heuristic search with control can be still faster but with much lower plan quality.
III. Can we learn domain knowledge from example plans?
Motivation
Control Rules used in TLPlan and Blackbox are hand-coded.
Idea: learn control rules on a sequence of small problems solved by planner.
Learning System Framework
Plan Justification / Type Inference
Blackbox Planner
Problem
ILP Learning Module / Verification
Control Rules
Target Concepts for Actions
Action Select Rule: indicate conditions under which the action can be performed immediately.
Action Reject Rule: indicate conditions under which it must not be performed.
Basic Assumption on Learning Control
Plan found by planner on simple problems are optimal or near-optimal.
Actions appear in an optimal plan must be selected.
Actions that can be executed but do not appear in the plan must be rejected.
Real action: action appears in the plan.
Virtual action: action that its preconditions are hold but does not appear in the plan.
Definition
An Toy Planning Example
GoalInitial
BOS SFONYC
Initial
a ba b
Real & Virtual Actions for UnloadAirplane
Time 1: LoadAirplane (P a BOS)Time 2: FlyAirplane (P SFO NYC) UnloadAirplane (P a BOS)Time 3: LoadAirplane (P b NYC) UnloadAirplane (P a NYC)Time 4: FlyAirplane (P NYC SFO) UnloadAirplane (P a NYC) UnloadAirplane (P b NYC)Time 5: UnloadAirplane (P a SFO) UnloadAirplane (P b SFO)
Real
Virtual
Heuristics for Extracting Examples
Select Rule Reject Rule
+ example - example + example - example
real virtual virtual real
Rule Induction
Literal: Xi = Xj , ex., loc1 = loc2 P(X1,…, Xn), ex., at (pkg, loc) goal (P(X1,…, Xn)), ex., goal (at (pkg, loc)) negation of the above
literalsaction )(
Based on Quinlan’s FOIL (Quinlan 1990; 1996).
Reject Rule: UnloadAirplane
time pln pkg apt
+ 2 P a BOS
+ 3 P a NYC
+ 4 P a NYC
+ 4 P a NYC
- 5 P a SFO
- 5 P a SFO
UnloadAirplane (pln pkg apt)
Reject Rule: UnloadAirplane
UnloadAirplane (pln pkg apt) goal(at (pkg loc))
time pln pkg apt loc
+ 2 P a BOS SFO
+ 3 P a NYC SFO
+ 4 P a NYC SFO
+ 4 P a NYC SFO
- 5 P a SFO SFO
- 5 P a SFO SFO
Reject Rule: UnloadAirplane
UnloadAirplane (pln pkg apt) goal(at (pkg loc)) ^ (apt != loc)
time pln pkg apt loc
+ 2 P a BOS SFO
+ 3 P a NYC SFO
+ 4 P a NYC SFO
+ 4 P a NYC SFO
- 5 P a SFO SFO
- 5 P a SFO SFO
Learning Time
0
10
20
30
40
50
logitics(10)
briefcase(3)
grid (6) gripper (2) mystery(6)
tireworld(5)
Tim
e (s
ec.)
Logistics Domain
1
10
100
1000
10000
100000
Problems
Tim
e (s
ec.)
w/ control w/o control
Learned Logistics Control Rules
If an object’s goal location is at different city, do NOT unload the object from airplanes.
p)) in(o c) incity(lc) incity(ml) goal(om) at(pp) (in(o
c) incity(ac) incity(ll) goal(aairport(a)a) at(tt) (in(o
Unload an object from a truck if the current location is an airport and it is not in the same city as the package’s goal location.
a)) at(o
Briefcase Domain
0.01
0.1
1
10
100
1000
10000
Problems
Tim
e (s
ec.)
w/ control w/o control
Grid Domain
1
10
100
1000
10000
100000
Problems
Tim
e (s
ec.)
w/ control w/o control
Gripper Domain
0.1
1
10
100
1000
10000
100000
Problems
Tim
e (s
ec.)
w/ control w/o control
Mystery Domain
0.1
1
10
100
1000
Problems
Tim
e (s
ec.)
w/ control w/o control
Tireworld Domain
0.1
1
10
100
1000
10000
Problems
Tim
e (s
ec.)
w/ control w/o control
Summary of Learning for Planning
Introduced inductive logic programming methodology into constraint-based planning framework to obtain “trainable planner”.
Demonstrated clear practical speedups on range of benchmark problems.
IV. Single-agent vs. Multi-agent planning
Observations: heuristic planners degrade rapidly in multi-agent settings. They tend to assign all work to a single agent.
We studied this phenomenon by exploring different work-load distributions.
Force the Planners
There is no easy way to modify the heuristic search planners to find better quality plans.
Limit the number of feature actions an agent can perform to force the planners to find plans with the same level of participation of all agents.
Sokoban Domain
Restricted Sokoban Domain
1
100
10000
1000000
sokoban-1 (4,4,4) sokoban-2 (3,3,3) sokoban-3 (5,5,5)
Blackbox HSP FF
Complexity Analysis on Restricted Domain
C.B.P H.P.
Sokoban PSPACE-Complete(Culberson, 1997)
V
Rocket NP-Complete(reduce from vertex feedback)
V
Grid Polynomial Solvable V
Elevator Polynomial Solvable V
Conclusions (a)
Demonstrated how performance of state-of-the-art general purpose planning systems can be boosted by incorporating control knowledge.
Knowledge encoded in purely declarative form using temporal logic formulas.
Obtained up to 2 orders of magnitude speedup on series of benchmarks.
Conclusions (b) Demonstrated feasibility of a “trainable” planning
system: system learns domain / control knowledge from many small example plans.
Based on concepts from inductive logic programming. Learned knowledge in temporal logic form.
First demonstration of practical speedups using learning in a planning system on realistic benchmarks.
Approach avoids learning “accidental truths” that can hurt system performance (problem in earlier systems)
Conclusions (c)
Uncovered link between performance of planners and inherent complexity of planning task.
Heuristic search planners work well on problems solvable in poly time with specialized algorithms.
Constraint-based planner dominate on NP-complete planning tasks.
Conclusion
Comparison of constraint-based planner and heuristic search planner shows that they complement each other on different domains.
Hand-coded control knowledge can be effectively applied in constraint-based planners.
Conclusion (cont.)
Our learning system is simple and modular; learning time is short.
Learned rules are on par with hand-coded ones and shown to improve the performance for over two orders of magnitude.
Learned rules are in logic form and can be used on other planning systems.
Demonstrated a way for effectively learning domain knowledge from small general plans. Learned control knowledge boosts performance on larger problems. First clear demonstration of boosting plan system performance through learning.
Declarative, logic-based approach is general and fits wide range of planning applications.
The End
Recommended