Hybrid Search for Generalized Plans Using Classical PlannersSiddharth SrivastavaUniversity of Massachusetts Amherst

(Joint work with Neil Immerman, Shlomo Zilberstein, and Tianjiao Zhang)

What is Generalized Planning?• Broadly applicable plans, procedures or workflows

with “rich” representations

Automated generation ofsafe controllers

Automated service compositionProgramming by demonstration

(c) Robert Sedgewick and Kevin Wayne

Generalized Planning Problem•Given: a domain, set of initial states from

potentially different state spaces, goal formula

•Find: A generalized plan that solves all of the initial states

X X X X

Grid TraversalReversing a singly linked list… algorithm synthesis problems

Other Approaches•Plan reuse [Fikes et al. ’72, Hammond ’86]

•Plans with loops▫Recognize repetitive patterns in plans [Hu and

Levesque ’10, Winner and Veloso ’07, Levesque ’05]▫Search for cyclic controllers for solving a

partially observable instance [Bonet et al., ’09] Plan termination/correctness not addressed Generalization cannot be easily directed towards a

given set of problems•Our objectives:

▫Termination, correctness, directed generalization

Our Approach

States

States = logical structures in a domain’s FO(TC)

vocabulary

V = {on, onTable, clear, on+}1

2

3

4

on 1 2 3 4

1 0 1 0 0

2 0 0 1 0

3 0 0 0 1

4 0 0 0 0

a

b

c

on 1 2 3 4

1 0 1 0 0

2 0 0 1 0

3 0 0 0 1

4 0 0 0 0

0 ½ ½ ½

0

on a b c

a 0 ½ 0

b 0 ½ ½

c 0 0 0

• Binary relationships with summary elements may become imprecise

• Represented using the truth value ½ , denoting “unknown”• Integrity constraints clarify the sets of states represented

by abstract states[AI Journal 2011]

Define:Role of an element = set of unary predicates it

satisfies

Action Application on Abstract States

• Branch depends on #elements (role-count) in the middle• In general, may need to draw out representative elements

prior to action application▫ This operation, in combination with changing the drawn

element’s role is sufficient to express any computation (abacus programs)

arg1mvToTable(arg1)

Choose x: clear(x) & onTable(x)

x

x

[ICAPS 2010; AI Journal 2011]

Generalized Plan Synthesis

Generalized Plans: Definition

•Connected, directed graph•Nodes = actions•Edges = conditions/ “abstract” states

Will use the dual representation in algorithms

•Start/Terminal nodes

A special class of finite state transducers

Recall: Generalized Planning Problem

•Given: a domain (vocabulary + actions + integrity constraints), set of initial states (abstract state S0), goal formula (FO(TC))

•Find: A generalized plan that solves every initial state represented by S0

•“Solves”: Every possible execution reaches the goal in a finite number of steps [Srivastava et al., AIJ-11]

Hybrid Search: Idea

<mvToTable(b1), mvToTable(b2)>

Classical Planner??

Objects b1 and b2 don’t exist in the general problem,or in the abstract state

Start node (Initial abstract state)

Dual Representation:

Nodes = Abstract StatesEdges = Actions

Hybrid Search: Idea

<mvToTable(b1), mvToTable(b2)>

choose x: role(x) = role(b1)mvToTable(x)choose x: role(x) = role(b2) mvToTable(x)

Add choice actions

Classical Planner

mTT(x)

mTT(x)

Goal State!

• Some states represented by the initial abstract state reach the goal along this path.

• Others end up at the “open” node

`

Open node:Terminal, non-goal

Hybrid Search: Idea

<mvToTable(b1), mvToTable(b2)>

choose x: role(x) = role(b1)mvToTable(x)choose x: role(x) = role(b2) mvToTable(x)

Add choice actions

Classical Planner

mTT(x)

mTT(x)

Goal State!

mTT(x)

mTT(x)

` Overall Algorithm:

• Repeat this process with new open nodes

Hybrid Search: Idea

<mvToTable(b1), mvToTable(b2)>

Classical Planner

mTT(x)

mTT(x)

Goal State!

mTT(x)

mTT(x)

choose x: role(x) = role(b1)mvToTable(x)choose x: role(x) = role(b2) mvToTable(x)

Add choice actions

mTT(x)

mTT(x)

`

Hybrid Search: Idea

mTT(x)

mTT(x)

Goal State!

mTT(x)

mTT(x) mTT(x)

mTT(x)

Overall Algorithm:

• Repeat this process with new open nodes

• Merge an abstract state with an existing one that subsumes it:• Loops must be

guaranteed to terminate [ICAPS 2010]

Hybrid Search: OptimizationsTwo main optimizations:

• If any open node is subsumed by an existing node, AND if it can be merged with an existing node while guaranteeing termination, merge it.

• If a previously generated instance is subsumed by the current open node, use the plan generated earlier.

Only two examples would be needed to solve the previous problem

Generalized Plan Synthesis: Hybrid Approach

Gen Plan w/ Branches and Loops

Get open node

Unsolved AbstractState

Classical Problem Instance

Classical Plan

Linear Gen Plan

Gen Plan w/ Branches

Generate Instance

Classical Planner

Add Choice AxnsTrace

Merge

a1(o1), a2(o2), a3…

ch(x:…), a1(x), ch(x:…), a2(x),…

Open node

New open node

Open node

Components of Hybrid Search: Instance Generation

on+

on

clear

onTable

a

b

c

on a b c

a 0 ½ 0

b 0 ½ ½

c 0 0 0

Write FO(TC) formulas for:1. Roles present: e.g. 2. Singletons present: e.g. 3. Predicate tuples that are true : e.g. on+(a,b) 4. Predicate tuples that are false: e.g. on(a,c)5. Integrity constraints: e.g.

½’s remain unspecified

Instance Generation

•First-order model generators are available, but our formulas use transitive closure (TC)

•TC cannot be expressed in first-order•Simulate it using:

This is accurate in acyclic models [Lev Ami et al., 2009]

Results

• At most 10 planning problems required to generate generalized plans for unbounded instances

• Problem instances to be solved by classical planners are “small” and deterministic

• Unified approach for algorithm and generalized plan synthesis

Conclusions• An approach for utilizing the powerful heuristic

search capabilities of classical planners for: Simultaneous exploration of infinitely many state

spaces using abstraction Generation of potentially useful paths of actions in

the abstract state space• Main questions addressed:

How to determine the current class of open problems during generalization

How to generate small instances of this class How to merge plans into a generalized plan with

well-defined executions: no open node in the plan complete, terminating solution!

Thank you!

Sets of States as Abstract States

clear

onTable

Define:Role of an element = set of unary predicates it

satisfies

“Summary” elements …. Represent collections

Abstract States: Truth values

• Binary relationships with summary elements may become imprecise

• Represented using the truth value ½ , denoting “unknown”• Integrity constraints clarify the sets of states represented

by abstract states [AI Journal 2011]

a

b

c

on 1 2 3 4

1 0 1 0 0

2 0 0 1 0

3 0 0 0 1

4 0 0 0 0

1

2

3

4

0 ½ ½

on a b c

a 0 ½ 0

b 0 ½ ½

c 0 0 0

½

0