8) Explain the concept of planning with state space search using suitable examples. Or(b) Explain the use of planning graphs in providing better heuristic estimates with suitable examples.

PLANNING WITH STATE-SPACE SEARCH

Planning Algorithms: The most straightforward approach is to use state-space search. Because the descriptions of actions in a planning problem specify both preconditions and effects, it is possible to search in either direction: either forward from the initial state or backward from the goal. We can also use the explicit action and goal representations to derive effective heuristics automatically.

Forward state-space search

Planning with forward state-space search is similar to the problem-solving approach . It is sometimes called progression planning, because it moves in the forward direction.

We start in the problem's initial state, considering sequences of actions until we find a sequence that reaches a goal state. The formulation of planning problems as state-space search problems is as follows:

The initial state of the search is the initial state from the planning problem. In general, each state will be a set of positive ground literals; literals not appearing are false.

The actions that are applicable to a state are all those whose preconditions are satisfied. The successor state resulting from an action is generated by adding the positive effect

literals and deleting the negative effect literals. The goal test checks whether the state satisfies the goal of the planning problem. The step cost of each action is typically 1. Although it would be easy to allow different

costs for different actions, this is seldom done by STRIPS planners. Recall that, in the absence of function symbols, the state space of a planning problem is

finite. Therefore, any graph search algorithm that is complete-for example, A*-will be a

complete planning algorithm.Disadvantage

Forward search does not address the irrelevant action problem-all applicable actions are considered from each state.

Second, the approach quickly bogs down without a good heuristic. requires a large amount of space

Backward Planning It is also possible to search backward from the set of states that satisfy the goal.

This approach is not practical when a large number of states satisfy the goal. Example: in the block problem a very large number of states satisfy the goal on(a,b).

Searching backwards is sometimes called regression planning

The main advantage of backward search is that it allows us to consider only relevant actions.

An action is relevant to a conjunctive goal if it achieves one of the conjuncts of thegoal. For example, the goal in our 10-airport air cargo problem is to have 20 pieces of cargoat airport B, or more precisely, At(C1,B ) A At(C2,B )A . . . At(C2o,B ) .

Now consider the conjunct At (C1,B, ).Working backwards, we can seek actions that havethis as an effect. There is only one: Unload (C1,p, , B),w here plane p is unspecified.Consistent Action

The purpose of applying an action is to ‘achieves a desired literal’ We should be careful that the action does not undo a desired literal (as a side effect) A consistent action is an action that does not undo a desired literal

example, the action Load(C2, p) would not be consistent with the current goal, because itwould negate the literal At (C2,B ).

Heuristic to Speed up Search1. Divide-and-conquer: sub-goal independence assumption

• Problem relaxation by removing2. all preconditions3. all preconditions and negative effects4. negative effects only: Empty-Delete-List

Subgoal Independence AssumptionThe cost of solving a conjunction of subgoals is the sum of the costs of solving each subgoal independently

• Optimistic– Where subplans interact negatively– Example: one action in a subplan delete goal achieved by an action in another

subplan • Pessimistic (not admissible)

Redundant actions in subplans can be replaced by a single action in merged planThe cost of solving a conjunction of subgoals is the sum of the costs of solving each subgoal independently

Problem Relaxation: Removing Preconditions Remove preconditions from action descriptions

All actions are applicable Every literal in the goal is achievable in one step

Given A goal G descriptionAn action A that is relevant and consistent

Generate a predecessor state wherePositive effects (literals) of A in G are deletedPrecondition literals of A are added unless they already appearSubstituting any variables in A’s effects to match literals in G Substituting any variables in A’s preconditions to match substitutions in A’s effects

Repeat until predecessor description matches initial state

Number of steps to achieve the conjunction of literals in the goal is equal to the number of unsatisfied literals

Alert Some actions may achieve several literals Some action may remove the effect of another action

Remove Preconditions & Negative Effects• Considers only positive interactions among actions to achieve multiple subgoals • The minimum number of actions required is the sum of the union of the actions’ positive

effects that satisfy the goal• The problem is reduced to a set cover problem, which is NP-hard

– Approximation by a greedy algorithm cannot guarantee an admissible heuristicRemoving Negative Effects

• Remove all negative effects of actions (no action may destroy the effects of another)• Known as the Empty-Delete-List heuristic• Requires running a simple planning algorithm• Quick & effective• Usable in progression or regression planning

2 What is Partial Order Planning?

Partial-Order Planning

A planner should be able to come up with the two-action sequence Rightsock followed by Rightshoe to achieve the first conjunct of the goal and the sequence Leftsock followed by LeftShoe for the second conjunct. Then the two sequences can be combined to yield the final plan. In doing this, the planner will be manipulating the two subsequences independently, without committing to whether an action in one sequence is before or after an action in the other. Any planning algorithm that can place two actions into a plan without specifying which PLANNER comes first is called a partial-order planner

The solution is represented as a graph of actions, not a sequence. The "dummy" actions called Start and Finish, which mark the beginning and end of the

plan The partial-order solution corresponds to six possible LINEARIZATION total-order

plans; each of these is called a linearization of the partial-order plan A set of actions that make up the steps of the plan. These are taken from the set of

actions in the planning problem. The "empty" plan contains just the Start and Finish actions. Start has no preconditions and has as its effect all the literals in the initial state of the planning problem. Finish has no effects and has as its preconditions the goal literals of the planning problem.

State-space searchYields totally ordered plans (linear plans)POP Works on subproblems independently, then combines subplansExampleGoal(RightShoeOn LeftShoeOn)Init()Action(RightShoe, PRECOND: RightSockOn, EFFECT: RightShoeOn)Action(RightSock, EFFECT: RightSockOn)Action(LeftShoe, PRECOND: LeftSockOn, EFFECT: LeftShoeOn)Action(LeftSock, EFFECT: LeftSockOn)

Components of a Pla

Consistent Plan

Consistent plan is a plan that hasNo cycle in the ordering constraintsNo conflicts with the causal links

SolutionIs a consistent plan with no open preconditions

To solve a conflict between a causal link A B and an action C (that clobbers, threatens the causal link), we force C to occur outside the “protection interval” by adding the constraint C p A (demoting C) or the constraint B p C (promoting C)

A set of actionsA set of ordering constraints A p B reads “A before B” but not necessarily immediately before BAlert: caution to cycles A p B and B p AA set of causal links (protection intervals) between actionsA B reads “A achieves p for B” and p must remain true from the time A is applied to the time B is appliedExample “RightSock RightShoe

A set of open preconditionsPlanners work to reduce the set of open preconditions to the empty set w/o introducing contradictions

Setting up the PoP

POP as a Search Problem

Example of POP: Flat tire problem

The successor function arbitrarily picks one open precondition p on an action BFor every possible consistent action A that achieves pIt generates a successor plan adding the causal link A B and the ordering constraint A p BIf A was not in the plan, it adds Start p A and A p FinishIt resolves all conflicts between the new causal link and all existing actions between A and all existing causal linksThen it adds the successor states for combination of resolved conflicts

It repeats until no open precondition exists

Add dummy states StartHas no preconditionsIts effects are the literals of the initial stateFinishIts preconditions are the literals of the goal stateHas no effects

Initial Plan:Actions: {Start, Finish}Ordering constraints: {Start p Finish}Causal links: {}Open Preconditions: {LeftShoeOn,RightShoeOn}

. The sequence of events is as follows:1. Pick the only open precondition, At(Spare, Axle) of Finish. Choose the only applicable action, Put On (Spare, Axle).2. Pick the At (Spare, Ground) precondition of PutOn(S$are, Axle). Choose the only applicable action, Remove(Spare, Trunk) to achieve it. The resulting plan is shown in

3. Pick the lAt(Flat, Axle) precondition of PutOn(Spare, Axle). Just to be contrary,choose the LeaveOvernight action rather than the Remove(Flat, Axle) action. Noticethat LeaveOvernight also has the effect lAt(Spare, Ground), which means it conflictswith the causal linkRenove(Spare, Trunk) At(spare,Ground) P utOn(Spare,A xle) .

To resolve the conflict we add an ordering constraint putting LeaveOvernight before Remove(Spare, Trunk). The resulting plan is shown in4 The only remaining open precondition at this point is the At (Spare, Punk) precondition of the action Remove(Spare, Trunk). The only action that can achieve it is the existing Start action, but the causal link from Start to Remove(Spare, Trunk) is in conflict with the ~At (Spar eT, r unk) effect of LeaveOvernight. This time there is no way to resolve the conflict with LeaveOvernight: we cannot order it before Start (because nothing can come before Start), and we cannot order it after Remove(Spare, 5. Consider again the lAt(Flat, Axle) precondition of lDutOn(Spare, Axle). This time, we choose Remove(Flat, Axle).6. Once again, pick the At(Spare, Trunk) precondition of Remove(Spare, Trunk) and choose Start to achieve it. This time there are no conflicts.7. Pick the At(Flat, Axle) precondition of Remove(Flat, Axle), and choose Start to achieve it. This gives us a complete, consistent. plan- in other words a solution-as shown in Trunk) (because there is already a constraint ordering it before Remove(Spare, Trunk)). Sowe are forced to back up, remove the LeaveOvernzght action and the last two causa links, and return to the state in Figure 11.8. In essence, the planner has proved thatLeaveOvernight doesn't work as a way to change a tire.

Planning Graph• Is special data structure used for

1. Deriving better heuristic estimates2. Extract a solution to the planning problem: GraphPlan algorithm

• Is a sequence áS0,A0,S1,A1,…,Siñ of levels1. Alternating state levels & action levels2. Levels correspond to time stamps3. Starting at initial state4. State level is a set of (propositional) literals

• All those literals that could be true at that level5. Action level is a set of (propositionalized) actions

• All those actions whose preconditions appear in the state level (ignoring all negative interactions, etc.)

• Propositionalization may yield combinatorial explosition in the presence of a large number of objects

Steps involved in the planning Graph• Building the Planning Graph• Using it for Heuristic Estimation• Using it for generating the plan•

At each state level, list all literals that may hold at that level

Init(Have(Cake))Goal(Have(Cake)Eaten(Cake))

Action(Eat(Cake)Precond: Have(Cake)Effect:

Have(Cake)Eaten(Cake))

Action(Bake(Cake)Precond: Have(Cake)Effect: Have(Cake))

Propositions true at the initial state

Action is connected to its preconds & effects

Persistence Actions (noop)

• At each action level, list all noops & all actions whose preconditions may hold at previous levels

• Repeat until plan ‘levels off,’ no new literals appears (Si=Si+1)• Building the Planning Graph is a polynomial process • Add (binary) mutual exclusion (mutex) links between conflicting actions and between

conflicting literals

Mutex Links between Actions1. Inconsistent effects: one action negates an effect of another

– Eat(Cake) & noop of Have(Cake) disagree on effect Have(Cake)2. Interference: An action effect negates the precondition of another

– Eat(Cake) negates precondition of the noop of Have(Cake): 3. Competing needs: A precondition on an action is mutex with the precondition of another

– Bake(Cake) & Eat(Cake): compete on Have(Cake) preconditionMutex Links between Literals

Mutex Links between Literals

Mutual exclusion links

S1 represents multiple states

Planning Graph for Heuristic Estimation

• A literal that does not appear in the final level cannot be achieved by any plan– State-space search: Any state containing an unachievable literal has cost h(n)=µ – POP: Any plan with an unachievable open condition has cost h(n)=µ

• The estimate cost of any goal literal is the first level at which it appears– Estimate is admissible for individual literals– Estimate can be improved by serializing the graph (serial planning graph: one

action per level) by adding mutex between all actions in a given level• The estimate of a conjunction of goal literals

– Three heuristics: max level, level sum, set level • A literal that does not appear in the final level cannot be achieved by any plan

– State-space search: Any state containing an unachievable literal has cost h(n)=µ – POP: Any plan with an unachievable open condition has cost h(n)=µ

• The estimate cost of any goal literal is the first level at which it appears– Estimate is admissible for individual literals– Estimate can be improved by serializing the graph (serial planning graph: one

action per level) by adding mutex between all actions in a given level• The estimate of a conjunction of goal literals

– Three heuristics: max level, level sum, set level Estimate of Conjunction of Goal Literals

• Max-level– The largest level of a literal in the conjunction– Admissible, not very accurate

• Level sum – Under subgoal independence assumption, sums the level costs of the literals– Inadmissible, works well for largely decomposable problems

• Set level– Finds the level at which all literals appear w/o any pair of them being mutex – Dominates max-level, works extremely well on problems where there is a great

deal of interaction among subplans GraphPlan algorithm

Example of GraphPlan Execution (1)

Example of GraphPlan Execution (2)

At(Spare,Axle) is not in S0

No need to extract solutionExpand the plan

GRAPHPLAN(problem) returns solution or failuregraph INITIALPLANNINGGRAPH(problem)goals GOALS[problem]loop do if goals all non-mutex in last level of graph then do solution

EXTRACTSOLUTION(graph,goals,LENGTH(graph)) if solution failure then return solution else if NOSOLUTIONPOSSIBLE(graph) then return failure graph EXPANDGRAPH (graph,problem)

Two main stagesExtract solutionExpand the graph

Example of GraphPlan Execution (3)

Solution Extraction (Backward)

Illustrates well mutex links: inconsistent effects, interference, competing needs, inconsistent support

Three actions are applicable3 actions and 5 noops are addedMutex links are addedAt(Spare,Axle) still not in S1

Plan is expanded

Backtrack Search for Solution Extraction

Termination of GraphPlan

Starting at the highest fact levelEach goal is put in a goal list for the current fact layerSearch iterates thru each fact in the goal list trying to find an action to support it which is not mutex with any other chosen actionWhen an action is chosen, its preconditions are added to the goal list of the lower levelWhen all facts in the goal list of the current level have a consistent assignment of actions, the search moves to the next levelSearch backtracks to the previous level when it fails to assign an action to each fact in the goal list at a given level

Search succeeds when the first level is reached.

Solve a Boolean CSP: Variables are actions, domains are {0=out of plan, 1=in plan), constraints are mutexSearch problem from last level backward

Optimality of GraphPlan• The plans generated by GraphPlan

– Are optimal in the number of steps needed to execute the plan– Not necessarily optimal in the number of actions in the plan (GraphPlan produces

partially ordered plans)

4 Explain the component necessary to define a problemA Problem can be defined formally by four components:

The initial state that the agent starts in. A description of the possible actions available to the agent. The most common

formulation uses a successor function. Given a particular state x, SUCCESSORFN(x) returns a set of < action, successor > ordered pairs, reachable from x by any single action.

Together, the initial state and successor function implicitly define the state space of the problem- the set of all states reachable from the initial state. The state space forms a graph in which the nodes are states and the arcs between nodes and actions.

A path in the state space is a sequence of states connected by a sequence of actions. The goal test, which determines whether a given state is a goal state. If more than one

goal state exists, then we can check whether any on of the goal state is reached or not.

A path cost function that assigns a numeric cost to each path. The cost of a path can be described as the sum of the costs of the individual actions along the path.

The step cost of taking action a to go from x to state y is denoted by c(x, a, y).The preceding elements define a problem can be gathered together into a single data

GRAPHPLAN is guaranteed to terminateLiteral increase monotonicallyActions increase monotonicallyMutexes decrease monotinicallyA solution is guaranteed not to exist whenThe graph levels off with all goals present & non-mutex, andEXTRACTSOLUTION fails to find solution

structure that is given as input to a problem-solving algorithm. A solution to a problem is a path from the initial state to a goal state. Solution quality is measured by the path cost function, and an optimal solution has the lowest path cost among all solutions.Formulating problemsWe derive a formulation of the problem in terms of the initial state, successor function, goal test, and path cost.

The process of removing detail from a representation is called abstraction. In addition to abstracting the state description, we must abstract the actions themselves.data type PROBLEMComponents: initial state, successor function, goal-test, path-cost.1.8 Example ProblemsToy problemsi. The 8 – puzzle problem

The 8 – puzzle consists of a 3 × 3 board with eight numbered tiles and a blank space a shown in figure . A tile adjacent to the blank space can slide into the space. The object is to reach a specified goal state. The standard formulation is as follows:

States: A state description specifies the location of each of the eight tiles and the blank in one of the nine squares.

Initial state: Any state can be designated as the initial state. Successor function: This generates the legal states that result from trying the four

actions (blank moves Left, Right, Up and Down). Goal test: This checks whether the state matches the goal configuration. Path cost: Each step costs 1, so the path cost is the number of steps in the path

(length of the path).

Two marks

1. What is Partial-Order PlanningA set of actions that make up the steps of the plan. These are taken from the set of actions

in the planning problem. The “empty” plan contains just the Start and Finish actions.

Start has no preconditions and has as its effect all the literals in the initial state of the

planning problem. Finish has no effects and has as its preconditions the goal literals of

the planning problem.

. Any planning algorithm that can place two actions into a plan without specifying which

PLANNER comes first is called a partial-order planner

2. What are the components that are needed for representing an actionThe components that are needed for representing an action are,

Action description.

Precondition;

Effect.

3. What is a consistent planA consistent plan is one in which there are no contradictions in the ordering or binding

constraints.

Consistent plan as a plan in which there are no cycles in the ordering constraints and no conflicts

with the causal links. A consistent plan with no open precondition is a solution.

4. Define Contingency planningA planning technique , which deals with incomplete ) information , in which the agent

finds out which part of the plan to execute by including sensing actions in the plan to test

for the appropriate conditions.