Non-Conservative Non-Conservative Cost Bound Increases Cost Bound Increases

in IDA*in IDA*

Doug DemyenDoug Demyen

OutlineOutline

► Introduction (Recap Introduction (Recap of IDA*)of IDA*)

► Advantages/ Advantages/ Disadvantages of Disadvantages of IDA*IDA*

► Alternate approachAlternate approach► Advantages/ Advantages/

DisadvantagesDisadvantages► Example: Travelling Example: Travelling

Salesman ProblemSalesman Problem

► TSP RepresentationTSP Representation► Creating the Pattern Creating the Pattern

DatabaseDatabase► Methods for Methods for

Increasing the Depth Increasing the Depth BoundBound

► Experiment Setup Experiment Setup and Parametersand Parameters

► Preliminary ResultsPreliminary Results► ConclusionConclusion

IDA* RecapIDA* Recap

►Search algorithm used for finding an Search algorithm used for finding an optimal solution to a problemoptimal solution to a problem

►Slower than A*, but requires is O(bd) Slower than A*, but requires is O(bd) memory whereas A* requires O(bmemory whereas A* requires O(bdd)) where b is the branching factor of the where b is the branching factor of the

space and d is the depth of the goalspace and d is the depth of the goal

►Usually used when there is insufficient Usually used when there is insufficient memory to run A*memory to run A*

IDA* Recap (Cont’d)IDA* Recap (Cont’d)

►For a node N, define:For a node N, define: g(N) = the distance from the start state to Ng(N) = the distance from the start state to N h(N) = an “at most” estimate of the distance h(N) = an “at most” estimate of the distance

from N to the goal statefrom N to the goal state f(N) = g(N) + h(N)f(N) = g(N) + h(N)

►Define also a Define also a depth bounddepth bound ӨӨ for IDA* for IDA*►When the algorithm begins, When the algorithm begins, ӨӨ = h(Start) = h(Start)

IDA* Recap (Cont’d)IDA* Recap (Cont’d)

►Does a depth-first search on all nodes N Does a depth-first search on all nodes N where f(N) ≤ where f(N) ≤ ӨӨ

► If the goal is not found in this iteration, If the goal is not found in this iteration, updates updates ӨӨ to be the minimum f(N) of the to be the minimum f(N) of the nodes N that were generated but not nodes N that were generated but not expanded, and searches againexpanded, and searches again

►That is, That is, ӨӨi+1i+1 := min := minNN {f(N)} {f(N)} for N for N єє {N {Nii | f(N | f(Nii) > ) > ӨӨii, N, Nii єє Succ(N Succ(Njj), f(N), f(Njj) ≤ ) ≤

ӨӨii}}

AdvantagesAdvantages

►Won’t search any deeper than the Won’t search any deeper than the goal:goal: first path found to goal is optimalfirst path found to goal is optimal avoids expanding any extra “levels”avoids expanding any extra “levels”

►Best method when a great number of Best method when a great number of nodes share the f-values (especially in nodes share the f-values (especially in an exponential state spaces)an exponential state spaces)

DisadvantagesDisadvantages

► In the worst case, each node has a In the worst case, each node has a different f-value – number of iterations different f-value – number of iterations will be O(n)will be O(n) where n is the number of nodes in the where n is the number of nodes in the

spacespace

►This is disastrousThis is disastrous► In this case, we want to update In this case, we want to update ӨӨ to to

include more than one more f-value in include more than one more f-value in each iterationeach iteration

Dangers of this ApproachDangers of this Approach

►Dangers of incrementing Dangers of incrementing ӨӨ in this way in this way stem from the possibility that in the stem from the possibility that in the last iteration, last iteration, ӨӨ > g(Goal): > g(Goal): The first time the goal is found, it might The first time the goal is found, it might

not be by an optimal pathnot be by an optimal path Searching deeper in an exponential space, Searching deeper in an exponential space,

one could expand more nodes deeper one could expand more nodes deeper than the goal than leading up to the goalthan the goal than leading up to the goal

Converging to OptimalConverging to Optimal

►Although the first path to the goal might Although the first path to the goal might not be optimal, we can find the optimal not be optimal, we can find the optimal path:path: when we find a path to the goal, when we find a path to the goal, ӨӨ := :=

g(Goal)-1g(Goal)-1 continue searching in the current iteration for continue searching in the current iteration for

other paths to the goal, updating other paths to the goal, updating ӨӨ similarly similarly when all nodes N with f(N) ≤ when all nodes N with f(N) ≤ ӨӨ have been have been

expanded, the last (shortest) path to the goal expanded, the last (shortest) path to the goal must be optimalmust be optimal

The Traveling Salesman The Traveling Salesman ProblemProblem

► One problem on which IDA* has classically One problem on which IDA* has classically performed poorly is the TSPperformed poorly is the TSP

► Involves a number of cities with a distance Involves a number of cities with a distance (or cost) between each pair(or cost) between each pair

► In the non-fully-connected problem, In the non-fully-connected problem, distances between unconnected states can distances between unconnected states can be thought of as infinitebe thought of as infinite

► The cost of traveling between two cities can The cost of traveling between two cities can be the same in both directions be the same in both directions (symmetrical) or different (asymmetrical)(symmetrical) or different (asymmetrical)

The TSP (Cont’d)The TSP (Cont’d)

► Want to visit each of the cities and return to Want to visit each of the cities and return to the starting city while incurring as little cost the starting city while incurring as little cost as possibleas possible

TSP RepresentationTSP Representation

► Distances (or costs) between cities are Distances (or costs) between cities are represented in a matrix as aboverepresented in a matrix as above

► In the symmetrical TSP the matrix is symmetricalIn the symmetrical TSP the matrix is symmetrical

From From ToTo aa bb cc dd

aa -- 44 66 77

bb 55 -- 33 88

cc 77 22 -- 99

dd 33 66 77 --

TSP Representation (Cont’d)TSP Representation (Cont’d)

►The state of the agent is defined as a The state of the agent is defined as a two-tuple of a set and an atom: ({a, two-tuple of a set and an atom: ({a, c}, b)c}, b)

►Representing the visited cities and the Representing the visited cities and the current location, respectivelycurrent location, respectively

► If we consider a to be the starting city:If we consider a to be the starting city: The start state is ({}, a)The start state is ({}, a) The goal state is ({a, b, c, d}, a)The goal state is ({a, b, c, d}, a)

Building PDBs with TSPsBuilding PDBs with TSPs

►Similarly to other problems, abstract Similarly to other problems, abstract cities within a TSP to the same constantcities within a TSP to the same constant Ex: Ex: ΦΦ: {a, b, c, d} : {a, b, c, d} {a, x, x, d} {a, x, x, d}

►When traveling from a city to another, When traveling from a city to another, take the distance to be the minimum of take the distance to be the minimum of the entries in the rows with cities the entries in the rows with cities mapped to the same constant as the mapped to the same constant as the origin city and the columns with cities origin city and the columns with cities mapped to the same constant as the mapped to the same constant as the destination citydestination city

ExampleExample

aa bb cc dd

aa -- 44 66 77

bb 55 -- 33 88

cc 77 22 -- 99

dd 33 66 77 --

aa dd

aa -- 77

dd 33 --

4

x

x 5 8

6

2

Creating the PDBCreating the PDB

►Moves are unidirectional, not invertibleMoves are unidirectional, not invertible►Easiest to enumerate the state space in Easiest to enumerate the state space in

the forward direction, then when the the forward direction, then when the goal is reached, for each node N in the goal is reached, for each node N in the path from Start path from Start Goal: Goal: h(N) := min {h(N), h(Goal) – g(N)}h(N) := min {h(N), h(Goal) – g(N)} where initially h(N) = ∞, for all nodes Nwhere initially h(N) = ∞, for all nodes N

►For this example, goal = ({a, x, x, d}, For this example, goal = ({a, x, x, d}, a)a)

Increasing the Depth BoundIncreasing the Depth Bound

►Several alternative methods have Several alternative methods have already been created for updating the already been created for updating the depth-bound:depth-bound: DFS* - double the depth bound each iterationDFS* - double the depth bound each iteration IDA*_CR - classify pruned nodes into IDA*_CR - classify pruned nodes into

“buckets” and increase the depth bound to “buckets” and increase the depth bound to include enough buckets containing a include enough buckets containing a predefined number of nodespredefined number of nodes

RIDA* - uses regression to set the depth RIDA* - uses regression to set the depth bound so the estimated increase in nodes bound so the estimated increase in nodes expanded next iteration is constantexpanded next iteration is constant

More MethodsMore Methods

► I am testing DFS* and IDA*_CR, along I am testing DFS* and IDA*_CR, along with a number of other methods:with a number of other methods: Multiply the IDA* depth bound by some Multiply the IDA* depth bound by some

constant (for example, 1.5)constant (for example, 1.5) Increase the depth bound to include a Increase the depth bound to include a

percentage of the “fringe” nodes (for percentage of the “fringe” nodes (for example, 50% = median, 100% = example, 50% = median, 100% = maximum)maximum)

Increase the depth bound to include a Increase the depth bound to include a constant number of the fringe nodesconstant number of the fringe nodes

More?More?

Experiment SetupExperiment Setup

►Currently using the 10-city TSPCurrently using the 10-city TSP►Use an abstraction for the space Use an abstraction for the space

(example: (example: ΦΦ(({abcdefghij}) = {abcdefghij}) = {VVWWXXYYZZ}){VVWWXXYYZZ})

►Populate a distance matrix randomly Populate a distance matrix randomly (either symmetrical or asymmetrical)(either symmetrical or asymmetrical)

►Enumerate the space to populate the Enumerate the space to populate the pattern databasepattern database

Experiment Setup (Cont’d)Experiment Setup (Cont’d)

►Run IDA* using each of the different Run IDA* using each of the different depth bound updating techniquesdepth bound updating techniques

►For each technique, record:For each technique, record: length of the first solutionlength of the first solution time expired and nodes expanded in time expired and nodes expanded in

reaching itreaching it time expired and nodes expanded in time expired and nodes expanded in

reaching the optimal solutionreaching the optimal solution time expired and nodes expanded by the time expired and nodes expanded by the

end of the algorithmend of the algorithm

Variables to ManipulateVariables to Manipulate

► I will try symmetrical and I will try symmetrical and asymmetrical TSPasymmetrical TSP

►Several different abstractions for the Several different abstractions for the PDBPDB

►Different parameters for methods (for Different parameters for methods (for example, include 8 fringe nodes)example, include 8 fringe nodes)

►Possibly different upper bounds on Possibly different upper bounds on inter-city distancesinter-city distances

Preliminary ResultsPreliminary Results

MethodMethod

First SolutionFirst Solution Optimal SolutionOptimal Solution FinalFinal

LengthLengthTime Time (s)(s)

# # NodesNodes

Time Time (s)(s)

# # NodesNodes

Time Time (s)(s)

# # NodesNodes

StandarStandard IDA*d IDA*

1613.71613.722

7.63877.638700

14533114533100

-- -- -- --

ӨӨminmin + + 50%50%

1839.11839.155

0.01550.015555

39213921 0.02450.024533

60006000 0.03510.035100

89358935

MedianMedian 1697.91697.933

0.03200.032033

65766576 0.03630.036388

77877787 0.04730.047300

1081810818

MaximuMaximumm

2042.02042.088

0.01600.016055

35803580 0.02860.028633

67656765 0.04020.040288

98849884

DFS*DFS* 2335.72335.788

0.01270.012788

35223522 0.04210.042155

75137513 0.05390.053933

1060510605

55thth Least Least on on

FringeFringe

1613.71613.722

5.24595.245955

10130310130300

5.25615.256155

10130310130300

5.26635.266300

10157010157000

IDA*_CR IDA*_CR (r = 5)(r = 5)

1726.61726.600

0.03330.033300

71647164 0.03800.038000

84008400 0.04990.049988

1141511415

Results so farResults so far

► Results taken from 40 runs of asymmetric Results taken from 40 runs of asymmetric 10-city TSP with a PDB using the domain 10-city TSP with a PDB using the domain abstraction: abstraction: ΦΦ({abcdefghij}) = ({abcdefghij}) = {VVWWXXYYZZ} (paired){VVWWXXYYZZ} (paired)

► DFS*, DFS*, ӨӨminmin+50%,+50%, and the maximum fringe f- and the maximum fringe f-value produce very similar results: long first value produce very similar results: long first solution paths found very quicklysolution paths found very quickly

► Interestingly, setting the depth bound to the Interestingly, setting the depth bound to the 55thth lowest fringe f-value always finds the lowest fringe f-value always finds the optimal path first, faster than IDA*optimal path first, faster than IDA*

► Other techniques form a middle ground in Other techniques form a middle ground in speed and initial solution path lengthspeed and initial solution path length

ConclusionConclusion

► In a state space like the TSP, non-In a state space like the TSP, non-conservative depth bound increments conservative depth bound increments perform much better than standard IDA*perform much better than standard IDA*

►Despite the “trade-off” between speed Despite the “trade-off” between speed and initial solution length, in my and initial solution length, in my experiments, non-conservative methods experiments, non-conservative methods still find the optimal solution more than still find the optimal solution more than 100 times after than standard IDA*100 times after than standard IDA*

►More to come . . . More to come . . .

ReferencesReferences

►B. W. Wah and Y. Shang, A B. W. Wah and Y. Shang, A Comparative Study of IDA*-Style Comparative Study of IDA*-Style Searches, Searches, Proc. 6Proc. 6thth Int’l Conference on Int’l Conference on Tools with Artificial IntelligenceTools with Artificial Intelligence, IEEE, , IEEE, Nov. 1994, pp. 290-296.Nov. 1994, pp. 290-296.

►R. E. Korf, Space-Efficient Search R. E. Korf, Space-Efficient Search Algorithms, Algorithms, ACM Computing Surveys ACM Computing Surveys (CSUR)(CSUR), Sept. 1995, pp. 337-339., Sept. 1995, pp. 337-339.

Questions?Questions?