Notes for CS3310Artificial Intelligence

Part 8: Heuristic search

Prof. Neil C. RoweNaval Postgraduate School

Version of January 2006

Search• Search = reasoning about "states".• A state is a set of facts true at some instant of time.

That is, a state is a snapshot of facts. (Or to save space, just the relevant facts in some problem.)

• Search and states permit reasoning about time.• In artificial intelligence, states are discrete, and

transitions between states ("branches") are discrete (all at once) too.

• Search among states permits hypothetical reasoning about effects of actions. Each possible action can result in a different state.

• A state can be represented by a linked list. Possible state transitions can be represented by a directed graph, a "search graph".

• Search is the oldest area of artificial intelligence.

More search terms• State: a set of facts (whose predicate names are usually past

participles, e.g. "opened", and abstract nouns, e.g. "location").• Successor state: an immediate next state.• Operator: a label on a state transition (usually a verb, e.g.

"open")• Starting state: the first state of a search.• Goal states: states in which objectives have been achieved, so

search can stop.• Level: the number of state transitions from the starting state to

a given state.• Preconditions of an operator: facts that must occur in a state

before you can use the operator to it.• Postconditions of an operator: facts that become true after you

use the operator from a state.

Search examples• Intercity route planning: given a road map, find the best route

between two cities for a car.• City route planning: Given intersections A and B, find a route

between them for a car.• Car repair: Given part to repair, access it, fix it, and put the car

back together.• Smarter forward chaining: Forward chaining but making an

intelligent decision at each step about which fact to pursue next. (The goal is to infer interesting facts sooner and save time.)

• Scheduling classes: At a university, assign classes to times so every student can take the classes they want.

• Mission planning: Given a model of targets and costs of attacking them, and a model of enemy responses, figure the best sequence of attacking actions.

Search example

The state is in brackets; air-mileage to San Francisco in curly brackets; mileages in parentheses; road names are the operators. Prefer westmost branch. Compute order of visit and solution path: Use abbreviations: M=Monterey, SC=Santa Cruz, G=Gilroy, SJ=San Jose, SF=San Francisco, Oak=Oakland.

[location(San Francisco)] {0}

[location(Santa Cruz)] {65}

[location(San Jose)] {45}

[location(Oakland)] {15}

[location(Gilroy)] {74}

[location(Monterey)] {100}

I-80 (20)

Cal-1 (85)

Cal-17 (30)

I-280 (50) Cal-17 (50)US-101 (35)

Cal-1 (45) US-101 (35)

Work page for comparison of search algorithms

Search method

States visited

Agenda Solution

Depth- first

Breadth-first

Best-first

A*

A search graph for car repair

Complete the search graph to the goal state [ok(battery), in(battery,car), cabled(battery), bolted(battery)]. Possible operators: "attach cables", "attach bolts", and "replace battery".

Note: not(cabled(battery)) is a precondition of the action "remove battery"; ok(battery) is a postcondition of the action "replace battery"; facts not mentioned are assumed false in a state.

[dead(battery), in(battery,car), bolted(battery), cabled(battery)]

[dead(battery), in(battery,car), cabled(battery)]

[dead(battery), in(battery,car), bolted(battery)]

[dead(battery), in(battery,car)]

remove bolts remove cables

remove cables

remove bolts

Work page for car battery problem

Classic search methods• These differ in the order in which states are "visited". "Visit”

means "find the successors of." • Depth-first: visit an unvisited successor of current state, else

back up and try a new successor of previous state.• Breadth-first: visit next-found unvisited state at same level,

else the first-found state at next level.• Hill climbing: visit the unvisited successor of current state

which has lowest evaluation-function value, else back up.• Best-first: visit the known unvisited state with lowest

evaluation-function value, which can be anywhere in search graph.

• Branch-and-bound: like best-first but use cost from start to state.

• A*: like best-first but use sum of evaluation-function value and cost from start to state.

Notes on the search methods• Search stops when you visit a goal state (that is, when you try to

find its successors).• Heuristics can give criteria for choosing the next state with all the

search methods (and break ties in best-first and A*).• Never visit the same state twice (or add a duplicate state to the

agenda) -- but for A*, consider new routes to the same state.• Depth-first can be implemented with a stack; breadth-first with a

queue; and the other methods with a sorted "agenda" of states known but not yet visited.

• When you find successors of a state, set the "backpointer" for each successor to point to the state. When done, follow backpointers from the goal state to the start state to get the solution path.

• Agendas should store for each state its description (list of facts true for it), its backpointer, and any numbers associated with the state.

• With A* search, if the evaluation function is a lower bound on the subsequent cost, the first path found to a state is guaranteed to be the best path to that state.

Heuristics

= Any piece of nonnumeric advice for choosing among possible branches in a search.

Heuristics are a form of meta-rules. Examples:

For city route planning, prefer not to turn twice in succession.

For car repair, do work from top of the engine before work from below the engine.

For buying decisions, don't buy anything advertised on television.

In mission planning, withdraw from an engagement if you do not have at least 2-to-1 superiority in assets.

Search terms that are easy to confuse• evaluation function: a "distance-to-goal" measure, a

number. A function of the state. Makes search more intelligent, if you pick direction to search with lowest evaluation function.

• heuristic: a non-numeric way for choosing a direction for search at a state.

• cost function: a "cost-accumulated" measure, a number. A function of the states in a path.

Note: Evaluation function concerns future; cost function concerns past.

An example route-planning program, not so goodMicrosoft Automap Road Atlas Route Planning DemoThe quickest route from Monterey, California, to San Francisco, California (going the speed limit), is as

follows:Total Time: 2 hours 31 minutes Total Distance: 121 milesTime Road For Dir Towards00:00 DEPART Monterey (California) 2 miles E on the Fremont St00:04 Go onto S1 17 miles E Santa Cruz00:23 At Castroville stay on the S1 14 miles N Santa Cruz00:39 At Freedom stay on the S1 15 miles NW Santa Cruz00:55 At Santa Cruz turn S9 23 miles N Saratoga right onto01:23 Turn left onto S35 14 miles N Woodside01:40 Turn right onto S84 5 miles E Redwood City01:46 At Woodside stay on the S84 1/2 mile NE Redwood City01:47 Turn left onto I280 16 miles W San Mateo02:07 At Burlingame stay on the I280 1 mile N Brisbane02:09 Turn off onto I380 1 mile E San Bruno02:10 At San Bruno stay on the I380 1 mile E02:12 Turn off onto U101 3 miles N South San Francisco02:17 At South San Francisco U101 3 miles NE Brisbane stay on the02:22 At Brisbane stay on the U101 3 miles N02:26 Turn off onto I80 3 miles E San Francisco02:31 ARRIVE San Francisco(California)If you are impressed with this demo, you'll be wowed by our full product, Microsoft Automap Road Atlas 4.0.

We invite you to join our mailing list, read more information about Automap Road Atlas, or give us feedback on this demo.

More about evaluation functionsWhat if states aren’t locations? Then add "amount-of-work-

necessary" numbers for certain facts to get the evaluation-function value. For instance for the car repair, start at 0 and then:

Add 10 if a dead(battery) fact is in the state;Add 5 if no bolted(battery) fact is in the state;Add 3 if no cabled(battery) fact is in the state.

So for instance:[dead(battery), in(battery,car), bolted(battery), cabled(battery)] has

evaluation 10[dead(battery), in(battery,car), cabled(battery)] has evaluation 15[dead(battery), in(battery,car)] has evaluation 18[ok(battery), in(battery,car)] has evaluation 8[ok(battery), in(battery(car), cabled(battery)] has evaluation 5[ok(battery), in(battery(car), bolted(battery), cabled(battery)] has 0

Problems with evaluation functionsEvaluation functions don't always work well. Some

classic problems:– "pond" problem: you may find a local (instead of

global) minimum– "valley" problem: most directions may be worse,

and it may be hard to find the few directions that are better

– "plateau" problem: you may have lots of ties in the evaluation function value

– "pit" problem: evaluation function may not be useful unless you're extremely close to the goal

In these case a set of heuristics may be better.

Another kind of cost function: The probability of an interpretation of a situation

Overall probability can be product of near-independent factors. So to find the best interpretation, minimize its logarithm:

Do this by an A* search where branches cost:

This is important in language understanding. For instance, interpret "navy fighter wing" with probabilities:

of each word sense of "navy” ; of each word sense of "fighter”; of each word sense of "wing”; that "navy" is a kind of "fighter”;that "navy" is part of "fighter”; that "fighter" is part of "navy”;that "navy" is owner of "fighter”; that "fighter" is a kind of "wing”; that "wing" is a part of "fighter”that "wing" is a kind of "fighter”. In a goal state each word has a sense and all senses are linked by

relationships.

)log( ip1 2 3

1log( ...) log( )

n

ii

p p p p

Stochastic state graph for rootkit-installation example5

39

25

install rootkit

close ftp

50

0.2

0.9

0.1

0.2

0.8

0.2

0.8

0.83

0.170.4

0.9

0

33

22 1 2 3 4

35

6

78

9

ping

pingresearchvulnerabilities

researchvulnerabilities scan open port

overflowbuffer

become admin

become admin

login

ftp

24

25

download rootkit

download rootkit download secureport

download secureport

10

11

122

13

14 15 16 17 18 19

download secureport

close ftp

decompress rootkit

install secureport

install rootkit

logout

21 22 23 28close ftp

decompress secureport

install secureport

test rootkit

test rootkit

test rootkit

test rootkit

test rootkit

26

close ftp

27

29

30

decompress secureport

decompress secureport

decompress rootkitdecompress

rootkit

install rootkit

32

31

ftp

download secureport

download secureport

180

70

167

107

61

36

101 10099

97

10075

7462

96

8685

4874

65

55

47 30 11

36 24 1 0

0.33

0.8

0.67 0.17

0.80.2

34

test rootkit

ftp

0.25 0.75

40

Numbers in boxes are state numbers.Integers outside boxes are duration from goal state.Decimals are transition probabilities in 10 runs.decompress

secureport

close port

close port

0.83

20

55

51

0.1

0.9

0.1

0.6

Definitions needed to solve a search problem

(1) successor: Defines all possible state transitions (with successor(State,Newstate) in Prolog; in Java, successor(State) which returns the array Newstates).

(2) goalreached: Defines all possible goal states.(3) eval (for best-first and A*): Returns the evaluation

function value for any state.(4) piece_cost (for A*): Returns the cost between any

pair of states.(5) To start search, call depthsearch, breadthsearch,

bestsearch, or astarsearch with argument the starting state; it should return the solution path.

Example Prolog definition of a search problem: finding a route to San Francisco

successor(R1,R2) :- successor2(R1,R2).successor(R1,R2) :- successor2(R2,R1).successor2(monterey,gilroy).successor2(monterey,santa_cruz).successor2(gilroy,san_jose).successor2(santa_cruz,san_jose).successor2(santa_cruz,san_francisco).successor2(san_jose,oakland).successor2(san_jose,san_francisco).successor2(oakland,san_francisco).goalreached(san_francisco).eval(monterey,100).eval(gilroy,74).eval(santa_cruz,65).eval(san_jose,45).eval(oakland,15).eval(san_francisco,0).

Example Prolog search definition, cont./* This totals up the pair_cost2 numbers along a path */ piece_cost(X,Y,C) :- piece_cost2(X,Y,C).piece_cost(X,Y,C) :- piece_cost2(Y,X,C).piece_cost2(monterey,gilroy,35).piece_cost2(monterey,santa_cruz,45).piece_cost2(gilroy,san_jose,35).piece_cost2(santa_cruz,san_jose,30).piece_cost2(santa_cruz,san_francisco,85).piece_cost2(san_jose,oakland,50).piece_cost2(san_jose,san_francisco,50).piece_cost2(oakland,san_francisco,20).Then give the starting state, and search returns to sequence of

states to the goal, like: ?- depthsearch(monterey,A).A=[san_francisco,san_jose,gilroy,monterey]

Another example: the robot housekeeper problemDefine behavior of a housekeeping robot.1. In the starting state:

– The robot is at the trash chute (use predicate at);– Offices 1 and 2 need dusting (use dusty);– Office 1 has trash in its trash basket, but Office 2 doesn't

(use trashy);– The carpet in Office 1 does not need vacuuming, while the

carpet in Office 2 does (use vacuumable).Write the starting state using predicate expressions.2. A goal state is one in which all these are true: Every office is

dusted; every carpet is vacuumed; every trash basket is emptied; the robot is not holding any basket (use holdingbasketfrom); and the robot is at office 1. Give the conditions defining a goal state in logical terms.

The cost function for the robot housekeeper3. Assume these energy costs:

– 10 units to vacuum a single room (the robot has a built-in vacuum);

– 6 units to dust a room (the robot has a built-in duster);– 3 units to pick up a trash basket (the robot can hold several

at once);– 1 unit to put down a trash basket;– 3 units to travel between offices;– 8 units to travel between an office and the trash chute;– 5 units to dispose of the contents of a trash basket down the

trash chute.Assume the last action taken by the robot is the argument to a

lastact fact in each state. Define a piece_cost function.

The eval function for the robot housekeeper4. For A* search, we prefer an evaluation function that is a lower

bound on the subsequent cost. So we'll add up numbers ("piece_eval"s) for each expression in the state.

The default number will be zero, but "dusty(X)" for instance will have a piece_eval of 6. Compute the other piece_evals.

Successor definition for the robot housekeeper5. The successor definition must follow these requirements:a. The robot can vacuum the floors, dust, and empty the trash

baskets.b. Vacuuming the floor generates dust that goes into the trash basket

of the room it is in.c. Emptying the trash baskets in each room requires a trip to the

trash chute.d. It doesn't make sense to vacuum or dust if the room isn't

vacuumable or dusty respectively.e. Dusting puts dust on the floor, requiring vacuuming.f. It doesn't make sense to vacuum if a room is dusty.g. It doesn't make sense to pick up a trash basket if a room is

vacuumable or dusty.h. A lastact fact in each state should hold what the last action was.

Write "successor" definition for the "putdown", "dust", and "go" actions

Example: For "dispose(Basket)” (meaning to empty the trash basket whose name is Basket). Preconditions: you are at the chute, have a basket, and the basket contains trash. Postconditions: Delete the fact the basket has trash, delete the old “lastact” fact, and add a new “lastact” fact with argument “dispose(Basket)”.

Successor definitions in Prologsuccessor(S,NS):-member(at(chute),S), member(fullbasket(X),S), member(holdingbasketfrom(X),S), delete(S, fullbasket(X),S2), delete(S2, lastact(A),S3), NS=[lastact(dispose(X)) | S3].successor(S,NS) :- member(at(X),S), \+ member(dusty(X),S), member(vacuumable(X),S), delete(S, vacuumable(X), S2), delete(S2, fullbasket(X), S3), delete(S3, lastact(A), S4), NS=[lastact(vacuum(X)), fullbasket(X) | S4].successor(S,NS) :- member(at(X),S), \+ member(dusty(X),S), \+ member(holdingbasketfrom(X),S), \+ member(vacuumable(X),S), member(fullbasket(X),S), delete(S, lastact(A), S2), NS=[lastact(pickup(X)), holdingbasketfrom(X) |S2].

successor(S,NS) :- member(at(X),S), member(dusty(X),S),

delete(S, dusty(X), S2), delete(S2, vacuumable(X), S3),

delete(S3, lastact(A), S4), NS=[lastact(dust(X)), vacuumable(X) | S4].successor(S,NS) :- member(at(X),S),

member(holdingbasketfrom(X),S), delete(S, holdingbasketfrom(X), S2), delete(S2, lastact(A), S3), NS=[lastact(putdown(X)) | S3].successor(S,NS) :- member(at(X),S), delete(S, at(X), S2), places(PL),

member(Y,PL),\+ X=Y, delete(S2, lastact(A), S3),

NS=[lastact(go(X,Y)), at(Y) | S3].

Eval function in Prologeval([],0).eval([X|L],N) :- piece_eval(X,N2), eval(L,N3),

N is N2+N3.

piece_eval(dusty(X),6).piece_eval(vacuumable(X),10).piece_eval(fullbasket(X),8).piece_eval(holdingbasketfrom(X),1).piece_eval(P,0)

Cost function in Prologcost([_],0).cost([S1,S2|L],C) :- S1=[FS1|_], S2=[FS2|_], piece_cost(FS1,FS2,C2), cost([S2|L],C3), C is C2+C3, !.piece_cost(lastact(vacuum(X)),_,10).piece_cost(lastact(dust(X)),_,6).piece_cost(lastact(pickup(X)),_,3).piece_cost(lastact(dispose(X)),_,5).piece_cost(lastact(putdown(X)),_,1).piece_cost(lastact(go(chute,Y)),_,8).piece_cost(lastact(go(X,chute)),_,8).piece_cost(lastact(go(X,Y)),_,3).piece_cost(lastact(none),_,0).

Statistics on the housekeeping problemThese programs all eliminate states whose fact lists are

permutations of another. "Cells" means the number of cons cells created. Breadth-first search ran extremely slowly. All runs were on SunOS machine ai9 in 1990.

AllegroLisp,interpreted

AllegroLisp,compiled

QuintusProlog,semicompiled

depth-firstsearch

433 msec. 33 msec. 267 msec.

breadth-first search

33144 msec. 3224 msec. > 1 hour

best-firstsearch

1450 msec. 83 msec. 1150 msec.

A* search 2000 msec. 117 msec. 9250 msec.

The depth-first search algorithm• Initialize S to current state; initialize stack to hold

starting state. • Until S is a goal state:

– Put (“push”) S onto the stack;– Find the successors of S;– Set S to its best successor (as per heuristics) that is

not in the stack;– If no such successor, remove (“pop”) the last state

P from stack and go to a different successor of P. • The solution path is the stack items when goal is

found.

The breadth-first search algorithm • Erase the oldstate array; initialize queue to the starting

state with a null backpointer.• Until the first state on the queue is a goal state:

– Remove the first item on the queue and call it S.– If S is a goal state, stop.– Otherwise, add successors of S not already on the

agenda or in the oldstate array to the end of the queue, using heuristics to determine the order you add them; give each a backpointer to S.

– Add S to the oldstate array.• The solution path is found by creating a list of the states

encountered in following the backpointers from the goal state (using the oldstate array), then reversing the list.

The A* search algorithm • Initialize agenda to just the starting state with null

backpointer and starting-state evaluation-function value.• Loop until you pick a goal state:

– Remove the first (best) item on the agenda and call it S.– If S is a goal state, stop.– Otherwise, insert successors of S into the agenda if

they are not already in the agenda or in the oldstate array, maintaining sorted order of total evaluation. Give each successor state T a backpointer to S and a total evaluation U(T) = U(S) + C(S,T) + E(T) - E(S) where C is the cost from S to T and E is the evaluation function. If T is already on the agenda and this U is better, replace the state on the agenda, else ignore.

– Add S to the oldstate array.• Create the solution path by following backpointers from

the goal state, then reversing the list.