68
Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 1 / 28 A Survey of Suboptimal Search Algorithms Jordan T. Thayer and Wheeler Ruml jtd7, ruml at cs.unh.edu slides at: http://www.cs.unh.edu/˜jtd7/papers/

A Survey of Suboptimal Search Algorithms - Computer Scienceruml/papers/suboptimal-tutorial.pdf · A Survey of Suboptimal Search Algorithms Jordan T. Thayer and Wheeler Ruml jtd7,

Embed Size (px)

Citation preview

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 1 / 28

A Survey of Suboptimal Search Algorithms

Jordan T. Thayer and Wheeler Ruml

jtd7, ruml at cs.unh.edu

slides at: http://www.cs.unh.edu/˜jtd7/papers/

This is Search

Introduction

■ Search

■ duplicates

■ structs

■ Search Types

■ Search Tips

■ Outline

■ Not Discussed

Suboptimal

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 2 / 28

■ initial state

■ expand

generates all successor states, implicitly computes g(n)

■ goal test

■ h(n)

estimates cost of reaching a goal

admissibility: non-overestimating

consistency: obeys triangle inequality

■ state: problem data

■ node: state, g, parent pointer

What is a duplicate?

Introduction

■ Search

■ duplicates

■ structs

■ Search Types

■ Search Tips

■ Outline

■ Not Discussed

Suboptimal

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 3 / 28

1. while open has nodes2. remove n from open with minimum h(n)3. if n is a goal then return n

4. otherwise expand n, inserting its children into open5. return failure

same state, different path

Common Data Structures

Introduction

■ Search

■ duplicates

■ structs

■ Search Types

■ Search Tips

■ Outline

■ Not Discussed

Suboptimal

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 4 / 28

1. while open has nodes2. remove n from open with minimum h(n)3. if n is a goal then return n

4. otherwise expand n, inserting its children into open5. return failure

■ openlist:

heap: handles real costs, large ranges of values, etc

bucket list: more efficient, requires integer values

■ closed list:

hash table: in practice, also includes open nodes

Different Kinds of Search

Introduction

■ Search

■ duplicates

■ structs

■ Search Types

■ Search Tips

■ Outline

■ Not Discussed

Suboptimal

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 5 / 28

Grid Four-way 35%

width800400

total nodes generated

160000

80000

A*

greedy

suboptimal search scales better than optimal search

Different Kinds of Search

Introduction

■ Search

■ duplicates

■ structs

■ Search Types

■ Search Tips

■ Outline

■ Not Discussed

Suboptimal

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 5 / 28

Grid Four-way 35%

width800400

total nodes generated

160000

80000

A*

greedy

Grid Four-way 35%

width800400

final sol cost relative to A* 1.3

1.2

1.1

1

greedy

A*

it does so at the cost of solution quality

Different Kinds of Search

Introduction

■ Search

■ duplicates

■ structs

■ Search Types

■ Search Tips

■ Outline

■ Not Discussed

Suboptimal

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 5 / 28

Grid Four-way 35%

Size800400

total nodes generated

160000

80000

A*

wA*, w = 1.5

greedy

Grid Four-way 35%

Size800400

fin

al

sol

cost

rela

tive t

o A

*

1.4

1.2

1

greedy

wA*, w = 1.5

A*

bounded suboptimal search presents a middle ground

Simple Tips for Efficient Search

Introduction

■ Search

■ duplicates

■ structs

■ Search Types

■ Search Tips

■ Outline

■ Not Discussed

Suboptimal

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 6 / 28

■ store open nodes in ’closed list’ as well

prevents multiple copies of a state being open at once

■ duplicate checking and delaying or dropping

■ correct tie breaking

varies by search, generally prefer high g

■ goal test on generation

then prune for bounded suboptimal search

■ recursive expansions (to reduce heap operations)

About the Tutorial

Introduction

■ Search

■ duplicates

■ structs

■ Search Types

■ Search Tips

■ Outline

■ Not Discussed

Suboptimal

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 7 / 28

■ Jordan and I will alternate

■ bibliography at the end

■ the pseudo code

not presented during talk

included for later review

■ not comprehensive due to time constraints

each section covers ’greatest hits’

our personal experience and biases

Things We Will Discuss

Introduction

■ Search

■ duplicates

■ structs

■ Search Types

■ Search Tips

■ Outline

■ Not Discussed

Suboptimal

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 8 / 28

■ suboptimal search minimize solving time

greedy best-first search

beam search

LSS-learning real-time Search (LSS-LRTA*)

■ bounded suboptimal search balance time and cost

weighted A*

optimistic search

■ anytime search unknown deadlines

anytime repairing A*

anytime weighted A*

restarting weighted A*

Things We Won’t Discuss

Introduction

■ Search

■ duplicates

■ structs

■ Search Types

■ Search Tips

■ Outline

■ Not Discussed

Suboptimal

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 9 / 28

■ optimal search strategies

■ bounded-depth tree search

■ local search strategies

■ constructing heuristics

■ using disk

■ parallel algorithms

■ anything relying on distance estimates

(come back next session)

Unboundedly Suboptimal Search

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 10 / 28

Suboptimal Search: Outline

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 11 / 28

■ greedy best-first search

variant of best-first search

■ beam search: best-first, breadth-first

irrevocable pruning

■ real-time search: LSS-LRTA*

interleaves planning and acting

next session: speedy search, A* on length, beam search on d.

Greedy Best-first Search Doran and Michie 1966

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 12 / 28

1. Best-first Search On Cost-to-go Estimate, h(n).

root

A*

5 6

8

6 6

6 88

129

with admissible h, f rises resulting in vacillationin optimal search, h prunes, in greedy it guides

Greedy Best-first Search Doran and Michie 1966

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 12 / 28

1. while open has nodes2. remove n from open with minimum h(n)3. if n is a goal then return n

4. otherwise expand n, inserting its children into open5. return failure

Greedy Best-first Search Doran and Michie 1966

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 12 / 28

1. while open has nodes2. remove n from open with minimum h(n)2a. break ties on h in favor of low g(n)3. if n is a goal then return n

4. otherwise expand n, inserting its children into open5. return failure

Greedy Best-first Search Doran and Michie 1966

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 12 / 28

1. while open has nodes2. remove n from open with minimum h(n)2a. break ties on h in favor of low g(n)3. if n is a goal then return n

4. otherwise for each child c of n4a. if c was ever expanded, discard it4b. if c is in open, keep c with smallest g4c. otherwise insert c into open5. return failure

Greedy Best-first Search Doran and Michie 1966

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 12 / 28

Grid Four-way 35%

width800400

total nodes generated

20000

10000

expand duplicates

drop duplicates

Grid Four-way 35%

Size800400

fin

al

sol

cost

2000

1000

drop duplicates

expand duplicates

dropping duplicates improves time, harms quality

Greedy Best-first Search Doran and Michie 1966

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 12 / 28

greedy best-first search: h(n) A*: f(n) = g(n) + h(n)

Greedy Best-first Search Doran and Michie 1966

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 12 / 28

greedy best-first search: h(n) A*: f(n) = g(n) + h(n)

basic ⇒ brittle

Beam Search Rich and Knight 1991, Bisani 1992

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 13 / 28

Best-First Beam Search

1. run A* with a fixed sized open list2. filter out nodes with high f(n)

fixed memory (sometimes)

conflates propulsion with pruningdoesn’t work well in practice Wilt et al SoCS-10

Beam Search Rich and Knight 1991, Bisani 1992

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 13 / 28

Best-First Beam Search

1. while open has nodes2. remove n from open with minimum f(n)3. if n is a goal then return n

4. otherwise for each child c of n5. insert c into open6. if open is larger than width7. discard worst node on open8. return failure

Beam Search Rich and Knight 1991, Bisani 1992

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 13 / 28

Breadth-First Beam Search

1. run breadth-first search with a fixed sized open list2. filter out nodes with high f(n)

Beam Search Rich and Knight 1991, Bisani 1992

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 13 / 28

Breadth-First Beam Search

1. while open has nodes2. for each n ∈ open3. if n is a goal, return n

4. otherwise expand n, adding to children5. open becomes best width nodes in children5a. best according to f(n) = g(n) + h(n)6. return failure

Beam Search Rich and Knight 1991, Bisani 1992

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 13 / 28

Breadth-First Beam Search

1. while open has nodes2. for each n ∈ open3. for each child c of n4. if c is a goal, return it5. otherwise add c to children6. open becomes best width nodes in children6a. best according to f(n) = g(n) + h(n)6b. break ties in favor of high g(n)7. return failure

Beam Search Rich and Knight 1991, Bisani 1992

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 13 / 28

for the uninitiated, the 15 puzzle

Beam Search Rich and Knight 1991, Bisani 1992

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 13 / 28

Korf's 100 15 Puzzles

log10 total raw cpu time0-3

Solution Quality

0.8

0.4

IDA*

breadth

wA*

greedy

Life Four-way Grids 35% Obstacles

log10 total raw cpu time210

Solution Quality

0.8

0.4

0

A*

wA*

greedy

breadth

beam search might be best approach, or it might be awful

Beam Search Rich and Knight 1991, Bisani 1992

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 13 / 28

no goals

dead ends (left) and many duplicates (right) cause problems

LSS-LRTA* Koenig And Sun, 2008

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 14 / 28

real-time search: interleave planning and actingtime bound on planning per action

1. run A* for a fixed number of expansions2. commit to bestf3. back-up heuristic values using djikstra variant4. act and repeat

LSS-LRTA* Koenig And Sun, 2008

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 14 / 28

real-time search: interleave planning and actingtime bound on planning per action

1. run A* for a fixed number of expansions2. commit to bestf3. back-up heuristic values using djikstra variant4. act and repeat

7

7 8

8

910

15 15

LSS-LRTA* Koenig And Sun, 2008

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 14 / 28

real-time search: interleave planning and actingtime bound on planning per action

1. run A* for a fixed number of expansions2. commit to bestf3. back-up heuristic values using djikstra variant4. act and repeat

7

7 8

8

910

15 15

LSS-LRTA* Koenig And Sun, 2008

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 14 / 28

real-time search: interleave planning and actingtime bound on planning per action

1. run A* for a fixed number of expansions2. commit to bestf3. back-up heuristic values using djikstra variant4. act and repeat

910

15 159

9

9

15

LSS-LRTA* Koenig And Sun, 2008

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 14 / 28

real-time search: interleave planning and actingtime bound on planning per action

1. run A* for a fixed number of expansions2. commit to bestf3. back-up heuristic values using djikstra variant4. act and repeat

910

15 159

9

9

15

LSS-LRTA* Koenig And Sun, 2008

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 14 / 28

Korf's 100 15 Puzzles

log10 total raw cpu time0-3

Solution Quality

0.8

0.4

breadth

LSS-LRTA*

greedy

Life Four-way Grids 35% Obstacles

log10 total raw cpu time210

Solution Quality

0.8

0.4

0

greedy

breadth

LSS-LRTA*

don’t commit if you don’t have to!

Section Summary

Introduction

Suboptimal

■ Greedy

■ Beam

■ LSS-LRTA*

■ Summary

Bounded Suboptimal

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 15 / 28

■ duplicate policy affects solution cost and solving time

■ beam searches

breadth-first generally better than best-first

need closed lists, see Wilt et al SoCS-10

■ topology dictates algorithm choice

greedy if problems are small

beam search if problems too big for greedy

many dead necessitates complete searches

many duplicates necessitates duplicate dropping

■ real-time only if you have real-time constraints

Bounded Suboptimal Search

Introduction

Suboptimal

Bounded Suboptimal

■ Weighted A*

■ Optimistic Search

■ Summary

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 16 / 28

Bounded Suboptimal Search: Outline

Introduction

Suboptimal

Bounded Suboptimal

■ Weighted A*

■ Optimistic Search

■ Summary

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 17 / 28

■ weighted A*

larger w 6= faster search

■ optimistic search

selecting an appropriate optimism

handling duplicates effectively

next tutorial: skeptical search, A∗

ǫ , EES

Weighted A* Pohl 1970

Introduction

Suboptimal

Bounded Suboptimal

■ Weighted A*

■ Optimistic Search

■ Summary

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 18 / 28

1. Best-first Search on f ′(n) = g(n) + w · h(n)

w ≥ 1placing additional emphasis on h should encourage progressadmissible h ensures bound.

Weighted A* Pohl 1970

Introduction

Suboptimal

Bounded Suboptimal

■ Weighted A*

■ Optimistic Search

■ Summary

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 18 / 28

1. while open has nodes2. remove n from open with minimum f ′(n)3. if n is a goal then return n

4. otherwise expand n, inserting its children into open5. return failure

placing additional emphasis on h should encourage progress

Weighted A* Pohl 1970

Introduction

Suboptimal

Bounded Suboptimal

■ Weighted A*

■ Optimistic Search

■ Summary

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 18 / 28

1. while open has nodes2. remove n from open with minimum f ′(n)2a f ′(n) = g(n) + w · h(n)2b. break ties on in favor of low f(n)3. if n is a goal then return n

4. otherwise for each child c of n4a. if c was ever expanded, discard it4b. if c is in open, keep c with smallest g4c. otherwise insert c into open5. return failure

discarding duplicates greatly improves performanceat the cost of solution qualityonly applicable with consistent heuristics

Weighted A* Pohl 1970

Introduction

Suboptimal

Bounded Suboptimal

■ Weighted A*

■ Optimistic Search

■ Summary

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 18 / 28

Life Four-way Grid World

Suboptimality42

tota

l nodes genera

ted relative to A

*4

2

expand duplicates

drop_duplicates

Life Four-way Grid World

Suboptimality42

final sol cost

2.7e+06

2.4e+06

drop duplicates

expand duplicates

discard duplicates when possible

Weighted A* Pohl 1970

Introduction

Suboptimal

Bounded Suboptimal

■ Weighted A*

■ Optimistic Search

■ Summary

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 18 / 28

Dynamic Robot Motion Planning

Suboptimality321

total nodes generated

2e+07

1e+07

wA*

Dock Robot

Suboptimality8040

total nodes generated

8e+06

4e+06

wA*

larger w does not always mean better performance

Bound for Weighted A*

Introduction

Suboptimal

Bounded Suboptimal

■ Weighted A*

■ Optimistic Search

■ Summary

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 19 / 28

f(n) = g(n) + h(n)f ′(n) = g(n) + w · h(n)

■ p is the deepest node onan optimal path to opt.

g(sol)f ′(sol) ≤ f ′(p)

g(p) + w · h(p) ≤ w · (g(p) + h(p))w · f(p) ≤ w · f(opt)

w · g(opt)

1. works for any f ′(p) ≤ w · f(p)2. g(p) + w · h(p) ≪ w(g(p) + h(p))!

Optimistic Search Thayer and Ruml, ICAPS-08

Introduction

Suboptimal

Bounded Suboptimal

■ Weighted A*

■ Optimistic Search

■ Summary

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 20 / 28

■ run weighted A∗ with a high weight.

■ expand node with lowest f value after a solution is found.

continue until w · bestf ≥ f(sol)

this ’clean up’ guarantees solution quality.

Optimistic Search Thayer and Ruml, ICAPS-08

Introduction

Suboptimal

Bounded Suboptimal

■ Weighted A*

■ Optimistic Search

■ Summary

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 20 / 28

■ p is the deepest node onan optimal path to opt.

■ bestf is the node with thesmallest f value.

f(p) ≤ f(opt)f(bestf ) ≤ f(p)

bestf provides a lower bound on solution cost

Determine bestf via priority queue sorted on f

Optimistic Search Thayer and Ruml, ICAPS-08

Introduction

Suboptimal

Bounded Suboptimal

■ Weighted A*

■ Optimistic Search

■ Summary

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 20 / 28

1. run weighted A∗ with weight (bound− 1) · 2 + 1

2. expand node with lowest f value after a solution is found.

Continue until w · bestf ≥ f(sol)

this ’clean up’ guarantees solution quality.

Optimistic Search Thayer and Ruml, ICAPS-08

Introduction

Suboptimal

Bounded Suboptimal

■ Weighted A*

■ Optimistic Search

■ Summary

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 20 / 28

1. run weighted A∗ with a high weight.

2. expand node with lowest f value after a solution is found.

continue until w · bestf ≥ f(sol)

this ’clean up’ guarantees solution quality.

Optimistic Search Thayer and Ruml, ICAPS-08

Introduction

Suboptimal

Bounded Suboptimal

■ Weighted A*

■ Optimistic Search

■ Summary

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 20 / 28

Optimistic Search Thayer and Ruml, ICAPS-08

Introduction

Suboptimal

Bounded Suboptimal

■ Weighted A*

■ Optimistic Search

■ Summary

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 20 / 28

1. w = (bound− 1) · optimism

2. while open and clean contain nodes3. if no incumbent has been found4. remove n from open with minimum f ′(n)5. remove n from clean5. if n is a goal, set incumbent to n

6. otherwise expand n, inserting its children into open and clean7. otherwise remove n from clean with minimum f(n)8. if bound · f(n) ≤ f(incumbent), return incumbent9. otherwise expand n, inserting its children into open10. return failure

In practice, clean isn’t constructed until an incumbent is found.

Optimistic Search Thayer and Ruml, ICAPS-08

Introduction

Suboptimal

Bounded Suboptimal

■ Weighted A*

■ Optimistic Search

■ Summary

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 20 / 28

Dynamic Robot Motion Planinng

Suboptimality42

log10 total nodes generated

8

7

6

5

4

wA*

Optimism 2

Optimism 5

Optimism 20

proper optimism depends on h accuracy

Section Summary

Introduction

Suboptimal

Bounded Suboptimal

■ Weighted A*

■ Optimistic Search

■ Summary

Anytime Search

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 21 / 28

■ use weighted A* as a first approach to a problem

■ use optimistic search if you know weighted A* works well

■ duplicate handling can be important

wA* can drop, requires consistent heuristic

other algorithms can delay

■ solving problems and showing bounds can be separate steps

Anytime Search

Introduction

Suboptimal

Bounded Suboptimal

Anytime Search

■ wA* Variants

■ Summary

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 22 / 28

Anytime Search Algorithms: Outline

Introduction

Suboptimal

Bounded Suboptimal

Anytime Search

■ wA* Variants

■ Summary

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 23 / 28

■ anytime repairing A*

■ anytime weighted A*

■ restarting weighted A*

come back next session for d-fenestration, size-cost search

Anytime Algorithms Based on Weighted A*

Introduction

Suboptimal

Bounded Suboptimal

Anytime Search

■ wA* Variants

■ Summary

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 24 / 28

anytime weighted A*, Hansen and Zhou 1997

1. run weighted A*2. if you find a goal, keep going.

anytime repairing A*, Likhachev et al, 2003

1. run weighted A*2. if you find a duplicate, don’t look at it just yet.3. if you find a goal4. dump duplicates into open, reduce w, keep going.

restarting weighted A*, Richter et al 2010

1. run weighted A*2. if you find a goal, start over with a lower weight.

Anytime Algorithms Based on Weighted A*

Introduction

Suboptimal

Bounded Suboptimal

Anytime Search

■ wA* Variants

■ Summary

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 24 / 28

anytime weighted A*, Hansen and Zhou 1997

1. run weighted A*2. if you find a goal, keep going.

anytime repairing A*, Likhachev et al, 2003

1. run weighted A*2. if you find a duplicate, don’t look at it just yet.3. if you find a goal4. dump duplicates into open, reduce w, keep going.

restarting weighted A*, Richter et al 2010

1. run weighted A*2. if you find a goal, start over with a lower weight.

Anytime Algorithms Based on Weighted A*

Introduction

Suboptimal

Bounded Suboptimal

Anytime Search

■ wA* Variants

■ Summary

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 24 / 28

continued search, Hansen and Zhou 1997

1. run some complete suboptimal search2. if you find a goal, keep going.

repairing search, Likhachev et al, 2003

1. run some complete parameterized search2. if you find a duplicate, don’t look at it just yet.3. if you find a goal4. dump duplicates into open, change parameter, keep going.

restarting search, Richter et al 2010

1. run some complete parameterized search2. if you find a goal, start over with a different parameter.

Anytime Algorithms Based on Weighted A*

Introduction

Suboptimal

Bounded Suboptimal

Anytime Search

■ wA* Variants

■ Summary

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 24 / 28

anytime weighted A*, Hansen and Zhou 1997

1. while open has nodes2. remove n from open with minimum f ′(n)3. if n is a goal set n as incumbent4. otherwise for each child c of n5. if f(c) < f(incumbent) insert c into open6. return incumbent

anytime repairing A*, Likhachev et al, 2003restarting weighted A*, Richter et al 2010

Anytime Algorithms Based on Weighted A*

Introduction

Suboptimal

Bounded Suboptimal

Anytime Search

■ wA* Variants

■ Summary

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 24 / 28

anytime weighted A*, Hansen and Zhou 1997anytime repairing A*, Likhachev et al, 2003

1. while open has nodes2. remove n from open with minimum f ′(n)3. if n is a goal4. set n as incumbent5. empty delay into open6. reduce w

7. otherwise for each child c of n8. if c was ever expanded, add it to delay9. otherwise insert c into open10. return incumbent

restarting weighted A*, Richter et al 2010

Anytime Algorithms Based on Weighted A*

Introduction

Suboptimal

Bounded Suboptimal

Anytime Search

■ wA* Variants

■ Summary

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 24 / 28

anytime weighted A*, Hansen and Zhou 1997anytime repairing A*, Likhachev et al, 2003restarting weighted A*, Richter et al 2010

1. while open has nodes2. remove n from open with minimum f ′(n)3. if n is a goal4. set n as incumbent5. reduce w

6. if w < 1, return incumbent7. otherwise restart the search8. otherwise for each child c of n9. if f(c) < f(incumbent) insert c into open10. return incumbent

Anytime Algorithms Based on Weighted A*

Introduction

Suboptimal

Bounded Suboptimal

Anytime Search

■ wA* Variants

■ Summary

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 24 / 28

Life Four-way Grids 35% Obstacles

raw cpu time40200

Solution Quality

0.8

0.4

0

ARA*

AwA*

RWA*

Korf's 100 15 Puzzles

raw cpu time3002001000

Solution Quality

0.96

0.88

0.8

RwA*

ARA*

AwA*

no one best anytime algorithm (or even framework)generally, try repairing first.

Anytime Algorithms Based on Weighted A*

Introduction

Suboptimal

Bounded Suboptimal

Anytime Search

■ wA* Variants

■ Summary

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 24 / 28

effort of anytime A*

Anytime Algorithms Based on Weighted A*

Introduction

Suboptimal

Bounded Suboptimal

Anytime Search

■ wA* Variants

■ Summary

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 24 / 28

effort of anytime repairing A*

Anytime Algorithms Based on Weighted A*

Introduction

Suboptimal

Bounded Suboptimal

Anytime Search

■ wA* Variants

■ Summary

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 24 / 28

effort of restarting A*

Summary Anytime Search

Introduction

Suboptimal

Bounded Suboptimal

Anytime Search

■ wA* Variants

■ Summary

Summary

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 25 / 28

■ ARA*, AwA*, RwA*

describe general frameworks (wA* not important)

continued – few duplicates, tight lower bound

repairing – many duplicates, high initial bound

restarting – low h bias, cheap expansions

■ great for unknown deadlines

■ for known deadlines, deadline aware search

(Dionne et al, SoCS-11)

Summary

Introduction

Suboptimal

Bounded Suboptimal

Anytime Search

Summary

■ Bibliography

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 26 / 28

Things We Discussed

Introduction

Suboptimal

Bounded Suboptimal

Anytime Search

Summary

■ Bibliography

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 27 / 28

■ suboptimal search minimize solving time

no guarantees on solution quality

so use inadmissible heuristics

and drop duplicate states

■ bounded suboptimal search balance time and cost

bounds on solution quality

drop duplicates when possible

looser bounds 6= always better performance

■ anytime search unknown deadlines

automatically trades time for quality

deadline agnostic

frameworks can be used with any complete algorithm

Bibliography

Introduction

Suboptimal

Bounded Suboptimal

Anytime Search

Summary

■ Bibliography

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 28 / 28

■ J. E. Dorand and D. Michie,“Experiments with the Graph Traverser Program”,Proceedings of the Royal Society of London. Series A,Mathematical and Physical Sciences, 1966.

■ Ira Pohl, “Heuristic Search Viewed as Path Finding in a Graph”,Artificial Intelligence volume 1, 1970.

■ Elain Rich and Kevin Knight, “Artificial Intelligence”, 1991.

■ Sven Koenig and Xiaoxun Sun,“Comparing Real-Time and Incremental Heuristic Search forReal-Time Situated Agents”,AAMAS-2008.

■ Jordan T. Thayer and Wheeler Ruml,“Faster Than Weigthed A*:An Optimistic Approach to Bounded Suboptimal Search”,ICAPS-2008.

Bibliography

Introduction

Suboptimal

Bounded Suboptimal

Anytime Search

Summary

■ Bibliography

Jordan Thayer and Wheeler Ruml (UNH) Suboptimal Search – 28 / 28

■ Eric A. Hansen, Shlomo Zilberstein and Victor A. Danilchenko,“Anytime Heuristic Search: First Results”,University of Massachusetts, Amherst Technical Repart 97-50,1997.

■ Maxim Likhachev, Geoff Gordon and Sebastian Thrun“ARA*: Anytime A* with Provable Bounds on Sub-Optimality”,NIPS-2003.

■ Eric A. Hansen and Rong Zhou,”Anytime Heuristic Search”,Journal of Artificial Intelligence Research volume 28, 2007.

■ Silvia Richter, Jordan T. Thayer and Wheeler Ruml,”The Joy of Forgetting: Faster Anytime Search via Restarting”,ICAPS-2010.

■ Chris Wilt, Jordan T. Thayer, and Wheeler Ruml,“A Comparison of Greedy Search Algorithms”, SoCS-2010.