All rights reservedL. Manevitz Lecture 31 Artificial Intelligence A/O* and Minimax L. Manevitz

All rights reserved L. Manevitz Lecture 3 1

Artificial Intelligence

A/O* and MinimaxL. Manevitz


Goals

• AO*

• Game Playing Algorithm :– Mini-max.– Mini-max with .

– References: Slides follow development in Nilsson; material is in Rich, Nilsson, or Winston.


A/O* algorithm

• Data Structure– Graph– Marked Connectors (down; unlike A*)– Costs q() maintained on nodes– SOLVED markings

– Note: We’ll discuss on acyclic graphs.


AND Connector

Or Connector


And/OR Graph


Solution Subgraph


Solution Subgraph


Solution Subgraph G’ of Gfrom n to terminals T

• If n is in T, G’ is just the singleton n.

• Otherwise n has one connector to set of nodes n1, n2, …, nk. – There is a Solution graph from each ni to T.– G’ is n, that connector, the nodes n1, …, nk– Plus … the solution graphs from each of the

ni.


Heuristic Values:estimated cost to solution set

0

2

2

4

1

1

0

0

4


Montone Restriction

• h(n) <= c + h(n1) + h(n2) + … h(nk)

Where c is cost of connector between n and

set of n1, … , nk.

This guarantees that h(n) <= h*(n).


Cost (q(n) ) Values

• If n has no successors q(n) = h (n)

• Otherwise working from bottom,– q(n) = connector cost + sum of q(successors)

Pick smallest of above; and mark direction.If that direction has all successors SOLVED

then n is marked SOLVED.


Basic Idea of A/O*

• First top-down graph growing picks out best available partial solution sub-graph from explicit graph.

• One leaf node of this graph is expanded

• Second, bottom-up cost-revising, connector-marking, SOLVE-labeling.


AO* Algorithm

1. Create G = <s> ; q(s) = h (s) If s TERM mark s SOLVED

2. Until s labeled SOLVED do:1. Compute G’ partial solution subgraph of G by

tracing down marked connectors in G from s.2. Select n in G’, n not in TERM, n a leaf.3. Expand n , place successors in G, for each successor

not already in G let q(successor)=h (successor). Label SOLVED all successors in TERM. (If no successors, reset q(n) := infinity ).


AO* Algorithm cont.

4. Let S := {n}.5. Until S = do :

1. Remove a node, m, from S which has no descendent in G also in S (minimal node).

2. Revise cost for m, (check each connector from m) q(m)=min [c +q(n1)+…+q(nk )]. Mark chosen connector. If all successors their connectors are SOLVED then mark m SOLVED.

3. If m SOLVED or changed q(m) then add to S all “preferred” parents of m.

4. End.

6. End.


Tracing the Algorithm

3

2

1

1



4

5

4

1

1

4



5

5

4

1

2

0

0

4

2



5

5

4

1

2

0

0

4

2



5

5

4

1

2

0

0



5

5

4

1

2

0

0


Optimality of A/O*?

• If h (n) <= c + h (n1) + … h (nk) for all nodes n and connectors

THEN OPTIMAL path.

• If h(term) = 0 this implies h (n) <= h*(n) (optimistic)


Note

• Acyclicity means that S is empty eventually.

• Choice of non-terminal node in partial solution graph to expand?– Best with HIGHEST h* (must expand

anyhow; quicker to change mind)


Game Playing Algorithm

• What are games ?

• Notion of Winning Strategy– Von – Neumann Morgernstern Theorem

• Use of heuristics to pick best move.

• Look ahead : So use backed up values to choose the best move.

• Mini-max algorithm.


Maximilian vs. MinervaWinning Strategy

(7,Min)

(6,1,Max) (5,2,Max) (4,3,Max)

(4,2,1,Min) (3,2,2,Min) (3,3,1,Min)(5,1,1,Min)

(4,1,1,1,Max) (3,2,1,1,Max) (2,2,2,1,Max)

(3,1,1,1,1,Min) (2,2,1,1,1,Min)

(2,1,1,1,1,1,Max)


Maximilian vs. Minerva

(7,Min)

(6,1,Max) (5,2,Max) (4,3,Max)

(4,2,1,Min) (3,2,2,Min) (3,3,1,Min)(5,1,1,Min)

(4,1,1,1,Max) (3,2,1,1,Max) (2,2,2,1,Max)

(3,1,1,1,1,Min) (2,2,1,1,1,Min)

(2,1,1,1,1,1,Max)


Mini-max Search

• See if limit of search has been reached :Yes :

• Compute static evaluation of current position + return value.

Otherwise :• If maximizing level – apply Mini-max to each

child. Return maximum of results.

• If minimizing level – apply Mini-max to each child. Return minimum of results.


Example

XX

OO

XX

OOX

XX

OOX

XX

OO

X XX

OO

X

XX

OOX


Example cont.

• e(p) = number of directions open for Max – number of directions open for Min

• e(p) = inf if win for Max

• e(p) = - inf if win for Min

• e(p) = 6 – 4 = 2XO


Example cont.

XX

OOX

XX

OOX

XX

OOX

XX

OOX

XX

OOXO

O OO

2-1=1 3-1=2 2-1=1 3-1=2

Min move1


Example cont.

XX

OOXO

- inf 3-2=1 2-2=0 3-2=1

Min move- inf

XX

OOX

XX

OOX

XX

OOX

XX

OOX

O OO


Example cont.

- inf 2-2=0 2-2=0

Min move- inf

XX

OO

X

XX

OO

X XX

OO

X XX

OO

X XX

OO

X

3-2=1

OO

OO


Example cont.

- inf 2-1=1 3-1=2

Min move- inf

3-1=2

XX

OO

X

XX

OO

XXX

OO

XXX

OO

XXX

OO

X

OO

OO


Example cont.

- inf 2-1=1 3-1=2

Min move- inf

2-1=1

XX

OOX

XX

OOX

XX

OOX

XX

OOX

XX

OOX

OO

O O


Example cont.

XX

OO

XX

OOX

XX

OOX

XX

OO

XX

XOO

X

XX

OOX

- inf

- inf- inf

- inf

1

Max move 1


Mini-max Algorithm

Mini-max (Position,Depth)

1. If DeepEnough (Position,Depth)1. then Return <Static (Position,Depth

2. Successors := MoveGen (Position,Depth)


Mini-max Algorithm cont.

3. If Depth Odd (Max node)1. then Loop until Successors =

1. Best = - inf

2. Remove m from Successors

3. <Value,Path> := Minimax (m,Depth+1)

4. If Value > Best then :

1. Best := Value

2. BestPath := m^Path

5. End Loop

2. Return <Best,BestPath>



4. If Depth Even (Min node)1. then Loop until Successors =

1. Best = inf

2. Remove m from Successors

3. <Value,Path> := Minimax (m,Depth+1)

4. If Value < Best then :

1. Best := Value

2. BestPath := m^Path

5. End Loop

2. Return <Best,BestPath>


Mini-max

–values at Max nodes never decrease.

values at Min nodes never increase.


Cut - Off

• Below any MIN node having value smaller than value of an ancestor.

• Below any MAX node having value greater than value of an ancestor.


Computing

of Max node : current largest backed up value of child.

of Min node : current smallest backed up value of child.


Mini-max

• Keep track of values and send on to children :

1) Search discontinued below any Min node with an of one of it’s ancestors. Set final value of node to be this value.

2) Search discontinued below any Max node with a of one of it’s ancestors. Set final value of node to be this value.


Example – Cut-Off

Max

Max

Min

2 7

=2

>=2

<=1

1

This is a Cut-Off


Mini-max Algorithm

1. Determine if the level is : 1. The top level.

2. The limit of search has been reached.

3. The level is a minimizing level.

4. The level is a maximizing level



2. If the level is the top level :1. Let alpha be –inf and let beta be inf.

3. If the limit of search has been reached : compute the static value of the current position relative to the appropriate player + Return result.



4. If the level is a minimizing level :1. Until all children are examined with

Minimax or alpha is bigger then beta :1. Set beta to the smaller of the given beta values

and the smallest value so far reported by Minimax working on the children.

2. Use Minimax on the next child of the current position, handing this new application of Minimax the current alpha and beta.

2. Report beta.



5. If the level is a maximizing level :1. Until all children are examined with

Minimax or alpha is bigger then beta :1. Set alpha to the larger of the given alpha values

and the biggest value so far reported by Minimax working on the children.

2. Use Minimax on the next child of the current position, handing this new application of Minimax the current alpha and beta.

2. Report alpha.


≥

Max node

Max node

Max node

Min node

Min node

Min node

0 5 -3 3

≤

3

-3 0 2 -2 3

≥

≥

≤

5 2 5 -5 0 1

=2

≥

≤

≥

≥


Max nodeMin node

0 5

-3 3

3

-3 0 2

-2 3

5 2

-55

0 1

5 1 -3 0

3 -3 0

-114 5

Cut Off≥

β=1

≥

β=-3

β≤1

≥

β≤1

≥

β=3

≥

β≤-3

β≤3

≥

β≤3

β=4

≥

β≤4

≥

≥


Example Max nodeMin node

0

5

-3

3

3

-3

0

2

-2 3

5 2

-55

0 1 5 1

-3 0

-5 5

-3 3

2

3

-3 0 0 1

-1 -2

-2

2

-3

3

-1

-11

4 5

Cut Off


Example Max nodeMin node

0

5

-3

3

3

-3

0

2

-2 3

5 2

-55

0 1 5 1

-3 0

-5 5

-3 3

2

3

-3 0 0 1

-1 -2

-2

2

-3

3

-1

-11

4 5

Cut Off


Other Adjustments

• Progressive Deepening (Analyze depth1,depth2,…).

• Heuristic Pruning (Order moves plausibly).

• Futility Cut-Off.• Heuristic Continuation (Waiting

for Quiescence).• Horizon effect.• Secondary Search.


Homework Number 1

• Implement BOTH the Mini-max and• The Alpha-Beta Algorithm for a

Beloved Game.• Suggestion : Othello (not allowed), Checkers,

Chinese Checkers, Othello (for losers), Chess etc.• Due date • (Penalty for lateness).


Other Adjustments

• Progressive Deepening (Analyze depth1,depth2,…).

• Heuristic Pruning (Order moves plausibly).

• Futility Cut-Off.• Heuristic Continuation (Waiting

for Quiescence).• Horizon effect.• Secondary Search.


Homework Number 1

• Implement BOTH the Mini-max AND• The Alpha-Beta Algorithm for a

Beloved Game.• Suggestion : Checkers (Damka), Othello (losers),

regular Othello not allowed, Chess etc.• Due date Dec 20 (before class)• (Penalty for lateness): -4 up to one week.

then -4 each class (before class)

Documents

All rights reservedL. Manevitz Lecture 31 Artificial Intelligence A/O* and Minimax L. Manevitz