1

GP End-Chess

Evolution of Chess Endgame Players

Ami Hauptman & Moshe Sipper

2

Outline Introduction

The Game of Chess – a solved problem? Important differences between human and

artificial chess players Chess Endgames - features & building

blocks GP problem definition Experiment and Results Future work

3

The game of Chess

First developed in India and Persia Considered an extremely complex game of

strategy and inventiveness Enormous search space

Roughly 50 possible moves at mid-game A typical game consists of a few dozen moves Estimated at 1043 in 40-move game (Shannon,

1950)

4

The game of Chess – AI history

First chess AI at 1958 – novice level Machine strength increasing linearly 1997 – defeat of former world champion,

Garry Kasparov, by IBM’s deep blue Last years – performance still increasing

Mainly Hardware Also Software

… The End ?

5

The game of Chess …NO! Deep(er) blue uses extreme brute-force,

traversing several millions of boards ps Very little generalization Virtually no human resemblance Deemed theoretically uninteresting

Chomsky: As interesting as a weight lifting competition between machine and man

Low A to I ratio; Low return

6

The game of Chess – Basic concepts

8x8 board Each player starts with 16 pieces, of 6

different types, and may only move 1 piece per turn

A piece can only move into an empty square or into one containing an opponent’s piece (a capture)

Win by capturing the opponent’s king

7

The game of Chess – pieces Pawn: may only move forward (or capture

diagonally) Bishop: diagonals Knight: L shaped moves. The only “unblockable”

piece Rook: Ranks & files Queen: Bishop & Rook combined King: 1 square in any direction. May not move

into attacked square

Values :

1 3 3/3.5 5 9 ∞

8

The game of Chess – example

White has over 30 possible moves

If black’s turn – can capture pawn at c3 and check (also fork)

9

The game of Chess – Check and Checkmate

“Checking” is attacking opponent’s king. Opponent must respond

“Mating” (Checkmate)

is when the opponent can’t avoid losing the king – and thus forfeiting the game

10

Human & Artificial Players - AI

AI uses search to assign a score to a board

Traverse the move tree from leaves - up

Select the best child using scores found

Only partial tree

Computer is O (Max) opponent is X

11

Human & Artificial Players – AI Search

Millions of boards (nodes) per second Little time for each board – less

knowledge Smart search algorithms –

pruning Alpha-beta variants (negascout etc.)

Still use heuristics at end – can’t see all tree

Most research revolves around search Human resemblance minimal – humans

use little search

12

Human & Artificial Players – Humans(1)

Humans use problem solving cognition Deeply knowledge based – extensive “Chess

theory” Massive use of pattern recognition Also use search but

Less deep Only develop “good” positions

More efficient – less nodes for “same” result Reminiscent of greedy search

Not only in chess

13

Human & Artificial Players – Grand Masters - Findings

Play against several opponents at the same level they play against a single opponent

GMs and novices: same level of performance when memorizing a random board; differ when memorizing real game positions

GM eye movements show they only scan “correct” parts of board

Strong Amateurs use the same meta-search as GMs - equally deep, same nodes, same speed; Differ in knowledge of domain (De Groot)

14

Endgames - example

White’s turn: mate in

5, with Qe6+

Features include: #moves for black king

minimal

Attacking, un-attacked

Checking

Officers same line\row

Black’s turn: draw

with: Rc1+, then Qg5

– fork & exchange

15

Endgames (2) - features

Few pieces remain (typically: king, 0-3

officers and sometimes pawns)

Fewer options, but more moves for each

piece

Trees still extremely large

16

Endgames - Building Blocks

Main goals Reduce search by “smart” features of the

board Use more game knowledge as humans do

Allow more complex features to be built by supplying basic ones (terminals) and building methods (functions)

Schemata evolution

17

Features Example - Fork

My piece is: Attacking 2 or more

pieces Protected or not

attacked Opponent pieces:

Unprotected OR protected but of

greater value Example: black must

exchange Q for R because of fork

18

Fork: Traditional AI search Only 3 legal moves for

black Find that one of white’s

next moves (out of 23 possible) captures black queen

Check all following moves for more piece exchanges

Sometimes, still check other moves (non capturing)

At end of search – compare remaining pieces

No fork “concept”

19

Features Example – Fork Feature Search (GP)

One of the features is isMyFork function – Checks all previously defined conditions

Also, use some smaller building blocks:

Is Opponent piece Attacked?

Is attacking piece protected?

Is opponent in check? Value of attacked piece

20

GP Problem Definition

Terminals & Functions

Fitness

Run parameters

Termination

We will see each element in the

following experiments

21

Endgame experiments conducted

KRKR – each player has 1 king and 1 rook

KRRKR – King with 2 Rook against King

and Rook

KRRKRR

KQKQ – Kings and Queens

KQRKQR – Combined

22

Basic program architecture

Generate all possible moves (depth=1)

Evaluate each board with GP individual

Select board with best score (or

stochastically decide between equal)

Perform best move

Repeat process with GP opponent until

game ends (or until only kings left)

23

KRKR Endgame Each player has 1 King, 1 Rook

“Toy” problem for chess endgames

Theoretical draw (experts never lose this)

Some interesting positions exist

24

KRKR Endgame - what needs to be learned

(1)

Avoid losing rook Don’t move to

attacked, unprotected squares

Vice versa - capture opponent’s rook if able

Black to move – white loses Rook

25

KRKR Endgame - what need to be learned (2) Avoid getting king

stuck in edges

Again, take advantage if opponent does this

Black to move – mate in 1

26

KRKR Endgames - Terminals Used in first runs: Is My Rook Attacked, Is Opp Rook

attacked Is {My, Opp} Rook Protected (two as

above) Is {My, Opp} Rook In Play Num Moves {My, Opp} king {My, Opp}-King’s proximity to edges Is Mate ERCs: ± {0.25, 0.5, 1} * MAX

MAX = 1000 (empirically)

27

KRKR Endgames - Functions Boolean

OR2, OR3, OR4

AND2, AND3, AND4

NOT

Arithmetic - +, -, *

Combined - <, =, >, IF

STGP

For now, no “chess” functions, only

terminals

28

KRKR Endgames - Fitness

Competitive, Random-2-ways Each individual plays against k randomly

selected opponents

Each game counts for both players

For each encounter Several games (typically 4) are played

Short games - ~5-8 moves per player

Each game starts at a random legal position

Safe start - no piece is attacked at the

beginning

29

KRKR Endgames – Fitness (2)

Scoring method: Victory: 1-2 points

Piece count advantage (theoretical win) – ¾

point

Draw: ½ point After advantage – 0 points

Loss: 0 points

30

KRKR Endgames – Parameters

Population size - 80

#Generations - 150..250

Operators: Reproduction 0.35

Crossover 0.5

Mutation 0.15 (including ERC mutation)

Termination – ~10-20 hours

31

KRKR Endgames – Results Every 10 generations, best individual

played against: Best of generation 0

An opponent performing random moves

Longer games: ~10-12 moves per player

50-150 games

Games were doubled – each player

staring from both positions

32

KRKR Endgames – Results

Bad results – no distinct improvement

Several reasons: Arithmetic operations problematic – we get

large numbers

Mate not distinct enough (traditionally

terminates the search)

Boolean functions not clear enough

Slow Runs due to large trees with

repeating functions

33

KRKR Endgames –Improvements

Boolean functions Divided to good and bad

Example: Is-My-King-In-Check changed to Is-

My-King-Not-In-Check

Mate changed to 1000*Mate

Added Not-My-Rook-Attacked-Unprotected

and Opp-Rook-Attacked-Unprotected

34

KRKR Endgames – Results - Improvements

Also consulted Chess

Experts – added

more:

Is-Opp-King-Behind-

Rook

Split to Opp-King-Prox-Rook

Opp-King-Behind-Rook

Is-Stalemate (only

kings left)Black moves and White loses Rook

35

KRKR Endgames – Results - Improvements

Arithmetic functions canceled Although Still using floats for terminals Also divide to good and bad:

NumNotMovesOppKing Theoretically justified – more “logical”

search in literature Empirically - need more logical rules, and

not : ( > (+ (#moves-k #moves-opp-k) 5.5)) Memoization – saves more than ½ the

time

36

KRKR Endgames – Final Results

Improvement Above 75% of games against random end in

advantage or mate

Still, too few mates, even when score for win

is increased – difficult to learn move sequence

Same against best of generation 0 (almost

random)

The main thing that was learned was avoiding

getting the rook captured

37

KRRKR Endgames Example (right)

Very good for white Black king

exposed 2 rooks close Next move –

captures rook (mate in 5)

38

KRRKR Endgames - goals One player has 2 rooks, the other – 1

Not theoretically drawn

We want one generalized individual for

all endgames and positions (Not one for

each endgame): Each player needs to play both advantage,

draw (KRKR) and disadvantage

Terminals need to be more general

39

KRRKR Endgames - changes Terminals - changed and added to cope

with changing state Material-Count (recall each rook = 5)

Num-My-Pieces-Not-Attacked, since now there

are more than 1

Is-My-King-Protecting-Piece and My-Officers-

Same-Line to allow more complex considerations

Functions If-Adv-Then-(left child)-Else-(right child)

Eventually divided to 3 trees

40

KRRKR Endgames - changes

Also added – comparing differences to

parent node Boolean Is-Material-Increase, which

compares to the parent node (board)

Material decrease is not needed since

considering only my move

Not-My-King-Moves-Decrease to further use

number of moves for king

41

KRRKR Endgames – Opponents

Random forsaken; Best-of-0 still used but less

Added new opponent – MASTER a program we wrote based on consultation with

experts, highest being International Master Boris Gutkin, ELO 2400 (only about 3000 of those…)

Used ~50 general positions and rules derived from them, together with scores for each

Defined a strategy (“Expert”) accordingly Tested evolved programs against it

Human competitive?

42

KRRKR Endgames – Fitness

Test were conducted by assigning each player both roles for each position

Fitness was refined – score effected by: Starting position (advantage or

disadvantage) End result – win, loss or draw Adv position ending in draw receives a

score of near zero Dis-adv ending in a draw will receive better

than 0.5

43

KRRKR Endgames – Results

Expert-defined performed extremely well

against Random and Best0

Evolved programs performed generally as

well as expert defined, sometimes better

Percent of favorable results in game outcomes

44

Main Experiment – KQRKQR Most complex

endgame we worked

with

Still theoretical draw

Highly position

dependant – “noisy”

Larger trees 2 officers

Queens

Easier to mate

45

KQRKQR Endgames - changes

Added – more “heavy” terminals (and

components) Boolean Is-Not-Mate-in-one, most time

consuming but necessary

Boolean Is-My-King-Not-Trapped Not all king’s moves lead closer to edges

Important but vague – usually happens with

complex terminals My-Officers-Same-Line

46

Simple TerminalsComplex TerminalsFunction SetNotMyKingInCheckEvaluateMaterialIf3(B,F,F)

IsOppKingInCheckIsMaterialIncreaseOr2(B,B)

MyKingDistEdgesIsMateOr3(B,B,B)

OppKingProximityToEdges

IsMateInOneAnd2(B,B)

NumMyPiecesNotAttacked

OppPieceCanBeCapturedAnd3(B,B,B)

NumOppPiecesAttackedMyPieceCannotBeCaptured

Smaller(B,B)

ValueMyPiecesAttackingIsOppKingStuckNot(B)

ValueOppPiecesAttackingIsMyKingNotStuck

IsMyQueenNotAttackedIsOppKingBehindPiece

IsOppQueenAttackedIsMyKingNotBehindPiece

IsMyForkIsOppPiecePinned

IsOppNotForkIsMyPieceNotPinned

NumMovesMyKing

NumNotMovesOppKing

MyKingProxRook

OppKingDistRook

MyPiecesSameLine

OppPiecesNotSameLine

IsOppKingProtectingPiece

IsMyKingProtectingPiece

Genome Summary

47

KQRKQR Endgames – New Opponent

CRAFTY, second in the 2004 Computer

Chess Championship (held at Bar-Ilan)

Uses brute force methods; State-of-the-

art search algorithms

Specializes in Blitz games (typically 3

minutes per game)

We limited to 5 secs per move, enough to

scan ~1.5 Million boards with pruning

48

KQRKQR Endgames – Our parameters

Used lookahead of 2 Typically ~5 secs per move

Simple Minimax search, but not Alpha-Beta

Played 5-6 moves per game

Never cancelled a game, even if it started

with mate-in-4 (which CRAFTY easily saw) Played each position 2 times, switching places

~100 games - reduce noises in starting

positions

49

Results: Master

50

Results: CRAFTY

51

Multiple Endgames Aim for general-purpose strategies

All endgames used during evolution

Results:

%Wins%Advs%Draws

Master6268

CRAFTY2472

52

Sample GP-EndchessTree 0: (If3 (Or2 (Not (Or2 (And2 OppPieceAttUnprotected NotMyKingInCheck) (Or2 NotMyPieceAttUnprotected 100*Increase))) (And2 (Or3 (And2 OppKingStuck NotMyPieceAttUnprotected) (And2 OppPieceAttUnprotected OppKingStuck) (And3 -1000*MateInOne OppKingInCheckPieceBehind NotMyKingStuck)) (Or2 (Not NotMyKingStuck) OppKingInCheck))) NumMyPiecesUNATT (If3 (< (If3 (Or2 NotMyPieceAttUnprotected NotMyKingInCheck) (If3 NotMyPieceAttUnprotected #NotMovesOppKing OppKingInCheckPieceBehind) (If3 OppKingStuck OppKingInCheckPieceBehind -1000*MateInOne)) (If3 (And2 100*Increase 1000*Mate?) (If3 (< NumMyPiecesUNATT (If3 NotMyPieceAttUnprotected -1000*MateInOne OppKingProxEdges)) (If3 (< MyKingDistEdges #NotMovesOppKing) (If3 -1000*MateInOne -1000*MateInOne NotMyPieceATT) (If3 100*Increase #MovesMyKing OppKingInCheckPieceBehind)) NumOppPiecesATT) (If3 NotMyKingStuck -100.0 OppKingProxEdges))) (If3 OppKingInCheck (If3 (Or2 NotMyPieceAttUnprotected NotMyKingInCheck) (If3 (< MyKingDistEdges #NotMovesOppKing) (If3 -1000*MateInOne -1000*MateInOne NotMyPieceATT) (If3 100*Increase #MovesMyKing OppKingInCheckPieceBehind)) NumOppPiecesATT) (If3 (And3 -1000*MateInOne NotMyPieceAttUnprotected 100*Increase) (If3 (< NumMyPiecesUNATT (If3 NotMyPieceAttUnprotected -1000*MateInOne OppKingProxEdges)) (If3 (< MyKingDistEdges #NotMovesOppKing) (If3 -1000*MateInOne -1000*MateInOne NotMyPieceATT) (If3 100*Increase #MovesMyKing OppKingInCheckPieceBehind)) NumOppPiecesATT) -1000*MateInOne)) (If3 (< (If3 100*Increase MyKingDistEdges 100*Increase) (If3 OppKingStuck OppKingInCheckPieceBehind -1000*MateInOne)) -100.0 (If3 (And2 NotMyPieceAttUnprotected -1000*MateInOne) (If3 (< NumMyPiecesUNATT (If3 NotMyPieceAttUnprotected -1000*MateInOne OppKingProxEdges)) (If3 (< MyKingDistEdges #NotMovesOppKing) (If3 -1000*MateInOne -1000*MateInOne NotMyPieceATT) (If3 100*Increase #MovesMyKing OppKingInCheckPieceBehind)) NumOppPiecesATT) (If3 OppPieceAttUnprotected NumMyPiecesUNATT MyFork)))))Tree 1: (If3 NotMyPieceAttUnprotected #NotMovesOppKing 1000*Mate?)Tree 2: (If3 1000*Mate? NumMyPiecesUNATT -1000*MateInOne)

53

Summary

Draw and better against Master-defined

Draw against a world class opponent On limited conditions (not a full game, time ,etc.)

Shows deep search may have an alternative

Fast, pattern-oriented search suggests

more human resemblance

Search and lookahead are still important

54

Future Work Add more pieces Improve evolution speed

Parallel nets Stronger board representations

Develop more cognitive models using evolution

Search scheme space as well as game space

Tackle beyond endgames Openings and mid-game

General game concept schemas (?)