UCT for tactical assault planningRadha-Krishna Balla and Alan Fern, 2009

Seminar paper

Đorđe Relić

dorde.relic@stud.unibas.ch

University of Basel

November 2015

Content overview

Problem: tactical planning in RTS games

AI planners for RTS games

Monte Carlo and UCT

UCT in tactical assault planning

Experimental results

November 2015 Đorđe Relić @ University of Basel 2

Real-time strategy games

November 2015 Đorđe Relić @ University of Basel 3

Warcraft II (1995 by Blizzard)

Problem : defeat the enemy

Three sub-problems:

– Unit control

– Strategic planning

– Tactical planning

November 2015 Đorđe Relić @ University of Basel 4

Real-time strategy games

AI planners for RTS games

Choosing an action sequence that leads to victory

– Scripting

– Dynamic scripting

– Monte Carlo planning

November 2015 Đorđe Relić @ University of Basel 5

Photo credits Microsoft Research

November 2015 Đorđe Relić @ University of Basel 6

Multi-armed bandit

November 2015 Đorđe Relić @ University of Basel 7

Monte Carlo planning

UCT

November 2015 Đorđe Relić @ University of Basel 8

November 2015 Đorđe Relić @ University of Basel 9

UCT

UCT

November 2015 Đorđe Relić @ University of Basel 10

UCT

November 2015 Đorđe Relić @ University of Basel 11

November 2015 Đorđe Relić @ University of Basel 12

UCT

November 2015 Đorđe Relić @ University of Basel 13

UCT

UCT

November 2015 Đorđe Relić @ University of Basel 14

UCT

November 2015 Đorđe Relić @ University of Basel 15

UCB rule: 𝑄+(𝑠, 𝑎) = 𝑄 𝑠, 𝑎 + 𝑐 ×log 𝑛 𝑠

𝑛 𝑠 , 𝑎

𝑛 𝑠, 𝑎 ← 𝑛 𝑠, 𝑎 + 1 ; 𝑛 𝑠 ← 𝑛 𝑠 + 1

𝑄 𝑠, 𝑎 ← 𝑄 𝑠, 𝑎 +1

𝑛 𝑠, 𝑎[𝑅 − 𝑄 𝑠, 𝑎 ]

UCT for tactical assault planning

November 2015 Đorđe Relić @ University of Basel 16

Experiment results

Wargus and Stratagus

12 scenarios

Two objective functions

Comparison with baseline algorithms:– Random

– Attack closest

– Attack weakest

– Stratagus AI

– Human player

November 2015 Đorđe Relić @ University of Basel 17

Experiment results

November 2015 Đorđe Relić @ University of Basel 18

Experimental results

November 2015 Đorđe Relić @ University of Basel 19

Experimental results

November 2015 Đorđe Relić @ University of Basel 20

Experimental results

November 2015 Đorđe Relić @ University of Basel 21

Conclusion

Brief intro to RTS games

UCT algorithm

UCT applied to RTS tactical assault

Experimental results

November 2015 Đorđe Relić @ University of Basel 22

Questions ?

Thank you for your attention