Embed Size (px)
Citation preview
HOW MACHINES LEARN TO
PLAY
O F E R EG O Z I
ARTIFICIAL INTELLIGENCELet’s start from the basicsActually, let’s go straight to the cutting edge!Here’s a demo for a multi-layered convolutional neural
network using feed-forward training to perform supervised learning
Well-defined rulesA clear, measurable goalA task one can train forBonus: human opponents!
THE ‘ADDITRON’
Josef Kates1950
Josef Kates1950
EARLY DAYS – MINIMAXBuild a tree of game states
(from current state) Well-defined transition rules
Define a function to score each state
How close are we to the goal (a winning board)?
Choose path that maximizes our gain and minimizes opponent’s gains
Toc-Tac-Toe has only 765 unique states… Solved!!
5 x 1020 (500 billion billion) possible positions
Arthur L. Samuel1956
1989Chinook
1992
Marion Tinsley
199219941995
“With his passing, we lost not only a feared adversary but also a friend. Every member of our team had the deepest respect and admiration for Tinsley. It was a privilege to know him”
2007
10120 possible positions (checkers squared)
1950
1957
160,000 positions per second1980
Alpha-beta pruning
19961997
200 million positions per second
GENERIC LEARNING
So far, humans were central in the learning process
Pre-encoding the allowed movesProviding the winning states
Can machines learn on their own, like real toddlers?
2013
https://arxiv.org/pdf/1312.5602v1.pdf
1) 33600 raw pixels2) Target scoreInput:
2013
Output: player!
• We’re skipping entire ML courses now
• What’s fundamentally different about Deep Learning?• No predefined rules – a generic system• “A bishop moves this way, and a knight this way…”
• No domain knowledge – system “finds” the features• “count #pieces within 3 steps from the king”
DEEP LEARNING
• We’re skipping even more entire ML courses now
• Uses Artificial Neural Network, with LOTS of data• Deep Learning == multiple hidden layers
DEEP LEARNING
Magic happens HERE…Magic built-in
2016#positions > #atoms in universe1202 CPUs and 176 GPUs
GO
HOW ABOUT AI BUILDING THE GAME?...
GENERATIVE LANGUAGE MODELS
• Not done with skipping ML courses just yet • First, let’s divert to literature for a bit, shall we?...
• “Robert Cohn was once middleweight boxi?”
https://medium.com/@ageitgey/machine-learning-is-fun-part-2-a26a10b68df3
GENERATIVE LANGUAGE MODELS
• Feed the book into an RNN, let it train itself…100 iterations:
1000 iterations:
GENERATIVE LANGUAGE MODELS
• Feed the book into an ANN, let it train itself…<10K iterations:
GENERATIVE MODELS
• Now back to games…
GENERATIVE MODELS
• Extracted and encoded all game levels• Initial iterations build junk• After enough training iterations – it works!
THANK YOU!
GENERATIVE ADVERSARIAL NETWORKS (GANS)
