Modelling Soccer ML/DM for Soccer Analytics2 © Fraunhofer Institut für intelligente Analyse- und Informationssysteme IAIS Seite 7 Arimaa played on a chess board four traps: c3, f3,

1

© Fraunhofer Institut für intelligente Analyse- und Informationssysteme IAIS

Thomas Gärtner

joint work with Olana Missura, Ulf Brefeld, Dirk Hecker, ...

ML/DM for Soccer Analytics


Seite 2

Modelling Soccer

multi-agent 3d game

11 vs 11 (active agents, „players“)

+ (passive agents, „ball“)

+ (stationary objects, „goals“)

goal:

goal:

abilities of the active agents vary and

depend on the position

dataevent streams

…

sensor readings(up to) 100Hz per leg

1000Hz goal keeper hands1000Hz ball(s)


Seite 3

How can ML/DM help Soccer Teams

in-game

tactics tracking / discovery / adaptation

player (team and opponent) performance monitoring

between games

player selection

team composition

team tactics

opponent tactics and performance

player recovery and training


Seite 4


ML/DM for Game Analyticsas a means to analyse tactics in soccer?

Dynamic Difficulty Adjustmentas a means to effective training / recovery?


Seite 5

Soccer Tactics vs Pattern Mining

patterns = abstractions from actual movements/positions

on subset of active players

frequent pattern samplingpro: patterns that occur more frequently in data

will appear more frequently whendrawn at random

pro: whole pattern space is covered

con: there are a lot of infrequent patternsso chance of drawing one is high

?: draw „discriminative“ patterns


Seite 6

Soccer Tactics vs Kernel Methods

patterns = groups of similar „situations“

kernels for structured data

represent individuals by terms

in a typed -calculus

represent sets, multisets, etc. naturally

pro: defines default kernel

con: dynamics make things expensive

?: how about kernels based onthe voronoi diagram

or its planar dual

2


Seite 7

Arimaa

played on a chess board

four traps: c3, f3, c6, f6

pieces set up at the beginning

pieces of different strength

Elephant, caMel, Horse, Dog, Cat, Rabbit

pieces can assist each other

stronger pieces can push/pull weaker pieces

winner is who reaches the other side of the board with one rabbit first


Seite 8

Arimaa

developed such that computers won’t be good

high branching factor per turn-> no fun using minimax, etc

each of the 16 pieces can move in four directions

online archive

every(?) game since 2002 on the original server

from 3MB/year (2003) to 15MB/month (jan, 2012)

for free!!!


Seite 9

Arimaa Logs

id wplayerid bplayerid wusername

busername wtitle btitle wcountry bcountry wrating brating

wratingk bratingk wtype btype event site timecontrol postal

startts endts result termination plycount

mode rated corrupt movelist events


Seite 10

Arimaa Example

25b ec1w Rb2e eb1n ra2s[-2 0 3 -2 0 0 0 0][ 0 -7 0 2 0 0 0 0][ 0 2 0 0 0 0 0 0][ 0 0 0 -2 0 0 0 0][ 2 0 0 5 0 0 0 0][ 0 2 0 4 0 0 6 -2][ 2 0 0 0 0 0 -2 0]

[ 2 5 0 0 0 0 -5 -2][ 2 4 0 0 0 0 -4 -2][ 2 3 0 0 0 0 -3 -2][ 2 6 0 0 0 0 -6 -2][ 2 7 0 0 0 0 -7 -2][ 2 3 0 0 0 0 -3 -2][ 2 4 0 0 0 0 -4 -2]

2w Md2n Md3w Db2n Dg2n[ 2 5 0 0 0 0 -5 -2][ 2 0 4 0 0 0 -4 -2][ 2 3 6 0 0 0 -3 -2][ 2 0 0 0 0 0 -6 -2][ 2 7 0 0 0 0 -7 -2][ 2 3 0 0 0 0 -3 -2][ 2 0 4 0 0 0 -4 -2]

...


Seite 11

Arimaa Starting State


Seite 12

Arimaa Final State

3


Seite 13

Arimaa Final State wpid:866


Seite 14

Arimaa Board Patterns (coupled from the past)

3357/3601 9:2 25:2 11:4 57:2915/ 776 47:-3 49:2 24:-2 25:2 33:2 8:-2 57:2 48:-2 16:-2 54:-4 17:22226/1870 47:-3 24:-2 25:2 56:-2 1:2 9:2 48:-2 41:2 64:-2 17:2 40:-23595/3952 47:-3 49:2 40:-2 33:2 57:2 64:-2862/ 968 33:2 8:-2 14:-3 9:2 41:2 16:-2 64:-2 17:21816/1991 49:2 24:-2 25:2 18:3 32:-2 8:-2 56:-2 57:2 1:2 9:2 64:-2 17:22730/2569 47:-3 49:2 25:2 32:-2 8:-2 9:2 16:-2 40:-22449/2571 49:2 23:-3 40:-2 33:2 32:-2 57:2 48:-2 41:2 64:-2 17:2

59/ 33 23:-3 25:2 8:-2 57:2 1:2 9:2 41:2 64:-2 35:6 17:2 63:-4190/ 181 23:-3 25:2 33:2 57:2 1:2 31:-6 14:-4 40:-2

(the board after 25%-75% of the moves)


Seite 15

Learning from Arimaa with Board-Kernels


Seite 16


0) material kernel

1) board kernel


Seite 17


0) material kernel

1) board kernel


Seite 18


2) position kernel

3) piece kernel

4


Seite 19

Winner Prediction (Top Players, Position Kernel)


Seite 20

So... Soccer Analytics?


Seite 21


ML/DM for Game Analyticsas a means to analyse tactics in soccer?

Dynamic Difficulty Adjustmentas a means to effective training / recovery?


Seite 22

What is the Goal of Game Design?


Seite 23

What is the Goal of Game Design?

skill level

dif

ficu

lty level

hence, we investigate

DynamicDifficultyAdjustmentin Games


Seite 24

Engagement leads to Flow

anxiety

arousal

flow

boredomrelaxationapathy

worry control

skill level

dif

ficu

lty level

“the mental state of operation in which a

person in an activity is fully immersed in a

feeling of energized focus, full involvement,

and successin the process of the

activity”

[Csíkszentmihályi, Mihály (1975), Beyond Boredom and Anxiety, San Francisco, CA: Jossey-Bass]

5


Seite 25

What is Difficulty Adjustment?

skill level

dif

ficu

lty leve

l

we want an algorithm which

adapts the difficulty of a game, e.g., the

strength of opponents

based on observations of the players behaviour

or success within one game


Seite 26

The Problem from a Game Developers Perspective

perceived difficulty(in games)

speed of enemieshealth of enemies

frequency of powerups

power of character

distraction of player

power of enemies

gameplayexperience

dif

ficu

lty

time


Seite 27

The Problem from a Game Developers Perspective

easy hard insanenormal

perceived difficulty(in games)

speed of enemieshealth of enemies

frequency of powerups

power of character

distraction of player

power of enemies

gameplayexperience


Seite 28

The Problem from a Players Perspective

?

easy hard insanenormal


Seite 29

[ Heuristic Approach for Adaptive Games ]

choose some evaluation function

for each turn:

order possible moves according to the evaluation function

if opponent picked i-th best move then also pick i-th best move


Seite 30

[ Heuristic Approach for Adaptive Games ]

how can we keep games `interesting’?

6


Seite 31

[ Player Modelling for Adaptive Games ]


Seite 32

A more sound Ansatz

Formulate Dynamic Difficulty Adjustment as a Prediction Game

1) game master predicts a difficulty setting

2) player plays one round of the game in this setting

3) game master observes whether the setting was

too easy, just right, or too difficult

this game is played on a partially ordered set

reflecting the `is more difficult’-relation

[ Learning on Posets ? ]

33

learning monotone concepts

active learning – yes

PAC learning – yes

online learning – yes

learning intersections

active learning – no

PAC learning – yes

online learning – yes

learning with inconsistent labels

with 1-ary & 2-ary queries – no

with 3-ary queries – yes


Seite 34

Partially Ordered Difficulty Settings

most difficult setting

easiest setting

suppose a fixed labelling:too easy, just right, too difficult


Seite 35

zero tolerance: observe

too easy


Seite 36

zero tolerance: infer

too easy

too easy

too easy

too easy

7


Seite 37

zero tolerance: update

xx x

x© Fraunhofer Institut für intelligente

Analyse- und Informationssysteme IAIS

Seite 38

zero tolerance: observe

too difficult x

x xx


Seite 39

zero tolerance: infer

too difficult

too difficult

too difficult

xx x

x© Fraunhofer Institut für intelligente

Analyse- und Informationssysteme IAIS

Seite 40

zero tolerance: update

xx x

xxx x


Seite 41

?zero tolerance: which setting to choose next?

xx x

xxx x


Seite 42

zero tolerance: bounds

in each turn choose the – in the worst case – most informative setting

i.e. such the the minimum of

{ | settings that are more difficult |, | settings that are more easy | }

is maximised

can not be worse that doing halving on each path

8


Seite 43

How to deal with changing skills / labels (-> POSM)

We maintain a weight reflecting our belief that each vertex is “just right”

Each round:

we update belief for all vertices for which we

can infer that they were not “just right”

we choose the difficulty setting such that we gain the most information

how well does our game master algorithm perform?

(compared with the best static difficulty setting chosen in hindsight)


Seite 44

POSM: maintains belief


Seite 45

POSM: observe

too easy


Seite 46

POSM: infer

too easy

too easy

too easy

too easy


Seite 47

POSM: update


Seite 48

POSM: update

9


Seite 49

POSM: observe

too difficult


Seite 50

POSM: infer

too difficult

too difficult

too difficult


Seite 51

POSM: update


Seite 52

POSM: which setting to choose next?

?


Seite 53

Algorithm: Partially Ordered Set Master


Each round:







Seite 54

POSM: (above, below) - belief

10


Seite 55

POSM: (above, below) - belief

how much belief can we update if we

observe “too difficult”

how much belief can we update if we

observe “too easy”


Seite 59

POSM: propose and play at argmax min( above, below )

(0.5, 8.5)

(1.0, 6.0)

(1.5, 5.0)

(2,0, 2.0)

(2.5, 1.0)

(1.5, 5.5)

(3.5, 4.0) (6.5, 1.0)

(1.5, 4.0) (6.0, 3.0) (7.0, 1.5) (8.5, 0.5)

0.5

1.0

1.5

1.5

3.5

1.5

3.0

2.0

1.0

1.5

1.0

0.5


Seite 60

Algorithm: Partially Ordered Set Master


Each round:







Seite 61

Bounds

zero tolerance:

on each path we are doing halving

partially ordered set master (simplified view):

on each path we are doing multiplicative updates

number of mistakes setting k*

would have made until time tnumber of mistakes POSM

made until time t

if a mistake was made


Seite 62

General Posets

on any chainon any chainfor any chain on the “heaviest” chain

over any vertex over any vertex on the “heaviest” chain

weights restricted to the respective chainweights restricted to the respective chainbeliefs restricted to the respective chain

in each iteration, the overall belief can be reduced at least by a constant factor


Seite 63

A Bound for the Partially Ordered Set Master

number of mistakes of POSM

number of mistakes of the best static difficulty setting (chosen in hindsight)

bound gets worse, the larger the maximum

antichain is

update strength

11


Seite 64

Simulations

chain grid

smo

oth

no

n-s

mo

oth


Seite 65

Adversarial


Seite 66

Adversarial (smooth)


Seite 67

Results

no

n-s

mo

oth

smo

oth

simulated human adversary


Seite 68

Experiments with Real Games

without human involvement:

how well do we match/estimate the opponent’s skill level (skill level = minimax search depth)

how often do we win/loose


Seite 69

Skill Estimate in Checkers

true le

vel

Move

pre

dic

ted

skil

l le

vel

2

3

4

5678

12


Seite 70

Skill Estimate in Chinese Chess

2

3

4

5

6

true le

vel

Move

pre

dic

ted

skil

l le

vel


Seite 71

Winning-Ratio in Real Games

8+

Opponent Strength

Op

po

nen

t’s W

inn

ing

-Rati

o

6+

5+

1

POSM

diffic

ulty

settin

g

ideal goal


Seite 72

Winning-Ratio in Real Games

8+

Opponent Strength

Op

po

nen

t’s W

inn

ing

-Rati

o

6+

5+

1

POSM

diffic

ulty

settin

g

ok results


Seite 73

Winning-Ratio in Checkers

8+

Opponent Strength

Op

po

nen

t’s W

inn

ing

-Rati

o

6+

5+

1+

POSM

diffic

ulty

settin

g


Seite 74

Human Player Results

** * * *

*

*


Seite 75

getting more human empirical evidence

experiments in “real” games, teaching systems, sports training, ...

extending POSM abstract convexity spaces, continuous parameters, ...

getting smaller pathcovers

learning/updating the partial order as we go

analysing the real world

real world is not worst case, take smoothness into account

getting things together automated content/level generation for games

Future Work with you?

Documents

Modelling Soccer ML/DM for Soccer Analytics2 © Fraunhofer Institut für intelligente Analyse- und Informationssysteme IAIS Seite 7 Arimaa played on a chess board four traps: c3, f3,