30
GAME ANALYTICS The challenges of mobile Free- To-Play games Christian Beckers Analyst at Outplay Entertainment Christian.beckers@outplay .com Dundee Data Day 5 th of December 2015

Game analytics - The challenges of mobile free-to-play games

Embed Size (px)

Citation preview

Page 1: Game analytics - The challenges of mobile free-to-play games

GAME ANALYTICSThe challenges of mobile Free-To-Play

gamesChristian BeckersAnalyst at Outplay [email protected]

Dundee Data Day5th of December 2015

Page 2: Game analytics - The challenges of mobile free-to-play games

Spend Hard Levels Fall Off

Page 3: Game analytics - The challenges of mobile free-to-play games

What we can learn• Levels with high drop off but

low spend -> improve balancing

• Losing lots of players early -> Rework tutorial to better make them stick

• Levels with high spend and low drop off -> the “perfect” levels, have a closer look to determine what people like and what makes them spend

…and this is only a very small picture of all the data we collect!

Page 4: Game analytics - The challenges of mobile free-to-play games

Established 2011 in Dundee

Outplay Entertainment is a fast growing, venture-backed Developer and Publisher of social and

mobile Games.

Page 5: Game analytics - The challenges of mobile free-to-play games

The role of a data analyst•Explore!

•Explain!•Experiment!

Excite!

Page 6: Game analytics - The challenges of mobile free-to-play games

Tasks of a game analyst• Measure KPI to understand performance of the games• Find and explain issues and potentials in the game from the

gathered data• Optimise different factors and settings • Understand the users playing the game • Aid the decision making process • Find in-depth answers for the questions and verify the assumptions

of the project team• Present all findings in a compact and understandable way

Page 7: Game analytics - The challenges of mobile free-to-play games

The challenges of a data analyst• You have to understand the game and the free-to-play model to

understand the data!• You have to use databases to gather exactly the data you need!• You have to write scripts and functions to help you clean transform

and summarise the data!• You have to deal with millions of data rows!• You have to know which changes are just random occurrences and

which are significant developments!• You have to communicate your findings to people unfamiliar with

statistics and probability!

Page 8: Game analytics - The challenges of mobile free-to-play games

No analyst can work without data!and

Your analyses are only as good as your data!

Page 9: Game analytics - The challenges of mobile free-to-play games

Getting data• Every piece of data has a certain

cost• Collect as little data as possible while

still being able to perform all analyses• Only gather data on a significant

sample of all usersMixpanel 3rd party service

Postgresql database OLGA

R / csv Files

Report / Summary

Export / Transform

Query

Analyse /Summarise

• API export and data import into database and transformation need loads of maintenance• Analyst working tightly with

DevOps team• Game might be buggy or outdated• Need to stay flexible to work

around missing data or inconsistency issues

Game app

Send data

• Games can be played offline• Some data will be reported late• We need a certain export delay to

allow offline mode data to come through

Page 10: Game analytics - The challenges of mobile free-to-play games

Understanding data!• The data comes as events and

properties• An event is triggered by a certain

condition and sends a new data point to the servers

• Every event has certain properties providing background information on the event and the user triggering it

• In the database, each event corresponds to a row in a table and the properties correspond to the columns

Example:I successfully finish level 12 in Alien Creeps on my iPhone and then close my session.The game now sends the two events to the server.After the export to the database we get the new rows:

Game

event Level_id

User_id Platform

Other properties

AC Level_finished

12 Christian’s Id

iOS e.g. Outcome = win

AC Session_closed

Christian’s Id

iOS e.g. Duration = 120 s

Page 11: Game analytics - The challenges of mobile free-to-play games

Understanding data!Challenges are:• Multiple games, all with slightly different analytics. Templates used

to prevent inconsistencies• Good knowledge of the game. Understanding how an event is fired

and what it means in the context of the game• Players on different versions. Need to be aware of major difference

and adapt analyses accordingly• Select data on the right users, especially if sampling was

introduced

Page 12: Game analytics - The challenges of mobile free-to-play games

Cleaning and transforming dataWe need to tidy up all of our data to be able to use it!

Reasons:• Hackers and company QA users skew the data and have to be

identified and removed. • Some data is incomplete due to communication errors or bugs and

has to be completed e.g. by extrapolating other events of the user• You might need data from different sources and combine the

information you get through different APIs

Page 13: Game analytics - The challenges of mobile free-to-play games

ANALYSING DATA!Time to get our hands dirty!

Page 14: Game analytics - The challenges of mobile free-to-play games

Types of Analyses

•Exploratory analysis

•Investigative analysis

•Experiment analysis

Page 15: Game analytics - The challenges of mobile free-to-play games

Exploratory Analysis• Dig into the data to find new correlations, causations or

segmentations • Look at the data in many different ways, e.g. plot lots of different

factors against each other and look for potential relations between variables

• Use machine learning techniques to find relations the eye cannot see by itself

Challenges:• Restrict to certain aspects of the game (e.g. social interactions) to

keep the amount of data manageable• Correlation does not equal causation!

Page 16: Game analytics - The challenges of mobile free-to-play games

Investigative Analysis• Starts with a questions, e.g. “Why do so many users quit after level x?”• Select data specifically to answer the question• Look into every aspect of the game connected to the questions (e.g. also

consider in-game messaging, tutorial steps, boosts used)

Challenges:• The question might be too wide or too specific• Collecting data to only answer the question might remove the big picture

(e.g. the drop off is caused by the levels before level x as well)• Correlation does not equal causation!

Page 17: Game analytics - The challenges of mobile free-to-play games

Experiment AnalysisMultivariate Experiment• Look at multiple factors with a

distinct high and low setting• Groups implement a combination of

high/low settings for the factors• Determine the impact each factor

has on the KPI while keeping control of interference effects

• Use experimental design to reduce the number of groups needed (otherwise 2^Number of factors groups needed)

A/B test

Page 18: Game analytics - The challenges of mobile free-to-play games

Multivariate experiment

Page 19: Game analytics - The challenges of mobile free-to-play games

Multivariate ExperimentCo

ntro

l gr

oup

Retention Day 7 / Day 14

Revenue Day 7 / Day 14

Page 20: Game analytics - The challenges of mobile free-to-play games

Experiment AnalysisMultivariate Experiment• Look at multiple factors with a

distinct high and low setting• Groups implement a combination of

high/low settings for the factors• Determine the impact each factor

has on the KPI while keeping control of interference effects

• Use experimental design to reduce the number of groups needed (otherwise 2^Number of factors groups needed)

A/B test• Look at a number of test groups

(usually 1-3) and compare them against the control

• The groups all change the same factor or factors to avoid unforeseen interference ruining the results

• When the result is a very significant change, follow up tests are used to further optimised the tested factors

• Make sure the observed differences are statistically significant!

Page 21: Game analytics - The challenges of mobile free-to-play games

HOW TO: A GAME ANALYSIS

Methods and tools used for analyses

Page 22: Game analytics - The challenges of mobile free-to-play games

SegmentationWe often segment the users we look at, some examples:• Country• Purchasers/Non-Purchasers• Platform/Store• Social activity• Date of first launch

First launches

Purchases

Page 23: Game analytics - The challenges of mobile free-to-play games

Lifestories – The backbone of any analysis

All events selected of a user or cohort of users with selected properties

This allows us to do all analyses!

user_id event_name timestamp store session_counter current_level outcome furthest_level cumulative_game_time_seconds attempts continues_used boosts_used

aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 first_launch 30/11/2015 21:36 Google Play 0 1 1 0 0 0

aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 adjust_attribution 30/11/2015 21:36 Google Play 0 1 1 1 0 0

aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 21:38 Google Play 0 1 won 2 65 1 0 0

aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 21:39 Google Play 0 2 won 3 142 1 0 0

aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 21:40 Google Play 0 3 won 4 221 1 0 0

aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 21:41 Google Play 0 4 won 5 287 1 0 0

aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 21:52 Google Play 0 5 won 6 403 1 0 0

aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 21:55 Google Play 0 6 won 7 522 1 0 0

aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 21:58 Google Play 0 7 won 8 619 1 0 0

aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 21:59 Google Play 0 8 won 9 710 1 0 1

aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 23:26 Google Play 1 10 won 10 1075 1 0 1

aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 23:27 Google Play 1 11 won 12 1186 1 0 0

aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 23:30 Google Play 1 12 won 12 1312 1 0 2

aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 23:31 Google Play 1 13 won 14 1423 1 0 0

aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 23:35 Google Play 1 14 retried 14 1650 1 3 3

aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 23:37 Google Play 1 14 retried 14 1752 2 0 0

aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 23:39 Google Play 1 14 retried 14 1848 3 0 0

aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 23:40 Google Play 1 14 retried 14 1926 4 0 0

aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 iap_confirmed 30/11/2015 23:42 Google Play 1 14 14 2025 0 0

aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 23:45 Google Play 1 14 won 15 2216 5 0 2

Page 24: Game analytics - The challenges of mobile free-to-play games

R & R StudioWe use the scripting language R to transform, visualise and analyse our data.

R Studio is a free GUI for R with many helpful features like easily accessible help and plot windows and an environment overview.

Page 25: Game analytics - The challenges of mobile free-to-play games

R & R StudioR can be expanded easily and additional packages allow for quicker data processing and better visualisation.

In R, you can make all sorts of plots very quickly and R Studio allows to quickly go through them.

Page 26: Game analytics - The challenges of mobile free-to-play games

Sankey diagram

R even allows more complex visualisation like Sankey diagrams for flow explorations.

Page 27: Game analytics - The challenges of mobile free-to-play games

Social graph

Using R to transform data and then exporting into Gephi allows for nice and quick network visualisations and analyses.

Page 28: Game analytics - The challenges of mobile free-to-play games

Priority is Player Satisfaction and Fun“If your game sucks, data won’t save you!”

Data Analystat Outplay

“Understand and love your game!”

Page 29: Game analytics - The challenges of mobile free-to-play games

Thanks for listening!

Any Questions?

Page 30: Game analytics - The challenges of mobile free-to-play games

• Outplay.com

• Outplay.com/careers

[email protected] interesting materials:• ‘Freemium Economics’, Eric B.

Seufert• https://cran.r-project.org/• https://www.rstudio.com/• http://www.cookbook-r.com/• https://gephi.org/