Embed Size (px)
Citation preview
WHAT YOU DON’T KNOW CAN KILL YOUACTIONABLE ANALYTICS – USING BETTER DATA BETTER
Reach Engage Earn
JAMES GWERTZMANCEO, PLAYFAB
DAMON DANIELICEO, APPURI
About Us
• Former Founder/CTO, Z2(acquired by King.com in 2015)
• Core Designer of Xbox Live (Gamertag ‘d’)
• Filed over 20 patents
• Masters in Computer Science, University of Washington
• BSEE, University of Washington
James GwertzmanCEO, PlayFab
• Former Founder, Sprout Games(acquired by PopCap in 2005)
• Launched and led PopCap’s 1st F2P studio in China
• At Code.org, led Hour of Code campaign
• Named on several patents for server technology developed while at Microsoft
• A.B. CS, Harvard
Damon DanieliCEO, Appuri
Vanity metrics are not actionable
Game Events Metrics
?
Good Acquisition
Stagnant Cohort
Performance
Poor Retention
Daily retention cohorts only show the symptoms
Now what?
Using metrics more effectively is about learning
Source: Lean Startup Framework
Idea
Try
Data
21,340
46,953
982 1,273 484 1,162 4,881 5,495 3,846 3,728 3,651
14,200
-44,257
-91,368
-12,092 -11,355-3,308
-7,042
-19,343
-9,996-2,423 -891 -543 -852
Acquired Level 1 Level 2 Level 3 Level 4 Level 5 Level 6+ Level 10+ Level 20+ Level 30+ Level 40+ Level 50+
Population Churned
“Ideas” takes work – explore the data
Where are users pooling and dropping
out?
What is different between users who progress and those
who don’t?
Idea
Try
Data
21,340
46,953
982 1,273 484 1,162 4,881 5,495 3,846 3,728 3,651
14,200
-44,257
-91,368
-12,092 -11,355-3,308
-7,042
-19,343
-9,996-2,423 -891 -543 -852
Acquired Level 1 Level 2 Level 3 Level 4 Level 5 Level 6+ Level 10+ Level 20+ Level 30+ Level 40+ Level 50+
Population Churned
Come up with a hypothesis Idea
Try
Data
“Users who progress are earning achievements early”
Test your idea with experiments Idea
Try
Data
Targeted push notification campaign
Test your idea with experiments Idea
Try
Data
Changing game behavior in real time
Test your idea with experiments Idea
Try
Data
A/B test to try multiple tests at the same time
21,340
46,953
1,212 1,354 578 1,239 5,388 6,323 4,231 3,923 4,123
15,300
-44,257
-76,368
-12,092 -10,447-3,043
-6,479
-17,796-9,196
-2,229 -820 -500 -784
Acquired Level 1 Level 2 Level 3 Level 4 Level 5 Level 6+ Level 10+ Level 20+ Level 30+ Level 40+ Level 50+
Population Churned
Measure the results of your change Idea
Try
Data
Did the change reduce the
churn?
Requirements
• Deeper access to your data than many analytics tools allow
– Consider Amazon Redshift – provides direct SQL access
• Rich set of events – you can’t measure what you don’t capture
• Tools to easily run experiments w/o writing new code
– Especially on mobile… don’t resubmit through Apple!
Questions? Tweet @playfabnetwork
Learn more: https://playfab.com
http://appuri.com
Follow us:@playfabnetwork, @appuricorp
https://playfab.com/blog/
