2
busuu is the world’s leading social network for language learning
Language courses Social network
+• Access to native speakers• Peer to peer text corrections
• High quality courses in 12 languages• Beginner to advanced intermediate level
1 2
How does busuu work?
Most important vocabulary
Key grammar
Practice with native speakers
Faster fluency
busuu is a complete self-study and language practice environment
3
busuu 2016
What sort of data do we use?
● Front end tracking data
● Progress data
● Backend db data
● Third party data
busuu 2016
Problems
My data says X, why does yours say Y?
Cloudwatch Alert!
“Why can’t i find the results of my A/B test till tomorrow again?
“Oh my god, do we really have to put yet another tracker in?”
busuu 2016
Snowplow delivery
How do we get Snowplow to deliver the events to everybody/thing that needs it, instead of adding more trackers to the frontend
busuu 2016
Data Collection Phase
Events Events Backend Data
API Calls
Yet to be done
Scala StreamCollector
busuu 2016
Processing
Validation● Customised busuu event
schemas● Different based on environment
Enrichments● ip lookup● Forex Conversion
busuu 2016
Plug & Play Integrations
18
● One source of truth● Scalability● Third party systems can be added very quickly
busuu 2016
Lambda?
19
Parse through each field of enriched data looking for custom schema
name
One lambda function per type of data and
per integration
Relay required data to third party service
through REST api or given python client
busuu 2016
Main implementation bugbears
1. Strict Multi Platform Schemas
2. Offline mode delay
3. Device vs Collector Timestamps
busuu 2016
Future projects
● Live A/B test trains & results
● Live machine learning results in app
● Automated alerting on complex company metrics.
busuu 2016
Frontend event data
● Track and find issues in user behavior.● Insight into product usage● A/B Testing● CRM cohorting● In-app message cohorting
busuu 2016
Progress Data
● What has a user learnt?● How is our content performing● What is their language level?● Vocabulary lists
busuu 2016
Backend Data
● What are the user’s attributes● Social relationships (friends)● Writing exercises and comments