50
at Dorota Kuleszo Francesco Mucio MicroStrategy Symposium, London, 22 april 2016

MicroStrategy at Badoo

Embed Size (px)

Citation preview

Page 1: MicroStrategy at Badoo

at Dorota KuleszoFrancesco Mucio MicroStrategy Symposium, London, 22 april 2016

Page 2: MicroStrategy at Badoo

What is ?● Social Network for meeting people near you

● Over 300M users in 190 countries

● 200 employees based in London and Moscow

● Over 100M downloads on Android

● Also on iOS, Windows Phone, Web and Mobile Web

● Very Agile working environment

Page 3: MicroStrategy at Badoo

● We needed a proper BI tool - among 14 candidates

● Our data volumes - user level data

● Environment - Linux, Database, SSO

● Technical users with high expectations

Why ?

Page 4: MicroStrategy at Badoo

● Hard to set up in our environment

● No real dimensional model

● Data/ETL Team had to prepare data for us

● Time to onboard users and earn their trust

vs

Page 5: MicroStrategy at Badoo

What does with now?● Fancy Dashboards around the office● Data Discovery tools● Analysis delivered by email● Self Service Reports● Weekly releases

90+ Users in Finance, Billing, Marketing, Developers, User Ops, Founders

Page 6: MicroStrategy at Badoo

‘s BI Architecture

Page 7: MicroStrategy at Badoo

‘s BI Architecture

Page 8: MicroStrategy at Badoo

Badoo’s Database

EXASOL is an Massive Parallel Processing (MPP) database.

It is an in memory columnar database.

● 8 Nodes (plus 1 spare) with 5.6 TB of RAM

● ~100 TB of Raw Data - ~30 TB of Data on Disk

● Each node has 8 TB of Disk, in RAID 2 and redundancy factor = 2

Page 9: MicroStrategy at Badoo

and Query Generation Time: 0:00:00.13Total Elapsed Time in Query Engine: 0:18:36.68

Sum of Query Execution Time: 0:16:08.46Sum of Data Fetching and Processing Time: 0:01:03.73

Sum of Data Transfer from Datasource(s) Time: 0:00:57.93Sum of Analytical Processing Time: 0:00:00.00Sum of Other Processing Time: 0:01:24.49

Sum of Cube Publish Time 0:19:06.37

Number of Rows Returned: 5759450Number of Columns Returned: 38Number of Temp Tables: 0Total Number of Passes: 15Number of Datasource Query Passes: 15Number of Analytical Query Passes: 0

Page 10: MicroStrategy at Badoo

Query Improvements

● Use real tables

● Use parallelization

● Use the Pre/Post Processing statements

and

Page 11: MicroStrategy at Badoo

and Query Generation Time: 0:00:00.13Total Elapsed Time in Query Engine*: 0:11:18.82Sum of Query Execution Time: 0:18:30.86Sum of Data Fetching and Processing Time: 0:01:04.82

Sum of Data Transfer from Datasource(s) Time: 0:00:59.37Sum of Analytical Processing Time: 0:00:00.00Sum of Other Processing Time: 0:01:55.12

* This report has some passes that have been executed in parallel. Individual time components may not add up to Total Elapsed Time in Query Engine.

Sum of Cube Publish Time 0:11:28.63

Number of Rows Returned: 5759450Number of Columns Returned: 38Number of Temp Tables: 17Total Number of Passes: 49Number of Datasource Query Passes: 49

Page 12: MicroStrategy at Badoo

and Query Generation Time: 0:00:00.09Total Elapsed Time in Query Engine: 0:02:43.29

Sum of Query Execution Time: 0:00:53.08Sum of Data Fetching and Processing Time: 0:00:52.09

Sum of Data Transfer from Datasource(s) Time: 0:00:47.83Sum of Analytical Processing Time: 0:00:00.00Sum of Other Processing Time: 0:00:58.10

Sum of Template Calculate Time 0:00:00.00Sum of AE Data Persisting Time 0:00:00.46Sum of Cube Publish Time 0:02:56.36

Number of Rows Returned: 4702678Number of Columns Returned: 38Number of Temp Tables: 12

Page 13: MicroStrategy at Badoo

Enable Your Users with Visual Insight

Page 14: MicroStrategy at Badoo

● High Level Dashboards

● Analysis Dashboards

● OLAP Reports

Enable Your Users with Visual Insight

Page 15: MicroStrategy at Badoo
Page 16: MicroStrategy at Badoo
Page 17: MicroStrategy at Badoo
Page 18: MicroStrategy at Badoo

● High Level Dashboards

● Analysis Dashboards

● OLAP Reports

Enable Your Users with Visual Insight

This is still not enough for our users!

Page 19: MicroStrategy at Badoo
Page 20: MicroStrategy at Badoo

Let Your Users Do the Legworkwith Transaction Services

Page 21: MicroStrategy at Badoo

Enable Your Users with Transaction Services

● Agile environment

● New analysis have an assessment period

● People just like to play with data

This was just bad

Time consumingSelf esteem problems

We would end up hating our users

Page 22: MicroStrategy at Badoo

Enable Your Users with Transaction Services

“We have a Coefficient that we would like to use in our calculation, this can be different for Campaign Media Source, Country, and Platform...”

200+ Media Sources

254 Countries

12 Platforms

200+ x 254 x 12 = 609600!

Page 23: MicroStrategy at Badoo

Enable Your Users with Transaction Services

We had to convince them to have a go withTransaction Services!

Page 24: MicroStrategy at Badoo

Enable Your Users with Transaction Services

Page 25: MicroStrategy at Badoo

Enable Your Users with Transaction Services

Page 26: MicroStrategy at Badoo

Enable Your Users with Transaction Services

Page 27: MicroStrategy at Badoo

Don’t Reinvent the Wheel

Just Use MicroStrategy

Page 28: MicroStrategy at Badoo

Don’t reinvent the wheel: use MicroStrategy

Problem: Deliver a csv file to an external location.

Proposed Solution:

❏ Generate the data

❏ Put the data on a local drive

❏ Create a tool to copy it remotely

Page 29: MicroStrategy at Badoo

Don’t reinvent the wheel: Use MicroStrategy

Page 30: MicroStrategy at Badoo

Beat The CommuteLearn to use

Command Manager

Page 31: MicroStrategy at Badoo

Save Time with Command Manager

Page 32: MicroStrategy at Badoo

Few things we do with Command Manager

● Cube Refresh

● Start Schedules

● Manage Our Users

● Configure Database Connections

Save Time with Command Manager

Page 33: MicroStrategy at Badoo
Page 34: MicroStrategy at Badoo

My two cents about Command Manager:

● Get familiar with it

● Try to script repetitive tasks

● Integrate it with other tools

Save Time with Command Manager

Page 35: MicroStrategy at Badoo

MicroStrategy Web Deployment Made Easy

Page 36: MicroStrategy at Badoo

MicroStrategy Web Deployment Made Easy

What we started with:

● MicroStrategy WAR File

● SDK Customizations

● Deployment scripts

● Settings changes

- Deployed Manually

- Deployed Manually

- Executed Manually

- Undocumented

Page 37: MicroStrategy at Badoo

MicroStrategy Web Deployment Made Easy

Page 38: MicroStrategy at Badoo

1. GIT

2. Maven

3. Jenkins

MicroStrategy Web Deployment Made Easy

Page 39: MicroStrategy at Badoo
Page 40: MicroStrategy at Badoo
Page 41: MicroStrategy at Badoo
Page 42: MicroStrategy at Badoo
Page 43: MicroStrategy at Badoo
Page 44: MicroStrategy at Badoo
Page 45: MicroStrategy at Badoo
Page 46: MicroStrategy at Badoo
Page 47: MicroStrategy at Badoo
Page 48: MicroStrategy at Badoo
Page 49: MicroStrategy at Badoo

that’s all folks… maybe

Page 50: MicroStrategy at Badoo

q & a