Data Driven Insights: Cypher 2017 presentation

Data-driven Insights

Sept 21st 2017

Presentation by:

Gurpreet Singh & Gopi Suvanam

G-Square Solutions

Data Driven Insights – An Introduction

Need of Insights for Business: Business leaders and analytics teams are looking to derive meaningful actionables from the business data, on the go.

Conventional BI tools wont work: MIS/BI reports or only visualizations will not help the cause – what is needed is an ability to get data-driven insights which can help them get the pulse of the business and suggest meaningful action points.

Key is AI and Machine Learning: Data driven insights are the fuel for every business decision being taken. Rather than relying on an analyst to get insights, business users can get insights directly from Robo-data-scientists using AI and machine learning.

A point of view can be a dangerous luxury when substituted for insight and understandingMarshall McLuhan, Canadian Communications Professor

Data-to-Insight-to-Action journey

Source: Genpact report on Data-to-insight

The critical aspect in the Analytics function

which is often neglected is the Data-to-

insight and Insight-to-action journey.

1. Provide Visibility – Descriptive

analytics

2. Manage effectiveness – Insighting on

the data

3. Execute actions – Prescriptive

analytics

‘He who searches for pearls must dive below’ – John Dryden

How will we prefer to infer data?

‘Once we know something, we find it hard to imagine what it was like not to know it’ Chip & Dan Heath, Authors of Made to Stick, Switch

Some insighting illustrations

Opportunities multiply as they are seized – Sun Tzu

Insig

hts

Solving the problem of automated insighting

Break insighting into

several sub problems Analyse data for each sub-

problem Do you care: Are the

insights interesting

enough?

Breaking down

the problem

Is there a seasonality of trend?

Are KPIs related to any factors strongly?

Are factors related to each other?

Is there any interaction effect?

Are there sub-trends in Factors?

What are the causal relationships?

Analysing each part: Deep data-mining

ARIMA

Model

Multivariate

analysis

Mutual

Information

Deeper

relationships

through

Recursive Trees

Causality

test

Outliers

A note on Mutual Information

› Variance gives dispersion in normal distribution

› Entropy gives a measure understand dispersion for any distribution

› 𝐻 𝑋 = 𝑋 𝑝 𝑋 𝑙𝑜𝑔1

𝑝(𝑋)

› Mutual information is a measure of the mutual dependence between the two variables.

› 𝐼(𝑋; 𝑌) = 𝑥,𝑦 𝑝 𝑥, 𝑦 𝑙𝑜𝑔𝑝(𝑥,𝑦)

𝑝(𝑥)𝑝(𝑦)

› Point-wise mutual information is similar to mutual information, but it refers to a single

event whereas mutual information is the average of all possible events.

Consolidating

Insights

Is the insight statistically

significant?

Are the underlying variables

important for the user?

This is where use of machine

learning becomes important to

identify the right set of insights,

curate them and present them in

an automated fashion

Narrating Insights: Natural Language Generation

Generate several templates

Keep the language simple and direct

Take care of grammar

Make it interesting

Use right ajectives

Insights to automated decision making

Automated

insighting tool

should also give

out insights in

structured format

{

"insight_type": "Trend_Analysis_Variable",

"sub_type": "month"

"priority": ""high",

"variable": "Region",

"level":"Vidarbha & Chattisgarh“

"insight": "Total PARValue has increased over one month

for WB & OR by 27.0% not changed muchVidarbha &

Chattisgarh by 0.0%", "Total PARValue has overall

decreased however comparatively lowest fall over one

quarter for Vidarbha & Chattisgarh by -27.5% and

significantly decreased for East - UP by 120.4%",

"campaign_text": "Focus on Vidarbha & Chattisgarh"

}

Automated Insights in Action : Narrator

Tech Stack

Columnar DB for

structured data:

MonetDB

NoSQL DB for

structured data:

MongoDB

PythonPandas, Numpy,

Scipy, NLTK

D3JS Angular

15

Q & A

Let data Insights lead the way forward.

+91 22 43470408

info@g-square.in

www.g-square.in

Mumbai, India

@company/g-square-

solutions-pvt-ltd

@GSquareSolution

@GSquareSolutions1