Big Data for Product Managers

© 2015, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 1

Ben Hopkins Product Marketing Manager - Pentaho

Jim Stascavage Vice President of Engineering - ESRG

Big Data for Product Managers


①  Data Trends & Data Types

②  Big Data Challenges & Technology Solutions

③  Big Data & Product Innovation

④  Case Study: ESRG

Quick Agenda


Pentaho We Enable & Empower Data-Driven Businesses

Customer momentum •  Over 1,500 commercial customers

•  Over 10,000 production deployments

Innovation through open source •  Open, pluggable, purpose-built for the future

•  Sustained leadership in Big Data ecosystem with technology innovation

Modern, cohesive business analytics & data integration platform •  Full spectrum of analytics for all key roles

•  Embeddable, cloud-ready analytics

•  Broadest and deepest Big Data integration


2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020

40000 30000 20000 10000

Exa

byte

s of

Dat

a

Source: IDC’s Digital Universe Study, sponsored by EMC, April 2014

We are ONLY here!

77% of data relevant to enterprises will be unstructured

At the Beginning of a Data Revolution

40% Machine Data

50X Growth


The Most Compelling Insights Come from Blending Data

38% 38%

16% 4% 4%

Existing Underutilized “Dark Data”

More Customer & Supplier Detail

Social Media Content

Commercially Available Data

Publicly Available Data

+ + + +

Big ROI Opportunity

“Which source of data represents the most immediate opportunity to transform your business?”

Summary of Analyst Surveys on Big Data: Gartner, Forrester and Ventana Research


Traditional ERP, CRM, and transactional data

Web logfile, clickstream and social media post data

Human language, text, email, audio, video, and image data

Sensor, machine-to-machine, network, and geospatial data

More Data Types of Interest


Addressing New Challenges

Flexibility to land a wide variety of data types in the data store thanks

to schema-on-read

Highly varied data structures are difficult to bring into relational DBs

and blend for analysis

‘Divide and conquer’ with distributed storage and processing on

affordable hardware

Explosive growth in data volume is straining existing data warehouse

infrastructure

Leverage high performance random read/write access, streamline

analytic queries for speed

Time-sensitive data is being rapidly generated, and there is pressure to

deliver faster analytics

Big Data Challenges Technology Solutions


Big Data Technologies Not an Exhaustive List

Hadoop •  Distributed file

system and MapReduce framework

•  Ideal for high volume, diverse data processing

NoSQL •  Broad group of

DBs with flexible data models: graph, key/value, document, etc

•  Often ideal for rapid ingestion, random read/write access

Analytic DB •  Relational DB

designed for high performance BI

•  Ideal for complex analytic/OLAP queries


Why Does it Matter for Product Management?

New Opportunities for Valuable Products

Potential for Built-in Intelligence

Future-Proof for Scale and Growth


Big Data Products

Source: Aaron Kimball, “The secrets of designing and building big data apps,” venturebeat.com, 12/24/2013

•  Data structured in pre-defined

(relational) way

•  Designed for specific problem; data access, interfaces & protocols reflect that purpose

•  Application data often isolated from other relevant data

•  Example: CRM system for storing customer info, prioritizes calls based on purchase data

Traditional Application

•  Accommodates various data

structures (and speeds) •  Framework that can potentially

solve multiple problems

•  Includes process for ingesting new data sources and building & iterating on predictive models

•  Example: Adds in prioritization of calls from predictive model on customer behavior & purchase data

Big Data Application


To create completely individualized apps •  Understand all relevant

user intents, actions

•  Leverage mobile, behavioral, & profile data

•  Incorporate more user data for a more complete view

•  Predictive modeling to anticipate and respond

Big Data + Predictive Analytics

Source: “Predictive Apps are the Next Big Thing in Customer Engagement,” Forrester, 6/25/2013


Architecture Patterns We See

Weblog & social media data

Machine, sensor, & device data

Customer profile data

Existing application data

Unstructured & Semi-structured

Structured & Relational

Hadoop Cluster

Relational Database

NoSQL Store

Analytic Database

Client-side User Interface

Web-based

experience including embedded visual

analytics

NoSQL

‘Massive Archive’ ‘Operational Speed Layer’

‘Existing System of Record’

‘Powerful BI Performance’


In Brief - Use Cases & Products

“Personalization Engines” •  Using predictive analytics on web-

sized data sets to individualize customer relationships

•  Example: RichRelevance delivers personalized content to customers online and in-store, based on Big Data predictive analytics on over 50 mln shopping sessions per day. •  Tech: Hadoop, Hbase, Hive,

others

“Machine Data Analysis” •  Monitoring and analyzing sensor

and network data to understand equipment/device performance

•  Example: Ruckus Wireless leverages Big Data to provide decade-long analysis of semi-structured WiFi network data for telecom carriers and enterprises •  Tech: Hadoop, Vertica, others

ESRG Customer Case Example Industrial Predictive Analytics

Jim Stascavage – VP, Engineering

Presentation overview

•  Introduc)on

•  Os)aEdge industrial predic)ve analy)cs and Pentaho integra)on

•  Marine case example: Saving fuel and avoiding failure

•  Liquid packaging case example: Predic)ng )me to failure

•  Conclusion

15

Founded 2000 10+ years product development

History

Exper6se

Product

Reliability engineering So>ware & architecture Big-‐Data

Currently remotely monitoring 3,000+ assets daily Comprehensive, 6ered solu)on

Markets served

Defense & Commercial Mari6me, Process, Power Gen

ESRG overview

Focus & results

Turn data into ac6onable informa6on; examples: •  $70,000 annual savings per gas turbine •  Defer ~60% of in-‐person equipment assessments

Industrial Analytics to create significant value…

17

Cisco es)mates 50B devices connected,

crea)ng addi)onal $14T in profits over next

decade

McKinsey & Co. es)mates up to $6.2T in annual value

created annually by 2025

General Electric es)mates the market for industrial internet technology and services to grow to $500B by 2020

Sources: Manyika, James, others, “Disruptive technologies: Advances that will transform life, business and the global economy” McKinsey Global Institute, May 2013 Chambers, John, “Internet of Evertyhing”, Cisco, February 21, 2013 General Electric press release, June 18, 2013

…ESRG uses data to improve return on industrial assets

Source: Bringing the industrial internet to the maritime industry and ships into the cloud ;http://www.esrgtech.com/company/ESRGcontent/ 18

Avoid breakdowns & down6me

Op6mize maintenance

Improve environmental compliance

Improve opera6ons

Improve energy efficiency

Marine Vessel $500K-‐$1.5M Per year savings poten6al

Liquid filling 10-‐20% up6me

10-‐50% maint cost Per year savings poten6al

Our technology monitors 3,000+ assets

~3,000 total assets monitored

AC plants Hydraulic systems Compressors GT engines GT lube oil systems Diesel engines GT Generators Reduc)on gears/transmissions Refrigera)on Desaliniza)on Fuel flow meters Misc

3/24/15 ESRG Confidential 19

20

Mechanical equipment

Fuel/energy consump)on

Control & opera)ons

Emissions, Discharges, etc

1000s of sensors per asset… …across an enterprise …automated analy6cs avoid need for large Department to analyze “Big Data”

Automated analy6cs & Experts overcome Big Data challenges

Once per second data = 86,400 points/day 1000 data points = 2.6B points/month

100 ships = 3.1 trillion data points per year

Automated analy)cs provide fuel/energy efficiency and equipment health

ESRG uses analytics to turn Big Data into actionable information

OstiaEdge overview

21

On-‐site / Onboard Local

Central / Shore Central or Cloud

Plant Edi)on Central Edi)on Business Intelligence

•  Local data acquisi)on & qualifica)on

•  Local analy)cs and presenta)on

•  Real-‐)me data viewer

•  Machine state and condi)on analy)cs

•  Embedded Keble (PDI) •  Web/cloud presenta)on •  Workflow management •  Security/user mgt

•  Dashboards & analyzers

•  ETL •  Email reports •  Mobile •  NEW: PDI: R & Weka

embedded

Example data flow and integration

3/24/15 ESRG Business Sensitive 22

Data Input

External

ETL PDI

RUL Algorithm & Engine

DSP R-‐Plugin

Customer maintenance

system

External

Os)aEdge Analy)cs

Os)aEdge Presenta)on

Analyzers & Dashboards

Custom Dashboards

BI Cube(s)

Email Reports

Configura)on Management

New

Saving Fuel and Avoiding Failure Case Example: Maritime

23

Op)mize generators

$50K-‐$250K Tune equipment

$50K-‐$150K Avoid failure

$10K-‐$500K

Large ship owner trying to reduce fuel & failures/down)me: •  $5-‐10M in fuel cost per year •  $10,000 per day for vessel down)me

Solu6on

Situa6on

Use Os)aEdge & embedded Pentaho ETL to make beber opera)ons & maintenance decisions •  Embedded Pentaho ETL for generator op)miza)on & dashboards •  Os)aEdge analy)cs for failure avoidance

New: Predicting Time to Failure Case Example: Liquid packaging

24

R & Weka based RUL algorithms

Predict Failure Standard & Custom Dashboards

Exec Transparency

Global liquid packaging OEM with two goals: •  Improve customer up)me •  Reduce unnecessary maintenance and extra parts consump)on

Solu6on

Situa6on

Leverage Pentaho PDI + Os)aEdge to predict Remaining Useful Life (RUL) •  ETL to bring in enterprise level data •  Data Science Pack (R & Weka) used to design algorithms to predict RUL •  Customized embedded dashboard

Industrial Analytics Opportunity

25

Os6aEdge + Pentaho •  ETL •  Diagnos)cs •  Analyzers &

Dashboards •  New:

Prognos?cs with R & Weka

Lower Cost & Faster •  Small team •  Rapid & agile

algorithm development

•  Easy integra)on •  Flexible

implementa)on

Avoid breakdowns & down6me

Op6mize maintenance

Improve environmental compliance

Improve opera6ons

Improve energy efficiency

Q and A …

Ask Questions. Our team is standing by to help. The webinar slides will be posted to our website and our Slideshare.net/aipmm page. The webinar recording will be posted at AIPMM.net for members.

Product Management Body Of Knowledge

We will pick one winner from our attendees.

15 Compe66ve Intelligence Ques6ons that Product Managers Need To Ask Mar 13

AIPMM Webinar Series: hbp://aipmm.com/aipmm_webinars

Topic Sugges)ons: [email protected] Announcements: hbp://www.aipmm.com/subscribe LinkedIn: hbp://www.linkedin.com/company/aipmm Membership: hbp://www.aipmm.com/join.php Cer)fica)on: hbp://aipmm.com/html/cer)fica)on/

Upcoming Webinar