Hot Technologies of 2013: Investigative Analytics

H T Technologies 2013

HOST: Eric Kavanagh

THIS YEAR is…

INVESTIGATIVE ANALYTICS =

�  Seeking previously unknown patterns in data

�  Extracting real-‐time insight from machine generated data

�  Being able to query data streams, i.e., mobile data, web logs, geospatial data, social media data, etc.

ANALYST:

Philip Howard Research Director, Bloor Research

ANALYST:

Robin Bloor Chief Analyst, The Bloor Group

GUEST:

Don DeLoach CEO & President, Infobright TH

E LINE UP

INTRODUCING

Philip Howard

Exploiting the Internet of Things with Investigative Analytics

Philip Howard Research Director, Bloor Research

telling the right story Confidential © Bloor Research 2013

Internet of Things

+ trains, golf courses, icebergs, ATMs, pipeline networks ….


Investigative Analytics

" What happened? " Why did it happen? Is this part of a pattern that indicates that it

might happen again? " How are we going to react? If it is part of a pattern how can we

can leverage this for business purposes in the future?


Some use cases

IBM X-Force survey


Requirements


INTRODUCING

Robin Bloor

INVESTIGATIVE ANALYTICS +

THE INTERNET OF THINGS

It Begins With State…

People, objects, systems, system

components, etc.

Things can report state

RFID tags, sensors, log files, tweets, etc.

Such snippets of data are events

EVERYTHING HAS STATE

Transactional Event Based

�  Corresponds to a system change

�  Process heavy/data light �  Analysis happens

downstream �  Flows as part of a

business process �  Fast

�  Corresponds to a state change

�  Process light/data heavy �  Analysis can happen

pre-transaction �  Can be a trigger in a

business process �  Faster

Transactions v Events

The Technology March

The Three Latencies

Time to develop

Time to deploy

User experience

Boiling It Down

It is all about TIME TO INSIGHT – as long as that is followed by ACTION

INTRODUCING

Don DeLoach

Infobright: Investigative Analytics for The Internet of Things

Don DeLoach, CEO, Infobright [email protected]

Internet of Things

Graphic from Sensor Mania! The Internet of Things, Wearable Computing, Objective Metrics, and the Quantified Self 2.0,JSAN, Nov. 2102

Requirements for Practical Investigative Analytics

LOW TOUCH HIGH AVAILABILITY

AFFORDABILITY TCO

AD HOC PERFORMANCE SCALABILITY

COMPRESSION

LOAD SPEEDS

§ Data management § Hadoop transforming this area

§ Transparent analytic stack § Operational, investigative, predictive § Machine-generated, text

§ User consumption: § Real-time, interactive visualization & query creation

Emerging Data Analytics Stack: Days of One-Size-Fits All Are Gone

“Yesterday’s BI-‐ETL-‐EDW stack is wrong-‐sided for tomorrow’s needs, and quickly becoming irrelevant.” Gigamon

Intelligence Not Hardware: Knowledge Grid

• Stores it in the Knowledge Grid (KG) • KG is loaded into memory • Less than 1% of total compressed data size

Creates informa?on (metadata) about the

data upon load, automa?cally

• The less data that needs to be accessed, the faster the response

• Sub-‐second responses when answered by the KG

Uses the metadata when processing a query to

eliminate / reduce need to access data

• No need to par??on data, create/maintain indexes, projec?ons or tune for performance

• Ad-‐hoc queries are as fast as sta?c queries, so users have total flexibility

Architecture Benefits

Infobright Analytic Suite

Investigative Analytics for Machine-generated Data: §  High performance ad-hoc query capabilities—enabling real-time information insights at

the speed of business

§  Extremely efficient (footprint, compression, data load) analytic engine designed for enterprise software deployments, OEM/embedded configurations and enterprise-ready appliance configurations proven in production

§  Install to analytics in hours: Infobright is designed for time to value

§  Integrated with the leading Hadoop, BI and ETL players

Operational Simplicity

High Performance

Efficient Form Factor

Infobright sets the bar for query performance, form

factor, and analytics business impact

AFTER BEFORE

What is needed today (and tomorrow)?

MACHINE DATA

MACHINE DATA

DATABASE ADMINISTRATORS

HARDWARE

HARDWARE

APP

LIC

ATIO

N

APP

LIC

ATIO

N

Embedded Database for M2M/Internet of Things

Low Admin: Do not want to force users to require DBAs to keep solution running

Load Speeds: Ingestion rates continue to increase, placing heavy burden on solutions

High Compression: Want to keep longer histories in less space

Lower TCO: Resulting in better value for customers, better margins for providers

Stripped Away “DBA” tax requirement required by previous versions

Ingesting over 1TB/Hour, with significant headroom beyond that

Over 3X the retention period and a 5X simultaneous reduction in storage requirement

Lower TCO for users, higher margins for JDSU

Little to No Admin

Fast Load

Speeds

20:1+ Compression

Exceptional Ad

Hoc Query Performance

Very Low TCO

REQUIREMENTS EXAMPLE: JDSU


Low Admin: Looking for would ensure customers have fast access to data

Load Speeds: Handle projected 70% growth rate in mobile messaging

High Compression: Need to increase data stored without increase in storage requirements

Lower TCO: Competitive flexibility of lower cost with higher value-add services

No indexes, data partitioning or manual tuning. No need for dedicated DBAs.

100,000 records per second at peak making data available for analysis within minutes Increased to 90 days of data stored in less hardware due to drastic compression

TCO only 20% of the cost of competitors. Major wireless carrier wins with this solution

Little to No Admin

Fast Load

Speeds

20:1+ Compression

Exceptional Ad


Very Low TCO

REQUIREMENTS EXAMPLE: MAVENIR


Low Admin: Do not want to force users to require DBAs to keep solution running

Fast Query Performance: Customers depend on this analysis to tune networks

High Compression and Fast Load Speeds: Need to meet business growth projections

Lower TCO: Resulting in better value for customers, better margins for providers

Low touch administration reduces friction and latency for queries

Sub-second web-based queries critical to customers to tune the network

High data compression rates and load speed allow for projected growth rate of data volume

Low OPEX = better margins and more confidence planning capacity to meet growth

Little to No Admin

Fast Load

Speeds

20:1+ Compression

Exceptional Ad


Very Low TCO

REQUIREMENTS EXAMPLE: POLYSTAR


High Compression: Projected data growth outpacing storage capacity

Ad hoc Query: Utilities want to drive customer participation in efficiency-related programs

Fast Load Speeds: Need to integrate several data streams quickly

Lower TCO: Solution needs to affordably meet business needs

No additional hardware or manual set-up in the form of data indexing or partitioning

Fast flexible reporting (20K reports in first 3 months) help utilities better drive business

Better business answers due to combined analysis of behavioral, demographic and log data

Low TCO translates to better pricing and stronger competitive positioning

Little to No Admin

Fast Load

Speeds

20:1+ Compression

Exceptional Ad


Very Low TCO

REQUIREMENTS EXAMPLE: OPOWER

Momentum in the M2M/Internet of Things

Applications in the Internet of Things will all require Low Touch, High Capacity and High Density; and Low Cost deployments

Smart Grids

Smart Vehicles,

Smart Cities

Mobile Health

Others..

BEFORE

MACHINE DATA

DATABASE ADMINISTRATORS

HARDWARE

APP

LIC

ATIO

N

AFTER

MACHINE DATA

HARDWARE APP

LIC

ATIO

N

Thank You

The Archive Trifecta: •  Inside Analysis www.insideanalysis.com •  SlideShare www.slideshare.net/InsideAnalysis •  YouTube www.youtube.com/user/BloorGroup

THANK YOU!