Are You Ready?

Adam Wray

CEO – Basho Technologies

FIREFOX iKETTLE

FITBIT

GOOGLE GLASS

APPLE WATCH

SMART THINGS

SENSORS IN AIRPLANE ENGINES

“The Internet of Things is all about catching things in progress rather than waiting until after the fact to analyze the data,” says Paul Maritz, CEO of Pivotal

*Source IDC

THE DATA EXPLOSION

2013

4.4 ZB2020

44 ZB

If the Digital Universe were

represented by the memory in a stack of tablets…

WHAT IS DRIVING THE

Gaining insights from previously

unconnected data

Connecting people for work and play

Innovation of connected devices

DATA EXPLOSION?

80 MILLION PEOPLE1300 MESSAGES A SECOND

24 Hours a Day / 365 Days a Year

Information is requested and amended more

than 2.6 BILLION times a year

Has enabled over 42 MILLION Summary

Care Records to be created and stored

Has transmitted over 1.3 BILLION

prescription messages

NHS Data Spine

Smart Thermostat Market is expected to grow

from $146.9 million in revenue in 2014 to $2.3

billion by 2023.

-Navigant

THE CONNECTED HOME

Store historical date for analytics

Store User Schedules

Sessions Storage for Connected

Users

1.5 TRILLION RECORDS PER DAY

400% Year Over Year Increase

Ability to Monitor entire IT

environment from Single Portal

Array of Real-Time Statistics

and Insight

Centralize & Correlate Events,

Alerts, and Notifications

Cloud-based management tools and data

analytics usually only accessible to larger

companies.

Millions of transactions each day.

POS FOR OVER 20K SMALL

BUSINESSES

Quick Service

Retail

Restaurant & Bar

20 TERABYTES OF DATA PER DAY

BILLIONS OF MOBILE DEVICES

10 BILLION data transactions a

day – 150,000 a second – Apple

Forecasting 2.8 BILLION locations

around the world

Generates 4GB OF DATA every

secondWe’re focusing on

helping people make

better decisions with

the weather.

WEATHER FORECAST

PREDICTS WALMART’S SALES

Ideal BERRY weather turns

out to be low wind with

temperatures below 80

degrees.

People are more likely to

eat STEAK when it's warm

out with higher winds but

no rain, but not if it gets too

hot.

WHAT ARE THE IMPLICATIONS?

Enterprises must choose: Modernize or Sink!

• Adopt technologies to successfully manage data generated

by new data sources and consumed by users accessing

complex data types from around the world, 24 hours a day.

• Organizations must make plans to handle data ingestion,

data storage, and analytics in order to bring value to the

business.

• Increasingly look to distributed systems to help manage the

influx of traffic, and ease current operational challenges.

DISTRIBUTED WORKLOADS

App App App App

Virtualization

Server

App

Aggregation

Server Server Server Server

Client-Server Era:

SMALL APPS

BIG SERVERS

ONE LOCATION

Cloud Era:

BIG APPS

SMALL SERVERS

MANY LOCATIONS

Everything works

at small scale

What happens when

something goes wrong

The customer

experience matters

WHY DISTRIBUTED SYSTEMS?

Scale out, up and down predictably

and linearly

Survive server, network or data

center failures

Data locality enables data operations

close to end-users

DISTRIBUTED SYSTEMS

DEVELOPERS OPERATIONS CUSTOMERS SALES

Challenge 1 – Isolation of Data

The growing hype surrounding data lakes is causing substantial confusion in the information management space

Gartner

Challenge 2 – Consistency of Data

To Be or Not To Be

Consistent?

Understand the

question…

C A

PX

The CAP TheoremDr Eric Brewer

Describes the trade-offs involved in distributed systems

Challenge 3 – Data Gravity

Apps

Services

Lower Latency

&

Higher Bandwidth

Growth

Over Time Data

“Perhaps the biggest challenge is that the IoT has the potential to generate orders of magnitude more data than any other source in existence today. So, in the world of the IoT we will test the limits of ‘big.’”

Bill Franks, Chief Analytics Officer for Teradata

Putting Challenges In Perspective

DISTRIBUTED SYSTEMS AND NoSQL

• Most powerful tool in distributed systems is NoSQL

• Efficient with active workloads

• More cost effective and scalable than RDBMS

• Key Value most flexible and preferred

Usage in web applications for blogs

Session management

Chat applications

Social network management

• Others – Document, Graph, Big Table

• Riak most operationally efficientMarket is Expected to Reach

$4.2 Billion, Globally,

by 2020 - Allied Market Research

Two Sigma's vision has been to develop

technological innovations that intelligently

analyze the world's data to consistently

deliver value for their clients.

MANAGING $18B WITH TRADE DATA

Modeling teams work

with vast data sets –

from corporate

earnings to news and

weather reports.

Seek to synthesize

quantitative and

qualitative information

sources to generate

new scientific insights

with systematic

precision.

Wall Street Journal Front Page April 1, 2015

5TB per Day in Real-Time Growing > 50% per Year

Over 3K Customers, 5K Appliances, 100K+ Data Sources

on Customer Networks

SECURITY AS A SERVICE