Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... ·...

Big Data Platform Overview

Alex Hay (athay@us.ibm.com), Big Data CTPMeridee Lowry (meridee@us.ibm.com), Big Data CTP

April 30th, 2014

Big Data is a Concept

Big Data

2

IBM Big Data and Analytics Offerings

Watson Foundations

Information Integration & GovernanceINFORMATION SERVER, MDM, G2, GUARDIUM, OPTIM

Exploration, landing and

archive

Trusted data

Reporting & interactive analysis

Deep analytics & modeling

Real-time processing & analyticsSTREAMS, DATA REPLICATION

Operational systems

BIGINSIGHTS

PUREDATAHADOOP

DB2, INFORMIX

PUREDATA TRANSACTIONS

PUREDATA ANALYTICS

DB2 BLU

PUREDATA ANALYTICS

DB2 WAREHOUSE

PUREDATA OPERATIONAL

ANALYTICS

Actionable insight

Decision management

Predictive analytics and modeling

Reporting, analysis, content

analytics

Discovery and exploration

SPSS MODELER

COGNOS BICOGNOS TM1

WATSON EXPLORER

SPSS MODELER GOLD

Image and video

Data types

Transaction andapplication data

Machine andsensor data

Enterprise content

Social data

Third-party data

3

Online Retailer Scenarios

4

The Big Outdoors Company

Fictitious company

IBM-generated data for sales

Company stats

144 Products

3,500 Unstructured Data Items

5

Scenario Zero Demo

6

Scenario 1: Populate the Landing ZoneValue: Process and explore data more quickly

Information Movement, Matching & Transformation

Landing, Exploration& Archive

Data

Sources

Structured

Operational

Unstructured

External

Social

BigSheets

Actionable

Insights

Explore Landed

data using

BigSheets

7

Scenario One Demo

8

BigSheets - Spreadsheet Interface

9

Scenario 2: Perform Contextual DiscoveryValue: More quickly search all forms of data

Landing, Exploration& Archive

Data

Sources

Structured

Operational

Unstructured

External

Social

Actionable

Insights

Exploration &

Discovery

10

Bringing Big Data to the Point of Impact

Commenting

Rating

SharedFolders

Tagging

Social Tools

FileSystems

RelationalData

ContentManagement

Email

CRM

SupplyChain

ERP

RSS Feeds

ExternalSources

Cloud

CustomSources

Application/Users

Wa

tso

n E

xp

lore

r

11

BIG DATA PLATFORM

Stream Computing

Hadoop System

Analytics – Clustering

12

Scenario 3: Perform Text Analytics & Merge DataValue: Report and Retain Lineage from Hadoop

Information Movement, Matching & Transformation

Landing, Exploration& Archive

Data

Sources

Structured

Operational

Unstructured

External

Social

Actionable

Insights

Apply Text

Analytics

Merge Structured

w/ Unstructured

Data

BI & Performance

Management

13

Scenario Three Demo

14

Text Analytics – Processing of Unstructured Data

How it works• Parses unstructured text and detects

meaning with annotators

• Understands the context in which the text is analyzed

• Hundreds of pre-built annotators for names, addresses, phone numbers, and others

Parts of speech support forp English, Spanish, French, German, Portuguese, Dutch, Japanese, Chinese

• Distills structured info from unstructured text

Benefits

• More precise and correct answers

• 50% faster than manual method

• Run faster text analysis

Football World Cup 2010, one team distinguished themselves well, losing to the eventual champions 1-0 in the Final. Early in the second half, Netherlands’

striker, Arjen Robben, had a breakaway, but the keeper for Spain, Iker Casillasmade the save. Winger Andres Iniesta

scored for Spain for the win.

Unstructured text (document, email, etc)

Classification and Insight

15

Scenario 4: Enhanced Product and Consumer Data Value: 360-degree view

Information Movement, Matching & Transformation

Landing, Exploration& Archive

PureData for

Operational

Analytics

Data

Sources

External

Social

BI & Performance

Management

Actionable

Insights

Apply Text

Analytics

with SDA

Enterprise Warehouse

Add value to

existing

Customer data

by applying

Social Profiles

16

Scenario Four Demo

17

Scenario 5: Better decision making using Predictive AnalyticsValue: Predict more often, more granular using more data

Landing, Exploration& Archive

Data

Sources

Structured

Operational

Unstructured

External

Social

Predictive Analytics

& Modeling

Actionable

Insights

18

PureData for

Operational

Analytics

Enterprise Warehouse

Scenario 6: Real-time data mining & Text Analytics Value: Improve Customer Satisfaction & Sales

Real-Time Analytics

Data

Sources

Structured

Operational

Unstructured

External

Social

Sensor

Geospatial

Time Series

Streaming Predictive Analytics

& Modeling

Actionable

Insights

Real-time

(Predictions)

Real-time Text

Analytics

19

Scenario 7: Encryption and Auditing of BigInsightsValue: Improve Data Security through Governance

Analytic Appliances

Security, Governance and Business Continuity

Landing, Exploration& Archive Enterprise

Warehouse

Data Marts

Data

Sources

Structured

Operational

Unstructured

Sensor

Geospatial

Time Series

BI & Performance

Management

Actionable

Insights

20

IBM Big Data and Analytics Offerings

Watson Foundations

Information Integration & GovernanceINFORMATION SERVER, MDM, G2, GUARDIUM, OPTIM

Exploration, landing and

archive

Trusted data

Reporting & interactive analysis

Deep analytics & modeling

Real-time processing & analyticsSTREAMS, DATA REPLICATION

Operational systems

BIGINSIGHTS

PUREDATAHADOOP

DB2, INFORMIX

PUREDATA TRANSACTIONS

PUREDATA ANALYTICS

DB2 BLU

PUREDATA ANALYTICS

DB2 WAREHOUSE

PUREDATA OPERATIONAL

ANALYTICS

Actionable insight

Decision management

Predictive analytics and modeling

Reporting, analysis, content

analytics

Discovery and exploration

SPSS MODELER

COGNOS BICOGNOS TM1

WATSON EXPLORER

SPSS MODELER GOLD

Image and video

Data types

Transaction andapplication data

Machine andsensor data

Enterprise content

Social data

Third-party data

21

Thank You!

22