Expand a Data warehouse with Hadoop and Big Data

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.1

Expand your Data Warehouse with new Data Streams

Jean-Pierre DijcksTeam Lead – Big Data Product Management


Agenda

Big Data Technologies

Big Data Trends and Patterns

Next Generation Data Infrastructure

Big Data Appliance

Summary


New as well as Proven Technologies

The Hadoop Family The NoSQL Family Event Processing Technologies Oracle Database / Oracle Exadata And more…


Hadoop

Framework for distributed processing

Large Data Sets

Clusters of Computers

Simple Computing Models

Highly Available Service


Cross-Platform Strengths

Exadata+

Oracle Database

Extreme Performance Highly Secure Analytic SQL Rich Tool Set Vast Expertise

Big Data Appliance+

Hadoop

Low-cost Scalability Flexible Schema on

Read Abstract Storage Model Open Rapid Evolution


New as well as Proven TechnologiesA simple segmentation of usage

NoSQL DB

RDBMSHadoop

Usage Logs

Aggregation

Correlation

Counters

ProfilesRecommendations

Transactions


New as well as Proven TechnologiesA simple segmentation of usage

NoSQL DB

RDBMSHadoop

Usage Logs

Aggregation

Correlation

Counters

ProfilesRecommendations

Transactions

Event Processing

Interaction


MEDIA/ENTERTAINMENTViewers / advertising effectivenessCross Sell

COMMUNICATIONSLocation-based advertising

EDUCATION &RESEARCHExperiment sensor analysis

Retail / CPGSentiment analysisHot productsOptimized Marketing

HEALTH CAREPatient sensors, monitoring, EHRsQuality of care

LIFE SCIENCESClinical trialsGenomics

HIGH TECHNOLOGY / INDUSTRIAL MFG.Mfg qualityWarranty analysis

OIL & GASDrilling exploration sensor analysis

FINANCIALSERVICESRisk & portfolio analysis New products

AUTOMOTIVEAuto sensors reporting location, problems

GamesAdjust toplayer behaviorIn-Game Ads

LAW ENFORCEMENT & DEFENSEThreat analysis - social media monitoring, photo analysis

TRAVEL &TRANSPORTATIONSensor analysis for optimal traffic flowsCustomer sentiment

UTILITIESSmart Meter analysis for network capacity,

Fact: Data Expansion

ON-LINE SERVICES / SOCIAL MEDIAPeople & career matchingWeb-site optimization


EDUCATION &RESEARCHExperiment sensor analysis

Retail / CPGSentiment analysisHot productsOptimized Marketing

HEALTH CAREPatient sensors, monitoring, EHRsQuality of care

LIFE SCIENCESClinical trialsGenomics

OIL & GASDrilling exploration sensor analysis

GamesAdjust toplayer behaviorIn-Game Ads

LAW ENFORCEMENT & DEFENSEThreat analysis - social media monitoring, photo analysis

UTILITIESSmart Meter analysis for network capacity,

Fact: Data Expansion

ON-LINE SERVICES / SOCIAL MEDIAPeople & career matchingWeb-site optimization

Finding and Monetizing Unknown Relationships

Analyze Bigger, More Diverse Data Sets

Data Driven Business Decisions

MEDIA/ENTERTAINMENTViewers / advertising effectivenessCross Sell

COMMUNICATIONSLocation-based advertising

HIGH TECHNOLOGY / INDUSTRIAL MFG.Mfg qualityWarranty analysis

FINANCIALSERVICESRisk & portfolio analysis New products

AUTOMOTIVEAuto sensors reporting location, problems

TRAVEL &TRANSPORTATIONSensor analysis for optimal traffic flowsCustomer sentiment


Capturing Data ExpansionExpand Data Warehouse with Data Reservoir

MartsData Warehouse

Σ Σ

BusinessIntelligence

• Online• Scalable• Flexible• Cost

Effective

Hadoop


Capturing Data ExpansionInstant Responses to Streaming Data

Data WarehouseBusinessIntelligence

• Online• Scalable• Flexible• Cost

Effective

Hadoop

Event Decisions

NoSQL


Oracle Big Data Solution

Oracle BI Foundation Suite

Oracle Real-TimeDecisions

Endeca Information Discovery

Business Analytics

Oracle Advanced Analytics

OracleDatabase

Oracle Spatial & Graph

Big Data Platform

Oracle Big Data Connectors

Oracle DataIntegrator

Fast Data

Oracle Event Processing

Apache Flume

OracleGoldenGate

Oracle NoSQL

Database

Cloudera Hadoop

Oracle R Distribution


Expanding the DWArchitecture

DataData Warehouse

Big Store

HadoopRDBMS



Data

Batch processing of data moves toHadoop and enables more cycles for analytics in the DW

1

Other diverse streams of data enter theHadoop reservoir for processing and correlation or exploration

2



Data

New data sets are continuouslygenerated and moved for bothmass-comsumption and further analysis in the DW

1


Expanding the DW for Streaming Data Logical Architecture

Data

Data Warehouse

Cache

Big Store

Event Processor



Data

Hold data for a short time then flush

Main analytics and publishing system

Long term retention and long running analytics

Act on data as it streams

EP NoSQL

Hadoop

RDBMS



Data

Data is received and kept for a “short duration” to enablereal-time monitoring and views

2

Without moving it, data is exposed (sometimes aggregated) to the real-time DW

3

Without moving it, data is exposed to Hadoop processingenabling an up-to-date view

4

Data is transported and actedupon while streaming using event processing logic

1



Data

New data sets (aggregations and more)are continuously created in the DW for mass-consumption and consolidatedanalysis

2Raw data is flushed into the

Hadoop storefor long term storage and

analytics onraw data (plus aggregation)

1

As data is flushed, aggregatesare materialized in the DW toavoid double work

1



Data

Business logic (scores, profiles) aremoved into the cache for fast look-upby the Event Processors

1

1

Event Processing reads relevantdata with low latency from cache

2

3

Decisions and results are stored with context in Big Store 4

Results are continuously evaluatedand scores etc are re-calculated






Business Analytics


OracleDatabase


Big Data Platform



Fast Data


Apache Flume

OracleGoldenGate

Oracle NoSQL

Database

Cloudera Hadoop



Big Data Connectors and Data Integrator

Big Data Appliance+

Hadoop

Exadata+

Oracle Database

15TB / hour10x Faster


Analyze All Your Data In-Place

Big Data Appliance+

Hadoop

Exadata+

Oracle Database

Advanced Analytics


Expose All Data to End Users

Big Data Appliance+

Hadoop

Exadata+

Oracle Database

Endeca

OBI EE






Business Analytics


OracleDatabase


Big Data Platform



Fast Data


Apache Flume

OracleGoldenGate

Oracle NoSQL

Database

Cloudera Hadoop


Scalable key-value store

Scalable, low-cost data storage and processing engine

Statistical analysis framework


Big Data Appliance for Hadoop and NoSQL

Multi-Purpose Big Data Platform

Simplified Operations

ComprehensiveSecurity

Lower TCO than DIY Hadoop


Big Data Appliance

Includes all components of Cloudera Enterprise and Add-ons – Cloudera CDH

– Cloudera Impala

– Cloudera HBase (with Apache Accumulo)

– Cloudera Search

– Cloudera Manager (incl. BDR and Navigator)

Oracle NoSQL Database Oracle R Distribution

Multi-Purpose Big Data Platform


Big Data Appliance

Beats a DIY Cluster on: Initial Cost Time to Value Operations Performance

Lower TCO than DIY Hadoop

40%Cost Savings

33%Faster Time to

Value

DIY BDA$0k

$700k


Big Data Appliance

Out-of-the-Box Performance One-command:

– Installation

– Patching

– Updating

– Expansion

Enterprise Manager Plug-in for Big Data Appliance Single Contact for all Support

Simplified Operations


Big Data Appliance

Authenticate Authorize Audit

Comprehensive Security

DatabasesRelational Data

HadoopNon-Relational Data

Operating Systems

Audit Vault


Western European Media CompanyCreating a linked customer analytics system

Objectives

Maximizing customer value Optimizing campaign cost through

Automation and Targeting

Solution

Single, rich customer repository based on Big Data Appliance and NG Data® Lily®

Analytics drive: subscriber management (up-sell/cross-sell,

churn, conservation) editorial use (article engagement, adapt content

over time)

- Toyota Global Vision

Customer Data Store

Digital, RDBMS, External

BDA

Mobile

Web

Subscribers

NG DataLily

Customer Analytics

Phase 1: Improved Data Quality Single View of all Customers improves

customer management

Benefits

Social

CustomerAnalytics &

Aggregated data

Oracle Data Warehouse

Business Objects


US-based BankLowering Costs by Simplifying IT Infrastructure

Objectives

Comply with regulations requiring more data to support stress testing

Reduce IT costs & streamline processing by eliminating duplicate data stores

Solution

Single, reliable BDA/Exadata-based ODS supporting all downstream systems

Landing zone & archival repository for both structured & unstructured data

Use Exadata as “19th” BDA node

- Toyota Global Vision

Operational Data StoreMainframe,

RDBMS, more

BDA Exadata

• Agile business model

• All data• De-normalized

& Partial-normalized

• Normalized• Aggregate

data• EDW

Oracle Enterprise Manager

Oracle Data Integrator

Data Delivery

Master

S1

Master

S2

Master

SnSOA/API

CRMSOther

Fast access to 85% more data Lower costs, simplified architecture and

fast time to value

Benefits


Thomson ReutersIdentifying Cross Sell and Upsell Opportunities

Objectives

Maximize cross-sell opportunities Lower cost and complexity

Solution

Economically capture all customer activity Testing 50M events/sec ingest rates into

BDA and Oracle NoSQL DB Feeds Exadata EDW for customer

profitability & segmentation analysis

Rick KingChief Operating Officer for TechnologyThomson Reuters

“Oracle's engineered systems… are geared toward high performance big data delivery - and that is exactly the type of work we do”

BDA Exadata Exalytics

EDWSandbox & DR

Event Capture & Store

Interactive Analytics

Research Applications

Upsell/Cross Sell


Summary

Business requirements are here now– Your end users want more analytics and more data

Technology is available– Hadoop, NoSQL, Event Processing are commercially available

Technology is increasingly easy to consume– Oracle Big Data Appliance, Oracle Exadata make deployment much

simpler compared to DIY systems

Your competitors are doing this today – so should you!

The Future is Now!

