38
1 BIG DATA Audio / Video Log Files Text/Image Social Sentiment Data Market Feeds eGov Feeds Weather Wikis / Blogs Click Stream Sensors / RFID / Devices Spatial & GPS Coordinates Mobile WEB 2.0 Advertising Collaboration eCommerce Digital Marketing Search Marketing Web Logs Recommendations Contacts Deal Tracking Sales Pipeline ERP / CRM Payables Payroll Inventory Data Complexity: Variety and Velocity Terabytes Gigabytes Megabytes Petabytes

Big Data From Microsoft and Atidan

  • Upload
    atidan

  • View
    594

  • Download
    2

Embed Size (px)

Citation preview

Page 1: Big Data From Microsoft and Atidan

1

BIG DATAAudio / Video

Log Files

Text/Image

Social Sentiment

Data Market Feeds

eGov Feeds

Weather

Wikis / Blogs

Click StreamSensors / RFID / Devices

Spatial & GPS CoordinatesMobile WEB 2.0

Advertising

Collaboration

eCommerceDigital Marketing

Search Marketing

Web Logs

Recommendations

Contacts

Deal Tracking

Sales Pipeline

ERP / CRMPayables

Payroll

Inventory

Data Complexity: Variety and Velocity

Terabytes

Gigabytes

Megabytes

Petabytes

Page 2: Big Data From Microsoft and Atidan

How do I optimize my

fleet based on weather

and traffic patterns?

What’s the social

sentiment for my

brand or products

How do I better

predict future

outcomes?

Page 3: Big Data From Microsoft and Atidan

Increases ad revenue by processing 3.5 billion events per day

Massive Volumes

Processes 464 billion rows per quarter, with average query time under 10 secs.

Measures and ranks online user influence by processing 3 billion signals per day

Cloud Connectivity

Connects across 15 social networks via the cloud for data and API access

Uses sentiment analysis and web analytics for its internal cloud

Real-Time Insight

Improves operational decision making for IT managers and users

Page 4: Big Data From Microsoft and Atidan
Page 5: Big Data From Microsoft and Atidan

MANAGE ANY DATA, ANY SIZE, ANYWHERE

010101010101010101101010101010101001010101010101101010101010

Page 6: Big Data From Microsoft and Atidan

Extremely large volume of unstructured web logs

Ad hoc analysis of logs to prototype patterns

Hadoop data cluster feeds large 24TB cube

Business users analyze cube data

E.g. STRUCTURED & UNSTRUCTURED DATA

Page 7: Big Data From Microsoft and Atidan
Page 8: Big Data From Microsoft and Atidan

OPEN & FLEXIBLE

100% compatible with

Apache Hadoop

Accelerating the delivery

of Hadoop for Windows

Tools from a rich

ecosystem of partners

Built with close

community collaboration

Hadoop for Windows

JavaScript libraries

Hive ODBC drivers

The Apache Software Foundation

Page 9: Big Data From Microsoft and Atidan
Page 10: Big Data From Microsoft and Atidan

ENRICH BY CONNECTING TO THE WORLDS DATA

Discover

Combine

Refine

Page 11: Big Data From Microsoft and Atidan

DISCOVER DATA

FROM

TOIDENTITY

DOC CONTEXT

SOCIAL GRAPHS

DATA EXPLORER

DATA HUB

Page 12: Big Data From Microsoft and Atidan

POWER OF COMBINING THE WORLDS DATA

Value

Page 13: Big Data From Microsoft and Atidan

REFINE DATA

Enterprise Information Management & Full Analytic Spectrum

Page 14: Big Data From Microsoft and Atidan

E.g. VALUE OF EXTERNAL DATA

Page 15: Big Data From Microsoft and Atidan
Page 16: Big Data From Microsoft and Atidan

INSIGHTS ON ANY DATA, ALL USERS, WHEREVER THEY ARE

010101010101010101101010101010101001010101010101101010101010

Page 17: Big Data From Microsoft and Atidan

DEMO: FROM DATA TO INSIGHTS!

Page 18: Big Data From Microsoft and Atidan

INSIGHTS FOR ALL USERS THROUGH FAMILIAR TOOLS

PB TB GB

Page 19: Big Data From Microsoft and Atidan

BIG DATA REQUIRES AN END-TO-END APPROACH

INSIGHTS

SELF-SERVICE | COLLABORATIVE | MOBILE | REAL-TIME

NON-RELATIONAL

100111

DATA MANAGEMENT

RELATIONAL STREAMING

SHARE

AND GOVERN

DISCOVER

AND RECOMMEND

TRANSFORM

AND CLEAN

DATA ENRICHMENT

Page 20: Big Data From Microsoft and Atidan

Parallel Data Warehouse

PowerPivot

Power ViewHadoop on Windows

Page 21: Big Data From Microsoft and Atidan

ADDITIONAL RESOURCES

LEARN MORE1. Microsoft Big Data Solution: www.microsoft.com/bigdata

2. Windows Azure: www.windowsazure.com/en-us/home/scenarios/big-data

3. Microsoft BI blog: http://blogs.msdn.com/b/microsoft_business_intelligence1/

TRY NOW1. Preview of the Hadoop-based service for Windows Azure:

https://www.hadooponazure.com

Page 22: Big Data From Microsoft and Atidan

1. McKinsey&Company, McKinsey Global Survey Results, Minding Your Digital Business, 20122. IDC Market Analysis, Worldwide Big Data Technology and Services 2012–2015 Forecast , 2012

49% of top CEOs and CIOs are

currently using Big Data for

customer analytics

2.7

3.9

5.1

6.5

0

2

4

6

8

2012 2013 2014 2015

Bil

lio

ns

$

39% compound

annual growth

rate2

Big Data Software Growth

1.82.5

3.4

4.6

0

2

4

6

2012 2013 2014 2015B

illi

on

s $ 34% compound

annual growth

rate

Page 23: Big Data From Microsoft and Atidan
Page 24: Big Data From Microsoft and Atidan

Discover data with Data Explorer

Combine with information from

other sources via Azure Marketplace

Refine with Advanced Analytics

Connecting

with the World’s Data

MICROSOFT BIG DATA

Immersive insights for all users

Insights on any data

Embedded insights with simplified

programming

Immersive Insight,

Wherever you are

Extend data warehouse with Hadoop

Windows simplicity for Hadoop

Scale & elasticity of the cloud

Any Data, Any Size

Anywhere

Page 25: Big Data From Microsoft and Atidan

Discover data with Data Explorer

Combine with information from

other sources via Azure Marketplace

Refine with Advanced Analytics

Connecting

with the World’s Data

MICROSOFT BIG DATA

Immersive insights for all users

Insights on any data

Embedded insights with simplified

programming

Immersive Insight,

Wherever you are

Extend data warehouse with Hadoop

Windows simplicity for Hadoop

Scale & elasticity of the cloud

Any Data, Any Size

Anywhere

Parallel Data Warehouse

PowerPivot

Power View

Page 26: Big Data From Microsoft and Atidan

TRADITIONALRelational Database Management System

NEWPetabyte-Scale Services

Page 27: Big Data From Microsoft and Atidan

Industry/Vertical Scenarios

Financial Services

Modeling True Risk

Threat Analysis

Fraud Detection

Trade Surveillance

Credit scoring and analysis

Web & E-Tailing

Recommendation Engines

Ad Targeting

Search Quality

Abuse and click fraud detection

Retail Point of Sales Transaction Analysis

Customer Churn Analysis

Sentiment Analysis

Telecommunications

Customer Churn Prevention

Network Performance optimization

Call Detail Record (CDR) Analysis

Analyzing Network to Predict Failure

Government Fraud Detection and Cyber Security

General (Cross Vertical) ETL & Processing Engine

Page 28: Big Data From Microsoft and Atidan

WHILE DRAMATICALLY SIMPLIFYING PROGRAMMING ON HADOOP

Integration with .NET and new

JavaScript libraries for Hadoop

JS

MapReduce

programs in

JavaScript

Simplified

programming

Simplified deployment of

MapReduce jobs

Benefit

sKey

Featu

res

Deploy JavaScript Hadoop jobs

from a simple web browser on

any supported device

Page 29: Big Data From Microsoft and Atidan

HADOOP ON PREMISES AND IN THE CLOUD

Enterprise-class Big Data

platform on-premises

Hadoop-based distribution

on Windows Server

Elastic Big Data platform

in the cloud

Hadoop-based Service on

Windows Azure platform

Hadoop connectors for

SQL Server

Extend your EDW

with Big Data

Page 30: Big Data From Microsoft and Atidan

MANAGE ANY DATA, ANY SIZE ANYWHERE

101010101010101001010101010101

101010101010

Hadoop Connectors & ETL

Page 31: Big Data From Microsoft and Atidan

SIMPLICITY AND MANAGEABILITY OF WINDOWS FOR HADOOP

Integration with Microsoft

System Center

Simplified management Enterprise-class security

Integration with Windows

Server® Active Directory

Easy setup on-premises

and in the cloud

Hadoop-based service on

Windows Azure

Smart packaging of

Hadoop on Windows

Page 32: Big Data From Microsoft and Atidan

STREAMING DATA WITH STREAMINSIGHT

Complex Event Processing with

SteamInsight (On-premise)

On-premises analysis of streaming

data in real time

Event Processing in the Cloud with

Windows Azure SQL StreamInsight

Cloud designed analysis of streaming

data

StreamInsight SQL StreamInsight

Page 33: Big Data From Microsoft and Atidan

EXTEND YOUR DATA WAREHOUSE WITH HADOOP

Integration with

enterprise BI solutions

Microsoft SQL Server

connector for Apache Hadoop

with SQOOP (SQL to Hadoop)

Integration with Microsoft

Data Warehousing

SQL Server Parallel Data

Warehouse connector for

Apache Hadoop with

SQOOP

Deeper insights from structured

and unstructured data

Page 34: Big Data From Microsoft and Atidan

CONNECT HADOOP TO THE WORLD VIA WINDOWS AZURE MARKETPLACE

Mashing up of internal and

public data sets

Integration with third-party

data and services

Sharing of data and

insights through Windows

Azure Marketplace

Integration with Windows

Azure Marketplace

through ODATA

Page 35: Big Data From Microsoft and Atidan

ENRICHMENT VIA INTEGRATION WITH SOCIAL MEDIA

Integration of social

media data with

business applications

Microsoft Codename

"Social Analytics"

Stronger customer

relationships

Integration with social

media sites

Models augmented with

publicly available data

from social media sites

Page 36: Big Data From Microsoft and Atidan

ADVANCED ANALYSIS WITH HADOOP

Unlock rare patterns from bespoke data

mining models

Mahout & Pegasus libraries

already supported on

Azure

New business insights with predictive

analytics from Microsoft

Hive ODBC Driver connects Hadoop to

SQL Server Data Mining tools in SSAS

Support for open source

Advanced Analytics tools

such as Mahout & Pegasus

Page 37: Big Data From Microsoft and Atidan

MICROSOFT ENTERPRISE DATA WAREHOUSING

Software AppliancesReference Architectures

Fast Track for

Dell Parallel Data Warehouse

HP Enterprise Data Warehouse

Dell QuickstartData Warehouse

HP Business Data Warehouse

Page 38: Big Data From Microsoft and Atidan

11/24/2010 © Microsoft Corporation, All rights reserved