Verizon Centralizes Data into a Data Lake in Real Time for Analytics

Verizon Centralizes Data into a Data Lake in Real Time for Analytics

© 2017 Attunity

Sanjay Kumar

General Manager, Telecom - Hortonworks

Arvind RajagopalanDirector – Global Technology Services – Verizon

Jordan MartzDirector of Technology Solutions - Attunity

Guest Speakers

© 2017 Attunity

• Hortonworks - Hortonworks Data Platform (HDP) for the telecommunications market

• Verizon - How we centralized data into a data lake for analytics

• Attunity - Migrating real-time data

• All - Q&A

Agenda

• Hortonworks Data Platform (HDP) for the telecommunications market

5 ©HortonworksInc.2011– 2016.AllRightsReserved

TheNewLandscapeoftheTelecomIndustry

ServiceProvidersSocialMedia&Mobile:ExplosionofrichcustomerdatathroughSocialMedia

andMobileAppsforcustomersentiment&

Interests

CustomerExpectation:Withtheculturalimpactofwebandmobile,customersareexpectinggreater levels

ofserviceandresponsiveness

CompetitiveDifferentiation:Asotherserviceprovidersdeliver

similar levelsoftelecomserviceandcoverage,otherareasofservice levels

areneeded

NewDigitalEcosystem:GreatervalueofDataondigitalecosystemfor

CustomersandPartnersdrivingDataMonetization

InternetOfThings:ExplosionofdatafromIOTwithbenefitsaligned

withinsightnotcorrelated todata

volumes


Service Providers Areas of FocusCustomerExperience&Marketing- EnhanceEnd-to-end ExperienceofCustomer- BecomeTrustedPartnertoCustomer- Awarenessofcustomer’sneedswhenandwhereneeded

NewBusiness&ConsumerServices- NewDigital&InfrastructureServices- DataMonetization- M2M,IoT,Analytics-as-a service

NetworkOptimization- MovetoSoftwareDrivenNetworks- LeverageNetworkDataAssets- Selfoptimizingandprovisioning Customer

Network Service


DATA-CENTRIC

OldModel:App-Centric

Limitations:• Multiplecopiesofdata• Difficultcross-systemintegration• Upper-limitondatavolumes

beforeharmingperformance

NewModel:Data-Centric

Advantages:• Oneversionofthedata• Noneedforcross-appintegration• Systemscaleslinearly

APP-CENTRIC

App1 App2 App3 App4 App5 App6

AppCentricwillbreakdownwithx10,x100,x1000…NeedtoshifttoDataCentric


TheFutureofDataActionableIntelligence

D ATA I N M O T I O N

STORA

GEST

ORA

GE

GROUP2GROUP1

GROUP4GROUP3

D ATA AT R E S T

INTERNETOF

ANYTHING

CombiningData-in-MotionandData-at-RestpowersActionableIntelligence


StreamingInjestion

(HDF)

Streaming:NetworkProbes,ClickStream,Sensor,Location

Batch:CallDetailRecords

On-Line:CustomerSentiment

Unstructured:Txt,Pictures,Video,Voice2Text

Clickstream

Web&Social

Geolocation

EventRecords

CallLogs

Files,emails

ServerLogs

ExistingOSS/BSS/DWHApp1 App2 App3 App4 App5 App6

IOT Solutions

Analytic Services

Network Planning

OptimizationContextual Marketing

360 Customer Awareness

DataDrivenBusiness

DELIVERY

PersonalDataAnalysis&CustomerInsightServicesToCustomers&Partners


SocialMedia

SentimentCallCenterInteraction

Lifestyle&Interests

StreamingInjestion

(HDF)

Streaming:NetworkProbes,ClickStream,Sensor,Location

Batch:CallDetailRecords

On-Line:CustomerSentiment

Unstructured:Txt,Pictures,Video,Voice2Text

HowtoSeeWhattheCustomerSeesandRespondwithAwareness

– ViewfromTheCustomer(360degreeview)• QualityofServiceExperience (Networkdetails)• CallCenterdialog(voicetotextanalysis)• CustomerInteractionChannel(CRM,Billing)• Customer’sSentiment(whattheyaresayingonsocial

media)

– UnderstandingCurrentStateoftheCustomer• Allkeycustomermetricsattributesandreferences

withinadynamiccustomerprofile• NetPromoterScore(dynamicallycomputed)• ChurnScore• AppetiteforInformation&otherCustomerMetrics• TargetAdvertisingProfile

– NextBestActions• Contextawarepromotions• Churnriskresponse• Retail StorewithStateoftheCustomer• Omni-channelcorrelatedactions

QualityofService

viaNetwork

CustomerExperience

ViaBSS


BusinessFunctions

CustomerGo-to-market

CyberSecurity

Hadoop-as-a-service

M2M/IoTSolutions

PersonalIntelligence

IndustryBrokering

ITOperationsEDWDataOffload HDP

LandingZoneHDP

DataLakeReal-timeStreaming

HDPDataLake

360 CustomerHousehold

View

Dynamic Customer

Profile

Next BestAction

Location &Context Based

CEM- CustomerExperience

Target Audience

Profile

Context Aware Target

Marketing

Location & Context Based

Promotions- Marketing

ThreatDetection

Real-TimeThreat

Analytics

DynamicPattern

Detection- Security

Next BestNetwork

Investment

EquipmentPredictive

Maintenance

Customer Aware Network

Provisioning

- Network

Customer Sentiment &

Churn

Dynamic Network

Provisioning

NetworkDecommissioning

© 2017 Attunity

CRM

Streaming Injestion(HDF)

Call Center Voice to Text

Social Media Feeds (Twitter..)

CDRs or Network Probe

© 2017 Attunity

Customer Sentiment &

Churn

360 CustomerHousehold

View

Dynamic Customer

Profile

Next BestAction

Location &Context Based

CEM

Target Audience

Profile

Context Aware Target

Marketing

Location & Context Based

Promotions

• How we centralized data into a data lake for analytics

Migrating real-time data

© 2017 Attunity

Attunity Corporate Overview

Data Integration & Big Data Management Software

Accelerate data delivery and availability

Automate data readiness for analytics

Optimize data management with intelligence

§Hadoop & Big Data

§Databases & Data Warehouses

§On premise & in the Cloud

Solutions for: Global OfficesOverview

§2000 customers in 65 countries

§250 people and growing

§NASDAQ traded (ATTU)

© 2017 Attunity

Seamless integration with Hortonworks Connected Data platforms and solutions

HortonworksConnection

Hortonworks Solutions

Enterprise DataWarehouse Optimization

Cyber Security andThreat Management

Internet of Thingsand Streaming Analytics

Hortonworks ConnectionSubscription SupportSmartSensePremier SupportEducational ServicesProfessional ServicesCommunity Connection

CloudHortonworks Data CloudAWS HDInsight

Data CenterHortonworks Data Suite

HDFHDP

© 2017 Attunity

Real-time Data Ingest with Attunity Replicate

SOURC

ES

OLTP,ERP,CRMSystems

Documents,Emails

Web Logs,Click Streams

SocialNetworks

MachineGenerated

SensorData

GeolocationData

Attunity Replicate for HDP & HDF

Accelerate time-to-insights by delivering solutions faster, with fresh data, from many sources - Automated data ingest - Incremental data ingest (CDC)- Support for multiple sources

© 2017 Attunity

Attunity Replicate architecture

Transfer

TransformFilterBatch

CDC Incremental

In-Memory

File Channel

Batch

Hadoop

Files

RDBMS

Data Warehouse

Mainframe

Cloud

On-prem

Cloud

On-prem

Hadoop

Files

RDBMS

Data Warehouse

Kafka

Persistent Store

© 2017 Attunity*Supported under early access program

Attunity Replicate sources and targets

RDBMS

OracleSQL ServerDB2 iSeriesDB2 z/OSDB2 LUW MySQLPostgreSQLSybase ASEInformix

DW

ExadataTeradataNetezzaVertica

HortonworksClouderaMapR

Hadoop

DB2 for z/OSIMS/DBVSAMSQL M/PEnscribeRMSHP NonStop

Mainframe

AWS RDSSalesforce

Cloud

RDBMS

OracleSQL ServerDB2 LUWMySQLPostgreSQLSybase ASEInformix

DW

Microsoft PDWExadataTeradataNetezzaVerticaSybase IQAmazon RedshiftActian VectorSAP HANA

HortonworksClouderaMapRPivotalAmazon EMR

Hadoop

MongoDB

NoSQL

Amazon RDSAmazon RedshiftAmazon EMRGoogle Cloud SQLGoogle Cloud DataprocAzure SQL Data WarehouseAzure SQL Database

Cloud

Azure Event Hubs*Kafka

Messaging

Targets

Sources

SAP

ECC on OracleECC on SQLECC on DB2*

SAP

HANA

© 2017 Attunity

In Memory and File Optimized Data Transport

CDC for data-at-rest and data-in-motion

R1R1R2R1R2

R1R2

Batch CDC

Data WarehouseIngest-Merge

SQLn 2 1

SQL SQLTransactional CDC

Message Encoded CDC

Data Sources

AttunityReplicate–ChangeProcessing

CDC

Many Databases and Data

Warehouses

....

© 2017 Attunity

CDC

Demo: Data Streaming into Kafka è HDF è HDP

MSG

n 2 1

MSG MSGData Streaming

Transaction logs

In memory optimized metadata management and data transport

Bulk Load

MSG

n 2 1

MSG MSGData Streaming

Message broker

Message broker

© 2017 Attunity

Attunity Replicate for SAPUniversal, Real-Time and Simplified Data Integration

• Replicate your SAP application data in bulk or real-time for data analytics§ Documents, transactions and business data§ All core and industry-specific SAP modules

• Integrate real-time with all major targets§ DBs, data warehouses, Hadoop – cloud or on premises§ Decode SAP data from complex source structures§ Enable business usage on common data model

• Move external data into SAP HANA

Attunity ReplicateBulk Load

CDC

Core and Industry-Specific SAP Modules

RDBMS | EDW | HadoopOn Premises or Cloud

Hadoop Data Lake

© 2017 Attunity

Attunity Replicate Server

TransformFilter

Batch

CDC IncrementalIn-Memory

File Channel

Batch

Demo: Attunity Replicate

Persistent Store Extract relationships for Pool and Cluster Tables

RDBMS(Oracle, DB2, etc.)

Redo/ Archivelogsor

JournalFile

----------------Transparent

Tables

On Premises

Hadoop RDBMS

Data WarehouseKafka

Cloud

Attunity Replicate Agent for SAP

SAP ECC (Enterprise Central

Component)

Data Model Mapping

Pool/Cluster table RFC

© 2017 Attunity

Demo

attunity.com

Questions?

Thanks!