Cox Business Intelligence & Oracle BI/DW Stack

  • Published on
    14-Jun-2015

  • View
    1.513

  • Download
    0

Embed Size (px)

DESCRIPTION

Cox Communications Gartner BI Summit 2012 presentation.

Transcript

<ul><li> 1. Cox Business Intelligence&amp; Oracle BI/DW StackJohn Landis, Sr. Director BI &amp; Data ArchitectureApril, 2012</li></ul> <p> 2. Cox Communications Company Overview Is the third-largest cableentertainment and broadbandservices provider in thecountry Has over 6 million customers Has over 22,000 employees 2 3. Cox Communications Services Residential TV, Internet,Phone, Tech Solutions, HomeSecurity Business TV, Internet, Phone,Security, Backups, IndustryServices for Real Estate,Residential Communities,Education, Government,Healthcare, and Hospitality Media On Air, Online and On-the-Go3 4. CEBI 2008 Problem Business customers not Proliferation of tools hassatisfied with multiple become expensive and hardplatforms. Not sure where to maintainto get data the right data. Data needs of the company Business intelligence are growing, offlineplatform has multiple databases at all sitesversions of the truth Development taking place in Data Integration is fractured multiple organizations Data Warehouse has not No standards exist in thehad investment in 3 years enterprise4 5. Cox Enterprise Business Intelligence (CEBI) 2008 As well as 5 6. CEBI 2008 Transition Adopt a enterprise reportingapplication to encourage Strategic Dynamiccollaborative enterprisedevelopment of reporting Modelingacross the organization andlower the cost of reportingthroughout Cox.Past Oriented Advanced Future Oriented Analytics Reuse and optimization of Ad-Hoc Query &amp; Reportingresources: Standardized People and Processes Reporting Application, Data, and Operational StaticServices Death by 1000 paper cuts Time and Cost 6 7. CEBI 2008 Solution Oracle Business IntelligenceEnterprise Edition 10G chosenas the enterprise BI platform Oracle Database chosen asthe Enterprise data platformand Infomatica chosen asintegration platform Business IntelligenceCompetency Center Deployed Data Warehouse Clean-UpBegins7 8. CEBI 2008 OBIEE 10G Total Cost of Ownership Common Semantic Layer Prebuilt Analytical Options Oracles Strategic BI Roadmap Single Sign On Embedded Metadata Self Service Reporting8 9. CEBI 2008 OBIEE 10G Interactive Charts and Graphs Personal Dashboards One Suite of Tools Open Source not ready forenterprise deployment Hyperion Integration Personalization9 10. CEBI 2008 Solution Cont.Cox lived happily ever after and I got to retire to my dream location..Not Exactly 10 11. CEBI 2008 Lessons Learned Data governance is required Garbage in, garbage out IT can only facilitate data Most people dont understandgovernance, business needs data, therefore carefully createto leadyour RPD Training is critical OBIEE resources are hard to find Self Service Reportingrequires supervision The business wants you to challenge them on requirements Start small Reports are only as fast as the Not everyone likes changedatabase11 12. CEBI 2010 Exadata With the growing data and reporting needs within theorganization, the platform needed to expand to handle theprojected growth. Business data needs went from daily updates to near realtime updates. Existing hardware reached its capacity and newtechnology was needed in order to meet the current andupcoming demands. Without a platform and technology upgrade, data andreporting would not be made available to the organization. 12 13. CEBI 2010 Exadata In April 2010, the EBI team partnered with Oracle EXADATA POC -OBIEE Queries Total Run Time (Total of 130 queries executed) BASELINEto perform a Proof of Concept (POC). 25,000 Based on the results of the POC, an executive20,000 EXADATA AS ISdecision was made to implement the full solution. 15,000 EXADATA NO INDEXES In July 2010 the EBI team began the planning and 10,000rollout of Exadata.EXADATA5,000COMPRESSIO N/ AGG With the help of the Operations Support group, all- REMOVAL EXADATA EXADATAEBI databases were implemented on Exadata in BASELINEEXADATA AS EXADATA NO COMPRESSI EXADATAMIXED LOAD MIXED LOAD ISINDEXES ON/ AGG TESTINGProduction.REMOVAL TESTING Total Time in Seconds22,077 9,774 3,998 2,316 2,399 13 14. CEBI 2010 Exadata Pre-Launch Concerns People Technology - Support structure is different - Oracle was new to the hardware market - Adoption - Technology had limited instances in - Learning curve for support and production.development - Switching from commodity based storageto appliance; risk of stranding assets. Process - Backup strategy changes and recovery - Compression can mask the lack of a model changesdata lifecycle management - Vendor lock in moving away from Oracle - It is not the way we have always becomes more expensive.done it - Performance increases can maskarchitectural issues Note: Degradations in performance caused by development code that should have been avoided. Nearly 600K IOPS. 14 15. CEBI 2010 Exadata Reporting Nearly 6X improvement out of the box Up to 200X query performance improvement. 9X on average Nearly 6X performance increase on the work orders load (non Exadata source). 2X on average for non Exadata sources and 10X on average for Exadata to Exadata loads. Some reports showed worse performance15 16. CEBI 2010 Exadata Results 5-10x Compression saves Cox money over traditional storageCompression Lowers backup time and tapes needed Estimated savings in space through 2012 range approx. $2.4M $4.8M Less tuning reduces project timelines Enables near real time processingPerformance Able to process data previously not possible Estimated savings in time in 2012 approx. 5% or $400K Highly available, has uncovered issues in other Cox Oracle systems helping to improve reliabilityEnterprise Availability Reduces complexity of environment because Oracle has tested the integration points, all hardware is tested to work together unlike commodity solution Oracle is our standard database today, no conversion costs were incurred.Leverage Existing Technologies Cox employees already had a skill set in this technology Development best practices were enhanced16 17. CEBI 201217 18. CEBI 2012 OBIEE 11G Score carding Mobility Improved Visualizations Spatial Intelligence via Map-based Visualizations Business Process Invocation Packaged Apps Exalytics18 19. CEBI 2012 Architecture Enterprise Business Intelligence Platform Reporting and AnalyticsInteractive Reporting andAdhocOffice Detect andCollaborate &amp;Mobile and Scorecards DashboardsPublishing Analysis Integration AlertsSeachEmbeddedEnterprise Metadata Layer Standards ProcessFinancial DataMaster DataOLTP Data Data Warehouse Financial ERP -HumanERP - HRFinance ConsolidationORGANIZATION FInancialsResourcesCUSTOMER Sales &amp; CRM Logistics FieldMarketingEMPLOYEEPlanning, BudgetingTime &amp;Business PROJECTBilling &amp; Forecasting Attendance OperationsData SourcesGovernance and Monitoring 19 20. CEBI 2012 Exadata Primary Database Areas:- Reporting- Applications- Web Services Standby Database Areas:- Analysis- What If- Predictions- Data Mining- Ad-Hoc20 21. CEBI 2012 HistoryProcess TimeData VolumeUsers2008 10.5 Hours/Night 2008 &lt; 2 Billion/Night 2008 &lt; 10002009 12.5 Hours/Night 2009 36+ Billion/Night 2009 2500+2010 4 Hours/Night2010 43+ Billion/Night 2010 5000+2011 3.5 Hours/Night2011 50+ Billion/Night 2011 9000+BICC Migrations/Reviews Errors Per 1M Sessions User Generated Reports2008 1002008 500 2008 6000 Usr Rpts2010 2218 2010 1252010 10000+ Usr Rpts2011 3000+2011 1002011 15000+ Usr RptsComplexity2008 Single Billing, Weekly/NightlyNumbers2011 Multi Billing, Near Real Time21 22. CEBI Customer GoalsPersonalProduct Personalization Customized Interfaces Personalized ServicesRecommendationsCustomers wantCross Team EffortsCross Product UsageGrowing AnalyticalNeedsInternal Cox Users wantProduct Planning and Data Analysis andReal-time OperationsData-driven Sales andOptimization ResearchMonitoring Marketing22 23. CEBI FutureAccording to Gartner, Enterprise Data will grow 650% by 2014. 80% of this data willbe Unstructured Data, with a CAGR of 62% per year, far larger than transactionaldata.Unstructured Data in Web Pages Growth is taking place in areas not well served by traditional databases This chart shows the growth over data over the next couple of years. It is projected that a large portion of this growth will be unstructured data (web logs, emails, social interactions, etc.). Unstructured data is driving an explosive growth in data Unstructured data does not work well with traditional databases. To achieve the low response times, traditional databases rely on strict data structures. These strict data structures work well for certain types ofStructured data. However, the growth of unstructureddata data in the enterprise and the proposed uses of it create the need for a new type of data processing to be introduced to the technology stack. The 2011 IDC Digital Universe Study Sponsored by EMC 23 24. CEBI GoalsCreate additional value from customer data Increase the perceived value of products by enabling a high degree of individual personalization. Give a highly tailored customer experience every time a customer interacts with Cox.Make Cox a more data-driven company Improve the efficiency and security of Cox operational processes. Allow the company to make decisions, spot trends, and react to competitive challenges more quickly.Democratize access to data Allow the company can make quick, innovative use of the data that is already being generated every day. Improve cross-team and company wide insight into how customers are using Coxs services. 24 25. CEBI GoalsCox DataData Decision FrameworkTraditional Data Architecture Big Data ArchitectureStructured DataUnstructured/Semi-structured DataCharacterized by well-known use cases, requiring onlyCharacterized by lack of established use cases and on-repeatable, static data cubes and ETLs. Highlythe-fly analysis in a Sandbox manner. Useful inproductized results. developing new insights, products.Example Use CasesExample Use Cases Ad-hoc Queries Ad-hoc Queries Financial and Operational Dashboards Data Mining/Discovery Large Datasets, Fast Response Times Ad Impression Analyzer Predictive Analysis Marketing AnalyzerChallenges Challenges Sizing to support new reporting dimensions is notLatency is greater than with traditional databases.always economically feasible. Large unstructured datasets will need to be monitored Analysis against new datasets can slow Time to Marketand managed at scale.for new products. 25 26. CEBI Sample Decision FrameworkLow Data Volumes Used to evaluate analysis useLatency Complex Simple casesHighExadata Any Can determine which system toLow Exadata Exadata use: Traditional databaseHigh Data Volumes Non-traditional data storeLatency Complex Simple Can standardize reporting andHighBig DataBig Data analysis use cases across theLow Big DataExadataenterprise26 27. Future Solution DesignPresent PresentationAnalytics ServicesApplicationsStore EDWMaster Store NOSQLVirtualProcess Data CleansingAnonymization Billing MediationTransformMDM ETL/ELTMap ReduceVirtualization ODSNOSQLHadoopFederatedStagieReplicationMediation Ingestion VirtualizationData Sources (Raw Data)27 28. POC in Progress Exalytics Oracle Big Data Appliance Endeca Information Discovery Packaged Apps 28</p>