Click here to load reader

Optimizing IT Costs & Services With Big Data (Little Effort!) - Case Studies - #Pink13

  • View
    285

  • Download
    0

Embed Size (px)

DESCRIPTION

IT organizations have a wealth of Service Management and Service Delivery tools, processes and metrics that typically exist in relative isolation. This session will present detailed real-life examples of how existing tools and metrics can be brought together using big data techniques to optimize costs and performance of IT environments.

Text of Optimizing IT Costs & Services With Big Data (Little Effort!) - Case Studies - #Pink13

  • 1. Optimizing IT Costs & Services with Big Data, Little EffortDavid Wagner TeamQuest AdvocateTeamQuest and the TeamQuest logo are registered trademarks in the US, EU and elsewhere.All other trademarks and service marks are the property of their respective owners.Copyright 2012 TeamQuest Corporation. All Rights Reserved.

2. Agenda Why? Big Data: conceptual overview 2013 Capacity Management 101: History Goals Obstacles New Big Data approaches Concepts Case Study ValueCopyright 2012 TeamQuest Corporation. All Rights Reserved. 3. Why does TeamQuest Exist? We passionately believe always having and usingthe right amount of resources is a societal imperative Anything less is failure Anything more is wasteful 20+ years sole focus ensuring our customers can continuously and automatically perform at their utmost level of efficiency ensuring business service performance, conserving scarce resources, saving money and improving productivity We call this: IT Service OptimizationCopyright 2012 TeamQuest Corporation. All Rights Reserved. 4. What this Presentation is and is not! IS: Applying Big data approaches to Capacity Management Faster and larger value More scalable New ways to think about optimization beyond ITIL Capacity Management Include ITIL Service Management and Delivery Not just technology anymore Is NOT! A Primer on Big Data or a Big Data how to Presentation Hadoop ecosystem deep dive, etcCopyright 2012 TeamQuest Corporation. All Rights Reserved. 5. Big Data at 50,000 feet Big Data is about: data actionable information Plethora of existing sources Technology Business (Sales, Marketing, ) Service (Transactions, SLAs, ) Learning new insights from old data Key is Analytics Deep Wide Adaptable But Optimizing costs with Capacity Management?Copyright 2012 TeamQuest Corporation. All Rights Reserved. 6. Technology Approaches Data Access and Aggregation Build huge data marts (aka: Data Warehousing) Integrate with multiple different data sources Technology (e.g. Server, Network, Storage, etc.) Service (Catalog, Metrics, Tickets, etc.) Business (KPIs, Plans, Transactions, etc.) Implement Analytics against/across Flexible and adaptive Turn data within, into actionable information across But Capacity Management???Copyright 2012 TeamQuest Corporation. All Rights Reserved. 7. 2013 Capacity Management 101 - History Answering what if questions Change in technology, demand, etc impact? Focus on Optimizing Server Cost versus Performance Extremely Technology-centric Servers, Mainframes Occasionally Storage or Network in isolation Big Value and Return, but also effort Highly trained staff Required building a central, massive datamart (CMIS) Scalability of Staff, Tools, , PoliticsCopyright 2012 TeamQuest Corporation. All Rights Reserved. 8. 2013 Capacity Management Goals: What Maintain traditional value, and add Optimize Amplify Accelerate Increase Business relevance Valuable predictive analytics in business and service context Optimize Efficiency Virtualization and Cloud Scale to everything Many to many inter-relationships; Capacity criticalCopyright 2012 TeamQuest Corporation. All Rights Reserved. 9. 2013 Capacity Management 101 Goals: How Integrate and Analyze across multiple sources Technology (e.g. Server, Network, Storage, etc.) Service (Catalog, Metrics, Tickets, etc.) Business (KPIs, Plans, Transactions, etc.) Single pane of Analytic Glass Ability to tie together, correlate, and operate across Tear down the wall! Dont force reinvention or data duplication! Flexible and adaptive Turn data within, into actionable information acrossCopyright 2012 TeamQuest Corporation. All Rights Reserved. 10. 2013 Capacity Management 101 - Obstacles Data Access and Aggregation Building huge data marts (fka: Data Warehousing) Complexity = (data ETL) x (# sources) x (maintenance effort) Compliance: Data duplication, privacy, audit, etc Costly and time consuming Implementing Analytics against/across General purpose BI Analytics for Capacity? Traditional Performance/Capacity for General Purpose? Big Data + ITIL = Optimized CapacityManagement?Copyright 2012 TeamQuest Corporation. All Rights Reserved. 11. Capacity Management with ITIL 2011 Service Strategy Financial management Service Design Service Level and Availability management Service Transition Asset, Change and Configuration Management Service Operations Service Desk Application and IT operations Event, Incident, Problem Or, in simpler terms Integrate Capacity across ITIL V2: Service Support and Service Delivery!Copyright 2012 TeamQuest Corporation. All Rights Reserved. 12. Optimized Capacity Management Leverage the data (and tools) you have! Dont reinvent or reimplement Quickly and easily with True Federation Use existing data/tools already in place Dont force data duplication, ETL Capacity Analysis across data sources Key ITIL discipline metrics amplify Capacity ManagementValue Strategy factor financials Design factor Service Levels, technology performance Transition track business and technology changes Operations factor Service risks, multiple technologiesCopyright 2012 TeamQuest Corporation. All Rights Reserved. 13. Integrated Case Study Walkthrough ITIL: Strategy Capacity Management integrated with Financial costing/reporting ITIL: Design Capacity Management integrated with Risk Registry ITIL: Transition Includes integration with Asset and Configuration Management ITIL: Operations Integration with Service Desk Operations factor Service risks, multiple technologiesCopyright 2012 TeamQuest Corporation. All Rights Reserved. 14. Very Large BankAs an IT Shop: Operate tens of thousands of servers Every server platform under the sun Manage dozens of data centers Huge mainframe with many thousands ofMIPS Thousands of VMs Thousands of VDIs & Citrix Many Petabytes of storageCopyright 2012 TeamQuest Corporation. All Rights Reserved. 14 15. Seamless data integration & analysis1. All capacity/performance data2. All platforms, OSs, 3. Configuration data4. Change records5. Risk registryCopyright 2012 TeamQuest Corporation. All Rights Reserved. 15 16. Deliverable: Fully AutomatedApplication ReportWe need:1. Risk detection and tracking2. Risk reporting3. Actionable informationReporting has to be:AutomatedRepeatableHuman-readable financials, business terms, not speeds and feedsCopyright 2012 TeamQuest Corporation. All Rights Reserved. 16 17. Analysis OverviewApplication and Configuration from Service Catalog and CMDBCopyright 2012 TeamQuest Corporation. All Rights Reserved. 17 18. Usage PatternsTime Series data from Performance and Event ManagementCopyright 2012 TeamQuest Corporation. All Rights Reserved. 18 19. Service Desk and Risk managementCopyright 2012 TeamQuest Corporation. All Rights Reserved. 19 20. Existing Capacity Issues Scaleably ID Possible Copyright 2012 TeamQuest Corporation. All Rights Reserved. 20 21. ID Possible FutureCapacity RisksCopyright 2012 TeamQuest Corporation. All Rights Reserved. 21 22. Fixed Costs / Variable Costs - MethodVariable CostsSource: wikipedia.orgFixedCostsCopyright 2012 TeamQuest Corporation. All Rights Reserved. 22 23. Capacity Management + Strategy (Financials) Fixed/Variable Cost server0009b01a - Excess Capacity Report Produced by the Server Capacity & Performance Management (SCPM) Team Analysis Period: August 01 2010 to August 31 2010 Run Time: 4:09 PM September 27 2010 (8 seconds) Purpose: To analyze the systems current resource consumption and compute the available headroom based on a fixed/variable costs methodology and our rules-of-thumb. This report also attempts to determine the nearest bottlenecks, from a consumption perspective.server0009b01a: Maximum Growth Capability by Resource server0009b01a: Top 10 PIDsNameGrowth VauleNAMEPIDGROWTH SLOPE MINCPU AVGCPU MAXCPU CPU RunQ Length Growth 2.15System:417.54 0.000.06 0.09 5.18 Disk - 0 4.41NTRtScan:1660 29.88-0.000.00 0.02 3.01 Memory Utilization Growth5.48beasvc:1080 47.72-0.000.00 0.14 1.89 FS - C:10.61 svchost:840184.41-0.000.03 0.07 0.52 Virtual Memory Growth20.27 svchost:872213.22 0.000.05 0.08 0.47 CPU Growth 38.10 TmListen:2160763.86-0.000.00 0.01 0.12 Net In 100MB - NIC1 260.82 python:1788848.86 0.000.00 0.03 0.11 Net Out 100MB - NIC1349.39 wmiprvse:268 987.95-0.000.00 0.04 0.09 Net In 1GB - NIC12608.18 wmiprvse:2228 1322.98 0.000.02 0.04 0.09 Net Out 1GB - NIC1 3493.92 wmiprvse:2044 1328.88-0.000.02 0.05 0.0923Copyright 2012 TeamQuest Corporation. All Rights Reserved. 24. Capacity Management + Strategy (Financials)Copyright 2012 TeamQuest Corporation. All Rights Reserved. 25. Capacity Management + Strategy (Financials)Copyright 2012 TeamQuest Corporation. All Rights Reserved. 26. VM Optimization AnalysisThousands of VMsSome too smallSome too bigSome idleWhich ones?What size should they be?Copyright 2012 TeamQuest Corporation. All Rights Reserved. 26 27. Physical to Virtual AnalysisCopyright 2012 TeamQuest Corporation. All Rights Reserved. 27 28. Capacity Optimization Candidates Total Virtual Machines Idle Virtual Machines Oversized Virtual Machines22 3 13Idle Virtual Machines Recommende Avg % Max Max %Average Avg % Max % CPU Avg %TotalRecommende d SSO vCPUs Max % CPUCPU Memory Memory MemoryMemory ReadyCPUMemoryd SSO vCPUsMemory in Ready Used Util Used UtilGBCLUSTER0019V0194 0 0 00 409600 0 0 12CLUSTER0019V0242 0 0 00 200000 0 0 12CLUSTER0019V029-OLD_DO_NOT_USE 4 0 0 00 409600 0 0 12Oversized Virtual Machines Avg % Max Max %Average Avg % Recommend Recommended Max % CPU Avg %Total vCPUs Max % CPUCPU Memory Memory MemoryMemoryed SSOSSO Memory in ReadyCPUMemory Ready UsedUtilUsedUtil vCPUsGBCLUSTER0019V001 2 4059 1 2044 1921 941729 8514CLUSTER0019V003 2 30 107 2 2048 1895 931737 8514CLUSTER0019V004 2 45 513 3 2048 1914 931333 6524CLUSTER0019V005 2 41 178 2 2048 1955 951716 8424CLUSTER0019V006 2 45 412 2 2048 1963 961510 7424CLUSTER0019V008 2 27 232 2 2048 1860 911232 6014CLUSTER0019V013 2 30 403 3 2048 1845 901326 6514CLUSTER0019V014 2 30 363 2 2048 1834 901286 6314CLUSTER0019V018 2 32 303 2 2000 1843 921581 7914CLUSTER0019V029-REAL4 42 305 4 4096 3612 882951 7248CLUSTER0019V030 2 47 232 2 2048 1881 921387 6824CLUSTER0019v009 2 43 142 1 4096 3117 762258 5524CLUSTER0019v010 2 30 182 1 4096 3102 762151 5314 Copyright 2012 TeamQuest Corporation. All Rights Reserved.28 29. Delivered:Repeatable processesQuicker analysisPowerfulFlexibleCopyright 2012 TeamQuest Corporation. All Rights Reserved. 29