Oracle Enterprise Manager Database Consolidation Workbench —Secret to Successful Consolidation
Prabhaker Gongloor (GP) Senior Director of Product Management, Oracle
Mikael Fries Director Product Management, Oracle
Oracle Confidential – Internal Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Oracle Confidential – Internal 2
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
Motivation & Consolidation Considerations
Database Consolidation Workbench
Database Consolidation Advisor
Consolidation: Making it Happen!
Consolidation: Validation
Demo and Q/A
1
2
3
4
5
Oracle Confidential – Internal 3
6
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
Motivation & Consolidation Considerations
Database Consolidation Workbench
Database Consolidation Advisor
Consolidation: Making it Happen!
Consolidation: Validation
Demo and Q/A
1
2
3
4
5
Oracle Confidential – Internal 4
6
Demogrounds: SLD-023: End-to-End
Database Performance Management
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Quality of Service
Consolidation
Cloud Services
Journey to the Cloud: Stages of Technology Adoption
Maximize Server Utilization,
Streamline Manageability and
Drive Down Capital Expenses
Reduce Operating Expenses
through Self-service
Administration
Maximize Performance with
Advanced Automation
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Database Consolidation: Considerations
– What database consolidation strategy to use?
– Can workloads co-exist together for the chosen strategy, what conflicts exist?
– Can the system handle peak workloads while maintaining performance?
– How to perform migration with reduced downtime?
– How to test the chosen consolidation strategy to minimize risk?
Oracle Confidential – Internal 7
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Consolidation Alternatives
Oracle Confidential – Internal 8
Virtual Machines
Share servers
Dedicated Schema(s)
Share servers, OS
and database
Increasing Consolidation
Dedicated DBs
Share servers and
OS
Pluggable DBs
Share servers, OS
and database
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Consolidation Architectural Choices
Virtual Machines Dedicated DB Dedicated Schemas Pluggable DB
Consolidation Density
Low - Moderate High Highest Highest
Management Very complex (VM
Sprawl) Easy
Easy to Involved
(based on required
resource isolation)
Easy
Isolation Excellent Good Low Good
Implementation & Onboarding
Easy Easy Difficult Easy
Application Suitability
Some (workload
dependent)
All Home grown; requires
app validation
All but have to be
certified for
Database 12c
Oracle Confidential – Internal 9
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Consolidation Tools Available Today
• XLS spreadsheet (seriously)
• Host Consolidation Planner - plan & consolidate multiple hosts to fewer machines
– Physical Virtual, Physical Physical server support
– Exadata, OVM, Oracle Compute Cloud shapes (OCPUs, OC3-7, OC1M-OC5M)
• Consolidation Scenario Report to pick the best strategy – Consolidation ratio & destination server utilization
– Source to Target server mapping
– Confidence of meeting requirements based on historical samples
– Manually mapped consolidation constraint violations and exceptions
Oracle Confidential – Internal 10
Enterprise Manager Consolidation Planner
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
Motivation & Consolidation Considerations
Database Consolidation Workbench
Database Consolidation Advisor
Consolidation: Making it Happen!
Consolidation: Validation
Demo and Q/A
1
2
3
4
5
Oracle Confidential – Internal 11
6
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Database Consolidation Workbench
• Comprehensive end-to-end solution for managing database consolidation
• Provides a risk-free and accurate approach to consolidation by eliminating guess work, human errors
– Analysis based on historical workload (DB, Host metrics)
– Automation in all phases of consolidation from planning to deployment
• Provides a flexible approach for various customer scenarios – 10.2 DB versions and higher support
– Support for consolidation to Oracle Private/Public Cloud or Exadata
– HA options to minimize downtime depending on source/destination DB platform and versions
Oracle Confidential – Internal 12
End-to-end Consolidation Solution
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 13
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Database Consolidation Workbench
Plan
– Gives consolidation advice by identifying candidate databases for the designated consolidation platform using AWR data
Migrate
– Implements consolidation plan by migrating databases to new consolidation platform using EM’s provisioning features
Validate – Validates consolidation plan with Real Application Testing
(SPA) by running test workloads on consolidated databases
Oracle Confidential – Internal 14
End-to-end Consolidation Solution
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
Motivation & Consolidation Considerations
Database Consolidation Workbench
Database Consolidation Advisor
Consolidation: Making it Happen!
Consolidation: Validation
Demo and Q/A
1
2
3
4
5
Oracle Confidential – Internal 15
6
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Database Consolidation Advisor
• Estimates resource utilization under various consolidation scenarios
– Ultra Conservative (peak maximum) Aggressive (average)
• Supported consolidation modes
– Database-to-Database (Multitenant) consolidation
– Database-to-Server (Server, e.g. Exadata) consolidation
– Oracle Public Cloud consolidation
• Consolidation optimization advice
– Identifies conflicts based on workload characteristics, Exadata suitability
– Storage/Platform advice: impact of compression on I/O & storage, impact of I/O offloading & Flash Cache
Oracle Confidential – Internal 16
Key Features
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Database Consolidation Advisor
• ADDM like rules engine to analyze AWR data from source databases
• Data driven rules that read inputs from AWR / EM Metrics / AWR Warehouse
• Two pass rules analysis generates findings which identify bottlenecks
– First pass on source databases
– Second pass when consolidating source databases on a single destination
• Rules Analysis can impact mapping of source databases to destination
• Currently there are 16+ rules on top of 27+ different metrics
• Rules can be easily extended & can be made self updatable
Oracle Confidential – Internal 17
Consolidation Advice: Performance Bottleneck Analysis
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 18
Database Consolidation Advisor: Findings
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Database Consolidation Advisor
• Findings and Recommendations (some examples)
– DATABASE_CPU_BOUND • Detected potentially CPU-bound source database with average active session count of 21.8 that
exceeds its core count. Consider tuning this database to eliminate any CPU bottleneck prior to consolidation planning. Its current workload measurements may not be true indicators of its resource requirements.
– OLTP_IOPS_RECOMMEND_EXADATA • The I/O Requests/Second value of 300 and on-line transaction processing workload indicate that this
database will benefit from consolidation to Oracle Exadata Database Machines to take advantage of Smart Flash Cache.
• Findings can be justified by showing performance charts & trends using AWR viewer
Oracle Confidential – Internal 19
Consolidation Advice: Performance Bottleneck Analysis
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 20
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 21
Consolidation Workbench: Source Server Heatmap
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 22
Consolidation Workbench: Source Server Resource Usage
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Database Consolidation Advisor
• Compression Advisor job
– Can be deployed from EM & runs using DB Scheduler during system maintenance windows
– Generates compression estimates at segment and Tablespace level
– Scans top-N segments at a time incrementally
– Estimates are based on sampling blocks in random and compressing them
• Compression Results are stored in DBSNMP and collected by a Metric
• Results are shown in Consolidation Advisor and Compression Home
Oracle Confidential – Internal 23
Storage and I/O Consolidation Benefits Estimation
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Database Consolidation Advisor
• Smart Scans
– Estimate smart scans using SPA Trials Exadata Simulation mode
– Uses an optimizer parameter to run all plans as if system was Exadata
– Shows predicted impact of Exadata on IO Interconnect bytes
• Flash Cache
– Calculate flash cache benefit from total small reads and writes
– Post EM 13.1: Flash Cache recommendation for OLTP workloads, identify objects to pin in memory based on AWR I/O intensity report
• Compression – Calculate storage reduction from enabling compression, & post EM 13.1 IO savings
Oracle Confidential – Internal 24
Exadata: Storage and I/O Consolidation Benefits
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 25
Database Consolidation Advisor: Storage
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 26
Consolidation Workbench: SPA Simulation
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 27
Consolidation Workbench: Destination Server Mapping
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
Motivation & Consolidation Considerations
Database Consolidation Workbench
Database Consolidation Advisor
Consolidation: Making it Happen!
Consolidation: Validation
Demo and Q/A
1
2
3
4
5
Oracle Confidential – Internal 28
6
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Migrate
• Multiple migration methods based on source and target database attributes (endian-ness, versions)
– Offline: RMAN clone, Data Pump
– Online: Data Guard standby , migrate as PDB with minimum downtime • Transportable TTS is used to minimize downtime in "Migrate as PDB". Any source database non CDB
version 11203, 11204 and 12.x can be moved to 12.x as PDB
• Supports 10g(10.2) to 12c (12.1) databases
• SI & RAC, Multitenant and non-CDB modes
• Added to EM’s provisioning DP library
• Requires O_H to be pre-provisioned
Oracle Confidential – Internal 29
Make it so: Database Migration to Consolidated Environment
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Mode Source DB Solution Minimal
Downtime
Destination Move DB Steps
DB Type Platform/version Uptime Downtime
D2S 10.2 and up Data Guard
Yes RAC/SI Same
1) Create Standby 2) Convert to RAC
3) Switchover standby
D2S 10.2 and up RMAN No RAC/SI Same 1) Create clone 2) Convert to RAC
D2D 11.2.0.3+ Full TTS Yes RAC/SI Cross Platform and Endian + DB 12.1+
1) RMAN Level 0 Clone
2) Level 1 incremental
3) Final Level 1 incremental & Metadata Export / Import with TTS
D2D/ D2S
10.2 and up Datapump No RAC/SI/
PDB
Cross Platform and Endian & Same or higher DB Version
1) Create DB/PDB if required
2) Export schema from source
3) Import
DB Consolidation Migration (Make it so)
Oracle Confidential – Internal 30
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Make it so: Migration to Consolidated Platform
• XML-based emcli script generated to implement the recommended consolidation plan
• Script can be edited to override
– Parameters
– Credentials
– Other fields
• Submitted as EM deployment procedures and can be tracked for completion
• Validation of SQL execution plans for the specified workload (STS) submitted as part of deployment script
Oracle Confidential – Internal 31
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 32
Note: In this case, we used D2D consolidation so mapping many databases to same server, hence only Data Pump and TTS options are enabled in this case
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
Motivation & Consolidation Considerations
Database Consolidation Workbench
Database Consolidation Advisor
Consolidation: Making it Happen!
Consolidation: Validation
Demo and Q/A
1
2
3
4
5
Oracle Confidential – Internal 33
6
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Database Consolidation: Why Validate?
• Database Consolidation Advisor recommends optimal consolidation strategy based on historical workload information
• However, actual database consolidation involves potentially many major changes and therefore can impact business SLAs and system stability
– Co-existence of workloads in the same database
– Platform changes (Operating Systems, Exadata), move to Oracle Public Cloud
– DB upgrades
Oracle Confidential – Internal 34
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Database Consolidation: Why Validate?
• Real Application Testing supports risk-free administration
– Supports day-to-day use cases
• Gathering optimizer statistics
• Validating indexes
• SQL Profiles
– Capacity planning and consolidation
– Validating newer DB releases, patchsets, options like RAC, ACO, TDE, In-memory, ,etc.
Oracle Confidential – Internal 35
Secret Behind Successful Consolidation: Real Application Testing
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Real Application Testing
Consolidation Method Method Performance Test
Server, OS, Schema, Multitenant SQL Performance Analyzer (SPA)
Before and after “Make it so” SPA trials, SQL unit testing for response time in all consolidation scenarios & Exadata simulation in planning phase
Server, OS Database Replay Load, performance testing for throughput
Schema, Multitenant
Concurrent Database Replay Replay multiple workloads against one database for throughput and scalability
Oracle Confidential – Internal 36
Capture Workload
Create Test System
Deploy Replay Clients
Use SQL Performance Analyzer to assess SQL workload impact
Replay Workload
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
• SQL Performance Analyzer (SPA) component of Real Application Testing used by Consolidation Workbench in current release
– SPA is used to validate SQL execution plans for the given workload on the consolidated platform
– Planned support for Database Replay in future releases
• DBA captures interesting SQL workload into STS on each of the source databases that should be validated for the consolidated platform – STS used for testing are picked up from the last successful SPA task on the source
databases
– Same STS are also used for Exadata Simulation in planning phase
Real Application Testing for Consolidation Validation
Oracle Confidential – Internal 37
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
• Automatic SPA consolidation testing
– Before trial on source databases in Explain Plan Mode
– After trial on consolidated database (after Migrate) in Explain Plan Mode
– SPA report highlighting plan regressions generated
• Review plan regressions and take appropriate action
– Iterative testing
– Remediation and tuning as required
• Deploy consolidated platform in production after satisfied with testing results
•
Real Application Testing for Consolidation Validation
Oracle Confidential – Internal 38
HR
ERP
SCM
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Database Consolidation Workbench: Roadmap
• Support for Sun Super Cluster, ODA
• Support for custom hardware system profiles (CPU, Memory, Storage, I/O)
• Migration Workbench UI (rich interview-based UI alternative to the current XML interface)
• Support for migrating individual source PDBs to new CDB (by measuring PDB-level workload)
• IO savings estimation
• Flash cache object placement recommendations
Oracle Confidential – Internal 39
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
Motivation & Consolidation Considerations
Database Consolidation Workbench
Database Consolidation Advisor
Making it Happen!
Testing
Demo and Q/A
1
2
3
4
5
Oracle Confidential – Internal 40
6
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Demo Database Consolidation Workbench
Oracle Confidential – Internal 41
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Oracle Confidential – Internal 42
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 43
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Backup/Demo slides
Oracle Confidential – Internal 45
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
DS Consolidation: 1x Scale Factor, Non-CDB, ASM-Normal, No Compression: Using Existing Servers
Oracle Confidential – Internal 46
Summary: Cannot use existing servers, severely under powered on CPU resource, let’s try DB Multitenant option next
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
DS Consolidation: 1x Scale Factor, CDB, ASM-Normal, No Compression: Using Existing Servers
Oracle Confidential – Internal 47
Summary: Multitenant option works much better, but not sure if I can accommodate future growth
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
DD, Exadata Consolidation: 5x Scale Factor, CDB, ASM-HA, Compression
Oracle Confidential – Internal 48
Summary: Try 5x CPU and Memory scale factor, CDB, ASM and Compression. Each option can be evaluated incrementally
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Storage Savings with Compression
Oracle Confidential – Internal 49
Selected compression option
for the workload
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 50
Classroom Training
Learning Subscription
Live Virtual Class
Training On Demand
Keep Learning with Oracle University
education.oracle.com
Cloud
Technology
Applications
Industries
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Session Surveys
Help us help you!! • The [Program Committee J1] [organizing committee OW] would like to
invite you to take a moment to give us your session feedback. Your feedback will help us to improve your conference.
• Please be sure to add your feedback for your attended sessions by using the Mobile Survey or in Schedule Builder.
Oracle Confidential – Internal 51