View
8
Download
0
Category
Preview:
Citation preview
This solution guide describes the Continuous Availability Modular add-on to the
Federation Enterprise Hybrid Cloud Foundation solution for SAP. This guide
introduces the architecture and features that ensure continuous availability of
SAP services and operations in the cloud.
October 2015
2
Copyright © 2015 EMC Corporation. All rights reserved. Published in the USA.
Published October 2015
EMC believes the information in this publication is accurate as of its publication date. The
information is subject to change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES
NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE
INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Use,
copying, and distribution of any EMC software described in this publication requires an
applicable software license.
EMC2, EMC, VMAX, VNX, XtremIO, VPLEX, ViPR, PowerPath, Data Domain, Avamar, and the
EMC logo are registered trademarks or trademarks of EMC Corporation in the United States
and other countries. All other trademarks used herein are the property of their respective
owners.
For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on
EMC.com.
Federation Enterprise Hybrid Cloud 3.1 - Data Protection for SAP: Continuous
Availability Solution Guide
Part Number H14266
Contents
3
Chapter 1 Executive Summary ............................................................. 5
Document purpose .............................................................................................. 6
Audience ............................................................................................................ 6
Essential reading ................................................................................................ 6
Solution purpose ................................................................................................. 6
Business challenge .............................................................................................. 7
Technology solution............................................................................................. 7
Terminology ....................................................................................................... 7
Chapter 2 Continuous Availability Solution Overview ............................... 9
Overview .......................................................................................................... 10
Solution architecture .......................................................................................... 10
Logical architecture ........................................................................................ 10
Physical architecture with VPLEX host Cross-Connect ......................................... 12
Software resources ............................................................................................ 12
Chapter 3 Design Considerations ........................................................ 14
Overview .......................................................................................................... 15
SAP system architecture ..................................................................................... 15
SAP instances outage impact ............................................................................... 16
SAP database instance outage ......................................................................... 16
SAP ASCS instance outage (enqueue and message server services) ..................... 16
SAP Global File Systems ................................................................................. 16
Considerations and best practices ........................................................................ 17
Auto start of SAP instances ............................................................................. 17
vSphere HA and DRS ..................................................................................... 18
Site affinity for virtual machines ...................................................................... 19
Chapter 4 Protecting SAP Applications with Continuous Availability ......... 21
Overview .......................................................................................................... 22
Implementing Continuous Availability for SAP applications ...................................... 22
Use case 1: Composing SAP provisioning service on CA-enabled infrastructure ...... 22
Use case 2: Setting costs for CA-enabled storage .............................................. 25
Validation.......................................................................................................... 26
Use case 3: Single ESXi host failure ................................................................. 28
Use case 4: One leg of VPLEX cluster failure ..................................................... 29
Use case 5: Single data center failure .............................................................. 30
Use case 6: Planned host maintenance ............................................................. 31
Use case 7: Compute resource balance ............................................................ 33
Chapter 5 Conclusion ........................................................................ 34
Conclusion ........................................................................................................ 35
Contents
4
Appendix A References ........................................................................ 36
References ........................................................................................................ 37
EMC Solution Guides ...................................................................................... 37
Product documentation .................................................................................. 37
Other documentation ..................................................................................... 37
Figures
Figure 1. Logical architecture ......................................................................... 11
Figure 2. Physical architecture ....................................................................... 12
Figure 3. SAP system architecture .................................................................. 15
Figure 4. Dependency among SAP instances .................................................... 17
Figure 5. Enable auto start of SAP instances during provisioning ........................ 18
Figure 6. Deploying SAP applications with site affinity ....................................... 20
Figure 7. Sample view of Site Affinity DRS Rule configuration ............................ 20
Figure 8. Adding a cloud template from vRealize Automation blueprints .............. 23
Figure 9. Adding cloud template mapping in the logical template ....................... 23
Figure 10. Creating new deployment profiles to use the infrastructure enabled for
continuous availability ..................................................................... 24
Figure 11. Mapping cloud templates enabled for continuous availability ................ 25
Figure 12. Creating a virtual machine storage profile .......................................... 26
Figure 13. Setting the cost for CA-enabled storage............................................. 26
Figure 14. VPLEX status after data center A VPLEX isolation ................................ 29
Figure 15. PowerPath status after VPLEX isolation .............................................. 30
Figure 16. No interruption of SAP CA2 after VPLEX cluster-2 failure on data center
B .................................................................................................. 30
Figure 17. Non-disruptive host maintenance ...................................................... 32
Tables
Terminology .................................................................................... 7 Table 1.
Solution software resources ............................................................. 12 Table 2.
Impact of SAP instances outages ...................................................... 16 Table 3.
VM restart priority for SAP instances ................................................. 18 Table 4.
List of ESXi hosts ............................................................................ 27 Table 5.
CA1 SAP system details ................................................................... 27 Table 6.
CA2 SAP system details ................................................................... 27 Table 7.
List of vSphere DRS rules ................................................................ 27 Table 8.
Use case 1: Reference metrics for ESXi failure .................................... 28 Table 9.
Use case 5: Reference metrics for single data center failure ................. 31 Table 10.
ESXi host resource balancing ........................................................... 33 Table 11.
Chapter 1: Executive Summary
5
This chapter presents the following topics:
Document purpose .............................................................................................. 6
Audience ............................................................................................................ 6
Essential reading ................................................................................................ 6
Solution purpose ................................................................................................. 6
Business challenge .............................................................................................. 7
Technology solution............................................................................................. 7
Terminology ....................................................................................................... 7
Chapter 1: Executive Summary
6
This solution guide describes a Continuous Availability (CA) add-on solution for the
Federation Enterprise Hybrid Cloud for SAP. This document:
Introduces the technical components needed to implement and operate the solution
Describes EMC’s testing and validation of the solution’s functionality
Evaluates the technical and business value of the solution in the context of a
Federation Enterprise Hybrid Cloud for SAP
This document is a companion to Federation Enterprise Hybrid Cloud 3.1: Foundation for
SAP Solution Guide, which provides critical reference information for planning and designing
a data protection solution for the hybrid cloud.
This solution guide is intended for executives, managers, SAP architects, cloud
administrators, and technical administrators of IT environments who want to implement CA
for a hybrid cloud infrastructure-as-a-service (IaaS) platform managing SAP landscapes. You
should be familiar with the VMware® vRealize® Suite, storage technologies, and general IT
functions and requirements, and how they fit into a hybrid cloud architecture.
To understand the concepts, architecture, and functionalities of the Federation Enterprise
Hybrid Cloud and how to operate SAP on top of it, you are advised to read the following
documents:
Federation Enterprise Hybrid Cloud 3.1: Foundation for SAP Solution Guide
Federation Enterprise Hybrid Cloud 3.1: Foundation Infrastructure Reference
Architecture Guide
Federation Enterprise Hybrid Cloud 3.1: Concepts and Architecture Solution Guide
Federation Enterprise Hybrid Cloud 3.1: Operations Solution Guide
Federation Enterprise Hybrid Cloud 3.1: Security Management Solution Guide
Organizations for which SAP is a major part of their core business operations must have a
business continuity management plan to manage risks in an IT environment and prepare for
recovery in the event of a disaster. Specifically, the plan should include measures to:
Safeguard the continuity of their business operations and protect revenue
Recover systems to at least a minimum level of operation if an outage occurs
A business continuity management plan has three components: backup, continuous
availability, and disaster recovery.
This guide focuses on the CA component. It describes a cloud computing solution that
increases the availability of an SAP landscape and provides rapid recovery after a severe,
isolated technical failure in one or more infrastructure or application components.
Chapter 1: Executive Summary
7
IT leaders are increasingly transferring mission-critical SAP applications to cloud
infrastructures to gain considerable cost savings, simplified management, and data center
efficiencies.
High availability and application mobility are essential for SAP applications. They provide
non-disruptive and live workload migrations within or across data centers, and implement
fully automated restarts when outages happen.
In traditional infrastructure designs, workloads across physical data centers are broken
when any of the infrastructure components fail, causing application downtime. Also,
traditional solutions are usually limited to specific combinations of operating
system/database software and hardware. These combinations make the solutions complex,
inflexible, costly, and difficult to implement and maintain.
To address these challenges for SAP applications, the Federation Enterprise Hybrid Cloud
offers:
High availability, which helps to manage risks in an IT environment and plans for
recovery when a technical failure occurs
Application mobility, which moves running workloads without disruption between
servers to avoid any downtime resulting from planned maintenance or a foreseeable
outage
This continuous availability solution uses EMC and VMware technologies, in particular EMC
VPLEX® and VMware vSphere® High Availability (HA), to enable continuous availability and
application mobility as a service within the Federation Enterprise Hybrid Cloud to provide:
Automated non-disruptive virtual machine movement within a cloud data center,
stretched across two separated data centers running SAP mission-critical systems
Simple automated infrastructure HA protection for complete SAP on-premises cloud
data center
Minimized RTO and near zero RPO
Additional benefits include:
Automatic restart of SAP instances in the event of server failures
Increased utilization of hardware and software assets:
Automatic load balancing between data centers
Zero downtime on maintenance
Table 1 defines the key terms relevant to this solution that are used in this document.
Terminology Table 1.
Term Definition
ABAP SAP central service (ASCS)
ABAP SAP Central Services. An instance in a distributed SAP system that hosts message server and enqueuer server
AccessAnywhere A technology of VPLEX distributed virtual storage to enable data access over distance within and between data centers.
Chapter 1: Executive Summary
8
Term Definition
Additional application server (AAS)
An SAP application server instance installed in a distributed SAP system
Application blueprint Logical topology of an application for deployment in vRealize Automation Application Services. An application blueprint captures the structure of an application with logical nodes, their corresponding services and operating systems, dependencies, default configurations, and network and storage topology requirements. The blueprint is published as a catalog item in the common service catalog.
High availability (HA) A system or component continuously operational for a desirably long length of time. Availability can be measured relative to "100% operational" or "never failing."
Logical template A predefined virtual machine definition in vRealize Application Services that can be mapped to a cloud template (and supporting services) in the cloud catalog
enabling an application blueprint to remain cloud-agnostic.
Primary application server
(PAS)
The first application server instance installed in a
distributed SAP system
VMware vSphere Metro
Storage Cluster (VMware vMSC)
VMware vMSC is a configuration option that allows for the
use of stretched clusters, meaning that servers within a cluster are spread across geographical locations. This configuration allows organizations to perform load balancing and nondisruptive live migrations between active data centers.
Chapter 2: Continuous Availability Solution Overview
9
This chapter presents the following topics:
Overview .......................................................................................................... 10
Solution architecture .......................................................................................... 10
Software resources ............................................................................................ 12
Chapter 2: Continuous Availability Solution Overview
10
This solution is built on the proven design of the VMware vSphere and EMC VPLEX Metro™
storage cluster. Using the features and functionality of Metro, the solution offers:
Workload mobility
Disaster avoidance
Location-optimized storage
Increased uptime and utilization
Storage provisioning automation
Storage for the solution is provided by XtremIO, VMAX and VNX arrays located at both of
the data center sites that host the infrastructure. The storage is presented to the VPLEX
arrays, also at both sites, forming distributed storage kept synchronized and presented to
the vSphere cluster as active/active LUNs.
VMware vCenter Orchestrator™ is central to all the customizations and operations that are
used in this solution. It manages operations across several EMC and VMware products,
including:
VMware vRealize™ Automation
VMware vCenter™
EMC ViPR®
This solution focuses on the implementation, management, and automation of the
continuous availability for SAP applications on the Federation Enterprise Hybrid Cloud.
Figure 1 shows the overall logical architecture of the Continuous Availability solution for
Federation Enterprise Hybrid Cloud. Logical
architecture
Chapter 2: Continuous Availability Solution Overview
11
Figure 1. Logical architecture
In this configuration, each pod is stretched across both sites in active/active fashion. The
underlying VPLEX distributed storage allows all the management components and workloads
to either proactively move before a known event using vMotion or reactively restart using
vSphere High Availability if an unpredicted failure event occurs.
Core Pod
The Core Pod is used to host a core set of resources that must exist before the remainder of
the cloud can be deployed. These core resources include vCenter Server, Microsoft SQL
Server 2012, and the vCloud Networking and Security or NSX Manager. The hardware that
hosts this pod need not managed by cloud components, but the virtual machines it hosts are
a critical foundation of the cloud.
Automation Pod
The Automation Pod hosts the virtual machines that automate and manage the cloud
infrastructure that supports the workloads consumed by the clouds tenants. The Automation
Pod supports the components responsible for functions such as the user portal and
automated provisioning, monitoring, metering, and reporting.
Chapter 2: Continuous Availability Solution Overview
12
NEI Pod
The NEI Pod hosts the NSX Edge appliances and NSX Controllers or the vCloud Networking
and Security components, and becomes the convergence point at which the physical and
virtual networks connect.
Workload Pods
The Workload Pods are configured and assigned in vRealize Automation as shared resources,
to host SAP virtual machines deployed by the different business groups in the hybrid cloud
environment. These Workload Pods are deployed as VMware vSphere clusters in VMware
vCenter endpoints.
EMC GeoSynchrony® supports the concept of a VPLEX Metro cluster with Cross-Connect.
This configuration provides a perfect platform for a uniform vSphere stretched-cluster
deployment.
ESXi hosts can access a distributed volume on the local VPLEX cluster and on the remote
cluster in the event of a failure. When this configuration is used with VPLEX Witness, ESXi
hosts are able to survive through multiple types of failure scenarios. For example, in the
event of a VPLEX cluster or back-end storage array failure, the ESXi hosts can still access
the second VPLEX cluster with no disruption in service.
Figure 2 shows the physical architecture of uniform host access configuration with VPLEX
host Cross-Connect:
Figure 2. Physical architecture
Table 2 details the software resources that are used in the solution.
Solution software resources Table 2.
Software Version Purpose
Federation Enterprise Hybrid
Cloud, Foundation module
3.1 Customization package for storage as a
service (STaaS) and foundation workflows
VMware vRealize Automation Application Services
6.2.1 Application provisioning
Physical
architecture with
VPLEX host
Cross-Connect
Chapter 2: Continuous Availability Solution Overview
13
Software Version Purpose
SAP provisioning module 3.1.0 Customization package for provisioning SAP systems
For a complete, up-to-date list of all software requirements for Federation Enterprise Hybrid
Cloud, refer to the EMC Simple Support Matrix (ESSM).
Chapter 3: Design Considerations
14
This chapter presents the following topics:
Overview .......................................................................................................... 15
SAP system architecture ..................................................................................... 15
SAP instances outage impact ............................................................................... 16
Considerations and best practices ........................................................................ 17
Chapter 3: Design Considerations
15
This solution is designed to eliminate single-points-of-failure (SPOFs) at the infrastructure
level, and to provide fast and automatic service recovery for SAP applications in case of
hardware failure. This chapter describes the SAP system architecture and analyzes the
impact of SAP instances outage. It also details the design considerations necessary to
minimize the duration and impact of service outages, and to automatically restore SAP
services.
This solution focuses on the SAP-specific design considerations, including:
Auto start of SAP instances
vSphere HA
vSphere Distributed Resource Scheduling (DRS) rule
For the general best practices of designing the CA solution on Federation Enterprise Hybrid
Cloud, refer to Federation Enterprise Hybrid Cloud 3.1: Concepts and Architecture Solution
Guide.
A typical SAP ERP 6.0 system consists of:
ABAP SAP Central Services (ASCS)
Application server instances
The first application server instance installed in an SAP system is called the Primary
Application Server instance (PAS). Additional Application Server (AAS) instances can
be installed later as needed to expand the compute resource capability.
Database (DB) instance
Also, a Global File System is required to provide shared access to /sapmnt and
/usr/sap/trans. In this solution, we enable the network file system (NFS) server on ASCS to
serve as the SAP Global File System. Figure 3 shows the SAP system architecture.
Figure 3. SAP system architecture
Chapter 3: Design Considerations
16
SAP PAS and AAS instances are installed to provide work processes such as dialog (DIA),
background (BGD), update (UPD), spool (SPO), and gateway. This enables workload balance
and protection against vSphere ESXi host or virtual machine failures.
The SAP ASCS, SAP DB, and SAP Global File Systems (NFS) server are SPOF components in
an SAP distributed architecture without additional operating system , DB-specific, or
complex-clustering technologies. Those application-level high-availability technologies can
be implemented with additional efforts, but their implementation is beyond the scope of this
document.
As shown in Table 3, this section analyzes the impact of SAP instances outage and
addresses these SPOFs.
Impact of SAP instances outages Table 3.
SPOF component Technical impact Business impact
Database server SAP system is suspended SAP service will become unavailable
Enqueue server Application transaction locks
are lost
SAP will become unavailable
Message server New connection or requests
cannot be executed
Users cannot login via
message server
RFC connections in other SAP systems via message server will be unavailable
Global File system /sapmnt/<SID>
Cannot start new application servers
Cannot write background job logs
Scalability of application servers will be affected
Tractability of background jobs will be affected
Global File system
/usr/sap/trans
Cannot use change and transport system
Development and customization will be affected
The whole SAP system is suspended if the SAP database instance component fails. When the
database becomes available again, the SAP work processes in the SAP application servers
reconnect automatically and users can resume their work. All in-flight transactions are rolled
back for consistency.
If the SAP ASCS instance component fails, both the enqueue and the message server
services become unavailable.
If the enqueue service fails, the SAP application transaction locks are lost, and the SAP
system is unavailable. If this happens, the SAP functional team must analyze the business
process impact after the service is restarted.
If the message server stops working, existing connections are not affected. New requests
cannot be executed for the dialog, update, and enqueue server.
If an SAP Global File Systems (NFS server) component fails, the NFS shares /sapmnt/<SID>
and the /usr/sap/trans are not available, with the following consequences:
NFS shares /sapmnt/<SID> not available: Additional SAP application servers are
prevented from starting. Active SAP application servers are not affected. The
background job will no longer be able to write logs under /sapmnt/<SID>/global.
SAP database
instance outage
SAP ASCS
instance outage
(enqueue and
message server
services)
SAP Global File
Systems
Chapter 3: Design Considerations
17
/usr/sap/trans transport directory is not available: The change and transport system
cannot be used.
In a large-scale cloud environment, a tenant may run hundreds of SAP systems. During
hardware failure, some SAP virtual machines are restarted by vSphere HA. To start a
distributed SAP system:
1. ASCS with Global file system
2. DB
3. PAS/AAS
Figure 4 shows the dependency among SAP instances.
Figure 4. Dependency among SAP instances
On the Linux operating system, however, the SAP and Oracle database, services are not
restarted by default until they are manually triggered. To eliminate manual intervention and
reduce recovery time, we developed the scripts to start each type of SAP instance
automatically on startup of the operating system.
For NFS services, the NFS client waits until the NFS server is available.
For the DB service, we developed shell scripts to monitor the availability of the
database in the background during boot up. Once the database is online, PAS/AAS is
started automatically.
The auto start of SAP instances can be enabled during provisioning by choosing Yes for the
ENABLE_AUTO_START parameter, as shown in Figure 5.
Auto start of SAP
instances
Chapter 3: Design Considerations
18
Figure 5. Enable auto start of SAP instances during provisioning
Admission control
Admission Control must be enabled. EMC recommends that you set the Admission Control
Policy to Percentage of Cluster Resources Reserved as failover spare capacity and
reserve 50% of CPU and memory for the DRS clusters where SAP production systems run.
This configuration reserves cluster compute resources to guarantee the restart of all virtual
machines to another surviving data center in the event of failure of a single data center.
VM restart priority
Override the VM Restart Priority according to the rules listed in Table 4.
VM restart priority for SAP instances Table 4.
SAP instance VM restart priority
ASCS High
DB Medium
PAS/AAS Low
If an ESXi host fails, vSphere HA powers on the SAP virtual machines according to the
virtual machine restart priority, from high to low. Although this setting does not guarantee
the proper start order of SAP services, it ensures that virtual machines with higher priority
are powered on first, avoiding the PAS or AAS instances being powered on before ASCS or
DB.
VM monitoring
When VM monitoring is enabled, vSphere HA restarts a virtual machine when both the
following conditions are met:
Heartbeats sent from VMware Tools are not received in a set period
The virtual machine is not generating storage or network IO for 120 seconds by
default. (You can change the default interval by using the das.iostatsInterval
advanced cluster level setting.)
vSphere HA and
DRS
Chapter 3: Design Considerations
19
VM monitoring is disabled by default. If VM monitoring is required, we recommend setting
the Monitor Sensitivity to Low.
vSphere DRS rule
We recommend creating vSphere DRS rules manually to separate SAP PAS and AAS
instances on different ESXi hosts. If the ESXi host running PAS or AAS instances fails, the
surviving PAS or AAS instances can provide redundancy of SAP work processes.
We also recommend creating vSphere DRS rules manually to separate SAP ASCS and DB on
different ESXi hosts in order to minimize the duration of service outage.
Federation Enterprise Hybrid Cloud 3.1 introduces a new feature called Site Affinity, which
uses VMware host Distributed Resource Scheduler (DRS) groups to subdivide the ESXi hosts
in each workload and management cluster into groupings of hosts corresponding to their
respective sites. The site affinity feature allows better network bandwidth utilization for a
distributed SAP system within one data center, avoiding additional network overhead across
two data centers. It does this by defining two groups in the format SiteName_Hosts where
the site names of both sites are defined during the installation of the Federation Enterprise
Hybrid Cloud foundation packaging.
VMware virtual machine DRS groups are also created in the format Sitename_VMs during
the preparation of the ESXi cluster for continuous availability.
Storage reservation polices created by the Federation Enterprise Hybrid Cloud storage as
service workflows are named to indicate the site in which that storage type is preferred to
run.
During composing of SAP provisioning services, the administrator will choose from a list of
storage reservation policies. Federation Enterprise Hybrid Cloud custom workflows use this
information to place the virtual machine on an ESXi cluster with access to the required
storage type as well as placing the virtual machine into the appropriate virtual machine DRS
group.
Virtual machine to host DRS rules are then used to bind virtual machines to the preferred
site by configuring the respective SiteName_VMs virtual machine DRS group to a value of
‘should run’ on SiteName_Hosts host DRS group. This ensures virtual machines run on the
required site, while allowing them the flexibility of failing over if the infrastructure on that
site becomes unavailable.
Figure 6 shows a simple example of two scenarios where SAP applications are deployed to a
VMware vMSC and how the logic operates to place those virtual machines on their preferred
sites.
Site affinity for
virtual machines
Chapter 3: Design Considerations
20
Figure 6. Deploying SAP applications with site affinity
Figure 7 shows how the virtual machine DRS groups and affinity rules might look in a
sample configuration.
Figure 7. Sample view of Site Affinity DRS Rule configuration
Chapter 4: Protecting SAP Applications with Continuous Availability
21
This chapter presents the following topics:
Overview .......................................................................................................... 22
Implementing Continuous Availability for SAP applications ...................................... 22
Validation.......................................................................................................... 26
Chapter 4: Protecting SAP Applications with Continuous Availability
22
This chapter demonstrates use cases to configure provisioning of an SAP system on
continuous availability (CA)-enabled infrastructure, including:
Composing SAP provisioning service on the CA-enabled infrastructure
Setting the cost for CA-enabled storage
This chapter also validates how to protect SAP applications from different failure scenarios
and demonstrates the flexibility of planned non-disruptive host maintenance. The following
scenarios are described:
Single ESXi host failure
One leg of VPLEX cluster failure
Single data center failure
Planned host maintenance
Compute resource balance
Before creating SAP provisioning services on CA-enabled infrastructure, follow the
Federation Enterprise Hybrid Cloud 3.1: Operations Solution Guide to configure CA services:
1. Provision CA-enabled storage on both data centers (Use case 2).
2. Create Reservation Policies (Use case 3).
3. Create a new Reservation to reserve CA-enabled storage (Use case 2).
4. Copy an existing SAP blueprint and specify the reservation policy and storage
reservation policy which connects to the CA-enabled infrastructure (Use case 4 and
6).
A CA environment presents new considerations for auto-provisioning SAP systems. After
configuring a reservation and blueprint in the vRealize Automation Center portal, you must
compose the SAP provisioning service on vRealize Automation Application Services to deploy
an SAP application on the CA-enabled infrastructure. All configurations in this section are
modifications and enhancements based on the existing components that were created as
described in Federation Enterprise Hybrid Cloud 3.1: Foundation for SAP Solution Guide You
must create and properly configure all of the components, including the cloud provider,
logical template, service, and application, for SAP provisioning.
To enable SAP provisioning service on the CA-enabled infrastructure:
1. Log in to the vRealize Automation application services portal as a service
architect.
2. Navigate to Cloud Providers and edit the cloud provider for HR business
group. Add the blueprints from vRealize Automation that use CA-enabled
infrastructure, as shown in Figure 8.
Use case 1:
Composing SAP
provisioning
service on CA-
enabled
infrastructure
Chapter 4: Protecting SAP Applications with Continuous Availability
23
Figure 8. Adding a cloud template from vRealize Automation blueprints
3. Navigate to Logical Templates. Edit the logical template SLES11 SP3 for SAP
Application and add the two cloud template mappings (see Figure 9) created in
step 2.
Figure 9. Adding cloud template mapping in the logical template
4. Navigate to Applications. Edit the SAP Distributed installation applications.
Navigate to a proper application version. For example, we navigated to SAP Distributed installation and clicked version 6.0.7. Create new deployment profiles to use the CA-enabled infrastructure.
Chapter 4: Protecting SAP Applications with Continuous Availability
24
As shown in Figure 10, we created a new deployment profile named “ERP 6.07 SR2
CA-Enabled” under the HR business group.
Figure 10. Creating new deployment profiles to use the infrastructure enabled for continuous availability
5. From the Deployment Profile Creation Wizard, map each node to the CA-
enabled cloud template that was added in step 3. For storage, select the CA-
enabled storage reservation policy created automatically by the cloud storage
provisioning service in vRealize Automation portal.
As shown in Figure 11, we mapped the cloud template SLES 11 SP3 for SAP
CA-Enabled to the ASCS and PAS instances, and mapped SLES 11 SP3 for SAP
DB instance CA-Enabled to the DB node. We also selected the S1-CA-Silver
CA-Enabled storage reservation policy for additional hard disks in each node.
Chapter 4: Protecting SAP Applications with Continuous Availability
25
Figure 11. Mapping cloud templates enabled for continuous availability
6. Finish the wizard with appropriate values for vCPUs, memory, and disk size for each
node.
7. Publish the deployment profile to the vRealize Automation portal.
8. Repeat steps 4 to 7 to create additional deployment profiles as needed. For example,
you may need to provision SAP systems on other storage tiers on different sites,
larger or smaller disk size.
9. Repeat steps 4 to 8 to configure SAP Standard Installation and SAP AAS instances.
10. Log in to the vRealize Automation portal as tenant administrator. Assign the
published SAP provisioning application to services and assign entitlements in
vRealize Automation portal.
End users can now request an SAP system, which will be provisioned on the CA-enabled
infrastructure.
This solution uses VMware vRealize Business, with its integration with vCenter and vRealize
Automation, to provide chargeback information on the storage provisioned to and consumed
in the hybrid cloud.
The ViPR storage provider running in vCenter automatically captures the underlying
capabilities of storage provisioned by ViPR. The storage administrator creates storage
profiles based on the storage capabilities of the ViPR storage, which is configured to provide
multiple storage service offerings. This integration enables vRealize Business to
automatically discover and group datastores based on predefined service levels of storage.
In this solution, we created separate virtual machine storage profiles for each of the storage
capabilities that were automatically captured by the ViPR storage provider, as shown in
Figure 12.
Use case 2:
Setting costs for
CA-enabled
storage
Chapter 4: Protecting SAP Applications with Continuous Availability
26
Figure 12. Creating a virtual machine storage profile
After the ViPR storage provider has automatically configured the datastores with the
appropriate storage capability, these datastores can be grouped and managed in vRealize
Business, according to their storage profile. Figure 13 shows that the cost profiles created in
vCenter are discovered by vRealize Business.
Figure 13. Setting the cost for CA-enabled storage
This enables the business management administrator to group tiered datastores provisioned
with ViPR and set the monthly cost per GB as needed to differentiate the cost of various
storage service levels. For example, to differentiate CA-enabled storages from other
storages, we set the monthly cost per GB of CA-enabled storage ”S1-CA-Silver” to $0.4 to
reflect the additional cost for VPLEX and other hardware required to build the solution.
In this section, we demonstrate how the CA-enabled infrastructure protects SAP systems
from different types of hardware failure. We simulated the following three hardware failure
scenarios:
Single ESXi host failure
Single leg of VPLEX and storage array failure
Single data center failure
We recorded the behavior of the SAP virtual machines during the hardware failures and
captured the duration of the SAP service restoration. We also simulated a planned host
Chapter 4: Protecting SAP Applications with Continuous Availability
27
maintenance, which allows non-disruptive host maintenance while keeping the SAP system
service availability.
In the test environment, six ESXi hosts were configured for a tenant resource cluster. Three
of the hosts came from data center A, and three from data center B. Table 5 shows the host
details.
List of ESXi hosts Table 5.
Data center ESXi hosts
A s1vccomp145, s1vccomp146, s1vccomp147
B s2vccomp45, s2vccomp46, s2vccomp47
The two data centers were configured according to the solution architecture, as shown in
Figure 1 and Figure 2. In order to simulate the worst connection between two data centers,
we configured a round trip latency of 10 ms for both the SAN and IP networks using a WAN
simulator.
We provisioned a 1 TB data center A preferred CA-enabled storage and a 1 TB data center B
preferred CA-enabled storage for the HR business group. We deployed two distributed SAP
systems with CA1 on data center A and CA2 on data center B. Table 6 and Table 7 list the
details of the SAP systems.
CA1 SAP system details Table 6.
Instance Virtual machine name
ASCS sapca1ascs
DB sapca1db
PAS sapca1pas
AAS sapca1aas
CA2 SAP system details Table 7.
Instance Virtual machine name
ASCS sapca2ascs
DB sapca2db
PAS sapca2pas
AAS sapca2aas
vSphere DRS rules were created to prevent the SAP DB and ASCS instances from running in
the same host. This is because both instances are SPOFs and must be protected from ESXi
host failure. Additional rules were created to keep SAP Application Servers running on
different ESXi hosts to provide redundancy for the SAP work processes. Table 8 lists details
about the vSphere DRS rules.
List of vSphere DRS rules Table 8.
DRS rule name Members Rule Type
SAP Application Server CA1 sapca1pas, sapca1aas Separate
SAP DB and ASCS CA1 sapca1db, sapca1ascs Separate
SAP Application Server CA2 sapca2pas, sapca2aas Separate
Chapter 4: Protecting SAP Applications with Continuous Availability
28
DRS rule name Members Rule Type
SAP DB and ASCS CA2 sapca2db, sapca2ascs Separate
S1_VM_to_Host_DRS_rule Virtual machines:
sapca1pas, sapca1aas, sapca1db, sapca1ascs
Hosts:
s1vccomp145, s1vccomp146, s1vccomp147
Should Run
S2_VM_to_Host_DRS_rule Virtual machines:
sapca2pas, sapca2aas, sapca2db, sapca2ascs
Hosts:
s2vccomp45, s2vccomp46, s2vccomp47
Should Run
Test scenario
This test scenario validates how the SAP virtual machines on a failed host can be restarted
on the surviving ESXi hosts when an unexpected ESXi host failure occurs in a tenant
resource cluster. To simulate this hardware failure scenario, we powered off two of the ESXi
hosts.
Objectives
Verify that the virtual machines running on the failed ESXi hosts are restarted by
vSphere HA on the surviving ESXi hosts, with the proper start order.
Verify that the configured DRS affinity rules are followed when placing the virtual
machines after the failure.
Procedure
1. Migrate the PAS and DB instances of SAP system CA1 to the same ESXi host
s1vccomp147.
2. Migrate the ASCS and AAS instances of SAP system CA2 to the same ESXi host
s2vccomp47.
3. Power off ESXi host s1vccomp147 and s2vccomp47.
4. Verify that vSphere HA automatically detects that the ESXi hosts have failed and
that VSphere HA initiates a restart of the SAP virtual machines on other ESXi hosts.
5. Check Oracle database startup log in /home/ca1adm/startdb.log
6. Check the log file /usr/sap/<SID>/DVEBMGS00/work/available.log to determine the
duration of SAP service unavailability.
Results and Analysis
Table 9 shows the observed behaviors of the system and reference metrics in minutes and
seconds (mm:ss) when ESXi hosts failed.
Use case 1: Reference metrics for ESXi failure Table 9.
Virtual
machine
ESXi host
after failure
VMHA restart
duration
(mm:ss)1
Service outage
duration
(mm:ss)2
DRS rules
compliance
CA1 DB s1vccomp145 0:44 2:30 Compliant
CA1 PAS s1vccomp146 0:47 3:28 Compliant
CA2 ASCS s2vccomp45 0:47 1:37 Compliant
1 VMHA restart duration measures the time from ESXi host failure to successful trigger of SAP virtual machine
power on. 2 Service outage duration measures the time from ESXi host failure to successful restart of services on SAP
instances.
Use case 3:
Single ESXi host
failure
Chapter 4: Protecting SAP Applications with Continuous Availability
29
Virtual machine
ESXi host after failure
VMHA restart duration
(mm:ss)1
Service outage duration
(mm:ss)2
DRS rules compliance
CA2 AAS s2vccomp45 0:49 3:18 Compliant
This use case demonstrates that vSphere HA automatically reacts to the vSphere ESXi host
failure incident and restarts the affected virtual machines on the available vSphere ESXi
host. vSphere DRS, with the configured affinity and anti-affinity rules, immediately,
automatically, and non-disruptively adjusts the placement of the restarted virtual machines
in the surviving ESXi hosts in the respective cluster, restoring the previous high-availability
service level provided.
Test scenario
This test scenario validates that, in the event of an unexpected VPLEX cluster isolation, the
SAP systems continue services without interruption. To simulate this isolation scenario, we
powered off VPLEX in data center B. With VPLEX powered off, we can assume the storage
that connects to the failed VPLEX is also unavailable.
Objective
Verify that there is no service interruption for the SAP systems running on data center
A.
Procedure
1. Power off VPLEX cluster-2 and storage in data center B.
2. Check the path status on datastore.
3. Verify the VPLEX cluster-1 functionality in data center A.
4. Verify the status of SAP Load Generator and check the log file
/usr/sap/CA2/D00/work/available.log to verify that there was no service
interruption during the outage of VPLEX.
Results and analysis
When VPLEX cluster-2 in data center B are powered off at 10:51:35, the VPLEX Witness
detects failure of VPLEX cluster-2.The storage served by VPLEX cluster-1 on data center A
remains available, as shown in Figure 14.
Figure 14. VPLEX status after data center A VPLEX isolation
Due to the VPLEX Cross-Cluster Connect configuration, EMC PowerPath®/VE sets the Cross-
Cluster paths on hosts at data center B to auto standby proxy before the isolation. When the
isolation event occurs, PowerPath/VE can detect the isolation of the VPLEX at data center B
and claims the paths are unresponsive. When the cluster is recovered, PowerPath/VE
automatically recovers the dead paths. Figure 15 shows the behavior of PowerPath/VE
during the isolation event.
Use case 4: One
leg of VPLEX
cluster failure
Chapter 4: Protecting SAP Applications with Continuous Availability
30
Figure 15. PowerPath status after VPLEX isolation
Since the ESXi hosts are connected to both VPLEX clusters, PowerPath/VE simply reroutes
the I/O to the surviving path.
During one leg of VPLEX cluster failure, the SAP CA2 system remains unaffected. No SAPGUI
drop is observed. Figure 16 shows that the SAP CA2 system remains available after
10:51:35, the time we powered off the VPLEX cluster-2 in data center B.
Figure 16. No interruption of SAP CA2 after VPLEX cluster-2 failure on data center B
This use case demonstrates that:
Even after one leg of VPLEX cluster failure, the SAP workload is not interrupted during
the transparent failover of the storage services from the VPLEX in data center B to
data center A.
PowerPath/VE identifies the failed paths and redirects the Fibre Channel (FC) traffic
through the active paths, which are connected to the VPLEX storage on data center A.
This makes the VPLEX failure transparent to the SAP application running inside the
SAP virtual machines on data center B, as well as avoiding the need to restart virtual
machines on other ESXi hosts, which would cause downtime.
Test scenario
This test scenario validates that, in the event of a complete data center failure, all of the
SAP virtual machines running on that data center are restarted on the remote data center.
To simulate this hardware failure scenario, we powered off all ESXi hosts and the VPLEX in
data center B.
Objective
Verify that the virtual machines running on the failed ESXi hosts in data center B are
restarted by vSphere HA on the hosts in data center A in the proper start order.
Verify that the configured DRS affinity rules are followed when placing the virtual
machines on the remote data center after the failure.
Use case 5:
Single data
center failure
Chapter 4: Protecting SAP Applications with Continuous Availability
31
Procedure
1. Power off the ESXi hosts and the VPLEX cluster-2 in data center B.
2. Verify that the ESXi hosts failure is detected by vSphere HA. vSphere HA
automatically detects that the servers have failed and then initiates a restart of the
SAP CA2 virtual machines.
3. Verify that the SAP CA2 instances are restarted in the proper order.
4. Check Oracle database startup log in /home/ca1adm/startdb.log
5. After automatic startup of SAP CA2, check the log file
/usr/sap/CA2/DVEBMGS00/work/available.log to determine the duration of the SAP
service unavailability.
6. After VPLEX cluster-2 in data center B is online, verify that SAP CA2 virtual
machines are automatically migrated to ESXi hosts in data center A according to
site affinity rule.
Results and analysis
Table 10 shows the observed behaviors of the system and reference metrics in minutes and
seconds (mm:ss), when the ESXi hosts failed.
Use case 5: Reference metrics for single data center failure Table 10.
Virtual
machine
ESXi host
after failure
ESXi host
after VPLEX cluster-2 is restored
VMHA
Restart duration
(mm:ss)
Service
outage duration
(mm:ss)
DRS rules
Compliance
CA2 ASCS s1vccomp147 s2vccomp46 0:56 2:23 Compliant
CA2 DB s1vccomp145 s2vccomp47 0:52 4:03 Compliant
CA2 PAS s1vccomp147 s2vccomp47 0:57 5:13 Compliant
CA2 AAS s1vccomp145 s2vccomp46 1:10 4:57 Compliant
This use case demonstrates that vSphere HA automatically reacts to the vSphere ESXi host
failure incidents and restarts the virtual machines on the available vSphere ESXi host. With
VPLEX, the SAP virtual machines are restarted on the other data center. vSphere DRS, with
the configured affinity and anti-affinity rules, immediately, automatically, and non-
disruptively adjusts the placement of the restarted virtual machines in the surviving ESXi
hosts in the respective cluster, restoring the previous high-availability service level provided.
Test scenario
Planned maintenance includes hardware maintenance, hypervisor maintenance, installation
of patches, and upgrades of the server BIOS, drivers, or the hypervisor itself. All these
activities can cause downtime in your environment. This solution provides the benefit of
allowing SAP or other workloads to be non-disruptively moved across data centers when
planned downtime results from an external event, such as a power upgrade planned by the
utility company.
This test scenario validates that no SAP service disruption occurs during planned downtime
of ESXi hosts. To simulate this scenario, we put all of the ESXi servers in data center B into
maintenance mode.
Objectives
Verify that the SAP virtual machines, which run under an SAP workload on the ESXi
host set on maintenance mode, are non-disruptively migrated across data centers.
Verify that the vSphere DRS affinity and anti-affinity rules are enforced by vCenter
during the migration of the SAP virtual machines.
Use case 6:
Planned host
maintenance
Chapter 4: Protecting SAP Applications with Continuous Availability
32
Procedure
1. Log in to SAP CA2 using SAPGUI. Run SAP Load Generator (t-code sgen) to
generate workload.
2. Temporarily disable Admission Control in the vSphere HA settings to make sure
virtual machines can be migrated to ESXi hosts in data center A.
3. Select all ESXi hosts in data center B and put them into maintenance mode.
4. Verify that SAP CA2 virtual machines are migrated data center A. Verify that SAP
CA2 is not interrupted. Verify that DRS rules are followed.
5. Select all ESXi hosts in data center B and exit maintenance mode.
6. Verify that SAP CA2 virtual machines are migrated back to data center B. Verify that
SAP CA2 is not interrupted. Verify that DRS rules are followed.
Results and analysis
All SAP virtual machines are migrated to data center A in less than one minute without any
service interruption, and DRS anti-affinity rules are complied with, as shown in Figure 17.
Figure 17. Non-disruptive host maintenance
This use case demonstrates that:
A data-center-wide planned host maintenance can be performed non-disruptively and
on demand without affecting the planned downtime commitments on your current
service level agreements (SLAs).
vSphere DRS automatically enforces the affinity rules configured to keep the SAP
virtual machines’ ASCS, DB, AAS, and PAS instances in separate physical vSphere ESXi
hosts, maintaining the same protection level that exists before the hosts are put into
maintenance mode.
An administrator can proactively avoid the impact of a foreseeable outage, to preserve
the availability of the business-critical SAP applications. Business-critical SAP
applications can be moved live and within seconds to a remote data center, outside
the area to be affected by the foreseeable outage.
Chapter 4: Protecting SAP Applications with Continuous Availability
33
Test scenario
This test scenario validates that, in the event of compute resource contention on ESXi hosts,
vSphere DRS can balance host resource utilization by migrating SAP virtual machines to
other ESXi hosts. To simulate this scenario, we stressed ESXi host CPU resources by running
SAP Load Generator (t-code sgen).
Objectives
Verify that when the resource utilization of a tenant resource cluster becomes
imbalanced, vSphere DRS can trigger non-disruptive virtual machine migration to
rebalance resource utilization.
Verify that the vSphere DRS affinity and anti-affinity rules are enforced by vCenter
during the non-disruptive migration of the SAP virtual machines
Procedure
1. Migrate DB and PAS instances of SAP CA2 to one ESXi host (s2vccomp45).
2. Verify Automation Level is set to Fully Automated in vSphere DRS settings for the tenant resource cluster.
3. Log in to SAP CA2. Run SAP Load Generator (t-code sgen) to generate workload on both SAP systems.
4. Verify that the resource utilization of the tenant resource cluster becomes imbalanced.
5. Verify that vSphere DRS can trigger non-disruptive virtual machine migration to balance resource utilization.
Results and analysis
During the process of compute resource balancing, there was no service interruption of
either SAP system, and the vSphere DRS rules were followed. Table 11 shows the observed
behaviors during ESXi host resource balancing.
ESXi host resource balancing Table 11.
SAP virtual
machine
ESXi host before
migration
ESXi host after
migration
DRS rule
compliance
CA2 DB s2vccomp45 s2vccomp45 Compliant
CA2 PAS s2vccomp45 s2vccomp46 Compliant
This use case demonstrates that vSphere DRS can balance compute resource utilization
while maintaining SAP service availability.
Use case 7:
Compute
resource balance
Appendix A: References
34
This chapter presents the following topics:
Conclusion ........................................................................................................ 35
Appendix A: References
35
The Federation Enterprise Hybrid Cloud for SAP integrates the best of EMC and VMware
products and enables customers to continuously protect SAP system landscapes from events
ranging from single host failure to single data center outage. vSphere HA can restart SAP
instances within a few minutes of a hardware failure.
With the integration of EMC ViPR, vCO, and vRealize Automation, cloud administrators can
request CA-enabled storage through the vRealize Automation self-service portal. End users
can provision SAP systems on CA-enabled infrastructures with just a few clicks.
VPLEX Cross-Connect enables high availability of SAP systems in case of VPLEX and storage
failure in one of the data centers.
With the VMware metro storage cluster powered by EMC VPLEX and vSphere, Federation
Enterprise Hybrid Cloud can provide non-disruptive planned host maintenance and resource
balance across data centers.
This solution provides these benefits for protecting SAP systems:
Eliminates single points of failure across all infrastructure layers
Eliminates recovery time in case of a single VPLEX or storage outage by using VPLEX
Metro with Cross-Cluster Connect
Minimizes downtime, with less than 6 minutes RTO and zero RPO, and provides high
availability across datacenters
Enables mission-critical business continuity for SAP applications running across local
and remote data centers
Appendix A: References
36
This appendix presents the following topics:
References ........................................................................................................ 37
Appendix A: References
37
These documents are available on www.emc.com. Access to Online Support depends on
your login credentials. If you do not have access to a document, contact your Federation
representative.
Federation Enterprise Hybrid Cloud 3.1: Foundation Infrastructure Reference
Architecture Guide
Federation Enterprise Hybrid Cloud 3.1: Concepts and Architecture Solution Guide
Federation Enterprise Hybrid Cloud 3.1: Operations Solution Guide
Federation Enterprise Hybrid Cloud 3.1: Security Management Solution Guide
Federation Enterprise Hybrid Cloud 3.1: Foundation for SAP Solution Guide
EMC Cloud-Enabled Infrastructure for SAP: HA and Application Mobility Bundle
These documents are available from www.emc.com or EMC Online Support. Access to online
support depends on your login credentials. If you do not have access to a document, contact
your EMC representative.
Implementation and Planning Best Practices for EMC VPLEX Technical Notes
EMC VPLEX Metro Witness Technology and High Availability
Using VPLEX Metro with VMWare High Availability and Fault Tolerance for Ultimate
Availability
VMware KB Topic 2007545: Implementing vSphere Metro Storage Cluster (vMSC)
using EMC VPLEX
For additional information on SAP HA, refer to the following documents, available on the SAP
support portal.
High Availability Solutions for SAP on VMware
SAP Note 1380654 - SAP support in public cloud environments
SAP Note 1492000 - General Support Statement for Virtual Environments
SAP Note 1122388 - Linux: VMware vSphere configuration guidelines
SAP Note 98051 - Database Reconnect: Architecture and function
SAP Note 24806 - Database Reconnect: technical details and settings
SAP Note 1122387 - Linux: SAP Support in virtualized environments
SAP Note 1122388 - Linux: VMware vSphere configuration guidelines
SAP Note 0171356 - SAP software on Linux: Essential information
SAP Note 1310037 - SUSE Linux Enterprise Server 11: Installation notes
EMC Solution
Guides
Product
documentation
Other
documentation
Recommended