High Availability for SAP in a DB2 pureScale …...process of installing and configuring SAP for HA Easy to use wizard guides through required steps to configure SA MP policies for

High Availability for SAP in a

DB2 pureScale Environment Hinnerk Gildhoff (IBM), Andreas Schauberer (IBM),

Isabell Schwertle (IBM), Steffen Siegmund (SAP AG)

May, 2011

© 2011 SAP AG. All rights reserved. 2

SAP Legal Disclaimer

The information in this presentation is confidential and proprietary to SAP and may not be disclosed without

the permission of SAP. This presentation is not subject to your license agreement or any other service or

subscription agreement with SAP. SAP has no obligation to pursue any course of business outlined in this

document or any related presentation, or to develop or release any functionality mentioned therein. This

document, or any related presentation and SAP's strategy and possible future developments, products and

or platforms directions and functionality are all subject to change and may be changed by SAP at any time

for any reason without notice. The information on this document is not a commitment, promise or legal

obligation to deliver any material, code or functionality. This document is provided without a warranty of any

kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness

for a particular purpose, or non-infringement. This document is for informational purposes and may not be

incorporated into a contract. SAP assumes no responsibility for errors or omissions in this document, except

if such damages were caused by SAP intentionally or grossly negligent.

All forward-looking statements are subject to various risks and uncertainties that could cause actual results

to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-

looking statements, which speak only as of their dates, and they should not be relied upon in making

purchasing decisions.


IBM Legal Disclaimer

© Copyright IBM Corporation 2011. All rights reserved.

U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract

with IBM Corp.

THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES

ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION

CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EXPRESS OR

IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY,

WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM SHALL NOT BE RESPONSIBLE FOR ANY

DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER

DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE

EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR

LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY AGREEMENT OR LICENSE GOVERNING THE

USE OF IBM PRODUCTS AND/OR SOFTWARE.

IBM, the IBM logo, ibm.com, AIX, DB2, DB2 Universal Database, pSeries, POWER6, POWER7 are trademarks of

International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be

trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at

www.ibm.com/legal/copytrade.shtml.

http://www.ibm.com/legal/copytrade.shtml


2 SAP NetWeaver Architecture Overview

3 IBM Tivoli System Automation for Multiplatforms

Agenda

4 SAP NetWeaver High Availability Cluster with SA MP

5 NFS Cluster with SA MP

6 DB2 pureScale Cluster with SA MP

1 SAP NetWeaver HA Customer Value

7 HA Cluster Setup and Integration

SAP NetWeaver HA Customer Value

System Automation for Multiplatforms


SA MP SAP High Availability Customer Value

SAP NetWeaver

Is usually business critical and has high

availability needs

Is a complex technology platform with

various subcomponents

Scripting HA for SAP NetWeaver is a

manual, error-prone and time consuming

process

SA MP

SA MP provides out-of-the box high

availability policies - they can be applied

in less than one week to make SAP

highly available and provide a central,

easy operating

SAP NetWeaver on DB2 pureScale

Shared Disks

Member 1

Application Servers Application Servers

Member n

Cluster Caching

Facility

Cluster

Services High speed

interconnect

Central Services

NFS


SA MP SAP High Availability Customer Value

SA MP for SAP

High availability and automation solution for SAP

NetWeaver

Support for all major SAP landscape components – SAP

Central Services, database and NFS servers

ONE technology can be used to AUTOMATE a full SAP

landscape

SA MP SAP HA Solution includes

Comprehensive documentation that guides through the

process of installing and configuring SAP for HA

Easy to use wizard guides through required steps to

configure SA MP policies for SAP, IBM DB2, and NFS

SAP NetWeaver Architecture Overview


SAP NetWeaver Solutions Structure

SAP NetWeaver

SAP NetWeaver is the infrastructure which allows to run different SAP solutions

like ERP, CRM, and others. SA MP provides a high availability solution for SAP

NetWeaver to cover mission critical SAP solutions

A single SAP NetWeaver instance can run a single solution. (1:1)

SAP NetWeaver stack consists of: ABAP, Java or Double stack (ABAP and Java)

SAP NetWeaver

(ABAP or

Double Stack)

SAP ERP

SAP NetWeaver

(ABAP)

SAP CRM

SAP NetWeaver

(JAVA)

SAP Portal

SAP NetWeaver

(Double Stack)

SAP

Solution Manager

End Users Service Desk

Solution

Developer


SAP NetWeaver Architecture

SAP NetWeaver

Application server(s)

SAP Central Services

Standalone Enqueue Service

Enqueue Replication server

Message Server

Gateway Server

Database

Additional Components

SAP WebDispatcher

SAP Router

SAP NFS Shares


Shared Disks

Member 1


Member n

Cluster Caching

Facility

Cluster

Services High speed

interconnect

Central Services

NFS


SAP NetWeaver Failure Scenarios: Central Services

Enqueue Server (Enqueue Replication Server)

Manages locks of business layer

Outage: SAP applications cannot acquire locks

anymore and therefore the SAP system hangs

SAP infrastructure allows transparent recovery, but

failover has to be automated

Automated recovery of Enqueue Server environment is

done by SA Multiplatforms

End-user(s) can continue work after short freeze

Message Server

Internal application server communication and Load

Balancing over logon groups

Outage: No communication and dispatching between

application server possible

SAP Gateway

RFC services

Outage: No RFC communication

This can also be used for

planned scenarios


Shared Disks

Member 1


Member n

Cluster Caching

Facility

Cluster

Services High speed

interconnect

Central Services

NFS


SAP NetWeaver Failure Scenarios: Application Server

Application Server Failure

Recovery of Application Server is automated by

SA Multiplatforms or in an End-to-End

automated environment by SA Application

Manager and Agentless Adapter

End-user(s) have to re-logon, previous session

information is lost

End-user(s) will automatically be re-routed to

another Application Server (SAP Group Logon)

This can also be used for

planned scenarios


Shared Disks

Member 1


Member n

Cluster Caching

Facility

Cluster

Services High speed

interconnect

Central Services

NFS



SAP NetWeaver Failure Scenarios: Database Server & NFS

Database Server Failure

Recovery of Database can be automated by

Tivoli System Automation for Multiplatforms

SAP automatic re-connect feature connects to

a surviving database member

End-user(s) remain logged on (either freeze or

re-enter not committed work)

NFS Failure

Recovery of NFS can be automated by Tivoli


In planned scenarios this can be

done transparently to end users.

Shared Disks

Member 1


Member n

Cluster Caching

Facility

Cluster

Services High speed

interconnect

Central Services

NFS

IBM Tivoli System Automation for Multiplatforms


IBM Tivoli System Automation for Multiplatforms Overview


Tivoli System Automation for

Multiplatforms is a high availability

clustering solution with advanced

automation capabilities

Manages business application

availability by:

Detection of outage through monitoring

Knowledge about application

components and their relationships

Recovery of failed resources and whole

applications, either in place or on another

system in a Linux, AIX, Solaris or

Windows cluster

Resources / Relationships

Depends On

Runs On Depends On


Resources / Relationships

IBM Tivoli System Automation for Multiplatforms Policy based automation


Clusterwide policy with

Resource information, like start, stop,

and monitor, and preferred systems

Groups of resources enable operations

at the business level

Relationships model your configuration:

Start/stop, DependsOn, Location

Resource harvesting

No programming required

Operators control applications at the

business level

Operators do not need to remember

application components and

relationships

Node 2 Node 1

Resource Group:RG_DB2

Equivalency:

Network

Floating Resource:

Mount Point

Floating Resource:

DB2

Floating Resource:

ServiceIP

DependsOn

DependsOn

NIC

NIC

NIC

NIC

DependsO

n

Service

IP

Service

IP

DB2 DB2

Mount

Point

Mount

Point

SAP NetWeaver Cluster with SA MP

Cluster Solution for SAP Central Services and SAP Application Server


SAP NetWeaver High Availability with SA MP

SAP NetWeaver HA

SA MP provides product support for

a SAP NetWeaver policy

Automated start, stop and failover of

the SAP Central Services

Automated start, stop and restart of

the Application Servers

Best-of-breed, with long and

successful customer relationships

Ease-of-use through comprehensive

documentation and policy

configuration wizard


Shared Disks

Member 1


Member n

Cluster Caching

Facility

Cluster

Services High speed

interconnect

Central Services

NFS


SAP NetWeaver High Availability with SA MP

ERS Resource Group (SAP_ABAP_HA5_ENQ)

ENQ Resource Group (SAP_ABAP_HA5_ENQ)

Serverlist

Node (db6xen003)

Node (db6xen004)

Virtual IP (SAP_ABAP_ha5_ENQ_IP)

Adresse: db6xenvscs

Network

Equivalency

Network (db6xen003)

Network (db6xen004)

Depends On

Message Server (SAP_ABAP_HA5_ENQ_MS)

System Log Collector (SAP_ABAP_HA5_ENQ_CO)

Log Send Daemon (SAP_ABAP_HA5_ENQ_SE)

SAP Gateway (SAP_ABAP_HA5_ENQ_GW)

Enqueue Server (SAP_ABAP_HA5_ENQ_ES)

Depends On

Enqueue Replication (SAP_ABAP_HA5_ENQREP_ERS)

StartAfter

AntiCollocated

IsStartable

Collocated

(IfNotOffline)


SAP Enqueue Server and Enqueue Replication Server Lock List Replication

Enqueue Replication Server (ERS)

SAP Application Servers lock business objects via enqueue server

SAP Enqueue server keeps lock in memory and replicates the locks to the enqueue replication server

(shared memory)

SA MP: Ensure correct start-up sequence / node of ES and prereq’s

SA MP: Ensure correct start-up sequence / node of ERS

Host A

Enqueue

Replication Thread

Application Servers Application Servers Application Servers Application Servers Application Servers

Host B

Enqueue Replication Server

Locks

(Copy)

Worker Thread

Locks


SAP Enqueue Server and Enqueue Replication Server Enqueue Server Failure

Enqueue Server Failure

Primary lock list is lost

SAP application server try to re-connect to enqueue server via virtual IP address

SA MP: Recognize Node 1 failure

Host A

Enqueue

Replication Thread

Worker Thread

Locks


Host B


Locks

(Copy)


SAP Enqueue Server and Enqueue Replication Server Re-create Lock List

Enqueue Server Recovery

SA MP: recovers enqueue server on standby node and moves the virtual IP address

SAP enqueue server restores the lock list based on the copy in shared memory

SAP application servers re-connect via VIP

Host A


Host B


Locks

(Copy)

Worker Thread

Locks


SAP Enqueue Server and Enqueue Replication Server Lock List Replication

Enqueue Replication Server (ERS)

SA MP: re-establish the replication of the lock list to prevent other outages on another node

in the cluster

2 node: as soon as the failed node comes back online;

3 and more node: re-establish the replication on another online node

Host A


Host B


Worker Thread


Locks

(Copy) Replication Thread Locks


Supported SAP HA Setup Overview

Linux SLES 10 SP3 or higher

RHEL 5.5 or higher

SAP NW 7.0 SR3 (or higher)

SA MP license is free for

SAP OEM DB2 pureScale customers

SAP Solutions

• ABAP SCS Setup

• JAVA SCS Setup

• ABAP/JAVA Double Stack Setup

IBM DB2 pureScale

AIX AIX 6.1 or higher

SAP NW 7.0 SR3 (or higher)

SA MP license is free for

SAP OEM DB2 pureScale customers

SAP Solutions

• ABAP SCS Setup

• JAVA SCS Setup

• ABAP/JAVA Double Stack Setup

IBM DB2 pureScale

NFS Cluster with SA MP


NFS High Availability with SA MP

NFS HA

Shared filesystem is used by

Application Servers

SA MP has a precanned policy for

NFS servers available on OPAL

SAP NFS Share

Executables, global and profiles

shared

Outage: No restart of application

server possible, no transports


Shared Disks

Member 1


Member n

Cluster Caching

Facility

Cluster

Services High speed

interconnect

Central Services

NFS

DB2 pureScale Cluster with SA MP


SAP NetWeaver on DB2 pureScale with SA MP

DB2 Cluster Services and SA MP

SA MP is DB2‘s build in high availability

solution

Programmatic integration of Tivoli


Integrated Installation, Configuration and

Operation

SA MP is bundled in a DB2 setup

SAP and SA MP

SAP provides tools and utilities to ease

cluster setup

SAP provides document how to use SA

MP for DB2 in SAP context


Shared Disks

Member 1


Member n

Cluster Caching

Facility

Cluster

Services High speed

interconnect

Central Services

NFS


High Availability Database Cluster Options

Shared Disk HA

Active/passive solution

Requires no CPU cycles on standby

Medium-fast fail-over

Rolling OS, HW and DB2 fixpack update

DB2 High Availability Disaster Recovery (HADR)

Active/semi-active solution

Protection against disaster

Very fast failover

Rolling OS, HW and DB2 fixpack update

DB2 9.8 pureScale

Active/active: Database is distributed over several servers

Scaling across multiple servers

Stealth OS and HW maintenance


SAP on DB2 pureScale

Shared

Disks

pureScale

cluster

Application

Servers

DB2 pureScale objectives

Continuous availability

Protection against unplanned outages

Protection against planned outages (maintenance)

Transparent application scaling

SAP Applications on DB2 pureScale

Connect to any server of the DB2 pureScale cluster

Automatically reroute to another DB2 server in case of

a server failure or planned server maintenance

Transparently route back to the original DB2 server

when it becomes available again

DB2 pureScale cluster

Consists of multiple DB2 servers (data sharing

members) that process requests from connected

applications

Exploits intra-cluster high-speed communication with

RDMA (remote direct memory access) technology

Stores data in a DB2 built-in cluster file system

(GPFS)

Automatically recovers failed components


SAP on DB2 pureScale Detailed View

Shared Disks

Member 1 Member 2

Application Servers Application Servers Application Servers

Member n

Cluster Caching

Facility

Cluster

Services High speed

interconnect

Central

Services

End Users

Work with an SAP application server

Can use SAP log-on groups to direct

similar work to same set of servers

SAP application servers

Are connected to a DB2 Member,

based on central configuration file

Automatic fail-over and fail-back

HA for SAP Central Services provided

by DB2-provided cluster manager

DB2 pureScale cluster

DB2 members process requests from

connected SAP application servers

Central locking and cache coherency

through Cluster Caching Facility (CF)

Intra-cluster high-speed

communication, exploiting RDMA

technology

Data resides in DB2 built-in cluster file

system

DB2 built-in cluster manager provides

HA of DB2 pureScale components


DB2 pureScale Availability Characteristics Unplanned Outages

Data availability

100%

time

Data in-flight on failed member

Member restart

Failure of a DB2 Member

Computing power of failed member becomes

unavailable

Connections from SAP application servers

automatically fail over to other members

Only data that was in-flight data at the failed member is

locked until member restart completes.

All other data and all other members remain fully

available

Automatic member restart

Attempt restart on member's home host

If restart is not possible on the member's home

host, DB2 will automatically perform the restart on

another host (light restart)

DB2 will bring the member up at its home host as soon

as it becomes available again

Connections from SAP application servers fail back to

the member at transaction boundaries, invisible to the

end users

Shared Disks

Member 1 Member 2


Member n

Cluster Caching

Facility

Cluster

Services

High speed

interconnect

Central

Services


DB2 pureScale Availability Characteristics Unplanned Outages

Data availability

100%

time

Failure of primary Cluster Caching Facility

Secondary CF takes over

Short cluster freeze during lock rebuild

100% of data available after lock rebuild

No loss of in-flight transactions

Failed CF will automatically be brought up as

secondary once its home host becomes

available again

Failure of sec. Cluster Caching Facility

Likely not perceived (very short hiccup)

No loss of in-flight transactions

Failed CF will automatically be brought up as

secondary once its home host becomes

available again

lock

rebuild

Shared Disks

Member 1 Member 2


Member n

Cluster Caching

Facility

Cluster

Services

High speed

interconnect

Central

Services


DB2 pureScale Availability Characteristics Planned Outages

Maintenance of a DB2 Member

Transactions are transparently drained out from

the member

Connections from SAP application servers fail over

to other members at transaction boundaries,

invisible to the end users

Once DB2 member maintenance is completed, the

member is brought up again

Connections from SAP application servers fail

back to the member at transaction boundaries,

invisible to the end users

Maintenance of Cluster Caching Facility

For maintenance of current primary:

Initiate role switch (secondary will take over)

CF is taken out of the cluster and maintenance

conducted

CF is integrated into the cluster again

100%

time Data availability

Data remains fully available

Shared Disks

Member 1 Member 2


Member n

Cluster Caching

Facility

Cluster

Services

High speed

interconnect

Central

Services

HA Cluster Setup and Integration


HA Setup for SAP Central Services and DB2 pureScale

SAP Installation Tool

Migrate from ESE to pureScale (GPFS)

Create db2dsdriver.cfg (Round Robin)

Topology Management

Add/drop Member/CF

Add/drop Application Sever

Update db2dsdriver.cfg

SAP Cluster Setup Tool *

Enqueue server (ES) and ERS

Message server

SAP gateway

SAP Web dispatcher

SAP router

SA MP

Usage of SA MP provided by DB2 pureScale for

HA of SAP Central Services is covered by

your pureScale license *

*) Applies AIX and Linux only. First released with SAP OEM DB2 pureScale.

Shared Disks

Member 1 Member 2


Member n

Cluster Caching

Facility

Cluster

Services

High speed

interconnect

Central

Services


Peer Domain B Peer Domain A

SAP/DB2 pureScale Recommended Setup Separate DB2 pureScale and SAP into different cluster domains

Host D (db6lpar17)

Host A (db6lpar14)

Host C (db6lpar16)

Host B (db6lpar15)

Storage A

MEMBER

Host F (db6lpar19)

Host E (db6lpar18)

Storage B

Prim. AS

Storage C

db6lparvwd

db6lparvsr Router Router

db6lparvscs

db6lparvascs

Add. AS

Prim. CF MEMBER Sec. CF

WebDisp

SCS

ERS

SCS

ERS

WebDisp

ASCS

AERS

ASCS

AERS

Thank You! Contact information:

Hinnerk Gildhoff

Software Engineer, IBM

[email protected]

Isabell Schwertle


[email protected]

Andreas Schauberer


[email protected]

Steffen Siegmund

Senior Developer, SAP AG

[email protected]


No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice.

Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.

Microsoft, Windows, Excel, Outlook, and PowerPoint are registered trademarks of Microsoft Corporation.

IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, System x, System z, System z10, System z9, z10, z9, iSeries, pSeries, xSeries, zSeries, eServer, z/VM, z/OS, i5/OS, S/390, OS/390, OS/400, AS/400, S/390 Parallel Enterprise Server, PowerVM, Power Architecture, POWER6+, POWER6, POWER5+, POWER5, POWER, OpenPower, PowerPC, BatchPipes, BladeCenter, System Storage, GPFS, HACMP, RETAIN, DB2 Connect, RACF, Redbooks, OS/2, Parallel Sysplex, MVS/ESA, AIX, Intelligent Miner, WebSphere, Netfinity, Tivoli and Informix are trademarks or registered trademarks of IBM Corporation.

Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.

Adobe, the Adobe logo, Acrobat, PostScript, and Reader are either trademarks or registered trademarks of Adobe Systems Incorporated in the United States and/or other countries.

Oracle is a registered trademark of Oracle Corporation.

UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.

Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of Citrix Systems, Inc.

HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts Institute of Technology.

Java is a registered trademark of Sun Microsystems, Inc.

JavaScript is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape.

SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP BusinessObjects Explorer, StreamWork, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries.

© 2011 SAP AG. All rights reserved

Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business Objects products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Business Objects Software Ltd. Business Objects is an SAP company.

Sybase and Adaptive Server, iAnywhere, Sybase 365, SQL Anywhere, and other Sybase products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Sybase, Inc. Sybase is an SAP company.

All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary.

The information in this document is proprietary to SAP. No part of this document may be reproduced, copied, or transmitted in any form or for any purpose without the express prior written permission of SAP AG.

This document is a preliminary version and not subject to your license agreement or any other agreement with SAP. This document contains only intended strategies, developments, and functionalities of the SAP® product and is not intended to be binding upon SAP to any particular course of business, product strategy, and/or development. Please note that this document is subject to change and may be changed by SAP at any time without notice.

SAP assumes no responsibility for errors or omissions in this document. SAP does not warrant the accuracy or completeness of the information, text, graphics, links, or other items contained within this material. This document is provided without a warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability, fitness for a particular purpose, or non-infringement.

SAP shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. This limitation shall not apply in cases of intent or gross negligence.

The statutory liability for personal injury and defective products is not affected. SAP has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of third-party Web pages nor provide any warranty whatsoever relating to third-party Web pages.

Documents

High Availability for SAP in a DB2 pureScale …...process of installing and configuring SAP for HA Easy to use wizard guides through required steps to configure SA MP policies for