Upload
lythuy
View
236
Download
2
Embed Size (px)
Citation preview
COPY AUTOMATION TOOL FOR SAP
IMPLEMENTATION GUIDE
Version 2.3
May, 2018
Pure Storage Proprietary Information 2 | 33
Copy Automation Tool for SAP Implementation Guide
Pure Storage Proprietary Information 3 | 33
INTRODUCTION
ABSTRACT
This document describes installation and configuration of the SAP CAT software for SAP systems on:
• UNIX (including Linux) servers, using either Oracle or SAP HANA as the DBMS.
• Windows servers using MSSQL Server as the DBMS.
LICENSE AGREEMENT
Use of SAP CAT is subject to the Pure Storage EULA for Plugin / Adaptor /
Provider / SDK / Management Pack license agreement.
The Plugin EULA can be found on the Product End User Page:
http://www.purestorage.com/legal/productenduserinfo.html
A text version of the EULA is also included in the software archive.
REPORT HISTORY
Version Date Author Description
1.0 01-Sep-2017 Expertum Initial version
1.1 13-Oct-2017 Expertum Updates to:
• prerequisites
• configuration file storage_<SID>
• conductor and client script: ssh configuration changes
1.2 24-Oct-2017 Expertum • Added Appendix: SSH setup
• Added Oracle prerequisite
1.3 25-Oct-2017 Expertum • Added sample config files
• Added License Agreement
1.5 17-Jan-2018 Expertum • Major architecture change: Purity//Run is used as a central machine.
• Initial Windows version
2.0 22-Mar-2018 Expertum • Initial SAP HANA version
• SAPCAT on Docker
2.1 20-Apr-2018 Expertum • Replaced the NFS share with Samba/CIFS
• Rebranding to CAT for SAP
2.2 02-May-2018 Expertum • Design update
Copy Automation Tool for SAP Implementation Guide
Pure Storage Proprietary Information 4 | 33
• Grammar review
2.3 03-May-2018 Expertum • Updated HANA Setup
• Updated sequence file format
• Added systemcopy parameter
Copy Automation Tool for SAP Implementation Guide
Pure Storage Proprietary Information 5 | 33
TABLE OF CONTENTS
1 Quick Start .................................................................................................................... 6
1.1 Installation and configuration ............................................................................ 6
1.2 Key terms ........................................................................................................ 6
2 Prerequisites ................................................................................................................. 7
2.1 Software versions ............................................................................................. 7
2.2 Languages ....................................................................................................... 8
2.3 Oracle ............................................................................................................. 8
2.4 FlashArray volumes .......................................................................................... 9
3 Infrastructure ................................................................................................................ 9
3.1 Hosts ............................................................................................................... 9
3.2 SAPCAT container ............................................................................................. 9
3.3 Software directory ........................................................................................... 10
3.4 Shared area .................................................................................................... 10
3.5 Security and access ......................................................................................... 11
4 Setting up SAP CAT..................................................................................................... 12
4.1 Software installation ........................................................................................ 12
4.2 Configuration .................................................................................................. 15
5 Running CAT for SAP .................................................................................................. 29
5.1 Conductor and client script ............................................................................... 29
5.2 Calling systemcopy .......................................................................................... 31
6 Appendix .................................................................................................................... 32
6.1 Password-less SSH setup.................................................................................. 32
6.2 Errors ............................................................................................................. 33
1 QUICK START
1.1 INSTALLATION AND CONFIGURATION
This section contains brief instructions for setting up and running the first system copy.
1. Check that the SAP systems, the database and the operating systems meet the prerequisites specified in section 2.1.
2. For Linux, ensure that the Korn shell (ksh) is installed. For Windows, ensure that PowerShell 3.0 and Microsoft .NET Framework 4.5 are installed. ( 2.1.3)
3. Determine the directories (shared Docker volumes) to be used by the system copy process (3.4)
4. Configure password-free SSH for Unix systems to enable fully automated system copies (
3.5)
5. For each Windows system, run the SAP Host Agent setup.ps1 script. For UNIX systems,
configure the mount in /etc/fstab.( 4.1)
6. Create a sequence file for each copy path using the sequence file delivered with the software as a template. There is usually no need to change the context list, but system IDs, host names, user names, and possibly the path to the SAPCOPY script must be adapted. Section 4.2.2 describes the sequence file.
7. Create a sapcopy.paths file (one file for all copy paths) or use the file delivered with the
software as a template. Section 4.2.6. describes the paths file.
8. Review the sapcopy.properties file and adapt as necessary. Section 4.2.7 describes the
properties file and the properties it contains. Properties that typically must be changed are dirshare and listener_name.
9. Create the clients.conf file (one file for all copy paths) or use the file delivered with the
software as a template. See 4.2.3 for more information.
10. Create storage_<SID> files containing FlashArray configuration information for all source
and target systems involved in copy processes. Section 4.2.8 describes storage configuration files.
1.2 KEY TERMS
The following terms are used throughout this document and by the software:
• A copy path is a system copy identified by the source and target systems. For example, a SAP ECC production system regularly copied to both an ECC test system and an ECC training system and a CRM production system regularly copied to a CRM test system define three copy paths:
o ECC production to ECC test
o ECC production to ECC training
o CRM production to CRM test
• A context is a major phase of a system copy. An end-to-end copy is a series of contexts. Examples of contexts are: pre-copy preparation of target, backup of source system, restore/recovery of target database, post-processing in target system.
• An action is a step in a context. Examples of actions are: exporting tables from the database, stopping SAP, stopping Oracle, etc. Each context is made up of a series of actions.
2 PREREQUISITES
2.1 SOFTWARE VERSIONS
The SAP installation and underlying database and server platform must meet the following prerequisites.
2.1.1 SAP
SAP system type ABAP
SAP NetWeaver version 7.0 and higher (SAP_BASIS >= 700)
2.1.2 DATABASE
Oracle DBMS 11.2.0.3 and higher
Microsoft SQL Server 2012 and higher
HANA 2.0 (Multi-tenant: only one tenant and system db)
The same schema name must be used in the source and target databases.
2.1.3 OPERATING SYSTEM
Linux SAP CAT supports all Linux distributions and versions supported by SAP according to the Product Availability Matrix (PAM)
Solaris
Solaris 10 and higher
AIX
(TBD)
Utilities Unix utilities:
• id
• awk
• sudo
• sed
Windows Windows Server 2008 R2 and higher
Utilities Required:
• PowerShell 3.0 or higher
• Microsoft .NET Framework 4.5 or higher
• SAP Host Agent
2.2 LANGUAGES
Version 1.5 and newer versions of CAT support both Unix/Linux and Windows. CAT for SAP is written in UNIX Korn shell (ksh) and PowerShell scripts. The Windows version also includes a batch script to
allow communication via the SAP Host Agent.
NOTE
The Korn shell is not always installed on Unix systems by default. If it is not present, use the Linux distribution’s installation tool to install the ksh package.
There is no compiled (C) code, Java code or code developed in any other scripting language.
2.3 ORACLE
The SAP CAT script automatically copies the Oracle database data files from the source to the target system. To recover the target Oracle database, the script requires information collected on the source system, including the collected Oracle archive files.
The names of these files are generated based on 2 Oracle parameters, Pure Storage highly recommends setting these as recommended in SAP note 1431798:
LOG_ARCHIVE_DEST_1 'LOCATION=<SAPDATA_HOME>/oraarch/<sid>arch'
LOG_ARCHIVE_FORMAT %t_%s_%r.dbf
This for both source and target systems.
For example, for SID ABC:
alter system set log_archive_dest_1 = 'LOCATION=/oracle/ABC/oraarch/ABCarch
SCOPE=BOTH;
alter system set LOG_ARCHIVE_FORMAT = '%t_%s_%r.dbf' SCOPE=spfile;
If different naming conventions are used, CAT for SAP may not be able to identify the needed files.
2.4 FLASHARRAY VOLUMES
During a system copy SAP CAT overwrites the FlashArray target volume(s) with snapshots of the source volume(s) contents. This implies the following prerequisites for FlashArray volumes:
• Each copied volume should contain only database data files. Because copied volumes are overwritten on the target, all other files, including software installations (database, SAP, etc.) should reside on volumes that are not part of the copy.
• All source volumes containing database data files should be in the same FlashArray protection group. The protection group must be defined in the storage file for the source system (see 4.2.8).
• Each source volume must be mapped to one target volume. Source and target volumes should have a 1-to-1 relationship (see 4.2.8).
Remark, it is possible to copy a source system that has more FlashArray volumes than the target system. To do this, create empty missing target volumes and map them to the corresponding source volumes in the target system’s storage configuration file.
3 INFRASTRUCTURE
3.1 HOSTS
For a given copy path, i.e. a system copy from a specific source system to a specific target system, five systems are involved:
1. The SAPCAT Docker host.
2. The Pure Storage FlashArray
3. The database server of the source system
4. The database server of the target system
5. The primary application server of the target system
If the database and central instance of the target system run on the same server, only three hosts would be involved. The copy scenario delivered with the SAP CAT software assumes that these hosts are separate, but the software can also be used in a single-host target environment.
NOTE
The copy process does not access the primary application server of the source system.
3.2 SAPCAT CONTAINER
CAT for SAP version 2.0 is delivered as a Docker image that can be used to start a container on the Purity//Run machine. It makes it possible to run CAT from a central location with minor setup steps.
3.3 SOFTWARE DIRECTORY
CAT for SAP scripts and configuration files occupy less than 2 MB. A copy operation creates about 2 MB of logs, control files, and temporary files. Pure Storage recommends removing these files periodically.
The CAT scripts are delivered in a Docker container, but need to be ran on each machine.
This facilitates software setup and management by maintaining a single central copy. Ensure that the Docker volumes containing the configuration files are mapped locally on the Docker host to keep them persistent (see 4.1).
3.4 SHARED AREA
A copy from a given source to a given target requires a directory shared between the source and target servers. This is achieved by the SAPCAT Docker container, it offers a shared file system. The CAT processes on all servers involved in the copy process read and write files in this directory.
By default, this shared area is provided by the SAPCAT Docker image as a Samba share.
CAT for SAP uses the shared area mainly for two types of files:
1. Non-persistent files shared by source and target during a copy
2. Backups of source database archive logs needed for database recovery on the target.
Non-persistent files in the shared area are small, typically 1-2 MB per copy. However, Oracle archive log backups may consume a substantial amount of space, which is difficult to predict. CAT for SAP compresses these backups (GZIP)to reduce their size, typically by 70-80%. Pure Storage nevertheless recommends 2-4 GB of free space in the shared area for this purpose.
Figure 11 illustrates the software directory and shared area and the servers requiring access to it. Note that the FlashArray accesses neither; all operations on the array are initiated from the SAPCAT Docker container.
Figure 1: Software directory and shared area
3.5 SECURITY AND ACCESS
The system copy process is controlled by a central "conductor" script, which starts actions on the different servers. The conductor script runs in the SAPCAT Docker container and connects to Unix servers via Secure Shell (SSH) or to Windows systems via the SAP Host Agent with HTTP(S).
A fully automated (-auto or -fullauto option) system copy in a Windows environment requests all
passwords when the conductor script starts. The script encrypts the passwords with a temporary key and stores them in a temporary file. The temporary password file cannot be used for the next refresh.
Fully automatic system copies in UNIX environments require password-free SSH logins from the user running the conductor script to the server-user combinations listed below:
Server User(s)
Source database server User authorized to connect to the database. For Oracle, as SYSDBA (ora<sid> or <sid>adm).
FlashArray Administrator authorized to execute snapshot commands
Target database server User authorized to connect to the database.
For Oracle, as SYSDBA (ora<sid> or <sid>adm).
Target application server <sid>adm
4 SETTING UP SAP CAT
4.1 SOFTWARE INSTALLATION
The application is delivered as a Docker image in a compressed archive (tar extension). After starting the container, the CAT for SAP scripts ares offered via a Samba share. Three important folders are made available as a volume to make them persistent: etc, log and share.
On Unix/Linux machines it is necessary to mount the share SAPCAT on the local mountpoint /mnt/sapcat. The mountpoint used must be the same on each server.
On Windows hosts, run the setup.ps1 script located in the Samba share to install the necessary
scripts onto the SAP Host Agent.
The Docker image provides the Samba services. Therefore, the following ports must be available for mapping on the Docker host:
• 137, 138, 139, and 445 (Samba)
Use the following command to import the image on the Docker host:
docker load --input sapcat2.0.tar
Create a Docker container from the image:
mountpoint="<SAPCAT persistent location>"
docker run -itd --privileged --name=SAPCAT \
--volume=$mountpoint/etc:/mnt/sapcat/etc \
--volume=$mountpoint/log:/mnt/sapcat/log \
--volume=$mountpoint/share:/mnt/sapcat/share \
-p 139:139 -p 445:445 \
-p 137:137/udp -p 138:138/udp purestorage/sapcat
The value of the mountpoint parameter is the location at which to store persistent files on the
Docker host. For example, /home/pureuser/SAPCAT.
4.1.1 STARTING SAPCAT
To start a system copy, log in to the Docker image and start the conductor script at /mnt/sapcat/bin as follows:
docker exec -it SAPCAT bash
Switch to our pureuser user:
su - pureuser
Next you can start the conductor script at /mnt/sapcat/bin:
# cd /mnt/sapcat/bin/
# ./systemcopy ../etc/<sequence file>
4.1.2 UNIX SETUP
Add the following line to the /etc/fstab of each Unix machine participating in the copy to
automatically mount SAPCAT when the system starts:
//<docker hostname>/sapcatunx /mnt/sapcat cifs
username=pureuser,password=<password>,uid=<sidadmuid>,gid=<sapsysgid>,
dir_mode=0777,file_mode=0774,forceuid,forcegid 0 0
The Samba/CIFS share comes with the default credentials:
- User: pureuser
- Password: sapcat
This will mount automatically SAPCAT when the system starts.
After creating the definition in /etc/fstab, login as root and mount the path manually:
mount /mnt/sapcat
4.1.3 WINDOWS SETUP
On Windows systems, install the scripts on the SAP Host Agent to map the Samba share automatically when a CAT instance is started.
With Windows Explorer, navigate to \\<Docker server hostname>\sapcat.
Log in with the default credentials:
• User: pureuser
• Password: sapcat
Remark: Log in to the Docker container and use normal Unix tools to change the pureuser
password.
Start PowerShell and launch the setup script:
cd \\<Docker server hostname>\sapcat
> .\setup.ps1
4.1.4 HANA SETUP
CAT for SAP HANA uses following hdbuserstore keys:
• DEFAULT: connection to the tenant database
• SAPCATSYSTEM: connection to the system database
• CATSYSTENANT: optional connection to the tenant database as an administrator
4.1.4.1 Tenant connection
The DEFAULT key is used on the target database to export/import tables with the native SAP HANA
database tools. To do this, the user linked to the DEFAULT key must have the following permissions:
Database System Privilege
Tenant IMPORT
Tenant EXPORT
Use the following command to list the user that corresponds to the DEFAULT key:
hdbuserstore LIST DEFAULT
4.1.4.2 Optional tenant admin connection
After a system copy, the user of the DEFAULT connection key will have the same permissions as in
the source system. Most likely the source is a production environment and granting the import rights pre-refresh isn’t wanted. Without any permissions you’ll run into problems when importing any native database exports on the target system.
A few optional scripts are defined to allow the complete automation of importing data during the copy process. To use these, you’ll need to configure a hdbuserstore key with a user that has the
permissions to grant EXPORT and IMPORT permissions and enable the scripts in the flow file.
4.1.4.2.1 Connection key
By default, CAT will look for the connection key CATSYSTENANT. This can be changed in the
sapcopy.properties file (see 4.2.7):
default/hdbtenantsys = CATSYSTENANT
Use the following command to create the CATSYSTENANT hdbuserstore key:
hdbuserstore -I SET CATSYSTENANT “hostname:port” “username”
4.1.4.2.2 Scripts
Uncomment the optional scripts in the flow file (see 4.2.5):
[target_prepare]
…
action="hana_grant_import" # Optional: automation of permissions, see
implementation guide
…
[target_sap_postprocess]
…
action="hana_grant_export" # Optional: automation of permissions, see
implementation guide
…
4.1.4.3 System database connection
A hdbuserstore key is required on both source and target database servers to connect to the SAP
HANA system database. On the source system, it is used to change the database backup mode before and after the storage snapshot. The default key SAPCATSYSTEM can be changed for each system
with the CAT property hdbsystemdb in the sapcopy.properties file (see 4.2.7):
default/hdbsystemdb = SAPCATSYSTEM
Use the following command to create the SAPCATSYSTEM hdbuserstore key:
hdbuserstore -I SET SAPCATSYSTEM “hostname:port” “username”
The -i option causes the tool to request a password for the specified username.
Pure Storage strongly recommends creating a user with the following permissions specifically for the SAPCATSYSTEM key:
Database System Privilege
SYSTEMDB BACKUP ADMIN
4.2 CONFIGURATION
This section describes the CAT for SAP configuration files.
All configuration files reside in the Docker volume etc. The etc directory contains examples with the
file extension .sample. These are overwritten each time the Docker container starts.
Copy the examples and edit them to adjust the configuration, e.g.:
cp sapcopy.flow.sample sapcopy.flow
vi sapcopy.flow
4.2.1 OVERVIEW OF CONFIGURATION FILES
Table 1 lists the configuration files used by the current version of CAT for SAP. The first column contains the names of the files in the etc directory. The second column describes each file briefly.
The third column specifies whether adapting the file delivered with the software is mandatory or optional. For example, files that refer to system names (SIDs) must be adapted to the specific configuration. In contrast, other files, such as the list of tables to be truncated in the restored target database, can be used as is in most cases, although they can be edited to meet local needs if necessary.
Some file names contain the source and/or target SID. In the following table, <SSID> and <TSID>
denote the SIDs of the source and target systems respectively.
Filename Description Change Req’d
<SSID>_<TSID>.seq Sequence file Mandatory
clients.conf Client definitions to export/reimport Mandatory
exportsets.conf Tables to export and reimport Optional
sapcopy.flow Flow file (contexts and actions) Optional
sapcopy.paths Authorized copy paths Mandatory
sapcopy.properties System copy properties Mandatory
storage_<SSID> FlashArray configuration data for source system
Mandatory
storage_<TSID> FlashArray configuration data for target system
Mandatory
truncate.conf Tables to be truncated Optional
Table 1: SAP CAT configuration files
4.2.2 SEQUENCE FILE
4.2.2.1 Function
There is a sequence file for each copy path (specific source system to specific target system). Sequence file names are arbitrary (they are referred to explicitly in copy commands), but the recommended location and name is:
<INSTDIR>/etc/<SSID>_<TSID>.seq
For example, with two copy paths, EEPEEQ and CRPCRQ, in installation directory /mnt/sapcat,
the recommended sequence files names are:
/mnt/sapcat/etc/EEP_EEQ.seq
/mnt/sapcat/etc/CRP_CRQ.seq
4.2.2.2 Format Example
The copy path in this example is EEPEEQ. The EEP DB server host name is eepdbhost, the EEQ
DB server host name is eeqdbhost and the EEQ SAP Central Instance server host name is
eeqapphost.
The context names are defined in the flow file (see sapcopy.flow in subsection 4.2.5).
# Sequence file for system copy process using SAP CAT
# 'sourcesid' and 'targetsid' define the system ID of the source and target
sourcesid EEP
targetsid EEQ
# 'sapcopypath' is the full path to the SAPCOPY script
# This must be the same on all servers involved!
sapcopypath /mnt/sapcat/bin/SAPCOPY
# The 'context' entries define the sequence of the copy process for this
source/target combination
# Each entry has the form:
# context connectiontype host user ctx
# connection type: The type of connection used to launch the SAPCAT script
# Currently supported connection protocols:
# - ssh
# - SOAP (Windows servers via the SAPHostAgent)
# host : The host where the conext must run.
# The format depends on the connection type:
# - ssh = hostname
# - soap = http(s)://hostname:port
# - purerun = connection to the Pure Storage FlashArray
# user : user ID for the run
# ctx : name of the context, as defined in the sapcopy.flow file
context ssh eeqdbhost eeqadm target_prepare
context ssh eepdbhost eepadm sourcedb_part1
context purerun eeqdbhost eeqadm dbsnapshot
context ssh eepdbhost eepadm sourcedb_part2
context ssh eeqapphost eeqadm target_stopsap
context ssh eeqdbhost eeqadm target_dbrecover
context purerun eeqdbhost eeqadm target_db_postprocess
context ssh eeqapphost eeqadm target_sap_postprocess
context ssh eeqapphost eeqadm target_final
4.2.3 CONFIGURATION FILE (CLIENTS.CONF)
4.2.3.1 Function
This file controls which client definitions (as configured in SCC4) must be exported from the target before the refresh and re-imported afterwards. This action replaces manual configuration in SCC4 after the refresh.
4.2.3.2 Format Example
The following example specifies that the following client definition records should be preserved:
• EEQ: clients 800 and 810
• CRQ: clients 800, 810 and 850
# Configuration file: clients.conf
# This file controls which client definitions (as configured in SCC4)
# must be exported from the target before the refresh and
# re-imported afterwards.
# This action replaces manual configuration in SCC4 after the refresh.
#
# Format of an entry:
# sid:cli
#
# Example:
# QAS:040
# means that SCC4 definition of client 040 in QAS must be
# preserved across refreshes
EEQ:800
EEQ:810
CRQ:800
CRQ:810
CRQ:850
4.2.4 CONFIGURATION FILE (EXPORTSETS.CONF)
4.2.4.1 Function
This file specifies the sets of tables to be exported from the target before the refresh and re-imported into the target afterwards.
During target system preparation, the specified export sets, which contain the tables whose data is to be preserved, are exported. During target post-processing, these sets are re-imported into the refreshed system. Of the specified schemas, only those that actually exist in the target system are re-imported. For example, in ABAP-only systems, export sets that refer to the Java schema are ignored.
4.2.4.2 Format Rules
Each set of tables is a separate section in the file, headed by an exportset entry; parameters for
the section are surrounded by curly brackets ({ }).
Name and arguments Meaning
exportset SETNAME Header for export set SETNAME. The name is arbitrary but must be unique.
method {exp | R3trans} Tool to be used for export and import, either Oracle exp/imp or R3trans. R3trans can only be used for tables in the ABAP schema.
critical {yes | no} If yes, failure of the export terminates the copy process. If no, failure of the export is non-fatal and the copy continues. The default is yes.
schema {abap | java | <name>} Owner of the tables in the list.
The keywords "abap" and "java" represent the SAP ABAP and SAP Java schemas; actual schema names are determined at runtime.
To request export/import of a non-SAP schema, specify its name.
sid *
sid SID1,SID2,…
Specifies the systems on which the export set must be handled. If omitted or set to *, the
export is performed on all target systems. To restrict an export to certain systems only, specify a comma-separated list of system names.
tables TAB1,TAB2,TAB3,…
tables *
Comma-separated list of tables in the export set, or * to export all tables in the schema (because of the number of tables in the ABAP schema, using '*' for the ABAP schema is
inadvisable).
4.2.4.3 Format Example
The example below is a excerpt from the export sets file delivered with the software:
exportset SAPLICENSE
{
critical yes
method exp
schema abap
tables SAPLIKEY
}
exportset PROFILES
{
critical yes
method R3trans
schema abap
tables TPFET,TPFHT
}
exportset SCOT
{
critical yes
method R3trans
schema abap
tables
BCSD_RQST,BCSD_STML,SOPR,SXADMINTAB,SXCOS,SXNODES,SXPARAMS,SXROUTE,SXSERV,S
XTELMOOUT
}
exportset STRUST
{
critical yes
method R3trans
schema abap
tables
RSECTAB,SSF_PSE_D,SSF_PSE_H,SSFAPPLIC,SSFAPPLICT,SSFARGS,SSFVKEYDEF,STRUSTS
SL,STRUSTSSLS,STRUSTSSLST,STRUSTSSLT,STRUSTWSSE,STRUSTWSSET
}
WARNING
The license key file (SAPLIKEY) requires export method "exp" (Oracle
export), not "R3trans", because this table is not known in the SAP dictionary.
4.2.4.4 Customization
For example, to preserve customer tables ZDATA1, ZDATA2, and ZDATA3, add the following export
set to the exportsets.conf file:
exportset OURTABLES
{
critical yes
method R3trans
schema abap
tables ZDATA1,ZDATA2,ZDATA3
}
4.2.5 CONFIGURATION FILE (SAPCOPY.FLOW)
4.2.5.1 Function
The flow file defines what CAT for SAP does and where it does it. Specifically, the flow file defines contexts and the actions to be performed in each.
A context is a phase in the system copy process. For example, there is almost always a pre-target context in which data that must be preserved is exported from the target system. Any number of contexts can be defined, provided that together they define an end-to-end system copy.
An action is an activity within a context. Each context therefore consists of a series of activities in a specified sequence. Physically, an action is a shell script.
NOTE
Several flow files are delivered with the CAT software that define a series of contexts that perform an end-to-end copy.
4.2.5.2 Format Rules and Examples
A flow file is divided into sections, each of which corresponds to a context. The first line of each section is the name of the context enclosed in square brackets.
Within a section there are three types of entries:
• A system entry, specifying whether the context operates on the source or target system.
• One or more action entries that specify the names of shell scripts that comprise the
context, optionally followed by command line arguments.
• (Optionally) one or more host entries that specify host(s) on which the context must run.
The example below illustrates a typical flow file section called [copytest]:
[copytest]
system=source
action=always:"first_action"
action="second_action -open"
action="third_action" recovery="fix_error"
action="last_action"
host=EEP:eppdbhost
host=CRP:crpdbhost
The context defined in this section runs on the source and consists of four actions:
• The first_action script is preceded by the qualifier always:, indicating that it is a
mandatory action. Mandatory actions are always executed; they cannot be skipped. Such actions typically collect information needed by the following actions in the context; skipping them would not make sense because the later actions then would be unable to do their work.
• The second_action script receives the command line argument -open.
• The action script third_action has an associated recovery script, described in
subsection 4.2.5.3 below. If third_action fails with a fatal error (i.e., with exit code ≥
64), then CAT invokes the script fix_error to undo the effect of the failed action.
• The last_action.script illustrates the simplest form of action. The action can be skipped
(there is no always: qualifier), it takes no parameters and there is no recovery action.
The section also contains two host entries. These indicate that for source system EEP, the
copytest context must run on the server eppdbhost, and for source system CRP, it must run on
server crpdbhost. If a section contains no host entries, CAT for SAP does not verify that it is
running on the correct host.
4.2.5.3 Recovery actions
If an action script encounters a fatal error, it returns an error code to the SAPCOPY controlling script,
which displays information about the error and stops. In some situations, however, simply stopping the partially complete process is undesirable. For example, if one action shuts down the Oracle database, and a subsequent action fails, it is very likely to be desirable to restart Oracle before SAPCOPY terminates. An action with a recovery script that starts Oracle accomplishes that.
4.2.6 CONFIGURATION FILE (SAPCOPY.PATHS)
4.2.6.1 Function and format
This file specifies authorized copy paths and optional embargo entries. Authorized copy paths protect against accidentally copying to the wrong system. For example, if EEP and EEQ are both SAP ECC systems and CRP and CRQ are SAP CRM systems, then it makes sense to copy EEP to EEQ and CRP to CRQ. However, copying EEP to CRQ or CRP to EEQ would be very unlikely, and EEQ should not be copied to EEP, nor CRQ to CRP.
To ensure that only acceptable paths are used, add these paths to the sapcopy.paths file. The
format for specifying authorized copy paths is source and target SIDs separated by a colon as shown in the examples below:
<source_sid>:<target_sid>
*:<target_sid>
The wildcard in the second example indicates that target_sid may be refreshed from any source.
This may not be appropriate, because, for example, it would allow copying EEP to CRQ.
4.2.6.2 Embargo entries
To prevent normally allowed copies to a specific system, for example:
• A system that has been reserved for training sessions over a 6-week period
• A test system reserved for several months for a critical development project. The test system cannot be refreshed during this period without destroying all development, customization, and data created by the project.
These can also be accomplished by deleting or commenting the copy path, but this is less reliable. The copy path would have to be removed at the start of the critical period and reinstated at its end. A more reliable solution is an embargo entry in the sapcopy.paths file. An embargo entry defines a
period during which copies to a system specified by its SID are not permitted. The form of an embargo entry is:
deny <SID> <fromdate> <todate>
The "from" date and "to" dates are specified as YYYYMMDD.
For example, if copies from production system EEP to test system EEQ are normally allowed, but must be suppressed during the fourth quarter of 2017, for example because EEQ is to be used for a new project, the sapcopy.paths file would contain the following entries:
# Normally allow copies EEP -> EEQ
EEP:EEQ
# Embargo: no EEQ refreshes in 4th quarter of 2017:
deny EEQ 20171001 20171231
4.2.7 CONFIGURATION FILE (SAPCOPY.PROPERTIES)
4.2.7.1 Function and format
This file contains the properties (parameters) to be used for the system copy.
Properties (parameters) take the form of a key-value pair:
property_name property_value
The property name (the key) and its value are separated by one or more spaces.
4.2.7.2 Default and system-specific properties
The name of a property has the form:
prefix/property
Most properties have default values which can be overridden. If the sapcopy.properties file does
not contain a value for a given SAP system, the default value is used. Default properties have the prefix default, whereas the prefix of a system-specific property is the system ID (SID).
In the example below, the default directory for sharing data between source and target system (property name: dirshare) is /usr/sap/trans/tmp, but systems CRP and CRQ use a different
share at /sapshare/data. All other systems use the default share.
default/dirshare /usr/sap/trans/tmp
CRP/dirshare /sapshare/data
CRQ/dirshare /sapshare/data
4.2.7.3 List of properties
The table below does not contain an exhaustive list of properties. As with action scripts these may differ between environments. The properties listed here are the most important ones for CAT in Oracle/UNIX configurations.
The two rightmost columns indicate whether a default (cross-system) value can be set (Dflt = X) and whether system-specific settings that override the default are possible (Sys = X).
Name Meaning Default System
deljoblogs Controls whether old job logs are deleted from /sapmnt/SID/global/*JOBLG directories
during post-processing. Depending on the number of logs, this can be slow.
The parameter takes a numeric value; if the number of job logs is less than or equal to this value, job logs are removed. To suppress removal, set to 0. To force removal regardless of the number of logs, set a high value, such as 999999999.
X X
dirshare Full path to shared directory X X
exportdir Directory for exporting data from target database during pre-processing on target
X X
hdbsystemdb hdbuserstore key used to connect to the SAP
HANA system database. Default: SAPCATSYSTEM
listener_control
See note after table
Stop/start Oracle Listener during target refresh
• false: do not stop/start Listener
• true: stop/start Listener (name must be in listener_name.<SID>)
- X
listener_name
See note after table
Name of Oracle Listener to start/stop during target refresh; only needed if property SID/listener_control is set to true
- X
maxage Maximum age in seconds of the source copy. If it is older than this, the target process does not use it for the refresh.
X X
redosize Redo log size for target database. If not specified, CAT for SAP creates redo logs on the target with the same size as those on the source. This option is useful when the source database has huge redo logs, which are not especially useful and can increase recovery time on the target.
- X
Name Meaning Default System
signalwait Number of seconds that the target process waits for the source process to complete.
X X
sourceops Defines what to do in the target database with the OPS$ users that have come over from the source. Possible values are "drop", "lock" and "keep.
X X
Table 2: SAP CAT properties
NOTE
The listener_control and listener_name properties must be adapted
because they are inherently SID-specific. Other properties may be left at their default settings or changed to system-specific settings. In addition, the default settings may be changed.
4.2.8 CONFIGURATION FILES (STORAGE_<SID>)
4.2.8.1 Function
These files contain settings for the FlashArray snapshot process. Each copy process normally involves two storage configurations:
• File storage_<SSID> for the source system
• File storage_<TSID> for the target system
4.2.8.2 Format
Each record in the storage file has the form:
type:value
Records of a given type may appear more than once in the same file.
The record types are:
• flashArray
Host name or IP address of the Flash Array. This entry must exist on both source and target.
• flashArrayUser
UserID for logging in to the Flash Array; This entry must exist on both source and target.
• pgroup
Name of the protection group. The source system protection group must contain only FlashArray volumes that hold database data files. Match the snapped volumes with the volumes mapped in the storage file of the target system. If more (or fewer) volumes are snapped, the copy will not start. On target systems, a protection group of the whole system can be used to create a snapshot backup before starting the copy procedure.
• fs: File system mount path; exists only on the target side.
NOTE For AIX source systems, this entry should also be created. These are used to:
• Gather the AIX LVM storage information.
• Temporarily freeze the file systems for maximum 60 seconds when making the snapshot.
WARNING
On AIX systems, file system paths are used to derive the physical volumes from the containing volume group (via the logical volume). These are unmounted; varied off and exported on the target system!
Thus, physical volumes may only contain logical volumes related to the database SAP data files. Database data files should therefore be created in a separate volume group on separate dedicated physical volumes/disks.
• volmap_<SSID>_<TSID>
Volume mappings from source to target. Present only on the target.
Volume mappings are the “brain” of CAT for SAP. They specify which FlashArray volumes are to be copied from the source to the target. As a prerequisite, the number of source volumes on should be equal to the number of target volumes.
NOTE For AIX filesystems only:
SAP CAT handles volume management on AIX target systems, including attaching disks and importing the volume groups on the target system based on storage information collected from the source system.
If the source system has more FlashArray volumes than the target system, empty FlashArray volumes must be created on the target system.
The final step is then listing the volume mapping between the volumes of system A and system B in the storage file of the target system.
End of: AIX filesystems
4.2.8.3 Example
Example of a storage file for a source system:
flashArray:10.148.25.2
flashArrayUser:pureuser
pgroup:OracleCRPGroup
Example of a storage file for a target system:
flashArray: 10.148.25.2
flashArrayUser:pureuser
fs:/oracle/SID/sapdata1
fs:/oracle/SID/sapdata2
fs:/oracle/SID/sapdata3
fs:/oracle/SID/sapdata4
fs:/oracle/SID/sapdata5
fs:/oracle/SID/sapdata6
fs:/oracle/SID/sapdata7
fs:/oracle/SID/sapdata8
volmap_CRP_CRQ:OracleDataVol1 OracleCPDataVolume1
volmap_CRP_CRQ:OracleDataVol2 OracleCPDataVolume2
volmap_CRP_CRQ:OracleDataVol3 OracleCPDataVolume3
volmap_CRP_CRQ:OracleDataVol4 OracleCPDataVolume4
volmap_CRP_CRQ:OracleDataVol5 OracleCPDataVolume5
volmap_CRP_CRQ:OracleOrigLogA OracleCPOrigLogA
volmap_CRP_CRQ:SAPOrigLogB OracleCPOrigLogB
volmap_CRP_CRQ:OracleMirrLogA OracleCPMirrLogA
volmap_CRP_CRQ:OracleMirrLogB OracleCPMirrLogB
4.2.9 CONFIGURATION FILE (TRUNCATE.CONF)
4.2.9.1 Function
This file defines the sets of target system tables to be truncated (emptied) after the refresh.
4.2.9.2 Format
An entry in the truncation list has one of the following forms:
abap:TABLE
<schema>:TABLE
The prefix abap refers to tables in the ABAP schema; actual schema names are substituted at
runtime. Tables in non-SAP schemas can be truncated by preceding their names with those of their schemas.
4.2.9.3 Example
The example below lists a set of ABAP tables to be truncated plus two tables in external (non-SAP) schema PERF.
# List of tables to truncate after refresh and before SAP start
# Format:
# abap:TABLE
# java:TABLE
# <name>:TABLE
#
# 'abap' and 'java' are placeholders for the names of the
# ABAP/Java schema owners (actual schmena names are substituted at
# runtime). <name> is the name of a non-SAP schema.
#
# Note: ABAP table DDLOG is *always* truncated, whether
# listed here or not.
abap:DDLOG
abap:SDBAD
abap:SDBAH
abap:SDBAP
abap:SDBAR
abap:DBMSGORA
abap:VBHDR
abap:VBMOD
abap:VBDATA
abap:VBERROR
abap:TPFHT
abap:TPFET
abap:SNAP
abap:TLOCK
abap:RZLLITAB
abap:DBSTATTORA
abap:DBSTATIORA
abap:DBSTATHORA
abap:DBSTAIHORA
abap:TPFID
abap:TPFBA
abap:BTCOMSET
abap:BTCOMSDL
abap:MONI
abap:SWNCMONI
abap:SWNCMONIINDEX
abap:APQI
abap:APQD
PERF:COLLECTOR_INFO
PERF:COLLECTOR_HIST
5 RUNNING CAT FOR SAP
5.1 CONDUCTOR AND CLIENT SCRIPT
To carry out a complete system copy, run a "conductor" script. This script takes as input a sequence file (see 4.2.2) to determine which contexts (phases) must be executed, in what order and on which server. The conductor is a shell script called systemcopy.
Depending on the connection type defined in the sequence file, the conductor connects via 2 possible protocols with the server responsible for executing the actions in that context:
• Secure Shell (SSH) for UNIX
• SOAP (HTTP/S) for SAP Host Agents on Windows
To run a context, the conductor logs in to the appropriate server via the specified protocol and invokes the client script responsible for executing the actions in that context.
5.1.1 UNIX
In general, when the systemcopy script for a CAT run on UNIX systems starts, the following steps
are taken:
• The systemcopy script is started on the SAPCAT Docker container with a given sequence
file.
• For each sequence in the file a CAT instance is started. Depending on the connection type, an instance is started on:
o The SAPCAT Docker container
o A source database or target database/SAP system
The complete connection flow with UNIX systems is shown in Figure 2:
Figure 2: Conductor script systemcopy
The client script, which is called SAPCOPY, receives the name of the context as one of its parameters.
SAPCOPY then looks up that context in the flow file (see 4.2.5), where it finds the list of actions that belong to the context. It then executes one action after the other. This is shown schematically in Figure 3: to execute a context do_xyz, the conductor launches SAPCOPY on the remote host.
SAPCOPY reads the action list of do_xyz from the flow file and runs the actions (scripts) action1,
action2, action3, etc.
Figure 3: Client script SAPCOPY
5.2 CALLING SYSTEMCOPY
5.2.1 WHICH USER?
Any server user that meets the following conditions can run systemcopy:
• Read and write access to the shared directory
• Password-less SSH to all servers involved in the system copy (source DB server, target DB server, target CI server and Flash Array)
• Member of the sudo group
NOTE
Several actions require the sidadm or orasid user to start the sudo
command. For example:
• AIX volume handling
• Run as another user: sudo su –orasid –c “command…”
• mount/umount
5.2.2 SYNTAX
The syntax of the systemcopy command is as follows:
systemcopy [ -dialog | -auto | -fullauto ]
[-flow flowfile]
[ -restart | -norestart ]
seqfile
Command arguments:
-dialog
Run in dialog (interactive) mode. In this mode, systemcopy prompts at the beginning of
every context and SAPCOPY at the beginning of every action.
-auto
Run in automatic mode. In this mode, systemcopy prompts at the beginning of every
context; the context itself, however, runs without interaction.
-fullauto
Run in fully automatic mode. There is no user interaction; the system copy runs from beginning to end without human intervention.
-flow flowfile
Run with the given flow file. Remark: it is mandatory to use the relative path to the flow file (see 4.2.5 for a description of the flow file).
-restart
If the previous system copy on this path failed, this option instructs systemcopy to continue at the point of failure. If the previous copy ended successfully, this option has no effect.
-norestart
If the previous system copy on this path failed, this option instructs systemcopy to start a fresh copy from the beginning. If the previous copy ended successfully, this option has no effect. Note: if neither option is specified and the previous copy failed, then the behavior is as follows:
• Dialog mode: systemcopy prompts interactively for a decision (continue failed copy or
start anew)
• Auto / full auto mode: the script reports an error and stops. Either -restart
or -norestart must be specified.
seqfile
Name of the sequence file for this copy path. This is the only mandatory argument. See section 4.2.2 for a description of the sequence file.
6 APPENDIX
6.1 PASSWORD-LESS SSH SETUP
To avoid password prompts during the execution of SAP CAT, set up a SSH key. The public and private key pair is used to automatically log on on the different systems.
This example sets up user ecqadm to automatically login onto the FlashArray’s pureuser account.
6.1.1 GENERATE PRIVATE AND PUBLIC KEY
If no key pair exists, generate a key pair for the user to automatically login onto the remote system as follows:
ecqadm> ssh-keygen -t ecdsa -b 521
Accept the default path:
Generating public/private ecdsa key pair.
Enter file in which to save the key (/home/ecqadm/.ssh/id_ecdsa):
Just press ENTER when asked for passwords:
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
This generates following output:
Your identification has been saved in /home/ecqadm/.ssh/id_ecdsa.
Your public key has been saved in /home/ecqadm/.ssh/id_ecdsa.pub.
…
Copy the contents of the public key file to a buffer:
ecqadm> cat /home/ecqadm/.ssh/id_ecdsa.pub
Select the output and copy it for example with CTRL+C.
6.1.2 ADD THE PUBLIC KEY TO THE PURE ARRAY
In this step, paste the public SSH key into the FlashArray GUI.
• Log via the web browser into the storage array with the user pureuser.
• Got to: System > User > Me
• Click on Setting (the gear icon)
• Select Update Public Key
• Paste the copied public key. Make sure additional line breaks are removed and click Save.
6.1.3 TEST
Start a ssh session as ecqadm to the array with user pureuser without a password prompt.
For example:
ecqadm> ssh pureuser@<array_name> purevol list
6.2 ERRORS
6.2.1 XSOAP
The SOAP protocol is used to connect with the SAP Host Agent on Windows servers.
This section lists the possible exit/error codes and the possible solutions. The error codes returned are those from the tool curl.
6.2.1.1 xsoap: 6
In this case there is clearly a connection problem:
• Is the hostname known inside the SAPCAT container? Can it resolve the ip address?
• Is there even a connection possible? Firewall? Network range/setup?
6.2.1.2 xsoap: 22
When error 22 is shown, it indicates a possible authorization issue:
• Wrong username and/or password?
Another possibility is that the program failed. Please check the SAP Host Agent and CAT log files.