Flexframe Ff4s-Sap Install

User Guide

English

FlexFrame for SAP 4.0Installation Guide for SAP Solutions

FlexFrame for SAPVersion 4.0

Installation Guide for SAP Solutions

Edition March 2007 Document Version 1.0

Fujitsu Siemens Computers GmbH Copyright Fujitsu Siemens Computers GmbH 2007FlexFrame, PRIMECLUSTER. PRIMEPOWER and PRIMERGY are trademarks of Fujitsu Siemens Computers SPARC64 is a registered trademark of Fujitsu Ltd. SAP and NetWeaver are trademarks or registered trademarks of SAP AG in Germany and in several other countries Linux is a registered trademark of Linus Torvalds SUSE Linux is a registered trademark of Novell, Inc., in the United States and other countries Java and Solaris are trademarks of Sun Microsystems, Inc. in the United States and other countries Intel and PXE are registered trademarks of Intel Corporation in the United States and other countries MaxDB is a registered trademark of MySQL AB, Sweden MySQL is a registered trademark of MySQL AB, Sweden NetApp , Network Appliance , Open Network Technology for Appliance Products, Write Anywhere File Layout and WAFL are trademarks or registered trademarks of Network Appilance, Inc. in the United States and other countries Oracle is a registered trademark of ORACLE Corporation EMC , CLARiiON , Symmetrix , PowerPath , Celerra and SnapSure are trademarks or registered trademarks of EMC Corporation in the United States and other countries SPARC is a trademark of SPARC International, Inc. in the United States and other countries Ethernet is a registered trademark of XEROX, Inc., Digital Equipment Corporation and Intel Corporation Windows , Excel and Word are registered trademarks of Microsoft Corporation All other hardware and software names used are trademarks of their respective companies. All rights, including rights of translation, reproduction by printing, copying or similar methods, in part or in whole, are reserved. Offenders will be liable for damages. All rights, including rights created by patent grant or registration of a utility model or design, are reserved. Delivery subject to availability. Right of technical modification reserved.

Contents1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 2 2.1 2.2 2.3 2.4 2.5 2.6 2.6.1 2.6.2 3 3.1 3.2 3.3 4 5 5.1 5.2 5.3 5.4 6 6.1 6.2 7 7.1 7.1.1 7.2 7.2.1 Introduction ..................................................................................................... 1 Purpose of this Document ................................................................................. 1 General Usage of SAP Installation Guide During Installation............................ 1 Overview ........................................................................................................... 1 Notational Conventions ..................................................................................... 2 Document History.............................................................................................. 2 Related Documents........................................................................................... 2 IP Addresses Used in this Document ................................................................ 3 Hotlinks for FlexFrame ...................................................................................... 4 Overview .......................................................................................................... 7 FlexFrame ......................................................................................................... 7 Virtualization of SAP Services ........................................................................... 7 Preparations ...................................................................................................... 8 Gathering SAP Services Information................................................................. 8 FlexFrame Installation Diagram Step by Step ................................................... 9 Application Nodes Released ........................................................................... 12 Linux................................................................................................................ 12 Solaris ............................................................................................................. 12 SAP Service Connection .............................................................................. 13 OSS Network Connection with Dedicated saprouter Server ........................... 13 OSS Network Connection without Dedicated saprouter Server....................... 14 Related SAP Notes ......................................................................................... 15 FA Agents ...................................................................................................... 17 SAP Service Scripts for Virtualizing Applications...................................... 19 Starting a Configuration................................................................................... 20 Configuring the Service Status Path................................................................ 22 Configuring the Interfaces ............................................................................... 23 Host Names..................................................................................................... 25 Pre-SAP Installation Steps ........................................................................... 27 General ........................................................................................................... 27 SAN Configuration........................................................................................... 29 Operating System in General ....................................................................... 31 Solaris 8/9 ....................................................................................................... 31 Integrating New Java SDK (JDK) in Solaris .................................................... 31 Linux................................................................................................................ 34 Integrating a new Java SDK (JDK) in Linux SLES8 / SLES9 32bit.................. 35


Contents

7.2.2 8 8.1 8.1.1 8.1.2 8.1.3 8.2 8.2.1 8.2.2 8.3 8.3.1 8.3.2 8.4 8.5 8.5.1 8.5.1.1 8.5.2 8.5.3 8.5.4 8.5.5 9 9.1 9.2 9.2.1 9.2.2 10 10.1 11 11.1 11.1.1 11.1.2 12 12.1 12.2 12.2.1 12.2.2 12.2.3 12.2.4 12.2.5

Integrating New Java SDK (JDK) in SLES9_x86............................................. 36 Database in General ...................................................................................... 37 Installation of Oracle in General ...................................................................... 37 TNS Listener for Oracle ................................................................................... 37 listener.ora .................................................................................................... 38 Host Name Changes in "tnsnames.ora" .......................................................... 39 Installation of Oracle 9.2 .................................................................................. 40 Post-Oracle 9.2 Installation Steps ................................................................... 41 Patch Installation ............................................................................................. 41 Installation of Oracle 10.1 (R1) ........................................................................ 41 Script root.sh ................................................................................................... 42 Patch Installation ............................................................................................. 43 Installation of Oracle 10.2 (R2) ........................................................................ 43 Installation of SAPDB / MaxDB........................................................................ 43 XUSER ............................................................................................................ 44 Same SID on Different Platforms..................................................................... 44 MaxDB 7.5/7.6 /etc/opt/sdb ............................................................................. 45 MaxDB 7.6....................................................................................................... 46 IO Balancing .................................................................................................... 47 Directories for sapdata / saplog ....................................................................... 47 SAP in General............................................................................................... 49 ASCS ABAP Central Service Instance in General........................................... 49 SAP Java Cryptographic Toolkit ...................................................................... 49 Linux................................................................................................................ 49 Solaris ............................................................................................................. 50 Installing SAP 4.6C SR2 ................................................................................ 51 SAPDB / MaxDB / Oracle ................................................................................ 52 Installing Web AS 6.20 .................................................................................. 57 Installating Web AS 6.20 with sapinst 642....................................................... 57 Installing the Central Instance (CI) .................................................................. 58 Installing the Database Instance (DB) ............................................................. 60 Installing Web AS 6.40 with NetWeaver04 SR1........................................... 65 Description of Web AS 6.40 JAVA................................................................... 66 Web AS 6.40 NetWeaver04SR1 JAVA MaxDB/Oracle ................................... 67 SCS ................................................................................................................. 67 DB ................................................................................................................... 71 JC .................................................................................................................... 72 J ...................................................................................................................... 72 Web AS 6.40 Deployment SP16...................................................................... 73


Contents

12.2.5.1 12.2.5.2 12.3 12.3.1 12.3.2 12.3.3 12.4 12.4.1 12.4.2 12.4.3 12.4.4 12.4.4.1 12.5 13 13.1 13.1.1 13.2 13.3 13.4 13.5 13.6 13.7 13.8 13.9 13.10 13.11 13.12 13.13 13.14 13.14.1 13.14.2 13.14.3 13.14.4 14 14.1 14.1.1 14.1.2 14.1.3 14.1.4 14.1.5 14.1.6 14.2

JC.................................................................................................................... 73 J ..................................................................................................................... 73 Web AS 6.40 NetWeaver04SR1 ABAP Oracle ............................................... 73 CI..................................................................................................................... 73 DB ................................................................................................................... 77 APP ................................................................................................................. 78 Web AS 6.40 ABAP + JAVA-AddIn NetWeaver04SR1 ................................... 78 CI..................................................................................................................... 78 DB ................................................................................................................... 78 SCS................................................................................................................. 79 Web AS 6.40 Deployment SP16 ..................................................................... 83 DVEBMGS (JAVA-AddIN) ........................................................................ 83 Web AS 6.40 Application Server JAVA-AddIn................................................. 83 Installing NetWeaver 7.0SR1/SR2 ................................................................ 85 Installation of SAP Solutions ........................................................................... 86 Prerequisite Checker (Linux) ........................................................................... 89 Sapinit File (Linux)........................................................................................... 90 installsapinit.sh symlink (Linux) .................................................................... 91 Services File (Linux)........................................................................................ 92 NetWeaver 7.0SR1/SR2 and Oracle 10(R2) ................................................... 93 NetWeaver 7.0SR1/SR2 and MaxDB76.......................................................... 94 NetWeaver 7.0SR1/SR2 / MaxDB76 and SLES9_x86 .................................... 96 NetWeaver 7.0SR1/SR2 MaxDB76 and Solaris9 ............................................ 98 NetWeaver 7.0SR1/SR2 and EP..................................................................... 98 NetWeaver 7.0SR1/SR2 ABAP+JAVA-AddIn ............................................... 100 NetWeaver 7.0SR1/SR2 JAVA only .............................................................. 105 LiveCache ..................................................................................................... 106 ERS Enque Replicated Server ................................................................... 106 Installation of SAP Solutions Post Phase ................................................... 106 Linux.............................................................................................................. 106 Solaris ........................................................................................................... 107 Linux and Solaris........................................................................................... 107 Solaris 10 Preparations ................................................................................. 108 Post-SAP Installation Steps ....................................................................... 109 Configuring the Virtual Host Name Manually................................................. 109 Central Instance (not valid for Web AS 6.40 J2EE)....................................... 111 Database Instance (also valid for WebAS 6.40 J2EE) .................................. 111 Application Instance (not valid for WebAS 6.40 J2EE).................................. 112 SCS Instance ................................................................................................ 113 ASCS Instance (WebAS 7.00)....................................................................... 115 NetWeaver 7.0 / NetWeaver 7.0SR1/SR2..................................................... 116 Configuring the Virtual Host Name for Web AS 6.40 ABAP and JAVA-AddIn / WebAS 6.40 JAVA.................................................................. 117


Contents

14.3 14.4 14.5 14.6 14.7 14.8 14.9 14.10 14.11 14.12 15 15.1 15.2 15.2.1 15.2.1.1 15.2.1.2 15.2.1.3 15.2.1.4 15.2.2 15.2.3 15.3 15.3.1 15.3.2 15.3.3 15.4 15.5 15.5.1 15.5.2 15.5.3 15.5.3.1 15.5.3.2 15.5.3.3 15.5.4 16 16.1 16.2 17 18 19

Start Script startdb for Oracle and SAPDB/MaxDB ....................................... 134 Stop Script stopdb for Oracle and SAPDB/MaxDB........................................ 134 Scheduling of Batch Jobs (CCMS) ................................................................ 134 Profile Parameters for DEFAULT.PFL........................................................... 135 Profile Parameters for the Start Profile .......................................................... 136 Profile Parameters for the Instance Profile .................................................... 136 SAP Start and Stop Mechanism (Release 4.6CSR2) .................................... 137 SAP License .................................................................................................. 139 SAPLogon SAPGUI.................................................................................... 139 SAPROUTER on the Control Node ............................................................... 142 SAP System Administration ....................................................................... 143 Upgrading Application Software .................................................................... 143 Software Administration................................................................................. 143 Starting and Stopping SAP Services ............................................................. 143 SAP Service Scripts ...................................................................................... 143 SAP Service Script Actions............................................................................ 145 Return Codes of the SAP Service Scripts ..................................................... 146 Joined SAP Service Scripts ........................................................................... 146 Removing an Application from Monitoring by FA Agents............................... 147 Application Software...................................................................................... 147 SAP Release Upgrade .................................................................................. 148 Services Port ................................................................................................. 148 Group Sapinst................................................................................................ 148 FA-Agent ....................................................................................................... 149 Transport Directory........................................................................................ 149 Printing in FlexFrame 3.2 .............................................................................. 149 Possible Printing Architectures and Methods ................................................ 149 Local Printing (Access Methods C and L)...................................................... 150 Remote Printing (Access Methods S and U) ................................................. 150 Architecture Prerequisites.............................................................................. 150 SAP System Prerequisites............................................................................. 151 Description..................................................................................................... 151 Front-end Printing on Local Printers .............................................................. 152 BRBACKUP and SAPDBA/BRCONNECT................................................... 153 Solution 1 for Oracle and SAPDB/MaxDB ..................................................... 153 Solution 2 for Oracle...................................................................................... 154 Troubleshooting .......................................................................................... 155 Abbreviations............................................................................................... 157 Glossary ....................................................................................................... 161


Contents

20

Index............................................................................................................. 167


1

Introduction

As we continue to improve the installation process, this document is permanently subject to change. The latest version can be found at http://extranet.fujitsu-siemens.com/FlexFrame Please make sure that you get the latest version before starting an installation, and always read the latest README file that contains further changes and fixes.

1.1

Purpose of this Document

This document provides brief instructions for installing SAP with Oracle or SAPDB / MaxDB in a FlexFrame 3.2 environment. It is absolutely essential to use the SAP Installation Guides and the latest SAP Notes for the SAP solution that is to be installed. It is intended for certified FlexFrame consultants only; sound knowledge of FlexFrame technology and its background is required.

1.2

General Usage of SAP Installation Guide During Installation

Please note that the entire documentation on installing SAP on FlexFrame is not intended as a stand-alone documentation. It solely describes the differences to a standard SAP installation as described by SAP.

1.31. 2. 3. 4. 5.

Overview

The SAP installation is performed in 5 steps: Information about the SAP Services is collected in an Excel spreadsheet; a configuration file is generated and stored on the LDAP server (Section 2.4).

SAP Service scripts for virtualizing the applications (Chapter 5). Oracle 9.2/10.x and SAPDB (7.3 / 7.4) /MaxDB (7.5 / 7.6) installation, LDAP configuration (Chapter 6). Installation of SAP software depending on the release (Chapters 9, 10, 11 ,12, and 13). Post-SAP installation steps have to perform (Chapter 14).1


Introduction

1.4

Notational Conventions

The following conventions are used in this manual: Additional information that should be observed.

Warning that must be observed. Names of paths, files, commands, and system output. Names of variables User input in command examples (if applicable using with variables)

fixed font fixed font

1.51.0

Document HistoryChanges First Edition Date 2006-05-25

Document Version

1.6

Related Documents

FlexFrame for SAP Planning Tool FlexFrame for SAP Installation of a FlexFrame Environment FlexFrame for SAP Administration and Operation FlexFrame for SAP Network Design and Configuration Guide FlexFrame for SAP FA Agents - Installation and Administration FlexFrame for SAP Installation ACC 1.0 SP13 FlexFrame for SAP myAMC.FA_LogAgent - Concept and Usage FlexFrame for SAP Upgrading FlexFrame 3.1 or 3.2 to 4.0 FlexFrame for SAP White Paper PRIMECLUSTER Documentation ServerView Documentation SUSE Linux Enterprise Server Documentation Solaris Documentation

2


Introduction

1.7

IP Addresses Used in this Document

All Application Nodes must belong to a dedicated FlexFrame pool; nevertheless it is possible to have only one pool in a FlexFrame 3.2 environment. It is not permissible to use a node in more than one pool at the same time. For this reason Application Nodes belonging to the same pool use the same IP subnet. All blades in the same cabinet can be members of different pools. Control Nodes and the NetApp Filer have IP addresses for all pools.

The Control LAN is independent of the pool number and has its own IP subnet addresses.Pool Client IP No. 1 2 1 2 1 2 1 1 1 1 2 2 2 2 1 2 10.0.2.11 10.0.2.12 10.0.2.13 10.0.1.21 10.0.2.22 192.168.2.11 192.168.101.11 192.168.2.12 192.168.101.12 192.168.2.13 192.168.101.13 192.168.1.21 192.168.100.21 192.168.2.22 192.168.101.22 192.168.200.21 192.168.200.22 10.0.1.1 10.0.1.2 10.0.1.3 192.168.1.1 192.168.1.2 192.168.1.3 192.168.100.1 192.168.100.2 192.168.100.3 Server IP Storage IP Control IP (RSB) Heartbeat (PCL) 192.168.200.201 192.168.200.202

10.0.1.201 192.168.1.201 192.168.100.201 192.168.200.221 control1 10.0.2.201 192.168.2.201 192.168.101.201 192.168.200.221 control1 10.0.1.202 192.168.1.202 192.168.100.202 192.168.200.222 control2 10.0.2.202 192.168.2.202 192.168.101.202 192.168.200.222 control2 192.168.100.203 192.168.200.203 filer 192.168.101.203 192.168.200.203 filer BX300-1 192.168.200.230 bx3001mg mt blade1 blade2 blade3 BX300-2 192.168.200.240 bx3002mg mt blade11 blade12 blade13 PW250_1 PW250_2

192.168.200.252 switch1 192.168.200.253 switch2 SAP instance 1 10.0.1.101 192.168.1.101 cic11


3

Introduction

Pool Client IP No. 1 1 1 1 2 2 2 2

Server IP 192.168.1.102

Storage IP

Control IP (RSB) dbc11 app01c11 app02c11 app03c11 cibw1 dbbw1 app01bw1 app02bw1

Heartbeat (PCL)

10.0.1.103 192.168.1.103 10.0.1.104 192.168.1.104 10.0.1.105 192.168.1.105 10.0.2.111 192.168.2.111 192.168.2.112 10.0.2.113 192.168.2.113 10.0.2.114 192.168.2.114

1.8

Hotlinks for FlexFrame

PRIMECLUSTER4

PRIMECLUSTER Wizards (downloads & documents) http://extranet.fujitsu-siemens.com/My.htm Primecluster RMS Home (information on the Reliant Monitor Services) http://athen.mch.fsc.net/~kon/reliant.htm PRIMECLUSTER http://www.primecluster.com/ SAP on Linux / FlexFrame SAP on Linux (released kernels, hardware and distributions for SAP) http://www.sap.com/linux SAP NetWeaver on Linux http://www.sap.com/solutions/netweaver/ Official FlexFrame home page http://www.fujitsu-siemens.com/flexframe Information on FlexFrame for SAP http://extranet.fujitsu-siemens.com/FlexFrame PRIMERGY manuals http://manuals.fujitsu-siemens.com/primergyservers.html

4


Introduction

SAP on Solaris / FlexFrame Global information and operating system manuals Fujitsu Siemens Computers and SAP http://manuals.fujitsu-siemens.com/unixservers.htmlPRIMEPOWER manuals

http://manuals.fujitsu-siemens.com/unixservers.html Knowledge SNMP protocol information http://www2.rad.com/networks/1995/snmp/snmp.htm Linux Knowledge Portal http://www.linux-knowledge-portal.org/de/index.php Official Linux documentation project http://www.tldp.org/ Official PRIMERGY documentation http://manuals.fujitsu-siemens.com/serverbooks/start.htm Internal information on PRIMERGY servers http://extranet.fujitsu-siemens.com/products/primergy Software GNU Homepage http://www.gnu.org/ Software by Fujitsu Siemens Computers http://support.fujitsu-siemens.de/de/support/downloads.html Open SSH website http://www.openssh.org/ SSH download http://www.ssh.com/


5

22.1

OverviewFlexFrame

FlexFrame is a completely revolutionary solution for creating and configuring the infrastructure required to run a SAP environment. An essential feature of FlexFrame is that the infrastructure is divided into the following elements: Computing, represented by the Application Nodes Network, completely implemented on the basis of IP networks Control and management, represented by the Control Nodes Storage, represented by the NetApp Filer

A number of fundamental features of FlexFrame are implemented on the basis of this division into the areas of computing, network, control and storage: Virtualized SAP Services Central, shared operating system Embedded high availability Network attached storage (NAS)

For further details, please refer to the Technical White Paper for FlexFrame 4.0.

2.2

Virtualization of SAP Services

To enable high availability and scalability of SAP Services (database, SAP R/3, etc.), the fixed assignment of SAP Services to Application Nodes has been abolished. Here, each SAP Service can run not only on one host in the high-availability cluster, as has been the case with customary high availability solutions to date, but on virtually any required Application Node in FlexFrame, provided it has resources available for this. Technically speaking, virtualization is achieved by the fact that an SAP Service is able to move, taking its IP address with it and retaining its virtual host name. Virtualization is achieved here by SAP Service scripts that ensure the virtual host names and virtual IP addresses are set accordingly when SAP Services are started and stopped.


7

Overview

2.3

PreparationsYour FlexFrame hardware A notebook with Microsoft Excel 2003 FlexFrame 3.2 Planning CD-ROM OS installation of Control Nodes and the necessary Application Nodes (Linux/Solaris) must have been completed Installation DVDs/CDs from SAP for the required SAP Releases

Make sure you have the following hardware and software available:

2.4

Gathering SAP Services Information

The FlexFrame Planning Tool is a program based on Microsoft Excel 2003. It was used during the setup of the hardware to collect all information for a FlexFrame landscape (original file name FFPLAN.XLS). You may use the Excel sheet that has already been completed or a copy of it. If all SAP Services were entered beforehand, you may skip the rest of this section. After you have opened the Excel sheet, go to the SAP Services tab at the bottom of the document (scroll to the right in the tab list). If there are no SAP instances listed, click the Add SID button and insert the SAP Services that have to be installed. You will be asked for the following details: SID Select pool (dropdown list) Select DB type (dropdown list) Select SAP version (dropdown list) Select one of the following (only Web AS 6.40) CI CI + SCS JC + SCS (ABAP only) (ABAP/JAVA-AddIn) (JAVA only)

8


Overview

Select one of the following (only Web AS 7.00) CI (ABAP only) CI + ASCS (ABAP/ASCS) CI + ASCS + SCS (ABAP/ASCS/JAVA-AddIn) CI + SCS (ABAP/JAVA-AddIn) JC + SCS (JAVA only) CI + ASCS + ERS (ABAP/ASCS/Enqueue Replicated Server) CI + ASCS + SCS + ERS (ABAP/ASCS/JAVA-AddIn/Enqueue Replicated Server) JC +ERS (Enqueue Replicated Server only)

Number of application instances (in addition to the CI/JC) LiveCache (WebAS 7.0/MaxDB only) If you want to install a SAP component like CRM or R/3 (ERP), you wont find it in the dropdown list of the SAP versions. You have to enter the underlying SAP Basis (Web AS) release which must be one of the four listed versions. If you need another Basis release or another DB type for your SAP component, please ask our SAP Competence Center for project-specific support. Please make sure to have the latest version before starting an installation, and always read the latest README file that contains further changes and fixes.

After all information for the complete SAP Services landscape has been entered in the tool, the configuration files ff_*.conf can be created by pressing the button labeled Generate configuration files. A dialog box will ask you for a directory where the configuration files are stored. We recommend that you create a new directory for each project and version of the files, including the FFPLAN32.XLS spreadsheet. The configuration file ff_services.conf now has all the information required to be stored on the LDAP server. This step must currently be performed manually until a tool is available to store this information in LDAP. For more information about LDAP please refer to the LDAP Installation Guide.

2.5

FlexFrame Installation Diagram Step by Step

The following chart shows the installation steps of a complete FlexFrame installation. As the final step of the platform basis, SAP installation is shown at the bottom of the second chart (red frame). The numbers in the grey bar on the left side of the chart refer to the sections of the chapter Quick Installation in the manual FlexFrame for SAP Installation of a FlexFrame Environment.


9

OverviewThe steps are numbered according to the sections in the quick installation chapter Preparations

FF Prep CD

Excel FlexFrame Planning Tool

Installation Steps FlexFrame 4.0

2.2

Configuration Files: ff_cn.conf ff_an.conf ff_misc.conf ff_wiring.pl ff_services.conf ff_switch_conf.pl

Cabling Plan

HW Installation

2.3OS Installation Control Node

Hardware Installation

Network Cabling

2.4Switch Configuration

External Medium: Configuration files; Patches

DVD CN Boot from DVD Control Node 1

Service CN ff_setup_cn.sh ff_apply_settings_cn.sh

Serial cable Control Node 1 OS Installation 2nd Control Node, see 2.4 DVD CN Serial cable Control Node

ff_wiring.pl Boot from DVD

Switches Control Node 2

2.5 2.6NAS Configuration

2.7MAC Addresses

Manual Setup

NAS system

2.8LDAP and SSH Configuration

Enter MAC adresses

ff_get_an_mac.pl

ff_an_mac.conf

2.9.1 2.6.3Filer QTrees; LDAP Initialisation

ff_cn.conf; ff_an.conf; ff_services.conf; ff_an_mac.conf

Generates Scripts: ff_ldap_conf.pl ff_slapd_init.sh ff_filer_qtree_create.sh

2.9.2PRIMECLUSTER Configuration

ff_cn.conf; ff_an.conf; ff_services.conf; ff_an_mac.conf

ff_filer_qtree_create.sh ff_slapd_init.sh NAS system

2.9.4

Commands: cftool -n sdtool -s hvdisp -a

Control Node

10


Overview

2.10Preparation for Linux Application Nodes

Application Node Configuration

DVD AN Linux

ff_install_an_linux_images.sh NAS system

2.10.2Preparation for Solaris Application Nodes

DVD AN Solaris nb_unpack_bi NAS system

2.10.3

netboot.cfg Generation of netboot config files

2.10.4

ff_cn.conf ff_an.conf ff_an_mac.conf

ff_create_an_cfg.pl

NAS system /tftpboot/config/netboot_ _.cfg

Setup Linux Application Nodes

LDAP Configuration Files

ff_new_an.sh

2.10.5

in /FlexFrame/volFF/FlexFrame/ldap/etc/ pools// ; /opt/FlexFrame/bin/ff_linux_ldap_client.sh /tftpboot/config/netboot__.cfg

Customized Boot Image

NAS system

Setup Solaris Application Nodes

/FlexFrame/volFF/ FlexFrame/ldap/ etc/pools// solaris_ldap_client.sh

ff_new_an.sh

Customized Boot Image

2.10.6NAS system

2.12 Proceed with SAP Installation


11

Overview

2.62.6.1

Application Nodes ReleasedLinux

All current SAP-certified, Intel//AMD-based servers from Fujitsu-Siemens Computers have been released. Please refer to: http://www.sap.com/solutions/netweaver/linux/platforms/fsc.asp

2.6.2

Solaris

The PRIMEPOWER servers PW250/PW450 PW650/PW850 from Fujitsu-Siemens Computers have been released. Please note: Relevant source for supported components including Application Nodes is the FlexFrame Support Matrix available at: http://extranet.fujitsu-siemens.com/flexframe

12


3

SAP Service Connection

It is very important that the customers FlexFrame environment can be accessed via SAPs OSS (Online Software Service). This is a mandatory requirement when customer support is requested, particularly during the installation phase. Please use the following OSS component for your customer calls to SAP about FlexFrame: CA-ADA-FSC

3.1

OSS Network Connection with Dedicated saprouter Server

The figure below shows an example of a possible network path from SAP to the FlexFrame environment. This figure includes a (possibly existing) server SAPROUTER, which runs the saprouter process:

SAPWalldorf Tokio Foster CityTELNET SAPGUI

CustomerGatewayTELNET SAPGUI

SAPROUTER

Control Node 1 Control Node 2

TELNET SAPGUI SSH SSH TELNET SAPGUI SSH SSH SAPGUI SSH SAPGUI SSH

Application Node 1 Application Node 2 Application Node ... Application Node n

Please enter the server SAPROUTER as External Connectivity in the FF Planning tool to get a reservation of an IP address for the first pool.


13


3.2

OSS Network Connection without Dedicated saprouter Server

The saprouter process can be configured on the Control Node cluster if there is no dedicated server for it (see section SAPROUTER on the Control Node on page 142). In this case, a virtual IP address for the Client LAN has to be defined for the saprouter service and for every pool, and PRIMECLUSTER is configured accordingly (only for the first pool today). The virtual IP addresses will be generated with the FlexFrame Planning Tool: go to the External Connectivity tab, select Add New, enter the pool name and Other as selected type. Afterwards enter Saprouter as comment and a valid host number. The host name for the first pool is predefined as saprouterINT and has to be used (check entry in /usr/opt/reliant/etc/hvipalias). To allow PRIMECLUSTER to control of the saprouter service, it is necessary to get the latest version of saprouter from SAP (Service Marketplace) and to install it in the directory /FlexFrame/volFF/FlexFrame/saprouter on the active Control Node. This directory must have been created, the saprouter program must be copied to it and the saprouttab file created in the same directory. The PRIMECLUSTER control of the service is already predefined and has to be activated! To add the saprouter application to RMS, call: cn1:~ # pcstool -s -i /etc/PRIMECLUSTER/scripts/addSaprouter FlexFrame To remove the saprouter application from RMS, call: cn1:~ # pcstool -s -i /etc/PRIMECLUSTER/scripts/delSaprouter FlexFrame To activate the modified configuration, first shut down RMS on both nodes: cn1:~ # hvshut -L cn2:~ # hvshut -L The capital letter -L keeps all services running. Then activate the modified configuration and distribute it to all Cluster Nodes: cn1:~ # pcstool -a FlexFrame Finally, start RMS on both Control Nodes: cn1:~ # hvcm cn2:~ # hvcm Dont forget to check the correct permissions in the saprouttab file if there are multiple pools!

14



SAPWalldorf Tokio Foster CityTELNET SAPGUI

CustomerGatewayTELNET SAPGUI

Control Node 1(saprouter)

SAPGUI SSH SAPGUI SSH SAPGUI SSH SAPGUI SSH

Application Node 1 Application Node 2 Application Node ... Application Node n

Control Node 2(saprouter)

3.3

Related SAP NotesTitle Service connections: Composite note (overview) Telnet link to customer systems

A table of related SAP Notes is provided below: SAP OSS note number 35010 37001


15

4

FA Agents

A FlexFrame Autonomous Agent (FA Agent) is an efficient component for highly autonomous operation of systems with distributed instances. A FlexFrame solution consists of Network Attached Storage, Application Nodes, and redundant Control Nodes. The FA Agents product has been developed for this solution, consisting of storage, servers, and connectivity. It permits fast and flexible installation of solutions, which simplify the operation of the applications and make them flexible. New nodes and application instances can be integrated without any changes to the configuration. Components of a FlexFrame Autonomous Agent: FlexFrame Autonomous Application Agent (FA_AppAgent) FlexFrame Autonomous CtrlAgent (FA_CtrlAgent)

The FlexFrame Autonomous Agent components also permit the highly available, partly autonomous operation of distributed applications. The application instances can be distributed over any required number of nodes within a FlexFrame solution. Monitoring of several services is implemented via the FlexFrame Autonomous Agent. The FA Application Agent components normally support SAP application instances (pure application instances and central instances), as well as SAPDB/MaxDB and Oracle databases. For more information, please refer to the FA Agents - Installation and Administration manual. Please make sure that the FA Application Agents are stopped on the hosts while you are installing SAP software: /etc/init.d/myAMC.FA_AppAgent stop (Linux/Solaris9) svcadm disable -s smawfa-agents (Solars10) Check the status with: /etc/init.d/myAMC.FA_AppAgent status (Linux/Solaris9) svcs smawfa-agents (Solars10)


17

5

SAP Service Scripts for Virtualizing ApplicationsMake sure that the scripts are located at /FlexFrame/scripts.

Scripts for starting, stopping and restarting the applications and services are provided by SAP.

The following scripts are installed: Script name sapapp sapascs sapci sapdb sapers sapj sapjc saplc sapscs sapservice_config sapservice_functions start_all_sapservices stop_all_sapservices Application Application server (ABAP) ABAP central services instance Central instance (ABAP) Database instance (SAPDB, MaxDB, Oracle) Enqueue Replicated Server J2EE Application server J2EE Central Instance LiveCache Java central services instance Global customer-specific parameters Global parameters and functions Initial start of all configured applications Stops all running applications

stop_all_sapservices_local Stops all running applications on the local node stop_all_sapservices_SID Stops all running applications of one SID


19

SAP Service Scripts for Virtualizing Applications

5.1

Starting a Configuration

The start_all_sapservices start script defines which applications are to be started in what sequence and on which virtual host. The script is executed on a Control Node. The script below illustrates its syntax: #!/bin/bash #----------------------------------------------------------------# Rule is: # All sapservices in one 'eval' must start one after another, # each 'eval' may start simultaneous. # C11 eval " ssh blade1 ssh blade1 ssh blade2 ssh blade3 " &

sapdb C11 sapci C11 sapapp 01 sapapp 02

start start C11 start C11 start

# BW1 eval " ssh PW250-1 sapdb BW1 start ssh PW250-1 sapci BW1 start " & #echo "" #echo "All services should be up now." #----------------------------------------------------------------The start sequence within a system ID must always be observed: ABAP 1. DB 2. CI 3. APP ABAP + ASCS 1. DB 1. ASCS 2. CI 3. APP ABAP + JAVA-AddIn 1. DB 1. SCS 2. CI 2. APP JAVA 1. DB 1. SCS 2. JC 2. J ABAP + ASCS + JAVA-AddIn 1. DB 1. ASCS 1. SCS 2. CI 3. APP Equal numbers mean that the features can be started in parallel.

20



The figure below shows the dependencies of the SAP Services (x dependent on y).

y means x is

The field containing JC and SCS is the minimal group of services for J2EE only. The field with CI and SCS is the minimum for ABAP and J2EE. Only ABAP is not mentioned here but it would be CI only. However it is possible to have a combination of CI and ASCS (ABAP central services instance). And still every service group needs its DB server.


21


5.2

Configuring the Service Status Path

The SERVICE_STATUS_PATH parameter describes the global location for the service control files of the applications (services). If such a file exists, the application or service has been started and therefore should be running. The naming convention is defined as follows: For the service types db and ci: For the service type app: Explanation: id sid __host app___host Consecutive two-digit number from 00-97 The lower-case three-digit system ID

Web AS 6.40 with JAVA/J2EE Engine and/or ABAP Engine There are additional service types when you run a J2EE engine based on WebAS 6.40. For the service types scs and jc: __host For the service type j: Explanation: id sid j___host Consecutive two-digit number from 00-97 The lower-case three-digit system ID

The content of the service control file is the name of the node the application or service is running on. Web AS 7.00 with JAVA/J2EE Engine and/or ABAP/ASCS Engine There are additional service types when you run a J2EE engine based on WebAS 7.00. For the service types scs, ascs and jc: For the service type j: Explanation: id sid __host j___host Consecutive two-digit number from 00-97 The lower-case three-digit system ID

WebAS 7.0 with JAVA/J2EE Engine and/or ABAP/ASCS Engine and ERS There are optional service types bound on WebAS 7.00. For the service types scs and ascs: sapers___host Explanation: id sid Consecutive two-digit number from 00-97 The lower-case three-digit system ID

22



WebAS 7.0 with ABAP and MaxDB (LiveCache) There is a additional service type which supports LiveCache. Service type: lc__host Explanation: sid The lower-case three-digit system ID

5.3

Configuring the Interfaces

Depending on which interfaces the nodes have, the LAN type must be assigned to the interfaces. The following section from the sapservice_config include script shows the connection between the LAN type, interface and netmask: #----------------------------------------------------------------# Netmask # #NETMASK_CLIENT=255.255.240.0 ls3195:255.255.255.0 NETMASK_CLIENT=255.255.255.0 NETMASK_SERVER=255.255.255.0 # # Interface # #----------------------------------------------------------------# SunOS link aggregation # ~~~~~~~~~~~~~~~~~~~~~~~ # # real interfaces look like this: # fjgi2005000 # this is the first interface # fjgi2005001 # this is the second interface # fjgi2005000:1 # this indicates that the first interface # # is active # fjgi2005001:1 # this indicates that the second interface # # is active # # These interface names consist of three parts: # fjgi2005001:1 # ^^^----------- interface number # ^^^^-------------- vlan # ^^^^------------------ 1GB-model # # For our network to interface relation we use only the first two # parts of the interface names, the model and the vlan. In our # example:


23


# fjgi2005 # #----------------------------------------------------------------# # Recommendations: # # for a clean linux environment without network redundancy, # FlexFrame = 3.1 # ETH_CLIENT=vlan2001 # ETH_SERVER=vlan2005 # # for a clean solaris environment with network redundancy, # FlexFrame >= 3.1 # ETH_CLIENT=fjgi2001 # ETH_SERVER=fjgi2005 # # for a mixed environment without network redundancy, # customer specific # ETH_CLIENT=eth0 belana:hme0 belana2:hme0 # ETH_SERVER=eth1 belana:hme1 belana2:hme1 # # for a mixed environment with network redundancy, # FlexFrame >= 3.1 # ETH_CLIENT=vlan2001 Baby_1:fjgi2001 Baby_2:fjgi2001 # klinge1:vlan2002 # ETH_SERVER=vlan2005 Baby_1:fjgi2005 Baby_2:fjgi2005 # klinge1:vlan2006 # # In a FlexFrame >= 3.1 environment with more than one pool, # we can have more than one configuration file, linked into an # pool-image. This avoid to define to much specials. The LAN types Server and Client are used. The syntax will be illustrated with reference to the example of Client and Server LAN. For example, the ETH_SERVER parameter describes a list of assignments between the node and the interface. The first element in the list is the standard interface for the LAN type server LAN; all of the subsequent elements are optional and describe a deviation from this type.

24



5.4

Host Names

The following (virtual) host names for the SAP Services must be used and will be generated by the Excel planning tool: Server LAN db-se ci-se ascs-se app-se scs-se jc-se j-se Explanation: id sid ci ascs app scs jc j Consecutive two-digit number 00-97 The lower-case three-digit system ID Client LAN

In FlexFrame 4.0, the following rules have to be regarded to define host names: 1. 2. 3. 4. 5. 6. The host name in the Storage LAN is built using the host name and adding -st . The host name in the Server LAN is built using the host name and adding -se . The host name in the Control LAN is built using the host name and adding -co . The host name in the Client LAN is built using the host name without an extension. Host names are written in lower case exclusively. Host names are supposed not to be longer than 13 characters.


25

66.1

Pre-SAP Installation StepsGeneral

The customary instructions and guide lines apply to the installation of the software components. The deviations from these which are required are described here. Dont change the host name of the Application Node where you are going to install the SAP and DB software. After the installation is done, further steps are described in this document to virtualize the SAP instance (except SAPINST 640 SR1 when used with the SAPINST_USE_HOSTNAME parameter). Please install CI (J2EE: SCS) first, then DB, (J2EE: JC 2nd) and the Application Server at the end. Setup the virtual interfaces before installing the SAP solution. It is absolutly essential to setup both virtual interfaces (client + server LAN) and make sure that they up an running. Please make sure that the FA Application Agents are stopped on the hosts while you are installing SAP software: /etc/init.d/myAMC.FA_AppAgent stop (Linux/Solaris9) svcadm disable -s smawfa-agents (Solars10) Check the status with: /etc/init.d/myAMC.FA_AppAgent status (Linux/Solaris9) svcs smawfa-agents (Solars10) The SAP Installation CDs/DVDs have to be copied to the NetApp Filer (from the Control Node into the directory /FlexFrame/volFF/FlexFrame/stage/SAP_SW/). The whole directory or each subdirectory has to be exported on the Filer with the necessary ro or rw options for the Application Node(s). On the Application Node, the mount will be done manually. If an SAP Installation is done after the initial setup of the FlexFrame environment and without entering the SAP Service data in the FF Planning tool before, the necessary LDAP data has to be created with the script ff_sid_adm.pl (see chapter Adding / Removing SAP Systems and Application in the manual FlexFrame for SAP Administration and Operation). The directories that are necessary for SAP installation should have been created in a step during the LDAP server setup. A script will have been created to create the SID folder structure, if necessary.


27

Pre-SAP Installation Steps

control1# ff_setup_sid_folder.sh Check the folder structure as described in the SAP installation guide. You may use ssh X server for the Sapinst GUI. $DISPLAY should also be adequate, like localhost:10.0. The X connection can be tested with xclock. RX600-01:~ # xclock An XCLOCK should pop up on your display. If not, check the DISPLAY variable or the.Xauthority file. Check the JAVA JDK: server:~ # java -version java version "1.4.2_11" Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_11-b04) Java HotSpot(TM) Client VM (build 1.4.2_11-b04, mixed mode) Check the JAVA environment on Linux: server:~ # echo $JAVA_HOME /usr/java/ Set the JAVA environment on Solaris: server:~ # echo $JAVA_HOME server:~ # JAVA_HOME=/usr/j2sdk/ Create the SAP installation directory: server:/ # mkdir /usr/sap//install server:/ # chmod 777 /usr/sap//install server:/ # cd /usr/sap//install Setup the following variables, before starting the installation: server:/ # export JAVA_HOME=/usr/java/ server:/ # export SAPINST_JRE_HOME=/usr/java/ server:/ # export TMP=

28


Pre-SAP Installation Steps

How to configure the interfaces manually: Solaris (client LAN: 10.1.7.106 server LAN: 20.1.7.106): server:/ # ifconfig fjgi2001:2 plumb 10.1.7.106 netmask 0xffffff00 broadcast 10.1.7.255 up server:/ # ifconfig fjgi2002:2 plumb 20.1.7.106 netmask 0xffffff00 broadcast 20.1.7.255 up Linux (client LAN: 10.1.7.101 server LAN: 20.1.7.106): server:/ # /sbin/ifconfig vlan2001:0 10.1.7.101 broadcast 10.1.7.255 netmask 0xffffff00 server:/ # /sbin/ifconfig vlan2002:0 20.1.7.101 broadcast 20.1.7.255 netmask 0xffffff00

6.2

SAN Configuration

Before installing a SAP system you have to prepare the SAPDATA and SAPLOG concerning the SAN requirements. Please refer to chapter Creating Volumes and Filesystems in the Administration and Operation Guide. Before the database instance is installed you have to execute ff_san_mount.sh pre sapdb SID start to make the mountpoints accessible. Important: Especially for MaxDB database installations there is a special reference needed to address the data and log volumes. If you are asked for datafile directories you have to enter /sapdb//sapdata for data volumes and /sapdb//saplog for log volumes. The access rights have to be set to the database user and database group.


29

7

Operating System in General

The following sections describe the installation of Java SDK (JDK). If you have installed an JDK in an old Application Node image and you change to a new image you have to install the previous JDK in the new image in the same way. You can find the installation path and version of JDK in the instance profile entry jstartup/jvm/home=.

7.17.1.1

Solaris 8/9Integrating New Java SDK (JDK) in Solaris

This section describes how a new SUN Java SDK (JDK) can be integrated in a Solaris Boot Image using the nb_get_j2sdk script. Starting a maintenance run is not required. It runs independently of the customary FlexFrame procedure and the commands used there on the Control Node. The script works interactively. For legal reasons the EULA from the SUN Java SDK archive must be displayed and acknowledged. The script does not belong to the basic functionality of "Netboot for Solaris" but is a project-specific script. It is therefore supplied in the SMAWnbpw-FF4mySAP package and normally packed to /opt/SMAW/SMAWnbpw/bin/nb_get_j2sdk. Because of legal problems it is not possible to distribute one or more versions of the SUN Java SDK unpacked within our Solaris Boot Images. Nevertheless this script is provided as a simple way of making one or more versions of the JDK available in parallel in a Solaris Boot Image. With version 12.3 of nb_get_j2sdk it is possible to install the "Java Cryptography Extension" (JCE) too. Now it is possible to make the same SUN Java SDK version available in parallel as a version without JCE and as a version which includes JCE in the same Solaris Boot Image. The versions will be distinguished by the directory name. After the script has been called, the new SUN Java SDK version is available in the /usr/j2sdk/ directory (e.g. /usr/j2sdk/j2sdk1.4.2_10) or /usr/j2sdk/_jce (e.g. /usr/j2sdk/j2sdk1.4.2_10_jce) if JCE is installed too on all Application Nodes which were booted from the Boot Image specified in the script call. Setting the environment variable JAVA_HOME to this path then enables this SDK version to be used by applications. The script has to be called once for each SUN Java SDK version which is to be stored, and also for each Boot Image in which a version is to be stored.


31


1.

The current J2SE SDK is loaded from http://java.sun.com/j2se/1.4.2/download.html or an older version of the SDK is loaded from http://java.sun.com/products/archive/index.html The following files should always be fetched here: Solaris SPARC Platform: 32-bit self-extracting file and 64-bit self-extracting file j2sdk-1_4_2_11-solaris-sparc.sh j2sdk-1_4_2_11-solaris-sparcv9.sh

Example:

If JCE should be installed too, the matching Java Cryptography Extension zip file must be loaded from http://java.sun.com/j2se/1.4.2/download.html Section Other Downloads and must be put beside the two self-extracting files mentioned above. 2. The script requires the SUN Java SDK version which is to be distributed to be locally available on the Control Node. The two files are located adjacently on the Control Node. In addition, an active Solaris Application Node which can be addressed via the Storage LAN is available temporarily for unpacking these files. Call parameters of the script: nb_get_j2sdk -s -a -i -p -s Complete path name of one of the two SUN Java SDK Solaris SPARC Platform self-extracting files on the Control Node (e.g. /tmp/j2sdk-1_4_2_11-solaris-sparcv9.sh) -a Name of the active Solaris Application Node in the Storage LAN which can be used temporarily for unpacking the self-extracting files (e.g. AN2-st) -i Complete path name of the Boot Image in which the SUN Java SDK version is to be stored in the /usr part, (e.g. /FlexFrame/volFF/os/Solaris/ FSC_5.8_202_20050211/bi_FJSV,GPUZC-M_PW-P) -p requests the unpacking of the JCE package. The installation directory is named /usr/j2sdk/_jce.

3.

32



Files: The script generates or modifies the /usr/j2sdk/ files in the Boot Image. Actions: The script performs the following actions: Check of the call parameter -s and its argument (SDK archive) A check is made to see if an absolute path was specified and all the required files exist. The name of the directory in which the data is stored automatically when it is unpacked from the SDK archive is determined. The name must be ascertained using the javahome pattern from the archive and is designated as in this text.

Check of the call parameter -a and its argument (Application Node) ping is used to check whether the Application Node can be addressed. The root file system of this Application Node is determined via the entry in /etc/bootparams. The SDK archive is later unpacked into this root file system under /JDK_PACK.

Check of the call parameter -i and its argument (Image) A check is made to see if an absolute path was specified, if the directory does exist on the Control Node and if it has a correct structure. If it does not yet exist, the j2sdk directory is created in the /usr part of this Boot Image. Checks if the SUN Java SDK version does exist already . If a SUN Java SDK version exists, it is not overwritten as it may already be in use by one of the Application Nodes.

The two SUN Java SDK archive files and the JCE archive file, if available, are now copied to /JDK_PACK in the root file system of the Application Node which is to be used temporarily. Afterwards, the unpacking of the two archive files on the Application Node is triggered by two ssh calls. Both times the EULA of the SUN Java SDK is displayed and must be acknowledged with yes. The unpacking of the JCE archive file - if available - on the Application Node is triggered by a ssh call. Afterwards, the unpacked files from the JCE archive are copied into the just before unpacked SUN Java SDK to jre/lib/security.


33


The original files in this directory will be saved by moving to _orig. The new files from the JCE archive will be copied to _jce and then linked to the "right" name. So it is possible to identify the orginal SUN Java SDK files and the new JCE files. Subsequently the /JDK_PACK/ resp. j2sdk/_jce (if JCE was installed too) directory which was generated during unpacking is moved to j2sdk/ in the /usr part of the Boot Image selected. Finally, a cleanup takes place on the Application Node that, among other things, deletes the /JDK_PACK directory .

7.2

LinuxIn case the startup of the lsnrctl should fail during the SAP installation on Oracle, please check section listener.ora on page 38.

Relevant SAP Note for installation on Linux If you want to install an SAP R/3 system on Linux, refer to SAP Note 171356. SAP advice to use the most recent kernel Please be advised to use one of the most recent 2.6 or 2.4 kernels for your hardware platform and distribution. In case of problems with the operating system, please try to reproduce it on a tested kernel and contact the support organization of your Linux distributor if the problem persists. (www.sap.com/linux/platforms) Errors while executing shell scripts The execution of shell scripts starting with : (instead of #!/bin/sh) causes errors or core dumps due to the Linux kernel. Try to solve this problem by rebooting your host or installing a new Linux kernel (see SAP Note 359566). Due to the fact that the root image of Linux is read-only, please check the existence of the following links and directory: control1# cd /FlexFrame/volFF/os/Linux/FSC_3.2/root_img appnode1# mkdir /oracle/ora-config/ORCLfmap

34



Create the DB instance installation directory: BLADE1:~ BLADE1:~ BLADE1:~ BLADE1:~ BLADE1:~ # # # # # mkdir /usr/sap/C11/install cd /usr/sap/C11/install mkdir C11DB chmod 777 C11DB cd C11DB

7.2.1

Integrating a new Java SDK (JDK) in Linux SLES8 / SLES9 32bit

This section describes how to integrate a new SUN Java SDK (JDK) into a Linux Boot Image on the FlexFrame environment without going through a maintenance cycle. The current J2SE SDK can be downloaded from http://java.sun.com/j2se/1.4.2/download.html An older version of the SDK can be found at http://java.sun.com/products/archive/index.html Example for an Application Node (IP address 172.16.2.19): install .rpm Mount root fs as rw: Export the root image as rw for only the maintenance Application Node: control1:/FlexFrame//vol0/etc # vi exports > /vol/volFF/os///root_img > -ro=172.16.1.0/24:172.16.2.0/24,anon=0,rw=172.16.2.19 control1:~ # rsh filer1_qa exportfs -a Do not forget to export the root image as ro after the maintenance! RX300-01:/ # remount rw /


35


Make sure that the RPM does not contain files for the var image! RX300-01:/ # rpm -qilp .rpm | less > search for "/var" install the RPM: RX300-01:/ # rpm -i .rpm remount the root file system read only: RX300-01:/ # remount ro / control1:/FlexFrame//vol0/etc # vi exports modify the entry containing the line: AN 172.16.2.19 "rw": #/vol/volFF/os///root_imgro=172.16.1.0/24:172.16.2.0/24,anon=0,rw=172.16.2.19 /vol/volFF/os///root_img -ro=172.16.1.0 /24:172.16.2.0/24,anon=0 control1:~ # rsh filer1_qa exportfs a

7.2.2

Integrating New Java SDK (JDK) in SLES9_x86

This section describes how to integrate a new IBM Java SDK (JDK) into a Linux Boot Image on the FlexFrame environment without going through a maintenance cycle. The IBM SDK can be downloaded from: https://www14.software.ibm.com/webapp/iwm/web/preLogin.do?source=j avasap To install the IBM Java, refer to the section Integrating a new Java SDK (JDK) in Linux SLES8 / SLES9 32bit on how to proceed the installation.

36


88.1

Database in GeneralInstallation of Oracle in GeneralInstalling more then one Oracle databases on a host: In a FlexFrame environment, Oracle binaries are installed to a shared filesystem. Thus you have to read the following SAP notes before installing: SAP note 98252 (Listener), 350251 (oraInventory) and 521230 (9i) or 819829 (10g) (Oracle client software) Before you start installing Oracle, make sure that all the required directories have been created. If the following error occurs during installation, check the mount option of all Oracle directories. nolock: ORA-01102: cannot mount the database in EXCLUSIVE mode On Solaris it has to be llock whereas on Linux it has to be nolock. Oracle Parameter: After successful installation of the oracle software, please check the value of the following parameters (init.ora / spfile.ora): disk_asynch_io=FALSE filesystemio_options=directio Before starting the Oracle RUNINSTALLER (runInstaller for patches or opatch for interim patches) please check the value of the environment variable ORACLE_BASE=/oracle/ o for the ora user (.dbenv*.*sh).

8.1.1

TNS Listener for Oracle

Virtualization of SAP Services means that it is possible to run any Oracle instance on any Application Node. As a result, the listener task (start and stop) is performed automatically by the FlexFrame sapdb script . Each TNS listener can have its own listener name. In the /FlexFrame/scripts/ ora_listener_names configuration file, an individual listener name can be assigned to each . If the entry for an is omitted, the standard listener name LISTENER is valid. Example for /FlexFrame/scripts/ora_listener_names: #: C11:MY_LISTENER C12:YOUR_THING P10:LISTENER_P10 P12:P12


37

Database in General

Technical background This configuration file for the SAP Service start and stop scripts is /FlexFrame/scripts/sapdb. sapdb starts the TNS listener in accordance with these entries. To avoid TNS errors, please do not forget to configure these individual listener names in listener.ora, too. Oracle 10 only LSNRCTL: Check if the correct host name is used in the following files: /oracle//10[1-2]_64/network/admin/lsnrctl.ora /oracle//10[1-2]_64/network/admin/tnsnames.ora /sapmnt//profile/oracle/tnsnames.ora

8.1.2

listener.oraEvery database instance must have its own listener.ora. The relevant parameter is HOST in the section LISTENER. The example below is valid for Solaris only. On a Linux platform the IP address of db-se has to be entered. This also applies for tnsnames.ora. Otherwise the startup of the lsnrctl will fail with Segmentation error.

Example for a virtual DB host: LISTENER = (ADDRESS_LIST = .. .. (ADDRESS = (COMMUNITY = SAP.WORLD) (PROTOCOL = TCP) (HOST = db-se) (PORT = 1527) ) )

# Virtual host of the listener

38


Database in General

Example for the ADDRESS_LIST section: LISTENER = (ADDRESS_LIST = (ADDRESS= (PROTOCOL=IPC) (KEY= .WORLD) ) (ADDRESS= (PROTOCOL=IPC) (KEY= ) ) Example for the SID_LIST section: (SID_LIST = (SID_DESC = (SDU = 32768) (SID_NAME = ) (ORACLE_HOME = /oracle//_) # Must be available per automount )

8.1.3

Host Name Changes in "tnsnames.ora"The physical node name must not be entered here as HOST. The virtual node name has to be entered here; the database is not tied to a physical node, but only to a virtual node. The example below is valid only for Solaris. On a Linux platform the IP address of db-se has to be entered. Otherwise the startup of the lsnrctl will fail with Segmentation error.


39

Database in General

Example: .WORLD= (DESCRIPTION = (SDU = 32768) (ADDRESS_LIST = (ADDRESS = (COMMUNITY = SAP.WORLD) (PROTOCOL = TCP) (HOST = db-se) (PORT = 1527) ) ) (CONNECT_DATA = (sid = ) (GLOBAL_NAME = .WORLD) ) )

# Virtual host of the database

8.2

Installation of Oracle 9.2During the Oracle DB installation, a password for ora has to be set. You may type in any password you want to, it has no effect. Only ff_scripts can change a user password. Linux Installation of Oracle patch 9.2.0.7: Check the existence of the following directory: /oracle/ora-config/ORCLfmap Solaris Before you start the Oracle installation, create the following link and directory: mkdir /oracle/ora-config/ORCLfmap After the installation has been completed, make sure to import the latest patch set (read the SAP Notes 539921, 539922, 355776, 509314, and 540021).

The installation procedure will ask for /oracle//920_ (Linux) or /oracle//920_64 (Solaris) and /oracle/client. All directories will be configured automatically with LDAP. They are also part of the /etc/auto.master file. Start installation as described in the SAP installation manual.

40


Database in General

8.2.1

Post-Oracle 9.2 Installation StepsAfter installation, execute root.sh and use the parameter /oracle/ instead of the default value (/usr/local/bin).

BLADE1:/oracle/C11/920_ # ./root.sh Running Oracle9 root.sh script... The following environment variables are set as: ORACLE_OWNER= oraC11 ORACLE_HOME= /oracle/C11/920_ Enter the full pathname of the local bin directory: [/usr/local/bin]: /oracle/C11 Copying dbhome to /oracle/C11 ... Copying oraenv to /oracle/C11 ... Copying coraenv to /oracle/C11 ... Adding entry to /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root.sh script. Now product-specific root actions will be performed.

8.2.2

Patch Installation

Please have a look to corresponding SAP-Sites to get the newest Patches. Before continuing SAP-Installation install the patches as described in corresponding readme files (either patch contains own runinstaller or patch is installed with Oracle's opatch command interface).

8.3

Installation of Oracle 10.1 (R1)Linux Installation: Check the existence of the following directory: /oracle/ora-config/oracle LSNRCTL: Check if the correct host name is used in the following files: /oracle//101_/network/admin/lsnrctl.ora /oracle//101_/network/admin/tnsnames.ora /sapmnt//profile/oracle/tnsnames.ora


41

Database in General

8.3.1

Script root.shAfter installation, execute root.sh and use the parameter /oracle/ instead of the default value (/usr/local/bin).

Example: SID=OLP BLADE1:/oracle/OLP/101_64 # ./root.sh Running Oracle10 root.sh script... The following environment variables are set as: ORACLE_OWNER= oraolp ORACLE_HOME= /oracle/OLP/101_64 Enter the full pathname of the local bin directory: [/usr/bin]: /oracle/OLP Copying dbhome to /oracle/OLP ... Copying oraenv to /oracle/OLP ... Copying coraenv to /oracle/OLP ... Adding entry to /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root.sh script. Now product-specific root actions will be performed. Successfully accumulated necessary OCR keys. Creating OCR keys for user 'root', privgrp 'root'.. Operation successful. Oracle Cluster Registry for cluster has been initialized /bin/cp: cannot create regular file `/etc/init.d/init.cssd': Read-only file system /bin/ln: creating symbolic link `/etc/rc.d/rc2.d/S96init.cssd' to `/etc/init.d/init.cssd': Read-only file system The deamon cssd is only necessary for ORACLE RAC. You can ignore the error message regarding Read-only file system. Solaris: The full pathname of the local bin directory is also /oracle/. However you will not receive such an error message.

42


Database in General

8.3.2

Patch Installation

Please have a look to corresponding SAP-Sites to get the newest Patches. Before continuing SAP-Installation install the patches as described in corresponding readme files (either patch contains own runinstaller or patch is installed with Oracle's opatch command interface).

8.4

Installation of Oracle 10.2 (R2)After installation, execute root.sh and use the parameter /oracle/ instead of the default value (/usr/local/bin). LSNRCTL: Check if the correct host name is used in the following files: /oracle//102_/network/admin/lsnrctl.ora /oracle//102_/network/admin/tnsnames.ora /sapmnt//profile/oracle/tnsnames.ora

8.5

Installation of SAPDB / MaxDBFor SAP DB 7.3 NetApp ONTAP Version 7.0.1R1P1 or higher GD release is required. SAPDB 7.3: If you receive the following error messages, the process for calculating the database space could not check the available space on NFS mounted file systems: Sum of all datadevspaces to small for the SAP System Please change the value of the parameter MIN_DB_SIZE in the DATABASE.R3S file: Section: [ADMINDBSIZE_IND_ADA] MIN_DB_SIZE=15000 cd /FlexFrame/volFF/os/Linux/FSC3.2xxx/root_img/usr/java control1:~#> cp -rp mount -t nfs filerxxxx:/........var-xxxxxxxx/ /usr/java/ (e.g. = j2sdk1.4.2_12) Now the JAVA files are writeable.


49

SAP in General

If the installation has finished successfully, do the following: appnode1:~#> umount /usr/java/ control1:~#> cd ..../var_img/var-xxxxxxxx/ control1:~#> cp -rp ...os/Linux/FSC3.2xxx/root_img/usr/java / (e.g. = j2sdk1.4.2_12)

9.2.2

Solaris

control1:~#> cd /FlexFrame/volFF/os/Solaris//bi..../Solaris_x/usr_sparc.all/usr/j2sdk control1:~#> cp -rp ../../../../root // appnode1:~#> mount -F nfs filerxxxx:/vol/volFF/os/Solaris //bixxxxxxxx/root/ \/ /usr/j2sdk/ Now the JAVA files are writeable. If the installation has finished succesfully, do the following: appnode1:~#> umount /usr/java/ control1:~#> cd /FlexFrame/volFF/os/Solaris//bi.... /Solaris_x/root/ control1:~#> cp -rp /FlexFrame/volFF/os/Solaris //bi..../Solaris_x/usr_sparc.all/usr/j2sdk

50


10

Installing SAP 4.6C SR2Before starting with the Installation of SAP 4.6C SR2 on FlexFrame, please refer to the original SAP installation guides. Please also get the applicable SAP notes from the SAP Marketplace. R3SETUP has a problem with NFS mounted file systems. NetApp has a script which is helpful to avoid such problems. The script must be edited. The lines which refers to chown -R root:root /oracle and chmod -R 777 /oracle must be deleted. Possible Error Message: 4.6C SR2 SAPCAR aborts with error 36: staging area is not empty. Solution: Delete all files and subdirectories in directory /oracle/stage/920_32. Please check the permissions of the users adm, ora and sqd.

A workaround must be observed if SAP is installed on the NetApp Filer using R3SETUP (up to release 4.6C). This is described in the following technical report: SAPDB http://www.netapp.com/library/tr/3143.pdf

R3SETUP does not yet permit installation of SAP on NFS drives in this version. If sapinst is used for installation (release 4.7 and above or Web AS 6.20 or 6.40), installation on the NetApp Filer with NFS is directly supported.


51

Installing SAP 4.6C SR2

10.1

SAPDB / MaxDB / Oracle

During the installation of SAP 4.6CSR2, the R3SETUP will fail several times due to the fact that the necessary directories, links and ownerships have already been created in FlexFrame 3.2. The current *.R3S file (e.g. CENTRAL.R3S or DATABASE.R3S) has to be edited and R3SETUP needs to be restarted.

The requested file system is not mounted automatically. To mount the file system, issue the command cd /sapmnt//exe and repeat the R3SETUP process until the next error occurs.

52



Edit the *.R3S file and change STATUS=ERROR to STATUS=OK.

Restart R3SETUP.


53


Edit the *.R3S file and change STATUS=ERROR to STATUS=OK.

Restart R3SETUP.

54



Edit the *.R3S file and set the parameter BREAK_ON_NFS_PERMISSION_ERROR=NO.

Restart R3SETUP.


55

11

Installing Web AS 6.20Before starting with the Installation of an SAP Solution (e.g. SAP R/3 Enterprise 4.7x2.00) on FlexFrame, please refer to the original SAP installation guides and also get the applicable SAP notes from the SAP Marketplace. Some new SAP Solutions (e.g. Solution Manager 3.2) are based on Web AS 6.20 but they contain the executable sapinst Version 642 from NetWeaver04 SR1. In that case we suggest using the option SAPINST_USE_HOSTNAME. See also chapter Installing Web AS 6.40 with NetWeaver04 SR1 on page 65. Change to the SAP installation directory and start INSTALL/SAPINST. Use the physical host name of the installation server when you are asked for a host name of the CI or DB. After finishing the SAP installation, perform the postinstallation steps. See also chapter Post-SAP Installation Steps on page 109. During the installation, the file /etc/services will be copied under /etc/services.x. This will fail because the root file system is mounted readonly. In that case, stop the installation and search for the string ERROR in the keydb.xml. Change the entry to OK and restart the installation. Please check the permissions of the users adm, ora, sqd and sdb. Starting SAPINST directly from the ssh X session will cause SAPINST to abort.

Use the physical host name of the installation server when you are asked for a host name of the CI or DB. After finishing the SAP installation, perform the post-installation steps (see chapter Post-SAP Installation Steps on page 109).

11.1

Installating Web AS 6.20 with sapinst 642

If you are not sure to have version 642 of the executable sapinst, run the command sapinst v; the output has to contain the string 642. Otherwise you have to continue with the installation using the physical host name.


57

Installing Web AS 6.20

11.1.1 Installing the Central Instance (CI)In case of a virtual installation the virtual interface must be up. Find out the IP address of the virtual hostname ciosm: # getent hosts ciosm 10.1.7.106 ciosm -ciosm Set the Client LAN interface up: Use the active interface which is not deprecated and always has a virtual interface running, e.g. fjgi2002001:1. Solaris: # ifconfig fjgi2002001:2 plumb 10.1.7.106 netmask 0xffffff00 broadcast 10.1.7.255 up Linux: # /sbin/ifconfig vlan2001:0 10.1.6.101 broadcast 10.1.6.255 netmask 255.255.255.0 baby_3:~ baby_3:~ baby_3:~ baby_3:~ # # # # cd /usr/sap/OSM/install mkdir OSMCI chmod 777 OSMCI cd OSMCI

# ..../sapinst -extract Extracting in /usr/sap/OSM/install/OSMCI... sapinst: ok iacdlib642.so: ok iaccdlib.so: ok iaejs642.so: ok iakdblib642.so: ok ... iarfclib642.so: ok Done # SAPINST_JRE_HOME=/usr/j2sdk/j2sdk1.4.2_10 # export SAPINST_JRE_HOME # ./sapinst SAPINST_USE_HOSTNAME=ciosm cd /usr/sap//install_SCS appnode1:~#> /sapinst extract appnode1:~#> ./sapinst SAPINST_USE_HOSTNAME= /product_ha.catalog


67

Installing Web AS 6.40 with NetWeaver04 SR1

68



Type in the virtual DB host name


69


70



12.2.2 DBMake sure that the IP interface address of the is up and running (e. g. ifconfig plumb up). appnode1:~#> cd /usr/sap//install_DB appnode1:~#> /sapinst extract appnode1:~#> ./sapinst SAPINST_USE_HOSTNAME= /product_ha.catalog


71


If you install a J2EE engine using the product_ha.catalog, and during the database installation step you see a dialog asking for an installation drive, stop the installation and insert the following three lines to your control.xml after var client = new ClientMapping(); var ids = ["ciHostIsUnix", "sapmntPath"]; var globalHostSID; -if (parseBool(installer.getGlobalParameter("SeparateSCS")) && (context.getBool("ci") || context .getBool("db"))) Insert: if (installer.onWindows() && parseBool(installer.getGlobalParameter("SeparateSCS")) && (context.getBool("ci") || context.getBool("db"))) { Now, restart SAPinst.

12.2.3 JCMake sure that the IP interface address of the is up and running (e. g. ifconfig plumb up). appnode1:~#> cd /usr/sap//install_JC appnode1:~#> /sapinst extract appnode1:~#> ./sapinst SAPINST_USE_HOSTNAME= /product_ha.catalog

12.2.4 JMake sure that the IP interface address of the is up and running (e. g. ifconfig plumb up). appnode1:~#> cd /usr/sap//install_J appnode1:~#> /sapinst extract appnode1:~#> ./sapinst SAPINST_USE_HOSTNAME= /product_ha.catalog

72



12.2.5 Web AS 6.40 Deployment SP16After installing JAVA or JAVA-AddIn using the NW04 SR1 DVD/CD, proceed as follows: 12.2.5.1 JC

appnode1:~#> cd /usr/sap//install_SP appnode1:~#> /sapinst extract appnode1:~#> ./sapinst SAPINST_USE_HOSTNAME= product.catalog 12.2.5.2 J

appnode1:~#> cd /usr/sap//install_SP appnode1:~#> /sapinst extract appnode1:~#> ./sapinst SAPINST_USE_HOSTNAME= product.catalog

12.3

Web AS 6.40 NetWeaver04SR1 ABAP OracleStart the installation using virtual host names. There is no difference between Linux/Solaris for the installation process.

12.3.1 CIMake sure that the IP interface address of the is up and running. appnode1:~#> cd /usr/sap//install_CI appnode1:~#> /sapinst extract appnode1:~#> ./sapinst SAPINST_USE_HOSTNAME= /product.catalog


73


74




75


Type in the virtual DB host name

76



12.3.2 DBMake sure that the IP interface address of the is up and running. appnode1:~#> cd /usr/sap//install_DB appnode1:~#> /sapinst extract appnode1:~#> ./sapinst SAPINST_USE_HOSTNAME= /product.catalog After successful loading the database, the automatic startup of SAP may fail sometimes. Please start the SAP manually Stop the SAPINST Modify the keydb.xml (search for ERROR in the section StartSAPSystem) Change the Status from ERROR to OK and startup the SAPINST


77


12.3.3 APPMake sure that the IP interface address of the is up and running. appnode1:~#> cd /usr/sap//install_APP appnode1:~#> /sapinst extract appnode1:~#> ./sapinst SAPINST_USE_HOSTNAME= /product.catalog

12.4

Web AS 6.40 ABAP + JAVA-AddIn NetWeaver04SR1

12.4.1 CIMake sure that the IP interface address of the is up and running. appnode1:~#> cd /usr/sap//install_CI appnode1:~#> /sapinst extract appnode1:~#> ./sapinst SAPINST_USE_HOSTNAME= /product.catalog

12.4.2 DBMake sure that the IP interface address of the is up and running. appnode1:~#> cd /usr/sap//install_DB appnode1:~#> /sapinst extract appnode1:~#> ./sapinst SAPINST_USE_HOSTNAME= /product.catalog

78



12.4.3 SCSMake sure that the IP interface address of the , DB and CI are up and running (in Web AS 6.40 you have to use the ). Before starting the installation procedure, add the sapms entry manually in the /etc/services (do not forget to remove the entry after the installation!). When installing on Oracle, there is one further option Distributed System, that allows to install the database schema and the JAVA part separately: Java Database Schema Installation: appnode1:~#> cd /usr/sap//install appnode1:~#> /sapinst extract appnode1:~#> ./sapinst SAPINST_USE_HOSTNAME= /product.catalog Central Instance Finalization: appnode1:~#> cd /usr/sap//install appnode1:~#> /sapinst extract appnode1:~#> ./sapinst SAPINST_USE_HOSTNAME= /product.catalog The example below shows an installation on MaxDB. appnode1:~#> cd /usr/sap//install appnode1:~#> /sapinst extract appnode1:~#> ./sapinst SAPINST_USE_HOSTNAME= /product.catalog


79


80



Type in the virtual CI host name


81


When the installation has finished successfully, you have to change the hostname of the SCS instance manually. If the installation of the SCS Installation aborts at the step Load Java Database Content (step 19) with the error message no connection to network adapter, please proceed as follows: Stop the Oracle listener process. Change the virtual host name db-se to the real host name (uname a) in the listener.ora and tnsnames.ora. Start the Oracle listener process and click the Retry button from the SAPINST GUI.

82



Check and modify the following: Copy the .sapenv_ci.csh file to .sapenv_scs.csh (also for the .sh file). Move the instance/start profile to the correct hostname. Exchange the hostname in the instance/start profile. Go to section 14.1 in this document to change the hostname for the SCS instance.

12.4.4 Web AS 6.40 Deployment SP1612.4.4.1 DVEBMGS (JAVA-AddIN)

appnode1:~#> cd /usr/sap//install_SPxx appnode1:~#> /sapinst extract appnode1:~#> ./sapinst SAPINST_USE_HOSTNAME= product.catalog If the installation of the SCS Installation aborts at the step Deploy JDDI (step 19) with the error message no connection to network adapter, please proceed as follows: Stop the Oracle listener process. Change the virtual host name db-se to the real host name (uname a) in the listener.ora and tnsnames.ora. Start the Oracle listener process and click the Retry button from the SAPINST GUI.

12.5

Web AS 6.40 Application Server JAVA-AddIn

We assume that all required components (DB, ABAP CI, JAVA-AddIn and SCS) have already been installed. An /etc/services entry for is necessary for the installation only You have already installed a ABAP DI Instance Use the CD/DVD of NW04 SR1 or higher

Start up the installation as follows: 1. Extract the sapinst in the installation directory (.../sapinst extract)

appnode1:~#> ./sapinst SAPINST_USE_HOSTNAME= /product.catalog


83


2.

The following screen appears:

3. 4.

Choose OK The instance number is the same as you have chosen for the ABAP DI

Make sure that you have deleted the /etc/services entry of your installed system before starting the APP.

84


13

Installing NetWeaver 7.0SR1/SR2Linux: During the installation, the file /etc/services will be copied under /etc/services.x. This will fail because the root file system is mounted readonly. In that case, stop the installation and search for the string ERROR in the keydb.xml. Change the entry to OK and restart the installation. For every SAP solution you have to follow the instructions of section 0.

During installation of any SAP solution the SAP or DB instance did not start up. In this case use the FlexFrame start scripts and start the instance manually, (e.g.): ERROR 2006-05-17 10:29:35 CJSlibModule::writeError_impl() CJS-20022 Could not start instance 'JC41' of SAP system OLP. ERROR 2006-05-17 10:29:35 CJSlibModule::writeError_impl() CJS-20022 Could not start instance 'JC41' of SAP system Before starting the installation of a SAP solution you should take into account that with NetWeaver 7.0/NetWeaver 7.0SR1/SR2 a new operating system group sapinst is requested. You need a group entry in LDAP database (pool specific) adapted to your FF environment like the following (example): dn:cn=sapinst,ou=Group,ou=p1,ou=Pools,ou=FlexFrame,dc=ff32,dc=wdf, dc=fsc,dc=net objectClass: top objectClass: posixGroup cn: sapinst userPassword:: eA== gidNumber: 3606 memberUid: orasm2 memberUid: sm2adm Please insert this entry before you start with the installation. During the upgrade from FlexFrame V3.1A to V4.0 this new group is not inserted automatically. Instruction how the entry can be brought into LDAP: LDIFIN=/tmp/ldapin rm $LDIFIN echo "dn: cn=sapinst,ou=Group,ou=pool1,ou=Pools,ou=FlexFrame, dc=jer,dc=fujitsu-siemens,dc=com" >> $LDIFIN


85

Installing NetWeaver 7.0SR1/SR2

echo echo echo echo echo

"objectClass: top" >> $LDIFIN "objectClass: posixGroup" >> $LDIFIN "gidNumber: 1234" >> $LDIFIN "userPassword: leer" >> $LDIFIN "cn: sapinst" >> $LDIFIN

ldapmodify -x -D"cn=root,ou=LDAPadmins,ou=FlexFrame, dc=jer,dc=fujitsu-siemens,dc=com" -w -f $LDIFIN If you upgrade from a version less than 3.2B we recommend the following commands (example for pool pool1): ff_sid_adm.pl --op add --pool pool1 --sid DMY --sapversion 4.6 --db ORACLE9:dbdmy-se:\* --sap ci:25:cidmy:\*:cidmy-se:\* ff_setup_sid_folder.sh pool1 DMY ff_sap_upgrade.pl --pool pool1 --sid dmy --sapvo 4.6 --sapvn 7.0 --dbvo ORACLE9 --dbvn ORACLE10 ff_sid_adm.pl --op del --