Upload
agcristi
View
897
Download
0
Embed Size (px)
DESCRIPTION
Citation preview
HP Integrity SAP ERP White Paper
Integrated Solutions Deployment Lab
Revision 1.0
Executive Summary......................................................................................................................... 3 Terms and Definitions................................................................................................................... 3
SAP Solution Overview.................................................................................................................... 4 Hardware Components ................................................................................................................ 4
Database Servers .................................................................................................................... 4 Application Servers.................................................................................................................. 4 DNS Server ............................................................................................................................ 4
Software Components.................................................................................................................. 4 Hardware Topology .................................................................................................................... 4 Application Environment............................................................................................................... 6 High Availability Environment........................................................................................................ 6 SAP R/3 Compliance Testing ........................................................................................................ 6
Test Activities .......................................................................................................................... 6 Test Results ............................................................................................................................. 7
High Availability Testing............................................................................................................... 7 Test Activities .......................................................................................................................... 7 Test Results ............................................................................................................................. 7
HP OpenView Storage Data Protector Testing .................................................................................. 7 Test Activities .......................................................................................................................... 7 Test Results ............................................................................................................................. 8
Functional Summary / Recommendations............................................................................................ 8 Appendix A – Detailed Hardware Environment.................................................................................... 9
Integrity Superdome Partitions ....................................................................................................... 9 SAP GUI Server ........................................................................................................................ 10 Data Protector Cell Manager Server ............................................................................................. 10
Appendix B - Network Topology ..................................................................................................... 11 Appendix C – Detailed Software Environment.................................................................................... 12
Database Servers...................................................................................................................... 12 Application Servers ................................................................................................................... 13 DNS Server.............................................................................................................................. 15 SAP GUI Server ........................................................................................................................ 15 HP OpenView Storage Data Protector Cell Manager Server ............................................................. 15
Appendix D - Integrity Superdome Database and Application Server Configuration ................................. 17 Database Server Configuration.................................................................................................... 17
HP-UX Configuration and Tunes ............................................................................................... 17 SAP R/3 Application Server Configuration .................................................................................... 18
HP-UX Configuration and Tunes ............................................................................................... 18 DNS Server Configuration .......................................................................................................... 19
Windows Configuration & Tunes.............................................................................................. 19 DNS Configuration ................................................................................................................ 19
Appendix E – Patches.................................................................................................................... 20 Appendix F – Problems and Workarounds........................................................................................ 20 Figure 1: Hardware Topology ................................................................................................................... 5 Figure 2: Partition and IO Layout............................................................................................................. 10 Figure 3: Subnet Layouts .......................................................................................................................... 11 Table 1: Partition Definitions ...................................................................................................................... 9 Table 2: Database Server Software......................................................................................................... 12 Table 3: Application Server Software...................................................................................................... 13 Table 4: DNS Server Software................................................................................................................. 15 Table 5: SAP GUI Server Software.......................................................................................................... 15 Table 6: Data Protector Cell Manager Server Software ......................................................................... 15 Table 7: Database Server File System Layout.......................................................................................... 17 Table 8: Database Server Kernel Tunable Parameters............................................................................ 18 Table 9: Application Server File System Layout ...................................................................................... 18 Table 10: Application Server Kernel Tunable Parameters...................................................................... 19 Table 11: HP OpenView Storage Data Protector Patches on Cell Manager Server and SAP DB/CI
Server................................................................................................................................................. 20
3
Executive Summary The purpose of this white paper is to assist sales representatives and solution architects with understanding, implementing, and consolidating ERP solutions using multi-partition HP Integrity platforms. By leveraging the work that HP performed in implementing a multi-tiered SAP R/3 application, customers can reduce their deployment time, which improves the Total Customer Experience for their HP-UX 11i HP Integrity implementation. Customer Focused Testing reduces deployment time by validating and documenting all major hardware and software elements in a given configuration.
This document demonstrates to customers and HP field personnel that this configuration information has been successfully validated in the test environment. Please note that while these configurations have been effectively validated, they have not been optimized for specific customer workloads and may therefore need further refinement.
Terms and Definitions • ABAP – SAP’s native configuration language • DB/CI – Database and Central Instance for SAP • Enqueue – The server that controls the lock request to the SAP Central Instance • lgtst – SAP Logon Group Test Tool • niping – An SAP utility to validate the TCP/IP configuration • SICK – SAP transaction verifying Installation and consistency check in ABAP dictionary • SGeSAP – Serviceguard Extension for SAP
4
SAP Solution Overview
Hardware Components Database Servers Two Integrity Superdome hard partitions were running SAP DB/CI and Oracle 9.2.0.1.0 in a SGeSAP cluster configuration. A HP StorageWorks Virtual Array 7410 was used for shared database storage. A HP Surestore disk system SC10 was used for the operating system.
Application Servers Two Integrity Superdome hard partitions were running SAP R/3 Enterprise 4.7 SR1.
DNS Server One Integrity Superdome hard partition was running the Windows 2003 64-bit Data Center Edition. A HP StorageWorks disk system 2110 was used as a boot device.
Software Components The software stack was comprised of HP ISU products and ISV software components. SAP was installed along with the high availability Serviceguard Extension for SAP (SGeSAP). When incorporated together, the following software components provide a highly available SAP/ERP application environment:
• HP-UX 11i v2 Operating Environments • HP Hardware Partitions (nPars) • HP C/ANSI C Developer ‘s Bundle • HP aC++ Compiler • Java2 1.3 SDK for HPUX • HP Serviceguard • HP Serviceguard Extension for SAP (SGeSAP) • HP OpenView Storage Data Protector • SAP R/3 Enterprise 4.7 • Oracle 9i Database
Hardware Topology In order to perform a server consolidation proof of concept, a 3-tier SAP R/3 installation was implemented using hard partitions on a single Integrity Superdome complex. An additional partition on the Integrity Superdome was used as a Windows-based DNS server. See Figure 1, below.
5
Figure 1: Hardware Topology
S APApplication
Servers
O racleD B
Servers
C lient
DN S
SuperD om enPar Partition
6
Application Environment The application stack was used to validate the Serviceguard failover, installation, configuration and basic functionality of SAP. The SAP R/3 compliance test was used to verify that the application in the software stack was working correctly. The HP OpenView Storage Data Protector backup and recovery product was also tested.
High Availability Environment At the database tier, the Serviceguard Extension for SAP (SGeSAP) was integrated with Oracle 9i on two Integrity Superdome partitions to provide high-availability for SAP and Oracle processes. The SAP and Oracle data files were stored on external disk arrays that were accessible from both the primary and secondary DB/CI servers. A dual path between the external storage and the database servers eliminated the single point of failure between the disks and systems. Serviceguard ensured that processes would failover whenever the primary server system became unavailable.
SAP R/3 Compliance Testing Test Activities The SAP compliance test consists of running SAP transactions through the SAPGUI client front-end. The test validates the application functionality at all three levels: the client, the application, and the database layer. It also verifies the network functionality between all of the SAP servers.
• Checking SAP Instance Processes This test verifies that all SAP R/3 processes are running.
• Basic Network Functions and Configuration This test validates the network configuration of the SAP R/3 network. The niping utility is used to verify network connectivity between the host machines.
• Load Balanced SAP Front Logon This test checks that the SAP “load balanced front logon” functionality is working properly. When a user connects as part of a logon group, one of the servers from the list of reachable application servers is selected by the load balance mechanism to handle the session.
• Checking Reachable Application Servers The lgtst tool is used to access and check the message server list on the Central Instance (CI), which keeps a list of application servers that can be reached within the system.
• Checking Address Entries of the Local Gateway The gateway service on the CI keeps a list of addresses that refers to the local host machine. This test checks the number of entries currently held by the gateway and determines whether all local addresses are included in the address list.
• Checking Consistency The SAP defined transaction SICK compares the kernel, database, and operating system release to determine whether certain tables exist and are consistent in the ABAP Dictionary.
• Checking the Message and Gateway Services This test verifies the message and gateway services are functioning properly. All the SAP R/3 instances of a SAP R/3 system should be displayed and the work processes should be viewable.
7
• Checking Logical Remote Function Call Destination These transactions check to see if the Remote Function Call destination is reachable
• System Wide Remote Function Call Connection This transaction uses remote function calls to all running SAP R/3 instances in order to show all of the users who are currently logged on to the entire SAP R/3 environment.
• Enqueue and Update Services The enqueue service locks objects at the SAP level using an enqueue table. This table is held in the Central Instance. In order to check communication with the enqueue service inside the system, this test accesses the enqueue service from a non-Central Instance.
Test Results The compliance tests passed with no errors.
High Availability Testing Test Activities In order to verify the high availability characteristics of SAP, Oracle, and SGeSAP, a failover test was performed. The Serviceguard cluster was brought up and the package was started on the primary DB/CI server. After validating that all of the SAP and Oracle processes were up and functioning correctly, a compliance test was run on the active DB/CI server. Next, a Serviceguard planned failover was performed on the cluster which migrated the application package from the primary node to the secondary node. Once again, a compliance test was performed on the active DB/CI server to verify application functionality. Finally, the Serviceguard package was migrated back to the primary node. All SAP and Oracle processes were verified and a compliance test was completed.
Test Results All SAP and Oracle processes were running correctly on the active node after the planned failover and each compliance test completed without any errors.
HP OpenView Storage Data Protector Testing Test Activities The HP OpenView Storage Data Protector Client software was locally installed on each clustered SAP R/3 database node. The client software components include Data Protector’s disk agent for file system backup, user interface for command-line access, and SAP R/3 integration for linking the SAP R/3 backup utilities (BRTOOLS) with Data Protector. The Data Protector Manager GUI was used to import the hostname of the application cluster package to the Data Protector Cell Manager so that backups could continue in the event of an application cluster package failover. The SAP R/3 integration was then configured using the hostname of the application cluster package. The validation testing sequence consisted of the following:
• Local installation of the client software to the clustered SAP R/3 database nodes • Importing the hostname of the application cluster package to the Data Protector server • Integration of the SAP R/3 backup utilities (BRTOOLS) with Data Protector • Online backup using both the Data Protector Manager GUI and the client command-line interface • Restoration of the SAP R/3 data file, using the Data Protector Manager GUI • Recovery of the database, using the SAP R/3 (BRRECOVER) utility • Online backup after an SAP R/3 application fail-over to the secondary database node in the cluster
8
Test Results As described in the test activities, this testing was intended to uncover any inconsistencies with the product usage specifications. The overall test results were successful. However, a documentation error was discovered due to a SAP product change. A detail of this discovery is presented in Appendix F.
Functional Summary / Recommendations The overall results of the tests indicate that multi-tiered SAP R/3 application environments can be successfully implemented and consolidated on multiple partitions of a single Integrity Superdome complex. The SAP R/3 application, Oracle database, networking and backup software all functioned as expected in this highly available configuration. In addition, the Windows Data Center Edition DNS server ran in a separate partition on the same Integrity Superdome complex. This demonstrates that HP-UX and Windows operating systems can run independently and simultaneously in their designated partitions within the same Integrity Superdome.
Accordingly, the Integrated Solutions Deployment Lab makes the following recommendations:
• Consider using Workload Manager. This product dynamically moves CPU resources between partitions as required to achieve the service level objective of SAP application. Again, this ensures higher levels of server utilization and increased agility since server resources are automatically and effectively adjusted on demand to meet changing business priorities and application usage levels.
• Consider the Integrity Superdome as well as other smaller cell architecture Integrity servers for server consolidation efforts.
• Follow the standard HP high availability implementation guidelines. • Follow the standard Oracle performance tuning process for each database server. • Follow the standard SAP performance tuning process for the application servers. • Consider using the Instant-Capacity-on-Demand (iCOD) and Pay-Per-Use (PPU) products. The ability
to pay only for the actual processors used could significantly reduce the Cost of Ownership, while leaving “head room” in the system available for rapid deployment. Also, the presence of the available processors could enhance system reliability since the iCOD software will automatically switch to a new CPU in the event that an error is detected in a currently used CPU.
9
Appendix A – Detailed Hardware Environment
Integrity Superdome Partitions A 16 cell Integrity Superdome with two IO expanders was used. Each of the cells was connected to its own IO chassis. Redundant IO paths, particularly on the database partitions, were implemented to maximize fault tolerance. To maximize availability and fault isolation, the replicated partitions (database servers, application servers) were defined in different cabinets. To optimize performance, all of the cells for any given partition were kept within a single cross-plane.
Table 1: Partition Definitions
Partition Cell # CPUs
Interleaved Memory (GB)
Cell Local Memory (GB)
0 (0/0)1 4 11 1
1 (0/1) 4 11 1
2 (0/2) 4 11 1
0 / Primary DB/CI Server
3 (0/3) 4 11 1
8 (1/0) 4 11 1
9 (1/1) 4 11 1
10 (1/2)
4 11 1
1 / Secondary DB/CI Server
11 (1/3)
4 11 1
4 (0/4) 4 11 1 2 / First Application Server
5 (0/5) 4 11 1
12 (1/4)
4 11 1 3 / Second Application Server
13 (1/5)
4 11 1
4 / Windows 2003 Data Center Edition DNS Server
7 (0/7)2 4 11 1
6 (0/6) 4 NA NA
14 (1/6)
4 NA NA
Unused cells
15 (1/7)
4 NA NA
1 This is the local format, which refers to “cabinet 0, cell 0”. See parcreate(1M). 2 This cell was selected because its IO chassis was closer to the peripheral device rack where the system console was housed.
10
Figure 2: Partition and IO Layout
ABC
LVD / SEDS21x0
DS2100
ABC
LVD / SEDS21x0
Page Title:SAP / ERP Sweet Spot
Date Revised:29 June, 2004
9:05 AM
Page:1 of 1
Author:Craig Fuget
Description:SAP / ERP Sweet Spot Test -Orca Server ConsolidationConfiguration. See: http://odin.rsn.hp.com/dscgi/ds.py/Get/File-8697/
Legend:®
i n v e n t
Cell 0
Cell 1
Cell 2
Cell 3
Cell4
Cell5
Cell6
Cell 7
BulkPS0
I/O Bay 0 - Front
BulkPS1 BulkPS2 BulkPS3 BulkPS4 BulkPS5
PDCA: Power Source 0 PDCA: Power Source 1
Cell 0I/O
Chas1
Cell 4I/O
Chas3
I/O Bay 1 - Rear
Cell 6I/O
Chas1
Cell 2I/O
Chas3
L
R
R
L
CPU 0CPU 3CPU 1CPU 2
RAM
CPU 0CPU 3CPU 1CPU 2
RAM
CPU 0CPU 3CPU 1CPU 2
RAM
CPU 0CPU 3CPU 1CPU 2
RAM
CPU 0CPU 3CPU 1CPU 2
RAM
CPU 0CPU 3CPU 1CPU 2
RAM
CPU 0CPU 3CPU 1CPU 2
RAM
CPU 0CPU 3CPU 1CPU 2
RAM
6 MBCache
6 MBCache
6 MBCache
Itanium 2 1.5GHz6 MBCache
Itanium 2 1.5GHz
Itanium 2 1.5GHz
Itanium 2 1.5GHz 6 MBCache
6 MBCache
6 MBCache
Itanium 2 1.5GHz6 MBCache
Itanium 2 1.5GHz
Itanium 2 1.5GHz
Itanium 2 1.5 GHz 6 MBCache
6 MBCache
6 MBCache
Itanium 2 1.5GHz6 MBCache
Itanium 2 1.5GHz
Itanium 2 1.5GHz
Itanium 2 1.5GHz
6 MBCache
6 MBCache
6 MBCache
Itanium 2 1.5GHz6 MBCache
Itanium 2 1.5GHz
Itanium 2 1.5GHz
Itanium 2 1.5GHz6 MBCache
6 MBCache
6 MBCache
Itanium 2 1.5GHz6 MBCache
Itanium 2 1.5GHz
Itanium 2 1.5GHz
Itanium 2 1.5GHz 6 MBCache
6 MBCache
6 MBCache
Itanium 2 1.5GHz6 MBCache
Itanium 2 1.5GHz
Itanium 2 1.5GHz
Itanium 2 1.5GHz
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
4X PCI Slot 64X PCI Slot 72X PCI Slot 82X PCI Slot 9
2X PCI Slot 02X PCI Slot 12X PCI Slot 22X PCI Slot 34X PCI Slot 44X PCI Slot 5
2X PCI Slot 102X PCI Slot 11
4X PCI Slot 64X PCI Slot 72X PCI Slot 82X PCI Slot 9
2X PCI Slot 02X PCI Slot 12X PCI Slot 22X PCI Slot 34X PCI Slot 44X PCI Slot 5
2X PCI Slot 102X PCI Slot 11
4X PCI Slot 64X PCI Slot 72X PCI Slot 82X PCI Slot 9
2X PCI Slot 02X PCI Slot 12X PCI Slot 22X PCI Slot 34X PCI Slot 44X PCI Slot 5
2X PCI Slot 102X PCI Slot 11
4X PCI Slot 64X PCI Slot 72X PCI Slot 82X PCI Slot 9
2X PCI Slot 02X PCI Slot 12X PCI Slot 22X PCI Slot 34X PCI Slot 44X PCI Slot 5
2X PCI Slot 102X PCI Slot 11
Core I/ORS-232
10BT
Cell 0
Cell 1
Cell 2
Cell 3
Cell4
Cell5
Cell6
Cell 7
BulkPS0
I/O Bay 0 - Front
BulkPS1 BulkPS2 BulkPS3 BulkPS4 BulkPS5
PDCA: Power Source 0 PDCA: Power Source 1
Cell 8I/O
Chas1
Cell 12I/O
Chas3
I/O Bay 1 - Rear
Cell 14I/O
Chas1
Cell 10I/O
Chas3
L
R
R
L
CPU 0CPU 3CPU 1CPU 2
RAM
CPU 0CPU 3CPU 1CPU 2
RAM
CPU 0CPU 3CPU 1CPU 2
RAM
CPU 0CPU 3CPU 1CPU 2
RAM
CPU 0CPU 3CPU 1CPU 2
RAM
CPU 0CPU 3CPU 1CPU 2
RAM
CPU 0CPU 3CPU 1CPU 2
RAM
CPU 0CPU 3CPU 1CPU 2
RAM
6 MBC ache
6 MBC ache
6 MBC ache
I tanium 2 1.5GHz6 MB
C ache
I tanium 2 1.5GHz
I tanium 2 1.5GHz
I tanium 2 1.5GHz 6 MBCache
6 MBCache
6 MBCache
Itanium 2 1.5GHz6 MBCache
Itanium 2 1.5GHz
Itanium 2 1.5GHz
Itanium 2 1.5GHz 6 MBCache
6 MBCache
6 MBCache
Itanium 2 1.5GHz6 MBCache
Itanium 2 1.5GHz
Itanium 2 1.5GHz
Itanium 2 1.5GHz 6 MBCache
6 MBCache
6 MBCache
Itanium 2 1.5GHz6 MBCache
Itanium 2 1.5GHz
Itanium 2 1.5GHz
Itanium 2 1.5GHz
6 MBCache
6 MBC ache
6 MBC ache
Itanium 2 1.5GHz6 MB
C ache
Itanium 2 1.5GHz
Itanium 2 1.5GHz
Itanium 2 1.5GHz6 MBCache
6 MBCache
6 MBCache
Itanium 2 1.5GHz6 MBCache
Itanium 2 1.5GHz
Itanium 2 1.5GHz
Itanium 2 1.5GHz6 MB
C ache
6 MBC ache
6 MBC ache
I tanium 2 1.5GHz6 MB
C ache
I tanium 2 1.5GHz
Itanium 2 1.5MHz
I tanium 2 1.5GHz6 MBCache
6 MBCache
6 MBCache
Itanium 2 1.5GHz6 MBCache
Itanium 2 1.5GHz
Itanium 2 1.5GHz
Itanium 2 1.5GHz
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
4X PCI Slot 64X PCI Slot 72X PCI Slot 82X PCI Slot 9
2X PCI Slot 02X PCI Slot 12X PCI Slot 22X PCI Slot 34X PCI Slot 44X PCI Slot 5
2X PCI Slot 102X PCI Slot 11
4X PCI Slot 64X PCI Slot 72X PCI Slot 82X PCI Slot 9
2X PCI Slot 02X PCI Slot 12X PCI Slot 22X PCI Slot 34X PCI Slot 44X PCI Slot 5
2X PCI Slot 102X PCI Slot 11
4X PCI Slot 64X PCI Slot 72X PCI Slot 82X PCI Slot 9
2X PCI Slot 02X PCI Slot 12X PCI Slot 22X PCI Slot 34X PCI Slot 44X PCI Slot 5
2X PCI Slot 102X PCI Slot 11
4X PCI Slot 64X PCI Slot 72X PCI Slot 82X PCI Slot 9
2X PCI Slot 02X PCI Slot 12X PCI Slot 22X PCI Slot 34X PCI Slot 44X PCI Slot 5
2X PCI Slot 102X PCI Slot 11
Core I/O R S - 2 3 2
1 0 B T
Cabinet 0 Cabinet 1
1000 Base T 1000 Base T
1000 Base T
U2-LVD
2G Fibre Ch
U2-LVD
LVD SCSI
CPU 0CPU 3CPU 1CPU 2
RAM2GB
2GB
2GB
2GB
2GB
2GB
2GB
2GB
6 MBCache
6 MBCache
6 MBCache
Itanium 2 1.5GHz6 MBCache
Itanium 2 1.5 GHz
Itanium 2 1.5GHz
Itanium 2 1.5GHz
6 MBCache
6 MBCache
6 MBCache
Itanium 2 1.5GHz6 MBCache
Itanium 2 1.5GHz
Itanium 2 1.5GHz
Itanium 2 1.5GHz
U2-LVD
U2-LVD
VGA
11
0
0
11
11
011
0
011
0
11 0
110
11
Core I/ORS-232
10BT
2G Fibre Ch
1000 BaseSX
U2-LVD
Core I/ORS-2 32
10BT
10/100 TX-4 10/100 TX-4
10/100 TX-4 10/100 TX-4
Core I/ORS-2 32
1 0BT
Core I/ORS-23 2
1 0BT
LVD SCSI1000 Base T1000 BaseSX
BulkPS0
I/O Bay 0 - Front
BulkPS1 BulkPS2 BulkPS3 BulkPS4 BulkPS5
PDCA: Power Source 0 PDCA: Power Source 1
Cell 3I/O
Chas1
Cell 1I/O
Chas3
I/O Bay 1 - Rear
Cell 6I/O
Chas1
Cell 2I/O
Chas3
L
R
R
L
4X PCI Slot 64X PCI Slot 72X PCI Slot 82X PCI Slot 9
2X PCI Slot 02X PCI Slot 12X PCI Slot 22X PCI Slot 34X PCI Slot 44X PCI Slot 5
2X PCI Slot 102X PCI Slot 11
4X PCI Slot 64X PCI Slot 72X PCI Slot 82X PCI Slot 9
2X PCI Slot 02X PCI Slot 12X PCI Slot 22X PCI Slot 34X PCI Slot 44X PCI Slot 5
2X PCI Slot 102X PCI Slot 11
4X PCI Slot 64X PCI Slot 72X PCI Slot 82X PCI Slot 9
2X PCI Slot 02X PCI Slot 12X PCI Slot 22X PCI Slot 34X PCI Slot 44X PCI Slot 5
2X PCI Slot 102X PCI Slot 11
4X PCI Slot 64X PCI Slot 72X PCI Slot 82X PCI Slot 9
2X PCI Slot 02X PCI Slot 12X PCI Slot 22X PCI Slot 34X PCI Slot 44X PCI Slot 5
2X PCI Slot 102X PCI Slot 11
BulkPS0
I/O Bay 0 - Front
BulkPS1 BulkPS2 BulkPS3 BulkPS4 BulkPS5
PDCA: Power Source 0 PDCA: Power Source 1
Cell 8I/O
Chas1
Cell 12I/O
Chas3
I/O Bay 1 - Rear
Cell 14I/O
Chas1
Cell 10I/O
Chas3
L
R
R
L
4X PCI Slot 64X PCI Slot 72X PCI Slot 82X PCI Slot 9
2X PCI Slot 02X PCI Slot 12X PCI Slot 22X PCI Slot 34X PCI Slot 44X PCI Slot 5
2X PCI Slot 102X PCI Slot 11
4X PCI Slot 64X PCI Slot 72X PCI Slot 82X PCI Slot 9
2X PCI Slot 02X PCI Slot 12X PCI Slot 22X PCI Slot 34X PCI Slot 44X PCI Slot 5
2X PCI Slot 102X PCI Slot 11
4X PCI Slot 64X PCI Slot 72X PCI Slot 82X PCI Slot 9
2X PCI Slot 02X PCI Slot 12X PCI Slot 22X PCI Slot 34X PCI Slot 44X PCI Slot 5
2X PCI Slot 102X PCI Slot 11
4X PCI Slot 64X PCI Slot 72X PCI Slot 82X PCI Slot 9
2X PCI Slot 02X PCI Slot 12X PCI Slot 22X PCI Slot 34X PCI Slot 44X PCI Slot 5
2X PCI Slot 102X PCI Slot 11
Cabinet 8 Cabinet 9
U2-LVD
2G Fibre Ch
1000 Base T
11
0
0
11
11
011
0
011
0
11 0
110
11
10/100 TX-4Core I/ORS-232
10BT
U2-LVD
2G Fibre Ch
Core I/ORS-232
10 BT
1000 BaseSX
987654321 10 11 12 13 14 15
VA CC B
FCOut
Host
VA CC APS + Fan PS + FanVA7410 Array Main
FCOut
Host
SC10
Fan
PS
Bus Cont rol Card ASCSI SCSI
9 8 7 6 5 4 3 2 1 0
Fan
PS
Fib re Ch ann el Arbitr at ed Lo op Hub
10BaseT
Pow erFault 1 2 3 4 5 6 7 8 9 10
RS -232
Reset
Fibre C hanne l Arb itra ted Loop Hub
10B aseT
PowerFault 1 2 3 4 5 6 7 8 9 1 0
RS-232
R eset
SC 10
Fan
PS
Bus Control Card ASCSI SCSI
9 8 7 6 5 4 3 2 1 0
Fan
PS
SC10
Fan
PS
B us Cont rol Card ASCSI SCSI
9 8 7 6 5 4 3 2 1 0
Fan
PS
SC10
Fan
PS
Bus Control Card ASCSI SCSI
9 8 7 6 5 4 3 2 1 0
Fan
PS
SC10
Fan
PS
Bus Cont rol Card ASCSI SCSI
9 8 7 6 5 4 3 2 1 0
Fan
PS
SC10 #1
SC10
Fan
PS
Bus Control Card ASCSI SCSI
9 8 7 6 5 4 3 2 1 0
Fan
PS
SC10 #3
SC10 #4
SC10 #6 SC10 #2 SC10 #5
P rocur ve Rout ing Swi t ch 9304M
H P J4139A
Site LANSite LAN
Site LAN
Site LANSite LAN
Site LAN
Site LAN
DS2110hp sure store d vd
External DVD
Heartbeat
“Classic” PC
Fibre Channel MS10/100 Base-T1000 Base-SX1000 Base-TU2 SCSIU2 SCSI, to be cabled later
U320 SCSIVideo outPrimary DB Partition1st App. Server PartitionLinux Partition (NOT USED)2nd App. Server Partition
Failover DB PartitionWindows DNS Partition
Site LAN
SAP GUI Server The SAP GUI software was installed on an HP Kayak XU800 32-bit PC. No special peripherals were required, beyond having two 100 Base-T network connections.
Data Protector Cell Manager Server The Storage Data Protector Cell Manager Server software was installed on an HP9000 L2000 system. The system had a fibre channel connection to an HP SureStore 2/20 tape library and a 100 Base-T connection to the network.
11
Appendix B - Network Topology
Figure 3: Subnet Layouts
Managem
ent (Timeshare) S
ubnet
Heartbeat Subnet (192.168.255.x)
Relocatable IP192.168.16.22
SAP Server-1rua21-0315.22.226.173192.168.16.14
Windows DNSrua21-0515.22.226.175192.168.16.12
SAP Server-2rua21-0415.22.226.174192.168.16.13
Secondary_dbrua21-0215.22.226.172192.168.16.15192.168.255.15
RemoteDesktop15.x.x.x
SAP GUIrua21-0715.22.226.177192.168.16.10
Primary_dbrua21-0115.22.226.171192.168.16.16192.168.255.16
Private S
ubnet (192.168.16.x, mask 255.255.255.0)
12
Appendix C – Detailed Software Environment
Database Servers
Table 2: Database Server Software
Product Version Software Description
B3701AA C.03.71.23 HP GlancePlus/UX Pak for s800 11.23
B3901BA C.05.50 HP C/ANSI C Developer's Bundle (S800)
B3913DB C.05.50 HP aC++ Compiler (S800)
B5140BA A.11.23.01 MC/Serviceguard NFS Toolkit
T1905BA A.11.15.00 MC/ Serviceguard
B6834AA B.01.03.01 HP-UX Security Patch Check Tool
B6848BA 1.4.gm.46.3 Ximian GNOME 1.4 GTK+ Libraries for HP-UX 11iV1.6
B6849AA B.02.01.02 Bastille Security Hardening Tool
B8339BA B.03.00.03 servicecontrol manager Server and Agent Bundle
B8465BA A.01.05.05 HP WBEM Services for HP-UX
B9073BA B.06.00 HP-UX iCOD (Instant Capacity on Demand)
B9788AA 1.3.1.09.07 Java2 1.3 SDK for HP-UX
B9789AA 1.3.1.09.07 Java2 1.3 RTE for HP-UX
B9901AA A.03.05.06 HP IPFilter 3.5alpha5
Base-VXVM B.03.50.IA.003 Base VERITAS Volume Manager Bundle 3.5 for HP-UX
CDE-English B.11.23 English CDE Environment
CMDVIEWSDM A.1.07.00 hp StorageWorks Command View SDM
FDDI-00 B.11.23.00 PCI FDDI;Supptd HW=A3739B;SW=J3626AA
FibrChanl-00 B.11.23.01 PCI FibreChannel;Supptd HW=A6795A,A5158A
GigEther-00 B.11.23.01 PCI GigEther;Supptd HW=A4926A/A4929A/A6096A;SW=J1642AA
GigEther-01 B.11.23.01 PCI GigEther;Supptd HW=A6825A/A6794A/A6847A/A8685A/A9782A/A9784A/A7109A
HPUX11i-OE-MC B.11.23 HP-UX Mission Critical Operating Environment Component
HPUXBaseAux B.11.23 HP-UX Base OS Auxiliary
HPUXBaseOS B.11.23 HP-UX Base OS
IEther-00 B.11.23.01 PCI IEther;Supptd HW=A6974A
Judy B.11.11.04.15 Judy Library - development and runtime libraries for handling dynamic arrays
MOZILLA 1.2.1.03.00 Mozilla 1.2 for HP-UX
MySQL 3.23.54a.00 MySQL open-source database
NPar B.01.00 nPartition Provider - HP-UX
13
OnlineDiag B.11.23.01.21 HPUX 11.23 Support Tools Bundle, Jun 2003
ParMgr B.11.23.01.00 Partition Manager - HP-UX
Sec00Tools B.01.00.03 Install-Time security infrastructure.
T1456AA 1.4.1.03.01 Java2 1.4 SDK for HP-UX
T1457AA 1.4.1.03.01 Java2 1.4 RTE for HP-UX
T1471AA A.03.10.007 HP-UX Secure Shell
T2357BA B.03.09 MC/Service Guard Extension for SAP
USB-00 B.11.23 Object Oriented USB Driver
hpuxwsApache B.1.0.06.01 HP-UX Apache-based Web Server
hpuxwsTomcat B.1.0.06.01 HP-UX Tomcat-based Servlet Engine
hpuxwsWebmin A.1.0.06.01 HP-UX Webmin-based Admin
hpuxwsXml A.1.0.06.01 HP-UX XML Web Server Tools
perl D.5.8.0.A Perl Programming Language
scsiU320-00 B.11.23 PCI SCSI U320; Supptd HW=CoreIO
Oracle RDBMS 9.2.0.1.0 Oracle 9i Database for IPF
SAP R/3 4.7 SRI SAP R/3 Enterprise
DATA-PROTECTOR
A.05.10 HP OpenView Storage Data Protector
Application Servers
Table 3: Application Server Software
Product Version Software Description
B3701AA C.03.71.23 HP GlancePlus/UX Pak for s800 11.23
B3901BA C.05.50 HP C/ANSI C Developer's Bundle (S800)
B3913DB C.05.50 HP aC++ Compiler (S800)
B5140BA A.11.23.01 MC/Serviceguard NFS Toolkit
B6834AA B.01.03.01 HP-UX Security Patch Check Tool
B6848BA 1.4.gm.46.3 Ximian GNOME 1.4 GTK+ Libraries for HP-UX 11iV1.6
B6849AA B.02.01.02 Bastille Security Hardening Tool
B8339BA B.03.00.03 servicecontrol manager Server and Agent Bundle
B8465BA A.01.05.05 HP WBEM Services for HP-UX
B9073BA B.06.00 HP-UX iCOD (Instant Capacity on Demand)
B9788AA 1.3.1.09.07 Java2 1.3 SDK for HP-UX
B9789AA 1.3.1.09.07 Java2 1.3 RTE for HP-UX
B9901AA A.03.05.06 HP IPFilter 3.5alpha5
14
Base-VXVM B.03.50.IA.003 Base VERITAS Volume Manager Bundle 3.5 for HP-UX
CDE-English B.11.23 English CDE Environment
FDDI-00 B.11.23.00 PCI FDDI;Supptd HW=A3739B;SW=J3626AA
FibrChanl-00 B.11.23.01 PCI FibreChannel;Supptd HW=A6795A,A5158A
GigEther-00 B.11.23.01 PCI GigEther;Supptd HW=A4926A/A4929A/A6096A;SW=J1642AA
GigEther-01 B.11.23.01 PCI GigEther;Supptd HW=A6825A/A6794A/A6847A/A8685A/A9782A/A9784A/A7109A
HPUX11i-OE-Ent B.11.23 HP-UX Enterprise Operating Environment Component
HPUXBaseAux B.11.23 HP-UX Base OS Auxiliary
HPUXBaseOS B.11.23 HP-UX Base OS
IEther-00 B.11.23.01 PCI IEther;Supptd HW=A6974A
Judy B.11.11.04.15 Judy Library - development and runtime libraries for handling dynamic arrays
MOZILLA 1.2.1.03.00 Mozilla 1.2 for HP-UX
MySQL 3.23.54a.00 MySQL open-source database
NPar B.01.00 nPartition Provider - HP-UX
OnlineDiag B.11.23.01.21 HPUX 11.23 Support Tools Bundle, Jun 2003
ParMgr B.11.23.01.00 Partition Manager - HP-UX
Sec00Tools B.01.00.03 Install-Time security infrastructure.
T1456AA 1.4.1.03.01 Java2 1.4 SDK for HP-UX
T1457AA 1.4.1.03.01 Java2 1.4 RTE for HP-UX
T1471AA A.03.10.007 HP-UX Secure Shell
T2357BA B.03.09 MC/Service Guard Extension for SAP
USB-00 B.11.23 Object Oriented USB Driver
hpuxwsApache B.1.0.06.01 HP-UX Apache-based Web Server
hpuxwsTomcat B.1.0.06.01 HP-UX Tomcat-based Servlet Engine
hpuxwsWebmin A.1.0.06.01 HP-UX Webmin-based Admin
hpuxwsXml A.1.0.06.01 HP-UX XML Web Server Tools
perl D.5.8.0.A Perl Programming Language
scsiU320-00 B.11.23 PCI SCSI U320; Supptd HW=CoreIO
Oracle RDBMS 9.2.0.1.0 Oracle 9i Database for IPF
SAP R/3 4.7 SRI SAP R/3 Enterprise
15
DNS Server
Table 4: DNS Server Software
Product Version Software Description
HP Data Center DTC.10B.ISO Windows 2003 Data Center Edition
SAP GUI Server
Table 5: SAP GUI Server Software
Version Software Description
NT Service Pack 5 Windows 2000
3.3.7 VNC
6.20 SAPGUI
HP OpenView Storage Data Protector Cell Manager Server
Table 6: Data Protector Cell Manager Server Software
Product Version Software Description
BUNDLE B.11.11 Patch Bundle
BUNDLE11i B.11.11.0102.2 Required Patch Bundle for HP-UX 11i, February 2001
Base-VXVM B.03.20.1 Base VERITAS Volume Manager 3.2 for HP-UX
CDE-English B.11.11 English CDE Environment
DATA-PROTECTOR A.05.10 HP OpenView Storage Data Protector
FCManagerSTAND 3.1.F Bundle of Fibre Channel Manager
FDDI-00 B.11.11.02 PCI FDDI;Supptd HW=A3739A/A3739B;SW=J3626AA
FDDI-02 B.11.11.01 HPPB FDDI;Supptd HW=J2157B;SW=J2658BA
FibrChanl-00 B.11.11.09 PCI/HSC FibreChannel;Supptd HW=A6684A,A6685A,A5158A,A6795A
GOLDAPPS11i B.11.11.0206.4 Gold Applications Patches for HP-UX 11i, June 2002
GOLDBASE11i B.11.11.0206.4 Gold Base Patches for HP-UX 11i, June 2002
GigEther-00 B.11.11.14 PCI/HSC GigEther;Supptd HW=A4926A/A4929A/A4924A/A4925A;SW=J1642AA
GigEther-01 B.11.11.04 PCI GigEther;Supptd HW=A6794A/A6825A/A6847A
HPBridgeManager A.01.00.0086 HP Bridge Manager for Enterprise Storage Mgmt
HPOVSAMAP 03.10.00.0126 HP OpenView Storage Area Manager Applications
HPOVSAMCA 03.10.04.0023 hp OpenView storage area manager Core Applications
16
HPOVSAMCP 03.10.04.0023 hp OpenView storage area manager Builder Application
HPOVSAMDA 03.10.04.0023 hp OpenView storage area manager HostAgent DIAL
HPOVSAMHA 03.10.04.0023 hp OpenView storage area manager HostAgent Framework
HPOVSAMHB 03.10.00.0126 hp OpenView storage area manager HostAgent HBA Gateway
HPOVSAMJR 1.4.1.3 hp OpenView storage area manager embedded JRE
HPOVSAMLM 03.10.00.0126 hp OpenView storage area manager Allocater Application
HPOVSAMPM 03.10.04.0023 hp OpenView storage area manager Optimizer Application
HPOVSAMSG 03.10.00.0126 hp OpenView storage area manager HostAgent SCSI Gateway
HPSCSIGateway A.01.00.0011 HP Bridge Manager for Enterprise Storage Mgmt
HPUX11i-OE B.11.11 HP-UX Internet Operating Environment Component
HPUXBase64 B.11.11 HP-UX 64-bit Base OS
HPUXBaseAux B.11.11.0206 HP-UX Base OS Auxiliary
HWEnable11i B.11.11.0206.5 Hardware Enablement Patches for HP-UX 11i, June 2002
ITOAgent A.06.00 ITO Agents for HP-UX 11.x English
IUX-Recovery B.3.6.82 Ignite-UX network recovery tool subset
J1608CA B.02.00.09 Netscape Fasttrack Server
MeasureWare C.02.65.00 MeasureWare Software/UX
MeasurementInt C.02.65.00 HP-UX Measurement Interface for 11i
OnlineDiag B.11.11.07.11 HPUX 11.11 Support Tools Bundle, Jun 2002
OnlineJFS B.11.11 Online features of the VxFS File System
RAID-00 B.11.11.01 PCI RAID; Supptd HW=A5856A
perl B.5.6.1.C Perl for HP-UX
17
Appendix D - Integrity Superdome Database and Application Server Configuration
Database Server Configuration HP-UX Configuration and Tunes
Table 7: Database Server File System Layout
File System Size
/ 720 MB
/stand 512 MB
/var 4600 MB
/usr 3000 MB
/tmp 720 MB
/opt 4600 MB
/home 256 MB
/oracle/client 280 MB
/oracle/S03 12000 MB
/oracle/S03/sapdata1 10016 MB
/oracle/S03/sapdata2 12416 MB
/oracle/S03/sapdata3 11616 MB
/oracle/S03/sapdata4 13600 MB
/oracle/S03/mirrlogA 320 MB
/oracle/S03/mirrlogB 320 MB
/oracle/S03/origlogA 320 MB
/oracle/S03/origlogB 320 MB
/oracle/S03/oraarch 2400 MB
/oracle/S03/saparch 800 MB
/oracle/S03/sapreorg 1600 MB
/oracle/stage 6400 MB
/sapmnt/S03 1600 MB
/usr/sap/trans 1600 MB
/usr/sap/S03 2016 MB
18
Table 8: Database Server Kernel Tunable Parameters
Tunable Value 11.23 Default
dbc_max_pct 8 50
dbc_min_pct 5 5
maxdsiz_64bit 17179869184 4294967296
maxuprc 400 256
ninode 8192 4880
shmmax 17179869184 1073741824
semvmx 32767 32767
semume 100 100
semmnu 256 256
semaem 16384 16384
msgmap 2048 1026
msgtql 2046 1024
msgmnb 65535 16384
msgseg 32767 8192
nstrpty 60 60
swchunk 8192 2048
SAP R/3 Application Server Configuration HP-UX Configuration and Tunes
Table 9: Application Server File System Layout
File System Size
/ 720 MB
/stand 512 MB
/var 4600 MB
/usr 3000 MB
/tmp 720 MB
/opt 4600 MB
/home 256 MB
/oracle/S03 1400 MB
19
/oracle/stage 1500 MB
/sapmnt/S03/exe 1228 MB
/usr/sap/S03 900 MB
Table 10: Application Server Kernel Tunable Parameters
Tunable Value 11.23 Default
dbc_max_pct 8 50
dbc_min_pct 5 5
maxdsiz_64bit 17179869184 4294967296
maxuprc 400 256
ninode 8192 4880
shmmax 17179869184 1073741824
semvmx 32767 32767
semume 100 100
semmnu 256 256
semaem 16384 16384
msgmap 2048 1026
msgtql 2046 1024
msgmnb 65535 16384
msgseg 32767 8192
nstrpty 60 60
DNS Server Configuration Windows Configuration & Tunes There were no special configuration changes or tunes for the Windows system, other than explicitly selecting and naming the network adapters.
DNS Configuration Windows 2003 Server no longer installs and enables Domain Name Services (DNS) by default. Configuration will have to be done manually.
20
Appendix E – Patches
Table 11: HP OpenView Storage Data Protector Patches on Cell Manager Server and SAP DB/CI Server
Patch Description Cell Manager
DB/CI Server
PHSS_29142 OV DP5.10 patch - EVAA packet
ü
PHSS_29143 OV DP5.10 patch - DOC packet
ü
PHSS_29414 OV DP5.10 patch - CC packet
ü ü
PHSS_29417 OV DP5.10 patch - MA packet
ü
PHSS_29423 OV DP5.10 patch - CM packet
ü
PHSS_29863 OV DP5.10 patch - ORACLE8 packet
ü
PHSS_29864 OV DP5.10 patch - SAP packet
ü ü
PHSS_29867 OV DP5.10 patch - CORE packet
ü ü
PHSS_29872 OV DP5.10 patch - SSEA packet
ü
Appendix F – Problems and Workarounds In order to restore a SAP R/3 data object, the “HP OpenView Storage Data Protector UNIX Integration Guide” provides examples using the BRRESTORE utility from SAPDBA. Using this method to perform the restore resulted in the following error:
SAPDBA: Bad format in detail BRBACKUP log file. (2004-xx-xx 17.21.04) Filename: "/oracle/S03/sapbackup/bdmdnmwr.pnf" Line : 41 Additional info: LOC_017:
Workaround: When the problem was reported to SAP, HP was told that SAP is no longer developing the SAPDBA functions and that SAPDBA will soon be obsolete. SAP strongly recommends that customers use the BRRECOVER functions rather than the equivalent SAPDBA functionsbecause BRRECOVER will always be fully up-to-date.
© 2004 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.