Reference Architecture: Oracle Database Real Application ... · 5.1 Server BOM ... two Oracle validated configurations for bare-metal deployment of Oracle RAC together with best practices

Reference Architecture: Oracle Database Real Application Clusters (RAC)

Describes reference architecture for Oracle RAC database

Contains three Oracle validated configurations for Lenovo hardware

Includes deployment details and best practices

Contains detailed bill of materials for servers, storage, and network switches

Mike Perks

Chandrakandh Mouleeswaran

Last update: 29 March 2019 Version 1.1

https://lenovopress.com/updatecheck/LP1073/90675dcbfe454dff3d9b649f9b5a75db

ii Reference Architecture: Oracle Database Real Application Clusters (RAC) Version 1.1

Table of Contents

1 Introduction ............................................................................................... 1

2 Business problem and business value ................................................... 2

2.1 Business problem .................................................................................................... 2

2.2 Business value ......................................................................................................... 2

3 Requirements ............................................................................................ 3

3.1 Functional requirements .......................................................................................... 3

3.2 Non-functional requirements .................................................................................... 3

4 Operational model .................................................................................... 4

4.1 Operational model scenarios ................................................................................... 4

4.2 Hardware components ............................................................................................. 4

4.3 Software components ............................................................................................ 10

4.4 Oracle validated configurations .............................................................................. 11

4.5 Best Practices ........................................................................................................ 17

4.6 Systems management ........................................................................................... 18

5 Appendix: Bill of Materials ..................................................................... 19

5.1 Server BOM ........................................................................................................... 19

5.2 Storage BOM ......................................................................................................... 22

5.3 Ethernet Networking BOM ..................................................................................... 23

5.4 SAN Networking BOM ........................................................................................... 23

5.5 Rack BOM.............................................................................................................. 23

Resources ..................................................................................................... 24

1 Reference Architecture: Oracle Database Real Application Clusters (RAC) Version 1.1

1 Introduction This document describes the reference architecture for the Oracle Database Real Application Clusters (RAC) using Lenovo® ThinkSystem servers, Lenovo DM series storage, and Lenovo switches for networking. The intended audience is IT professionals, technical architects, sales engineers, and consultants to assist in planning, designing, and implementing Oracle RAC.

This document provides an overview of the business problem and business value that is addressed by Oracle RAC followed by a description of customer requirements The operational model describes the architecture of two Oracle validated configurations for bare-metal deployment of Oracle RAC together with best practices for deploying into small to large Enterprises. The last section features detailed Bill of Materials configurations for Lenovo ThinkSystem servers, Lenovo DM series storage, and Lenovo network switches that are used in the solution.


2 Business problem and business value The following section provides a summary of the business problems that this reference architecture is intended to help address, and the value that this solution can provide.

2.1 Business problem As one of the fastest growing database platforms, Oracle database deployments are becoming increasingly critical to organizations. They are used in everything from departmental databases to business-critical workloads, including enterprise resource planning, customer relationship management and business intelligence. At the same time, enterprises are require highly scalable and highly available databases to allow for 24x7x365 continuous operation.

2.2 Business value Database performance has long been the primary criteria for selecting infrastructure. Multicore processors and large system memory capacity have now moved the performance conversation away from compute to the storage system. Storage solutions that support Oracle database need to handle a dynamic mix of transactional (OLTP) and analytical (OLAP) databases, along with their unique storage I/O profiles and active datasets. This requires efficiently delivering random and sequential read/write at high performance, across sizable amounts of active or hot data.

This solution utilizes Lenovo DM series storage array which provides an all-flash array to meet the demand for higher speed storage and provide higher IOPs and bandwidth with lower power usage and total cost of ownership than hybrid or HDD-based solutions.

This solution has been validated by Oracle and is listed in their list of published validated solutions: linux.oracle.com/pls/apex/f?p=102:1:::NO:::.

https://linux.oracle.com/pls/apex/f?p=102:1:::NO


3 Requirements This section descirbes the functional and non-functional requirements for this reference architecture.

3.1 Functional requirements Table 1 lists the functional requirements of a database management system (database) such as Oracle database 12c.

Table 1: Functional requirements

Requirement name Description

Stores any kind of data A database management system should be able to store any kind of data including binary such as images and video.

Support ACID Properties Database must support ACID (Accuracy, Completeness, Isolation, and Durability) properties.

Represents complex relationships between data

Database must represent the complex relationships between data to make the efficient and accurate use of data.

Database schema Database must provide a method to create and maintain the database schema using both GUI and command line.

Database operations Database must provide a method to submit SQL queries and return results using GUI, command line, and other interfaces.

Reporting Database should provide a method to generate formatted reports in various file formats, on-screen or printed

Multiple views Depending on role, users may see different views of the data. Concurrent use Database must be able to respond to multiple requests at a time

from multiple sources.

3.2 Non-functional requirements Table 2 lists the non-functional requirements that are needed for deployment.

Table 2: Non-functional requirements

Requirement name Description

Data integrity Integrity ensures the quality and reliability of database system High availability This critical part of IT infrastructure must always be available Disaster recovery Provide ability for secondary data center to take over if the primary

data center suffers a catastrophic failure and all components fail Scalability Solution components such as compute and storage capacity and

performance scale with an increase in number of concurrent users or transactions

Security Solution provides ways to secure data based on authorized role Ease of installation Reduced complexity of database deployment Ease of management/operations Simple management of infrastructure including support for rolling

upgrades of hardware and software Backup/Recovery Solution support for integrated backup


4 Operational model This section describes the operational model for Oracle Rapid Application Cluster (RAC) using Lenovo servers, storage, and network switches. The BOM configurations for the hardware are described in chapter 5.

4.1 Operational model scenarios This reference architecture presents three validated configurations that can be used for different size deployments. Table 3 gives an overview of these three models.

Table 3: Operational model scenarios

Small Enterprise Medium Enterprise Large Enterprise

Servers (min of 2) 2U SR650 2U SR850 4U SR950

Processors 2 x Intel Scalable 6126 4 x Intel Scalable 8158 8 x Intel Scalable 8158

System Memory 768 GB 1.5 TB 3 TB

SAN Storage DM5000F DM5000F DM7000F

Storage Drives 24 x 3.84TB SSD 24 x 3.84TB SSD 48 x 3.84TB SSD

Transaction Rate Up to 3 million TPM Up to 6 million TPM Up to 10 million TPM

4.2 Hardware components The following section describes the hardware components used in the Oracle RAC solution.

4.2.1 Servers The following Lenovo ThinkSystem servers can be used with the Oracle RAC solution.

Lenovo ThinkSystem SR630

Lenovo ThinkSystem SR630 (as shown in Figure 1) is an ideal 2-socket 1U rack server for small businesses up to large enterprises that need industry-leading reliability, management, and security, as well as maximizing performance and flexibility for future growth. The SR630 server is designed to handle a wide range of workloads, such as databases, virtualization and cloud computing, virtual desktop infrastructure (VDI), infrastructure security, systems management, enterprise applications, collaboration/email, streaming media, web, and HPC. It offers up to twelve 2.5-inch hot-swappable SAS/SATA HDDs or SSDs together with up to 4 on-board NVMe PCIe ports that allow direct connections to the U.2 NVMe PCIe SSDs.

Figure 1: Lenovo ThinkSystem SR630

For more information, see this website: lenovopress.com/lp0643

https://lenovopress.com/lp0643



Lenovo ThinkSystem SR650 (as shown in Figure 2) is similar to the SR630 but in a 2U form factor.


The key differences compared to the SR630 server are more expansion slots and chassis to support up to twenty-four 2.5-inch or fourteen 3.5-inch hot-swappable SAS/SATA HDDs or SSDs together with up to 8 on-board NVMe PCIe ports that allow direct connections to the U.2 NVMe PCIe SSDs. The SR650 server also supports up to two NVIDIA GRID cards for graphics acceleration.



Lenovo ThinkSystem SR850 (as shown in Figure 3) is a 4-socket server that features a streamlined 2U rack design that is optimized for price and performance, with best-in-class flexibility and expandability. Models of the SR850 are powered by up to four Intel Xeon Processor Scalable Family processors, each with up to 28 cores, for an efficient 4-socket solution.




Lenovo ThinkSystem SR950 (as shown in Figure 4) is 8-socket server with up to 96 DIMMs in a 4U rack space. It provides high density and most processing power and designed for mission critical applications. Models of the SR950 are powered by up to eight Gold or Platinum level Intel Xeon Processor Scalable Family processors, each with up to 28 cores.






4.2.2 10 GbE networking The following Lenovo 10 GbE top of rack (ToR) switch is recommended:

• Lenovo RackSwitch G8272

For more information about network switches, see this website: shop.lenovo.com/us/en/systems/networking/ethernet-rackswitch

Lenovo RackSwitch G8272

The Lenovo RackSwitch G8272 that uses 10 Gb SFP+ and 40 Gb QSFP+ Ethernet technology is specifically designed for the data center. It is ideal for today's big data, cloud, and optimized workload solutions. It is an enterprise class Layer 2 and Layer 3 full featured switch that delivers line-rate, high-bandwidth switching, filtering, and traffic queuing without delaying data. Large data center-grade buffers help keep traffic moving, while the hot-swap redundant power supplies and fans (along with numerous high-availability features) help provide high availability for business sensitive traffic.

The RackSwitch G8272 (shown in Figure 5), is ideal for latency sensitive applications, such as high-performance computing clusters and financial applications. In addition to the 10 Gb Ethernet (GbE) and 40 GbE connections, the G8272 can use 1 GbE connections. The G8272 supports the newest protocols, including Data Center Bridging/Converged Enhanced Ethernet (DCB/CEE) for Fibre Channel over Ethernet (FCoE), iSCSI and network-attached storage (NAS).

Figure 5: Lenovo RackSwitch G8272

For more information, see this website: lenovopress.com/tips1267

4.2.3 1 GbE network The following Lenovo 1GbE ToR switch is recommended for the solution management network:

• Lenovo RackSwitch G8052


http://shop.lenovo.com/us/en/systems/networking/ethernet-rackswitch

http://lenovopress.com/tips1267


Lenovo RackSwitch G8052

The Lenovo System Networking RackSwitch G8052 (as shown in Figure 6) is an Ethernet switch that is designed for the data center and provides a virtualized, cooler, and simpler network solution. The Lenovo RackSwitch G8052 offers up to 48 1 GbE ports and up to 4 10 GbE ports in a 1U footprint. The G8052 switch is always available for business-sensitive traffic by using redundant power supplies, fans, and numerous high-availability features.

Figure 6: Lenovo RackSwitch G8052

For more information, see this website: lenovopress.com/tips0813

4.2.4 SAN Storage The following Lenovo SAN storage can be used with the Oracle RAC solution.

Lenovo ThinkSystem DM5000F storage array

Lenovo ThinkSystem DM5000F is a unified, all flash entry-level storage system that is designed to provide performance, simplicity, capacity, security, and high availability for medium to large businesses. Powered by the ONTAP software, ThinkSystem DM5000F delivers enterprise-class storage management capabilities with a wide choice of host connectivity options and enhanced data management features. The ThinkSystem DM5000F is a perfect fit for a wide range of enterprise workloads, including big data and analytics, artificial intelligence, engineering and design, enterprise applications, and other storage I/O-intensive applications.

ThinkSystem DM5000F models (as shown in Figure 7) are 2U rack-mount controller enclosures that include two controllers, 64 GB RAM and 8 GB battery-backed NVRAM (32 GB RAM and 4 GB NVRAM per controller), and 24 SFF hot-swap drive bays (2U24 form factor). Controllers provide universal 1/10 GbE NAS/iSCSI or 8/16 Gb Fibre Channel (FC) ports for host connectivity.

A single ThinkSystem DM5000F Storage Array scales up to 144 solid-state drives (SSDs) with the attachment of Lenovo ThinkSystem DM240S 2U24 SFF Expansion Enclosures. Up to 12 DM5000F Storage Arrays can be combined into a clustered system in a NAS environment, or up to 6 DM5000F Storage Arrays can be combined into a clustered system in a SAN environment.

http://lenovopress.com/tips0813


Figure 7: Lenovo DM5000F storage array

The ThinkSystem DM5000F offers the following key features and benefits:

• All-flash array capabilities to meet the demand for higher speed storage and provide higher IOPs and bandwidth with lower power usage and total cost of ownership than hybrid or HDD-based solutions.

• Improved performance and data protection with RAID-DP and RAID-TEC, as well as support for traditional RAID 4.

• Flexible host connectivity to match diverse client needs with support for unified NAS and SAN storage protocols, including 1/10 GbE NAS and iSCSI, and 8/16 Gb Fibre Channel connectivity.

• 12 Gb SAS drive-side connectivity with multipathing with up to 24x 2.5-inch small form factor (SFF) drives in the 2U24 SFF enclosures.

• Rich set of standard storage management functions available at no extra cost, including snapshots, volume copy, quality of service, thin provisioning, compression, deduplication, encryption, disk-based backup, application- and virtual machine-aware backup, quick data recovery, and asynchronous mirroring.

• Optional WORM (write once, read many) (SnapLock) licensed function to reinforce permanence and integrity of stored data and to ensure compliance with applicable regulations.

• Intuitive, web-based GUI for easy system setup and management. • Lenovo XClarity support for centralized systems management of Lenovo x86 servers, switches,

and storage, which provides automated agent-less discovery, inventory, monitoring, and additional platform-specific functions across multiple systems.

• Designed for high availability with redundant hot-swap components, including controllers and I/O modules, power supplies, and non-disruptive firmware upgrades.

For more information, see this website: lenovopress.com/lp0911.

Lenovo ThinkSystem DM7000F storage array

Lenovo ThinkSystem DM7000F is a scalable, unified, all flash mid-range storage system that is designed to provide high performance, simplicity, capacity, security, and high availability for medium to large businesses. Powered by the ONTAP software, ThinkSystem DM7000F delivers enterprise-class storage management capabilities with a wide choice of host connectivity options, flexible drive configurations, and enhanced data management features, including support for NVMe over Fabrics. The ThinkSystem DM7000F is a perfect fit for a wide range of enterprise workloads, including big data and analytics, artificial intelligence, engineering and design, hybrid clouds, and other storage I/O-intensive applications.



ThinkSystem DM7000F models (as shown in Figure 8) are 3U rack-mount controller enclosures that include two controllers, and 256 GB RAM and 16 GB battery-backed NVRAM (128 GB RAM and 8 GB NVRAM per controller). Universal 1/10 GbE NAS/iSCSI or 4/8/16 Gb Fibre Channel (FC) ports and 1/10 GbE RJ-45 ports provide base host connectivity, with an option for additional 1/10 GbE or 40 GbE NAS/iSCSI, or 8/16/32 Gb FC connections with the adapter cards.

A single ThinkSystem DM7000F Storage Array scales up to 384 SFF solid-state drives (SSDs) with the attachment of Lenovo ThinkSystem DM240S 2U24 SFF Expansion Enclosures. Up to 12 DM7000F Storage Arrays can be combined into a clustered system in a NAS environment, or up to 6 DM7000F Storage Arrays can be combined into a clustered system in a SAN environment.

Figure 8: Lenovo DM7000F storage array

The ThinkSystem DM7000F offers the same key features and benefits as the DM5000F but with the following enhancements:

• Support for NVMe over Fabrics to help achieve up to two times higher performance at a half of the latency.

• Additional support for 40 GbE NAS and iSCSI, and 32 Gb Fibre Channel connectivity.


4.2.5 SAN switches The following Lenovo SAN switches for Fibre Channel can be used with the Oracle RAC solution.

Lenovo ThinkSystem DB610S

The Lenovo ThinkSystem DB610S FC SAN Switch provides exceptional price/performance value by delivering market-leading 32 Gb Gen 6 Fibre Channel technology and combining flexibility, simplicity, and enterprise-class functionality to meet the demands of growing flash-based storage environments.

Designed to enable maximum flexibility and reliability, the ThinkSystem DB610S is a compact, 1U rack-mount FC switch that offers low-cost access to industry-leading Storage Area Network (SAN) technology while providing “pay-as-you-grow” scalability to meet the needs of an evolving storage environment.



The DB610S FC SAN Switch (as shown in Figure 9) offers 24x SFP+ ports that support 4/8/16/32 Gbps speeds. The DB610S FC SAN switch provides easy integration into the existing SAN environments while realizing the benefits of Gen 6 Fibre Channel connectivity, and the switch offers a rich set of standard features with the options to expand its capabilities as needed.

The DB610S FC SAN Switch features the EZSwitch Setup wizard and can be configured in Access Gateway Mode to simplify deployment. The switch provides full non-blocking performance with Ports On Demand scalability to support SAN expansion and enable long-term investment protection.

Figure 9: Lenovo ThinkSystem DB610S SAN Switch


Lenovo ThinkSystem DB620S

The DB620S FC SAN Switch (as shown in Figure 10) is similar to the DB610S but offers 48x SFP+ ports that support 4/8/10/16/32 Gbps speeds and 4x QSFP+ ports that support 128 Gbps (4x 32 Gbps) or 4x 4/8/16/32 Gbps speeds.

Figure 10: Lenovo ThinkSystem DB620S SAN Switch


4.3 Software components The following software components are used in the solution.

4.3.1 Oracle Linux Oracle Linux 7 is used by the solution. It provides the latest innovations, tools, and features that enable you to innovate, collaborate, and create solutions across traditional, cloud-based, and virtual environments. Providing advanced scalability and reliability for enterprise applications and systems, Oracle Linux delivers truly open cloud with great performance. Oracle Linux is free to use, free to distribute, free to update, and easy to download.

Oracle Linux 7 offers a number of new features and improvements, most notably a new version of the Unbreakable Enterprise Kernel.




Oracle Linux customers can access a number of unique and highly useful features, including:

• The Ksplice offline client is a feature in our Ksplice service. Ksplice provides updates for your kernel with no downtime. Therefore, this eliminates the requirement for an internet connection for applying Ksplice patches.

• Support for the journaling file system (XFS): is now available for customers with Premier Support subscriptions using Oracle Linux 7.

• DTrace for Oracle Linux: DTrace for Oracle Linux is a comprehensive dynamic tracing framework available to Oracle Linux customers. Supported customers can download this from the Oracle Unbreakable Linux Network (ULN).

For more information, see this website: oracle.com/linux.

4.3.2 Oracle 12c database As organizations embrace the cloud, they seek technologies that will transform business and improve their overall operational agility and effectiveness. Oracle Database 12c is a next-generation database designed to meet these needs, providing a new multitenant architecture on top of a fast, scalable, reliable, and secure database platform. Oracle Database 12c also includes in-memory data processing capabilities for breakthrough analytical performance. By plugging into the cloud with Oracle Database 12c, customers can improve the quality and performance of applications, save time with maximum availability architecture and storage management, and simplify database consolidation by managing hundreds of databases as one.

For more information, see this website: docs.oracle.com/en/database/oracle/oracle-database/12.2/index.html.

4.4 Oracle validated configurations This section describes the Oracle validated configurations as described in oracle.com/technetwork/topics/linux/validated-configurations-085828.html

“The goal of this program is to validate hardware with Oracle Linux and Oracle VM, and to identify issues and configuration details that would impact customer deployments. Oracle developed the program to go far beyond the standard certification testing normally performed with operating systems. Hardware vendors participating in this program perform thorough testing of the hardware in real-world configurations with Oracle Linux and Oracle VM.

This is not a database certification. Oracle Database is a component of the environment because it is demanding on hardware resources (processor, memory, cache, storage, and network) and the operating system, and well suited for this advanced level of testing. The validation program uses Oracle Database as a tool and hardware is not certified with Oracle Database as a result of the validation program.”

The Oracle Linux Test is used to provide the validation tests. For more information, see oss.oracle.com/projects/olt/dist/documentation/OLT_UserGuide.pdf.

After installing Oracle Linux 7 and configuring Oracle Linux Test environment, the following OLT test cases should be run:

• Automated installation of Oracle Real Application Clusters (RAC)

http://www.oracle.com/linux

https://docs.oracle.com/en/database/oracle/oracle-database/12.2/index.html


http://www.oracle.com/technetwork/topics/linux/validated-configurations-085828.html

https://oss.oracle.com/projects/olt/dist/documentation/OLT_UserGuide.pdf


• Automated creation of the Oracle database • Automated performance stress tests for the RAC • Automated destructive tests • Manual multipath networking tests to verify redundancy for the host-side, storage-side, and switches.

The following sections describe the cluster configurations that have been tested by Lenovo and validated by Oracle.

4.4.1 SR650 2 socket validated configuration Figure 11 shows the SR650 2 socket configuration. The black lines show 10 GbE data connections, green lines 1GbE management connections, and orange lines 16Gb fibre connections.

Figure 11: Lenovo ThinkSystem SR650 validated configuration

The following tables contain the configuration details:

Server and Storage Platform Details

Server Model 2 X Lenovo ThinkSystem SR650 (2U)

Server Details Other servers models in this family: Lenovo ThinkSystem SR630 (1U)


Processors 2 X 12 Core Intel Xeon - 6126, 125W 2.6GHz

Memory 768GB RAM

On Board Storage 2 X 480Gb ThinkSystem M.2 5100 SATA 6Gbps Non-Hot-Swap SSD Mirrored M.2

Network/Interconnect 1 X Quad SFP+ Ports Intel Corporation X722 10GbE Ethernet Connection

HBA 1 X QLE2742 PCIe 32Gb 2-port SFP+ Fibre Channel Adapter

Multipath device mapper multipath 0.4.9-119.el7.x86_64

Storage Model Lenovo ThinkSystem DM5000F All Flash Storage Array

Note that the memory can be smaller than 768GB and that a faster CPU could also be used instead. The DM7000F all flash storage array can be used instead of the DM5000F to provide additional performance and capacity. Multiple Fibre Channel adapters can be added for improved storage performance.

Switch Details

Switch Model Lenovo RackSwitch G8272

Switch Type 10 GbE Network switch

Switch Model Lenovo ThinkSystem DB620S 32Gb FC SAN Switch

Switch Type 32Gbps FC switch

Note that the DB610S can be used instead of the DB620S if less ports are needed.

Software Details

OS Oracle Linux 7 Update 5 x86_64

Kernel 4.1.12-112.16.4.el7uek.x86_64 or higher

Oracle Software Oracle Database 12c Release 2 (12.2.0.1) for Linux x86_64

For more details about the validated configuration, including the packages used, parameters used in configuration files, and driver details, see the website: linux.oracle.com/pls/apex/f?p=102:2:::NO::P2_VC_ID,P2_VERSION:706,1.1.


https://linux.oracle.com/pls/apex/f?p=102:2:::NO::P2_VC_ID,P2_VERSION:706,1.1







Memory 768GB RAM

OnBoardStorage 2 X 480Gb ThinkSystem M.2 5100 SATA 6Gbps Non-Hot-Swap SSD Mirrored M.2





Note that the memory can be smaller than 768GB and that a faster CPU could also be used instead. The DM7000F all flash storage array can be used instead of the DM5000F to provide additional performance and capacity. Multiple Fibre Channel adapters can be added for improved storage performance.


Switch Details






Software Details


Kernel

4.1.12-124.22.4.el7uek.x86_64

Note: The default UEK version 4.14.35.xxxx that is packaged with Oracle Linux 7.6 does not currently support Oracle ASM Filter Driver (ASMFD) and hence the UEK version 4.1.12-124 should be installed as it does support the ASMFD.


For more details about the validated configuration, including the packages used, parameters used in configuration files, and driver details, see the website: linux.oracle.com/pls/apex/f?p=102:2:::NO::P2_VC_ID,P2_VERSION:707,1.1.









Memory 3TB RAM

OnBoardStorage 2 X 480Gb ThinkSystem M.2 5100 SATA 6Gbps Non-Hot-Swap SSD Mirrored M.2






Note that the memory can be smaller than 3TB and that a faster CPU could also be used instead. The DM7000F all flash storage array can be used instead of the DM5000F to provide additional performance and capacity. Multiple Fibre Channel adapters can be added for improved storage performance.

Switch Details






Software Details


Kernel

4.1.12-124.22.4.el7uek.x86_64

Note: The default UEK version 4.14.35.xxxx that is packaged with Oracle Linux 7.6 does not currently support Oracle ASM Filter Driver (ASMFD) and hence the UEK version 4.1.12-124 should be installed as it does support the ASMFD.


For more details about the validated configuration, including the packages used, parameters used in configuration files, and driver details, see the website: linux.oracle.com/pls/apex/f?p=102:2:::NO::P2_VC_ID,P2_VERSION:708,1.0

4.5 Best Practices 4.5.1 No single point of failure The hardware infrastructure should have high availability with as much elimination of single points of failure as possible using redundancy:

• Multiple servers in a high availability cluster using Oracle RAC • Dual processors • Dual power supplies (and dual data center power if available) • Mirrored boot drives • Dual network adapters • Redundant wiring to ToR Ethernet and Fibre Channel switches from servers and storage • Dual ToR Lenovo G8272 10GbE switches connected using VLAG



• Dual ToR Lenovo ThinkSystem fibre channel switches • Lenovo ThinkSystem 5000F storage array with dual controllers

4.5.2 Use hardened software The following software features should be used:

• Oracle Linux 7.5 or greater running the Unbreakable Enterprise Kernel (UEK) • Oracle users, groups, permissions, and recommended kernel and database parameters are set in

accord with Oracle’s validated pre-install RPM for Oracle Database 12c. • Use Oracle RAC Database 12c release 2

4.5.3 Thin Provisioning The storage volumes on the DM5000F storage array should be configured using thin provisioning to conserve storage capacity and only utilize drives for the actual data stored.

4.5.4 Use Oracle ASMFD The Oracle Automatic Storage Management Filter Driver (ASMFD) should be used to overcome the limitations of ASMLIB such as compromising the security of data and accidental corruption of the data in the ASM disks.

Note that during validation there was one issue related to using ASMFD. This is described in Oracle bug 28875565 whereby the device mapper with multipath (DMMP) is not compatible with ASMFD. The resolution is to apply the patch 28875565.

4.6 Systems management Lenovo XClarity™ Administrator is a centralized resource management solution that reduces complexity, speeds up response, and enhances the availability of Lenovo® server systems and solutions.

The Lenovo XClarity Administrator provides agent-free hardware management for Lenovo’s ThinkSystem® rack servers, System x® rack servers, and Flex System™ compute nodes and components, including the Chassis Management Module (CMM) and Flex System I/O modules. Figure 14 shows the Lenovo XClarity administrator interface, in which Flex System components and rack servers are managed and are seen on the dashboard. Lenovo XClarity Administrator is a virtual appliance that is quickly imported into a virtualized environment server configuration.

Figure 14: XClarity Administrator interface


5 Appendix: Bill of Materials This appendix features the Bill of Materials (BOMs) for Oracle validated configurations using Lenovo servers, storage, and switches.

For connections between ToR switches and devices (servers and storage), the connector cables are configured with the device. The ToR switch configuration includes only transceivers or other cabling that is needed for failover or redundancy.

5.1 Server BOM Table 4, Table 5, and Table 6 list the server BOMs.

Table 4: ThinkSystem SR650

Code Description Quantity

7X06CTO1WW ThinkSystem SR650 - 3yr Warranty 1 AUQB Lenovo ThinkSystem Mainstream MB - 2U 1 AUVX ThinkSystem MS 2U 2.5" Chassis, up to 16 1 AWEN Intel Xeon Gold 6130 16C 125W 2.1GHz Processor 2 AUND ThinkSystem 32GB TruDDR4 2666 MHz (2Rx4 1.2V) RDIMM 12 AURA ThinkSystem 2U/Twr 2.5" SATA/SAS 8-Bay Backplane 1 AUMV ThinkSystem M.2 with Mirroring Enablement Kit 1 B11V ThinkSystem M.2 5100 480GB SATA 6Gbps Non-Hot Swap SSD 2 AUNU ThinkSystem QLogic QLE2742 PCIe 32Gb 2-Port SFP+ Fibre Channel Adapter 1 AUKK ThinkSystem 10Gb 4-port SFP+ LOM 1 AUPW ThinkSystem XClarity Controller Standard to Enterprise Upgrade 1 AXCH ThinkSystem Toolless Slide Rail Kit with 2U CMA 1 AUS8 ThinkSystem SR550/SR590/SR650 EIA Latch w/ VGA Upgrade Kit 1 AVWF ThinkSystem 1100W (230V/115V) Platinum Hot-Swap Power Supply 2 6311 2.8m, 10A/100-250V, C13 to IEC 320-C14 Rack Power Cable 2 5977 Select Storage devices - no configured RAID required 1




7X19CTO1WW ThinkSystem SR850 - 3yr Warranty 1 AUR4 ThinkSystem 2U x8/x8/x8 PCIE FH Riser 1 1 B0MJ Feature Enable TPM 1.2 1 AVUU ThinkSystem XClarity Controller Advanced to Enterprise Upgrade 1 AXDX USB 2.0 Front Panel 1 AUR5 ThinkSystem 2U/Twr 2.5" AnyBay 8-Bay Backplane 1 AURA ThinkSystem 2U/Twr 2.5" SATA/SAS 8-Bay Backplane 1 AUPT 1600W Platinum (230V) Power Supply 2 5977 Select Storage devices - no configured RAID required 1 AUMW Front Operator Panel with LCD Display 1 AXCA ThinkSystem Toolless Slide Rail 1 AUKK ThinkSystem 10Gb 4-port SFP+ LOM 1 AUPP SR850 2U Chassis Base 1 AX7B Intel Xeon Platinum 8158 12C 150W 3.0GHz Processor 2 AUND ThinkSystem 32GB TruDDR4 2666 MHz (2Rx4 1.2V) RDIMM 24 B11V ThinkSystem M.2 5100 480GB SATA 6Gbps Non-Hot Swap SSD 2 AUMV ThinkSystem M.2 with Mirroring Enablement Kit 1 AUNU ThinkSystem QLogic QLE2742 PCIe 32Gb 2-Port SFP+ Fibre Channel Adapter 1 AUPX ThinkSystem SR850 Front VGA Connector Upgrade Kit 1 A4VP 1.0m, C13 to C14 Jumper Cord, Rack Power Cable 2 AUPQ SR850 System Board 1 AUPZ SR850/SR860 2S Power Paddle 1




7X12CTO1WW ThinkSystem SR950 - 3yr Warranty 1 AUKF SR950 2S to 8S Base 1 AX7B Intel Xeon Platinum 8158 12C 150W 3.0GHz Processor 8 AUNE ThinkSystem 64GB TruDDR4 2666 MHz (4Rx4 1.2V) LRDIMM 48 AX3Z 24 Hard Drive Maximum System 1 5977 Select Storage devices - no configured RAID required 1 AUMV ThinkSystem M.2 with Mirroring Enablement Kit 1 B11V ThinkSystem M.2 5100 480GB SATA 6Gbps Non-Hot Swap SSD 2 AUN1 ThinkSystem SR950 (2) x16 PCIe Riser 2 AUKK ThinkSystem 10Gb 4-port SFP+ LOM 1 AUNU ThinkSystem QLogic QLE2742 PCIe 32Gb 2-Port SFP+ Fibre Channel Adapter 1 AUPJ High Efficiency 1600W Power Supply for SR950 (200-240VAC only) 4 6311 2.8m, 10A/100-250V, C13 to C14 Jumper Cord 4 AUMY SR950 Lift Handles 1 A4AA Rail Kit 1 AUMW Front Operator Panel with LCD Display 1 AUMT SR950 Main I/O Planar 1 AUN0 ThinkSystem SR950 2-CPU, 24 DIMM Compute System Board 4 B37G SR950 16k rpm Chassis Fan Assembly 12 AUMZ ThinkSystem SR950 Upper Compute Tray 1 B0ML Feature Enable TPM on MB 1


5.2 Storage BOM Table 7 and Table 8 list the storage array BOMs. Table 9 lists the storage expansion unit BOM (at least one is needed for the DM7000F).

Table 7: ThinkSystem DM5000F


7Y41CTO1WW Lenovo ThinkSystem DM5000F All Flash Array 1 B38L Lenovo ThinkSystem Storage 2U24 Chassis 1

B39F Lenovo ThinkSystem DM Series DM3000/DM5000 Cntr, 16Gb FC/10Gb 2

ASR7 Lenovo 3m LC-LC OM3 MMF Cable 4 B3WK Lenovo ThinkSystem DM5000F 92TB SSD Pack (24x 3.8TB SSD) 1 B4BP Lenovo ThinkSystem Storage USB Cable, Micro-USB 1 6311 2.8m, 10A/100-250V, C13 to C14 Jumper Cord 2 B4D0 Lenovo ThinkSystem DM Series ONTAP Software, Encryption Capable 1 B39L Lenovo ThinkSystem DM Series 2U24 Bezel 1 B4CX Lenovo ThinkSystem DM Series 2U Accessory 1

Table 8: ThinkSystem DM7000F


7Y40CTO1WW LES : Lenovo ThinkSystem DM7000F All Flash Array 1 B38K Lenovo ThinkSystem DM Series 3U Chassis 1 B39H Lenovo ThinkSystem DM Series DM7000 Controller 2 B38U Lenovo ThinkSystem DM Series 32Gb 2 port Fibre Channel Card 4 ASR7 Lenovo 3m LC-LC OM3 MMF Cable 4 B4BP Lenovo ThinkSystem Storage USB Cable, Micro-USB 1 6311 2.8m, 10A/100-250V, C13 to C14 Jumper Cord 2 B4D0 Lenovo ThinkSystem DM Series ONTAP Software, Encryption Capable 1 B39M Lenovo ThinkSystem DM Series 3U Bezel 1 B4CW Lenovo ThinkSystem DM7000 Accessory 1 AU16 0.5m External MiniSAS HD 8644/MiniSAS HD 8644 Cable 2

Table 9: ThinkSystem DM240S


7Y58CTO1WW Lenovo ThinkSystem DM240S 2U24 SFF Expansion Enclosure 1 B38L Lenovo ThinkSystem Storage 2U24 Chassis 1 B39J Lenovo ThinkSystem DM Series DM120S/240S/600S IOM 2 B3WH Lenovo ThinkSystem DM7000F 92TB SSD Pack (24x 3.8TB SSD) 1 AU17 1m External MiniSAS HD 8644/MiniSAS HD 8644 Cable 2 6311 2.8m, 10A/100-250V, C13 to C14 Jumper Cord 2 B39L Lenovo ThinkSystem DM Series 2U24 Bezel 1


5.3 Ethernet Networking BOM Table 10 and Table 11 list the BOMs for the network switches for 1 GbE and 10 GbE connectivity respectively

Table 10: RackSwitch G8052


7159G52 Lenovo System Networking RackSwitch G8052 (Rear to Front) 1 6201 1.5m, 10A/100-250V, C13 to IEC 320-C14 Rack Power Cable 2 3802 1.5m Blue Cat5e Cable 1 A3KP Lenovo System Networking Adjustable 19" 4 Post Rail Kit 1

Table 11: RackSwitch G8272


7159CRW Lenovo System Networking RackSwitch G8272 (Rear to Front) 1 6201 1.5m, 10A/100-250V, C13 to IEC 320-C14 Rack Power Cable 2 A3KP Lenovo System Networking Adjustable 19" 4 Post Rail Kit 1 A1DP 1m QSFP+-to-QSFP+ cable 1

5.4 SAN Networking BOM Table 12 and Table 13 lists the BOM for the SAN network (Fibre Channel)

Table 12: ThinkSystem DB620S


6415HC4 Lenovo ThinkSystem DB620S Gen6 FC Switch 24x32Gb SWL SFP (1yr) 1 B4QE Lenovo ThinkSystem DB620S Gen6 FC Switch 24x32Gb SWL SFP (1yr) 1 6201 1.5m, 10A/100-250V, C13 to IEC 320-C14 Rack Power Cable 2 AVFX 1.0m Green Cat6 Cable 1

Table 13: ThinkSystem DB610S


6559HC4 Lenovo ThinkSystem DB610S Gen6 FC Switch 24x16Gb SWL SFP (1yr) 1 B4QB Lenovo ThinkSystem DB610S Gen6 FC Switch 24x16Gb SWL SFP (1yr) 1 6201 1.5m, 10A/100-250V, C13 to IEC 320-C14 Rack Power Cable 2 AVFX 1.0m Green Cat6 Cable 1

5.5 Rack BOM Table 14 lists the BOM for a rack.

Table 14: Rack BOM


93634PX 42U 1100mm Enterprise V2 Dynamic Rack 1 39Y8941 DPI Single Phase C19 Enterprise PDU (without line cord) 6 40K9614 DPI 30a Line Cord (NEMA L6-30P) 6


Resources For more information about the topics that are described in this document, see the following resources:

• Oracle Linux oracle.com/linux

• Oracle 12c docs.oracle.com/en/database/oracle/oracle-database/12.2/index.html.

• Oracle RAC oracle.com/database/technologies/rac.html

• Oracle validated configurations oracle.com/technetwork/topics/linux/validated-configurations-085828.html

http://www.oracle.com/linux


https://www.oracle.com/database/technologies/rac.html

http://www.oracle.com/technetwork/topics/linux/validated-configurations-085828.html


Document History Version 1.0 4 February 2019 • Initial version

Version 1.1 29 March 2019 • Added validated configuration and BOM for ThinkSystem SR950 server


Trademarks and special notices © Copyright Lenovo 2019.

References in this document to Lenovo products or services do not imply that Lenovo intends to make them available in every country.

Lenovo, the Lenovo logo, ThinkSystem, ThinkAgile, ThinkCentre, ThinkVision, ThinkVantage, ThinkPlus and Rescue and Recovery are trademarks of Lenovo.

IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both.

Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.

Intel, Intel Inside (logos), MMX, and Pentium are trademarks of Intel Corporation in the United States, other countries, or both.

Other company, product, or service names may be trademarks or service marks of others.

Information is provided "AS IS" without warranty of any kind.

All customer examples described are presented as illustrations of how those customers have used Lenovo products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer.

Information concerning non-Lenovo products was obtained from a supplier of these products, published announcement material, or other publicly available sources and does not constitute an endorsement of such products by Lenovo. Sources for non-Lenovo list prices and performance numbers are taken from publicly available information, including vendor announcements and vendor worldwide homepages. Lenovo has not tested these products and cannot confirm the accuracy of performance, capability, or any other claims related to non-Lenovo products. Questions on the capability of non-Lenovo products should be addressed to the supplier of those products.

All statements regarding Lenovo future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Contact your local Lenovo office or Lenovo authorized reseller for the full text of the specific Statement of Direction.

Some information addresses anticipated future capabilities. Such information is not intended as a definitive statement of a commitment to specific levels of performance, function or delivery schedules with respect to any future products. Such commitments are only made in Lenovo product announcements. The information is presented here to communicate Lenovo’s current investment and development activities as a good faith effort to help with our customers' future planning.

Performance is based on measurements and projections using standard Lenovo benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput or performance improvements equivalent to the ratios stated here.

Photographs shown are of engineering prototypes. Changes may be incorporated in production models.

Any references in this information to non-Lenovo websites are provided for convenience only and do not in any manner serve as an endorsement of those websites. The materials at those websites are not part of the materials for this Lenovo product and use of those websites is at your own risk.

Documents

Reference Architecture: Oracle Database Real Application ... · 5.1 Server BOM ... two Oracle validated configurations for bare-metal deployment of Oracle RAC together with best practices