Oracle Optimized Solution for Oracle WebLogic Serverhosteddocs.ittoolbox.com/ser_us_en_wp_serveroos.pdf · Oracle Optimized Solution for Oracle WebLogic Server . ... internal configuration

An Oracle Technical White Paper October 2011

Oracle Optimized Solution for Oracle WebLogic Server


Introduction ......................................................................................... 1 Solution Overview ............................................................................... 1

Integrated Enterprise Application Stack .......................................... 2 Oracle's SPARC T4 Servers ........................................................... 3

Solution Architecture ........................................................................... 3 Physical Architecture ....................................................................... 3 Virtual Architecture .......................................................................... 4 WebLogic Clustering for Improved Availability and Scalability ........ 6

Test Environment ................................................................................ 6 Workload Description ...................................................................... 8 Tuning Guidelines ........................................................................... 8

Test Results ........................................................................................ 9 CPU Results .................................................................................. 10 Memory Utilization Results ............................................................ 10

Sizing Guidelines .............................................................................. 11 Security: T-Series Cryptography ....................................................... 11

T-Series Cryptography Performance ............................................ 12 For More Information ......................................................................... 13


1

Introduction

Oracle WebLogic Server has long set the enterprise gold standard for a high-performance, high-value application server platform. The Oracle Optimized Solution for Oracle WebLogic Server builds upon this success, providing a pretested and optimized architecture for deploying enterprise Java applications.

Oracle Optimized Solutions are predefined solutions with detailed build instructions and best practices for assembling a best-of-breed combination of Oracle's Sun servers and storage, Oracle applications, and the Oracle Solaris operating system. Oracle's hardware and software engineers work together to integrate the complete stack and to ensure that Oracle applications, databases, and middleware are integrated and optimized with compute, storage, networking, and operating systems to deliver extreme performance. Employing this pretested and preconfigured infrastructure can help mitigate risk, reduce complexity, and accelerate deployment of new applications.

This document focuses on the configuration and performance characteristics of the Oracle Optimized Solution for WebLogic Server on Oracle's SPARC T4 servers.

The remainder of this paper describes the solution architecture, presents a test environment used to gather performance characterization information, and provides sizing guidelines and recommended optimizations in order to achieve a reliable solution.

Solution Overview

The Oracle Optimized Solution for Oracle WebLogic Server features a complete infrastructure for deploying Oracle WebLogic Server version 10.3.5 in an enterprise environment. The following sections describe the integrated enterprise application stack and the underlying hardware components.

Performance expectations, sizing recommendations and results were derived from the most recent record-setting performance test conducted using the SPECjEnterprise® benchmark standards1. While the architecture of this solution paper departs somewhat from the test environment specified in the benchmark tests, it is essentially the same simple design as what was tested, but with more provisions for availability, proper sizing configuration, cost effectiveness and practicality.

This solution whitepaper omits certain details as to the configuration of the Oracle Database in favor of leaving the database options open to the end user. Ample documentation already exists, including a dedicated Oracle Optimized Solution for Oracle Database 11g, for deploying a complementary database solution.

1 SPEC and SPECjEnterprise are registered trademarks of Standard Performance Evaluation Corporation.


2

Note: While Oracle WebLogic Server forms a core part of the of the Oracle Fusion Middleware portfolio, the information in this paper is relevant only to the Oracle WebLogic application server.

Integrated Enterprise Application Stack

The Oracle Optimized Solution for Oracle WebLogic Server provides a complete environment for developing and deploying enterprise Java EE applications. The solution utilizes an integrated enterprise application stack that includes the Solaris 10 operating system with built-in Oracle Solaris Containers virtualization and Oracle WebLogic Server.

Oracle Solaris

The Oracle Solaris 10 operating system includes innovative, built-in functionality, such as near wire-speed networking throughput and high availability features that deliver industry-leading performance. Built-in virtualization features help to optimize resource utilization, and advanced security features provide the isolation and control required by enterprise environments. Oracle Solaris Containers are used in this architecture to securely and conveniently host and deploy Java EE applications. Some of the major features of the Oracle Solaris operating system include:

• High-performance 64-bit operating environment

• Support for large memory and high CPU count systems

• Excellent scalability for richly threaded environments

• Predictive, self-healing features

• Extensive instrumentation and diagnostic capabilities to assist performance and availability

• Specialized Security features

Oracle Solaris Containers

Oracle Solaris Containers allow kernel-level separation of applications running in a single Oracle Solaris 10 instance. An included feature of the Oracle Solaris 10 operating system, Oracle Solaris Containers provide built-in no-cost virtualization. Oracle Solaris Containers are rapid to deploy, impose extremely low overhead, and are used in this architecture to separate instances of Oracle WebLogic Server.

Oracle WebLogic Server 11g

Oracle WebLogic Server Enterprise Edition is the application server of choice for demanding customer environments the world over. Comprehensive management capabilities enable administration of sophisticated systems via a well-designed graphical console. Proven clustering technology, cross-domain management, and comprehensive diagnostic tools are standard with Oracle WebLogic Server Enterprise Edition, and are well integrated with the Oracle portfolio for superior interoperability and support across the technology stack.


3

The Oracle WebLogic Server 11g is a fully compliant Java EE 5.0 application server which is feature rich and holds benchmark world records for Java EE performance. Oracle WebLogic Server 11g takes full advantage of the 64-bit addressable memory and also the large number of hardware threads available in servers such as the Sun SPARC T4 servers used in this architecture.

Oracle's SPARC T4 Servers

Oracle's SPARC T4 servers are powered by the SPARC T4 processor that combines a new level of single-threaded performance along with high overall system throughput. Integrated on-chip cryptographic acceleration provides high levels of security without sacrificing application performance. The SPARC T4 servers run Oracle Solaris 10 and 11, both of which include advanced integrated virtualization technology that enables real-time scaling and optimal resource utilization. SPARC T4 based servers are especially ideal for deploying Java applications on Oracle WebLogic Server, due to their optimally balanced single and multithreaded performance capabilities. Older Java applications that were coded for and performed better on single-thread, clockspeed-centric servers can operate happily alongside their more modern, multithreaded counterparts.

Options within this family of servers include the SPARC T4-1, SPARC T4-2, and SPARC T4-4 servers configured with one, two, or four SPARC T4 processors respectively. The systems scale to a maximum configuration of up to four 3.0 GHz processors (eight cores per processor, for a maximum of 256 threads), 1 TB of memory, and up to eight internal SAS or SSD drives in the SPARC T4-4 server.

Solution Architecture

The following sections describe the physical and virtual architecture of the Oracle Optimized Solution for WebLogic Server.

Physical Architecture

The physical architecture of this Oracle Optimized Solution for WebLogic Server consists of, at its smallest and most compact, dual SPARC T4-1 servers attached to a 10 Gigabit Ethernet network switch, as shown in Figure 1.

Figure 1: The most basic physical architecture configuration includes a pair of SPARC T4-1 servers.


4

In the largest configuration, seen in Figure 2, this solution architecture consists of four SPARC T4-4 servers attached to a 10 Gigabit Ethernet network switch using the on-board 1 Gigabit Ethernet interfaces that are a standard part of the server configuration. The 10 Gigabit Ethernet connectivity provides further flexibility and options for extending the function of this solution. A database tier, additional servers, or NAS options can be connected easily into this central network switch, for instance.

Figure 2. This solution scales to a larger configuration with four SPARC T4-4 servers.

Virtual Architecture

Using Oracle Solaris Containers, each SPARC T4 server is logically divided into two separate containers, as shown in Figure 3. Each container hosts an instance of WebLogic Server. The CPU and memory resources of the SPARC T4 server are allocated to these individual containers, to help ensure that each one has sufficient resources available to it.


5

Figure 3. WebLogic Server runs in separate Oracle Solaris Containers on the SPARC T4 servers.

Given the low resource overhead and ease-of-use characteristics of Oracle Solaris Containers, it is both efficient and convenient to separate these multiple application server instances by installing them into different containers. This virtualized architecture provides flexible resource allocation, security, and scalability, as well as power and space savings afforded by consolidation onto a single high-performance server such as the SPARC T4 server.

Extending scale and increasing resiliency from this basic unit is simply a matter of adding more server units and interconnecting them via the 10 Gb Ethernet switch, as shown in Figure 4. At that point WebLogic Clustering could be configured to manage failover between WebLogic Server containers running on separate servers.

Figure 4. WebLogic clustering can be configured for increased availability and scalability of the solution.


6

WebLogic Clustering for Improved Availability and Scalability

Application availability could be further enhanced in this architecture by utilizing WebLogic clustering. A WebLogic cluster is a collection of WebLogic Server instances that work together to provide a reliable, scalable environment for Java EE applications. WebLogic Server clusters increase reliability by supporting failover: the WebLogic software automatically switches requests and processing to a redundant server upon the failure or abnormal termination of the currently active server. A WebLogic cluster always contains one administration server, and this could run in a dedicated container on one of the existing SPARC T4 servers in the configuration.

The architecture of this Oracle Optimized Solution for WebLogic Server is easily configurable and highly scalable. Depending on expected workloads, the solution can be deployed with either SPARC T4-1, SPARC T4-2, or SPARC T4-4 servers. If workload requirements increase, additional Oracle Solaris Containers can be dynamically deployed to run additional instances of Oracle WebLogic Server.

Test Environment

Figure 3 depicts the test environment used to study the performance and scalability of the Oracle Optimized Solution for WebLogic Server on SPARC T4 servers. A total of four SPARC T4-4 servers were configured as application servers running WebLogic Server software. An additional two SPARC T4-4 servers were configured as database servers running Oracle Database 11g. In addition, a Sun Fire X4170 M2 Server was used as a DNS server during testing. For networking and communication, multiple 10 Gb Ethernet switches were employed.


7

Figure 3. The test environment included SPARC T4-4 servers and Sun Storage F5100 Flash Arrays.

Design and configuration of the database storage is independent of the Oracle Optimized Solution for WebLogic Server. In this test environment, eight Sun Storage F5100 Flash Arrays were used to contain the database storage, and eight Sun Fire X4270 M2 servers were configured as database storage servers that exported the storage via Fibre Channel. This configuration was deployed in the lab environment so that the database would not be a bottleneck and so that Oracle WebLogic Server performance could be more clearly evaluated.

While this test architecture was intentionally sized to handle the largest enterprise-scale workloads, the base reference architecture can be deployed on a pair of SPARC T4-1 servers. The number and choice of SPARC T4-1, T4-2, or T4-4 servers and their internal configuration depends on the transaction processing requirements for each environment. See Sizing Guidelines on page 10 for more details.


8

Configuration details of the test hardware are included in Table 1.

TABLE 1. TEST ENVIRONMENT CONFIGURATION

FUNCTION SYSTEMS CONFIGURATION DETAILS

WebLogic Servers 4x SPARC T4-4 Servers • 4x 3.0 GHz SPARC T4 processors

• 512 MB RAM

• 2x 300 GB SAS disks

• 2x 100 GB SSD

Workload Description

To determine the application server capacity, the average number of transactions the server can support for a credible WebLogic Server solution must be determined. The SPECjEnterprise2010 benchmark was used in this study to generate the test workload.

Created by the Standard Performance Evaluation Corporation (SPEC®) as a standard Java EE benchmark, the SPECjEnterprise2010 benchmark is a full system benchmark that enables performance characterization of Java EE 5.0 servers and supporting infrastructure including database and disks. In this reference architecture, the SPECjEnterprise2010 application is used both as a benchmark and as a good example of a complex and resource-intensive application typical of enterprise businesses that run and rely on Java EE and Oracle WebLogic for their core business functions.

Tuning Guidelines

The following tuning was performed on the SPARC T4 servers used to deploy Oracle WebLogic Server in the test environment.

The following tuning was performed in the /etc/system file:

set kernel_cage_enable = 0

set autoup = 345600

set ip:ip_squeue_fanout = 1

set ip:ip_soft_rings_cnt= 32

The following network tuning was performed:

ndd -set /dev/tcp tcp_conn_req_max_q 40000

ndd -set /dev/tcp tcp_conn_req_max_q0 40000

ndd -set /dev/tcp tcp_xmit_hiwat 524288

ndd -set /dev/tcp tcp_recv_hiwat 524288

ndd -set /dev/tcp tcp_smallest_anon_port 4096

ndd -set /dev/tcp tcp_naglim_def 1

ndd -set /dev/tcp tcp_time_wait_interval 10000

ndd -set /dev/tcp tcp_fin_wait_2_flush_interval 10000


9

The following kernel tuning was performed in the /kernel/drv/ixgbe.conf file to configure the number of transmit queues, receive queues, and the interrupt throttling rate used by the network driver:

ixgbe2: tx_queue_number=16, rx_queue_number=16, intr_throttling=1000



Test Results

Oracle engineers looked at system resource utilization and throughput metrics as they scaled the number of transactions using the SPECjEnterprise2010 benchmark. The performance results from this testing were used to determine the recommended sizing guidelines for different configurations.

It is important to examine throughput and response time metrics together when analyzing application performance and configuration scalability. As a rule of thumb, as the number of users increases there is a corresponding increase in throughput. As the number of transactions increases, response time must remain within acceptable bounds.

The performance results were measured using the SPECjEnterprise 2010 benchmark, showing the performance on the SPECj Dealer Mix and the SPECj Manufacturing Mix benchmark subsets, are shown in Figure 4.1 The average observed latency during testing was 0.075 seconds per thread, demonstrating the solution's ability to handle large enterprise-level workloads with outstanding user response time.

While Figure 4 illustrates more granular performance results for throughput on a sample Java application simulating a demanding but generally typical Java application, ultimately the benchmark tests distill into one standardized throughput score known as EjOPS, or Enterprise Java Operations Per Second. This score is an objective and realistic evaluation of a system’s performance running a complex Java application. This Oracle Optimized Solution for WebLogic Server, as architected at maximum configuration, achieves record-setting 40,167 EjOPS.

1 SPECjEnterprise2010 models contemporary Java-based applications that run on large JEE (Java Enterprise Edition) servers, backed by network infrastructure and database servers. Oracle: Application tier—4x SPARC T4-4 servers (16 chips, 128 cores), Database tier—2x SPARC T4-4 servers (8 chips, 64 cores). 40,104.86 SPECjEnterprise2010 EjOPS, 1,671 SPECjEnterprise2010 EjOPS performance per processor across the configuration. All results from www.spec.org as of September 24, 2011.


10

Figure 4. Performance results measured using the SPECjEnterprise®2010 benchmark.

CPU Results

Table 2 shows CPU utilization for the software components in the Oracle Optimized Solution for WebLogic Server. Results are shown for four configurations, differing in the number of transactions per hour (TPH) for the WebLogic Server. As shown, CPU reaches a maximum utilization rate of 71% on the Oracle WebLogic Servers, and a maximum utilization rate of 77% for the Oracle Database 11g server. There is additional compute capacity available to handle peaks in utilization in all three configurations.

TABLE 2. CPU UTILIZATION (%)

CONFIGURATION ENTRY LEVEL

12,000,000 TPH

MEDIUM

24,000,000 TPH

LARGE

48,000,000 TPH

EXTRA LARGE

90,000,000 TPH

Oracle WebLogic Server 69% 69% 70% 71%

Oracle Database 11gR2 77% 77% 75% 77%

Memory Utilization Results

Table 3 shows memory allocation requirements (in gigabytes) for the software components in this solution, Oracle WebLogic Server and Oracle Database 11g. Results are shown for four configurations, differing in the number of transactions per hour (TPH) for the WebLogic Server. As shown, memory utilization scales as the number of transactions increases.

TABLE 3. MEMORY UTILIZATION (DEDICATED GB)

CONFIGURATION ENTRY LEVEL

12,000,000 TPH

MEDIUM

24,000,000 TPH

LARGE

48,000,000 TPH

EXTRA LARGE

90,000,000 TPH

Oracle WebLogic Server 32 GB 55 GB 64 GB 75 GB

Oracle Database 11gR2 68 GB 75 GB 100 GB 130 GB

These memory utilization results are used to help determine the recommended sizing guidelines and the choice of SPARC T4-1, T4-2, or T4-4 servers.


11

Sizing Guidelines

Based on the testing performed in this study, the sizing guidelines in Table 4 are recommended for entry level, medium, large, and extra large configurations. These four configurations are based on the expected number of transactions on the Oracle WebLogic Server and are sized accordingly to provide the best performance while keeping costs in mind. There is additional compute and memory capacity available to handle peaks in utilization in all configurations.

These recommended sizing guidelines serve as starting points when planning a WebLogic Server deployment on SPARC T4 servers. The architecture is flexible and highly scalable, providing an easy upgrade path should workload requirements increase.

TABLE 4. CONFIGURATION SIZING

ENTRY LEVEL MEDIUM LARGE EXTRA LARGE

Number of transactions/hour

(TPH)

Up to 12,000,000 Up to 24,000,000 Up to 48,000,000 Up to 90,000,000

WebLogic Server 2x SPARC T4-1

Servers, each with:

• 1 CPU (8 cores)

• 128 GB RAM

2x SPARC T4-2

Servers, each with:

• 2 CPUs (16 cores)

• 256 GB RAM

2x SPARC T4-4

Servers, each with:


• 512 GB RAM

4x SPARC T4-4

Servers, each with:


• 640 GB RAM

Security: T-Series Cryptography

A truly unique feature of the T-Series servers is the on-CPU cryptography acceleration capabilities, which act as a perfectly complementary technology to Solaris’ security functions. Cryptographic workloads can greatly increase the amount of overhead imposed on a system. As a rule of thumb, simply using HTTPS (SSL) at the web tier to secure server to browser (client) connections can place an additional 20% overhead on the CPU resources, essentially robbing an ordinary server of its performance value.

In most circumstances, specialized cryptography cards or network appliances are used to offload the additional workload imposed by cryptography. However, both these approaches impose penalties of their own. A card generally adds complexity and cost while also putting the additional cryptography workload on the bus, imposing a system-wide performance penalty and potential bottleneck. Using a network appliance to intercept inbound SSL, strip out the encryption and then pass it in the clear to the application introduces problems with increased resources (electricity, cooling, footprint) and can present security compliance issues. In both cases, these approaches to mitigating security overhead represent additional cost of acquisition and operation.

With Oracle SPARC T4 servers, however, hardware cryptography acceleration is built into the CPU and is easily configured using a number of methods without additional costs.

The most immediate and direct means of taking advantage of T-Series hardware acceleration features is to use the KSSL (Kernel SSL) approach. KSSL acts essentially as a two-way proxy for intercepting SSL


12

workloads and executing the encryption and decryption on the CPU rather than placing that burden on the overall system.

Actual configuration of WebLogic Server to make use of the SPARC T4 cryptographic acceleration features is beyond the scope of this document. However, a comprehensive guide with a full set of instructions on this topic may be found at http://blogs.oracle.com/BestPerf/entry/20100920_sparc_t3_weblogic_security.

T-Series Cryptography Performance

Figures 5 and 6 present the overall performance characteristics of using CMT-based cryptographic acceleration and study the effect of cryptographic overheads on the application. Figure 5 shows throughput performance for three scenarios: using KSSL acceleration, SSL without acceleration, and no encryption. Figure 6 shows the measured response times for these three same scenarios.

Figure 5. T-Series Server cryptography throughput performance.

Figure 6. T-Series Server cryptography response time performance.

Table 5 includes the cryptography performance data for the T-Series server, shown graphically in Figures 5 and 6.

TABLE 5. T-SERIES CRYPTOGRAPHY PERFORMANCE

TRANSACTIONS/SEC HITS/SEC RESPONSE TIME

No encryption 402 1214 0.705

SSL without acceleration 243 712 1.615

KSSL acceleration 339 1022 0.915


13

LoadRunner 9.1 was used as the load driver for driving the workload, simulating 1000 concurrent users ramped up in increments of 10 users per minute until reaching 150 users. Each user queried the web application as many times as possible per minute, clearing caches in between. Once 1000 concurrent users were reached, the workload was sustained for 10 minutes. The load test run captured numbers for three key aspects of any web transaction: throughput (or peak transfer), hits per second and tests per minute. This Jmeter load test was not intended to push the upper limits of the server but rather to demonstrate the overheads of cryptography at a reasonable load and the effects of using hardware assisted cryptographic acceleration.

The results showed only a minor difference due to SSL overhead between the unsecured application versus on-chip cryptographic accelerated solution, which yielded tangible, immediate and cost-efficient results in the form of faster secure transactions and better response times—all without adding any additional security equipment costs, changes in power usage profiles or elaborate system configurations. Additionally, the results clarify the massive burden un-accelerated cryptographic workloads can have on a server.

For More Information

For more information on the Oracle Optimized Solution for WebLogic Server on SPARC T4 Servers, see the references listed in Table 6.

TABLE 6. REFERENCES FOR MORE INFORMATION

Oracle WebLogic Server www.oracle.com/us/products/middleware/application-server/index.html

Oracle's SPARC T4 Server www.oracle.com/servers

Oracle Solaris www.oracle.com/solaris

Oracle Database 11g www.oracle.com/us/products/database/index.html

SPECjEnterprise®2010 Result www.spec.org/jEnterprise2010/results/res2011q3/jEnterprise2010-20110907-00027.html

High Performance Security for Oracle WebLogic Applications Using Oracle SPARC Enterprise T-Series Servers

www.oracle.com/technetwork/articles/systems-hardware-architecture/security-weblogic-t-series-168447.pdf

Oracle Fusin Middleware Security Blog fusionsecurity.blogspot.com/2011/04/ssl-offloading-and-weblogic-server.html

Deploying Oracle WebLogic (Java EE) applications on Oracle Sun T-series Servers and Sun Storage 7000 Unified Storage

www.oracle.com/technetwork/articles/systems-hardware-architecture/deploy-weblogic-t-series-ss7000-168444.pdf

Oracle Optimized Solution for Oracle WebLogic Server October 2011, Version 1.0 Oracle Corporation World Headquarters 500 Oracle Parkway Redwood Shores, CA 94065 U.S.A. Worldwide Inquiries: Phone: +1.650.506.7000 Fax: +1.650.506.7200 oracle.com

Copyright © 2010, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only and the contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or fitness for a particular purpose. We specifically disclaim any liability with respect to this document and no contractual obligations are formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without our prior written permission. Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. UNIX is a registered trademark licensed through X/Open Company, Ltd. 0410

Documents

Oracle Optimized Solution for Oracle WebLogic Serverhosteddocs.ittoolbox.com/ser_us_en_wp_serveroos.pdf · Oracle Optimized Solution for Oracle WebLogic Server . ... internal configuration