Upload
rams08
View
127
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Oracle EPM 11-1-2 3 Tuning Guide v2
Citation preview
1
Disclaimer:
All tuning information stated in this guide is only for orientation, every modification has to be tested
and its impact should be monitored and analyzed. Before implementing any of the tuning settings, it
is recommended to carry out end to end performance testing that will also include to obtain baseline
performance data for the default configurations, make incremental changes to the tuning settings and
then collect performance data. Otherwise it may worse the system performance.
Oracle EPM, Fusion Edition Infrastructure Performance Tuning Guide
APPLIES TO EPM RELEASE 11.1.2.2, 11.1.2.3
An Oracle White Paper
October 2013
Change History
Date Author Version Change Reference
03/2012 Ahmed Awan (Product Assurance Engineering)
1.5
10/2013 CEAL Team (Customer Engineering & Advocacy Laboratory)
2
2
TABLE OF CONTENTS
INTRODUCTION ................................................................................................................. 4
1.0 PERFORMANCE OVERVIEW ................................................................................... 4
1.1 INTRODUCTION TO ORACLE EPM SYSTEM PERFORMANCE ................................. 4
1.2 PERFORMANCE TERMINOLOGY ................................................................................ 4
1.3 UNDERSTANDING KEY PERFORMANCE DRIVERS ..................................................... 5
2.0 TOP TUNING RECOMMENDATIONS FOR EPM SYSTEM ............................. 6
2.1 TUNE OPERATING SYSTEMS PARAMETERS .................................................................... 7
2.2 TUNE ORACLE WEBLOGIC SERVER (WLS) PARAMETERS .......................................... 20
2.3 TUNE IBM WEBSPHERE SERVER (IBM WAS) PARAMETERS ..................................... 24
2.4 TUNE 64BIT JAVA VIRTUAL MACHINES (JVM) .......................................................... 33
2.5 TUNE 32BIT JAVA VIRTUAL MACHINES (JVM) .......................................................... 38
2.6 TUNE HTTP SERVER PARAMETERS ............................................................................ 39
2.7 TUNE HTTP SERVER COMPRESSION / CACHING ....................................................... 43
2.8 TUNE ORACLE DATABASE PARAMETERS ................................................................... 53
3.0 PERFORMANCE MONITORING EPM SYSTEM ............................................... 55
4.0 TUNING FINANCIAL PM APPLICATIONS ....................................................... 56
4.1 HYPERION FINANCIAL MANAGEMENT (HFM) ......................................................... 56
4.2 HYPERION PLANNING ................................................................................................ 58
4.3 ESSBASE APPLICATION LINK FOR HFM ..................................................................... 64
4.3 FINANCIAL MANAGEMENT ANALYTICS .................................................................... 68
4.5 HYPERION PROFITABILITY AND COST MANAGEMENT (STANDARD COSTING) ........ 69
5.0 TUNING FOUNDATION SERVICES .................................................................... 70
5.1 ENTERPRISE PERFORMANCE MANAGEMENT WORKSPACE ....................................... 70
5.2 ENTERPRISE PERFORMANCE MANAGEMENT ARCHITECT (EPMA) .......................... 70
5.3 HYPERION SHARED SERVICES ..................................................................................... 71
5.3 SMARTVIEW FOR OFFICE ............................................................................................ 73
6.0 TUNING ESSBASE ..................................................................................................... 75
6.1 ESSBASE AGGREGATE STORAGE (ASO) TUNING ....................................................... 75
6.2 ESSBASE BLOCK STORAGE (BSO) TUNING ................................................................. 81
3
6.3 TROUBLESHOOTING .................................................................................................... 86
7.0 TUNING REPORTING AND ANALYSIS ............................................................. 89
7.1 TUNE REPORTING AND ANALYSIS FRAMEWORK (RAF) SERVICES ........................... 89
7.2 FINANCIAL REPORTING .............................................................................................. 91
7.3 WEB ANALYSIS ............................................................................................................ 95
7.0 TUNING DATA MANAGEMENT .......................................................................... 96
7.1 TUNE DATA RELATIONAL MANAGEMENT (DRM) ................................................... 96
7.2 TUNE FINANCIAL DATA QUALITY MANAGEMENT (FDQM) .................................... 98
8.0 WEB BROWSER OPTIMIZATIONS ..................................................................... 100
8.1 TUNE INTERNET EXPLORER ...................................................................................... 100
8.2 TUNE FIREFOX ........................................................................................................... 105
9.0 DISK TUNING .......................................................................................................... 107
9.1 RAID CONSIDERATION FOR EPM ............................................................................ 107
9.2 FILE SYSTEM CONSIDERATION FOR EPM .................................................................. 107
9.3 RECOMMENDED IOPS FOR EPM .............................................................................. 108
10.0 ORACLE VIRTUAL MACHINE TUNING ...................................................... 109
10.1 CONSIDERATIONS WITH OVM V3 ........................................................................ 109
10.2 REDUCE THE NUMBER OF DOM0 CPUS................................................................. 109
10.3 TUNE GUEST VM .................................................................................................. 109
10.4 TUNE PROCESS AFFINITY ...................................................................................... 109
4
INTRODUCTION
This document is written for people who monitor performance and tune the
components in an EPM environment. It is assumed that readers know server
administration and hardware performance tuning fundamentals, web servers, java
application servers and database.
1.0 PERFORMANCE OVERVIEW This chapter discusses performance and tuning concepts for Oracle EPM. This
chapter contains the following sections:
1.1 Introduction to Oracle EPM System Performance
1.2 Performance Terminology
1.3 Understanding Key Performance Drivers
1.1 Introduction to Oracle EPM System Performance
To maximize Oracle EPM System performance, you need to monitor, analyze, and
tune all the components. This guide describes the tools that you can use to monitor
performance and the techniques for optimizing the performance of Oracle EPM
System components, for example Financial Management, Essbase, Reporting and
Analysis and Planning.
1.2 Performance Terminology
This guide uses the following performance terminologies:
Scalability
o System's ability to perform within specification under increasing
user load, data load and hardware expansion.
Latency
o Time between the issuing of a request and the time when the work
actually begins on the request.
Think time
o The time a real user pauses to think between actions.
Resource utilization
o A consumption metric, for example, the percent of CPU usage.
Response time
o A time metric, for example round-trip time it takes the server to
deliver a Web page.
5
Throughput
o A rate metric (requests per unit of time), for example, requests per
second, bits per second. For example, if an application can handle
20 customer requests simultaneously and each request takes one
second to process, this site has a potential throughput of 20
requests per second.
1.3 Understanding Key Performance Drivers
To optimize your deployment, you must understand the elements that influence
performance and scalability. A factor that dictates performance is called a key
performance driver. Knowing how the drivers behave in combination further
enhances your ability to deploy Oracle EPM system optimally, based on the unique
requirements of each deployment.
- Hardware Capacity - Factors such as number of servers, quantity and speed
of processors, available RAM, network speed etc.
- Technical Platforms Tuning Fine tuning other third party software
required for installing and running Oracle EPM products; for example:
relational databases, Java application servers, Web servers, Server / Client
Operating System and browsers.
- Business Application Design - Application design is an important factor in
system performance i.e. structure, size, and use of product features in
designing applications databases, reports, Web data entry forms, calculations
and consolidations.
- Business process usage - Activities carried out by users in the normal flow of
your business cycle.
o Business process usage has three components:
User activityActivities available to users for data load or data
entry, database processing (consolidations, copy, clear, and so on),
and reporting and analysis.
Rate of user activity A number of transactions executed by one
user per one hour.
User concurrencyNumber of users for each activity being carried
out simultaneously.
6
2.0 TOP TUNING RECOMMENDATIONS FOR EPM SYSTEM Performance tuning Oracle EPM system is a complex and iterative process. To get
you started, we have created a list of recommendations to help you optimize your
Oracle EPM system performance.
This chapter includes the following sections that provide a quick start for
performance tuning Oracle EPM products. Note these performance tuning
techniques are applicable to nearly all Oracle EPM products such as Financial PM
Applications, Essbase, Reporting and Foundation services.
2.1 Tune Operating Systems parameters.
2.2 Tune Oracle WebLogic Server (WLS) parameters.
2.3 Tune IBM WebSphere Application Server (IBM WAS) parameters.
2.4 Tune 64bit Java Virtual Machines (JVM), including ADF parameters.
2.5 Tune 32bit Java Virtual Machines (JVM).
2.6 Tune HTTP Server parameters.
2.7 Tune HTTP Server Compression / Caching.
2.8 Tune Oracle Database Parameters.
2.9 Tune Reporting And Analysis Framework (RAF) Services.
2.10 Tune Oracle ADF parameters.
Note: While the list in each of the above stated section is a useful tool in starting
your performance tuning, it is not meant to be comprehensive list of areas to tune.
You must monitor and track specific performance issues within your EPM
application to understand where tuning can improve performance. See
"Performance Monitoring EPM System" for more information.
7
2.1 Tune Operating Systems parameters
Proper OS tuning might improves system performance by preventing the occurrence
of error conditions. Operating system error conditions always degrade performance.
The following sections describe issues related to operating system performance:
2.1.1 Linux Tuning Parameters
This topic describes how to tune the Linux operating system to optimize the
performance of your Oracle EPM System.
Parameter Default Value Suggested Value
tcp_fin_timeout
By reducing the value of this entry, TCP/IP can release
closed connections faster, providing more resources for new
connections.
Consult your Linux (OEL / RHEL / SLES) documentation for
more information on how to permanently change the value
for this parameter.
Important Note: It is hardcoded into Linux kernel directly
sometimes, which will imply that it is not possible to modify
even with root.
60 30
File descriptors (open files=ulimit n value)
If the value set for this parameter is too low, a file open
error, memory allocation failure, or connection
establishment error might be displayed.
Important Note: When you increase the soft limit, make
sure to increase the hard limit as well. The hard limit is the
maximum value which can be reached before the user gets
the error messages Out of file handles.
Consult your Linux (OEL / RHEL / SLES) documentation for
more information on how to permanently change the value
for this parameter.
1024 4096 (32 bit)
32000 (64 bit)
Backlog connections queue (tcp_max_syn_backlog)
When the server is heavily loaded or has many clients with
bad connections with high latency, it can result in an
increase in half-open connections.
Consult your Linux (OEL / RHEL / SLES) documentation for
more information on how to permanently change the value
for this parameter.
1024 4096
Local Port Range (ip_local_port_range)
netstat is the tool to use to determine what ports are in use
currently. As to the limits, the number of ports available is a
16bit unsigned integer which gives you the range 0-65535.
The ports that are available for applications to bind to are
9000-65535 1024-65535
8
the reserved privileged/root ports (0-1024) plus whatever is
not covered by your ephemeral port range.
Important Note: cat /proc/sys/net/ipv4/ip_local_port_range
will display actual values. To modify it, youll have to
choose between 3 methods:
- echo 1024 65535 > /proc/sys/net/ipv4/ip_local_port_range
- sudo sysctl -w net.ipv4.ip_local_port_range="1024 65535"
- You may need to edit /etc/sysctl.conf file, to make changes
to /proc filesystem permanently i.e. append the following to
your /etc/sysctl.conf file:
# increase system IP port limits
net.ipv4.ip_local_port_range = 1024 65535
Recycling of Time Wait connections (tcp_tw_recycle)
It enables fast recycling of TIME_WAIT sockets. The default
value is 0 (disabled). The sysctl documentation incorrectly
states the default as enabled. It can be changed to 1
(enabled) in many cases.
Important Note: Known to cause some issues with host
stated (load balancing and fail over) if enabled, should be
used with caution.
0 1
Reuse of Time Wait connections (tcp_tw_reuse)
This allows reusing sockets in TIME_WAIT state for new
connections when it is safe from protocol view point.
Default value is 0 (disabled).
Important Note: It is generally a safer alternative to
tcp_tw_recycle, especially when lot of TIME_WAITs are
generated
0 1
For more information about Linux tuning, you should consult your Linux
documentation. Note the above TCP/kernel tunable parameters and their
corresponding values are provided as examples and rough guidelines only. You can
re-adjust these and other parameters based on actual system load, usage patterns
such as the number of concurrent users and sessions and so on.
9
2.1.2 Windows 2008 Tuning Parameters
This topic describes how to tune the Windows 2008 operating system to optimize
the performance of your Oracle EPM System.
Parameter Default
Value
Suggested Value
MaxUserPort
Under heavy loads it may be necessary to adjust the
MaxUserPort. This parameter determines the availability of
user ports requested by an EPM application e.g. SmartView,
Planning etc.
Tip: By default, the start port is 49152, and the default end
port is 65535, this means there are 16383 usable dynamic
ports. *Use the following netsh command to configure start
port and the range:
netsh int ipv4 set dynamicport tcp start=1025 num=64508
Important Note: If you are using firewalls to restrict traffic on
your internal network you will need to update the
configuration of those firewalls to compensate for the new
port range. You can view the current dynamic port range on
the server by using the following netsh command:
netsh int ipv4 show dynamicport tcp
16383 65532
Receive Window Auto-Tuning Level
Windows Server 2008 has a TCP window scaling feature. The
operating system can adjust the TCP receive window size to
maximize the network performance. However, some old
routers, firewalls and operating systems that are incompatible
with the window scaling feature may cause slow data transfer
or lost of connectivity.
At the command prompt, run the following command:
netsh interface tcp show global
The default value for the autoTuningLevel parameter is listed
as Receive Window Auto-Tuning Level:
Important Note: In Windows 2008 SP1/SP2, If you try to
disabled (auto-tuning) and then again set it to normal, then it
is strongly recommended to read MS KB article:
http://support.microsoft.com/kb/967224
normal If your user
experience slow
network performance
(e.g. server sends
data very slowly or
drops the data
completely) then only
disable the TCP
window scaling feature
on Windows Server
2008.
10
Set the power option setting to "High performance"
With High Performance power scheme, processors are always
locked at the highest performance state. Refer to below
section 2.1.2.1 for performance benchmark on two power
plans. Important Note: For more details, read MS KB article:
http://support.microsoft.com/kb/2207548
Tip: How-to change a power plan:
1. Under Control Panel click on Power Options.
2. Choose the High Performance option
3. Close the Power Option window.
Balanced High Performance
Disable Data Execution Prevention (DEP)
On Windows Vista and later, DEP and PAE are enabled at
boot time and are configured by setting values for the nx and
pae parameters using the BCDEdit /set command.
To disable DEP on Windows Server 2008 type the following
on an elevated Command Line.
bcdedit /set nx AlwaysOff /pae
Important Note: Available documentation is here :
http://msdn.microsoft.com/en-us/library/ff542202.aspx
http://msdn.microsoft.com/en-us/library/ff542275.aspx
bcdedit
/set nx
AlwaysOn
bcdedit /set nx
AlwaysOff /pae
Scanning mechanism for ORACLE_HOME folder
When using EPM or java based products, you should turn off
scanning mechanisms (for example, anti-virus, Windows
Automatic Updates). Running these in conjunction with EPM
or java based products may result in unexpected behavior.
When enabled noticed very negative impact on performances
by giving extra CPU time (up to 50%) and slowness on
processes running.
Scan all Exception for
ORACLE_HOME
folder
11
2.1.2.1 Performance benchmark using Balanced versus High Performance
HFM consolidation test showed 25% performance improvement when using the
"High Performance" power plan versus the "Balanced" (default) power plan. Note on
HFM application / database servers it has been observed CPU core down-clocking
from 2.66GHz to 1.5GHz. This means that Balanced power management is in
effect on the Xeon processor, which has lowered its clock speed to save power.
12
2.1.3 Windows 2003 Tuning Parameters
This topic describes how to tune the Windows 2003 operating system to optimize
the performance of your Oracle EPM System.
Parameter Default Value Suggested Value
TcpTimedWaitDelay
This parameter controls the amount of time the OS
waits to reclaim a port after an application closes a TCP
connection, has a default value of 4 minutes. During a
heavy users load, these limits may be exceeded
resulting in an address in use: connect exception.
Tip: In registry set this parameter using following:
HKLM\System\CurrentControlSet\Services\Tcpip\Para
meters
Value: TcpTimedWaitDelay
Value Type: dword
Data: 30 (decimal)
240 30
Disable Data Execution Prevention (DEP)
Data Execution Prevention (DEP) is a set of hardware
and software technologies that perform additional
checks on memory to help prevent malicious code from
running on a system. In Microsoft Windows 2003
Server, DEP is enforced by hardware and by software.
The problem is that java and EPM programs are
considered as malicious codes by default, which will
have a very negative impact on performances by giving
extra CPU time (up to 50%) and slowness on processes
running.
To completely disable DEP in Windows Server 2003,
perform the following with administrative credentials:
1. Open Windows Explorer
2. Tools > Folder Options > View
3. Uncheck Hide Protected operating system files
(Recommended) and Hide extensions for known file
types
4. Click apply > OK
5. Browse to C:\
6. Right click on boot.ini, select properties and ensure
the read-only tab is unchecked and click OK
7. Edit boot.ini
8. Modify the /noexecute=
For example, set /noexecute=AlwaysOff to disable DEP
entirely
9. File > Save, close boot.ini file
10. Right click on boot.ini, select properties and ensure
the read-only tab is checked and click OK
11. Reboot the computer
Important Note: Available documentation is here :
/noexecute=alwayso
n /pae
/noexecute=always
off /pae
13
http://msdn.microsoft.com/en-us/library/ff542275.aspx
http://support.microsoft.com/kb/875352
Scanning mechanism for ORACLE_HOME folder
When using EPM or java based products, you should
turn off scanning mechanisms (for example, anti-virus,
Windows Automatic Updates). Running these in
conjunction with EPM or java based products may
result in unexpected behavior. When enabled we saw
very negative impact on performances by giving extra
CPU time (up to 50%) and slowness on processes
running.
Scan all Exception for
ORACLE_HOME
folder
MaxUserPort
The number of user-accessible ephemeral ports that can
be used to source outbound connections is configurable
using this parameter.
Tip: In registry set this parameter using following:
HKLM\System\CurrentControlSet\Services\Tcpip\Para
meters
Value: MaxUserPort
Value Type: dword
Data: 65534 (decimal)
5000 65534
Tune Windows 2003 (x86 32 bit) /3GB switch
Important Note: Using only the /3GB switch allocates 1 GB to the kernel and 3 GB to the User-mode
space. Therefore it is strongly recommended to implement following settings when /3GB switch is
applied, in order to make sure Essbase, HFM and Windows stability:
1. Using a range of memory for the /userva=xxxx switch that is within the range of 2900 to 3030.
The following sample boot.ini file demonstrates how to use the new switch to tune a server to allocate
2,900 MB of User-mode virtual memory and 1,196 MB of Kernel-mode virtual memory. This increases
the available kernel space by 172 MB:
[boot loader]
timeout=30
default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows Server 2003, Enterprise" /fastdetect
/NoExecute=OptOut /3GB /Userva=2900
2. Increase the SystemPages in registry: The setting being updated controls the allocable memory for
the operational system caches, file caches among others, and the value being set will tell Windows to
calculate the value itself on the side of the maximum allowable resources. HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management
Open the Data: SystemPages
Enter its new value (hex): FFFFFFFF
3. After applying above settings, reboot the server.
14
2.1.4 AIX Tuning Parameters
This topic describes how to tune the AIX (6.1 / 7.x) operating system to optimize the
performance of your Oracle EPM System.
Parameter Default Value Suggested Value
tcp_timewait
By reducing the value of this entry, TCP/IP can
release closed connections faster, providing more
resources for new connections. Tip: Set using command no -r -o tcp_timewait=1
1 1
tcp_keepidle
KeepAlive packet ensures that a connection stays in
an active/ESTABLISHED state. Tip: Set using command no -r -o tcp_keepidle=600
14400
(2 hours)
600
(5 minutes)
tcp_keepintvl
Specifies the interval between packets that are sent
to validate the connection. Tip: Set using command no -r -o tcp_keepintvl=10
150 10
tcp_keepinit
Specifies the initial timeout value for TCP
connection. Tip: Set using command no -r -o tcp_keepinit=40
150 40
Listen Backlog
Increase this to handle peak incoming connection
rates under heavy load. Tip: Set using command no -r -o somaxconn=8192
1024 8192
Socket send buffer size
Tip: Set using command no -r -o tcp_sendspace=16384 16384 For optimal value for your
Ethernet adapter, consult
IBM support.
Socket receive buffer size Tip: Set using command no -r -o tcp_recvspace=16384
16384
For optimal value for your
Ethernet adapter, consult
IBM support.
User Limit Parameter
It is highly recommended to apply the following values to set for user limits when running Essbase
(x64bit) Server on a 64-bit AIX platform. Tip: ulimit -a command displays all the ulimit limits.
time (seconds) Default unlimited
file (blocks) Default unlimited
data (kbytes) Default unlimited
stack (kbytes) Default unlimited on 64-bit and
64 MB on 32-bit.
memory (kbytes) Default unlimited
coredump (blocks) Default unlimited
nofiles (descriptors)
Important Note: In majority of deployments
Essbase does not need too many files to be open.
It is pure waste of resources.
Default 8192 on 64 bit and 4096
on 32 bit.
15
AIXTHREAD_COND_DEBUG
Maintains a list of condition variables for use by
the debugger. If the program (e.g. Essbase)
contains large number of active condition
variables and frequently creates and destroys
condition variables, this might create higher
overhead for maintaining the list of condition
variables. Setting the variable to OFF disables the
list.
-
AIXTHREAD_COND_
DEBUG=OFF
-
AIXTHREAD_COND_DEB
UG=OFF
AIXTHREAD_MUTEX_DEBUG
Maintains a list of active mutexes for use by the
debugger.
If the program (e.g. Essbase) contains a large
number of active mutexes and frequently creates
and destroys mutexes, this might create higher
overhead for maintaining the list of mutexes.
Leaving the variable set to OFF disables the list.
-
AIXTHREAD_MUTEX_
DEBUG=OFF
-
AIXTHREAD_MUTEX_DE
BUG=OFF
AIXTHREAD_MNRATIO
Controls the scaling factor of the library. This ratio
is used when creating and terminating pthreads.
Two positive values (p:k), where k is the number
of kernel threads that should be employed to
handle the number of executable pthreads defined
in the p variable. Might be useful for applications
with a very large number of threads. However,
always test a ratio of 1:1 because it might provide
better performance.
-
AIXTHREAD_MNRATI
O=8:1
-
AIXTHREAD_MNRATIO=
1:1
AIXTHREAD_MUTEX_FAST
Enables the use of the optimized mutex locking
mechanism.
Setting the variable to ON forces threaded
applications to use an optimized mutex locking
mechanism, resulting in increased performance.
If the program experiences performance
degradation due to heavy mutex contention, then
setting this variable to ON will force the pthread
library to use an optimized mutex locking
mechanism that works only on process private
mutexes. These process private mutexes must be
initialized using the pthread_mutex_init routine
and must be destroyed using the
pthread_mutex_destroy routine.
-
AIXTHREAD_MUTEX_
FAST=OFF
-
AIXTHREAD_MUTEX_FA
ST=ON
AIXTHREAD_SCOPE
Controls contention scope. A value of P signifies
process-based contention scope (M:N). A value of
S signifies system-based contention scope (1:1). If
fewer threads are being dispatched than expected,
try system scope.
-
AIXTHREAD_SCOPE=
P
-AIXTHREAD_SCOPE=S
16
YIELDLOOPTIME
Controls the number of times to yield the
processor before blocking on a busy lock (only for
libpthreads). The processor is yielded to another
kernel thread, assuming there is another runnable
kernel thread with sufficient priority.
If threads are going to sleep often (lot of idle
time), then the YIELDLOOPTIME might not be
high enough.
Increasing the value from the default value of 0
may benefit you if you do not want the threads to
go to sleep while they are waiting for locks.
- YIELDLOOPTIME=0 -YIELDLOOPTIME=20
SPINLOOPTIME
Controls the number of times the system will
retry a busy lock before yielding to another
process. The default value is 40. This should be
increased to 500 or higher because a busy lock
retry is inexpensive compared to the alternative.
Use the tprof command to determine if the
check_lock routine has high CPU usage. If it does,
you can increase the value even more.
- SPINLOOPTIME=40 -SPINLOOPTIME=500
MALLOCOPTIONS
This environment variable enables the use of
multiple memory heaps. In a highly threaded
environment (e.g. Essbase), many threads can
request memory at the same time. With only one
heap, these requests can serialize and create wait
situations. With this variable, multiple heaps are
generated that are used to serve memory-
allocations requests and reduce memory-
allocation latency
Tip: Malloc multiheap is not enabled by default. It
is enabled and configured by setting the
MALLOCOPTIONS environment variable. To
enable malloc multiheap with the default settings,
set MALLOCOPTIONS=multiheap in Essbase
startup environment file.
Disabled -
MALLOCOPTIONS=multi
heap:16,considersize
For more information about AIX tuning, you should consult your AIX vendors
documentation. Note the above TCP/kernel tunable parameters and their
corresponding values are provided as examples and rough guidelines only. You can
adjust these and other parameters based on actual system load, usage patterns such
as the number of concurrent users and sessions and so on.
17
Important Note to remember for Essbase only: In some cases for AIX 5.3/6.1 the
default Virtual Memory Manager (VMM) settings are not optimal for an Essbase
environment. Common symptoms include high paging, high system CPU time and
slow overall system throughput. Therefore in this scenario, it is recommended to
lower maxperm/maxclient to get good performance.
Important Note to remember for Essbase BSO only: In some cases for AIX
5.3/6.1/7.x the disks are configured in JFS2 for Essbase. AIX classifies memory into
two categories: file cache and working storage. Over time, memory use grows with
both types of memory. AIX will eventually need to free memory and it can do so by
freeing either file cache or working storage. To maximize performance, you would
want AIX to release the type of memory not being used by the application. So for
databases, which use working storage, you would want AIX to release file cache
memory first. To do so in the past, you would put a strict limit on JFS2 file cache
memory using maxclient. It is not unusual to see database servers with file cache
limited to 15% of total memory. However, a strict limit is not always desirable.
The new "lru_file_repage" option solves this problem. By setting "lru_file_repage=0",
AIX only frees file cache memory. This guarantees working storage stays in
memory, and allows file cache to grow.
So in the past you might have set maxclient at 20% for database servers. Today you
could set maxclient at 90% and "lru_file_repage=0". The exact setting will vary
based on your application and amount of memory. Contact IBM Support if you need
help determining the optimum setting.
Full example on AIX for Essbase:
vmo -p -o maxperm%=90 -o maxclient%=90 -o minperm%=3 -o lru_file_repage=0
Useful documentations:
http://www-01.ibm.com/support/docview.wss?uid=swg21328602
https://www.ibm.com/developerworks/community/blogs/aixpert/entry/ne
w_aix_memory_tuning_option?lang=en
18
2.1.5 Oracle Sun Solaris Tuning Parameters
This topic describes how to tune the Solaris operating system to optimize the
performance of your Oracle EPM System. Note: Solaris default network tuning
parameters should be sufficient in most cases. For heavy, network centric
workloads, its always best practice to increase the size of key TCP-related
parameters to improve system performance and reduce dropped connections.
Parameter Default Value Suggested Value
File descriptors (nofiles)
Important Note: On any box that hosts an Oracle
HTTP Server / WebLogic Server / Essbase server it is
recommended to increase nofiles (descriptors) to
optimal value. On Solaris this means setting the
rlim_fd_cur in the /etc/system file and rebooting the
box.
Please make sure to check your operating system
documentation and release notes; there are some
negative performance implications on some older
versions of Solaris if you set these numbers too high.
Tip: The syntax for adjusting this parameter in the
/etc/system file is shown below:
set rlim_fd_cur=16384
256 16384
tcp_time_wait_interval
It is recommended to reduce this setting to 60,000
milliseconds, or 1 minute; this is the default value starting
in Solaris 9/10. By reducing the value of this entry,
TCP/IP can release closed connections faster, providing
more resources for new connections.
Tip: The command netstat -a | grep TIME_WAIT | wc -l
will count all of the TCP connections that are in the
TIME_WAIT state. As this number approaches the
maximum number of file descriptors per process, your
applications throughput will suffer dramatic
degradations because new connection requests may
have to wait for a free space in the applications file
descriptor table. Tip: Set using command:
ndd -set /dev/tcp tcp_time_wait_interval 30000
60000
30000
tcp_xmit_hiwat (Solaris 9/10) / send_buf (Solaris 11)
The tcp_xmit_hiwat (tcp_send_buf) and
tcp_recv_hiwat (tcp_recv_buf) parameters control the
default size of the send window and receive window
for each TCP connection. On very fast networks, you
should make sure that the values are set to at least 32K.
By default, Solaris 9 changes the default settings for
49152 This parameter is calculated based on the network
bandwidth and latency. You
could follow this rule to
calculate:
*
* 8
[bits/byte]
19
both parameters to 48K.
tcp_recv_hiwat (Solaris 9/10) / recv_buf (Solaris 11)
The tcp_xmit_hiwat (tcp_send_buf) and
tcp_recv_hiwat (tcp_recv_buf) parameters control the
default size of the send window and receive window
for each TCP connection, respectively. On very fast
networks, you should make sure that the values are set
to at least 32K. By default, Solaris 9 changes the default
settings for both parameters to 48K.
49152 (in Solaris 9/10)
128000 (in Solaris 11)
This parameter is calculated
based on the network
bandwidth and latency. You
could follow this rule to
calculate:
*
* 8
[bits/byte]
tcp_conn_hash_size (Solaris 9) / ipcl_conn_hash_size
(Solaris 10/11)
This parameter controls the size of a hash table that
helps quickly locate the TCP sockets data structure in
the kernel. If the size is too small, it will result in long
hash chains in each bucket that force the operating
system into a linear search for the socket entry of
interest, and performance will suffer accordingly.
512 (in Solaris 9)
0 (in Solaris 10/11)
1024 (in Solaris 9)
In Solaris 10, the default
value of 0 means that the
system automatically
sizes an appropriate
value for this parameter
at boot time, depending
on the available
memory.
Connection backlog
This parameter controls the maximum allowable
number of completed connections waiting to return
from an accept call (that have completed the three-way
TCP connection handshake). You should increase this
parameter only if you notice that your system is
dropping connections.
Increase these values if you do see non-zero values for
tcpListenDrop in the output for the following
command:
netstat s | fgrep i listendrop
Increase the values in steps of 256 starting from the
default value
Tip: Set using commands:
ndd -set /dev/tcp tcp_conn_req_max_q 1024
ndd -set /dev/tcp tcp_conn_req_max_q0 4096
128 (max_q)
1024 (max_q0)
1024 (max_q)
4096 (max_q0)
Important Note: It is important to keep systems up to date with Solaris patches
because the patches provide fixes for security vulnerabilities, performance, and
other improvements of the OS. Certain patches specifically target the Java platform, which
benefits WebLogic Server in regards to performance and functionalities.
20
2.2 Tune Oracle WebLogic Server (WLS) parameters
This topic describes how to tune the WebLogic Server to optimize the performance
of your Oracle EPM System.
2.2.1 Tuning JDBC Data Sources
You can improve EPM system performance and stability by properly configuring
the attributes in JDBC data sources in your EPMSystem domain.
2.2.1.1 Increase the Number of Connection Pool
If EPM data sources are running out of connections to the relational database then
you need to set the maximum number of connections to high value for the following
EPM data sources:
Important Note: Reason why the default value initial and maximum capacity size
are set to lower value because it is important to protect database resources. So make
sure you have enough resources on the database server (available memory is a constraint). If
you increase initial / maximum capacity to very high value, you may exceed the limit of
PROCESSES / OPEN_CURSORS on Oracle database server.
EPM Data Source Name Default Value Suggested Value
EPMSystemRegistry
Initial Capacity: 1
Maximum Capacity: 150
Initial Capacity: 10*
Maximum Capacity: 150
*Note: Each managed server target to
EPMSystemRegistry data source creates
1 pool on start-up. Ensure you have
enough Oracle processes; refer to Oracle
Database Tuning section of this guide.
raframework_datasource Initial Capacity: 1
Maximum Capacity: 15
Initial Capacity: 1
Maximum Capacity: 150
planning_datasource Initial Capacity: 1
Maximum Capacity: 15
Initial Capacity: 1
Maximum Capacity: 150
financialclose_datasource (FCM) Initial Capacity: 1
Maximum Capacity: 30
Initial Capacity: 1
Maximum Capacity: 150
eas_datasource (Essbase Admin
Services)
Initial Capacity: 1
Maximum Capacity: 15
Initial Capacity: 1
Maximum Capacity: 30
calc_datasource (CalcMgr) Initial Capacity: 1
Maximum Capacity: 15
Initial Capacity: 1
Maximum Capacity: 30
profitability_datasource (Profitability) Initial Capacity: 1
Maximum Capacity: 15
Initial Capacity: 1
Maximum Capacity: 30
hps_datasource (HpsAlerter,
HpsWebReports)
Initial Capacity: 1
Maximum Capacity: 15
Initial Capacity: 1
Maximum Capacity: 30
discman_datasource
(DisclosureManagement)
Initial Capacity: 1
Maximum Capacity: 15
Initial Capacity: 1
Maximum Capacity: 30
aif_datasource (ErpIntegrator) Initial Capacity: 1
Maximum Capacity: 120
Initial Capacity: 1
Maximum Capacity: 200
Important Note: You need to determine the optimal value for the Initial and Maximum
Capacity as part of your pre-production performance testing. Once optimal values are
21
determined then set the value of Initial Capacity equal to the value for Maximum
Capacity in order to boost performance of JDBC in WebLogic Server web
applications.
Tip: Here is the example on how to size open cursors value on Oracle database, if
you have a data source with 150 connections deployed on 2 servers and if you use
default Statement Cache Size 10, you may open 3000 (150 x 2 x 10) cursors on your
database server for the cached statements. You can increase the open cursors limit in
Oracle database server. See section Tune Oracle Database Parameters in this guide.
Tip: To increase this value:
1. Log in to the WebLogic Server administration console.
2. Select your data source (for example EPMSystemRegistry), and then click on
Connection Pool tab, and then Maximum Capacity.
Tip: Monitor the EPM data source connections from Console (Deployments-
>Monitoring->JDBC) i.e. Active Connections Count and if wait for connections has a
greater than 0 value then increase the pool size.
22
2.2.2 Tune Connection Backlog Buffering
Tune the number of connection requests that a WebLogic Server instance will accept
before refusing additional requests. The Accept Backlog parameter specifies how
many Transmission Control Protocol (TCP) connections can be buffered in a wait
queue.
Important Note: In WebLogic Server 10.3.x, it is not required to increase Accept
Backlog as default listen thread Accept Backlog value is increased to 300 which is a
better default value.
2.2.3 Stuck thread detection behavior Tuning
WebLogic Server automatically detects when a thread in an execute queue becomes
stuck. Because a stuck thread cannot complete its current work or accept new
work, the server logs a message each time it diagnoses a stuck thread.
Some operations (for example, long running large FR / WA reports, Database /
Essbase call, computation loop inside an application) can get into an infinite
loop. This can cause the thread to be stuck with that operation for whatever time
it takes to complete that operation.
Performance of the server can degrade significantly if many such threads get
stuck.
Tip: Changing the stuck threads values will only change the detection of stuck
threads. This does not mean that there are no stuck threads any more. Only the
reporting is changing. Meaning how long usually a specific operation should take in
an EPM application. If such an operation takes longer than 600 seconds then you
should increase the stuck thread value to a higher one to prevent the reporting of a
stuck thread. The performance is not impacted or controlled through the stuck
thread detection. WebLogic Server will not "delete" or "kill" a thread in case it
becomes stuck.
Use the below Stuck Threads suggested settings as a starting point, and then, after
careful testing of your long running transactions, adjust as needed.
EPM Managed Servers Default Value Suggested Value
Planning server Stuck Thread Max Time=600
Stuck Thread Timer Interval=60
Max Stuck Thread Time=600
Stuck Thread Max Time=1200
Stuck Thread Timer Interval=1200
Max Stuck Thread Time=1200
Foundation Services
server
Stuck Thread Max Time=600
Stuck Thread Timer Interval=60
Max Stuck Thread Time=600
Stuck Thread Max Time=1200
Stuck Thread Timer Interval=1200
Max Stuck Thread Time=1200
EPMA Web & DataSync
server
Stuck Thread Max Time=600
Stuck Thread Timer Interval=60
Max Stuck Thread Time=600
Stuck Thread Max Time=1200
Stuck Thread Timer Interval=1200
Max Stuck Thread Time=1200
23
Financial Reporting
server
Stuck Thread Max Time=600
Stuck Thread Timer Interval=60
Max Stuck Thread Time=600
Stuck Thread Max Time=1200
Stuck Thread Timer Interval=1200
Max Stuck Thread Time=1200
Web Analysis server Stuck Thread Max Time=600
Stuck Thread Timer Interval=60
Max Stuck Thread Time=600
Stuck Thread Max Time=1200
Stuck Thread Timer Interval=1200
Max Stuck Thread Time=1200
Provider Services server Stuck Thread Max Time=600
Stuck Thread Timer Interval=60
Max Stuck Thread Time=600
Stuck Thread Max Time=1200
Stuck Thread Timer Interval=1200
Max Stuck Thread Time=1200
Profitability and Cost
Management server
Stuck Thread Max Time=600
Stuck Thread Timer Interval=60
Max Stuck Thread Time=600
Stuck Thread Max Time=1200
Stuck Thread Timer Interval=1200
Max Stuck Thread Time=1200
Essbase Administration
Services
Stuck Thread Max Time=600
Stuck Thread Timer Interval=60
Max Stuck Thread Time=600
Stuck Thread Max Time=1200
Stuck Thread Timer Interval=1200
Max Stuck Thread Time=1200
RA Framework Web Stuck Thread Max Time=600
Stuck Thread Timer Interval=60
Max Stuck Thread Time=600
Stuck Thread Max Time=1200
Stuck Thread Timer Interval=1200
Max Stuck Thread Time=1200
Calc Mgr Stuck Thread Max Time=600
Stuck Thread Timer Interval=60
Max Stuck Thread Time=600
Stuck Thread Max Time=1200
Stuck Thread Timer Interval=1200
Max Stuck Thread Time=1200
ErpIntegrator Stuck Thread Max Time=600
Stuck Thread Timer Interval=60
Max Stuck Thread Time=600
Stuck Thread Max Time=1200
Stuck Thread Timer Interval=1200
Max Stuck Thread Time=1200
FM Web Services Stuck Thread Max Time=600
Stuck Thread Timer Interval=60
Max Stuck Thread Time=600
Stuck Thread Max Time=1200
Stuck Thread Timer Interval=1200
Max Stuck Thread Time=1200
Tip: To configure stuck thread detection behavior, click on each EPM Managed
Server and then click on Tuning tab:
2.2.4 Enable Native IO Performance Pack
Performance can be improved by enabling native IO in production mode. WebLogic
Server benchmarks show major performance improvements when native
performance packs are used on machines that host Oracle WebLogic Server
instances.
Important Note: Always enable native I/O, if available, and check for errors at
startup to make sure it is being initialed properly.
Tip: The use of NATIVE performance packs are enabled by default in the
configuration shipped with your distribution. You can use the Administration
Console to verify that performance packs are enabled by clicking on each managed
server and click on Tuning tab.
24
2.3 Tune IBM WebSphere Server (IBM WAS) parameters
This topic describes how to tune the IBM WebSphere Server to optimize the
performance of your Oracle EPM System.
Required documentation to read before proceeding:
http://docs.oracle.com/cd/E40248_01/epm.1112/epm_install/ch08.html
http://www.oracle.com/technetwork/middleware/bi-foundation/epm-websphere-
manual-deploy11122-1887166.pdf
http://pic.dhe.ibm.com/infocenter/wasinfo/v7r0/topic/com.ibm.websphere.base.doc/i
nfo/aes/ae/welc6toptuning.html
http://pic.dhe.ibm.com/infocenter/wasinfo/v7r0/index.jsp?topic=%2Fcom.ibm.websp
here.express.doc%2Finfo%2Fexp%2Fae%2Ftprf_hotspot_jvm.html
2.3.1 Tuning JDBC Data Sources
You can improve EPM system performance and stability by properly configuring
the attributes in JDBC data sources in your EPMSystem domain.
2.3.1.1 Increase the Number of Connection Pool
If EPM data sources are running out of connections to the relational database then
you need to set the maximum number of connections to high value for the following
EPM data sources:
IBM WAS behavior is the following:
Each time an application attempts to access a back-end store (such as a database), it
requires resources to create, maintain, and release a connection to that data store.
To mitigate the strain that this process can place on overall application resources, the
application server enables you to establish a pool of back-end connections that
applications can share on an application server.
Connection pooling spreads the connection overhead across several user requests,
thereby conserving application resources for future requests.
Important Note: Reason why the default value initial and maximum capacity size
are set to lower value because it is important to protect database resources. So make
sure you have enough resources on the database server (available memory is a constraint). If
you increase initial / maximum capacity to very high value, you may exceed the limit of
PROCESSES / OPEN_CURSORS on Oracle database server.
EPM Data Source Name Default Value Suggested Value
EPMSystemRegistry
Minimum connections: 1
Maximum connections: 10
Minimum connections: 10*
Maximum connections: 150
25
raframework_datasource Minimum connections: 1
Maximum connections: 10
Minimum connections: 1
Maximum connections: 150
planning_datasource Minimum connections: 1
Maximum connections: 10
Minimum connections: 1
Maximum connections: 150
financialclose_datasource (FCM) Minimum connections: 1
Maximum connections: 10
Minimum connections: 1
Maximum connections: 150
eas_datasource (Essbase Admin
Services)
Minimum connections: 1
Maximum connections: 10
Minimum connections: 1
Maximum connections: 30
calc_datasource (CalcMgr) Minimum connections: 1
Maximum connections: 10
Minimum connections: 1
Maximum connections: 30
profitability_datasource (Profitability) Minimum connections: 1
Maximum connections: 10
Minimum connections: 1
Maximum connections: 30
hps_datasource (HpsAlerter,
HpsWebReports)
Minimum connections: 1
Maximum connections: 10
Minimum connections: 1
Maximum connections: 30
discman_datasource
(DisclosureManagement)
Minimum connections: 1
Maximum connections: 10
Minimum connections: 1
Maximum connections: 30
aif_datasource (ErpIntegrator) Minimum connections: 1
Maximum connections: 10
Minimum connections: 1
Maximum connections: 200
Important Note: You need to determine the optimal value for the Minimum and Maximum
connection as part of your pre-production performance testing. Once optimal values are
determined then set the value of Minimum connection equal to the value for
Maximum connection in order to boost performance of JDBC in IBM Websphere
Application Server web applications.
Tip: To increase this value:
3. Log in to the IBM Websphere Application Server administration console.
4. Select Resources => JDBC => Data Sources => data_source (for example
EPMSystemRegistry) => Connection pool properties and then input some
values in Minimum and Maximum connection boxes.
26
Tip: Monitor the EPM data source connections from IBM WAS Console (Tivoli
Performance Viewer) i.e. Free Pool Size and if Wait Time has a greater than 0
value then increase the pool size.
2.3.2 Tune Thread Pool
Each task performed by the server runs on a thread obtained from one of
WebSphere Application Servers many thread pools. A thread pool enables
components of the server to reuse threads, eliminating the need to create new
threads at run time to service each new request. Three of the most commonly used
(and tuned) thread pools within the application server are:
Web container: Used when requests come in over HTTP.
Default: Used when requests come in for a message driven bean or if a particular
transport chain has not been defined to a specific thread pool.
ORB: Used when remote requests come in over RMI/IIOP for an enterprise bean
from an EJB application client, remote EJB interface, or another application server.
Use the below Thread Pool suggested settings as a starting point, and then, after
careful testing of your long running transactions, adjust as needed.
EPM Managed Servers Web Container Default ORB
Planning server Minimum Size =50
Maximum Size = 300
Minimum Size =20
Maximum Size = 100
Minimum Size =10
Maximum Size = 50
Foundation Services server Minimum Size =50 Minimum Size =20 Minimum Size =10
27
Maximum Size = 300 Maximum Size = 100 Maximum Size = 50
EPMA Web & DataSync
server
Minimum Size =50
Maximum Size = 300
Minimum Size =20
Maximum Size = 100
Minimum Size =10
Maximum Size = 50
Financial Reporting server Minimum Size =50
Maximum Size = 300
Minimum Size =20
Maximum Size = 100
Minimum Size =10
Maximum Size = 50
Web Analysis server Minimum Size =50
Maximum Size = 300
Minimum Size =20
Maximum Size = 100
Minimum Size =10
Maximum Size = 50
Provider Services server Minimum Size =50
Maximum Size = 300
Minimum Size =20
Maximum Size = 100
Minimum Size =10
Maximum Size = 50
Profitability and Cost
Management server
Minimum Size =50
Maximum Size = 300
Minimum Size =20
Maximum Size = 100
Minimum Size =10
Maximum Size = 50
Essbase Administration
Services
Minimum Size =50
Maximum Size = 300
Minimum Size =20
Maximum Size = 100
Minimum Size =10
Maximum Size = 50
RA Framework Web Minimum Size =50
Maximum Size = 300
Minimum Size =20
Maximum Size = 100
Minimum Size =10
Maximum Size = 50
Calc Mgr Minimum Size =50
Maximum Size = 300
Minimum Size =20
Maximum Size = 100
Minimum Size =10
Maximum Size = 50
ErpIntegrator Minimum Size =50
Maximum Size = 300
Minimum Size =20
Maximum Size = 100
Minimum Size =10
Maximum Size = 50
FM Web Services Minimum Size =50
Maximum Size = 300
Minimum Size =20
Maximum Size = 100
Minimum Size =10
Maximum Size = 50
Tip: Thread pool settings can be changed in the administrative console by
navigating to Servers => Application Servers => server_name => Thread Pool
28
Tip: Monitoring Thread Pool Size using IBM WAS Console
2.3.3 Increasing JVM Size of Admin Server
In order to make sure successful deployments of several EPM web applications. It is
highly recommended to tune the Admin Server java heap size.
Tip: This can be done in the WebSphere Application Server administrative console
by navigating to Servers => Application servers => server_name => Process definition
=> Java Virtual Machine.
29
Then
Then
You will see here that you can modify Initial and Maximum heap size. (Should be
set both to 2048)
2.3.4 Tuning Servlet Caching
WebSphere Application Servers DynaCache provides a general in-memory caching
service for objects and page fragments generated by the server. The DistributedMap
and DistributedObjectCache interfaces can be used within an application to cache
and share Java objects by storing references to these objects in the cache for later use.
Servlet caching, on the other hand, enables servlet and JSP response fragments to be
stored and managed by a customizable set of caching rules. Servlet caching can be
30
enabled in the administrative console by navigating to Servers => Application
servers => server_name => Web container settings => Web container. The URI path to
the servlet or JSP to be cached must be defined in a cachespec.xml file, which is
placed inside the Web module's WEB-INF directory.
31
2.3.5 Tuning HTTP Transport persistent connections
Persistent connections specify that an outgoing HTTP response should use a
persistent (keep-alive) connection instead of a connection that closes after one
request or response exchange occurs. In many cases, a performance boost can be
achieved by increasing the maximum number of persistent requests that are
permitted on a single HTTP connection.
SSL connections can see a significant performance gain by enabling unlimited
persistent requests per connection because SSL connections incur the costly
overhead of exchanging keys and negotiating protocols to complete the SSL
handshake process. Maximizing the number of requests that can be handled per
connection minimizes the impact of this overhead.
Also, high throughput applications with fast response times can realize a
performance gain by keeping the connections open, rather than building up and
closing the connection on each request. When this property is set to 0 (zero), the
connection stays open as long as the application server is running.
However, if security is a concern, then careful consideration should be placed on
this setting, as this parameter can help prevent denial of service attacks when a
client tries to hold on to a keep-alive connection.
HTTP transport persistent connections settings can be set in the administrative
console by navigating to Servers => Application servers => server_name => Ports.
Once there, click on View associated transports for the port associated with the
HTTP transport channel settings you want to change:
32
2.3.6 IBM WebSphere PMI (Performance Monitoring Infrastructure):
Available via IBM WAS Console, browsing to Application servers > > Performance Monitoring Infrastructure (PMI)
2.3.7 IBM WebSphere PMI (Performance Monitoring Infrastructure):
Useful to monitor and analyze your application performances
Available here:
http://www.ibm.com/developerworks/websphere/downloads/peformtuning.html#F
ORUM144
33
2.4 Tune 64bit Java Virtual Machines (JVM)
Newly deployed web application server instances use default memory heap
settings, which are often too small to accommodate EPM System 64bit requirements.
Optimal JVM settings can be determined only by close monitoring of application
server performance under peak realistic load.
2.4.1 Tuning 64bit JRockit JVM
In JRockit JVM (R28), the heap grows faster than before. The JVM also ensures that
the heap size grows up to the maximum Java heap size (-Xmx) before an
OutofMemory error is thrown.
This table lists the memory setting, the suggested value (depending on available
physical memory on each server), and the Oracle JRockit Java argument:
Setting Suggested Value Java Argument*
Minimum heap
1024 MB -Xms1024m
Maximum heap
4096 MB -Xmx4096m
Important Note: Ensure that the sum of the maximum heap size (-Xmx) of all the EPM JVMs running
on your server does not exceed the amount of available physical RAM. If this value is exceeded, the
Operating System starts paging and performance degrades significantly.
For SOA, increase the size of
Compressed References to resolve native
OutOfMemory (OOM) error.
Tip: This can be done for other EPM web
components besides the SOA, only if you
encounter a native OutOfMemory error
despite memory being available.
32 GB -XXcompressedRefs:size=32GB
Thread-local areas (TLA) is used for
object allocation. This option (-XXtlaSize)
can be used to tune the size of the
thread-local areas.
Important Note: Only apply this setting if
you see java.lang.OutOfMemoryError: getNewTla
error in web app logs. You may also need to
consider increasing the physical memory on
server to permanently resolve the
getNewTla error occurrences.
min=128k
preferred=512k
wasteLimit=128k
-
XXtlaSize:min=128k,preferred=512k
,wasteLimit=128k
Note: It is recommended to use the default garbage collection mode i.e. -Xgc:throughput. Also large
heap reduces the garbage collection frequency and fragmentation, improving the throughput of the
application; however, a large heap increases the memory footprint of the Java process.
34
*Warning: Make sure you have enough physical memory allocated / available on
servers (deployed with several EPM 64bit web applications) and then use the
above settings as a starting point, and after load testing, adjust as needed. For a
stable / an even performance over time, you should set the initial heap size (-Xms) to
the same value as the maximum heap size (-Xmx).
2.4.2 Tuning 64bit Sun JVM
This table lists the memory setting, the suggested value, and the Suns Java
argument:
Setting Suggested Value Java Argument*
Minimum heap
1024 MB -Xms1024m
Maximum heap
4096 MB -Xmx4096m
Important Note: Ensure that the sum of the maximum heap size (-Xmx) of all the EPM JVMs
running on your server does not exceed the amount of available physical RAM. If this value is
exceeded, the Operating System starts paging and performance degrades significantly.
Minimum permanent generation heap
256 MB -XX:PermSize=256m
Maximum permanent generation heap If you receive java.lang.OutOfMemoryError:
PermGen space errors, you may also need to
increase the permanent generation space.
512 MB -XX:MaxPermSize=512m
Stack size for Java Threads
512 KB -Xss512k
*Warning: Make sure you have enough physical memory allocated / available on
servers (deployed with several EPM 64bit web applications) and then use the
above settings as a starting point, and after load testing, adjust as needed. For a
stable / an even performance over time, you should set the initial heap size (-Xms) to
the same value as the maximum heap size (-Xmx).
35
2.4.3 Tuning 64bit AIX JVM
This table lists the memory setting, the suggested value, and the AIXs Java
argument:
Setting Suggested Value Java Argument*
Minimum heap
1024 MB -Xms1024m
Maximum heap
4096 MB -Xmx4096m
Important Note: Ensure that the sum of the maximum heap size (-Xmx) of all the EPM JVMs
running on your server does not exceed the amount of available physical RAM. If this value is
exceeded, the Operating System starts paging and performance degrades significantly.
Garbage collector policy
gencon -Xgcpolicy:gencon
Nursery size
768 MB -Xmn768m
Stack size for Java Threads
2 MB -Xss2m
*Warning: Make sure you have enough physical memory allocated / available on
servers (deployed with several EPM 64bit web applications) and then use the
above settings as a starting point, and after load testing, adjust as needed. For a
stable / an even performance over time, you should set the initial heap size (-Xms) to
the same value as the maximum heap size (-Xmx).
2.4.4 Tuning Oracle ADF
We can tune default Application Module settings for Application Module (AM)
pools, connection pools & data sources as per our requirements. It's mostly depends
on use or access of an Application Module.
As a performance optimization, when an instance of an AM is returned to the pool
in "managed state" mode, the pool keeps track that the AM is referenced by that
particular session. The AM instance is still in the pool and available for use, but it
would prefer to be used by the same session that was using it last time because
maintaining this so-called "session affinity" improves performance.
Application Pool Parameters to take into consideration:
1. Pool Behavior Parameters
Failover Transaction State Upon Managed Release (jbo.dofailover)
Row-Level Locking Behavior Upon Release (jbo.locking.mode)
36
Disconnect Application Module Upon Release (jbo.doconnectionpooling)
Enable Application Module Pooling (jbo.ampool.doampooling)
Support Dynamic JDBC Credentials (jbo.ampool.dynamicjdbccredentials)
Reset Non-Transactional State Upon Unmanaged Release
(jbo.ampool.resetnontransactionalstate))
2. Pool Sizing Parameters
Initial Pool Size (jbo.ampool.initpoolsize)
Maximum Pool Size (jbo.ampool.maxpoolsize)
Referenced Pool Size (jbo.recyclethreshold)
Maximum Instance Time to Live (jbo.ampool.timetolive)
3. Pool Cleanup Parameters
Pool Polling Interval (jbo.ampool.monitorsleepinterval)
Maximum Available Size (jbo.ampool.maxavailablesize)
Minimum Available Size (jbo.ampool.minavailablesize)
Idle Instance Timeout (jbo.ampool.maxinactiveage)
Maximum Instance Time to Live (jbo.ampool.timetolive)
See that for more details:
http://docs.oracle.com/cd/E15523_01/web.1111/b31974/bcampool.htm
This table lists the memory setting, the suggested value (depending on available
physical memory on each server), and the Oracle Java argument:
Setting Suggested Value Java Argument*
Maximum Available Size
(jbo.ampool.maxavailablesize) The ideal
maximum number of available
application module instances in the pool
when not under abnormal load. When
the pool monitor wakes up to do
resource cleanup, it will try to remove
available application module instances
to bring the total number of available
instances down to this ideal maximum.
Instances that have been not been used
for a period longer than the idle instance
time-out will always get cleaned up at
this time, then additional available
instances will be removed if necessary to
400 -Djbo.ampool.maxavailablesize=400
37
bring the number of available instances
down to this size.
The default maximum available size is 25
instances. Configure this to leave the
maximum number of available instances
desired after a resource cleanup. A lower
value generally results in more
application module instances being
removed from the pool on a cleanup.
Referenced Pool Size
(jbo.recyclethreshold) The maximum
number of application module instances
in the pool that attempt to preserve
session affinity for the next request made
by the session which used them last
before releasing them to the pool in
managed-state mode.
The referenced pool size should always
be less than or equal to the maximum
pool size. The default is to allow 10
available instances to try and remain
"loyal" to the affinity they have with the
most recent session that released them in
managed state mode.
Configure this value to maintain the
application module instance's affinity to
a user's session. A general guideline is to
configure this to the expected number of
concurrent users that perform multiple
operations with short think times. If
there are no users expected to use the
application with short think times, then
this can be configured to 0 zero to
eliminate affinity.
Maintaining this affinity as much as
possible will save the CPU processing
cost of needing to switch an application
module instance from one user session to
another.
320 -Djbo.recyclethreshold=320
Best Practice: When you specify the length of time between application module pool cleanup passes,
set all application modules to use the same Pool Polling Interval value. Since there is only a single
application monitor pool monitor per Java VM, the value that will effectively be used for the
application module pool monitor polling interval will be the value found in the application module
configuration read by the first application module pool that gets created. Setting all application
modules to use the same value ensures that this value is set in a predictable way.
38
2.5 Tune 32bit Java Virtual Machines (JVM)
Newly created application server instances use default memory settings, which are
often too small to accommodate EPM System requirements. For 32bit JVM in order
to allocate a heap of more than 1024 MB, you need to use a 64bit JVM in order to
allocate more than 1024 MB of maximum heap. See section Tune 64bit Java Virtual
Machines (JVM) in this guide.
2.5.1 Tuning 32bit JRockit JVM
This table lists the memory setting, the suggested value, and the Oracle JRockit Java
argument:
Setting Suggested Value Java Argument
Minimum heap 1024 MB -Xms1024m
Maximum heap 1024 MB -Xmx1024m
Important Note: By setting minimum heap size = maximum heap size gives you a
controlled environment where you get a good heap size right from the start and also
to minimize minor/major garbage collections.
Important Note: By default, JRockit JVM uses the dynamic garbage collection
strategy to optimize for throughput. For EPM component, it is not recommended to
change to other available dynamic strategies.
2.5.2 Tuning 32bit Sun JVM
This table lists the memory setting, the suggested value, and the Suns Java
argument:
Setting Suggested Value Java Argument
Minimum heap 1024 MB -Xms1024m
Maximum heap 1024 MB -Xmx1024m
Minimum permanent generation heap 128 MB -XX:PermSize=128m
Maximum permanent generation heap 256 MB -XX:MaxPermSize=256m
Important Note: By setting minimum heap size = maximum heap size gives you a
controlled environment where you get a good heap size right from the start and also
to minimize minor/major garbage collections.
Important Note: Also for application servers that use Sun's Java Virtual Machine
(JVM), make sure that the size of the permanent generation heap must be increased
(as stated in above table). When the permanent area of the heap is too small, the JVM
will do a full garbage collection of the entire heap before resizing the permanent
area.
39
2.6 Tune HTTP Server parameters
This topic describes how to tune the HTTP server to optimize the performance of your Oracle EPM System.
2.6.1 IIS 7.0+ HTTP Server Tuning
Parameters Default Value Suggested Value
Disable Idle Time-out (minutes)
setting for all the EPM Application
Pools.
Tip: In IIS manager, on Application Pool
page, selection an application pool and
then click on Advanced Settings ... to
set this configuration.
20 0
Disable Regular Time Interval
(minutes) setting for all the EPM
Application Pools.
Tip: On the Application Pools page,
select an application pool, and then click
Recycling in the Actions pane and
deselect this option.
1740 0
minBytesPerSecond property
Tip: If Timer_MinBytesPerSecond error
message noted in httperr1.log file on the
IIS server. It is recommended to reduce
or disable the minBytesPerSecond
property in IIS 7.0+ to prevent a slow
client connection from being closed
prematurely.
240 50 How-To: Navigate to Path: C:\Windows\System32\Inetsrv\ , Run the Command: appcmd.exe set config -section:system.applicationHost/webLimits /minBytesPerSecond:"50" /commit:apphost
Connection Timeout
Tip: In order to avoid random
disconnections for remote users, it is
recommended to increase this value.
120 7,200
How-To: Navigate to Path: C:\Windows\System32\Inetsrv\, Run the Command: appcmd.exe set config -section:system.applicationHost/sites "/[name='Default Web Site'].limits.connectionTimeout:02:00:00" /commit:apphost
40
Tune the following memory limits for
all the EPM (64bit) Application Pools:
Private Memory Limit (KB)
Virtual Memory Limit (KB)
Tune the following memory limits for
all the EPM (32bit) Application Pools:
Private Memory Limit (KB)
Virtual Memory Limit (KB)
Important Note: It is recommended to
set no memory limits (0) only for 64bit
IIS. Only force memory limits for 32bit IIS.
Tip: On the Application Pools page,
select an application pool, and then click
Recycling in the Actions pane and
deselect all the options.
= Default
= Default
= Default
= Default
= 0
= 0
= 1048576
= 1572864
ASP Session Timeout (hfm)
Tip: Set to longer than the longest consol.
20 360
ASP Response Buffering Limit (hfm)
To allow Downloads e.g. Data Extracts
and avoid truncated files.
4194304 1073741824
which equates to 1GB
maxAllowedContentLength (hfm)
In HFM, during loading of the files if
you see error HTTP Error 404.13 - Not
Found then increase this value.
30 MB 1073741824
which equates to 1GB How-To: Navigate to Path: C:\Windows\System32\Inetsrv\, Run the Command: appcmd set config "Default Web Site/hfm" /section:system.webServer/security/requestfiltering /requestlimits.maxallowedcontentlength:1073741824
Note: You may need to further increase this value depending on the total size of all the HFM files.
Important Note: Use the above suggested settings as a starting point, and then, after careful testing, adjust as needed. To avoid potential
performance issues, values for IIS 7.0+ parameters should be set only after considering the nature of the workload and the system capacity
(i.e. CPU and memory).
41
2.6.2 Oracle HTTP Server (ohs) Tuning
Oracle HTTP Server uses directives in httpd.conf file. The directives for each Multi-
Processing Module (MPM) type are defined in the
MIDDLEWARE_HOME/user_projects/epmsystem1/httpConfig/ohs/config/OHS/ohs_component/httpd.conf file. The default
MPM type is Worker MPM.
Parameters Default Value Suggested Value
MaxKeepAliveRequests 100 0
KeepAliveTimeout 5 100
KeepAlive On On
Timeout 300 6000
# WinNT MPM (this MPM is for Windows Only
ThreadsPerChild
ListenBackLog
MaxRequestsPerChild
= 512
= 511
= 0
= 1048
= 1000
= 0
#worker MPM (by default OHS use multithreaded mode in
UNIX)
StartServers
MaxClients
ListenBackLog
MinSpareThreads
MaxSpareThreads
ThreadsPerChild
MaxRequestsPerChild
AcceptMutex fcntl
LockFile "${ORACLE_INSTANCE}/diagnostics/logs/${COMPONENT_TYPE}/${COMPONENT_NAME}/http_lock"
= 2
= 512
= 511
= 25
= 75
= 64
= 0
= 2
= 1024
= 1000
= 25
= 75
= 64
= 0
SSLSessionCache
1. Open ssl.conf and find SSLSessionCache. It will be a line like:
SSLSessionCache "shmcb:${ORACLE_INSTANCE}/diagnostics/logs/${COMPONENT_TYPE}/${COMPONENT_NAME}/ssl_scache(512000)
2. Update SSLSessionCache as follows:
SSLSessionCache none
Important Note: Use the above suggested settings as a starting point, and then, after
careful testing, adjust as needed. To avoid potential performance issues, values for
OHS parameters should be set only after considering the nature of the workload and
the system capacity (i.e. CPU and memory).
42
2.6.3 IBM HTTP Server (IHS) Tuning
IBM HTTP Server uses directives in httpd.conf file. The httpd.conf configuration file
is located in the conf directory of your server installation. There is also an
httpd.conf.default file, if you need to use another copy of the original file. The
product provides a sample configuration file called httpd.conf.sample that illustrates
basic IBM module directives and advanced security options.
IBM HTTP Server also provides the admin.conf.default, magic.default, and
mime.types.default configuration files. IBM HTTP Server configuration files are
located in [IBM HTTP Server installation directory]\(/)conf.
Parameters Default Value Suggested Value
MaxKeepAliveRequests 100 0
KeepAliveTimeout 5 10 for high network
bandwidth
100 for low network
bandwidth
KeepAlive On On
Timeout 300 900
# WinNT MPM (this MPM is for Windows Only)
ThreadsPerChild
ThreadLimit
MaxRequestsPerChild
= 64
= 1920
= 0
= 1024
= 4096
= 0
#worker MPM
StartServers
MaxClients
ListenBackLog
MinSpareThreads
MaxSpareThreads
ThreadsPerChild
ThreadLimit
MaxRequestsPerChild
ServerLimit
AcceptMutex fcntl
LockFile [IBM HTTP Server installation directory]/logs/accept.lock
= 3
= 400
= 511
= 75
= 250
= 25
= 64
= 0
= 16
= 10
= 1024
= 2048
= 75
= 250
= 64
= 64
= 0
= 16
Important Note: Use the above suggested settings as a starting point, and then, after
careful testing, adjust as needed. To avoid potential performance issues, values for
IHS parameters should be set only after considering the nature of the workload and
the system capacity (i.e. CPU and memory).
43
2.7 Tune HTTP Server Compression / Caching
Why use Web Server Compression / Caching for Oracle EPM products?
1. Bandwidth Savings: Enabling HTTP compression can have a dramatic
improvement on the latency of responses, while improving the throughput
capacity of the system. By compressing static files and dynamic application
responses, it will significantly reduce the remote (high latency) user response
time.
2. Improves request/response latency: Caching components (i.e. images, js and css)
provide an accurate control of every cache in the proxy chain including the
browser's one. Such an accurate use of the cache makes it possible to suppress
the payload of the HTTP reply using the 304 status code. Minimizing round trips
over the Web to revalidate cached items can make a huge difference in browser
page load times.
Important Note: For only LAN based usage of Oracle EPM applications, it is not
recommended to enable compression / caching for HTTP servers. Therefore it is only
recommended to enable compression / caching when Oracle EPM applications are used
over the WAN and remote users are accessing from high latency remote locations. Once
compression is enabled in http server, it is strongly recommended to conduct series of tests
with multiple concurrent users in test environment and during the test it is important to
evaluate how much of CPU is typically being utilization for http server.
2.7.0.1 Web Server Compression Flow
To better understand compression flow, below are the possible compression enabled
scenarios that shows handling of compressed http requests / responses on Oracle
and IIS web server.
44
Scenario 1 Compression enabled on Oracle HTTP server (ohs) level:
HTTPVPN (WAN)HTTP
:19000
HFM:80
Reporting and Analysis Web:45000
Foundations Services:28080
Planning:8300
Compression Layer
(gzip, deflate)
IE / Firefox
Decompression Layer
(gizp, deflate)
Web Browser Rendering
Scenario 2 Compression enabled on IIS HTTP server level:
45
2.7.1 Oracle HTTP Server (ohs)
This topic describes how to enable compression in Oracle HTTP Server of your
Oracle EPM System.
Tip: The module named "mod_deflate.so" that can compress outgoing data is now added in
current Oracle HTTP Server release included in Oracle Fusion Middleware platform.
a. How to Enable Compression
1. Create file deflate.conf and copy & paste the following lines in this file: SetOutputFilter DEFLATE #Don't compress images SetEnvIfNoCase Request_URI \ \.(?:gif|jpe?g|png)$ no-gzip dont-vary #Don't compress PDFs,doc,ppt,xls
SetEnvIfNoCase Request_URI \.(?:pdf|doc?x|ppt?x|xls?x)$ no-gzip dont-vary #Don't compress compressed file formats
SetEnvIfNoCase Request_URI \.(?:7z|bz|bzip|gz|gzip|ngzip|rar|tgz|zip)$ no-gzip dont-vary
Header append Vary User-Agent #check whether compression ratio is working as expected DeflateFilterNote Input instream DeflateFilterNote Output outstream DeflateFilterNote Ratio ratio LogFormat '"%r" %{outstream}n/%{instream}n (%{ratio}n%%)' deflate CustomLog "|${ORACLE_HOME}/ohs/bin/odl_rotatelogs ${ORACLE_INSTANCE}/diagnostics/logs/OHS/ohs_component/deflate 43200" deflate
#Important Note: Above deflate.log can grow large in size under high users load. Logging
#degrades performance because of the (possibly significant for large file) I/O overhead. Therefore it is
#recommended to disable the generation of deflate log. To disable this log, comment the section
#.
2. Copy deflate.conf file to the ohs_component folder of the OHS folder (path: MIDDLEWARE_HOME/user_projects/epmsystem1/httpConfig/ohs/config/OHS/ohs_component/)
3. Enable mod_deflate module, to do this add the following to httpd.conf
(path:MIDDLEWARE_HOME/user_projects/epmsystem1/httpConfig/ohs/config/OHS/ohs_component/) file: LoadModule deflate_module "${ORACLE_HOME}/ohs/modules/mod_deflate.so"
46
4. Add the following line to httpd.conf file after LoadModule lines: Include /Oracle/Middleware/user_projects/epmsystem1/httpConfig/ohs/config/OHS/ohs_component/deflate.conf
Important Note: In UNIX deployment, ensure the above path is correct and
update accordingly.
5. Save the changes in httpd.conf and Restart the Oracle HTTP Server (ohs).
6. To check that compression is enabled, open deflate file at ${ORACLE_INSTANCE}/httpConfig/ohs/diagnostics/logs/OHS/ohs_component
Tip: In above steps the default EPM Oracle instance location is used i.e.
MIDDLEWARE_HOME/user_projects/epmsystem1. Use your EPM Oracle instance
location in case if it is different.
Tip: After applying all the above steps, if OHS server isnt starting then verify all the
paths (e.g. modules) are valid
For details about