View
228
Download
0
Category
Preview:
Citation preview
8/11/2019 11i Ebs TDB Version 2A
1/20
1Copyright 2009, IBM Corporation Copyright 2009 Oracle Corporation
PSR Test of Pr oj ect Anal yt i cs f orPeopl eSof t Ent er pr i se, OBI EE 10. 1. 3. 4, DAC7. 9. 6, and I nf or mat i ca 8. 6
Harry Doby
Oracle Corporation
February 2009
Abst r act
This white paper documents performance results of the Extract, Transform, and Load
(ETL) component of the Oracle Business Intelligence Applications for PeopleSoft Project
Analytics.
8/11/2019 11i Ebs TDB Version 2A
2/20
2Copyright 2009, IBM Corporation Copyright 2009 Oracle Corporation
Table of contents
Execut i ve Summar y 2
Test Envi r onment 3
Test Obj ect i ves 5
Dat a Expansi on 6
Test Resul t s 7
Concl usi ons 15
Appendi x A - - Dat abase and OS Tuni ng 16
Executive Summary
As a global leader in enterprise applications, Oracles Applications Unlimited Business Intelligence (AUBI) team is
committed to delivering high performance solutions that meet our customers expectations. Business software must
deliver rich functionality with robust performance. This performance must be maintained at volumes that are
representative of customer environments.
Oracles benchmark testing demonstrates our softwares performance characteristics for a range of processing volumes in
a specific configuration. But it is not intended to include all potential processing requirements. Customers should contact
their hardware vendor for help in determining the software, hardware, and network configurations necessary to support
their complete processing volumes.
A Business Intelligence Application such as Project Analytics provides business executives with a complete solution for
Project Lifecycle Reporting, including for each project:
Profitability Analysis
Cost to Budget
Resource Utilization
Bookings and Backlog
This analytical information is drawn from the enterprise application transaction processing system through the ETL
process. In this case, the test utilized data from PeopleSoft Enterprise 8.9, although it could just as easily come from
Oracle EBS or several other enterprise systems.
Figure 1 below gives a high level view of Oracles Business Intelligence Applications
8/11/2019 11i Ebs TDB Version 2A
3/20
3Copyright 2009, IBM Corporation Copyright 2009 Oracle Corporation
Figure 1
Test Environment
The hardware for this project was provided by IBM. Each server identified below is a logical partition (LPAR) within a
single server. A total of 6 LPARs have been configured for two independent Business Intelligence test environments.
PSR will use these environments to test the scaling and performance of various BI development projects. The storage forall LPARs is provided by a single storage array attached to the server. Below is the hardware and software configuration
specific to each of the three servers configured for the Project Analytics PSR Test.
OLTP Server Configuration
Hardware confiiguration:
Machine IBM Power 570 Model 9117-570
Processor POWER5 64-bit
1 processor
Speed 2.2 Ghz
Memory 8192 MB
Storage IBM System Storage DS4700 Fiber array
Fifty-Six 146 GB 10K rpm disks
500 GB allocated
paging: 3072 MB
OS AIX 5.3.7 64-bit
Software configuration:
Oracle DB 10gR2 10.2.0.4
OLTP PeopleSoft Enterprise 8.9
8/11/2019 11i Ebs TDB Version 2A
4/20
4Copyright 2009, IBM Corporation Copyright 2009 Oracle Corporation
ETL Server Configuration
Hardware configuration:
Machine IBM Power 570 Model 9117-570
Processor POWER5 64-bit
1 processor
Speed 2.2 Ghz
Memory 8192 MB
Storage IBM System Storage DS4700 Fiber array
Fifty-Six 146 GB 10K rpm disks
500 GB allocated
paging: 3072 MB
OS AIX 5.3.7 64-bit
Software configuration:
Oracle DB 10gR2 10.2.0.4
Informatica 8.6 HotFix4
DAC Server 7.9.6
OBIEE 10.1.3.4
Java 1.5 JDK
OLAP Server Configuration
Hardware configuration:
Machine IBM Power 570 Model 9117-570
Processor POWER5 64-bit
1 processor
Speed 2.2 Ghz
Memory 8192 MB
Storage IBM System Storage DS4700 Fiber array
Fifty-Six 146 GB 10K rpm disks
500 GB allocated
paging: 3072 MB
OS AIX 5.3.7 64-bit
Software configuration:
Oracle DB 10gR2 10.2.0.4
8/11/2019 11i Ebs TDB Version 2A
5/20
5Copyright 2009, IBM Corporation Copyright 2009 Oracle Corporation
Client Machine Configuration
Hardware configuration:
Machine Dell GX240
Processor 1.5 GHz P4
Memory 1024 MB
Storage C: 38GB
D: 38GB
OS Windows XP SP2
Software configuration:
Informatica 8.6 HotFix4 Client
OBIEE 10.3.4 Client
OBIA 7.9.6
DAC 7.9.6 (necessary for installation of DAC on the ETL server)
Oracle Client 10gR2 10.2.0.4
Java 1.5.0_15
Test Objectives
The goal of this test was to determine the amount of time and resources required to Extract, Transform and Load a fixed
workload using the Project Analytics BI application for PeopleSoft.
ETL Process
Figure 2
8/11/2019 11i Ebs TDB Version 2A
6/20
6Copyright 2009, IBM Corporation Copyright 2009 Oracle Corporation
Test Methodology
There are two modes that ETL typically operates under: Full Load and Incremental Load. A full load happens whenever
a data warehouse is initially created, or when the time comes to completely refresh the dataset under analysis.
Incremental loads occur on a periodic basis to update the existing warehouse with the latest periods data. The
incremental frequency is determined by the business requirements, and may be daily, weekly or some other increment
that meets the organizations needs.
For the purposes of this test, we created 45 million rows of seed data across 10 key OLTP tables in PeopleSoft 8.9. A full
load processed a total of 112 million rows in approximately 6 hours, while the smaller incremental load processed 37
million rows in slightly more than 1 hour.
Scenario 1: Full Load
Purpose: This test will measure the performance of the extract, transform and load steps with a fully populated OLTP
source database with 4 million unique records to key off of the 9 other supporting tables containing the rest of the seed
data.
Description: Two hundred and seventy workflows were executed in Full Load mode into an empty warehouse database.
Approximately 112 million rows in all will be processed.
Scenario 2: Incremental Load
Purpose: This test will measure the performance of the extract, transform and load steps with a fully populated OLTP
source database, and OLAP target database by adding 1 million unique records to key off of the 9 other supporting tables
containing the rest of the seed data.
Description: Two hundred and seventy workflows were executed in Incremental Load mode into an existing warehouse
database. Approximately 37 million rows in all will be processed.
Data Expansion
Ten key tables from the PeopleSoft 8.9 OLTP database were expanded with test data in order to provide a well defined
starting point for this test. The expansion rules are illustrated in the following table.
Source Table # of rows per table
PS_PROJECT 1,000,000
PS_CA_CONTR_HDR 1,000,000
PS_CA_DETAIL 8,000,000
PS_PROJ_ACTIVITY 8,000,000
PS_CA_DETAIL_PROJ 8,000,000
PS_PROJ_RESOURCE 8,000,000
PS_BI_HDR 1,000,000
PS_BI_LINE 4,000,000
PS_PC_BUD_PLAN 1,000,000
PS_PC_BUD_DETAIL 8,000,000
Total 48,000,000
8/11/2019 11i Ebs TDB Version 2A
7/20
7Copyright 2009, IBM Corporation Copyright 2009 Oracle Corporation
Test Results
Full Load Summary Results
Overall Summary APP_ID END_ ETL_R PLAN_RUN_TIME
----------------------------------- --------------- --------- ---------- ------------------------------
Listed in descending time to process PSFT89 22-JA Project 0 days 6 hrs 3 min 33 sec
SESSION_NAME SUCCESS READ WRITE INFA_RUN_TIME
------------------------------------------------------------ ----------- --------- ---------- ------------------------------
SIL_ProjectInvoiceLine_Fact 4000020 327 373 0 days 3 hrs 25 min 7 sec
SIL_ProjectFundingLine_Fact 8000008 701 800 0 days 3 hrs 10 min 51 sec
SIL_ProjectFundingHeader_Fact 8000008 776 878 0 days 2 hrs 52 min 42 sec
SIL_ProjectBudget_Fact 1000000 135 145 0 days 2 hrs 4 min 55 sec
SIL_Project_TaskDimensionHierarchy_Full 8000480 1301 2614 0 days 1 hrs 42 min 50 sec
SIL_ProjectRetention_Fact 1000000 180 208 0 days 1 hrs 33 min 25 sec
SIL_ProjectRevenueLine_Fact 2000000 600 801 0 days 0 hrs 56 min 22 sec
SIL_ProjectCostLine_Fact 1000000 316 364 0 days 0 hrs 53 min 23 sec
SDE_PSFT_ProjectContract_Full 9000024 2965 5242 0 days 0 hrs 51 min 51 sec
SDE_PSFT_ProjectFundingHeaderFact_Full 8000008 2852 4657 0 days 0 hrs 47 min 25 sec
SDE_PSFT_ProjectInvoiceLineFact_Full 4000020 1479 2047 0 days 0 hrs 46 min 57 sec
SDE_PSFT_PersistedStage_ProjectTransactio 2000000 803 2017 0 days 0 hrs 44 min 3 sec
SDE_PSFT_ProjectFundingLineFact_Full 8000008 3669 5010 0 days 0 hrs 37 min 42 sec
SIL_ProjectTaskDimension 8000479 4612 4614 0 days 0 hrs 30 min 21 secSDE_PSFT_PersistedStage_ProjectTransactio 1000000 664 1764 0 days 0 hrs 29 min 40 sec
SDE_PSFT_ProjectRetentionFact_Full 1000000 796 2778 0 days 0 hrs 23 min 9 sec
SDE_PSFT_TaskDimension_Task_Full 8000479 6066 6553 0 days 0 hrs 22 min 51 sec
SIL_ProjectRevenueHdr_Fact_Full 500000 472 6173 0 days 0 hrs 18 min 14 sec
SDE_PSFT_ProjectBudgetFact_Primary 1000000 1187 16394 0 days 0 hrs 14 min 59 sec
SIL_ProjectContractsDimension_Full 9000024 11598 11598 0 days 0 hrs 13 min 31 sec
SDE_PSFT_ProjectFundingHeaderFact_Primar 0 15445 0 0 days 0 hrs 9 min 32 sec
SIL_ProjectRoleDimension 59 59 59 0 days 0 hrs 8 min 38 sec
SDE_PSFT_ProjectRevenueLineFact_PS_Full 2000000 5264 5264 0 days 0 hrs 8 min 33 sec
SDE_PSFT_ProjectFundingLineFact_Primary 0 20888 0 0 days 0 hrs 7 min 15 sec
SIL_DayDimension_GenerateRows5 64 64 64 0 days 0 hrs 6 min 58 sec
SDE_PSFT_ProjectBudgetFact_PS_Full 1000000 4311 4292 0 days 0 hrs 5 min 36 sec
SIL_ProjectCostLineFact_IdentifyDelete 0 0 0 0 days 0 hrs 5 min 8 secSIL_DayDimension_GenerateRows4 32 32 32 0 days 0 hrs 5 min 7 sec
SDE_PSFT_PersistedStage_ProjectTransactio 1000000 3985 4609 0 days 0 hrs 5 min 5 sec
SIL_ProjectContractsDimension_Unspecif ied 1 15 1 0 days 0 hrs 4 min 38 sec
8/11/2019 11i Ebs TDB Version 2A
8/20
8Copyright 2009, IBM Corporation Copyright 2009 Oracle Corporation
Full Load Performance Graphs
OLTP CPU Utilization
0
10
20
30
40
50
60
70
80
90
100
hr
17:41
:42
18:02
:13
18:22
:43
18:43
:13
19:03
:43
19:24
:13
19:44
:43
20:05
:14
20:25
:44
20:46
:14
21:06
:44
21:27
:14
21:47
:44
22:08
:15
22:28
:45
22:49
:15
23:09
:45
Time (30 Second samples)
CPU% CPU %
CPU Wait %
Chart 1
As the chart indicates, the OLTP system is heavily stressed in the first hour of the ETL process as data is extracted from
the database. Once the transformation and load steps that dominate the workload finish, the OLTP system returns to a
normal state.
8/11/2019 11i Ebs TDB Version 2A
9/20
9Copyright 2009, IBM Corporation Copyright 2009 Oracle Corporation
Informatica CPU Utilization
0
10
20
30
40
50
60
70
80
90
100
hr
17
:41:44
18
:02:14
18
:22:44
18
:43:15
19
:03:45
19
:24:15
19
:44:45
20
:05:15
20
:25:46
20
:46:16
21
:06:46
21
:27:16
21
:47:46
22
:08:17
22
:28:47
22
:49:17
23
:09:47
Time (30 second samples)
CPU% CPU %
CPU Wait %
Chart 2
The ETL server running Informatica carries a major portion of the workload, and so the utilization is extremely high
throughout the test. The work being accomplished on this server is the transformation of OLTP-structured data into an
OLAP star schema which will lend itself to answering questions posed by upper management to better run an enterprise.
8/11/2019 11i Ebs TDB Version 2A
10/20
10Copyright 2009, IBM Corporation Copyright 2009 Oracle Corporation
BI CPU Utilization
0
10
20
30
40
50
60
70
80
90
100
hr
17
:41:36
18
:02:06
18
:22:36
18
:43:06
19
:03:36
19
:24:07
19
:44:37
20
:05:07
20
:25:37
20
:46:08
21
:06:39
21
:27:09
21
:47:39
22
:08:09
22
:28:39
22
:49:09
23
:09:39
Time (30 second samples)
CPU% CPU %
CPU Wait %
Chart 3
Since the goal is to get the most data loaded into the data warehouse as quickly as possible, it is normal to have the
processor almost completely consumed.
8/11/2019 11i Ebs TDB Version 2A
11/20
11Copyright 2009, IBM Corporation Copyright 2009 Oracle Corporation
BI Disk Throughput
0
20000
40000
60000
80000
100000
120000
140000
160000
180000
1 50 99 148 197 246 295 344 393 442 491 540 589 638 687
Time (30 second samples)
DiskThroughput(KB/sec)
Disk Throughput
Chart 4
The disk throughput peaked at 170 MB/sec over a span of approximately 20 minutes.
8/11/2019 11i Ebs TDB Version 2A
12/20
12Copyright 2009, IBM Corporation Copyright 2009 Oracle Corporation
Incremental Load Summary Results
Incremental Summary Listing
One Million rows added APP_ID END_TIM ETL_RUN PLAN_RUN_TIME
----------------------------------- --------------- -------------- -------------- ------------------------------
Listed in descending order of processed rows PSFT89 23-JAN-2 Projects - 0 days 1 hrs 12 min 1 sec
SESSION_NAME SUCCESS READ_TH WRITE_T STEP_RUN_TIME
------------------------------------------------------------ ----------- ------------ ------------- ------------------------------------------------------
SDE_PSFT_ProjectFundingHeaderFact_Full 8000008 4148 6612 0 days 0 hrs 32 min 35 sec
SDE_PSFT_ProjectFundingLineFact_Full 8000008 5359 7783 0 days 0 hrs 25 min 26 sec
SDE_PSFT_ProjectInvoiceLineFact_Primary 4000020 36037 36037 0 days 0 hrs 2 min 34 sec
SDE_PSFT_PersistedStage_ProjectTransactio 2500000 1201 4085 0 days 0 hrs 44 min 32 sec
SDE_PSFT_ProjectRevenueLine_Fact_Primar 2500000 5695 6330 0 days 0 hrs 8 min 0 sec
SDE_PSFT_ProjectRevenueLineFact_PS 2000000 10696 10696 0 days 0 hrs 3 min 30 sec
SDE_PSFT_PersistedStage_ProjectTransactio 1250000 716 1774 0 days 0 hrs 34 min 24 sec
SDE_PSFT_ProjectRetentionFact_Full 1250000 924 4033 0 days 0 hrs 23 min 53 sec
SDE_PSFT_ProjectBudgetFact_Primary 1250000 1266 30488 0 days 0 hrs 17 min 14 sec
SDE_PSFT_PersistedStage_ProjectTransactio 1250000 2126 2541 0 days 0 hrs 12 min 44 sec
SDE_PSFT_ProjectCostLineFact_Primary 1250000 2718 3165 0 days 0 hrs 8 min 25 sec
SDE_PSFT_ProjectRetentionFact_Primary 1250000 3592 3592 0 days 0 hrs 6 min 19 sec
SDE_PSFT_BudgetDimension_Full 1000009 5781 6136 0 days 0 hrs 3 min 24 sec
SDE_PSFT_ProjectCostLineFact_PS 1000000 5051 5000 0 days 0 hrs 4 min 45 sec
SDE_PSFT_ProjectBudgetFact_PS 1000000 6061 6025 0 days 0 hrs 3 min 30 sec
Incremental Load Performance Graphs
OLTP CPU Utilization
0
10
20
30
40
50
60
70
80
90
100
hr
1
0:06:19
1
0:10:49
1
0:15:19
1
0:19:49
1
0:24:19
1
0:28:49
1
0:33:19
1
0:37:49
1
0:42:19
1
0:46:49
1
0:51:19
1
0:55:49
1
1:00:19
1
1:04:49
1
1:09:19
1
1:13:49
Time (30 second samples)
CPU% CPU %
CPU Wait %
Chart 5
The incremental load curve for the OLTP source database was similar to the full load. However, given the shorter time
span needed to process the additional 1 million seed records, it is a much faster process.
8/11/2019 11i Ebs TDB Version 2A
13/20
13Copyright 2009, IBM Corporation Copyright 2009 Oracle Corporation
Informatica CPU Utilization
0
10
20
30
40
50
60
70
80
90
100
hr
10
:06:19
10
:10:49
10
:15:20
10
:19:50
10
:24:20
10
:28:50
10
:33:20
10
:37:50
10
:42:20
10
:46:50
10
:51:20
10
:55:50
11
:00:20
11
:04:50
11
:09:20
11
:13:50
Time (30 second samples)
CPU% CPU %
CPU Wait %
Chart 6
The ETL server running Informatica was being heavily utilized, transforming the OLTP data into the OLAP schema
needed for the warehouse.
8/11/2019 11i Ebs TDB Version 2A
14/20
14Copyright 2009, IBM Corporation Copyright 2009 Oracle Corporation
BI CPU Utilization
0
10
20
30
40
50
60
70
80
90
100
hr
10
:06:14
10
:10:44
10
:15:14
10
:19:44
10
:24:15
10
:28:45
10
:33:15
10
:37:45
10
:42:15
10
:46:45
10
:51:15
10
:55:45
11
:00:15
11
:04:45
11
:09:15
11
:13:45
Time (30 second samples)
CPU% CPU %
CPU Wait %
Chart 7
The BI server was almost fully utilized for the ETL processing, and as the next chart will show, the disk throughput was
extremely high.
8/11/2019 11i Ebs TDB Version 2A
15/20
15Copyright 2009, IBM Corporation Copyright 2009 Oracle Corporation
BI Disk Throughput
0
10000
20000
30000
40000
50000
60000
1 11 21 31 41 51 61 71 81 91 101 111 121 131 141
Time (30 second samples)
DiskThroughput(KB/Sec)
Disk Throughput
Chart 8
The disk subsystem peaked at almost 60mb/sec of throughput.
Conclusions
This test represents the first test of Project Analytics BI application, so there is no data to compare from previous tests.
However, we have been able to capture valuable performance and capacity planning information from these tests.
The test environment only consisted of single CPU servers, but with SMT (simultaneous multithreading) execution
enabled. The disk subsystem was a DS4700 Fiber array equipped with 56 10K rpm disk drives of 146gb capacity each.
500GB was allocated to each of the three servers. The disk array has an approximate capacity to support 5,000 transfers
per second. Our tests were able to absorb about half of that capacity, while driving the CPUs to 100% utilization for
extended periods of time.
So the overall results of processing 112 million rows in 6 hours for a full load, and 37 million rows in just over 1 hour for
the incremental load is a reflection of both the speed of the ETL software and the capacity of the test environment.
8/11/2019 11i Ebs TDB Version 2A
16/20
8/11/2019 11i Ebs TDB Version 2A
17/20
17Copyright 2009, IBM Corporation Copyright 2009 Oracle Corporation
OLTP SGA Info
Figure 4
8/11/2019 11i Ebs TDB Version 2A
18/20
18Copyright 2009, IBM Corporation Copyright 2009 Oracle Corporation
Informatica SGA Info
Figure 5
8/11/2019 11i Ebs TDB Version 2A
19/20
19Copyright 2009, IBM Corporation Copyright 2009 Oracle Corporation
AIX Config Changes:
VMO:
-Default settings:
Minperm%=20
Maxperm%=80
Maxclient%=80
Lru_file_repage=1
-Changes:
# vmo -p -o minperm%=5 -o maxperm%=80 -o maxclient%=80 -o lru_file_repage=0
Setting minperm% to 5 in nextboot file
Setting maxperm% to 80 in nextboot file
Setting maxclient% to 80 in nextboot file
Setting lru_file_repage to 0 in nextboot file
Setting minperm% to 5
Setting maxperm% to 80
Setting maxclient% to 80Setting lru_file_repage to 0
SMIT AIO:
-Default settings:
Minservers = 1
Maxservers=10
Maxreqs=4096
-Changes:
Minservers = 10
Maxservers=50
Maxreqs=16384
Network Settings:
-Default settings:
tcp_recvspace= 16384
tcp_sendspace= 16384
-Changes:
no -p -o tcp_recvspace=262144
no -p -o tcp_sendspace=262144
8/11/2019 11i Ebs TDB Version 2A
20/20
20
Copyright IBM Corporation 2009
IBM Corporation
1 New Orchard Road
Armonk, NY 10504
U.S.A.
Produced in the United States of America
June 2009
All Rights Reserved
IBM, the IBM logo, ibm.com, AIX, Power, POWER5, and
System Storage are trademarks or registered trademarks
of International Business Machines Corporation in the
United States, other countries, or both. These and other
IBM trademarked terms are marked on their first
occurrence in this information with the appropriate symbol
( or TM), indicating U.S. registered or common lawtrademarks owned by IBM at the time this information
was published. Such trademarks may also be registered
or common law trademarks in other countries. A current
list of IBM trademarks is available on the Web at
ibm.com/legal/copytrade.shtml.
References in this document to IBM products or services
do not imply that IBM intends to make them available in
every country.
The information provided in this document is distributed
AS IS without any warranty, either express or implied.
The information in this document may include technical
inaccuracies or typographical errors.
Performance is based on measurements and projections
using standard IBM benchmarks in a controlled
environment. The actual throughput or performance that
any user will experience will vary depending upon
considerations such as the amount of multiprogramming
in the user's job stream, the I/O configuration, the storage
configuration, and the workload processed. Therefore, no
assurance can be given that an individual user will
achieve throughput or performance improvements
equivalent to the ratios stated here.
Copyright 2009 Oracle Corporation
Oracle Corporation
500 Oracle Parkway
Redwood Shores, CA 94065
Oracle is a registered trademark of Oracle Corporation
and/or its affiliates.
Recommended