101
Performance Tuning the Peoplesoft Database Tier on Microsoft SQL Server Peoplesoft Deployment May 2022

People Soft Performance Tuning on SQL Server

Embed Size (px)

Citation preview

Page 1: People Soft Performance Tuning on SQL Server

Performance Tuning the Peoplesoft Database Tier on Microsoft SQL Server

Peoplesoft Deployment April 2023

Page 2: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-2

Performance Tuning the Peoplesoft Database Tier on Microsoft SQL Server..................1

Introduction........................................................................................................................5Web Server..........................................................................................................................................5Application Server...............................................................................................................................6Batch Server........................................................................................................................................6

Database Server Configuration........................................................................................6Storage Configuration.........................................................................................................................6

RAID Type Recommendations.................................................................................................6

Files, Filegroups and Object Placement Strategies.................................................................7

TempDB....................................................................................................................................9

SQL Server Configuration...............................................................................................11Installation Considerations................................................................................................................11Memory..............................................................................................................................................11Network.............................................................................................................................................12Configuration Settings.......................................................................................................................12

awe enabled...........................................................................................................................12

cost threshold for parallelism..................................................................................................12

max degree of parallelism......................................................................................................13

max server memory (MB).......................................................................................................14

network packet size (B)..........................................................................................................14

The Peoplesoft Workload................................................................................................15Overview............................................................................................................................................15Prepare / Execute..............................................................................................................................15Cursors..............................................................................................................................................17

Correlating Execution to Prepared Statements and/or Cursors.............................................17

Identifying Cursor Options from SQLTrace RPC....................................................................19

Cursor Type Mask.........................................................................19

Cursor Concurrency Mask............................................................20

sp_cursorprepexec:.......................................................................20

sp_cursorprepare:.........................................................................21

sp_cursorexecute:.........................................................................21Batch Processing...............................................................................................................................22

Process Scheduler.................................................................................................................22

PSPRCSRQST Table.............................................................................................................23

SQRs......................................................................................................................................23

Use of Keyset cursor in SQR applications....................................24

Application Engine..................................................................................................................25

Tracing Application Engine Applications.......................................25

Page 3: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-3

Reducing SQL Compilation with Application Engine Jobs............27

Restrictions on enabling the Reuse Statement Setting.................28

COBOL...................................................................................................................................30

Changing SQL Statements in COBOL Applications.....................32

Database Considerations................................................................................................33Peoplesoft Tables..............................................................................................................................33

“Tools” Tables.........................................................................................................................33

Data Tables............................................................................................................................33

Temporary Tables..................................................................................................................33Peoplesoft Indexes............................................................................................................................36

Introduction.............................................................................................................................36

Naming Conventions..............................................................................................................36

Primary Index................................................................................36

Additional Indexes.........................................................................36

Application Designer...............................................................................................................36

Computed Column Indexes....................................................................................................37Optimizer Statistics............................................................................................................................40

Auto-Stats...............................................................................................................................40

Statistics on Composite Indexes............................................................................................40

Lock Contention..............................................................................................................41Background.......................................................................................................................................41Blocking.............................................................................................................................................41

Indexing Considerations.........................................................................................................41

Volatile Index Columns..................................................................41

Index Column Sequence...............................................................41

Covering Indexes..........................................................................43

Mismatching initial lock sizes..................................................................................................43

Page Lock.....................................................................................43

Table Lock.....................................................................................44

Lock Escalation......................................................................................................................44

Determining if Lock Escalation is Causing Contention.................45

Controlling Lock Escalation...........................................................45

Known Table/Index Contention Issues...................................................................................46

PSVERSION.................................................................................46

PS_BUS_UNIT_TBL_xx................................................................46

Online Users...........................................................................................................................48Locking and Lock Classes.................................................................................................................50

Overview.................................................................................................................................50

QFE 856.................................................................................................................................51

Page 4: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-4

Identifying Lock Class Use through SHOWPLAN..................................................................52

Bookmark Lookup Example..........................................................52

Nested Loop Example...................................................................52

SELECT of Long Column Followed by Sort:.................................53

Query Processor Considerations...................................................................................55Predicate Implication.........................................................................................................................55

Overview.................................................................................................................................55

Implied Predicates Example...................................................................................................56

Manually Adding Redundant Predicates................................................................................56SET FMTONLY meta-data calls........................................................................................................62

Monitoring........................................................................................................................64SQLTrace Events..............................................................................................................................64Correlating Batch Applications to SQL Server Activity......................................................................64Sysprocesses....................................................................................................................................65

Context_info...........................................................................................................................65

Sql_handle..............................................................................................................................66

Appendix..........................................................................................................................67SP_PSWHO......................................................................................................................................67SP_BLOCK........................................................................................................................................71

Page 5: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-5

Introduction

The purpose of this document is to provide guidance and recommendations for tuning the Peoplesoft database tier when deployed on Microsoft SQL Server.

It is important to understand that a successful Peoplesoft deployment will address performance in a wholistic way across each of the tiers that make up the Peoplesoft application architecture.

The diagram below is a high level summary of the basic Peoplesoft architecture:

Web ServerThe Peoplesoft web tier is required to run the PeopleSoft Pure Internet Architecture (PIA), and works with web servers that support Java servlet execution, such as:

The PeopleSoft Internet Architecture works with any Web server that supports

Java servlet execution, such as:

• BEA WebLogic Server

• IBM WebSphere Server

Page 6: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-6

Application ServerThe Tuxedo application server is a central component of the Peoplesoft architecture. Tuxedo provides the client and application software for both from web or Windows based clients. It connects to the PeopleSoft database via ODBC. Tuxedo is packaged with PeopleSoft.

BEA Jolt is a Java-based interface that enables Tuxedo to service requests from Weblogic’s servlets.

Batch ServerThe Batch Server hosts the Peoplesoft batch processing service called Process Scheduler. Process Scheduler provides basic scheduling capability to process jobs either immediately or at a specified date and time.

Process Scheduler passes reports to the Report Repository which authenticates requests for viewing reports and makes report content available to users via the web server.

Database Server Configuration

Storage Configuration The Peoplesoft applications represent a particularly intensive database workload. Therefore, the storage system which will be used for the database server should be configured for optimal performance.

The I/O workload of the Peoplesoft database is similar to that of most OLTP systems, except that there are periods of high-volume write activity during various batch processing operations. So the configuration of the storage system must accommodate write performance as well as read performance.

RAID Type RecommendationsA debate that is common when discussing the topic of RAID options is the relative performance of RAID 5 versus RAID 10.

It is generally true that RAID 10 will outperform a RAID 5 set of the same number of volumes, for the following reasons:

Page 7: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-7

Write performance for RAID 10 is superior. A write operation on RAID 5 requires 4 physical I/O operations, whereas RAID requires two.

Read performance of RAID is enhanced on most implementations by balancing read requests across the two drive participating on the mirror.

Note that configuration settings, available cache, specific vendor capability can introduce many variables into this discussion.

RAID 0 is unsuitable for use in a SQL Server environment. A loss of a single drive will result in the loss of data. Even tempdb should not be placed on RAID 0 – the loss of one drive on RAID 0 would result in an outage to the SQL Server instance.

A possible use of RAID 0 could be the temporary location of disk backups, prior to writing disk backups to tape or another location.

RAID 1 is appropriate for objects such as the SQL Server binaries, the master database, and the MSDB database. I/O requirements for these objects are minimal (thus they do not generally benefit from striping), but they require fault tolerance for availability reasons.

The transaction log also benefits from RAID 1. Fault tolerance is a must for continuous operation. Also, the nature of transaction log I/O is continuous sequential write. RAID 1 is ideal for this type of I/O pattern.

Note: the transaction log should be entirely isolated from all other I/O activity – no other files should exist on the drives that contain the log file. This ensures that, with the exception of transaction log backup and the occasional rollback – nothing disturbs the sequential nature of transaction log activity.

RAID 10 affords the best overall performance, making it the preferred choice for all database files.

Files, Filegroups and Object Placement StrategiesThe complex pattern of I/O activity and large number of tables and indexes in the Peoplesoft database makes attempting strategic placement of objects and object types (tables vs. indexes) a difficult task. A better strategy will be to spread a single user-defined filegroup across as many physical drives as possible in order to put the entire storage system to work completing as many I/O requests as possible in parallel.

Page 8: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-8

A possible exception to this strategy is the isolation of what are called “temporary” tables (described later in this document). Temporary tables are used to store intermediate results during batch processing; therefore isolating temporary tables from the rest of the Peoplesoft database objects has the potential to reduce the interference that batch can have with OLTP processing.

Note: Peoplesoft does provide the means assign tables and indexes to specific filegroups. This is accomplished through updating Peopletools tables PSDDLMODEL and PSDDLDEFPARMS. When used, table and index scripts are generated with their specific filegroup included.

A script to perform this update is supplied by Peoplesoft; the script is called DDLMSS.DMS. The script is executed by Peoplesoft Data Mover, but upon viewing the script it should be self evident what values can be modified via standard SQL executed via Query Analyzer or other means. Below is a recent copy of DDLMSS.DMS:

SET LOG DDLMSS.LOG;

REMARK\ This script loads the PeopleTools DDL tables for MicroSoft.;

DELETE FROM PSDDLMODELWHERE PLATFORMID=7;

INSERT INTO PSDDLMODEL (STATEMENT_TYPE,PLATFORMID,SIZING_SET,PARMCOUNT,MODEL_STATEMENT)VALUES(:1,:2,:3,:4,:5)\$DATATYPES NUMERIC,NUMERIC,NUMERIC,NUMERIC,CHARACTER1,7,0,1,$longCREATE TABLE [TBNAME] ([TBCOLLIST]) ON **FILEGRP**;//2,7,0,1,$longCREATE [UNIQUE] [CLUSTER] INDEX [IDXNAME] ON [TBNAME] ([IDXCOLLIST]) WITH SORT_IN_TEMPDB ON **INDXGRP** ;//3,7,0,0,$long

Placeholder for Table Filegroup name

Placeholder for Index Filegroup name

Page 9: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-9

;///

DELETE FROM PSDDLDEFPARMSWHERE PLATFORMID=7;

INSERT INTO PSDDLDEFPARMS (STATEMENT_TYPE,PLATFORMID,SIZING_SET,PARMNAME,PARMVALUE)VALUES(:1,:2,:3,:4,:5)\$DATATYPES NUMERIC,NUMERIC,NUMERIC,CHARACTER,CHARACTER1,7,0,"FILEGRP","TABLEGROUP"2,7,0,"INDXGRP","INDEXGROUP";

TempDBThe Peoplesoft database workload makes heavy use of tempdb. It is recommended that tempdb be isolated from other database activity onto its own RAID set of physical disks.

Use of RAID 10 especially important for tempdb – this system database is used heavily in most environments and has approximately 50% read to 50% write ratio.

Note: during periods of heavy activity in the Peoplesoft database, contention has been detected on the allocation structures in the tempdb database files. The contention can be detected as pagelatch wait time on GAM, SGAM, PFS, and IAM pages.

Refer to KB article 328551 for details on tempdb contention and corrective actions that are available.

One way to preempt this form of tempdb contention is to allocate multiple files to the tempdb database, even if those files exist on the same set of physical disks. Consider the recommendation in the KB article to allocate a number of tempdb files equal to the number of processors available to SQL Server.

Optional Index create clause can be added here, such as fillfactor or sort_in_tempdb

Actual filegroup names are specified here

Page 10: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-10

The SP_PSWHO script included with this document can assist in detecting tempdb contention; this procedure translates the waittype column in sysprocesses to a descriptive text value. Look for “Pagelatch” in the Waittype column produced by this procedure. If further examination of the waitresource from sysprocesses shows allocation page from tempdb then the corrective action documented in KB article 328551should be taken.

Page 11: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-11

SQL Server Configuration

Installation ConsiderationsPeoplesoft requires the use of binary collation on SQL Server. When installing SQL Server, choose the custom setup type. Under Collation Settings, select the Collation designator radio button and then check the Binary sort order check box.

You should refer to the Peopletools Installation Guide for SQL Server for installation background and requirements. The installation guide for Peoplesoft 8.4 is file name is PT84Install_MSS.PDF, and is available on Peoplesoft’s support website.

MemoryLike all database-intensive applications, Peoplesoft can benefit from the extended memory capability offered by SQL Server 2000 Enterprise Edition. The options for using additional memory are:

/3GB flag set in the server’s boot.ini file. SQL Server will have 3 GB allocated to its base address space.

/PAE flag set in the server’s boot.ini file. This flag, along with setting awe enabled through sp_configure, will statically allocate memory above 4GB upp to the amount specified in the max server memory (MB).

Not all consumers of memory can use memory allocated through AWE; most notably, lock manager optimizer and procedure cache are constrained to SQL Server’s process address space. Peoplesoft consumes relatively large amounts of memory for these purposes, so many deployments will benefit from using a combination of /3GB and AWE memory.

Note: there are conditions where the combination of /3GB and the use of PAE/AWE memory can result in a shortage of system memory for page table entries (PTEs). Some services or drivers may consume space within the system address space which could lead to memory related errors when /3GB is enabled.

It is crucial that testing take place in an environment that resembles the production environment as possible to ensure that any memory-related conditions associated with memory settings are detected early. It is also good practice to have all memory related Performance monitor counters collected on a continuous basis to provide information to assist with memory-related configuration questions should they arise.

Page 12: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-12

Data Center deployments are constrained to no more than 16GB if both /3GB and /PAE are enabled.

The 64 bit editions of Windows Server and SQL Server provide a large process address space for SQL Server to use; the options for /3GB and PAE/AWE are not applicable in 64 bit configurations.

NetworkThe Peoplesoft application is relatively “chatty” in its communication with the database tier. High number of Batch Requests/sec are normal due to the extensive use of cursors in the Peoplesoft workload.

As a rule of thumb, you should consider 1Gb Ethernet if the anticipated batch requests/sec approaches 3,000; this level of traffic can saturate 100Mb connections.

Sufficient network bandwidth is especially important if there is a large volume of time-critical batch activity and the Process Scheduler (Peoplesoft’s batch scheduling service, described later) is located on a server other than the SQL Server instance.

A configuration option that should be considered for Peoplesoft customers that have high volumes of batch activity concurrent with OLTP activity is the use of the virtual interface (VI) architecture. VI provides a connection-oriented remote memory “pipe” between the application server (in this context, the batch application server) and SQL Server. It is ideal for the type of traffic between Peoplesoft’s batch applications and the database tier.

SQL Server 2000 sp3 provides renewed support for the Virtual Interface (VI) architecture using host adapters from QLogic.

Configuration SettingsThe following are the recommended sp_configure settings for a Peoplesoft SQL Server instance:

awe enabledRecommended in order to take advantage of memory above 4GB for 32 bit configurations.

cost threshold for parallelismThe default value for this setting is 5; this means that if the optimizer’s estimated elapsed time for a query is 5 seconds or greater and the execution plan is being prepared on an SMP server, then a parallel plan will be created in addition to a serial plan.

Page 13: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-13

Experience has shown that this setting may be too low and makes SQL Server too “eager” to create a parallel plan. For estimated elapsed times in and around 5 seconds, it is quite possible that parallel execution may exceed that of the serial plan.

It is recommended to set cost threshold for parallelism to a minimum of 60; thus, parallelism will only be considered for queries that have a high likelihood of benefiting from parallel operations.

max degree of parallelismThe default value for this setting is 0, which means that a single parallel operation can have threads up to the number of processors available to SQL Server.

Peoplesoft has a heavily mixed workload of batch, OLTP, reporting, and query activity. It is advisable that parallelism be constrained during periods of peak OLTP activity to reduce the impact of batch, reporting, and query work.

Parallel execution has a locking implication that is not well understood: row/key locks are never the initial lock size acquired by parallel operators. The minimum lock size is page. If page locks have been disallowed, then the lock size chosen is table.

As of this writing (SP3), an issue exists where a parallel plan that also includes prefetch operations does not decrement page lock reference counts. Under this condition, the page locks acquired by parallel operators prior to prefetch are not released until the end of transaction, further exacerbating the locking issues surrounding parallelism.

This issue is documented in KB article 843266 and is resolved by QFE 949

Use the following guidelines for max degree of parallelism in a Peoplesoft environment:

Set max degree of parallelism to 1 during peak OLTP periods.

Increase max degree of parallelism during periods of low OLTP and high batch, reporting, and query activity. Also note that index creation and recreation can take advantage of parallelism, so it is advisable to enable parallelism through this setting when planning to build or rebuild indexes.

Do not set max degree of parallelism greater than 8.

Do not set max degree of parallelism to exceed the number of physical processors on systems with hyperthreading enabled.

Page 14: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-14

max server memory (MB)Observe standard best practices for this setting. If enabling AWE, remember that AWE memory is statically allocated and non-pageable; plan to leave at least 1 GB for OS and other ancillary services on the database server. For example, if the database server has 16 GB, set max server memory to 15 GB. Monitor Memory: Available Bytes to determine if max server memory should be reduced or increased.

network packet size (B)Peoplesoft currently recommends 8192 for this setting, but will be amending its documentation in the future to suggest leaving this setting at its default value of 4096.

Page 15: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-15

The Peoplesoft Workload

Overview

Prepare / ExecutePeoplesoft makes extensive use of the prepare/execute model in both their on-line and batch applications. When tracing Peoplesoft activity using SQLTrace, trace RPC events rather than SQL Statement or SQL Batch events.

Below is a typical RPC event text for a statement that is issued under the prepare/execute model:

declare @P1 intset @P1=550exec sp_prepexec @P1 output, N'@P1 varchar(508),@P2 varchar(508),@P3 decimal(38,0)', N'DELETE FROM PS_PAYMENT_ITEM WHERE DEPOSIT_BU = @P1 AND DEPOSIT_ID = @P2 AND PAYMENT_SEQ_NUM = @P3 AND ITEM_SELECTED = ''N'' AND EXISTS ( SELECT ''X'' FROM PS_PAYMENT_ITEMTMP WHERE DEPOSIT_BU = PS_PAYMENT_ITEM.DEPOSIT_BU AND DEPOSIT_ID = PS_PAYMENT_ITEM.DEPOSIT_ID AND PAYMENT_SEQ_NUM = PS_PAYMENT_ITEM.PAYMENT_SEQ_NUM AND BUSINESS_UNIT = PS_PAYMENT_ITEM.BUSINESS_UNIT AND CUST_ID = PS_PAYMENT_ITEM.CUST_ID AND ITEM = PS_PAYMENT_ITEM.ITEM AND ITEM_LINE = PS_PAYMENT_ITEM.ITEM_LINE ) ', '003', '16724', 1select @P1The above statement issued with the deferred-prepare option available in SQL2K, which is the default setting. Deferred prepare reduces round trips to SQL Server by sending a combined prepare and execute statement. An additional advantage of deferred prepare is that the parameter values are available at compile (preparation) time. Therefore, the optimizer has greater visibility to optimizer statistics than was the case with the older, non-deferred prepare.Some parts of Peoplesoft still issue requests with the deferred prepare disabled. These statements will be seen as RPC events executing the internal procedures sp_prepare and sp_cursorprepare.

Page 16: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-16

Note: It may be useful to include SQL Batch events also. Portions of the Peoplesoft application use the SET FMTONLY ON option to initially retrieve metadata. The structure of the batches using this setting (from Peoplesoft) is:

SET FMTONLY ON

SELECT B.EMPLID FROM PSOPRDEFN A, PS_EX_EE_AUTH_VW B

SET FMTONLY OFF

The WHERE clause is typically removed, since the statement’s purpose is to return metadata, not to execute.

The SET FMTONLY ON setting causes the statement following to not execute; it does, however, still compile. And in some cases the absence of the WHERE clause results in an extremely complex optimization challenge for the optimizer, which takes a great deal of time and optimizer memory to resolve.

Page 17: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-17

CursorsA significant portion of the database processing issued by Peoplesoft is performed by cursors. Peoplesoft uses the Fast Forward Only / Autofetch cursor when possible, and achieves relatively good performance through this cursor model.

An example of a cursor issued from Peoplesoft (also using the prepare execute model) follows:

declare @P1 intset @P1=24declare @P2 intset @P2=180150012declare @P3 intset @P3=8declare @P4 intset @P4=1declare @P5 intset @P5=3201exec sp_cursorprepexec @P1 output, @P2 output, N'@P1 varchar(5)', N'SELECT A.BUSINESS_UNIT, A.CUST_ID, SUM(A.BAL_AMT) FROM PS_ITEM A, PS_CUSTOMER B WHERE A.CUST_ID = B.CUST_ID AND A.ITEM_STATUS = ''O'' AND A.DISPUTE_STATUS = '' '' AND B.SETID = @P1 AND B.SETID=(SELECT X.SETID FROM PS_SET_CNTRL_RECWHERE X.SETCNTRLVALUE = A.BUSINESS_UNIT AND X.RECNAME = ''CUSTOMER'') GROUP BY A.BUSINESS_UNIT, A.CUST_ID ORDER BY A.BUSINESS_UNIT, A.CUST_ID ', @P3 output, @P4 output, @P5 output, 'XXXX'select @P1, @P2, @P3, @P4, @P5

The sp_cursorprepexec is followed by a series of sp_cursorfetch RPC calls to consume the rows of the cursor.

sp_cursorexecute may also be called to reuse the existing prepared statement with a new set of parameter values.

Correlating Execution to Prepared Statements and/or CursorsIn many cases it will be necessary to relate and RPC call for sp_execute or sp_cursorexecute back to the RPC call issuing the prepare. Assume, for example, the following statement has an unacceptable duration:

declare @P1 intset @P1=180150576declare @P2 intset @P2=1declare @P3 intset @P3=1declare @P4 int

Page 18: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-18

set @P4=0exec sp_cursorexecute 90, @P1 output, @P2 output, @P3 output, @P4 output, '03100', 'ACTUALS', 2004.0, 1.0, 1.0select @P1, @P2, @P3, @P4

In order to address the performance issue, we must locate the cursor or SQL statement that was prepared in advance of the current statement’s execution.

1. To do this, note the 1st argument of the sp_cursorexecute. In the example above, the value is 90.

2. Next, locate the RPC call issuing a sp_cursorprepare or sp_cursorprepexec with this same value supplied as @P1:

declare @P1 intset @P1=90declare @P2 intset @P2=180150078declare @P3 intset @P3=1declare @P4 intset @P4=1declare @P5 intset @P5=0exec sp_cursorprepexec @P1 output, @P2 output, N'@P1 varchar(5),@P2 varchar(7),@P3 decimal(5,1),@P4 decimal(2,1),@P5 decimal(2,1)', N'SELECT LDGR.ACCOUNTING_PERIOD, LDGR.ACCOUNT,LDGR.POSTED_TOTAL_AMT, LDGR.STATISTICS_CODE FROM PS_LEDGER LDGR WHERE NOT EXISTS (SELECT ''X'' FROM PS_JRNL_LN JRLN, PS_JRNL_HEADER HDER WHERE LDGR.BUSINESS_UNIT = HDER.BUSINESS_UNIT AND HDER.BUSINESS_UNIT = JRLN.BUSINESS_UNIT AND HDER.FISCAL_YEAR = LDGR.FISCAL_YEAR AND HDER.ACCOUNTING_PERIOD = LDGR.ACCOUNTING_PERIOD AND HDER.JOURNAL_ID = JRLN.JOURNAL_ID AND HDER.JOURNAL_DATE = JRLN.JOURNAL_DATE AND HDER.UNPOST_SEQ = JRLN.UNPOST_SEQ AND LDGR.CURRENCY_CD = JRLN.FOREIGN_CURRENCY AND JRLN.LEDGER = LDGR.LEDGER AND JRLN.ACCOUNT = LDGR.ACCOUNT AND ((LDGR.STATISTICS_CODE = JRLN.STATISTICS_CODE AND LDGR.CURRENCY_CD = '' '') OR (JRLN.FOREIGN_CURRENCY = LDGR.CURRENCY_CD AND LDGR.STATISTICS_CODE = '' ''))) AND LDGR.BUSINESS_UNIT = @P1 AND LDGR.LEDGER = @P2 AND LDGR.FISCAL_YEAR = @P3 AND LDGR.ACCOUNTING_PERIOD >= @P4 AND LDGR.ACCOUNTING_PERIOD <= @P5 AND LDGR.POSTED_BASE_AMT <> 0 ', @P3 output, @P4 output, @P5 output, '02000', 'ACTUALS', 2004.0, 1.0, 1.0select @P1, @P2, @P3, @P4, @P5

3. This event will contain the SQL statement for the prepared statement or cursor.

The fetch operations that follow a cursor execute can be correlated to their respective cursor in a somewhat similar exercise. The value set in @P2 of sp_cursorprepexec or sp_cursorexecute will appear as the 1st argument in subsequent fetch operations for that cursor:

Page 19: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-19

declare @P1 intset @P1=90declare @P2 intset @P2=180150078declare @P3 intset @P3=1declare @P4 intset @P4=1declare @P5 intset @P5=0exec sp_cursorprepexec @P1 output, @P2 output…………………exec sp_cursorfetch 180150078, 2, 1, 10

Identifying Cursor Options from SQLTrace RPCServer-side cursor operations are implemented in SQL Server through a series of internal stored procedures:Procedure Functionsp_cursorprepare Cursor prepare – deferred prepare disabledsp_cursorprepexec Cursor prepare and 1st execute – deferred prepare

enabledsp_cursorfetch Cursor fetchsp_cursorexecute Subsequent execution of a prepared cursor

(through sp_cursorprepare or sp_cursorprepexec)

Cursor Type Mask

Cursor type is actually a bit map.  You can interpret the values from the following table:

 

1: KEYSET 2: DYNAMIC 4: FORWARD_ONLY 8: STATIC 16: FAST_FORWARD 4096: PARAMETERIZED_STMT *8192: AUTO_FETCH *16384: AUTO_CLOSE *32768: CHECK_ACCEPTED_TYPES *65536: KEYSET_ACCEPTABLE *131072: DYNAMIC_ACCEPTABLE *262144: FORWARD_ONLY_ACCEPTABLE *524288: STATIC_ACCEPTABLE *1048576: FAST_FORWARD_ACCEPTABLE *

Page 20: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-20

Cursor type values marked above with ‘*’ are only visible at RPC Starting events for sp_cursorprepexec. The basic cursor types (1, 2, 4, 8, and 16) are supplied at RPC Completed events.

Cursor Concurrency Mask

Cursor concurrency options are also represented by a bit map.  You can interpret the values from the following table:

1 READ_ONLY 2 SCROLL_LOCKS 4 OPTIMISTIC 8 OPTIMISTIC 8192 ALLOW_DIRECT *16384 UPDT_IN_PLACE *32768 CHECK_ACCEPTED_OPTS *65536 READ_ONLY_ACCEPTABLE *131072 SCROLL_LOCKS_ACCEPTABLE* 262144 OPTIMISTIC_ACCEPTABLE *524288 OPTIMISTIC_ACCEPTABLE *

Cursor concurrency values marked above with ‘*’ are only visible at RPC Starting events for sp_cursorprepexec. The basic cursor concurrency (1, 2, 4, and 8) are supplied at RPC Completed events.

sp_cursorprepexec:declare @P1 intset @P1=24declare @P2 intset @P2=180150012declare @P3 intset @P3=8declare @P4 intset @P4=1declare @P5 intset @P5=3201exec sp_cursorprepexec @P1 output, @P2 output, N’@P1 varchar(5)’, N’SELECT A.BUSINESS_UNIT, A.CUST_ID, SUM(A.BAL_AMT) FROM PS_ITEM A, PS_CUSTOMER B WHERE A.CUST_ID = B.CUST_ID AND A.ITEM_STATUS = ‘’O’’ AND A.DISPUTE_STATUS = ‘’ ‘’ AND B.SETID = @P1 AND B.SETID=(SELECT X.SETID FROM PS_SET_CNTRL_RECWHERE X.SETCNTRLVALUE = A.BUSINESS_UNIT AND X.RECNAME = ‘’CUSTOMER’’) GROUP BY A.BUSINESS_UNIT, A.CUST_ID ORDER BY A.BUSINESS_UNIT, A.CUST_ID ‘, @P3 output, @P4 output, @P5 output, ‘SETID1’

If the statement is parameterized, the parameters used are declared here;

The parameters’ used are located here

Cursor Concurrency Option

Cursor Type

Correlates to subsequent sp_cursorfetch

Correlates to subsequent sp_cursorexecute

Page 21: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-21

select @P1, @P2, @P3, @P4, @P5

sp_cursorprepare:

Note: sp_cursorprepare is only seen when deferred prepare is not enabled. Still observable in SQR applications.

set @P1=4007

declare @P2 int

set @P2=4097

declare @P3 int

set @P3=8193

exec sp_cursorprepare @P1 output, N'@P1 varchar(5),@P2 varchar(7),@P3

decimal(5,1),@P4 decimal(2,1)', N'SELECT SUM(LEDGER.POSTED_TOTAL_AMT) FROM

PS_LEDGER LEDGER WHERE LEDGER.BUSINESS_UNIT = @P1 AND LEDGER.LEDGER = @P2 AND

LEDGER.FISCAL_YEAR = @P3 AND LEDGER.ACCOUNTING_PERIOD = @P4 AND LEDGER.ACCOUNT

= ''20310004'' AND LEDGER.CURRENCY_CD = ''JPY'' AND

LEDGER.STATISTICS_CODE = '' '' ', 1, @P2 output, @P3 output

select @P1, @P2, @P3

sp_cursorexecute:declare @P1 int

set @P1=180154744

declare @P2 int

set @P2=8

declare @P3 int

set @P3=1

declare @P4 int

set @P4=1

exec sp_cursorexecute 4007, @P1 output, @P2 output, @P3 output, @P4 output,

'05034', 'ACTUALS', 2004.0, 1.0

select @P1, @P2, @P3, @P4

Correlates to subsequent sp_cursorexecute

Cursor Type

Cursor Concurrency Option

Correlates to subsequent sp_cursorfetch

Cursor Type

Cursor Concurrency Option

Correlates to subsequent sp_cursorexecute

From @P1 of sp_cursorprepare or sp_cursorprepexec

Page 22: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-22

Batch ProcessingOne of the complexities of the Peoplesoft workload on the database tier is that it is a combination of OLTP, batch update, and batch query. The most significant issues associated with this mixed workload are concurrency issues; blocking and deadlocking between OLTP and batch, or between multiple batch applications.

There are several types of batch applications in Peoplesoft:

SQR

COBOL Applications

Application Engine (AppEngine) Applications

There an effort underway by Peoplesoft to reduce or eliminate some types. The direction from Peoplesoft has been to replace older COBOL and SQR applications with equivalent application engine versions.

Process SchedulerThe process scheduler is the scheduling process for the Peoplesoft applications. Process scheduler runs as a service on the designated batch server. There are three potential locations for the batch server:

1. Database server. In this scenario, process scheduler runs directly on the database server. This is not the preferred configuration because it consumes resources that could otherwise be dedicated to SQL Server.

2. Application servers. The process scheduler can be collocated with the Tuxedo instances on one or more of the application servers. This is also not an ideal configuration; the application servers are memory intensive processes and it is best to dedicate the application servers to that singular purpose.

3. Dedicated batch scheduler. This is the preferred configuration. One or more dedicated servers assigned to run the process scheduler service provides the best overall scalability for batch processing and the isolation needed to effectively tuning the various tiers of the Peoplesoft infrastructure.

Note: You may need to work closely with the hardware vendor providing the hardware sizing document for the customer. In an effort to provide the most cost effective hardware solution, the vendor may opt to locate the process scheduler on the database server without understanding the tradeoffs associated with this configuration decision.

Page 23: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-23

PSPRCSRQST TableThe PRCSRQST table is a very important table in the Peoplesoft schema (it is also a “tools” table, which defined later in the Database Considerations section of this document). PRCSRQST is the location that process scheduler(s) record information about the batch jobs and their status. Some of the key columns in this table are:

PRCSINSTANCE Process Instance – a unique identifier for a batch process in the Peoplesoft system

PRCSNAME Process name. A name assigned by Peoplesoft (or the customer in the event of a custom application) for the batch application

PRCSTYPE Process Type. Identifies the process as SQR, Cobol, etc.

RUNSTATUS Current status of the process. RUNSTATUS of 7 means the application is currently processing.

BEGINDTTM Process starting time

ENDDTTM Process completion time

SQRsSQR is a 4GL language originally marketed by SQRIBE, which was then acquired by BRIO. SQR is bundled into the Peoplesoft application suite and is one of the standard types of Peoplesoft batch.

SQR programs are stored in text files, making them easy (perhaps too much so) to modify. SQL statements, although wrapped in procedural extensions implemented by SQR, are generally quite recognizable. Relative to the other types of batch applications from Peoplesoft, modification to SQL for performance tuning purposes is easier to accomplish in SQR programs.

Following is an example of a SQL statement from a delivered SQR from Peoplesoft:BEGIN-SELECTIA.BOOK{IA-SELECT-SEL}IA.IU_MAP_KEYIA.FROM_CURIA.COSTIA.SALVAGE_VALUE

let $IUT_Add_Book = &IA.BOOK let #IUT_Add_Map_Key = &IA.IU_MAP_KEY let #IUT_Add_Cost = &IA.COST

Page 24: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-24

let #IUT_Add_Salvage_Value = &IA.SALVAGE_VALUE let $IUT_Add_Currency_Cd = &IA.FROM_CUR

let $Add_Entry_FND_SW = 'Y'

do Update-IUT-COST

do Get-To-Led-Group

if ($TO_Ledger_Group <> $TO_Ledger_Group_Prev or $Second_CF_SW = 'Y') and $TO_Ledger_Group <> ' '

do IUT-ADD-Other-Bks move $TO_Ledger_Group to $TO_Ledger_Group_Prev end-if

FROM PS_COST IA WHERE IA.BUSINESS_UNIT = $line_Transfer_BU AND IA.ASSET_ID = $Line_Transfer_Asset_Id AND IA.TRANSFER_BOOK = $OT_BOOK AND IA.DTTM_STAMP = {DATETIMEIN-PREFIX}$OT_Dttm_Stamp{DATETIMEIN-SUFFIX} AND IA.TRANS_TYPE = $TYPE_ADD AND IA.IU_MAP_KEY = #Line_IU_Map_Key END-SELECT

Use of Keyset cursor in SQR applications

The PeopleSoft SQR engine defaults to using Keyset cursors for all its queries. An example T-SQL query generated by the Payslip SQR job is shown below.

declare @P1 intset @P1=183908640declare @P2 intset @P2=1declare @P3 intset @P3=1declare @P4 intset @P4=53exec sp_cursorexecute 41, @P1 output, @P2 output, @P3 output, @P4 output, 'KA AU HRLY ', '2000-01-01 00:00:00.000', 1

While functionally a valid cursor type, a Keyset cursor is known to be less efficient than Forward-Only or Fast Forward Auto Fetch (FFAF) cursors (used by OLTP and AppEngine components of Peoplesoft).

While the SQR engine at present does not support FFAF cursors, the SQR configuration file PSSQR.INI file can be modified to change the cursor type to Forward Only by adding the ‘ODBCCursorstyle=forward’ option under the [Default-Settings] section. Example:

….[Default-Settings]

Page 25: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-25

AllowDateAsChar=TRUEoutput-file-mode=longOutputTwoDigitYearWarningMsg=FALSEUseY2KCenturyAlgorithm=TRUEDefault-Numeric=Decimal(26)ODBCCursorstyle=forward….

The above configuration change should yield some performance gains. Meanwhile the SQR team is working on a solution to enable FFAF cursors for SQR.

Application EngineApplication Engine applications afford somewhat greater control over some performance-related elements of the application, such as:

The ability to programmatically determine whether SQL is to be issued as parameterized or not. Parameterization increases the probability of execution plan reuse, but also increases some risk that execution plans may get reused under circumstances when such reuse does not yield good performance. The ability to specify parameterization is an important option for Application Engine. More follows on this topic under Application Engine below.

It is easier to introduce commit logic into Application Engine applications. Doing so requires careful consideration of the transaction consistency issues that may arise should the application terminate and need to be restarted. Application Engine provides a model for establishing restartability in the event that commits must be issued at points other than well-defined transaction boundaries. (consult Peoplesoft training and documentation, along appropriately trained Peoplesoft developers before recommending this course of action)

Development and modification to Application Engine programs is performed via the Peoplesoft App Designer tool.

Tracing Application Engine Applications.

Peoplesoft provides extensive tracing capability for Application Engine processes. Tracing can be enabled through the App Designer. Work with the customer or partner Peoplesoft Administrator to enable tracing for the specific Application Engine Process that you are interested in tracing.

Page 26: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-26

Note that tracing in Application Engine does introduce a moderate amount of overhead, and so should be used as a troubleshooting measure rather than a continuous practice.

Note: the description of Peoplesoft tracing that is contained in this document is not comprehensive. It is intended to give the reader a general idea of the fact that tracing capabilities exist, not to provide a detailed how-to guide for AppEngine tracing.

Location of trace files produced by AppEngine tracing is site-dependent, but they will always be located in a subfolder with the following naming convention:

AE_program-name_process-instance

Tracing options available for AppEngine programs are:

Timings Trace (example below). Trace data

Page 27: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-27

1. Below is an example of an application engine trace:

PeopleSoft Application Engine Timings

(All timings in seconds)

2003-07-07 09.20.11

C o m p i l e E x e c u t e F e t c h Total SQL Statement Count Time Count Time Count Time Time ------------------------------ ------- -------- ------- -------- ------- -------- --------

Application Engine

COMMIT 0 0.0 2170 17.3 0 0.0 17.3 -------- 17.3

AE Program: AP_APY2015

B0000001.Step01.S 1 0.0 1 0.0 0 0.0 0.0B0000001.Step02.S 1 0.0 1 0.0 0 0.0 0.0B0000001.Step03.S 1 0.0 1 0.0 0 0.0 0.0………………………………………………B0170000.Step01.H 1 0.0 1 0.0 1 0.0 0.0B0170000.Step01.S 1 0.0 1 0.4 0 0.0 0.4B0197000.Step02.H 575 0.3 575 0.7 575 0.0 1.0B0197000.Step04.H 575 0.2 575 1.1 575 0.0 1.3B0197000.Step05.S 575 0.3 575 3.6 575 0.0 3.9………………………………………………

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Total run time : 545.8

Total time in application SQL : 433.3 Percent time in application SQL : 79.4%

Total time in PeopleCode : 8.1 Percent time in PeopleCode : 1.5%

Total time in cache : 15.6 Number of calls to cache : 8900

------------------------------------------------------------------------------------------

PeopleTools SQL Trace value: 7 (0x7)PeopleTools PeopleCode Trace value: 0 (0)Application Engine Trace value: 131 (0x83)Application Engine DbFlags value: 0 (0)

Reducing SQL Compilation with Application Engine Jobs

By default, the majority of Application Engine SQL statements are not parameterized. This means that statements executed repeatedly but with different values will be recompiled. For complex statements that are executed many times, the recompilation can consume significant amounts of CPU.

Application Engine SQL statements can be parameterized by setting the “ReUse Statement” value to “Yes”.

Compile, Execute, and Fetch times by SQL Statement.

Note: Compile times are not necessarily accurate. The trace is unaware of the deferred prepare used by SQL Server 2000

Breakdown of run time can be useful in isolating whether to focus in SQL Server or the application

This column identifies the section(B0170000), step(Step01), and SQL statement(H) in the AppEngine program (AP_APY2015). This information can ve used to locate and view the SQL through AppDesigner

Page 28: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-28

Restrictions on enabling the Reuse Statement Setting

It is acceptable to specify ‘ReUse Statement’ if %Bind is used to supply a value to a column in a WHERE predicate, SET clause, or INSERT VALUES list.

Example:UPDATE PS_PF_DL_GRP_EXEC SET PF_ODS_STATUS = 'C',

PROCESS_INSTANCE = %Bind(PROCESS_INSTANCE) WHERE PF_DL_GRP_ID = %Bind(PF_DL_GRP_ID) AND PF_DL_ROW_NUM = %Bind(PF_DL_ROW_NUM)

Do not specify ‘ReUse Statement’ if %Bind is used to supply a column name or portion of a table name.

Example:SELECT DISTINCT KPI_ID , CALC_ID , ' ' ,0 ,0 ,KP_CALC_SW ,KP_OFFCYCLE_CALC FROM PS_%Bind(KP_CALC_AET.KP_KPI_LST1,NOQUOTES)

%Bind(EPM_CORE_AET.FACT_TABLE_APPEND ,NOQUOTES) WHERE LOOP_CNT = %Bind(KP_CALC_AET.LOOP_CNT) AND LOOP_PROGRESSION='B'

Reuse Statement Setting in Application Engine

Page 29: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-29

Do not enable ‘ReUse Statement’ if %Bind appears in the SELECT list.

Example:SELECT DISTINCT %Bind(EPM_CORE_AET.PROCESS_INSTANCE) , %Bind(EPM_CORE_AET.ENGINE_ID) , %CurrentDateTimeIn , 10623 , 31 , 'GL_ACCOUNT' , ' ' , ' ' , ' ' , ' ' , A.MAP_GL_ACCOUNT , ' ' , ' ' , ' ' , ' ' , 'LEDMAP_SEQ' FROM …………

Do not enable ‘ReUse Statement’ If %Bind is being used to resolve to a value other than a standard Bind value and the contents of the Bind will change each time the statement executes.

Example:

%Bind(GC_EQTZ_AET.GC_SQL_STRING,NOQUOTES)

In this case the SQL is different each time (at least from the database perspective) and therefore can't be "reused."

If NOQUOTES modifier is being used inside %Bind, there is an implied STATIC. For dynamic SQL substitution, the %Bind has a CHAR field and NOQUOTES to insert SQL rather than a literal value. If ReUse is enabled, then the value of the CHAR field gets substituted inline, instead of using a Bind marker (as in :1, :2, and so on). The next time that Application Engine Action executes, the SQL that it executes will be the same as before, even if the value of a static bind has changed.

Example:

INSERT INTO PS_PF_ENGMSGD_S

Page 30: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-30

%Bind(EPM_CORE_AET.TABLE_APPEND,NOQUOTES) (PROCESS_INSTANCE , ENGINE_ID , MESSAGE_DTTM , MESSAGE_SET_NBR , MESSAGE_NBR , FIELDNAME1 , FIELDNAME2 , FIELDNAME3 , FIELDNAME4 , FIELDNAME5 , FIELDVAL1 , FIELDVAL2 , FIELDVAL3 , FIELDVAL4 , FIELDVAL5 , SOURCE_TABLE) ……………

To force a ReUsed statement to be re-prepared because one of the static binds has changed and the SQL has to reflect that, use %ClearCursor.

COBOL

Cobol, like SQR, is slowly being replaced by Application Engine as the preferred method of defining and executing batch processing. As of this writing, a moderate amount of COBOL still remains in the Peoplesoft application suites.

Note: COBOL should be modified only as a last resort. Unlike Peoplecode and Application Engine, COBOL is not designed to be extensible by the customer.

Cobol provides similar tracing capability to that found in Application Engine. Below is a sample of a trace produced by COBOL:

Page 31: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-31

PeopleSoft Batch Timings Report: cobsql_4088385.trc

Encoding Scheme Used: Ansi

Time Line Elapsed SQL Cursor Return DB API Statement-------- --------- ------- ------- ------ ------- ----------------10:00:07 1 0.078 0.093 #1 RC=0 Connect=PROD8S/people/?10:00:07 2 0.000 0.000 #1 RC=0 CEX Stmt=SELECT OWNERID, UNICODE_ENABLED FROM PSSTATUS 10:00:07 3 0.000 0.000 #1 RC=0 SSB column=1, type=SQLPSTR, length=810:00:07 4 0.000 0.000 #1 RC=0 SSB column=2, type=SQLPSIN, length=410:00:07 5 0.000 0.000 #1 RC=0 CEX Stmt=SELECT A.ACCESSID, A.ACCESSPSWD, A.ENCRYPTED FROM PSOPRDEFN O, PSACCESSPRFL A WHERE O.OPRID='BAROK' AND A.SYMBOLICID = O.SYMBOLICID 10:00:07 6 0.000 0.000 #1 RC=0 SSB column=1, type=SQLPSTR, length=810:00:07 7 0.000 0.000 #1 RC=0 SSB column=2, type=SQLPSTR, length=810:00:07 8 0.000 0.000 #1 RC=0 SSB column=3, type=SQLPSIN, length=410:00:07 9 0.016 0.000 #1 RC=0 Disconnect10:00:07 10 0.031 0.000 #1 RC=0 Access Connect=PROD8S/sa/?10:00:07 11 0.000 0.000 #1 RC=0 SET OprId=BAROK10:00:07 12 0.000 0.000 #1 RC=0 SET type= cursor=persistent..................................................................................................10:00:08 133 0.016 0.000 #1 RC=0 Commit10:00:08 134 0.000 0.000 #1 RC=0 DYNAMIC Stmt=GLPJREQS_S_DG00010:00:08 135 0.000 0.000 #1 RC=0 COM Stmt=SELECT DISTINCT 'X' FROM PS_JRNL_HEADER WHERE PROCESS_INSTANCE=0004088385 AND BUSINESS_UNIT='175' AND LEDGER_GROUP='ACTUALS' AND SYSTEM_SOURCE <> 'PNL' ....................................................................................................10:00:09 178 0.016 0.000 #7 RC=0 Disconnect10:00:09 179 0.000 0.000 #1 RC=0 Commit10:00:09 180 0.000 0.000 #1 RC=0 Commit10:00:09 181 0.000 0.000 #6 RC=0 Disconnect10:00:09 182 0.000 0.000 #5 RC=0 Disconnect10:00:09 183 0.000 0.000 #4 RC=0 Disconnect10:00:09 184 0.000 0.000 #3 RC=0 Disconnect10:00:09 185 0.000 0.000 #2 RC=0 Disconnect10:00:09 186 0.000 0.000 #1 RC=0 Disconnect PeopleSoft Batch Statistics (All timings in seconds)

Encoding Scheme Used: Ansi

R e t r i e v e C o m p i l e E x e c u t e F e t c h STMT TOTALSStatement Count Time Count Time Count Time Count Time Time % SQL

COMMIT 0 0.00 0 0.00 9 0.03 0 0.00 0.03 2.56CONNECT 0 0.00 0 0.00 8 0.16 0 0.00 0.16 12.47DISCONNECT 0 0.00 0 0.00 8 0.02 0 0.00 0.02 1.28GLPJDLOG_U_DLOG 1 0.01 1 0.00 1 0.00 0 0.00 0.01 1.20GLPJEDIT_S_INSTALL 1 0.01 1 0.00 1 0.00 1 0.00 0.01 1.20GLPJRECF_S_RECFLD 3 0.00 3 0.00 3 0.02 137 0.00 0.02 1.28GLPJRECF_S_SUBREC 3 0.00 3 0.00 3 0.00 3 0.00 0.00 0.00GLPJREQS_S_DG000 0 0.00 1 0.00 1 0.02 1 0.00 0.02 1.28GLPJREQS_S_DJ000 0 0.00 1 0.00 1 0.01 1 0.00 0.01 1.20GLPJREQS_S_OPRDFLT 1 0.00 1 0.00 1 0.02 1 0.00 0.02 1.28GLPJREQS_S_REQUST 2 0.00 2 0.00 2 0.02 2 0.00 0.02 1.28GLPJREQS_U_DC000 0 0.00 1 0.00 1 0.86 0 0.00 0.86 68.67GLPJREQS_U_ILOG 1 0.00 1 0.00 1 0.00 0 0.00 0.00 0.00GLPJREQS_U_INITREQ 1 0.00 1 0.00 1 0.00 0 0.00 0.00 0.00GLPJUPDR_U_UPDREQ 1 0.00 1 0.00 1 0.00 0 0.00 0.00 0.00PTPLOGMS_I_LOGMSG 3 0.00 3 0.00 3 0.00 0 0.00 0.00 0.00PTPLOGMS_I_LOGPARM 1 0.02 1 0.00 3 0.00 0 0.00 0.02 1.28PTPLOGMS_S_GETMSG 1 0.00 1 0.00 3 0.00 3 0.00 0.00 0.00PTPLOGMS_S_MAXSEQ 1 0.00 1 0.00 1 0.00 1 0.00 0.00 0.00PTPLOGMS_S_OPRDEFN 1 0.02 1 0.00 1 0.00 1 0.00 0.02 1.28PTPUSTAT_S_JOBINST 2 0.00 2 0.00 2 0.00 2 0.00 0.00 0.00PTPUSTAT_U_PRCQUE 1 0.00 1 0.00 1 0.00 0 0.00 0.00 0.00PTPUSTAT_U_PRCRQSB 1 0.00 1 0.00 1 0.02 0 0.00 0.02 1.28PTPUSTAT_U_PRCRQSE 1 0.00 1 0.00 1 0.03 0 0.00 0.03 2.48 ------- -------- ------- -------- ------- -------- ------- --------Total: 26 0.06 29 0.00 58 1.19 153 0.00 ----- ----- ----- -----Percent of Total: 4.96% 0.00% 95.04% 0.00%

Time % TotalTotal in SQL: 1.25 90.98Total in Application: 0.12 9.02 -----Total Run Time: 1.38

Page 32: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-32

Changing SQL Statements in COBOL Applications

The majority of SQL statements executed by Peoplesoft COBOL applications are located in table PS_SQLSTMT_TBL. It is relatively easy to modify these statements if necessary in order to change the statement structure or to add hints to improve performance:

1. Locate the SQL statement that is exhiiting poor performance. This can be done through SP_PSWHO (See Appendix) or SQL Server Profiler.

2. Using a portion of the captured SQL statement, locate its entry in PS_SQLSTMT_TBL:

SELECT * FROM PS_SQLSTMT_TBL WHERE STMT_TEXT LIKE '%SELECT S.CALENDR_ID ,L.LEDGER FROM PS_LEDGER_SET_TBL S%

3. The output of this query will identify the application and statement name for the SQL text:

PGM_NAME STMT_TYPE STMT_NAME STMT_TEXT

-------- --------- --------- ------------------------------------

GLPOSLST S LEDSET SELECT S.CALENDAR_ID , …………………………………

Confirm by examining the full text of the SQL statement returned that it is the same as the problem statement captured.

4. Once confirmed. Locate the DMS script used to store the SQL statement into PS_SQLSTMT_TBL. The DMS script will be located in subdirectory src\cbl\base and will be named as program-name.DMS, where program_name will equal PGM_NAME from the query above.

5. Search the DMS script for the specific statement name (LEDSET in the example above). This action will locate the STORE statement that imports the SQL into PS_SQLSTMT_TBL.

6. Modify the statement text. This activity must be done carefully and adhere to syntax requirements of SQLL Server and Peoplesoft. Perform the statement modification with the assistance of a knowledgeable Peoplesoft developer.

7. Execute the DMS script to store the modified SQL statement.

Note: all modifications of this type must be documented thoroughly!

Page 33: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-33

Database Considerations

Peoplesoft Tables“Tools” TablesA subset of tables located in the Peoplesoft schema are referred to as “tools” tables. These tables represent the only portion of the “application” that actually resides on the database server.

Tools tables contain the following types of information:

Object definitions, such as tables (records), columns (fields), indexes, etc.

Control information and definition of Peoplesoft applications, including in some cases the SQL statements that are issued by COBOL and Application Engine program.

Reference tables.

Tables that interact with the batch scheduling service in Peoplesoft. This is called the Process Scheduler.

You can typically identify tools tables by their naming convention. Table names that begin with “PS” but do not have ‘_’ in position 3 of the name are generally tools tables.

Data TablesData tables make up the majority of objects defined in the Peoplesoft schema. Almost all data tables have the naming convention of ‘PS_”, followed by a more or less meaningful name.

Note: Currently, all character columns in the Peoplesoft schema generated for the SQL Server platform are defined as fixed length. This may be a factor in correctly sizing a customer’s database when converting from Oracle (only schema currently that Peoplesoft supports VARCHAR). Resist the urge to modify column definitions to allow VARCHAR; it is not supported by Peoplesoft at this time, but is under consideration for a future Peoplesoft version.

Temporary TablesTemporary tables in Peoplesoft are not temporary; they are permanently defined in the schema and used as working tables for intermediate results produced in batch processes.

Page 34: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-34

You can recognize “temp” tables by their naming convention; a suffix of the form TMP, TAO, or STG, possibly followed by a numeric sequence, are the common practice for naming temporary tables.

Temp tables represent a frequent source of performance challenges within the Peoplesoft workload for the following reasons:

Their contents are temporary. A batch process may start with an empty temp table, populate it with many rows (in some cases thousands or even millions), and then truncate the table at the end of the process. This can make attempting to tune and accurately index for a SQL statement referencing a temp table very difficult; SHOWPLAN, when used interactively, will always have a low (as in 0) estimate of rows emitted by an operator that accesses a temp table.

Temp tables are frequently implicated in blocking and deadlocking episodes. In some cases, multiple batch processes are sharing the same temp tables without sufficient indexing to isolate access. In the worst cases, batch processes complete their processing by issuing a truncate to the temp table; obviously, two concurrent processes cannot successfully complete this statement without encountering lock contention.

It is not uncommon for temp tables to be insufficiently indexed for moderate to large volume batch.

Some general guidelines for handling temp table performance issues are:

Most Application Engine processes are designed to work with a series of temp tables. So, if a process needs temp table PS_PO_UOMCNV_TMP, the a series can be created and defined to App Engine to accommodate the expected number of concurrent processes:

PS_PO_UOMCNV_TMP001

PS_PO_UOMCNV_TMP002

And so on, such that the tables will not be shared. If insufficient number of tables are defined, subsequent processes will revert to using the “original” temp table, and contention may result.

If a temp table is discovered to have indexes, and no clustered index exists, then it is important to change the definition of one of the indexes to be clustered.

Page 35: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-35

If the potential exists to have concurrent access to the temp table by multiple batch processes, at least one index must be defined on the “Process Instance” column (likely followed by other columns). Process Instance is an identifier that uniquely identifies a batch process; indexing on this column helps to eliminate some contention between the processes.

If a temp table is truncated at the end of a process, it cannot be shared. Monitor blocking and deadlocking of batch processes (outlined later in this document) and also Profile the application filtering on %TRUNCATE% in the TextData column to identify all temp tables which are truncated. An easy way to serialize access to these tables is to force table-level locking:

EXEC sp_indexoption ‘PS_PO_UOMCNV_TMP’,’allowrowlocks’,false

EXEC sp_indexoption ‘PS_PO_UOMCNV_TMP’,’allowpagelocks’,false

Using this method may be easier than other approaches to attempt to schedule around concurrency issues on these tables, and it also has the added benefit of reducing locking overhead on the batch processes that are the exclusive user of a table.

If temporary tables must be shared, and there is resulting contention from this, consider using the following technique to reduce lock contention:

1. First, rename the temporary table. One guideline you may use for this is to rename the table with a lower-case version of the original name:

sp_rename ‘PS_GP_NEW_RTO_WRK', 'ps_gp_new_rto_wrk'

2. Next, create a view on the renamed temporary table, giving the view name as the original name, and including in the view text a NOLOCK hint:

CREATE VIEW PS_GP_NEW_RTO_WRK AS SELECT * FROM ps_gp_new_rto_wrk WITH (NOLOCK)

3. The NOLOCK prevents the possibility of lock contention for read operations in the event that the shared temporary table is scanned. The normal concerns around using NOLOCK do not exist with temporary tables: if another concurrent process rolls back, it will not affect the data consumed by any other process.

Page 36: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-36

Peoplesoft IndexesIntroductionThe Peoplesoft schema makes extensive use of composite indexes. Rather than assigned a more compact surrogate key to logically represent a single row of a table, a composite logical key of the business identifiers are used. In practice, the use of the composite keys

Naming Conventions

All index names in the Peoplesoft schema are derived from the name of the table on which they are created.

Primary Index

By default, the index which defines a table’s de facto “primary key” has the same name as the table. For example, on table PS_PAY_EARNINGS, index PS_PAY_EARNINGS will be the unique index that serves as primary key.

Note: Peoplesoft handles data relationships within the application; thus, no true declared primary or foreign keys exist in the schema. So the reference above to primary key means the index key treated by the application as the primary key; it is created as a unique index.

This index is, by default, also the clustered index on each table.

Additional Indexes

Many tables in the Peoplesoft schema have indexes in addition to the default primary. The naming convention for these indexes replaces the “_” in position three of the table name with another character (called the INDEXID in Peoplesoft terminology).

Application Designer It is important that all index changes to the Peoplesoft schema be made through Application Designer. It is through this facility that and index key and other index characteristics are entered. The script to create and/or drop an index is generated from Application designer; the script can be executed directly or saved for subsequent execution.

If index changes are made outside of Application Designer, there is a risk that changes will be lost anytime a new environment is established or refreshed from the scripts that can be generated from Application Designer.

Page 37: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-37

The database object management features of Application Designer operate on a set of Peoplesoft tools tables that contain the definitions of the database objects. The definitions are stored generically; it is the scripting of the object definitions that take into account the platform specific characteristics.

Note: when replatforming a Peoplesoft system to SQL Server, special care should be taken to ensure that any schema changes – indexes or otherwise – that may have been made outside of Application Designer are identified and reconciled. A Peoplesoft utility called DDDAUDIT exists for this purpose; the utility will examine the tools tables that contain object definitions and compare them to the system tables on the target DBMS and database; discrepancies are reported.

Computed Column IndexesMicrosoft SQL Server indexes are currently limited to 16 key columns. In other words, a composite index cannot be made up of more than 16 individual columns.

In some cases, Peoplesoft requires more than 16 columns to enforce uniqueness within a table. When first implemented, the approach Peopleosft took to accomplish this was to create a computed column – called MSSCONCATCOL – that was the concatenation of all columns needed to establish uniqueness. The unique clustered index was then created on this column. Here is an example:

CREATE TABLE PS_LEDGER_BUDG (…………………………………………………MSSCONCATCOL AS BUSINESS_UNIT + LEDGER + ACCOUNT + ALTACCT + DEPTID+ OPERATING_UNIT + PRODUCT + AFFILIATE + AFFILIATE_INTRA2 +PROJECT_ID + BUDGET_PERIOD + SCENARIO + BOOK_CODE + GL_ADJUST_TYPE +CURRENCY_CD + STATISTICS_CODE + CONVERT(CHAR,FISCAL_YEAR) +CONVERT(CHAR,ACCOUNTING_PERIOD))

CREATE UNIQUE CLUSTERED INDEX PS_LEDGER_BUDG ON PS_LEDGER_BUDG(MSSCONCATCOL)

Page 38: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-38

At present SQL Server’s query processor cannot evaluate search arguments on the columns within the MSSCONCATCOL computed column. As a result, the index created on this column is useful only for enforcing uniqueness. It is not useful for any other index operations such as those chosen by the optimizer to speed query processing.

Here are the steps necessary to identify and correct this condition within the Peoplesoft database:

1. Determine if MSSCONCATCOL is the only column in the clustered index on any table in the Peoplesoft schema. The following query can be used for this purpose:select object_name(i.id), i.namefrom sysindexkeys k,

sysindexes iwhere i.id = k.idand i.indid = k.indidand col_name(k.id,k.colid) = 'MSSCONCATCOL'and keyno = 1and indid = 1

2. For each table located in step1, examine the definition of the MSSCONCATCOL and list the columns that are concatenated:

MSSCONCATCOL ASBUSINESS_UNIT + LEDGER + ACCOUNT + ALTACCT + DEPTID + OPERATING_UNIT + PRODUCT + AFFILIATE + AFFILIATE_INTRA2 +PROJECT_ID + BUDGET_PERIOD + SCENARIO + BOOK_CODE + GL_ADJUST_TYPE +CURRENCY_CD + STATISTICS_CODE + CONVERT(CHAR,FISCAL_YEAR) +CONVERT(CHAR,ACCOUNTING_PERIOD))

3. For each table located in step1, find an existing non-clustered that most closely matches on the leading key columns to the columns that are concatenated in MSSCONCATCOL.

Page 39: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-39

CREATE NONCLUSTERED INDEX PSWLEDGER_BUDG ON PS_LEDGER_BUDG (BUSINESS_UNIT, LEDGER, ACCOUNT, ALTACCT, DEPTID, OPERATING_UNIT, PRODUCT, AFFILIATE, AFFILIATE_INTRA2, PROJECT_ID, BUDGET_PERIOD, SCENARIO, BOOK_CODE, GL_ADJUST_TYPE, CURRENCY_CD,STATISTICS_CODE)

4. Indexes located in step 3 are to become their tables’ clustered index. The existing clustered index on MSSCONCATCOL is to remain in place as a unique but non-clustering index:

CREATE CLUSTERED INDEX PSWLEDGER_BUDG ON PS_LEDGER_BUDG (BUSINESS_UNIT, LEDGER, ACCOUNT, ALTACCT, DEPTID, OPERATING_UNIT, PRODUCT, AFFILIATE, AFFILIATE_INTRA2, PROJECT_ID, BUDGET_PERIOD, SCENARIO, BOOK_CODE, GL_ADJUST_TYPE, CURRENCY_CD,STATISTICS_CODE)

CREATE UNIQUE NONCLUSTERED INDEX PS_LEDGER_BUDG ON PS_LEDGER_BUDG (MSSCONCATCOL)

Page 40: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-40

Note: future versions of Peoplesoft will not have this issue; MSSCONCATCOL will exist as the last column of the composite unique index and computed from just the additional columns needed to enforce uniqueness when combined with the columns that precede it in the index’s key column.

Optimizer StatisticsAuto-StatsDatabase options AUTO_CREATE_STATISTICS and AUTO_UPDATE_STATISTICS should be enabled for the Peoplesoft database.

Statistics on Composite IndexesThe majority of indexes in the Peoplesoft schema are composite indexes. This is due to the Peoplesoft approach of using natural – as opposed to surrogate – keys for most cases.

By default, the distribution histogram portion of optimizer statistics is created for the leading column of a composite index. Eventually, with AUTO_CREATE_STATISTICS enabled, statistics entries will be created for the interior columns of the composite key as well.

There are a couple of reasons why preempting the automatic creation of statistics on the interior columns of composite indexes is a good idea:

The creation of statistics does take time and consumes CPU resources. This will be noticeable during the initial period of a workload before optimizer requirements for statistics are known. The initial “burst” of statistics creation can distort some performance test results.

Statistics generated through AUTO_CREATE_STATISTICS are not kept current at the same rate as either manually created statistics or those present through an existing index. This behavior may not be an issue for non-indexed columns, but statistics within a composite index key are vital to accurate optimization.

An approach to address these issues is to create the statistics on all columns of composite indexes in advance using sp_createstats. This procedure, when used with the ‘indexonly’ argument, looks for columns that appear in any position of any index that does not have a statistical entry. It then creates the entry in sysindexes using the column name as the statistics name.

Page 41: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-41

Lock Contention

BackgroundPeoplesoft represents a mixed workload in which a combination of batch read, batch update, and end—user query activities compete with concurrent OLTP transactions. Such a workload is susceptible to lock contention problems.

A majority of contention issues will occur due to batch applications. Batch programs delivered by Peoplesoft or customized by the customer may not be committing transactions at an interval necessary to avoid contention, or may not have suitable indexing for a specific customer’s mix of concurrent batch and OLTP.

Make certain to review the batch processing schedule to determine the requirements for running batch update applications concurrent with other batch and OLTP.

BlockingThere are several strategies available to address blocking issues that arise within the Peoplesoft environment:

Indexing ConsiderationsVolatile Index Columns

Columns that appear in an index that are frequently updated can be a common source of contention. Columns such as flags or switches (with a maximum domain of values being 2) may not afford a great deal of selectivity, and can create conditions of contention when they are updated.

Index Column Sequence

The vast majority of indexes defined in the Peoplesoft schema are composite keys, so the discussion of index column sequence is of some importance to this section of the document.

A commonly stated best practice regarding the design of composite indexes states that the leading column of a composite key should be the column with the greatest selectivity, or expressed in other terms, the lowest density. The theory behind this guideline is that an index key structure with high leading-column selectivity is favored by the optimizer, and therefore is more likely to be used.

Page 42: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-42

This assertion is not necessarily correct; at minimum, it is incomplete in establishing adequate guidelines for designing composite keys that are useful in improving concurrency as well as overall access efficiency.

All indexing decisions on the Peoplesoft schema should factor in the probability and frequency of concurrent OLTP and batch processing.

The commonly used left-key subset of a composite index should be selective enough that it is demonstrably useful to the optimizer. Note that selectivity requirements for index usefulness are considerably lower on clustered indexes than on non-clustered indexes.

The leading columns should be chosen in such a way that they can help concurrent processes remain isolated from one another.

As an example, consider the following hypothetical update statement for a table in the Peoplesoft Financials schema: PS_PYMNT_VCHR_XREF:

UPDATE PS_PYMNT_VCHR_XREF SET SCHEDULED_PAY_DT=@P1 WHERE BUSINESS_UNIT='C0033'AND VOUCHER_ID='00005880'

The update uses the clustered index - PS_PYMNT_VCHR_XREF - on PS_PYMNT_VCHR_XREF to seek on columns BUSINESS_UNIT and VOUCHER_ID. From the density information on index PS_PYMNT_VCHR_XREF (as presented from DBCC SHOW_STATISTICS), it is clear that the leading column BUSINESS_UNIT is not as selective as the 2nd column VOUCHER_ID:

All density Average Length Columns ------------------------ ------------------------ ------------------------------------3.3444816E-3 5.0 BUSINESS_UNIT3.1294536E-7 13.0 BUSINESS_UNIT, VOUCHER_ID3.1279529E-7 17.0 BUSINESS_UNIT, VOUCHER_ID, PYMNT_CNT

However, from the way that this index is being used by this update, it is not the selectivity of the leading column that counts; it is the combined selectivity of the 1st and 2nd columns as expressed by the composite search argument ON BUSINESS_UNIT and VOUCHER_ID.

Page 43: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-43

If we alter the index according to the recommendation to place the most selective columns 1st (which in this example would be VOUCHER_ID), the update statement will still execute quickly, but an opportunity is opened for lock contention. Examine the following abbreviated SHOWPLAN for the UPDATE; its execution plan was created after index PS_PYMNT_VCHR_XREF was altered to move BUSINESS_UNIT to the last position:

|--Clustered Index Update(OBJECT:(PS_PYMNT_VCHR_XREF.PS_PYMNT_VCHR_XREF), SET:(SCHEDULED_PAY_DT=@P1)) |--Compute Scalar(DEFINE:(Expr1005]=@P1)) |--Top(ROWCOUNT est 0) |--Clustered Index Seek(SEEK:(VOUCHER_ID='00005880'),WHERE:(BUSINESS_UNIT='C0033'))

We are seeking on VOUCHER_ID, which in reality is highly selective. The predicate on BUSINESS_UNIT appears in the execution plan’s WHERE argument; it is evaluated after the SEEK and during an ordered scan. Since VOUCHER_ID is not unique across BUSINESS_UNITs, our update will touch those rows 1st (due to the SEEK) and then discard them (based on the WHERE). It is during the evaluation of the BUSINESS_UNIT predicate columns that we can experience contention if there are concurrent updates taking place by BUSINESS_UNIT.

Covering Indexes

A covering index is one in which one or more queries can materialize a result set without having to access a data row. BY avoiding the actual data row the covering index can, in a sense, “skip over” potential lock contention. The indexing recommendations for tables PS_INSTALLATION_xx and PS_BUS_UNIT_TBL_xx are examples of this strategy to avoid locking issues

Mismatching initial lock sizesThe initial lock size chosen for accessing an object is dynamic; lock size can be row (key), page, or table. It is not uncommon for an initial locksize of page to be chosen and result in subsequent contention in the form of blocking or deadlocking.

Page Lock

By examining the output of the SQLTrace and blocking scripts described in the Appendix of this document, it should be a simple matter to determine if page locking is causing contention.

In SQLTrace, look for Page lock requests in the Deadlock Chain event.

In the blocking scripts’ output, look for a blocked lock request of Page.

Page 44: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-44

In either case, there are a couple of courses of action that can be taken to reduce or eliminate the contention.

A quick solution is to use sp_indexoption to disallow page locking, either on the table and all of its indexes, or on the specific index where contention occurred. If this option is taken, careful monitoring of lock memory is advised to ensure that excessive consumption does not occur; lock memory is limited to 60% of SQL Server memory (does not include AWE memory), but performance issues can arise well in advance of this threshold being reached.

A better (but more time consuming) approach to eliminating the page locking is to identify the process that holds or is requesting a real page lock, and then tune the process’s specific SQL. In this context, “real” page locks (S, U, X) are differentiated from intent locks (IS, IU, IX) which are taken in preparation for row or key locks within a page.

It is possible that an initial locksize of table can be chosen by SQL Server for a specific SQL statement; in this case you would observe a table lock being requested, but waiting, in either the deadlock chain or the blocked process scripts. There is currently no method to disallow table locks from being chosen; the SQL would have to include a lock hint to direct SQL Server to acquire a row, key or page lock.

Table Lock

An initial lock size of table is actually very rare. If table locks are observed to be causing contention, then the cause is most likely to be lock escalation.

Lock EscalationLock escalation is an automatic internal event that attempts to mitigate the cost associated managing locks at a low grain. Once an internal threshold is reach by a transaction, lock manager will attempt to escalate from the initial lock size chosen for a table to a table lock.

Note: Lock escalation will not take place of the table lock to be acquired is incompatible with any existing locks currently granted on the table. For example: if there is uncommitted activity on a table and one or more transactions currently hold IX locks on the table, lock escalation will not take place (because the escalated lock of S or X is not compatible with IX. The transaction will defer the escalation attempt until later, after another set of locks are acquired and the threshold is breached again. Therefore, in theory, escalation will only happen on lightly accessed tables and under conditions when escalation will not cause contention.

Lock escalation is a common occurrence in the Peoplesoft and other workloads. In many cases it does not result in contention, and has the positive effect of reducing lock memory consumption.

Page 45: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-45

Determining if Lock Escalation is Causing Contention

1. The 1st step in this determination is to verify that the contention is due to table lock. Using either the SP_BLOCK or SP_TODAYS_BLOCKS stored procedures, look for evidence that processes are blocked by an existing S or X lock on the table. This will appear as a blocked lock request of IS or IX on the table. An example from SP_BLOCK appears below:

BLOCKED_DTTM BLOCKER_SPID TABLENAME LOCK_TYPE LOCK_MODE----------------------- ------------ --------------- --------- ---------2004-04-28 13:48:06.507 110 PS_SHIP_INF_INV TAB IS

2. Next, determine if an escalation event has occurred for the blocker SPID prior to the time of the blocking event.

SELECT * FROM SQLTRACE_VWWHERE SPID = 110AND STARTTIME >= ‘2004-04-28’AND EVENTCLASS = ‘Lock:Escalation’

Controlling Lock Escalation

Lock escalation can be disabled on a SQL Server instance level through use of trace flag 1211. All lock escalation is suppressed when this trace flag is active.

Be certain to monitor the effect of using this trace flag – ideally in a stress test environment that can closely simulate a production environment. Specifically, monitor the consumption of lock memory (found under the SQL Server: Memory Manager object in System Monitor.

An alternative method allows lock escalation to be suppressed on a per-table basis. This approach can be used if it is determined that a small number of tables are implicated in escalation-related contention.

The alternate method uses a script that acquires an innocuous IX lock on a specified table, and does so for a predetermined period of time. Here is the sample script:

WHILE 1 = 1 BEGIN

BEGIN TRANSACTION IF DATEPART(HOUR,(GETDATE()) ) BETWEEN 6 AND 18 SELECT * FROM PS_SHIP_INF_INV

WITH (HOLDLOCK,UPDLOCK) WHERE 0=1 WAITFOR DELAY '00:00:15'

COMMIT END

Page 46: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-46

The script will acquire an IX lock on a specific table (PS_SHIP_INF_INV in this example) between the hours of 6 AM and 6 PM. The IX lock is held for the transaction duration (15 seconds in this example). The script will loop continuously.

The statement above does not change any data, and due to the WHERE predicate will not touch any rows. The lock hints will cause an IX lock to be held to the end of the transaction. WAITFOR specifies the transaction will end after one minute duration. WHILE will cause this transaction to loop continuously.

Note there are activities such creating or dropping indexes that will also be blocked by the above script; it is important to realize that its use does introduce some administrative burden. The 15 second WAITFOR statement in the example above does mitigate this somewhat by allowing blocked table lock requests to be acquired after a maximum wait of 15 seconds (Note: lock escalation requests do not wait but time out immediately and are attempted after another quantity of locks are acquired at the initial lock size).

Known Table/Index Contention IssuesPSVERSION

PSVERSION is a table used to ensure that object version consistency is maintained between the database tier and application tier (cache).

There will be frequent queries to PSVERSION, but update activity should be minimal. Any update activity to PSVERSION can create conditions of blocking which affect many users and batch processes.

Events that may require PSVERSION to be updated include:

Migrating a Peoplesoft project

Update user profiles

Update of tree structures used for query and security

These activities should be performed during periods of relatively low activity to minimize lock contention on PSVERSION.

PS_BUS_UNIT_TBL_xx

A table with this naming convention exists for each of the major subsystems within the Peoplesoft Financials package. The tables are often access and updated within frequently occurring transactions. For example, PS_BUS_UNIT_TBL_AP is used to assign voucher numbers:

UPDATE PS_BUS_UNIT_TBL_AP SET VCHR_NUM_LAST_USED = VCHR_NUM_LAST_USED + 1 WHERE BUSINESS_UNIT = @P1

Page 47: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-47

Blocking can result if the table is queried during periods of frequent updates. Such blocking can be reduced or eliminated by adding indexes to cover frequently occurring queries. For example, if the following common query is being frequently blocked:

SELECT ERS_OPT_DFLT, BUSINESS_UNIT_GL FROM PS_BUS_UNIT_TBL_AP WHERE BUSINESS_UNIT = @P1

The blocking can be eliminated by creating the following non-clustered index on the queries columns:

CREATE INDEX PSZBUS_UNIT_TBL_APON PS_BUS_UNIT_TBL_AP(BUSINESS_UNIT, BUSINESS_UNIT_GL, ERS_OPT_DFLT)

Providing that care is taken to avoid indexing on frequently updated columns (such as VCHR_NUM_LAST_USED in the example above), then PS_BUS_UNIT_TBL_AP can be indexed very aggressively to reduce blocking.

Page 48: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-48

Online UsersIt can be difficult to identify the deadlock participants that are online users. If the online deadlock participant is the deadlock victim, and an error message is recorded in the application server log that can assist in locating the specific Peoplecode and SQL being executed at the time of the deadlock.

1. Examine the output of SQLTrace deadlock events. Joining the SQLTrace table to the PSTPRCSRQST table will identify the batch applications that are participating in the deadlock. SQLTrace rows for SPIDs not represented in the PSTPRCSRQST table will be online users.

2. From the SQLTRACE table, note the time of the deadlock as recorded in the Deadlock Chain event, the table name being referenced in the Deadlock event, and the hostname of the deadlock victim; the latter identifies the application server whose appserv log will contain additional information about the deadlock victim:

Page 49: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-49

3. Locate the appserv log on the server named in the hostname column from SQLTrace.

4. Perform a search on the following string: “SQL error in Exec.”. Continue the search to find the place where this string is preceded with a date and time matching the deadlock chain event.

PSAPPSRV.2800 [06/05/03 13:58:02 [email protected] …… SQL error in Exec………

Additional information located following the date and time of the error in the appserv log identifies the user’s OPRID and IP address of the user’s workstation.

5. An entry in the appserv log resembling the text below will appear nearby. This entry contains the Peoplesoft “record” (SUBCUST_PRF_WRK), the record’s field (CUST_PROFILE_FCN), the field’s action (FieldFormula), and function name (select_nopost). This data can be used to located the portion of Peoplecode where the SQL statement was constructed and issued.

Note: Navigating through the objects identified above is performed through the Application Designer. Be certain to have the assistance of a trained developer or administrator while performing this navigation.

At SUBCUST_PRF_WRK.CUST_PROFILE_FCN.FieldFormula select_notpost PCPC:7071 Statement:79

Page 50: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-50

Locking and Lock ClassesOverviewA lock class is an object used under circumstances when it is necessary to hold a lock for a period longer than normally required by the connection’s isolation level. A common use of lock class is to retain locks acquired under isolation level read-committed until the end of the SQL statements execution. Examples where lock classes are used:

Isolation is read-committed, and:

o Execution plan includes bookmark lookup with prefetch, or:

Page 51: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-51

o Execution plan includes Nest Loop join with prefetch, or:

o Execution plan contains Sort operator or the Table Spool operator for queries that select a text, ntext, or image column.

The effect of lock class is retention of locks until end of statement, even if this isolation is read-committed. This longer-term retention of locks can lock contention, escalation, and large amounts of lock memory consumed.

The lock class can be observed in several ways:

It is exposed in syslockinfo through column req_lifetime. A value of 1 in req_lifetime denotes a lock class applied for the circumstances listed above.

It is present in the output of deadlock trace flag 1204 in the lifetime field. A value of 00000001 denotes a lock class applied for the circumstances listed above.

Deadlock encountered .... Printing deadlock informationWait-for graphNode:1KEY: 5:2121058592:2 (a70064fb1eac) CleanCnt:1 Mode: S Flags: 0x0 Grant List:: Owner:0x19165c00 Mode: S Flg:0x0 Ref:0 Life:00000001 SPID:58 ECID:0 SPID: 58 ECID: 0 Statement Type: SELECT Line #: 1

………

It is also visible when tracing locking detail through trace flag 1200:

Process 52 acquiring S lock on KEY: 7:2009058193:2 (020068e8b274) (class bit1 ref1)

Note: trace flag 1200 introduces significant overhead and should not be used on a production system. Its use here is for demonstration purposes only.

QFE 856Prior to QFE 856, locks acquired in keys or rows that are subsequently disqualified through residual predicates were also retained until end of statement.

The Index Seek operator or the Index Scan operator passes a residual filter criterion to the storage engine, and the row does not meet the criteria that are specified in the filter. If a residual predicate is passed, the showplan output contains a WHERE clause on the

Page 52: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-52

Index Seek operator (and the SEEK condition) or the Index Scan operator. The rows or keys which are disqualified by the WHERE clause of the SEEK or SCAN will not appear in the result set. In spite of this, locks were retained until end of statement.

Post sp3 QFE 856 resolves this issue. Consult detail on QFE 856 in KB article 828096.

Identifying Lock Class Use through SHOWPLAN

At present, showplan does not accurately report prefetch when it is associated with bookmark lookup.

Until the detection of prefetch for showplan is corrected, Trace flag 8719 can be used. 8719 will show the delay operator (for nested loop join prefetch) and a double bookmark lookup (for prefetch with bookmark lookup) in the plan.

The examples below should simplify the identification of circumstances where a lock class is being used:

Bookmark Lookup Example

SELECT * FROM ORDER_ITEM

WHERE SHIP_DATE < '1992-02-01'

|--Bookmark Lookup(BOOKMARK:(Bmk1000), OBJECT:(ORDER_ITEM))

|--Bookmark Lookup(BOOKMARK:(Bmk1000), OBJECT:(ORDER_ITEM))

|--Index Seek(OBJECT:(IX_SHIP_DATE), SEEK:(SHIP_DATE< Convert(@1)))

The 1st bookmark lookup is the prefetch operation. The 2nd is the actual fetch and consumption of the row.

Nested Loop Example

SELECT ORDER_DATE, PRICE_AMTFROM ORDERS, ORDER_ITEMWHERE ORDERS.ORDER_ID = ORDER_ITEM.ORDER_ID AND ORDER_ITEM.ORDER_ID < 100

|--Nested Loops(Inner Join, OUTER REFERENCES:([ORDER_ITEM].[ORDER_ID]) WITH PREFETCH)

|--Delay(OBJECT:([ORDERS].[dbo].[ORDERS].[IX_ORDER_ID]))

| |--Clustered Index Seek(OBJECT:IX_ORDER_ITEM_LINE, SEEK:(ORDER_ID] < 100)

Page 53: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-53

|--Index Seek(OBJECT:IX_ORDER_ID), SEEK:(ORDER_ID=ORDER_ID) ORDERED FORWARD)

When trace flag 8719 is enabled, the delay operator associated with prefetch is recorded in showplan, but it is not useful since the prefetch string is also included in showplan.

SELECT of Long Column Followed by Sort:

There is indirect evidence in showplan for the situation where the lock class is used to protect consistency when a long column is accessed and the execution plan includes a sort.

The following query:

SELECT ORDER_DATE, ORDER_COMMENT FROM ORDERS

WHERE ORDER_ID < 3

ORDER BY ORDER_DATE DESC

Retrieves the text column ORDER_COMMENT, and sorts the result set. The (abbreviated) execution plan is:

|--Compute Scalar(DEFINE:(ORDERS.ORDER_COMMENT=ORDERS.ORDER_COMMENT))

|--Sort(ORDER BY:(ORDERS.ORDER_DATE DESC))

|--Bookmark Lookup(BOOKMARK:(Bmk1000), OBJECT:(ORDERS.dbo.ORDERS))

|--Index Seek(OBJECT:(ORDERS.IX_ORDER_ID), SEEK:(ORDER_ID < Convert(@1))

The subsequent retrieval of the text column via its pointer following the sort operator can be inferred from the Compute Scalar operator in the plan.

Using lock trace 1200, the query’s execution confirms the locks held under the lock class until end of statement:

Process 52 acquiring IS lock on PAG: 7:4:81926 (class bit1 ref1) result: OK

Process 52 acquiring S lock on KEY: 7:2009058193:1 (b200b8c06a27) (class bit1 ref1)

Process 52 releasing lock on KEY: 7:2009058193:2 (010086470766)

Process 52 acquiring S lock on KEY: 7:2009058193:2 (020068e8b274) (class bit0 ref1)

Process 52 releasing lock reference on PAG: 7:4:81926

Process 52 acquiring IS lock on PAG: 7:4:86122 (class bit1 ref1) result: OK

Process 52 releasing lock reference on KEY: 7:2009058193:1 (b200b8c06a27)

Process 52 acquiring S lock on KEY: 7:2009058193:1 (b200b28b0084) (class bit1 ref1) Process 52 releasing lock on KEY: 7:2009058193:2 (020068e8b274)

Process 52 releasing lock on PAG: 7:3:22288

Process 52 acquiring IS lock on PAG: 7:4:47944 (class bit0 ref1) result: OK

Process 52 acquiring S lock on RID: 7:4:47944:2 (class bit0 ref1) result: OK

Page 54: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-54

ORDER_DATE ORDER_COMMENT

------------------- ---------------------------------------------------------------------

1996-12-01 00:00:00 5PRxL1nM7xhQNzP2hnjhy1zz ykhg4P2A MMg5Px3OCN 0B0iyCRgiC2

Process 52 acquiring IS lock on PAG: 7:4:47944 (class bit0 ref1) result: OK

Process 52 acquiring S lock on RID: 7:4:47944:1 (class bit0 ref1) result: OK

ORDER_DATE ORDER_COMMENT

------------------- ---------------------------------------------------------------------

1996-01-02 00:00:00 A0xCm5ARNL mxjChn2kC64xA4L6zBg2O5jhg M42izyPO QlymN1ky5kmSiSgBAQA

Process 52 releasing lock class 0x1

Note: trace flag 1200 introduces significant overhead and should not be used on a production system. Its use here is for demonstration purposes only.

These locks indicate the retrieval of the text data after the row emerges from sort

Page 55: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-55

Query Processor Considerations

Predicate ImplicationOverviewPredicate implication refers to the optimizer’s application of the rule of transitivity. Predicate implication is sometimes referred to as predicate transitive closure. Here is an example.

If the optimizer sees a WHERE clause of the following form:WHERE column_a = ‘X’AND column_a = column_b

Adding and additional redundant predicate:

WHERE column_a = ‘X’AND column_a = column_bAND column_b = ‘X’

Will clearly not alter the functionality of the query, but it does provide the optimizer with more information that could affect the execution plan. In this example, the optimizer would now be able to evaluate the selectivity of AND column_b = ‘X’ , and as a result possibly choose the join order differently as a result.

The optimizer attempts to apply this rule during the simplification phase of optimization by introducing redundant predicates. The presence of the redundant predicates can have the effect of encouraging a more optimal execution plan.

In the current implementation of SQL Server’s optimizer, predicate implication is considered for the following conditions:

Equality predicates on join columns

Equality predicates on correlation columns in correlated subqueries.

Equality predicates on correlation columns in correlated subqueries using EXISTS (as of sp3)

Equality predicates on correlation columns in correlated subqueries using aggregate function such as MIN, MAX (as of post-sp3 QFE 837)

Note that any predicate other than equality is not a candidate for optimizer to consider predicate implication.

Page 56: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-56

Implied Predicates ExampleThe implied predicate for inequalities problem explained above also applies to queries that involve equalities. Example query:

SELECT R.PRCSINSTANCE, R.ORIGPRCSINSTANCE, R.JOBINSTANCE, R.MAINJOBINSTANCE,

R.MAINJOBNAME, R.PRCSITEMLEVEL, R.PRCSJOBSEQ, R.PRCSJOBNAME, R.PRCSTYPE,

R.PRCSNAME, R.PRCSPRTY, R.RUNDTTM, R.GENPRCSTYPE, R.OUTDESTTYPE, R.RETRYCOUNT, R.RESTARTENABLED, R.SERVERNAMERQST, R.OPSYS, R.SCHEDULENAME, R.PRCSCATEGORY, R.P_PRCSINSTANCE, C.PRCSPRIORITY, S.PRCSPRIORITY, R.PRCSWINPOP, R.MCFREN_URL_ID

FROM PSPRCSQUE R,PS_SERVERCLASS S,PS_SERVERCATEGORY C

WHERE R.RUNDTTM <= GETDATE() AND R.SERVERASSIGN = @P1 AND R.RUNSTATUS = @P2 AND S.SERVERNAME = @P3 AND R.PRCSTYPE = S.PRCSTYPE AND R.PRCSCATEGORY = C.PRCSCATEGORY AND C.SERVERNAME = S.SERVERNAME AND C.MAXCONCURRENT > 0 AND R.PRCSTYPE NOT IN ('PSJob') ORDER BY C.PRCSPRIORITY DESC, R.PRCSPRTY DESC, S.PRCSPRIORITY DESC, R.RUNDTTM ASC

Assume in this case S.SERVENAME is assigned a value of ‘PBNTDB007’ at execution time. Since C.SERVERNAME = S.SERVERNAME, it is implied that C.SERVERNAME = ‘PBNTDB007’. The redundant predicate enables the optimizer to consider additional join sequences based on the selectivity of the implied predicate that it introduced.

Manually Adding Redundant PredicatesMost PeopleSoft application tasks are multi-stream enabled, implying that the unit of work to be processed can be divided into sub-units and processed in parallel thereby making efficient use of a multi-processor system and cutting down the execution time.

To enable multi-stream processing PeopleSoft applications often use range operators within the T-SQL queries to delimit the sub-units of work. An example of this would be the text in bold blue in the query below.

INSERT INTO PS_GP_PYE_ITER_LST(EMPLID, CAL_RUN_ID, PRC_NUM, PRC_ACTN, PRC_RSN,

SEQNUM, CURRENT_IND, RUN_CNTL_ID, OPRID, PRC_SAVE_TS)

Page 57: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-57

SELECT DISTINCT A.EMPLID, A.CAL_RUN_ID, B.PRC_NUM, 'C', 'N', 60, 'Y',

B.RUN_CNTL_ID, B.OPRID, B.PRC_SAVE_TSFROM PS_GP_PYE_STAT_WRK A, PS_GP_RUNCTL BWHERE A.EMPLID BETWEEN :1 AND :2

AND A.CAL_RUN_ID=:3AND A.RUN_CNTL_ID = :4AND A.OPRID = :5AND A.RUN_CNTL_ID = B.RUN_CNTL_IDAND A.OPRID = B.OPRIDAND NOT EXISTS(SELECT 'X' FROM PS_GP_PYE_ITER_WRK C

WHERE A.EMPLID = C.EMPLIDAND A.CAL_RUN_ID = C.CAL_RUN_IDAND B.PRC_NUM = C.PRC_NUMAND C.PRC_ACTN ='C')

In this query the parameters :1 and :2 corresponding to the EMPLID are passed in at run time corresponding to the range assigned to the stream.

This EMPLID range defined on table PS_GP_PYE_STAT_WRK (in blue bold above) is then implied for table PS_GP_PYE_ITER_WRK as well via the predicate A.EMPLID = C.EMPLID.

While a valid query construct, this syntax often results in a poor execution query plan with SQL Server 2000 because the range operator does not get automatically implied for the PS_GP_PYE_ITER_WRK table. This can result in degraded performance and at times lead to deadlocks due to the wider range scan on the table PS_GP_PYE_ITER_WRK table than which is absolutely necessary. The effects of this are particularly noticeable when the tables involved in the query are large.

Queries such as these can be trivially optimized by manually adding in the implied predicate for the relevant tables. With this optimization the predicate C.EMPLID BETWEEN :1 AND :2 would need to be added to the above example query as shown in bold red below.

INSERT INTO PS_GP_PYE_ITER_LST(EMPLID,CAL_RUN_ID,PRC_NUM,PRC_ACTN,PRC_RSN,

SEQNUM,CURRENT_IND,RUN_CNTL_ID,OPRID,PRC_SAVE_TS)SELECT DISTINCT A.EMPLID,A.CAL_RUN_ID,B.PRC_NUM,'C','N',60,'Y,

B.RUN_CNTL_ID,B.OPRID,B.PRC_SAVE_TSFROM PS_GP_PYE_STAT_WRK A, PS_GP_RUNCTL BWHERE A.EMPLID BETWEEN :1 AND :2

AND A.CAL_RUN_ID=:3AND A.RUN_CNTL_ID = :4AND A.OPRID = :5AND A.RUN_CNTL_ID = B.RUN_CNTL_IDAND A.OPRID = B.OPRID

Page 58: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-58

AND NOT EXISTS(SELECT 'X' FROM PS_GP_PYE_ITER_WRK C

WHERE A.EMPLID = C.EMPLIDAND C.EMPLID BETWEEN :1 AND :2AND A.CAL_RUN_ID = C.CAL_RUN_IDAND B.PRC_NUM = C.PRC_NUMAND C.PRC_ACTN ='C')

This change would keep the query semantically identical to the original one, but limit the range for table PS_GP_PYE_ITER_WRK to the range delimited by :1 and :2.

The query below presents a slightly more complex example for the same optimization.

Original query:SELECT RA.EMPLID, P.PRC_ORD_TS, RA.EMPL_RCD, RA.RSLT_SEG_NUM, RA.PIN_NUM,

RA.EMPL_RCD_ACUM, RA.USER_KEY1, RA.USER_KEY2, RA.USER_KEY3, RA.USER_KEY4,

RA.COUNTRY, RA.SLICE_BGN_DT, RA.SLICE_END_DT, RA.ACM_TYPE, RA.ACM_PRD_OPTN, RA.ACM_FROM_DT, RA.ACM_THRU_DT,

RA.CALC_RSLT_VAL, RA.CALC_VAL, RA.USER_ADJ_VAL, RA.PIN_PARENT_NUM,

RA.CORR_RTO_INDFROM PS_GP_RSLT_ACUM RA, PS_GP_PYE_PRC_STAT POLD, PS_GP_PYE_PRC_STAT P, PS_GP_PYE_SEG_STAT SWHERE P.CAL_RUN_ID =:1 AND P.EMPLID BETWEEN :2 AND :3 AND (('N'=:4) OR P.EMPLID IN (SELECT EMPLID FROM PS_GP_GRP_LIST_RUN WHERE RUN_CNTL_ID=:5 AND OPRID=:6) ) AND S.EMPLID=P.EMPLID AND S.EMPL_RCD=P.EMPL_RCD AND S.GP_PAYGROUP=P.GP_PAYGROUP AND S.CAL_RUN_ID=P.CAL_RUN_ID AND S.CAL_ID=P.CAL_ID AND S.ORIG_CAL_RUN_ID=P.ORIG_CAL_RUN_ID AND S.EMPLID=RA.EMPLID AND S.EMPL_RCD=RA.EMPL_RCD AND S.GP_PAYGROUP=RA.GP_PAYGROUP AND S.CAL_ID=RA.CAL_ID AND S.ORIG_CAL_RUN_ID=RA.CAL_RUN_ID AND POLD.EMPLID=RA.EMPLID AND POLD.EMPL_RCD=RA.EMPL_RCD AND POLD.GP_PAYGROUP=RA.GP_PAYGROUP AND POLD.CAL_ID=RA.CAL_ID AND POLD.CAL_RUN_ID=RA.CAL_RUN_ID AND POLD.ORIG_CAL_RUN_ID=RA.ORIG_CAL_RUN_ID AND S.RSLT_SEG_NUM=RA.RSLT_SEG_NUM

Page 59: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-59

AND S.RSLT_VER_NUM=POLD.RSLT_VER_NUM AND S.RSLT_REV_NUM=POLD.RSLT_REV_NUM AND RA.ACM_PRD_OPTN <> '1' AND ((S.CALC_ACTION='L' AND NOT EXISTS (SELECT 'X' FROM PS_GP_PYE_SEG_STAT SS WHERE S.CAL_RUN_ID=SS.CAL_RUN_ID AND S.EMPLID=SS.EMPLID AND SS.PYE_CALC_STAT >'00') ) OR (S.PYE_CALC_STAT IN ('50','60','65') AND P.PRC_ORD_TS = (SELECT MAX(PP.PRC_ORD_TS) FROM PS_GP_PYE_PRC_STAT PP ,PS_GP_PYE_SEG_STAT SS WHERE S.CAL_RUN_ID=PP.CAL_RUN_ID AND S.EMPLID=PP.EMPLID AND PP.CAL_RUN_ID=SS.CAL_RUN_ID AND PP.EMPLID=SS.EMPLID AND PP.EMPL_RCD=SS.EMPL_RCD AND PP.GP_PAYGROUP=SS.GP_PAYGROUP AND PP.CAL_ID=SS.CAL_ID AND PP.ORIG_CAL_RUN_ID=SS.ORIG_CAL_RUN_ID AND SS.PYE_CALC_STAT IN ('50','60','65'))

AND EXISTS (SELECT 'X' FROM PS_GP_PYE_SEG_STAT SS WHERE S.CAL_RUN_ID=SS.CAL_RUN_ID AND S.EMPLID=SS.EMPLID AND SS.CALC_ACTION<>'L' AND SS.PYE_CALC_STAT ='00') ) )ORDER BY RA.EMPLID,P.PRC_ORD_TS,RA.EMPL_RCD ,RA.RSLT_SEG_NUM, RA.PIN_NUM

Optimized query (implied predicated highlighted in bold blue)

SELECT RA.EMPLID, P.PRC_ORD_TS, RA.EMPL_RCD, RA.RSLT_SEG_NUM, RA.PIN_NUM,

RA.EMPL_RCD_ACUM, RA.USER_KEY1, RA.USER_KEY2, RA.USER_KEY3, RA.USER_KEY4,

RA.COUNTRY, RA.SLICE_BGN_DT, RA.SLICE_END_DT, RA.ACM_TYPE,

RA.ACM_PRD_OPTN, RA.ACM_FROM_DT, RA.ACM_THRU_DT, RA.CALC_RSLT_VAL,

RA.CALC_VAL, RA.USER_ADJ_VAL, RA.PIN_PARENT_NUM, RA.CORR_RTO_INDFROM PS_GP_RSLT_ACUM RA, PS_GP_PYE_PRC_STAT POLD, PS_GP_PYE_PRC_STAT P, PS_GP_PYE_SEG_STAT SWHERE P.CAL_RUN_ID =:1 AND P.EMPLID BETWEEN :2 AND :3 AND

Page 60: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-60

(('N'=:4) OR P.EMPLID IN (SELECT EMPLID FROM PS_GP_GRP_LIST_RUN WHERE RUN_CNTL_ID=:5 AND OPRID=:6) ) AND S.EMPLID BETWEEN :2 AND :3 AND S.EMPLID=P.EMPLID AND S.EMPL_RCD=P.EMPL_RCD AND S.GP_PAYGROUP=P.GP_PAYGROUP AND S.CAL_RUN_ID=P.CAL_RUN_ID AND S.CAL_ID=P.CAL_ID AND S.ORIG_CAL_RUN_ID=P.ORIG_CAL_RUN_ID AND RA.EMPLID BETWEEN :2 AND :3 AND S.EMPLID=RA.EMPLID AND S.EMPL_RCD=RA.EMPL_RCD AND S.GP_PAYGROUP=RA.GP_PAYGROUP AND S.CAL_ID=RA.CAL_ID AND S.ORIG_CAL_RUN_ID=RA.CAL_RUN_ID AND POLD.EMPLID BETWEEN :2 AND :3 AND POLD.EMPLID=RA.EMPLID AND POLD.EMPL_RCD=RA.EMPL_RCD AND POLD.GP_PAYGROUP=RA.GP_PAYGROUP AND POLD.CAL_ID=RA.CAL_ID AND POLD.CAL_RUN_ID=RA.CAL_RUN_ID AND POLD.ORIG_CAL_RUN_ID=RA.ORIG_CAL_RUN_ID AND S.RSLT_SEG_NUM=RA.RSLT_SEG_NUM AND S.RSLT_VER_NUM=POLD.RSLT_VER_NUM AND S.RSLT_REV_NUM=POLD.RSLT_REV_NUM AND RA.ACM_PRD_OPTN <> '1' AND ((S.CALC_ACTION='L' AND NOT EXISTS (SELECT 'X' FROM PS_GP_PYE_SEG_STAT SS WHERE S.CAL_RUN_ID=SS.CAL_RUN_ID AND SS.EMPLID BETWEEN :2 AND :3 AND S.EMPLID=SS.EMPLID AND SS.PYE_CALC_STAT >'00') ) OR (S.PYE_CALC_STAT IN ('50','60','65') AND P.PRC_ORD_TS = (SELECT MAX(PP.PRC_ORD_TS) FROM PS_GP_PYE_PRC_STAT PP ,PS_GP_PYE_SEG_STAT SS WHERE S.CAL_RUN_ID=PP.CAL_RUN_ID AND PP.EMPLID BETWEEN :2 AND :3 AND S.EMPLID=PP.EMPLID AND PP.CAL_RUN_ID=SS.CAL_RUN_ID AND SS.EMPLID BETWEEN :2 AND :3 AND PP.EMPLID=SS.EMPLID AND PP.EMPL_RCD=SS.EMPL_RCD AND PP.GP_PAYGROUP=SS.GP_PAYGROUP AND PP.CAL_ID=SS.CAL_ID AND PP.ORIG_CAL_RUN_ID=SS.ORIG_CAL_RUN_ID AND SS.PYE_CALC_STAT IN ('50','60','65'))

Page 61: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-61

AND EXISTS (SELECT 'X' FROM PS_GP_PYE_SEG_STAT SS WHERE S.CAL_RUN_ID=SS.CAL_RUN_ID AND SS.EMPLID BETWEEN :2 AND :3 AND S.EMPLID=SS.EMPLID AND SS.CALC_ACTION<>'L' AND SS.PYE_CALC_STAT ='00') ) )ORDER BY RA.EMPLID,P.PRC_ORD_TS,RA.EMPL_RCD ,RA.RSLT_SEG_NUM, RA.PIN_NUM

Optimizing Subqueries with MAX / MIN OperatorsSome (not all) complex PeopleSoft application queries that include the MAX and MIN operators take a long time to compile and may result in a sub-optimal query execution plan. An example of such a query is:

SELECT H.CAL_RUN_ID, B.EMPLID, B.EMPL_RCD, B.GP_PAYGROUP, B.CAL_ID, B.ORIG_CAL_RUN_ID, B.RSLT_VER_NUM, B.RSLT_REV_NUM,

B.PRC_ORD_TS, B.SEL_STAT, B.SEL_RSN, H.RTO_PRC_ID, B.SEL_ACTION, B.PYMT_DT, B.PRC_BGN_DT, B.PRC_END_DT

FROM PS_GP_CAL_RUN_DTL A,PS_GP_PYE_PRC_STAT B,PS_GP_RTO_PRC_WRK H

WHERE H.EMPLID BETWEEN '000029127' AND 'ZZZZZ' AND H.CAL_RUN_ID = 'PE200311' AND (('N'='N') OR H.EMPLID IN (SELECT EMPLID FROM PS_GP_GRP_LIST_RUN WHERE RUN_CNTL_ID='CALCEXT_08' AND OPRID='PS') ) AND H.COUNTRY=A.COUNTRY AND A.GP_PAYGROUP = B.GP_PAYGROUP AND A.CAL_ID = B.CAL_ID AND A.CAL_FINAL_TS IS NOT NULL AND A.CAL_FINAL_TS >= H.CAL_FINAL_TS AND B.EMPLID = H.EMPLID AND B.SEL_STAT IN ('A', 'I') AND B.PRD_TYPE IN ('R', 'C') AND B.SEL_RSN NOT IN ('0A','0B') AND B.RSLT_VER_NUM = (SELECT MAX(M.RSLT_VER_NUM) FROM PS_GP_PYE_PRC_STAT M WHERE M.EMPLID = B.EMPLID AND M.EMPL_RCD = B.EMPL_RCD AND M.CAL_ID = B.CAL_ID AND M.GP_PAYGROUP = B.GP_PAYGROUP AND M.PRC_ORD_TS = (SELECT MAX(Q.PRC_ORD_TS) FROM PS_GP_PYE_PRC_STAT Q WHERE Q.EMPLID=M.EMPLID

Page 62: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-62

AND Q.EMPL_RCD=M.EMPL_RCD AND Q.CAL_ID=M.CAL_ID AND Q.GP_PAYGROUP=M.GP_PAYGROUP) ) AND B.RSLT_REV_NUM = (SELECT MAX(N.RSLT_REV_NUM) FROM PS_GP_PYE_PRC_STAT N WHERE N.EMPLID = B.EMPLID AND N.EMPL_RCD = B.EMPL_RCD AND N.CAL_ID = B.CAL_ID AND N.GP_PAYGROUP = B.GP_PAYGROUP AND N.RSLT_VER_NUM = B.RSLT_VER_NUM AND N.PRC_ORD_TS = (SELECT MAX(Q.PRC_ORD_TS) FROM PS_GP_PYE_PRC_STAT Q WHERE Q.EMPLID=N.EMPLID AND Q.EMPL_RCD=N.EMPL_RCD AND Q.CAL_ID=N.CAL_ID AND Q.GP_PAYGROUP=N.GP_PAYGROUP))

The root of this issue has been traced back to the way the SQL optimizer generates the query execution plan. There are two solutions to work around this problem.

(a) Precede all the MAX and MIN operators with a ‘TOP 1’ clause. This change would preserve the semantics of the query and help the optimizer to generate a better execution plan faster. With this change the first MAX operator in the query above would be converted to:

…AND B.SEL_RSN NOT IN ('0A','0B')AND B.RSLT_VER_NUM =

(SELECT TOP 1 MAX(M.RSLT_VER_NUM) FROM PS_GP_PYE_PRC_STAT M WHERE M.EMPLID = B.EMPLID AND M.EMPL_RCD = B.EMPL_RCD

(b) An alternate solution is to install the SQL Server 2000 post SP3a QFE 933 (or higher) on the database server and enable trace flag 9051 via the startup parameters (SQL Server Enterprise Manager select SQL instance select Properties On the General tab select Startup Parameters… Add –T9051 as a startup parameter).

SET FMTONLY meta-data callsQueries encapsulated via SET FMTONLY are used to retrieve the meta-data from the server. In a sense they can be considered as overhead and should be avoided wherever possible. An example:

SET FMTONLY ON

Page 63: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-63

SELECT A.GPFR_PYSL_B_A, A.GPFR_ELEMENT_ORDER, A.GPFR_DESCR_TYPE, A.DESCR, A.GPFR_PIN_TYPE_EE, A.GPFR_PIN_NUM_EE, A.GPFR_PIN_TYPE_ER, A.GPFR_PIN_NUM_ER, A.GPFR_RETRO_TYPE, A.GPFR_DET_INSTANCES, R.DESCR15, R.PIN_NUM, R.INSTANCE, (CONVERT(CHAR(10),R.SLICE_BGN_DT,121)), (CONVERT(CHAR(10),R.SLICE_END_DT,121)), R.CALC_RSLT_VAL, R.BASE_RSLT_VAL, R.RATE_RSLT_VAL, R.UNIT_RSLT_VAL, R.PCT_RSLT_VAL, R.CURRENCY_CD, R.CURRENCY_CD2 FROM PS_GPFR_PAY_SLIP_1 A, PS_GPFR_RSLT_EA_VW R

SET FMTONLY OFF

With PeopleSoft workloads there are actually two issues related to SET FMTONLY calls.

(a) SQL Server 2000 not processing the SET FMTONLY meta-data calls efficiently (fully optimizes the calls).

(b) PeopleSoft applications generate an extremely large number of SET FMTONLY meta-data calls, at least some of which can be eliminated. These calls result in an unnecessary round-trip to the server causing excessive network traffic and degrading performance.

Issue (a) has been fixed by Microsoft by way of an MDAC QFE (836830) for MDAC driver versions 2.7 SP1 and 2.8.

Issue (b) relates to a sub-optimal coding sequence in the PeopleTools SAM layer and has been identified for a fix in PT release 8.46 (ICE 671044000).

NOTE: To check if your application is generating SET FMTONLY meta data calls the SQL:BatchCompleted trace event needs to be captured.

Page 64: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-64

Monitoring

This section briefly describes the functionality of a set of tools whose scripts and installation notes accompany this document. References to the tools are made throughout this document.

If the scripts are not found, contact Mark Prazak at

[email protected]

SQLTrace EventsThe attached scripts include stored procedure SP_SQLTRACE that captures the following events into table SQLTRACE:

Auto Update Stats

Lock:Deadlock

Lock:Deadlock Chain

Lock:Escalation

Hash Warning

Execution Warnings

Sort Warnings

Data File Auto Grow

Log File Auto Grow

These events are inexpensive to collect on a continuous basis, yet are important in identifying common performance issues

Correlating Batch Applications to SQL Server Activity

Batch applications represent part of the complexity of the Peoplesoft workload that can make performance tuning a challenge. It will often be necessary to relate events that occur within SQL Server identified by a SPID (such as sysprocesses and SQLTrace data) to the information that identifies a particular batch process (Process Instance.

Page 65: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-65

The toolset that accompanies this document creates a trigger on the Process Request table PSPRCSQRST. The trigger name is PSUPSPRCSRQST. Whenever a batch process enters running status, the trigger performs three actions:

1. Appends process instance and process name onto the context_info column of sysprocesses. This allows monitoring of current activity to be quickly identified if it is a batch process.

2. Writes a row to an additional “trigger” table - PSTPRCSRQST – that is added to the Peoplesoft schema. PSTPRCSRQST is a column-subset of PSPRCSRQST, but includes the SPID for the batch process. Having SPID correlated to process instance allows longer-term identification of batch processes implicated in performance issues; any reference to SPID, for example in the SQL Server errorlog or in the SQLTrace covered earlier, can be related to its associated batch process.

3. Optionally, initiate a server side trace that records detail events but is filtered on the process’s SPID.

When a process moves out of running status, trigger PSUPSPRCSRQST performs the following actions:

1. Updates the row in PSTPSPRCSRQST to include the end date/time for the process as well as the first runstatus following running status. Note: the trigger does not update with subsequent post-processing runstatuses; table PSPRCSRQST or the Peoplesoft process monitor must be used to determine final sun status.

SysprocessesContext_infoThe Peoplesoft application servers populate the context_info column of sysprocesses to include information that can be used to identify Peoplesoft users that are associated with a spid.

The default format for Web users is:

OPRID, Windows Login, App Server, Client

Batch processes except SQR:

OPRID, Windows Login, Batch Server

SQR:

OPRID, Process ID

The attached scripts include a trigger on the Process Request table (PSPRCSRQST) that appends batch process information onto context info.

Page 66: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-66

Sql_handleThe sql_handle column introduced in SP3 is especially useful in observing the Peoplesoft workload. Due to the extensive use of cursors and the prepare/execute programming model, active SQL statements are not fully exposed through the DBCC inputbuffer command that was available previously.

The SP_PSWHO and SP_BLOCK stored procedure scripts included with this document are examples that take advantage of this new feature that returns the full SQL text associated with an open handle (Prepare) or cursor.

Page 67: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-67

Appendix

SP_PSWHO-- Copyright © Microsoft Corporation.  All rights reserved. -- THIS SCRIPT IS MADE AVAILABLE TO YOU WITHOUT ANY EXPRESS, IMPLIED OR STATUTORY WARRANTY, -- NOT EVEN THE IMPLIED WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, -- OR THE WARRANTY OF TITLE OR NON-INFRINGEMENT.  -- THE ENTIRE RISK OF THE USE OR THE RESULTS FROM THE USE OF THIS SCRIPT REMAINS WITH YOU.

SET QUOTED_IDENTIFIER ON GOSET ANSI_NULLS ON GO

-- ***********************************************************************-- This stored procedure is provided AS IS with no warranties and confers no rights.-- ***********************************************************************

-- ***********************************************************************-- ***********************************************************************-- ***********************************************************************-- ***********************************************************************CREATE PROC SP_PSWHO -- ***********************************************************************-- The @CONTEXT parameter allows the filtering of output by information-- that is suppplied by Peoplesoft in the context_info column in sysprocesses.-- For example, OPRID is present in context_info, so we can easily filter-- on this:-- SP_PSWHO 'JOE'-- ***********************************************************************

@CONTEXT VARCHAR(128) = '%',-- ***********************************************************************-- The @WAIT parameter allows the filtering for activity that is waiting on-- some resource. The value we pass in this parameter establishes the -- threshhold, in milleseconds, for wait time for activity to be viewed.-- We then examine the WAITTYPE and WAITRESOURCE output of SP_PSWHO to -- determine the exact reason the connection is waiting.-- Consult Knowledge Base article 822101 for details on sysprocesses waittype-- column.-- ***********************************************************************@WAIT INT = 0

Page 68: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-68

ASSET NOCOUNT ON

CREATE TABLE #processes (spid smallint NOT NULL ,waittime int NOT NULL ,waittype varchar(32),waitresourcevarchar(32),cpu int NOT NULL ,physical_io bigint NOT NULL ,open_tran int NOT NULL ,login_time datetime NOT NULL ,hostname char (128) NOT NULL ,program_name char (128) NOT NULL ,nt_domain char (128) NOT NULL ,nt_username char (128) NOT NULL ,loginame char (128) NOT NULL ,context_info varchar(128),sql_handle binary (20) ,stmt_text text NULL)DECLARE@spid smallint ,@waittime int ,@waittype varchar(32),@waitresource varchar(32),@cpu int ,@physical_io bigint ,@open_tran int,@login_time datetime,@hostname char (128),@program_name char (128),@nt_domain char (128) ,@nt_username char (128) ,@loginame char (128),@context_info VARCHAR(128),@sql_handle binary (20) ,@stmt_text VARCHAR(8000)

DECLARE PROCESSES CURSOR FORSELECTspid,waittime,waittype = caseWHEN waittype <> 0X0000 THEN lastwaittypeELSE ' 'END,waitresource,cpu,physical_io,open_tran ,login_time,hostname,program_name,nt_domain,nt_username,

Page 69: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-69

loginame,caseWHEN context_info = 0X0 THEN ' 'ELSE convert(VARCHAR(128),context_info)end,sql_handle,cmdFROM master..sysprocesses-- ***********************************************************************-- ***********************************************************************-- The following criteria may need to be modified if the site's default-- context for unused connections from the application servers is not PSAPPS-- ***********************************************************************-- ***********************************************************************

WHERE spid > 50AND waittime >=@WAITAND dbid = DB_ID()

OPEN PROCESSES

FETCH PROCESSESINTO @spid,@waittime,@waittype,@waitresource,@cpu,@physical_io,@open_tran ,@login_time,@hostname,@program_name,@nt_domain,@nt_username,@loginame,@context_info,@sql_handle,@stmt_text

WHILE @@FETCH_STATUS = 0BEGINIF @sql_handle <> 0x0 AND IS_SRVROLEMEMBER ('sysadmin') = 1SELECT @stmt_text = text from ::fn_get_sql(@sql_handle)INSERT INTO #processesVALUES( @spid,@waittime,@waittype,@waitresource,@cpu,@physical_io,@open_tran ,@login_time,@hostname,@program_name,@nt_domain,@nt_username,@loginame,@context_info,

Page 70: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-70

@sql_handle,@stmt_text)FETCH PROCESSESINTO @spid,@waittime,@waittype,@waitresource,@cpu,

@physical_io,@open_tran ,@login_time,@hostname,@program_name,@nt_domain,@nt_username,@loginame,@context_info,@sql_handle,@stmt_textEND

DEALLOCATE PROCESSES

SELECT * FROM

(SELECT[CONTEXT_INFO] = CASEWHEN context_info <> ' ' THEN context_infoWHEN nt_username <> ' ' THEN RTRIM(nt_username)+','+program_nameELSE RTRIM(loginame)+','+program_nameEND,'SQL' = stmt_text,Server = hostname,spid as SPID,open_tran ,login_time AS [Start Time],[Wait(ms)] = waittime,waittype,waitresource,CPU = cpu,'IO' = physical_io

FROM #processes P) AS PROCESSES

WHERE [CONTEXT_INFO] LIKE '%'+@CONTEXT+'%' -- ***********************************************************************-- * You may want to filter out the unused connections from the app servers-- * Typically, the OPRID for these connections are PSAPPS AND [CONTEXT_INFO] NOT LIKE 'PSAPPS%'-- ***********************************************************************ORDER BY 1

RETURN @@ERROR

Page 71: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-71

SP_BLOCKCREATE procedure SP_BLOCK @BATCH CHAR(1) = NULL AS-- ***********************************************************************-- This stored procedure is provided AS IS with no warranties and confers no rights.-- ***********************************************************************

CREATE TABLE #DBCC(PARENTOBJECTNVARCHAR(128),OBJECTNVARCHAR(128),FIELD NVARCHAR(128),VALUE NVARCHAR(128))DECLARE @BLOCKED TABLE(BLOCKER_SPIDSMALLINT,BLOCKER_CONTEXT VARCHAR(128),BLOCKER_STATUS VARCHAR(18),BLOCKED_SPIDSMALLINT,BLOCKED_CONTEXT VARCHAR(128),WAITTIME INT,LOCK_MODE VARCHAR(7),LOCK_TYPE CHAR(3),DBID SMALLINT,LOCK_RESOURCE VARCHAR(30),BLOCKER_SQL TEXT,BLOCKED_SQL TEXT)DECLARE @BLOCKED2 TABLE(BLOCKER_SPIDSMALLINT,BLOCKER_CONTEXT VARCHAR(128),BLOCKER_STATUS VARCHAR(18),BLOCKED_SPIDSMALLINT,BLOCKED_CONTEXT VARCHAR(128),WAITTIME INT,LOCK_MODE VARCHAR(7),LOCK_TYPE CHAR(3),DBNAMECHAR(8),TABLENAME CHAR(18),INDEXID INT,BLOCKER_SQL TEXT,BLOCKED_SQL TEXT)

SET NOCOUNT ON

DECLARE@BLOCKER_SPID SMALLINT,@BLOCKER_CONTEXT VARCHAR(128),@BLOCKER_STATUS VARCHAR(18),@BLOCKED_SPID SMALLINT,@BLOCKED_CONTEXT VARCHAR(128),@WAITTIME INT,@LOCK_MODE VARCHAR(7),@LOCK_TYPE CHAR(3),@DBID SMALLINT,

Page 72: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-72

@OBJECTID INT,@INDEXID INT,@LOCK_RESOURCE VARCHAR(30),@BLOCKER_HANDLE BINARY(20),@BLOCKER_SQLVARCHAR(8000),@BLOCKED_HANDLE BINARY(20),@BLOCKED_SQLVARCHAR(8000),@CMD VARCHAR(1000),@DELIMITER1 TINYINT,@DELIMITER2 TINYINT,@DELIMITER3 TINYINT,@FILEID VARCHAR(10),@PAGEID VARCHAR(10)

-- --------------------------------------------------------------------------------------- Populate temporary table #BLOCKED from sysindexes for blocked and blocking processes-- -------------------------------------------------------------------------------------

DECLARE PROCESSES CURSOR FORSELECTBLOCKER.spid, -- BLOCKER_SPIDRTRIM(convert(varchar(128),BLOCKER.context_info)), -- BLOCKER_CONTEXTCASE BLOCKER.blocked -- BLOCKER_STATUSWHEN 0 THEN 'Lead Blocker'ELSE 'In Blocking Chain'END,BLOCKED.spid, -- BLOCKED_SPIDRTRIM(convert(varchar(128),BLOCKED.context_info)), -- BLOCKER_CONTEXTBLOCKED.waittime, -- BLOCKED_WAITTIMECASE CONVERT(TINYINT, BLOCKED.waittype) -- LOCK_MODEWHEN 1THEN 'SCH-ST'WHEN 2THEN 'SCH-MOD'WHEN 3THEN 'S'WHEN 4THEN 'U'WHEN 5THEN 'X'WHEN 6THEN 'IS'WHEN 7THEN 'IU'WHEN 8THEN 'IX'WHEN 9THEN 'SIU'WHEN 10 THEN 'SIX'WHEN 11 THEN 'UIX'WHEN 12 THEN 'BU'WHEN 13 THEN 'RangeS-S' WHEN 14 THEN 'RangeS-U' WHEN 15 THEN 'RangeIn-Null' WHEN 16 THEN 'RangeIn-S' WHEN 17 THEN 'RangeIn-U' WHEN 18 THEN 'RangeIn-X' WHEN 19 THEN 'RangeX-S' WHEN 20 THEN 'RangeX-U' WHEN 21 THEN 'RangeX-X'

Page 73: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-73

ELSE 'UNKNOWN'END,SUBSTRING(BLOCKED.waitresource,1,3), -- LOCK_RESOURCE_TYPEBLOCKED.dbid, -- DBIDSUBSTRING(BLOCKED.waitresource,6,30), -- LOCK_RESOURCEBLOCKER.sql_handle,BLOCKER.cmd,BLOCKED.sql_handle,BLOCKED.cmdFROM master..sysprocesses BLOCKERJOIN master..sysprocesses BLOCKED ON BLOCKER.spid = BLOCKED.blockedWHERE BLOCKED.blocked <> 0AND BLOCKER.dbid = db_id()

OPEN PROCESSESFETCH PROCESSESINTO @BLOCKER_SPID ,@BLOCKER_CONTEXT ,@BLOCKER_STATUS ,@BLOCKED_SPID ,@BLOCKED_CONTEXT ,@WAITTIME ,@LOCK_MODE ,@LOCK_TYPE ,@DBID ,@LOCK_RESOURCE ,@BLOCKER_HANDLE ,@BLOCKER_SQL,@BLOCKED_HANDLE ,@BLOCKED_SQL

WHILE @@FETCH_STATUS = 0BEGIN

IF @BLOCKED_HANDLE <> 0x0 AND IS_SRVROLEMEMBER ('sysadmin') = 1SELECT @BLOCKED_SQL = text from ::fn_get_sql(@BLOCKED_HANDLE)

IF @BLOCKER_HANDLE <> 0x0 AND IS_SRVROLEMEMBER ('sysadmin') = 1SELECT @BLOCKER_SQL = text from ::fn_get_sql(@BLOCKER_HANDLE)

INSERT INTO @BLOCKED VALUES( @BLOCKER_SPID ,@BLOCKER_CONTEXT ,@BLOCKER_STATUS ,@BLOCKED_SPID ,@BLOCKED_CONTEXT ,@WAITTIME ,@LOCK_MODE ,@LOCK_TYPE ,@DBID ,@LOCK_RESOURCE ,@BLOCKER_SQL,@BLOCKED_SQL)

FETCH PROCESSES INTO@BLOCKER_SPID ,@BLOCKER_CONTEXT ,

Page 74: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-74

@BLOCKER_STATUS ,@BLOCKED_SPID ,@BLOCKED_CONTEXT ,@WAITTIME ,@LOCK_MODE ,@LOCK_TYPE ,@DBID ,@LOCK_RESOURCE ,@BLOCKER_HANDLE ,@BLOCKER_SQL,@BLOCKED_HANDLE ,@BLOCKED_SQLEND -- @@FETCH_STATUS = 0DEALLOCATE PROCESSES

DECLARE BLOCKED CURSOR FOR SELECTBLOCKER_SPID,BLOCKER_CONTEXT ,BLOCKER_STATUS ,BLOCKED_SPID,BLOCKED_CONTEXT ,WAITTIME ,LOCK_MODE ,LOCK_TYPE ,DBID ,LOCK_RESOURCE ,BLOCKER_SQL ,BLOCKED_SQLFROM @BLOCKED

OPEN BLOCKED

FETCH BLOCKEDINTO @BLOCKER_SPID ,@BLOCKER_CONTEXT ,@BLOCKER_STATUS ,@BLOCKED_SPID ,@BLOCKED_CONTEXT ,@WAITTIME ,@LOCK_MODE ,@LOCK_TYPE ,@DBID ,@LOCK_RESOURCE ,@BLOCKER_SQL,@BLOCKED_SQL

WHILE @@FETCH_STATUS = 0 BEGIN-- --------------------------------------------------------------------------------------- Decode the waitresource column from sysprocesses.-- The 1st 5 bytes have already been trimmed off and stored in LOCK_TYPE-- --------------------------------------------------------------------------------------- Establish position of the delimiters between the fields of the lock

Page 75: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-75

-- resource. In order to establish a uniform delimiter to look for, replace ':' with ' '-- -------------------------------------------------------------------------------------SET @LOCK_RESOURCE = REPLACE(@LOCK_RESOURCE,':',' ')SET @DELIMITER1 = CHARINDEX(' ',@LOCK_RESOURCE)SET @DELIMITER2 = CHARINDEX(' ', @LOCK_RESOURCE, (@DELIMITER1+1))SET @DELIMITER3 = CHARINDEX(' ', @LOCK_RESOURCE, (@DELIMITER2+1))-- --------------------------------------------------------------------------------------- Delimiter positions are then used to substring fields from @LOCK_RESOURCE-- -------------------------------------------------------------------------------------IF @LOCK_TYPE IN ('RID','PAG')BEGIN-- --------------------------------------------------------------------------------------- Extract objectid and indexid from file/page for resources of RID or PAG.-- LOCK_RESOURCE_TYPE 'RID': dbid:fileid:pageid:row#-- Example: 7:3:47912:10-- LOCK_RESOURCE_TYPE 'PAG': dbid:fileid:pageid-- Example: 7:3:47912-- -------------------------------------------------------------------------------------SET @FILEID = SUBSTRING(@LOCK_RESOURCE,@DELIMITER1+1,(@DELIMITER2-@DELIMITER1)-1)SET @PAGEID = SUBSTRING(@LOCK_RESOURCE,@DELIMITER2+1,(@DELIMITER3-@DELIMITER2)-1)-- --------------------------------------------------------------------------------------- Execute DBCC PAGE to determine the object owner of the page. We use the rowset-- version of DBCC (WITH TABLERESULTS) to make it easy to retrieve objext id and index id.-- -------------------------------------------------------------------------------------SET @CMD = 'DBCC PAGE ('+CONVERT(VARCHAR(3),@DBID)+','+@FILEID+','+@PAGEID+') WITH TABLERESULTS, no_infomsgs'INSERT INTO #DBCC EXEC(@CMD)SELECT @OBJECTID = CONVERT(INT,SUBSTRING(VALUE,1,30)) FROM #DBCC WHERE FIELD = 'm_objId' OPTION(KEEP PLAN)SELECT @INDEXID = CONVERT(INT,SUBSTRING(VALUE,1,30)) FROM #DBCC WHERE FIELD = 'm_indexId' OPTION(KEEP PLAN)TRUNCATE TABLE #DBCCEND

IF @LOCK_TYPE = 'TAB'BEGIN-- --------------------------------------------------------------------------------------- LOCK_RESOURCE_TYPE 'TAB': dbid:objectid-- Example: 7:1993058136-- -------------------------------------------------------------------------------------SET @OBJECTID = SUBSTRING(@LOCK_RESOURCE,@DELIMITER1+1,(@DELIMITER2-@DELIMITER1)-1)

-- --------------------------------------------------------------------------------------- For table locks, set indexid to '0'

Page 76: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-76

-- -------------------------------------------------------------------------------------SET @INDEXID = 0-- --------------------------------------------------------------------------------------- If the waitresource contains the keyword COMPILE, then the object name is actually a-- stored procedure.-- Example: 6:834102 [[COMPILE]]-- -------------------------------------------------------------------------------------IF @LOCK_RESOURCE LIKE '%COMPILE%'SET @LOCK_TYPE = 'PRC'END

IF @LOCK_TYPE = 'KEY'BEGIN-- --------------------------------------------------------------------------------------- LOCK_RESOURCE_TYPE 'KEY': dbid:objectid:indexid (hash of key value)-- Example: 7:1993058136:4 (0a0087c006b1)-- -------------------------------------------------------------------------------------SET @OBJECTID = SUBSTRING(@LOCK_RESOURCE,@DELIMITER1+1,(@DELIMITER2-@DELIMITER1)-1)SET @INDEXID = SUBSTRING(@LOCK_RESOURCE,@DELIMITER2+1,(@DELIMITER3-@DELIMITER2)-1)END

-- --------------------------------------------------------------------------------------- If executing in batch, write directly to SQLPERF..BLOCKED_PROCESSES-- -------------------------------------------------------------------------------------IF @BATCH IS NOT NULL

INSERT INTO SQLPERF..BLOCKED_PROCESSES WITH (TABLOCKX) VALUES (GETDATE() ,@BLOCKED_SPID ,@BLOCKED_CONTEXT ,@BLOCKER_SPID ,@BLOCKER_CONTEXT ,@BLOCKER_STATUS ,@WAITTIME ,LEFT(DB_NAME(@DBID),8) ,LEFT(OBJECT_NAME(@OBJECTID),18) ,@INDEXID ,@LOCK_TYPE ,@LOCK_MODE ,@BLOCKER_SQL,@BLOCKED_SQL)ELSE-- --------------------------------------------------------------------------------------- Otherwise, insert into temp table for subsequent OUTPUT-- -------------------------------------------------------------------------------------INSERT INTO @BLOCKED2 VALUES (

Page 77: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-77

@BLOCKER_SPID ,@BLOCKER_CONTEXT ,@BLOCKER_STATUS ,@BLOCKED_SPID ,@BLOCKED_CONTEXT ,@WAITTIME ,@LOCK_MODE ,@LOCK_TYPE ,LEFT(DB_NAME(@DBID),8) ,LEFT(OBJECT_NAME(@OBJECTID),18) ,@INDEXID ,@BLOCKER_SQL,@BLOCKED_SQL)

FETCH BLOCKEDINTO @BLOCKER_SPID ,@BLOCKER_CONTEXT ,@BLOCKER_STATUS ,@BLOCKED_SPID ,@BLOCKED_CONTEXT ,@WAITTIME ,@LOCK_MODE ,@LOCK_TYPE ,@DBID ,@LOCK_RESOURCE ,@BLOCKER_SQL,@BLOCKED_SQL

END -- WHILE @@FETCH_STATUS = 0

DEALLOCATE BLOCKED-- ------------------------------------------------------------------ Format output.-- ----------------------------------------------------------------

IF @BATCH IS NULLSELECT BLOCKED_DTTM = GETDATE(),BLOCKED_SPID,BLOCKED_CONTEXT, BLOCKER_SPID,BLOCKER_CONTEXT,BLOCKER_STATUS ,WAITTIME AS[WAITING(ms)],DBNAME,TABLENAME ,INDEXID , LOCK_TYPE ,LOCK_MODE ,BLOCKER_SQL ,BLOCKED_SQLFROM @BLOCKED2 ORDER BY BLOCKER_STATUS desc, [WAITING(ms)] DESCRETURN

Page 78: People Soft Performance Tuning on SQL Server

Tuning the Database Tier 1-78