Performance Tune Your iSeries for E1

Session: 27960Tom DavidsonIBM Certified ILE SpecialistRS Infocontdavidson@rsinfocon.com

Ground Rules• Please set cell phones to ‘stun’

• Feel free to ask questions as we go along, I may defer to later in the session or offline

• Not all slides in handout will be on-screen, they are included for your reference

Speaker Bio• CNC for a $3B Chemical Company for 7

years• 24+ years on the System i &

predecessors• 10+ years performance tuning• Now a consultant, working with Xe

through 8.12 installations

RS Infocon• Consulting firm, not a network of contractors• “Rightsourcer” On-site, off-site, off-shore• Specialties include complex implementations

& architectures

• Milwaukee Metropolitan Area of Commerce "Future 50" award in 2006 and 2007

Agenda• Theory

– How work is processed on System i– General performance info– Segregation of work– E1 units of work– SQL Indexes

• Practice– Tools– System Values– Segregation of work– SQL Indexes– Caching

Definitions

• Job, task, thread – I use these inter changeably. Normally they are equivalent (major exception is WAS)

• Indexes - I use index to indicate both an SQL index and a System I Logical File

Definitions (cont)• Resource – Something a job needs to

continue it’s work. i.e. CPU, memory, data (disk)

• Performance Tuning – Process of improving throughput of a system, while simultaneously ensuring the important work gets done first

Theory

• Goal is to reduce faults, this will increase CPU & Memory utilization.

• Ensure that all tasks get the appropriate amount of resources, when they need it.

• Cost is an important element of performance

Theory

• 3 Primary elements of tuning– CPU Utilization– Memory Utilization– Disk Utilization

Acceptable CPU utilization

• Depends upon the # of CPU’s– # Max Utilization– 1 70%– 4 85%– 10 92%– 20 95%

Timeslices

• Used to determine how long a task can hold the CPU

• Default settings the same as they were in 1983

• Batch use 1500

• Interactive use 1200

Acceptable Memory Utilization

• This the most subjective, but arguably the most important– Memory is ALWAYS over committed– More is not always better (but normally is)– Hardest to get a ‘read’ on.

Acceptable Disk Utilization• Depends on # of arms and total disk space• In general:

– < 500 G, 70% used– < 1T, 75% used– > 5T, 80-85% used

• This assumes that you are using smaller drives, if you are using large drives these numbers will be significantly lower

• The number of disk arms is as important as the amount of disk space

How things work

• Once a job goes active it will hold the CPU until one of the following occur:– It consumes its timeslice (known as TSE)– It reaches a ‘long’ wait.

• Waiting for disk I/O• Waiting for user input• Waiting for Communications

How things work (cont)

• Lifecycle of a task– Job goes active– Job structures/data paged in (if needed)– Begins execution– One of 3 things happen:

• Job reaches TSE• Job faults• Job ends

How things work (cont)– If TSE, question is asked:

• Is there a HIGHER priority task waiting?– Yes, give up the CPU– No, continue

– If a fault, question is asked:• What is the highest priority task waiting

– If job ends, highest priority task waiting is started

The DATABASE:

• The job of an System I is:– Maintain the integrity of the database

– Serve the DB to requestors

– Perform other tasks as required

The DATABASE (cont)• Integrity

– When a record is added/changed/deleted the file can not be used until ALL indexes are updated, this is called a seize. Seizes are normal, and are a required element of a DB

– The number of indexes can affect the length of a seize

The DATABASEServe the DB to requestors• This is where performance comes in.• The database is predictive, it knows when

you are doing sequential operations and will ‘pre-fetch’ data to prevent faults.

• It also knows when you are doing random I/O and reduces how much it brings in.

• Determines best way to get the data

The DATABASE:

• Perform other tasks– This is not optimized, DB processes within

these tasks do get optimized.– This is where you can shine

Segregation of Work

• Performed by isolating tasks in their own subsystems and memory pools.

• Allows tasks to keep resources in memory

• Helps prevent ‘thrashing’ & promotes ‘sharing’.

• Implemented via memory pools

Memory pool basics

• All tasks run within a memory pool

• Basic memory pools are:– 1 *MACHINE - OS runs here– 2 *BASE - Everything else– *SPOOL – Printing– *INTERACT – Green Screen– Custom Pools

Memory pool basics (cont)– Max Active – How many tasks can compete for the

CPU, at a time– Paging and faulting

• Definitions– Pages is the number of pages brought in– Faults is the number of times a task requested a page that

was not in memory• Two types

– Database» Shows numbers for DB operations

– Non-Database» Shows numbers for all other operations

Memory pool basics (cont)– *MACHINE Pool

• If the machine waits EVERYONE waits• Very little DB work goes on here• Base system internal tables are a large portion

of this area• IBM recommends that the number of faults (DB

& Non-DB) be less than 10

Memory pool basics (cont)– *BASE

• Default pool for most stuff ‘out of the box’• Holds all memory not currently allocated to

other pools

– *SPOOL• Used for printing• It is OK to have large faulting numbers in this

pool

Memory pool basics (cont)– *INTERACT

• Green Screen jobs run here• If you are not running many GS apps, keep this

memory constrained

– Custom pools• User defined• You will use these pools to do your JDE work

Memory pool basics (cont)

Types of memory pools

• Private– Dedicated to a single subsystem

• Shared– May be shared among subsystems– Allows you to adjust paging characteristics

if you want (you want)

Memory pool basics (cont)

How to determine what pools you need

• Determine what different TYPES of work you have– Kernels, ODBC, UBE’s, etc– Interactive, batch, client/server, etc

• Determine what resources are shared– Programs, *SRVPGM (DLL), files

Memory pool basics (cont)• Goal is to reduce faulting

– Reduced faulting normally indicates that the CPU is being well utilized

– With reduced faulting, active jobs spend less time waiting and more time processing

– EXCEPTION: If you have a large amount of jobs at the same priority, you may want to allow faulting to perform task switching

SQL Indexes

• The only generic way to affect database performance

• An effective indexing scheme can improve performance even in the absence of proper tuning

• A bad indexing scheme can make good performance impossible

E1 Units of work

• Kernels– Do most of the ‘real’ work in E1– You define how many of each type may run

• UBE’s– Essentially batch jobs

E1 Units of work (cont)• ODBC (JDBC)

– Not technically a JDE unit of work– Handles retrieval of data for non-kernel,

non-UBE tasks

• WAS– Not a JDE unit of work– Acts as a platform for JDE unit’s of work

SQL Indexes

• Improves performance multiple ways– Indexes hold information about the data– Many queries can be implemented in the

index. COUNT, WHERE clause if no matches

– Reduces amount of data which needs moving

Agenda Theory

– How work is processed on System i– General performance info– Segregation of work– E1 units of work– SQL Indexes

• Practice– Tools– System Values– Segregation of work– SQL Indexes– Caching

Tools• Green Screen

– WRKSYSSTS– DSPACTPJ

• Client Access– Work Management Tasks

• System Values• Shared pools• Databases

– SQL Plan Cache– Index Advisor

Tools (cont)

• Third Party– Centerfield– MPG Performance Navigator– IBM

System Values

• QDYNPTYADJ – Set to 1

• QDYNPTYSCD – Set to 1

• QPFRADJ – Set to 0 or 2

• QQRYDEGREE – Set to *OPTIMIZE

• QQRYTIMLMT=86400 (1day)

System Values (cont)

• QACTJOB = 600

• QADLACTJ = 100

• QTOTJOB = 5000

• QADLTOTJ = 100

• QMAXJOB may need changing if you have a large shop

Disk Tuning - Journaling• No need anymore to separate journal to

it’s own ASP• By default journals are maintained on 1st

10 disk drives• You may change the journal so that up

to 100 disks are used• May be cached

Disk Tuning - Journaling

Segregation of work

• Create memory pools for different types of work

• Create new subsystems for E1 related tasks

• Modify existing subsystems to use shared memory pools

Performance and E1

• Elements of E1 on the System I– JDE Code itself– ODBC (JDBC)– UBE’s– 3rd Party products– Other products

JDE Elements (examples)

• Kernels

• QZDASOINIT/QSQSRVR

• WAS

• R09801

• Scheduler

Memory pools• You will need the following pools:

– *MACHINE– *BASE– *INTERACT– *SPOOL– Kernel Pool– Batch Pool– ODBC/JDBC Pool– WebSphere*– Package build*– Cache*

Memory pools (cont)

Tuning

Kernel Memory Pool

Batch • Create a new subsystem• Components

– Subsystem description– Class– Job Queue– Routing entry

• Handout contains the ‘How-To’

JDBC/ODBC - Monitoring

• Use DSPACTPJ command

JDBC• Connections handled by QSQSRVR job in

QSYSWRK• By default they run in *BASE• Shipped with 5 pre-started• You want it to share with ODBC• Rule of thumb: 5 + (3 for each JAS server) + (2 for

each CO kernel)• Set threshold to 1/3 or 10 which ever is larger• Set additional to 2 times threshold

ODBC• Used mostly by developers• Handled by QZDASOINIT (not SSL) or

QZDASSINIT (SSL)• Run in it’s own subsystem• Share memory with JDBC• Rule of thumb: Start with 100 jobs, Threshold

of 10, Additional twice threshold

ODBC (cont)• Process

– Create Subsystem– Add job queue– Add pre-start job entry– Create the job description– Create the ODBC class– Start subsystem– Use iSeries Access to route IP’s to new

subsystem

Create ODBC Subsystem

Create the class

Add ODBC Pre-start job entry

Add ODBC Pre-start job entry (cont)

Route requests

• Notes:– Requires static IP’s OR IP address range

• Static IP’s usually not a problem for servers• Address range works best for DHCP &

developers

Routing Requests (cont)

Routing requests (cont)

SQL Indexing• You WILL need to add indexes• I recommend that when you add an index, you do it

via E1• Several different advisors for what indexes to add.

– iSeries Nav– QSYS2/SYSIXADV– Centerfield (third party)– DBMON (V5R3 & before)

• No matter what method you use, you will need to determine what the value of the index is.

SQL Indexing• Considerations

– What is the business purpose for the index.– Can I extend an existing index?– Is the index for batch or interactive use?– How often is the index really used?– Does it make sense? Can training remove the

need?– If over 60 or so indexes, is it more important than

one of them?– Remember NOTHING IS FREE.

SQL Indexes – iSeries Navigator• Index Advisor

– Works at system, database, and file level– Advantages

• Free, kind of• Covers a lot

– Disadvantages• Voluminous• No selection criteria

SQL Indexes – iSeries Navigator

SQL Indexes - QSYS2/SYSIXADV

• System wide collection – Base O/S• Advantages

– Absolutely free– Covers everything

• Disadvantages– No real interface, must use SQL or

programs

SQL Indexes - Centerfield• Third party tool• Advantages

– E1 Module– Comprehensive– Data selection capability– Reporting capability

• Disadvantages– Extra cost

SQL Indexes - Centerfield

Caching

• Optional, you may do just fine without it

• Pins programs and files in memory so there is no wait (fault).

• I recommend starting with a cache no larger than 1% or your system memory

• Best for often used objects.

Caching (cont)

• Reduces the likelihood of data being paged out

• You really need to know how your DB is used

• Easy to get carried away, can waste memory

Caching - Implementation

• Determine what to cache– Small highly used files, Next numbers

come to mind– Should be pretty ‘static’– Programs, look at invocation stacks, don’t

be afraid of QSYS objects

Caching - Implementation (cont)

• Process– Create subsystem, with private pool– Create job description– Add auto-start job– Schedule periodic refreshes via scheduler

Caching – Implementation (cont)

Caching – Implementation (cont)

Caching – Implementation (cont)

Caching – Implementation (cont)

Caching - SETOBJACC

Summary• Lots of things you can tweak• Proper indexing is most important• Segregation of work helps a lot

• NOTHING HELPS MORE THAN KNOWING YOUR BUSINESS AND YOUR DATABASE

Questions

Further Reading• General Performance Info

http://publib.boulder.ibm.com/infocenter/iseries/v5r4/index.jsp?topic=/rzamy/50/admin/prftunedb.htm

• WAS 6 tuning http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100985

• IBM Techdocs http://www-03.ibm.com/support/techdocs/atsmastr.nsf/Web/TechDocs

Third party vendors

• RS Infocon http://www.rsinfocon.com/

• Centerfield Technologies http://www.centerfieldtechnology.com

• Midrange Performance Group http://www.mpginc.com/