2015 IBM Corporation 2016 IBM Corporation
Boosting SQL Performance with Indexing Technology
Rob [email protected] - DB2 for i
IBM Training
2016 IBM CorporationIBM Power Systems2
index
Something that serves to guide, point out, or otherwise facilitate reference.
NEUG
C 2016
NEUG
C 2016
1
IBM Training
2016 IBM CorporationIBM Power Systems3
Indexing
IBM Training
2016 IBM CorporationIBM Power Systems4
Index Types
Two types of indexing technologies are supported Radix Index. The default unless you explicitly say otherwise Encoded Vector Index
Each type of index has specific uses and advantages
Radix is the best general purpose index
Respective indexing technologies complement each other
NEUG
C 2016
NEUG
C 2016
2
IBM Training
2016 IBM CorporationIBM Power Systems5
Index EssenceIndexes: Are helper objects
In SQL they exist simply to improve performance (try referencing them in a FROM clause)
Are used for implementation and statistics
Are maintained automatically by the database as the table rows come and go
Only contain active rows (no deleted rows!)
Contain data (the key) and a row reference (RRN)
Are most often used in conjunction with table access
They assist in determining the desired rows of the table, then the rows are retrieved from the table
Can be used without having to access the table index only access
IBM Training
2016 IBM CorporationIBM Power Systems6
Index Usage - Probe vs. Scan
Indexes can be probed or scanned Probe using the leading (contiguous) key columns for fast lookups
If more than one key is used, equal selection is required for all but the last key Scan can occur on any key column
Probe and scan can be used together Probe to a subset of entries, then scan each of those
A Radix index can provide ordering of data (ORDER BY) Can leverage this side effect with either probe or scan
Indexes are cooperative More than one index can be used for a table in a given query
Requires use of RRN list or bitmap of RRNs Mixing and matching Radix and EVI supported
NEUG
C 2016
NEUG
C 2016
3
IBM Training
2016 IBM CorporationIBM Power Systems7
CUSTOMER ORDER ITEM
001 B507 AB-2700
001 B607 CD-2000
002 B100 XY-1005
002 B102 AZ-5000
003 B709 HH-6500
004 B043 HH-6500
Index Key Columns (CUSTOMER, ORDER, ITEM)
Probe (key positioning)with leading, n contiguouskey columns11+21+2+3
Scan (test)with any otherkey columns232+3
...WHERE ORDER = B102 AND CUSTOMER = 002
...WHERE ITEM = HH-6500
WHERE ORDER = B102 OR CUSTOMER = 002
Using Indexes - Probe v Scan
Why is the last WHERE example a scan??
IBM Training
2016 IBM CorporationIBM Power Systems8
SELECT *FROM EMPLOYEEWHERE STATE = MINNESOTA'AND WORKDEPT IN ( 'B01', C01, 'E01')
IntermediateRRN list
IntermediateRRN list
FinalRRN list
AND(Merge)
State Workdept
State Workdepts
351015
10001005100730013050
371027
10001010203530014100
310
10003001
Two Radix - ANDing Example
Represents all the local selection
RadixRadix
NEUG
C 2016
NEUG
C 2016
4
IBM Training
2016 IBM CorporationIBM Power Systems9
Encoded Vector Index (EVI)
Index for delivering fast data access in analytical and reportingenvironments Used to produce dynamic bitmaps and RRN lists Fast access to statistics to improve query optimizer decision making Can also include non-key summary data (SUM, COUNT)
Not a tree structure. Contains two parts:1. Symbol table of unique key values. Each unique value is encoded
(assigned) to a number2. Vector with one entry per row in the underlying table. Each row contains
the associated key values assigned number
Can only be created through an SQL interface or System i Navigator GUI
CREATE ENCODED VECTOR INDEXSchemaName/IndexName ON SchemaName/TableName(ColumnName)INCLUDE ( SUM(ColumnName));
IBM Training
2016 IBM CorporationIBM Power Systems10
SELECT *FROM EMPLOYEEWHERE STATE = MINNESOTA'AND WORKDEPT IN ( 'B01', C01, 'E01')
IntermediateRRN list
IntermediateRRN list
EVI
FinalRRN list
AND(Merge)
State Workdept
State Workdepts
EVI
351015
10001005100730013050
371027
10001010203530014100
310
10003001
Two EVIs - ANDing Example
Represents all the local selection
NEUG
C 2016
NEUG
C 2016
5
IBM Training
2016 IBM CorporationIBM Power Systems11
001100001...1...
000100000...1...
001000001...1...
SELECT *FROM EMPLOYEEWHERE STATE = IOWA'OR WORKDEPT IN ( 'B01', C01, 'E01')
IntermediateBitmap
IntermediateBitmap
RadixEVI
FinalBitmap
OR(Merge)
State Workdept
State Workdepts
Represents all the local selection
Radix + EVI Usage - ORing Example
IBM Training
2016 IBM CorporationIBM Power Systems
Search / Scansymbol table
for key(s) and counts
CREATE ENCODED VECTOR INDEX idx2 ON employee (state) INCLUDE (SUM(x), SUM(y), COUNT(*))
SELECT COUNT(*) FROM EMPLOYEE WHERE STATE = Wisconsin;SELECT COUNT(DISTINCT STATE) FROM EMPLOYEE;SELECT STATE, SUM(x), SUM(y) FROM EMPLOYEE GROUP BY STATE;
EVI Index for Aggregate Answers
Symbol Table
Key Value Code CountInclude
Sum(x)
Include
Sum(y)Arizona 1 5000 1500 2005Arkansas 2 7300 3200 450Wisconsin 49 340 575 1200Wyoming 50 2760 210 0 NEU
GC 20
16
NEUG
C 2016
6
IBM Training
2016 IBM CorporationIBM Power Systems13
cardinality The number of elements in a setHigh cardinality = large number of distinct values
Low cardinality = small number of distinct values
In generalA radix index is best when accessing a small set of rows and the key cardinality is high
An encoded vector index is best when accessing a subset of rows and the key cardinality is low
Understanding the data and query is essential
The right index for the job
IBM Training
2016 IBM CorporationIBM Power Systems14
Creating Indexes How?
CREATE INDEX SQL statement
CREATE INDEX MY_IX on MY_TABLE (KEY1, KEY2)
CREATE ENCODED VECTOR INDEX SQL statement
CREATE ENCODED VECTOR INDEX MY_EVI on MY_TABLE (KEY1)
System i Navigator Database graphical interface
CRTPF and CRTLF CL commands (under the covers) Keyed access path within the physical file or logical file
Join logical file
Caution: Do not use this (native) approach for SQL! Illustration purposes only
Primary Key, Foreign Key and Unique Key Constraints have indexes created automatically under the covers
NEUG
C 2016
NEUG
C 2016
7
IBM Training
2016 IBM CorporationIBM Power Systems15
Creating Indexes through iNav
IBM Training
2016 IBM CorporationIBM Power Systems16
Derived Keys
Creation of indexes with derived keys via SQL supported
CREATE INDEX ORDERPRIORITYUPPER ON T1(UPPER(ORDERPRIORITY) AS UORDERPRIORITY ASC);
CREATE ENCODED VECTOR INDEX YEARQTR ON T1(YEAR(ORDERDATE) AS ORDYEAR ASC, QUARTER(ORDERDATE)AS ORDQTR ASC);
CREATE INDEX TOTALEXTENDEDPRICE ON T1(QUANTITY * EXTENDEDPRICE AS TOTEXTPRICE ASC);
NEUG
C 2016
NEUG
C 2016
8
IBM Training
2016 IBM CorporationIBM Power Systems17
When to create a Derived Index?
Derived indexes can be useful for Case insensitive searches
Data extracted from a column e.g. SUBSTR, YEAR, MONTH
Derive Common Grouping columns e.g. YEAR(ORDERDATE)
Results of operations e.g. COL1+COL2 , QTY * COST
Might be useful to allow index only access in more cases
Including the INCLUDE support for EVIs
Could replace some logical files with SQL indexes for use by RLA native, high level language programs
Modernize those objects
Big logical page size (8K v 64K)
Query Optimization(using indexes)
NEUG
C 2016
NEUG
C 2016
9
IBM Training
2016 IBM CorporationIBM Power Systems19
Cost based optimization dictates that the fastest access method for a given table will vary based upon selectivity of the query
Number of rows searched / accessedFew Many
ResponseTime Table Scan
Low
High
Probe
Clustered,Skip Seq
Data Access Methods
IBM Training
2016 IBM CorporationIBM Power Systems20
Optimizing indexes will generally follow this (simplified) strategy:
Gather list of indexes for statistics and costing implementation methodsConsider the indexes based on how the index can be used
Local selectionJoiningGroupingOrderingIndex only access
One index may be useful for statistics, and another useful for implementation
Strategy for Query Optimization
If an index is missing that could be useful for the query, the optimizer will often
Advise it! NEUGC 2
016
NEUG
C 2016
10
IBM Training
2016 IBM CorporationIBM Power Systems21
Advised IndexCatalog
SYSIXADV
SQE Plan Cache
Query Optimization
SQL requestDetailed
DB Monitor Data
VisualExplain
Query Optimization Feedback
SQE Plan Cache
Snapshots
IBM Training
2016 IBM CorporationIBM Power Systems22
