2004 Presentation 549

8/3/2019 2004 Presentation 549

1/42

1

www.SageLogix.Com

International Oracle Users GroupLive 2004

Technical Session #549:

Understanding Indexes

Tim Gorman

Principal - SageLogix, Inc.

Email: [email protected]


2/42

2

B*Tree index architecture

Issues with B*Tree indexes in Oracle

Real issues

Sparsely-populated indexes

Contention on INSERT

Uneven data distribution

Low data cardinality (a.k.a. selectivity)

Imagined or mythical issues

Indexes needing rebalancing

???

Some interesting options for indexing

Function-based, descending, compression

Agenda


3/42

3


Oracle implements balanced B*Treeindexesas its primary indexing method

Tree structure consisting of root, branch, and leafnodes

each nodeis one database block in Oracle

rootnode is the top-level branchnode

starting point for searches into the index

branchnodes are connectors point to other branchnodes or to leafnodes

leafnodes

contain data values and pointers to rows


4/42

4


root

branch branch branch branch

leaf leaf leaf leafleaf leaf leaf

Branchentries contain:

Max data value piece DBA of next level branch or leafLeafentries contain:

Data value

ROWID of table row

Index: PK_X

Table: X


5/42

5


Index read I/O is sequential in nature Oracle wait-event for single-block I/O requests used during

indexed access

db file sequential read

Used with RANGE, UNIQUE, and FULL scans

FAST FULL index-scans are an exception

Uses sequential multiblock I/O similar to FULL table-scans

Conventional wisdomthat tables and indexes must be

separated to different tablespaces to enable some form ofparallel I/O is a myth

Think of how a treasure hunt is performed.


6/42

6


When indexes are built on populatedtables using CREATE INDEX or ALTERINDEX xxx REBUILD

1. Table is scanned and column values aresorted

2. Leaf blocks of index are populated first

3. Then supporting branchlevels are built to

support those leaves Allblocks are filled up to PCTFREE

threshold


7/427


When indexes are built transactionallyusing conventional-path INSERT orUPDATE statements

How the indexes grow is dependent upontwo possible design decisions

Optimize either for:

random data values

Or:

sequentially ascending data values


8/428


If optimizing for randomdata values As blocks fill, split them in half, move upper half to

new block, leave lower half behind

Leaves behind half-full blocks

Allows back-fillingas lower data values occur

If optimizing for sequentialdata values

As blocks fill, simply overflow into a new block

Leaves behind full blocks

No need to worry about back-fillingas lowerdata values could never occur


9/429


Overflow: Grows like this

Split: Grows like this

Splitting anticipates back-filling in a sorteddatastructure Therefore uses space more efficiently when

optimizingrandom data values

Overflowing uses space more efficiently in a heap(non-sorted) data structure Also uses space more efficiently when optimizingsequentialdata values


10/4210


If the last index entry inserted is the highestdata value in the block

Then data will overflow into the next block on thesame level

If the last index entry inserted is not thehighest data value in the block

Then the current block will split, with the lower setof values remaining in place and the higher set ofvalues moving to the next block on the same level


11/4211


If the branch block above cannotaccommodate another child

Then this algorithm is applied recursively first

If this algorithm iterates recursively all theway back to the root block, and the rootbecomes full

Then the root splits/overflows to generate a newbranch level (BLEVEL)

Root block stays in place, generates two new blocks at anew 1st BLEVEL below


12/4212


Balanced B*Tree indexes grow balanced bydesign, automatically

New levels are added to the tree structure above,dynamically, as needed

Indexes do not shrinkautomatically whendeletions occur

Instead, empty index entries are left in place

For possible reuse by newly inserted data

ROWID (pointer to data) portion is set to NULL

Deletions may cause an index to become sparseand thus less efficient over time


13/4213

Issues with B*Tree indexes

B*Tree indexes optimize:

Both the randomand sequentialdata values!

Even distribution of distinct data values

High data cardinality or selectivity of values

As a result, issues with B*Tree indexes are:

Sparseness

Resulting from data deletions

Block contention when inserting sequential data Uneven distribution of data values

Popularand unpopulardata values

Low data cardinality


14/4214


Indexes never become unbalancedover time

If so, then why does performance sometimesdeteriorateover time?

Instead, indexes can become sparselypopulateddue to:

Deletions of row data

Unfortunate patterns of inserted random data

Sparsenesssimply means that more I/O isrequired to perform the same work


15/4215


How is this condition detected? ANALYZE INDEX VALIDATE STRUCTURE

command

Populate session-private view named

INDEX_STATS with only one row Column PCT_USED is the average percentage of

space utilized in the blocks belonging to the index

Derived from the ratio of the value of the

columns USED_SPACE and BTREE_SPACE Expect PCT_USED to be 90 by default

Lesser values may indicate sparsenessdeveloping

Best to watch values over time


16/4216


How is this condition detected? If PCT_USED is much less than 90

Non-zero values in the column

DEL_LF_ROWSCause of sparseness is probably row

deletions, if the value in the columnDEL_LF_ROWS is a large percentage of

value in column LF_ROWS


17/4217


Rebuild or coalesce are two possiblesolutions for index sparseness

Dont guess!!!

Make certain using ANALYZE INDEX

VALIDATE STRUCTURE

Be aware that the ANALYZE command locksthe index, however

To rebuild an index: Use ALTER INDEX REBUILD

To coalesce space within an index:

Use ALTER INDEX COALESCE


18/4218


ALTER INDEX REBUILD is the most commonly-used solution for sparsely-populated indexes

Unlike CREATE INDEX, uses the existing index as thesource, which is faster because:

Index is usually smaller than the table (less I/O) Index is already sorted (some sorting still needed)

Other features of ALTER INDEX REBUILD include:

Parallel execution (throw more resources at task)

Direct-path operations (no rollback/undo generated)

NOLOGGING (no redo generated)

COMPUTE STATISTICS (better CBO stats cheaply)

ONLINE (allows full usage of index during task)


19/4219


ALTER INDEX COALESCE is anotheralternative to rebuilding

Merges unused space within indexes to free blocksfor reuse

Slower than ALTER INDEX REBUILD

because it is a transactionaloperation, not abulkoperation

No direct-path, parallel, nologging, etc But COALESCE is implicitly an ONLINE

operation


20/4220


Multiple concurrently-executing INSERTs intoan index on column(s) with sequentialdatavalues can also bottle-neck on buffer busywaits (a.k.a. block-level contention)

Pseudo-randomizing such data with REVERSEkey indexes relieves this performance problem

Instead of many processes attempting to insertindex entries into the right-most leaf block

Insertions are evenly scattered over all of theavailable leaf blocks


21/4221


[ CREATE | ALTER ] INDEX REVERSE Pseudo-randomizes non-random data

By flipping or reversing the physical orderof the data value during storage

Causes sequentialdata values to become morerandomby simply reversing data

123456 becomes 654321

123457 becomes 754321, etc...

Index data values are transparentlyconverted(flipped) and unconverted (unflipped) uponinsert and retrieval, respectively


22/4222


Adverse impacts of using REVERSE indexes: only equivalence operations will use the index

=, !=, , IN, and NOT IN

range-scans will not use the index

>, >=,


23/42

23

Uneven distribution of data values

Basic rule of thumb for using indexes: Use indexes when selecting unpopulardata

Otherwise, use FULL table-scan

Or a different index? Or partitioning? Or

By default, the Oracle cost-based optimizer(CBO) assumes even data distribution

Only LOVAL, HIVAL, and DISTINCT_KEYSgathered during ANALYZE or DBMS_STATS

Real-life often invalidates this assumption

Selectivelygathering column-level statisticsprovides the CBO with information on data valueswhich are popularand which are unpopular


24/42

24


There are two commands for gathering CBOstatistics

ANALYZE TABLE FOR [ ALL | INDEXED ]COLUMNS

DBMS_STATS.GATHER_COLUMN_STATS Populates data dictionary views

DBA_TAB_HISTOGRAMS andDBA_PART_HISTOGRAMS with rows

representing buckets of data values All buckets assumed to have the same number of

rows

Each bucket is defined by its highest value

By default, only one row populated


25/42

25

Please dont gather column-level statisticsunless the problem is proven

Gathering column-level statistics is not a goodchoice as a default operation

Perform SQL tuning with SQL Trace/TKPROF orSTATSPACK to provide proof of a problem

If the CBO fails to choose an index

it could be because it has detected low cardinality

on average If the CBO chooses an index

it could perform poorly if the data value is overlypopular



26/42

26

Illustrating the importance of data

SQL> create table t1

2 ( c1 varchar2(30),

3 c2 number,

4 c3 number

5 ) tablespace tools;

Table created.

SQL> begin

2 for i in 1..100000 loop

3 insert into t1

4 values(to_char(mod(i,187)), i, mod(i,187));5 end loop;

6 end;

7 /

PL/SQL procedure successfully completed.


27/42

27


SQL> create index i1 on t1(c1) tablespace tools;

Index created.

SQL> analyze table t1 compute statistics;

Table analyzed.

SQL> set autotrace on

SQL> select c1 from t1 where c1 = '10000';

no rows selected

Execution Plan

-----------------------------------------------------0 SELECT STATEMENT Optimizer=CHOOSE (Cost=37 Card=535

Bytes=1605)

1 0 TABLE ACCESS (FULL) OF 'T1' (Cost=37 Card=535 Bytes=1605)


28/42

28


SQL> select num_rows, blocks

2 from user_tables3 where table_name = 'T1';

NUM_ROWS BLOCKS

------------- ---------

100,000 232

SQL> select num_rows, distinct_keys,

2 avg_leaf_blocks_per_key, avg_data_blocks_per_key

3 from user_indexes where index_name = 'I1';

Avg Leaf Avg Data

Distinct Blocks Blocks

Nbr Rows Key Per Key Per Key

-------- -------- ---------- ---------

100,000 187 1 231


29/42

29


SQL> truncate table t1

Table truncated.

SQL> begin

2 for i in 1..100000 loop3 insert into t1

4 values(to_char(round(i/187,0)), i,

5 round(i/187,0));

6 end loop;

7 end;

8 /

PL/SQL procedure successfully completed.


30/42

30


SQL> create index i1 on t1(c1) tablespace tools;

Index created.

SQL> analyze table t1 compute statistics;

Table analyzed.

SQL> set autotrace on

SQL> select c1 from t1 where c1 = '10000';

no rows selected

Execution Plan

-----------------------------------------------------0 SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=187 Bytes=561)

1 0 TABLE ACCESS (BY INDEX ROWID) OF 'T1' (Cost=2 Card=187Bytes=561)

2 1 INDEX (RANGE SCAN) OF 'I1' (NON-UNIQUE) (Cost=1 Card=187)


31/42

31

SQL> select num_rows, blocks

2 from user_tables3 where table_name = 'T1';

NUM_ROWS BLOCKS

------------- ---------

100000 242

SQL> select num_rows, distinct_keys,

2 avg_leaf_blocks_per_key, avg_data_blocks_per_key

3 from user_indexes where index_name = 'I1';

Avg Leaf Avg Data

Distinct Blocks Blocks

Nbr Rows Key Per Key Per Key

-------- -------- ---------- ---------

100000 536 1 1



32/42

32


B*Tree indexes work best with highcardinality

Think of the example of a telephone book

Would you use the standard indexing method

to find all occurances of a name comprising25% of the book?

Or, would it be better to just scan through thebook?


33/42

33


Bitmap indexes are extremely compact That is their primary advantage

No real magic involved

Less I/O is faster, plain and simple

Still, no matter how compact they are a FULL table scan is still likely faster than a

single bitmap index on low cardinality data

However, merging two or more bitmap indexes

brings out the real power of the mechanism

Scanning of merged bitmaps using fast low-level bitmasking operations

Sifts through huge numbers of rows quickly


34/42

34


Bitmap indexes are designed for quickly scanning low

cardinalitydata each database block is comprised of two bitmap segments

each bitmap segmentis comprised of a bitmap, a ROWID list,and a list of distinct data values

ROWID listis a forward-compressedlist of ROWIDs, positioned according tobits in bitmap

Bitmapis organized into columnsfordistinct data values and rowsof bitsrepresenting rows

More distinct data values means lessroom for rows within each data value

ROWID list

Bitmap

Distinct values list

ROWID list

Bitmap

Distinct values list


35/42

35


Bitmap indexes extremely slow during DML INSERT, UPDATE, and DELETE

For each change, entries within the bitmap segmentsmust be: Expanded/decoded

Manipulated Re-encoded

For multiple concurrently-executing DML changes Slower individual changes cause queueing

Only two bitmap segments per block increases contention

Because of this, bitmap indexes are most feasible onpartitioned tables in non-transactional applications Data changes should be performed in bulk using some

variation on the technique of EXCHANGE PARTITION


36/42

36

Function-based indexes

Indexes can be based on functions andexpressions

create index xxx_ix1 on xxx(upper(c1));

create index xxx_ix2 on

xxx(upper(c1)||yadda_yadda(c2));


37/42

37


User-defined functions usable along with standardbuilt-in functions

Restrictions for using user-definedfunctions

But functions and expressions must be deterministic

repeatable: given the same inputs, the function orexpression must always provide the same results

user-defined functions cannot accesstables or package variables

PRAGMA RESTRICT_REFERENCES(, RNDS, WNDS,RNPS, WNPS)

CREATE FUNCTION DETERMINISTIC


38/42

38

Function-based indexes must have statisticsgathered before they can be used

System permissions

[ GLOBAL ] QUERY REWRITE needed to CREATE or ALTER

REBUILD

Parameter QUERY_REWRITE_ENABLED = TRUE to

use



39/42

39

Descending indexes

Implemented as a form of function-basedindexes

CREATE INDEX DESC

data values in index leaf blocks are sorted in

descending order must have statistics before it will be utilized

not usable with rule-based optimizer

system permission QUERY REWRITE or GLOBAL

QUERY REWRITE needed to CREATE or ALTER REBUILD

parameter QUERY_REWRITE_ENABLED notnecessary to use


40/42

40

Index compression

Since Oracle8i, compression has been available onB*Tree indexes:

Reduces repeated storage of prefix columndata values

Index compression

[ COMPRESS [ #-prefix-cols ] | NOCOMPRESS ]

UNIQUE: default #-prefix-colsis (#-cols)-1

NONUNIQUE: default #-prefix-colsis #-cols

Same syntax for IOT in USING clause as for indexcompression

DML is supported on compressed indexes, but it becomesmuch slower

Index compression is best used for read-mostlyor read-onlysituations

Mixing of compressed and uncompressed indexpartitions is possible


41/42

41

Solutions for B*Tree indexes

Dont assume that there is anything wrong with a

B*Tree index Prove it first!!!

To address sparse index structures

ALTER INDEX [ REBUILD | COALESCE ]

To address block contention on INSERTs ALTER INDEX REVERSE

To address uneven data distribution

Gather column-level statistics or histograms

To address low cardinality Use bitmap indexes or no indexes at all

Additional groovystuff

Function-based, descending, and compressed indexes


42/42

Technical Session #549

Slides, paper, and scripts downloadable from

http://www.SageLogix.com

and

http://www.EvDBT.com/papers.htm

Email: [email protected]
http://www.sagelogix.com/http://www.evdbt.com/papers.htmhttp://www.evdbt.com/papers.htmhttp://www.sagelogix.com/

Documents

2004 Presentation 549