Upload
trivadis
View
127
Download
3
Embed Size (px)
DESCRIPTION
Dani Schnider, Principal Consultant bei der Trivadis AG, an der Oracle Open World 2011 in San Francisco
Citation preview
2014 © Trivadis
BASEL BERN BRUGG LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN
2014 © Trivadis
MORE THAN JUST SOME BITS…Data Warehousing and Bitmap Indexes
Dani Schnider, Trivadis AG
Oracle Open World 2011,San Francisco
02.10.2011Data Warehouseing & Bitmap Indexes
1
2014 © Trivadis
Fehlertolerante Ladeprozesse2
Dani Schnider Principal Consultant und
DWH/BI Lead Architectbei Trivadis in Zürich
Teacher for courses about Data Warehousing, SQL Optimization and Oracle Warehouse Builder
Co-author of book «Data Warehousing mit Oracle»
18. April 2012
2014 © Trivadis
11 Trivadis Niederlassungen mitüber 600 Mitarbeitenden
200 Service Level Agreements
Mehr als 4'000 Trainingsteilnehmer
Forschungs- und Entwicklungs-budget: CHF 5.0 / EUR 4 Mio.
Finanziell unabhängig undnachhaltig profitabel
Erfahrung aus mehr als 1'900 Projekten pro Jahr bei über 800 Kunden
Stand 12/2012
Hamburg
Düsseldorf
Frankfurt
FreiburgMünchen
Wien
Basel
ZürichBernLausanne
3
Stuttgart
Mit über 600 IT- und Fachexperten bei Ihnen vor Ort
02.10.2011Data Warehouseing & Bitmap Indexes
3
2014 © Trivadis
02.10.2011Data Warehouseing & Bitmap Indexes
4
SOME BITS ABOUT
BITMAP INDEXES
AND THEIR USAGE IN
DATA WAREHOUSES
2014 © Trivadis
Bitmap Indexes – Concept
Oracle® Database Concepts 11g Release 2 (11.2)
02.10.2011Data Warehouseing & Bitmap Indexes
5
2014 © Trivadis
Bitmap Indexes – Concept
02.10.2011Data Warehouseing & Bitmap Indexes
6
2014 © Trivadis
Bitmap Indexes – Example
02.10.2011Data Warehouseing & Bitmap Indexes
7
2014 © Trivadis
Comparison B-tree Index / Bitmap Index
02.10.2011Data Warehouseing & Bitmap Indexes
8
2014 © Trivadis
Comparison B-tree Index / Bitmap Index
02.10.2011Data Warehouseing & Bitmap Indexes
9
2014 © Trivadis
Concatenated Bitmap Indexes
Bitmap Indexes on two or more columns are possible, but not useful
Why?
Concatenated bitmap index can be used only when all columns are filtered
Two or more separate bitmap indexes are more flexible and can be used for all combinations
02.10.2011Data Warehouseing & Bitmap Indexes
10
2014 © Trivadis
Index Strategy on Star Schema
Dimension Tables: Unique b-tree index on primary key column Bitmap indexes on additional columns (optional)
Fact Tables: Bitmap indexes on all dimension key (foreign key) columns Bitmap join indexes on often used filter columns (optional) Usually no primary key on fact tables
02.10.2011Data Warehouseing & Bitmap Indexes
11
2014 © Trivadis
Index Strategy on Star Schema
02.10.2011Data Warehouseing & Bitmap Indexes
12
2014 © Trivadis
Query Optimization on Star Schemas
Typical Queries: Filter criteria on (multiple) dimension
tables Facts are selected by join with
dimensions
Problem: Tables with restrictions should be
read first Only two tables can be joined at a
time No relationships between dimension
tables
02.10.2011Data Warehouseing & Bitmap Indexes
13
2014 © Trivadis
Star Transformation – Concept
02.10.2011Data Warehouseing & Bitmap Indexes
14
2014 © Trivadis
Star Transformation – Example
What is the total revenue for product “Shorts” for all customers in Germany, grouped by male and female customers?
02.10.2011Data Warehouseing & Bitmap Indexes
15
2014 © Trivadis
Star Transformation – Example
02.10.2011Data Warehouseing & Bitmap Indexes
16
2014 © Trivadis
Bitmap Join Index – Concept
Bitmap index is defined on fact table
Indexed column is part of dimension table
Possible for star and snowflake schemas
02.10.2011Data Warehouseing & Bitmap Indexes
17
2014 © Trivadis
Bitmap Join Index – Example
Same example with Bitmap Join Indexes
02.10.2011Data Warehouseing & Bitmap Indexes
18
Demo
2014 © Trivadis
Bitmap Indexes and ETL
Bitmap indexes are efficient for set-based ETL Index maintenance is deferred until end of each DML operation
Efficient index maintenance for Parallel DML
Avoid “hand-made” parallel ETL with multiple sessions Locking behavior of bitmap indexes
For large data loads: Rebuild indexes after ETL operation Set all indexes to UNUSABLE Load data into table Rebuild all indexes
02.10.2011Data Warehouseing & Bitmap Indexes
19
2014 © Trivadis
Bitmap Indexes and Partitioning
Bitmap Indexes must always be LOCAL
Local indexes are recommended in Data Warehouse Only purpose of global indexes in DWH: unique indexes without partition key Restriction of local bitmap indexes is not a problem
02.10.2011Data Warehouseing & Bitmap Indexes
20
2014 © Trivadis
Bitmap Indexes and Compression
Bitmap Indexes are always compressed Zeros are compressed, Ones are not compressed Bitmap contains number of zeros to next 1 bit
Bitmap Indexes need less disk space than B-tree Indexes Even for columns with high number of distinct keys
Detail description see «Bitmap Index Internals», © Julian Dyke http://www.juliandyke.com/Presentations/BitmapIndexInternals.ppt
02.10.2011Data Warehouseing & Bitmap Indexes
21
2014 © Trivadis
Bitmap Indexes and Compression
Number of distinct keys has impact on size of index Bitmap index is usually smaller than b-tree index For high number of keys, size of bitmap index increases Size of b-tree index is more or less constant Sort order affects size of bitmap index, but not of b-tree index
02.10.2011Data Warehouseing & Bitmap Indexes
22
Demo
10 100 1000 10000 1000000
50
100
150
200
250
300
350
Bitmap (scattered) Bitmap (sorted) B-tree index
2014 © Trivadis
Bitmap Indexes and Compression
Enabling table compression When a table is enabled for compression, all bitmap indexes must be disabled
Reason: Hakan factor of table Maximum number of rows that can be stored in a table block Hakan factor changes when table is enabled for compression (only first time)02.10.2011
Data Warehouseing & Bitmap Indexes23
Demo
2014 © Trivadis
Bitmap Indexes – Conclusion
Bitmap Indexes are a powerful Data Warehouse feature Can be combined in ad hoc queries Star Transformation Useful for selective and non-selective columns
Bitmap Indexes are useful for large databases Efficient for set-based ETL and Parallel DML Local bitmap indexes on partitioned tables Compression of bitmap indexes
There are almost no reasons to use b-tree indexes in a Data Warehouse B-tree indexes are mainly used for Primary Key or Unique Key constraints
02.10.2011Data Warehouseing & Bitmap Indexes
24
2014 © Trivadis
References
Oracle® Database Data Warehousing Guide 11g Release 2 (11.2) Oracle Corporation, http://www.oracle.com/pls/db112/
Oracle® Database Concepts 11g Release 2 (11.2) Oracle Corporation, http://www.oracle.com/pls/db112/
Christian Antognini: Troubleshooting Oracle Performance Apress, http://www.apress.com/9781590599174
Jonathan Lewis: Cost-Based Oracle Fundamentals Apress, http://www.apress.com/9781590596364
Julian Dyke: Bitmap Index Internals http://www.juliandyke.com/Presentations/BitmapIndexInternals.ppt
02.10.2011Data Warehouseing & Bitmap Indexes
25
2014 © Trivadis
BASEL BERN BRUGG LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN
2014 © Trivadis
THANK YOU.Trivadis AG
Dani Schnider
Europa-Strasse 5CH-8152 Glattbrugg/ZürichSwitzerland
Tel. +41 44 808 70 20Fax +41 44 808 70 21
02.10.2011Data Warehouseing & Bitmap Indexes
26