© Copyright 2006 1 - 1 Oracle and Essbase - Building a Data Warehouse

© Copyright 2006 www.bravesoft.com 1 - 1

Oracle and Essbase - Building a Data Warehouse


Agenda

• Oracle’s Fusion

• Essbase

• OLAP, ROLAP, run laps

• Dimensional Modeling

• The Integration Console

• Metadata

• Creating a Cube

• Hierarchies

• OLAP Metaoutline

• Dense and Sparse

• Creating the Essbase Application

• The Administrative Console

• OBIEE Integration

• What is OBIEE?

• An OBIEE Dashboard


Oracle’s Fusion

• Oracle has acquired a number of companies • A link between acquired technologies often creates

synergies• This is especially true in the Business Intelligence (BI)

space• Oracle’s Fusion is approach to middleware that creates

the links and synergies• Example: Essbase now includes the Integration Console

– A full GUI development Environment– Data from Relational Sources to OLAP


Relational and OLAP

• The Relational Model is very powerful way to abstract and solve business problems– Objects in the world of business: invoices, orders and

so forth can be captured using relational tables– The business object could be understood as a table,

and the business rules as the relationship– Very powerful in supporting transactions and single

entries: Online Transaction Processing (OLTP)– Reporting, however, had it’s problems

• Executives want to analyze summaries• The enterprise view


OLAP

• Online Analytical Processing– New approach to thinking

• What are the ways we want to slice and dice (dimensions)

• What the measures that are important to the business (facts)

• Also can use the subject based approach to analysis

• This talk is based on the dimensional modeling approach

– Use transaction or operational data and create a data warehouse


Definition

According to Wikipedia

A data warehouse is a repository of an organization's electronically stored data.

Data warehouses are designed to facilitate reporting and analysis [1]

1. Inmon, W.H. Tech Topic: What is a Data Warehouse? Prism Solutions. Volume 1. 1995

http://en.wikipedia.org/wiki/Repository


Two LeadersBill Inmon

Bill Inmon is universally recognized as the "father of the data warehouse." He has over 26 years of database technology management experience and data warehouse design expertise, and has published 36 books and more than 350 articles in major computer journals. His books have been translated into nine languages. He is known globally for his seminars on developing data warehouses and has been a keynote speaker for every major computing association. Before founding Pine Cone Systems, Bill was a co-founder of Prism Solutions, Inc.

Ralph Kimball

Ralph Kimball was co-inventor of the Xerox Star workstation, the first commercial product to use mice, icons, and windows. He was vice president of applications at Metaphor Computer Systems, and founder and CEO of Red Brick Systems. He has a Ph.D. from Stanford in electrical engineering, specializing in man-machine systems. Ralph is a leading proponent of the dimensional approach to designing large data warehouses. He currently teaches data warehousing design skills to IT groups, and helps selected clients with specific data warehouse designs. Ralph is a columnist for Intelligent Enterprise magazine and has a relationship with Sagent Technology, Inc., a data warehouse tool vendor. His book "The Data Warehouse

Toolkit" is widely recognized as the seminal work on the subject.


Kimball’s Definition

A data warehouse is "a copy of transaction data specifically

structured for query and analysis".


Kimball’s Rules

• The data warehouse must make an organization’s information easily accessible

• The data warehouse must present the organization’s information consistently

• The data warehouse must be adaptive and resilient to change

• The data warehouse must be a secure bastion that protects our information

• The data warehouse must serve as a foundation for improved decision making

• The business community must accept the data warehouse if it is to be deemed successful


Differences?

• Kimball and Inmon are very close in their definitions of a data warehouse and the functions it provides. So how do they differ?

– Inmon believes a data warehouse should be based on a relational database using 3NF.

– Kimball designs a data warehouse with intentional de-normalization with ease of use for the business person as a key criteria


Kimball’s Four Step Method

1. Identify the business process

2. Identify the grain

3. Choose dimensions

4. Identify numeric facts


Identify the Business Process

• Natural business process• Supported by a data gathering activity• NOT a department or division – it is a process

– Multiple departments may contribute to a process– Forces use of consistent vocabulary and labels– Departmental approach results in duplicate data

flows in a single process

• “Economical” Data Usage


Identify the Grain

• This is the level of detail that will be associated measurements in the model

• This is an extremely important step• Will impact the selection of dimensions• You may find in analysis for steps 3 and 4 that the

original selection was wrong. Don’t hesitate to go back to step 2 and start over as soon as you find the problem.

Examples?


Identify the Grain

Examples!


Dimension Tables

• Dimensions are the “by” words in business intelligence– Count Customers by Region– Order Sales by Product– Shipments by Quarter

• They are the entry point for business intelligence• They are textual

– Words and terms business users understand• They are discrete• Often represent hierarchies• Typically highly de-normalized


Dimensions• Nearly every BI application includes some type of time or date

dimension– Include attributes in the date dimension for all units of time

measurements– Don’t break time into separate measures, month dimension, day

dimension, etc.• Always include a dimension total attribute• Don’t be afraid to include all of the attributes of a dimension

– 50 attributes is not uncommon– Think about the drill down

• Use surrogate keys for dimension– Smart keys are never smart enough– Natural Keys change

• Resist “Snowflaking”• Be careful of too many dimensions, use dimension attributes in a

de-normalized fashion


Fact Tables

• A row in a fact table corresponds to a measurement– A measurement is a row in a fact table– All measurements must have the same grain

• The most useful facts are numeric and additive• Generally, we do not enter zeros when something has

not happened• Three types are

– Transaction– Periodic Snapshot– Accumulating Snapshot

• Percentages and ratios are non-additive– Store the numerator and denominator– Remember to calculate the ratio of the sums, not the sum of the

ratio

KPFs

KPFs

KPFs

KPFs


Star Schema

• The result is refereed to as a star schema, as the dimensions arranged around the fact table resemble a star!

From Wikipedia


Cube

• Star Schema is also called a Cube or a Multidimensional Cube. The fundamental structure for data in a multidimensional (OLAP) system.

• A cube contains dimensions, hierarchies, levels, and measures. Each individual point in a cube is referred to as a cell.

Source: http://www.sdgcomputing.com/glossary.htm


Degenerate Dimensions• Control numbers which are never added,

only used to count or group are candidates for degenerate dimensions

• These are dimensions, but the descriptive information has been pulled into other dimensions– Consider an order

• Customer• Ship to• Order date

– All pulled into other dimensions, only the order number remains

• In fact table, identified with prefix dd_• Degenerate Dimensions are often the grain

of the fact table


What is OLAP

• An approach to business intelligence very similar to dimensional modeling

• Facts and Dimensions are identified

• Then, multi dimensional cubes are built– Each fact is associated with all appropriate

dimensions– This where term multi-dimensional arises

• Data is gathered and aggregated from operational systems


OLAP Database Architectures

• There are several options for the physical structure of an OLAP database

• The key is supporting data in cubes for user interactions


Simplest OLAP example

May 2008 June 2008 July 2008 August 2008

Online Sales 23456234 5235 45 2345345

Internet 21110611 3403 18 2110811

Text Message 234623 1832 27 234535

Offline Sales 2345234 2345234 3434 2345234

Catalog 2110711 1524402 1374 2110711

In Store 234523 820832 2060 234523

Introduce Hierarchies

May 2008 June 2008 July 2008 August 2008

Online Sales 23456234 5235 45 2345345

Offline Sales 2345234 2345234 3434 2345234


Product Group A

Apparel Type 1

SKU 3490857

SKU 2389057

Apparel Type 2

SKU 934785

SKU 90348

SKU 348907

Online Sales

Internet

Text Message

Offline Sales

Catalog

In Store

More Dimensions And Hierarchies

Year 1

Quarter 1

Month 1

Month 2

Month 3

Quarter 2

Month 4


Logical View Fact Table

Sales Cube

Date Key

Product Key

Channel Key

Sales Fact 1

Sales Fact 2

• We add dimensions as appropriate• Facts are sliced and diced as need• Each entry is a cell• Not every cell will have a fact – eg.

Sunday sales in a B2B environment in cube with a daily grain

• The percentage of populated cubes is called the sparsely or density of the cube

People with an accounting background often think of cubes as

big pivot tables


Query Language for an OLAP database

• Unlike relational databases, which had SQL there is no equivalent in the OLAP world

• The first real standard API was OLE DB for OLAP specification from Microsoft – Appeared in 1997– The MDX query language– Adopted by most server and client OLAP vendors

• In 2001 Microsoft and Hyperion announced the XML for Analysis specification,

• XML for Analysis also uses MDX


The Exploding Database

• Exploding multidimensional databases are a common– Happens when a Multidimensional database gets so

large query performance degrades– The amount of actual data is small, yet the database

is huge

• How does this happen?– In part, because people are unfamiliar with multidimensional

databases, so the phenomenon is widely misunderstood– multidimensional geometry can be counterintuitive


Some Facts You Need to Understand

• Multidimensional databases usually take data from other sources, such as legacy systems, relational databases or desktop tools, such as spreadsheets.

• A few slides back we saw some of the common approaches to building a multidimensional database

– ROLAP -- the data is still physically stored in an RDBMS, usually in some form of star or snowflake schema

– MDB (multidimensional database), the data is physically stored in a different file structure, optimized for multidimensional processing and fast retrieval.

– Hybrid OLAP products, which allow both direct access to relational data for multidimensional processing, as well as having their own optimized multidimensional disk storage for aggregates and pre-calculated results.

Data is stored in an MDB will normally take much less space than it did in the source system, even if it is not summarized

• Multidimensional storage takes between a tenth and a half of the space taken to store exactly the same information in a relational database.

– This is mainly because the keys, indexes and dimensional structures are either not required at all or take far less space.

– Also, the sparsity is often better suppressed and the data may even be compressed


Then, your database gets huge…

What happens next?• OLAP applications are intended for interactive use, people expect to get a

fast response to queries — ideally, not more than a few seconds– Queries get slower and more complicated if a significant amount of

calculations have to be done– hierarchical consolidations– calculations of variances– analyzing trends– deriving computed measures

• In order to get a fast response, large multidimensional applications need to pre-calculate some of the information for analysis.

This might include high level consolidations• In MDBs, the storage of pre-calculated data is usually automatic and

transparent• In ROLAPs, summary tables are normally used


Queries die and never come back

The details of the implementation differ, there is no difference in principle between MOLAP and ROLAP products in this.

• Multidimensional cross relationships exist in all OLAP applications • The data is usually very sparse -- the vast majority of possible cells,

combinations of dimension members, contain no data.• Thinly distributed data values may have hundreds of computed dependent

cells• hierarchies in each dimension. • The ‘computed space’ is much denser than the original data

While it is hard to predict the exact value in advance, adding an extra dimension to a multidimensional object with no increase in the amount of input data will at least double the size of the fully computed database.


What to Do?

• Avoid fully pre-calculating any multidimensional object with more than five sparse dimensions– Ratios– Variances– simple time series conversions– rarely viewed consolidations may all be computed on-the-fly

• Many products classify data using a concept variously called attributes, properties or characteristics

• These are used for on-the-fly groupings, selections and aggregations, without requiring the full overhead of a dimension

• Reduce the sparsity of individual data objects by good application design

• Use a multicube rather than a hypercube approach• so each object has the minimum number of necessary dimensions


Essbase: An OLAP Product

• Essbase is a multidimensional database management system (MDBMS) – A platform to build analytic applications. Essbase, – Its derives from "Extended Spread Sheet dataBASE“

• Originally developed by Arbor Software• Arbor merged with Hyperion Software in 1998• Hyperion Solutions Corporation was acquired by Oracle

Corporation in 2007


Multidimensional

• Essbase is a general-purpose multidimensional database• Developed to address the scalability issues associated with

spreadsheets• Consider financial data in spreadsheet format:

Jan Feb Mar Apr

Sales 232,234 342,342 1,234,345 34,345

Expenses 131,345 193618 691,233 20,607

Profit 100,889 148,724 543,112 13,738

Units 343 505 1,822 50


Looks Like a Cube Already…

• Ask a question: How does it break down by region? Now we have multiple spreadsheets


How about a customer breakdown?


Spreadsheet Nightmare, or…• Apply our knowledge of dimensional modeling

– What is the business process?• Sales

– What is the grain?• Monthly

– What are the dimensions?• Time• Region• Customer• What else?!?!?

– What are the facts?• Sales Amounts• Expenses• Units Sold

Profit is calculated


Create a Hierarchy • Essbase includes an administrative tool which allows the analyst to define the

dimensions and their hierarchies• There are two "storage options“• Block Storage Option (Essbase BSO) or Essbase Analytics

– Dense blocks allocate space for every potential cell in that block– Sparse dimensions only create blocks when required.– This implementation is hidden from front-end tools– Block storage effectively minimizes storage requirements without impacting

retrieval time, but it is limited by its treatment of aggregate data in large applications

• The second storage option is Aggregate Storage Option (Essbase ASO) or Enterprise Analytics– Essbase ASO does not store any aggregate values, but instead calculates

them on demand.– Where runtime generation of these values is inconvenient, the database

materializes one or more aggregate "views“– One aggregate level from each dimension– This process can be partially automated


And the Front End is familiar

• The front end tools for Essbase are familiar to users

– An Excel plug-in is the basic tool– Users pull in data from a cube and Excel it as

rolled-up totals on dimension hierarchies– Uses can then drill down or across to

investigate the cube• SmartView is a new product that allows user to employ

any Microsoft Office product as their front end


Dashboards and Essbase

• Oracle’s acquisition of Hyperion and Oracle’s Fusion efforts means Essbase can be integrated with many of Oracle’s other BI tools

– A star schema cube in an Oracle database can be a data source for Essbase

– BI Publisher reports can connect to an Essbase database

– An OBIEE dashboard can use an Essbase cube as its data source


Migration

• So, what about data stored in a Star Schema in a relational data base

– How do we move it to an Essbase database?

http://www.johnkittelsonwoodcarvings.com/images/conestoga_wagon_L.jpg


The Data Warehouse


Configure and Use the Integration Services Console

• Oracle has built a tool which allows you to map objects in a relational database to an Essbase cube

• It is included as a component in Essbase System – 9.

• Confirm that your Integration Server is up and configured properly.• Start the Integration Console, you should see this dialogue• Although there are drop down boxes, you will have to type each of the entries.


Attach to the Star Schema• The Server is the name of the server on

which your Oracle database is running. • The Catalog ODBS DSN in this case is an

entry in the tnsname.ora file. • The prefix Oracle: identifies it as a

tnsnames entry to Essbase. • BSORCL is the identifier for this tnsnames

entry.• The Code Page is the language.• User Name here refers to the database

schema in which you would like an OLAP Metadata catalog created.

– Here the catalog is being installed in Scott/Tiger (bad practice)

• Finally, press the Create button to create the catalog and you are ready to log on.


The next dialogue facilitates the connection between your newly created metadata catalog and the Essbase Server


OLAP Model or an OLAP Metaoutline?

Create the OLAP model first.


Data Source?

• For simplicity in this demo, the same database that was used to store the metadata catalog is also used to as the data source.


Build the OLAP Model


Add FactsStart by creating a

fact table. You can select

Tools/Create Fact Table from the main menu and select the table to use from the list of tables.

Or simply highlight the table name you want to use for your fact table and drag it to diagram.


Time?

You will then be asked if you want to create a time dimension. Press No – this demo will use the Transaction Date table for its time dimension.


Accounts are Our Metrics

Finally, you will be asked if you want to create an Accounts dimensions. Answer Yes, this is where the measures are stored in a multi-dimensional model.


Link our Transaction Facts to Accounts

Highlight the Transactions, right click and use the tear-off menu to display the columns

You can move and re-size objects with standard GUI functionality, grabbing focus, sizing arrows, etc.


Add Dimensions

Select Tools/Create Dimensions


Next…


Answer Yes and the Model is Nearly Complete


One table at a Time Offers More Control

Drag and drop adds tables to the model and lets you navigate to the dialogue to create links and properties


Now it is Time!Finally, this model needs a time dimension. Transaction Date will work, just double click on the table object in the diagram and a properties dialogue will be displayed. Change the type to Time:


Your Time Dimension will now show a red title bar when selected.


Give the Model a Name and…

Save Your Work!!


Create Hierarchies

The meat of Business Intelligence are the hierarchies – the ability to accumulate and drill down on dimensions to spot trends, correlations and relationships.


Retailers and the Other Dimensions


OLAP Metaoutline

• An Essbase installation is organized into applications.• An application is a data set and a set of business rules

that forms a unit of interest to an analyst.• There are several components, one of the chief

elements being a database to contain and organize the cubes in the application.

• The next is a Metaouline that is used by the Essbase application to organize and describe data and its relationships.


Point and Click…

• Begin by selecting File/Open and then in the New tab, selecting OLAP Metaouline while the model selection is the OLAP Model just created.


…And Let the Tool do

the Work..


Nice, eh?


Drill Down


Time to Think – are You Dense?

• There are two types of storage available in Essbase applications– Aggregate– Block.

According to Edward Roske and Tracy McMullen compare the two storage options in their book Look Smarter than You Are with Hyperion Essbase (interRel Press, 2007).

Block Storage Option (BSO) databases utilize…member formulas, blocks, dense and sparse dimensions, and page and index files. Unfortunately, the block storage architecture…starts to have performance issues as dimensionality and outline sizes grow…

Aggregate storage option (ASO) databases were created to deal with … very large sparse data sets with a high number and potentially millions of members. ASO utilizes a new kind of storage mechanism that allows for improved calculation times…the calculations just aren’t as complex.”

• “Use BSO for applications that require complex calculations and write back capabilities.

• Use ASO for applications that require a large number of dimensions and members that simply ‘roll up’ (i.e. minimal complex calculations are required).”

And Beware the exploding database!!


This Cube is Dense


Create an Essbase Application and Database

• Time has come to use the Model and Metaoutline you have created to generate an Essbase application and database.

• Select Outline\Member Load.

• A dialogue will appear.

• Tell Essbase what to name the new Essbase application


In the Calc Scripts section…

make sure “Use default calc script”

is selected.

This insures that when the new application is created,

the loader calculates aggregates as well as adding

the detail level data to the new application.


Essbase creates a database with approximately a million and half

records in about one minute


Operations from Essbase’s Administration Console

• We will want to access the Essbase Outline

• The outline is the organization of the cube

• Clean up Essbase assigned names to make them more user friendly

• Create Generations

• All done from the Essbase Administration Console


We can Review Data


You can also view hierarchies in the outline


Prepare for Integration with OBIEE

• Most of the work before you make the actual connection between Essbase and OBIEE has to do with supplying meaningful names and descriptive information for the entries in your outline.

Start with the Retailers dimension.A right click, select Generations

Enter the labels that are to appear in the dashboard


Working in OBIEE

• OBIEE stands for Oracle Business Intelligence Enterprise edition

• Formerly known as Siebel Analytics

• When Oracle Corp. has acquired Siebel System and decided to make Siebel Analytics their flagship analytic engine and renamed it OBIEE

– A reporting component DASHBOARD– Metadata Layer– A repository


Creating a New Repository


Map it to Essbase

Essbase is a Multidimensional

database


The login is the Essbase owner.

Select Example1 Application


Drag and Drop to Create…

The Physical layer



The Physical layer

The Metadata layer



The Physical layer

The Metadata layer

The Presentation layer


Building a Dashboard

• The interface and reports are a system to users. • Careful design of an interface is one of the critical

success factors in a BI project. • OBIEE’s two main user facing components are

Dashboards and BI Publisher. • Our valid repository is Example1.rpd1• This step covers making that .rpd available to an OBIEE

dashboard• Then, we will examine generation of a display.


Step by Step

• Start by shutting down your BI servers


by step by step

• Now create a folder for Example1 in your OBIEE catalog


Configuration Settings

YOUR_ORACLEBI_HOME\OracleBIData\web\config\instanceconfig.xml


No Worries

YOUR_ORACLEBI_HOME \OracleBI\server\Config\ NQSConfig.INI


Restart the Servers, then…

We have Data!!!!


Oracle Answers

Click on column names to build an Answer


Check Your Answer

And Save


Navigate to Dashboards

Select Edit Dashboard


Add a page, a section, and your saved answer to the new page.

Then Save.


A dashboard with full functionality


Conclusions

• Clearly, this dashboard is first step. • The plumbing is in place, now you must complete the

rest of the house.

– Prompts, – Meaningful displays– pivot tables

• Small features add up to a big BI experience• There are many other fine tutorials and online papers

that will help you refine your dashboard – now that you have a multidimensional database supplying the cube with the data.


Thank You!

BravesoftTECH.

www.bravesoft.com

3131 South State Street

Suite 307

Ann Arbor MI 48108

877-734-2780

Fax: 734-786-8476

http://www.bravesoft.com/

Documents

© Copyright 2006 1 - 1 Oracle and Essbase - Building a Data Warehouse