Previews of TDWI course books offer an opportunity TDWI ...download.1105media.com/tdwi/Remote-assets/Sessions/... · x Collections of quality data and metadata important to the business

TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY.

Previews of TDWI course books offer an opportunity to see the quality of our material and help you to select the courses that best fit your needs. The previews cannot be printed. TDWI strives to provide course books that are content-rich and that serve as useful reference documents after a class has ended. This preview shows selected pages that are representative of the entire course book; pages are not consecutive. The page numbers shown at the bottom of each page indicate their actual position in the course book. All table-of-contents pages are included to illustrate all of the topics covered by the course.

TDWI Dimensional Data Modeling Primer

TDWI Dimensional Data Modeling Primer From Requirements to Business Analysis


ii TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY.

The Data Warehousing Institute takes pride in the educational soundness and technical accuracy of all of our courses. Please give us your comments—we’d like to hear from you. Address your feedback to:

[email protected] Publication Date: February 2011

© Copyright 2004-2011 by The Data Warehousing Institute. All rights reserved. No part of this document may be reproduced in any form, or by any means, without written permission from The Data Warehousing Institute.


TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY. iii

TAB

LE O

F C

ON

TEN

TS

Module 1 Dimensional Modeling Concepts ……....................................... 1-1

Module 2 Requirements Gathering for Dimensional Modeling ……….. 2-1

Module 3 Logical Dimensional Modeling ………………...….....……….... 3-1

Module 4 From Logical Model to Star Schema …………..…………...…. 4-1

Module 5 Dimensional Data and Business Analysis ……....……..…….... 5-1

Appendix A Bibliography and References ……………………..……...……… A-1

Exercises Exercise Instructions and Worksheets ………………………… W-1


iv TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY.

CO

UR

SE O

BJE

CTI

VES To learn:

Concepts of dimensional data modeling

The relationship between business metrics and dimensional data

Similarities and differences between relational and dimensional data models

Requirements gathering techniques for business metrics and dimensional data

How to build a logical dimensional model

Understand different types of dimensions and where to use each

How to translate a logical dimensional model to a star schema design

Testing and validating the dimensional model

How dimensional data is used to deliver business analytics and OLAP capabilities

TDWI Dimensional Data Modeling Primer Dimensional Modeling Concepts

TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY. 1-1

Module 1

Dimensional Modeling Concepts

Topic Page Dimensional Modeling in Context 1-2

Dimensional Modeling Basics 1-10

Comparing E-R and Dimensional Models 1-18

Concepts Summary 1-32

Dimensional Modeling Concepts TDWI Dimensional Data Modeling Primer

1-2 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY.

Dimensional Modeling in Context Business Intelligence Defined

Business Intelligence Defined

• The goal is improved decision making.• Concepts and methodologies – not just

technology.• Facts and fact-based systems are

necessary to implement.

Business Intelligence Defined









Dimensional Modeling in Context Business Intelligence Defined

BUSINESS INTELLIGENCE

Howard Dresner (Gartner Group) first defined business intelligence as shown on the facing page. As BI became a mainstream term, additional definitions have emerged. Among them is the following definition: “BI is neither a product nor a system. It is an architecture and a collection of integrated operational as well as decision-support applications and databases that provide the business community easy access to business data.” (Larissa T. Moss and Shaku Arte, Business Intelligence Roadmap, Pearson Education, 2003) David Loshin defines business intelligence as: “The processes, technologies, and tools needed to turn data into information, information into knowledge, and knowledge into plans that drive profitable business actions. Business intelligence encompasses data warehousing, business analytic tools, and content knowledge management.” (Business Intelligence: The Savvy Manager’s Guide, Addison Wesley, 2003) Business intelligence provides the ability to transform data into usable, actionable information for business purposes. BI requires:

Collections of quality data and metadata important to the business The application of analytic tools, techniques, and processes The knowledge and skills to use business analysis to

identify/create business information The organizational skills and motivation to develop a BI program

and apply the results back into the business The foundation that enables BI is the enterprise architecture—business, data, and technical. A well-implemented data warehousing program provides much of that foundation.



Dimensional Modeling Basics Dimensional Model Defined



Dimensional Modeling Basics Dimensional Modeling Defined

A BUSINESS MODEL OF PROCESS AND ACTIVITY

Chris Adamson’s definition of a dimensional model makes it clear that this is a process of modeling much more than data. While data is important as the means to collect and store measures, the modeling process is business oriented because the data simply represents the quantitative properties of business processes and activities.



Comparing E-R and Dimensional Models A Quick Review of Entity-Relationship Modeling



Comparing E-R and Dimensional Models A Quick Review of Entity-Relationship Modeling

ENTITY-RELATIONSHIP MODELS

This diagram illustrates a simple entity-relationship (ER) model. The primary components of an ER model are: Entity—An entity is a subject about which the business has the need,

will, and means to collect data—a person, place, thing, concept, or event that is of business interest. Entities are represented as labeled boxes in the ER model. Examples in the diagram include EMPLOYEE, JOB, and DEPARTMENT.

Attribute—An attribute is a property or characteristic of an entity that

can be collected as data. Attributes are listed inside the box of the entities that they describe. Job_shift_code and job_title, for example, are attributes of the entity JOB.

Relationship—A relationship is an important association between

pairs of entities that is of business interest and may be collected as data. Examples of relationships in this diagram include EMPLOYEE PERFORMS JOB and DEPARTMENT OWNS JOB.

Other important ER concepts include: Cardinality, which describes the number of occurrences of each

entity type that may participate in an occurrence of a relationship. Cardinality options include zero or one, exactly one, one or more, and zero or more. In this diagram some of the relationship cardinalities are: a PERSONNEL ACTION changes exactly one JOB; a JOB is changed by one or more PERSONNEL ACTIONs; a JOB is held by zero or one EMPLOYEEs. Sometimes cardinality as defined above is divided with “cardinality” indicating the maximum number and “optionality” indicating the minimum number.

Specialization (also called subtyping) creates a hierarchy or

parent/child relationship between an ENTITY super-type and its sub-types. Specialization makes sense when the entity sub-types have unique attributes or participate in relationships not common to all sub-types. In this diagram, PERSONNEL ACTION is an entity super-type with three sub-types.



TDWI Dimensional Data Modeling Primer Requirements Gathering for Dimensional Modeling


Module 2 Requirements Gathering for Dimensional Modeling

Topic Page Business Context for Data Modeling 2-2

Business Questions as Requirements Models 2-8

Fact/Qualifier Analysis 2-14

Requirements Gathering Summary 2-24

Requirements Gathering for Dimensional Modeling TDWI Dimensional Data Modeling Primer


Business Context for Data Modeling Business Value



Business Context for Data Modeling Business Value

IMPORTANCE OF BUSINESS CONTEXT

Business context is essential to delivering business value. The context determines the nature of BI program—the people who will use the information, the kinds of information they need, the business processes to be affected, and the information services to be provided. Business context provides the means to align business intelligence results with business goals.

BUSINESS VALUE PERSPECTIVES

As previously described, dimensional modeling is really a business modeling process—modeling of business processes and activities. The “big picture” view of modeling context begins by understanding how we create business value. The facing page illustrates three perspectives of value creation:

Business management views value as positive results of business activities that align with the tactics, strategies, goals, and drivers of the business.

Data management views value as business outcomes that are enabled through useful data that enhances knowledge to inform business decisions and actions.

Performance management views value as successful execution of continuously evolving strategy and planning where data and information are critical elements of a feedback system for continuous growth and improvement.



Business Questions as Requirements Models A Framework for Business Questions

businessquestions

(CRM / Product)

Management Disciplines

Subj

ect

Are

as

CRM BPM SCM BAM etc.

Customer

Product

Supplier

Workforce

etc.

Cost

Value

Duratio

netc.

businessquestions

(BPM / Customer)

businessquestions

(BAM / Workforce)

businessquestions

(SCM / Supplier)

ActivityProcess

OrganizationEnterprise

businessquestions

(CRM / Product)

Management Disciplines

Subj

ect

Are

as

CRM BPM SCM BAM etc.

Customer

Product

Supplier

Workforce

etc.

Cost

Value

Duratio

netc.

Cost

Value

Duratio

netc.

businessquestions

(BPM / Customer)

businessquestions

(BAM / Workforce)

businessquestions

(SCM / Supplier)

ActivityProcess

OrganizationEnterprise



Business Questions as Requirements Models A Framework for Business Questions

LIST OF BUSINESS QUESTIONS

Requirements modeling begins with business questions—the foundation for all subsequent dimensional modeling activities. The first context-level model is a robust and representative list of business questions that are within the scope of the project. It is not possible to develop a complete and exhaustive list of all questions that might be asked. The objective is to create a list of questions that is fertile enough to drive development of a highly adaptable data structure capable of answering questions that are not yet known. A major strength of the dimensional model is its ability to provide such a data structure.

A DIMENSIONAL VIEW OF METRICS

The diagram on the facing page illustrates a three-dimensional view that is useful in managing a comprehensive set of business metrics. The dimensions are:

Levels of metrics—enterprise, organization, process, and activity—that were previously discussed.

Subject areas of your business, typically found in a subject area model and the basis for the subject-oriented data warehouse. Subjects vary depending on the industry and the things that make your business unique. They include such things as customer, supplier, product, service, finance, etc.

Performance management methods used by your business. These include disciplines such as business performance management (BPM), customer relationship management (CRM), supply chain management (SCM), etc.

INSIDE THE FRAMEWORK

Each intersection of the three dimensions is an opportunity to find business questions and to manage business metrics. At the intersection of enterprise, product, and CRM, you might ask:

What needs to be known about products at the executive level to make CRM effective?

What are the KPIs for products that are important to CRM? How are product measures for CRM related to those for BPM? How are product and customer measures for CRM related?

METRIC CLASS WORDS

Similar to the class words of common data elements (date, code, etc.) metrics have class words (e.g., cost, value, duration, count) that describe the kind of quantity that they express.



Fact/Qualifier Analysis From Business Requirements to Data Requirements

FACTS:Discrete items of business information.

QUALIFIERS: Criteria used to access, sort, group,summarize, and present information. ass

ociatio

ns am

ong

facts &

qualif

iers

Mapping ofBusiness Questions

FACTS:Discrete items of business information.

QUALIFIERS: Criteria used to access, sort, group,summarize, and present information. ass

ociatio

ns am

ong

facts &

qualif

iers

Mapping ofBusiness Questions



Fact/Qualifier Analysis From Business Requirements to Data Requirements

STRUCTURING REQUIREMENTS FOR DATA

Business questions are the basis of requirements analysis, but they are not the final result. Full understanding of requirements is achieved through analysis of those questions. Requirements are much more than simply knowing which data is needed. To provide a foundation for logical design, you’ll also need to know how it will be accessed, navigated, and used. Business questions begin the transition to more formal data models through a matrix modeling technique known as fact/qualifier analysis. Each business question is examined to determine the fact(s) to be contained in the answer and the qualifiers that give the fact(s) meaning. The terms fact and qualifier describe roles that a data item assumes in a specific instance of business usage. Facts are discrete items of business information. Qualifiers are criteria used to access, sort, group, summarize, and present information. A single data item may be a fact in one instance and a qualifier in another.



TDWI Dimensional Data Modeling Primer Logical Dimensional Modeling


Module 3 Logical Dimensional Modeling

Topic Page Modeling Meters and Measures 3-2

Modeling Dimensions 3-4

More about Meters and Measures 3-12

Model Verification 3-18

Logical Modeling Summary 3-20

Logical Dimensional Modeling TDWI Dimensional Data Modeling Primer


Modeling Meters and Measures A Group of Related Business Measures



Modeling Meters and Measures A Group of Related Business Measures

FINDING METERS The first step of developing a logical dimensional model is identifying meters and measures. A meter is a group of related measures that can collectively be given a business name. The set of facts in the fact/qualifier matrix are the source of measures. Group those facts that are measures of the same business performance concept—EMPLOYEE SATISFACTION, CUSTOMER LOYALTY, PRODUCT PROFITABILITY, WORKFORCE PRODUCTIVITY, etc. Each business performance concept becomes a meter and the set of related facts the measures contained by the meter. The example domain has been limited to EMPLOYEE SATISFACTION; thus, all of the facts are related by that domain. Sometimes business questions are listed and fact/qualifier analysis performed for a broader or more complex domain. You may, for example, have a set of questions that yield facts about both CUSTOMER VALUE and PRODUCT PROFITABILITY—a combination of facts that aren’t readily combined in a single meter. A further examination of questions 6 and 7 reveals that these are not measures of EMPLOYEE SATISFACTION. Question 8 is already addressed by question 3. These are therefore excluded from the scope of this model and would be addressed by a separate model. This relatively simple example yields one EMPLOYEE SATISFACTION meter containing three measures: job_turnover_rate, avg_length_of_employment, and number_of_complaints.



Modeling Dimensions Adding Dimensions from the Qualifiers



Modeling Dimensions Adding Dimensions from the Qualifiers

DIMENSIONS IN THE LOGICAL MODEL

Once dimension hierarchies are known, the logical dimensional model is extended by adding dimensions and associating them with the meter. For every measure contained in the meter, all associated qualifiers must be represented by a dimension. To add dimensions to the model: 1. Include each dimension hierarchy previously identified. Associate

only the lowest level of each hierarchy with the meter as a one-to-many relationship.

2. Examine the remaining qualifiers to find those that may be

dimension attributes instead of dimension levels. In this example, employee gender and employee age are attributes that describe employee. Thus employee becomes the dimension level, with age and gender modeled as attributes. The dimension level employee is associated with the meter as a one-to-many relationship.

3. All remaining qualifiers are mapped as flat (single-level, non-

hierarchical) dimensions, and each is associated with the meter using a one-to-many relationship.



More about Meters and Measures Granularity and the Meter



More about Meters and Measures Granularity and the Meter

BUSINESS METRICS LEVEL OF DETAIL

Granularity describes the level of detail contained in the meter. Every measure in a meter must be at the same level of detail. Declaring the grain of the meter is an important verification step and yields an essential piece of information for the next steps of dimensional modeling. The grain is determined by the set of dimensions associated with the meter. By reading the cardinality of meter-to-dimension relationships, it is clear that every instance of EMPLOYEE SATISFACTION contains measures for a unique combination of:

One month One employee One job One location, and One trade



Model Verification Testing the Model



Model Verification Testing the Model

ADEQUACY TESTING

At a minimum, the dimensional model structure must provide answers to the questions that were posed by the business representatives. One of the tests, therefore, is to refer to the original set of questions and verify that the structure supports answering each of the questions.

CONSISTENCY TESTING

If conformed dimensions exist, the model dimensions should be compared against those dimensions. While the best practice is to adopt the conformed dimension, this is not always done. Even if the conformed dimensions are not used directly, there should be no violations of the definitions and structures prescribed by those dimensions. If conformed dimensions do not exist, then dimensions should be compared to similar ones that are used in existing data marts. Any inconsistencies in the definitions and structures will likely lead to differing conclusions.

EXPANSION TESTING

As stated earlier, not all the business questions can be known in advance. A major strength of the dimensional model is its ability to answer some questions that have not been predefined. In testing the model, review the dimension attributes and compare them to attributes available in the business data model (if it exists) and in the operational systems. If any are missing, determine whether they should be added. If the definitions differ, be sure to resolve the differences. Similarly, there may be additional measures related to the model meter that could be useful to the business. Explore these with the business representatives and augment the model accordingly. Often period-to-period comparisons are made, and inclusion of the previous period (e.g., same month last year) data as a distinct data element enables retrieval of the data for comparison from a single row.



TDWI Dimensional Data Modeling Primer From Logical Model to Star Schema


Module 4

From Logical Model to Star Schema

Topic Page Star Schema Dimensions 4-2

Star Schema Fact Tables 4-8

Star Schema Design Challenges 4-16

Modeling Process Summary 4-30

From Logical Model to Star Schema TDWI Dimensional Data Modeling Primer


Star Schema Dimensions Naming the Dimensions



Star Schema Dimensions Naming the Dimensions

FROM LOGICAL TO PHYSICAL

Logical dimensional models are typically implemented in either of two forms:

Multi-dimensional databases, also called cubes, are used with multi-dimensional OLAP (MOLAP) tools. For these implementations, the MOLAP tool includes a cube generation capability. The logical dimensional model captures the information needed to describe a desired cube to the tool.

Star schema databases are used with relational OLAP (ROLAP)

tools. For these implementations, physical modeling activities design the star schema.

NAMED DIMENSIONS

With either implementation, each dimension needs to be named. Dimension names are key navigation components for business analysts and others using OLAP tools; thus, it is important that the names have business meaning. In flat (single-level, non-hierarchical) dimensions, it is common practice for the dimension to have the same name as the dimension level. For multi-level dimensions, a collective name that encompasses all dimension levels is established. In the EMPLOYEE SATISFACTION example:

Employee is a flat dimension and retains the name employee. Location is a flat dimension that retains the name location. Labor union contains trade, and is a multi-level dimension with

the collective name labor organization. Year contains month, and is a multi-level dimension collectively

named month. Department contains job, and is a hierarchical dimension named

employment organization.



Star Schema Fact Tables Modeling the Fact Table



Star Schema Fact Tables Modeling the Fact Table

PHYSICAL MODELING CONVENTIONS

The fact table in a star schema begins as a translation of the meter from the logical model. Little or no structural change occurs. The most significant changes here are in modeling standards and conventions—primarily the naming standards that are applied.



Star Schema Design Challenges Slowly Changing Dimensions



Star Schema Design Challenges Slowly Changing Dimensions

DESIGNING FOR DIMENSION VOLATILITY

One of the challenges of OLAP analysis is analyzing data over time when the dimensions don’t remain constant across the time period of interest. Changing data values are common in dimensional data—a problem known as “slowly changing dimensions.” The terminology doesn’t mean to imply that some dimensions change more quickly than others; rather, that dimension values change less frequently than the values of measures in a fact table. When the data values in a dimension change, three options are possible to implement the dimension. Using Ralph Kimball’s widely accepted terminology, these options are:

Type 1 dimensions that overwrite dimension rows without retaining a history of dimension changes.

Type 2 dimensions that preserve dimension history by inserting new rows into the dimension table.

Type 3 dimensions that keep versions of dimension values by creating new columns for changed data.



TDWI Dimensional Data Modeling Primer Dimensional Data and Business Analytics


Module 5 Dimensional Data and Business Analytics

Topic Page Delivering Business Value 5-2

Effective Dimensional Modeling 5-6

Dimensional Data and Business Analytics TDWI Dimensional Data Modeling Primer


Delivering Business Value Data Enabled Business Analysis



Delivering Business Value Data Enabled Business Analysis

MANY DATA MARTS

A typical business intelligence environment encompasses many different data marts at different levels of detail, with different focus and purpose, yet with substantial overlap of dimensions and some overlap of measures. The example that we’ve followed throughout the course—employee satisfaction—is but a single data model in a more complex environment. The employee satisfaction model is a summary, and it is likely that a data mart exists to contain more detailed, atomic level data. It is also likely that many other data marts exist with similar measures and similar dimensions. Consistency and conformity across data marts is essential to support cross-functional business analysis.



Effective Dimensional Modeling Critical Success Factors



Effective Dimensional Modeling Critical Success Factors

KEYS TO SUCCESS The facing page lists several best practices and success factors for high-impact dimensional modeling.



Documents

Previews of TDWI course books offer an opportunity TDWI ...download.1105media.com/tdwi/Remote-assets/Sessions/... · x Collections of quality data and metadata important to the business