Enterprise Data Warehouse (EDW) With SAP BW

Enterprise Wide Data Warehousing with SAP BW

SAP NetWeaver Regional Implementation Group -BISAP AG

© SAP AG 2004, Enterprise Wide BW, J. Haupt / 2

Content

Enterprise Wide BW Enterprise Wide BW -- BackgroundBackground

Elements of a Corporate BW StrategyElements of a Corporate BW Strategy

The BW Data Model Prevents SilosThe BW Data Model Prevents Silos

BW Data Layers Guarantee Reliability and FlexibilityBW Data Layers Guarantee Reliability and Flexibility

BW Corporate Landscape PatternsBW Corporate Landscape Patterns

Content


Enterprise Wide BW

Background Continuously increasing number of BW installationsSteady growth and coverage of BW installations High degree of customer-satisfaction with BWBroad analyst acceptance of BW and the packaged solution approach In general a renaissance of data warehousing

Introduce an enterprise wide BW Strategy and Guidelines that guarantees :FlexibilityReliability

at low costs

What offers BW to help you realizing an enterprise Data Warehouse Strategy ?What are the elements of an enterprise BW Strategy ?


Enterprise Perspective on BW

Enterprise perspective on BW:Higher level organizational units wantto harmonize the lower level BW initiatives

Architecture-based BW implementationfrom the very beginning Stabilize a mature BW landscape(Overhaul the architecture)

GroupGroup

Division ADivision A Division BDivision B Division CDivision C Division DDivision D

............ RetailRetailBusinessMarketing

BusinessMarketing

..............

EuropeEurope AmericasAmericasAsia/

PacificAsia/

Pacific

GermanyGermanySpainSpain

UKUKFranceFrance

NordicNordic


Multiple BW Implementations – Challenges

Source

Source

Source

Source

Source

Source

Source Source

LocalBW

LocalBW

Source

Source

SourceLocalBW

Unit 2BW

Successful Data Warehousing means: controlled redundancy !

LocalBW

LocalBW

Stream BBW

Stream C BW

GroupBW Stream A

BW

Unit 1BW

Consistency issues:Uncontrolled data flowsMultiple extractions of same dataCostly developmentNot aligned data models

Danger of ‘high level’ Silos


Local BW Implementation – Challenges

....Real World

Informationrequirements

Schema-Modeling

Data Marts-InfoCubes

Extraction

Sources

Business Rules

Transformation

BW

Project Team

Observations:Incremental set upSolution-Focus (Data Marts)Departmental viewProject is responsible for the entire Data Warehouse Process

Potential consequences:Departmental islands (silos) Consistency problemsCompleteness not sufficientFlexibility restrictedTrace back not possible

Successful Data Warehousing means: controlled redundancy !


Corporate Data Warehousing Strategy

Missing corporate strategy and guidelinesPotential consequences:

Silo solutionsDoubtful reliabilityRestricted flexibilityHigh costs

Means:Uncontrolled RedundancyLimited Solution Focus

Prerequisites to set up a proper strategy:Awarenessabout accepted data warehousing concepts and their benefits

Support of corporate management (Sponsorship)

If there is no organizational momentum toward a common goal, then the best architecture, the best framework in the world is bound to fail.

W.H. Inmon“


Who needs a Corporate BW Strategy?All, but the prerequisites are different!

Group

Stream A Stream B Stream C Stream D

...... Retail BusinessMarketing ......

Regions/ countries

Headquarter

Region A Region B Region C

countries

Architecture-, SAP-based,

‚single Instance‘

Legacy R/3

Legacy R/3

Legacy Legacy

Legacy

Organization,Business

OLTP landscape

Necessity of a corporate BW Strategy


Content






Content


Elements of a Corporate BW Strategy

BW Enterprise Data Warehousing Decision Areas

Architecture Application Development

IntegrationStrategySupport

Data Layer Data Model Landscape

Tactical/Strategic

Decision-Making

OperationalDecision-Making

Think of: Building a ‘Corporate Information Factory’ *- Avoid Silos

*The Corporate Information Factory: introduced by Inmon, Imhoff


The Role of a Consistent Data Warehouse Data Model

Consistent Data Warehouse Data Model ⇒ long term success

A challenge forData Warehouse

consistency

MaterialManagement

OLTP

SalesOLTP

Consistent corporate Data Model

?

HROLTP

Not aligned operational Data Models

Inconsistent Data Warehouse Data Models ⇒ Island-solutions (silos, stovepipes)

?


The BW Data Warehouse Data Model

BW offers as part of the Business Content a predefined expandable Data Warehouse Data Model

BW Data Warehouse Data Model for SAP und other

applications(InfoObjects & InfoSources)

SAP Corporate Data Model

Legacy Data Models

Consistent mapping of operative data models

For all SAP applicationsFor SAP industrial solutions(e.g. Retail, Utilities..)For non SAP application

Ascential PartnershipSiebel, Oracle ....


BW Extended Star Schema built on the BW Data Model

BW Extended Star Schemasbuilt on BW Data Model

0SALESPERS0SALESORG.....

0CUSTOMER00CUSTGR......

0MATERIAL0MATGR0MATTYPE......

BW InfoObject Master Data:

0CUSTOMER 0MATERIAL 0AMOUNT

BW InfoCube:

Customer Material Sales Person

Sales Transaction

Materialgroup Sales ORGMaterialType

Corporate Data Model:SAP Components

Shared/ Conformed Dimensions

scope-specific

BCT Extractors/ DataSources

BCT InfoSources ≡Subject Areas

Master Data Transaction Data

0MATERIAL 0MATGR 0MATTYPE 0DOCNO 0CUSTOMER 0SALESPERS 0MATERIAL 0AMOUNT

BW Data Model definedby Business Content


BW Extended Star Schemas Prevent Silos

Conformed dimensions enable virtual BW MultiProvider scenariosConformed dimensions enable virtual BW MultiProvider scenarios

FACT Table

Material_Dimension_ID..........._Dimension_ID.............Dimension_ID.............Dimension_ID

Key figure 1Key figure 2InfoCubeInfoCube

BB

LocalLocal Part of Material Dimension

Material_Dimension_ID

0MATERIAL0PROD_HIERMaterial Dimension

Table

FACT Table

Material_Dimension_IDSalesOrg_Dimension_IDTime_Dimension_IDCustomer_Dimension_ID

Sales AmountQuantity InfoCubeInfoCube

AA

Material_Dimension_ID

0MATERIALMaterial Dimension

Table

Gebiet 1 Gebiet 2 Gebiet 3

Bezirk 1

Gebiet 3a

Bezirk 2

Region 1

Gebiet 4 Gebiet 5

Bezirk 3

Region 2

Gebiet 6

Bezirk 4

Gebiet 7 Gebiet 8

Bezirk 5

Region 3

Vertriebsorganisation

Material Hierarchy Table

0MATERIALLanguage CodeMaterial Name

Material Text Table

Material Master Table

0MATERIALMaterial Type

LocalLocal Part of Material Dimension

Conformed, SharedConformed, SharedPart of

Material Dimension

Material Dimension:Material Dimension:local & conformed partlocal & conformed part


BW DWH Data Model and Enterprise Wide BW

The BW Data Warehouse Data Model is the glue,which ties everything together!

It protects against ‘silos’ (‘stove pipe solutions‘, ‘islands’)Within a BW InstanceWithin a BW Landscape

It is the basis for communication‘Drill Thru’ between BW solution structures (InfoCubes, ODS-Objects)

within a BW Instancewithin distributed BW Instances

‘Drill Thru’ between BW solution structures and SAP applicationsBetween people

Enterprise BW Strategy:Usage of delivered BW DWH data model whenever possibleCentral controlled guidelines for modifications and expansions


Content






Content


Introducing a BW Layer Architecture to Guarantee Reliability and Flexibility

PrimaryPrimaryData Data

ManagementManagement

DataDataAcquisitionAcquisition

Data Data DeliveryDelivery

SAP Business Information Warehouse andthe Corporate Information Factory (CIF)*

OperationalOperationalDataDataStoreStore

StagingStagingAreaArea

Extr

acti

on /

Ope

n St

agin

g

any

sour

ce

CleansingCleansingTransformTransform

MasterReference

Data

Data Data WarehouseWarehouse

TransactionData

any

targ

et

Open

Dis

tribu

tion

ArchitectedArchitectedData MartsData Marts

Finance

Logistic

‘The Data MartThe Data Mart is customized and/or summarized data derived from the data warehouse’

* Source: Bill Inmon


BW Architected Data Mart Layer: The Departmental Layer – What can we expect?

0CUSTOMERmaster

0MATERIALmaster

BW Extended Star Schema:InfoCubesInfoObject Master Data(‘conformed dimensions’)

0CUSTOMER

0MATERIAL

InfoCube A

0CUSTOMER

InfoCube B

0CUSTOMER

0MATERIAL

InfoCube C

The BW Data Mart Layer based on the BW Data Model and a proper Project Method offers

Consistency within the Data Marts (InfoCubes, ODS-Objekte)Control of data redundancy (MultiProvider, Queries)All desired historical views ...


BW Architected Data Mart Layer: The Departmental Layer – Corporate Needs ?

It is not the task of the scope-oriented Data Mart layer to anticipate all kinds of future arising needs – this would overload the schemas and corrupt performance

It cannot be expected that the data mart project teams have the 360°corporate view

We are limited from a corporate perspective in terms ofFlexibility, Completeness Reliability, Consistency

if we concentrate ourselves just on Data Marts.

BW Data Warehouse Layer – the corporate layer


The BW (Enterprise) Data Warehouse Layer:The Corporate Layer


Open

Dis

tribu

tion



MasterReference

Data





Extr

acti

on /

Ope

n St

agin

gDataData

AcquisitionAcquisitionData Data

DeliverDeliver

any

targ

et

any

sour

ce TransactionData



Finance

LogisticThe result of the data transformation and cleansing process is stored persistently:

Subject orientedIntegratedGranularNon volatile (historical)Not scope-specific (not flavored)


The BW (Enterprise) Data Warehouse Layer The Corporate Layer – What can we expect?Reliability, Trace back – Prevent Silos

‘Single point of truth’All data have to pass this layer on it’s path from the source to the summarized edw-managed data marts

Controlled Extraction and Data Staging (transformation, cleansing)

Data are extracted only once and deployed manyMerging data that are commonly used together

Flexibility, Reusability, CompletenessThe data are not manipulated to please specific project scopes (unflavored)Coverage of unexpected ad hoc requirements (Support the Unknown)The data are not aggregatedOld versions are not overwritten or changed but useful information may be added

IntegrationData are integrated (as far as possible)Realization of the corporate data integration strategy

...

The BW Data Warehouse Layer is the corporate memory, thecorporate information repository


Introducing the BW Data Warehouse Layer

Business Logic

Web templates,ReportsQueries

Data MartsData Marts(InfoCubes) (InfoCubes)

(E)DW Layer(E)DW Layer(ODS(ODS--Objects)Objects)

Extraction

Sources

Staging-Transformation,

Cleansing

Project Team(E)D

WA

dmin Team

1

1

1 1

1

1

2

2

Business Logic

Set upIncremental set up in parallelwith data marts guarantees buy in from business users

TechnologyBW ODS-ObjectsExpanded InfoSources define ODS-Object structures BW flexible staging for Master Data

OwnershipCorporate Guidelines must be defined and administrators must be establishedto achieve the goals of a Data Warehouse layer

1 Increment


BW and Operational Decision-Making


Open

Dis

tribu

tion



MasterReference

Data





Extr

acti

on /

Ope

n St

agin

gDataData

AcquisitionAcquisitionData Data

DeliverDeliver

any

sour

ce TransactionData


any

targ

et

Finance

Logistic



Operational, Tactical and Strategic Decision-Making in ‘Traditional’ BW Implementations

Most BW customers do not distinguish between strategic/ tactical and operational decision-making using BW features:

Precalculated aggregates of InfoCubes (aggregation awareness)‘Drill thru’

tactical/ strategicon

summarized data:Data Marts

operationalon detailed, slightlysummarized data:

Standard CollectionsInfoCube withprecalculated aggregates

Two InfoCubes with ‘Drill Thru’

ODS-Object

InfoCube and ODS-Object

with ‘Drill Thru’

BW Architected D

ata Marts


Flexibility of BW Architected Data Marts: Data Marts & Standard Collections

Focus of traditionalFocus of traditionalBW implementations:BW implementations:

support support the unknownthe unknown

support support the knownthe known

data martscenarios• InfoCubes

• summarized

tactical / strategicdecision-making

edw base tables• granular• complete history• ODS-Objects

flexibility &reliability

standard collections•detailed, slightly summarized• InfoCubes, ODS-Objects

operationaldecision-making,flexibility


Limits of Using BW the Traditional Way: Introducing BW Operational Data Stores

The traditional usage of BW (with or without dwh-layer), which supports all kinds of decision-making using Architected Data Marts has its limits

If we want to provide event-level or close to event-level operational reporting (‘near real time data warehousing’) and/ orIf we are confronted with high data volumes (e.g. Retail)

s ev eral -> ev ent l oads

G ranul arity XP SA

Sou rce G ranul arity X

DWHDWH

G ranul arity >=X

DWHOD S-Ob j ect

Data MartsData Marts

G ranul arity > XI nf oCub e

d aily-w eekly…

Dedicated ODS-Objects (or InfoCubes) with granular data support operational decision making

Directly loaded from PSA Short life cycle of data

( Archiving, Near line Storage)They build a BW Operational DataStore

G ranul arity X

I nf oCub e

OD S-Ob j ect

BW BW Operational Operational Data StoreData Store


Abstract Layer Architecture and the Reality of BW as ‚Packaged Solution‘

Focus of traditionalFocus of traditionalBW implementations:BW implementations:

support the unknown

support the known

data martscenarios• InfoCubes

• summarized

tactical / strategicdecision-making

edw (base tables)• granular• complete history• ODS-Objects

flexibility,reliability &data mining

standard collections•detailed, slightly summarized• InfoCubes, ODS-Objects

operationaldecision-making& flexibility

operational decision-making& data mining

supportlow latency &high volume

Operational Data Stores• BW ods / BW PIPE for retail• others• most granular


Content






Content


Enterprise Data Warehouse as ‘Single Point of Truth’

DSS Applications Departmental Data Marts

EDW

MarketingAcctg Finance

Sales ERPERP

ERP

CRM

eComm.

Bus. Int.

ETL

GlobalODS

Oper.Mart

Exploration warehouse/data mining


Stag

ing

Are

a

localODS

DialogueManager

CookieCognition

Preformatteddialogues

Cross mediaStorage Management

Near lineStorage

Web Logs

SessionAnalysis

Internet

ERPCorporate

Applications

ChangedData

GranularityManager

Archives

The Corporate Information Factory*


Basic Landscape Types

The ideal corporate BW landscape that supports all kindof decision-making needs ?

EnterpriseBW

Hub & SpokeBW Enterprise

Data Warehouse SpokeBW

SpokeBW

Others

SpokeBW

‘Local’BW

‘Local’BW

‘Local’BW

GlobalBW

Outside-INMultiple Hubs &

Global BW

Others

Select the [IT] approach that fits most closely with corporate strategy Prof. Sethi, University of Texas

Information&Management, 2/01“

There is no ‘one size fits all’ BW landscape!


Some Influencers of a Corporate BW Landscape

More and more operational, low-latency decision-making using BW

operationaldecision making

tactical/ strategic decision making

edw &edw managed

data marts

operational BW Objects

Mission critical BW applications in BW

mission criticalapplications

business criticalapplications

Local interests

central governancelocal freedom

Flexibility for time of migration and future acquisitions


Single Corporate Wide BW Instance

ParametersCompany time zones Mission critical dominates othersWorkload from operational reportingScalability

BW EDWBW EDW

BW BW Architected Data MartsArchitected Data Marts

PSAPSA

Stag

ing

Area

BW ODSBW ODSlatency: near real time

External Data MartsExternal Data Marts

• mission critical• business critical• others

Standard Standard Collections Collections

DataDataMartsMarts

• mission critical• business critical

latency: >= 1 day


Handling Mission Critical and Operative Decision-Making Separately

• mission critical• strategic

BW EDWBW EDW

BW O

DS

BW O

DS


BW O

DS

BW O

DS

• business critical• others

BW O

DS

BW O

DS

ParametersCompany time zones Mission critical dominatesEDWMigration path from singleInstanceEDW separated from majority of data marts


BW EDWBW EDW

BW O

DS

BW O

DS

ParametersCompany time zones EDW close to majority ofdata marts


Multiple Central Managed Local BWs to Support Business Critical Scenarios

Asia Europe Americas

EDWEDWO

DS

OD

S





Local BWLocal BW Local BWLocal BW Local BWLocal BW

Global BWGlobal BWParameters

ScalableMission critical dominatesEDWEDW separated frommajority of data marts

OD

SO

DS

OD

SO

DS

OD

SO

DS


Summary

BW offers a wide range of features that support enterprise wide data warehousing needs.

Nevertheless a strategy and an organizational momentum must exist to realize an enterprise wide BW strategy.


Questions

Any Questions?


Further Information

Public Web:www.sap.com > Solutions > SAP NetWeaver

SAPNet:Use ALIAS: /BW

SAP Service Marketplace:

www.service.sap.com/bwBW InfoIndex – Data ModellingBW InfoIndex – Enterprise Data WarehousingBW InfoIndex – ODS FunctionsEDW Whitepaper :

http://service.sap.com/~sapidb/011000358700003703852003E/EnterpriseDWOverviewEN.pdf

www.service.sap.com/educationPDEBW1 ‘Enterprise Wide Data Warehousing with

SAP BW’ (Workshop - 2 days)

http://www.sap.com/solutions/

http://www.service.sap.com/bw



http://www.service.sap.com/education


Copyright 2004 SAP AG. All Rights Reserved

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice.

Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.

Microsoft®, WINDOWS®, NT®, EXCEL®, Word®, PowerPoint® and SQL Server® are registered trademarks of Microsoft Corporation.

IBM®, DB2®, DB2 Universal Database, OS/2®, Parallel Sysplex®, MVS/ESA, AIX®, S/390®, AS/400®, OS/390®, OS/400®, iSeries, pSeries, xSeries, zSeries, z/OS, AFP, Intelligent Miner, WebSphere®, Netfinity®, Tivoli®, Informix and Informix® Dynamic ServerTM are trademarks of IBM Corporation in USA and/or other countries.

ORACLE® is a registered trademark of ORACLE Corporation.

UNIX®, X/Open®, OSF/1®, and Motif® are registered trademarks of the Open Group.

Citrix®, the Citrix logo, ICA®, Program Neighborhood®, MetaFrame®, WinFrame®, VideoFrame®, MultiWin® and other Citrix product names referenced herein are trademarks of Citrix Systems, Inc.

HTML, DHTML, XML, XHTML are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts Institute of Technology.

JAVA® is a registered trademark of Sun Microsystems, Inc.

JAVASCRIPT® is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape.

MarketSet and Enterprise Buyer are jointly owned trademarks of SAP AG and Commerce One.

SAP, SAP Logo, R/2, R/3, mySAP, mySAP.com and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned are trademarks of their respective companies.

Documents

Enterprise Data Warehouse (EDW) With SAP BW