LSA Layered Scalable Architecture

The BW Layered Scalable Architecture (LSA)An Introduction

Juergen Haupt, SAP NetWeaver RIG BI EMEAKHNC March, 11th 2009

1. What is the LSA?2. SAP LSA Overview

LSA Building BlocksLayersDomains

LSA Implementation ReferenceLSA Operations Reference

3. Lifecycle of the Customer LSA

Agenda

Note: All slides are taken from the Workshop PDEBW1‘‘Blueprinting an Enterprise Data WarehouseBlueprinting an Enterprise Data Warehouse ––The BW Layered, Scalable Architecture (LSA)The BW Layered, Scalable Architecture (LSA)’’

© SAP 2009 / KHNC- BW Layered, Scalable Architecture (LSA) Intro, Juergen Haupt /200903/ Page 2

SAP introduces the termLSA – Layered, Scalable Architecture

in order to describe the design ofservice-level oriented, scalable, best practice BW

architectures founded on accepted EDW principles*.

The LSA serves as a reference architectureto design transparent, complete, comprehensive

customer DWH architectures (Customer LSA).

The Customer LSA describes corporate standardsto build BI applications in a

performant, maintainable, flexible manner.

Enterprise Data Warehouse (EDW) And TheLayered Scalable Architecture (LSA)

* As introduced in Bill Inmon‘s Corporate Information Factory (CIF)© SAP 2009 / KHNC- BW Layered, Scalable Architecture (LSA) Intro, Juergen Haupt /200903/ Page 3

SAP LSA and Customer Adoption

SAP LSA: The BW Reference Architecture

Customer LSA Standards - Handbook

BI-Projekt-DesignBI-Projekt-DesignBI Project Design

Step 1:DesignDesign

LSALSADesignprocessDesignprocess

ApplyApplyCustomerCustomer

LSA StandardsLSA Standards

Step 2:ApplyApply



1. LSA Building Blocks1. Layers2. Domains

2. LSA Implementation Reference3. LSA Operations Reference


Agenda


SAP LSA: The Reference Architecture

LSABuilding Blocks

Reference

LSAImplementation

Reference

LSAOperationsReference

Describescore structures &

definitions

Describesdesign standards tobuild BI applicationsfounded on building

blocks

DescribesSupport

Scenarios

Life CyclesInformationMeta ObjectLSA

Meta Data ManagementNaming ConventionsOrganization (InfoAreas)

AdministrationData BaseHousekeepingMonitoring

Transport

Security

Data Staging/ Management fortransactional & master data

Persistent ObjectsFlows - scheduled/ recoveryTransformationProgramming (Abap)Organization (Process Ch.)

SAP LSA

Building BlocksLayer

Data QualityData IntegrationETL

DomainsData ModelLandscapeStorageOwnershipDevelopment concept


From LSA Reference to Customer LSA

SAP LSAReference

Customer-LSAImplementation

Standards

Customer-LSAOperationsStandards

Customer-LSABuildingBlocks

Customer-LSAStandards

LSABuilding Blocks

Reference

Core structures &definitions

LSAImplementation

Reference

Design standards tobuild BI applicationsfounded on building

blocks


SupportScenarios


LSA Building BlocksReference LSA & Customer LSA

The LSA is a reference architecture.A concrete customer LSA will always be unique

As background and targets of each customer differ, the preferred servicesdiffer and thus the customer LSA (layers & domains)

Customers differ with respect to:Priority of services

Painful experiences -> Control & Influence (outsourcing)State of integration (master data, source systems (processes))SkillsOverall governance, sponsorship, commitmentIndustriesTheir starting point

Enterprise Application Rollout Driven BW EDWHeterogeneous Source Driven BW EDW

All in one EDW for local and group reportingClassic EDW for group reporting






Agenda



LSABuilding Blocks

Reference

LSAImplementation

Reference



definitions


blocks

DescribesSupport

Scenarios




Transport

Security



SAP LSA





LSA Reference LayersQuick Intro

LSA

Reporting Layer (Architected Data Marts)Reporting Layer (Architected Data Marts)

Business Transformation Layer

Operational D

ata StoreO

perational Data Store

Data Propagation Layer

Quality & Harmonisation Layer

CorporateMemory

Data Acquisition LayerData Acquisition Layer

Accepts extracted data 1:1Temporary

Digestible data - ready toconsume for BI applications

Harmonized view on dataApplication neutralCorporate ownedGranularBusiness Key

Source system like service levelLong term, granularComprehensive, completeMaster the unknown

Create harmonized viewGuarantee quality

Apply business logic

Reporting & Analysis readyOften Near Real Time

ReportingGranular, operational like


Detailing LSA Reference LayersAcquisition to Propagator Layer

SourceSystems

EDW

Data flowData flow

CorporateMemory

BusinessTransformation

Layer

Reporting Layer

PropagationLayer

AcquisitionLayer

BI-Applications

Harmonize/Quality

PropagatorsThis flow describes daily, weekly,.. recurringstaging of data to feed finally the BIapplication layersThe staging creates data in PropagatorDSOs, which are easy to digest for BIapplications on top.Easy to digest means standards like:

Additive delta i.e. data can be directprocessed into InfoCubes

Data are integrated if the BI applications askfor integrated data

Data are local if the BI applications ask forlocal, not harmonized data

Data have no flavor with respect to specificbusiness rule transformations but offeradditional data with respect to the loadeddata, which are commonly or frequentlyneeded by the BI applications

Manageable portions of data to fulfillReport Availability, Recovery, AdministrationSLAs(-> Domains)

......

LSA Example


Detailing LSA Reference LayersPropagation Layer & Digestible Data

The core service of a Propagation Layer is to offer digestible data to applications.

Digestible may mean:

harmonized data in the broadest sense (for more details s. Chapter on Data Model)1. integrating data: common semantics, common values2. smoothing data: common semantics, technically unified values (e.g. compounding)

trimmed to fit DataSources and Data persistency‘s toReduce data complexity for applications1. Extending DataSources by looking up information, which applications frequently

ask for. Note: introduced parent-child relationship must be stable otherwiserealignment issues!

2. Merging different but highly related DataSources and store data in a singlepropagator, If application always or frequently request them together (e.g. HRInfoTypes, avoiding extractor enhancements)

Provide sound data portions for better support of application services (availability etc)3. Collecting data from the same (or similar) DataSource but from different source

systems to less or a single source system independent propagator (s) ( LSAdomains)

4. Splitting data from a DataSources into multiple persistency’s with identical structure( LSA domains)

LSA Example


Detailing LSA Reference LayersContent of EDW Layers

Dat

aSou

rce

Dat

aSou

rce

no history

complete history‘booking history’Source system SLA

completeHistory

Business key

limited historyBusiness key

DataSourceDataSource

Less comprehensive + add fieldsNone or lightHarmonization:

compoundingconcatenationor 1:1


comprehensive: all fields+ add fields

CorporateMemory Layer

DataSourceDataSourceComplex Harmonization:common semanticsmappingcontent consolidation

Harmonized fields + add fields

Propagation

Layer M

oldingsAcquisitionLayer


comprehensive: all fields


comprehensive: all fields

LSA Example


Detailing LSA Reference LayersPropagation Layer & Trimming Data

DataSource ASource 2




‘DataSource A’Propagator


3. Collect

DataSource BDataSource BDataSource ADataSource A

‘DataSource A & B’Propagator


2.Merge

DataSource ADataSource A

‘DataSource A +’Propagator


1.ExtendAdddata


‘DataSource A’Propagator 1‘DataSource A’Propagator 1

4.Split


Note on Collecting and Splitting DataSources:This is very close related to LSA Domains!Both may not be applied without regarding

volume of data!

LSA Example


Data flowData flow

2LIS_11_VASTI

2LIS_11_VASCL

YGTCS_SUMMARY

2LIS_12_VCSCL

Acquisition

Corp Memory

EDWpropagationR/3

Corp Memory

Corp Memory

Corp MemoryCorporate Mem.

EDW LayersCustomer Example

LSA Example


LSA Reference LayersData Propagation Layer Flier

‚normal‘ DSO in overwritesemantical/ logical partitioned for large scale DWH/ time-zone support

store & deploy

Minimum history defined by requirements of target-applications/ dependent fromCorporate Memory existence

history

DataSource specificas comprehensive as possible, if propagator is expecting volatile requiermentsMerge of different DataSources to reduce complexity

content

driven by BI application requirements (report availability)update

Criteria Characteristics

potential sources Data Acquisition Layer, Harmonization Layer, Corporate Memory

potential targets Business Transformation Layer, Reporting Layer

reusability Yes

transformations Additional, stable fields to increase (re-)useability & accessibility (e.g. currencytranslation). No application-specific rules!

granularity single records, granularity defined by DataSource business-key

main services ‘Single Point of Truth’ for BI applications (Business Transf. & Reporting Layers)Provide digestible (additive delta, content, performance) data for BI applicationsapplication recovery, rebuilt

housekeeping Regular delete of DSO change-log content

DW

H S

ervi

ces

Impl

.O

p.

LSA Example






Agenda



LSABuilding Blocks

Reference

LSAImplementation

Reference



definitions


blocks

DescribesSupport

Scenarios




Transport

Security



SAP LSA





LSA Domains: Background –The Broad View of BW on EDW

Traditional EDW SAP BW EDW

Companiessituation

operational world fragmentationno view across the business

Increasing operational worldharmonization

Yet limited view across the business

Scope-AreasCross-/

Corporate BI

In scopelittle information freshness (monthly)

In scopehigh information freshness (daily)high flexibility

Local- /depart. BI

Not in scope In scopestandard reporting with local adoptionsflexible roll out

Operational BI Not in scope Partly in scope

Evaluation Long time for ROIHigh Latency (e.g. monthly)High costsLow synergies for local/departmental BIOverall acceptance problems

Incremental approach to EDWImmediate ROI (local BI)Std. DWH Latency (day) to Low (RDA)Standardization lowers costsHigh synergies for all BI flavorsIncreasing acceptance over time

LSA Example


LSA Domains: Background –The Broad View of BW on EDWThe goals of an EDW providing cross-organizational insights are generally accepted.

But what happens with the BI needs of all the organizational units (e.g. markets) and what’sabout overall BI/ DWH consolidation ?

The originally limited view of EDW promoters is one of the main reasons for missingacceptance of the EDW investments.

A lot of SAP customers thinking of a BW EDW have a broader scope:Yes, of course we urgently want this cross-corporate BI but on the other hand side we urgently need aconsolidation of DWHousing & BI for all their organizational units to support the daily business based oncorporate best practice

Nobody can afford (from business & cost point of view) on the long run that allorganizational units build their own BI & DWH reality.

Why not consolidate and standardize BI & DWHousing ‘local’ BI requirements having EDWprinciples in mind and building incrementally the foundation for cross-corporate BI, the BWEDW, but having immediate ROI standardizing ‘local’ BI?

Note: of course we find also the ‘traditional’ EDW approach using BW!

The LSA addresses an evolutionary EDW approach introducingData Domains to support ‘local’ BI services without neglecting the

broader EDW picture.

LSA Example


LSA Domains:Services addressed by Data Domains

Non/Light-Architecture

Domains mean structuring / modelling of data within the layersTransparent, disjoint structuring of transactional data using stable criteria.Target is the support of:

Independency / autonomy of organizations24x7, time-zonesScalability / performance/ low latency(parallel vers. sequential)Challenging recovery windowEmbedding into RDBMSImplementation & operational robustness

Layered, Scalable-Architecture

D a

t a

f l o

w

D a

t a

f l o

w

Advantages+ Transparency & Flexibility

+Development, Maintenance+Administration, Operations

+ Scalability & Robustness+Application+Load / Query Performance+Database Integration

Acquisition Layer

LSA Example


I) BW EDW Data Domains Supporting BI/ BWConsolidation

EuropeJapan

Asia Pacific

ERPERPERPERP

ERPERP

ERPERP

ERPERP

BWBWBWBW

BWBW

BWBW

BWBW ERPERP

North America

South America

A BW EDW replaces a bunch of existing BWs and/ or legacy DWHs(BI Consolidation) spread across the organization

To enable comparable services like we had in a distributed, multiple DWH instanceworld (yes, there are some nice things) we introduce Data Domains in a BW EDW that

divide the transactional data but use identical meta data.BWBWX

XX

Using Domains in a BW EDW stands for manageability & flexibilityDomains allow SLAs in a BW EDW like in a distributed BW world

BW EDWBW EDW

X

Domain AmericasDomain Americas

X

Domain EuropeDomain Europe

X

Domain Asia PacificDomain Asia Pacific

LSA Example


II) BW EDW Data Domains Supporting EnterpriseERP Rollout

EuropeJapan

Asia Pacific

Global ERPGlobal ERP

North America

South America

A single BW EDW shall offer standard reporting & analytics for all organizational unitsin a global ERP rollout.

To enable comparable services like we had in a distributed, multiple BW instanceworld we introduce Data Domains in a BW EDW that divide the transactional data but

use identical meta data.

BW EDWBW EDWAMSAMS EMEAEMEA APAAPAUSUS GermanyGermany


LSA Example


III) BW EDW Data Domains to Divide Data bySources-‘Quality’

BW EDWBW EDW

mainERP

mainERP

remoteERP 1

remoteERP 1

remoteERP 2

remoteERP 2

Main DomainMain Domain Remote DomainRemote Domain

less stable, no controlstable, controlled


LSA Example


Summary: Domains makes always sense keepinglarge BW EDWs manageable & flexible

Dom

ain

Wes

tD

omai

n W

est

Dom

ain

Cen

tral

Dom

ain

Cen

tral

Dom

ain

East

Dom

ain

East

BW EDWBW EDWCompany

is operatingIn North America


North America

South America

Domain SouthDomain SouthAMSAMS

Company isexpanding to

South America

Dom

ain

Dom

ain

Wes

tW

est

Dom

ain

Dom

ain

Cen

tral

Cen

tral

Dom

ain

Dom

ain

East

East

BW EDWBW EDWEurope

North America

South America

Domain SouthDomain SouthAMSAMS

Dom

ain

Dom

ain

Wes

tW

est

Dom

ain

Dom

ain

Cen

tral

Cen

tral

Dom

ain

Dom

ain

East

East

DomainDomainEUEU

Company isexpanding to

Europe

BW EDWBW EDW

LSA Example


Transparent, Scalable Structuring Your BW:LSA Domains & Layers

LSA


Business Transformation LayerBusiness Transformation Layer

Data Propagation LayerData Propagation Layer

Quality & Harmonisation LayerQuality & Harmonisation Layer

Corporate

Memory

Corporate

Memory


Access Abstraction Layer(MultiProvider)


Single Source System (Layer)Single Source System (Layer)

LSA


Business Transformation LayerBusiness Transformation Layer

Data Propagation LayerData Propagation Layer

Quality & Harmonisation LayerQuality & Harmonisation Layer

Corporate

Memory

Corporate

Memory




Multiple Source Systems (Layer)

Distributionactively designed:

Domains

Distributioninherited

LSA Domains distribute the transactional data for the entire BW EDW or certain areas(flows) in a disjunctive manner. The meta data definitions of domains are common.

LSA Example


LSA Data Domains and LayersProperties of LSA Domains

As a rule of thumb:Domains organize data by a general criteria driven by reporting, BI andmanageability requirements, i.e. domains often differ from operational dataorganization.Domains are disjoint from a transactional data point of view – harmonized masterdata is shared.Domains use identical meta data definitions.Cross-views are achieved through MultiProviders or dedicated persistentInfoProviders.Domains should be stable.Domains should be consistent across all layers (across all flows).Domains should be general for all layers, exceptions:

The Acquisition Layer inherits the structuring from source systems / extractors.Domains do not apply on the Corporate Memory.

LSA Example


LSA Building BlocksReference LSA & Customer LSA – Example

Acquisition/PSA Layer

CorporateMemory

Layer

PropagationLayer

BusinessTransformation

Layer

FlexibleLayer

DimensionalLayer

VirtualLayer

(YADSS100)

YCDSS100

YPDSS1DX

YPDSS1GX

YPDSS1WX

YPDSS1UX

YBAPP1AX

YBAPP1DX

YBAPP1GX

YBAPP1WX

YBAPP1UX

YFAPP1AX

YFAPP1DX

YFAPP1GX

YFAPP1WX

YFAPP1UX

YDAPP1AX

YDAPP1UX

YVAPP1XX

YVAPP1XXYDAPP1WX

YDAPP1DX

YDAPP1GX

Lookup-tables

Asia

Europe

Americas

Germany

US

YPDSS1AX

Controltables

LSA

Reporting Layer (Architected DataMarts)

Reporting Layer (Architected DataMarts)

Business Transformation Layer

Operational D

ata Store

Operational D

ata Store


Quality & HarmonisationLayer

Corporate

Memory


From Reference LSA to Customer LSA:Individual Domains

Reference LSA Customer LSA LSA Example






Agenda



LSABuilding Blocks

Reference

LSAImplementation

Reference



definitions


blocks

DescribesSupport

Scenarios




Transport

Security



SAP LSA





LSA/ EDW Implementation Reference

An EDW means managing high volumes in all areas: data store size, data loads,querying and this under often extremely challenging conditions: 24x7

The customer LSA building blocks provide the basic framework (layers & domains) toreconcile the competing services even under extreme conditions.

The customer LSA implementation blueprint has then the task to provide standardsthat translate services like flexibility, thru put, robustness, completeness,comprehensiveness.... into BW functionality.

Customer LSA ImplementationCustomer LSA building blocks

LSA Example


LSA/ EDW Implementation ReferenceLessons Learned

Standardize data management as much as possible regardless the origin ofdata

Observe 80:20 rule – first provide guidelines for core BI application requirementsImplementations standards are developed incrementallyExceptions to implementation guidelines must be approvedThe more exceptions the less robustness and the higher TCOThe bigger the expected EDW (meta data) will become the more generic theimplementation must be

Anticipate growth – implementation standards must be able to manage growthAvoid serialization of data processing – parallelize data flows

Strategic – follow customer domain concept (general logical/ semantical partitionedimplementation)Strategic + Tactical – expand domain concept by tactical parallelization to meet individualapplication requirementsTactical – no general domains chosen – use parallelization to meet individual applicationrequirementsBranch out services – observe core services an put other services aside the main dataflow

Advertise & Train the idea of Customer LSA and implementation guidelines

LSA Example


LSA Layer ImplementationPropagation Layer & Trimming Data







3.Collect

DataSource BDataSource BDataSource ADataSource A



2.Merge




1.ExtendAdddata



4.Split


LSA Example


SRC3SRC2SRC1

DS DS DS DS DS DS DS DS DS DS DS DS DS DS DS DS

0EMPLOYEE_ATTR

0PERSON_ATTR

0HRPOSITION_ATTR

PA-Info-types

OM-Info-types

Customer Infot-types

DS

0EMPLOYEE_ATTR

0PERSON_ATTR

0HRPOSITION_ATTR

DS

0EMPLOYEE_ATTR

0PERSON_ATTR

0HRPOSITION_ATTR

DSO DSO DSO DSO DSODSO

DS DS DS DS DS DS DS DS DS

0EMPLOYEE_ATTR 0PERSON_ATTR 0HRPOSITION_ATTR PA-Infotypen OM-Infotypen Kundeneigene Infotypen

PA-Info-types

OM-Info-types


PA-Info-types

OM-Info-types


DSODSO DSO

ZEMPLOYEE

ZEMPLOYEE ZPERSON ZHRPOSITION

Data Acquisition Layer

Corporate Memory


Reporting Layer

Harmonization Layer

PROVIDE +gather

PROVIDE +gather

PROVIDE +gather

PSA PSA PSA PSA PSA PSA PSA PSA PSA PSA PSA PSAPSA PSA PSA PSA PSA PSAPSA PSA PSA PSA PSA PSAPSA PSA PSA

InfoSource

DSO

DSO

DSO

ZEMPLOYEE

ZPERSON

ZHRPOSITIONDSO

DSO DSO

ZPERSON ZHRPOSITION

LSA Implementation ReferenceTrimming Data: Merge DataSources (Example)

MergedPropagators

CM: MergedTransformed Data

CM: extracteddata 1:1

LSA Example


Filling DomainsFlow Splitting Implementation: Data Unification

The early PSA-based split

APA EMEA Americas

PSA

The Pass Thru DSO based split

APA EMEA Americas

PSA

Pass ThruWO-DSO

Unification InfoSource

Propagators InfoSource Propagators InfoSource

Unification of datadata managementadministration

LSA Example






Agenda


LSA Operations

Data Warehouse Performance Optimization

Recommendation Customer LSA & Housekeeping:Use and adapt existing best practices (SAP or own) to standardize operations

LSA Example


LSA OperationsPotential Content Topics

Load Performance GuidelinesInfoCubes– Efficient load of data into aggregates– InfoProvider properties– Line item & high cardenality– Inventory– Multi-dimensional clustering– Compression– Number range bufferingDataStore Objects– Run time parametersMaster data– Reorganization– DeleteDTPsFlat FileABAP-Programming

Load BalancingFor Data Warehouse processesFor load processes into BWFor BW background processesFor DataStore-Object processes

Optimization data storage

Tools for run time analysis of BW processes

Information Lifecycle ManagementData archiving processes (DAPs)archiving dataarchiving of request dataNLS

HousekeepingDelete requests of PSADelete Change Log dataSelektive delete löschenDelete master data & texts

Transports

Security´´

LSA Example






Agenda


BI-Project-Design

Life Cycle of The Customer LSA


Customer LSA : Standards - Handbook

BI-Project-DesignBI Project Design

Step 3:PerfectPerfect

Customer LSA

Step 4:UpdateUpdate

Customer LSA

Step 1:DesignDesign

Customer LSA

Step 2:ApplyApply

Customer LSA


© SAP 2009 / Page 42

Thank you!


Copyright 2009 SAP AGAll Rights Reserved

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained hereinmay be changed without prior notice.

Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.

Microsoft, Windows, Excel, Outlook, and PowerPoint are registered trademarks of Microsoft Corporation.

IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, System x, System z, System z10, System z9, z10, z9, iSeries, pSeries, xSeries, zSeries,eServer, z/VM, z/OS, i5/OS, S/390, OS/390, OS/400, AS/400, S/390 Parallel Enterprise Server, PowerVM, Power Architecture, POWER6+, POWER6, POWER5+,POWER5, POWER, OpenPower, PowerPC, BatchPipes, BladeCenter, System Storage, GPFS, HACMP, RETAIN, DB2 Connect, RACF, Redbooks, OS/2, Parallel Sysplex,MVS/ESA, AIX, Intelligent Miner, WebSphere, Netfinity, Tivoli and Informix are trademarks or registered trademarks of IBM Corporation.

Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.

Adobe, the Adobe logo, Acrobat, PostScript, and Reader are either trademarks or registered trademarks of Adobe Systems Incorporated in the United States and/or othercountries.

Oracle is a registered trademark of Oracle Corporation.

UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.

Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of Citrix Systems, Inc.

HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts Institute of Technology.

Java is a registered trademark of Sun Microsystems, Inc.

JavaScript is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape.

SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP Business ByDesign, and other SAP products and services mentioned herein as well as their respective logosare trademarks or registered trademarks of SAP AG in Germany and other countries.

Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business Objects products andservices mentioned herein as well as their respective logos are trademarks or registered trademarks of Business Objects S.A. in the United States and in other countries.Business Objects is an SAP company.

All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only.National product specifications may vary.

These materials are subject to change without notice. These materials are provided by SAP AG and its affiliated companies ("SAP Group") for informational purposes only,without representation or warranty of any kind, and SAP Group shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP Groupproducts and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construedas constituting an additional warrant.

Documents

LSA Layered Scalable Architecture