Train-BW

Solution Management and Planning for mySAP

SCM (including BW and APO)

May 2002

Agenda

BW

APO

BW & APO Integration

BW Architecture

Infosource

InfoCubes

Communication

structure

BW Server

Extracting Transactional Data and Master Data

Transfer Structure

Extract Structure

Source System

Transaction

Data

DataSource

Transfer Structure DataSource

(replicas)

Master Data

Transfer Structure

Extract Structure

Transfer Structure

Communication

structure

Master Data

InfoSource: Features

l Types of InfoSources: e.g cost center, division

n Transaction Data

n Master Data: Attributes, Texts, Hierarchies

l Communication structures are generated from the

InfoSources

l Transfer structures and transfer rules are generated

from the combination of InfoSource, Source System

and DataSource

l An InfoSource represents a set of data that is logically

grouped together based on some business logic

l Component of an InfoSource

l An InfoSource is made up of InfoObjects that logically

belong together

l Master data InfoSource

l A characteristic that contains master data, texts or

hierarchies and is assigned to an application

component

l Forming an InfoCube

l An InfoCube/ODS Object consists of a number of

InfoObjects.

l The characteristics, units and time characteristics are

essentially the key fields of the InfoCube/ODS Object.

l The key figures are the data fields (or facts) or values

of the InfoCube/ODS Object.

InfoObjects: Integration

Update Rules: Features

InfoCube

Communication

Structure

Characteristics Time Units Key Figures

InfoObjects

1 Simple move

More or less complex

computation within a routine

1

2

2 1 1 23

3 Lookup in an external table to

determine the value of a characteristic

Reference of a key figure to a unit4

4

4

SAPI

Extract Structure

Extractor

Transfer Structure

ALEInbox

Outbox

Data IDOCInfo IDOC

tRFC Connecting protocol

Transfer Structure

Connecting protocol tRFC

Data Packet

No PSA

SAP BW

Source System

Non-SAP

DataSource

ALE

Inbox

Outbox

Extraction Methodologies Summary

PSA

BAPI

PSA possible

InfoCubes

Communication

structure

BW Server

Transfer Structure DataSources

(replicas)

Non-R/3 Data Source - Using Flat Files

Transfer Structure

Extract Structure

SAP application

Transfer StructureData Sources

(user-defined)

Flat File Source System

Transaction

Data Transaction

Data

DataSources

Transaction

InfoSource

Update Rules

Transfer Rules

Demo InfoObjects/InfoSource Exercise

Architecture Datatargets

l Infocubes are the primary data targets in BW.

l Primary Object for data storage for data to be used

for analysis.

l Contains two types of data

n Key figures—Values or amounts (e.g. quantity & sales)

n Characteristics—are master data or organizational elements

such as company code and product

l 1 fact table (in 2 parts) and up to 16 dimension tables

n 3 Dimensions are predefined by SAP

Time

Unit

Data package ID

Basic InfoCube

BasicCube

Dimension 2

Facts

Dimension 1 Dimension 3

Dimension 4Dimension n

l An InfoCube is designed, or

“modeled” to meet a set of

business reporting requirements.

l Modeling is the process by which

reporting requirements are

structured into an object with the

facts and characteristics that will

l Characteristics are structured

together in related branches

called “dimensions” .

l The key figures form the “facts”.

l The configuration of dimension

tables in relation to the fact table

results in what is known as the

“star schema”

SAP BW Data Model

FACT

Dimension

table

Text

SID Table

Master

Hierarchy

Text

SID Table

Master

HierarchyText

SID Table

Master

Hierarchy

Text

SID Table

Master

Hierarchy

Text

SID Table

Master

Hierarchy

Text

SID Table

Master

Hierarchy

Text

SID Table

Master

Hierarchy

Text

SID Table

Master

Hierarchy

Text

SID Table

Master

HierarchyText

SID Table

Master

Hierarchy

Text

SID Table

Master

Hierarchy

Dimension

table

Dimension

table

Dimension

table

Dimension

table

BW: Extended Star Schema / Snowflake Schema

What is it? An Indented and nested listing of all InfoCube tables and

supporting tables with access to each table via the data browser transaction

n Review table design and links n Navigate to table contents n Review nested structure

Example for the

Costs and Allocations

InfoCube

Note the BW table naming

convention. All BW object

tables start with the prefix

“/BI*/*”. “/BI0/” is for SAP-

delivered objects and and

“/BIC/” means customer-

created. “F” is for “Fact table”,

“D” is for “Dimension table”.

Transaction 'LISTSCHEMA'

ls star time dep yes

ls star time dep yes

Master data

tables

Master Data Tables of a Characteristic

Multi Cube

Basic Cube Remote Cube Basic Cube

MultiCube Concept

B

W

S

E

R

V

E

R

U

s

e

r

Data Flow Architecture Including PSA and ODS

Transfer Rules

OLAP Processor

Transfer Rules & Update

Rules


Rules

Update RulesUpdate RulesUpdate Rules

Operational Data Store

PSAPSA

Update Rules

Transfer Rules

Transfer Structure Data Source

(replicas)

Transfer StructureData Sources

(user-defined)

Object Object

Source System

PSA PSA

Transfer StructureTransfer Structure

PSA

Persistent Staging Area (PSA)

Inbound layer for initial data storage

Based upon transfer structure

Source system dependent

Stores Transaction data and Master data

Partitioning supported (2.0B)

PSA PSA

ODS ObjectObject

Transfer Rules & Update Rules

l Can store transaction data

l Data can be loaded from the ODS to

InfoCubes

l Can serve as a quality checkpoint for

data

l Can merge data from different InfoSources

l Is a denormalized data structure

l Transparent table

l Not a star schema

l Holds data that is available for reporting

l Not optimized for query

l performance though – optimized

for storage

l Can be defined as a DataSource for

another BW System

l Data extraction using TRFC is then

possible

l Can dump ODS data to a flat file

Operational Data Store Properties

Demo ‘Creation Of a InfoCube’

BW Reporting

InfoCube

Query Definition BEx Browser

BEx Analyzer

Document

Storage

Business Information Warehouse Server

MS ExcelBEx Map

OLAP Processor

InfoCubes,

ODS

Business Explorer

BEx Navigation Toolbar

OPEN

SAVE REFRESH

QUERY

CHANGE

QUERY VIEW

BACKGOTO

OLAP FUNCTIONS

FOR ACTIVE CELL

FORMAT

LAYOUT

TOOLS

SETTINGS

HELP

• Additional toolbar appears in SAP Business Explorer

• All toolbar functionality also available through the SAP Business Explorer

pull-down menu

Demo ‘BexAnalyzer to Create a Query’

Technical Implementation

BW is a READ based system rather than

an UPDATE based system.

RAID 5 generally provides better READ

performance than RAID 1.

Example BW Disk Configuration

using RAID 1 and RAID5:

Operation System - RAID1

DB, SAP SWAP - RAID1

Redo Logs - RAID1

Mirrored Redo Logs - RAID1

PSAPTEMP - RAID1

SAPDATA - RAID5

BW Pre-Installation: Disk Configuration and Layout

Maintain entries in customizing table RSADMINC

Packet Size is in KB and determines the size of Idoc data

packages of data loads from flat file data sources

and data loads from BAPIs. A setting of 20000 is

the standard recommendation.

FrequencyStatus-IDOC is a ratio. The setting determines the

frequency that status Idocs are sent.

BW Setup: Maintain Data Transfer Parameters

Technical Setup

Source System Connections: Some Facts

l A BW system can serve as a source system for another BW

system. This scenario is referred to as a “data mart”.

l The ODS can be a DataSource for another BW system.

l An InfoCube can be a DataSource for another BW system.

l A BW system can be a source system for an APO system.

l An APO system can be a source system for a BW system.

n Two-way data exchange is possible.

l BW can be a source system for other SAP strategic

initiative products, such as SEM and CRM

Maintain Data Packet Size in Source System

Allows tuning of packet sizesl Optimal settings depend upon network configuration.

l Packet sizes between 20000 Kb and 50000 Kb are

recommended.

l Frequency represents the threshold to send statistical

information = 10.

l Max proc. is the maximum number of parallel extraction

processes. This setting depends on the number of CPUs and

configured BTC processes.

The recommended method is to maintain systems using the Transport Management System rather than doing a database copy.

If a BW system is copied, source systems and source system-dependent objects must be maintained.

Generally, a source system is deleted in the copied BW system, and a connection is established to a new source system.

Source-system specific objects need to be transported in order to avoid re-configuring manually.

System Copy Possibilities

BW Query Performance/Statistics

Statistics Overview

source

system

transfer

rule

InfoSource update

rule

infoCube query workbook role

Data flow Before Data flow Afterwards

SAP BW Metadata Visualized: Flow of Information

l Operations apply to

l Secondary F- and E-facttable indexes

l But not to user-defined indexes containing a letter, e.g. /BIC/FIUSALES~A12

l And not to primary indexes

l Check: checks status (mainly existence, index type)

l Delete: drops indexes

l Repair: re-creates non-existing indexes, coalesces existing indexes

How to get there:

l InfoCube

l Right mouse menu

l "Manage"

l Tabstrip "Performance"

Tool for Index and Statistical Maintenance

Tool for InfoCube Analysis

Transaction :RSRV

Tool to Check DB Parameter Settings

Aggregates

l Evaluation-oriented summary InfoCubes

l Strategies for building aggregates:

n Summarize on a characteristic

n Summarize on hierarchy levels

n Filter on specific characteristic values

l One query navigation step -> read InfoCube or aggregate

Aggregates - Example

Country Customer Sales

USA

Germany

USA

Austria

Austria

Germany

USA

Mega Soft Inc.

Ocean Networks

Funny Duds Inc.

Ocean Networks

Thor Industries

Funny Duds Inc.

Mega Soft Inc.

10

15

5

10

10

20

25

Fact Table: Sales Data Aggregate Tables: Sales Data

Country *

Country Sales

40

35

20

USA

Germany

Austria

Data for queries like ‘sales for all countries’, ‘sales in Germany’,

or ‘overall sales’ can be read out of the aggregate (country *).

Aggregates - Maintenance Switch

on/off

Show

aggregate

hierarchy

Activate

& Fill

unsave

d

change

s

Estimate of value

Aggregates - Facts in BW

Time-dependent navigational attributes

Hierarchy levels where the structure is time-dependent

Non-BasicCubes

Aggregatescan be created on

Aggregatescannot be created on

Dimension characteristics

Navigational attributes

Hierarchy levels of dimensional

characteristics

reduced retrieval

costs

costs of

aggregate

maintenance

Few vs. Many Aggregates

l tradeoff: aggregate maintenance vs. improved query performance

l BW provides tools to find that tradeoff BW statistics

Query-Analyzing Tool:

Query - Analyze

Query monitor - Transaction RSRT

Query - Analyze

Query-Analyzing Tool:

l Trace Tool - Transaction RSRTRACE

APO-Indexing-Specifics

possible additional secondary index on F facttable

– OSS note 363092

– supports APO-mass-update jobs

– index name contains a letter, e.g. /BIC/FIUSALES~A12

primary index for APO infocubes

– APO infocubes = infocubes with "BW application" APO

– APO cannot cope with duplicates in infocubes

– APO infocubes have a primary index on F facttable

unique (rather than non-unique) index on a dimension tables

– same reason as for primary index

l Operations apply to

l Secondary F- and E-facttable indexes

l But not to user-defined indexes containing a letter, e.g. /BIC/FIUSALES~A12

l And not to primary indexes

l Check: checks status (mainly existence, index type)

l Delete: drops indexes

l Repair: re-creates non-existing indexes, coalesces existing indexes

How to get there:

l InfoCube

l Right mouse menu

l "Manage"

l Tabstrip "Performance"

RSA1 - Admin Workbench

How to get there:

l Transaction RSRV

l Provide technical name of infocube

l Tabstrip database "indices ..."

l Analysis performs a check on

facttable and aggregate

indexes

l Results provides a detailed list

of the analysis

l Repair repairs the indexes

Tool to Handle Indexes-RSRV - BW Check Routines

DB02 - Missing Indexes

DB02 button "missing indexes"

if many facttable indexes are missing then ...

run ABAP reports (OSS note 157918):

– SAP_UPDATE_DBDIFF

– SAP_INFOCUBE_INDEXES_REPAIR

check DB02 again

Partitioning

MultiCubes

l conceptual level

l "high-level partitioning"

l e.g. basic infocubes by department

Table Partitioning

l database level

l "low-level partitioning"

l e.g. fact table range partitioned by month

BW Partitioning: Concepts

ConceptualPartitioning

TablePartitioning

MultiCube

Tables

BasicCubes

"North" "South"

Partitioning: Example

l Records added to InfoCube fact tables

have several “keys” which uniquely

identify the record.

l Request ID is just one of several fields in

a record that helps identify the data.

l But, Request ID can be removed, and

each record can still be uniquely

identified.

l Compression finds records which are

identical except for Request ID, then

deletes all but one of these records.

l If a compression is not performed, the

“Group by” condition of any query’s SQL

statement will remove duplicates. This

results in decreased query performance.

Compression of InfoCube Requests

Same scheduling and event

chain functionality available as

with aggregate rollup

Option to delete only requests older than

X number of days: provides ability to

prevent compression of recent requests

Zero Elimination option: delete records

where key figures are all zeros

InfoCube Compression

Size of each PSA partition,

value in number of records

Where? Enterprise IMG (transaction SPRO)

Business Information Warehouse > Links to Other

Systems > Maintain Control Parameters for the Data

Transfer

Set Threshold Value for PSA Partition

System Operation

Monitor for Extraction and Data Load

Choose appropriate

selection criteria to view

extraction / load status of

InfoPackages.

Use the Goto menu or the monitor icon

Yellow: extraction /

load not finished

Red: extraction /

load failed

Drill down on failed

InfoPackage for details

Green: successful data extraction / load

Monitor for Data Extraction and Load

green

yellow

red

l Status OK

l No errors occurred

l Data loaded

l Status indifferent

l Data update not finished (All IDocs)

l Incorrect number of data packages

(updated IDocs)

l Incorrect data request (scheduler)

l No data in source system (general view)

l Status not OK or

l Data loaded with error

l Indifferent status after time-out

l Data only booked into ALE incoming

IDoc

Data Loading Status Indicator in the Monitor

Source System (e.g. R/3)

Infocube: F fact table

Aggregates

Extraction

Rollup

Infocube Upload

(from PSA)

Infocube Upload (from ODS)

ODS Upload

Requestsexist

consolidated(no requests)

transactional

Infocube: E fact table

Compression

ODS Object

Persistant Staging Area (PSA)

Data Flow Summary

Start:

Confirm

InfoPackage

configuration

Monitor

Extraction &

load

Schedule

InfoPackage

DataPackages arrive into

ODS - data verification

possible

DataPackages

arrive into

InfoCube

Aggregate

Rollup

Compression of

old requests in

InfoCube

DB statistics

update

DataPackages

arrive into

PSA

Data extraction

from source

Typical Data Extraction / Load Cycle

Data Load Performance

Step 1

If a large quantity of transaction data should be uploaded into

the BW, the number range buffer should be increased for the

dimensions with a high expected number of rows.

Get the number range of all involved objects.

Buffer Number Range Objects


Change the Buffering of the Number Range ObjectsStep 2



Issue

l Existence of secondary indexes on fact tables slows data

load performance drastically.

l It requires that inserts are made into indexes as well as fact

table - much more work involved.

l Dropping secondary indexes prior to data load means that

inserts are made to fact table only.

l Indexes can be quickly rebuilt after data load is finished.

Delete Secondary Indexes Before Loading Data

Using InfoCube Data Load Performance Tools

Create Index button:

set automatic index

drop / rebuild

Statistics Structure button:

possibility to recalculate DB

statistics after a data load

Possible Settings for Indices and DB Statistics

Issue

l The master data load creates the SIDs to the key-

values and the navigational attribute values.

l This avoids your transaction data from containing

many characteristic attributes for which a new SID

must be determined.

l The insert on the characteristic table for loading the

transaction data is no longer necessary.

Load Master Data before Transaction Data

Issue

l When loading large quantities of data it is recommended

that you split the data into several files and take advantage

of multiple processors to improve performance.

l These files can then be loaded in parallel into the BW with

several requests.

l Prerequisites

l Multiple processors

l Quick Raid

Load Concurrent

l You can start several processes manually, e.g.

l InfoPackages

l Manually

l Via InfoPackage groups

l Creation of several aggregates

Manual Parallelization

l Several processes can be parallelized by the SAP

system, e.g.

l Data Packets when extracting from an SAP system

l Loading into PSA and data targets

l By the underlying database

l Partitioning

l Parallel query (Oracle)

Automatic Parallelization

l Extraction can be parallelized manually

l By starting several extraction jobs (InfoPackages) at the

same time

l These jobs should extract separate data sets

l Source system: via selection criteria (if possible)

l File: split up file

l InfoPackage groups don‘t extract in parallel for files

Manual Parallelization

BW sends request IDOC (RSRQST)RSAP_IDOC_DISPATCHER

Only in case of consistency problems:

Info IDOC (RSINFO): error occuredSubmit -> extraction in batch-process

consistency checksInfo status 5

Info IDOC (RSINFO): data selection schedulednew task / commit work

Info status 1

Info IDOC (RSINFO): data selection running, no. of records

Info status 2

Send data packets:

‘starting new task’

Info IDOC (RSINFO): data request received

Info status 0 or 5

...

Call extractor:

• initialization

• fetch dataData IDOC (RSSEND) or via ODS

Data packet 1new task / commit work

Data IDOC (RSSEND) or via ODS


new task / commit work

Data IDOC (RSSEND) or via ODS


Info IDOC (RSINFO): data selection ended

Info status 9

Parallel Processing For Transfer of Data (SAPI)

Parallel

Processes

Data Packets

/ Requests

Staging

Engine

Extraction

1

2

3

4

2

4

1

3

Data Packets / Requests can be loaded into an InfoCube in parallel

Loading into an InfoCube

B

W

S

E

R

V

E

R

User

Transfer Rules

OLAP Processor


Rules


Rules

Update RulesUpdate RulesUpdate Rules

PSAPSA

Update Rules

Transfer Rules

Transfer StructureData Source

(replicas)Transfer Structure

Data Sources

(user-defined)

Operational Data StoreObject Object

Data Flow Architecture Including PSA and ODS

Data Packets / Requests can NOT (yet) be loaded into an ODS object in parallel

(because of overwriting functionality)

Staging

Engine

2

4

1

3

ODSObject

Staging

Engine

2

4

1

3

1234

ODSObject

Loading into ODS

l PSA and data targets can be loaded in parallel

l Data is loaded into the PSA by the first work process,

start new task is done for data targets (for each data

packet)

PSA and Data Targets

l Size of the table for new data should not exceed 1 million

data records

l Initialization before deltas

l Avoiding SIDs

l DB partitioning on the table for active data

l Indexing

l Locking by parallel loading

ODS Data Load Performance

Issue

l The size of a data packet determines the granularity for the

database COMMITS and as a result can have an effect on

performance.

Set Package Size

Maintain entries in customizing table RSADMINC

Packet Size is in KB and determines the size of Idoc data

packages of data loads from flat file data sources

and data loads from BAPIs. A setting of 20000 is

the standard recommendation.

FrequencyStatus-IDOC is a ratio. The setting determines the

frequency that status Idocs are sent.

BW Setup: Maintain Data Transfer Parameters

Data Packet ODSObject

l Loading from PSA into a data target is always done

sequentially (for one request)

l One Data Packet is written sequentially into multiple

data targets

Sequential processing

Staging

Engine

2413

1234

ODS

Object

l ODS object might be one data target of a multiple

target InfoSource

l Sequential processing of data packets into the ODS

object forces the same for the InfoCube

Sequential Processing

l Use a predefined record length (ASCII file) when loading

from a file.

l If possible load the data from a file from the application

server and not from the client workstation.

Additional Data Load Tips

l Create InfoCube in proper tablespaces

(see note 156784).

l Run loads at off hours.

l Avoid large loads across a network.

l Avoid reading load files from tape.

(copy to disk first)

l Avoid placing input load files on the same disk drives or

controllers as the tables being loaded.

Additional Data Load Tips

Authorizations

BW Security

There are two different areas(Object Classes) in BW. One is the

administration and the other one is reporting. The reporting part is the

one differing most from R/3.

Administration –Object Class RS

-Concept very close to standard R/3

-All authorization relevant objects are delivered by SAP

-Administration of authorizations like in R/3

Reporting—Object Class RSR

-no authorization relevant object definition is delivered

-set of tools to define customer specified concept embedded in

SAP BW administration

Mark characteristics as "Authorization Relevant”

Create an Authorization Object for Reporting

(use Basic Settings -> Authorizations -> Reporting Objects)

Create Authorizations with the values3

2

1

Three Steps to Create Reporting Authorizations

Useful Transactions in BW

/nrsimg BW Customizing Implementation Guide

/nsale ALE Communication

/nbale Checking IDOCs

/nwedi IDOC Protocols

/nse53 Check Permission of active User

/nrsd1 View Info Objects

/nse38 ABAP Workbench => LRREXU01 BW Exit

/nse37 Function Builder => RSD_CUBE_GET InfoCubes

/nrszv Variable Maintenance

/nrsrtReporting Monitor, trace tool - find aggregates

/nrsa1 Administrator Workbench

/nrsrv Checking InfoCubes, Cube BW Statistics

/nrsddv Aggregate Maintenance

/nsnro Number Range Maintenance

/nlistcube, listschema

l /nsm21 Systemlog

l /nst22 ABAP Shortdumps

l /nsm50 Work Process Overview

l /nsm59 RFC Destinations

l /nsm65 Analyze Batchjob Server

l /nst02 Setup/Tune Buffers

l /nst03 Workload Statistics

l /nst04 Database Monitor

l /nst06 Operating System Monitor

l /nse11 View SAP Tables

l /nse38 ABAP Workbench => RMCSBIWC InfoSource R/3

l /ndb20 New Optimizer Statistics for single Object

l /ndb21 Extended Table Maintenance for DBSTATC

l /nrz06 Alerts/Thresholds

l /nrz20 SAP Performance Monitore

Monitoring Transactions in R/3 and BW

Documents

Train-BW