02 01 Pc Architecture&Func

1

SAP AG 2005, BW System Administration / 1

BW Systemadministration – Portfolio

Extraction and Data Transfer

Data Load Monitoring, Reload and Repair

ODS and Datamart Handling

Process ChainModelling Aspects

Process ChainArchitecture & Functionality

Process ChainMonitoring & Error Handling

BW Transport Management System

Periodical jobs and tasks

Monitoring Changerun

Load Balancing & Number Range Buffering

BW Statistics and Query Tuning with Aggregates(RSRT)

Data Modelling from thetechnical point of view

Usage of Basis Transactions(SM50, …) for applicational people

2


Process Chain Architecture & Functionality

First steps & Basics

Specific settings

Background Processing & Load Distribution to different Servers

Implementing new Processes

Tips & Tricks

Transporting Process Chains

3


Introduction: Typical Data Load Cycle

Data Load Data Load Data Load Data Load

MonitorMonitorMonitorMonitor

Data Target Data Target Data Target Data Target

MaintenanceMaintenanceMaintenanceMaintenance

Start

Load into PSA Load into ODS

Activate Data in ODS

Load into Cube

Build Indices

Build DB Statistics

Roll up Aggregates

Drop Indices

4


Motivation

Automating the BW administrationAutomating the BW administrationAutomating the BW administrationAutomating the BW administration

Graphical modelling and monitoringGraphical modelling and monitoringGraphical modelling and monitoringGraphical modelling and monitoring

Central Central Central Central monitoringmonitoringmonitoringmonitoring of all BW of all BW of all BW of all BW processesprocessesprocessesprocesses

Openness for all kinds of processesOpenness for all kinds of processesOpenness for all kinds of processesOpenness for all kinds of processes

Moving responsibility into the individual processesMoving responsibility into the individual processesMoving responsibility into the individual processesMoving responsibility into the individual processes

5


3 Different Views to Process Chains

Planning view forchecking the plan status of theprocess chain

Checking view forcheck consistencyof the processchain

Log View formonitoringmaintenance activitiesand executions

Different views to thethe activities whichare possible:� Process chains� Process types� Data targets� InfoSources� Logs

workingarea

Views related to the working area

There are three main views in the icon bar with different views:

Planning view (shows if the process chain is active)

- Grey: unplanned processes (e.g. not activated process chain)

- Green: planned processes (process chain is active and start process is released)

- Yellow: planned but unknown processes

- Red: multiple planned processes

Checking view (consistency check like double used start variants, missing index deletion and recreation, wrong references in variants, ...)

- Green: Error-free processes

- Yellow: Process with warnings

- Red: Process with errors

Log view

- Grey: Not yet run

- Green: Finished without error

- Yellow: running

- Red: aborted or failed

note that the log information is a usually mixture of

- log information of maintenance activites (e.g. new process, change of design,activation)

6


Different Object Trees for Process Chain Administration

Available Process Types

FFFFor easy administration on process chains different object trees can be displayed.

Display component

Processchain

Folders in the process chain are called display components.

For mainentance of display components you have to use process chains � attributes �display components.

For reassignment of process chain to different display component

For creation of new display component

7


Different object trees when building process chains

For easy administration on process chains different object trees can be displayed. Log

Display the log treeInfoAreas

Search in InfoProvider tree

InfoSources

Search in InfoSources tree for InfoPackages

Possible processes on data target are displayed

In InfoPackage reference to process chain is displayed

8


Application Processes

Application processes represent BW activities that are typicallyperformed as part of BW operations. Examples inclu de:

� Data load

� Attribute/Hierarchy Change run

� Aggregate rollup

� Reporting Agent Settings

Other special types of application processes exist:

� Starter – process that exists to initiate process ch ain execution

� ABAP program

� Another process chain

� Customer built process

A starter process is part of every process chain.

9


Application Process Types

Operating system command

� Access to OS commands like ‘dir’ (list contents of application server file system)

� Note: There is only an call “out” to OS, only Standar d-Out is given back to BW

Remote process chains

� Supports process chains in a data mart environment

Master data reorganization

Reporting Agent Settings

� Exception reporting

� Print queries in background

� Pre-calculating web templates

� Pre-calculating characteristic variables of type pr e-calculated value sets

10


Collector Processes

CCCCollectors are used to manage multiple processes tha t feed into the same subsequent process. The collectors availab le for BW are:� AND: All of the processes that are direct predecessors must send

an event in order for subsequent processes to be ex ecuted

� OR: A least one predecessor process must send an even t

� The first predecessor process that sends an event t riggers the subsequent process

� EXOR: Exclusive “OR”

� Similar to regular “OR”, but there is only ONE execut ion of the successor processes, even if several predecessor pr ocesses raise an event

Collector processes allow the designer of a process chain to trigger a subsequent process based on whether certain conditions are met by multiple predecessor processes.

Application processes are the other type of processes – these represent BW activities such as aggregate rollup, etc.

Although the “AND” condition is implemented for process chains using the event chain functionality from 2.x, this event chain is internal and cannot be edited.

11


Start

TR

Load Data

TR Texts

Attrib. Change

TR Characteristics

RSPC User Interface: Building a Process Chain

1. Drag and drop

2. Draw line to connect processes

Load Process and Subsequent Processing

Data Loading Process

Read PSA and update data target

Save Hierarchy

Further Processing of ODS Object Data

Data Export into External Systems

Delete Overlapping Requests from InfoCube

Data Target Administration

Delete Index

Generate Index

Construct Database Statistics

Initial Fill of New Aggregates

Roll Up of Filled Aggregates

Compression of the InfoCube

Activate ODS Object Data

Complete Deletion of Data Target Contents

Other BW Processes

Attribute Change Run

Adjustment of Time-Dependent Aggregates

Deletion of Requests from PSA

General Services

Start Process

AND (Last)

OR (Each)

EXOR (First)

3. Press ‚Refresh‘

When you draw a line to connect processes, you are prompted to indicate whether the subsequent process should execute based on the success or the failure of the predecessor process. In other words, it is possible to schedule a process to run only if the predecessor process fails.

12


Start Process

When and how process chain will be scheduled

Variant

Note:Note:Note:Note: For each process chain the start process is mandato ry. It is not possible to use the same variant more than o nce !!!

The process variant could be used just by one process chain.

Each process chain could be started manually or via RFC connection with the function module ‚RSPC_API_CHAIN_START‘ .

13


Structure of a process – Example on variant

Maintenance of variant is specific for every proces s type

A variant is the configuration of a process of a pa rticular type set at the time of definition (e.g. the variant for a d ata load process is an InfoPackage)

Process = process type + variant

Attribute Change Run

ps_attrib2_long

PA_ATTRIB2

Execute InfoPackage: ZPAK_4QVBBF65GZONPAP96APH2P3KQ

Type – Kind of task

Variant – Configuration

14


Structure of a process – Example of process type

Type –––– Attribute Change Run� Execute the hierarchy and attribute change run

� Process types can be maintained via Settings – Proce ss Types� Do not change standard process types (if allowed)

If standard process types can be changed depends on the changeability of name ranges (transaction SE03)

15


Structure of a process – Example of variant

Variant – Configure the Hierarchy and Attribute Change run� Execute the specific hierarchy and attribute change run for which

hierarchy, which InfoObject or which data loading p rocess

16


Process Types – Example Execute InfoPackage

17


[email protected]

Maintain Message

Planning view context menu

Write a message and fill in recipient and type. Info

saved within process variant.

Send with Note

[email protected]

Internet Mail Access has to be configured (with SAPCONNECT) for sending Emails from SAP-System to the Internet via SMTP.

18




Specific settings



Tips & Tricks


19


Automatic Insertion of Corresponding Process Types

ExampleExampleExampleExample: You drag and drop a data load process to your process chain, the Index drop and the Index create process are automatically inserted.

IIIIf you want no corresponding processes to be inserted automatically, flag the (user specific) setting in the menu under Settings �� Default Chains

IIIIf a process is inserted into the process chain the corresponding process variants are inserted into the process chain automa tically:

20


Polling Flag

WWWWith setting the polling flag you can control the behavio r of the main process when you have distributed processes (Pr ocessChain �� Attributes �� Polling).

TTTThe main job run all the time in parallel (flag is set) or is onlytriggered by a finished process of the process chain (flag is notset: default).

With setting the polling flag you can control the behavior of the main process when you have distributed processes. Distributed processes, such as the loading process, are characterized as having different work processes involved in specific tasks.

With the polling flag you determine whether the main process needs to be kept as long as the actual process has ended.

Setting the indicator has the following advantages:

A high level of process security is guaranteed

External scheduling tools that react only to the SAP internal event 'Batch Process Complete', are also informed about the status of distributed processes.

Setting the indicator also has the following disadvantages:

Increased demand on resources. Although the CPU is not under pressure during the waiting time, it is during the status checks that run at two minute intervals.

You require one more background process.

21


WAIT Time ...

Note:Note:Note:Note:

TTTThis wait time is just for Debugging purposes, in order to have some time to catch a process for debugging in SM50!

DDDDon‘t use this feature to getting somewait time for a productive processchain.

RRRReason:Sometimes there are alos otherprocesses influenced by this wait time.

22


WAIT Process

SSSSometimes it is necessary to have some wait time before the s ubsequent processis started ...

23


Background Server

WWWWith this setting you can assign this process chain (especial lythe Process BI_PROCESS_TRIGGER to a specific backgroun dserver.

DDDDue to the fact that the different processes of this chain a rescheduled via event, they are usually started also on th is server.Only if there are to less background processes they aredistributed to other servers.

Note:

This setting is just valid, if theprocess chain is not started directly, but as sub chain!

For the case that you start the process chain with directly the the setting for background server is given as proposal, when you execute the process chain.

But you can change the background server within this step.

Please refer to notes:

604448 Collection note on job management

24092 Distribution of background jobs on application servers

519059 FAQ: Background processing system

24




Specific settings



Tips & Tricks


25


Event triggered

Execution of a direct scheduled process chain

Start processchain

Time scheduledAll processes except theBI_PROCESS_TRIGGER are eventscheduledAll other processes are event triggered

After process chain is started, the differentprocesses are scheduled and released.

For subchains just process‚BI_PROCESS_CHAIN‘ is released.

26


General Rules for Time Scheduled Jobs

DDDDescription of Time scheduled Jobs

� On each SAP Instance with BTC processes a system program runsperiodically (rdisp/btctime=(60)) and picks up as muc h jobs from the‚time scheduling‘ queue until all the BTC processes of t he Instanceare occupied

� After installation: periodicity of the time scheduler is et to 60s on eachSAP Instance. This means there is a fixed realationship f or the pickuptimes between the different SAP Instances.

Result :

� one instance is usually the favourite execution server

Recommendation:� Set rdisp/btctime on the different SAP Instances to di fferent values,

like 59, 60,61,.... With this setting there is no fixed relationship for the pic kup timesanymore

Recommendation is just useful for SAP Systems with more than one SAP Instance

27


General Rules for Event Triggered Jobs (1)

DDDDescription of Event Triggered Jobs

� When an event is triggered, the system checks the SAP para meterrdisp/btcname on the actual SAP Instance and starts the e vent triggerprogram on the Instance which is specified with this parame ter(default: Central Instance). After this the event sche duler tries to start the Job immediately on this Instance or in case of that al l BTC processes occupied, the Job is converted to ‚time sched uled‘

� If the Job is converted to time scheduled the mechanism fo r time scheduled jobs is working (see slide before!)

Result :

� If rdisp/btcname is the same on all Instances and there ar e sufficientBTC processes running on this Instance, this Instance wi ll takealmost all event triggered BTC Jobs (this means no load d istributionfor Process Chain Jobs!)

28



CaseCaseCaseCase 1: 1: 1: 1: TypicallyTypicallyTypicallyTypically forforforfor moremoremoremore thanthanthanthan 2 SAP 2 SAP 2 SAP 2 SAP InstancesInstancesInstancesInstances

Recommendation:� Use one btcname – Server for all Instances. This BTC name server

(rdisp/btcname) must have less BTC processes (e.g. 2 - 4 ), so thatusually these BTC processes are not sufficient for all job s which aretriggered by event -> conversion to time scheduled takes place

� Set rdisp/btctime on the different SAP Instances to di fferent values, like 59, 60,61,.... With this setting you get an almost equal distribution of B TC Jobs over all other servers except rdisp/btcname

� No Backgound Server have to be assigned to the START pro cesses of the different process chains.

� in order to minimize delay times: rdisp/btctime could b e set to 30,31,32, …, but not smaller!

Disadvantage:� Delay times are occuring for event triggered jobs , but the m ore

Instances the less delay times.

29



CaseCaseCaseCase 2: 2 SAP 2: 2 SAP 2: 2 SAP 2: 2 SAP InstancesInstancesInstancesInstances

Recommendation:� Assign for rdisp/btcname in the Instance profile the own In stance

name. Therefore you get different settings for this param eter on different Instances. After this you have to assign dedi cated servernames for the START processes of the process chains. If th ere aresufficient BTC processes are configured on the Instances the entireprocess chain runs on the same Instance.

� Set rdisp/btctime on the different SAP Instances to di fferent values, like 59, 60, ...

Disadvantage:

� Unflexible and getting optimal Job distribution on th e different servers is more difficult.

30


Goal: Equal distribution of BTC Jobs over all servers

RZ01:

31


Dedicated server for a direct scheduled processchain

After starting a process chain you are able to assign a dedicated server for the Job ‚BI_TRIGGER_PROCESS‘

DDDDue to the fact that the other processes of the chain areevent triggered, they are usually running on the serverwhich is determined with the SAP Parameter /rdisp/btc_name (instance specific!) (see note 24092)

604448 Collection note on job management

24092 Distribution of background jobs on application servers

519059 FAQ: Background processing system

700479 Load balancing of background jobs does not work

636854 Jobs remain in the 'Ready' status for a long time

32


Number of required BTC Processes

If no subchains are used there is no special requirement to thenumber of BTC work processes.

If subchain are used there is a minimum number of BTC workprocesses necessary in order to avoid dead lock situations

� # BTC (minimum) = number of parallel starting subchains at the widest part of the chain + 1

� details: see note 621400

If complex chains (also in parallel) are executed, you c an improve theoverall runtime of the process chains, if there are suffic ient BTC processes configured, in order to avoid unnecessary de lay times.Sometimes it makes sense to configure more than 2 BTC proc essesper CPU.

rough calculation for the number of BTC processes in order to get best performance and avoid delay times in the batch queue:

# BTC (optimal) = number of parallel processes at the widest part of the chain + number of parallel subchains at the widest part + 1

33



Widest part:

2 subchains

3 processes

Minimum mumberof BTC processes:

2 + 1 = 3

Optimal number of BTC Processes:

3 + 2 + 1 = 6

34



SM37: Job Overview: active jobs

SM50: Workprocess Overview:

5 BTC configured

Note:Note:Note:Note:

BI_Process_Trigger for the second subchain could not be started immediately , because there are just 5 BTC configuredon the SAP Application Server

meaning of this slide

•there are 5 BTC work processes configured for the BW System (in this casejust Central Instance exists)

•After execution of the process chain all BTC processes areallocated

•BI_Process_Trigger for the Process Chain itself. This process startssubsequent processes and finishes after the subsequent processes aresending status information back to BI_Process_Trigger

•PI_Process_Chain processes are starting subsequently theBI_Process_Trigger jobs of their assigned local process chains.

• Due to the fact that the there are just 5 BTC configured on theSAP Instance, there is one free BTC process missing forBI_Process_Trigger of the second subchain.This job is still waiting in the batch queue for free resources on this orfurther SAP Instances.

35


Subchains

If you have a complex structure of subchains which are runni ng in parallel, you have to provide a minimum number of BTC processes:# BTC (minimum) = number of parallel subchains at the widest part of the chain + 1

Otherwise you sometimes get a Deadlock situation……

SM 37: Job Overview SM50: Configuration with 3 BTC processes

All BGD processesare blocked by‚BI_PROCESS_CHAIN‘

Please also refer to the note: 621400

36


Number of BTC processes

� If process chains are used intensively and complexscenarios with lots of subchains are designed, then asufficient number of BTC processes shall be configured

� Rule of thumb: #BTC = 2 * #CPUs not valid in all casesanymore

� In specific cases the runtime for process chains couldbe optimized by increasing number of BTC processes(e.g. process chains with many parallel running masterdata loads)

37


Optimize Load distribution

AAAApply different parameter values for rdisp/btctime on dif ferent servers (e.g. 59, 60, 61) (see note 519059)� Each server has its own batch scheduler, which checks peri odically

after rdisp/btctime the central BTC Queue for waiting BTC jobs

� Event triggered and immediately started jobs are usually not startedview batch scheduler

IIIIf subchains are used, a different background server can b eassigned to the subchains with attribute ‚background serv er‘ in the maintenance view� Usually the processes within one process chain are running o n the

same server (because they are event-scheduled!)

BBBBTC process definition on the application server

� General rule: don‘t apply too much BTC work processes for an SAP instance (e.g. 1-2 BTC per CPU)

rdisp/btctime: Start Interval for Background Scheduler

Specifies the frequency in seconds with which the background processing job scheduler is to be started in a server which can run jobs in the background.

38


Remark: General Rules for immediately scheduledJobs

UUUUsually immediately scheduled Jobs are not used in BW, bes idesexceptional cases like restart of process chains, ...

DDDDescription of Immediately scheduled Jobs

� default setting: Jobs that have the 'Immediate star t' start condition and no target server are not distributed consistent ly on the different batch servers. The jobs mostly run on the same serv er

RRRResult :

� The jobs mostly run on the same server

RRRRecommendation:

� apply note 700479 in order to get a round robin procedur e

39




Specific settings



Tips & Tricks


40


Process Chains: Transport

Process chains are transportable� Button for writing to a change request

when maintaining a process chain in RSPC

� Process chains available in the transport connection wizard (administrator workbench)

� Tlogo Object R3TR RSPC

41


Transport – Import Post-Processing Destination

42


Transport - Changeability

Note 337950 “Queries (and other objects) cannot be edited”

This setting is only intended for systems in

production or test systems

that are set to not changeable !

This option allows to deactivate the Change and Transport System forspecific BW components

43




Specific settings



Tips & Tricks


44


Consideration of automatisms in process chains

The following loading process automatisms areignored within process chains:

� Flag for automatic ODS-activation in ODS-maintenance

� Flags for automatic deletion/creation of infocube indexes/statistics(Performance-Tabstrip in Infocube � Manage)

� Flags for automatic Infocube Content Deletion or Deletion of overlapping requests in infopackage

� Flag in Infopackage: PSA + update subsequently in datatargets

� There are differences between manualexecution and execution in process chain

� For all these cases there are own process typesto get the same functionality and to avoidinterdependency / deadlock problems.

45


Consideration of automatisms in process chains

These flags aren‘tprocessed within a

Process Chain !

Instead of this:Process Type „Read PSA and Update Data Targets“

Instead there is:Process Type „Delete

Overlapping Requests fromInfoCube“

Instead of this:Process Type „CompleteDeletion of Data Target

Contents“

Only the BW Quality flag is processed, but it is a prerequisite for ProcessChains that this flag is set, otherwise unknown kinds of error messagescan be displayed…

46


Start Process

Direct scheduling:Job BI_PROCESS_TRIGGER will bescheduled when the process chain

is executed.

Start using Meta Chain of API:No BI_PROCESS_TRIGGER will be scheduled.

Start of the process chain have to be donevia FM ‚RSPC_API_CHAIN_START‘

orwith another process chain

Variant name and

description

47


Manual Start of Process Chain

Technical Name of process chain

SSSSometimes you want to start a process chain immediately.

YYYYou don‘t want to change thevariant of the start process.

Function Builder: SE37

Each process chain could be startedalwaysmanually or via RFC connection with thefunction module‚RSPC_API_CHAIN_START‘ .

48


Display Component

Folderstructuremissing!

Apply yourown displaycomponents

Typical situation: Proposal:

49


Restartability of Process Types

FFFFor each process type (self definedor standard) re-starting is a featurewhich is set in table: RSPROCESSTYPES

50


Copying Process Chain – Step 1 + 2

Transaction Code: ====copycopycopycopy

Assign new name:

51


Copying Process Chain – Step 3 + 4

Exchange start variant:

Assign new unused start variant:

52


Maintain Process Chains – Detail View On

Note:Note:Note:Note: In detail view, a collector process is displayed as multiple collector processes (needed for conditions)

RSPC -> View -> Detail View On:� Technical names

� Ability to move boxes to re-design the process chain

� Hidden collector processes are displayed

When collector processes are built into a process chain, there are actually several background jobs scheduled with events in order to construct the conditional nature of collector processes.

The simple view displays the processing chain as it exists logically. The detail view displays the processing chain with the extra collector processes.

53


� Available as process type in process chains

� Minimize table size: improve performance� Attribute/Hierarchy Change Run

� Query performance

� Often required in HR

� Two features supported:� Compressing time intervals for time-dependent maste r data

� Compressing attributes with same A- and M-version

Master Data Reorganization

54


Compressing time intervals for time-dependent MD� Combining records with same attributes and adjacent time intervals

01/2001 11/2001 01/2001 11/2001

Development manager

Development manager Development manager

Reorganization

2 records 1 record

Example: An employee‘s position (time-dependent master data)

Master Data Reorganization - I

55


Compressing attributes with same A- and M-version� Deletion of equal A- and M-version of master data re cords

� Scenario:�Active data (A-version) already exists

� Load of new (different) data �� M-version inserted

�No activation of loaded master data� Load of new data again which equals A-version

blue

Example: Master data for article 4711 (attribute: colo ur)

time

blue

green

blue

blue

no activation

blue

Reo

rgan

izat

ion

1st upload 2nd upload

active data

modified data

Master Data Reorganization - II

56




Specific settings



Tips & Tricks


57


Appendix: Openness: Implementing a Process

If you...� ...Have a simple, stand alone program to run in a c hain

� ...Have a program, which is planned in batch by ano ther program or user

...then you can use the process type “ABAP” to run your program in a chain.

However if you...� ...Want to ask predecessor processes of your progra m

� ...Write a log , which should be displayed in the p rocess monitor

� ...Have implemented an own monitor

� ...Have a customizing screen, which is more difficu lt than the “parameters“ of a program

...then it is recommended to custom develop own pro cess type

58


Appendix: Implementing a Custom Designed Process

Interfaces: IF_RSPC_*� EXECUTE – Execution of the process

� GET_INFO – Give Information to successors

� GET_LOG – Give back messages

� MAINTAIN – Maintenance of variants

� GET_VARIANT – F4 for variants

� GET_DEFAULT_CHAIN – Give default-chain for a process

� CHECK – Check consisteny

� GET_STATUS – Give status of instance

� CALL_MONITOR – own monitoring tool

� TRANSPORT – give back TLOGO-entry

� CONTEXT_MENU – Enhance context-menu

Make entry in view RSPROCESSTYPES

The interfaces are documented in the system

59


Appendix: Implementing a Custom Designed Process

XXTRANSPORT

XXCONTEXT_MENU

XXCALL_MONITOR

XXXGET_STATUS

XXCHECK

XXGET_DEFAULT_CHAIN

XXGET_VARIANT

XXMAINTAIN

XXXGET_LOG

XXXGET_INFO

XXEXECUTE

optionalalterna-

tively

manda-

tory

Log View

RuntimeMain-

tainance

Interfaces:Implementing is...Method called at...

means:

e.g.:

Interface: IF_RSPC_EXECUTE is executed at runtime of the Process Chain. Usage of this Interface within your self defined process type is mandatory

Process Type ABAP just uses IF_RSPC_EXECUTE and IF_RSPC_GET_Variant. No status is given back.

e.g.:

Interface: IF_RSPC_Get_Status is executed at runtime and when startig log view. It is not mandatory to use this interface within your own process type.

60


Appendix: Generic Services

There are several services in Paket RSPC which shall help you implementing a process type:

� Generic variant storage class: CL_RSPC_VARIANT

� Generic dynpro service for variantmaintainance: Function moduleRSPC_VARIANT_MAINTAIN

� Generic instance storage class: CL_RSPC_INSTANCE

� Wrapper class for application log forunambigous assignment of logs to instance CL_RSPC_APPL_LOG

� Service function group: RSPC_SERVICES

Documents

02 01 Pc Architecture&Func