Advanced Rules & Feeders

Outline•Rules Basics

•Feeders Completeness

•Performance

•Rules vs. TI

1 Rule Basics

TM1 Calculations

Cell Calculations

View Calculations

Feeders Basics

1.1 TM1 Calculations

Calculations are On Request and Recursive

Cell by Cell Calculation

View Calculation

Intermediate results are stored for re-use.

1.1 TM1 Calculations (cont’d)

Why Calculations on Demand: Real Time Performance.

OLAP Geometric Explosion - One change affects many cells.

Example: 10 Dimensions, 3 Levels each. Change one leaf cell affects 310 -1 calculated cells = 59,048.

In-memory architecture and special algorithms permit fast last-minute calculations.

Only relevant cells are calculated.

1.2 Cell Calculations

Get Cell

Simple Cell

Rule

Interpret Expression

Get CellsGet Value from Cube

Done

Calculate Expression

Consolidation

Go through nonempty cells

Get Cells

Accumulate

Determine Cell Type

1.3 Views Calculations

View consolidations are calculated “wholesale” using an algorithm called “Stargate”.

Performance ~ n times faster than cell by cell, where n is the number of cells. Get n for ~ the price of 1.

1.3 Rules in Views

When rule

Is evaluated in the context of a view

Requests a consolidated cell

Cell some cell members occur in the axes of the view.

Then

Consolidation is done with Stargate and result saved for use by other cells in view.

1.3 Storage of Views

VMT and VMM control storage of views.

When view requested is included in stored view, stored view is used.

When view requested overlaps stored view, the axes are “joined” and a new expanded view is constructed and stored.

1.4 Feeders Basics

Consolidation speed derives from “skipping” empty areas of cube.

In most applications “density” is < 10-6.

Thus need to “tag” cells that have values calculated by rules.

Otherwise calculation could take more than 1 million times longer. The price of forgetting SKIPCHECK!

1.4 Feeders Basics – Q&A

Are there any effects of feeders if SKIPCHEK is forgotten?

No, feeders are still done, but ignored. Instead every cell is evaluated.

Would that happen if feeder and rules are in different cubes?

Same

Even an empty rule file switches sparsity algorithm off?

Yes - VERY IMPORTANT

1.4 Feeders Basics (cont’d)

Feeder statements have the form:TriggerRange => Target.

Feeders are triggered by data input to an empty cell

If A = k * B - Feed A from B.

If A = B +C - Feed from B AND C

If A = B * C - Feed from either B OR C

Feeders propagate recursively.

1.4 Feeders Basics – Q&A

• Concerning B*C, should you feed from B or from C? – from whichever is more likely to be zero.

• If Cube X feeds Y, and Y feeds Z, would it make sense to feed Z directly from X

– No, because rules will be hard to understand and maintain, and you still have to feed Y

• If what if X, Y and Z are portions of the same cube? – Answer unchanged

• If Y has rules that use values from X, where are feeders written?

– always in the source cube, i.e., in X.

1.4 Feeders Basics (Cont’d)

Zero Suppression

Feeders are necessary for proper zero suppression.

Even if you do not use SKIPCHECK, zero suppression will not work without feeders.

Feeding Strings

String cells are normally not fed.

Must use FEEDSTRINGS statement to enable.

1.4 Feeders Basics (Cont’d)

Feeding from Strings

Feeders from string cells re-fire whenever value changes.

Re-fire does not propagate – if target cell is already fed (string or numeric) it will not fire.

Watch out when feeding from strings – Entering a string at a consolidated level will cause overfeeding.

1.4 Feeders Basics (Cont’d)

Feeding from and to consolidations

To consolidation: feed all leaf components. Can result in overfeeding.

From consolidation: feed from each leaf component.

1.4 Feeders Basics - Case

Q. What if Y uses values from X, and the time dimension in X (Month) goes down to months, but the one in Y (Year) only down to years?

A. The months in X must feed the corresponding years in Y. However, you cannot write !Year on the right had side. You must use ELPAR or preferably an attribute that maps month to year.

2 Feeders Completeness

Locating Missing Feeders

String-Driven Feeders

2.1 Locating Missing Feeders

Using Rules Tracer

Needle in a haystack. Must examine every cell.

Judiciously look at subareas.

2.2 String Driven Feeders

Strings parameters are often used to direct calculations an feeders.

Care must be used to define feeders so they re-trigger when the string parameters are changed.

2.2 String Driven Feeders – A common trap

Feeders are often written in such a way that the one of the elements in the target (written using a db() expression) are specified by a string stored in some cube.TriggerRange => DB( …, DB(stringcubecell, ...), …)

The problem with this is that if the string in question is changed, the feeder will not re-fire because the cells in TriggerRange are already populated. And the consolidations will be WRONG.

To correct the situation, the trigger of the feeder must use the string in question which, as mentioned before, will fire whenever the string is changed.

3. Performance

Missing SKIPCHEKs

Overfeeding

Other

3.1 Missing Skipchecks

Symptom: Very slow performance

Approach:

Check all rule files and put them in.

Put in appropriate feeders

An empty rule will cause sparse consolidations to be turned off and slow performance

3.2 Overfeeding• Why worry about overfeeding?

• Locating Overfeeds

• Conditional Feeders

3.2.1 Why worry about overfeeding?

• Slow cube loading.

• Unnecessary memory consumption.

• Slow Performance.

Which is the most significant?

3.2.2 Technique for locating Overfeeds

Create cube OVERFEED with the same dimensions as subject cube.

Add rule to OVERFEED as follows:SKIPCHECK;

[]=N:IF(DB(’01Basic’,!dim1 , !dim2, ...)=0,1,0);

Add feeder to subject cube as follows:[]=>DB(‘OVERFEED’, !dim1 , !dim2, ...);

3.2.3.1 Conditional Feeders Basics

Purpose - to minimize number of “overfeeds”.

Trick: Write if expression for the target cube name that yields a blank when feeders are NOT to take place.

Classic example:

A = B * C

B feeds A if C is nonzero

C feeds A if B is nonzero

3.2.3.2 Different Dimensionalities

= *

Conditional

AxB A B

Baux

3.2.3.2 Different Dimensionalities - ExampleTo Calculate:

Commission( Product, SalesPerson, Time) = Sales( Product, SalesPerson, Time) * CommissionRate( Product, Time)

Create CommissionRateAux(Product, SalesPerson, Time)

In CommissionRate:[] => DB(‘CommissionRateAux’, !Product, ‘All Salespersons’, !Time);

In CommissionRateAux:[] => DB( IF( DB(‘Sales’, !Product, !SalesPerson, !Time) = 0, ‘’,‘Commission’), !Product, !SalesPerson, !Time);

In Sales:[] => DB( IF( DB(‘CommissionRate’, !Product, !Time) = 0. ‘’,‘Commission’), !Product, !SalesPerson, !Time);

3.3 Other Methods

• Avoiding Long Expressions

• Avoiding Ratio Feeders

• Avoiding Matrix Multiplication Feeders

• Freezing Rules

• Miscellaneous

3.3.1 Avoiding Long Expressions

Complex expression (with identical data) used for multiple cells - Calculation is done over and over.

Approach: “Factor out” by defining calculation as cells in auxiliary cube.

3.3.2 Avoiding Ratio FeedersRatio calculations such as:

[‘Sales Percent of Year'] = ['Sales']\['Year','Sales']*100;

apply at all levels of hierarchy. I.e., rule is not preceded by N: or C:

Therefore no feeders are needed for consolidations because leaf values are never consolidated. Problem is, how to support zero suppression.

Answer: Define “Sales Percent of Year” as a consolidation of “Sales”. This effectively acts as a feeder from “Sales”, but costs no memory.

3.3.3 Avoiding Matrix Multiplication Feeders

= *AxB A B

d2

d1d3

d2

d3

d1

d2

3.3.3 Case– “Multicurrency” Company

Large reporting cube

Need all data in 12 currencies

Long time to load

Hardware memory limits exceeded

Client not happy

3.3.3 Approach

Currency rate varies only with TIME - not with other dimensions.

Can express currency rule for both “C” and “N” cells.

All consolidations would be right except TIME.

Express TIME consolidation using rules (ConsolidateChildren)

No feeders needed.

But need to express calculated currencies as consolidation of base currency in order to support zero suppression.

3.3.4 Freezing Rules

One way to improve performance of a TM1 application is to “freeze” rule calculated values for periods or versions that are not subject to change. Typically history or frozen scenarios.

This way, the cells thus frozen need not be recalculated when other values in the same cube are updated.

3.3.4 Freezing Rules - Method

User selects a cube, a dimension within the cube, and leaf members in the dimension to be frozen.

The leaf values for those members are extracted and output to an ASCII file, including leaf values calculated by rules.

The rules that apply to leaf cells are deactivated for the members frozen.

The values that were output to the ASCII file are re- imported into the cube.

3.3.4 Freezing Rules - Continued

In order to disable the rules for the frozen members automatically, rather than editing the rule file, do as follows:

Put disabled members in a parallel dimension.

Put rules that do not apply only to leaf cells at the beginning.

Follow it by a conditional expression of the form:

[] = IF( DIMIX(‘Dimx_Freeze', !Dimx) =0, CONTINUE, STET);

One for each dimension with frozen members.

3.3.5 Miscellaneous

Inadequate optimization of consolidations in rules (rare cases).

Rules not taking full advantage of “Stargate”.

Can easily make performance worse by a factor of 10.

Report to support / development.

Rules TIEase of writing / maintaining + Declarative expression more

intuitive for user.- Feeders a challenge for all.- Limited ability for complex calculations.

+ Procedural calculations more familiar to developers.- Requires additional setup to trigger calculations.+ Able to handle iterative calculations.

Error proneness -Locating incomplete feeders-Value-directed feeders

- User must remember to trigger.+ Procedural calculations easier to test and fix.

Performance + Only necessary cells are calculated- Invalidated when values change.O Calculation speed comparable

- Must calculate all cells. No trigger based on change.O Calculation speed comparable

Dynamism + Dynamic - always up to date - Not Dynamic

Memory O No significant difference O No significant difference

4 Rules vs. TI

Thank You

Advanced Rules and Feeders