View
104
Download
3
Category
Preview:
DESCRIPTION
Essbase Calculation
Citation preview
459A
Calc Scripts for Mere Mortals
Presented by:Edward Roske, interRel Consulting
Agenda
Review the Basics Calc Script Commands Using Functions Controlling Execution Creating Blocks Organizing Calc Scripts Real World Calculations Optimization
Types Of Calculations
Outline consolidations Unary operators + - * / % Relies on hierarchy
Member formulas Allows complex calculations Can use functions and conditional processing
Calculation scripts Override outline operations Offer most control
Calculation Order
Two basic categories Outline order
Dense Sparse
Dimension order
Calculation Order All Dimensions First, Accounts Second, Time Third, all remaining dense dimensions Forth, all remaining sparse dimensions Last, Two-pass calculations
Calc Scripts
Reasons to use calc scripts: Override outline operators Specific calculation commands available Iterative calculations Variables Optimization
Calc Script Editor
Choice of the Essbase XTD Analytic Services Calc Script Editor or another text editor
If you use another editor: Save file with .csc extension Easier to perform some functions
Find/Replace See more lines of the script Printing options
Can mix and match
Essbase XTD Analytic Services Calc Script Editor Has a number of functions built in Command buttons simplify entries Validation function to detect errors Limited work area
Calc Script Editor Functions
Most simply enter the character that are on the face of the button
The is a notable exception Provides a list of available functions Grouped by function type Can include parameters
Member Selection
Associate the outline Can select members from the member list
Automatically adds quotes if needed Frequently easier/faster to just type the
member names
Online Help
Use menu option F1 does not work
Functions and commands are included Can also use Essbase XTD Analytic Services
documentation Technical Reference
Calculation Commands
Limited number of true calculation commands Not functions they are in another chapter Not Set statements they are covered under
optimization
Agg
Syntax AGG(Dimname, Dimname,);
Summary Aggregates dimension(s) according to the outline
operators Member formulas are ignored Very fast
Fewer than 6 levels
Array
Syntax Array ArrayName[DimName] = {initial values };
Summary Declares an array variable with as many
members as the dimension name that follows it Can be initialized with values if desired
Optional
Calc All
Syntax Calc All;
Summary Calculates all dimensions in the database Uses default calculation order This is typically the default calc Very efficient Can add Except to remove
dimensions/members
Calc First
Syntax Calc First;
Summary Calculates members tagged as TB First Does not update Time dimension, only Accounts
Calc Last
Syntax Calc Last;
Summary Calculates members tagged as TB Last Does not update Time dimension, only Accounts
Calc Average
Syntax Calc Average;
Summary Calculates members tagged as TB Average Does not update Time dimension, only Accounts
Calc Dim
Syntax Calc Dim(DimName, DimName,)
Summary Calculates the dimensions that are listed All dense dimensions are calculated first in the order
they appear Sparse dimensions are calculated last - also in the
order they appear If you need a different order, use multiple Calc Dim
commands Includes member formulas
Calc TwoPass
Syntax Calc TwoPass;
Summary Calculates members tagged as Two-Pass in the
Accounts dimension
CCONV
Syntax CCONV type->Currency name
Summary Special command reserved for currency
conversion Converts values to the currency name specified Only usable if you have the currency conversion
module
ClearBlock
Syntax ClearBlock ALL | UPPER | NONINPUT |
DYNAMIC Summary
Designed to clear blocks by replacing values with #Missing
Within a FIX block, may only clear a portion of a block
Options for which types of block to clear
ClearData
Syntax ClearData MemberName;
Summary Designed to set a member value to #Missing Works within a fix
ClearBlock vs. ClearData
ClearBlock is designed to clear an entire block At least a large portion of the block
No Member name, just block type ClearData is designed to clear a member
Usually before it is recalculated In many cases either one will work
DataCopy
Syntax DataCopy MemberName To MemberName;
Summary Copies data from one member to another All intersections are included Great for copying versions Will create blocks if they do not exist
If/Else/ElseIf/EndIf
Syntax IF (condition) ElseIf/Else EndIf;
Summary Condition must be if parenthesis In a calc script, associate with a member name
Does not really matter which member name Each If must end with EndIf Most effective on dense members
Fix/EndFix
Synatx Fix(MemberNames, Functions Etc.) ENDFIX
Summary Used to focus or limit a calculation Most effective on sparse members Used extensively
Loop/EndLoop
Syntax Loop (integer, break) EndLoop;
Summary Causes a block of code to be repeatedly
executed The integer determines the number of times the
loop runs The break value allows early exit Although it is a loop, its not a true programming
loop
Using Commands
Need to combine commands with functions Execute in the order they are written None by itself is that useful Used together, they provide a powerful tool
set to apply business rules to the database
Not Commands But
A member name by itself Calculates the member formula if present Calculates outline rollup if there is no formula
Example East;
Multiple members can be listed Member set functions can also be used
Example @IDESCENDANTS(2003)
Assignment Statements
Assign a value to a member Example
Budget = 0; Budget = Budget * 0.95;
There are other math operators More later
Every command ends with a semicolon (;) Okay, there are exceptions but most do
More on Member Names
All the members are calculated at once if possible
To force multiple passes of the database, group the member names in parenthesis One pass for each group in parenthesis Not usually what you want, so use with caution
Function Categories
Conditional Boolean Relationship Calculation Operators Mathematical Member Set Range and Financial
Allocation Forecasting Statistical Date and Time Miscellaneous Custom Defined
Calculation Operators
Nothing too exciting + - * / % Use parenthesis to control order of
operations
Conditional
Typically used within an If command >, >=,
Boolean
More traditional functions@ISACCTYPE@ISANCEST@ISCHILD@ISDESC@ISGEN@ISIANCEST@ISICHILD@ISIDESC@ISIPARENT
Typically used in If Statement Returns True/False
@ISISIBLING@ISLEV@ISMBR@ISPARENT@ISSAMEGEN@ISSAMELEV@ISSIBLING@ISUDA
Relationship Functions
Returns Values Frequently used to get other values in
database @ANCESTVAL@ATTRIBUTEBVAL@ATTRIBUTESVAL@ATTRIBUTEVAL@CURGEN@CURLEV@GEN
@LEV@MDANCESTVAL@MDPARENTVAL@PARENTVAL@SANCESTVAL@SPARENTVAL@XREF
Mathematical Functions
Most functions available@ABS@AVG@EXP@FACTORIAL@INT@LN@LOG@LOG10@MAX@MAXS
@MIN@MINS@MOD@POWER@REMAINDER@ROUND@SUM@TRUNCATE@VAR@VARPER
Member Set Functions
Returns a List of Member Names
@ALLANCESTORS@ANCEST@ANCESTORS@ATTRIBUTE@CHILDREN@CURRMBR@DESCENDANTS@GENMBRS@IALLANCESTORS@IANCESTORS@ICHILDREN
@MATCH
@IDESCENDANTS@ILSIBLINGS@IRSIBLINGS@IRDESCENDANTS@ISIBLINGS@LEVMBRS@LIST@LSIBLINGS
@MEMBER
@RDESCENDANTS
@MERGE@PARENT@RANGE
@RELATIVE@REMOVE@RSIBLINGS@SIBLINGS@UDA@WITHATTR
Range and Financial Functions
Accepts a range of members instead of individual members
@MINRANGE@MINSRANGE@NEXT@NEXTS@NPV@PTD@PRIOR@PRIORS@SHIFT@SLN@SUMRANGE@SYD
@ACCUM@AVGRANGE@COMPOUND@COMPOUNDGROWTH@CURRMBRRANGE@DECLINE@DISCOUNT@GROWTH@INTEREST@IRR@MAXRANGE@MAXSRANGE@MDSHIFT
Allocations Functions
Only two functions in this category Allocate MDAllocate
Used in simple situations to allocate an amount down to lower levels
Forecasting Functions
Similar to what you may have used in Excel @MOVAVG@MOVMAX@MOVMED@MOVMIN@SPLINE@TREND
Statistical Functions
Most common are available@CORRELATION@COUNT@MEDIAN@MODE@RANK
@STDEV@STDEVP@STDEVRANGE@VARIANCE@VARIANCEP
Date and Time
Just one to convert strings to a date ToDate
Miscellaneous
The catch-all category@CALCMODE@CONCATENATE@SUBSTRING@NAME
Using Functions
Frequently, functions are nested or used as parameters for another function
Functions are prefixed with @ Parameters are separated by commas Used in many places
Right of = FIX statements IFs etc.
Focusing Calculations
Two most common methods Fix If
Other methods: CrossDim operator MemberSet functions Relationship functions Variables
FIX
FIX limits the members that are processed Most effective when used on members of
sparse dimensions Only blocks that meet the Fix criteria are
processed Remaining blocks are simply skipped
Typical uses Only calculate one scenario Limit calc to current year
More on FIX
FIX works properly on dense members May result in all blocks being processed All statements between the FIX and ENDFIX
are executed as a block FIX statements an be nested Multiple members from the same dimension
are treated as or In other words, all members will be processed
What You Cannot Do With FIX
You cannot assign values to a member that is included in the FIX statement Actually, values cannot be assigned to any
member from a dimension in the FIX statement Example
Fix(Sales)Sales = 100;
ENDFIX Would result in an error
FIX On
Members from one or more dimensions Can list specific member names Can include member set functions Can include AND and OR
Use AND to get the intersection of two or more functions
Caution on empty sets
IF Statements
Conditional processing is often needed to process business rules
Can include complex conditions And, Or, Not, etc.
What You Can Check For
Most common are Members
Specific members MemberSet comparisons Use boolean functions
Values >, < = etc. Use #Missing for test
True is treated as 1 and false is 0
Syntax for IF
Place condition in parenthesis Associate with a member
Doesnt matter which member MemberName (
If (condition) Statements
EndIf ) Not required for member formulas
More on IF
Can have ElseIf (condition) Can Have Else Can have both Still terminate with single EndIf
Be sure to include ; All ElseIF/Else conditions have to remain
within parenthesis for the member association
To If or FIX
Generally, If works best on dense dimension members Particularly when ElseIF or Else are
includedBlock is only brought into memory once and all related
conditions are processedFix would bring block in multiple times
Conversely, FIX works best on sparse dimension members
Old saying Fix on Sparse, If on dense If youre dense, fix on sparse?
There are always exceptions
Other Method to Focus Calculations Cross Dim operator
Allows reference to specific cells Frequently in another block Technique for complex allocations
Relationship functions Can be used to get a value from another part of the
database Example is @ParentVal()
MemberSet functions Return member names Can be used to calculate those members or in a Fix/If
Loops
Can use the Loop/EndLoop to repeatedly execute calc commands
Not designed to increment variables and reference different members
Used to solve goal seeking problems
Why Create Blocks
Often need blocks that do not exist Typically when performing allocations
Actually, one of the most challenging aspects of writing calc scripts If a block does not exist, not processed by a calc script
Except outline calculations Frequent cause of calculation problems
ExampleIF (Budget > 100)
Actual = 50;ENDIF Assumes Scenario is sparse
Ways to Create Blocks
There are basically four methods to create blocks: Load data Calculate the database Use the DataCopy command Place a sparse member on the left side of an
assignment statement A little more complex than this
Load Data
During a data load, a block is created if it doesnt already exist Can use this to create blocks Create data records based on source information Load the file One trick is to use the Replace function of the
load rule
Calculate the Database
An outline rollup will create upper level blocks
Both Calc All/Dim and AGG will cause blocks to be created
Only blocks containing data are created Empty blocks are not written
DataCopy Command
Easiest way to create blocks If an intersection for the To member does
not exist, it is created Frequently, the only practical method to
create the needed blocks Danger in that it can create excessive
numbers of blocks
Assignment Statements
If The member on the left of the assignment
statement is sparseAnd
The database setting Create bocks on equations is selected
Blocks are created Not always possible to have a sparse
member on the left side of the assignment statement Try fixing on a dense member and assigning to
a sparse memberE.g. Fix on the account member and assign to another
May need to change dense/sparse settings
Changing Database Settings
A quick diversion When you change database settings, you
should stop and then start the database (or application) for the settings to take effect
Organizing Calc Scripts
Most calc scripts have common parts or sections Housekeeping Initial Calculation Mainline Rollup Back Calc
Housekeeping
The calculation environment is established Intelligent calculation is turned on/off Other Set commands are issue
Variables are declared Comments describing the purpose of the calc
script are entered Use /* and */ for comments
Key Set Commands
Set AggMissg ON | OFF Controls if #Missing values are aggregated If all children and #Missing, parent is et to
#Missing Upper level step-on
Caution needed if values loaded at upper levels SET MSG ERROR | WARN | INFO | SUMMARY |
DETAIL | NONE; Detail lists each block being processed Good for debugging Generates a large log and is slow
Initial Calculation
This is where business rules are applied Frequently include allocations
This usually requires a partial database rollup There may be many calculation blocks with
FIX statements Or If statements
Typically the most complex portion of the calc script
Mainline Calculation
This is the rollup of the database May be focused
e.g. FIX(Actual) Regardless, this is the major rollup of the
database
Back Calc
This is where two-pass members and any other members requiring special treatment are handled
Often requires recalculation of all or most of the blocks
Corrects percentages and ratios that may have been summed
Testing Calc Scripts
The following methodology works well: Use a test database Create test data in Excel
Keep it on its own sheet Clear the database Use Lock & Send to load test data Run the calc from Excel Retrieve onto a test sheet
More on Testing
Best to know expected outcome before retrieving
Can create a calc script to clear the database Minimizes switching between Excel and App
Manager The key is small amounts of data which
allows short test cycles Many people test with large amounts of data
resulting in long cycles and inability to get many tests in per day
When initial testing is complete, add to the test data set to make sure everything is working properly
Typical Issues
Intelligent Calc Prevents blocks from being calculated
FIX on too few or too many members Nested FIXs are ORs, not ANDs
Dynamic Calc runs after Batch Calc Dynamic calc members do not make good counters
Referring to the wrong block Remember, every intersection is processed unless
the calculation is focused
Real World Calcs
These exercises are derived from calculations created for customers but modified for the Sample application
Exercise
Simple Allocation Complex Allocation Comparable Sales Compute LQ (last quarter) in the scenario
dimension Compute 12 month rolling average Compute Market Share and Product Mix Get data from another database
Exercise (Continued)
Putting it together This is for the Budget dimension only Clear upper level blocks Add the member Insurance to the Measures
dimension and load a value at the top level Allocate the insurance expense to all products based
on Budgeted Sales Calculate Market Share in the calc script (may need
to change storage settings); insure it is correct at all levels
Include values for Last Quarter and a 6 month rolling average
Perform a rollup of the database
Goal
Make the calculation run faster Variety of settings/changes available to make
things run faster Many limiting factors
Existing reports/processes Source systems Hardware
Parallel Calculation
Apply multiple processors to a calc Only straight forward calcs can use this
If order dependent portions, calculated in serial Maximum of 4 processors Recommend 1 less than number on server Available in Essbase XTD Analytic
Services 6.5
Cache Settings
More is not always better Sometime Hyperion Essbase seems to spend
more time maintaining caches than calculating Usually find a point where increasing cache
does not improve performance Typically a trial and error process Set uncommitted / 0 on Transaction tab
Dense/Sparse Settings
This will make the largest difference Other factors influence decisions
Attribute dimensions If only a portion of the database is calculated
ideally you want to Fix on it as sparse members Reduces the number of blocks processed
May need to try a number of combinations Smaller blocks almost always calc faster
Review Dimension Order
Largest dense dimension to smallest Smallest sparse dimension to largest Hourglass shape Dimension calc order is influenced by order
of dimensions in the outline Goal is to process fewest blocks possible
Eliminate Database Fragmentation Previously calculated database takes longer
to calculate Faster to create blocks than to update them
Non-adjacent blocks can take longer to access As data is added to blocks, they grow and cannot
be rewritten to same place Export, ClearData (All), Import OR
Restructure Old DeFrag Command no longer available
Want to calc everything in the block at same time
Review Blocks that are Processed Can use SET MSG DETAIL Each block is listed as it is calculated
See application log Can determine number of times each block is
processed Can also see if expected blocks are being
processed Often, extra blocks are being read Correct by updating FIX
Control Calculation Environment Use SET Commands
Set CACHE HIGH; New cache Tracks the block used in the calc
Set CALCHASHTBL ON; Good for flat dimensions
Set FromBottomUp ON; Faster than TopDown Make sure you get correct results
Other Methods
Remove dimensions from the cube Can make a huge difference Some companies keep multiple databases
One with all dimensions and other with limited Partition
Move data that does not need to be calculated to another database
Intelligent Calc
More SET Commands
SET LOCKBLOCK HIGH; Allows more blocks to be locked Not really optimization but
SET CALCTASKDIMS n; Determines how many dimensions are used to
generate opportunities for the parallel calculation
Simulate the Calculation
Method to predict the actual calc time Based on the number of blocks that would be
created SET MSG ONLY; SET NOTICE HIGH; CALC ALL; Does not actually calc the database but
provides timings Run the calc, and use ratio of actual time to
simulated Not perfect but
Dynamic Calc
Can improve calc times significantly Make entire dimension dynamic calc
Never needs to be calculated Especially effective for Time dimension
Normally calculates very slowly All upper level members of dense dimension
Keeps block smaller Upper 1-2 levels of sparse dimension(s)
Be aware of how many blocks are needed to calc highest level
Custom Functions
Can write functions in Java Compile, save on server, and register Call from calc scripts Run slower than same calc command May run faster than complex series of
database passes required to perform the same thing
Custom Macros
Can predefine combinations of commands SumRange is actually a Macro
Combines Sum and Range functions Not necessarily faster but might avoid errors Need to define parameters May be possible to save a pass of blocks
Database Statistics
Not a optimization technique but a tool Can monitor calculation impacts
Number of blocks created Can refresh while calc is running
Block creation rate Not necessarily best for the calc
Copy and paste into Excel
Comments / Questions ?
Edward Roske
Office: 972-735-8716
Email: eroske@interrel.com
Please Complete Your Session Evaluation.
459ACalc Scripts for Mere Mortals
Presented by:Edward Roske,interRel Consulting
459AAgendaTypes Of CalculationsCalculation OrderCalculation Order All DimensionsCalc ScriptsCalc Script EditorEssbase XTD Analytic Services Calc Script EditorCalc Script Editor FunctionsMember SelectionOnline HelpCalculation CommandsAggArrayCalc AllCalc FirstCalc LastCalc AverageCalc DimCalc TwoPassCCONVClearBlockClearDataClearBlock vs. ClearDataDataCopyIf/Else/ElseIf/EndIfFix/EndFixLoop/EndLoopUsing CommandsNot Commands ButAssignment StatementsMore on Member NamesFunction CategoriesCalculation OperatorsConditionalBooleanRelationship FunctionsMathematical FunctionsMember Set FunctionsRange and Financial FunctionsAllocations FunctionsForecasting FunctionsStatistical FunctionsDate and TimeMiscellaneousUsing FunctionsFocusing CalculationsFIXMore on FIXWhat You Cannot Do With FIXFIX OnIF StatementsWhat You Can Check ForSyntax for IFMore on IFTo If or FIXOther Method to Focus CalculationsLoopsWhy Create BlocksWays to Create BlocksLoad DataCalculate the DatabaseDataCopy CommandAssignment StatementsChanging Database SettingsOrganizing Calc ScriptsHousekeepingKey Set CommandsInitial CalculationMainline CalculationBack CalcTesting Calc ScriptsMore on TestingTypical IssuesReal World CalcsExerciseExercise (Continued)GoalParallel CalculationCache SettingsDense/Sparse SettingsReview Dimension OrderEliminate Database FragmentationReview Blocks that are ProcessedControl Calculation EnvironmentOther MethodsMore SET CommandsSimulate the CalculationDynamic CalcCustom FunctionsCustom MacrosDatabase StatisticsComments / Questions ?Please Complete Your Session Evaluation.
Recommended