Matthias Ender, SAS Institute Inc., Cary, NCsupport.sas.com/resources/papers/proceedings/proceedings/sugi29/029-29.pdfADDED FOR IMPLEMENTATION Legal Entity ID Date Of Birth Gender

1

Paper 029-29

Taking Your SAS/MDDB® Server Applications to the Next Level Matthias Ender, SAS Institute Inc., Cary, NC

ABSTRACT The new SAS® OLAP Server implements the industry standard MDX (Multidimensional Expression) query language and a new threaded server that’s designed to support many concurrent users. While there is no automatic conversion of Version 8, SAS/MDDB applications to the new server because of the different functionalities, you can use all the knowledge gained when designing your MDDBs to lay out the new cubes quickly and easily. This paper presents some guidelines and case studies for moving your MDDB applications to the next level and includes examples of how to take specific application customizations into the new SAS OLAP Server.

INTRODUCTION This paper describes features of SAS System 9 OLAP Server technology and, where applicable, compares them to SAS Version 8 OLAP (online analytical processing) technology. This paper provides an overview and high-level introduction to this technology. For more detailed information, see the References section at the end of this paper.

DESIGN AND DATA PREPARATION An important reason for building an OLAP application is to improve end-user access to flexible reports. The necessary design work and data preparation steps represent, by far, the most effort when developing an OLAP application. The final step, mapping those designs and data to a particular OLAP technology, is different for SAS Version 8 and SAS 9 OLAP technology. This section describes the elements of an OLAP application that are independent of the actual implementation technology. If you have an existing Version 8 SAS/MDDB or HOLAP application or an OLAP application in another ROLAP or MOLAP system, you can reuse these elements when moving to SAS 9 OLAP technology.

DIMENSION DESIGN One of the central steps of preparing any OLAP database is the process of mapping data entities from your operational system to dimensions, levels, drill hierarchies, and measures for your cubes, as illustrated in Figure 1. This process includes determining the granularity of your OLAP database and the time intervals for updating your OLAP data. This logical design step has been done if you already have an OLAP application. If you are building a new application, it is a preparatory step to be performed independently of the OLAP technology that you’re using.

Figure 1. Moving your operational data to a dimensional model is the first step in designing an OLAP application.

BUSINESS REQUIREMENT TO SEPARATEEXTERNALLY SUPPLIED CONTENT FROM

LEGAL ENTITY STATUS CONTENT

OPTIONAL AGENT STATUS INSTEADOF USING LEGAL ENTITY STATUS

Product Heriarchy LevelADDED FOR IMPLEMENTATION

MERGED FOR IMPLEMENTATION

LEGAL ENTITYLegal Entity IDCompany IDLegal Entity Type

INDIVIDUALDate Of BirthFirst NameGenderLast NameMiddle NameSalutationName SuffixTitled NameNationalityName Prefix

ORGANISATIONOrganisation NameOrganisation SIC CodeBusiness TypeIncorporation DateRegistered NumberRegistered Country Name

AGENT DETAILAgent IDAgent NameAgent TypeIncorporation DateSIB Number

PRODUCTProduct IDProduct DescriptionProduct Line NameProduct TypeProduct End DateProduct NameProduct Short NameAdministration Company IDProduct Start DateProduct BrandBrand CodeProduct LAUTRO Code

PENSION

UNIT TRUST

LIFE

BANK

SHARE

PEP

ARRANGEMENTArrangement IDArrangement TypeArrangement Start DateArrangement End DateDefault Payment MethodDefault Payment FrequencyPayment Type CodePayment Type Description

ARRANGEMENT LEGAL ENTITYRelationship TypeRelationship Start DateRelationship End Date

ARRANGEMENT RELATIONSHIPRelationship TypeRelationship Start DateRelationship End Date

PRODUCT RELATIONSHIPRelationship TypeRelationship Start DateRelationship End Date

ADDRESSAddress ID

ADDRESS LEGAL ENTITYRelationship TypeRelationship Start DateRelationship End DateSpecial Delivery Instruction

LEGAL ENTITY STATUSStatus TypeStatus Start DateStatus End DateStatus ValueStatus Reason Description

ARRANGEMENT STATUSStatus TypeStatus Start DateStatus End DateStatus ValueStatus Reason Description

SUB CAMPAIGNSub Campaign IDSub Campaign DescriptionSub Campaign Mai ling DateSub Campaign Promotion DateSub Campaign TypeSub Campaign ObjectiveSub Campaign Start DateSub Campaign End DateSub Campaign Product Promoted

CAMPAIGNCampaign IDCampaign DescriptionCampaign Distribution ArmCampaign End DateCampaign Start DateCampaign TypeCampaign ObjectiveCampaign Product Promoted

PROMOTIONPromotion IDPromotion TypePromotion NamePromotion Investment EnvironmentPromotion Product Promoted

FREE RIDEFree Ride Carrier Type DescriptionFree Ride Total Cost AmountFree Ride Style CodeFree Ride Type DescriptionCost Per 1000Free Ride Carrier Name

ADVERTAdvert Total Cost AmountAdvert Carrier Type DescriptionAdvert DimensionAdvert Style CodeAdvert Type DescriptionAdvert Appearance DateAdvert Carrier NameAdvert Approval DateEditorial Information IndicatorAdvert CategoryAdvert PositionCampaign FrequencyPoster Site PostcodeAdvert Media

NON ATTRIBUTABLENon Attributable IDNon Attributable Description

MAILINGCost Of Mai lingCost Per 1000Mailing Description

SELECTION LISTSelection List IDSelection List Description

PACKPack IDPack Total Cost Amount

MAILING SELECTION PACKRelationship End DateRelationship TypeRelationship Start DateNumber Of Packs Mai led

LEGAL ENTITY MAILING SELECTION PACKRelationship TypeRelationship Start DateRelationship End Date

ARRANGEMENT BALANCEArrangement Balance TypeArrangement Balance Start DateArrangement Balance End DateArrangement Balance Value

AGENT STATUSStatus TypeStatus Start DateStatus End DateStatus ValueStatus Reason Description

TRANSACTIONTransaction TimeTransaction DateTransaction Type CodeTransaction AmountTransaction Closing Balance AmountCommission AmountDiscount AmountExpected Future Income AmountGross Credit AmountPayment MethodPayment FrequencyTransaction Type Description

SALES INDIVIDUALSales Individual IDSales Individual NameSales Individual Type Description

LEGAL ENTITY SALES INDIVIDUALRelationship TypeRelationship Start DateRelationship End Date

ORGANISATION UNITOrganisation Unit IDOrganisation Unit NameOrganisation Unit Type

SALES INDIVIDUAL ORGANISATION UNITRelationship TypeRelationship Start DateRelationship End Date

SALES INDIVIDUAL RELATIONSHIPRelationship TypeRelationship Start DateRelationship End Date

ORGANISATION UNIT RELATIONSHIPRelationship TypeRelationship Start DateRelationship End Date

CAMPAIGN STATUSStatus End DateStatus Start DateStatus TypeStatus ValueStatus Reason Description

ANNUITY

Conceptual Data ModelProject : Save & Prosper Data WarehouseModel : Data Warehouse Logical Data Model v3Author : Derek Price Version: DRAFT 02/08/99

COMMUNICATIONCommunication IDSub Channel Type CodeCommunication PurposeCommunication Channel DescriptionCommunication DateGenerated Campaign CodeGenerated Sub Campaign CodeCommunication TimeCommunication ContentCommunication DirectionSub Channel Type Description

LEGAL ENTITY RATINGRating TypeRating ValueRating Start DateRating End Date

LEGAL ENTITY ELECTRONIC ADDRESSRelationship TypeRelationship Start DateRelationship End Date

PROSPECT LISTProspect Group ID

PROSPECT GROUP SUPPLIERProspect Group Supplier IDProspect Group Supplier Name

ELECTRONIC ADDRESSElectronic Address IDElectronic Address TypeElectronic Address Value

INVESTMENT TRUST

OEIC

ISA

LEGAL ENTITY RELATIONSHIPRelationship TypeRelationship Start DateRelationship End Date

PRODUCT FEATUREProduct Feature TypeProduct Feature ValueProduct Feature Start DateProduct Feature End Date

ADDRESS COMPONENTAddress Component TypeAddress Component ValueAddress Component Start DateAddress Component End Date

ENCLOSUREEnclosure IDEnclosure Description

COMMUNICATION ENCLOSURERelationship TypeRelationship Start DateRelationship End Date

MAILING SELECTIONRelationship End DateRelationship Start DateRelationship TypeSelection List Priority

AGENT PRODUCT

SUB CAMPAIGN STATUSStatus TypeStatus Start DateStatus End DateStatus Reason DescriptionStatus Value

SUB CAMPAIGN PRODUCTRelationship End DateRelationship Start DateRelationship Type

Arrangement Hierarchy LevelADDED FOR IMPLEMENTATION

DEMOGRAPHIC COMPONENTDemographic Component TypeDemographic Component Start DateDemographic Component End DateDemographic Component Value
















PENSION

UNIT TRUST

LIFE

BANK

SHARE

PEP






PENSION

UNIT TRUST

LIFE

BANK

SHARE

PEP





ADDRESSAddress ID





















SALES INDIVIDUAL ORGANISATION UNITRelationship Type





ADDRESSAddress ID

























ANNUITY








INVESTMENT TRUST

OEIC

ISA







AGENT PRODUCT















PENSION

UNIT TRUST

LIFE

BANK

SHARE

PEP





ADDRESSAddress ID




SUB CAMPAIGNSub Campaign IDSub Campaign DescriptionSub Campaign Mailing DateSub Campaign Promotion DateSub Campaign TypeSub Campaign ObjectiveSub Campaign Start DateSub Campaign End DateSub Campaign Product Promoted









MAILING SELECTION PACKRelationship End DateRelationship TypeRelationship Start DateNumber Of Packs Mailed












ANNUITY








INVESTMENT TRUST

OEIC

ISA







AGENT PRODUCT




















PENSION

UNIT TRUST

LIFE

BANK

SHARE

PEP






PENSION

UNIT TRUST

LIFE

BANK

SHARE

PEP





ADDRESSAddress ID





















SALES INDIVIDUAL ORGANISATION UNITRelationship Type





ADDRESSAddress ID

























ANNUITY








INVESTMENT TRUST

OEIC

ISA







AGENT PRODUCT





SUGI 29 Applications Development

2

ETL / DATA SCRUBBING The steps needed to move your operational data into the information repository for your OLAP applications are commonly known as Extract/Transform/Load (ETL) and related steps, such as removing duplicate data values and resolving ambiguities. These steps typically also involve joins and partial summarizations. Often, staging also includes loading additional reference data from outside your operational systems, such as census data, customer or competition information, or market research data. All these data preparation steps are part of building your information repository or data warehouse, but are independent of the OLAP technology that you’re using.

AGGREGATION DESIGN Building an OLAP cube provides pre-summarized aggregations of your data to minimize server CPU and I/O use and maximize end-user query performance. The combinations of dimension levels that you choose to pre-summarize have an immediate impact on cube size, cube build-time, and end-user query performance. Performing an initial analysis of expected user drill patterns or using experience with a well-performing cube gives you a head-start when building your SAS 9 OLAP application.

SECURITY DESIGN The design work put into authentication and authorization schemes for existing OLAP applications can be moved into SAS 9 OLAP applications. The SAS 9 security framework is more comprehensive than the security features offered with SAS/EIS software and SAS/MDDB software. SAS 9 security is part of the SAS Metadata framework and applies to all units known to the metadata, including servers, tables, and cubes.

DEPLOYMENT Finally, the setup of server machines, mid-tier architecture, client machines and applications can stay the same or be used as a starting point for your SAS 9 OLAP application. With these steps, the main work in preparing an OLAP application is finished. To show you what an implementation using SAS 9 OLAP technology looks like, the next two sections introduce features of the SAS 9 OLAP Server and the MDX query language.

SAS 9 OLAP SERVER FEATURES The SAS 9 OLAP Server is based on the SAS 9 Scalable Architecture (SSA), which includes a threaded kernel. Using this SSA core technology, the SAS 9 OLAP Server is designed to execute multiple tasks in parallel and to take full advantage of multi-processor hardware and multiple I/O controllers. The SAS 9 OLAP Server is designed as a multi-user server. Although some multi-user capability was included in Version 8, its core architecture was that of a single-user server. The SAS 9 OLAP Server takes advantage of parallel processing both on the query side and the data access side.

QUERY PROCESSING Query processing breaks up into the following threads: • Each client connection is set up and runs in its own thread. • Each query submitted through a client session spawns its own thread. • Each query is further broken up into one or multiple data access regions that are being submitted in parallel and

are each executed in their own thread. This multi-threaded design allows as much parallel execution as possible. To illustrate this point, as soon as the data region analysis is concluded, the data access threads are spawned. At the same time, the structures for the return row set are being prepared. For more information about the threading model, see Ressler (2002) in the References section at the end of this paper.

DATA ACCESS PROCESSING On the data access side, the OLAP server's MOLAP aggregation store takes advantage of the threading capability of the new, multi-partition SAS Scalable Performance Data (SPD) Engine. The data and the index section of the files are stored in different physical files, which enables parallel access to the data and index sections. Also, the data and index files themselves are stored in partitions, which enables parallel data retrieval within the same file. The partitions of the data and index section can be distributed over multiple disk controllers, thus adding a further boost to the threaded and partitioned architecture by reducing contention and possible bottlenecks in the physical I/O.


3

As each query is broken up into steps that can be independently executed, and data regions are being queried against the cube store, the cube store itself is optimized for the fastest and most scalable access. The portable technology leveraged for the MOLAP storage of the SAS 9 OLAP Server is based on the experience gained with the SAS SPDS server product.

IN-MEMORY CACHE The results of previously executed queries are stored in a cache in memory, which speeds up subsequent queries against the same data. The cache size and the number of queries to cache can be configured. The cache is shared among user sessions, which ensures that the most frequently submitted queries are being returned quickly and efficiently.

ACCESS TO ROLAP DATA SOURCES External summarized tables and star schemas can be used to store and access cube data. The SAS 9 OLAP Server sends simple SQL (Structured Query Language) roll-up queries for execution by the RDBMS (relational database management system). More details about the SAS 9 Cube ROLAP store are presented later in this paper in the section on OLAP cube features.

SERVER TUNING AND PERFORMANCE The interactions between the OLAP server, the size and structure of the cubes it serves, the number of user connections, and the hardware used can become quite complex. An introduction to the art of tuning an OLAP server can be found in Polzin, et. al. (2002). Figure 2 graphs the results of an internal performance study conducted in 2003. The test was performed using LoadRunner by Mercury Interactive to simulate multiple user connections. Each user repeatedly executed 4 different queries with varying “think time” against 1 of 5 pre-aggregated 4 GB cubes, on a 4-way server on an isolated network. Fifteen users were added every 15 seconds, until a total of 1000 concurrent users was reached. The internal software requirements for SAS 9 servers are stated as being able to run 100 concurrent users with no “think time,” 24 hours a day for 7 days.

Figure 2. OLAP Server Resource Usage in a Multi-User Scenario

- the green line represents the number of concurrent virtual users who are executing the series of roll-up queries as the scenario progresses. - the blue line represents the throughput in transactions per second. There are 6 transactions per OLAP query (enclosing transaction, open method, get axis info method, get tuples method, read cells method, and close method). The peak transaction/sec rate during this run is 218 tx/s. Dividing by 6 and multiplying by 3600 equals 36 queries per second (130,800 per hour at peak rate). - the red line represents CPU usage in percent busy.


4

OLE DB FOR OLAP The SAS 9 OLAP Server supports and is based on the OLE DB for OLAP standard (ODBO) with its query language MDX, which has evolved as the most wide-spread industry standard for OLAP servers. ODBO includes an API that is implemented for C++ and in ADO MD (in its different flavors for Visual Basic (VB) and .NET). Additionally, in its BI platform, SAS has used this API as a basis for a set of Java classes. ODBO (and MDX) provides a comprehensive set of OLAP functionality that is being used by an increasing number of end-user clients and servers, and is the basis for emerging standards such as XML for Analysis. Figure 3 shows how ODBO fits into the SAS OLAP Server architecture.

Figure 3. OLAP Server Architecture

MDX QUERY LANGUAGE FEATURES MDX is a query language for multidimensional queries and a means of communication between OLE DB for OLAP providers and consumers. This section provides an overview of MDX concepts and offers examples of usage and syntax for calculations and security expressions. MDX queries are typically generated by client GUI applications. The end user does not typically use MDX or know about it. OLAP server administrators can use it to define calculations and security conditions. MDX uses the concept of dimensions and dimension members. For example, the members of a dimension named “Time” would be hierarchically organized names of years and months. In MDX syntax, dimension and members names and members are identified by enclosing them in square brackets that are separated by dots. For example, members of theTime dimension could be [Time].[2003], or [Time].[2004].[March], or [Time].[2004].[May].[09] Other elements identified by square brackets are Hierarchies, Levels, and Measures.

SIMPLIFIED CALCULATIONS The power and relative ease-of-use of MDX can best be illustrated by giving an example of calculations that were difficult to achieve with SAS Version 8 OLAP technology.

OLAP Server Architecture

SAS OLAP Server

Interface

Scalable Storage

SAS Dataset Storage

SAS Management

Console

SAS OLAP Cube

Studio

Java

SAS Metadata Server

SAS Enterprise Guide

Microsoft Excel SAS Web Report Studio

OLE DB for OLAP

Proclarity SAS Report Studio

SAS AppDev Studio

In Memory Cache


5

- The Version 8 SAS/MDDB uses the statistics PCTSUM and PCTN to calculate the percentage contribution of the current value to its parent value for sums and counts. An Example: Number Car Sold PCT Profit PCT N N SUM SUM ----------------------------------------- Toyota Camry 5 17% $3.000 29% Ford Explorer 10 50% $2.400 23% Mazda Miata 15 33% $5.000 48% ----------------------------------------- All Cars 30 100% $10.400 100% MDX enables you to easily define calculated members that express not only the ratio or percentage contribution to the parent member, but also to any other ancestor or to the ALL member. Some examples: Ratio to Parent (corresponds to PCTN): [Measures].[Profit] / ([Measures].[Profit], [Cars].CurrentMember.Parent) Ratio to Parent (corresponds to PCTSUM): [Measures].[Number Sold] / ([Measures].[Number Sold], [Cars].CurrentMember.Parent) Ratio to Ancestor: [Measures].[Number Sold] / ([Measures].[Number Sold], Ancestor([Cars].CurrentMember,[Cars].[Manufacturer])) Ratio to [ALL] [Measures].[Number Sold] / ([Measures].[Number Sold], [Cars].[All Cars]) To display the corresponding percentage contribution, either use a percentage format or multiply the expression by 100.

- To calculate a rolling average of the last 6 months for the Invoice measure, you can use the following MDX expression: avg( {ParallelPeriod([Time].[Month], 6, [Time].CurrentMember): [Time].CurrentMember}, [Measures].[Invoices]) The PARALLELPERIOD function returns the dimension member that lags six members in the Month level of the Time dimension. The AVG function calculates the mean for the Invoices measure for all members between (the colon ":" operator is used for ranges) that lagged member and the current member of the Time dimension.

- One recurring problem is to avoid double counting; a problem that is addressed with the NUNIQUE feature in SAS Version 8 OLAP. For example, on a given day 20 customers in a store bought one product and 30 customers bought another product. This doesn’t necessarily mean that the store had 50 customers that day; some customers might have bought both products. Count(CrossJoin( {[Measures].[Price]}, Descendants([Customers].CurrentMember, [Customer Name])), ExcludeEmpty) This expression evaluates each Customer Name – Price combination for the current member of the Customers dimension and counts the number of combinations that are not NULL.

- In Version 8 MDDBs, you can specify WEIGHT variables for your analysis variables. Basically, a WEIGHT variable enables you to specify the relative importance of each observation. In SAS 9 OLAP Cubes, make sure you define the WEIGHT variable as a measure, and then use MDX to create a calculated member:


6

[Measures].[Actual_sum] * [Measures].[Weight_sum] or , to calculate weighted averages: [Measures].[Actual_sum] / [Measures].[Weight_sum]

- Sometimes a measure needs to be summed across all dimensions except TIME. For the TIME dimension, it needs to be averaged. For example, the inventory count of a given item in a warehouse changes weekly and cannot add up over time. To solve that problem, store that measure by using the SUM statistic in the cube and define a calculated measure with the following formula. This formula divides the value of SUM by the number of weeks that contribute to the current Time member. [Measures].[Actual_sum] / count(descendants([Time].CurrentMember, [Time].[Month], SELF))

MDX allows you to create calculated members like these and store the definition permanently in the cube. Calculated members appear just like regular members to the end user.

SECURITY EXPRESSIONS SAS 9 OLAP Server uses MDX to define security authorization conditions. You can set access flags on servers, cubes, and dimensions. To restrict or allow access to a subset of the data in a cube, define the permission conditions (as shown in the example in Figure 4) that are set on a cube’s dimension in SAS Management Console’s Authorization Manager:

Figure 4. OLAP Cube Dimension Permission Conditions This permission condition is set on the CUSTOMER dimension and allows the user to see the following: 1. The All member for the dimension 2. All the states (FL, NC, AZ, etc.) 3. The direct children of FL. In this case, that's the CITIES in FLORIDA For a comprehensive look at the MDX language, see the SAS OLAP Server: MDX Guide.

SAS 9 OLAP CUBE FEATURES The SAS 9 OLAP Cube offers many new features that allow improvements in the process of preparing your OLAP application. Support for ragged and unbalanced hierarchies and member properties offer dimension design choices not previously available in SAS Version 8 OLAP technology. Logging capabilities give you the data that are needed for tuning your cubes’ aggregation design. Choices in aggregation storage include scalable and tunable MOLAP storage and several of ROLAP summary storage options. Cube creation is possible via individual batch jobs, scheduled processes integrated in the ETL process, and a portable Java-based administration GUI. Multiple Language Support makes it possible to build mutli-lingual cubes. Server monitoring tools provide the necessary administrative framework for long-running, multi-user servers.


7

RAGGED AND UNBALANCED HIERARCHIES Hierarchies can have different depths ( "holes" ) in their branches. An example of an unbalanced hierarchy is a geographical hierarchy for the United States where the geographic regions are not always subdivided into States and where States do not always break down into cities.

Figure 5. An example of a ragged and unbalanced hierarchy. The geographic hierarchy in Figure 5 is both ragged and unbalanced. Some regions only drill down to the State level. The City level is missing, making it unbalanced (not all branches have the same number of levels.) Other regions drill directly to cities and skip the State level, which makes the hierarchy ragged. An important aspect of the support for ragged and unbalanced hierarchies is in how MDX functions return the relationship between members, for example, what is considered a sibling or a direct child, and so forth. Let’s call the parent-child relationships among hierarchy members 'vertical’, and let’s call the relationships across a level 'horizontal'. For the North region in the above example, descendants are found only at the City level. This means that when the .CHILDREN function is called for the member [Geographic].[All Geographic].[North], the set returned will contain the three city members. It will not be empty. Similarly, the .FIRSTCHILD function called for the member [Geographic].[All Geographic].[Central] will return Ohio (State level) and .LASTCHILD will return Chicago (City level). For horizontal relationships, the member [Geographic].[All Geographic].[Central].[Ohio] has no siblings, even though [Geographic].[All Geographic].[Central] has three children. Applying the .NEXTMEMBER function to the member [Geographic].[All Geographic].[Central].[Ohio] will return Georgia. In other words, each member remains at the level at which it was defined. In the field, unbalanced hierarchies are often defined in so-called Parent-Child tables. A simple data transformation can make a parent-child table into an input table for an unbalanced hierarchy:

MEMBER PROPERTIES Not every attribute of a dimension needs to be expressed as a level in a drill-down hierarchy. Some attributes provide additional information for analysis and subsetting, but you don't need them in a drill-down hierarchy. For example, much of the demographic information that is provided by your customer database, such as income level, number of children, or street address or marital status might not be essential categories for your daily slice-and-dice, but you might want to keep it in the cube as useful lookup information or for occasional specialized queries. If you store attributes such as these as member properties, your cube size and creation time will not increase because the attributes are not included in your aggregations, but the information will be available for subsetting or reporting.


8

Figure 6. OLAP Cube Studio Cube Designer dialog window for selecting Member Property input columns To subset on all employees with an annual salary larger then $100,000, use an MDX expression similar to this one:

FILTER([Employees].[All Employees].children, [Employees].currentmember.properties('ANNUAL_SALARY') > ‘100000’)

To transform a member property into a measure, create a calculated member like this one:

WITH MEMBER [measures].[annual_salary] as 'iif([Employees].currentmember.level IS [Employees].[Employee], [Employees].currentmember.properties(''ANNUAL_SALARY''),NULL)'

This calculated member returns the value of the ANNUAL_SALARY property for all members of the Employee level of the Employees dimension, and a NULL value for all other members of the Employees dimension.

MOLAP AGGREGATION STORAGE MOLAP storage of SAS 9 OLAP Cubes takes advantage of key compression and allows data access using the cube's internal data representation directly. In other words, for its internal processing, the OLAP server represents each hierarchy member (for example, [1999].[January].[01]) as a numeric key. MOLAP storage uses that internal key to reduce the number of columns that are needed to identify dimension levels and allows direct communication between the OLAP server and the physical storage, thereby eliminating the need for data conversion and lookup. As mentioned above, the file structure and access engine of MOLAP storage supports a partitioned design that enables parallel threads to read the same file.


9

ROLAP AGGREGATION STORAGE ROLAP tables used in SAS 9 OLAP Cubes can be SAS data sets, SAS data views, or any tables or views accessible via a SAS engine. This capability extends the choice of available storage options for SAS OLAP Cubes to include SPDE, SPDS, and any RDBMS product for which SAS/ACCESS software is available. Data requests against ROLAP tables that are part of a cube are sent as simple SQL statements for execution and aggregation to the respective database engine. ROLAP data requests can also read the data that is used to create and load the cube, whether it’s a detail table or a star schema. The cube's input data can be used in place of the N-way aggregation.

HOLAP AGGREGATION STORAGE A hybrid approach that mixes MOLAP and ROLAP aggregations is also possible. For example, you can start off with an existing set of ROLAP tables, either from a SAS Version 8 HOLAP application or from a star schema that is designed for another ROLAP system, and then tune your cube by adding additional MOLAP aggregations.

CUBE CREATION AND DEPLOYMENT You can create a cube by using either the OLAP procedure (a familiar way to do things for SAS users), or you can use the Cube Designer wizard GUI that is available in SAS OLAP Cube Studio and SAS ETL Studio, which is shown in Figure 7. The definition of the cube created by the Cube Designer is stored in a SAS Metadata Repository. Cube re-creation can be scheduled either within SAS ETL Studio or as a PROC OLAP batch job. A cube can be loaded from a star schema, a fully de-normalized detail table, or a summarized table (from a ROLAP\ N-way aggregation).

Figure 7. The main window of the Cube Designer wizard.

METADATA When creating a cube, metadata about the cube is stored within a SAS Open Metadata Repository on a SAS Open Metadata Server. Each component of the SAS Intelligence Architecture leverages the sharing of common metadata and a common repository. By making cubes and servers available in a SAS Open Metadata Repository, they become available to client applications (both inside and outside of SAS) and can be secured and managed from a single point


10

of control. For more information on these aspects of the SAS Open Metadata, see Ryals (2003), In the References section at the end of this paper. SAS OLAP Cubes and the SAS OLAP Server use a version of Unicode for their internal data representation and storage. Input data with different character encodings can be loaded and displayed. You can load member captions in different languages and encodings into a cube. Then, the locale of the client that is connecting to the server determines which member captions to return to the client applications. For example, your MLS cube might have product name captions stored in English, French, and Chinese. Three users with client systems in their preferred locale connect to the server and see the product names in their language!

CUBE DEPLOYMENT The SAS OLAP Server's Application Response Measurement (ARM) facility not only enables you to track aggregation usage and query performance, but also server load and user identities. ARM is a vendor initiative to create an API for an open, vendor-neutral approach to manage the performance of distributed applications. By default, the SAS OLAP Server’s ARM records are written into a flat file, and they can be extracted and analyzed by using a separate step. The ARM system can also be connected to an external ARM agent such as HP's Workload Server to allow live monitoring of server activities. You can set ARM options in the Advanced Options window of an OLAP Server definition in SAS Management Console (SMC), shown in Figure 8. The integrated security in SMC allows you to control access to parts of your cubes or cube data.

Figure 8. Advanced Options window of SAS Management Console

CUBE TUNING The OLAP world has long been struggling to come up with the best way to choose which aggregations to summarize and store with the cube. In reality, the most important factor that needs to be taken into account is user behavior. Here are some initial assumptions about which combination of hierarchies are most likely to be requested. Most useful, though, is insight into actual usage patterns. Best practices indicate that you start with an initial aggregation design based on a minimal set of aggregations or on your best assumptions about usage patterns. After the cube is deployed to users, use the SAS 9 OLAP Server's ARM logging capabilities to collect data about the usage pattern


11

and the performance of individual queries. You can then analyze the collected data to find out which cube aggregations are used most, which ones you can safely eliminate without harming query performance, and which ones are often requested but don't exist in the cube. The ARM data provides all the information that you need to get the optimal query improvement for your build time and cube storage space. SAS 9 OLAP Cubes give you the option not to store the aggregation that represents the crossing of the lowest, most granular level of your dimensions (often called N-way or N-way aggregation, a name that is borrowed from the SUMMARY and MEANS procedures where it denotes the combination of all CLASS variables). If your input data comes in at the same granularity as the cube, your N-way might contain data that is very similar to your input data. If you want to avoid that duplication and have a well-designed star schema as data input, there is no need to replicate those data. Also, queries against a cube with a good aggregation design, depending on the usage patterns, should rarely hit the N-way.

AGGREGATION STORAGE The aggregations of a SAS OLAP Cube can either be generated during the cube creation step and stored in the cube's internal format as MOLAP aggregations, or aggregation tables can be created outside of the cube creation process and made available to the cube.

HOW TO VIEW A CUBE: CLIENTS AND APPLICATIONS DEVELOPMENT The SAS Version 8 OLAP technology came with viewers like those in SAS/EIS software, the web-based MRV and SAS AppDev Studio. SAS Version 8 MDDBs could also be viewed via the Open OLAP Server, a limited OLE DB for OLAP server and provider implementation. Clients that viewed data by using the Open OLAP Server will now be able to take full advantage for a comprehensive ODBO implementation. These clients include SAS Enterprise Guide, and external ODBO compliant clients such as MS Excel 2000 or ProClarity. The SAS Business Intelligence platform is offering clients and components that communicate with the OLAP server via ODBO and MDX. SAS 9 cube viewers are included in SAS Enterprise Guide software, SAS Web Report Studio, and the SAS Visual Data Explorer. Under Windows, applications development is possible via OLE DB for OLAP for C++ and ADOMD for Visual Basic and .NET. In Java, the SAS Business Intelligence platform provides an applications development environment.

COMPARISON OF V8 SAS/MDDB AND V9 OLAP CUBE CREATION TASKS The following table lists some of the basic tasks that are performed when you create a SAS OLAP cube and the differences in how these tasks are handled in SAS Version 8 and SAS 9.

Functionality How it is handled in Version 8 How it is handled in SAS 9.1

Defining hierarchy levels for dimensions

PROC MDDB CLASS statement PROC OLAP LEVEL statement

Specifying statistics PROC MDDB VAR statement In the MDDB procedure, you specify stored statistics. Derived statistics are available at query time.

PROC OLAP MEASURE statement In the OLAP procedure, you have one MEASURE statement for each combination of an input column plus its associated statistic (which includes defining derived statistics).


12

Grouping levels for drill-down capability

PROC MDDB HIERARCHY statement, DISPLAY=YES/NODATA option In the MDDB procedure, individual classes (levels) are not linked but can be grouped into drill-down hierarchies by using the DISPLAY option in the HIERARCHY statement.

PROC OLAP DIMENSION statement The OLAP procedure uses the DIMENSION statement to accomplish the same goal. Each dimension consists of a hierarchy that is composed of grouped levels. Each level is fed by one input column.

Creating NWAYs and other aggregations

The MDDB procedure always creates an NWAY subtable, which is the combination of all class (level) variables. Any additional subtables can be explicitly requested by using HIERARCHY statements.

In the OLAP procedure, subtables are called aggregations. As with the MDDB procedure, the NWAY aggregation is created by default, although you can choose not to create the NWAY. Aggregation creation is handled by the AGGREGATION statement.

Specifying hierarchy navigation HIERARCHY statement, DISPLAY=YES/NODATA option In the MDDB procedure, the HIERARCHY statement can be used to specify navigation hierarchy and, unless you specify DISPLAY=NODATA, also to create an aggregation.

HIERARCHY statement In the OLAP procedure, the navigation hierarchies are handled by the HIERARCHY statement.

Loading cubes from star schemas Star schema support is available through the Distributed Multidimensional Metabase (DMM) facility.

You can load your cubes from star schemas and indicate your dimension tables by using the DIMTABL=, DIMKEY=, and FACTKEY= options in the DIMENSION statements. You can also use a specified star schema instead of an NWAY aggregation by using the NO_NWAY option in the PROC OLAP statement. To load the cube from a star schema, use the FACT= option instead of the DATA= option.

Using externally summarized data sources

HOLAP data groups You use HOLAP data groups to point to externally summarized data sources, which might be available through RDBMS tables; SAS data sets; or, in the case of multiple aggregations combined into one table, a PROC SUMMARY output data set with multiple types.

AGGREGATION statement or TABLE= option You use the OLAP procedure’s AGGREGATION statement to point to externally summarized sources.


13

Cube metadata registration The SAS/EIS metabase facility and the Distributed Multidimensional Metadata (DMM) facility for HOLAP data groups serve as an extension to the MDDB's own structural information. Information about MDDBs, such as drill-down hierarchies and distributed data sources, is stored in those repositories and accessed there by the different data model extensions.

All relevant structural information is contained within the cube and most of it is also replicated within the SAS Open Metadata Architecture. This is done to gain the following advantages: The ability to disassociate the cube definition process from cube creation. Later, you can create a cube by using its stored definition. The ability to define and enforce security at the SAS Open Metadata Architecture level. The ability to manage and control the data source in the centralized SAS Open Metadata Architecture.

COMPARISON OF PROC MDDB AND PROC OLAP SYNTAX The following example code illustrates the difference between the MDDB procedure and the OLAP procedure when you create a simple cube. To create a Version 8 MDDB and make it ready for use with SAS or external viewers, execute PROC MDDB. After you run the code, you register the cube in the SAS/EIS metabase facility in order to transform the display hierarchies into the HIERARCHY attribute of the metabase registration. For partitioned or distributed cubes, you must also create a HOLAP data group definition by using the Distributed Multidimensional Metadata (DMM) facility.

proc mddb data=sashelp.prdsale out=work.prdmddb; class country region division prodtype product year quarter month; var actual / sum n; var predict / sum n; hierarchy year quarter month / name='Time YQM' display=nodata; hierarchy year month / name='Time YM' display=nodata; hierarchy country region division / name='Geography' display=nodata; hierarchy prodtype product / name='Product Line' display=nodata; hierarchy country prodtype year; run;

To create the same cube in the SAS 9 OLAP Server, use the following PROC OLAP code. This PROC step also registers the cube in the specified metadata repository. Note: You can also create this cube by using the Cube Designer wizard, which is available in SAS OLAP Cube Studio and SAS ETL Studio.

proc olap data=sashelp.prdsale cube=prdcube path='c:\tmp'; metasvr olap_schema='banking schema' repository='financial repository' host='misdept.us.mar.com' port=9999 protocol=com userid=jjones pw='my password'; dimension Geography hierarchies=(Geography); hierarchy Geography levels=(country region division); level country caption='Country'; level region caption='Region'; level division caption='Division'; dimension Time hierarchies=(Time); hierarchy Time caption='Time' levels=(year quarter month); hierarchy Time caption='Time' levels=(year month); level year caption='Year' type=year ; level quarter caption='Quarter' type=quarters ; level month caption='Month' type = months;


14

dimension prdln caption='Product Line' hierarchies=(prdln); hierarchy prdln caption='Product Line' levels=(prodtype product); level prodtype caption='Product Type'; level product caption='Product Name'; measure actual_sum caption='Actual Sum' stat=sum column=actual; measure actual_n caption='Actual Count' stat=n column=actual; measure predict_sum caption='Predict Sum' stat=sum column=predict; measure predict_n caption='Predict Count' stat=n column=predict; aggregation country prodtype year /name='Product Types by Country'; run;

CONCLUSION The SAS 9 OLAP Server is built to allow SAS OLAP applications to move out of departmental and specialized use into full scale enterprise information architecture. SAS 9 OLAP Server has the features to satisfy the back end and the server administrator, and the breadth of query functionality to satisfy the front end, the application implementer, and the end-user. Moving existing OLAP applications involves primarily logistical and organizational steps. Recoding can be kept to a minimum.

TRADEMARKS SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Other brand and product names are registered trademarks or trademarks of their respective companies.

REFERENCES Levine, Stuart B. 2002. “Designing SAS 9.1 OLAP Structures for Optimum Performance and Scalability.” Powerpoint Presentation. The Twenty-Seventh Annual SAS Users Group International Conference Proceedings, Orlando, FL http://support.sas.com/rnd/scalability/papers/SR-67.ppt Polzin, Jeff, Barbara Walters, Vicki Jones, and Bob Janka. 2002. “OLAP Server: Focusing on Performance.” The Twenty-Seventh Annual SAS Users Group International Conference Proceedings, Orlando, FL http://support.sas.com/rnd/scalability/papers/OLAP_Performance.pdf Ressler, Duane. 2002. “V9 OLAP: An Architectural Overview.” The Twenty-Seventh Annual SAS Users Group International Conference Proceedings, Orlando, FL http://www2.sas.com/proceedings/sugi27/p176-27.pdf Ryals, Michelle . 2003. “SAS® Metadata, Authorization and Management Services – Working Together for You.” The Twenty-Eighth Annual SAS Users Group International Conference Proceedings, Seattle, WA http://www2.sas.com/proceedings/sugi28/175-28.pdf SAS Institute Inc. 2003. SAS OLAP Server: MDX Guide. Cary, NC: SAS Institute Inc. SAS Institute Inc. 2003. SAS OLAP Server: : Administrator’s Guide. Cary, NC: SAS Institute Inc. Spofford, George 2003. “SAS OLAP Server: Concepts and Excerpts from ‘MDX Solutions with Microsoft SQL Server Analysis Services’.” http://www.sas.com/apps/pubscat/bookdetails.jsp?catid-1&pc=59287

CONTACT INFORMATION Your comments and questions are valued and encouraged. Contact the author at:

Matthias Ender SAS Institute Inc. SAS Campus Drive Cary NC 27513 Work Phone: (919) 531 2148 Email: [email protected] Web: www.sas.com


Documents

Matthias Ender, SAS Institute Inc., Cary, NCsupport.sas.com/resources/papers/proceedings/proceedings/sugi29/029-29.pdfADDED FOR IMPLEMENTATION Legal Entity ID Date Of Birth Gender