Real-Time Business Intelligence with SQL Server 2005 Analysis Services

• What are the barriers to real-time

business intelligence (BI)?

• How can Microsoft SQL Server 2005

Analysis Services be used to make BI

more real-time?

What Will We Cover?

Difficulties of Real-Time BI

SSISSSIS

POS OLTP Cleanse andEnrich

DW

POS OLTP

UDMStaging

Cube

ValidationUDMProduction

Cube

Barriers to Real-Time BI

Barrier Solution

Historical view None

Coordination with business processes

across systems

None

Data quality management None

Integration of multiple data sources Heterogeneous Query Processing (HQP)

Data consistency Snapshot Isolation

More Barriers to Real-Time BI

Barrier Solution

Managing aggregates

Isolating the OLTP system from long-

running queries

Data Push & Proactive Caching

Knowing what has changed Notification services

Linking back to the source system Actions

Pushing Data into UDM

Data can be pushed directly into a Unified Dimensional Model

SQL Server 2005 Integration Services processing transforms

Includes fact and dimension tables

SSISSSIS

POS OLTP Cleanse andEnrich

POS OLTP

UDMProduction

Cube

Demo

Linking Integration Services (SSIS) and Analysis Services Directly

View an SSIS Package Run an SSIS Package Browse the Updated Cube

demonstration

Updating with Trickle Feeds

Trickle feeds can get data directly into the UDM

Integration Services updates the cube every few minutes

SSISSSIS

POS OLTP

POS OLTP

UDMProduction

Cube

Cleanse andEnrich

Building the Cube Directly

UDM can combine data from multiple sources

One of the underlying sources must be SQL Server

Not applicable for all scenarios

SSISSSIS

POS OLTP

POS OLTP

UDMProduction

Cube

Cleanse andEnrich

Continuously Changing Data

Problem Solution

How to handle updated data

Source data might be continually changing

How to ensure consistency during processing

Use Snapshot Isolation

Proactive Caching

• Policy-based managementHas source data changed?When to refresh?How to answer queries during refresh

• Proactive caching combinesOLAP query performanceReal-time data access as needed

• No more explicit “cube processing”

Proactive Caching – An Example

UDMUDM

MOLAPCache

OLTP

MDX

Analysis Services

Proactive Caching – An Example

UDMUDM

MOLAPCache

Events

OLTP

POS

SQL

Analysis Services

Proactive Caching – An Example

UDMUDM

MOLAPCache

NewVersion

OLTP

Data

Analysis Services

Using Policies to Refresh the Cache

UDMUDM

POS OLTP

POS OLTP

Policy-based refresh of the cache

UDMProduction

Cube

Demo

Using MOLAP and Reverting to ROLAP when Latency Exceeded

View Partition Settings Cause Latency Revert to ROLAP

demonstration

Proactive Caching Challenges

• EfficiencyHow to avoid overloading Analysis Services with frequent updates

How fast can the caches catch up?

• PerformanceHow to balance between latency and performance

• NotificationsIs the cache refreshed on change or periodically?How does AS know that the RDBMS has changed?

Policy Settings

Property Description

SilenceInterval After an update, for how long must there be a quiet time with no further updates before rebuild starts?

-1 (infinite) = no quiet time

SilenceOverrideInterval If no quiet time, start anyway after this time

-1 (infinite) = no override

ForceRebuildInterval How long after last cache was built should rebuild of a new cache always commence?

-1 (infinite) = no periodic rebuild

Latency How out-of-date can the cache be before reverting to ROLAP?

-1 (infinite) = never revert to ROLAP mode

Scaling Up

Problem Solution

How to handle large quantities of data

Re-creating the whole cache on every change is expensive

Use ROLAP

Use partitions

Use incremental cache updates to add data

Demo

Using Automatic MOLAP with Polling Queries and Incremental Refresh

View Cube Settings View Reports Add New Data

demonstration