Building a Data Warehouse from SAP using iWay and WebFOCUS

O. Julian Plys PMP, Sunoco, Inc.Carole Benoit, Tek Systems, Inc.

What you will learn

Why create a DW outside of SAP?

How can you use iWay tools for this

How to structure your data for easiest extraction and efficient reporting

What WebFocus BI tools to use

The story of how we did this at Sunoco

Why create a DW from SAP?Control & flexibility over data design

history merged with new data merging with outside data conformed dimensionsSCD – Slowly Changing Dimension

Reduce the transaction load on SAP - separate transaction from queriesFlexibility in BI front end toolsCost reduction

SQL development vs. BW developmentLicense costs – SQL Server vs BW

A Little Data Warehouse History

1980’s – the Information Center

1990’s – the Data Warehouse

2000’s – New Tools, New SpecialtiesCRM – Customer Relationship Management

MDM – Master Data Management

Data Quality; Data Cleansing

Business Intelligence tools

It was 20 Years ago today…

IBI taught us how to play…

The Sunoco Decision

We wanted the flexibility

We wanted the cost reduction

Decision: Build a reporting oriented Data Warehouse alongside SAP implementation. Use WebFocus reporting and BI tools to expose SAP + other data to end users

The Action Plan

Connections to SAP – iWay

ETL processes shared between iWay and SSIS

Data Warehouse design based on Dimensional modeling techniques

Structured Reports

OLAP and AdHoc Tools – Report Assist

The iWay SAP Query Adapter

SAP Environment

WebFocus / SharepointWeb Servers WebFocus

Reporting Server

`

Developer

BrowserSharepoint

Enterprise DW SQL Server

SAP Appl Server SAP DB Server

iWay SAP R/3 Query Adapter

-Talks to application for data and other logical structures-Can perform real-time extract-SAP metadata

iWay Adapter Mgr --Moves data from SAP using Query Adapter to connect

Reports and Reporting ApplicationsAnalytical Processing

ETL Processes

Move SAP data to Staging Area (MSSQL) in raw form (overnight and near real time)

Use ETL tools to load dimensional structures (SSIS)

Load dimensions first, then measures

Combine with outside data while loading

Error check and Log while loading

Data Design Principles

Dimensional models simplify SAP complexity

Further complexity simplified in views

Masters created from views

Dimensions and measures joined using native SAP keys

Can perform SCD (Slowly Changing Dimension)

Before – SAP Structure

VIQMELNotification Header

VIQMELSTPM Notification Selection by Status

VIQMFEPM Notification - Item

VIQMMANotification: Activities

VIQMSMNotification – Tasks

VIQMURPM Notification: Item Cause

CategoryNotification # is PK

Join via OBJNR

Items

VIQMFE2 is abbreviated version of this table

VIQMFEL = VIQMFE + VIQMEL

VIQMAMALActivities + Actions

Code GroupActivity Text = Short TextCodingActivity CodeLong Text Indicator = X

STXH - STXLObject = QMMANotification # + ItemID = LTQM

STXLLong Text field not accessible to iWAY

Long Text

Notifications StructureSAP

AFTER – DW Structure

F_NOTIFICATION

Material

Customer

Division

Plant

NOTIFICATION Status

Carrier (D_VENDOR)

NOTIFICATION Type

NOTIFICATION Priority

WebFocus Reporting

MRE Dashboard for standard reports

Every Dimensional Structure / View becomes a Report Object

Each report documented on Dashboard page

Metadata is built into the WebFocus Master

Metadata comes from SAP Query Adapter (in English !)

WebFocus BI Tools

OLAP Reports become embarrassingly easy to create with dimensional model and metadata in Master File Description

Every View becomes a Report Object

End User training customized with existing report objects

Dramatis Personae

1 Project Manager1.5 Analysts1 Data Designer / ETL Specialist1 Report DeveloperThe Cavalry Charge at the end

4 additional developers 2 additional analysts 3 IBI consultants

Time FrameJan 2006-Apr 2006 Interviews, Environment set

up

May 2006-Aug 2006 Analysis, Requirements

Sep 2006 – Nov 2006 Development, Testing and Delivery

Dec 2006 – Feb 1, 2007 Intense Dev, Test, Deliver

Feb 2007 – Apr 2007 User training, New Requirements, Project wrap-up

More InformationiWay Manual = Adapter Administration for UNIX, Windows, OpenVMS, i5/OS, and z/OS 7.6.4 – Chapter 54 Using the Adapter for SAP R/3-ECC

Ralph Kimball Books on DW design, ETL:The Data Warehouse Lifecycle ToolKit, Ralph Kimball et al., John Wiley & Sons, 1998 – 2nd edition availableThe Data Warehouse ETL ToolKit, Ralph Kimball et al., John Wiley & Sons, 2004The Data Warehouse Toolkit 2nd Edition, Ralph Kimball et al., John Wiley & Sons, 2002

Questions