Upload
phungxuyen
View
222
Download
0
Embed Size (px)
Citation preview
SAP BW Archiving with Nearline Storage at Esprit
Claudia Ottilige, Esprit Europe GmbH Dr. Michael Hahne, Hahne Consulting GmbH 27. Februar 2013
Agenda
• Company Esprit • Initial situation • NLS Best Practices • Solution options • Conclusions • Project results so far
SAP BW Archiving with Nearline Storage at Esprit 2
Agenda
• Company Esprit • Initial situation • NLS Best Practices • Solution options • Conclusions • Project results so far
SAP BW Archiving with Nearline Storage at Esprit 3
Esprit Company Overview
• One of the most well-known apparel brands in the world. • Founded in San Francisco in 1968 by Susie and Doug Tompkins • Offering style and quality made to last. Effortless style, authentic and
easygoing living are at the heart of the brand philosophy. • Company is represented in more than 40 countries • More than 1,000 directly managed retail stores • Distributes via more than 10,800 controlled space wholesale point-of-sale • Total selling space over 1 million square meters. • Since 1993 listed on the Hongkong stock exchange. • Group turnover: 30,165 m HK$ (FY 2011/2012) • Product lines: Casual, Collection, edc, others: accessoires, bodywear, shoes, kids, sports, de.corp • Distribution channels: Wholesale, Retail, Licences
Company overview
SAP BW Archiving with Nearline Storage at Esprit 4
Agenda
• Company Esprit • Initial situation • NLS Best Practices • Solution options • Conclusions • Project results so far
SAP BW Archiving with Nearline Storage at Esprit 5
6
Background / initial situation (I)
• Esprit‘s SAP BW has currently 16 TB of data, data volume still growing – nightly data loading times too long à reports are often available too
late for the users – potential recovery of the system would take several days – backup time very long à one night can become too short for a backup – in case of further growth of data volume à new backup infrastructure
needed – makes copy of whole production system to test-system inacceptable – database reorganisation takes very long – data storage cost could be reduced – assumption: SAP BW is not meant to archive data relevant for audits,
audit relevant data are stored in source systems
SAP BW Archiving with Nearline Storage at Esprit 6
Background / initial situation (II)
During the last 12 months the used dabase size increased from 14,5 to 16 TB. Average growth per month: 125 GB Several housekeeping activities had positive impact on the data volume.
SAP BW Archiving with Nearline Storage at Esprit 7
8
Data volume reduction by DVM
Data volume management (DVM) is a concept which tries to keep the data volume of the SAP BW system as small as possible. This includes:
– deletion of unused data – deletion of temporary data – data archiving (ADK) – near-line storage – database re-organisation – aggregation of data – data compression (e.g. Infocubes) – processes and guidelines to keep the amount of data as small as
possible – ...
SAP BW Archiving with Nearline Storage at Esprit 8
Archive Development Kit (ADK) versus Nearline Storage (NLS)
ADK (SAP BW 3.X) NLS (SAP NW 7.0 BI)
• Classical Archiving solution for Infocubes and ODS objects based on Archive development kit (ADK)
• Archived data must be reloaded into the SAP BW system when needed
• Cost reduction by using cheaper storage media and through data compression
• read-only
• Archiving solution for Infocubes and DS objects
• Direct access to NLS data for reporting and progagation
• Archived data does not need to be reloaded into the SAP BW system when needed
• Cost reduction possible through compression of data
• NLS data can be stored on file system, DBMS, also in ADK files
• read-only
SAP BW Archiving with Nearline Storage at Esprit 9
Agenda
• Company Esprit • Initial situation • NLS Best Practices • Solution options • Conclusions • Project results so far
SAP BW Archiving with Nearline Storage at Esprit 10
Data access vs. Data growth
• Typical Data Growth from year to year
• Typical Data Access vs. Data Growth
• With increasing Data Growth access likelihood changes significantly SAP BW Archiving with Nearline Storage at Esprit 11
0.5 TB 1.5 TB 2.5 TB
4 TB
12 TB
100% 90%
75%
30%
10% Frequently read Infrequently read Rarely read
Information Lifecycle Management
Online Database Storage Near Line Storage
Data Archiving
Frequently read/updated data
Infrequently read data
Very rarely read data
• Online: – Persistent in the Data Warehouse – Data Modelling and Layered Architectures are relevant – Housekeeping
• Nearline: – Transparent Access – „BW on a diet“
• Offline: – Classical
Archiving
SAP BW Archiving with Nearline Storage at Esprit 12
Roll-Up & Transformation
Process
Roll-Up & Transformation
Process
Data Load Process
Data Marts
Data Acquisition Layer
Data Integration Layer “Ready to use”Data è Feeding the “Data Marts“
Granular, untransformed Data è 1:1 mapping to ERP System
Analytical layer è Mainly aggregated data
50-70% of overall
data volume
Data Management Best Practice: Enterprise Data Warehousing
Archiving mostly daily
Archiving weekly or monthly
Archiving monthly to yearly
SAP BW Archiving with Nearline Storage at Esprit 13
Look-Ups in data flows
Data Warehouse Layer
Acquisition Layer
Reporting Layer
History Objects Staging ODS
Look up of historical data in
Update Rules
Nearline - Object
Adhoc reporting, Analysis Process
Designer
… Look-
Up
SAP BW Archiving with Nearline Storage at Esprit 14
Look-Up API
NLS - APIDB Interface
Data AccessAPI
Analysis Process
Single Point of access to all data
– archived and non-archived
NLS - APIDB Interface
Data AccessAPI
Analysis Process
Single Point of access to all data
– archived and non-archived
SAP BW Archiving with Nearline Storage at Esprit 15
DSO Look-up in Transformations with BW 7.30
• With BW 7.30 there‘s a DSO look-up as Transformation Rule available
• Reads also NLS partitions automatically
SAP BW Archiving with Nearline Storage at Esprit 16
7.30
NLS Access in Analysis Processes with BW 7.30
• With BW 7.30 the Source-Typ DSO in Analysis Processes reads the NLS Partitions automatically if required
SAP BW Archiving with Nearline Storage at Esprit 17
Data from NLS can be included as option
7.30
Data Management Best Practice: Integration of NLS in Reporting Layer
SAP BW Archiving with Nearline Storage at Esprit 18
Data Management Best Practice: Transparent Access in Queries with NLS
SAP BW Archiving with Nearline Storage at Esprit 19
NLS Support für MultiProvider with 7.30
NearLine Readmode • disabled at all • enabled at all • InfoProvider settings
7.30
SAP BW Archiving with Nearline Storage at Esprit 20
NLS Properties in a Query with 7.30
Fixed NLS Settings • read NLS • do not read NLS • see InfoProvider settings
7.30
SAP BW Archiving with Nearline Storage at Esprit 21
NLS Query Variables with BW 7.30
Variable NLS Settings • read NLS • do not read NLS • see InfoProvider settings
SAP BW Archiving with Nearline Storage at Esprit 22
Advantages of a Data Volume Management Strategy
• Performance – Faster Data Loads – Faster Queries
• Costs – Less storage costs – Less administrative effort – controlled system growth
• Availability – of data:
faster rollups and change runs etc. – of system:
less downtime for backups, upgrades etc. • Enterprise Data Warehousing
– „Data Marts on Demand“: more flexibility, faster rollout
SAP BW Archiving with Nearline Storage at Esprit 23
Agenda
• Company Esprit • Initial situation • NLS Best Practices • Solution options • Conclusions • Project results so far
SAP BW Archiving with Nearline Storage at Esprit 24
SAP Defined Integration Scenario NW-BI-NLS 04S - Nearline Storage 2004s for SAP NetWeaver BW
Certified Partners (June 2012)
(see also http://www.sap.com/ecosystem/customers/directories/SearchSolution.epx )
Informatica ILM Nearline for SAP BW (Former SAND Nearline)
• Reduce production storage costs by moving infrequently accessed data to a nearline data store.
• Does not require any database administration.
• Data is stored in encrypted files on standard file system, simplifying backup and restore processes as well as replication to other environments.
• Reduce non-compliance risk with continued access to data in immutable, compressed files - compression rates as high as 98%, with average of 90%+ compression rate.
• Contact: [email protected]
SAP NetWeaver BW Nearline Storage on DataVard OutBoard™
• DataVard‘s OutBoard™ is the all-in-SAP nearline storage solution • Built in ABAP, runs in your existing SAP landscape, no additional database or
hardware required • OutBoard™ is priced per productive system landscape. No matter how much data
you load onto it. • Compresses all InfoProviders, PSA and
Changelogs up to 95% • Contact: [email protected]
Dat
a Fl
ow C
ontr
ol /
Proc
ess
Cha
ins
Enterprise Query, Reporting & Analysis
Enterprise Data Warehouse
BW
Acc
eler
ator
Met
a D
ata
Rep
osito
ry /
Doc
umen
ts
Info
Obj
ects
/ M
aste
r Dat
a
Out
Boa
rd™
N
ear-
Line
Sto
rage
DataSource / PSA
Data Store Object
(volatile) Data Warehouse Layer (historical)
SAP NLS Interface
(Architected) Data Marts
Analytical Engine
Change Logs
…
Database
SAP NetWeaver BW Nearline Storage on PBS CBW • With CBW PBS provides an add-on solution for the data retrieval of archived/
nearlined BW data. • On the basis of the SAP ADK files, the CBW solution creates aggregates on the
archive data and (optionally, if needed) index structures to enable a fast retrieval option for the archived data directly from the end-user's original query.
• PBS CBW NLS IQ for Sybase Analytics Server IQ utilizes the Sybase Analytics Server IQ as a nearline database and offers extremely fast query response times.
• Contact: [email protected]
Agenda
• Company Esprit • Initial situation • NLS Best Practices • Solution options • Conclusions • Project results so far
SAP BW Archiving with Nearline Storage at Esprit 29
Conclusions (I)
SAP BW Archiving with Nearline Storage at Esprit 30
During the next 5 years, approx. 4 – 7 TB of data can be archived only for the integration layer. (data vary because of co-existance phase of two ERP systems)
In addition to NLS data, ~ 3 GB could be stored via ADK files, as data are older than requested retention times.
vs. PY: ∑Storage:
Conclusions (II)
§
Integration layer is the main focus for archiving. Reporting layer is already supported by BWA to increase performance, archiving of data is not a must (alternative: keep only aggregated data for historical data). PSA and extraction layer are deleted very promptly.
SAP BW Archiving with Nearline Storage at Esprit 31
Esprit‘s Evaluation (criteria)
SAP BW Archiving with Nearline Storage at Esprit
32
To compare different providers the following evaluation criteria have been applied:
• Is separate hardware necessary? • How is the Query performance in comparison to BWA? • How large is the compression rate? • How well does the software assists with the archiving process? • Reference customers • Support via OSS • Compatibility with HANA • Architectural fit with other archiving tools already available at Esprit • Is it possible to access productive data from the test system? • Company background
1. Data archiving / storing DSO & Cubes in NLS
2. Loading of data via DTP 3. Reporting on NLS data 4. Reloading of requests 5. Usage in process chains
1. OutBoard™ Cockpit § OutBoard™ Browser § OutBoard™ Analyzer § Statistics
2. Performance-optimization for fast access to NLS Data
3. Mass archiving / Automation 4. Storage Management (via
separate Tablespaces a.o.) 5. Archiving Request deletion 6. Archiving PSA & Change log
Datavard OutBoard
SAP NLS Interface OutBoard™ Features
ADK vs. NLS To have 2 archiving systems available increases complexity and cost therefore we decided to only implemented NLS solution. Motivation:
§ NLS is necessary anyway: • fast response to new reporting requirements • quick reload of cubes • quick reload of data for realignments • look-ups on archived data during data load possible
§ In case amount of data needs to be further reduced also in Reporting layer (e.g. because BWA is at it‘s limits) there are 2 possible strategies how to deal with it using the Datavard NLS solution:
à either data model can be adapted (aggregation of data) OR à data are saved in NLS, but are not made available for reporting (can be quickly retrieved if absolutely necessary à but probably performance issue)
SAP BW Archiving with Nearline Storage at Esprit 34
Agenda
• Company Esprit • Initial situation • NLS Best Practices • Solution options • Conclusions • Project results so far
SAP BW Archiving with Nearline Storage at Esprit 35
36
Project results up to now (I)
• Several housekeeping activities implemented in 2011 • Several workshops regarding archiving concept in 2011 • Data volume scoping by SAP in October 2011 • Analysis of SAP BW system by SAP in February 2012 • Data volume management workshop with SAP (1 day) in Feb. 2012 • Defined & prioritized workpackages to reduce SAP BW data
volume • Implemented some workpackages (e.g. deleted some large unused
DSO‘s)
SAP BW Archiving with Nearline Storage at Esprit 36
37
Project results up to now (II)
• defined all objects to be archived, distinguishing between: – Archiving development kit (ADK) or – Nearline storage
• defined business requirements for data retention times • Near-line Storage
– presentations by different nearline storage solution providers – received offers from solution providers – workshop with nearline storage expert – telephone call with reference customer – checked already available SAP archiving solutions at Esprit Europe (PBS
solution for current FI/CO system) • Calculated business case for NLS solution • Installed Outboard Software • Started with archiving of first DSO‘s in October 2012
SAP BW Archiving with Nearline Storage at Esprit 37
38
Summary
• Near-line storage is state-of-the-art alternative to classical archiving
• NLS can solve various problems: – Load run time – Query performance – Data volume and storage costs – Backup and recovery
• SAP data volume management workshop identifies the biggest potential issues and provides recommendations
• Quick return on investment (often less than a year)
SAP BW Archiving with Nearline Storage at Esprit 38
26.02.13 Copyright Hahne Consulting GmbH 2013 39
How to get started…
• Analysis Workshop / Fitnesstest – Objective: identify „Quick Wins“ / low hanging fruits – Benchmarking and Recommendations in areas of
• System Performance and Robustness • Data Management • Housekeeping
– ROI in a couple of months • Proof of Concept
– Test installation – Proof of Concept
• Project and usage – Analysis and design – Implementation – Operation – Check-up with Fitnesstest