Database Platforms - UKISUG · PDF fileWhere does SAP Sybase ASE fit in? Database Platforms – Best ... Client Server Architecture ... DBA Cockpit provides simplified and better administration

Database Platforms Best Practices and What’s New

Darrell Gosden

Principal Consultant

SAP Global Database and Technology Services

2 UKI SAP User Group 2013

Speaker Introduction Darrell Gosden – My Background

• Long term Sybase Principal Consultant (>20 Years) prior to SAP acquisition (in

Oct 2012 in the UK).

• Project / Customer Engagement Delivery – Financial Markets Bias.

• Relational Database / Data Management (including Oracle and Ingres) and

associated technologies.

• SAP Sybase Database Technology Specialist with core skill set in SAP Sybase

ASE, SAP Sybase IQ, SAP Sybase Replication Server and associated design /

operational management products & tools.

Contact Details: Darrell Gosden SCN Entry or you can always find me through LinkedIn!

Mobile: +44 (0) 7977257074

E-Mail: [email protected]

Public SAP Community Blog(s) – or search “Darrell Gosden Running SAP Sybase Virtualized”:

Running SAP Sybase Data Management Products in a Virtualised World

The SAP Real-time Data Platform (RTDP)

“The Vision”

Database Platforms – Best Practices and What’s New Outline Agenda

Speaker Introduction Darrell Gosden – My Background

http://scn.sap.com/people/darrell.gosden

mailto:[email protected]

http://scn.sap.com/community/services/blog/2012/12/10/running-sap-sybase-data-management-products-in-a-virtualised-world



1. The SAP Real-time Data Platform (RDTP) “Vision” explained.

2. SAP Sybase Adaptive Server Enterprise (ASE) – What is it?

3. Migrating to SAP Sybase ASE.

4. ASE Customer Success Stories.

5. SAP Sybase IQ (IQ) – What is it / What’s New?

6. IQ Customer Success Stories.

7. The SAP Sybase IQ Near-line Storage (NLS) Solution – Overview.

8. SAP Sybase IQ NLS – The Rapid Deployment Service (RDS).

Please feel free to ask questions throughout the session.



The SAP Real-time Data Platform (RTDP)

“The Vision”


The SAP Real-time Platform (RDTP) The “Vision”


The SAP Real-Time Data Platform (RTDP) Where does SAP Sybase ASE fit in?


The SAP Real-time Platform (RDTP) Where does SAP Sybase ASE fit in?


SAP Sybase ASE Designed for mission-critical transactional environments

• Easily scale the amount

of users and data

• Execute millions of

transactions per minute

with terabyte-sized

databases

• Support rapid growth in

data volumes and

transaction levels

• Easy to maintain,

minimized downtime

• Automated tools for

monitoring and

diagnostics

• Reduce operational risk

by ensuring data security

and system stability

• Reliable and responsive

even under the most

punishing workloads

• More efficient use of

servers and storage

• Improve efficiency by

reducing administrative

tasks

Performance Efficiency Usability


SAP Sybase ASE Designed for mission-critical transactional environments


Sybase ASE – A Quick History

Certification with SAP Business Suite & NetWeaver

DBMS Clustering Database Clustering, Cluster Workload Mgmt,

Virtual Resource Management

Commodity & Low Cost Linux, Java-in-the-Database, XML

Heterogeneous Data Mgmt Heterogeneous Data Mgmt., Replication

Distributed Query Processing

Client/Server Client Server Architecture,

Stored Procedures, Triggers

1980S

1994: Microsoft releases SQL Server 4.2 for NT, MS & Sybase Split Marketing SQL Server

1990S 2009: Leader in Forrester Wave for Enterprise Database Management Systems

2000S

2010: “Best enterprise data management vendor on Wall Street four years running.”

2010S

IDC Finds ASE’s TCO to be 28% Less Than Other RDBMS Database Platforms – Best Practices and What’s New Outline Agenda

SAP Sybase ASE – A Record of Innovation


IDC Finds ASE’s TCO to be 28% Less Than Other RDBMS

IDC research conducted in-depth interviews with organizations to develop insights into the true costs associated with deploying ASE in comparison with other RDBMS products.

Sybase ASE Five-Year Savings per 100 Users

IDC White Paper (Dec 2011): Calculating the True Cost of RDBMS Ownership and How Sybase ASE Stacks Up: A Guide for SAP Business Suite Users

“Sybase ASE installations were

more efficient than the other

systems, requiring 24% less

storage capacity and running on

29% less server processor

power.”

“Sybase ASE required

27% fewer IT staff resources

overall”


IDC Finds SAP Sybase ASE’s TCO to be 28% Less Than Other RDBMS


Migrating to SAP Sybase ASE Best Practices Thoughts – A (Very!) Quick Review

• Database Schema - Sybase PowerDesigner is your friend!

• Reverse Engineering.

• Schema Migration.

• Data Migration Approaches

• Big Bang Materialisation (Flat-file, ETL tool, Sybase CIS & “proxy tables”).

• Trickle Feed / Incremental (Flat-file, ETL tool, SAP Sybase Replication Server, Sybase CIS).

• Continuous Data Movement – SAP Sybase Replication Server.

• SQL / “code” Migration

• Hand craft / “home brew”.

• Tools provided as part of SAP Services packages – semi automated – targeted at Oracle to ASE.

• Parallel Running

• SAP Sybase Replication Server.


Migrating to SAP Sybase ASE Best Practices – A (Very!) Quick Review


Migrating to SAP Sybase ASE Best Practices – SAP Services Can Help You

A number of packaged SAP Services

offerings are available to support your

migration to SAP Sybase ASE

including:

• RDS Rapid Database Migration to SAP

Sybase ASE (inc. SAP applications).

• RDS Rapid Database Migration POC to

SAP Sybase ASE – to support a

modest proof-of-concept (PoC)

exercise. “Try before you buy!”.

• Database Migration of Custom

Environments to SAP Sybase ASE.

Please contact your SAP Account

Executive for fuller details – or –

myself during this conference if you

prefer.


Migrating to SAP Sybase ASE Best Practices – SAP Services can help you!


Migrating to SAP Sybase ASE Best Practices – Additional Migration Support

Go to the Public SAP Community Network (SCN) SAP Services Content Library

(http://scn.sap.com/docs/DOC-34352) where you will find an excellent series of

blogs by my college Jonathan Wesley-James on planning and executing SAP

Sybase ASE Migrations:


Migrating to SAP Sybase ASE Best Practices – Useful Blogs

http://scn.sap.com/docs/DOC-34352




Day & Zimmermann Engineering, Construction and Operations

“ As part of our technology transformation

initiative we migrated our SAP applications

running on the Oracle database to SAP

Sybase ASE. This project virtualized 10 SAP

applications consisting of 70+ servers with

VMware on Windows. Significant cost

savings were achieved on Sybase ASE

annual maintenance costs vs. Oracle, and,

using compression technology in SAP

Sybase ASE, our storage requirements

were reduced by 30% for additional

savings. SAP provided expertise and training

to ensure the migration went smoothly, and as

a satisfied long-time SAP customer we believe

SAP’s strategic roadmap for its data

management products will meet our needs

well into the future.

Trushar Desai, Director, SAP Technology,

Day & Zimmermann

”


Day & Zimmermann Engineering, Construction and Operations


Redington Gulf IT distribution and support services

Business Challenges

High Oracle database licensing and maintenance costs

Solution Implementation

Migrate SAP ERP system off Oracle to SAP Sybase ASE

Deploy new SAP CRM system on SAP Sybase ASE

SAP Services assisting with ERP migration

Benefits

100% performance improvement over Oracle

50% more compression than Oracle

DBA Cockpit provides simplified and better administration

Lower database licensing and maintenance costs

Lower database TCO

2x less expensive

than Oracle database

100% performance

improvement over

Oracle

Redington Gulf replaces Oracle database

with SAP Sybase ASE for lower TCO

IN BRIEF

Redington Gulf is the largest

an end-to end supply chain

provider for IT distribution and

support services across the

Middle East, Africa and

Turkey.


Redington Gulf IT distribution and support services


SAP Sybase IQ – Position within Real-time Data Platform

“Near Line Storage” (NLS)* & many other scenarios possible

* Discussed later.

IQ

16.0


SAP Sybase IQ Real-time Data Platform (RTDP) Positioning


SAP Sybase IQ Engine

Mu

ltiple

x g

rid a

rch

itectu

re

Storage area network

Loading engine

In-database analytics

Text search

Web-enabled analytics

Info

rmatio

n life

cycle

managem

ent

Low latency, write optimized store

Role-based access control

Communications and security

N-bit and tiered indexing

Column indexing

subsystem

Query engine

Column store

Aggressive scale out

Fully parallel

Re

silie

nt

We

b b

ase

d a

dm

inis

tratio

n a

nd

mo

nito

ring

LDAP authentication

Hash partitioned tables

and data affinity

New Generation

PETABYTE SCALE store

New indexing technology for increased compression and fast, incremental batch loads

SAP Sybase IQ In a Nutshell!

High performance, fully parallel bulk loading

Enterprise-grade RBAC authorization and LDAP authentication

Comprehensive web-based monitoring and administration

Low latency, concurrent write optimized delta store

Re-engineered column store for extreme data volumes

Resilient Multiplex grid withstands network interrupts and server failures

Aggressively scaled out query engine

Intelligent query engine with data affinity for “shared nothing” performance on a flexible “shared everything” architecture


SAP Sybase IQ In a Nutshell!


SAP Sybase IQ IQ Column Store and Compression – A cornerstone capability

IQ stores data vertically

Values of same column are stored next to each other, column values of a row are in

separate locations

Easy reference for fixed length values

Efficient evaluation for very large number of rows with a limited set of columns

Rows exist as row numbers which mark a position in each individual column vector

IQ compresses values

Dictionary Compression for repeating values, Storage Compression for all values

IQ is optimized for very large databases

IQ 16 holds the Guinness World Record for Big Data Load (> 34 TB / hr.)

Massive parallel Multicore and Multinode query execution, table partitioning


SAP Sybase IQ IQ Column Store and Compression


SAP Sybase IQ 16 What’s New?

Notable Enhancements in SAP Sybase IQ 16:

Improved table concurrency – multiple “writers” against a single table – “row level

locking/versioning” (RLV) feature NOTE: Part of REAL TIME option for IQ > uses memory based row versioning and row merge back techniques very similar

to SAP HANA. Some usage restrictions apply.

(Even!) better storage compression (NBit dictionary tokenization etc) – targeted at

VARCHAR/VARBINARY/other variable length data

(Even!) faster data loading – higher degrees of internal parallelism possible and data load

query plans available allowing for low level tuning/optimization Breaking News!: Early “like for like” IQ 15.4 v. IQ 16 comparisons indicate 6-8x faster loads with load

rates of 5,000,000 > 7,000,000 rows/sec easily achievable with IQ 16!

Query (inc. DQP) processing improvements – partition based hash algorithms & others

LDAP support.

Enhanced user security/authorisations based on fine grained access privileges/roles.

HADOOP Integration…HOT!


SAP Sybase IQ 16 What’s New?


SAP Sybase IQ 16 HADOOP Integration – Of particular note!

The data returned from a Hadoop analysis can be integrated into an SAP

Sybase IQ database in several ways:

• ETL Processing – Bulk load data from Hadoop data stores into SAP Sybase IQ using the

open source utility SCOOP.

• Data Federation – Expose HDFS files as tables in an SAP Sybase IQ database that

participate in SQL queries. The HDFS files do not need to be loaded into SAP Sybase IQ.

• Query Federation – Allow SQL queries in SAP Sybase IQ to execute Hadoop processes

that return data that is incorporated into the SQL result set.

• Client-side Federation – Federate queries across SAP Sybase IQ databases and Hadoop

files using the TOAD© SQL tool.

New capabilities such as these help position SAP Sybase IQ as part of the “SAP HANA

Platform for Big Data” – next slides!


SAP Sybase IQ 16 HADOOP Integration – Of particular note!


SAP

Enterprise

data

Non-SAP

Enterprise

data

Mobile data

Machine data (Sensors, SCADA, Machine

Logs, Etc.)

Data Sources

Analytics & Applications

HANA In Memory

Transactional Planning &

Simulation

Graph Analytical

Predictive

Analysis

Spatial

Extended

Storage (IQ)

NLS Solution

Tie

red S

tora

ge (H

ot-w

arm

-cold

)

Sm

art D

ata

Access

Dashboard / Reporting in Real-

Time

Large Low Cost Data Platform

(Hadoop / IQ)

Stream Processing

Real-Time

Replication

Synchronization

Historical Data, Offline Batch Processes,

Model Training etc.

SAP HANA Platform for Big Data

Use Cases •Energy Optimization

•Predictive

maintenance

•Remote asset mgmt.

•Supply/demand

forecast

•Inventory mgmt.

•Route optimization

SAP HANA Platform for Big Data Where does SAP Sybase IQ fit in?

Real-time data stream

(Billions of events/day)

Millions of events/day

correlated with Enterprise

Data

Enable real-time operations,

analysis and actions


SAP HANA Platform for Big Data Where does SAP Sybase IQ fit in?


SAP Sybase IQ In Summary

Ecosystem

Most mature

column store

Comprehensive

lifecycle tiering

MPP queries + virtual

marts + user scaling

High-speed

loads

Structured +

unstructured store

Comprehensive ANSI

SQL with OLAP

Built-in full-

text search

In-database analytics with

MapReduce + simulator

Web 2.0

APIs

Big data OpnSrc

APIs

Optimized BI and EIM;

model and replicate

Development and

administration tools Predictive analytics

Packaged ILM

applications

Bradmark, Symantec,

Whitesands, Quest,

and ZEND

SAS, SPSS, KXEN,

Fuzzy Logix,

Zementis, and Visual

Numerics

BMMSoft,

SOLIX, and PBS

SAP Sybase PowerDesigner,

SAP Sybase Replication

Server,

SAP, ISYS, and Panopticon

App. services

DBMS Hadoop

and R


SAP Sybase IQ In Summary


Financial Services Customers Succeed with IQ for

Mission-Critical Applications

Customer IQ Deployment

US Securities &

Exchange

Commission

With Sybase IQ, the SEC built a flexible, scalable and

reliable data-rich warehouse that delivers faster load

times and query results, and more comprehensive

information views—all at a lower cost.

Citigroup With Sybase powering their data archive, Citigroup

users can now research historical data themselves,

whereas in the past IT specialists were required to do

so.

Morgan Stanley

(Spain)

With Sybase IQ, Morgan Stanley Spain now has access

to heterogeneous system-stored critical data with single,

unified delivery that has dramatically reduced analysis

response times.

ICICI Bank ICICI bank has achieved improvement in system uptime

and query performance over its previous Teradata

implementation, in addition to the host of other benefits

of the Sybase IQ data warehouse migration.

BNP Paribas

Securities Services

Sybase IQ has improved the robustness of its online

reporting system. Now the bank can compile larger amounts

of data and anticipate growth in its customer base by

proactively providing real-time data.


Customers Succeed with SAP Sybase IQ for Mission-Critical Applications


SAP HANA and SAP Sybase ASE / IQ Working Together

Smart Data Access – A interesting new capability!

SAP HANA SP6 was released recently and

one of the key features included in this

release is the Smart Data Access, which

allows customers to combine SAP HANA

data with other heterogeneous data sources

like Hadoop, Teradata, SAP Sybase ASE

and SAP Sybase IQ.

With Smart Data Access (SDA) you can

expose data from remote sources as “virtual

tables” and combine these with other data in

HANA physical tables using HANA models.

There are few restrictions on usage of SDA

currently:

• Data in virtual tables is read only.

• Virtual tables do not support BLOB/CLOB “type”

data.

• Virtual tables cannot be used in multi node SAP

HANA clusters.

• Some analytical views do not support virtual

tables.


SAP HANA and ASE / IQ Working Together Smart Data Access – A interesting new capability!


Near-line Storage (NLS) for Business Warehouse What is Hot and Cold Data?

Hot Data is permanently accessed with extremely short acceptable response

time.

Dashboard style information continuously updated.

Tactical level data.

Aberrations must be recognized with minimal latency to identify disruption and react.

Warm Data is occasionally accessed with a somewhat higher acceptable

response time

Kept as non-active data in primary BW database. Loaded into memory on demand, after

which full query performance is available

Cold Data continues to be relevant but is less frequently accessed or has a

more relaxed response time constraint.

Paper Report style information evaluated on demand.

Strategic level data.

Long-range evaluation for long-term decisions.


Near-line Storage (NLS) for Business Warehouse What is Hot and Cold Data?


Near-line Storage (NLS) for Business Warehouse Why would you use a SAP Sybase IQ based NLS with BW on SAP HANA?

SAP HANA platform can deliver the best possible Business Analytics

performance. There are, however, limitations.

Hard Limitations

In- Memory database size is restricted by (combined) RAM size of server machine(s).

Dynamic growth of data size may be faster than your reaction speed in improving hardware.

Soft Limitations

You may not want to cover the cost for the best possible technology for all of your Big Data.

You may want to protect your system from data growth by only keeping a moving time

window In- Memory.

You may want to pursue not the best possible performance but the best value for money.

Split your data between subsets demanding the best possible query

performance and subsets less performance critical.

For non-compressing DBMS, NLS will be more space efficient.


Near-line Storage (NLS) for Business Warehouse SAP Sybase IQ based NLS with BW on SAP HANA


Near-line Storage (NLS) for Business Warehouse Overview – BW on SAP HANA Example

In-Line Storage

Hot current data

Limited size

Rapid real-time access

Near line Storage

Cold Historical data

Massive volumes

Slightly slower access

SAP HANA

In-memory

Database

High Value,

Current Data

(- OR - OTHER

SUPPORTED

DBMS)

Sybase IQ

Disk Based

Database

Historical Data

Trend Analysis

SAP NetWeaver

Business

Warehouse


Near-line Storage (NLS) for Business Warehouse Solution Overview


Near-line Storage (NLS) for Business Warehouse How does NLS work?

Beside the primary BW database connection, a second connection to the SAP

Sybase IQ database is maintained

NetWeaver maintains and executes the aging rules

Aging happens along the time axis

Controls data transfer to the NLS instance

Cleans up data in the primary database

Query federation is handled by NetWeaver

Transparent to BW application logic

Evaluation of InfoProviders / DSOs is directed to primary connection, NLS connection or

both

Intermediate results are consolidated by NetWeaver where required




Near-line Storage (NLS) for Business Warehouse How does NLS work? – Simple Example

1. Data partitioned

by fiscal

Quarter/year

2. Partitions being moved to

Near-Line Storage are

locked for update

3. Once the data is moved to

Near-Line Storage, it is

deleted from BW




Business benefits Measurable success

Reduced SAP NetWeaver BW downtimes

Reduced SAP NetWeaver BW maintenance efforts

Reduced amount of data residing within SAP

NetWeaver BW

Faster querying and reporting

Reduced total cost of ownership for SAP HANA

IMPORTANT NOTE:

Traditional DBMS based BW implementations are

supported by the RDS.

Near-line Storage (NLS) for Business Warehouse Rapid Deployment Solution (RDS) - The Value Proposition

SAP NetWeaver BW Near-line Storage rapid-

deployment solution will:

Help gain a high performing near-line storage with

the functionalities of reporting, ETL, lookup

Reduce your total cost of ownership for SAP

NetWeaver BW (powered by HANA) by

implementing a clear and scheduled data aging

strategy

Enable faster online querying and reporting

Avoid further interfaces to near-line storage as

data is accessed by SAP NetWeaver BW

seamlessly

Decrease maintenance efforts for SAP NetWeaver

BW

Decrease system downtimes for SAP NetWeaver

BW



Near-line Storage (NLS) for Business Warehouse Rapid Deployment Solution (RDS) Overview


Near-line Storage (NLS) for Business Warehouse Rapid Deployment Solution – Migration to NLS – Options Overview

Option 1

Option 2


Near-line Storage (NLS) for Business Warehouse Migration to NLS – RDS Options


Near-line Storage (NLS) for Business Warehouse Rapid Deployment Solution - Go live in as little as 12 weeks!

A special step-by-step guide describes each activity

during the deployment

Service Name: Rapid deployment of SAP NetWeaver BW Near-Line Storage

Implementation Duration: 12 Weeks*

* Based on scope

** Provided as option and impacts overall efforts of the service

Core Offerings

Implementation of SAP NetWeaver BW Near-

Line Storage

Identification of objects to be archived

Installation and Configuration of SAP

Sybase IQ database

Installation of SAP Sybase IQ Client and

configuration of SAP NetWeaver BW to

Near-Line Storage

Create up to 10 DAPs (Data Archiving

Processes)

Create up to 2 Process Chains

Adjust settings for up to 10 Queries and 2

multiproviders

Optional Task**

Support Package Update


Near-line Storage (NLS) for Business Warehouse RDS - Go live in as little as 12 weeks!


Near-line Storage (NLS) for Business Warehouse

Rapid Deployment Solution – SAP Service Marketplace Database Platforms – Best Practices and What’s New Outline Agenda

Near-line Storage (NLS) for Business Warehouse RDS – SAP Service Marketplace

Documents

Database Platforms - UKISUG · PDF fileWhere does SAP Sybase ASE fit in? Database Platforms – Best ... Client Server Architecture ... DBA Cockpit provides simplified and better administration