Upload
sap-technology
View
1.313
Download
8
Embed Size (px)
Citation preview
SAP ASE 16 SP02 Performance FeaturesReinoud Reynders (UZ Leuven)
Ashok Swaminathan (SAP)
January 2016
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 2Internal
SAP ASE 16 SP02 Performance Features
• Overview of SAP ASE 16 SP02 Performance Features
• Case Study - Uz Leuven Experience With ASE 16 SP02
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 3Internal
Disclaimer
This presentation outlines our general product direction and should not be relied on in making a purchase decision. This presentation is not subject to your license agreement or any other agreement with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or to develop or release any functionality mentioned in this presentation. This presentation and SAP's strategy and possible future developments are subject to change and may be changed by SAP at any time for any reason without notice. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this document, except if such damages were caused by SAP intentionally or grossly negligent.
Overview of SAP ASE 16 SP02 Performance Features
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 5Internal
SAP ASE Investments In High Performance
Version Features Benefits15.7 • ASE In-memory Database (IMDB) - entire database in
memory, but without support for durability of transactions• Typical usage : IMDB as temp DB or use of IMDB for fast
computations without the need to persist changes• Performance increase by 2X (up to 10x in some customer cases)
16.0 • Run Time Logging Enhancements (Buffer Unpinning)• Metadata Management Enhancements• Lock Management Enhancements
• Allowed scale up beyond 32 cores (up to 80 cores), near linear scaling
• Achieved over 1 million/minute business transactions
16 SP02 • Compiled Queries• Transactional Memory• Lockless Data cache• Latchfree Btree• Non Volatile Cache Management
• Latency and Throughput benefits
• Investments starting from ASE 15.7• Early vendor to introduce In-Memory technology• Continued focus with each release
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 6Internal
ASE 16 – Start Of High Performance Focus
16 32 48 64 800
200000400000600000800000
10000001200000
Scale-up Performance
Transactions per minute
> 1 million tpm
Engines
Environment
SUSE Linux Enterprise Server 11
Platform: 8 sockets/80 cores Intel® Xeon® [email protected] with 1TB RAM
Feature Description
Scale up on large SMP systems
• Minimize locking/latch contention, in highly concurrent environments• Enhancements in the area of run time logging optimization, metadata management, lock management
Partition level Locking • Granular locking for partitioned table• Enables Concurrent DDL and DML operations on a table
Dynamic Thread Assignment
• Allows executing query plans in parallel with fewer resources/threads• If # of threads < work units, threads completing execute remaining tasks
Index Compression • Index compression enabling storage savings for large indexes • Completes current capabilities - data compression (row/page), LOB, backup
Optimized Star Join Queries
• Allows hints in the join syntax• Can use the syntax “plan (use fact_table tablename)” –query processing optimizations related to star join
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 7Internal
SAP ASE 16 SP02 – High Performance Features
To Be Developed
T/SQLJDBCODBC
XOLTP QP
Parallel Dynamic ThreadingCode Compile
MVC
C
In-memory Row Store
Main MemoryLockless Metadata
Transaction Memory
Kernel Threading
DiskFlash Disk
Upcoming New In SP02
Buffer Cache
Scripts
Available
Latch Free BTree
Lockless Data Cache
Simplified Native Access Plans (Compiled Queries)• Compiled query plans – faster execution• Transparent to applications and users
Latchfree B-Tree on Indexes• Reduces contention• Increases concurrency and performance
Lockless Buffer Manager• Decreases cache contention• Increases concurrency and performance
Transactional Memory• Minimize contention leveraging hardware for identifying memory
conflicts
Non Volatile Cache Management• Leverage SSD for storing frequently accessed/updated pages
Latch Free BTree
Lockless Data Cache
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 8Internal
Simplified Native Access Plan (SNAP) - Overview
A new query plan execution feature Avoids repetitive query exec code generation from the physical
operators output from query optimization– Works ono Cached SQL Statements in statement cacheo Statements within stored procedureso Fully prepared SQL statements
Available on x86-64/amd64 Linux
Transparent to the userEnabled by configuration parameter; ASE uses SNAP automatically when possible.Unsupported plans continue to use lava execution engine.
SELECT {column list}FROM table COND1 due_dt <=getdate()COND2 (AND) recv_date is null
SELECT {column id’s & datatypes}FROM objid=123456COND1 col_id=3 (dt) >= (dt) ‘Jan 1 2015’COND2 (AND) col_id=4 (dt) IS NULL
SQL Parsing
Normalization
Pre-Processing
Query Optimization
Native Access Plan (Query Exec Code)
Query Compilation
TDSLANG select * from table where due_dt =getdate() and recv_date is null
Receive Buffer
Query Execution
Query Execution
SNAP
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 9Internal
Latch Free BTree
Latch is used to ensure physical consistence of page between DMLs and query.
Minimize contention where SH_LATCH and EX_LATCH block each other, when multiple threads need to synchronize between each other when modifying/reading index pages
Logical lock semantics not changed
BLOCK
page
Query DMLs
SH_LATCH EX_LATCH
Select * from customer where money > 5000 Insert into customer values(6000)
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 10Internal
Lockless Data Cache
Cache ContentionAccess to pages in named caches are through hashtables. Spinlocks protect buckets in named cache’s hashtable.
Contention on cache spinlock can be resolved by Moving object(s) to separate named caches
or Partitioning the cache
For example, lockless data cache can help when there is significant contention on a single cache partition but not significant contention on other cache partitions
This means less waits and spins on spinlocks, and thus lower contention. In turn lower CPU utilization, higher performance and improved scalability
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 11Internal
Transactional Memory
Objective: Increase concurrency (and minimize contention) by leveraging hardware for tracking memory conflicts as a
result of concurrent execution of code Background
– Spinlock (used to protect certain data structures) has been fragmented and distributed in the ASE code– Contention is on Lock – Many times, not on actual data which we want to protect
Solution Allow multiple threads to modify data structures, unless there is conflict at memory level Newer hardware from Intel (Haswell-EX/Xeon E7 V3 processor), and IBM (Power), keeps track of memory
reads/writes and memory contention By allowing concurrent processes to modify memory, and undoing the offending process/thread when there is a
memory conflict, concurrency is increased
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 12Internal
Non-Volatile Cache Management
DB ACCESS/STORAGE
Buffer Manager
DISK
• SSD cache will act as a intermediate storage.
• Buffer manager reads pages from disk (cold pages)
• Hot and warm pages that cannot fit in Buffer manager will be evicted to the SSD cache.
• Pages which are dirty will also be shifted to the SSD cache. This dirty pages then will be written in a delayed manner to the HDD disk.
• Frequently read pages will be in SSD
SSD CACHE
XOLTP
UZ Leuven – Case Study
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 14Internal
UZ Leuven
• The largest healthcare provider in Belgium
• 1955 Beds
• 9000+ Employees
• Centralized IT dept.: 145 staff• 55 Infrastructure & operations• 73 Patient – related software development• 17 Admin/logistics software development
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 15Internal
Clinical Workstation (CWS/KWS)
• Electronical medical record:• Patient mgmt., planning, order entry, e-prescriptions, bedside scanning, drug-drug interactions, lab
results, …
• Consolidated workspace for physicians, nurses, GPs and patients
• Development started in 1988
• Release cycle: 1 release/week
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 16Internal
KWS Infrastructure
• Mostly 2-Tier
• Database: • 20.764 stored procedures (1.123.544 lines)• 4.411 triggers (309.275 lines)
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 17Internal
Nexuz Health
•19 other hospitals (will) use our medical file system (the Clinical Workstation, CWS)
•1 instance, 1 shared medical file per patient
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 18Internal
Advantages - Medical
• One group of hospitals with the same procedures, the same quality proposal
• Bring the care closer to home
• Patients can transparently be referred between these hospitals
• Pooling of knowledge
• Transmural clinical pathways
• Clinical trials
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 19Internal
Nexuz Health Service Points
9643 beds total (37% of Flemish hospital beds; will grow to 45%)
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 20Internal
Expected Growth
• Current situation• Partner complexity per bed: 30% of UZ Leuven• Many partners waiting for startup
• Expectations next 24m• New partners startup• Partner complexity per bed -> 50%
• Need to grow 50% in less than 2 years
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 21Internal
Expected Growth: Example e-nursing
#registrations
#plans
18 months
36 months
6 months
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 22Internal
1 Shared Database
• Ergo: huge system, catering for +8000 bed• +25.000 named active users / 6.000 simultaneously (/3 min• 85M transactions per day / 8.000 processes sim/3m
• 150K rows inserted|updated|deleted/sec
• Database size: 12 TB
• Cache size: 140GB
• Cache hit ratio: >99%
• One of the largest worldwide
• 24/7/365 operation
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 23Internal
Scalability: Today’s Utilisation
Avg % (CPU busy + I/O) utilisation (64 engines):
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 24Internal
Challenge: Scalability
• Current ASE 15.7 SP130, threaded mode with 64 engines
• Better scaling in ASE 16.0, even without the new mem scale options
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 25Internal
UZ Leuven Benchmark Test
• We take the top 1400 queries with parameters from production
• We use real data (copy of production)
• Try to execute as many as possible transactions in 5 min time (use different engines)
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 26Internal
UZ Leuven Benchmark Test
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 27Internal
Extreme Scalability
X 8
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 28Internal
One Of The Challenges For Our Growth: Spinlock
• We have status tables that are used by everybody everytime• The status, the basic info and pointers of the current patients.• The status of each patient in a specific physical location
• This info is always in cache (important!) but is dynamic and it’s there where locking occurs
• Solution: lockless data cache
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 29Internal
Lockless Data Cache
• If we take the top 1400 queries:• Some have advantage, others not• Depends on the size of the table/data, is there cache contention or not?
• If we take queries in the top 1400 where lockless data cache fits well (smaller hot tables):• Select our status tables: we know our application and we know they can have an issue and are relative
small.:• Tables with a high amount of logical IO: detect by monOpenObjectActivity• Some examples: x#chemoVersieStatus : 60 000 rows , 2 MB / x#chemoSchemaDefinitie : 145 000 rows : 100 MB
• Performance gain 3 times!!!
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 30Internal
Improvement Using Lockless Cache
One top query 1000 queries Status tables0
50
100
150
200
250
300
350
"normal" cache Lockless cache
3 x
+5 % +11 %
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 31Internal
Conclusion
• With SAP ASE 16 SP2• We can grow more than 8 times • Reducing of spinlocks• Perfect architecture for extreme transactions• 100 % adapted for multicore and multi thread hardware
• With high performance features in SP02• A big benefit in general• For lockless cache we see improvements of 3x better performance.• Will give us less locking issues, better performance but more important: better scalability
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 32Internal
Reinoud Reynders [email protected]@reinoudreynders
Ashok [email protected]
Contacts