Embed Size (px)
Citation preview
#JSS2015
Les journéesSQL Server 2015
Un événement organisé par GUSS
@GUSS_FRANCE
#JSS2015
Les journéesSQL Server 2015
Un événement organisé par GUSS
In-Memory 2016 Operational Analytics
David BarbarinFrédéric Pichaut
#JSS2015
Merci à nos sponsors
#JSS2015
Speakers
David Barbarinhttp://blog.developpez.com/mikedavem/[email protected]@mikedavem
Frédéric Pichaut
Senior Escalation Engineer EMEA, Microsoft France @FPichaut
#JSS2015
Columnstore ArchitectureC1 C2 C3 C4 C5
Row groupSet of rows (typically 1 million rows) SegmentContains values from one column for the row groupUnit of transfer between disk and memory
1- Encoding
Dictionary> Value: Base / Scale> Bit packing> RLE…
Vertipaq
2- Binary compression
(3)- Archive compressionXpress 8
+
+
C1 C2 C3
C2 C4
C1 C5
C3
C5
LOB
C1
C2
C3
C4
C5
Tuple mover
#JSS2015
Columnstore & Batch mode
Scan
Predicate
I/O significantly reduced by> Encoding & compression> Segment elimination
Next challenge: CPU usage – how to improve performance in this area?
Get row
Filtered row
X
X
X
Batch objectColumn vector
C1 C2Scan
Predicate
Get batch
Filtered batch
Process multiple rows in batch for efficiency (~1K rows)> Using of SIMD instructions > Optimized for 64 bits values of the register> Significant reduction in function calls = less CPU time(7 – 40x)
Reduction of CPU latency and cache misses > Optimized for CPU L2 cache and avoiding cache misses> Aggressive memory pre-fetch (sequential vs random)
#JSS2015
SQL 2016 Columnstore Improvements
• Tables:• Primary key, foreign keys, CDC, Triggers, temporal tables• Change tracking (NCCI only)• Transaction isolation level SI and RCSI• Availability groups and readonly secondary replicas support• CI introduced in operational analytics• Disk-based table + NCCI (updatable and filter capabilities)• CCI + nonclustered indexes • In-memory table + NCCI
• ALTER TABLE .. REORGANIZE for dealing with fragmentation• New or enhanced DMVS • sys.dm_db_column_store_row_group_*• sys.dm_db_index_* and sys.dm_xtp_*
#JSS2015
SQL 2016 Columnstore Improvements
• Batch mode• Sort operator• Aggregate operators and distinct • AVG, SUM, COUNT, CHECKSUM_AGG, STDEV, STDEVP
• Windows functions • COUNT, COUNT_BIG, SUM, AVG, MIN, MAX
• Analysis functions• LAG, LEAD, FIRST_VALUE, LAST_VALUE, PERCENTILE_CONT,
PERCENTILE_DISC, CUME_DIST, PERCENT_RANK• Batch mode with maxdop 1• Parallelized insert into the delta stores• String predicate pushdown
#JSS2015
Relational Table(Clustered Index/Heap)
Btree Index
Hot
Dele
te b
itmap
Delta rowgroups
Operational analytics: columnstore indexes & In-Memory tables
Unified view for OLTP & DW developers
High performance past period DW
Data Lifecycle
Real-time operational OLTP
Create Columnstore only on cold data – using filtered predicate to minimize maintenancecreate nonclustered columnstore index ….. where order_status = ‘SHIPPED’
CCI w / High compression
Dele
te b
itmap
Delta rowgroups
#JSS2015
demo
#JSS2015
Memory-optimized Table Filegroup
SQL Server.exe
Hekaton Engine: Memory_optimized Tables &
Indexes
TDS Handler and Session Management
OLTP In-Memory Architecture
Natively Compiled SPs and Schema
Buffer Pool for Tables & Indexes
Proc/Plan cache for ad-hoc T-SQL and SPs
Client App
Transaction Log
Query Interop
Non-durable Table
T1T4T3
T2
T1T4T3
T2
T1
TablesIndexes
Interpreter for TSQL, query plans, expressions
T1 T4T3
T4T3
Checkpoint & Recovery
Access Methods
Parser, Catalog, Algebrize
r, Optimize
r
Hekaton Compiler
Hekaton Component
KeyExisting
SQL Component
Generated .dll
Data File Group
T4T3
T4T3
#JSS2015
SQL 2016 - Improved Scaling - Storage• SQL Server 2014
• SQL Server 2016– Multiple offline checkpoint threads– Goal: 1GB/s of log generation [work in progress]
–
Single Offline Checkpoint Thread
Del Tran2(TS 450)
Del Tran3(TS 250)
Del Tran1(TS150)
Insert into Hekaton T1
Log in SQL Table
Del Tran1(TS150)
Del Tran2(TS 450)
Del Tran3(TS 250)
Insert into Hekaton T1SQL Transaction log
disk
SQL Transaction log Del Tran2(TS 450)
Del Tran3(TS 250)
Del Tran1(TS150)
Insert into Hekaton T1
Log in SQL Table
Del Tran1(TS150)
Del Tran2(TS 450)
Del Tran3(TS 250)
Insert into Hekaton T1
disk
Offline Checkpoint Thread
Offline Checkpoint Thread
Offline Checkpoint Thread
#JSS2015
SQL 2016 In-Memory Improvements
• Tables:• ALTER TABLE (offline, requires 2X memory)• Identity columns• Indexes on NULLable columns• COLUMNSTORE indexes• FOREIGN KEY, CHECK, UNIQUE constraints. • Change HASH index bucket_count through index REBUILD• Add/drop index supported• Stats improvements: auto-update and sampled stats• More than 8 indexes.• DML Triggers (Only AFTER triggers; no INSTEADOF; Natively
compiled)• 2TB of user data in durable tables (in SQL 2014 Max 256GB)• Full collations support
#JSS2015
SQL 2016 In-Memory Improvements• Native Procedure:
• ALTER PROC and sp_recompile (online and recompilation Durant execution)• Nested native procedures• Natively Compiled Scalar UDFs (Access from both native and interop)• Native Inline Table-Valued Functions• EXECUTE AS CALLER• Security and math built-ins• Full collations support in native modules
• Query surface area• Subqueries• LIKE operator• {LEFT|RIGHT} OUTER JOIN• Disjunction (OR, NOT)• UNION [ALL]• SELECT DISTINCT• Subqueries (EXISTS, IN, scalar)• Parallel scan for memory-optimized indexes• MARS (Multiple Active Result Sets) support• TDE (Transparent Data Encryption) (on-disk data files encrypted once TDE is enabled)
#JSS2015
Remaining In-Memory Unsupported FeaturesTables• DDL triggers and Transactional DDL• Data types: XML, Spatial, CLR types,
datetimeoffset, rowversion, sql_variant• ALTER TABLE ONLINE• Cross-container transaction limitations:
snapshot/snapshot, serializable/serializable; SAVEPOINT
• Online Migration of disk-based tables to memory-optimized
Interop: cross-database queries, MERGE INTO, locking hints
Native Compilation• CASE, MERGE, JOIN in UPDATE/DELETE;
DML with OUTPUT; VIEWs• Natively Compiled Table-Valued Functions• Automatic and statement-level recompile• Access to disk-based tables• Must be schema-bound; no dynamic T-
SQL• Parallelism; limitations in query
operators (hash join/agg, merge join)Management: replication; DB snapshot; CDC; compression
#JSS2015
In-Memory OLTP Table
Updateable CCI
TailDRT
Range Index
Hash Index
TailLike
Delta rowgroup
HotRelational Table
(Clustered Index/Heap)
Btree Index
Hot
Dele
te b
itmap
Delta rowgroups
Operational analytics: columnstore indexes & In-Memory tables
Unified view for OLTP & DW developers
High performance past period DW
Data Lifecycle
Real-time operational OLTP
Create Columnstore only on cold data – using filtered predicate to minimize maintenancecreate nonclustered columnstore index ….. where order_status = ‘SHIPPED’
CCI w / High compression
Dele
te b
itmap
Delta rowgroups
#JSS2015
demo
#JSS2015#JSS2015
Les évaluations des sessions,c’est important !!
http://GUSS.Pro/jss
#JSS2015
Merci à nos volontaires…
#JSS2015#JSS2015
