Introduction to Columnstore Indexes

Introduction to Columnstore Indexes

Jason Strate

e: [email protected]

e: [email protected]

b: www.jasonstrate.com

t: StrateSQL

Resources jasonstrate.com/go/indexing

Introduction

MAKING BUSINESS INTELLIGENT www.pragmaticworks.com


• Industry leaders in Microsoft BI and SQL Server Platform

• SQL Server Professionals - PASS Board of Directors, Speakers, Authors and MVP’s

• National Sales Team Divided by Microsoft Territories

• National System Integrator (NSI)• Gold Certified in Business Intelligence and Data

Platform• Platform Modernization/Safe Passage• Premier Partner for PDW SI Partner Program

MS PDW Partner of Year FY13Frontline Partnership Partner of the Year for Big DataExecutive sponsor - Andy Mouacdie, WW sales director PDW

• Over 7,200 customers worldwide• Over 186,000 people in PW database for demand

generation

About Pragmatic Works


Everyone wants fast queries


Sometimes Normal Doesn’t Make Sense

We need to change what we do


To get things back to normal

Agenda

Introduction Columnstore BasicsSQL Server

Columnstore

Non-Clustered Columnstore

Clustered Columnstore

Summary

Agenda


Columnstore



Summary

Session Goals

• Identify differences between rowstore and columnstore indexes

• Describe implementation of columnstore index in SQL Server

• Define pros and cons with columnstore indexes

• Demonstrate use of columnstore indexes


Agenda


Columnstore



Summary

Index Problems

• Indexes traditionally are row-based

– All columns for row stored together

• Any column read brings over all columns

– Impacts IO, CPU, and memory

– Typically 15% of DW columns in queries

• Databases growing 10x every 5 years

• Tuning for over indexing, under indexing, and bookmark lookups


Columnstore Basics

• Change data storage from row to column based

– Rowstores

– Columnstores

• Data grouped by columns

– One column per grouping (segment)

– Data access at columns level

– Only return columns required for query



Say What?!

Traditional Index


Columnstore Index


Columnstore Index


Why Columnstore?

• Improved query performance

– Column centric reads

– Heavy data compression

– Processing by batch versus row

• Improved resource utilization

– Smaller index space

– Less memory required


Agenda


Columnstore



Summary

SQL Server Columnstore

• Introduced to improve data warehouse performance

• Available since SQL Server 2012

• SQL Server 2012

– Non-clustered columnstore

• SQL Server 2014

– Clustered columnstore


Columnstore Attributes

• Introduced in SQL Server 2012• Supports batch mode processing• SQL Server 2012

– Non-clustered columnstore– No key columns– Support partitioning– Read only index

• SQL Server 2014– Clustered columnstore– Columnstore archive format– Read\write when clustered


Batch-mode Processing


1 record versus 415 records

Columnstore Restrictions

• Cannot be a unique index.• Cannot be created on a view or

indexed view.• Cannot include a sparse

column.• Cannot act as a primary key or

a foreign key.• Cannot be created with

the INCLUDE keyword• Cannot include

the ASC or DESC keywords for sorting the index

• No seek operations

• Cannot contain a column with a FILESTREAM attribute

• Limited data types– Except varchar(max),

nvarchar(max), binary, varbinary, ntext, text, image, uniqueidentifier, rowversion, timestamp, sqlvariant, xml, CLR types

• Feature restrictions– Replication– Change tracking– Change data capture– Filestream


Agenda


Columnstore



Summary


• Available in:– SQL Server 2012– SQL Server 2012 Parallel Data Warehouse– SQL Server 2014

• One non-clustered columnstore per table– Support tables heap or clustered index

• Add in all columns from table– No key columns

• Best support star-join style queries• Avoid OUTER JOIN and NOT IN

– Prevent batch processing


Updating Non-Clustered Columnstore

• Drop the columnstore

• Perform DML

• Recreate columnstore

Drop and Rebuild

• Create new partition with modified data

• Create columnstore

• Switch in partition

Partition Switch

• Apply changes to separate table

• UNION ALL columnstore or change table

• Manage DML through trigger

Partitioned View


NON-CLUSTERED COLUMNSTORE

Demo


Agenda


Columnstore



Summary


• Available in:– SQL Server 2012 Parallel Data Warehouse

– SQL Server 2014

• One clustered columnstore per table– Is primary storage for data

– Reduces space requirements

• Can leverage columnstore archive format

• Same design patterns as non-clustered

• Preferred over non-clustered columnstore


CLUSTERED COLUMNSTORE

Demo


Agenda


Columnstore



Summary

Summary

• Columnstore indexes change data can be accessed

• Provides increased data compression

• Retrieves only the columns that are needed

• Primary benefit in data warehouses

– Slowly or unchanging data


More Information

Expert Performance IndexingFor SQL Server 2012

Jason StrateTed Krueger

OverviewStatistics

MaintenanceTools

Analysis

http://amzn.com/1430237414


ServicesSpeed development through training, and rapid development services from Pragmatic Works.

ProductsBI products to covert to a Microsoft BI platform and simplify development onthe platform.

FoundationHelping those who do not have themeans to get into information technologyachieve their dreams.

For more information…Name: Jason Strate

Email: [email protected]

Blog: www.jasonstrate.com

Resource: jasonstrate.com/go/indexing

Need Help? jasonstrate.com/go/vmdba

Technology

Introduction to Columnstore Indexes