SQL Server Windowing

Functions

Enrique Catalá Bañuls Mentor, SolidQ

MAP 2012 – Microsoft Technical Ranger – Microsoft Certified Trainer

ecatala@solidq.com

Twitter: @enriquecatala

© 2012 SolidQ

Enrique Catalá Bañuls

• Computer engineer

• Mentor at SolidQ in the relational engine team

• Microsoft Technical Ranger

• Microsoft Active Professional

• Microsoft Certified Trainer

© 2012 SolidQ

ecatala@solidq.com

Twitter: @enriquecatala

3 © 2012 SolidQ

About SolidQ…

• SolidQ sets the standard for information management services by providing the world's most trusted and reliable business intelligence, data management, collaboration and custom solutions for Microsoft's cloud and on-premise platforms. SolidQ is recognized as an expert among its peers:

• 80 of the world’s top information experts in our BI and data management team, including more than 30 SQL MVPs

• Authored over 30 industry leading books

• Frequent speakers at local and international SQL Server technical conferences such as TechEd around the world, the BI Conference, SQL PASS Summit, and SQL Saturdays

• Serving over 800 clients in 22 countries

• We are so confident in the capabilities of our team and the value we provide, we unconditionally guarantee your satisfaction with our services.

• For more information visit www.solidq.com

SolidQTM Journal

• Free, monthly e-magazine providing answers you need to be successful with

your Microsoft SQL Server, Business Intelligence, and software development

solutions

• Featuring in-depth technical articles, real-life case studies, practical columns,

and seasoned perspectives from our lineup of influential global experts—

both inside SolidQ and around the community

• Helping busy IT pros make the best decisions about which technologies to

use when and where, how to get the most out of their systems, and how to

be more productive and effective in their organizations

• Read today at www.solidq.com/sqj

Real Practices for

the Real World

© 2012 SolidQ – Think fast, move fast 4

Objectives of this session

• Introduction

• The past, the present and the future

• Window Functions

• Why do we need window functions?

• Syntax

• Performance

• Why almost complete?

• Q&A

5 | © 2012 SolidQ

Window Functions

• Window function

• function applied to a set of rows defined by a window

descriptor and returns a single value for each row

from the underlying query

• Window descriptor

• Define which rows will be used with the function

© 2012 SolidQ

Objectives of this session

• Introduction

• The past, the present and the future

• Window Functions

• Why do we need window functions?

• Syntax

• Performance

• Why almost complete?

• Q&A

7 | © 2012 SolidQ

The past, the present and the future

• The SQL 2000 way

• SQL Server 2000 does not provide any syntax to support windowing functions

• The SQL 2005 way

• SQL Server 2005 to 2008R2 introduced a partial implementation of the windowing functions using the OVER clause

• The SQL 2012 way

• SQL Server 2012 will have an almost complete implementation of the windowing functions

© 2012 SolidQ

The SQL 2005-2008R2 way

• SQL Server 2005 introduces

• OVER clause (partially implemented)

• New window functions

o ROW_NUMBER()

o RANK()

o DENSE_RANK()

o NTILE()

• SQL Server 2008/2008R2 added no new

implementations in this particular area

© 2012 SolidQ

The SQL 2005-2008R2 way

• OVER clause

• Two different implementations depending on type of windowing

• Aggregate window functions does not provide the ORDER BY clause

© 2012 SolidQ

The SQL 2012 way

• SQL Server 2012 are getting close to the full implementation of Windowing functions

• SQL Server 2012 introduces:

• OVER clause almost complete o Order by

o Window Frame

• 8 new window functions o LAG(), LEAD()

o FIRST_VALUE(), LAST_VALUE()

o CUME_DIST()

o PERCENT_RANK()

o PERCENTILE_DISC(), PERCENTILE_COUNT()

© 2012 SolidQ

OVER clause syntax

• SQL Server 2005/2008/R2

• SQL Server 2012

© 2012 SolidQ

Objectives of this session

• Introduction

• The past, the present and the future

• Window Functions

• Why do we need window functions?

• Syntax

• Performance

• Why almost complete?

• Q&A

13 | © 2012 SolidQ

Why do we need windows functions?

• Why do we need windows functions?

• What if we want to get the sum and the value column? (group and detail)

• If we are in SQL 2005/2008/2008R2…am i correct with this approach?

select id_table, value,

sum(value) as [sum(value)]

from table1 group by id_table Msg 8120, Level 16, State 1, Line 1

Column 'table1.value' is invalid in the select list because

it is not contained in either an aggregate function or the

GROUP BY clause.

id_table value

1 1

2 1

2 2

3 1

3 2

3 3

SUM(value)

1

3

6

id_table value sum(value)

1 1 1

2 1 3

2 2 3

3 1 6

3 2 6

3 3 6

Select sum(value) as [sum(value)]

from table1 group by id_table

© 2012 SolidQ

Why do we need windows functions?

• Now we know that the OVER clause exists…

• This must be our solution

select id_table,

value,

sum(value) over(partition by id_table)

from table1

id_table value sum(value)

1 1 1

2 1 3

2 2 3

3 1 6

3 2 6

3 3 6

© 2012 SolidQ

New syntax to solve old

problems

• Partitioning

• Ordering

• Slicing/framing

© 2012 SolidQ

Key concepts: All together Partition

UNBOUNDED

FOLLOWING

UNBOUNDED

PRECEDING

CURRENT

ROW

© 2012 SolidQ

Key concepts

© 2012 SolidQ

Key concepts: Partition

• Partition is like a group of

rows with similar

“characteristics” inside a set

© 2012 SolidQ

Key concepts: Partition

• Partition is like a group of

rows with similar

“characteristics” inside a set

© 2012 SolidQ

Key concepts: Slicing/Framing

• RANGE/ROWS

• ROWS | RANGE BETWEEN <B1> AND <B2>

• ROWS | RANGE <B1>

© 2012 SolidQ

Key concepts: Slicing/Framing

• B1 and B2 can be

• UNBOUNDED PRECEDING

• UNBOUNDED FOLLOWING

• CURRENT ROW

• For “ROWS” clause only o <scalar expression> PRECEDING

o <sclara expression> FOLLOWING

• Note

• B1 <= B2 or NULL will be returned o Except in COUNT() that 0 will be returned

© 2012 SolidQ

Key concepts: All together Partition

UNBOUNDED

FOLLOWING

UNBOUNDED

PRECEDING

CURRENT

ROW

© 2012 SolidQ

New windowing functions

• New analytic functions in SQL 2012

• offset

• LAG()

• LEAD()

• FIRST_VALUE()

• LAST_VALUE()

• Distribution

• PERCENT_RANK()

• CUME_DIST()

• PERCENTILE_CONT()

• PERCENTILE_DIST()

© 2012 SolidQ

DEMO 1: NEW FUNCTIONS

AND SYNTAX

Basics on framing, RANGE vs ROWS, new window functions

© 2012 SolidQ

Objectives of this session

• Introduction

• The past, the present and the future

• Window Functions

• Why do we need window functions?

• Syntax

• Performance

• Why almost complete?

• Q&A

26 | © 2012 SolidQ

Objectives of this session

• Introduction

• The past, the present and the future

• Window Functions

• Why do we need window functions?

• Syntax

• Performance

• Why almost complete?

• Q&A

27 | © 2012 SolidQ

Performance: IN-MEMORY vs ON

DISK

• Directly affected by Window Spool Operator

• Used to store the framing data

• In-memory vs disk-based worktable

• In-memory worktable

• Fastest

• Prerequisites: Framing defined using ROWS and working with <10k rows

• Disk-based worktable

• Default

• Used when the window was defined using RANGE

• Used when the windows was defined using ROWS and the frame has more than 10k rows

© 2012 SolidQ

Performance: Indexing

• Index order by the window partitioning

elements and then by the order elements

• Include the rest of the columns

• Optimized execution operators:

1. Scan

2. Segment

3. Window spool

4. Aggregate

© 2012 SolidQ

© 2012 SolidQ

DEMO 2: PERFORMANCE

Running Totals, in memory vs on disk, …

© 2012 SolidQ

Why almost complete?

• RANGE only supports

• UNBOUNDED PRECEDING

• UNBOUNDED FOLLOWING

• CURRENT ROW

• Ranking functions does not support framing:

• ROW_NUMBER, RANK, NTILE,DENSE_RANK, PERCENTILE_DIST, PERCENTILE_CONT

• Not implemented

• WINDOW aliases

• Clauses: NULLs FIRST and NULLs LAST

© 2012 SolidQ

Objectives of this session

• Introduction

• The past, the present and the future

• Window Functions

• Why do we need window functions?

• Syntax

• Performance

• Why almost complete?

• Q&A

32 |

THANK YOU!

Enrique Catalá Bañuls Mentor, SolidQ

MAP 2012 – Microsoft Technical Ranger – Microsoft Certified Trainer

ecatala@solidq.com

Twitter: @enriquecatala