Still using Windows 3.1? - modern-sql.com · ‣Literate SQL Organize SQL code to improve maintainability ‣Assign column names to tables produced by values or unnest. ‣Overload

Still using Windows 3.1? So why stick with

SQL-92?

@ModernSQL - https://modern-sql.com/ @MarkusWinand

SQL:1999

WITH (Common Table Expressions)

WITH (non-recursive) The ProblemNested queries are hard to read:

SELECT…FROM(SELECT…FROMt1JOIN(SELECT…FROM…)aON(…))bJOIN(SELECT…FROM…)cON(…)

Understand

this first



Then this...



Then this...



Finally the first line makes sense



CTEs are statement-scoped views:

WITHa(c1,c2,c3)AS(SELECTc1,c2,c3FROM…),

b(c4,…)AS(SELECTc4,…FROMt1JOINaON(…)),

c(…)

Keyword

WITH (non-recursive) Since SQL:1999




c(…)

Name of CTE and (here optional) column names





c(…)

Definition





c(…)

Introduces another CTE

Don't repeat WITH





c(…)

May refer toprevious CTEs




c(…)AS(SELECT…FROM…)

SELECT…FROMbJOINcON(…)

Third CTE






No comma!






Main query







Read top down


‣ Literate SQL

Organize SQL code toimprove maintainability

‣ Assign column names

to tables produced by valuesor unnest.

‣ Overload tables (for testing)

with queries hide tablesof the same name.

Use-CasesWITH (non-recursive)

https://modern-sql.com/use-case/literate-sql

https://modern-sql.com/use-case/naming-unnamed-columns

https://modern-sql.com/use-case/unit-tests-on-transient-data

http://modern-sql.com/use-case/literate-sql

http://modern-sql.com/use-case/naming-unnamed-columns

http://modern-sql.com/use-case/unit-tests-on-transient-data

WITH are the "private methods" of SQL

WITH is a prefix to SELECT

WITH queries are only visible in the SELECT they precede

WITH in detail: https://modern-sql.com/feature/with

WITH (non-recursive) In a Nutshell

http://modern-sql.com/feature/with

AvailabilityWITH (non-recursive)1999

2001

2003

2005

2007

2009

2011

2013

2015

2017

5.1 10.2 MariaDB8.0 MySQL

8.4 PostgreSQL3.8.3[0] SQLite

7.0 DB2 LUW9iR2 Oracle

2005[1] SQL Server[0]Only for top-level SELECT statements[1]Only allowed at the very begin of a statement. E.g. WITH...INSERT...SELECT.

WITHRECURSIVE (Common Table Expressions)

(This page is intentionally left blank)

WITHRECURSIVE The Problem

CREATETABLEt(idNUMERICNOTNULL,parent_idNUMERIC,…PRIMARYKEY(id))

Coping with hierarchies in the Adjacency List Model[0]


[0] Hierarchies implemented using a “parent id” — see “Joe Celko’s Trees and Hierarchies in SQL for Smarties”

CREATETABLEt(idNUMERICNOTNULL,parent_idNUMERIC,…PRIMARYKEY(id))




SELECT*FROMtASd0LEFTJOINtASd1ON(d1.parent_id=d0.id)LEFTJOINtASd2ON(d2.parent_id=d1.id)



WHEREd0.id=?





WHEREd0.id=?





WHEREd0.id=?



WITHRECURSIVE Since SQL:1999

WHEREd0.id=?

WITHRECURSIVEd(id,parent,…)AS(SELECTid,parent,…FROMtblWHEREid=?UNIONALLSELECTid,parent,…FROMdJOINtblON(tbl.parent=d.id))SELECT*FROMsubtree

Recursive common table expressions may refer to themselves in a leg of a UNION[ALL]:

WITHRECURSIVEcte(n)AS(SELECT1UNIONALLSELECTn+1FROMcteWHEREn<3)SELECT*FROMcte

Keyword

Since SQL:1999WITHRECURSIVE



Column list mandatory here




Executed first




Result sent there




Result visible twice




n---123(3rows)

Once it becomes

part of the final result




n---123(3rows)




n---123(3rows)

Second leg of UNION is executed




n---123(3rows)

Result sent there again




n---123(3rows)




n---123(3rows)

It's a loop!




n---123(3rows)

It's a loop!




n---123(3rows)

It's a loop!




n---123(3rows)

n=3 doesn't match




n---123(3rows)

n=3 doesn't matchLoop terminates


Use Cases‣ Row generators

To fill gaps (e.g., in time series), generate test data.

‣ Processing graphs

Shortest route from person A to B in LinkedIn/Facebook/Twitter/…

‣ Finding distinct values

with n*log(N)† time complexity. […many more…]

As shown on previous slide

http://aprogrammerwrites.eu/?p=1391

“[…] for certain classes of graphs, solutions utilizing relational database technology […] can offer performance superior to that of the dedicated graph databases.” event.cwi.nl/grades2013/07-welc.pdf

http://wiki.postgresql.org/wiki/Loose_indexscan

† n … # distinct values, N … # of table rows. Suitable index required

WITHRECURSIVE

http://aprogrammerwrites.eu/?p=1391

http://event.cwi.nl/grades2013/07-welc.pdf

http://wiki.postgresql.org/wiki/Loose_indexscan

WITHRECURSIVE is the “while” of SQL

WITHRECURSIVE "supports" infinite loops

Except PostgreSQL, databases generally don't require the RECURSIVE keyword.

DB2, SQL Server & Oracle don’t even know the keyword RECURSIVE, but allow recursive CTEs anyway.

In a NutshellWITHRECURSIVE

AvailabilityWITHRECURSIVE1999

2001

2003

2005

2007

2009

2011

2013

2015

2017


8.4 PostgreSQL3.8.3[0] SQLite

7.0 DB2 LUW11gR2 Oracle

2005 SQL Server[0]Only for top-level SELECT statements

GROUPINGSETS

Only one GROUPBY operation at a time:

GROUPINGSETS Before SQL:1999

SELECTyear,month,sum(revenue)FROMtblGROUPBYyear,month

Monthly revenue Yearly revenue

SELECTyear,sum(revenue)FROMtblGROUPBYyear

GROUPINGSETS Before SQL:1999SELECTyear,month,sum(revenue)FROMtblGROUPBYyear,month


GROUPINGSETS Before SQL:1999SELECTyear,month,sum(revenue)FROMtblGROUPBYyear,month


UNIONALL

,null

GROUPINGSETS Since SQL:1999SELECTyear,month,sum(revenue)FROMtblGROUPBYyear,month


UNIONALL

,null

SELECTyear,month,sum(revenue)FROMtblGROUPBYGROUPINGSETS((year,month),(year))

GROUPINGSETS are multiple GROUPBYs in one go

() (empty parenthesis) build a group over all rows

GROUPING (function) disambiguates the meaning of NULL(was the grouped data NULL or is this column not currently grouped?)

Permutations can be created using ROLLUP and CUBE(ROLLUP(a,b,c) = GROUPINGSETS((a,b,c),(a,b),(a),())

GROUPINGSETS In a Nutshell

GROUPINGSETS Availability1999

2001

2003

2005

2007

2009

2011

2013

2015

2017

5.1[0] MariaDB5.0[1] MySQL

9.5 PostgreSQLSQLite

5 DB2 LUW9iR1 Oracle

2008 SQL Server[0]Only ROLLUP (properitery syntax).[1]Only ROLLUP (properitery syntax). GROUPING function since MySQL 8.0.

SQL:2003

OVER and

PARTITIONBY

OVER (PARTITION BY) The ProblemTwo distinct concepts could not be used independently:

‣Merge rows with the same key properties

‣ GROUPBY to specify key properties

‣ DISTINCT to use full row as key

‣ Aggregate data from related rows ‣ Requires GROUPBY to segregate the rows

‣ COUNT, SUM, AVG, MIN, MAX to aggregate grouped rows

SELECTc1,SUM(c2)totFROMtGROUPBYc1

OVER (PARTITION BY) The Problem

Yes ⇠

Mer

ge ro

ws ⇢

No

No ⇠ Aggregate ⇢ Yes

SELECTc1,c2FROMt

SELECTDISTINCTc1,c2FROMt

SELECTc1,c2FROMt




Yes ⇠

Mer

ge ro

ws ⇢

No


SELECTc1,c2FROMt


SELECTc1,c2FROMt




Yes ⇠

Mer

ge ro

ws ⇢

No


SELECTc1,c2FROMt


SELECTc1,c2FROMtJOIN()taON(t.c1=ta.c1)




Yes ⇠

Mer

ge ro

ws ⇢

No


SELECTc1,c2FROMt




,tot



Yes ⇠

Mer

ge ro

ws ⇢

No


SELECTc1,c2FROMt




,tot


OVER (PARTITION BY) Since SQL:2003

Yes ⇠

Mer

ge ro

ws ⇢

No


SELECTc1,c2FROMt


SELECTc1,c2FROMt

FROMt

,SUM(c2)OVER(PARTITIONBYc1)

SELECTdep,salary,SUM(salary)OVER()FROMemp

dep salary1 1000 600022 1000 600022 1000 6000333 1000 6000333 1000 6000333 1000 6000

OVER (PARTITION BY) How it works


dep salary1 1000 600022 1000 600022 1000 6000333 1000 6000333 1000 6000333 1000 6000


Look here


dep salary1 1000 600022 1000 600022 1000 6000333 1000 6000333 1000 6000333 1000 6000



dep salary1 1000 600022 1000 600022 1000 6000333 1000 6000333 1000 6000333 1000 6000



dep salary1 1000 100022 1000 200022 1000 2000333 1000 3000333 1000 3000333 1000 3000


)PARTITIONBYdep

OVER and

ORDERBY(Framing & Ranking)

acnt id value balance

1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

OVER (ORDER BY) The Problem

SELECTid,value,FROMtransactionst


1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

OVER (ORDER BY) The Problem

SELECTid,value,

(SELECTSUM(value)FROMtransactionst2WHEREt2.id<=t.id)

FROMtransactionst

Range segregation (<=)not possible with

GROUP BY orPARTITION BY

OVER (ORDER BY) Since SQL:2003

SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)


1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

ORDERBYid


SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)


1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

ORDERBYidROWSBETWEENUNBOUNDEDPRECEDING


SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)


1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

ORDERBYidROWSBETWEENUNBOUNDEDPRECEDINGANDCURRENTROW


SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)


1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20



SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)


1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20



SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)


1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20



SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)


1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20



SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)


1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20



SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)


1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20



SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)


1 1 +10

22 2 +20

22 3 -10

333 4 +50

333 5 -30

333 6 -20



SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)


1 1 +10 +10

22 2 +20 +20

22 3 -10 +10

333 4 +50 +50

333 5 -30 +20

333 6 -20 .0


PARTITIONBYacnt

OVER (ORDER BY) Since SQL:2003With OVER(ORDERBYn) a new type of functions make sense:

n ROW_NUMBER RANK DENSE_RANK PERCENT_RANK CUME_DIST1 1 1 1 0 0.252 2 2 2 0.33… 0.753 3 2 2 0.33… 0.754 4 4 3 1 1

‣ Aggregates without GROUPBY

‣ Running totals, moving averages

‣ Ranking‣ Top-N per Group

‣ Avoiding self-joins

[… many more …]

Use Cases

SELECT*FROM(SELECTROW_NUMBER()OVER(PARTITIONBY…ORDERBY…)rn,t.*FROMt)numbered_tWHERErn<=3

AVG(…)OVER(ORDERBY…ROWSBETWEEN3PRECEDINGAND3FOLLOWING)moving_avg

OVER (SQL:2003)

OVER may follow any aggregate function

OVER defines which rows are visible at each row

OVER() makes all rows visible at every row

OVER(PARTITIONBY …) segregates like GROUPBY

OVER(ORDERBY…BETWEEN) segregates using <, >

In a NutshellOVER (SQL:2003)

1999

2001

2003

2005

2007

2009

2011

2013

2015

2017


8.4 PostgreSQLSQLite[0]

7.0 DB2 LUW8i Oracle

2005 SQL Server[0]Expected for release 3.25.0 (available in snapshot release).

OVER (SQL:2003) AvailabilityHive

ImpalaSpark

NuoDB

NULLSFIRST/LAST

The sorting of NULL is implementation defined (some DBs sort NULL as great, others as very small value)

NULLS FIRST/LAST Before SQL:2003

SELECT…FROM…ORDERBYCOALESCE(nullable,?);

If you know a valuelarger/smaller than any

actual value…

The sorting of NULL is implementation defined (some DBs sort NULL as great, others as very small value)

NULLS FIRST/LAST Before SQL:2003

SELECT…FROM…ORDERBYCOALESCE(nullable,?);

ORDERBYCASEWHENnullableISNULLTHEN0ELSE1END,nullable;

Using an extra sort keyto put NULL and NOT NULL

apart is more robust

This shows NULLs first (no matter if nullable

is sorted ASC or DESC)

SQL:2003 introduced ORDERBY…NULLSFIRST/LAST

NULLS FIRST/LAST Since SQL:2003

SELECT…FROM…ORDERBYnullableNULLSFIRST

Note: PostgreSQL accepts NULLSFIRST/LAST in index definitions.

This returnsNULLs first

(for ASC and DESC)

NULLS FIRST/LAST Since SQL:20031999

2001

2003

2005

2007

2009

2011

2013

2015

2017

MariaDB[0]

MySQL[0]

8.3[1] PostgreSQLSQLite[0]

11.1[1] DB2 LUW11gR1[1] Oracle

SQL Server[0][0]By default sorted as smallest[1]By default sorted as greatest

FILTER

SELECTYEAR,SUM(CASEWHENMONTH=1THENrevenueELSE0END)JAN,SUM(CASEWHENMONTH=2THENrevenueEND)FEB,…FROMsalesGROUPBYYEAR

FILTER The ProblemPivot table: Years on the Y axis, month on X:



Optional:ELSE NULL is default

Aggregatesignore NULL*

*Exceptions:array_agg, json_objectagg, xmlagg See: https://modern-sql.com/concept/null#aggregates

http://modern-sql.com/concept/null#aggregates



SELECTYEAR,SUM(revenue)FILTER(WHEREMONTH=1)JAN,SUM(revenue)FILTER(WHEREMONTH=2)FEB,…FROMsalesGROUPBYYEAR;

FILTER Since SQL:2003SQL:2003 allows FILTER(WHERE…) after aggregates:

FILTER Since SQL:2003

Year

2016

2016

2016

2016

2016

Month

1

2

3

...

12

Revenue

1

23

345

...

1234

Year

2016

Jan

1

Feb

23

Mar

345

...

...

Dec

1234

SUM(…) FILTE

R(WHERE …)

SUM(…)

FILTER(WHER

E month=2)

SUM(rev

enue) FILTER(WHERE

month=3)

SUM(rev

enue) FILTER(WHERE mo

nth=…)

SUM(rev

enue) FILTER(WHERE month=

12)

Pivot in SQL1. Use GROUP BY

to combine rows2. Use FILTER to pick

rows per column

See: https://modern-sql.com/use-case/pivot

http://modern-sql.com/use-case/pivot

FILTER Availability1999

2001

2003

2005

2007

2009

2011

2013

2015

2017

5.1 MariaDBMySQL

9.4 PostgreSQLSQLite[0]

DB2 LUWOracleSQL Server

[0]Only with OVER clause

Inverse Distribution Functions (percentiles)

SELECTd1.valFROMdatad1JOINdatad2ON(d1.val<d2.valOR(d1.val=d2.valANDd1.id<d2.id))GROUPBYd1.valHAVINGcount(*)=(SELECTFLOOR(COUNT(*)/2)FROMdatad3)

Inverse Distribution Functions The ProblemGrouped rows cannot be ordered prior aggregation.

(how to get the middle value (median) of a set)




Number rows

Pick middle one




Number rows

Pick middle one

SELECTPERCENTILE_DISC(0.5)WITHINGROUP(ORDERBYval)FROMdata

Median

Which value?

Since SQL:2003Inverse Distribution Functions

1

2

3

4

0 0.25 0.5 0.75 11

2

3

4

0 0.25 0.5 0.75 11

2

3

4

PERCENTILE_DISC0 0.25 0.5 0.75 1

1

2

3

4

PERCENTILE_DISC0 0.25 0.5 0.75 1

1

2

3

4

PERCENTILE_DISC(0.5)0 0.25 0.5 0.75 1

1

2

3

4

PERCENTILE_CONT

PERCENTILE_DISC(0.5)0 0.25 0.5 0.75 1

1

2

3

4

PERCENTILE_CONT(0.5)

PERCENTILE_DISC(0.5)

SELECTPERCENTILE_DISC(0.5)WITHINGROUP(ORDERBYval)FROMdata

Since SQL:2003Inverse Distribution Functions

Two variants: ‣ for discrete values (categories) ‣ for continuous values (linear interpolation)

Inverse Distribution Functions Availability1999

2001

2003

2005

2007

2009

2011

2013

2015

2017

5.1 10.3[0] MariaDBMySQL

9.4 PostgreSQLSQLite

11.1 DB2 LUW9iR1 Oracle

2012[0] SQL Server[0]Only as window function (requires OVER clause)

SQL:2006

XMLTABLE

SELECTid,c1,nFROMtbl,XMLTABLE('/d/e'PASSINGxCOLUMNSidINTPATH'@id',c1VARCHAR(255)PATH'c1',nFORORDINALITY)r

XMLTABLE Since SQL:2006Stored in tbl.x:

<d><eid="42"><c1>…</c1></e></d>

XPath* expression to identify rows

*Standard SQL allows XQuery



<d><eid="42"><c1>…</c1></e></d>




<d><eid="42"><c1>…</c1></e></d>


XPath* expressions to extract data



<d><eid="42"><c1>…</c1></e></d>


Row number (like for unnest)



<d><eid="42"><c1>…</c1></e></d>


Result id|c1|n----+----+---42|…|1

XMLTABLE Availability1999

2001

2003

2005

2007

2009

2011

2013

2015

2017

MariaDBMySQL

10[0] PostgreSQLSQLite

9.7 DB2 LUW11gR1 Oracle

SQL Server[0]No XQuery (only XPath). No default namespace declaration.

SQL:2008

FETCHFIRST

SELECT*FROM(SELECT*,ROW_NUMBER()OVER(ORDERBYx)rnFROMdata)numbered_dataWHERErn<=10

FETCHFIRST The ProblemLimit the result to a number of rows. (LIMIT, TOP and ROWNUM are all proprietary)

SQL:2003 introduced ROW_NUMBER() to number rows.But this still requires wrapping to limit the result.

And how about databases not supporting ROW_NUMBER()?

SELECT*FROM(SELECT*,ROW_NUMBER()OVER(ORDERBYx)rnFROMdata)numbered_dataWHERErn<=10

FETCHFIRST The ProblemLimit the result to a number of rows. (LIMIT, TOP and ROWNUM are all proprietary)

SQL:2003 introduced ROW_NUMBER() to number rows.But this still requires wrapping to limit the result.

And how about databases not supporting ROW_NUMBER()?

Dammit! Let's takeLIMIT

SELECT*FROMdataORDERBYxFETCHFIRST10ROWSONLY

FETCHFIRST Since SQL:2008SQL:2008 introduced the FETCHFIRST…ROWSONLY clause:

FETCHFIRST Availability1999

2001

2003

2005

2007

2009

2011

2013

2015

2017

5.1 MariaDB3.19.3[0] MySQL6.5[1] 8.4 PostgreSQL

2.1.0[1] SQLite7.0 DB2 LUW

12cR1 Oracle7.0[2] 2012 SQL Server

[0]Earliest mention of LIMIT. Probably inherited from mSQL[1]Functionality available using LIMIT[2]SELECT TOP n ... SQL Server 2000 also supports expressions and bind parameters

SQL:2011

OFFSET

SELECT*FROM(SELECT*,ROW_NUMBER()OVER(ORDERBYx)rnFROMdata)numbered_dataWHERErn>10andrn<=20

OFFSET The ProblemHow to fetch the rows after a limit?

(pagination anybody?)

SELECT*FROMdataORDERBYxOFFSET10ROWSFETCHNEXT10ROWSONLY

OFFSET Since SQL:2011SQL:2011 introduced OFFSET, unfortunately!

SELECT*FROMdataORDERBYxOFFSET10ROWSFETCHNEXT10ROWSONLY

OFFSET Since SQL:2011SQL:2011 introduced OFFSET, unfortunately!

OFFSETGrab coasters & stickers!

https://use-the-index-luke.com/no-offset

OFFSET Since SQL:201119

99

2001

2003

2005

2007

2009

2011

2013

2015

5.1 MariaDB3.20.3[0] 4.0.6[1] MySQL

6.5 PostgreSQL2.1.0 SQLite

9.7[2] 11.1 DB2 LUW12c Oracle

2012 SQL Server[0]LIMIT[offset,]limit: "With this it's easy to do a poor man's next page/previous page WWW application."[1]The release notes say "Added PostgreSQL compatible LIMIT syntax"[2]Requires enabling the MySQL compatibility vector: db2setDB2_COMPATIBILITY_VECTOR=MYS

OVER

WITHnumbered_tAS(SELECT*)

SELECTcurr.*,curr.balance-COALESCE(prev.balance,0)FROMnumbered_tcurrLEFTJOINnumbered_tprevON(curr.rn=prev.rn+1)

OVER (SQL:2011) The ProblemDirect access of other rows of the same window is not possible.

(E.g., calculate the difference to the previous rows)

currbalance … rn

50 … 190 … 270 … 330 … 4

FROMt





currbalance … rn

50 … 190 … 270 … 330 … 4

FROMt,ROW_NUMBER()OVER(ORDERBYx)rn





currbalance … rn

50 … 190 … 270 … 330 … 4






currbalance … rn

50 … 190 … 270 … 330 … 4


prevbalance … rn

50 … 190 … 270 … 330 … 4





currbalance … rn

50 … 190 … 270 … 330 … 4


prevbalance … rn

50 … 190 … 270 … 330 … 4





currbalance … rn

50 … 190 … 270 … 330 … 4


prevbalance … rn

50 … 190 … 270 … 330 … 4

+50+40-20-40

SELECT*,balance-COALESCE(LAG(balance)OVER(ORDERBYx),0)FROMt

Available functions:LEAD/LAGFIRST_VALUE/LAST_VALUENTH_VALUE(col,n)FROMFIRST/LASTRESPECT/IGNORENULLS

OVER (SQL:2011) Since SQL:2011SQL:2011 introduced LEAD, LAG, NTH_VALUE, … for that:

OVER (LEAD, LAG, …) Since SQL:20111999

2001

2003

2005

2007

2009

2011

2013

2015

2017

5.1 10.2[0] MariaDB8.0[0] MySQL

8.4[0] PostgreSQLSQLite[0]

9.5[1] 11.1 DB2 LUW8i[1] 11gR2 Oracle

2012[1] SQL Server[0]No IGNORENULLS and FROMLAST[1]No NTH_VALUE

System Versioning (Time Traveling)

INSERTUPDATEDELETE

are DESTRUCTIVE

System Versioning The Problem

CREATETABLEt(...,start_tsTIMESTAMP(9)GENERATEDALWAYSASROWSTART,end_tsTIMESTAMP(9)GENERATEDALWAYSASROWEND,

PERIODFORSYSTEM_TIME(start_ts,end_ts))WITHSYSTEMVERSIONING

System Versioning Since SQL:2011Table can be system versioned, application versioned or both.

ID Data start_ts end_ts1 X 10:00:00

UPDATE...SETDATA='Y'...

ID Data start_ts end_ts1 X 10:00:00 11:00:001 Y 11:00:00

DELETE...WHEREID=1

INSERT...(ID,DATA)VALUES(1,'X')

System Versioning Since SQL:2011




DELETE...WHEREID=1






DELETE...WHEREID=1

ID Data start_ts end_ts1 X 10:00:00 11:00:001 Y 11:00:00 12:00:00



Although multiple versions exist, only the “current” one is visible per default.

After 12:00:00, SELECT*FROMt doesn’t return anything anymore.




With FOR…ASOF you can query anything you like: SELECT*FROMtFORSYSTEM_TIMEASOFTIMESTAMP'2018-09-1210:30:00'

ID Data start_ts end_ts

1 X 10:00:00 11:00:00



2001

2003

2005

2007

2009

2011

2013

2015

2017

5.1 10.3[0] MariaDBMySQLPostgreSQLSQLite

10.1[1] DB2 LUW10gR1[2] Oracle

2016 SQL Server[0]Transaction time not immutable. See MDEV-16236.[1]Third column required (tx id), history table required.[2]Functionality available using Flashback

SQL:2016 (released: 2016-12-15)

MATCH_RECOGNIZE (Row Pattern Matching)

Row Pattern Matching

Time

30 minutes

Example: Logfile


Example: Logfile

Time

30 minutes

Session 1 Session 2

Session 3

Session 4


Example: Logfile

Time

30 minutes

Session 1 Session 2

Session 3

Session 4

Example problem:

‣ Average session duration

Two approaches:

‣ Row pattern matching

‣ Start-of-group tagging

SELECTCOUNT(*)sessions,AVG(duration)avg_durationFROMlogMATCH_RECOGNIZE(ORDERBYtsMEASURESLAST(ts)-FIRST(ts)ASdurationONEROWPERMATCHPATTERN(anycont*)DEFINEcontASts<PREV(ts)+INTERVAL'30'minute)t

Since SQL:2016Row Pattern Matching

Time

30 minutes

Oracle doesn’t support avg on intervals — query doesn’t work as shown



Time

30 minutes

definecontinued




Time

30 minutes


undefinedpattern variable: matches any row



Time

30 minutes

any numberof “cont”

rows




Time

30 minutes

Very muchlike GROUP BY




Time

30 minutes

Very muchlike SELECT




Time

30 minutes


Row Pattern Matching Before SQL:2016

Time

30 minutes

Now, let’s try using window functions

SELECTcount(*)sessions,avg(duration)avg_durationFROM(SELECTMAX(ts)-MIN(ts)durationFROM(SELECTts,COUNT(grp_start)OVER(ORDERBYts)session_noFROM(SELECTts,CASEWHENts>=LAG(ts,1,DATE'1900-01-1')OVER(ORDERBYts)+INTERVAL'30'minuteTHEN1ENDgrp_startFROMlog)tagged)numberedGROUPBYsession_no)grouped


Time

30 minutes

Start-of-group tags



Time

30 minutes

number sessions

2222 2 33 3 44 42 3 41



Time

30 minutes 2222 2 33 3 44 42 3 41

Row Pattern Matching Since SQL:2016

https://www.slideshare.net/MarkusWinand/row-pattern-matching-in-sql2016

Row Pattern Matching Availability1999

2001

2003

2005

2007

2009

2011

2013

2015

2017

MariaDBMySQLPostgreSQLSQLiteDB2 LUW

12cR1 OracleSQL Server

SQL has evolved beyond the relational idea.

Modern SQL? @MarkusWinand


If you are using SQL like 25 years ago,you are doing it wrong!



If you are using SQL like 25 years ago,you are doing it wrong!

A lot has happened since SQL-92.


https://www.flickr.com/photos/mfoubister/25367243054/

I have shown you a few features today


I have shown you a few features today


There are hundreds more to discover

@ModernSQL modern-sql.comMy other website:

https://use-the-index-luke.com

Training & co: https://winand.at/

Documents

Still using Windows 3.1? - modern-sql.com · ‣Literate SQL Organize SQL code to improve maintainability ‣Assign column names to tables produced by values or unnest. ‣Overload