Embed Size (px)
Citation preview
Copyright © 2018 Oracle and/or its afliates. All rights reserved.
More SQL in MySQL 8.0
Norvald H. RyengSofware Development Senior ManagerMySQL Optmizer TeamNovember, 2018
3Copyright © 2018 Oracle and/or its afliates. All rights reserved.
Program AgendaCommon table expressions (CTEs)
Window functons
JSON_TABLE
Demo
1
2
3
4
5
6
7
4Copyright © 2018 Oracle and/or its afliates. All rights reserved.
Theory
5Copyright © 2018 Oracle and/or its afliates. All rights reserved.
Common Table Expressions (CTEs)● Alternatve to derived tables or views
SELECT … FROM (subquery) AS derived, t1 …
CREATE VIEW derived AS (subquery);SELECT … FROM derived, t1 …
WITH derived AS (subquery)SELECT … FROM derived, t1 …
● Can refer to other CTEsWITH cte1 AS (subquery), cte2 AS (SELECT … FROM cte1 …)SELECT … FROM cte2 …
● Can refer to itself, creatng a recursive CTE
6Copyright © 2018 Oracle and/or its afliates. All rights reserved.
Recursive CTEsWITH RECURSIVE cte AS
(SELECT … FROM table_name /* "seed" SELECT */UNION [DISTINCT|ALL]SELECT … FROM cte, table_name) /* "recursive" SELECT */
SELECT … FROM cte;
● A recursive CTE refers to itself in a subquery
● The “seed” SELECT is executed once to create the inital data subset
● The “recursive” SELECT is executed repeatedly
– Stops when iteraton doesn’t generate any new rows
– Set cte_max_recursion_depth to limit recursion (default 1000)
● Useful to traverse hierarchies (parent/child, part/subpart)
7Copyright © 2018 Oracle and/or its afliates. All rights reserved.
Recursive CTE ExampleCREATE TABLE employee (number INTEGER PRIMARY KEY, name VARCHAR(100), manager INTEGER REFERENCES employee (number));
INSERT INTO employee VALUES (1, 'Alice', NULL), (2, 'Bob', 1), (3, 'Bety', 1), (4, 'Charlie', 2), (5, 'Cherise', 2), (6, 'Chandler', 2), (7, 'Chris', 3), (8, 'Camilla', 3), (9, 'Dave', 8), (10, 'Denise', 8);
WITH RECURSIVE emps AS ( SELECT *, name AS reportng_chain FROM employee WHERE manager IS NULL UNION ALL SELECT e.number, e.name, e.manager, CONCAT(e.name, ', ', m.reportng_chain) FROM employee AS e JOIN emps AS m ON e.manager = m.number)SELECT * FROM emps;
number name manager reportng_chain
1 Alice NULL Alice
2 Bob 1 Bob, Alice
3 Bety 1 Bety, Alice
4 Charlie 2 Charlie, Bob, Alice
5 Cherise 2 Cherise, Bob, Alice
6 Chandler 2 Chandler, Bob, Alice
7 Chris 3 Chris, Bety, Alice
8 Camilla 3 Camilla, Bety, Alice
9 Dave 8 Dave, Camilla, Bety, Alice
10 Denise 8 Denise, Camilla, Bety, Alice
8Copyright © 2018 Oracle and/or its afliates. All rights reserved.
Recursive CTE ExampleCREATE TABLE employee (number INTEGER PRIMARY KEY, name VARCHAR(100), manager INTEGER REFERENCES employee (number));
INSERT INTO employee VALUES (1, 'Alice', NULL), (2, 'Bob', 1), (3, 'Bety', 1), (4, 'Charlie', 2), (5, 'Cherise', 2), (6, 'Chandler', 2), (7, 'Chris', 3), (8, 'Camilla', 3), (9, 'Dave', 8), (10, 'Denise', 8);
WITH RECURSIVE emps AS ( SELECT *, name AS reportng_chain FROM employee WHERE manager IS NULL UNION ALL SELECT e.number, e.name, e.manager, CONCAT(e.name, ', ', m.reportng_chain) FROM employee AS e JOIN emps AS m ON e.manager = m.number)SELECT * FROM emps;
number name manager reportng_chain
1 Alice NULL Alice
2 Bob 1 Bob, Alice
3 Bety 1 Bety, Alice
4 Charlie 2 Charlie, Bob, Alice
5 Cherise 2 Cherise, Bob, Alice
6 Chandler 2 Chandler, Bob, Alice
7 Chris 3 Chris, Bety, Alice
8 Camilla 3 Camilla, Bety, Alice
9 Dave 8 Dave, Camilla, Bety, Alice
10 Denise 8 Denise, Camilla, Bety, Alice
9Copyright © 2018 Oracle and/or its afliates. All rights reserved.
Window Functons● Similar to aggregaton functons
– Computes one value based on multple rows
– But does not group
Aggregaton functon Window functon
10Copyright © 2018 Oracle and/or its afliates. All rights reserved.
Window Functon ExampleSum up total salary per departmentSELECT name, dept_id, salary, SUM(salary) OVER (PARTITION BY dept_id) AS dept_totalFROM employeeORDER BY dept_id, name;
The OVER keyword signals a window functon
PARTITION ⇒disjoint set of rows in result set
name dept_id salary dept_total
Newt NULL 75000 75000Dag 10 NULL 370000Ed 10 100000 370000Fred 10 60000 370000Jon 10 60000 370000Michael 10 70000 370000Newt 10 80000 370000Lebedev 20 65000 130000Pete 20 65000 130000Jef 30 300000 370000Will 30 70000 370000
11Copyright © 2018 Oracle and/or its afliates. All rights reserved.
Window Functon ExampleSum up total salary per departmentSELECT name, dept_id, salary, SUM(salary) OVER (PARTITION BY dept_id) AS dept_totalFROM employeeORDER BY dept_id, name;
The OVER keyword signals a window functon
PARTITION ⇒disjoint set of rows in result set
name dept_id salary dept_total
Newt NULL 75000 75000Dag 10 NULL 370000Ed 10 100000 370000Fred 10 60000 370000Jon 10 60000 370000Michael 10 70000 370000Newt 10 80000 370000Lebedev 20 65000 130000Pete 20 65000 130000Jef 30 300000 370000Will 30 70000 370000
12Copyright © 2018 Oracle and/or its afliates. All rights reserved.
JSON_TABLEINSERT INTO t1 (json_col) VALUES ( '{ "people": [ { "name":"John Smith", "address":"780 Mission St, San Francisco, CA 94103"}, { "name":"Sally Brown", "address":"75 37th Ave S, St Cloud, MN 94103"}, { "name":"John Johnson", "address":"1262 Roosevelt Trail, Raymond, ME 04071"} ] }');
SELECT people.*FROM t1, JSON_TABLE(json_col, '$.people[*]' COLUMNS ( name VARCHAR(40) PATH '$.name', address VARCHAR(100) PATH '$.address')) AS people;
name address
John Smith 780 Mission St, San Francisco, CA 94103
Sally Brown 75 37th Ave S, St Cloud, MN 9410
John Johnson 1262 Roosevelt Trail, Raymond, ME 04071
13Copyright © 2018 Oracle and/or its afliates. All rights reserved.
JSON_TABLE Nested Arrays[
{ "father":"John", "mother":"Mary",
"marriage_date":"2003-12-05",
"children": [ { "name":"Eric", "age":10 },
{ "name":"Beth", "age":12 } ]
},
{ "father":"Paul", "mother":"Laura",
"children": [ { "name":"Sarah", "age":9},
{ "name":"Noah", "age":3} ,
{ "name":"Peter", "age":10} ]
}
]
id father married child_id child age
1 John 1 1 Eric 10
1 John 1 2 Beth 12
2 Paul 0 1 Sarah 9
2 Paul 0 2 Noah 3
2 Paul 0 3 Peter 10
14Copyright © 2018 Oracle and/or its afliates. All rights reserved.
JSON_TABLE Nested Arrays
JSON_TABLE (families, '$[*]' COLUMNS ( id FOR ORDINALITY, father VARCHAR(30) PATH '$.father', married INTEGER EXISTS PATH '$.marriage_date', NESTED PATH '$.children[*]' COLUMNS ( child_id FOR ORDINALITY, child VARCHAR(30) PATH '$.name', age INTEGER PATH '$.age' )))
id father married child_id child age
1 John 1 1 Eric 10
1 John 1 2 Beth 12
2 Paul 0 1 Sarah 9
2 Paul 0 2 Noah 3
2 Paul 0 3 Peter 10
15Copyright © 2018 Oracle and/or its afliates. All rights reserved.
Playtme
16Copyright © 2018 Oracle and/or its afliates. All rights reserved.
Demo Warning!● The following queries are intended to demonstrate the power
of the features
● This is not necessarily the best way of doing things
● This is not necessarily the easiest way of doing things
● This is not necessarily smart …
… but it's fun! :-)
17Copyright © 2018 Oracle and/or its afliates. All rights reserved.
JSON_TABLE
18Copyright © 2018 Oracle and/or its afliates. All rights reserved.
Birth Order# First child in a family = 1, second child = 2, etc.SELECT fam.*,
RANK() OVER w AS nth_childFROM families, JSON_TABLE (families, '$[*]' COLUMNS ( id FOR ORDINALITY, father VARCHAR(30) PATH '$.father', mother VARCHAR(30) PATH '$.mother', married INTEGER EXISTS PATH '$.marriage_date', marriage_date DATE PATH '$.marriage_date', NESTED PATH '$.children[*]' COLUMNS ( child_id FOR ORDINALITY, child VARCHAR(30) PATH '$.name', age INTEGER PATH '$.age' ) )) AS famWINDOW w AS (PARTITION BY id ORDER BY age DESC)ORDER BY id, child_id;
19Copyright © 2018 Oracle and/or its afliates. All rights reserved.
Families as Table# Use JSON_TABLE to convert JSON document to a table representng the same infoCREATE TABLE families_tabSELECT fam.*FROM families, JSON_TABLE (families, '$[*]' COLUMNS ( id FOR ORDINALITY, father VARCHAR(30) PATH '$.father', mother VARCHAR(30) PATH '$.mother', marriage_date DATE PATH '$.marriage_date', NESTED PATH '$.children[*]' COLUMNS ( child_id FOR ORDINALITY, child VARCHAR(30) PATH '$.name', age INTEGER PATH '$.age' ) )) AS fam;
20Copyright © 2018 Oracle and/or its afliates. All rights reserved.
Families from Table to JSON# Use JSON_ARRAYAGG to produce JSON
SELECT JSON_PRETTY( JSON_OBJECT( 'father',ANY_VALUE(father), 'mother',ANY_VALUE(mother), 'marriage_date',ANY_VALUE(marriage_date), 'children',JSON_ARRAYAGG(JSON_OBJECT('name',child,'age',age)) )) FROM families_tab GROUP BY id;
# Getng rid of "marriage_date" : null is lef as an exercise to the reader …
21Copyright © 2018 Oracle and/or its afliates. All rights reserved.
GPS Tracking
22Copyright © 2018 Oracle and/or its afliates. All rights reserved.
Timeline CREATE TABLE tmeline ( ts TIMESTAMP, pos POINT SRID 4326);
INSERT INTO tmeline VALUES ('2018-11-03 20:30:00', ST_SRID(POINT(8.627229352645486, 50.11755950283356), 4326)), ('2018-11-03 20:30:05', ST_SRID(POINT(8.627438564948648, 50.11767989367171), 4326)), ('2018-11-03 20:30:10', ST_SRID(POINT(8.627234717063516, 50.11775212802929), 4326)), ('2018-11-03 20:30:15', ST_SRID(POINT(8.627057691268533, 50.11782436227788), 4326)), ('2018-11-03 20:30:20', ST_SRID(POINT(8.62689139430961, 50.117879397822705), 4326)), ('2018-11-03 20:30:25', ST_SRID(POINT(8.626735826186746, 50.117941312735), 4326)), ('2018-11-03 20:30:30', ST_SRID(POINT(8.62662853782615, 50.11786219922173), 4326)), ('2018-11-03 20:30:35', ST_SRID(POINT(8.626714368514627, 50.11778996503024), 4326)), ('2018-11-03 20:30:40', ST_SRID(POINT(8.626837750129312, 50.11774180884201), 4326)), ('2018-11-03 20:30:45', ST_SRID(POINT(8.626977224998086, 50.11770053207072), 4326)), ('2018-11-03 20:30:50', ST_SRID(POINT(8.62712206428489, 50.11764893605653), 4326)), ('2018-11-03 20:30:55', ST_SRID(POINT(8.627218623809426, 50.11761453868286), 4326));
INSERT INTO tmeline VALUES ('2018-11-03 20:41:00', ST_SRID(POINT(8.627277632407754, 50.117590460506584), 4326)), ('2018-11-03 20:41:05', ST_SRID(POINT(8.627492209128945, 50.11751134641354), 4326)), ('2018-11-03 20:41:10', ST_SRID(POINT(8.62763704841575, 50.11746318994507), 4326)), ('2018-11-03 20:41:15', ST_SRID(POINT(8.627755065612405, 50.117425352685885), 4326)), ('2018-11-03 20:41:20', ST_SRID(POINT(8.62788917606315, 50.117518225905094), 4326)), ('2018-11-03 20:41:25', ST_SRID(POINT(8.627733607940286, 50.117573261801816), 4326)), ('2018-11-03 20:41:30', ST_SRID(POINT(8.627578039817422, 50.11762485789756), 4326)), ('2018-11-03 20:41:35', ST_SRID(POINT(8.627417107276528, 50.11768677313905), 4326)), ('2018-11-03 20:41:40', ST_SRID(POINT(8.627277632407754, 50.117590460506584), 4326));
INSERT INTO tmeline VALUES ('2018-11-03 20:47:00', ST_SRID(POINT(8.627309818915933, 50.11758702076614), 4326)), ('2018-11-03 20:47:05', ST_SRID(POINT(8.627089877776712, 50.11764549632028), 4326)), ('2018-11-03 20:47:10', ST_SRID(POINT(8.626848478965371, 50.117734929382614), 4326)), ('2018-11-03 20:47:15', ST_SRID(POINT(8.626660724334329, 50.11781404310618), 4326)), ('2018-11-03 20:47:20', ST_SRID(POINT(8.626655359916299, 50.11797227016113), 4326)), ('2018-11-03 20:47:25', ST_SRID(POINT(8.626515885047525, 50.11805138349252), 4326)), ('2018-11-03 20:47:30', ST_SRID(POINT(8.62637641017875, 50.11807202173137), 4326)), ('2018-11-03 20:47:35', ST_SRID(POINT(8.626215477637857, 50.11800322756724), 4326)), ('2018-11-03 20:47:40', ST_SRID(POINT(8.626076002769082, 50.11791035528887), 4326)), ('2018-11-03 20:47:45', ST_SRID(POINT(8.625904341392129, 50.11781404310618), 4326)), ('2018-11-03 20:47:50', ST_SRID(POINT(8.625738044433206, 50.117721170460825), 4326));
23Copyright © 2018 Oracle and/or its afliates. All rights reserved.
Window Functon to Add Rank# Order by tmestamp. First point = 1, second point = 2, etc.CREATE VIEW ranked_tmeline AS SELECT *, RANK() OVER w AS num, TIME_TO_SEC(TIMEDIFF(ts, LAG(ts) OVER w)) AS dif FROM tmeline WINDOW w AS ( PARTITION BY DATE(ts) ORDER BY ts );
24Copyright © 2018 Oracle and/or its afliates. All rights reserved.
Generate GeoJSON Path# 5 minutes (300 seconds) or more between points -> new trip.WITH RECURSIVE cte AS ( SELECT num AS trip, ts AS start, ts AS stop, num, JSON_ARRAY(JSON_ARRAY(ST_Longitude(pos), ST_Lattude(pos))) AS path FROM ranked_tmeline WHERE dif IS NULL OR dif >= 300 UNION ALL SELECT cte.trip, cte.start, ranked_tmeline.ts, ranked_tmeline.num, JSON_ARRAY_APPEND(cte.path, '$', JSON_ARRAY(ST_Longitude(ranked_tmeline.pos), ST_Lattude(ranked_tmeline.pos))) FROM ranked_tmeline, cte WHERE ranked_tmeline.num - 1 = cte.num AND ranked_tmeline.dif < 300)SELECT DISTINCT trip, start, FIRST_VALUE(stop) OVER w AS stop, JSON_OBJECT('type', 'LineString', 'coordinates', FIRST_VALUE(path) OVER w) AS path_jsonFROM cteWINDOW w AS (PARTITION BY trip ORDER BY num DESC);
25Copyright © 2018 Oracle and/or its afliates. All rights reserved.
Trips Ploted from GeoJSON
26Copyright © 2018 Oracle and/or its afliates. All rights reserved.
Feature descriptons and design detailsdirectly from the source.
htp://mysqlserverteam.com/
27Copyright © 2018 Oracle and/or its afliates. All rights reserved.
28Copyright © 2018 Oracle and/or its afliates. All rights reserved.
Safe Harbor Statement
The preceding is intended to outline our general product directon. It is intended for informaton purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functonality, and should not be relied upon in making purchasing decisions. The development, release, and tming of any features or functonality described for Oracle’s products remains at the sole discreton of Oracle.
