SunshinePHP 2017 - Making the most out of MySQL

Embed Size (px)

Text of SunshinePHP 2017 - Making the most out of MySQL

  1. 1. Making the most out of MySQL Gabriela DAvila @gabidavila Feb/2017
  2. 2. Who? Data Engineer Lego Hoarder @gabidavila http://gabriela.io 2
  3. 3. What to Expect? DDL Changes SQL Modes Generated Columns JSON Data Type `sys` Schema 3
  4. 4. DDL Changes on InnoDB
  5. 5. DDL Changes In Place (ALGORITHM=INPLACE) Rename Index VARCHAR from 1B to 255B VARCHAR from 256B to 65353B* Add a Virtual Column Table-copy (ALGORITHM=COPY) Add a Column VARCHAR between 255B and >=256 Decreasing VARCHAR size Type Conversion 5 More Info
  6. 6. SQL Modes
  7. 7. SQL modes (SELECT @@GLOBAL.sql_mode;) 7 On MySQL 5.6: +--------------------------------------------+ | @@GLOBAL.sql_mode | +--------------------------------------------+ | STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION | +--------------------------------------------+ 1 row in set (0.00 sec) On MySQL 5.7: +-----------------------------------------------------------------------+ | @@GLOBAL.sql_mode | +-----------------------------------------------------------------------+ | ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE, | | ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION | +-----------------------------------------------------------------------+ 1 row in set (0.00 sec)
  8. 8. SQL modes comparison On MySQL 5.6.35: STRICT_TRANS_TABLES (Bug #69743) NO_ENGINE_SUBSTITUTION On MySQL 5.7.17: STRICT_TRANS_TABLES NO_ENGINE_SUBSTITUTION ONLY_FULL_GROUP_BY NO_ZERO_IN_DATE* NO_ZERO_DATE* ERROR_FOR_DIVISION_BY_ZERO* 8 Full list of sql_mode
  9. 9. YYYY-MM-DD NO_ZERO_DATE 0000-00-00 0000-00-00 00:00:00 NO_ZERO_IN_DATE 2017-01-00 2017-00-04 0000-01-04 9
  10. 10. ERROR_FOR_DIVISION_BY_ZERO 10 mysql> UPDATE users SET id = id/0 WHERE id = 2; ERROR 1365 (22012): Division by 0 Write Operation: UPDATE mysql> SELECT id, mod(id,0) FROM users WHERE id = 2; +----+-----------+ | id | mod(id,0) | +----+-----------+ | 2 | NULL | +----+-----------+ 1 row in set, 1 warning (0.00 sec) Read Operation: SELECT mysql> SHOW WARNINGS; +---------+------+---------------+ | Level | Code | Message | +---------+------+---------------+ | Warning | 1365 | Division by 0 | +---------+------+---------------+ 1 row in set (0.00 sec)
  11. 11. ONLY_FULL_GROUP_BY
  12. 12. Table `users` Field Type Null Key Default Extra id int(10) unsigned NO PRI auto_increment first_name varchar(255) NO NULL last_name varchar(255) NO NULL email varchar(255) NO NULL twitter_info json YES created_at datetime NO CURRENT_TIMESTAMP updated_at datetime NO CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP 13
  13. 13. Repeated emails SELECT * FROM users GROUP BY email HAVING count(*) > 1 +-------+------------+-----------+---------------------+-------------------+---------------------+---------------------+ | id | first_name | last_name | email | twitter_info | created_at | updated_at | +-------+------------+-----------+---------------------+-------------------+---------------------+---------------------+ | 8965 | Conor | Quitzon | yjacobson@yahoo.com | {"id": 100838242, | 2017-01-02 20:57:39 | 2017-01-02 20:57:39 | | 69772 | Junius | Mante | nkuhlman@gmail.com | {"id": 20039476, | 2017-01-02 20:59:34 | 2017-01-02 20:59:34 | | 66525 | Katelynn | Feil | qgottlieb@gmail.com | {"id": 111644778, | 2017-01-02 20:59:29 | 2017-01-02 20:59:29 | | 92577 | Tillman | Nienow | zzieme@yahoo.com | {"id": 1359073920,| 2017-01-02 21:00:14 | 2017-01-02 21:00:14 | | 18046 | Guillermo | Lebsack | cdoyle@gmail.com | {"id": 120064855, | 2017-01-02 20:57:59 | 2017-01-02 20:57:59 | +-------+------------+-----------+---------------------+-------------------+---------------------+---------------------+ 5 rows in set (0.65 sec) 5.6 14
  14. 14. Repeated emails SELECT * FROM users WHERE email = 'cdoyle@gmail.com' +-------+------------+-----------+------------------+--------------+---------------------+---------------------+ | id | first_name | last_name | email | twitter_info | created_at | updated_at | +-------+------------+-----------+------------------+--------------+---------------------+---------------------+ | 18046 | Guillermo | Lebsack | cdoyle@gmail.com | NULL | 2017-01-02 14:57:59 | 2017-01-02 14:57:59 | | 20559 | Marietta | Quitzon | cdoyle@gmail.com | NULL | 2017-01-02 14:58:03 | 2017-01-02 14:58:03 | +-------+------------+-----------+------------------+--------------+---------------------+---------------------+ 2 rows in set (0.04 sec) 15
  15. 15. Repeated emails SELECT * FROM users GROUP BY email HAVING count(*) > 1 mysql> SELECT * FROM `users` GROUP BY `email` HAVING count(*) > 1; ERROR 1055 (42000): Expression #1 of SELECT list is not in GROUP BY clause and contains nonaggregated column store.users.id which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by 5.7 16
  16. 16. Why? "Reject queries for which the select list, HAVING condition, or ORDER BY list refer to nonaggregated columns that are neither named in the GROUP BY clause nor are functionally dependent on (uniquely determined by) GROUP BY columns." 17
  17. 17. Translation This happens because the query is grouping by email and MySQL is magically trying to guess which eld it should bring as a result to the other columns (id, rst_name, etc.) since there are more than one result per row. This mode is known as ONLY_FULL_GROUP_BY. 18 More Info
  18. 18. Fix
  19. 19. AGGREGATORS!
  20. 20. MySQL Aggregators min(column) max(column) group_concat(column) count(column), count(DISTINCT column) or count(*) sum(column) 21 More Info
  21. 21. How are we used to doing it SELECT * FROM users GROUP BY email HAVING count(*) > 1