MySQL 5.7 - Everything You Need to Know

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

1

Dave Stokes MySQL Community Manager [email protected] @Stoker

MYSQL 5.7 – EVERYTHING YOU NEED TO KNOW

mailto:[email protected]


2

The following is intended to outline our general product direction. It is

intended for information purposes only, and may not be incorporated into

any contract. It is not a commitment to deliver any material, code, or

functionality, and should not be relied upon in making purchasing decision.

The development, release, and timing of any features or functionality

described for Oracle’s products remains at the sole discretion of Oracle.

Safe Harbor Statement


3

MySQL Enterprise Monitor 2.2

MySQL Cluster 7.1

MySQL Cluster Manager 1.0

MySQL Workbench 5.2

MySQL Database 5.5

MySQL Enterprise Backup 3.5




All GA!

Oracle Products Certifications

MySQL Windows Installer

MySQL Enterprise Security

MySQL Enterprise Scalability

MySQL Enterprise Audit

MySQL Cluster 7.2


MySQL Utilities

MySQL Workbench 6.0

All GA!

MySQL Windows Tools

MySQL Database 5.6

MySQL Cluster 7.3


MySQL Workbench 6.1


MySQL Fabric

MySQL Workbench 6.2

MySQL Database 5.7 DMRs*

MySQL Cluster 7.4 DMR

*Development Milestone Release

All GA!

Available Now!

Driving MySQL Innovation: 2010 - 2014

Partial List of Releases Delivered

3


4

MySQL 5.6 GA 15 Months Ago – Best GA Ever

● Performance

– Scales to 48 CPU threads

– Up to 230% faster than 5.5

● Faster Optimizer

– Performance

– Diagnostics

– Better Instrumentation

● Improved Replication

● NoSQL

– Fast – 9x

– Key/value, ACID

● InnoDB

– Better Transactional

throughput


5

MySQL Repositories

● Benefits users and distros

● Simple/convenient way to install/upgrade

● Supports

– RHEL/Oracle

– Fedora

– Debian/Ubuntu

● Database, Workbench, Utilities, ODBC & Python Connectors


6

0

100,000

200,000

300,000

400,000

500,000

600,000

700,000

8 16 32 64 128 256 512 1,024

Qu

eri

es

pe

r Se

con

d

Connections

MySQL 5.7: Sysbench Read Only (Point Select)

MySQL 5.7

MySQL 5.6

MySQL 5.5

MySQL 5.7: Sysbench Benchmark

Intel(R) Xeon(R) CPU E7-4860 x86_64 4 sockets x 10 cores-HT (80 CPU threads) 2.3 GHz, 512 GB RAM Oracle Linux 6.5

2x Faster than MySQL 5.6 3x Faster than MySQL 5.5

645,000 QPS

6


7

MySQL 5.7: Optimizer – New Cost Model

Improving Query Performance

● New cost model enables storage engines to provide

– Accurate and dynamic cost estimates

– Key lookups, table scans, range scans, index scans, etc…

● Configurable costs for additional factors

– Speed of disk I/O operations

– Speed of memory operations

● Improved records per key estimates

● Cost values included in JSON Explain output

7

labs.mysql.com


8

Improving Manageability

● Single repository of metadata for Storage Engine and MySQL server

– Leveraging InnoDB Strengths

– Reliable crash-safe tables

– Replacing the FRMs

● INFORMATION_SCHEMA tables can be simple VIEWs

– Queries can be optimized

– Improved performance

● Extensible

– Designed to provide backward-compatibility

– Designed with plugins in mind

New Data Dictionary

labs.mysql.com


9

Improving Cloud Operations

● MySQL Fabric

– HA & Sharding

● Provisions machines & MySQL

– OpenStack Nova

– Coming Soon: Trove, AWS

● Server setup

– Clones slave

– Sets up replication

9

MySQL Fabric: OpenStack Server Provisioning labs.mysql.com


11

Single Dashboard to manage Oracle stack for Web & Cloud

● An estimated 70% of Oracle customers also use MySQL

– For Web, custom, departmental and embedded applications

– #1 Requested integration

1

Oracle Enterprise Manager for MySQL GA


12

MySQL 5.7: Innovation & Quality

● Worklogs Implemented in 5.7: 244

● New MTR Tests for 5.7: 497

● Total Bugs Fixed in 5.7: 1263

● Total Bugs Fixed since 5.5: 8259

– Since MySQL 5.5 GA Dec 15, 2010

1


13

MySQL 5.7: InnoDB, NoSQL With Memcached 6x Faster than MySQL 5.6

Thank you, Facebook

0

200,000

400,000

600,000

800,000

1,000,000

1,200,000

8 16 32 64 128 256 512 1,024

Qu

eri

es

pe

r Se

con

d

Connections

MySQL 5.7 vs 5.6 - InnoDB & Memcached

MySQL 5.7

MySQL 5.6

1 Million QPS

1



14

MySQL 5.7: Connections per Second 1.7x Faster than MySQL 5.6

2.5x Faster than MySQL 5.5

67,000 Connections/Sec

0

10,000

20,000

30,000

40,000

50,000

60,000

70,000

80,000

MySQL 5.5 MySQL 5.6 MySQL 5.7

Co

nn

ect

ion

s/Se

con

d

Connections Per Second

MySQL 5.5

MySQL 5.6

MySQL 5.7

1



15

Optimizer: New Cost Model - Why?

● More accurate cost estimates

– Better decisions by the optimizer should improve query performance

● Adapt to new hardware architectures

– SSDs, larger memory sizes, improved caches

● More maintainable cost model implementation

– Avoid hard coded “cost constants”

– Refactoring of existing cost model code

● Configurable and tunable


16

MySQL 5.7: Optimizer - Cost Info in JSON EXPLAIN

● Expanded JSON EXPLAIN

– Now includes all available cost info

– Used for Visual Explain In MySQL Workbench

1

{ "query_block": { "select_id": 1, "cost_info": { "query_cost": "200.40" }, "table": { "table_name": "nicer_but_slower_film_list", "access_type": "ALL", "rows_examined_per_scan": 992, "rows_produced_per_join": 992, "filtered": 100, "cost_info": { "read_cost": "2.00", "eval_cost": "198.40", "prefix_cost": "200.40", "data_read_per_join": "852K" }, "used_columns": [ "FID", "title", "description", "category", "price", "length", "rating", "actors" ], ...


17

MySQL 5.7: Optimizer - Query Rewrite Plugin ● Query rewrite (pre and post parse)

● Post-parse query plugin

– Rewrite problematic queries without application changes

– Add hints

– Modify join order

– Many more …

● Improve problematic queries from ORMs, third party apps, etc.

1


18

InnoDB

New Data Dictionary: Architecture

1

Query Executioner Parser Optimizer

Data Dictionar

y Tablespac

e

Data Dictionary Internal API

Internal SE

Data Dictionary External API

Plugin

Storage Engine

Plugin

Storage Engine

Plugin Plugin Plugin

User Table

Tablespace


19

Memory Instrumentation

● Aggregates statistics by

– Type of memory used (caches, internal buffers, …)

– Thread/account/user/host indirectly performing the memory operation

● Attributes include

– Memory used (bytes)

– Operation counts

– High/Low Water Marks

Statement Instrumentation

● Stored Procedures

● Stored Functions

● Prepared Statements

● Transactions

Additional Information

● Replication slave status

● MDL lock instrumentation

● User variables per thread

● Server stage tracking

● Track long running SQL

● All while reducing total footprint and overhead

MySQL 5.7: Performance Schema

1


20

MySQL SYS Schema

Helper objects for DBAs, Developers and Operations staff

● Helps simplify DBA / Ops tasks

- Monitor server health, user and/or host statistics, and other key metrics

- Spot, diagnose, and tune performance issues

● Provides easy to understand views with insights into

- IO hot spots, Locking and Costly SQL statements

- Schema, table and index statistics

● SYS is similar to

- Oracle V$ catalog views

- Microsoft SQL DMVs (Dynamic Mgmnt Views)

2


21

MySQL 5.7: InnoDB Compression Thank you, SanDisk Fusion-io

● Transparent Page Level Compression

– Happens transparently in background threads

– Managed entirely within the IO layer

– Uses sparse file and "hole punching" support in OS kernels and File Systems

● Reduces IO

– Improves MySQL performance

– Improves storage efficiency

– Reduces write cycles, thus increasing SSD lifespan

● Applies to all InnoDB data, including the system tablespace and UNDO logs

2

labs.mysql.com


22

MySQL 5.7: InnoDB - New Features ● InnoDB support for MySQL Group Replication

– High priority transactions

● InnoDB Native Partitioning

– Will allow us to eliminate previous limitations on partitioned tables

– Removes memory usage problems when many partitions are used

● Support for 32K and 64K pages

● General TABLESPACE support

– Store multiple tables in user defined tablespaces

2

labs.mysql.com


23

● Consolidate updates from multiple Masters into one Slave

– Consolidated view of all shards

– More flexible topologies

– Centralized point for backups

● Compatible with Semi-Sync Replication & enhanced MTS

● Master-specific slave filters

MySQL 5.7: Multi-Source Replication

Binlog

Master 1

Binlog

Master 2

…

…

Binlog

Master N

IO 1

Relay 1

Coordinator

W1 W2 … WX

IO 2

Relay 2

Coordinator

W1 W2 … WX

…

…

Coordinator

W1 W2 … WX

IO N

Relay N

Coordinator

W1 W2 … WX

Slave

2

labs.mysql.com


24

● Up to 5X performance vs single threaded slave

– No changes to the application

– No manipulation (delaying) of Binary log Group Commit

● GTIDs & Crash-Safe Slave enabled

● Sysbench OLTP test

– 10M rows

– SSD / 48 core HT / 512 GB RAM

MySQL 5.7: Intra-Schema Multi-Threaded Slave

0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

0 4 16 25 50 100 200

Worker Threads

Slave Transactions per Second

Baseline

50 clients

100 clients

150 clients

200 clients

2

labs.mysql.com


25

● Shared-nothing virtually synchronous database system

● Multi-master update anywhere

– Conflict detection and resolution (transaction rollback)

– Optimistic State Machine Replication

● Automatic group membership management and failure detection

– No need for server fail-over

– Elastic scale out/in

– No single point of failure

– Automatic reconfiguration

● Well integrated

– InnoDB

– GTID-based replication

– PERFORMANCE_SCHEMA

MySQL 5.7: Group Replication

Application

MySQL Masters

Replication Plugin

API

MySQL Server

Group Comms (Corosync)

2

labs.mysql.com


26

MySQL Fabric 1.5: High Availability & Sharding

● OpenStack Integration

● High Availability

– Server monitoring; auto-promotion and transparent application failover

● Sharding for Scale-out

– Application provides shard key

● Integer, date or string

– Range or Hash

– Tools for resharding

● Fabric-aware connectors rather than proxy: Python, Java, PHP, .NET, C (labs)

– Lower latency, bottleneck-free

MySQL Fabric

Connector

Application

Read-slaves

SQL

HA group

Read-slaves

HA group

Connector

Application

26

GA


27

MySQL HA & Scaling Solutions MySQL

Replication MySQL Fabric

Oracle VM Template

Oracle Clusterware

Solaris Cluster

Windows Cluster

DRBD MySQL Cluster

App Auto-Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ ✔

Data Layer Auto-Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ ✔

Zero Data Loss MySQL 5.7 MySQL

5.7 ✔ ✔ ✔ ✔ ✔ ✔

Platform Support All All Linux Linux Solaris Windows Linux All

Clustering Mode Master +

Slaves Master +

Slaves Active/Pa

ssive Active/Pass

ive Active/Passive

Active/Passive

Active/Passive

Multi-Master

Failover Time N/A Secs Secs + Secs + Secs + Secs + Secs + < 1 Sec

Scale-out Reads ✔ ✖ ✖ ✖ ✖ ✖ ✔

Cross-shard operations N/A ✖ N/A N/A N/A N/A N/A ✔

Transparent routing ✖ For HA ✔ ✔ ✔ ✔ ✔ ✔

Shared Nothing ✔ ✔ ✖ ✖ ✖ ✖ ✔ ✔

Storage Engine InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ NDB

Single Vendor Support ✔ ✔ ✔ ✔ ✔ ✖ ✔ ✔

27


28

MySQL 5.7 : Syslog Support for Linux/Unix platforms

● Native support for syslog

● Simple option to (re)direct log output to native syslog facility

● Start-up server configuration option

● Dynamically in the running server

– System variable log_syslog (ON/OFF, defaults to OFF).

2

Thank you, Simon Mudd at booking.com


29

MySQL 5.7: Locking

Multiple User Level Locks per Connection

● User-level locks can be used to organize mutual exclusion

– When accessing some resource

– When table or row-level locks are not appropriate

● Request multiple locks by issuing a series of GET_LOCK statements

● Replaces custom user-level lock implementation

– With one based on the MDL lock manager

– Deadlocks between different connections acquiring user-level locks, metadata locks, and those waiting for table flushes are properly detected and reported as errors.

2

Thank you, Konstantin Osipov!


30

MySQL 5.7: Server-side Statement Timeouts

Thank you Davi Arnaut!

● Server side statement timeouts

– Global for server, per session, or for individual SELECT statements

● Expanded to Windows and Solaris, restricted by removing USER option

SELECT MAX_STATEMENT_TIME = 109 * FROM my_table;

3


31

● Replaced custom code

– For spatial calculations

– For spatial analysis

● Provides OGC compliance

– With improved performance

● Boost.Geometry contains

– Field and domain experts

– Bustling and robust community

● We’re also Boost.Geometry contributors!

MySQL 5.7: GIS - Integrating Boost.Geometry

3


32

HTTP Plugin for MySQL

● Server Plugin adds HTTP(S) endpoints to MySQL

● Results are serialized to JSON format encoded as UTF8

● Provides 3 choices of User Endpoint Types

– SQL

– CRUD - Key-Value

– JSON - Document

3

labs.mysql.com


33

●MySQL Workbench 6.2 ● Fabric

– Add fabric node, browse, view, connect

● Performance Dashboard

– Performance Schema Reports & Graphs

● Visual Explain

● GIS Viewer

● Migration

– New Microsoft Access

– Microsoft SQL Server, Sybase, PostgreSQL

3

GA


34

● Simple and convenient way to install & update MySQL products

● Supports the following distributions

– Oracle, Red Hat, CentOS

– Fedora

– Ubuntu, Debian

● Coming Soon

– SUSE

– Preconfigured Containers

– Improved support for popular DevOps deployment tools

The latest releases of

– MySQL Database – MySQL Workbench – MySQL Connector/ODBC – MySQL Connector/Python – MySQL Connector/NET – MySQL Utilities

MySQL Community Repositories: Yum, APT,NuGET

3


35

MySQL Source Code Available on GitHub

● Git for MySQL Engineering

– Fast, flexible and great for a distributed team

– Great tooling

– Large and vibrant community

● GitHub for MySQL Community

– Easy and fast code availability to the community and to downstream projects

– Beta repo available today at https://github.com/mysql

– More info at http://mysqlrelease.com

3

https://github.com/mysql

https://github.com/mysql

http://mysqlrelease.com/


36

Better performance and operational simplicity

MySQL Cluster 7.4

● Performance gain over 7.3

– 47% (Read-Only)

– 38% (Read-Write)

• Faster maintenance operations

– Nodal & Rolling restarts

– Upgrades

3


37

●MySQL Cluster: Active-Active Geo-Replication

● Asynchronous replication between MySQL Clusters

● Active-Active

– Update anywhere

– Conflict detection

● Application notified through exception tables

● Can opt to have conflicts resolved automatically

– Auto-conflict-resolution

● Conflicting transaction and dependent ones are rolled-back

● No required changes to app schema

3


40

Security

● AES 256 Default

● Password Rotation Policies

– Globally and at user level

● Deploy unattended default secure install

– Random password set on install

– No anonymous accounts

– No test account, test schema, or demo files


43

Performance Schema

● Memory usage

– 200+ types

– Aggregate stats by type cache, buffers, etc.

– Thread/account/user/host performing operation

– Memory used, operation counts, high/low watermarks

● Statement instruments extended

– Stored procedures, stored functions, prepared statements, transactions

● Replication status


44

SYS Schema

● Simplified views on PS and IS

– Help simplify DBA tasks by monitoring health, growth rates

– Spot, diagnose and tune performance problems

● Easy to understand insights into

– IO hot spots

– Costly SQL statements

– Dynamic statistics on tables, indexes, and schemas

– Wait times, locking

– InnoDB stats


45

Replication – Faster, higher, stronger

● Higher throughput

– Slave : Apply transactions in parallel even within same database

– Master :Better sync between replication-user sessions

● Better semi-sync performance

– Loss less replication

● PS tables for monitoring

● Dynamic replication filters

● Check out PECL mysqlnd_ms

– Replication and load balancing, does read/write splitting for you!


47

Multi-Threaded Slave

● 6x salve output

● Tune size of group commit buckets on master

● Evolution

– 2010 inter-schema 5.6 labs

– 2013 Inter-schema 5.6 GA

– 2014 Intra-schema 5.7.2


50

Proposal: enable SQL MODE to ONLY_FULL_GROUP

Default behavior for other databases

In 5.6 and lower, MySQL was guessing what to use for non-grouped fields.

–New ANY_VALUE() SQL Function to allow 'non-deterministic' statements so things can work 'old way'


51

Replication

5.6 provided more durable replication with group commit

5.7 proposal – make durable the default

–sync_binlog =1

–Master-info-repository = TABLE

–Relay-log-infor-repository = TABLE


52

Proposal

Deprecate SHOW ENGINE INNODB MUTEX

–Overlaps with Performance Schema

Deprecate InnoDB Monitor tables

–Developed long before Performance Schema


53

Proposal – make STRICT the default SQL Mode

Lot of criticism that MySQL is too permissive, truncating out-of-range, zero dates, etc.

Roll ERROR_FOR_DIVISION_BY_ZERO, NO_ZERO_DATE, and NO_ZERO_IN_DATE

into STRICT Mode


54

Proposal

Deprecate EXPLAIN PARTITIONS

Deprecate EXPLAIN EXTENDED


55

Alter Ignore Table

Useful for adding PRIMAY/UNIQUE keys on tables with duplicate keys

–Will silently drop rows for you

–Deprecated 5.7.17

–Removed in DMR4

–Was a MySQL-only extension


56

Query Cache

● Defaults to DISABLED in 5.6

● Plans for other options in later releases

● If you have data that should be cached like sports scores, put in a cache (memcached) ot store it in the application.


57

Deprecate NULL synonym \N

Anyone using? We DO want feedback!!

For removal in 5.7 or 5.8


58

FEDERATED & MERGE Storage Engines

We recommend multi-source replication over Federated

We want your feedback if you are still using Federated!!

Similar to Partitioning but MyISAM ONLY

–Major limitations with Partition Exchange

–Anyone using?


59

Slide to check if audience is still awake


60

InnoDB temp tables

● New separate table space for temp tables

● Better create/drop – 11x faster

● 2-4x faster INSERT/DELETE/UPDATE

● Optimized DML operations

● Undo logs in temp table spaces


62

Triggers

● Multiple Triggers per table

● Set execution order

● Check column constraints at end of trigger execution


63

Error logging

● Three levels

– Errors Only

– Errors + Warnings

– Errors + Warnings + Notes (default)

● --log-error-verbosity = 1

● SET GLOBAL log_error_verbosity=1;

● --log_timestampes = SYSTEM UTC (default)


64

Optimized for Web, Cloud-based, Embedded use cases

Simplified, Pluggable architecture

-Maintainability, more extensible

-More NoSQL options (HTTP, JSON, JavaScript, etc.)

Re factoring

-Data Dictionary in InnoDB

-Optimizer/Parser/Protocol

InnoDB

-Optimized for SSD

-GIS

Easy HA, Replication and Sharding

MySQL Database Development Priorities


65

mysql.com

- MySQL Products, Editions, Training, Consulting

- TCO calculator

- Customer use cases and success stories

dev.mysql.com

- Downloads, Documentation

- Forums

- PlanetMySQL

eDelivery.oracle.com

- Download and evaluate all MySQL products

Learn More


66

Free/Cheap ways to learn MySQL

● MySQL Marinate

– Free online virtual class by the Boston MySQL Users Group

– One chapter a week in O'Reilly Learning MySQL

– Homework checked by Mozilla's Senior DBA

● GirlDevelopIT.com

– Cheap

– Weekend/Nights

– Local chapter in your area?


67

MySQL 5.7 [email protected] @stoker slideshare.net/davestokes

Technology

MySQL 5.7 - Everything You Need to Know