Upload
davie-stokes
View
234
Download
3
Embed Size (px)
DESCRIPTION
Presentation from Mountain Moodle Moot, Helena MT
Citation preview
Query Tuning, System Tuning, and Little Known Tricks Using MySQL
Workbench [email protected]
@stokerSlideshare.net/davestokes
Mountain Moodle Moot
Databases Can Be a Mystery, Misery, or Magnificent
● You do not need to be a DBA● Nothing wrong with being a DBA
● You should think in sets● 'Relational Calculus for $500, Alex.'
● Let the Database do the Heavy Lifting
● Select What you need for speed
● Yes, YOU do need to do backups● And know how to restore from them
Server Tuning
Run latest release of MySQL Better performance Bug Fixes New features
Don't skimp on hardware Memory is better ROI than disk New SSDs are amazing FusionIO cards OMG
Configuration
Configuration should match hardware Remove Query Cache Nasty Mutex But what if you have 'repeatable data', i.e. sports score
Memcached In Application
Know your parameters Session, Global Needs reboot to change or on the fly
I_S, P_S, SYS
Information_Schema Performance_Schema Similar to V$ variables for Oracle 2-5% performance overhead
SYS Predefined for most common questions
You will not need them (now) but know that they are there when you need to eek out last %
Intro to Query Tuning
Reading from disk to memory is 100,000 slower than memory alone
'SELECT *' on a wide table with many col-umns when you need only two columns MUL-TIPLED by N thousands access a minute adds up
Read what you need for speed, no SELECT * Use joins to get desired sets
Our First Query
SELECT City.Name, Country.name, City.PopulationFROM CityJOIN Country ON
(Country.code = City.CountryCode)
Results
Our First Query
SELECT City.Name, Country.name, City.PopulationFROM CityJOIN Country ON
(Country.code = City.CountryCode)
Columns Selected (note Name & Name)
Our First Query
SELECT City.Name, Country.name, City.PopulationFROM CityJOIN Country ON
(Country.code = City.CountryCode)
Tables used
Venn Diagram
City is A, Country is B
Make query match diagram
SELECT a.Name, b.name, a.PopulationFROM City AS a JOIN Country AS b
ON (b.code = a.CountryCode)
Make query match diagram
SELECT a.Name, b.name, a.PopulationFROM City a JOIN Country b
ON (b.code = a.CountryCode)
Can also use AS to alias column names
Aliased Column Names
SELECT a.Name as 'City', b.name as 'Country' , a.Population as 'Pop.'FROM City aJOIN Country b ON (b.code = a.CountryCode)
Updated Results
Query Plan
EXPLAIN
EXPLAIN is pre-pended to a SQL statement and produces output like:
EXPLAIN
We see how many roads to be read:
Guess on rows needing to be read
4079 x 1 = 4079 rows to be read
EXPLAIN
In English – We read all the rows in the City table and find via an index to find the match in the Country table:
So what is an index?
http://en.wikipedia.org/wiki/Database_indexA database index is a data structure that improves the speed of data retrieval operations on a database table at the cost of additional writes and the use of more storage space to main-tain the extra copy of data. Indexes are used to quickly locate data without having to search every row in a database table every time a database table is accessed. Indexes can be cre-ated using one or more columns of a database table, provid-ing the basis for both rapid random lookups and efficient ac-cess of ordered records.
Uh, in English
Indexes let you go directly to the record you want Rolodex Card Catalog
Overhead Maintenance Sometimes you
DO have read all the table
B-Tree Index
A Peek at an Index
SHOW INDEX FROM <table>;
Statistics
Optimizer uses table statistics to generate cost data.
Out of data stats can make optimizer plan poorly
See 14.13.17.1 Con-figuring Persistent Optimizer Statistics Parameters in MySQL Manual
Run ANALYSE TA-BLE to update stats
Run ANALYSE TA-BLE after adding a new index
Store stat data at SHUTDOWN, Re-store at REBOOT
Lets look at the tables
EER Map of tables in World Database
Just City and Country
The two tables we have been using
Just City and Country
JOIN Country b ON (b.code = a.CountryCode)
Without Indexes
Plural of Moose List of definitions not alphabetized Duplicates
Compound Index
CREATE TABLE test ( id INT NOT NULL, last_name CHAR(30) NOT NULL, first_name CHAR(30) NOT NULL, PRIMARY KEY (id), INDEX name (last_name,first_name));
This index can use be used for searching on last_name + first_name or just last_name
Multi Column Index
CREATE INDEX CityStateZip ONCity (City, State, Zip);
This index can be used to search onCity + State + Zip, City + State, or City
This index can not be used to search onZip, State + Zip
Speeds up finding Zip when used in City + State
Why is it Important to Optimize?
MySQL, unlike other databases, has no way to lock a query plan. That means each time a query is submitted it goes through the opti-mizer.
Optimize Data
Ideally you want to allocate just enough stor-age for any possible record.
Have to use best judgment Human age range 0-999 Postal codes Address lines Phone numbers 01-123-123-1234 and extension ?! PROCEDURE ANALYSE will examine records, make suggestion
Not all Integers BIGINTs Not all text fields VARCHAR(255) Excess column widths are extra baggage
Integers
Type Storage (bytes)
Minimum Signed
Maximum Signed
Minimum Unsigned
Maximum Unsigned
TINYINT 1 -128 127 0 255
SMALLINT 2 -32768 32768 0 65535
MEDIUMINT
3 -8388608 8388607 0 16777215
INT 4 -2147483648 2147483647 0 4294967295
BIGINT 8 -9223372036854
775808
9223372036854775807
0 18446744073709551615
Are you really going to have 18,446,744,073,709,551,615 student id numbers?
Making it up in volume
Consider the worst case Data needs to be read off disk Transferred into memory Sent across network to application server Application server loads data into memory Application sends that data over network to end user
INT VS BIGINT – 4 extra bytes each times X number of records X number of users
Flexibility
Making changes to a schema is costly. It is a lot better for 5.6 Much better for 5.7
Must update replication masters and slaves (double check if done through replication)
Gender Example 2, 3, 17, or 58+
Replication
Changes made to master Replicated to slave server(s) Three threads
BINLOG dump thread Slave I/O thread Slave SQL thread
Read/Write Splits
Writes to Master Reads from slave(s)
Types of Replication
Statement Based – SQL Row Based – Binary changes after SQL exe-
cuted Mixed – Uses Statement Based but switches
when advantageous
Smarter Row Based Replication
MySQL 5.6 and later No need to copy unmodified parts of a row
Global Transaction ID
Previously had to match Master's log position to when starting replication
Each transaction unique Point slave at master and ask it to catch up
MySQL Utilities
Free, open sources Python scripts to auto-mate functions
mysqldbcompare Compare databases on two servers or the same server Compare definitions and data Generate a difference report Generate SQL transformation statements
mysqldbcopy Copy databases between servers Clone databases on the same server Supports rename
mysqldbexport Export metadata and/or data from one or more databases Formats: SQL, CSV, TAB, Grid, Vertical
mysqldbimport Import metadata and data from one or more files Reads all formats from mysqldbexport
mysqldiff Compare object definitions Generate a difference report
These utilities are those designed to perform general operations such as
reporting and searching.
mysqldiskusage Show disk usage for databases Generate reports in SQL, CSV, TAB, Grid, Vertical
mysqlfrm Reads .frm files, optionally in byte-by-byte diagnostic mode Generates CREATE statements from table definition data
mysqlindexcheck Read indexes for one or more tables Check for redundant and duplicate indexes Generate reports in SQL, CSV, TAB, Grid, Vertical
mysqlmetagrep Search metadata Regexp, database search Generate SQL statement for search query
mysqlprocgrep Search process information Generate SQL statement for search Kill processes that match query
mysqluserclone Clone a user account, to the same or different server Show user grants
mysqluc Command line client for running MySQL Utilities Allows a persistent connection to a MySQL Server Tab completion for utility names and options Allows calling the commands with shorter names, such as using "serverinfo" instead of mysqlserverinfo
These utilities are those designed to support replication and high availability operations
for MySQL servers.
mysqlfailover Provides automatic failover on a replication topology Uses Global Transaction Identifiers (GTID, MySQL Server 5.6.5+)
mysqlreplicate Setup replication Start from beginning, current, specific binlog, pos
mysqlrplms Provides round-robin multi-source replication (a slave server continually cycles through multiple masters in order to store
a consolidated data set) Uses Global Transaction Identifiers (GTID, MySQL Server 5.6.9+)
mysqlrpladmin Administers the replication topology Allows recovery of the master Commands include elect, failover, gtid, health, start, stop, and switchover
mysqlrplcheck Check replication configuration Tests binary logging on master
mysqlrplshow Show slaves attached to master Can search recursively Show the replication topology as a graph or list
mysqlrplsync Check data consistency between servers in a replicated setup Uses Global Transaction Identifiers (GTID) Requires MySQL Server 5.6.14 and higher
MySQL Fabric
Also written in Python and free Sharding High
Availability
Exercise
Mysqldbcopy –source=root:pass@localhost --destination=root:pass@localhost world:xworld
Use MySQL Utility mysqldbcopy to make a copy of world database
Exercise
Using world database, run query from earlier to see query plan.
SELECT a.Name as 'City', b.name as 'Country' , a.Population as 'Pop.'FROM City aJOIN Country b ON (b.code = a.CountryCode);
Remove Indexes
Use worldx schema which redefines the World database without indexes
Exercise
Using worldx database again, run query from earlier to see query plan to see cost.
SELECT a.Name as 'City', b.name as 'Country' , a.Population as 'Pop.'FROM City aJOIN Country b ON (b.code = a.CountryCode)
Back to World
How does a 'qualifier' effect query plan?
SELECT a.Name as 'City', b.name as 'Country' , a.Population as 'Pop.'FROM City aJOIN Country b ON (b.code = a.CountryCode)WHERE a.Population > 500000;
Back to World Again
How does a second 'qualifier' effect query plan?
SELECT a.Name as 'City', b.name as 'Country' , a.Population as 'Pop.'FROM City aJOIN Country b ON (b.code = a.CountryCode)WHERE a.Population > 500000ORDER By b.name;
Run those last two queries on worldx
Did indexes help/hurt? Are the qualifiers run before/after join?
Moodle 2.7
A quick look at the schema
More Info on Tuning
High Performance MySQL - 3rd Edition Schwartz, Zaitsev, & Tkachenko
Effective MySql Optimizing Sql Statements Bradford
MySQL Marinate
Free on-line Virtual class Boston MySQL Users Group One Chapter a week Homework Checked by
Mozilla's senior DBA
MySQL Training from Oracle
Introduction to MySQL MySQL Quickstart Fundamentals MySQL for Beginners MySQL for Database Administrators MySQL for Developers MySQL Performance Tuning MySQL High Availability MySQL Cluster MySQL Developer Techniques MySQL and PHP - Developing Dynamic Web Applica-
tions MySQL Advanced Stored Procedures
MySQL Central @ Oracle Open World
Keynotes from Booking.com, Twitter & Dropbox 50+ session, tutorials, BOFs and reception
Q/A
[email protected] @stoker
MySQL Central at Oracle Open WorldSan Francisco Septermber 29th to October 3rd