Upload
keith-hollman
View
349
Download
3
Embed Size (px)
DESCRIPTION
The OpenSUSE presentation I shared in Thessaloniki on using MySQL Replication and the Blackhole storage engine.
Citation preview
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 1
A MySQL Odyssey - a Blackhole crossover
Keith Hollman
MySQL Principal Sales Consultant EMEA
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 3
The following is intended to outline our general product direction. It is
intended for information purposes only, and may not be incorporated
into any contract.
It is not a commitment to deliver any material, code, or functionality,
and should not be relied upon in making purchasing decisions.
The development, release, and timing of any features or functionality
described for Oracle’s products remain at the sole discretion of Oracle.
Safe Harbor Statement
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 4
Agenda MySQL Replication: the crossover.
From
Dev…
…thru’ the
BLACK
HOLE…
Knowledge
is key
Non-
implosion
non-
destructive
…into Pre /
Production
Windows
¿Why?
Oracle Linux
The
Future
Nebulae
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 5
¿Why?
Sharing how to automate a cross-platform data transfer with
MySQL, applying data filters at the same time.
Small, quick and easy to setup, minimizing performance & resource
overheads.
Maneuvering the High Availability door to other possibilities, such as
Security control, Auditing, horizontal & vertical scalability, read-write
segregations.
What’s this all about:
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 6
¿Why?
A Developer using MySQL on Win (Testing).
Need to export & import data / schema definition into DBA’s /
Sys.Admin’s Linux environment (Pre-Prod, UAT, Production).
Certain changes are required and it has to be easy, and unobtrusive.
And data sensitive columns?
Wouldn’t it be nice if I could have it all transferred directly, automatically
and not have to worry about platform differences?
Just “a” Scenario
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 7
¿Why? The Setup
“Dev” pc
Win.7 Pro SP1 Intel i5 2.67GHz 8Gb RAM 64bit
CREATE TABLE `crossover` …… ENGINE=InnoDB;
“BLACKHOLE” env
Oracle Linux 6.3 UEK 2.6.39 VM 1xCPU 512Mb RAM 32bit 5Gb disk
ALTER TABLE `crossover` ENGINE=BLACKHOLE;
“Prod” env
SLES 11 SP2 3.0.74 VM 1xCPU 1Gb RAM 10Gb disk
ALTER TABLE `crossover` ENGINE=InnoDB;
MySQL 5.6.11 EE Replication Setup
using GTID’s: Master (Win)
-> Slave / Master (OL) -> Slave (SLES)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 8
¿Why?
BLACKHOLE storage engine:
“.. accepts data but throws it away and does not store it. Retrievals always return an empty
result”
– Row Based Replication best (vs Statement, autoinc & PK’s)
– UPDATE’s & DELETE’s triggers aren’t activated: will break replication.
– All operations on BLACKHOLE will be replicated to SLES slave.1
Others:
– Replicating with lower_case_table_names=1.
– replicate-ignore-db, replicate-do-table, replicate-wild-*.
– Double check FW (iptables –F for easiness).
MySQL BLACKHOLE & Other Awareness
1 Should we be tempted to temporarily ‘fix’
replication by changing SE and back.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 9
The BLACKHOLE Crossover
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 10
From Dev… thru’ the BLACKHOLE…
Logical Dump of the master database “sakila” from Win to OL: [mysql@ol63uek01 ~]$ mysqldump -uroot -poracle -h 192.168.56.101 -P3307 -B sakila \ --master-data=2 | mysql -uroot
ol63 10:40 sakila> show databases;
Set Replication up…
… and make sure we only replicate “sakila”: [mysql@ol63uek01 ~]$ vi /etc/my.cnf : [mysqld] replicate-do-db =sakila
Check it’s working: ol63 11:00 sakila> show slave status\G
Win 11:00 sakila> call BlackholeInsert (1000); ..& look for the “*–relay.bin*” logs.
[ From Win to Oracle Linux ]
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 11
Win 11:01 sakila> select count(*) from crossover;
Stepping into the BLACKHOLE: ol63 11:02 sakila> select count(*) from crossover; ol63 11:02 sakila> show create table crossover; ol63 11:02 sakila> alter table crossover engine=blackhole;
Query OK, 1001 rows affected (0.16 sec) Records: 1001 Duplicates: 0 Warnings: 0
Confirming: ol63 11:02 sakila> show create table crossover; ol63 11:03 sakila> select count(*) from crossover;
From Dev… thru’ the BLACKHOLE… Continued…
+----------+ | count(*) | +----------+ | 1001 | +----------+ 1 row in set (0.00 sec)
+----------+ | count(*) | +----------+ | 1001 | +----------+ 1 row in set (0.00 sec)
| crossover | CREATE TABLE `crossover` ( `ID` int(7) NOT NULL AUTO_INCREMENT, `Name` char(20) NOT NULL DEFAULT '', PRIMARY KEY (`ID`) ) ENGINE=InnoDB AUTO_INCREMENT=1002 DEFAULT CHARSET=utf8 | 1 row in set (0.00 sec)
| crossover | CREATE TABLE `crossover` ( `ID` int(7) NOT NULL AUTO_INCREMENT, `Name` char(20) NOT NULL DEFAULT '', PRIMARY KEY (`ID`) ) ENGINE=BLACKHOLE DEFAULT CHARSET=utf8 | 1 row in set (0.00 sec)
+----------+ | count(*) | +----------+ | 0 | +----------+ 1 row in set (0.00 sec)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 12
… thru’ the BLACKHOLE… into Prod.
Logical dump of ol63uek BLACKHOLE table to SLES Production env: [mysql@suse01 ~]$ mysqldump -uroot -p -h 192.168.56.102 -B sakila --master-data=2 \ | mysql -uroot suse01 11:11 sakila> show databases;
Set Replication up… again (ol63 is our Master now) & check it works: suse01 11:12 sakila> show slave status\G
Coming out of the BLACKHOLE, into InnoDB@SLES: suse01 11:12 sakila> alter table crossover engine=InnoDB;
Pull data from Win into SLES (non-blackhole tables): [mysql@suse01 ~]$ mysqldump -uroot –p –h 192.168.56.101 -P3307 --no-create-info \ --no-create-db –B sakila | mysql -uroot
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 13
Non-Implosion and Non-Destructive.
http://ourwikiworld.blogspot.com.es/2011/04/amazing-physics-stunning-building.html
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 14
Non-implosion & non-destructive
Truncate table crossover;
Bulk INSERT’s.
DDL modifications.
Finer Replication capabilities.
Schema manoeuvrability.
Performance analysis.
Located on same Master server.
Isolated env’s & modular.
! UPDATE & DELETE triggers.
! Added Operational commitments.
! Data models, PK & Autoinc cols.
! IO monitoring / impact evaluation
removed.
! Infrastructure becomes inter-
dependent (replication).
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 15
Non-implosion & non-destructive cont. Win ol63uek SLES
OBJECT_NAME crossover Diff vs orig. crossover diff crossover diff
COUNT_STAR 10,002 -1 10,001 0 40,007 -20,004
SUM_TIMER_WAIT 231,905,415,096 -26,335,040 7,504,322,436 0 3,783,883,427,646 -9,161,505,807
MIN_TIMER_WAIT 11,728,944 0 22,464 0 24,384 0
AVG_TIMER_WAIT 23,185,664 -376 750,087 0 94,580,202 94,127,574
MAX_TIMER_WAIT 50,530,335,064 0 426,302,487 0 428,082,829,761 0
COUNT_READ 1 -1 0 0 30,006 -20,004
SUM_TIMER_READ 26,335,040 -26,335,040 0 0 15,641,005,167 -9,161,505,807
MIN_TIMER_READ 26,335,040 -26,335,040 0 0 24,384 0
AVG_TIMER_READ 26,335,040 -26,335,040 0 0 521,208 126,492
MAX_TIMER_READ 26,335,040 -26,335,040 0 0 374,708,547 0
COUNT_WRITE 10,001 0 10,001 0 10,001 0
SUM_TIMER_WRITE 231,879,080,056 0 7,504,322,436 0 3,768,242,422,479 0
MIN_TIMER_WRITE 11,728,944 0 22,464 0 24,384 0
AVG_TIMER_WRITE 23,185,288 0 750,087 0 376,786,521 0
MAX_TIMER_WRITE 50,530,335,064 0 426,302,487 0 428,082,829,761 0
COUNT_FETCH 1 -1 0 0 30,006 -20,004
SUM_TIMER_FETCH 26,335,040 -26,335,040 0 0 15,641,005,167 -9,161,505,807
MIN_TIMER_FETCH 26,335,040 -26,335,040 0 0 24,384 0
AVG_TIMER_FETCH 26,335,040 -26,335,040 0 0 521,208 126,492
MAX_TIMER_FETCH 26,335,040 -26,335,040 0 0 374,708,547 0
COUNT_INSERT 10,001 0 10,001 0 10,001 0
SUM_TIMER_INSERT 231,879,080,056 0 7,504,322,436 0 3,768,242,422,479 0
MIN_TIMER_INSERT 11,728,944 0 22,464 0 24,384 0
AVG_TIMER_INSERT 23,185,288 0 750,087 0 376,786,521 0
MAX_TIMER_INSERT 50,530,335,064 0 426,302,487 0 428,082,829,761 0
COUNT_UPDATE 0 0 0 0 0 0
SUM_TIMER_UPDATE 0 0 0 0 0 0
We could use
common O.S. activity
measuring tools.
Using MySQL’s own
Performance_Schema:
After a: truncate table crossover;
Obtained via a: select * from table_io_waits_summary_by_table where object_schema ='sakila' and object_name = 'crossover‘ \G
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 16
Knowledge & The Future
http://ourwikiworld.blogspot.com.es/2011/04/amazing-physics-stunning-building.html
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 17
mysql CLI & plenty of commands.
MySQL Workbench Utilities.
– mysqldbcopy, mysqlfailover, mysqlreplicate, mysqlrpladmin, mysqlrplcheck, etc.
MySQL Performance_Schema.
MySQL Enterprise Monitor.
– Replication Advisor
– Query Analyzer
Knowledge is key: Monitoring & Utilities.
MySQL Solutions
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 18
The Future Nebulae
Existing / inherited Applications.
– eg.Using CSV type tables, from Win to Linux or vice-versa.(Address books, User
provisioning, etc.)
Cloud / Virtualization platforms.
– HA Potential: Replication, 1x Blackhole for 10+ slaves) & MySQL Cluster.
– Column filtering / Replication filtering / Triggers from Blackhole.
Eliminate sensitive or un-required columns.
Create a number of slaves from BLACKHOLE each with it’s own dataset.
Big Data / BI / custom solutions.
– Repeat bulk loads for troubleshooting / performance comparisons.
– Analyze binlogs on BLACKHOLE / backup strategies, etc.
Who can benefit?
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 19
Reference material
The BLACKHOLE Storage Engine (ref.man.)
MySQL Replication Tutorial (TWP)
The CSV Storage Engine (ref.man.)
MySQL Workbench download. (documentation)
Download Utilities without Workbench: http://dev.mysql.com/downloads/tools/utilities/1.3.html
MySQL Enterprise Monitor:
– Trial 30 day evaluation http://edelivery.oracle.com
Performance Schema documentation. (table_io_waits_summary_by_table descr.)
Oracle Linux & Unbreakable Enterprise Kernel (Getting Started).
SUSE Linux Enterprise Server 11 SP2 for x86 included packages.
Advanced Linux Command Mastery
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 20
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 21
Efxaristo OSC13, SUSE, Thessaloniki, everyone. (Thank you)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 22
Graphic Section Divider
Keynotes
Conferences Sessions
Hands-on Labs
Birds-of-a-feather sessions
Tutorials
Receptions
Demo Pods
Exhibition Hall
San Francisco, September 21-23 Additional Day of Tutorials Oracle.com/mysqlconnect
Early Bird
Discount: Register
by July 19 and Save
US$500!
Email to [email protected] for
exhibition and sponsorship opportunities
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 23
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 24