View
900
Download
4
Category
Tags:
Preview:
DESCRIPTION
Do you have the background necessary to take full advantage of Tungsten Replicator in your environments? Tungsten offers enterprise-quality replication features in an open source package hosted on Google Code. This virtual course will teach you how to set up innovative topologies that solve complex replication problems. We start with single MySQL servers running MySQL replication and show a simple path migration path to Tungsten. Course Topics - Checking host and MySQL prerequisites - Downloading code from http://code.google.com/p/tungsten-replicator/ - Installation using the tungsten-installer utility - Transaction filtering using standard filters as well as customized filters you write yourself - Enabling and managing parallel replication - Configuring multi-master and fan-in using multiple replication services - Backup and restore integration - Troubleshooting replication problems - Logging bugs and participating in the Tungsten Replicator community. Replication is a powerful technology that takes knowledge and planning to use effectively. We give you the background that makes replication easier to set up, and allows you to take full advantage of the Tungsten Replicator benefits. Learn how to configure and use it more effectively for your projects in the cloud as well as on-premise hardware.
Citation preview
©Continuent 2013
Tungsten University: Set Up and Operate
Tungsten Replication Robert Hodges, CEO
©Continuent 2013
Introducing Continuent
2
• The leading provider of clustering and replication for open source DBMS
• Our Product: Continuent Tungsten
• Clustering - Commercial-grade HA, performance scaling and data management for MySQL
• Replication - Flexible, high-performance data movement
©Continuent 2013
Introducing Tungsten Replicator
3
©Continuent 2013
Tungsten Replicator Overview
4
Master
(Transactions + Metadata)
Slave
THL
DBMSLogs
Replicator
(Transactions + Metadata)
THLReplicator
Download transactions via network
Apply using JDBC
©Continuent 2013
Tungsten Replication Service
5
Extract Filter Apply
StageExtract Filter Apply
StageExtract Filter Apply
Stage
Pipeline
MasterDBMS
TransactionHistory Log
In-MemoryQueue
SlaveDBMS
©Continuent 2013
Overview of Installation Process
1. Set up hosts
2. Prepare MySQL replicas
3. Download software
4. Install using tpm utility
6
Amazon Setup:https://docs.continuent.com/wiki/display/TEDOC/
Preparing+EC2+Servers
©Continuent 2013
Old Way of Installing Tungsten...
• tungsten-installer - Perform initial installation
• con!gure-service - Set up individual replication services
• update - Change parameters on services after installation
7
The tpm utility replaces all of this!
©Continuent 2013
How tpm Works
8
db1
db2
db3
Staging copy of files
check prereqscopy code
configure topology
©Continuent 2013
What Is Tungsten Doing?
9
©Continuent 2013
Commands to Check Replication
• trepctl services -- All services
• trepctl status -- Overall pipeline status
• trepctl status -name stages -- Stage summary
• trepctl status -name stores -- Store summary
• trepctl status -name tasks -- Task detail
• trepctl [-service name] heartbeat -- Is it Alive?
10
©Continuent 2013
Changing Replicator Con!guration
11
©Continuent 2013
Commands to Change Con!guration
• tpm update - Alter con"guration
• tpm update - Upgrade the replicator to a new version
12
Hint: Don’t change replicator configuration files directly...
©Continuent 2013
Recon!guring a Replicator
• No restart on local host
trepctl offlinetools/tpm update ... --no-restarttrepctl configuretrepctl online
• With automatic restart
tools/tpm update ...
13
©Continuent 2013
Enabling Parallel Replication
14
©Continuent 2013
Parallel Replication of Sharded Data
15
Extract Filter Apply
StageExtract Filter Apply
StageStage
Pipeline
MasterDBMS
TransactionHistory Log
ParallelQueue
SlaveDBMS
Extract Filter ApplyExtract Filter ApplyExtract Filter Apply
©Continuent 2013
Fixing Problems
16
©Continuent 2013
The Scene of the Crime
17
trepctl -host db2 statusProcessing status command...NAME VALUE---- -----...pendingError : Event application failed: seqno=1957469 fragno=0 message=java.sql.SQLException: Statement failed on slave but succeeded on masterpendingErrorCode : NONEpendingErrorEventId : mysql-bin.000008:0000000461736666;0pendingErrorSeqno : 1957469pendingExceptionMessage: java.sql.SQLException: Statement failed on slave but succeeded on master create table trouble(id int primary key) /* ___SERVICE___ = [university] */...Finished status command...
©Continuent 2013
Correcting problems
1. Look at the transaction (thl list -seqno NNN)
2. Look at the replicator message log
3. Fix the problem and go online
OR
4. Skip over the problem and go online
OR
5. Reprovision the slave
18
©Continuent 2013
Tungsten Logs
19
©Continuent 2013
Simple Facts about Logs
• Replicator log = THL (Transaction History Log)
• One THL per replication service
• 2x the size of MySQL binlog
• Default retention of 7 days
• Take service o#ine to move or delete logs safely
• The thl tool shows you log contents
20
©Continuent 2013
Resetting Replication
21
©Continuent 2013
Resetting and Reinstalling
Reset
1. Stop replicator
2. Remove THL "les
3. Remove tungsten_<svc> databases
Reinstall
1. All of the above, plus delete /opt/continuent/*
2. Rerun installation
22
©Continuent 2013
Backup and Restore
23
©Continuent 2013
What Backup Tools Does MySQL Have?
24
MySQL Server
Process
File System
DBMS Files
mysqldump(dataset < 10Gb)
xtrabackup(10Gb < dataset < 1Tbtar/rsync(dataset < 1Tb)
file system snapshot(1Tb << dataset)
©Continuent 2013
Advanced Installation using Cookbooks
25
©Continuent 2013
How Do I Install Fan-In Replication?
26
db1Replicator
db3
Service db1
db2Replicator
Service db2
Replicator
Service db1
Service db2
©Continuent 2013
How Do I install Multi-Master?
27
db1 Replicator
Service db1
Service db2
Service db3db3
Service db1
Service db2
Service db3db2
Replicator
Service db1
Service db2
Service db3
©Continuent 2013
Tpm Understands Topologies!
• Use --topology argument!
28
Fan-In StarAll-Masters
--topology=fan-in --topology=all-masters --topology=star
©Continuent 2013
For More Help: Tungsten Cookbook
29
• Installs complex topologies automatically
• Directions in cookbook/README
• Parameters in cookbook/USER_VALUES.sh
• Node names in COMMON_NODES.sh
• Installation script for each supported topology
http://datacharmer.blogspot.com/2013/07/tpm-multi-master-composer.html
©Continuent 2013
Joining the Community
30
©Continuent 2013
Tungsten Replicator is Open Source
31
• Project home:
http://code.google.com/p/tungsten-replicator/
• Log bugs, "nd builds, post in replicator discussion group
• Documentation:
https://docs.continuent.com/wiki/display/TEDOC/Tungsten+Documentation+Home
©Continuent 2013
Wrapping Up
32
©Continuent 2013
More Webinars and Talks!
33
• How Gittigidiyor (a subsidiary of eBay) Replicates Data in Real Time from MySQL to Oracle(6 November 2013 10am EDT/3pm GMT)
• Come see us at Percona Live London! (11-12 November 2013)
Send any feedback to: tu@continuent.com
©Continuent 2012.
Continuent Web Page:http://www.continuent.com
Tungsten Replicator 2.0:http://code.google.com/p/tungsten-replicator
Our Blogs:http://scale-out-blog.blogspot.comhttp://datacharmer.org/bloghttp://www.continuent.com/news/blogshttp://"yingclusters.blogspot.com/
560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel +1 (866) 998-3642 Fax +1 (408) 668-1009e-mail: sales@continuent.com
Recommended