Upload
enterprisedb
View
1.137
Download
0
Embed Size (px)
Citation preview
© Copyright EnterpriseDB Corporation, 2015. All rights reserved. 1
Active / Active Database Solutions with Log Based Replication in xDB 6.0
© Copyright EnterpriseDB Corporation, 2015. All rights reserved. 2
• xDB Overview − Features − Use Cases − Components of a Multi Master Replication System
• Supporting Active / Active Database Solutions with xDB 6.0 − Faster and more efficient replication with log-based Multi Master replication for
Postgres Plus and PostgreSQL − Easier to configure publication tables in bulk with pattern matching selection rules − Ensuring High Availability with integration of the 'Control Schema’ − Improved performance in conflict detection rules
Agenda
© Copyright EnterpriseDB Corporation, 2015. All rights reserved. 3
xDB is a highly flexible database replication tool to address multiple problem sets
What is xDB?
Multi Master: Consolidated view from any node and multiple conflict detection / resolution rules • Write Availability & Scalability • Warm Standby • Localized Data Access
Single Master: Heterogeneous (Oracle, SQL Server, PG, PPAS) & filtered data sources • Offload Reporting or BI Queries from the
OLTP system • Migrate or test systems at low cost
© Copyright EnterpriseDB Corporation, 2015. All rights reserved. 4
Features • Distributed Publication / Subscription
Architecture • Snapshot and continuous synchronization
modes • Replicate one or more tables • Define and apply row filters • Flexible replication scheduler • Supports cascading replication • Replication History Viewer • Graphical Console and CLI
Use Cases • Write Availability • Write Scalability • Warm Standby Servers • Localized Data Access • Offloading Reporting and Business
Intelligence Queries • Testing Systems in Parallel • Migrating Data
xDB Feature Highlights
© Copyright EnterpriseDB Corporation, 2015. All rights reserved. 5
Why xDB over other replication tools?
• Rich graphical management console − complete control over your replication system − a history viewer − a replication scheduler
• Command line operations suitable for scripting.
• Multi-Master is an ideal solution for geographically dispersed data centers or co-located redundant data sets. It provides a critical component of a 24x7 near zero downtime maintenance operation.
• Single Master Heterogeneous Sources provide integration and a path to reduce other license and maintenance costs
© Copyright EnterpriseDB Corporation, 2015. All rights reserved. 6
• Publication Server & Configuration Files: − Java application; configures publication database and
master nodes for replication; performs replication. • Controller database (Can be any Master Node):
− Primary provider of the replication system information to the publication server.
− Must be first one running and accessible to publication server; can be set to any node.
• Master Definition Node − First node added to the MMR system − Table definitions and optionally initial data to publish − Role can be reassigned
• Master Node − Replicated & updatable data stores
• More Info in Documentation Section 2.3.1.x – Physical Components of xDB system
Architecture and Physical Components of xDB MMR xDB is a set of programs + datastores containing configuration information and metadata that form a replication system.
© Copyright EnterpriseDB Corporation, 2015. All rights reserved. 7
• xDB Overview − Features − Use Cases − Components of a Multi Master Replication System
• Supporting Active / Active Database Solutions with xDB 6.0 − Faster and more efficient replication with log-based Multi Master replication for
Postgres Plus and PostgreSQL − Easier to configure publication tables in bulk with pattern matching selection
rules − Ensuring High Availability with integration of the 'Control Schema’ − Improved performance in conflict detection rules
Agenda
© Copyright EnterpriseDB Corporation, 2015. All rights reserved. 8
• Changes to rows in the source tables are extracted from the Write-Ahead Log segments (WAL files).
• xDB connects via walsender interface. Changes are streamed on continuous basis. Eliminates need to poll.
• The extracted changes are converted to an in-memory data structure and applied to the target tables by means of SQL statements executed using JDBC.
• See Documentation section 2.2.10 for information on the log-based method.
Faster and more efficient replication with log-based MMR for PPAS & PG 9.4 Capture data changes without impacting the online transaction processing rate
High-level architecture of how xDB communicates with database server to fetch logical changes.
© Copyright EnterpriseDB Corporation, 2015. All rights reserved. 9
• Changes to rows in the source tables are extracted from the Write-Ahead Log segments (WAL files).
• xDB connects via walsender interface. Changes are streamed on continuous basis. Eliminates need to poll.
• The extracted changes are converted to an in-memory data structure and applied to the target tables by means of SQL statements executed using JDBC.
• See Documentation section 2.2.10 for information on the log-based method.
Faster and more efficient replication with log-based MMR for PPAS & PG 9.4 Capture data changes without impacting the online transaction processing rate
© Copyright EnterpriseDB Corporation, 2015. All rights reserved. 10
• When you have a lot of tables to choose from when building your publication, the wildcard selection dialog is a useful tool
• Here we see a few examples − % as a wildcard character to identify all
tables of the _gb designation − [character lists] to identify more patterns
with specific characters
• Detailed Help is available in the client • See Documentation section 7.1 for
information on the wildcard selector UI.
Easier to Configure Publication Tables in Bulk Use sophisticated pattern matching rules to identify tables for publication
© Copyright EnterpriseDB Corporation, 2015. All rights reserved. 11
• In an xDB MMR systems, if any master goes offline, remaining nodes continue to synchronize amongst themselves.
• When an offline master is brought back, pending transactions are synchronized.
• Each node serves as a backup of other nodes* • As part of the Control Schema integration project,
normal replication configuration (control tables, procedures/functions, sequences…) and history tables are also replicated amongst master nodes.
• Thus, there is resiliency built into the system such that you could lose any database node and replication operations will continue
• See Documentation 6.11 for more information
*Keep care when using Table Filters, as some nodes may not have all data replicated onto it.
Ensuring High Availability
Control Schema
Control Schema
Control Schema
© Copyright EnterpriseDB Corporation, 2015. All rights reserved. 12
• There is however special significance of the Controller Database − Publication server accesses it for configuration
information − So any changes made to system config using GUI
or CLI is done on Controller DB first
• For planned maintenance − You can identify a different node to be your
Controller using the GUI Console or CLI
• For unplanned maintenance − Edit the xDB Configuration file to contain
connection information of another master node − Restart publication server. − You can now use the CLI or GUI tool to connect to,
monitor and manage the replication system.
• See Documentation 6.11 for more information
Ensuring High Availability
© Copyright EnterpriseDB Corporation, 2015. All rights reserved. 13
• xDB 5.x conflict detection query is based on IN clause for doing the comparison of PK attribute(s).
• i.e. SELECT s.* FROM _edb_replicator_pub.rrst_public_orders WHERE order_id IN (1,5,10,15,20,21);
• Performance starts to degrade as table grows or if PK type is not numeric.
• We replaced the existing query with a ‘common table expression’ optimize the query execution time. • i.e. WITH temp_table AS (SELECT…
Improved Performance in Conflict Detection Over 100x improvement with large tables and non-numeric PK
Internal Testing: Create table and seed 10 million rows of text data
create table x (t text not null); insert into x select md5(n::text) from (select generate_series(1, 10000000)) w(n);
Show timing with old and new algorithm\timing ontest=# select xdb60_style(); --Time: 3,629.105 ms (3 seconds)
test=# select xdb51_style(); --Time: 2,198,287.369 ms (36 minutes!)
© Copyright EnterpriseDB Corporation, 2015. All rights reserved. 14
• Active / Active Databases with fast and efficient log based replication and high performance conflict detection algorithms
• Pattern matching selection rules make it easier to configure publication tables in bulk
• Simple High Availability is now possible with integration of the 'Control Schema’
Summary – xDB 6.0 Enables New Replication Solutions!
© Copyright EnterpriseDB Corporation, 2015. All rights reserved. 15