Embed Size (px)
DESCRIPTION
PostgreSQL Conference 2009 Japan Keynote Lecture http://www.postgresql.jp/events/pgcon09j/e/program_1#2a http://www.postgresql.jp/events/pgcon09j/j/program_1#2a
Citation preview
Streaming Replication
Masao FujiiNTT OSS Center
Copyright(c)2009 NTT, Inc. All Rights Reserved.
• Postgres’ built-in replication– Now under development in the community– Will be available in v8.5
• Why replication?– Fail Over– Load Balancing
What’s Streaming Replication?
Client
query
SlaveMaster
changes
query
• Streaming Replication (SR)– Capability to send changes to the slave, to keep it current
• Hot Standby (HS)– Capability to run queries on the slave
Streaming Replication vs Hot Standby
Client
query
Slave
changes
Master
query
Hot Standby
Streaming Replication
• Historical policy– Avoid putting replication into core Postgres– "one size fits all" replication solution
• Replication war!
History
pgpool
Cybercluster
Sequoia
Postgres-R
PGCluster-II
RepDB
PG on DRBDSlony-I
Bucardo
warm-standby
Londiste
PGClusterPostgresForest
Monmoth
syncreplicator
PyReplica
PL/Proxy
pgpool-II
DBmirror
twin
rubyrep
PG on Shared DiskGridSQL
• Growing frustration– Too complex to install and use for simple cases– vs. other dbms
• Proposal of built-in replication– by NTT OSSC @ PGCon 2008 Ottawa
• Core team statement– It is time to include a simple, reliable basic
replication feature in the core system– NOT replace the existing projects
Road to core
• One master and one or more slaves– Only master accepts write queries– One-way replication: master slaves
Master - Slaves
Master
write query
Client
changes
Slaves
WALRecovery
DatabaseWALWAL
write query
Master
Client
Slave
• WAL is shipped as data changes– Slave can be kept current by replaying WAL– WAL records in partially-filled WAL file can be shipped⇔Cannot in warm-standby
Log-shipping
H/W architecture and major postgres release level must be the same
• No migration required– Provides the same interface as the stand-alone
Postgres does⇔Tables must have a primary key for Slony-I⇔Some queries (e.g., sequences) would have
different values on different servers in pgpool
Master
Client Client
Slave
Stand-alone
Log-shipping
No need to change the existing application code for replication
Master Slave Master Slave
Per database cluster granularity
• Replicates all database objects– No need to specify which object is replicated– NOT replicate: server’s log file, statistics
information, ...etc⇔Per-table in Slony-I
Per database cluster Per table
Shared nothing Shared disk
Master Slave Master Slave
Shared nothing architecture
• WAL is shipped via network– No single point of failure– No special H/W like shared disk required
• Fail Over– Anytime, slave can be brought up– Automatic failover is not supported⇒Need to use clusterware like heartbeat for that
• Split– Anytime, master can become stand-alone
Management
Master Slave
Client
Master Slave
Client
Fail Over Split
Master
Slave
Client
Management
• Online-Resync– Anytime, new slave can be added to replication
without downtime
Master Slave
fsync write
✔
✔
✔
fsync
async ✔
apply
✔
write ✔
fsync ✔
apply ✔ ✔
modes
fast
durable
Synchronization mode
• async– Asynchronous replication– Possible risk of data loss– No master overhead
• write, fsync– Semi-synchronous– No data loss
• apply– Synchronous– READ COMMITTED on slave is guaranteed– Large master overhead
Synchronization mode
Needs durability
SET mode TO fsync
Needs speed
SET mode TO async
Synchronization mode
• The mode can be specified per transaction– Mission critical trasanction like banking would
requires write, fsync or apply mode– async would be sufficient for transaction like
web crawling
• Easy to install and use– Just install postgres– Replication connection can be treat the same as normal
backend connection: pg_hba.conf, keepalive, SSL, ...
• Highly active community– Bug fix soon– Active development and collaboration– Many users
Built-in
• Needs your help– Comment about interface, design and architecture– Test and review the code– Correct my English on the doc
• Project site– http://wiki.postgresql.org/wiki/Streaming_Replication
• Where to comment and report bug ...– [email protected]– [email protected]
Road to v8.5
