Upload
others
View
49
Download
0
Embed Size (px)
Citation preview
PostgreSQL Backupsthe Modern Way
PGDay Russia 2016St Petersburg, Russia
Magnus Hagander [email protected]
Magnus HaganderRedpill Linpro
Infrastructure servicesPrincipal database consultant
PostgreSQLCore Team memberCommitterPostgreSQL Europe
So, backups...Do you make them?
BackupsAre not superseded by replicationOr cloudOr containers..
BackupsAre boringBut I'm glad you have them
BackupsWhen did you last restore?
PostgreSQL backupsOk, enough genericWhat about backups in PostgreSQL?
Seen this before?pg_dump options:
Fc = custom formatZ = compressionj = parallela = data only, s = schema onlyn = schema, t = table...
pg_dumpDon't use for backups
Has other good usecasesToo slow to restoreToo much overheadNo PITRExceptions, of course
Physical backupsBase backupsWith or without log archiveFast restoreFull cluster onlyPlatform specific
Base backups#!/bin/bashset e
psql U postgres q "SELECT pg_start_backup('foo')"
tar cfz /backup/$(date +%Y%m%d) /var/lib/pgsql/data
psql U postgres q "SELECT pg_stop_backup()"
Base backupsSo many ways to get that wrong
Spot one?
Base backupsThis used to be the only wayMany scripts around that does itMany of those are broken...
pg_basebackupBase backup over replication protocolSafeError handling and recoveryFor most cases
(we'll cover other options later)
pg_basebackup#!/bin/bashset e
pg_basebackup D /backup/$(date +%Y%m%d).tar Ft x
Needs replicationDefaults need to changeBut for now:
wal_level = hot_standbymax_wal_senders = 5
local replication postgres peer
Backup formatsplain
Safe copy of data directoryNot good with multiple tablespaces
tarDestination still a directoryEach tablespace gets one file
base.tar
Transaction logxlog required to restore backupFrom beginning of backup to endIn the log archive, right?
Including xlogAlways use -x or -X to include xlogMakes backup independently consistent
With or without log archiveMay back up xlog twice
Use even with log archive!
Including xlog-X fetch
Fetches xlog at end of backupCan fail if xlog rotated
-X streamReplicates xlog over secondary connectionFewer failure scenariosDoes not work with tar
Backup compressionpg_basebackup -Z
Compression happens in pg_basebackupTar format onlyCPU usageRemote server?
Transfer compressionSSL compression
Much harder these daysssh tunneling
ssh mydbserver c "pg_basebackup Ft D Z9" > backup.tgz
That's it!With that, you have backupsThat workAnd are (reasonably) safe
PITRPoint in time recoveryYou all want itA bit more setting up
archive_commandTo use PITR, we use log archivinglike this?
archive_command = 'test ! f /mnt/archivedir/%f && cp %p /mnt/archivedir/%f'
Don't do that!
pg_receivexlogRuns on archive serverUses streaming replicationGenerates log archive
pg_receivexlogMore granular recoverySafe against server restartsCan follow timeline switches on master
pg_receivexlogAlways use with replication slot
As of 9.4But we said modern..
Backups should block
pg_receivexlogpg_receivexlog D /log/archive h master S backup
Ensure it's restarted!
Backup retentionHow long to keep around?What granularity?...
Backup retentionRecovery needs:
Base backupAll xlog from start to endAll xlog from end to pitr
(that's why we use -x!)
Backup retentionfind is o�en enoughDelete logs older than X, base older than Y
Safe if -x was used!
#!/bin/bash
find /var/backups/basebackup type f mtime +30 print0 | xargs 0 r /bin/rm
find /var/backups/xlog type f mtime +7 print0 | xargs 0 r /bin/rm
Not enough?Handles the simple casesBut has limitationsParticularly in management
Other toolsBarmanpgBackRest
BarmanBackup schedulingLog archivingRetention managementMulti-serverRestore shortcuts
BarmanDeveloped by 2ndQuadrantPythonGPLv3Primarily ssh+rsync
1.6 just learned about pg_receivexlog!Does not use pg_basebackupNo (safe) concurrent backup support
pgBackRestBackup schedulingLog archivingRetention managementMulti-serverRestore shortcuts
pgBackRestDeveloped by CrunchyDataPerlMIT licensessh but not rsync
pgBackRestCustom protocolParallel backup sessionsFull/Differential/Incremental
Segment based
pgBackRestNo pg_receivexlog supportNo concurrent backup supportYet
Summary
Don't roll your own!
Don't roll your ownToo many pitfallsBoth base backups and archivingBackups are too important!
Don't roll your ownPrimary choice
Built-inIf it's enough
Secondary choicepgBackRestBarman
Tertiary choiceRestart from top of slide
Thank you!Magnus Hagander
[email protected] @magnushagander
http://www.hagander.net/talks/
This material is licensed