46
PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander [email protected]

PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander [email protected]. Magnus

  • Upload
    others

  • View
    49

  • Download
    0

Embed Size (px)

Citation preview

Page 1: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

PostgreSQL Backupsthe Modern Way

PGDay Russia 2016St Petersburg, Russia

Magnus Hagander [email protected]

Page 2: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Magnus HaganderRedpill Linpro

Infrastructure servicesPrincipal database consultant

PostgreSQLCore Team memberCommitterPostgreSQL Europe

Page 3: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

So, backups...Do you make them?

Page 4: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

BackupsAre not superseded by replicationOr cloudOr containers..

Page 5: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

BackupsAre boringBut I'm glad you have them

Page 6: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

BackupsWhen did you last restore?

Page 7: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

PostgreSQL backupsOk, enough genericWhat about backups in PostgreSQL?

Page 8: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Seen this before?pg_dump options:

­Fc    = custom format­Z     = compression­j     = parallel­a = data only, ­s = schema only­n = schema, ­t = table...    

Page 9: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

pg_dumpDon't use for backups

Has other good usecasesToo slow to restoreToo much overheadNo PITRExceptions, of course

Page 10: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Physical backupsBase backupsWith or without log archiveFast restoreFull cluster onlyPlatform specific

Page 11: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Base backups#!/bin/bashset ­e

psql ­U postgres ­q "SELECT pg_start_backup('foo')"

tar cfz /backup/$(date +%Y%m%d) /var/lib/pgsql/data

psql ­U postgres ­q "SELECT pg_stop_backup()"    

Page 12: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Base backupsSo many ways to get that wrong

Spot one?

Page 13: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Base backupsThis used to be the only wayMany scripts around that does itMany of those are broken...

Page 14: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

pg_basebackupBase backup over replication protocolSafeError handling and recoveryFor most cases

(we'll cover other options later)

Page 15: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

pg_basebackup#!/bin/bashset ­e

pg_basebackup ­D /backup/$(date +%Y%m%d).tar ­Ft ­x    

Page 16: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Needs replicationDefaults need to changeBut for now:

wal_level = hot_standbymax_wal_senders = 5    

local  replication  postgres  peer    

Page 17: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Backup formatsplain

Safe copy of data directoryNot good with multiple tablespaces

tarDestination still a directoryEach tablespace gets one file

base.tar

Page 18: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Transaction logxlog required to restore backupFrom beginning of backup to endIn the log archive, right?

Page 19: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Including xlogAlways use -x or -X to include xlogMakes backup independently consistent

With or without log archiveMay back up xlog twice

Use even with log archive!

Page 20: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Including xlog-X fetch

Fetches xlog at end of backupCan fail if xlog rotated

-X streamReplicates xlog over secondary connectionFewer failure scenariosDoes not work with tar

Page 21: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Backup compressionpg_basebackup -Z

Compression happens in pg_basebackupTar format onlyCPU usageRemote server?

Page 22: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Transfer compressionSSL compression

Much harder these daysssh tunneling

ssh mydbserver ­c "pg_basebackup ­Ft ­D­ ­Z9" > backup.tgz    

Page 23: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

That's it!With that, you have backupsThat workAnd are (reasonably) safe

Page 24: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

PITRPoint in time recoveryYou all want itA bit more setting up

Page 25: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

archive_commandTo use PITR, we use log archivinglike this?

archive_command =  'test ! ­f /mnt/archivedir/%f && cp %p /mnt/archivedir/%f'    

Page 26: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Don't do that!

Page 27: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

pg_receivexlogRuns on archive serverUses streaming replicationGenerates log archive

Page 28: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

pg_receivexlogMore granular recoverySafe against server restartsCan follow timeline switches on master

Page 29: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

pg_receivexlogAlways use with replication slot

As of 9.4But we said modern..

Backups should block

Page 30: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

pg_receivexlogpg_receivexlog ­D /log/archive ­h master ­S backup    

Ensure it's restarted!

Page 31: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Backup retentionHow long to keep around?What granularity?...

Page 32: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Backup retentionRecovery needs:

Base backupAll xlog from start to endAll xlog from end to pitr

(that's why we use -x!)

Page 33: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Backup retentionfind is o�en enoughDelete logs older than X, base older than Y

Safe if -x was used!

#!/bin/bash

find /var/backups/basebackup ­type f ­mtime +30 ­print0 |   xargs ­0 ­r /bin/rm

find /var/backups/xlog ­type f ­mtime +7 ­print0 |   xargs ­0 ­r /bin/rm    

Page 34: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Not enough?Handles the simple casesBut has limitationsParticularly in management

Page 35: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Other toolsBarmanpgBackRest

Page 36: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

BarmanBackup schedulingLog archivingRetention managementMulti-serverRestore shortcuts

Page 37: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

BarmanDeveloped by 2ndQuadrantPythonGPLv3Primarily ssh+rsync

1.6 just learned about pg_receivexlog!Does not use pg_basebackupNo (safe) concurrent backup support

Page 38: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

pgBackRestBackup schedulingLog archivingRetention managementMulti-serverRestore shortcuts

Page 39: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

pgBackRestDeveloped by CrunchyDataPerlMIT licensessh but not rsync

Page 40: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

pgBackRestCustom protocolParallel backup sessionsFull/Differential/Incremental

Segment based

Page 41: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

pgBackRestNo pg_receivexlog supportNo concurrent backup supportYet

Page 42: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Summary

Page 43: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Don't roll your own!

Page 44: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Don't roll your ownToo many pitfallsBoth base backups and archivingBackups are too important!

Page 45: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Don't roll your ownPrimary choice

Built-inIf it's enough

Secondary choicepgBackRestBarman

Tertiary choiceRestart from top of slide

Page 46: PostgreSQL Backups the Modern Way Backups the... · 2019-10-10 · PostgreSQL Backups the Modern Way PGDay Russia 2016 St Petersburg, Russia Magnus Hagander magnus@hagander.net. Magnus

Thank you!Magnus Hagander

[email protected] @magnushagander

http://www.hagander.net/talks/

This material is licensed