Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
© 2021, Amazon Web Services, Inc. or its Affiliates.
Chirag Dave
Sr. Database Specialist Solutions Architect
Move to Managed: Migrating Open-source
Databases from Self-managed to Amazon RDS or
Amazon Aurora
Peter Celentano
Database Specialist Solutions Architect
© 2021, Amazon Web Services, Inc. or its Affiliates.
Agenda
• Why move to managed database services
• An overview of AWS managed database services
• Available options to migrate self-managed database to
RDS/Aurora
• Demo 1: MySQL migration
• Demo 2: PostgreSQL migration
• Migration best practices
• Q&A
© 2021, Amazon Web Services, Inc. or its Affiliates.
Hardware & software installation, configuration, patching, backups
Performance and high availability issues
Capacity planning and scaling clusters for compute and storage
Security and compliance
Self-managing databases is time consuming, complex,
and expensive
© 2021, Amazon Web Services, Inc. or its Affiliates.
Amazon Relational Database Service (Amazon RDS) M A N A G E D R E L A T I O N A L D A T A B A S E S E R V I C E W I T H A C H O I C E O F P O P U L A R D A T A B A S E E N G I N E S
Easy to administerEasily deploy and maintain hardware, OS, and DB software; built-in monitoring
Performant & scalableScale compute and storage with a few clicks; minimal downtime for your application
Available & durableAutomatic multi-AZ data replication; automated backup, snapshots, and failover
Secure & CompliantData encryption at rest and in transit; assists customers achieve their compliance goals through tooling
Microsoft SQL Server
Oracle
© 2021, Amazon Web Services, Inc. or its Affiliates.
Amazon AuroraM Y S Q L - A N D P O S T G R E S Q L - C O M P A T I B L E R E L A T I O N A L D A T A B A S E W I T H P E R F O R M A N C E A N D A V A I L A B I L I T Y O F C O M M E R C I A L - G R A D E D A T A B A S E S A T 1 / 1 0 T H T H E C O S T
Performance
and scalability
Availability
and durabilityHighly secure Fully managed
Performance of commercial-
grade databases; scale-out up
to 15 read replicas
Fault-tolerant, self-healing
storage; six copies of data
across three Availability
Zones; continuous backup to
Amazon S3
Network isolation,
encryption at
rest/transit, compliance
and assurance programs
Managed by RDS:
No server provisioning,
software patching, setup,
configuration, or backups
© 2021, Amazon Web Services, Inc. or its Affiliates.
Amazon RDS & Amazon Aurora - fully managedS P E N D T I M E I N N O V A T I N G & B U I L D I N G N E W A P P S , N O T M A N A G I N G I N F R A S T R U C T U R E
Schema design
Query construction
Query optimization
Automatic fail-over
Backup & recovery
Isolation & security
Industry compliance
Push-button scaling
Automated patching &
upgrades
Advanced monitoring
Routine maintenance
You AWS
© 2021, Amazon Web Services, Inc. or its Affiliates.
Choosing your relational database deployment in AWS
Amazon RDS
Self-managed
© 2021, Amazon Web Services, Inc. or its Affiliates.
Move to managed relational databasesM I G R A T E O N - P R E M I S E S O R C L O U D - H O S T E D R E L A T I O N A L D A T A B A S E S T O M A N A G E D S E R V I C E S
Reduce DB administrative burden
No need to rearchitect existing applications
Get better performance, availability, scalability, and security
Amazon AuroraMySQL, PostgreSQL
Amazon RDSMySQL, PostgreSQL, MariaDB,
Oracle, SQL Server
© 2021, Amazon Web Services, Inc. or its Affiliates.
Migration options for MySQL
© 2021, Amazon Web Services, Inc. or its Affiliates.
MySQL migration options at a glanceC O N S I D E R A L L O P T I O N S I N C O N T E X T O F Y O U R R E Q U I R E M E N T S A N D L I M I T A T I O N S
Physical Restore
from Backups
Logical Export
and Import
AWS Database
Migration Service
Physical migration
+ logical replication
Using native or
ecosystem tools
+ logical replication
Using AWS DMS and
SCT for conversion and
data migration
1 2 3
Homogenous OnlyHeterogeneous
Homogenous
© 2021, Amazon Web Services, Inc. or its Affiliates.
Physical restore from backupsT Y P I C A L L Y T H E P R E F E R R E D O P T I O N C O M I N G F R O M M Y S Q L
• Simple option if you already use
Percona XtraBackup for backups
• Upload backup files to Amazon S3
• Physical restore is typically faster
than logical export and import
• Supports MySQL 5.6, 5.7 and 8.0 as
source
• Use native binlog replication for
capturing changes and minimizing
cutover downtime
Corporate
data center
AWS Cloud
Amazon S3
Amazon Aurora
MySQL
MySQL
Some limitations apply: encrypted backups not supported; stored procedures, functions and timezone setting not imported
automatically; only works with default file name for innodb_data_file_path parameter; some definer policies not usable
1.Percona Xtrabackup
3. Binlog Replication
2. Aurora MySQL
Restore
© 2021, Amazon Web Services, Inc. or its Affiliates.
Migrate using logical export and importT Y P I C A L L Y A R E L I A B L E O P T I O N I F D A T A S E T S A R E S M A L L , O R I M P O R T S A R E P A R A L L E L I Z E D
• Typically only used for homogenous migrations
• Using native tools or third party tools
- Native tools: mysqldump tool suitable only for small data sets
- Third party tools: mydumper/myloader suitable for parallelized export and import
operation
• Initial load depends on data set, degree of parallelization,
DB instance size, storage characteristics
• Use native logical replication for capturing changes
and minimizing downtime
© 2021, Amazon Web Services, Inc. or its Affiliates.
Migration options summary for MySQL
Options
Near Zero
Downtime
Required?
Supported
VersionsMigration Type
Physical restore from
backups using Percona
XtraBackup Can minimize
downtime with
native logical
replication5.6, 5.7, 8.0 (RDS
MySQL only)
Homogenous
Logical replication using
mysqldumpHomogenous
Logical replication using
mydumper/myloaderHomogenous
AWS DMS with CDC YesHomogeneous,
Heterogeneous
© 2021, Amazon Web Services, Inc. or its Affiliates.
Demo 1:
Creating an RDS MySQL DB Instance from
S3-based backups
© 2021, Amazon Web Services, Inc. or its Affiliates.
Migration options for PostgreSQL
© 2021, Amazon Web Services, Inc. or its Affiliates.
PostgreSQL Migration options at a glanceC O N S I D E R A L L O P T I O N S I N C O N T E X T O F Y O U R R E Q U I R E M E N T S A N D L I M I T A T I O N S
Logical Dump Logical
Replication
AWS Database
Migration Service
pg_dump/pg_restore Native logical
replication/pglogical
Using AWS DMS and
SCT for conversion and
data migration
1 2 3
Homogenous Only
Heterogeneous
Homogenous
© 2021, Amazon Web Services, Inc. or its Affiliates.
Migration to RDS PostgreSQL – Native tools
pg_dump/pg_restore
• Simple option
• Does not support CDC
• Supports all major versions
• May not be suitable for 24/7
workloadsSelf-
managed
pg_dump/
pg_restore
Amazon RDS
© 2021, Amazon Web Services, Inc. or its Affiliates.
When to choose pg_dump?
We recommend that you use nat ive PostgreSQL database migrat ion too ls
under the fo l lowing cond i t ions :
• You have a homogenous migrat ion , where you are migrat ing f rom a
database w i th the same database eng ine as the target database .
• You are migrat ing an ent i re database .
• Smal l database where downt ime dur ing backup and restore i s accepted .
© 2021, Amazon Web Services, Inc. or its Affiliates.
On-Premise/EC2 to Amazon RDS replicationI N T R O D U C I N G I N B U I L T L O G I C A L R E P L I C A T I O N A N D P G L O G I C A L E X T E N S I O N
• PostgreSQL 10 (onwards)
supports inbuilt logical
replication using
publication/subscription
• Pglogical Extension available in
Amazon RDS for PostgreSQL >=
9.6.10
9.6.10
Amazon RDS>=10
10.x +
© 2021, Amazon Web Services, Inc. or its Affiliates.
Migration Options Summary for PostgreSQL
Options
Near Zero
Downtime
required
Supported Versions Migration Type
pgdump/pg_restore No
All PostgreSQL
versions Homogenous
AWS DMS with CDC Yes 9.4 and higherHomogeneous
Heterogeneous
Pglogical extension Yes 9.4 and higherHomogenous
PostgreSQL Native
Pub/Sub replicationYes 10.x and higher
Homogenous
© 2021, Amazon Web Services, Inc. or its Affiliates.
AWS DMS: Migration Option for both MySQL
& PostgreSQL
© 2021, Amazon Web Services, Inc. or its Affiliates.
Migrate using AWS DMS and SCT
AWS Database Migration Service (DMS)easily and securely migrates and/or replicate
your databases and data warehouses to AWS
AWS Schema Conversion Tool (SCT)converts your commercial database and data
warehouse schemas to open-source engines or
AWS-native services, such as Amazon Aurora and
Redshift
© 2021, Amazon Web Services, Inc. or its Affiliates.
How does AWS DMS work? F I R S T , C O N V E R T S C H E M A S A N D P E R F O R M I N I T I A L D A T A L O A D S
Source DB
AWS SCT
Native Tool
Target DB
AWS DMS
Database
Source DBTarget DB
Step 1:
Step 2:
© 2021, Amazon Web Services, Inc. or its Affiliates.
How does DMS work? - contN E X T , C A P T U R E C H A N G E S ( K E E P A P P S R U N N I N G D U R I N G M I G R A T I O N )
Customer
premises
Application users
AWS
Internet
VPN
Start a replication instance
Connect to source and target databases
Select tables, schemas, or databases
Let AWS DMS create tables, load data, and keep them in sync
Switch applications over to the target at your convenience
AWS
DMS
© 2021, Amazon Web Services, Inc. or its Affiliates.
Demo 2:
Migrating from Self Managed PostgreSQL to
Amazon Aurora using Logical Replication
© 2021, Amazon Web Services, Inc. or its Affiliates.
Migration Best Practices
© 2021, Amazon Web Services, Inc. or its Affiliates.
Migration best practices
• Do not use Mul t i -AZ for in i t ia l load
• Ensure a l l the parameters and set t ings you need for the source and
target are set beforehand
• Test per formance by restor ing the snapshot of the migrated instance
• Use nat ive too ls where i t makes sense
• For AWS DMS: The s i ze and instance c lass for your DMS instance i s a
common i ssue for per formance
© 2021, Amazon Web Services, Inc. or its Affiliates.
How to
get started
• Learn more about AWS database services
• aws.amazon.com/products/databases/
• Engage with your AWS account team
early. They are here to help!
• Participate in workshops, Immersion
Days, or Data Lab
• Get support from ProServe or
AWS Partner for database migration
© 2021, Amazon Web Services, Inc. or its Affiliates.
Chirag Dave: [email protected]
Peter Celentano: [email protected]