Upload
amazon-web-services
View
621
Download
4
Embed Size (px)
Citation preview
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Brian Tracy, Solutions Architect
June 21, 2016
Amazon RDS Including Amazon Aurora
What is Amazon RDS?
• Managed relational database in the cloud• 6 familiar engines and multiple versions to chose from
• Managed for you:• Amazon RDS handles routine database tasks such as
provisioning, patching, backup, recovery, failure detection, and repair.
Why did AWS build Amazon RDS?
• There’s a lot of repetitive labor that must be done, but doesn’t directly add value
• Backups and restores• Software installs and
patching• Managing hardware
• Achieving many important capabilities requires lots of spend, lots of engineering, or both
• Scaling• High availability• Migration
Managing relational databases is hard
With a traditional DB you:• Acquire hardware (purchase, rack and stack)• Load OS• Load clustering software• Load database software• Create a database• Optimize your query logic• Design and implement a backup strategy• Perform patching (OS, clustering software,
database)• Perform software and hardware upgrades• Deal with hardware failures
How does RDS compare to traditional DB hosting?
With RDS you:• Create a database (selecting
options for backup and maintenance)
• Optimize your query logic and execution
Amazon RDS is simple and fast to deploy
• Get a production-ready database instance in minutes
• No need to acquire servers, rack and stack, install OS and database software
Amazon RDS simplifies administration
• Automated backups• Easy point-in-time
recovery• Enhanced monitoring• Automated patch
management• Automated replication
with Multi-AZ
A simple application architecture
RDS database instance
Application, in an Amazon EC2 instance
Elastic Load Balancing load balancer instance
DB snapshots in
Amazon S3
Choose Multi-AZ for greater availability, durability
• An Availability Zone is a physically distinct, independent infrastructure
• With Multi-AZ operation, your database is synchronously replicated to another AZ in the same AWS Region
• Failover occurs automatically in response to the most important failure scenarios
• Planned maintenance is applied first to backup
A resilient, durable, still simple application architecture
RDS database instances:master and Multi-AZ standby
Application, in Amazon EC2 instances
Elastic Load Balancing load balancer instance
DB snapshots in
Amazon S3
Amazon RDS offers fast, predictable storage
General Purpose (SSD) for most workloads
Provisioned IOPS (SSD) for OLTP workloads up to 30,000 IOPS
Magnetic for small workloads with infrequent access
Amazon RDS Read Replicas offer scale-out
• Offload read traffic to automatically maintained Read Replicas
• Load-share traffic across multiple Read Replicas
• Easy to set up
Amazon RDS provides levels of security difficult to achieve on-premises
• AWS has achieved major compliances • Amazon RDS gives each database instance IP firewall
protection • Amazon VPC lets you isolate and control network
configuration and connect securely to your IT infrastructure• AWS Identity and Access Management provides resource-
level permission controls• Amazon RDS offers encryption at rest and SSL protection for
data in transit
13
Amazon RDS is easy to monitor with Amazon CloudWatch
CloudWatch RDS Metrics CPU utilization Storage Memory Swap usage DB connections I/O (read and write) Latency (read and write) Throughput (read and write) Replica lag Many more
CloudWatch Alarms Similar to on-premises custom
monitoring tools
Amazon RDS is cost-effective
Monthly bill = GB+
Assumes DB instance accessed only from Amazon EC2Further details at http://aws.amazon.com/rds/pricing/
= 720 hrs * $0.35 + 100 GB * $0.115
= $263.50
db.m4.xlarge; MySQL; N. Virginia; Single-AZ;
On-Demand
100 GB General Purpose
(SSD)4 vCPUs;
16 GiB RAM
• Pay only for what you use; no minimum chargeExample:
Save money with Amazon RDS Reserved Instances
• Pay a low up-front fee to get a lower hourly price on database instances for a 1- or 3-year term
• Your lower Reserved Instance price applies to any running instance matching the description you specified at purchase time
Start saving here
Cumulative spend
Month 1 Month 2 Month 3 Month 4 Month 5 Month 6 Month 7 Month 8 Month 9 Month 10 Month 11 Month 12On-Demand 263.50 527.00 790.50 1,054.00 1,317.50 1,581.00 1,844.50 2,108.00 2,371.50 2,635.00 2,898.50 3,162.00 1-Yr RI 1,777.50 1,789.00 1,800.50 1,812.00 1,823.50 1,835.00 1,846.50 1,858.00 1,869.50 1,881.00 1,892.50 1,904.00
How Amazon RDS backups work
Automated backups• Restore your database to a
point in time• Enabled by default• Choose a retention period, up
to 35 days
Manual snapshots• Build a new database instance
from a snapshot when needed• Initiated by you• Persist until you delete them• Stored in Amazon S3
Choose cross-region snapshot copy for even greater durability, ease of migration
• Copy a database snapshot to a different AWS Region
• Warm standby for disaster recovery
• Or use it as a base for migration to a different region
Easily migrate to Amazon RDS
• AWS Schema Conversion Tool• Move schema to new DB• Convert schema to new DB platform
• AWS Database Migration Service• Homogenous (A to A)• Heterogeneous (A to B)
More info: http://aws.amazon.com/dms/
Why should I use RDS?
Let AWS handle these
So you can focus on these
MigrationBackup and recoveryConfigurationPatchingSoftware upgradesStorage upgradesServer upgradesHardware issues
Database schemaQuery designQuery optimization
Amazon Aurora
MySQL-compatible relational database
Performance and availability of commercial databases
Simplicity and cost-effectiveness of open source databases
Delivered as a managed service
What is Amazon Aurora?
Re-imagined for the cloud
Architected for the cloud—that is, we moved the logging and storage layer into a multitenant, scale-out database-optimized storage service
Leverages existing AWS services: Amazon EC2, Amazon VPC, Amazon DynamoDB, Amazon SWF, and Amazon S3
Maintains compatibility with MySQL—customers can migrate their MySQL applications as-is and use all MySQL tools
Control PlaneData Plane
Amazon DynamoD
B
Amazon SWF
Amazon Route 53
Logging + Storage
SQL
Transactions
Caching
Amazon S3
1
2
3
WRITE PERFORMANCE READ PERFORMANCE
MySQL SysBench results
R3.8XL: 32 cores / 244 GB RAM
5X faster than RDS MySQL 5.6 and 5.7
Five times higher throughput than stock MySQL based on industry standard benchmarks.
0
25,000
50,000
75,000
100,000
125,000
150,000
0
100,000
200,000
300,000
400,000
500,000
600,000
700,000
Aurora MySQL 5.6 MySQL 5.7
WRITE PERFORMANCE READ PERFORMANCE
Scaling with instance sizes
Aurora scales with instance size for both read and write.
Aurora MySQL 5.6 MySQL 5.7
Real-life data—gaming workloadAurora vs. RDS MySQL—r3.4XL, MAZ
Before : 15ms
After : 5.5ms
Aurora Migration
Aurora 3X faster on r3.4xlarge
Do fewer I/Os
Minimize network packets
Cache prior results
Offload the database engine
DO LESS WORK
Process asynchronously
Reduce latency path
Use lock-free data structures
Batch operations together
BE MORE EFFICIENT
How did we achieve this?
DATABASES ARE ALL ABOUT I/O
NETWORK-ATTACHED STORAGE IS ALL ABOUT PACKETS/SECOND
HIGH-THROUGHPUT PROCESSING DOES NOT ALLOW CONTEXT SWITCHES
I/O traffic in MySQL
BINLOG DATA DOUBLE-WRITELOG FRM FILES
T Y P E O F W R I T E
MYSQL WITH REPLICA
EBS mirrorEBS mirror
AZ 1 AZ 2
Amazon S3
EBSAmazon Elastic
Block Store (EBS)
PrimaryInstance
ReplicaInstance
1
2
3
4
5
Issue write to EBS—EBS issues to mirror, ack when both doneStage write to standby instance through DRBDIssue write to EBS on standby instance
I/O FLOW
Steps 1, 3, 5 are sequential and synchronousThis amplifies both latency and jitterMany types of writes for each user operationHave to write data blocks twice to avoid torn writes
OBSERVATIONS
780 K transactions7,388 K I/Os per million txns (excludes mirroring, standby)Average 7.4 I/Os per transaction
PERFORMANCE
30 minute SysBench write-only workload, 100 GB dataset, RDS Multi-AZ, 30 K PIOPS
IO traffic in Aurora
AZ 1 AZ 3
PrimaryInstance
Amazon S3
AZ 2
ReplicaInstance
AMAZON AURORA
ASYNC4/6 QUORUM
DISTRIBUTED WRITES
BINLOG DATA DOUBLE-WRITELOG FRM FILES
T Y P E O F W R I T E
IO FLOW
Only write redo log records; all steps asynchronousNo data block writes (checkpoint, cache replacement)6X more log writes, but 9X less network trafficTolerant of network and storage outlier latency
OBSERVATIONS
27,378 K transactions 35X MORE950 K I/Os per 1M txns (6X amplification) 7.7X LESS
PERFORMANCE
Boxcar redo log records—fully ordered by LSNShuffle to appropriate segments—partially orderedBoxcar to storage nodes and issue writesReplica
Instance
“In RDS MySQL, we saw replica lag spike to almost 12 minutes, which is almost absurd from an application’s perspective. The maximum read replica lag across 4 replicas never exceeded beyond 20 ms.”
Real-life data—read replica latency
I/O traffic in Aurora Replicas
PAGE CACHEUPDATE
Aurora Master
30% Read
70% Write
Aurora Replica
100% New Reads
Shared Multi-AZ Storage
MySQL Master
30% Read
70% Write
MySQL Replica
30% New Reads
70% Write
SINGLE-THREADEDBINLOG APPLY
Data Volume Data Volume
Logical: Ship SQL statements to replica
Write workload similar on both instances
Independent storage
Can result in data drift between master and replica
Physical: ship redo from master to replica
Replica shares storage; no writes performed
Cached pages have redo applied
Advance read view when all commits seen
MYSQL READ SCALING AMAZON AURORA READ SCALING
“Performance only matters if your database is up”
Storage durability
Storage volume automatically grows up to 64 TB
Quorum system for read/write; latency tolerant
Peer to peer gossip replication to fill in holes
Continuous backup to S3 (built for 11 9s durability)
Continuous monitoring of nodes and disks for repair
10 GB segments as unit of repair or hotspot rebalance
Quorum membership changes do not stall writes
AZ 1 AZ 2 AZ 3
Amazon S3
Six copies across three Availability Zones
4 out 6 write quorum; 3 out of 6 read quorum
Peer-to-peer replication for repairs
Volume striped across hundreds of storage nodes
SQL
Transaction
AZ 1 AZ 2 AZ 3
Caching
SQL
Transaction
AZ 1 AZ 2 AZ 3
Caching
Read and write availability Read availability
Fault-tolerant storage
Survivable caches
We moved the cache out of the database process
Cache remains warm in the event of database restart
Lets you resume fully loaded operations much faster
Instant crash recovery + survivable cache = quick and easy recovery from DB failures
SQLTransactions
Caching
SQL
Transactions
Caching
SQLTransactions
Caching
Caching process is outside the DB process and remains warm across a database restart
Aurora Replicas are failover targets
Aurora cluster contains primary node and up to 15 secondary nodes
Failing database nodes are automatically detected and replaced
Failing database processes are automatically detected and recycled
Secondary nodes automatically promoted on persistent outage, no single point of failure
Customer application can scale out read traffic across secondary nodes
AZ 1 AZ 3AZ 2
PrimaryNodePrimaryNodePrimaryNode
PrimaryNodePrimaryNode
SecondaryNode
PrimaryNodePrimaryNode
SecondaryNode
Customer specifiable failover order
Read balancing across Aurora Replicas
ALTER SYSTEM CRASH [{INSTANCE | DISPATCHER | NODE}]
ALTER SYSTEM SIMULATE percent_failure DISK failure_type IN
[DISK index | NODE index] FOR INTERVAL interval
ALTER SYSTEM SIMULATE percent_failure NETWORK failure_type
[TO {ALL | read_replica | availability_zone}] FOR INTERVAL interval
Simulate failures using SQL
To cause the failure of a component at the database node:
To simulate the failure of disks:
To simulate the failure of networking:
Compatible with the MySQL ecosystem
Well-established MySQL ecosystem
Business Intelligence Data Integration Query and Monitoring SI and Consulting
Source: Amazon
“We ran our compatibility test suites against Amazon Aurora and everything just worked." —Dan Jewett, Vice President of Product Management at Tableau
Many third-party monitoring tools
Migrate to Amazon Aurora easily
Why Aurora?
• Architected for 99.99% availability• Enterprise performance (5x) at 1/10 the cost• Compatible with MySQL 5.6• Automatically grows storage as needed, up to 64 TB• Easy migration from MySQL• Up to 15 Aurora Replicas in a region• Cross-region replication• Encryption in-transit and at rest• Continuous backup to S3 (11 9’s data durability)• Fully managed
Recent feature releases for Amazon RDS
May 18, 2016 Amazon Aurora now supports sharing database snapshots across accounts
May 6, 2016 Deploy Siebel CRM applications on Amazon RDS for Oracle
May 4, 2016 RDS Enhanced Monitoring is now available in South America (Sao Paulo) and China (Beijing)
April 27, 2016 MariaDB audit plug-in now available for RDS MySQL and MariaDB
April 26, 2016 Amazon RDS MySQL now supports point-and-click upgrade from MySQL 5.6 to 5.7
April 22, 2016 Enhanced Monitoring is now available for Amazon RDS for SQL Server
April 8, 2016 Amazon RDS for PostgreSQL now supports version 9.5 with minor version 9.5.2, and minor versions 9.4.7 and 9.3.12
April 1, 2016 Cluster view for Amazon Aurora in RDS console
April 1, 2016 Amazon RDS now supports January PSU patches, improved custom Oracle directories and read privileges support
Detailed listing available here: http://aws.amazon.com/rds/whats-new/
Try Amazon RDS for free
• For your first year, at no charge…• 750 free instance-hours allow you to run a micro database
instance continuously• 20 GB of database instance storage• 20 GB for automated backups• 10,000 I/Os
• Learn more about the AWS free tier:http://aws.amazon.com/free/
Thank you!