Deep dive on Amazon Aurora - Percona – The …...Aurora MySQL has 5x the throughput of MySQL 0 100,000 200,000 300,000 400,000 500,000 600,000 700,000 800,000 MySQL 5.6 MySQL 5.7

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Deep dive on Amazon AuroraPercona Live Amsterdam 2019

Yoav EilatSenior Product ManagerAmazon Aurora and Amazon RDSAWS


Note: some slides were removed from this PDF since we didn’t get to them during the event.


Amazon Aurora…Enterprise database at open source price

Delivered as a managed service

Speed and availability of high-end commercial databases

Simplicity and cost-effectiveness of open source databases

Drop-in compatibility with MySQL and PostgreSQL

Simple pay as you go pricing

Amazon Aurora


Aurora scale-out, distributed architecture

Purpose-built log-structured distributed storage system designed for databases

Storage volume is striped across hundreds of storage nodes distributed over 3 different availability zones

Six copies of data, two copies in each availability zone to protect against AZ+1 failures

Data is written in 10GB “protection groups”, growing automatically up to 64TB

Shared storage volume

Storage nodes with SSDs

AvailabilityZone 1

AvailabilityZone 2

AvailabilityZone 3

SQL

Transactions

Caching

SQL

Transactions

Caching

SQL

Transactions

Caching


Six-way replicated storageAvailability

Zone 1Availability

Zone 2Availability

Zone 3

SQL

Transactions

Caching

AvailabilityZone 1

AvailabilityZone 2

AvailabilityZone 3

SQL

Transactions

Caching


Write quorum in actionApplication

1 2 3 4 5 6


I/O flow in Aurora storage node1. Receive log records and add

to in-memory queue

2. Persist records in hot log and ACK

3. Organize records and identify gaps in log

4. Gossip with peers to fill in holes

5. Coalesce log records into new page versions

6. Periodically stage log and new page versions to Amazon S3

7. Periodically garbage collect old versions

8. Periodically validate CRC codes on blocks

Notes:All steps are asynchronousOnly steps 1 and 2 are in foreground latency path

Log records

Database Instance

Incoming queue

S3 BACKUP

1

2

3

4

5

6

7

8

Update queue

ACK

HotLog

Datapages

Continuous backup

GC

ScrubCoalesce

Sortgroup

Peer to peer gossipPeerStorageNodes


Write and read throughputAurora MySQL has 5x the throughput of MySQL

0

100,000

200,000

300,000

400,000

500,000

600,000

700,000

800,000

MySQL 5.6 MySQL 5.7 MySQL 8.0Aurora 5.6 Aurora 5.7

Write throughput Read throughput

0

50,000

100,000

150,000

200,000

250,000

MySQL 5.6 MySQL 5.7 MySQL 8.0Aurora 5.6 Aurora 5.7


Aurora read replicas

Master Readreplica

Readreplica

Readreplica

Reader end-point #1 Reader end-point #2

Shared distributed storage volume


MYSQL with replica Amazon Aurora

MySQL vs. Aurora I/O profile

Aurora IO profile for 30 min Sysbench run

27MM transactions

0.95 I/Os per transaction

35X More

7.7X Less

MySQL I/O profile for 30 min Sysbench run

0.78MM transactions

7.4 I/Os per transaction

1

2

3

4

5

ASYNC 4/6 QUORUM

Distributed writes

Type of write


Aurora read replicas are dedicated to reads

Physical using delta changes

No writes on replica

Shared storage

MySQL Master

30% Read

70% Write

MySQL Replica

30% New Reads

70% Write

Single-threadedBinlog apply

Data volume Data volume

MYSQL read scaling

Page cache update

Aurora Master

30% Read

70% Write

Aurora Replica

100% New Reads

Shared Multi-AZ storage

Amazon Aurora read scaling

Logical using delta changes

Same write workload

Independent storage


Aurora read scaling options

15 promotable read replicas per cluster

Auto-scaling to automatically add & remove replicas

Physical replication across regions (Aurora Global Database)

Logical (e.g. binlog) replication to any database

Asynchronous replication

Read onlyRead/write


Aurora Global DatabaseFaster disaster recovery and enhanced data locality

Oregon

M R R

StorageR R

Storage

OhioR R

Storage

R R

Storage

Northern Virginia

Ireland

(secondary region)

(secondary region)

(secondary region)

(primary region)

Inbo

und

repl

icat

ion

Inbo

und

repl

icat

ion

Inbo

und

repl

icat

ion

Outbound replication


Global replication performanceLogical vs. physical MySQL replication

Logical replication with MTS Physical replication

0

100

200

300

400

500

600

0

50,000

100,000

150,000

200,000

250,000

seco

nds

QPS

QPS

Lag

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

4.50

5.00

0

50,000

100,000

150,000

200,000

250,000

seco

nds

QPS

QPS

Lag

SysBench OLTP (write-only) stepped every 600 seconds on R4.16xlarge


Every node in the cluster is a writer

node that can serve both read and

write requests


Instances can fail and recover

independently

No additional cost for the feature

Introducing Amazon Aurora Multi-Master

Master 1

Master 2

Application1 Application 2



Using Aurora Multi-Master with your application

Allocate applications to writers

Implement writer node health checks in

applications

Re-distribute connections from failed

writers to healthy writers

Client side drivers used with Aurora

single master will also work with Multi-

Master


Multi-Master: example of transactions with no conflict

Transactions T1 and T2 from BLUE and PINK masters update different tables Table 1 and Table 2

No logical or physical conflicts because data from Table 1 and Table 2 are stored on different pages

No conflict resolution required

1 1 1 1 11 2 2 2 2 22

PAGE 1 PAGE 2

MASTER 1 MASTER 2

TABLE 1 TABLE 2


Multi-Master: example of physical conflict resolutionArbitration at storage node

1 1 1 2 21

PAGE 1 PAGE 2

TABLE 1 TABLE 2

Transactions T1 and T2 from BLUE and PINK masters update Table 1 at the same time

Data may be on the same page and cause a conflict

Transaction T1 from BLUE master achieves quorum (first to reach storage) – transaction T1 is committed

Transaction T2 from PINK master fails to achieves quorum – transaction T2 is rolled back and returns error to client

Storage nodes act as arbitrator for conflict resolution

MASTER 1 MASTER 2


Thank you!


Yoav Eilat

[email protected]

Documents

Deep dive on Amazon Aurora - Percona – The …...Aurora MySQL has 5x the throughput of MySQL 0 100,000 200,000 300,000 400,000 500,000 600,000 700,000 800,000 MySQL 5.6 MySQL 5.7