Click here to load reader

Deep Dive on Amazon Aurora - Amazon Web Servicesaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS...Deep Dive on Amazon Aurora Open source compatible relational database Performanceand

  • View
    0

  • Download
    0

Embed Size (px)

Text of Deep Dive on Amazon Aurora - Amazon Web...

  • © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

    Ian RobinsonSpecialist Solutions Architect, Data & Analytics, EMEA

    18 May, 2017

    Deep Dive on Amazon Aurora

  • Open source compatible relational database

    Performance and availability of commercial databases

    Simplicity and cost-effectiveness of open source databases

    What is Amazon Aurora?

  • Fastest growing

    service in AWS

    history

    Business applications

    Web and mobile

    Content management

    E-commerce, retail

    Internet of Things

    Search, advertising

    BI, analytics

    Games, media

    Aurora Customer Adoption

  • Performance

  • WRITE PERFORMANCE READ PERFORMANCE

    Scaling with Instance Sizes

    Aurora scales with instance size for both read and write.

    Aurora MySQL 5.6 MySQL 5.7

  • Real-Life Data – Gaming WorkloadAurora vs. RDS MySQL – r3.4XL, MAZ

    Aurora 3X faster on r3.4xlarge

  • How Did We Achieve This?

    I/OPackets/secondContext switching

  • A Service-Oriented Architecture Applied to DatabasesControl PlaneData Plane

    Amazon DynamoDB

    Amazon SWF

    Amazon Route 53

    Logging + Storage

    SQL

    Transactions

    Caching

    Amazon S3

  • I/O Traffic in MySQL

    BINLOG DATA DOUBLE-WRITELOG FRM FILES

    T Y P E O F W R I T E

    EBS mirrorEBS mirror

    AZ 1 AZ 2

    Amazon S3

    EBSAmazon Elastic

    Block Store (EBS)

    PrimaryInstance

    ReplicaInstance

    1

    2

    3

    4

    5

    780K transactions7,388K I/Os per million txns (excludes mirroring, standby)Average 7.4 I/Os per transaction

    PERFORMANCE

    30 minute SysBench writeonly workload, 100GB dataset, RDS MultiAZ, 30K PIOPS

  • I/O Traffic in Aurora

    AZ 1 AZ 3

    PrimaryInstance

    Amazon S3

    AZ 2

    ReplicaInstance

    ASYNC4/6 QUORUM

    DISTRIBUTED WRITES

    BINLOG DATA DOUBLE-WRITELOG FRM FILES

    T Y P E O F W R I T E

    27,378K transactions 35X MORE950K I/Os per 1M txns (6X amplification) 7.7X LESS

    PERFORMANCE

    ReplicaInstance

  • I/O Traffic in Aurora (Storage Node)

    LOG RECORDS

    Primary Instance

    INCOMING QUEUE

    STORAGE NODE

    S3 BACKUP

    1

    2

    3

    4

    5

    6

    7

    8UPDATE QUEUE

    ACK

    HOTLOG

    DATABLOCKS

    POINT IN TIMESNAPSHOT

    GC

    SCRUBCOALESCE

    SORTGROUP

    PEER TO PEER GOSSIPPeerStorageNodes

    All steps are asynchronousOnly steps 1 and 2 are in foreground latency pathInput queue is 46X less than MySQL (unamplified, per node)Favor latency-sensitive operationsUse disk space to buffer against spikes in activity

    OBSERVATIONS

  • Asynchronous Group Commits

    Read

    Write

    Commit

    Read

    Read

    T1

    Commit (T1)

    Commit (T2)

    Commit (T3)

    LSN 10

    LSN 12

    LSN 22

    LSN 50

    LSN 30 LSN 34LSN 41LSN 47

    LSN 20

    LSN 49

    Commit (T4)Commit (T5)Commit (T6)Commit (T7)

    Commit (T8)

    LSN GROWTHDurable LSN at head-node

    COMMIT QUEUEPending commits in LSN order

    TIME

    GROUPCOMMIT

    TRANSACTIONS

    Read

    Write

    Commit

    Read

    Read

    T2

    Read

    Write

    Commit

    Read

    Read

    Tn

  • MySQL Read Scaling

    MySQL Master

    30% Read

    70% Write

    MySQL Replica

    30% New Reads

    70% Write

    SINGLE-THREADEDBINLOG APPLY

    Data Volume Data Volume

  • Aurora Read Scaling

    PAGE CACHEUPDATEAurora Master

    30% Read

    70% Write

    Aurora Replica

    100% New Reads

    Shared Multi-AZ Storage

  • “In MySQL, we saw replica lag spike to almost 12 minutes which is almost absurd from an application’s perspective. With Aurora, the maximum read replica lag across 4 replicas never exceeded 20 ms.”

    Real-Life Data - Read Replica Latency

  • CLI

    ENT

    CO

    NN

    ECTI

    ON

    CLI

    ENT

    CO

    NN

    ECTI

    ON

    LATCH FREETASK QUEUE

    epol

    l()

    MYSQL THREAD MODEL AURORA THREAD MODEL

    Adaptive Thread Pool

  • Availability

    “Performance only matters if your database is up”

  • Storage Durability

    AZ 1 AZ 2 AZ 3

    Amazon S3

  • What can fail?Segment failures (disks)Node failures (machines)AZ failures (network or datacenter)

    Optimizations4 out of 6 write quorum3 out of 6 read quorumPeer-to-peer replication for repairs

    SQL

    Transaction

    AZ 1 AZ 2 AZ 3

    Caching

    SQL

    Transaction

    AZ 1 AZ 2 AZ 3

    Caching

    Amazon Aurora Storage Engine Fault-Tolerance

  • Aurora Replicas

    AZ 1 AZ 3AZ 2

    PrimaryNodePrimaryNodePrimaryNode

    PrimaryNodePrimaryNode

    SecondaryNode

    PrimaryNodePrimaryNode

    SecondaryNode

  • Continuous Backup

    Segment snapshot Log records

    Recovery point

    Segment 1

    Segment 2

    Segment 3

    Time

  • Checkpointed Data Redo Log

    Crash at T0 requiresa re-application of theSQL in the redo log sincelast checkpoint

    T0

    Traditional Crash Recovery

  • T0

    Crash at T0 will result in redo logs being applied to each segment on demand, in parallel, asynchronously

    Amazon Aurora – Instant Crash Recovery

  • Survivable Caches

    SQLTransactions

    Caching

    SQL

    Transactions

    Caching

    SQLTransactions

    Caching

    Caching process is outside the DB process and remains warm across a database restart

  • Faster Failover

    AppRunningFailure Detection DNS Propagation

    Recovery Recovery

    DBFailure

    MYSQL

    AppRunning

    Failure Detection DNS Propagation

    Recovery

    DBFailure

    AURORA WITH MARIADB DRIVER

    1 5 - 2 0 s e c

    3 - 2 0 s e c

  • Database Failover Time

    0.00% 5.00%

    10.00% 15.00% 20.00% 25.00% 30.00% 35.00%

    1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35

    0 - 5s – 30% of fail-overs

    0.00%

    10.00%

    20.00%

    30.00%

    40.00%

    50.00%

    1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35

    5 - 10s – 40% of fail-overs

    0% 10% 20% 30% 40% 50% 60%

    1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35

    10 - 20s – 25% of fail-overs

    0%

    5%

    10%

    15%

    20%

    1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35

    20 - 30s – 5% of fail-overs

  • Thank You