Click here to load reader

BERLIN - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit... · Aurora RDS MySQL;30,000 IOPS (Single AZ) Updates/ Second Amazon Aurora RDS MySQL 30K IOPS (Single

  • View
    0

  • Download
    0

Embed Size (px)

Text of BERLIN - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit... · Aurora RDS...

  • BERLIN

    ©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved

  • ©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved

    Amazon Aurora: Amazon’s New

    Relational Database Engine

    Carlos Conde – Technology Evangelist

    @caarlco

  • Current DB architectures are monolithic

    Multiple layers of

    functionality all on a

    single box

    SQL

    Transactions

    Caching

    Logging

  • Current DB architectures are monolithic

    Even when you scale

    it out, you’re still

    replicating the same

    stack

    SQL

    Transactions

    Caching

    Logging

    SQL

    Transactions

    Caching

    Logging

    Application

  • Current DB architectures are monolithic

    SQL

    Transactions

    Caching

    Logging

    SQL

    Transactions

    Caching

    Logging

    Application Even when you scale

    it out, you’re still

    replicating the same

    stack

  • Current DB architectures are monolithic

    SQL

    Transactions

    Caching

    Logging

    SQL

    Transactions

    Caching

    Logging

    Storage

    Application Even when you scale

    it out, you’re still

    replicating the same

    stack

  • This is a problem.

    For cost. For flexibility. And for availability.

  • Reimagining the relational database

    What if you were inventing the database today?

    You wouldn’t design it the way we did in 1970. At least not entirely.

    You’d build something that can scale out, that is self-healing, and that

    leverages existing AWS services.

  • Relational databases reimagined for the cloud

    Speed and availability of high-end commercial databases

    Simplicity and cost-effectiveness of open source databases

    Drop-in compatibility with MySQL

    Simple pay as you go pricing

    Delivered as a managed service

  • A service-oriented architecture applied to the database

    • Moved the logging and storage layer

    into a multi-tenant, scale-out

    database-optimized storage service

    • Integrated with other AWS services

    like Amazon EC2, Amazon VPC,

    Amazon DynamoDB, Amazon SWF,

    and Amazon Route 53 for control

    plane operations

    • Integrated with Amazon S3 for

    continuous backup with

    99.999999999% durability

    Control PlaneData Plane

    Amazon

    DynamoDB

    Amazon SWF

    Amazon Route 53

    Logging + Storage

    SQL

    Transactions

    Caching

    Amazon S3

  • • Sign up for preview access at: https://aws.amazon.com/rds/aurora/preview

    • Now available in US West (Oregon) and EU (Ireland), in addition to US East (N. Virginia)

    • Thousands of customers already invited to the limited preview

    • Now moving to unlimited preview; accepting all requests in 2–3 weeks

    • Full service launch in the coming months

    Aurora preview

    https://aws.amazon.com/rds/aurora/preview

  • Simple pricing

    • No licenses

    • No lock-in

    • Pay only for what you use

    Discounts

    • 44% with a 1-year RI

    • 63% with a 3-year RI

    vCPU Mem Hourly Price

    db.r3.large 2 15.25 $0.29

    db.r3.xlarge 4 30.5 $0.58

    db.r3.2xlarge 8 61 $1.16

    db.r3.4xlarge 16 122 $2.32

    db.r3.8xlarge 32 244 $4.64

    • Storage consumed, up to 64 TB, is $0.10/GB-month

    • IOs consumed are billed at $0.20 per million I/O

    • Prices are for Virginia

    Enterprise grade, open source pricing

  • Aurora Works with Your Existing Apps

  • Establishing our ecosystem

    Business Intelligence Data Integration Query and Monitoring SI and Consulting

    “It is great to see Amazon Aurora remains MySQL compatible; MariaDB connectors

    work with Aurora seamlessly. Today, customers can take MariaDB Enterprise with MariaDB MaxScale drivers and connect to Aurora, MariaDB, or MySQL without worrying about

    compatibility. We look forward to working with the Aurora team in the future to further

    accelerate innovation within the MySQL ecosystem.”— Roger Levy, VP Products, MariaDB

  • Amazon Aurora Is Easy to Use

  • Simplify database management

    • Create a database in minutes

    • Automated patching

    • Push-button scale compute

    • Continuous backups to S3

    • Automatic failure detection and failover

    Amazon RDS

  • Simplify storage management

    • Read replicas are available as failover targets—no data loss

    • Instantly create user snapshots—no performance impact

    • Continuous, incremental backups to S3

    • Automatic storage scaling up to 64 TB—no performance or availability

    impact

    • Automatic restriping, mirror repair, hot spot management, encryption

  • Simplify data security

    • Encryption to secure data at rest– AES-256; hardware accelerated

    – All blocks on disk and in Amazon S3 are encrypted

    – Key management via AWS KMS

    • SSL to secure data in transit

    • Network isolation via Amazon VPC by default

    • No direct access to nodes

    • Supports industry standard security and data protection

    certifications

    Storage

    SQL

    Transactions

    Caching

    Amazon S3

    Application

  • Amazon Aurora Is Highly Available

  • Aurora storage

    • Highly available by default– 6-way replication across 3 AZs

    – 4 of 6 write quorum

    • Automatic fallback to

    3 of 4 if an AZ is unavailable

    – 3 of 6 read quorum

    • SSD, scale-out, multi-tenant storage– Seamless storage scalability

    – Up to 64 TB database size

    – Only pay for what you use

    • Log-structured storage– Many small segments, each with

    their own redo logs

    – Log pages used to generate data pages

    – Eliminates chatter between database and storage

    SQL

    Transactions

    AZ 1 AZ 2 AZ 3

    Caching

    Amazon S3

  • Consistent, low-latency writes

    AZ 1 AZ 2

    Primary

    Instance

    Standby

    Instance

    Amazon Elastic

    Block Store (EBS)

    Amazon S3

    EBSmirror

    EBS

    EBSmirror

    AZ 1 AZ 3

    Primary

    Instance

    Amazon S3

    AZ 2

    Replica

    Instance

    Improvements

    • Consistency—tolerance to outliers

    • Latency— synchronous vs. asynchronous replication

    • Significantly more efficient use of network I/O

    Log records

    Binlog

    Data

    Double-write buffer

    FRM files, metadata

    Type of writes

    MySQL with standby Amazon Aurora

    async

    4/6 quorum

    PiTR

    Sequential

    write

    Sequential

    write Distributed

    writes

  • Self-healing, fault-tolerant

    • Lose two copies or an AZ failure without read or write availability impact

    • Lose three copies without read availability impact

    • Automatic detection, replication, and repair

    SQL

    Transaction

    AZ 1 AZ 2 AZ 3

    Caching

    SQL

    Transaction

    AZ 1 AZ 2 AZ 3

    Caching

    Read and write availability Read availability

  • Traditional databases

    • Have to replay logs since the last

    checkpoint

    • Single-threaded in MySQL;

    requires a large number of disk

    accesses

    Amazon Aurora

    • Underlying storage replays redo

    records on demand as part of a

    disk read

    • Parallel, distributed, asynchronous

    Checkpointed Data Redo Log

    Crash at T0 requires

    a re-application of the

    SQL in the redo log since

    last checkpoint

    T0 T0

    Crash at T0 will result in redo

    logs being applied to each segment

    on demand, in parallel, asynchronously

    Instant crash recovery

  • Survivable caches

    • We moved the cache out of

    the database process

    • Cache remains warm in the

    event of a database restart

    • Lets you resume fully loaded

    operations much faster

    • Instant crash recovery +

    survivable cache = quick and

    easy recovery from DB failures

    SQL

    Transactions

    Caching

    SQL

    Transactions

    Caching

    SQL

    Transactions

    Caching

    Caching process is outside the DB process

    and remains warm across a database restart

  • Multiple failover targets—no data loss

    Page cache

    invalidation

    Aurora Master

    30% Read

    70% Write

    Aurora Replica

    100% New

    Reads

    Shared Multi-AZ Storage

    MySQL Master

    30% Read

    70% Write

    MySQL Replica

    30% New Reads

    70% Write

    Single-threaded

    binlog apply

    Data Volume Data Volume

    MySQL read scaling

    • Replicas must replay logs

    • Replicas place additional load on master

    • Replica lag can grow indefinitely

    • Failover results in data loss

  • Simulate failures using SQL

    • To cause the failure of a component at the database node:ALTER SYSTEM CRASH [{INSTANCE | DISPATCHER | NODE}]

    • To simulate the failure of disks:ALTER SYSTEM SIMULATE percent_failure DISK failure_type IN

    [DISK index | NODE index] FOR INTERVAL interval

    • To simulate the failure of networking:ALTER SYSTEM SIMULATE percent_failure NETWORK failure_type

    [TO {ALL | read_replica | availability_zone}] FOR INTERVAL interval

  • Amazon Aurora Is Fast

  • Write performance (console screenshot)

    • MySQL Sysbench

    • R3.8XL with 32 cores

    and 244 GB RAM

    • 4 client machines with

    1,000 threads each

  • Read performance (console screenshot)

    • MySQL Sysbench

    • R3.8XL with 32 cores

    and 244 GB RAM

    • Single client with

    1,000 threads

  • Read replica lag (console screenshot)

    • Aurora Replica with 7.27 ms replica lag at 13.8 K updates/second

    • MySQL 5.6 on the same hardware has ~2 s lag at 2 K updates/second

  • Writes scale with table count

    -

    10

    20

    30

    40

    50

    60

    70

    10 100 1.000 10.000

    Th

    ou

    san

    ds o

    f W

    rite

    s p

    er

    Seco

    nd

    Number of Tables

    Write Performance and Table Count

    Aurora

    MySQL on I2.8XL

    MySQL on I2.8XL with RAM Disk

    RDS MySQL with 30,000 IOPS (Single AZ)

    TablesAmazon

    Aurora

    MySQL

    I2.8XL

    Local SSD

    MySQL

    I2.8XL

    RAM Disk

    RDS MySQL

    30K IOPS

    (Single AZ)

    10 60,000 18,000 22,000 25,000

    100 66,000 19,000 24,000 23,000

    1,000 64,000 7,000 18,000 8,000

    10,000 54,000 4,000 8,000 5,000

    Write-only workload

    1,000 connections

    Query cache (default on for Amazon Aurora, off for MySQL)

  • Better concurrency

    -

    20

    40

    60

    80

    100

    120

    50 500 5.000

    Th

    ou

    san

    ds o

    f W

    rite

    s p

    er

    Seco

    nd

    Concurrent Connections

    Write Performance and Concurrency

    Aurora RDS MySQL with 30,000 IOPS (Single AZ)

    ConnectionsAmazon

    Aurora

    RDS MySQL

    30K IOPS

    (Single AZ)

    50 40,000 10,000

    500 71,000 21,000

    5,000 110,000 13,000

    OLTP Workload

    Variable connection count

    250 tables

    Query cache (default on for Amazon Aurora, off for MySQL)

  • Caching improves performance

    -

    50

    100

    150

    200

    250

    300

    350

    400

    100/0 50/50 0/100

    Th

    ou

    san

    ds o

    f O

    pera

    tio

    ns/S

    eco

    nd

    Read/Write Ratio

    Performance with query cache on and off

    Aurora without Caching

    Aurora with Caching

    RDS MySQL;30,000 IOPS (Single AZ) - without caching

    RDS MySQL;30,000 IOPS (Single AZ) - with caching

    R/W Ratio

    Amazon

    Aurora

    Without

    Caching

    Amazon

    Aurora

    With

    Caching

    RDS MySQL

    30K IOPS

    Without

    Caching

    RDS MySQL

    30K IOPS

    With

    Caching

    100/0 160,000 375,000 35,000 19,000

    50/50 130,000 93,000 24,000 20,000

    0/100 64,000 64,000 16,000 16,000

    OLTP workload

    1,000 connections

    250 tables

    Query cache on/off tested

  • Replicas have up to 400 times less lag

    2,6 3,4 3,9 5,4

    1.000 2.000 5.000 10.000

    0

    50.000

    100.000

    150.000

    200.000

    250.000

    300.000

    350.000

    Updates per Second

    Read

    Rep

    lica L

    ag

    in

    Milliseco

    nd

    s

    Read Replica Lag

    Aurora RDS MySQL;30,000 IOPS (Single AZ)

    Updates/

    Second

    Amazon

    Aurora

    RDS MySQL

    30K IOPS

    (Single AZ)

    1,000 2.62 ms 0 s

    2,000 3.42 ms 1 s

    5,000 3.94 ms 60 s

    10,000 5.38 ms 300 s

    Write workload

    250 tables

    Query cache on for Amazon Aurora, off for MySQL (best settings)

  • BERLIN