MySQL and SSD

Vadim Tkachenko

Percona Inc, co-founder, CTO

www.percona.com

www.SSDPerformanceBlog.com

vadim@percona.com

This talk online

•PowerPoint

•http://bit.ly/MySQL-SSD-2012

•PDF

•http://bit.ly/MySQL-SSD-2012-PDF

•Contacts

• vadim@percona.com

•Twitter @VadimTk

MySQL and SSD

World is spinning

MySQL and SSD

Physical limits

MySQL and SSD

Rotate faster

7200 RPM

10000 RPM

15000 RPM

Kaboom

MySQL and SSD

milliseconds

Access time

MySQL and SSD

More spindles

MySQL and SSD

Still milliseconds

MySQL and SSD

Access time

MySQL and SSD

MySQL and SSD

Data fits into

memory

10% data growth

70% throughput drop

MySQL and SSD

Flash Access time

MySQL and SSD

MySQL throughput with Flash

MySQL and SSD

Flash

MySQL and SSD

MySQL and SSD

Erase size

MySQL and SSD

Write once

MySQL and SSD

Write Erase

•slow

MySQL and SSD

No rewrites

MySQL and SSD

MySQL and SSD

Garbage collector

MySQL and SSD

Write amplification

Flash writes more than application

MySQL and SSD

Software matters

MySQL and SSD

Hardware is less important

MySQL and SSD

MySQL and SSD

Flash quality is defined by software

Log-structured file system

Wear leveling

Garbage collector

MySQL and SSD

Flash types

SLC MLC

MySQL and SSD

Single Level Cell – 1bit

Erase

0

1

MySQL and SSD

Multi Level Cell – 2 bit

Erase

00

01

10

11

MySQL and SSD

Multi Level Cell – 3 bit

Erase

000

001

010

011

100

101

110

111

MySQL and SSD

SLC vs MLC

MySQL and SSD

Erase cycles

MySQL and SSD

• SLC 100.000 cycles

• MLC 10.000 cycles

• 25nm MLC 5.000 cycles

SLC

Benefits

• Reliability

• Performance

Drawbacks

• Up to 800GB

• Expensive

• 30-50$/GB

MySQL and SSD

MLC

•Over 1TB

•10-15$/GB

•Life time

•Reliability

MySQL and SSD

Space provisioning – Virident FlashMax 1400

MySQL and SSD

User space

1.4TB

Internal Space

0.6TB

SATA vs PCI Express

MySQL and SSD

SATA SSD

MySQL and SSD

My benchmarks story

2 Intel 320 SSD cards

MySQL and SSD

How do I install it?

MySQL and SSD

I need

Space Power

Controller +cables

MySQL and SSD

Initial setup

MySQL and SSD

Polished setup - AccuSTOR AS108X

MySQL and SSD

RAID controllers

LSI 9260 LSI 9211

MySQL and SSD

Last component - cable

LSI - SFF8087

Enclosure - SFF8088

MySQL and SSD

PCIe is different

MySQL and SSD

PCIe

MySQL and SSD

Just plug into a PCIe slot

But SATA is hot-swap

MySQL and SSD

Benchmarks lie

MySQL and SSD

Benchmark challenge: internal state

MySQL and SSD

Benchmark challenge: capacity

MySQL and SSD

Unrepeatable results

MySQL and SSD

Benchmark challenge: filesystems

Ext3/4 – synchronous IO

“bug” in O_DIRECT

MySQL and SSD

Benchmark challenge: filesystems

xfs - asynchronous

“bug” - serialization

MySQL and SSD

Xfs already fixed bug in source

code

4x improvement

MySQL and SSD

Benchmark challenge: filesystems

btrfs – not ready yet

MySQL and SSD

To add to confusion: in MySQL

Reads - sync Writes –

async

Readahead reads - async

MySQL and SSD

Ext4 vs xfs – your choice

MySQL and SSD

Comparing apples

• 2.5” 15K RPM HP Smart Array

8xHDD RAID10

• SATA SLC

STEC MACH16 200GB

• RAID10 – LSI 9211-4i

4xSTEC MACH16

• SATA MLC

Intel 320 SSD 160GB

• PCIe MLC

Virident FlashMax 1400

MySQL and SSD

Flash vendors

~50 on market

MySQL and SSD

Random 16KiB reads

MySQL and SSD

50

222

417

195

1300

0

200

400

600

800

1000

1200

1400

MiB

/ s

ec

8 HDD

STEC

4 STEC

Intel

Virident

Random 16KiB writes

MySQL and SSD

35

135

223

92

520

0

100

200

300

400

500

600

MiB

/ s

ec

8 HDD

STEC

4 STEC

Intel

Virident

Price

MySQL and SSD

24001500

6000

270

18000

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

20000

Pri

ce, $

8 HDD

STEC

4 STEC

Intel

Virident

PCIe vs SATA

Which one to choose?

MySQL and SSD

PCIe for absolute performance

I use it because I have free samples

MySQL and SSD

SATA for performance per $

I would use it if I had to buy…

MySQL and SSD

When should I use flash?

MySQL and SSD

Very good for random reads

Both SLC and MLC

MySQL and SSD

Random Writes

Maybe challenge for MLC

MySQL and SSD

SLC lifetime

20 years?

MySQL and SSD

MLC lifetime

8PB 15PB

MySQL and SSD

Write amplification

Flash writes more than application

MySQL and SSD

Experiment – tpcc-mysql

Virident FlashMAX 1400

Write amplification: 1.143

1125.65 GiB writes per hour

Lifetime: 1.52 years

MySQL and SSD

Flash for MySQL

MySQL and SSD

When Flash helps

Low-latency requirement

Joins, large tables, mixed workloads, replication

High throughput workloads

High concurrency workloads

MySQL and SSD

Most important decision

MySQL version

MySQL and SSD

MySQL 5.1 with builtin InnoDB

Not good

MySQL and SSD

You need

Multiple IO threads

Async

MySQL and SSD

Choices

Percona Server 5.5

MySQL 5.5

Percona Server 5.1

MySQL and SSD

Benchmarks again

MySQL and SSD

Percona Server 5.5

4x STEC MACH16 RAID10

• LSI 9211

• LSI 9260 – with cache

Sysbench oltp

• 100GB database, 50GB memory

MySQL and SSD

STEC SSD vs HDD: 8x gain

MySQL and SSD

750

90

0

100

200

300

400

500

600

700

800

SSD HDD

tps

STEC innodb_flush_log_at_trx_commit

MySQL and SSD

750

490

0

100

200

300

400

500

600

700

800

trx_commit=2 trx_commit=1

tps

MySQL IO workloads

MySQL

InnoDB data

InnoDB redo logs

Binary logs

InnoDB system tablespace

MySQL and SSD

innodb_flush_log_at_trx_commit=1

Location matters

SSD

RAID with cache

MySQL and SSD

STEC: InnoDB log location

MySQL and SSD

740

490

0

100

200

300

400

500

600

700

800

logs on RAID with cache logs on SSD

tps

STEC – RAID card

MySQL and SSD

750810

0

100

200

300

400

500

600

700

800

900

LSI 9211 (no cache) LSI 9260 (with cache)

tps

Log size matters

MySQL and SSD

Big log file 8GB (Percona Server)

MySQL and SSD

Flushing algorithm is important

MySQL and SSD

Innodb_adaptive_checkpoint=keep_average

(Percona Server)

MySQL and SSD

DoubleWrite area is important

MySQL and SSD

DoubleWrite is a protection

MySQL and SSD

DoubleWrite is rewriting the same area

MySQL and SSD

Rewrites of the same area

MySQL and SSD

Consider moving doublewrite

• Innodb_doublewrite_file =

• Percona Server

• ibdata1

• For general MySQL

MySQL and SSD

Fusion-io to support atomic

writes

1.5x performance improvement

MySQL and SSD

Misc

innodb_flush_neighbor_pages= ON | OFF

innodb_log_block_size = 512 | 4096

MySQL and SSD

Misc filesystem

Mkfs.xfs –s size=4096

Mount –o nobarrier

MySQL and SSD

Is Flash expensive?

MySQL and SSD

Consolidation

MySQL and SSD

Power savings

MySQL and SSD

New Relic

MySQL and SSD

Dell PowerEdge R610

Dell PowerVault MD1220

MySQL and SSD

Perc H800 RAID5

11 Intel 320 SSD 600GB

MySQL and SSD

Scale Up, not Scale Out

MySQL and SSD

“Flash made everything faster,

but more confusing”

MySQL and SSD

Pictures credits

• http://www.sunrainet.com/hdd-vs-ssd-speed-test-video-

windows-7-boot-up.html

• http://blog.familytreemagazine.com/insider/content/binary/datace

nter-2.jpg

MySQL and SSD

Thank you!

Questions ?

Flash is exciting!

vadim@percona.com

MySQL and SSD