<Insert Picture Here>

InnoDB: Status, Architecture and New FeaturesMySQL Conference and Expo, 2010

3:05 pm, April 13, 2010

Calvin Sun, Principal Software Engineer, Oracle/InnoDBRob Young, Sr. Product Manager, Oracle/MySQL

2

Agenda

• What’s New Since Last UC?

• InnoDB Architecture

3

• Where were you on 2005-Oct-7 at 5:14pm EDT?

• 81% of MySQL shops use InnoDB

• Oracle’s commitment• Be the technology leader for InnoDB/MySQL• Community involvement• Strategic, integrated product direction• Align Engineering/Product teams• The best technical support. Period.

• Goal = “Make MySQL a better MySQL”

• First fruits in MySQL 5.1, 5.5

Make MySQL a better MySQL

4

• Barracuda FILE FORMAT (more animals to come…)• Default for executing threads = unlimited• Faster Index Creation• Efficient Data Compression• Off-page storage options for Large Objects, Variable

Length Columns• New INFORMATION_SCHEMA tables• More…

From UC 2009InnoDB 1.0 Announced!

5

Performance/Availability improvements (since 1.0.3):• Better control of Spin Lock polling• Dynamic control of insert buffering, adaptive hash indexing • Group commit is back!• Improved read ahead algorithms• Enabled multiple background threads• Making buffer cache scan resistant• Improved crash recovery performance• More…

Better Instrumentation/Diagnostics• Optimized SHOW ENGINE INNODB MUTEX

GA!Making MySQL 5.1 a better MySQL

InnoDB 1.0 is now option in MySQL 5.1.46

6

MySQL Server

CSV

Enterprise Management Tools and

Utilities

MySQL Enterprise MonitorMySQL Query Analyzer

MySQL WorkBenchBackup & Recovery

SecurityReplication

ClusterPartitioning

INFORMATION_SCHEMAPERFORMANCE_SCHEMA

MySQL 5.1 Pluggable Architecture

1.0

GA!

7

My.cnf ignore_builtin_innodb plugin-load=innodb=ha_innodb_plugin.so;innodb_trx=ha_innodb_plugin.so; ...

Command Linemysqld --ignore-builtin-innodb --plugin-load=innodb=ha_innodb_plugin.so;...

Options for Installing/Enabling in 5.1

INSTALL PLUGININSTALL PLUGIN INNODB SONAME 'ha_innodb_plugin.so';...

GA!

8

Performance improvements (relative to 1.0):• Multiple Buffer Pool Instances• Multiple Rollback Segments• Extended Change Buffering with delete buffering and purge

bufferingBetter isolation of threads and mutex contention

• Improved Purge Scheduling • Improved Log Sys mutex• Separate Flush List mutex

Better Instrumentation/Diagnostics• InnoDB stats in 5.5 PERFORMANCE_SCHEMA

Beta

“InnoDB Plugin: Performance Features and Benchmarks”, InnoDB Team, Thurs, 10:50 am, Ballroom C

Making MySQL 5.5 a better MySQL

InnoDB 1.1 is a better InnoDB 1.0!

9

MySQL 5.5/InnoDB 1.1 SysBench Benchmarks

Intel Xeon X7460 x86_64 4 CPU x 6 Cores/CPU 2.66 GHz, 32GB RAM Fedora 10

MySQL 5.1.40(InnoDB built in)

MySQL 5.1.40 (InnoDB 1.0)

MySQL 5.5.4 (InnoDB 1.1)

At 1024 Connections:• 188% performance gain for MySQL 5.5 over 5.1.40 (InnoDB 1.0)• 200% performance gain for MySQL 5.5 over 5.1.40 (InnoDB built in)

10

MySQL 5.5/InnoDB 1.1 SysBench Benchmarks

MySQL 5.1.40(InnoDB built in)

MySQL 5.1.40 (InnoDB 1.0)

MySQL 5.5.4 (InnoDB 1.1)

Intel Xeon X7460 x86_64 4 CPU x 6 Cores/CPU 2.66 GHz, 32GB RAM Fedora 10

At 1024 Connections:• 164% performance gain for MySQL 5.5 over 5.1.40 (InnoDB 1.0)• 364% performance gain for MySQL 5.5 over 5.1.40 (InnoDB built in)

11

MySQL 5.5 / InnoDB 1.1 ScalabilitydbStress

MySQL 5.5.4 is reaching higher TPS levels than othersMySQL 5.5.4 is better prepared now to scale up to 32 coreshttp://dimitrik.free.fr/blog/archives/2010/04/mysql-performance-554-dbstress.html

Read-only

Read-write

cores

TPS

12

InnoDB Architecture

13

InnoDB Architecture: Component Model

Handler API

Server

Access Methods

Storage and IO Manager

Concurrency Control / Locking

Memory / Buffer Pool Manager

Transaction Manager

Logging and Recovery

Monitoring and Diagnostics

14

InnoDB Architecture: Runtime Model

InnoDB Code

Memory Threads:-master-read io-write io-ibuf io-log io-lock timeout-monitor

Buffer Pool-data-index-undo-adaptive hash index

Background threads

FilesLog

buffer buffer pool Misc buffer

15

Just Like Traffic System

Threads

Synchronization

Resources Scheduling

16

How to Solve Traffic Problem? Threads

Have more lanes• multiple background I/O threads

Build another road• Split purge thread

Car pool / batch• group commit• delete buffering and purge buffering

17

How to Solve Traffic Problem?Synchronization

Better / faster signals• faster locking for improved scalability • using a portable PAUSE to InnoDB spin loop• control over Spin lock polling

Change signal location• Improved log sys mutex • Separate flush list mutex

18

How to Solve Traffic Problem?

ResourcesMore resource providers

• multiple buffer pool instances• multiple rollback segments

Better resource management• using operating system memory allocators• native asynchronous I/O on Linux

19

How to Solve Traffic Problem?Scheduling

Better scheduling / flow control• Changes in read ahead algorithm• Making buffer cache scan resistant • Controlling master thread I/O rate• Controlling flushing rate of dirty pages

Faster accident handling• Improvements to crash recovery performance

Monitoring• Performance schema for InnoDB

20

InnoDB Architecture: Data Storage

Extent

Segment

Extent

Extent Extent

an extent = 64 pages

Extent

Trx id

Row

Field 1

Roll pointer

Field pointers

Field 2 Field n

Row

Page

Row

Row

Row Row

Leaf node segment

Tablespace

Rollback segment

Non-leaf node segment

RowRow

21

InnoDB Architecture: Data Storage

Data compression: Compress data & index pages from normal 16 KB size to specified compressed page size

New row format: New DYNAMIC mode stores long columns entirely “off-page”.

Multiple rollback segments: from 1 to 128 TRUNCATE TABLE re-creates the *.ibd file to

reclaim space

22

What about InnoDB Hot Backup?

Now called “MySQL Enterprise Backup” Backs up running databases without

locking InnoDB data Supports InnoDB and MyISAM Features:

• Compressed backup• Partial backup• Point-in-time recovery

“Backup Strategies with MySQL Enterprise Backup”, InnoDB Team, Wed, 3:05 pm, Ballroom D

23

New Features in Backup 3.5

Incremental backup Support of Barracuda file format

• Backup of compressed tables Backup of additional files, such as partition

files• Backup of in-memory database with

--exec-when-locked option

Email mike.frank@oracle.com to register for beta.

Beta

24

Recap: Plugin 1.0 Top Ten Performance and Scalability Enhancements1. Faster locking for improved scalability2. Using operating system memory allocators3. Multiple background I/O threads4. Group commit is back5. Controlling master thread I/O rate6. Controlling flushing rate of dirty pages7. Changes in read ahead algorithm8. Using a portable PAUSE to InnoDB spin loop9. Making buffer cache scan resistant10. Improvements to crash recovery performance

25

Recap: Plugin 1.1 Performance and Scalability Enhancements Multiple buffer pool instances Multiple rollback segments Improved purge scheduling Improved Log Sys mutex Separate flush list mutex Extended change buffering with delete buffering and

purge buffering: speed up random deletes Native async I/O support on Linux Performance schema for InnoDB

26

Deep Dive

Backup Strategies with MySQL Enterprise Backup, InnoDB Team, Wed, 3:05 pm, Ballroom D

InnoDB Plugin: Performance Features and Benchmarks, InnoDB Team, Thurs, 10:50 am, Ballroom C

BLOGS•http://blogs.innodb.com/innodb-performance-recovery, Inaam Rana•http://blogs.innodb.com/innodb-multiple-rollback-segments, Sunny Bains•http://blogs.innodb.com/innodb-performance-schema, Jimmy Yang•http://mikaelronstrom.blogspot.com/, Mikael Ronstrom•http://dimitrik.free.fr/blog/archives/2010/04/mysql-performance-554-dbstress.html, Dimitri Kravtchuk

TALKS

27

AQ&Q U E S T I O N SQ U E S T I O N S

A N S W E R SA N S W E R S

28

InnoDB: Status, Architecture and New FeaturesMySQL Conference and Expo, 2010

3:05 pm, April 13, 2010calvin.sun@oracle.com rob.young@oracle.com

Thanks for attending!