Traditional RDBMS Wisdom

is All Wrong -- In Three Acts

Michael Stonebraker

The “Stonebraker Says” Webinar Series

The first three acts:

1. Why main memory is the answer for OLTP

– Recording available at VoltDB.com

2. Main memory and ACID, How to Go Fast - Today!

3. OLTP RDBMSs are the Right Tool for Fast + Big Data Apps - June 10

– • Extensions to main memory

– • Support for streaming

– • Larger-than-memory applications

Our Speaker

Dr. Mike Stonebraker of

MIT, co-founder of VoltDB

Overview of Today

• Trends

• Why ACID is essential

• HA

• Concurrency Control

• Summary and Q&A

Version 4.2 launched in March 2014

Offices in Bedford, MA & Santa Clara, CA

400 Customers

Mike Stonebraker Founder, Advisor

VoltDB – The Company

Magic Quadrant

“Operational Databases”

2013 - One of the Top

Companies that Matter

Most in Data

Trends or “Why is it important to go fast?”

• Internet: people + content + things

• Data: Fast + Big

• Smart: raw data + intelligence

The RDBMS is Getting Crushed

Reports

Analytics

Dashboards

Alerts

Mobile

M2M

Market Data

Clickstreams

Web Interactions

Social Media

Internet of Things

RDBMS (In Real-Time)

vs. Batch

Making do? • SSD or Fusion IO cards • Sharding • Cache/Grid • NoSQL, No ACID

Bottleneck

VoltDB: A Modern DB Architecture

In-Memory performance

NewSQL - ACID & SQL & Java

Elastic Scaling

Reliability and fault tolerance

Real-time analytics

Preliminaries….

• Some people argue that you don’t need ACID

• Some people argue that eventual consistency is the best thing

since sliced bread

Who Needs ACID

• Anybody with integrity constraints

– Back out if fails

– Anybody for whom “usually ships in 24 hours” is not an acceptable

outcome

• Anybody with a multi-record state

– E.g. move and shoot

Give Up ACID

• If you need data accuracy, giving up

ACID is a decision to tear your hair out

by doing database “heavy lifting” in user

code

• Can you guarantee you won’t need ACID

tomorrow?

ACID = goodness, in spite of what the critics say

Even Jeff Dean (Google) agrees

Eventual Consistency and Replication

• Do the transaction at the primary, then send to secondary

asynchronously

• If a failure, then make secondary the primary

• Guarantee eventual delivery of transactions to all replicas

• When the dust settles, the claim is that both replicas are the

same

• Example of two people buying a widget

Fails to Work….

• Two people buying a widget when only one widget is in inventory….

• What goes wrong?

– Eventual consistency depends on transactions running in either order at the various replicas

– Fails if you cannot oversell inventory

• More generally…

– Fails unless transactions are commutative

Non Commutative Updates

• For example, + and * don’t commute – 10% raise

– $500.00 bonus

• Anybody with integrity constraints – Can’t sell the last item twice….

• In these cases, eventual consistency means “creates garbage”

• In the rest of this talk, we assume ACID

The Three Big Decisions

• How to do crash recovery

• How to do replication

• How to do concurrency control

Reality Check – High Availability (HA)

• Requirement in today’s OLTP systems

• Nobody will take down time

• Must be solved through replication

Reality Check – Main Memory Performance

• TPC-C CPU cycles

• On the Shore DBMS prototype

• “Elephants” should be similar

Some Data From Nirmesh Malvaiya

• Implemented Aries in VoltDB

• Compared against the VoltDB scheme

– Asynchronous transaction-consistent checkpoints

– Command logging

In Round Numbers

• Command logging 50% faster at run time

• But data logging 50% faster at recovery time

• However…..

– You only stop and recover on cluster-wide failures, e.g.

power failures

– Which are not very common (every few months???)

How to Implement HA

• Active-Passive

– As in the traditional wisdom

• Active-Active

– Send update transactions to all copies

– Each executes transaction logic

How to Implement HA

• Active-Passive

– Effectively requires you to write a log

– One of the four pie slices

• Active-Active

– Send only the transaction, not the effect of the

transaction

– Allows read-queries to be sent to any replica

Some Data From Chaomin Yu (CMU)

• Extended Nirmesh’s VoltDB code to replicas

– 1.5 becomes 1.9 at run time

– Roll forward stays at 1.5

• i.e. active-active wins!

Concurrency Control

• MVCC popular (NuoDB, Hekaton)

• Time stamp order popular (H-Store/VoltDB)

• Lightweight combinations of time stamp order and dynamic

locking (Calvin, Dora)

• I don’t know anybody who is doing normal dynamic locking

– It’s too slow!!!!

VoltDB Scheme

• Tables are replicated or fragmented

• Physical data base design cuts up fragmented tables

horizontally and allocates them to partitions

• Stored procedures can be classified into

– Single-partition ones

– Multi-partition ones

• “one shots”

• Multi-step xacts

Concurrency Control

• Single partition transactions

– Sent directly from user code to the “controller” for the

correct partition

– Who executes them serially (single threaded)

Concurrency Control

• Multi-partition one shots

– Sent to a multi-partition controller

– Who then sends the various pieces to the correct single-

partition controller

– Who inserts the piece somewhere in the stream of

single-partition transactions

• Doesn’t matter where

– With some cleverness, VoltDB does parallel one-shot

reads

Concurrency Control Summary

• Single-partition transactions

– Parallelism determined by number of partitions

– Essentially no overhead

• Multi-partition one-shot reads

– Extensive parallelism

– Essentially no overhead

Concurrency Control Summary

• Multi-partition one shot writes

– One-at-a-time

– Parallelized with single-partition transactions

• Multi-partition multi-step transactions

– Currently slow

– Can be dramatically accelerated using “speculative

execution”

Concurrency Control Advice

• VoltDB is unbeatable on a workload on single partition and

one-shot transactions

• Multi-partition multi-step transactions are slow

– “specEx” is part of the VoltDB roadmap discussion

– If you have high conflict, then you should redesign your

applications

• Nobody will do well

The Nail in the Coffin

• VoltDB scheme compatible with active-active

– As are any deterministic schemes

• Locking and MVCC are not

– Give up a factor of 2!

Net-Net on Main memory ACID • Deterministic concurrency control

– VoltDB has one scheme

• Crash recovery using command logging

– X 1.5

• HA/replication via active-active

– X 1.9

Traditional wisdom (and essentially all DBMS textbooks) are all

wrong!

Summary

• Data is growing in volume and velocity

• Smart data requires adding intelligence in real time

• New architectures are required,

but compromises are not…check out VoltDB

34

Questions?

• Use the chat window to type in your questions.

• Try VoltDB yourself:

Free trial of the Enterprise Edition:

• www.voltdb.com/download

Open source version is available on github.com

Register for the June 10th webinar

Thank You