©2013 DataStax Confidential. Do not distribute without consent.

@PatrickMcFadin

Patrick McFadinChief Evangelist for Apache Cassandra

Keynote: The State of Cassandra

1

Five Years of Cassandra

Jun-09 Mar-10 Jan-11 Nov-11 Sep-12 Jul-13

0.1 0.3 0.6 0.7 1.0 1.2...

2.0

DSE

Jul-08

•Massively scalable •High performance • Reliable/Available

Core values Cassandra HBase Redis MySQL

CREATE TABLE users ( id uuid PRIMARY KEY, name text, state text, birth_date int);!!CREATE INDEX ON users(state);!SELECT * FROM users WHERE state=‘Texas’ AND birth_date > 1950;

New Core Value

•Massively scalable •High performance •Reliable/Available •Productivity + ease of use

CREATE TABLE users ( id uuid PRIMARY KEY, name text, state text, birth_date int);

CREATE TABLE users_addresses ( user_id uuid REFERENCES users, email text);!SELECT *FROM users NATURAL JOIN users_addresses;X

Collections

UPDATE usersSET email_addresses = email_addresses + {‘jbellis@gmail.com’, ‘jbellis@datastax.com’};

CREATE TABLE users ( id uuid PRIMARY KEY, name text, state text, birth_date int, email_addresses set<text>);

Collections

Cassandra Community

Your community

We are growing!

Great projects!

•Storage backend for Graphite •Replacement of whisper •Plugs into carbon •Real users: Love it!

Cyanite

https://github.com/pyr/cyanite

Great projects!

The Kiji Project

http://www.kiji.org/2014/02/11/working-on-cassandra-integration-with-datastax/

•Real-time application framework •Product and content recommendation systems •Risk analysis and fraud monitoring •Customer profile and segmentation applications

Get Involved!

•Talk at a meet up •5 minute interview? •Stack overflow •Mailing list •Apache Jira

Cassandra 2.0

Reads in a cluster

Client Coordinator

40% busy

90% busy

30% busy

A failure

Client Coordinator

40% busy

90% busy

30% busyXtimeout

Rapid read protection

Client Coordinator

40% busy

90% busy

30% busyXsuccess

Rapid Read Protection

NONE

Latency (mid-compaction)

Cold data

10,000 req/s 5,000 req/s

4,000 req/s 10 req/s

Cold data

10,000 req/s 5,000 req/s

4,000 req/s 10 req/s

Cold data compaction

10 req/s

10,000 req/s

Cold Data Compaction•Disabled in 2.0

•Default in 2.1

•Configuration per Table:

ALTER TABLE mykeyspace.mytable WITH compaction = {'class': 'SizeTieredCompactionStrategy', 'cold_reads_to_omit': 0.05};

http://www.datastax.com/dev/blog/optimizations-around-cold-sstables

Cassandra 2.1

User defined typesCREATE TYPE address ( street text, city text, zip_code int, phones set<text>)!CREATE TABLE users ( id uuid PRIMARY KEY, name text, addresses map<text, address>)!SELECT id, name, addresses.city, addresses.phones FROM users;! id | name | addresses.city | addresses.phones--------------------+----------------+-------------------------- 63bf691f | jbellis | Austin | {'512-4567', '512-9999'}

Collection indexingCREATE TABLE songs ( id uuid PRIMARY KEY, artist text, album text, title text, data blob, tags set<text>);!CREATE INDEX song_tags_idx ON songs(tags);!SELECT * FROM songs WHERE tags CONTAINS 'blues';! id | album | artist | tags | title----------+---------------+-------------------+-----------------------+------------------ 5027b27e | Country Blues | Lightnin' Hopkins | {'acoustic', 'blues'} | Worrying My Mind!!

(UDT indexing?)

Counters++•simpler implementation, no more edge cases

•possible to properly repair now

•significantly less garbage and internode traffic generated

•better performance for 99% of uses •RF>1, replicate_on_write=true

•topology changes not leading to data loss (#4071)

•commitlog now 100% safe to replay (#4417)

•Internal format overhaul still coming in 3.0 (#6506)

What hasn’t changed•same API

•same average throughput

•same restrictions on mixing counter and non-counter columns

•same restrictions on mixing counter and non-counter updates

•same restrictions on counter deletes

•same retry limitations

Writes (low contention)

Writes (high contention)

Data directories (2.0)/var/lib/cassandra/data/foo/bar/foo-bar-jb-1-CompressionInfo.db/var/lib/cassandra/data/foo/bar/foo-bar-jb-1-Data.db/var/lib/cassandra/data/foo/bar/foo-bar-jb-1-Filter.db/var/lib/cassandra/data/foo/bar/foo-bar-jb-1-Index.db/var/lib/cassandra/data/foo/bar/foo-bar-jb-1-Statistics.db/var/lib/cassandra/data/foo/bar/foo-bar-jb-1-Summary.db/var/lib/cassandra/data/foo/bar/foo-bar-jb-1-TOC.txt

Data directories (2.1)/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/foo-bar-ka-1-CompressionInfo.db/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/foo-bar-ka-1-Data.db/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/foo-bar-ka-1-Filter.db/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/foo-bar-ka-1-Index.db/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/foo-bar-ka-1-Statistics.db/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/foo-bar-ka-1-Summary.db/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/foo-bar-ka-1-TOC.txt

Inefficient bloom filters

+

= ?

+

=

Inefficient bloom filters

+

=

Inefficient bloom filters

Inefficient bloom filters

HyperLogLog applied

HLL and compaction

HLL and compaction

More-efficient repair

More-efficient repair

More-efficient repair

Implications for LCS (and STCS)

Performance•Memtable memory use cut by 40% (85% by beta2)

• larger sstables, less compaction

•~50% better write performance

•Full results after beta2

The new query cache

The new row cacheCREATE TABLE notifications ( target_user text, notification_id timeuuid, source_id uuid, source_type text, activity text, PRIMARY KEY (target_user, notification_id))WITH CLUSTERING ORDER BY (notification_id DESC) AND caching = 'rows_only' AND rows_per_partition_to_cache = '3';!

The new row cachetarget_user notification_id source_id source_type activity

nick e1bd2bcb- d972b679- photo jbellis liked

nick 321998c- d972b679- photo rbranson commented

nick ea1c5d35- 88a049d5- user mbulman created account

nick 5321998c- 64613f27- photo jbellis commented

nick 07581439- 076eab7e- user thobbs created account

rbranson 1c34467a- f04e309f- user jbellis created account

The new row cachetarget_user notification_id source_id source_type activity

nick e1bd2bcb- d972b679- photo jbellis liked

nick 321998c- d972b679- photo rbranson commented

nick ea1c5d35- 88a049d5- user mbulman created account

nick 5321998c- 64613f27- photo jbellis commented

nick 07581439- 076eab7e- user thobbs created account

rbranson 1c34467a- f04e309f- user jbellis created account

2.1 roadmap•Beta1 today •Beta2 beginning of April •RC end of March •Final release end of April?

Questions?