N-O-SQL, new database technologies on the rise

IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

N-O-SQLnew database technologies on the rise

http://www.flickr.com/photos/wolfgangstaudt/2215246206/


Who am I

» Steven Noels - [email protected]

»Outerthought : scalable content applications

»makers of Daisy and Lily open source CMS

2


Agenda

3

» raison d’être: what brought us here

» concepts: required theory readings

»market overview: trees & the forest

» experiences and (h)in(d)sights


Raison d’être


History

5

hierarchical databases

IMS

OODBMS

XMLDB RDBMS

1. standardization

2. simplification


Inconsistency through slave lag

6

John

Qui

nn (

Dig

g)


Scaling writes (1)

7

John

Qui

nn (

Dig

g)


Scaling writes (2)

8

John

Qui

nn (

Dig

g)


Issues with partitioning

» lose the ability to make arbitrary queries

» have to predict data access patterns when formulating partitioning strategy

» complex and fragile systems

9


Replication complexity

10


Scaling relational systems

11

»When scaling relational systems you loose their advantages but retain their overhead


History

12

RDBMS NOSQL

cachingdenormalisationshardingreplication ...

3. scaling

4. rethinkingthe problem


Moore vs Kryder

» seek time isconstant (networklatency as well?)

» transfer rate ! spindles !

» as a principle, writes arehard to scale

13

✖


Cambrian Explosion

14

IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 15

?Buzz-oriented development


Cambrian Explosion

16

N-O-SQL



The Perspective of Cost


Common themes

19

» SCALE SCALE SCALE

» new datamodels

» devops

»N-O-SQL

»The Cloud :technology is of no interest anymore


Numbers of scale

20

http://qos.doubleclick.net/counters/


Types of scaling

21

» scaling for usage» volume of users

» volume of data

availabilityreplication

» scaling types of ops» concurrent read

» concurrent write

partioningconsistency

distribution


Distributed systems are hard !


8 fallacies of distributed computing» The network is reliable.

» Latency is zero.

» Bandwidth is infinite.

» The network is secure.

» Topology doesn't change.

» There is one administrator.

» Transport cost is zero.

» The network is homogeneous.

23

Pete

r D

euts

ch a

nd Ja

mes

Gos

ling

http://en.wikipedia.org/wiki/Computer_network

http://en.wikipedia.org/wiki/Computer_network

http://en.wikipedia.org/wiki/Latency_%28engineering%29

http://en.wikipedia.org/wiki/Latency_%28engineering%29

http://en.wikipedia.org/wiki/Throughput

http://en.wikipedia.org/wiki/Throughput

http://en.wikipedia.org/wiki/Computer_security

http://en.wikipedia.org/wiki/Computer_security

http://en.wikipedia.org/wiki/Network_topology

http://en.wikipedia.org/wiki/Network_topology

http://en.wikipedia.org/wiki/Network_administrator

http://en.wikipedia.org/wiki/Network_administrator


New Data

» sparse structures

»weak schemas

» graphs

» semi-structured

» document-oriented

24


N-O-SQL =not only SQL !


The NOSQL footprint

26

AC

ID,

sim

ple

oper

atio

nal

const

rain

ts

free-structured or sparse data

SQL

NOSQL

referential integrity,typed data

high

ly scalable an

davailab

le (com

plex

ity)

HBase

Cassandra

CouchDB

MongoDB

neo4j


NOSQL, if you need ...

» horizontal scaling (out rather than up)

» unusually common data (aka free-structured)

» speed (especially for writes)

» the bleeding edge

27


SQL/RDBMS, if you need ...

» SQL

»ACID

» normalisation

» a defined liability

28


Theory


Robust systems

30


Academic background

»Amazon Dynamo

»Google BigTable

» Eric Brewer CAP theorem

31


Amazon Dynamo

32

» coined the term ‘eventual consistency’

» consistent hashing


Consistent hashing

33

http://horicky.blogspot.com/2009/11/nosql-patterns.html


Consistent hashing

34

- node C+ node D

http://www.lexemetech.com/2007/11/consistent-hashing.html

»multi-dimensional column-oriented database

» on top of GoogleFileSystem

» object versioning


Google BigTable

35


CAP theorem

36

strong consistency

highavailability

partition-tolerance


CAP

»Strong Consistency: all clients see the same view, even in the presence of updates

»High Availability: all clients can find some replica of the data, even in the presence of failures

»Partition-tolerance: the system properties hold even when the system is partitioned

37


Consistency

38

»Where is my data I just updated?

» Ideal world :

The result of every write-operation is reflected by subsequent read-operations.


Consistency

39


Sunny-day scenario

40


Network partioning

41


Culture Clash

42

»Classic distributed systems: focus on ACID

» atomic

» consistent

» isolated

» durable

»Modern internet systems: focus on BASE

» basically available

» soft-state (or scalable)

» eventually consistent


Culture Clash

43

»ACID» highest priority: strong

consistency for transactions

» availability less important

» pessimistic

» rigorous analysis

» complex mechanisms

» BASE» availability and scaling

highest priorities

» weak consistency

» optimistic

» best effort

» simple and fast

spectrum


Building for failure

» defensive programming

» creating replicas

» disk flushing

»watch out for failure of utility infrastructure

» conscious sync/async decisions

44


Possible storage failures

45

» Application errors

» Repeatable DB failures

» Unrepeatable DB failures

» OS errors

» Local cluster HW failure

» Local cluster network partitioning

» Disaster

» WAN network failure between remote clusters Mic

hael

Sto

nebr

eake

r


Availability ≠ total async !


The Enterprise Service Bus

47

✘bus =

congestion


Bus systems

48

» objects don’t fit in a pipe

» object ➙ message

» serialization / de-serialization cost

»message size

» queuing = cost


Use a mixture of both

»async + sync

49

stuff which matters !


Numbers of scale

50

http://qos.doubleclick.net/counters/


Processing large datasets :

Map/Reduce


Smart Data

» sparse as a feature

»weak schemas

» ad-hoc indexing

» organic analytics

» near-data processing

» live(ly) datawarehouse

» distribution ➙ parallellization ➙ performance

52


Hadoop: HDFS + MapReduce» single filesystem + single execution-space

53


MapReduce example: WordCount

54


MapReduce

55


MapReduce and HDFS

56

© lars george


Physical architecture

57


Processing large datasets with MR

58

»Benefit from parallellisation

» Less modelling upfront (ad-hoc processing)

»Compartmentalized approach reduces operational risks

»AsterData et al. have SQL/MR hybrids for huge-scale BI


Market overview


Categories

» key-value stores

» column stores

» document stores

» graph databases

60


Key-value stores

»Redis

»Voldemort

»Tokyo Cabinet

61


Redis

»REmote DIctionary Server

» http://code.google.com/p/redis/

» vmware

62


Redis Features» persisted memcache, ‘awesome’

» RAM-based + persistable

» key ➙ values: string, list, set

» higher-level ops

» i.e. push/pop and sort for lists

» fast (very)

» configurable durability

» client-managed sharding

63


Voldemort

» http://project-voldemort.com/

» LinkedIn

64


Voldemort

» persistent

» distributed

» fault-tolerant

» hash table

65


Voldemort

66

API: GET, PUT,DELETE


Voldemort

67

routing logic moving up the stack,smaller latency


Voldemort data format

68

» key+values = arrays of bytes

» So how do we objects ⬌ bytes ?

» json

» string

» java-serialization

» protobuf

» identity


Tokyo Cabinet

» http://1978th.net/tokyocabinet/

»mixi.jp (i.e. Facebook Japan)

69


Product Family

70


Tokyo Cabinet

71

»memory or filesystem

» hash, b-tree, fixed-length, table


Column stores

»BigTable

»HBase

»Cassandra

72


BigTable

» http://labs.google.com/papers/bigtable.html

»Google

» layered on top of GFS

73


HBase

» http://hadoop.apache.org/hbase/

» StumbleUpon / Adobe / Cloudera

74


HBase

» sorted» distributed» column-oriented»multi-dimensional» highly-available» high-performance

» persisted» storage system

» adds random access reads and writes atop HDFS

75


HBase data model

76

»Distributed multi-dimensional sparse map

»Multi-dimensional keys:(table, row, family:column, timestamp) → value

»Keys are arbitrary strings

»Access to row data is atomic


Storage architecture

77

© lars george


Cassandra

» http://cassandra.apache.org/

»Rackspace / Facebook

78


Cassandra

»Key-value store (with added structure)

»Reliability (identical nodes)

» Eventual consistent

»Distributed

»Tunable

» Partitioning

» Replication

79

CA

P


Cassandra write pattern

80


Cassandra applicability

81

FIT

» Scalable reliability (through identical nodes)» Linear scaling»Write throughput» Large Data Sets

NO FIT

» Flexible indexing»Only PK-based

querying»Big Binary Data» 1 Row must fit in

RAM entirely


Document stores

»CouchDB

»MongoDB

»Riak

»MarkLogic

82


CouchDB

» http://couchdb.apache.org/

» couch.io

83


CouchDB

» fault-tolerant

» schema-free

» document-oriented

» accessible via a RESTful HTTP/JSON API

84


CouchDB documents

{ “_id”: ”BCCD12CBB”, “_rev”: ”AB764C”, “type”: ”person”, “name”: ”Darth Vader”, “age”: 63, “headware”: [“Helmet”, “Sombrero”], “dark_side”: true }

85


CouchDB REST API

»HTTP

» PUT /db/docid

»GET /db/docid

» POST /db/docid

»DELETE /db/docid

86


CouchDB Views»MapReduce-based

» Filter, Collate, Aggregate

» Javascript

87

function (Key, Values) { var sum = 0; for(var i in Values) sum += Values[i]; return sum; }

function (doc) { for(var i in doc.tags) emit(doc.tags[i], 1); }

map reduce


CouchDB

» be careful on semantics

» replication ≠ partioning/sharding !

» distributed database = distributable database

» sharded / distributed deploymentrequires proxy layer

88


MongoDB

» http://www.mongodb.org/

» 10gen

89


MongoDB

» cfr. CouchDB, really

» except for:

»C++

» performance focus

» runtime queries (mapreduce still available)

» native drivers (no REST/HTTP layering)

» no MVCC: update-in-place

» auto sharding (alpha)

90


Riak

» http://riak.basho.com/

»Basho Technologies

91


Riak

» buckets/keys, links

» values/content = bucket + metadata

» pluggable storage engines (fs, (D)ETS, InnoDB)

»HTTP/REST API

» automatic distribution

»mapreduce using Javascript

92


Jackrabbit

» http://jackrabbit.apache.org/

»Day Software

93


Jackrabbit

» reference implementation for JSR 170 & 283

» remoting: WebDAV & RMI

» persistence: RDBMS, fs, memory

94


Jackrabbit

» Java-centric (duh)

» complex repository model (nodes+properties)

»mixins, inheritance

»workspaces

» query language

» no partioning/sharding

95


JCR API levels

96


Graph databases

»Neo4j

»AllegroGraph (RDF)

97


Neo4j

» http://neo4j.org/

»Neo Technology

98


Neo4j» data = nodes + relationships + key/value properties

99


Neo4j

»many language bindings, little remoting

» ‘whiteboard’ friendly

» scaling to complexity (rather than volume?)

» lots of focus on domain modelling

» SPARQL/SAIL impl for triple geeks

»mostly RAM centric (with disk swapping & persistence)

100


Experiences & (h)in(d)sights


NOSQL applicability

»Horizontal scaling

»Multi-Master

»Data representation

» search of simplicity

» data that doesn’t fit the E-R model(graphs, trees, versions)

» Speed

102


Tools for the trade

» non-relational data: Couch, Mongo, Riak

»massive quantities: Cassandra, HBase

» persistent caching: Redis, Voldemort

» graphs: neo4j

103


Tool selection

» be careful on the marketeese:smoke and mirrors beware!

»monitor dev list, IRC, Twitter, blogs

»monitor project ‘sponsors’

»mix-and-match

»DON’T NOSQL WITHOUT INTERNAL SYS ARCHS & DEV(OP)S !

104


aptness

complexity

inte

rnet

ente

rpri

seco

rpor

ate

com

mun

ity

NOSQL}S

QL}

105


Our NOSQL-based project: Lily

» (open source)

» scalable store (Apache HBase)

» and search (Apache SOLR)

» content repository

»α due mid 2010

»www.lilycms.org or @outerthought

106


Lily architecture

107

Lily client

client

client

Lily store node

store node

store node

distributed process coordination

and configuration (ZooKeeper)

query indexerupdate

WAL M/RMQ

documents2ary

indexes

WAL /

MQ

}

}

}

Lily Store Server

HBase Region Server

Hadoop DFS

index replica

replica replica

} SOLR

inverted index

REST


When combining store and search, make sure your (search) index doesn’t become the store.


Key lessons learned

109

» importance of keyspace design

» secondary indexing

» data de-normalization

» schema vs. code flexibility?

» distribution is everywhereand you shouldn’t forget about it


Reading material

»Amazon Dynamo, Google BigTable, CAP

» http://nosql.mypopescu.com/

» http://nosql-database.org/

» http://twitter.com/nosqlupdate

» http://highscalability.com/

110


Questions?

111

http://www.flickr.com/photos/leehaywood/4237636853/


» [email protected]

» @stevenn

Thanks for your attention !