Embed Size (px)
DESCRIPTION
This talk was due to be presented at JAX London 2013, but the speaker was unfortunately unable to attend. Distributed data stores give us increased availability, linear scalability, predictable latency and improved fault tolerance. The flip-side is having to deal with inconsistencies: most distributed databases will ask your application layer how to resolve such inconsistencies. Conflict-free Replicated Data Types (CRDTs) are a way for a distributed database, such as Riak, to resolve those inconsistencies logically and automatically. Unlike traditional data structures, there is always a single state on which they converge. In this talk, I’ll look at the development of CRDTs from an academic project to implementation in Riak.
Citation preview
Conflict Free Replicated Data-types in Eventually Consistent SystemsJoel Jacobson, Basho Technologies
JAX London2013
Friday, 8 November 13
Retweet to win a copy!@joeljacobson
Friday, 8 November 13
RiakDistributed, Masterless, Key/Value Database+ Extras
Friday, 8 November 13
Buckets, Keys and ValuesSimple operations;GET, PUT, DELETE
Key ValueKey ValueKey ValueKey ValueKey ValueKey Value
Friday, 8 November 13
Distributed & Scalable
Friday, 8 November 13
Distributed & Scalable
Friday, 8 November 13
Fault Tolerance
Any node can serve R/W requests
Data is replicated
Fallback Node
Hinted Handoff
Friday, 8 November 13
Dynamo Systems
Distributed
High Availability
Masterless
Eventually Consistent
Friday, 8 November 13
A 1
3
2
Client
Friday, 8 November 13
A 1
3
2
Client
Friday, 8 November 13
N 1
3
2
N
Client
Client
Friday, 8 November 13
Conflicts
Friday, 8 November 13
Conflict Resolution
Last-Write Wins (LWW)
Vector Clocks (Siblings)
Friday, 8 November 13
Sibling Resolution
Friday, 8 November 13
Conflict-Free Replicated Data Types
Friday, 8 November 13
http://hal.upmc.fr/docs/00/55/55/88/PDF/techreport.pdf
Friday, 8 November 13
Counters
Sets
Maps
Registers
Booleans
Primitives
Friday, 8 November 13
Counters
Friday, 8 November 13
Counters
{ 'type': 'g-‐counter', 'e': { 'a': 1, 'b': 5, 'c': 2 }}
Friday, 8 November 13
Counters
{ 'type': 'g-‐counter', 'e': { 'a': 1, 'b': 5, 'c': 2 }}
{ 'type': 'pn-‐counter', 'p': { 'a': 10, 'b': 2 }, 'n': { 'c': 5, 'a': 1 }}
Friday, 8 November 13
Counters
{ 'type': 'g-‐counter', 'e': { 'a': 1, 'b': 5, 'c': 2 }}
{ 'type': 'pn-‐counter', 'p': { 'a': 10, 'b': 2 }, 'n': { 'c': 5, 'a': 1 }}
P=12, N=6, so the value is 6
Friday, 8 November 13
The Problem of Absence
Friday, 8 November 13
Sets
Friday, 8 November 13
Sets{ 'type': 'g-‐set', 'e': ['a', 'b', 'c']}
Friday, 8 November 13
Sets{ 'type': 'g-‐set', 'e': ['a', 'b', 'c']}
'type': 'or-‐set'{ [ {a, 3}, {b, 2}, {c, 2}], %% Version Vector [{ 'bob', [{a, 1}]}, %% Bob was added by 'a' at logical time 1 (a's first event) {'joel', [{a, 3}]}, %% Joel was added by 'a' at logical time 3 and by 'c' at logical time 1 {'sean', [{a, 2}, {c, 1}]}, {'matt', [{b, 1}]}, {'stu', [{c, 2}, {b, 2}]}]]}
Friday, 8 November 13
Sets
'type': 'or-‐set'{ [ {a, 3}, {b, 2}, {c, 2}], %% Version Vector [{ 'bob', [{a, 1}]}, %% Bob was added by 'a' at logical time 1 (a's first event) {'joel', [{a, 3}]}, %% Joel was added by 'a' at logical time 3 and by 'c' at logical time 1 {'sean', [{a, 2}, {c, 1}]}, {'matt', [{b, 1}]}, {'stu', [{c, 2}, {b, 2}]}]]}
Friday, 8 November 13
Sets
'type': 'or-‐set'{ [ {a, 3}, {b, 2}, {c, 2}], %% Version Vector [{ 'bob', [{a, 1}]}, %% Bob was added by 'a' at logical time 1 (a's first event) {'joel', [{a, 3}]}, %% Joel was added by 'a' at logical time 3 and by 'c' at logical time 1 {'sean', [{a, 2}, {c, 1}]}, {'matt', [{b, 1}]}, {'stu', [{c, 2}, {b, 2}]}]]}
VV, element-‐>dots
Friday, 8 November 13
Maps{ 'type': 'map', gold=100, stone = 10, weapons=[sword, knife, shotgun], armour=[helmet, shield]}
Friday, 8 November 13
Maps{ 'type': 'map', gold=100, stone = 10, weapons=[sword, knife, shotgun], armour=[helmet, shield]}
Hey Riak with Map at Key K -‐>
[{increment gold by 10, decrement stone by 4, add uzi to weapons and
remove helmet from armour}]
Friday, 8 November 13
Use-Cases
Advert clicks (G-Counter)
Shopping cart (Modified OR-Set)
Logged in users (P-N Counter)
Maps can be composed of complex data
Friday, 8 November 13
Common Questions
What do they cost me?
How big are they?
What can’t they do?
Friday, 8 November 13
Useful linkshttp://hal.upmc.fr/docs/00/55/55/88/PDF/techreport.pdf
http://arxiv.org/pdf/1210.3368.pdf
https://gist.github.com/russelldb/f92f44bdfb619e089a4d
http://gsd.di.uminho.pt/members/cbm/ps/scadt3.pdf
http://arxiv.org/abs/1011.5808
Friday, 8 November 13
