Eventual Consitency with CRDTS

@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT

Eventual Consistency without Consensus with CRDTs

Sam BESSALAH - @samklr

“ This talk is filled with words and terms, that might make you sound too nerdy or pedantic at dinners with non developers friends. ”

Use with caution.

“ Distributed Programming, generally a bad idea, best avoided. ”

-Peter Bourgon

- Availability- Fault tolerance- Throughtput- Architecture- Economics

Why we use distributed systems?

CAP Theorem

S. Gilbert and N. A. Lynch. Brewer’s conjecture and the feasibility of consistent, available, partition-tolerant web services. SIGACT News, 33(2):51–59, 2002.

http://bravenewgeek.com/from-mainframe-to-microservice-an-introduction-to-distributed-systems/

Data Consistency?

Consensus Systems ,

Locking Services and

“barbaric“ algorithms

Data Consistency?

- Distributed Consensus : Costly and nearly impossible. Multi Phase Commit, Stae Replication

- Two Phase Commit : Blocking, dependent on coordinator, deadlocks inducing

- Three Phase commit : abort on timeouts, non blocking but easily fails on network partitions

Data Consistency?

Data Consistency? RAFT

Data Consistency? Locking Services

Chubby (Google) Zab (Yahoo)

http://bravenewgeek.com/from-mainframe-to-microservice-an-introduction-to-distributed-systems/

@aphyr

“ Consistency is a property of your data, not of your nodes”

@aphyr

AP Systems Eventual Consistency

- High Availability- Low Latency- Fault tolerant

Dynamo Systems

Voldemort Cassandra

Conflicts resolution

- Semantic Resolution

- Vector Clocks

- Last write wins (LWW)

Vector Clocks LWW

Convergent Replicated Data Types

CRDTs- Take the consistency problem to the

level of Data Structures

- Their state resolves automatically (eventually) to a single coherent value

- Maintain multiple copies of your data

CRDTs : Monotonicity

- Every new operation adds information- Data is never immediately destroyed- Most things are trasparent to the

application

CRDTs : 2 types

- State based or convergent ( CvRDTs)

- Commutative or Operations based (CmRDTs)

CvRDTs

- All replicas connected - At least once semantics usually- State Changes advance upwards according to

partial order

CmRDTs

- All replicas connected - Need Reliable broadcast with ordering

guarantees- Best suitable for commutative updates

CRDTs : Fancy words

CRDTs are idempotent, commutative monoids !!!

CRDTs : Fancy words

CRDTs are idempotent, commutative monoid !!!

Joint Semi Lattice

- Idempotence : A * A = A- Commutative : A*B = B* A- Associative : A*(B*C) = (A*B)*C

Joint SemiLattice

CvRDTs

Some common CRDTs

- Registers (LWW, Multi Valued Register)

- G-Counter

- PN Counter

- G-Set, 2P-Set and OR-Set

- Graph

- Maps

G- Counter : grow only counter

PN-Counter

- Positive and Negative Counters- Uses two G-counter- One for increments(P) and another for

decrement(N)- Result is the difference

Grow only set, that only allows to add an element

2P-Set

- Two Phase Set- Built with 2 G-set for add and removal- Can’t add an already removed element :

Tombstone set- Has a tombstone that maintains

deleted elements

LWW-Element-set

- Add Timestamp to “add” and “remove” states wit h a timestamp.

- Greatest timestamps wins

- Close to cassandra model

CRDTs in the Wild

www.jepsen.io

- Riak- Cassandra- Kafka- etc.

Roshihttps://github.com/

soundcloud/roshi

Eventuate

Limitations

- Garbage Collection- Not always easy to have Semi

Lattice for all use case- Inducing some strange behaviours- Might require adding a stronger

consistency models: Raft, Paxos, etc- Use when availability is really

important- Don’t use them for your billing

application- Definitely not a panacea

Eventual Consitency with CRDTS

Software

CRDTs and Coordination Avoidance (Lecture 8, …...CRDTs and Coordination Avoidance (Lecture 8, cs262a) Ion Stoica & Ali Ghodsi UC Berkeley February 12, 2018 Today’s Papers CRDTs:

Dolo Eventual penal sentencia

Tesina Dolo Eventual

Bulletin - Laboratoire d'informatique de Paris 6 · the problem of garbage-collecting CRDTs; an exercise in applying CRDTs to a practical example, the bookstore shopping cart. 2 Background

WHITE PAPER Under the Hood: Redis CRDTs (Conflict-free ...lp.redislabs.com/rs/915-NFD-128/images/WP-RedisLabs-Redis-Conflic… · WHITE PAPER Under the Hood: Redis CRDTs (Conflict-free

The CRDTS Report Report 10-13.pdf · CRDTS Report The 41st CRDTS Annual meeting was held August 22 – 24, 2013 at the Hilton Kansas City Airport Hotel. Over two hundred attend the

RELACIÓN DE CONTRATOS DE PERSONAL SEMANAL EVENTUAL · ortega medina raul 060209 eventual por tiempo determinado padilla rodriguez gonzalo 060209 eventual por tiempo determinado rangel

Catálogo Eventual - Verão 2014

Dolo Eventual y Dolo Directo

2019 - Central Regional Dental Testing Service › uploads › CRDTS 2019 Local Anesthesia Man… · CRDTS 2019 LOCAL ANESTHESIA EXAMINATION SUPPLEMENT TO CANDIDATE’S MANUAL Introduction

Evetrything's Eventual

Coleção Eventual Inverno 2012

Central Regional Dental Testing Service, Inc. The CRDTS Report Newsletter 03-10.pdf · NCR forms introduced for independent evaluations CRDTS computerizes with a custom-written scoring

Understanding Eventual Consistency

Central Regional Dental Testing Service, Inc. The CRDTS Report Newsletter 7-11.pdf · Central Regional Dental Testing Service, Inc. Volume 9, Issue 1 July, 2011 The CRDTS Report

CRDTS’ National Dental Hygiene Examination DentalHygieneTechnicalReport.pdfDescription of the CRDTS National Dental Hygiene Examination ... synonym for testing ... Construct representation

Dolo Eventual e Culpa Consciente

- Dolo Eventual vs. Imprudencia

The CRDTS Report Report 06-13.pdf · The CRDTS Report Central Regional Dental Testing Service, Inc. Summer 2013 Mark G O’Farrell, DDS President’s Message Volume 12, Issue 1 Laura

Catalogo Eventual Inverno 2013