Apache Cassandra, part 1 – principles, data model

April 9, 2023 www.ExigenServices.com

Apache Cassandra, part 1 – principles, data model

2 www.ExigenServices.com

I. RDBMS Pros and Cons


Pros

1. Good balance between functionality and usability. Powerful tools support.

2. SQL has feature rich syntax

3. Set of widely accepted standards.

4. Consistency


Scalability

RDBMS were mainstream for tens years till requirements for scalability were increased dramatically.

Complexity of processed data structures was increased dramatically.


Scaling

Two ways to achieve scalability:

– Vertical scaling– Horizontal scaling


CAP Theorem


Cons

Cost of distributed transactions

a) No availability support . Two DB with 99.9% have availability 100% - 2 * (100% - DB availability) = 99.8% (43 min. downtime per month).

b) Additional synchronization overhead.

c) As slow as slowest DB node + network latency.

d) 2PC is blocking protocol.

e) It is possible to lock resources forever.


Cons

Usage of master - slave replication.

Makes write side (master) performance bottleneck and requires additional CPU/IO resources.

There is no partition tolerance.


Sharding

a) Feature sharding

b) Hash code sharding

c) Lookup table - Node that contains lookup table is performance bottleneck and single point of failure.


Feature sharding

DB instances are divided by DB functions.


Hash code sharding

Data is divided through DB instances by hash code ranges.


Sharding consistency

For efficient sharding data should be eventually consistent.


Feature vs. hash code sharding

Feature sharding allows to perform consistency tuning on the domain logic granularity. But load may be not well balanced.

Hash code sharding allows to perform good load balancing but does not allow consistency on domain logic level.


Cassandra sharding

Cassandra uses hash code load balancing

Cassandra better fits for reporting than for business logic processing.

Cassandra + Hadoop == OLAP server with high performance and availability.


II. Apache Cassandra. Overview


Cassandra

Amazon Dynamo

(architecture)

DHT Eventual consistency Tunable trade-offs,

consistency

Google BigTable(data model)

Values are structured and indexed

Column families and columns

+


Distributed and decentralized

No master/slave nodes (server symmetry) No single point of failure


DHT

Distributed hash table (DHT) is a class of a decentralized distributed system that provides a lookup service similar to a hash table; (key, value) pairs are stored in a DHT, and any participating node can efficiently retrieve the value associated with a given key.


DHT

Keyspace Keyspace partitioning Overlay network


Keyspace

Abstract keyspace, such as the set of 128 or 160 bit strings.

A keyspace partitioning scheme splits ownership of this keyspace among the participating nodes.


Keyspace partitioning

Keyspace distance function δ(k1,k2)

A node with ID ix owns all the keys km for which ix is the closest ID, measured according to δ(km,ix).



Imagine mapping range from 0 to 2128 into a circle so the values wrap around.



Consider what happens if node C is removed



Consider what happens if node D is added


Overlay network

For any key k, each node either has a node ID that owns k or has a link to a node whose node ID is closer to k

Greedy algorithm (that is not necessarily globally optimal): at each step, forward the message to the neighbor whose ID is closest to k


Elastic scalability

Adding/removing new node doesn’t require reconfiguring of Cassandra, changing application queries or restarting system


High availability and fault tolerance

Cassandra picks A and P from CAP Eventual consistency


Tunable consistency

Replication factor (number of copies of each piece of data)

Consistency level (number of replicas to access on every read/write operation)

Consistency level Read / Write

ONE 1 replica

QUORUM N/2 + 1

ALL N


Quorum consistency level

R = N/2 + 1W = N/2 + 1R + W > N


Hybrid orientation

Column orientation– columns aren’t fixed– columns can be sorted– columns can be queried for a certain range

Row orientation– each row is uniquely identifiable by key– rows group columns and super columns


Schema-free

You don’t have to define columns when you create data model

You think of queries you will use and then provide data around them


III. Data Model


Table1 Table2

Database

Relational data model

Column1 Column2

Row1 value value

Row2 null value

…

Column1 Column2 Column3

Row1 value value value

Row2 null value null

…


Cassandra data model

Keyspace

Column Family

RowKey1

RowKey2

Column1 Column2 Column3

Value3Value2Value1

Value4Value1

Column4Column1


Keyspace

Keyspace is close to a relational database

Basic attributes:– replication factor– replica placement strategy– column families (tables from relational model)

Possible to create several keyspaces per application (for example, if you need different replica placement strategy or replication factor)


Column family

Container for collection of rows Column family is close to a table from relational

data model

Column Family

Row

RowKeyColumn1 Column2 Column3

Value3Value2Value1


Column family vs. Table

Store represents four-dimensional hash map[Keyspace][ColumnFamily][Key][Column]

The columns are not strictly defined in column family and you can freely add any column to any row at any time

A column family can hold columns or super columns (collection of subcolumns)


Column family vs. Table

Column family has an comparator attribute which indicated how columns will be sorted in query results (according to long, byte, UTF8, etc)

Each column family is stored in separate file on disk so it’s useful to keep related columns in the same column family


Column

Basic unit of data structure

Column

name: byte[] value: byte[] clock: long


Skinny and wide rows

Wide rows – huge number of columns and several rows (are used to store lists of things)

Skinny rows – small number of columns and many different rows (close to the relational model)


Disadvantages of wide rows

Badly work with RowCash

If you have many rows and many columns you end up with larger indexes

(~ 40GB of data and 10GB index)


Column sorting

Column sorting is typically important only with wide model

Comparator – is an attribute of column family that specifies how column names will be compared for sort order


Comparator types

Cassandra has following predefined types:– AsciiType– BytesType– LexicalUUIDType– IntegerType– LongType– TimeUUIDType– UTF8Type


Super column

Super column

name: byte[] cols: Map<byte[], Column>

• Cannot store map of super columns (only one level deep)

• Five-dimensional hash:

[Keyspace][ColumnFamily][Key][SuperColumn][SubColumn]

Stores map of subcolumns


Super column

Sometimes it is useful to use composite keys instead of super columns.• Necessity more then one level depth• Performance issues


Super column family

Column families:

– Standard (default)

Can combine columns and super columns

– Super

More strict schema constraints

Can store only super columns

Subcomparator can be specified for subcolumns


Note that

There are no joins in Cassandra, so you can

– join data on a client side– create denormalized second column family


IV. Advanced column types


TTL column type

TTL column is column value of which expires after given period of time.

Useful to store session token.


Counter column

In eventual consistent environment old versions of column values are overridden by new one, but counters should be cumulative.

Counter columns are intended to support increment/decrement operations in eventual consistent environment without losing any of them.


CounterColumn internals

CounterColumn structure:

name

…….

[

(replicaId1, counter1, logical clock1),

(replicaId2, counter2, logical clock2),

………………..

(replicaId3, counter3, logical clock3)

]


CounterColumn write - before

UPDATE CounterCF SET count_me = count_me + 2

WHERE key = 'counter1‘

[

(A, 10, 2),

(B, 3, 4),

(C, 6, 7)

]


CounterColumn write -after

A is leader

[

(A, 10 + 2, 2 + 1),

(B, 3, 4),

(C, 6, 7)

]


CounterColumn Read

All Memtables and SSTables are read through using following algorithm:

All tuples with local replicaId will be summarized, tuple with maximum logical clock value will be chosen for foreign replica.

Counters of foreign replicas are updated during read repair , during replicate on write procedure or by AES


CounterColumn read - example

Memtable - (A, 12, 4) (B, 3, 5) (C, 10, 3) SSTable1 – (A, 5, 3) (B, 1, 6) (C, 5, 4) SSTable2 – (A, 2, 2) (B, 2, 4) (C, 6, 2)

Result:

(A, 19, 9) + (B, 1,6) + (C, 5, 4) =19 + 1 + 5 = 25


Resources

Home of Apache Cassandra Project http://cassandra.apache.org/

Apache Cassandra Wiki http://wiki.apache.org/cassandra/ Documentation provided by DataStax

http://www.datastax.com/docs/0.8/ Good explanation of creation secondary indexes

http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html

Eben Hewitt “Cassandra: The Definitive Guide”, O’REILLY, 2010, ISBN: 978-1-449-39041-9

http://cassandra.apache.org/

http://wiki.apache.org/cassandra/

http://www.datastax.com/docs/0.8/




Authors

Lev Sivashov - [email protected]

Andrey Lomakin - [email protected], twitter: @Andrey_LomakinLinkedIn: http://www.linkedin.com/in/andreylomakin

Artem Orobets – [email protected]: @Dr_EniSh

Anton Veretennik - [email protected]

mailto:[email protected]


http://www.linkedin.com/in/andreylomakin



Technology

Apache Cassandra, part 1 – principles, data model