Upload
primalanthony
View
236
Download
1
Embed Size (px)
Citation preview
7/31/2019 Primal Ladis
1/30
DISTRIBUTED INDEXING AND LOCKING :IN SEARCH OF SCALABLE CONSISTENCY
Mahdi Tayarani Najaran, Primal Wijesekera, AndrewWarfield and Norman C. Hutchinson
Networks, Systems,andSecurity Lab (NSS)
7/31/2019 Primal Ladis
2/30
Why isnt there just one instance ofgiant SQL Database ?
2
7/31/2019 Primal Ladis
3/30
SQL
3
7/31/2019 Primal Ladis
4/30
Alternative - 01
Data Partitioning
4
7/31/2019 Primal Ladis
5/30
Alternative - 02
NOSQLKV StorageSystems
5
7/31/2019 Primal Ladis
6/30
What about SQL
Highly Optimized Sound mathematical base
Highly adopted across domains Very rich query model
6
7/31/2019 Primal Ladis
7/30
What is wrong with SQL
Why we think that we can do better
Our Solution
Evaluation
What is next
7
7/31/2019 Primal Ladis
8/30
We think
SQL is just great but
Developers arent
8
7/31/2019 Primal Ladis
9/30
We have a great platform
SinfoniaEx: A Distributed Transactional Memory Service An open source implementation of Sinfonia
9
7/31/2019 Primal Ladis
10/30
Minitransaction
A given minitransaction has A read set A compare set A conditional write set
All updates occur atomically only if all comparisonssucceed
Locks are held only for a very small time
10
7/31/2019 Primal Ladis
11/30
What is wrong with SQL
Why we think that we can do better
Our Solution
Evaluation
What is next
11
7/31/2019 Primal Ladis
12/30
Replacethe single-nodestoragelayer of an off-the-shelf DBMS engine(eg. MySQL) with a distributed storage
layer
12
7/31/2019 Primal Ladis
13/30
The Design
13
7/31/2019 Primal Ladis
14/30
The Design contd.
14
7/31/2019 Primal Ladis
15/30
Why it is hard to do so
Distributed Indexing With high expressibility
Distributed Locking With complex range locking
( Distributed Storage )
Fundamental Requirements Distributed Scalability even in the presence of write heavy workloads Fault tolerance Strict consistency guarantees
15
7/31/2019 Primal Ladis
16/30
Getting Expressibility
B-Tree Single-attribute range queries Suffers for write-intense workloads Balanced tree
Spatial Partitioning Tree Multi-attribute range queries Scales for all types of workloads Un-balanced tree
We picked Spatial Partitioning Tree (SPT) over B-Tree
Store data only in leaf nodes to avoid contention SPTs increase partitioning granularity by re-partitioningleaves when required
16
7/31/2019 Primal Ladis
17/30
3-D Spatial Partitioning Tree
src : wikipedia.org
17
18
7/31/2019 Primal Ladis
18/30
SPT for Indexing
ALTER TABLE orders
ADD CONSTRAINTpkey_ordersPRIMARY KEY(o_w_id, o_d_id, o_id);
18
19
7/31/2019 Primal Ladis
19/30
SPT for Locking Manager
Modified SPT from Index Manager Key is a range. Range insert => lock acquiring Range removal => lock releasing
Supports both records locking and complex rangelocking.
All other locking semantics are intact.
Read and write locks.
19
20
7/31/2019 Primal Ladis
20/30
SinExTree
Is the SPT implementation on top of SinfoniaEx Gives a transactionaly consistent and scalable approach Fulfills all the fundamental requirements
20
21
7/31/2019 Primal Ladis
21/30
Minitransactions in SPT
Writing a value to Node L
In this instance, Compare set = {A, C, E, H, J, L} Conditional write set = {L}
21
22
7/31/2019 Primal Ladis
22/30
Final Architectural Setup
22
23
7/31/2019 Primal Ladis
23/30
What is wrong with SQL
Why we think that we can do better
Our Solution
Evaluation
What is next
23
24
7/31/2019 Primal Ladis
24/30
Evaluation Setup
The main objective is to measure the throughput of the indexunder maximum stress at different scales
Log and replay Logged the index operations of TPC-C benchmark Replayed logs on the distributed index
Used Amazon EC2 using 72 High CPU Extra Large instances
Varied the number of index hosts from 1 to 64 with 8 fixed
memnodes
24
25
7/31/2019 Primal Ladis
25/30
Evaluation ResultsThroughput
25
26
7/31/2019 Primal Ladis
26/30
Evaluation ResultsThroughput
26
7/31/2019 Primal Ladis
27/30
28
7/31/2019 Primal Ladis
28/30
What is wrong with SQL
Why we think that we can do better
Our Solution
Evaluation
What is next
28
29
7/31/2019 Primal Ladis
29/30
Next
Future Work Integrate all the components into one system Use this setup for a DBMS as a Service
Future Challenges Making sure the correctness is preserved through out all layers Implementing Security measures to guarantee privacy Define a pricing model with a emphasize on fluctuating workloads
29
30
7/31/2019 Primal Ladis
30/30
Thank You !!!
30