Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
On the Growth of the Prime Numbers Based EncodedVector Clock
Ajay D. Kshemkalyani Bhargav Voleti
University of Illinois at Chicago
Kshemkalyani, Voleti (UIC) Encoded Vector Clock 1 / 22
Overview
1 Introduction
2 Encoded Vector Clock (EVC)Operations on the EVC
3 SimulationsNumber of Events until EVC size exceeds 32nSize of EVC as a function of Number of EventsNumber of Events until Overflow 32n bits (function of Event Types)
4 Scalability of EVCs
5 Case Study
6 Discussion and Conclusions
Kshemkalyani, Voleti (UIC) Encoded Vector Clock 2 / 22
Introduction
Scalar clocks: e → f ⇒ C (e) < C (f )
Vector clocks: e → f ⇐⇒ V (e) < V (f )
Fundamental tool to characterize causalityTo capture the partial order (E ,→), size of vector clock is the dimension ofthe partial order, bounded by the size of the system, nNot scalable!
encoding of vector clocks (EVC) using prime numbers to use a single numberto represent vector time
big integer EVC grows fast, eventually exceeds size of vector clock
ContributionEvaluate and analyze the growth rate of EVC using simulations
Kshemkalyani, Voleti (UIC) Encoded Vector Clock 3 / 22
Vector Clock Operation at a Process Pi
1 Initialize V to the 0-vector.
2 Before an internal event happens at process Pi , V [i ] = V [i ] + 1 (local tick).
3 Before process Pi sends a message, it first executes V [i ] = V [i ] + 1 (localtick), then it sends the message piggybacked with V .
4 When process Pi receives a message piggybacked with timestamp U, itexecutes∀k ∈ [1 . . . n],V [k] = max(V [k],U[k]) (merge);V [i ] = V [i ] + 1 (local tick)before delivering the message.
Kshemkalyani, Voleti (UIC) Encoded Vector Clock 4 / 22
Encoded Vector Clock (EVC) and Operations
A vector clock V = 〈v1, v2, · · · , vn〉 can be encoded by n distinct primenumbers, p1, p2, · · · , pn as:
Enc(V ) = pv11 ∗ p
v22 ∗ · · · ∗ p
vnn
EVC operations: Tick, Merge, Compare
Tick at Pi : Enc(V ) = Enc(V ) ∗ pi
Kshemkalyani, Voleti (UIC) Encoded Vector Clock 5 / 22
EVC Operations (contd.)
Merge: For V1 = 〈v1, v2, · · · , vn〉 and V2 = 〈v ′1, v ′2, · · · , v ′n〉, merging yields:
U = 〈u1, u2, · · · , un〉, where ui = max(vi , v′i )
The encodings of V1, V2, and U are:
Enc(V1) = pv11 ∗ p
v22 ∗ · · · ∗ p
vnn
Enc(V2) = pv ′
11 ∗ p
v ′2
2 ∗ · · · ∗ pv ′n
n
Enc(U) =n∏
i=1
pmax(vi ,v
′i )
i
However,
Enc(U) = LCM(Enc(V1),Enc(V2)) =Enc(V1) ∗ Enc(V2)
GCD(Enc(V1),Enc(V2))
Kshemkalyani, Voleti (UIC) Encoded Vector Clock 6 / 22
EVC Operations (contd.)
Compare:
i) Enc(V1) ≺ Enc(V2) if Enc(V1) < Enc(V2) and
Enc(V2) mod Enc(V1) = 0
ii) Enc(V1)‖Enc(V2) if Enc(V1) 6≺ Enc(V2) and
Enc(V2) 6≺ Enc(V1)
Thus, to manipulate the EVC,
Each process needs to know only its own prime
Merging EVCs requires computing LCM
Use Euclid’s algorithm for GCD, which does not require factorization
Kshemkalyani, Voleti (UIC) Encoded Vector Clock 7 / 22
Correspondence of Operations
Table: Correspondence between vector clocks and EVC.
Operation Vector Clock Encoded Vector Clock
Representing clock V = 〈v1, v2, · · · , vn〉 Enc(V ) = pv11 ∗ p
v22 ∗ · · · ∗ p
vnn
Local Tick V [i ] = V [i ] + 1 Enc(V ) = Enc(V ) ∗ pi(at process Pi )Merge Merge V1 and V2 yields V Merge Enc(V1) and Enc(V2) yields
where V [j] = max(V1[j],V2[j]) Enc(V ) = LCM(Enc(V1), Enc(V2))Compare V1 < V2: Enc(V1) ≺ Enc(V2):
∀j ∈ [1, n], V1[j] ≤ V2[j], Enc(V1) < Enc(V2),and ∃j , V1[j] < V2[j] and Enc(V2) mod Enc(V1) = 0
Kshemkalyani, Voleti (UIC) Encoded Vector Clock 8 / 22
Operation of the Encoded Vector Clock
1 Initialize ti = 1.
2 Before an internal event happens at process Pi ,ti = ti ∗ pi (local tick).
3 Before process Pi sends a message, it first executes ti = ti ∗ pi (localtick), then it sends the message piggybacked with ti .
4 When process Pi receives a message piggybacked with timestamp s, itexecutesti = LCM(s, ti ) (merge);ti = ti ∗ pi (local tick)before delivering the message.
Figure: Operation of EVC ti at process Pi .
Kshemkalyani, Voleti (UIC) Encoded Vector Clock 9 / 22
Illustration of Using EVC
3240
P
P
P1
2
3
2
3
5
primenumber
[1,0,0] [2,0,1] [3,0,1]
[1,2,0] [1,3,0]
[0,0,1] [1,3,2]
[3,4,1]
2 20 40
18 54
5 1350
3
[0,1,0]
Figure: The vector timestamps and EVC timestamps are shown above and below eachtimeline, respectively. In real scenarios, only the EVC is stored and transmitted.
Kshemkalyani, Voleti (UIC) Encoded Vector Clock 10 / 22
Simulation Assumptions
Simulated distributed executions with a random communication pattern
prs : probability of a send (versus internal) event
Used first n primes for the n processes
Overflow process: that process which is earliest to have its EVC size exceed32n bits
Kshemkalyani, Voleti (UIC) Encoded Vector Clock 11 / 22
Number of Events until EVC Size exceeds 32n bits
Figure: Average of 10 runs. prs= 0.6.
Typically, 21-25 events/process before EVC size exceeded 32nKshemkalyani, Voleti (UIC) Encoded Vector Clock 12 / 22
Strawman Analysis
prs = 0.6 implies every third event is a receive event.
Consider n = 60. 60 lowest prime numbers needs 8 bit representation.
At each event, EVC size increases by 8 bits (local tick)
At a receive event, (every 3rd event), size of EVC doubles due to LCM
Worst-case progression of size of EVC in bits approx. as:
8, 16, 32 and 40 (event e3i ), 48, 56, 112 and 120 (event e6
i ),
128, 136, 272 and 280 (event e9i ), 288, 296, 592 and 600 (event e12
i ),
608, 616, 1232 and 1240 (event e15i ), 1248, 1256, 2512 and 2520 (event e18
i )
At the 18th event at Pi , the EVC size exceeds 60× 32 = 1920 bits
As per simulation, overflow happens at the 1250/60th event, or the 21stevent, at the overflow process
Kshemkalyani, Voleti (UIC) Encoded Vector Clock 13 / 22
Size of EVC as a Function of Number of Events
Figure: prs= 0.5.
About 900 events until EVC size reached 960 (= 30× 32) bits at overflowprocessKshemkalyani, Voleti (UIC) Encoded Vector Clock 14 / 22
Size of EVC as a Function of Number of Events
Figure: prs= 0.5.
About 1800 events until EVC size reached 1920 (= 60× 32) bits at overflowprocessKshemkalyani, Voleti (UIC) Encoded Vector Clock 15 / 22
Size of EVC as a Function of Number of Events
Figure: prs= 0.5.
About 3000 events until EVC size reached 3200 (= 100× 32) bits at overflowprocessKshemkalyani, Voleti (UIC) Encoded Vector Clock 16 / 22
Number of Events until Overflow 32n bits (function ofEvent Types)
0
2000
4000
6000
8000
10000
10 20 30 40 50 60 70 80 90
Num
ber
of e
vent
s un
til E
VC
siz
e is
32n
bits
The percentage of internal events
n = 10n = 40n = 60n = 80
Figure: Varied percentage of internal events (out of internal, send, and receive events)
Receive events cause EVC to grow very fast due to LCM
Kshemkalyani, Voleti (UIC) Encoded Vector Clock 17 / 22
Strawman Analysis
Consider n = 60, and prob(int. event) = 0.9, prob(receive event) = 0.05
For n = 60, 60 lowest prime numbers needs 8 bit representation.
At each event, EVC size increases by 8 bits (local tick)
At a receive event, (every 20th event), size of EVC doubles due to LCM
Worst-case progression of size of EVC in bits approx. as:
8, · · · 152, 304 and 312 (event e20i ),
320, · · · 464, 928 and 936 (event e40i ),
944, · · · 1088, 2176 and 2184 (event e60i )
At the 60th event at Pi , or 3600th event in execution, the EVC size exceeds60× 32 = 1920 bits
As per simulation, overflow happens at the 6000th event. Apply correction:
In the initial window before steady state, more than 20 non-receive events perreceive event
Kshemkalyani, Voleti (UIC) Encoded Vector Clock 18 / 22
Scalability of EVCs
EVC timestamps grow very fast. To alleviate this problem:1 Tick only at relevant events, e.g., when the variables alter the truth value of
a predicate
On social platforms, e.g., Twitter and Facebook, max length of any chain ofmessages is usually small
2 Application requiring a vector clock is confined to a subset of processes
3 Reset the EVC at a strongly consistent (i.e., transitless) global state4 Use logarithms to store and transmit EVCs
Local tick: single additionMerge and Compare: Take anti-logs and then logs,
complexity is subsumed by that of GCD computationextra space is only scratch space
Kshemkalyani, Voleti (UIC) Encoded Vector Clock 19 / 22
Case Study
Detecting memory consistency errors in MPI one-sided applications using EVC inthe MC-CChecker tool
Relevant events were the synchronization events; only these weretimestamped
Each concurrent region in the code was a unit of computation; boundarybetween two concurrent regions corresponded to a global transitless state
MC-CChecker safely reset EVCs at the start of each concurrent region
Execution time and memory usage using EVC in MC-CChecker were muchlower than using traditional vector clocks
Kshemkalyani, Voleti (UIC) Encoded Vector Clock 20 / 22
Conclusions
Studied the encoding of vector clocks using prime numbers, to use a singlenumber to represent vector time
Simulations show that the single integer EVCs grow fast
Analyzed the growth rateReceive events cause EVCs to grow much faster due to LCMProposed several solutions to deal with this problem
Tick at relevant events; detection regions; reset EVC at transitless global state;use logs of EVCs
Case study: Detecting memory consistency errors in MPI one-sidedapplications
Using EVCs far more memory- and time-efficient than using traditional vectorclocks
Kshemkalyani, Voleti (UIC) Encoded Vector Clock 21 / 22
Thank You!
Kshemkalyani, Voleti (UIC) Encoded Vector Clock 22 / 22