Sequential reductions for verifying serializabilityHagit Attiya Technion & EPFLG. Ramalingam MSR IndiaNoam Rinetzky University of London

The goal

Verify concurrent data structures• Pre-execution static analysis

E.g., linked list with hand-over-hand locking

• memory safety (no memory leaks)• shape invariants (it’s a list)• “atomicity” (serializability)

Find sequential reductions Consider only sequential executions But conclude that properties hold in all

executions

Back-of-envelop estimate of gainStatic analysis of a linked-list

algorithm[Amit, Rinetzky, Reps, Sagiv, Yahav, CAV 2007]

– Verifies e.g., memory safety, sortedness, pointed-to by a variable, heap sharing One thread (sequential) 10s 3.6MB

Two threads (interleaved) ~4h 886MB

Three threads (interleaved) > 8h ----

Serializability

operation

interleaved execution

complete non-interleaved execution

~~~~~~ ~~~ ~ thread-local views

[Papadimitriou ‘79]

Serializability assists verificationConcurrent code M Π = all executions of Mcni-Π = all complete non-interleaved

executions of M Πφ = some thread-local

propertyIf M is serializableThen Π ⊨ φ cni-Π ⊨ φ

How do we know that M is serializable, w/o considering all executions?I.e., by checking only cni-executionsIf M is serializableThen Π ⊨ φ cni-Π ⊨ φ

Guard access to data with locks Access data only after getting a lock for it

(well-lockedness)

Follow a locking protocol that guarantees

conflict serializability

E.g., two-phase locking (2PL) or tree locking (TL)

Special (and common) case: Disciplined programming with locks

Two-phase locking

[Papadimitriou `79]

• Locks acquire (grow) phase followed by locks release (shrink) phaseNo lock is acquired after some lock is

released

t1

H

t1t1

t2

t1

Tree (hand-over-hand) locking[Bayer & Scholnick ‘77] [Kedem & Sliberschatz ‘76] [Smadi ‘76]

• Except for the first lock, acquire a lock only when holding the lock on its parent

• No lock is acquired after being released

t1

H

t1t1

t2

Tree (hand-over-hand) locking[Bayer & Scholnick ‘77] [Kedem & Sliberschatz ‘76] [Smadi ‘76]

• Except for the first lock, acquire a lock only when holding the lock on its parent

• No lock is acquired after being released

t1

t2

t2

H

t1

Yes!– for databases– concurrency control monitor

ensures that M follows the locking policy at run-time M is serializable

No!– for static analysis– no central monitor

void p() { acquire(B)B = 0int b = Brelease(B)if (b) Y = 2

acquire(A)}

void q() {acquire(B)B = 1

release(B)}

Not two-phase lockedBut only in interleaved executions

How to (statically) verify that M follows a locking policy?

Our approach

Applies for local conflict-serializable (LCS) locking protocols – Use only thread-local information (see

next)

E.g., two phase locking, tree locking, (dynamic) DAG locking…

But not protocols that rely on a concurrency control monitor!

Thread-local properties

A thread-owned view contains the values of thread’s local variables & global variables locked by it

A property φ is thread-local if it – Can be expressed in terms of thread-

owned views – Is prefix closed

A thread-local property of an execution holds in every execution indistinguishable from it

Our contribution: Easy step

cni-Π = all complete non-interleaved executions of M Π

For any LCS locking policy LPΠ ⊨ LP ni-Π ⊨ LP

non-interleaved execution

For any thread-local property φΠ ⊨ φ ni-Π ⊨ φ

Two phase locking

Tree lockingDynamic tree

lockingDynamic DAG

locking

If all ni-executions follow LP all executions follow LP

σ is the shortest execution that does not follow LP

σ’ follows LP, guarantees conflict-serializabilitythere is a ni-execution that is “equivalent” to

σ’

Ni-reduction: Proof sketch

σ (t,e)

σ’

Ni-reduction: Proof sketch (cont.)there is a ni-execution that is “equivalent” to

σ’

there is a ni-execution that is “equivalent” to

σwhere LP is violated

σ’

σ’ni

σni

(t,e)

(t,e)

acni-Π = all almost-complete non-interleaved executions of M Π

For any LCS locking policy LPΠ ⊨ LP acni-Π ⊨ LP

almost complete non-interleaved execution

Further reduction

Need to argue about termination

Acni-reduction: A complication

int X=0, Y=0

void p() {acquire(Y)y = Yrelease(Y); if (y ≠ 0)

acquire(X)X = 3release(X)

}

void q() {if (random(5) == 3){

acquire(Y)Y = 1release(Y)while (true) nop

}}

Y is set to 1 & the method

enters an infinite

loop

Observe Y == 1 & violates

2PL

Can use sequential reduction to verify termination

For any “terminating” LCS locking policy LPΠ ⊨ LP acni-Π ⊨ LP

Acni-reduction: Termination

Acni-reduction: Proof ideas

Start from a ni-execution (and rely on the previous, ni-reduction to get there)

Create its equivalent completion, if possible

Not always possible, e.g.,

Does not access variables accessed by later threads

t1:lock(v), t1:lock(u), t2:lock(u) u

v

Implications for statis analysis• Pessimistic analysis (over approximate)

– Analyze a module from every possible state

• Semi-optimistic analysis– Analyze a module only from states that occur

after a sequence of modules ran one after the other

• Optimistic analysis (precise)– Analyze a module only from states that occur

after a sequence of modules ran to completion (one after the other) Acni-reduction

Ni-reduction

Initial analysis results

Shape analysis of hand-over-hand lists

* Does not verify sortedness of list and fails to verify linearizability in some cases

Shape analysis of hand-over-hand trees (for the first time)

Our method 3.5s 4.0MBTVLA prior 596.1s 90.3MBSeparation logic*

0.4s 0.2MB

Our method 124.6s 90.6MB

What’s next?

• Further extensions, esp., for software transactional memory– aborted transactions– non-locking non-conflict based

serializability (e.g., using timestamps)

• Combine with other reductions [Guerraoui, Henzinger, Jobstmann, Singh]

And beyond…

• The cost of verifying adherence to a locking policy in an acni- / ni-execution

• Automatic insertion of lock acquire / release commands… Or fences?

Thank you!