A Contention-Sensitive Fine-Grained Locking Protocol forMultiprocessor Real-Time Systems∗

Catherine E. Jarrett, Bryan C. Ward, and James H. AndersonDept. of Computer Science, University of North Carolina at Chapel Hill

ABSTRACTPrior work on multiprocessor real-time locking protocolshas shown how to support fine-grained lock nesting withasymptotically optimal worst-case priority-inversion blocking(pi-blocking) bounds. However, contention for each resourcehas heretofore been considered an unconstrained variable.This paper presents the first fine-grained multiprocessor real-time locking protocol with contention-sensitive worst-casepi-blocking bounds. Contention-sensitive pi-blocking is madepossible by incorporating knowledge of maximum critical-section lengths—which must be known a priori for analysisanyway—into the lock logic.

1. INTRODUCTIONWhen designing software for a multiprocessor platform, the

need for efficient synchronization looms large. In the domainof real-time systems, multiprocessor locking protocols existthat can be used to realize synchronization requirements,but protocols that can be flexibly applied and do not overlyinhibit parallelism have remained somewhat elusive. This ismainly due to difficulties caused by nested lock requests.

There are two principal means for supporting nested locks:coarse-grained locking and fine-grained locking. Under coarse-grained locking, all resources that potentially could be lockedsimultaneously by any task are statically coalesced underone lock. This approach clearly inhibits parallelism. How-ever, under the alternative of fine-grained locking, whereinresources are locked individually, correctness-related issuessuch as deadlock-avoidance become problematic.

Perhaps because of such difficulties, the first fine-grainedmultiprocessor real-time locking protocol was proposed onlyrecently, in the form of the real-time nested locking protocol(RNLP ) [12, 14]. When expressed in terms of the proces-

∗Work supported by NSF grants CNS 1115284, CNS 1218693,CPS 1239135, CNS 1409175, and CPS 1446631, AFOSR grantFA9550-14-1-0161, ARO grant W911NF-14-1-0499, and agrant from General Motors. The second author was supportedby an NSF graduate research fellowship.

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than theauthor(s) must be honored. Abstracting with credit is permitted. To copy otherwise, orrepublish, to post on servers or to redistribute to lists, requires prior specific permissionand/or a fee. Request permissions from permissions@acm.org.

RTNS 2015, November 04 - 06, 2015, Lille, Francec© 2015 Copyright held by the owner/author(s). Publication rights licensed to ACM.

ISBN 978-1-4503-3591-1/15/11. . . $15.00

DOI: http://dx.doi.org/10.1145/2834848.2834874

sor count, m, and the task count, n, the RNLP has worst-case priority-inversion blocking (pi-blocking) bounds that areasymptotically optimal. However, these bounds are actuallynot an improvement over those possible under coarse-grainedlocking. This may seem surprising, given the improved run-time parallelism afforded by fine-grained locking. A key mo-tivation for designing a fine-grained protocol is to avoid theartificial inflation of lock contention caused by coarse-grainedlocking. Clearly, analysis that does not consider contentionas a first-class parameter (like m and n) cannot demonstratecontention-related benefits. Similarly, protocols that are notdesigned to be contention sensitive in the worst case will notexhibit worst-case pi-blocking improvements.

Realizing this, we consider for the first time in this paperthe issue of contention sensitivity in the design of multi-processor real-time locking protocols. We show that suchprotocols do exist by presenting an asymptotically optimalvariant of the RNLP that is contention sensitive. A numberof challenges arise when attempting to design a contention-sensitive real-time locking protocol. We discuss some of thesechallenges below, and how they were addressed in the proto-col proposed herein. To provide needed context, however, wefirst briefly review prior related work.

Prior related work. While the first multiprocessor real-time locking protocols were presented over two decadesago [9, 10], the first protocols to be asymptotically opti-mal with respect to pi-blocking began to appear only a fewyears ago [2, 3, 4], and a contention-sensitive protocol hasnever been presented. The aforementioned RNLP is the onlymultiprocessor real-time locking protocol proposed to datethat supports fine-grained lock nesting [12, 14]. The RNLPis actually a family of protocols, where specific variants areobtained by determining how waiting is realized (spinning vs.suspending), the mechanism used to ensure that tasks makeprogress, and how pi-blocking is analyzed. (A more detaileddescription of the RNLP is given later in Sec. 2.) Worst-casepi-blocking under the RNLP is either O(m) or O(n) per re-source request, depending on analysis assumptions. As notedabove, these worst-case bounds are asymptotically optimalwhen such bounds are expressed in terms of m and n only.Intuitively, the RNLP achieves optimality by delaying thesatisfaction of some lock requests to ensure that earlier-issuedoutermost requests are never blocked by later-issued ones.Wieder and Brandenburg [15] recently showed that the prob-lem of obtaining precise worst-case pi-blocking bounds forFIFO- or priority-ordered nested locks is NP-hard. This re-sult does not directly apply to a protocol such as the RNLPthat can opt to delay the satisfaction of some requests.

R3

R1

R2R2

R3

`a `b `c `d

R4R4Proposed C-RNLP cuttingahead. Acquisition time forR3 would not be delayed.

Wait queues

Resource holder(s)

Figure 1: Illustration of the worst-case blocking behav-

ior of the RNLP and the proposed improvement in the

C-RNLP. R4 is transitively blocked by R1 even though

they do not conflict unless R4 cuts ahead of R3.

Key problem: transitive blocking chains. The delay-ing of some requests in the RNLP gives rise to the possibilityof long transitive blocking chains, which are a major compli-cation in the design of a contention-sensitive locking protocol.Consider the example in Fig. 1, which depicts four requestsR1, . . . ,R4 contending for four resources `a, . . . , `d. Blockedrequests wait within per-resource FIFO queues. In the de-picted situation, R1 has acquired `d, R2 is waiting to acquireboth `c and `d (and is enqueued on the waiting queue forboth resources), R3 is waiting to acquire both `b and `c, andR4 is waiting to acquire both `a and `b. In this example,despite the fact that R1 and R4 do not conflict, or access acommon resource, they are serialized due to requests R2 andR3, forming a transitive blocking chain. Such transitive block-ing chains are the very reason why the current RNLP is notcontention sensitive. Observe that the worst-case pi-blockingfor R4 is not upper bounded by the worst-case contentionfor the resources it accesses, `a and `b, but is a function ofthe maximum number of queued requests.

Contributions. We present the C-RNLP, a contention-sensitive variant of the RNLP. Under the C-RNLP, per-request pi-blocking bounds are asymptotically optimal whensuch bounds are expressed as a function of m, n, and lockcontention. In describing the C-RNLP, we mainly focus on aspin-based variant due to space constraints, but variants canbe defined corresponding to all RNLP variants.

The key new idea in the C-RNLP is a “cutting ahead”mechanism that breaks long transitive blocking chains. Indesigning such a mechanism, care must be taken to ensurethat requests that are cut ahead of do not experience in-creased worst-case blocking. The C-RNLP addresses thisconcern through the incorporation of critical-section lengthinformation into the lock and unlock logic. Such informationis necessary for a priori worst-case blocking and schedulabil-ity analysis, so it is reasonable to assume lock requests couldbe annotated with such information.

The C-RNLP’s lock/unlock logic is more complicated thanthe RNLP’s. To address this concern, we implemented aspin-based variant of the C-RNLP and conducted an exper-imental evaluation on a dual-socket 36-core Intel machine.Our experimental results demonstrate a clear trade-off be-tween lock/unlock overheads and the increased parallelismafforded by using a contention-sensitive locking protocol.These experiments show that the additional overheads canbe worth incurring if nesting is common.

Organization. In the rest of the paper, we provide neededbackground (Sec. 2), describe the C-RNLP, first in a rather

abstract way (Sec. 3) and then in a more implementation-oriented way (Sec. 4), present our experimental results (Sec. 5),and finally, conclude (Sec. 6).

2. BACKGROUNDIn this section, we provide relevant background material.

Task model. We consider the classic sporadic real-timetask model and assume familiarity with this model. Specifi-cally, we consider a task system Γ = {τ1, . . . , τn} scheduledon m processors. We denote an arbitrary job of τi as Ji. Welimit our attention to job-level fixed-priority schedulers, suchas static-priority and earliest-deadline-first (EDF) schedulers,under which the base priority of each job is fixed. As dis-cussed below, a synchronization protocol may change thepriority of a job.

Resource model. We extend the task model above byassuming that there are nr shared resources to which tasks inΓ require synchronized access. We denote these resources asL = {`1, . . . , `nr}. We assume that all resources are seriallyreusable, i.e., no reader/writer sharing. We assume familiaritywith synchronization-related terminology, such as criticalsection, lock, unlock, outermost critical section, etc. Whena job Ji requires a resource `a, it issues a request Ri for`a.1 Ri is satisfied as soon as Ji holds `a, and completeswhen Ji releases `a. Two requests conflict if they accessa common resource. We define the number of issued andnot-yet-completed requests that conflict with Ri when Ri isissued to be the contention that Ri faces, denoted ci. Themaximum critical-section length is denoted Lmax.

Priority inversions. Locking protocols designed for real-time systems must enable bounds on priority-inversion block-ing (or pi-blocking) to be determined; such bounds are appliedin schedulability analysis. Intuitively, pi-blocking occurs whena job is delayed while lower-priority work is executing in itsplace. A job may experience pi-blocking while it waits for aresource it has requested—this is called request blocking—oras a result of a progress mechanism (described later)—thisis called progress-mechanism-related (PMR) blocking.

Analysis assumptions. For asymptotic analysis, we con-sider critical-section lengths, as well as the number of requestsper job to be constant. We consider m and n to be variableswhen discussing both the proposed C-RNLP and prior pro-tocols, and each ci for τi ∈ Γ to also be a variable whendiscussing the C-RNLP. All other parameters are consideredto be variable unless stated otherwise.

Progress mechanisms. To ensure that pi-blocking timescan be reasonably bounded, real-time locking protocols em-ploy progress mechanisms that may temporarily raise a job’seffective priority. In this paper, we limit attention (due tospace constraints) to spin-based protocols that employ theprogress mechanism of priority boosting [3, 6, 10, 11], whereina resource-holding job’s priority is unconditionally elevatedabove the highest-possible base (i.e., non-boosted) priority.In suspension-based locks, other progress mechanisms areused such as priority inheritance [11] or priority donation [4].

Basic RNLP structure. The C-RNLP builds directly onthe structure of the RNLP, so a basic understanding of that

1We letRi denote an arbitrary request of Ji. We have no needto disambiguate different requests from Ji as our analysisfocuses on individual outermost requests.

RSMToken Lock Wait Queue

Resource Wait Queues

ResourceHolders

Figure 2: Architecture of the RNLP [12].

protocol is required to understand the results of this paper.As noted in Sec. 1, the RNLP is actually a family of protocols:a given instantiation is obtained by determining how waitingis realized (spinning or suspending), the progress mechanismto employ (priority inheritance, priority boosting, or prioritydonation), and potentially other factors (such as whether“long” critical sections are to be distinguished from “short”ones [12], and whether reader/writer sharing constraintsexist [13]). All such instantiations share a common structure,which consists of two components: a token lock and a requestsatisfaction mechanism (RSM). This is illustrated in Fig. 2.When a job Ji requires a shared resource, it requests a tokenfrom the token lock. Once Ji has acquired a token, it canissue requests for different resources within the RSM. TheRSM orders the satisfaction of such requests.

Each particular RNLP instantiation is defined by specifyingthe token-lock algorithm to use, the total number of tokens,and the RSM implementation to be employed. For the caseof spin-based locking with priority boosting (our focus here),each task is assigned a token when it becomes non-preemptive,so the token lock is essentially obviated. For the spin-basedvariant, worst-case request and PMR blocking are both O(m),which is asymptotically optimal. The other variants haveasymptotically optimal pi-blocking bounds as well.

The RSM. When a job is granted a token, it is assigned atimestamp. Within the RSM, each resource has a timestamp-ordered wait queue of pending requests for that resource.When requests can be nested, deadlock is a potential concern.A simple way to obviate any potential deadlock is to imposea partial ordering on resources, and to require that nestedlocks are acquired according to this ordering [5]. This is anold technique that is commonly used in practice (e.g. in theLinux kernel).

In some instantiations of the RNLP, resource orderings canbe avoided by using dynamic group locks (DGLs) [12]. LetDi be the set of all resources that may be acquired withinan outermost lock request Ri. When DGLs are used, insteadof issuing requests for resources in Di in a piecemeal fashionas nested requests are encountered, all such resources arerequested simultaneously at the time of token acquisition. (Ina real-time system, the set of all such requests would have tobe known for schedulability analysis.) In essence, a resourcegroup is identified dynamically and coalesced under one lock.The disadvantage of using DGLs is that requests may beissued for resources that are not actually needed. For example,if after acquiring resource `1, one of `2 and `3 is acquired,then requests would be issued for all three resources eventhough only two are needed. The main advantages are thatresource orderings are not required and runtime overheadstend to be lower. Also, worst-case pi-blocking bounds are notaltered under our existing analysis techniques when DGLsare used. Note that the usage of DGLs is not the same ascoarse-grained locking: coarse-grained locking groups are

statically defined offline, while DGLs can be used to acquirea subset of the resources that are a part of such a group.

3. C-RNLPIn this paper, we extend the RNLP to the C-RNLP, the first

fine-grained locking protocol with contention-sensitive worst-case blocking. We show that blocking under the C-RNLPis O(min(m, ci)) for each task τi. This is accomplished byallowing some lock requests to “cut ahead” of other queuedrequests, thereby limiting the length of transitive blockingchains. For ease of explanation, we assume that lock nestingis realized through the use of DGLs. Thus, each request Ri

specifies a set of resources Di to be locked.We define the C-RNLP in two passes: first, we give an

abstract description based on how the wait-for graph isupdated as requests are issued and completed, similarly toDijkstra’s early work on the dining philosophers problem [5];later, in Sec. 4, we describe a more concrete implementationthat conforms to the abstract specification.

The ordering of requests is maintained in a directed, acyclicwait-for graph G = (V,E), where vertices denote requestsand edges denote waiting relationships, i.e., (Ri,Rj) ∈ Emeans that Ri is blocked by Rj . Initially G is empty, withV = ∅ and E = ∅. When a request is issued, it is addedto the graph. This addition of a request along with anyassociated edges is called an insertion. Likewise, when arequest completes, the removal of it and all of its edges is aremoval. We denote the graph that results from G after aninsertion or a removal as G′ = (V ′, E′). (We similarly useprimes in referring to notation relevant to G′.) We say thatthe graph G′ is instantiated when it results from applyingan insertion or removal operation on G. For now, we assumethat these operations are atomic and take zero time to apply.Ri is satisfied when it has no outgoing edges. A resource`a requested by Ri is locked by Ri when Ri is satisfied. Aprotocol is considered safe if at most one request can lock anyone resource at a time. We assume that once Ri is satisfied,Ri completes within Li time units. Later we will explore theimplications of the violation of this assumption. We now usea series of examples to motivate the rules of the C-RNLP.

Safety. Our first example motivates the rules presentedlater that ensure that the C-RNLP is safe.

Ex. 1. Consider the wait-for graph G shown in Fig. 3.Each request requires only `a. R1 is satisfied and holds `aand blocks R2, as shown by the directed arrow to R1. Nowsuppose that R3 is issued and requires `a. R3 is added to G,and we must consider which edges to add. Several positions forinserting R3 are displayed in Fig. 3, denoted as positions P1–P5. For now, it suffices to understand the notion of a positionintuitively, but later we shall see that a formal definition isneeded. Intuitively, when a request is inserted into G, it isimplicitly reserving a position. (Actually, with DGLs, a setof positions is reserved, but we will ignore this additionalcomplication for now.) We examine these positions belowafter presenting several definitions.

Def. 1. For any request Ri, let In(Ri) denote its in-coming edges, In(Ri) = {(Rj ,Ri) : (Rj ,Ri) ∈ E}, and letOut(Ri) denote its outgoing edges, Out(Ri) = {(Ri,Rj) :(Ri,Rj) ∈ E}.

Def. 2. Let S be the set of satisfied requests: S = {Ri :Out(Ri) = ∅}.

P4

P3

R1

R2{a}

{a}

P2

P5

P1

Ri

Pi

{a, b}

node in graph for request Ri

potential position Pi

edge in graph with weight i

potential edge with weight i

resources needed for node

i

i

Ri

satisfied node for request Ri

Figure 3: A wait-for graph G and several positions at

which R3 could be inserted. (The legend also applies to

subsequent figures. Note that edge weights are not used

in this particular figure.)

Def. 3. A request Ri precedes Rj , denoted Ri ≺ Rj , ifthere exists a directed path from Rj to Ri.

Def. 4. A request Ri has cut ahead of Rj ∈ V if G′ isobtained by the insertion of Ri and Ri ≺ Rj is established.

Ex. 1 (continued). We examine each of the positionsP1–P5:

• P1. Here R3 would have no edges, and thus would besatisfied. However, this would lead to `a being locked byboth R1 and R3, which violates safety. Therefore, ourprotocol should not allow R3 to be placed at P1.

• P2. Here R3 would cut ahead of R2 and would be wait-ing for R1 to finish. This position ensures safety, i.e.,`a will be locked by at most one request at a time.

• P3. Here R3 would not cut ahead of any request, andwould be waiting for R2. This position is also safe.

• P4. Here R3 would cut ahead of R1 and wait for R2.This maintains safety, but creates deadlock in the sys-tem.

• P5. Here R3 would be cutting ahead of R1, which shouldbe disallowed since R1 is already satisfied.

Motivated by the above example, we note that to ensuresafety, there must be a single order in which each requestfor the same resource will be processed. A newly insertedrequest should also avoid cutting ahead of a request that hasalready been satisfied.

Def. 5. A set of requests Q ⊆ V has a unique orderingif and only if for any two distinct requests Ri and Rj in Q,either Ri ≺ Rj or Rj ≺ Ri.

Def. 6. Let Qa be the set of requests that require theresource `a: Qa = {Ri : Ri ∈ V ∧ `a ∈ Di}.

R4

R6

R5

R7

R2

R3

R1

5

38

4

2

44

{b}

{a, b}

{a}

{a, b, c}

{c}

{d}

{a}

Figure 4: A wait-for graph G that includes seven re-

quests.

Def. 7. An insertion into G resulting in G′ is a safe in-sertion if: (i) for each resource `a, there is a unique orderingon the set Q′

a; and (ii) a new request Ri does not cut aheadof a satisfied request, i.e., for any Rj in V , (Rj ,Ri) ∈ E′ ⇒Rj /∈ S. (Note that S is the set of satisfied requests in G.)

Delay preservation. Now that we have determined whichinsertions are safe, we investigate which are “best.” In orderto do so, we add information to G about how long eachrequest will run.

Def. 8. The weight of an edge (Ri,Rj) ∈ E is given byW (Ri,Rj) = Lj.

Ex. 2. Suppose we start with G containing nodes R1,R2, and R3 and the edges depicted between them as shownin Fig. 4, after which, R4, R5, R6, and R7 are inserted inorder into G. We examine each of those insertions.

• R4. The insertion of this request with no edges is clearlysafe, as no other request in G requires `d.

• R5. This request cuts ahead of R2. While this is a safeinsertion, it increases R2’s blocking time, as R2 mustnow wait for up to L1 + L5 = 6 time units to execute,as opposed to waiting for at most L1 = 4 time units.

• R6. This request cuts ahead of R3 and waits for R1 tocomplete. Since R3 is already waiting for as much asL1 +L2 = 9 time units, R6 cutting ahead is acceptable,as it would cause at most L1 + L6 = 7 time units ofwaiting time.

• R7. This request cuts ahead of R3. Using the samereasoning as used with R6, we see that L7 < L1 + L2.However, since L7 > L2, if R1 had already been runningfor close to L1 time units, then inserting R7 in thisposition could delay R3.

In order to reason about where to insert a request, wemust know how long a satisfied request has been satisfied,as demonstrated by Ex. 2 above.

Def. 9. The running time of a request Ri, denoted ri,is the time for which Ri has been satisfied.

Note that under our current assumption that Ri completeswithin Li time units, ri < Li.

Def. 10. A path in G from Ri ∈ V to Rj ∈ V is denotedRi;Rj. The length of this path is given by the sum of theweights of the included edges. This is denoted |Ri;Rj |.

As seen in Ex. 2, to determine a request Ri’s maximumblocking time, we are only concerned with the maximumdistance traversed through G from Ri’s node to a satisfiednode, and we must take into account rj for any Rj that issatisfied. Ri’s maximum blocking time is given by the value|A(Ri)|, defined next.

Def. 11. If Ri /∈ S, then let A(Ri) denote a path Ri;

Rj such that Rj ∈ S and |Ri;Rj | − rj is maximal; in thiscase, we define |A(Ri)| = |Ri;Rj | − rj . If Ri ∈ S, then wedefine |A(Ri)| = 0.

As suggested by Ex. 2, we want to preserve the existingmaximum delays for each request. Requiring preservationof maximum delays would also prevent the possibility ofdeadlock as shown in Ex. 1.

Def. 12. An insertion into G is delay-preserving if andonly if (∀Ri ∈ V :: |A′(Ri)| ≤ |A(Ri)|).

C-RNLP rules. Motivated by the examples above, ourC-RNLP implementation must satisfy the following rules:

Rule 1: All requests wait until satisfied.

Rule 2: Ri is removed when Ri completes.

Rule 3: A node is inserted at a safe, delay-preserving posi-tion in G that gives the lowest |A(Ri)|.

Rule 4: Insertions into and removals from G are atomic.

Using the rules of the C-RNLP, we sometimes insert re-quests into G in different positions than if we were using theRNLP. As shown in Fig. 1, the RNLP orders requests in theorder they are issued, with no cutting ahead. However, theC-RNLP would allow R4 in this example to cut ahead ofR3, if this would not increase R3’s blocking time, therebyavoiding transitive blocking.

Refining Rule 3. We have presented the abstract rulesthat define the C-RNLP, but Rule 3 lacks sufficient informa-tion to guide an actual implementation. Therefore, we willrefine Rule 3 after giving some necessary definitions. We firstrefine the notion of a position.

Def. 13. Ri and Rj are consecutive with respect to `aif {Ri,Rj} ⊆ Qa ∧Ri ≺ Rj ∧¬(∃Rl : Rl ∈ Qa :: Ri ≺ Rl ≺Rj).

Ex. 3. Given the graph shown in Fig. 5, R1 and R2 areconsecutive with respect to `a, and R1 and R3 are consecutivewith respect to `b.

Def. 14. A position Pk has at most one incoming edgeand at most one outgoing edge, which in an abuse of previousnotation, we denote In(Pk) and Out(Pk), respectively.

P1

P2

R1

R2

{a, b}

L1

R3 {a, b}

{a}

L2

L1

L3

L4

Figure 5: A wait-for graph G with two possible positions

for R4.

• If In(Pk) = ∅, then Pk is called a top-most position.

• If Out(Pk) = ∅, then Pk is called a bottom-most posi-tion.

• Otherwise, if In(Pk) = Ri and Out(Pk) = Rj , then Pk

is called an inner position. An inner position must haveOut(Pk) ≺ In(Pk).

A position is said to be an `a-position if In(Pk) and Out(Pk)are each either ∅ or a request that includes `a.

When a request Rl is inserted into a graph G, it re-serves a set of positions X, and In′(Rl) =

⋃Pk∈X In(Pk)

and Out′(Rl) =⋃

Pk∈X Out(Pk).We now use Ex. 3 to motivate the discussion of a position’s

capacity, defined below.

Ex. 3 (continued). Suppose G is as shown in Fig. 5,with request R4 for resource `b about to be added to thegraph at either position P1 or position P2. Both positionswould yield a safe insertion. Whether R4’s insertion at P1

is delay-preserving depends on the values of L2 and L4 andhow much longer R1 will be executing in the worst case. Forthis insertion to be delay-preserving, R4 must finish at thelatest when R2 would finish in the worst case (where eachrequest takes exactly its stated maximum time to execute),as this would ensure that all later requests—only R3 in thisscenario—would experience no additional worst-case blocking.To reason about what values of L4 would meet this condition,we introduce the concept of position capacity.

Def. 15. The capacity of a position Pk is defined as:

cap(Pk) =

∞ if In(Pk) = ∅|A(Ri)| if In(Pk) = Ri ∧Out(Pk) = ∅ω − β otherwise

(1)

where ω = |A(Ri)| − |A(Rj)| and β = (Lj − rj).

Ex. 3 (continued). We can now reason about the ca-pacities of P1 and P2. For P1 we obtain, cap(P1) = |A(R3)|−|A(R1)| − (L1 − r1) = (L1 + L2 − r1) − 0 − L1 + r1 = L2.This value indicates how long R3 will be waiting in the worstcase due solely to its blocking on R2 and taking into accountthat any request reserving position P1 would also be waitingfor R1 to finish. Therefore, if L4 were at most this value, R4

R1

R2

{a}

R3

{a}

R4

R5

{b}

R6

{b, c}

{a}R7

{b} {c}

P6

P5

P9

P10P7

P3

P2

P8 P11

P4

P1

Figure 6: A wait-for graph G with all possible positions

shown. P1–P4 are `a-positions, P5–P8 are `b-positions, and

P9–P11 are `c-positions.

could be inserted into P1 and be delay-preserving. BecauseIn(P2) = ∅, cap(P2) =∞. Intuitively, a request of any lengthcould be inserted into P2.

Def. 16. Let PDi be a Di position set, such that for each`a ∈ Di there is an `a-position in PDi .

When a request Ri is issued, it must reserve all posi-tions in a Di position set, PDi . In turn, Ri is insertedinto G, with In(Ri) =

⋃Pk∈PDi

{In(Pk)} and Out(Ri) =⋃Pk∈PDi

{Out(Pk)}.

Def. 17. The capacity of the position set PDi is thesmallest |A(Ri)| where Ri ∈ In(PDi) (or ∞ if In(PDi) = ∅)minus the largest |A(Rj)|+ Lj where Rj ∈ Out(PDi) (or 0if Out(PDi) = ∅)

Note that the capacity of a position set is at most that ofeach individual position in the set.

Now that we have more fully developed relevant conceptspertaining to positions, we replace Rule 3 with Rule 3′, whichupholds Rule 3 and refines how safe and delay-preservingpositions are found.

Rule 3′: Ri is inserted by reserving all positions in PDi

where Li ≤ cap(PDi) and |A(Ri)| is minimized.

Ex. 4. Suppose G is as shown in Fig. 6. (Note that allpossible positions are shown.) Now that we have defined Rule3′, we can easily examine all possible positions that a requestRi may reserve and determine which minimizes |A(Ri)|. IfDi = {`a}, then Ri must reserve one of the positions P1–P4.If instead Di = {`a, `b}, then Ri must reserve both an `aposition P1–P4 and an `b position P5–P8. Thus, Ri will haveat least two edges.

The fact that Rule 3′ upholds Rule 3 follows from the nextthree lemmas, the latter two of which are stated withoutproof, as they are straightforward.

Lemma 1. Ri inserted under Rule 3′ is delay-preserving.

Proof. Suppose to the contrary that Ri reserves a set ofpositions PDi and the insertion of Ri is not delay-preserving.Then, there is at least one node that obtains a new edgedirected to Ri that experiences increased blocking. Let Rj

denote such a node. Because Rj ’s blocking increases,

|A′(Rj)| > |A(Rj)|. (2)

Let Rl denote a node to which an outgoing edge from Ri isdirected due to the insertion such that the value |A(Rl)|+Ll

is maximized. (If no such node Rl exists, then a slightlysimpler version of the proof that follows can be applied. Weomit this case due to space constraints.) Then, by Def. 11,

|A′(Ri)| = |A′(Rl)|+ Ll − rl. (3)

Note that, after the insertion of Ri, there can be no pathfrom Rl to Ri, for then a cycle would exist (and hence,deadlock). More technically, if such a cycle were caused, thenthe set of positions PDi would have zero or negative capacityby Def. 17, and thus Ri would not have been inserted byRule 3′. Because the insertion of Ri does not result in anypath from Rl to Ri,

|A′(Rl)| = |A(Rl)|. (4)

Because the insertion of Ri caused Rj ’s blocking to increase,|A′(Rj)| depends on Ri, which upon insertion has not yetaccessed any resource, as it waits onRl. Therefore, by Def. 11,

|A′(Rj)| = |A′(Ri)|+ Li. (5)

The following reasoning establishes cap(PDi) < Li.

cap(PDi) ≤ {by Def. 17}|A(Rj)| − (|A(Rl)|+ Ll)

< {by (2)}|A′(Rj)| − (|A(Rl)|+ Ll)

< {by (4)}|A′(Rj)| − (|A′(Rl)|+ Ll)

= {by (5)}|A′(Ri)|+ Li − (|A′(Rl)|+ Ll)

= {by (3)}|A′(Rl)|+ Ll − rl + Li − (|A′(Rl)|+ Ll)

= {by simplification}Li − rl

≤ {because rl ≥ 0}Li

Therefore, for Rj to have experienced increased blockingupon the insertion of Ri, Ri must have reserved a set ofpositions with cap(PDi) < Li, which violates Rule 3′. Thus,such a set of reservations cannot occur.

Lemma 2. Ri inserted under Rule 3′ is safe.

Lemma 3. Ri is inserted by following Rule 3′ with thesame |A(Ri)| as by following Rule 3.

Establishing a bound. Based on the rules for the C-RNLP, we can bound the latest time at which a requestcould be satisfied.

Lemma 4. Suppose that G is instantiated at time t andG′ is instantiated at time t′. If |A(Ri)| > 0, then |A′(Ri)| ≤|A(Ri)| − (t′ − t).

Proof. The lemma follows because reservations are delay-preserving, and because all satisfied requests blocking Ri

execute continuously between times t and t′, due to priorityboosting.

Def. 18. Let Bi = |A(Ri)| at the time of Ri’s insertion.

Lemma 5. Ri is satisfied within Bi time units from itsinitial insertion.

P2

P1 R1

R3

{a, b}

L5

R4

L2

L1

L3

L4

P3R5

R6

{b}

{b}

R2

P4

{a, b}

{a, b}

{b}

L4

L2

L6

Figure 7: G with worst-case blocking for request Ri re-

quiring `a, with Li just greater than L1, L3, and L5.

Proof. Follows directly from Lem. 4.

Lemma 6. Bi < ci(Lmax + Li).

Proof. SupposeRi requires only one resource. By Rule 3′,Ri cannot be inserted into a position Pk where cap(Pk) < Li.In the worst-case scenario, Ri would have to be inserted intoa top-most position with all other relevant positions havingcapacity just less than Li. This scenario is shown in Fig. 7,where Ri requires `a and L1, L3, and L5 are all just lessthan Li. In this case, Bi < ciLmax + ciLi.

In the more general case, Ri requires a set of resources.Again, the worst-case scenario would only allow the insertionof Ri at a top-most position, with each other set of positionshaving capacity less than Li. This scenario establishes thesame bound. Therefore, Bi < ci(Lmax + Li).

Theorem 1. A request Ri is satisfied within ci(Lmax +Li) time units.

Proof. Follows from Lem. 5 and Lem. 6.

Uniform C-RNLP. Referring back to the proof of Lem. 6,when request Ri is inserted, call a potential position for itproblematic if that position has a non-zero capacity thatis less than Li. In the bound established in Thm. 1, theterm ciLi arises because there can be up to ci problematicpositions whenRi is inserted. We now briefly discuss a secondvariant of the C-RNLP, which we call the uniform C-RNLP,in which problematic positions cannot occur.

In this variant, the time line is segmented into fixed-lengthframes of size Lmax, wait-for graph modifications are allowedto occur only at frame boundaries, with all node removals oc-curring before any node insertions (we are still assuming thatthese graph modifications take zero time), and all satisfiedrequests are required to remain satisfied for exactly Lmax

time units. With these modifications, all parameters thataffect the graph’s basic structure are a multiple of Lmax andit can be formally shown by examining (1) that each positionhas a capacity that is a multiple of Lmax, or is 0, or is ∞.This implies that problematic positions cannot exist. As aresult, the bound in Thm. 1 reduces to ciLmax. However,because any request that occurs within a frame is delayeduntil the next frame boundary, this blocking bound must beincreased by Lmax (the frame size). From this discussion, wehave the following theorem.

Theorem 2. Under the uniform C-RNLP, a request Ri

is satisfied within (ci + 1)Lmax time units.

Corollary 1. Under either variant of the C-RNLP,worst-case request blocking is O(min(m, ci)).

Proof. Follows from Thms. 1 and 2, and our analysisassumptions and assumed progress mechanism of priorityboosting, which limits contention to at most m.

Removal of assumptions. In the two variants of theC-RNLP just described, safety is maintained even if resource-holding times are longer than specified. In fact, our statedblocking bounds actually remain unaltered if a request Ri

can hold a resource for longer than Li time units, providedno request holds any resource for longer than Lmax timeunits. However, if a request is allowed to hold a resourcefor longer than Lmax time units, then our bounds no longerhold. From a practical point of view, this really is not adeficiency, because if reliable bounds on resource-holdingtimes are not known, then it is pointless to attempt toconduct schedulability analysis.

So far, we have assumed that all modifications to a wait-forgraph take zero time. In reality, of course, such modificationswill entail some overhead. Such overheads can be factoredinto our blocking bounds using straightforward techniques.If these overheads are regarded as constants (which is some-thing assumed in all prior work on real-time locking protocolsknown to us), then blocking times under both protocol vari-ants remain contention sensitive.

4. IMPLEMENTATIONIn the prior section, we described two variants of the C-

RNLP at an abstract level. In this section, we present aconcrete implementation of the uniform variant.

Data structures. Fig. 8 depicts the key data structures ofour implementation. A request acquires resources by reservinga set of positions in a reservation table, Table, which is anarray of bit masks. Each bit in a bit mask represents aresource that can be reserved or locked for a frame of time oflength Lmax. We use bit masks because modern processorsprovide fast register-level operations for manipulating them.A request reserves a set of positions by selecting a row in Table(wrapping if necessary) and by setting the bits correspondingto its needed resources in that row’s bit mask. The arraysEnabled and Blocked are both indexed by frame. A requestRi

that has reserved a position given by row k is satisfied whenEnabled[k] = 1 holds; we say that such a row is enabled. Themanner in which Blocked is used is explained later. Severalother variables are used as well. Head indicates the currentlyenabled row of Table. Pending requests records the numberof pending (issued but not completed) requests. Size givesthe size of each array, which must be at least the number ofpending requests at any time.

Pseudo code. The lock and unlock routines of our im-plementation are shown separately in Alg. 1 and Alg. 2,respectively. Note that shared variables are capitalized, whilevariables specific to a request or a function call are lowercase.

In Fig. 8, we show how a set of requests in a wait-for graphG would be represented in our implementation. As in G,requests that will be fulfilled later appear higher in Table. R1

and R2 are satisfied, as is indicated by Enabled[0] = 1. R3

0

0

0

0

1

Table

Pending_requests: 4

la lb lc ld le

R1

{a, b}

1

{b, c}

{a, b}

R3

R4

1

R2

{c}

1

R1 R2

R3

R4

R1

R3

R4time

0

1

1

2

0

Lmax

bitmask

Head

0

1

2

3

4

Figure 8: Illustration of G and Table.

has reserved `b and `c in Table[1]. The value 2 in Blocked[1]indicates that R3 is waiting for both R1 and R2 to finish. R4

has similarly reserved `a and `b in Table[2]. We now explainthe lock and unlock routines by means of examples.

Ex. 5. Suppose R5 for `b and `c is issued when the sys-tem state is as in Fig. 8. First, a lock on Sublock is obtainedat Line 2 of Alg. 1 to serialize access to the protocol’s sharedstate. For the request under consideration, the test in Line 3would evaluate to true, so searching in Table for an availableset of positions would begin. This search occurs in the loopat Lines 5 and 6, where the bit masks of different rows arechecked. Upon termination of this search, the variable startindicates the corresponding row where the available positionswere found. Following the search, next is set, Blocked[next]and Pending requests are incremented, and the appropriaterow of Table is updated at Lines 11–14. The lock on Sublockis then released at Line 15. The task issuing the request thenspins on Enabled[start] at Lines 16 and 17.

Ex. 6. Now suppose that R2 completes. A lock on Sublockmust first be obtained at Line 21 in Alg. 2. At Line 22,Table is cleared of R2 by bit-wise ANDing the negation ofR2’s requested bitmask and its row in Table. Note that Headnow points to the acquired row of R2. Blocked[next] andPending requests are then decremented at Lines 23 and 24.If there are no more requests blocking requests in the rowindicated by next, i.e., Blocked[next] = 0 at Line 25, then thatrow is enabled and Head is updated at Lines 26–28. Finally,the lock on Sublock is released at Line 30.

The above implementation limits the total number of re-sources to be at most the number of bits per bit mask, e.g.,on a 64-bit machine, there could be at most 64 resources intotal. This restriction can be eased by applying results onthe renaming problem, which is a classic problem in work onconcurrent algorithms [1]. In this problem, tasks that haveidentifiers over a large name space are “renamed” by givingthem identifiers over a small name space—such names canbe both acquired and released [8]. A renaming algorithmcould be applied in our context to assign a unique identi-fier to any resource while it is being used. As a result, wewould merely need to limit the total number of concurrentlyrequested resources to be at most the bit mask size. Whilerenaming algorithms can be implemented with low overheadusing appropriate atomic instructions, we have not yet fullyexplored their use. We note also that it is possible to extend

Algorithm 1 Uniform C-RNLP Lock

1: procedure C-RNLP Lock(requested)2: lock(Sublock)3: if Pending requests > 0 then4: start← (Head+ 1) mod SIZE5: while (Table[start] & requested) 6= 0 do6: start← (start+ 1) mod SIZE7: end while8: else9: start← Head

10: end if11: next← (start+ 1) mod SIZE12: Blocked[next]← Blocked[next] + 113: Pending requests← Pending requests+ 114: Table[start]← Table[start] | requested15: unlock(Sublock)16: while Enabled[start] 6= 1 do17: /∗ null ∗/18: end while19: end procedure

Algorithm 2 Uniform C-RNLP Unlock

20: procedure C-RNLP Unlock(requested)21: lock(Sublock)22: Table[Head]← Table[Head] & ∼requested23: Blocked[next]← Blocked[next]− 124: Pending requests← Pending requests− 125: if Blocked[next] = 0 then26: Enabled[Head]← 027: Head← next28: Enabled[Head]← 129: end if30: unlock(Sublock)31: end procedure

our implementation by using several bit masks per row ofTable, though this would increase lock/unlock overheads.

5. EXPERIMENTAL EVALUATIONTo evaluate our C-RNLP implementation, we conducted a

series of experiments, measuring lock/unlock overheads, aswell as observed blocking and runtime performance.

We performed these experiments on a dual-socket, 18-cores-per-socket Intel Xeon E5-2699 platform. We compared theC-RNLP to the RNLP [14] and Mellor-Crummey and Scott’squeue lock (the MCS lock) [7], applied as a coarse-grainedlock, treating all resources as one resource group.

Measuring lock/unlock overheads. We measured lockand unlock overheads (i.e., the time it takes to perform thelock and unlock calls of each protocol) as a function of thenumber of requested resources, i.e., |Di|, the total numberof managed resources, nr, and the number of contendingtasks, n. Tasks were statically pinned one per core.2 Weconsidered n ∈ {2, 4, . . . , 36}, nr ∈ {1, 2, 4, 6, . . . , 64}, and|Di| ∈ {1, 2, 4, 8, 12, . . . , 48} where |Di| ≤ nr. Each contend-ing task executed lock and unlock calls in a tight loop 1,000times, with a negligible critical section, so as to maximizecontention for shared variables within the lock and unlockcalls.

The C-RNLP lock and unlock calls themselves acquire alock (which is true of the RNLP as well). Thus, blockingcan occur within the lock/unlock logic. This blocking is partof the lock/unlock overhead, and we therefore refer to it as

2Tasks were pinned to cores on the same socket when possible.

0 10 20 30 40 50Resources Requested

0

20

40

60

80

100

Lock

Ove

rhea

d (m

icro

seco

nds) parallel

serialparallel analyticserial analytic

Figure 9: Measured C-RNLP lock overhead as a function

of |Di| for n = 36 and nr = 64.

overhead blocking, to differentiate it from the protocol blocking(Lines 16 and 17 in the C-RNLP) experienced while waitingfor C-RNLP-protected resources to be released. Therefore,when measuring lock/unlock overheads, we included overheadblocking in the measurement, but not protocol blocking. Asimilar methodology was applied to the RNLP and the MCSlock (the latter has no overhead blocking). We measuredthese overheads using the cycle counter, and report the 99th

percentile of observed lock/unlock overheads, so as to filterany spurious measurements. Due to space constraints, wefocus only on lock overheads here; a similar story emergeswhen considering unlock overheads. Also, we can only presenta subset of the data we collected.

Fig. 9 gives several curves pertaining to the lock-overheaddata we collected for the C-RNLP. Each curve is plottedwith respect to the number of requested resources, |Di|. Forthe curve labeled serial, Di was defined to be the same forall tasks (i.e., resources are accessed serially). For the curvelabeled parallel, Di was determined at random (i.e., resourcescan be accessed in parallel). These two curves include anyoverhead blocking. One might argue that it is better toaccount for such blocking analytically rather than by relyingon measurement. To assess this possibility, we present twoadditional curves, serial analytic and parallel analytic, whichwere derived by measuring lock overheads with overheadblocking excluded and by inflating that measurement byaccounting for overhead blocking analytically. Based on Fig. 9,we make the following two observations.

Obs. 1. The complexity of the lock logic in the C-RNLPrequires an analytical estimation of worst-case overhead block-ing as opposed to a purely measurement-based approach.

We observed a surprising overhead trend, supported byFig. 9, in the parallel case in that, as the number of requestedresources |Di| increased, the observed worst-case lock andunlock overheads decreased. In comparison to the serial case,the observed overheads were higher, which was also surprisinggiven the potential that less of Table needed to be consideredif more requests could be processed in parallel. Initially, weconjectured this was because our experimental process wasnot able to produce the worst-case overhead blocking. Toaddress this, we considered overhead blocking analytically.The overheads using this analytical approach are indeed muchhigher, and therefore in a truly hard-real-time safety-criticalsystem, an analytically rigorous approach must be taken toaccount for overhead blocking.

Interestingly, this observation has implications for otherlocking protocols that themselves employ a lock. In particular,

0 5 10 15 20 25 30 35 40Tasks

0

2

4

6

8

10

12

Lock

Ove

rhea

d (m

icro

seco

nds) C-RNLP

RNLPMCS

Figure 10: Lock overhead as a function of task count n

for nr = 64 and |Di| = 4.

suspension-based locks, which are implemented in the kernel,often acquire kernel-based spin locks. To truly bound theworst-case overheads of such protocols, a similar analyticalapproach should be taken to account for overhead blocking.

Obs. 2. Runtime parallelism can increase worst-case lockand unlock overheads for the C-RNLP.

This can be seen by comparing the curves for the serialcases in Fig. 9 to those for the parallel cases. We found thebetter performance in the serial cases quite surprising, be-cause in these cases tasks do not “share” rows of Table andhence longer searches through Table are needed (indeed, weconfirmed that less of Table was typically searched in theparallel case). However, at least in the context of our experi-mental framework, greater parallelism allows requests to beissued at a faster rate, since they experience less protocolblocking. This in turn increases contention for the sharedvariables in the C-RNLP implementation. We conjecture thisincreased contention resulted in cache invalidations and ad-ditional cohearance traffic that resulted in increased memorylatency and therefore higher overheads.

The measurements discussed so far pertain only to theC-RNLP. In Fig. 10, we plot measured lock overheads forall three considered protocols as a function of the task countn (recall that in our experimental framework, n ≤ m). Twoobservations are supported by this data.

Obs. 3. For the C-RNLP, observed overheads increasedramatically when resources are shared across sockets.

This observation applies to the RNLP as well. It can beconfirmed by examining the sharp rise in the curves for bothprotocols between n = 18 and n = 20. This rise is due toincreased memory latencies due to cross-socket interactions.

Obs. 4. Fine-grained locking protocols have higher over-head than coarse-grained ones.

This can be easily seen in Fig. 10. This result is notsurprising, as coarse-grained protocols require far simplerlock/unlock logic. This exposes an interesting tradeoff: fine-grained protocols offer decreased blocking with higher over-heads, while coarse-grained protocols offer decreased over-heads at the expense of increased blocking.

Runtime performance. To examine this tradeoff, wemeasured the total lock and unlock overhead (including over-head blocking) and protocol blocking, which we hereaftersimply refer to as total blocking. Specifically, we tested the

0 20 40 60 80 100Critical-section Length (microseconds)

0

500

1000

1500

2000

2500

3000

3500

4000

Tota

l Blo

ckin

g Ti

me

(mic

rose

cond

s) C-RNLPRNLPMCS

Figure 11: Total blocking time of lock call as a function

of critical-section length for n = 36, nr = 64, and |Di| = 2.

same configuration parameters as in the previous experi-ments, and we also varied critical-section lengths within{1, 10, 20, . . . , 100} microseconds. Fig. 11 is a sample graphfrom this study, where n = 36, nr = 64, and |Di| = 2, thoughmany others are available online.3 Based on these results, wemake the following observation.

Obs. 5. When critical-section lengths are greater thanseveral microseconds and some parallelism is possible, theC-RNLP has less total blocking than previous protocols.

This can be seen quite dramatically in Fig. 11, where boththe RNLP and C-RNLP substantially outperform the MCSlock, with the C-RNLP (because of the greater parallelism itaffords) besting the RNLP. Fig. 11 was chosen to highlightthe best-case scenario for the C-RNLP, i.e., the case in whichthe most cutting ahead is possible. Obviously, for cases inwhich there is little if any cutting ahead (e.g., the serial casedescribed previously), the C-RNLP has inferior total blockingto the MCS lock as it results in the same request ordering, butwith higher overheads. Additionally, in the graphs availableonline, there exist cases in which the overhead of the C-RNLPresults in higher total blocking than the other protocols.

6. CONCLUSIONWe have presented two variants of the C-RNLP, the first

multiprocessor real-time locking protocol to be contentionsensitive. The C-RNLP uses novel techniques, which incorpo-rate knowledge of critical-section lengths, that enable requeststo be ordered in a way that breaks long transitive-blockingchains. These techniques increase lock and unlock overheads.However, the experimental results presented herein suggestthat these overheads may be worth incurring in some usecases.

Due to space constraints, we have limited our attentionto spin-based variants of the C-RNLP in this paper. In afuture expanded version of the paper, we will fully presentC-RNLP variants corresponding to all RNLP variants andshow that worst-case pi-blocking for each C-RNLP variant isasymptotically optimal when lock contention is considered.

To complement the measurement-based study presentedherein, we plan in future work to examine overhead/parallel-ism tradeoffs in the context of a full overhead-aware schedu-lability study that considers several coarse- and fine-grainedlocking protocols. Such a study will require implementations

3http://www.cs.unc.edu/˜anderson/papers.html

of all C-RNLP variants. While the spin-based implementa-tion presented herein entails only user-level code, suspension-based variants require kernel-level implementations. For allvariants, we intend to consider the possibility of using a “lockserver” as an option to reduce overheads. The idea here is tobind such a server to one processor and have tasks acquireand release resources by invoking the server. Such a serverwould tend to run “cache hot,” which might significantly re-duce overheads. We also intend to more fully investigate thepotential of using renaming algorithms (see Sec. 4), as wellas other techniques that might reduce overheads or increaseflexibility.

References[1] H. Attiya, A. Bar-Noy, D. Dolev, D. Koller, D. Peleg,

and R. Reischuk. Achievable cases in an asynchronousenvironment. In FOCS ’87.

[2] A. Block, H. Leontyev, B. Brandenburg, and J. Anderson.A flexible real-time locking protocol for multiprocessors.In RTCSA ’07.

[3] B. Brandenburg and J. Anderson. Optimality resultsfor multiprocessor real-time locking. In RTSS ’10.

[4] B. Brandenburg and J. Anderson. Real-time resource-sharing under clustered scheduling: Mutex, reader-writer, and k-exclusion locks. In EMSOFT ’11.

[5] E. Dijkstra. Two starvation free solutions to a generalexclusion problem. EWD 625.

[6] K. Lakshmanan, D. Niz, and R. Rajkumar. Coordi-nated task scheduling, allocation and synchronizationon multiprocessors. In RTSS ’09.

[7] J. Mellor-Crummey and M. Scott. Algorithms for scal-able synchronization of shared-memory multiprocessors.Transactions on Computer Systems, 9(1):21–64, 1991.

[8] M. Moir and J. Anderson. Wait-free algorithms for fast,long-lived renaming. Science of Computer Programming,25(1):1–39, 1995.

[9] R. Rajkumar. Real-time synchronization protocols forshared memory multiprocessors. In ICDCS ’90.

[10] R. Rajkumar, L. Sha, and J. Lehoczky. Real-time syn-chronization protocols for multiprocessors. In RTSS’88.

[11] L. Sha, R. Rajkumar, and J. Lehoczky. Priority in-heritance protocols: An approach to real-time systemsynchronization. IEEE Trans. on Comp., 39(9):1175–1185, 1990.

[12] B. Ward and J. Anderson. Fine-grained multiprocessorreal-time locking with improved blocking. In RTNS ’13.

[13] B. Ward and J. Anderson. Multi-resource real-timereader/writer locks for multiprocessors. In IPDPS ’14.

[14] B. Ward and J. Anderson. Supporting nested lockingin multiprocessor real-time systems. In ECRTS ’12.

[15] A. Wieder and B. Brandenburg. On the complexity ofworst-case blocking analysis of nested critical sections.In RTSS ’14.