CONCURRIT: Testing Concurrent Programs withProgrammable State-Space Exploration

Jacob Burnim Tayfun Elmas George Necula Koushik SenDepartment of Electrical Engineering and Computer Sciences, University of California, Berkeley

{jburnim,elmas,necula,ksen}@cs.berkeley.edu

AbstractTesting is the most widely-used methodology for soft-ware validation. However, due to the nondeterministicinterleavings of threads, traditional testing for concur-rent programs is not as effective as for sequential pro-grams. To attack the nondeterminism problem, softwaremodel checking techniques have been used to system-atically enumerate all possible thread schedules of atest program. But such systematic and exhaustive explo-ration is typically too time-consuming for many test pro-grams. We believe that the programmer’s help to guidethe model checker towards interesting executions is crit-ical to circumvent this problem.

We propose a testing technique and a supporting toolcalled CONCURRIT, which provides a model checkerthat can be guided programmatically within test code.While writing a test, the programmer specifies a partic-ular thread interleaving scenario in mind using an em-bedded domain-specific language (DSL), and CONCUR-RIT explores all and only the executions realizing theintended scenario. During the exploration, the program-mer is also able to observe the execution (e.g., assert in-variants) and constrain the future decisions of the modelchecker, all within the test code. We believe that provid-ing the programmer the ability to observe and controlthe exploration of executions will lead to more effectiveand efficient testing for concurrent programs.

1. IntroductionTesting is the most widely-used methodology for soft-ware validation. Concurrency brings new challenges tosoftware testing. The biggest challenge is the nondeter-minism in the scheduling of concurrently running com-putations, i.e., threads. The output of a concurrent pro-gram can be highly sensitive to timings of the interac-tions and interference between threads. The same testprogram can result in a large number of nondeterminis-tic executions producing different outcomes. This makesconcurrency-related defects notoriously difficult to de-tect and reproduce. As a result, testing has not been ap-

plied on concurrent programs as effectively in practiceas it has on sequential programs.

We believe that to address this problem, it is criticalfor testing to provide the programmer techniques andtools (i) to conveniently express interesting executionsof a test program with respect to a particular threadinterleaving scenario in mind and (ii) to examine theseexecutions systematically and efficiently. We observe aspectrum of approaches to this, as described next.

1.1 BackgroundAt one end of the spectrum, there are manual ap-proaches. In the extreme case, a multithreaded test pro-gram is run under a completely nondeterministic sched-uler. There is no guarantee about whether the result-ing execution will be of any interest (e.g., with suffi-cient degree of interaction between threads) to the pro-grammer and whether re-running the test will producethe same or different executions. To control the non-determinism, the programmer implements a synchro-nization/communication mechanism (e.g., inserts sleepstatements throughout the code) to restrict the possi-ble schedules of threads towards a particular scenario.However, such mechanisms are not portable and reli-able in general and may require nontrivial modificationsin the program text. Techniques have been proposedto guide the execution to intended thread schedules inmore portable and reliable ways, where the intendedschedules are specified by the programmer relative toa global timer (ConAn [16], MultithreadedTC [21]) or asequence of user-defined events expressed in linear tem-poral logic (IMUnit [10]). While these techniques givethe programmer the ability to impose constraints on theinterleaving of threads, they do not support systematicexploration of all executions satisfying these constraints.

At the other end of the spectrum, there are fully au-tomated approaches. To alleviate the nondeterminismin the execution, software model checking techniqueshave been combined with testing to control the threadscheduler so that distinct interleavings of the threadsin the test are systematically enumerated and checked

against the test criteria [14, 19, 20]. Many model check-ing exploration techniques have been developed thatseek to achieve high coverage of executions in a scal-able way [12]—such as partial-order reduction [7, 8],symmetry reduction [9], thread-modular reasoning [6],and preemption bounding [19]. These techniques donot directly help to efficiently examine interesting andpotentially-problematic interleaving scenarios. Due tothe state-space explosion problem, examining a particu-lar scenario involving large components of the programand identifying a bug may require to wait for an expen-sive (in time and resources) model checking process.

Recent studies proposed techniques to control thescheduling of a model checker to target suspicious ex-ecutions more quickly than traditional state-space ex-ploration. Among them, active testing [22], probabilisticscheduling [2], and change-aware preemption prioriti-zation [11] are fully automated and rely on heuristics.However, we believe that programmers help to guidethe exploration of executions is also valuable, and test-ing tools should be designed to allow the programmer tointeract with the test runtime to express her intents andinsights about the test scenario. In fact, work on preemp-tion sealing [1] proposed to disable preemptions that theprogrammer thinks are not interesting or can cause falsewarnings. In our work, we would like to give more con-trol and flexibility to the programmer in this direction.

1.2 Our ApproachWe propose a modeling and implementation of a testingtechnique and supporting tool (CONCURRIT) that com-bines testing and software model checking in a novelway. Our main goal is to make the model checking moreaccessible and controllable to the programmer. In par-ticular, we provide a domain-specific language (DSL)embedded in a host language (in our case C/C++) us-ing which the programmer can specify at high level howan execution of the test should develop, indicating con-straints and nondeterministic choices on thread sched-ules. For example, a constraint on the schedule may dic-tate which threads are allowed to be interleaved at whichpoint of the execution, or at which code location or un-der what conditions control will pass from a thread toanother. Our DSL provides to the programmer a conciseand convenient way of imposing constrains on thread in-terleavings programmatically—i.e., within the test code.In this way, one can describe a test scenario with variousdegrees of flexibility in the thread schedule ranging fromfully deterministic to fully nondeterministic. Section 2gives example test programs written in our DSL. CON-CURRIT implements a testing framework and a statelessmodel checker that, guided by our DSL, enumerates allpossible executions satisfying the scenario constraints.

Our approach also makes the underlying modelchecker more accessible to the programmer for imple-menting and evaluating a custom search strategy withinthe test code. While existing model checkers also offerplug-in mechanisms to customize and extend the searchalgorithms, we believe that interacting with the tool di-rectly from the test code and using a high-level DSL(rather than a low-level API) is more convenient for pro-grammers. In this aspect, our work is similar to [15],which separates a compact and high-level fixed-pointformulation of a model checking algorithm for Booleanprograms from a general-purpose fixed-point solver.

The initial prototype implementation of CON-CURRIT for C/C++ programs is available online athttp://code.google.com/p/concurrit/.

1.3 Testing Programs in Cooperative SemanticsOne of our key insights is that from the perspective of aprogrammer, cooperative executions of a program arerelatively much simpler to describe and reason aboutthan its traditional preemptive executions. Thus, we pro-pose to test a concurrent program by only examiningits cooperative executions. In this case, code locationsthat a thread may yield control to another thread are ex-plicitly marked, and a thread may not release controlat a different location. In other words, code betweentwo yield locations are guaranteed to run atomically.Once the functionality of the program is tested in thecooperative semantics, there are techniques (e.g., Yi etal. [23–25]) that can validate the choice of yield pointsby checking that the program is cooperative [25] withthese yield points—i.e., when run under a traditionalpreemptive scheduler, the program does not exhibit anyextra behavior other than it exhibits when running undera cooperative scheduler.

Let P refer to the program under test. We propose toperform the validation of P in the following steps:

S1 The programmer obtains from P a program P

coop

byannotating P with yield points. (Section 3.1 )

S2 The programmer writes multithreaded tests for P

coop

and runs them in the cooperative semantics. Thispaper is mainly about the testing of P

coop

in this step.

S3 The programmer (using [24] or [25]), either (i) showsthat P

coop

is cooperative, or (ii) identifies potentialinterference points that are not covered by the exist-ing yield annotations in P

coop

. If (ii) happens, backin S1, the programmer adds yield points to cover de-tected interference and re-runs the tests in S2.

S4 The programmer compiles and deploys the programin the preemptive semantics by simply ignoring theyield annotations. In our case, yield annotations areC macros that translate to no-ops in the release build.

1 // Program code under test (annotated with yield points for testing)

2 producer(Buffer *buff, int product) { ... ... ... }3 consumer(Buffer *buff) {4 ...

5 if (buff is empty ) { ... yield(‘‘Await’’); ... }6 ...

7 }8 ----------------------------------------------------------------------------------------------

9 // Test script in our DSL describing all interleavings of four threads

10 test producer consumer 1() {11 Buffer buff(2); // Create empty bounded buffer of size two

12 // Create producer and consumer threads, but not activate them yet

13 thread t P1 = thread(producer, &buff, 1), P2 = thread(producer, &buff, 2);

14 thread t C1 = thread(consumer, &buff), C2 = thread(consumer, &buff);

15 while(!all ended()); // Loop until both threads terminate

16 transfer(*).until(*); // Run a thread until some yield point

17 assert( ... ); // Check an invariant on the buffer

18 }19 assert(buff.size() == 0); // Check a post-condition on the buffer

20 }21 ---------------------------------------------------------------------------------------------

22 // Test script in our DSL describing intended interleavings of a scenario

23 test producer consumer 2() {

24 ... // Same definitions in lines 11-14 above

25 // T1

26 transfer(C1).until(‘‘Await’’); // Run C1 until yield point labeled ‘‘Await’’

27 // T2

28 with(P1, P2, C2) { // Execute lines 29-31 with only P1, P2 and C2

29 while(!ended(C2)); // Loop until C2 terminates

30 transfer(*).until(*); // Run a thread until some yield point

31 }32 }33 assert(buff.size() †= 1); // Check a condition on the buffer

34 // T3

35 while(!all ended()) { // Loop until all threads terminate

36 transfer(*).until(end); // Run a thread until it terminates

37 }38 assert(buff.size() == 0); // Check a condition on the buffer

39 }

Figure 1. Example producer-consumer tests.

2. Example: A Producer-Consumer TestFigure 1 shows two tests for checking a concur-rent producer-consumer module, which uses a shared,bounded buffer of integers. We omit the details of theprogram under test (shown at the top of the figure), ex-cept we highlight that the code is annotated with yield

points each with a unique label. The figure shows onlyone of these annotations labeled “Await” (line 5), thoughothers exists so that threads running producer and con-

sumer can interleave with each other. We focus on thetest procedures, we call test scripts, written using ourDSL. In the code, keywords specific to our DSL areshown underlined. For simplicity of exposition, we usea conceptual language, whereas in our implementation,the DSL is embedded in C/C++, and the correspondinglanguage constructs are provided as C macros that trans-late to API calls to the CONCURRIT library.

The first test test producer consumer 1 describes ascenario involving a bounded buffer to hold 2 integers(line 11), two threads P1, P2 to act as producers to in-

sert 1 and 2 into the buffer (line 13), and two threadsC1, C2 to act as consumers (line 14). Using our DSL,the programmer specifies that she expects to examine allpossible nondeterministic interleavings of the threads.The DSL expression all endedpq returns true iff all thethreads (visible in the current scope) have terminated.The statement transferp˚q.untilp˚q at line 16 instructsthe model checker to run one of the threads P1, P2, C1,

C2 until it reaches some yield point; ˚’s indicate thatboth the thread and yield point are chosen nondetermin-istically. The transfer statement blocks during this exe-cution. We point out that, in addition to checking a post-condition of the test (line 19), the programmer can alsoinsert an assertion at each interleaving point to check aprogram invariant (line 17). In this way, the programmeravoids to add these checks inside the program code un-der test, and thus, decouples her program from its tests.

The real novelty of our DSL comes into play whenthe programmer is interested in a particular set ofinterleavings rather than a completely nondeterminis-tic one. The second test test producer consumer 2 de-scribes such a situation. Suppose that the programmerdoes not want to examine all interleavings of these pro-ducer and consumer threads, because she suspects thata concurrency error is likely to occur during a particularinterleaving scenario, and she wants to ensure that theintended scenario does not trigger any errors. Note that,looking at all interleavings of the threads (i.e., withoutconstraining the interleavings) would help the program-mer check her claim about the error-freedom of that in-terleaving scenario. However, exploring all interleavingswould be too costly in time. Thus, the programmer in-tends to check all and only the possible executions ofthe scenario in mind, which develops in three steps:

T1 Thread C1 runs first until it detects that the buffer isempty and starts waiting for the buffer to get full.

T2 Then, both producer threads and consumer threadC2 run concurrently, until C2 consumes the integereither P1 or P2 inserts in the buffer and finishes. Atthe end of this step, size of the buffer must be § 1,since C2 consumes one of the integers produced.

T3 Finally, both producer threads and C1 run sequen-tially in some arbitrary order. At the end, the buffermust be empty, since in order to terminate C1 mustconsume the remaining integer not consumed by C2.

As shown in Figure 1, our DSL provides a clean, con-cise, and high-level way describing the scenario above.First, the DSL constructs help the programmer to con-strain the nondeterminism in the interleavings of pro-ducer and consumer threads, so that at each point onlyallowed threads can run (using with), and a thread can-not execute earlier than or beyond a given point (using

until). For example, C1 should run up to a yield pointlabeled “Await” at the beginning of the test (line 26),and then it should not run until C2 ends (line 28-32). (Infact, one can describe in this style a fully deterministicinterleaving indicating which threads to transfer at eachpoint and until which yield point.) Second, the DSL con-structs also allow the programmer to indicate expectednondeterminism about which thread to choose and untilwhen that thread must run (using ˚). For example, in T2,P1, P2, and C2 are interleaved with each other nondeter-ministically, and in T3, threads are run sequentially untiltermination (we use the special label end to refer to theend of a thread) but in a nondeterministic order. There-fore, this test script specifies not a single execution buta set of executions, which our integrated model checkertargets to systematically enumerate. We note that var-ious model checking techniques, e.g., dynamic partialorder reduction [5], can still be applied in this setting tomake the exploration efficient.

2.1 When a Test Passes or FailsWhile testing a sequential program, the common ques-tion asked by the programmer is: Does the (only) exe-cution of the test satisfy all the assertions? Having theability to search all executions of a concurrent test pro-gram, we can talk about two questions to answer whenrunning a test with a model checker. Each question de-termines when the overall test passes or fails (in otherwords, when the model checker finishes the explorationof executions). A successful execution of the test scriptis a “terminating” execution that (i) does not violate anassertion and (ii) satisfies all the constraints specified bythe programmer. Section 3.2.1 overviews our DSL con-structs to impose constraints on executions. A failing ex-ecution is one that violates an assertion. If an assertionviolation is detected, the model checker terminates theexploration immediately and declares the test failed.

The model checker can be run one of two modes toanswer the following questions, respectively:Are there any successful executions of the test script?In this mode, the model checker seeks to find a success-ful execution. The test passes if the model checker ex-plores at least one successful execution without detect-ing any failing execution meanwhile. The test fails oth-erwise. Notice that, a test in this mode can pass, produc-ing a successful execution, even though there also existsa failing execution.Are there any failing executions of the test script?In this mode, the model checker enumerates all suc-cessful executions of the test script. The test fails if themodel checker detects a failing execution during the ex-ploration and passes otherwise. Differently from the pre-vious mode, a test in this mode can pass even thoughthere is no successful execution.

3. Testing Programs with CONCURRIT

3.1 Annotations for Cooperative ExecutionCooperative concurrency builds on the idea of (symmet-ric) coroutines [4, 13], which generalize subroutines toallow multiple entry points for suspending and resum-ing the execution. In CONCURRIT, we model coroutinesfor C/C++ using pthreads [18] and cooperation at yieldpoints using proper synchronization operations. In ourcase, the cooperation among threads is performed ex-plicitly using two kinds of operations: transfer and yield.The former is used in the test script to pass control toother threads. The latter is used for a thread (for exam-ple, P1, P2, C1, C2 in Figure 1) to relinquish the controlback to the test script, and for this, the program’s codeunder test needs to be explicitly annotated with yield be-fore testing, as explained next.

Every yield annotation is given a unique label, us-ing which the yield location can be referred to in thetest script. Let l range over labels. A yieldplq statementcan be placed anywhere in the code to indicate a lo-cation at which a thread executing that code in a co-operative execution may yield control to other threads.Figure 1 shows an example to this at line 5. The yieldpoints in general indicate source locations, such as ac-cesses to shared memory, that are subject to interferenceby other threads. Thus, similarly to [23], yield pointscan also be annotated as follows: A read from a sharedvariable x can be replaced by yield readpl , xq. This ex-presses that the current thread first yields the executionto other threads, and when the control is transferred backto it again, it reads from x and returns the value. A writeto a variable x in the form of x “ e can be replaced byyield writepl , xq “ e. This is similar to yield read, ex-cept that the current thread writes the (previously com-puted) value of e to x after it gains the control back.

In the extreme case, one can treat every shared vari-able access a potential yield point and imagine a toolthat examines shared variable accesses and automati-cally inserts yield annotations accordingly. We providesuch a tool (using Pin [17]) that monitors sample exe-cutions of the test program and marks accesses that in-volve in a data race as potential yield points. However,in reality the number of shared variable accesses subjectto harmful interference may be less than all shared ac-cesses [3, 25]. In this case, the programmer can chooseto add the annotations gradually (for example, by prun-ing out the yield annotations inserted by a tool) as moretests pass. In fact, we see the gradual, test-driven addi-tion of yield annotations as a systematic way of increas-ing the concurrency of the program.

3.2 Test Scripts: Syntax and SemanticsFigure 2 shows our DSL for writing tests. Note that, forsimplicity we present here a conceptual language rather

f , g P Functions t P ThreadVariables

l ::“ next | end | “label text” Yield labelse ::“ ended (t) | all ended() | ¨ ¨ ¨ Boolean expressionst ::“ ˚ | t | t1, ¨ ¨ ¨, tk Thread expressionsu ::“ ˚ | l | e Until expressionsc ::“ except(t) | until(u) | count(k) Transfer clausess ::“ t = thread (f, args) Thread creation

| t = transfer (t).c1. . . ck Transfer to a thread| assert (e) Assertion| assume (e) Assumption| with/without (t1, ¨ ¨ ¨, tk) { s } Thread scope| x “ e | s ; s | if(e) { s } | ¨ ¨ ¨ Standard C/C++ stmts.

Figure 2. Syntax of our DSL, embedded in C/C++, forwriting test scripts. (We do not show yield, as it is notused in test scripts, but in the program under test.)

(Test&threads)&(Controller&thread)&

1"

2"

3"

4"

f(args) {// Program code:· · ·yield(l1);· · ·yield(l2);· · ·

}

// Test script:test func() {

t1 = thread(f, . . . );t2 = thread(g, . . . );transfer(t1).until(l1);assert x == 0;transfer(�).until(�);assume y == 1;assert x == 1;

}

g(args) {t1

t2Main

Figure 3. Execution of a test script.

than the actual C/C++ syntax. A test script in this settingis a function (in a recognizable signature by our testingframework), whose body is a statement s from Figure 2.A test script combines our DSL expressions and state-ments with other, standard expressions and statementsof the host language (in our case, C/C++).

Figure 3 depicts the execution of a test. The test isexecuted by a set of threads: tMain, t1, ..., tnu whoseinterleavings are controlled by a model checker. Mainrefers to the special controller thread running the testscript written in our DSL. In Figure 1 the body oftest producer consumer 1/2 are executed by Main . Ev-ery execution of the test starts with Main being the onlythread. Other threads (t1, ..., tn), we call test threads, arecreated by Main within the script using the thread op-eration; each test thread executes a given function callconcurrently (and cooperatively) with Main and othertest threads. In Figure 1 threads P1, P2, C1, C2 are testthreads and created at lines 13-14. Creating a threaddoes not activate the new thread; the thread is activatedlater by a transfer statement as described next.

At any time during the execution, either Main orone of t1, ..., tn is allowed to run, and that thread does

not pass the control to another thread until it reaches atransfer or yield.

• Control passes from the Main thread to a test threadti by a transfer operation in the test script. (Arrows1, 3 in Figure 3) As only one thread is active at anytime, Main then gets into a waiting state.

• Control passes from a test thread ti to the Mainthread by a yield operation. (Arrows 2, 4 in Fig-ure 3) Not every yield call is granted (taken) (see Sec-tion 3.2.1), but whenever it is, control is always givento Main .

When Main becomes active, it can check some con-dition on the current state using assert or assume (seebelow), perform local computation and update some(possibly global) variables, and/or can transfer controlto another test thread. An execution of the test endswhen the test script finishes. The model checker maythen backtrack and start a new execution of the script.

We point out that the test script may also describean infeasible interleaving. For example, a thread mayblock (on a synchronization operation) between twoyield point, preventing Main from taking the controlback. To detect and avoid such cases, we set a time limitfor each thread to execute between two yield points. Ifthis time expires, the model checker stops the currentexecution and backtracks for a distinct execution.

3.2.1 DSL Constructs to Constrain Test Executions

Assert and assume. Given a boolean expression on thecurrent state (global variables plus the locals of Main)we distinguish two ways to reason about a condition onthe current program state: assert e and assume e. Bothstatements do not have any side effects if e holds (i.e.,evaluates to true), but they differ when e does not hold.

If e does not hold, assert e causes the test to failimmediately, as usual. On the other hand, assume e

in this case does not terminate the entire test. Instead,the model checker discards the currently explored ex-ecution, backtracks, and restarts with a distinct threadschedule. Thus, we use assumptions to impose a “soft”constraint on the executions to be explored by the modelchecker, especially when expressing at which conditionsan interleaving is expected to happen is more convenientthan referring to a particular yield label in the program.Nondeterministic transfers. While a test thread alwaysyields to Main , the target of a transfer from Main canbe decided statically or at runtime. In the simplest case,transferptiq passes the control to thread ti. The pro-grammer can also leave the (nondeterministic) decisionabout which thread to run to the model checker usingtransferpt1, ..., tkq or transferp˚q. In the former case oneof t1, ..., tk and in the latter case any thread in the cur-

rent scope (see below) is chosen.1 The chosen targetthread is returned by transfer for the script’s observa-tion when control is given back to Main . An except

clause can be attached to transferp˚q to prevent a threador threads from being scheduled at that transfer point(e.g., transferp˚q.exceptpt2q).Transfer clauses. When transferring to a test thread, theprogrammer can also choose at which yield point thetarget thread should yield the control back to Main . Forthis, transfer statements are augmented with count anduntil clauses, which specify the required conditions forthe target thread to yield. By default, each transfer isaugmented with untilp˚q and countp1q clauses, indicat-ing that the target thread can relinquish the execution atany point.1 Given a label l , untilplq.countpkq indicatesthat the target thread should yield at the k

th visit of ayield point labeled with l but no earlier. We also de-fine special labels next and end for convenience to referto the next reached yield label and the end of the targetthread. For example, transferptq.untilpendq transfers tot and executes it until t terminates. Finally, the program-mer can also specify a boolean expression in untilpeq toindicate that the target thread should yield only whene evaluates to true. Given that, transferp¨ ¨ ¨ q.untilpeq isequivalent to transferp¨ ¨ ¨ q; assumepeq.Thread scopes. Our DSL allows the programmer todefine a temporary thread scope in which the modelchecker works with only a subset of threads vis-ible in the current C scope. For this, we providewithpt1, ¨ ¨ ¨, tkq and withoutpt1, ¨ ¨ ¨, tkq statements. Theformer defines a thread scope in which only threadst1, ¨ ¨ ¨, tk are used in any scheduling decision while ex-ecuting the given statement (referring to another threadcauses an error). In Figure 1, we use with to constrainpart of the execution to threads P1, P2 and C2 (line 28).The latter also defines a thread scope but in a dual way;it temporarily removes t1, ¨ ¨ ¨, tk from the current scope.

4. ConclusionWe propose a modeling and implementation of a testingtechnique and supporting tool (CONCURRIT) combiningtesting and software model checking in a novel way. Ourtechnique provides a domain-specific language (DSL)embedded in a host language (in our case C/C++). Theprogrammer, using this DSL, can specify at high levelhow an execution of the test should develop; s/he canexplicitly indicate constraints on thread schedules. Inthis way, the programmer can control the degree of free-dom in nondeterministic choices a model checker wouldmake during the state-space exploration. We believe thatsuch a DSL can make the model checking more acces-

1 The nondeterministic decision may be biased by the model checker’ssearch algorithm, e.g., the partial-order reduction [7] being used.

sible and controllable to the programmer, and thus, canlead to a more effective use of the state-space explo-ration in unit and system testing.

References[1] Thomas Ball, Sebastian Burckhardt, Katherine Coons,

Madanlal Musuvathi, , and Shaz Qadeer. Preemptionsealing for efficient concurrency testing. Technical Re-port MSR-TR-2009-143, Microsoft Research, 2009.

[2] Sebastian Burckhardt, Pravesh Kothari, Madanlal Musu-vathi, and Santosh Nagarakatte. A randomized schedulerwith probabilistic guarantees of finding bugs. In Pro-ceedings of the fifteenth edition of ASPLOS on Architec-tural support for programming languages and operatingsystems, ASPLOS ’10, pages 167–178, New York, NY,USA, 2010. ACM.

[3] Jacob Burnim, Tayfun Elmas, George Necula, andKoushik Sen. NDSeq: Runtime checking for nondeter-ministic sequential specifications of parallel correctness.In Programming Language Design and Implementation(PLDI), 2011.

[4] Melvin E. Conway. Design of a separable transition-diagram compiler. Commun. ACM, 6:396–408, July1963.

[5] Cormac Flanagan and Patrice Godefroid. Dynamicpartial-order reduction for model checking software. InPOPL ’05: Proc. of the 32nd ACM SIGPLAN-SIGACTsymposium on Principles of programming languages,pages 110–121, New York, NY, USA, 2005. ACM Press.

[6] Cormac Flanagan and Shaz Qadeer. Thread-modularmodel checking. In Proceedings of the 10th internationalconference on Model checking software, SPIN’03, pages213–224, Berlin, Heidelberg, 2003. Springer-Verlag.

[7] Patrice Godefroid. Partial-order methods for the verifi-cation of concurrent systems: an approach to the state-explosion problem, volume 1032. Springer-Verlag Inc.,New York, NY, USA, 1996.

[8] Guy Gueta, Cormac Flanagan, Eran Yahav, and MoolySagiv. Cartesian partial-order reduction. In Proceed-ings of the 14th international SPIN conference on Modelchecking software, pages 95–112, Berlin, Heidelberg,2007. Springer-Verlag.

[9] Radu Iosif. Symmetry reduction criteria for softwaremodel checking. In Proc. of the 9th International SPINWorkshop on Model Checking of Software, pages 22–41,London, UK, 2002. Springer-Verlag.

[10] Vilas Jagannath, Milos Gligoric, Dongyun Jin, QingzhouLuo, Grigore Rosu, and Darko Marinov. Improved mul-tithreaded unit testing. In Proceedings of the 19th ACMSIGSOFT symposium and the 13th European conferenceon Foundations of software engineering, ESEC/FSE ’11,pages 223–233, New York, NY, USA, 2011. ACM.

[11] Vilas Jagannath, Qingzhou Luo, and Darko Marinov.Change-aware preemption prioritization. In Proceedingsof the 2011 International Symposium on Software Testing

and Analysis, ISSTA ’11, pages 133–143, New York, NY,USA, 2011. ACM.

[12] Ranjit Jhala and Rupak Majumdar. Software modelchecking. ACM Comput. Surv., 41:21:1–21:54, October2009.

[13] Gilles Kahn and David B. Macqueen. Coroutines andNetworks of Parallel Processes. In Information Process-ing 77, pages 993–998. North Holland Publishing Com-pany, 1977.

[14] Moonzoo Kim, Yunho Kim, and Hotae Kim. A com-parative study of software model checkers as unit testingtools: An industrial case study. IEEE Trans. Softw. Eng.,37:146–160, March 2011.

[15] Salvatore La Torre, Madhusudan Parthasarathy, and Gen-naro Parlato. Analyzing recursive programs using afixed-point calculus. In Proceedings of the 2009 ACMSIGPLAN conference on Programming language designand implementation, PLDI ’09, pages 211–222, NewYork, NY, USA, 2009. ACM.

[16] Brad Long, Daniel Hoffman, and Paul Strooper. Toolsupport for testing concurrent java components. IEEETrans. Softw. Eng., 29:555–566, June 2003.

[17] Chi-Keung Luk, Robert Cohn, Robert Muth, HarishPatil, Artur Klauser, Geoff Lowney, Steven Wallace, Vi-jay Janapa Reddi, and Kim Hazelwood. Pin: buildingcustomized program analysis tools with dynamic instru-mentation. In Proceedings of the 2005 ACM SIGPLANconference on Programming language design and imple-mentation, PLDI ’05, pages 190–200, New York, NY,USA, 2005. ACM.

[18] F. Mueller. Pthreads library interface, 1993.[19] Madanlal Musuvathi and Shaz Qadeer. Iterative con-

text bounding for systematic testing of multithreaded pro-

grams. In Proceedings of the 2007 ACM SIGPLAN con-ference on Programming language design and implemen-tation, PLDI ’07, pages 446–455, New York, NY, USA,2007. ACM.

[20] V. Mutilin. Concurrent testing of java components usingjava pathfinder. In Leveraging Applications of FormalMethods, Verification and Validation, 2006. ISoLA 2006.Second International Symposium on, pages 53 –59, nov.2006.

[21] William Pugh and Nathaniel Ayewah. Unit testing con-current software. In Proceedings of the twenty-secondIEEE/ACM international conference on Automated soft-ware engineering, ASE ’07, pages 513–516, New York,NY, USA, 2007. ACM.

[22] Koushik Sen. Race directed random testing of concurrentprograms. In Proceedings of the 2008 ACM SIGPLANconference on Programming language design and imple-mentation, PLDI ’08, pages 11–21, New York, NY, USA,2008. ACM.

[23] Jaeheon Yi, Tim Disney, Stephen N. Freund, and CormacFlanagan. Types for precise thread interference. Tech-nical Report UCSC-SOE-11-22, University of Californiaat Santa Cruz, 2011.

[24] Jaeheon Yi and Cormac Flanagan. Effects for cooperableand serializable threads. In Proceedings of the 5th ACMSIGPLAN workshop on Types in language design andimplementation, TLDI ’10, pages 3–14, New York, NY,USA, 2010. ACM.

[25] Jaeheon Yi, Caitlin Sadowski, and Cormac Flanagan. Co-operative reasoning for preemptive execution. In Pro-ceedings of the 16th ACM symposium on Principlesand practice of parallel programming, PPoPP ’11, pages147–156, New York, NY, USA, 2011. ACM.