15
Hindawi Publishing Corporation International Journal of Distributed Sensor Networks Volume 2013, Article ID 953593, 14 pages http://dx.doi.org/10.1155/2013/953593 Research Article Verification of Data Races in Concurrent Interrupt Handlers Guy Martin Tchamgoue, Kyong Hoon Kim, and Yong-Kee Jun Department of Informatics, Gyeongsang National University, 660-701 Jinju, Republic of Korea Correspondence should be addressed to Yong-Kee Jun; [email protected] Received 22 May 2013; Accepted 11 September 2013 Academic Editor: Tai-hoon Kim Copyright © 2013 Guy Martin Tchamgoue et al. is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Data races are common in interrupt-driven programs and have already led to well-known real-world problems. Unfortunately, existing dynamic tools for reporting data races in interrupt-driven programs are not only unsound, but they also fail to verify the existence of data races in such programs. is paper presents an efficient and scalable on-the-fly technique that precisely detects, without false positives, apparent data races in interrupt-driven programs. e technique combines a tailored lightweight labeling scheme to maintain logical concurrency between the main program and every instance of its interrupt handlers with a precise detection protocol that analyzes conflicting shared memory accesses by storing at most two accesses for each shared variable. We implemented a prototype of this technique, called iRace, on top of the Avrora simulation framework. An empirical evaluation of iRace revealed the presence of data races in some existing TinyOS components and applications with a worst-case slowdown of only about 6 times on average and an increased average memory consumption of only about 20% in comparison with the original program execution. e evaluation also proved that the labeling scheme alone generates an average runtime overhead of only about 0.4x while consuming only about 12% more memory than the original program execution. 1. Introduction Asynchronous interrupt handling is a common mechanism in various cyber-physical systems including sensor networks, avionics, and automotive systems, or various medical devices. However, such interrupt-driven applications are hard to be thoroughly tested and debugged since they have to deal with external asynchronous interrupts that exponentially increase the number of their execution paths at runtime. Moreover, asynchronous interrupts introduce fine-grained concurrency into interrupt-driven programs making them prone to data races [1, 2] which occur when two threads access a shared memory location without proper synchronization, and at least one of the accesses is a write. Data races in interrupt-driven programs have already led to real-world problems like the well-known accident of the erac-25 [3] where, as the result of a data race caused by the keyboard interrupt handler, many people received fatal radiation doses. Another example of data race is certainly the GE XA/21 bug [4] that, in August 2003, caused a total blackout for about 50 million people in the Northeast United States and Canada. It becomes therefore crucial to precisely detect data races ahead of the exploitation phase. Data races are oſten classified into feasible data races that are data races based on the possible behavior of a program and apparent data races, which are defined in the context of a specific program execution [2]. Apparent data races which are only based on the behavior of the explicit synchronization in a program, represent an approximation of the feasible data races that are the real data races that might happen in any specific execution of the program. Apparent data races are less accurate but easier to detect than feasible data races, whose detection has been proven NP-hard [2]. Few dynamic race detectors [5, 6] have recently been pro- posed for detecting data races in interrupt-driven programs. However, legacy race detectors for multithreaded programs [79] are very intrusive and cannot directly be applied to interrupt-driven due not only to differences between the programming models, but also to the high overhead they incur. us, Lee et al. [6] proposed to convert an interrupt- driven program into a corresponding multithreaded program and to use an existing dynamic race detection tool to detect data races into the newly generated program. is technique fails to capture the semantics of interrupts and incurs high space and time overheads due to the use of tools basically

Research Article Verification of Data Races in Concurrent ...downloads.hindawi.com/journals/ijdsn/2013/953593.pdf · photoTask I20 W 11 R 12 W 13 W 24 R 15 T 2 J12 T 1 F : A POEG

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Research Article Verification of Data Races in Concurrent ...downloads.hindawi.com/journals/ijdsn/2013/953593.pdf · photoTask I20 W 11 R 12 W 13 W 24 R 15 T 2 J12 T 1 F : A POEG

Hindawi Publishing CorporationInternational Journal of Distributed Sensor NetworksVolume 2013 Article ID 953593 14 pageshttpdxdoiorg1011552013953593

Research ArticleVerification of Data Races in Concurrent Interrupt Handlers

Guy Martin Tchamgoue Kyong Hoon Kim and Yong-Kee Jun

Department of Informatics Gyeongsang National University 660-701 Jinju Republic of Korea

Correspondence should be addressed to Yong-Kee Jun jungnuackr

Received 22 May 2013 Accepted 11 September 2013

Academic Editor Tai-hoon Kim

Copyright copy 2013 Guy Martin Tchamgoue et alThis is an open access article distributed under the Creative Commons AttributionLicense which permits unrestricted use distribution and reproduction in anymedium provided the originalwork is properly cited

Data races are common in interrupt-driven programs and have already led to well-known real-world problems Unfortunatelyexisting dynamic tools for reporting data races in interrupt-driven programs are not only unsound but they also fail to verify theexistence of data races in such programs This paper presents an efficient and scalable on-the-fly technique that precisely detectswithout false positives apparent data races in interrupt-driven programs The technique combines a tailored lightweight labelingscheme to maintain logical concurrency between the main program and every instance of its interrupt handlers with a precisedetection protocol that analyzes conflicting shared memory accesses by storing at most two accesses for each shared variable Weimplemented a prototype of this technique called iRace on top of the Avrora simulation framework An empirical evaluation ofiRace revealed the presence of data races in some existing TinyOS components and applications with a worst-case slowdown ofonly about 6 times on average and an increased average memory consumption of only about 20 in comparison with the originalprogram executionThe evaluation also proved that the labeling scheme alone generates an average runtime overhead of only about04x while consuming only about 12 more memory than the original program execution

1 Introduction

Asynchronous interrupt handling is a common mechanismin various cyber-physical systems including sensor networksavionics and automotive systems or variousmedical devicesHowever such interrupt-driven applications are hard to bethoroughly tested and debugged since they have to deal withexternal asynchronous interrupts that exponentially increasethe number of their execution paths at runtime Moreoverasynchronous interrupts introduce fine-grained concurrencyinto interrupt-driven programs making them prone to dataraces [1 2] which occur when two threads access a sharedmemory location without proper synchronization and atleast one of the accesses is a write

Data races in interrupt-driven programs have already ledto real-world problems like the well-known accident of theTherac-25 [3] where as the result of a data race caused bythe keyboard interrupt handler many people received fatalradiation doses Another example of data race is certainlythe GE XA21 bug [4] that in August 2003 caused a totalblackout for about 50 million people in the Northeast UnitedStates and Canada It becomes therefore crucial to preciselydetect data races ahead of the exploitation phase Data races

are often classified into feasible data races that are data racesbased on the possible behavior of a program and apparentdata races which are defined in the context of a specificprogram execution [2] Apparent data races which are onlybased on the behavior of the explicit synchronization in aprogram represent an approximation of the feasible dataraces that are the real data races that might happen in anyspecific execution of the programApparent data races are lessaccurate but easier to detect than feasible data races whosedetection has been proven NP-hard [2]

Few dynamic race detectors [5 6] have recently been pro-posed for detecting data races in interrupt-driven programsHowever legacy race detectors for multithreaded programs[7ndash9] are very intrusive and cannot directly be applied tointerrupt-driven due not only to differences between theprogramming models but also to the high overhead theyincur Thus Lee et al [6] proposed to convert an interrupt-driven program into a correspondingmultithreaded programand to use an existing dynamic race detection tool to detectdata races into the newly generated program This techniquefails to capture the semantics of interrupts and incurs highspace and time overheads due to the use of tools basically

2 International Journal of Distributed Sensor Networks

designed for multithreaded programs Erickson et al [5]presented a technique that aims to detect data races in low-level kernel codes by randomly sampling parts of the programto be used as candidates for the race detection and using dataand code breakpoints to detect conflicting accesses to sharedvariables Although some data races caused by interruptsmaybe reported the tool still reports many false positives andfails to guarantee that programs are free from data racesMoreover existing testing methods for interrupt-driven pro-grams [11ndash14] are not sufficient for debugging since they canonly reveal the presence of data races but fail to adequatelydetect their root causes and therefore require importantmanual inspections for real data race detection Thus somedata races still remain undetectable andmake their way to theexploitation phase leading the application into unpredictableexecutions sometimes with severe consequences

Dynamic data race detection methods are generallyclassified into on-the-fly [5ndash9] and postmortem [15 16] tech-niques While postmortem techniques collect runtime datafor post-execution analysis on-the-fly techniques on theother handwill dynamically detect data races at runtime On-the-fly techniques still require less space overhead than post-mortem techniques due to the fact that unnecessary informa-tion is discarded at runtime as the detection progresses On-the-fly techniques are generally based either on the happens-before analysis [7 8 17 18] the lockset analysis [19] orthe hybrid analysis [9 20] that is a combination of the firsttwo However on-the-fly techniques still suffer from the highruntime overhead they incur Dynamic analysis methods canbe a good complement to static analysis methods [21 22]since they guarantee to isolate real data races in a particularexecution instance of a program given an input from thepotential data races reported by static analysis

This paper presents an on-the-fly technique that effi-ciently detects apparent data races in interrupt-driven pro-grams without false positives The technique combines atailored lightweight labeling scheme to maintain the logicalconcurrency between a program and every instance of itsinterrupt handlers with a precise detection protocol thatanalyzes conflicting accesses to shared memories by storingat most two accesses for each shared variable The labelingscheme generates unique identifiers to track and analyzethe happens-before relation between the main program andeach of its interrupt handlers The precise detection protocoleases the debugging process since it guarantees to verify theexistence of data races in themonitored programby reportingat least one data race for each shared variable involved ina data race Moreover the detection protocol shows a spacecomplexity of only 119874(1198811198792) while any pair of labels can becompared in time 119874(Log119879) where 119879 designates the totalnumber of interrupts supported by the underlying system and119881 the number of shared variables in the program

We implemented a prototype of our technique callediRace on top of the Avrora simulation and analysis frame-work [23] that provides a time accurate simulator and aninstrumentation engine [24] for programs written for theAVR microcontrollers and sensor networks An empiricalevaluation of iRace revealed the presence of data races insome existing TinyOS [25] components and applications

with a worst-case slowdown of only about 6 times onaverage and an increased average memory consumption ofonly about 20 in comparison with the original programexecution Moreover this evaluation also proved that thelabeling scheme alone generates an average runtime overheadof only about 04x with an average memory consumption ofonly about 12 relatively to the original program execution

The remainder of this paper is organized as followsSection 2 presents some properties of interrupt-driven pro-grams and introduces the notion of data races Section 3focuses on our lightweight labeling scheme for interrupt-driven programs Section 4 presents our precise data racedetection protocol In Section 5 a theoretical analysis ofthe proposed race detection technique is given Section 6discusses the implementation challenges of the technique andpresents our evaluation results The conclusion and futurework come in Section 7

2 Background

This section presents the interrupt-driven programs withsome important properties of interrupts and introduces thenotion of data races together with some useful definitions

21 Interrupt-Driven Programs Interrupts are generallydefined as hardware signals as they are generated by thehardware in response to an external operation or envi-ronment change A system is then interrupt driven if asignificant amount of its processing is initiated and influ-enced by external interrupts [13 26] For systems based onmore powerful processors interrupts are commonly man-aged inside the operating system kernel hiding their com-plexity from programmers However for highly resource-constrained embedded systems with a thin kernel layer likeTinyOS [25] applications have to provide and manage theirown interrupt handlers [13 26] We will focus our attentionon the latter since this kind of applications is likely to containdata races Interrupt handling is a common mechanism invarious cyber physical systems including embedded sensornetworks avionics and automotive systems or various med-ical devices However since interrupt-driven programs comewith different flavors our description will be based on thenesC language [21] used for TinyOS applications

An interrupt handler is an asynchronous callback routineattached to each interrupt to service it whenever it arrivesin the system When an interrupt is received it remainspending until it is delivered or handled Contrarily to threadsinterrupt handlers cannot block they run to completionexcept when preempted by another interrupt handler [1321 26] Interrupt handlers have an asymmetric preemptionrelation with the noninterrupt code or task in nesC taskscan be preempted by interrupt handlers but not the contraryTasks in nesC are not allowed to preempt themselves theyare ordered and run in a first-in first-out manner Interruptsare nested when they preempt each other Nested interruptsare used to allow time-sensitive operations to be handledwith low latency An interrupt is reentrant when it directlyor indirectly preempts itself

International Journal of Distributed Sensor Networks 3

(1) TOS Msg msg lowast shared variable lowast(2) uint8 t packet Index lowast shared variable lowast(3) async event result t ADCdataReady (uint16 t data) (4) struct OscopeMsg lowastpack(5) pack = (struct OscopeMsg lowast) msgdata(6) pack-gtdata [packetIndex++] = data(7) if (packetIndex == BUFFER SIZE) (8) post dataTask ( )(9)

(10)

(11) async event result t PhotoSensordataReady (uint16 t data) (12) post photoTask ( )(13)

(14) task void dataTask ( ) (15) struct OscopeMsg lowastpack(16) atomic (17) pack = (struct OscopeMsg lowast)msgdata(18) check (pack)(19) packetIndex = 0(20)

(21) pack-gtchannel = 1(22) pack-gtsourceMoteID = TOS LOCAL ADDRESS(23) send (ampmsg)(24)

(25) task void photoTask ( ) lowast do something lowast

Algorithm 1 An example nesC program

In order to minimize the impact of interrupt handlerson tasks all long-latency operations must run in split-phaseWith this mechanism interrupt handlers immediately returnafter servicing critical operations and post heavy compu-tations for later execution as new tasks The concurrencymodel of interrupt-driven programs allows at runtime anexponentially increasing number of execution paths makingthem difficult to reason about Since the only way to sharedata with interrupt handlers is through global variablesinterrupt-driven programs are prone to concurrency bugslike data races [1 2 13 26] To protect sensitive parts of theprogram interrupts must be disabled before and enabled onlyafter critical sections Interrupts can be either asynchronousor synchronous Asynchronous interrupts are signaled byexternal devices such as network interfaces and can fireat any time that the corresponding device is enabled [13]Synchronous interrupts on the other hand are those thatarrive in response to a specific action taken by the processorsuch as setting a timer

22 Data Races In shared-memory concurrent programsdata races [1 2 27] represent one of the most notorious classof concurrency bugs Data races occur when two threadsaccess a shared memory location without proper synchro-nization and at least one of the accesses is a write It is impor-tant to detect data races because they may lead programs tonondeterministic executions with unpredictable results

For illustration we consider the example program ofAlgorithm 1 that is adapted from theOscilloscope applicationdistributed with TinyOS [10] This application allows a node

of a sensor network to periodically sample cache and senddata from its sensors We assume that interrupt handlers inAlgorithm 1 are not reentrant and that they automaticallydisable themselves before they run We define a partial orderexecution graph (POEG) [28] to represent an executioninstance of an interrupt-driven program Unless explicitlymentioned wewill use the term thread to refer to either a taskor an interrupt handlerTherefore we define a thread segmentas a sequence of instructions executed by a single thread anddelimited by either a disabling or an enabling instructionThus a thread119879

119896is a finite set of 119899 sequentially ordered thread

segments that is 119879119896= 1199051 1199052 119905

119899 where 119899 represents the

number of disableenable instructions plus one and 119896 a taskidentifier or an interrupt number We therefore introduce thefollowing definitions

Definition 1 (POEG) A POEG is a directed acyclic graph of2-tuple 119866 = (119878 119864) where 119878 is a set of thread segments and119864 a set of synchronization edges A synchronization edge iseither a task posting edge for posting operations or a taskordering edge to represent the implicit order relation thatexists between tasks

Definition 2 (happens-before [29]) Given a program execu-tion if a thread segment 119905

119894runs earlier than another thread

segment 119905119895 then 119905

119894happens-before 119905

119895 denoted 119905

119894

ℎ119887

997888997888rarr 119905119895 If

the happens-before relation (ie 119905119894

ℎ119887

997888997888rarr 119905119895or 119905119895

ℎ119887

997888997888rarr 119905119894) is

not satisfied between 119905119894and 119905119895 the two thread segments are

concurrent and denoted by 119905119894 119905119895

4 International Journal of Distributed Sensor Networks

ADC20

20

20

20

12

12

Task posting edgeTask ordering edge

lowast

lowast

PhotoSensor

dataTask

photoTask

I20W11

R12

W13

W24

R15

T2

J12

T1

Figure 1 A POEG of an execution instance of the program inAlgorithm 1 Only accesses to msg are represented

In a POEG two accesses are concurrent if and onlyif there is no path between them Otherwise they areordered by the happens-before relation An example POEGfor the program in Algorithm 1 considering only accesses tothe shared variable msg is represented in Figure 1 In thisPOEG vertical lines symbolize threads while read and writeaccesses to shared variables are respectively represented bythe letters 119877

119894119895and119882

119894119895 where 119894 is the thread instance number

and 119895 an observed execution order We use the couple ofsymbols⊓119899 and⊔119899 to respectively represent the disabling andenabling of interrupts where 119899 is the corresponding interruptnumber that will be replaced with the star symbol (lowast) ifall interrupts are concerned The POEG of Figure 1 depictsan execution where a first instance of the ADC (Anolog-to-Digital Converter) interrupt handler collected new dataand posted an instance of dataTask Furthermore anotherinstance of the ADC interrupt arrives and preempts theexecution of dataTask

Although the first instance of the ADC interrupt handlerposted dataTask if the second ADC interrupt is receivedright before the message is sent at line 23 of Algorithm 1 dataraces will still occur between the write access in ADC andthe accesses 119882

13and 119877

15in dataTask making a corrupted

message to be delivered Those are the data races representedin Figure 1 by the pair of accesses 119882

13-11988224

and 11987715-11988224 If

a task is posted by an interrupt handler the task postingedge guarantees a happens-before relation only between theposted task and the actual instance of the interrupt handlerthat posted it but not with future instances of the sameinterrupt handler Consequently we ignore the task postingedge when traversing the graph for analyzing the happens-before relation because we use a single arrow to represent

all instances of the same interrupt Thus we can also reportthe data races 119882

13-11988211

and 11987715-11988211

between 1198791and 119868

20

Therefore in our POEG we assume that two thread segments119905119894and 119905119895 respectively from two different threads119879

119894and119879119895are

always concurrent unless they are ordered by a task orderingedge or one of them is disabled by the other one The nesClanguage [21] contains a static data race detector that howeveris still unsound since it cannot follow pointers [12 21 22]Thus the data races presented in Figure 1 cannot be reportedby the native nesC data race detector

3 Lightweight Labeling Scheme

For an effective dynamic race detection it is essentialto adequately determine the order of precedence betweenthread segments accessing shared memory locations Forthis purpose this section presents our lightweight labelingscheme used for capturing the logical concurrency betweenthread segments of interrupt-driven programs

31 GeneratingThread Labels Happens-before based on-the-fly race detection techniques generally rely on a variationof Lamportrsquos logical timestamp [29] in order to preciselyidentify concurrent thread segments of a program Theeffectiveness of on-the-fly techniques is highly dependenton the cost and accuracy of this identification process Ourlabeling scheme generates a unique identifier called label foreach thread segment 119905

119894during an execution instance of an

interrupt-driven program For the labeling scheme we needto introduce the following definitions

Definition 3 (critical section) A critical section with respectto a given interrupt 119896 is a portion of program comprisingbetween a disable and probably an enable instruction tointerrupt 119896

Definition 4 (interrupt set) The interrupt set or 119894119904119890119905 of athread segment 119905

119894represents the set of all interrupts 119896disabled

by 119905119894Thus this guarantees a happens-before relation between

119905119894and every interrupt in its 119894119904119890119905 Formally for all 119896 isin 119905

119894sdot 119894119904119890119905

for all 119905119895isin 119879119896 119905119894

ℎ119887

997888997888rarr 119905119895or 119905119895

ℎ119887

997888997888rarr 119905119894

Definition 5 (critical section violation) A thread segment 119905119894

of a thread 119879119896violates the critical section of another thread

segment 119905119895 denoted by 119905

119894

V999492999484 119905119895 when 119905

119895can be preempted by

119905119894

The label of each thread segment 119905119894consists of two compo-

nents an identifier referred to as segment descriptor 119878119863119894 and

an interrupt set 119894119904119890119905119894 Put together we use 119878119863

119894⟨119894119904119890119905119894⟩ to desig-

nate the label of a thread segment 119905119894 The segment descriptor

119878119863119894of a thread segment 119905

119894is defined as a triplet ⟨119905119905

119894 119905119894119889119894 119888119904119894⟩

where 119905119905119894specifies whether 119905119894119889

119894is a task or an interrupt

handler 119905119894119889119894contains the task identifier of a task or the

interrupt number of an interrupt and 119888119904119894indicates whether

the thread segment has disabled all interrupts in the systemCreating labels for thread segments of an interrupt-driven

program is straightforward with our labeling scheme as

International Journal of Distributed Sensor Networks 5

suggested by the algorithms in Algorithm 2 A new labeledthread 119879

119894is created when a new task starts its execution or

whenever an interrupt handler is invoked When a new taskstarts a new label is generated for the first thread segment ofthe task by simply initializing the component of the label asin Algorithm 2(a) A new label is similarly created when aninterrupt handler is invoked in Algorithm 2(b) Each threadthen creates additional thread segments whenever it disablesor enables interrupts

When a thread segment 119905119894of a thread 119879

119894disables an

interrupt 119896 the label of the new thread segment 119905119895is generated

by adding the interrupt number 119896 into the interrupt set ofthe current label owned by 119905

119894 However if all interrupts in

the system are disabled by 119905119894 the new label for 119905

119895is obtained

by setting 119888119904119894and by emptying 119894119904119890119905

119894from the current label

owned by 119905119894as shown in Algorithm 2(c) Thus as seen in

Algorithm 2(d) if 119905119894enables an interrupt 119896 a new label for a

new thread segment 119905119895is obtained by simply removing 119896 from

the interrupt set of the current label in 119905119894 If all interrupts are

enabled at the same time the flag 119888119904119894and the interrupt set

119894119904119890119905119894are both reset from the current label of 119905

119894to generate

the new label for the new thread segment 119905119895 This helps

maintain a reduced-size interrupt set for each thread segmentsince programmers generally disable and enable all interruptsrather than manipulating individual interrupts

Moreover the size of an interrupt set is bounded by thetotal number of interrupts supported by the hardware onwhich the program runs which is generally a reasonableinteger number By reusing the same label for differentinstances of the same interrupt handler our labeling schemedoes not depend on the total number of interrupts receivedat runtime which can be very high

Lemma 6 Given two labeled thread segments 119905119894and 119905119895 with

119905119894belonging to an interrupt handler 119879

119896 119905119894

V999492999484 119905119895rArr 119896 notin 119894119904119890119905

119895or

119888119904119895= 119891119886119897119904119890

Proof Let us consider two labeled thread segments 119905119894and 119905119895

and let us assume that 119905119894belongs to an interrupt handler 119879

119896

We also assume that 119905119894violates the critical section of 119905

119895 By

definition thread segments are delimited by either the disableor enable instructions Therefore if 119905

119895was created after a

the disable instruction targeting interrupt 119896 the interruptnumber 119896 will be in the interrupt set 119894119904119890119905

119895of 119905119895according

to the labeling algorithms of Algorithm 2 However if 119905119895was

created after the disabling of all interrupts 119888119904119895would then be

true by the labeling algorithms of Algorithm 2 to signify thatall interrupts including 119896 are disabledThus for all 119905

119894isin 119879119896 119905119894

cannot violate 119905119895 which is contradictory since we assumed

119905119894

V999492999484 119905119895 Therefore 119905

119895has necessarily enabled 119896 that is

for all 119905119894isin 119879119896 119905119894

V999492999484 119905119895rArr 119896 notin 119894119904119890119905

119895or 119888119904119895= 119891119886119897119904119890

32 Concurrency Relation Using our labeling scheme itbecomes easy to analyze the happens-before relation betweenany pair of thread segments 119905

119894and 119905119895 by comparing their

labels

[a Task Starting]a1 tsk start()a2 Label Lba3 Lbt = truea4 Lbtid = getTaskid()a5 Lbcs = falsea6 Lbiset =

a7 return Lb[b Interrupt Invocation]b1 int invoke()b2 Label Lbb3 Lbt = falseb4 Lbtid = getIntNumber()b5 Lbcs = falseb6 Lbiset =

b7 return Lb[c Interrupt Disabling]c1 int disable(Label Lb Interrupt n)c2 if(n gt 0) Lbiset cup 119899

c3 elsec4 Lbcs = truec5 Lbiset =

c6 endifc7 return Lb[d Interrupt Enabling]d1 int enable(Label Lb Interrupt n)d2 if(n gt 0) Lbiset-nd3 elsed4 Lbcs = falsed5 Lbiset =

d6 endifd7 return Lb

Algorithm 2 Labeling algorithms

Theorem7 A labeled thread segment 119905119894happens before anoth-

er labeled thread segment 119905119895 denoted by 119905

119894

ℎ119887

997888997888rarr 119905119895 if and only if

either that 119905119894and 119905119895belong to tasks or that 119905

119895does not violate

119905119894 that is not(119905

119895

V999492999484 119905119894)

Proof Let us assume two labeled thread segments 119905119894and

119905119895 such that 119905

119894

ℎ119887

997888997888rarr 119905119895 Thus according to the labeling

algorithms of Algorithm 2 we should have 119905119905119894= 119905119905119895= 119905119903119906119890

meaning that 119905119894and 119905

119895are either from the same or from

different tasks By definition tasks are all ordered and cannotpreempt each other However if 119905

119895is a thread segment of

an interrupt handler 119879119896 according to Definition 4 either

119888119904119894= 119905119903119906119890 because 119905

119894disabled all interrupts or the thread

identifier 119905119894119889119895of 119905119895that corresponds to 119896 the interrupt

number of 119879119896 is in 119894119904119890119905

119894 the interrupt set of 119905

119894 This means

that 119905119894is a critical section thread segment that disables either

all interrupts or particularly the interrupt number 119896 Thusaccording to Lemma 6 119905

119895does not violate the critical section

of 119905119894 Therefore we can say that 119905

119894

ℎ119887

997888997888rarr 119905119895implies either that 119905

119894

and 119905119895belong to tasks or that not(119905

119895

V999492999484 119905119894)

6 International Journal of Distributed Sensor Networks

Now let us assume that 119905119894and 119905

119895belong to tasks

Naturally 119905119894and 119905119895are ordered and the labeling algorithms of

Algorithm 2 will set 119905119905119894= 119905119905119895= 119905119903119906119890 meaning that 119905

119894

ℎ119887

997888997888rarr 119905119895

On the other hand if we assume that not(119905119895

V999492999484 119905119894) then 119905

119895

must be part of a given interrupt handler 119879119896and either 119905

119894has

disabled all interrupts to set 119888119904119894= 119905119903119906119890 or the thread identifier

119905119894119889119895= 119896 of 119905

119895is in the interrupt set 119894119904119890119905

119894of 119905119894 according to

Definition 4 and Lemma 6 Therefore we can conclude that119905119894

ℎ119887

997888997888rarr 119905119895

Any two labeled thread segments 119905119894and 119905119895are therefore

concurrent if they do not satisfy the happens-before relationas stated by Theorem 7 which can formally be rewritten asfollows

119905119894

ℎ119887

997888997888rarr 119905119895lArrrArr

119905119905119894= 119905119905119895= 119905119903119906119890

or(119905119905119895= 119891119886119897119904119890) and (119888119904

119894= 119905119903119906119890 or 119905119894119889

119895isin 119894119904119890119905119894)

(1)

By comparing based on Theorem 7 the labels of thethread segments in the POEG represented by Figure 2 wecan easily obtain that the thread segments of 119868

20 1198941 and 119894

2

are concurrent with other thread segments 1199051 1199053 1199054 and

1198951 Therefore the write accesses in 119894

1and 1198942conflict with

the other accesses in 1199053 leading to four data races 119882

11-11988213

11988211-1198771511988224-11988213 and119882

24-11987715 However since the accesses

11988211and119882

23 respectively from 119894

1and 1198942 represent the same

write access the set of data races to be reported for an effectivedebugging process should be reduced to only 119882

11-11988213

and11988211-11987715 By disabling all interrupts 119905

2enforces the happens-

before relation with other thread segments Although 1198791is

posted by 11986820 they are still concurrent Also the thread

segment 1198951is concurrent with 119894

1 1198942 1199051 1199053 and 119905

4

4 Precise Detection Protocol

The race detection protocol monitors memory accesses bymaintaining a special data structure called access history intowhich are stored the labels of prior accesses to each sharedvariable Our detection protocol stores in each entry of theaccess history of a shared variable a unique label representingthe thread segment that accessed the variable as describedin Section 3 At each access to a shared variable data racesare reported by analyzing based onTheorem 7 the happens-before relation between the current thread segment and thelabels of prior thread segments stored in the access historyTherefore the access history for each shared variable hasto be efficiently organized in order to reduce the overheadgenerated by the detection process

Our precise detection protocol maintains an access his-tory consisting of two sets for each shared variable a Readset and a Write set to respectively store labels of threadsegments that read and write the variable We use 119860119867(119909)

to designate the access history of a shared variable 119909 Thus119860119867(119909) sdot 119877119890119886119889 and 119860119867(119909) sdot 119882119903119894119905119890 refer to the read set andthe write set of 119860119867(119909) respectively The algorithms used bythe detection protocol are presented in Algorithm 3 In thesealgorithms we assume that the variable 119871119887 always contains

20

20

20

20

t1

t2

t3

t4

12

12

lowast

lowast

i1⟨0 20 0⟩⟨20⟩

⟨mdash⟩⟨1 1 1⟩

⟨mdash⟩⟨1 1 0⟩

⟨mdash⟩⟨1 1 0⟩

⟨0 12 0⟩⟨12⟩

⟨0 20 0⟩⟨20⟩

⟨mdash⟩⟨1 2 0⟩

I20

W11

R12

W13

W24

R15

T2

J12

T1

j1

i2

Figure 2 An example labeled POEG for an execution instance ofthe program in Algorithm 1

the label of the current thread segment accessing a sharedvariable and that 119901119871119887 represents any prior label stored in theaccess history

When a read access occurs to a shared variable 119909the detection protocol calls the ah read() function ofAlgorithm 3(a) that then reports data races by analyzing thehappens-before relation between the label of the accessingthread segment and the labels of prior accesses stored in119860119867(119909) sdot119882119903119894119905119890 Similarly upon a write access to a shared vari-able the monitoring function ah write() of Algorithm 3(b)reports data races by looking into both 119860119867(119909) sdot 119877119890119886119889 and119860119867(119909) sdot 119882119903119894119905119890 We assume the existence of a functionOrdered() that based on Theorem 7 decides whether athread segment 119905

119894happens before another thread segment 119905

119895

by comparing their labelsIn order to maintain a reduced size access history

accesses from critical sections that concern all interruptswithin a task are not inserted in the access history sincethey will never run into a data race with other accessesfrom interrupt handlers Also only the first read and writeaccesses to a shared variable from the same thread segmentare kept into the access history Subsequent similar accessesto the same shared variable from the same thread segmentare filtered outThis filtering operations are performed by theisFiltered() function of Algorithm 3(c) that is called by bothah read() and ah write() before any other action

After data races have been reported with the currentaccess the function maintain ah() of Algorithm 3(d) is usedfor keeping the size of the access history balanced Thisfunction replaces some previous accesses in the access historywith the current access Thus accesses from different tasksare considered from the same thread since tasks by defini-tion are all ordered by happens-before Similarly accesses

International Journal of Distributed Sensor Networks 7

(1) ah read(Label Lb Access History AH Variable x)(2) if (isFiltered(Lb AH(x)Read)) then return(3) for all labels pLb in AH(x)Write do(4) if (Not Ordered(Lb pLb)) then(5) report race(x pLb Lb)(6) end for(7) maintain ah(Lb AH(x)Read)

(a) Monitoring Protocol for Read Accesses(1) ah write(Label Lb Access History AHVariable x)(2) if (isFiltered(Lb AH(x)Write)) then return(3) for all labels pLb in(4) AH(x)Write cup AH(x)Read do(5) if (Not Ordered(Lb pLb)) then(6) report race(x pLb Lb)(7) end for(8) maintain ah(Lb AH(x)Write)

(b) Monitoring Protocol for Write Accesses(1) isFiltered(Label Lb Access History AH)(2) if (Lbcs = true) and (Lbtt = true)(3) then return true(4) if (any access with a label pLb exists in AH(5) such that pLb = Lb) then return true(6) else(7) return false

(c) Filtering Redundant Accesses(1) maintain ah(Label Lb Access History AH)(2) if (Lbtt = true) then(3) delete all entries in AH with label pLb(4) such that pLbtt = true(5) else(6) delete all entries in AH with label pLb such(7) that pLbtt = Lbtt and pLbtid = Lbtid(8) end if(9) add ah(Lb AH)

(d) Maintaining Access History

Algorithm 3 Labeling algorithms

from different instances of the same interrupt handler areconsidered from a single instance of the interrupt handlerThis property is already enforced by the labeling scheme thatreuses the same label for all instances of the same interrupthandler Again we assume the existence of another functionadd ah() which is able to insert the label of the current threadsegment into the appropriate set of the access history given ashared variable

Table 5 shows the different states of an access historyfor detecting data races with our protocol in the exampleprogram of Figure 2 where accesses to the shared variablemsg appear in the first column When the first access 119882

11

occurs the label ⟨0 20 0⟩⟨20⟩ of the thread segment 1198941

is simply added into the access history and no data racereported as there is no other access prior to 119882

11 The next

access 11987712 from the thread segment 119905

2is filtered out because

1199052that belongs to task 119879

1has disabled all interrupts The

following access 11988213

occurs on 1199053with interrupts enabled

Thus a data race11988213-11988211is reported by comparing the label

of 1199053 ⟨1 1 0⟩⟨minus⟩ with the label of 119894

1previously stored into the

access history Afterwards the label of 1199053is inserted into the

write set of the access history The thread segment 1199053is then

preempted by 1198942 but the access 119882

24is filtered out because

1198942which is a new instance of the interrupt 119868

20 also shares the

same label ⟨0 20 0⟩⟨20⟩ with 1198941which previously performed

a write and whose label is already in the access history Afterthe execution of 119894

2 1199053resumes to perform the read access

11987715 Another data race 119877

15-11988211

is reported and the label of1199053 ⟨1 1 0⟩⟨minus⟩ stored into the read set of the access history

Although the program received two instances of the ADCinterrupt our protocol precisely reported only the set of twodata races needed for effectively debug the program Thisshows that our detection protocol does not depend on thenumber of interrupts received at runtime which can be veryhigh

Although it keeps only a reduced set of labels in the accesshistory our precise detection protocol even with a minimalnumber of interrupts still guarantees to verify the existenceof data races in a program For this purpose the protocol isdesigned to report without false positives at least one datarace for each shared variable involved in a data race

8 International Journal of Distributed Sensor Networks

Theorem 8 Given an interrupt-driven program 119875 the detec-tion protocol reports a data race over a shared variable 119909 if andonly if there exists a data race over 119909 in an execution instanceof 119875

Proof We will first prove that if a data race is reported on ashared variable 119909 by the detection protocol then there alsoexists a data race in an execution instance of the program forthe same shared variable 119909 For this we assume a snapshotof the access history 119860119867(119909) of the shared variable 119909 at agiven time and the current access event 119886

119894to 119909 from a thread

segment 119905119894that owns a label we refer to as 119871119887

119894in an execution

instance of the program 119875 We can then distinguish twodifferent cases

Case A 119860119867(119909) is empty and there is obviously no data racefor 119909 In this case the current access 119886

119894is the only access to

119909 from the program 119875 and 119871119887119894will be added by the function

add ah() into119860119867(119909)sdot119877119890119886119889 if 119886119894is a read access or into119860119867(119909)sdot

119882119903119894119905119890 if it is a write access

Case B 119860119867(119909) is not empty Data races are reported usingthe ah read() function if 119886

119894is a read access or the ah write()

in case of a write access In both cases the happens-beforerelation is analyzed between the label 119871119887

119894of 119905119894and previous

labels stored in 119860119867(119909) according to Theorem 7 The outputsof these analyses can be classified into two categories

(1) There is no data race between 119886119894and119860119867(119909) therefore

all previous accesses stored in 119860119867(119909) happenedbefore 119886

119894meaning that no thread segment prior to

119905119894in 119875 is concurrent with 119905

119894 they are all ordered by

happens-before Thus for all 119886119895isin 119905119895and 119871119887119895isin 119860119867(119909)

119905119895

ℎ119887

997888997888rarr 119905119894 In which case 119875 also has no data race

(2) There are data races between 119886119894and prior accesses

in 119860119867(119909) this means that there are some labels in119860119867(119909) belonging to some prior thread segments thatare not ordered with 119905

119894 Concretely there are thread

segments in 119875 that are concurrent with 119905119894 Formally

exist119905119895isin 119875119871119887

119895isin 119860119867(119909) 119905

119894 119905119895Therefore the execution

instance of 119875 also contains data races over 119909

Now we show that if there exists a data race over a sharedvariable 119909 in an execution instance of the program 119875 thenour detection protocol also reports a data race over 119909 Usingthe contrapositive we will prove that if the detection protocoldoes not report any data race the program 119875 also does notcontain any for a shared variable 119909 We again consider asnapshot of the access history 119860119867(119909) for a shared variable119909 at a given time We also assume that the detection protocolreported no data races for the shared variable 119909 This meansthat 119860119867(119909) is either empty and there is clearly no data racesince there is no access to 119909 from 119875 yet or contains onlylabels of thread segments that are all ordered by the happens-before relation in which case there is no concurrent threadsegment in119875 which also implies that119875 has no data races over119909 Therefore we can formally write for all 119905

119894 119905119895isin 119875 and 119871119887

119894

119871119887119895isin 119860119867(119909) 119905

119894

ℎ119887

997888997888rarr 119905119895or 119905119895

ℎ119887

997888997888rarr 119905119894 Thus 119875 also contains no

data race for 119909

Table 1 Efficiency of the precise detection protocol

Space TimeNew label comparison Per access

O(1198811198792) O(1) O(LogT) O(TLogT)

5 Theoretical Analysis

On-the-fly data race detection techniques are known to havehigh runtime overhead that is generally provoked by the timespentmonitoring accesses to shared variables and to comparethread labels In this section we theoretically analyze theefficiency of our technique which depends on119881 the numberof shared variables and 119879 the total number of interruptsin the underlying system plus one the number of tasksIn general 119879 is very small since the number of interruptshandled by a system is also kept small For example thepopular Atmel ATmega128 [30] processor supports only amaximum of 34 interrupts Also because of the implicithappens-before relation that exists between all tasks they areviewed as a single task by the detection protocol

Table 1 presents the space and time complexities of ourlabeling and detection techniques The access history usedby our technique stores from each thread segment at mosttwo accesses for each shared variable Each label in the accesshistory has a worst-case size of 119874(119879) that represents thesize of its interrupt set Therefore the total labeling spacerequirement for each shared variable is 119874(1198792) Thus thetotal space complexity of our technique is 119874(1198811198792) Howeverbecause the number of interrupts is generally a reasonableinteger number the space complexity of our technique canfurther be reduced to 119874(119881)

A new label is always created in a constant amount of timeby our labeling algorithms Interrupts are generally givennumbers in a sequence order Therefore a binary searchtechnique is applied to locate values in interrupt sets whichcontain only interrupt numbers Thus the time complexityfor comparing any two labels is 119874(Log119879) in the worst-caseand 119874(1) in most cases where the interrupt sets are emptyAt each access to a shared variable the total time complexityto report data races with the previous accesses stored in theaccess history is then119874(119879Log119879)The labels and the detectionprotocol do not depend on the total number of interruptsreceived at runtimemaking our technique highly scalable andpractical

6 Experimental Results

We empirically evaluated the preciseness and efficiency ofour technique using a set of well-known programs providedwith TinyOS [10] This section details the implementationchallenges and presents the analysis results of the experimen-tation

61 Implementation We implemented a prototype of ourdata race detection tool for interrupt-driven programs callediRace based on the concurrency model of TinyOS [21] andits nesC language [25] We implemented iRace on top

International Journal of Distributed Sensor Networks 9

Table 2 Characteristics of the benchmark applications from TinyOS 210 [10]

Applications LOC Int SV Brief description6LowPanCli 19116 4 9 Implements a command line interface using the 6lowpan stackAnthiTheft 27421 5 4 Detects and reports a stolen mote by checking some sensorsBaseStation 14905 6 13 Sends and receives simple active messages on the radio and serial linksBlink 3688 1 0 Periodically blinks the three LEDs of a moteMultihopOscilloscope 24310 9 8 Periodically samples and broadcasts sensor data to a base stationOscilloscope 11686 5 5 Periodically samples sensor data and broadcasts a message every 10 timesPowerup 2352 0 0 Turns on the red LED when the mote powers upRadioCountToLeds 11165 4 3 Periodically samples sensors and displays the last 3 bits on its LEDsRadioSenseToLeds 11472 5 2 Periodically update a counter and displays the last 3 bits on its LEDsSense 5743 2 0 Periodically takes sensor readings and displays the values on the LEDs

of the Avrora [23] simulation and analysis framework thatprovides a time accurate simulator different analysis toolsknown asmonitors and a binary instrumentation engine [24]for programs written for AVR microcontrollers and sensornetworks All these features make Avrora just suitable forour tool Simulation is commonly used in the developmentcycle of embedded software because target systems generallylack conventional inputoutput interfaces With iRace thefunctionalities of Avrora are extendedwith a precise data racedetection tool that can easily be loaded as one of Avrorarsquosmonitors

As shown in Figure 3 iRace consists of two maincomponents the Instrumenter which exploits the Avrorainstrumentation engine [24] and the core Race Detectoritself The Instrumenter monitors task and interrupts activ-ities through its iMonitor component and provides entrypoints for the labeling routines Instrumentation points areinserted at the beginning of each task at the start andexit of each interrupt handler and also before interruptsare either enabled or disabled Similarly accesses to sharedmemory locations are monitored using the AccessMonitorcomponent that also calls routines of the detection proto-col The AccessMonitor instruments before read and writeaccesses to shared variables The Race Detector is composedof labeling routines that generate and manage labels for eachactive thread segment at runtime and detection routines formaintaining the access history and reporting data races on-the-fly as shared variables are manipulated

In TinyOS [21 25] interrupt handlers and tasks all runto completion unless preempted by other interrupt handlersThus only a single thread runs at any point of time givingus the opportunity to use only a single variable to maintainthe active thread label at detection time However whenpreemption happens the label of the preempted threadsegment must be saved and restored again when the activeinterrupt handler exits Otherwise there will be no way toknow about the concurrency information of the preemptedthread segment when it gains back the execution control Inorder tomanage the preemption we use a stack data structurewhere labels can be pushed and restored from since interruptscan be nested This provides a simple but powerful way tohandle preemption with negligible overhead

Our labeling scheme needs to uniquely identify eachinterrupt handler whenever it is invoked and each taskwhen it starts its execution Each interrupt handler is easilyidentified by the interrupt number of the interrupt it servesFor task we chose to generate a unique task identifier bysimply maintaining a counter that increases at each timea task starts its execution Thus each task is attributed aunique identifier thanks to the fact that once posted TinyOStasks run in a first-in first-out manner and cannot preempteach other By manipulating only binary files iRace doesnot require any source file nor source code annotation toeffectively detect data races However compiling the programwith debugging information may help iRace generate moredetailed debugging information on data races it reports

62 Environment We implemented and evaluated our tech-nique on a system that runs the kernel 30 of the Linuxoperating system using an Intel dual-core 320GHz and 4GBof mainmemory On this machine was installed TinyOS 210[10] along with the nesC compiler 131 the GNU C compiler46 the Java Runtime Environment 17 and the latest Avroraframework version 17115

Table 2 presents the characteristics and a brief descriptionof the benchmark applications selected for our evaluationfrom the TinyOS 210 repository [10] These applications arefreely distributed with each version of TinyOS The columnldquoLOCrdquo indicates the number of line of C codes generatedby the nesC compiler after compilation while the columnldquoIntrdquo contains the number of interrupts handled by eachapplicationThe number of shared variables for each programappears in column ldquoSVrdquo It is worthmentioning that this num-ber represents only the shared variables in the source code ofeach program The exact number of variables shared by eachapplicationmay finally depend on the number of componentsit shares with the operating system itself Applications writtenin nesC share TinyOS components by wiring themselvesto these components increasing therefore their number ofshared resources We compiled each application for theMica2 platform and measured the average runtime overheadover five executions The Mica2 motes are sensor networknodes based on the ATmega128 [30] microcontrollers Weconfigured Avrora to simulate a sensor network during 10

10 International Journal of Distributed Sensor Networks

iRace

and instrumentationframework

iMonitor

AccessMonitor

Race detector

Labeling routines

Detection protocolroutines

Binarycode Data

races

Avrora simulation

Figure 3 The Architecture of the Data Race Detection Tool

Table 3 Reported data races on the benchmark applications

Applications Data races Verif races Time (Sec) Memory (Kb)Original iRace Original iRace

6LowPanCli 0 0 269 357 1936960 2168160AnthiTheft 15 15 292 457 1944400 2320000BaseStation 0 0 279 376 1912240 2148240Blink 0 0 161 180 1887280 1949600MultihopOscilloscope 0 0 278 400 1939600 2196960Oscilloscope 13 13 179 328 1909360 2201760Powerup 0 0 151 186 1873600 1879920RadioCountToLeds 17 17 196 1255 1942880 2326320RadioSenseToLeds 15 15 181 1305 1944480 2298240Sense 0 0 158 192 1866400 2050560

seconds allowing each application to run exactly for 73728000instruction cycles We set to five the number of nodes in thesimulated sensor network

63 Evaluation For data race detectionwe instructedAvrorato simulate a sensor network with five nodes while thebenchmark applications are monitored by iRace Table 3represents the results of the evaluation of iRace on thebenchmark applications in its column ldquoData Racesrdquo whichcontains the number of shared variables involved in adata race for each application Our tool reported no datarace for all applications except for four AnthiTheft with15 shared variables Oscilloscope with 13 shared vari-ables RadioCountToLeds with 17 shared variables andRadioSenseToLeds with 15 shared variables Surprisinglymost of the reported data races reside inside some sharedsystem components of TinyOS itself The reported data racesare caused by two different interrupts namely the serialperipheral interface (SPI) interrupt and the timer interruptAlso we carefully analyzed the reported data races andinspected the source code of the applications for a manualverification process as shown in the column ldquoVerified racesrdquoof Table 3

In the AnthiTheft application for example which isused to detect and report a stolen mote by checking the levelof the light or the acceleration data races were reported over

15 shared variables Most of the reported data races are how-ever due to the fact that the SPI interrupt handler can preemptsome functions or tasks owned by the underlying radio stackThis implies that if a message is being received or sent whena SPI interrupt occurs shared variables like radioState orcount in the component CC1000SendReceiveP might becorrupted However all shared variables involved in a datarace do not directly belong to AnthiTheft but to functionsof the CC1000 radio stack in use in Mica2 motes Similardata races are also reported in RadioCountToLeds andRadioSenseToLeds since they all access the radio stack

Another data race reported both in RadioCountToLedsand RadioSenseToLeds and worthy to be commentedconcerns the shared variable reqResId inside thecomponent ArbiterP (Recently this data race hasalso been pointed out and a patch discussed in theTinyOS Forum) This data race occurs when the functionimmediateRequest() called within the function read ofthe component immediateRequest is preempted by a SPIinterrupt handler An excerpt of this function is presentedin Algorithm 4 At line 2 of Algorithm 4 the handle of aresource is requested and eventually locked out if grantedHowever if a SPI interrupt is received somewhere afterline 2 a data race will occur and the resource handlerstored in reqResId (line 6 Algorithm 4) may eventually beoverwritten by the SPI interrupt handler Also other shared

International Journal of Distributed Sensor Networks 11

(1) async command error t ResourceimmediateRequest [uint8 t id] ( ) (2) signal ResourceRequested immediateRequested [resId] ( )(3) atomic (4) if (state == RES CONTROLLED) (5) state = RES IMM GRANTING(6) reqResId = id(7)

(8) else return FAIL(9)

(10) signal ResourceDefaultOwner immediateRequested( )(11) sdot sdot sdot (12)

Algorithm 4 An excerpt of the request function from the component ArbiterP

(1) command error t Send send[uint8 t clientId] (message t lowastmsg(2) uint8 t len) (3) sdot sdot sdot (4) queue [clientId] msg = msg(5) call Packet setPayloadLength (msg len)(6) sdot sdot sdot (7)

Algorithm 5 An excerpt of the send function from the component AMSend

variables in ArbiterP like state can be corrupted as wellby the SPI interrupt handler

In the case of Oscilloscope which is a data collectionapplication that periodically samples the default sensor ofa mote and broadcasts a message over the radio every 10readings the reported data races are similar to those reportedin AnthiTheft Moreover our results suggest that if a SPIinterrupt arrives when a message is being transmitted viathe send() function of the AMSend interface the sharedbuffer sendBuf used by Oscilloscope to store incomingmessages will be updated and a corruptedmessage deliveredThe extract of code from the function Send presentedin Algorithm 5 shows that the variable msg which inthis case is a pointer to the shared variable sendBuf inOscilloscope can be updated at any time if the functionsend() is preempted because operations are not performedin a critical section This data race is somehow similar to theone we described in the example program of Algorithm 1Wealso find in Oscilloscope some of the data races reportedin the other applicationsThe results in Table 3 show that ourtool precisely detects data races on-the-fly in interrupt-drivenprograms without any false positives

64 Detection Overhead In order to evaluate the impactof our technique on the monitored programs we config-ured Avrora to simulate a single node sensor network Wethen measured the average runtime overhead and memoryspace required by each application of the benchmark ofTable 2 to execute 73728000 instruction cycles The totaloverhead of our technique mainly comes from actions of theinstrumentation engine the labeling routines and the race

Exec

utio

ns ti

me (

s)

14

12

10

8

6

4

2

0

TinyOS 210 programsOriginalWith iRace

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 4 Average runtime overhead

detection engine which also has the responsibility to createandmaintain the access historiesThe average execution timeand memory consumption collected for each application arepresented in Table 3 The columns ldquoOriginalrdquo and ldquoiRacerdquorepresent the execution time and memory consumption foreach application without and with iRace respectively

Figure 4 which plots the data from the column ldquoTimerdquoof Table 3 compares the average execution times of each

12 International Journal of Distributed Sensor Networks

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

TinyOS 210 programsOriginalWith iRace

Figure 5 Average memory consumption

application for five runs with and without iRace The captionldquoOriginalrdquo represents the native execution of the applica-tions without iRace The results of Figure 4 suggest thatthe average runtime overhead of our tool is only about1x for all applications except for RadioCountToLeds andRadioSenseToLeds that reaches 6x the original executiontime These two applications sample and display data at highfrequency rate (eg 4Hz in our case) and therefore arelikely to receive a higher number of interrupts at runtimeThis low runtime overhead is achieved by exploiting the factthat shared variables are stored by Avrora at program loadingtime Instead of monitoring all memory accesses to filterthose targeting shared variables iRace precisely instrumentsonly accesses to the well-known set of shared memory loca-tions obtained from the Avrora loader This technique allowsthe tool to drastically cuts down its monitoring overhead andto increase its scalability

Likewise wemeasured the averagememory consumptionfor each application during five runs The collected data arerepresented in Figure 5 The overall average memory con-sumption of the tool is very small at just about 20 above theoriginal memory requirement even for high frequency appli-cations like RadioCountToLeds and RadioSenseToLedsthanks to our labeling scheme that reuses the same labelfor different instances of the same interrupt handler and thedetection protocol that maintains only a reduced size accesshistoryThis result corroborates the theoretical analysis of ourtechnique as summarized in Table 1 The low runtime over-head and memory consumption of our on-the-fly techniquemake it very scalable and suitable even for interrupt-drivenprograms that handle interrupts at high frequency rates

65 Labeling Overhead Since the race detection subcompo-nent of iRace is in charge of implementing the labeling and

Table 4 Average labeling overhead

Applications Time (Sec) Memory (Kb)6LowPanCli 313 2101760AnthiTheft 308 2148880BaseStation 309 2077680Blink 174 1964640MultihopOscilloscope 313 2141920Oscilloscope 249 2092640Powerup 176 1894720RadioCountToLeds 218 2167440RadioSenseToLeds 224 2140080Sense 184 2011040

Table 5The states of an access history for detecting data races withour protocol considering the example of Figure 2

119886119894

Read Write Races11988211

⟨0 20 0⟩ ⟨20⟩

11987712

⟨0 20 0⟩ ⟨20⟩

11988213

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11988213-11988211

11988224

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩11987715

⟨1 1 0⟩ ⟨mdash⟩ ⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11987715-11988211

the detection routines it was legitimate to analyze the impactof the labeling scheme alone on the monitored programsat runtime For this purpose we deactivated the detectionengine of iRace before running it again on the benchmarkapplications We collected the average execution time andmemory space consumed by each application after five runsas shown in Table 4 The simulation was done using a singlenode network

Figure 6 compares the original execution time of eachprogram from Table 3 with the execution time of the labelingengine as presented in the column ldquoTimerdquo of Table 4 Thecomparison shows that the average runtime overhead ofiRace with just the labeling engine activated is only about04x in the worst-case This represents only a small fractionof the average runtime overhead of 6x imposed by iRace asdiscussed in Section 64Themain reason of this performancecan be found at source level When browsing the source codeof the benchmark applications we came to understand thatprogrammers generally disable and enable all the interruptsat once instead of focusing on particular interrupts In thiscase the size of each label generated by the labeling enginenever approaches its worst-case of 119874(119879) meaning that thecomparison overhead of labels is also reduced This is veryimportant because as discussed in Section 64 althoughthe size of labels does not depend on the total number ofinterrupts received at runtime this number still has impacton the overall detection overheadThus we can conclude thatthe labeling engine of iRace incurs a very low overhead atruntime making it a highly scalable tool

We also compared the memory consumption of thelabeling engine with the memory requirement of the original

International Journal of Distributed Sensor Networks 13

TinyOS 210 programs

4

35

3

25

2

15

1

05

0

Exec

utio

n tim

e (s)

With iRace labeling engineOriginal

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 6 Average runtime overhead of the labeling engine

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

With iRace labeling engineOriginal

TinyOS 210 programs

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 7 Average memory consumption by the labeling engine

program execution presented inTable 3The results plotted inFigure 7 show that when the labeling engine alone is runningiRace consumes only about 12 more memory on averageHowever during the detection phase the overall averagememory consumption of iRace is only about 20 Theseresults also tell us that the detection engine does not increasethat much the memory consumption of iRace This is alsoreasonable because we only disabled the detection engine ofiRace but did not remove the resources it used

7 Conclusions

Asynchronous interrupt handling introduces fine-grainedconcurrency into interrupt-driven programs making themprone to data races In this paper we presented an efficientand scalable on-the-fly technique to precisely detect dataraces without false positives in such programs Our techniqueintroduced a lightweight labeling scheme for maintainingconcurrency information for the main program and each ofits interrupt handlers and a precise detection protocol thatguarantees to verify the existence of data races in a programby reporting at least one data race for each shared variableinvolved in a data race

We implemented a prototype of our technique as a racedetection tool called iRace on top of the Avrora simulationframework An evaluation of iRace revealed the existence ofdata races in some programs distributed with TinyOS andin some TinyOS components while incurring a slowdownof only about 6 times on average in the worst-case with anaverage memory consumption of only about 20 above theoriginal memory requirements of the program Moreoverthis evaluation also proved that the labeling engine alonegenerates an average runtime overhead of only about 04xwith an average memory consumption of only about 12relatively to the original program execution These resultscoupled with a theoretical analysis showed that the proposedtechnique is highly scalable and practical even for largeinterrupt-driven programs

Detecting data races that occur first in a program isan important issue since it may make other data races todisappear It is therefore interesting to detect this class of dataraces in interrupt-driven programs to facilitate the debuggingprocess Also we may be interested in extending our work tosupport multithreaded programs that handle interrupts andalso to detect data races in low-level kernel codes

Acknowledgment

This research was financially supported by the Ministryof Education (MOE) and National Research Foundation ofKorea (NRF) through the Human Resource Training Projectfor Regional Innovation and BK21+ program

References

[1] U Banerjee B Bliss Z Ma and P Petersen ldquoA theory of datarace detectionrdquo in Proceedings of the Workshop on Parallel andDistributed Systems Testing and Debugging (PADTAD rsquo06) pp69ndash78 July 2006

[2] R H B Netzer and B P Miller ldquoWhat are race conditionsSome issues and formalizationsrdquo ACM letters on programminglanguages and systems vol 1 no 1 pp 74ndash88 1992

[3] N G Leveson and C S Turner ldquoInvestigation of theTherac-25accidentsrdquo Computer vol 26 no 7 pp 18ndash41 1993

[4] USA Canada Final Report on the August 14 2003 Blackout inthe United States and Canada Causes and RecommendationsUS-Canada Power System Outage Task Force 2004

[5] J Erickson M Musuvathi S Burckhardt and K OlynykldquoEffective data-race detection for the kernelrdquo in Proceedings of

14 International Journal of Distributed Sensor Networks

the 9th Conference on Operating Systems Design and Implemen-tation (USENIX rsquo10) 2010

[6] B-K Lee M-H Kang K C Park J S Yi S W Yang andY-K Jun ldquoProgram conversion for detecting data races inconcurrent interrupt handlersrdquo Communications in Computerand Information Science vol 257 pp 407ndash415 2011

[7] L Effinger-Dean B Lucia L Ceze D Grossman and H JBoehm ldquoIFRit interference-free regions for dynamic data-racedetectionrdquo in Proceedings of the ACM International Conferenceon Object Oriented Programming Systems Languages and Appli-cations (OOPSLA rsquo12) vol 47 pp 467ndash484 2012

[8] C Flanagan and S N Freund ldquoFastTrack efficient and precisedynamic race detectionrdquo in Proceedings of the ACM Interna-tional Conference on Object Oriented Programming SystemsLanguages and Applications (OOPSLA rsquo09) vol 44 pp 121ndash1332009

[9] X Xie and J Xue ldquoAcculock accurate and efficient detection ofdata racesrdquo in Proceedings of the 9th International Symposium onCode Generation and Optimization (CGO rsquo11) pp 201ndash212 April2011

[10] TinyOSCommunity ForumTinyOSAnopen source operatingsystem for low-power wireless devices 2008 httpwwwtinyosnet

[11] M Higashi T Yamamoto Y Hayase T Ishio and K InoueldquoAn effective method to control interrupt handler for datarace detectionrdquo in International Workshop on Automation ofSoftware Test (ACM rsquo10) pp 79ndash86 May 2010

[12] Z Lai S C Cheung and W K Chan ldquoInter-context control-flow and data-flow test adequacy criteria for nesC applicationsrdquoin Proceedings of the 16th ACM SIGSOFT International Sympo-sium on the Foundations of Software Engineering (SIGSOFT rsquo08)pp 94ndash104 November 2008

[13] J Regehr ldquoRandom testing of interrupt-driven softwarerdquo inPro-ceedings of the 5th ACM International Conference on EmbeddedSoftware (EMSOFT rsquo05) pp 290ndash298 September 2005

[14] Y Zhou X Chen M R Lyu and J Liu ldquoSentomist Unveil-ing transient sensor network bugs via symptom miningrdquoin Proceedings of the 30th IEEE International Conference onDistributed Computing Systems (ICDCS rsquo10) pp 784ndash794 June2010

[15] G Gracioli and S Fischmeister ldquoTracing interrupts in embed-ded softwarerdquo in Proceedings of the ACM SIGPLANSIGBEDConference on Languages Compilers and Tools for EmbeddedSystems (LCTES rsquo09) vol 44 pp 137ndash146 2009

[16] V Sundaram P Eugster and X Zhang ldquoLightweight tracingfor wireless sensor networks debuggingrdquo in Proceedings of the4th International Workshop on Middleware Tools Services andRun-Time Support for Sensor Networks (ACM rsquo09) pp 13ndash18December 2009

[17] O K Ha and Y K Jun ldquoMonitoring of programs with nestedparallelismusing efficient thread labelingrdquo International JournalFo U- and E- Service Science and Technology vol 4 pp 21ndash362011

[18] Y C Kim S S Jun and Y K Jun ldquoA tool for space-efficient on-the-fly race detectionrdquo International Journal of DatabaseTheoryand Application vol 4 pp 25ndash38 2011

[19] S Savage M Burrows G Nelson P Sobalvarro and T Ander-son ldquoEraser a dynamic data race detector for multithreadedprogramsrdquo ACM Transactions on Computer Systems vol 15 no4 pp 391ndash411 1997

[20] A Jannesari and W F Tichy ldquoOn-the-fly race detection inmulti-threaded programsrdquo in Proceedings of the 6th Workshop

on Parallel and Distributed Systems Testing Analysis andDebugging (PADTAD rsquo08) July 2007

[21] D Gay M Welsh P Levis E Brewer R Von Behren and DCuller ldquoThe nesC language a holistic approach to networkedembedded systemsrdquo inACMSIGPLANConference on Program-ming Language Design and Implementation pp 1ndash11 June 2003

[22] J Regehr and N Cooprider ldquoInterrupt verification via threadverificationrdquo Electronic Notes in Theoretical Computer Sciencevol 174 no 9 pp 139ndash150 2007

[23] B L Titzer D K Lee and J Palsberg ldquoAvrora scalable sensornetwork simulation with precise timingrdquo in Proceedings ofthe 4th International Symposium on Information Processing inSensor Networks (IPSN rsquo05) pp 477ndash482 April 2005

[24] B L Titzer and J Palsberg ldquoNonintrusive precision instru-mentation of microcontroller softwarerdquo in Proceedings of theConference on Languages Compilers and Tools for EmbeddedSystems (Lctes rsquo05) vol 40 pp 59ndash68 2005

[25] J Hil R Szewczyk A Woo S Hollar D Culler and K PisterldquoSystem architecture directions for networked sensorsrdquo inProceedings of the 9th International Conference on ArchitecturalSupport for Programming Languages and Operating Systems(ASPLOS rsquo00) pp 93ndash104 2000

[26] GMTchamgoue KHKim andYK Jun ldquoTesting anddebug-ging concurrency bugs in event-driven programsrdquo InternationalJournal of Advanced Science and Technology vol 40 pp 55ndash682012

[27] GM Tchamgoue O K Ha K H Kim and Y K Jun ldquoA frame-work for on-the-fly race healing in ARINC-653 applicationsrdquoInternational Journal of Hybrid Information Technology vol 4pp 1ndash12 2011

[28] A Dinning and E Schonberg ldquoDetecting access anomaliesin programs with critical sectionsrdquo in Proceedings of theACMONR Workshop on Parallel and Distributed Debugging(PADD rsquo91) vol 26 pp 85ndash96 1991

[29] L Lamport ldquoTime clocks and the ordering of events in adistributed systemrdquo Communications of the ACM vol 21 no7 pp 558ndash565 1978

[30] Atmel inc ATmega128 Datasheet 2011 httpwwwatmelcomImagesdoc2467pdf

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 2: Research Article Verification of Data Races in Concurrent ...downloads.hindawi.com/journals/ijdsn/2013/953593.pdf · photoTask I20 W 11 R 12 W 13 W 24 R 15 T 2 J12 T 1 F : A POEG

2 International Journal of Distributed Sensor Networks

designed for multithreaded programs Erickson et al [5]presented a technique that aims to detect data races in low-level kernel codes by randomly sampling parts of the programto be used as candidates for the race detection and using dataand code breakpoints to detect conflicting accesses to sharedvariables Although some data races caused by interruptsmaybe reported the tool still reports many false positives andfails to guarantee that programs are free from data racesMoreover existing testing methods for interrupt-driven pro-grams [11ndash14] are not sufficient for debugging since they canonly reveal the presence of data races but fail to adequatelydetect their root causes and therefore require importantmanual inspections for real data race detection Thus somedata races still remain undetectable andmake their way to theexploitation phase leading the application into unpredictableexecutions sometimes with severe consequences

Dynamic data race detection methods are generallyclassified into on-the-fly [5ndash9] and postmortem [15 16] tech-niques While postmortem techniques collect runtime datafor post-execution analysis on-the-fly techniques on theother handwill dynamically detect data races at runtime On-the-fly techniques still require less space overhead than post-mortem techniques due to the fact that unnecessary informa-tion is discarded at runtime as the detection progresses On-the-fly techniques are generally based either on the happens-before analysis [7 8 17 18] the lockset analysis [19] orthe hybrid analysis [9 20] that is a combination of the firsttwo However on-the-fly techniques still suffer from the highruntime overhead they incur Dynamic analysis methods canbe a good complement to static analysis methods [21 22]since they guarantee to isolate real data races in a particularexecution instance of a program given an input from thepotential data races reported by static analysis

This paper presents an on-the-fly technique that effi-ciently detects apparent data races in interrupt-driven pro-grams without false positives The technique combines atailored lightweight labeling scheme to maintain the logicalconcurrency between a program and every instance of itsinterrupt handlers with a precise detection protocol thatanalyzes conflicting accesses to shared memories by storingat most two accesses for each shared variable The labelingscheme generates unique identifiers to track and analyzethe happens-before relation between the main program andeach of its interrupt handlers The precise detection protocoleases the debugging process since it guarantees to verify theexistence of data races in themonitored programby reportingat least one data race for each shared variable involved ina data race Moreover the detection protocol shows a spacecomplexity of only 119874(1198811198792) while any pair of labels can becompared in time 119874(Log119879) where 119879 designates the totalnumber of interrupts supported by the underlying system and119881 the number of shared variables in the program

We implemented a prototype of our technique callediRace on top of the Avrora simulation and analysis frame-work [23] that provides a time accurate simulator and aninstrumentation engine [24] for programs written for theAVR microcontrollers and sensor networks An empiricalevaluation of iRace revealed the presence of data races insome existing TinyOS [25] components and applications

with a worst-case slowdown of only about 6 times onaverage and an increased average memory consumption ofonly about 20 in comparison with the original programexecution Moreover this evaluation also proved that thelabeling scheme alone generates an average runtime overheadof only about 04x with an average memory consumption ofonly about 12 relatively to the original program execution

The remainder of this paper is organized as followsSection 2 presents some properties of interrupt-driven pro-grams and introduces the notion of data races Section 3focuses on our lightweight labeling scheme for interrupt-driven programs Section 4 presents our precise data racedetection protocol In Section 5 a theoretical analysis ofthe proposed race detection technique is given Section 6discusses the implementation challenges of the technique andpresents our evaluation results The conclusion and futurework come in Section 7

2 Background

This section presents the interrupt-driven programs withsome important properties of interrupts and introduces thenotion of data races together with some useful definitions

21 Interrupt-Driven Programs Interrupts are generallydefined as hardware signals as they are generated by thehardware in response to an external operation or envi-ronment change A system is then interrupt driven if asignificant amount of its processing is initiated and influ-enced by external interrupts [13 26] For systems based onmore powerful processors interrupts are commonly man-aged inside the operating system kernel hiding their com-plexity from programmers However for highly resource-constrained embedded systems with a thin kernel layer likeTinyOS [25] applications have to provide and manage theirown interrupt handlers [13 26] We will focus our attentionon the latter since this kind of applications is likely to containdata races Interrupt handling is a common mechanism invarious cyber physical systems including embedded sensornetworks avionics and automotive systems or various med-ical devices However since interrupt-driven programs comewith different flavors our description will be based on thenesC language [21] used for TinyOS applications

An interrupt handler is an asynchronous callback routineattached to each interrupt to service it whenever it arrivesin the system When an interrupt is received it remainspending until it is delivered or handled Contrarily to threadsinterrupt handlers cannot block they run to completionexcept when preempted by another interrupt handler [1321 26] Interrupt handlers have an asymmetric preemptionrelation with the noninterrupt code or task in nesC taskscan be preempted by interrupt handlers but not the contraryTasks in nesC are not allowed to preempt themselves theyare ordered and run in a first-in first-out manner Interruptsare nested when they preempt each other Nested interruptsare used to allow time-sensitive operations to be handledwith low latency An interrupt is reentrant when it directlyor indirectly preempts itself

International Journal of Distributed Sensor Networks 3

(1) TOS Msg msg lowast shared variable lowast(2) uint8 t packet Index lowast shared variable lowast(3) async event result t ADCdataReady (uint16 t data) (4) struct OscopeMsg lowastpack(5) pack = (struct OscopeMsg lowast) msgdata(6) pack-gtdata [packetIndex++] = data(7) if (packetIndex == BUFFER SIZE) (8) post dataTask ( )(9)

(10)

(11) async event result t PhotoSensordataReady (uint16 t data) (12) post photoTask ( )(13)

(14) task void dataTask ( ) (15) struct OscopeMsg lowastpack(16) atomic (17) pack = (struct OscopeMsg lowast)msgdata(18) check (pack)(19) packetIndex = 0(20)

(21) pack-gtchannel = 1(22) pack-gtsourceMoteID = TOS LOCAL ADDRESS(23) send (ampmsg)(24)

(25) task void photoTask ( ) lowast do something lowast

Algorithm 1 An example nesC program

In order to minimize the impact of interrupt handlerson tasks all long-latency operations must run in split-phaseWith this mechanism interrupt handlers immediately returnafter servicing critical operations and post heavy compu-tations for later execution as new tasks The concurrencymodel of interrupt-driven programs allows at runtime anexponentially increasing number of execution paths makingthem difficult to reason about Since the only way to sharedata with interrupt handlers is through global variablesinterrupt-driven programs are prone to concurrency bugslike data races [1 2 13 26] To protect sensitive parts of theprogram interrupts must be disabled before and enabled onlyafter critical sections Interrupts can be either asynchronousor synchronous Asynchronous interrupts are signaled byexternal devices such as network interfaces and can fireat any time that the corresponding device is enabled [13]Synchronous interrupts on the other hand are those thatarrive in response to a specific action taken by the processorsuch as setting a timer

22 Data Races In shared-memory concurrent programsdata races [1 2 27] represent one of the most notorious classof concurrency bugs Data races occur when two threadsaccess a shared memory location without proper synchro-nization and at least one of the accesses is a write It is impor-tant to detect data races because they may lead programs tonondeterministic executions with unpredictable results

For illustration we consider the example program ofAlgorithm 1 that is adapted from theOscilloscope applicationdistributed with TinyOS [10] This application allows a node

of a sensor network to periodically sample cache and senddata from its sensors We assume that interrupt handlers inAlgorithm 1 are not reentrant and that they automaticallydisable themselves before they run We define a partial orderexecution graph (POEG) [28] to represent an executioninstance of an interrupt-driven program Unless explicitlymentioned wewill use the term thread to refer to either a taskor an interrupt handlerTherefore we define a thread segmentas a sequence of instructions executed by a single thread anddelimited by either a disabling or an enabling instructionThus a thread119879

119896is a finite set of 119899 sequentially ordered thread

segments that is 119879119896= 1199051 1199052 119905

119899 where 119899 represents the

number of disableenable instructions plus one and 119896 a taskidentifier or an interrupt number We therefore introduce thefollowing definitions

Definition 1 (POEG) A POEG is a directed acyclic graph of2-tuple 119866 = (119878 119864) where 119878 is a set of thread segments and119864 a set of synchronization edges A synchronization edge iseither a task posting edge for posting operations or a taskordering edge to represent the implicit order relation thatexists between tasks

Definition 2 (happens-before [29]) Given a program execu-tion if a thread segment 119905

119894runs earlier than another thread

segment 119905119895 then 119905

119894happens-before 119905

119895 denoted 119905

119894

ℎ119887

997888997888rarr 119905119895 If

the happens-before relation (ie 119905119894

ℎ119887

997888997888rarr 119905119895or 119905119895

ℎ119887

997888997888rarr 119905119894) is

not satisfied between 119905119894and 119905119895 the two thread segments are

concurrent and denoted by 119905119894 119905119895

4 International Journal of Distributed Sensor Networks

ADC20

20

20

20

12

12

Task posting edgeTask ordering edge

lowast

lowast

PhotoSensor

dataTask

photoTask

I20W11

R12

W13

W24

R15

T2

J12

T1

Figure 1 A POEG of an execution instance of the program inAlgorithm 1 Only accesses to msg are represented

In a POEG two accesses are concurrent if and onlyif there is no path between them Otherwise they areordered by the happens-before relation An example POEGfor the program in Algorithm 1 considering only accesses tothe shared variable msg is represented in Figure 1 In thisPOEG vertical lines symbolize threads while read and writeaccesses to shared variables are respectively represented bythe letters 119877

119894119895and119882

119894119895 where 119894 is the thread instance number

and 119895 an observed execution order We use the couple ofsymbols⊓119899 and⊔119899 to respectively represent the disabling andenabling of interrupts where 119899 is the corresponding interruptnumber that will be replaced with the star symbol (lowast) ifall interrupts are concerned The POEG of Figure 1 depictsan execution where a first instance of the ADC (Anolog-to-Digital Converter) interrupt handler collected new dataand posted an instance of dataTask Furthermore anotherinstance of the ADC interrupt arrives and preempts theexecution of dataTask

Although the first instance of the ADC interrupt handlerposted dataTask if the second ADC interrupt is receivedright before the message is sent at line 23 of Algorithm 1 dataraces will still occur between the write access in ADC andthe accesses 119882

13and 119877

15in dataTask making a corrupted

message to be delivered Those are the data races representedin Figure 1 by the pair of accesses 119882

13-11988224

and 11987715-11988224 If

a task is posted by an interrupt handler the task postingedge guarantees a happens-before relation only between theposted task and the actual instance of the interrupt handlerthat posted it but not with future instances of the sameinterrupt handler Consequently we ignore the task postingedge when traversing the graph for analyzing the happens-before relation because we use a single arrow to represent

all instances of the same interrupt Thus we can also reportthe data races 119882

13-11988211

and 11987715-11988211

between 1198791and 119868

20

Therefore in our POEG we assume that two thread segments119905119894and 119905119895 respectively from two different threads119879

119894and119879119895are

always concurrent unless they are ordered by a task orderingedge or one of them is disabled by the other one The nesClanguage [21] contains a static data race detector that howeveris still unsound since it cannot follow pointers [12 21 22]Thus the data races presented in Figure 1 cannot be reportedby the native nesC data race detector

3 Lightweight Labeling Scheme

For an effective dynamic race detection it is essentialto adequately determine the order of precedence betweenthread segments accessing shared memory locations Forthis purpose this section presents our lightweight labelingscheme used for capturing the logical concurrency betweenthread segments of interrupt-driven programs

31 GeneratingThread Labels Happens-before based on-the-fly race detection techniques generally rely on a variationof Lamportrsquos logical timestamp [29] in order to preciselyidentify concurrent thread segments of a program Theeffectiveness of on-the-fly techniques is highly dependenton the cost and accuracy of this identification process Ourlabeling scheme generates a unique identifier called label foreach thread segment 119905

119894during an execution instance of an

interrupt-driven program For the labeling scheme we needto introduce the following definitions

Definition 3 (critical section) A critical section with respectto a given interrupt 119896 is a portion of program comprisingbetween a disable and probably an enable instruction tointerrupt 119896

Definition 4 (interrupt set) The interrupt set or 119894119904119890119905 of athread segment 119905

119894represents the set of all interrupts 119896disabled

by 119905119894Thus this guarantees a happens-before relation between

119905119894and every interrupt in its 119894119904119890119905 Formally for all 119896 isin 119905

119894sdot 119894119904119890119905

for all 119905119895isin 119879119896 119905119894

ℎ119887

997888997888rarr 119905119895or 119905119895

ℎ119887

997888997888rarr 119905119894

Definition 5 (critical section violation) A thread segment 119905119894

of a thread 119879119896violates the critical section of another thread

segment 119905119895 denoted by 119905

119894

V999492999484 119905119895 when 119905

119895can be preempted by

119905119894

The label of each thread segment 119905119894consists of two compo-

nents an identifier referred to as segment descriptor 119878119863119894 and

an interrupt set 119894119904119890119905119894 Put together we use 119878119863

119894⟨119894119904119890119905119894⟩ to desig-

nate the label of a thread segment 119905119894 The segment descriptor

119878119863119894of a thread segment 119905

119894is defined as a triplet ⟨119905119905

119894 119905119894119889119894 119888119904119894⟩

where 119905119905119894specifies whether 119905119894119889

119894is a task or an interrupt

handler 119905119894119889119894contains the task identifier of a task or the

interrupt number of an interrupt and 119888119904119894indicates whether

the thread segment has disabled all interrupts in the systemCreating labels for thread segments of an interrupt-driven

program is straightforward with our labeling scheme as

International Journal of Distributed Sensor Networks 5

suggested by the algorithms in Algorithm 2 A new labeledthread 119879

119894is created when a new task starts its execution or

whenever an interrupt handler is invoked When a new taskstarts a new label is generated for the first thread segment ofthe task by simply initializing the component of the label asin Algorithm 2(a) A new label is similarly created when aninterrupt handler is invoked in Algorithm 2(b) Each threadthen creates additional thread segments whenever it disablesor enables interrupts

When a thread segment 119905119894of a thread 119879

119894disables an

interrupt 119896 the label of the new thread segment 119905119895is generated

by adding the interrupt number 119896 into the interrupt set ofthe current label owned by 119905

119894 However if all interrupts in

the system are disabled by 119905119894 the new label for 119905

119895is obtained

by setting 119888119904119894and by emptying 119894119904119890119905

119894from the current label

owned by 119905119894as shown in Algorithm 2(c) Thus as seen in

Algorithm 2(d) if 119905119894enables an interrupt 119896 a new label for a

new thread segment 119905119895is obtained by simply removing 119896 from

the interrupt set of the current label in 119905119894 If all interrupts are

enabled at the same time the flag 119888119904119894and the interrupt set

119894119904119890119905119894are both reset from the current label of 119905

119894to generate

the new label for the new thread segment 119905119895 This helps

maintain a reduced-size interrupt set for each thread segmentsince programmers generally disable and enable all interruptsrather than manipulating individual interrupts

Moreover the size of an interrupt set is bounded by thetotal number of interrupts supported by the hardware onwhich the program runs which is generally a reasonableinteger number By reusing the same label for differentinstances of the same interrupt handler our labeling schemedoes not depend on the total number of interrupts receivedat runtime which can be very high

Lemma 6 Given two labeled thread segments 119905119894and 119905119895 with

119905119894belonging to an interrupt handler 119879

119896 119905119894

V999492999484 119905119895rArr 119896 notin 119894119904119890119905

119895or

119888119904119895= 119891119886119897119904119890

Proof Let us consider two labeled thread segments 119905119894and 119905119895

and let us assume that 119905119894belongs to an interrupt handler 119879

119896

We also assume that 119905119894violates the critical section of 119905

119895 By

definition thread segments are delimited by either the disableor enable instructions Therefore if 119905

119895was created after a

the disable instruction targeting interrupt 119896 the interruptnumber 119896 will be in the interrupt set 119894119904119890119905

119895of 119905119895according

to the labeling algorithms of Algorithm 2 However if 119905119895was

created after the disabling of all interrupts 119888119904119895would then be

true by the labeling algorithms of Algorithm 2 to signify thatall interrupts including 119896 are disabledThus for all 119905

119894isin 119879119896 119905119894

cannot violate 119905119895 which is contradictory since we assumed

119905119894

V999492999484 119905119895 Therefore 119905

119895has necessarily enabled 119896 that is

for all 119905119894isin 119879119896 119905119894

V999492999484 119905119895rArr 119896 notin 119894119904119890119905

119895or 119888119904119895= 119891119886119897119904119890

32 Concurrency Relation Using our labeling scheme itbecomes easy to analyze the happens-before relation betweenany pair of thread segments 119905

119894and 119905119895 by comparing their

labels

[a Task Starting]a1 tsk start()a2 Label Lba3 Lbt = truea4 Lbtid = getTaskid()a5 Lbcs = falsea6 Lbiset =

a7 return Lb[b Interrupt Invocation]b1 int invoke()b2 Label Lbb3 Lbt = falseb4 Lbtid = getIntNumber()b5 Lbcs = falseb6 Lbiset =

b7 return Lb[c Interrupt Disabling]c1 int disable(Label Lb Interrupt n)c2 if(n gt 0) Lbiset cup 119899

c3 elsec4 Lbcs = truec5 Lbiset =

c6 endifc7 return Lb[d Interrupt Enabling]d1 int enable(Label Lb Interrupt n)d2 if(n gt 0) Lbiset-nd3 elsed4 Lbcs = falsed5 Lbiset =

d6 endifd7 return Lb

Algorithm 2 Labeling algorithms

Theorem7 A labeled thread segment 119905119894happens before anoth-

er labeled thread segment 119905119895 denoted by 119905

119894

ℎ119887

997888997888rarr 119905119895 if and only if

either that 119905119894and 119905119895belong to tasks or that 119905

119895does not violate

119905119894 that is not(119905

119895

V999492999484 119905119894)

Proof Let us assume two labeled thread segments 119905119894and

119905119895 such that 119905

119894

ℎ119887

997888997888rarr 119905119895 Thus according to the labeling

algorithms of Algorithm 2 we should have 119905119905119894= 119905119905119895= 119905119903119906119890

meaning that 119905119894and 119905

119895are either from the same or from

different tasks By definition tasks are all ordered and cannotpreempt each other However if 119905

119895is a thread segment of

an interrupt handler 119879119896 according to Definition 4 either

119888119904119894= 119905119903119906119890 because 119905

119894disabled all interrupts or the thread

identifier 119905119894119889119895of 119905119895that corresponds to 119896 the interrupt

number of 119879119896 is in 119894119904119890119905

119894 the interrupt set of 119905

119894 This means

that 119905119894is a critical section thread segment that disables either

all interrupts or particularly the interrupt number 119896 Thusaccording to Lemma 6 119905

119895does not violate the critical section

of 119905119894 Therefore we can say that 119905

119894

ℎ119887

997888997888rarr 119905119895implies either that 119905

119894

and 119905119895belong to tasks or that not(119905

119895

V999492999484 119905119894)

6 International Journal of Distributed Sensor Networks

Now let us assume that 119905119894and 119905

119895belong to tasks

Naturally 119905119894and 119905119895are ordered and the labeling algorithms of

Algorithm 2 will set 119905119905119894= 119905119905119895= 119905119903119906119890 meaning that 119905

119894

ℎ119887

997888997888rarr 119905119895

On the other hand if we assume that not(119905119895

V999492999484 119905119894) then 119905

119895

must be part of a given interrupt handler 119879119896and either 119905

119894has

disabled all interrupts to set 119888119904119894= 119905119903119906119890 or the thread identifier

119905119894119889119895= 119896 of 119905

119895is in the interrupt set 119894119904119890119905

119894of 119905119894 according to

Definition 4 and Lemma 6 Therefore we can conclude that119905119894

ℎ119887

997888997888rarr 119905119895

Any two labeled thread segments 119905119894and 119905119895are therefore

concurrent if they do not satisfy the happens-before relationas stated by Theorem 7 which can formally be rewritten asfollows

119905119894

ℎ119887

997888997888rarr 119905119895lArrrArr

119905119905119894= 119905119905119895= 119905119903119906119890

or(119905119905119895= 119891119886119897119904119890) and (119888119904

119894= 119905119903119906119890 or 119905119894119889

119895isin 119894119904119890119905119894)

(1)

By comparing based on Theorem 7 the labels of thethread segments in the POEG represented by Figure 2 wecan easily obtain that the thread segments of 119868

20 1198941 and 119894

2

are concurrent with other thread segments 1199051 1199053 1199054 and

1198951 Therefore the write accesses in 119894

1and 1198942conflict with

the other accesses in 1199053 leading to four data races 119882

11-11988213

11988211-1198771511988224-11988213 and119882

24-11987715 However since the accesses

11988211and119882

23 respectively from 119894

1and 1198942 represent the same

write access the set of data races to be reported for an effectivedebugging process should be reduced to only 119882

11-11988213

and11988211-11987715 By disabling all interrupts 119905

2enforces the happens-

before relation with other thread segments Although 1198791is

posted by 11986820 they are still concurrent Also the thread

segment 1198951is concurrent with 119894

1 1198942 1199051 1199053 and 119905

4

4 Precise Detection Protocol

The race detection protocol monitors memory accesses bymaintaining a special data structure called access history intowhich are stored the labels of prior accesses to each sharedvariable Our detection protocol stores in each entry of theaccess history of a shared variable a unique label representingthe thread segment that accessed the variable as describedin Section 3 At each access to a shared variable data racesare reported by analyzing based onTheorem 7 the happens-before relation between the current thread segment and thelabels of prior thread segments stored in the access historyTherefore the access history for each shared variable hasto be efficiently organized in order to reduce the overheadgenerated by the detection process

Our precise detection protocol maintains an access his-tory consisting of two sets for each shared variable a Readset and a Write set to respectively store labels of threadsegments that read and write the variable We use 119860119867(119909)

to designate the access history of a shared variable 119909 Thus119860119867(119909) sdot 119877119890119886119889 and 119860119867(119909) sdot 119882119903119894119905119890 refer to the read set andthe write set of 119860119867(119909) respectively The algorithms used bythe detection protocol are presented in Algorithm 3 In thesealgorithms we assume that the variable 119871119887 always contains

20

20

20

20

t1

t2

t3

t4

12

12

lowast

lowast

i1⟨0 20 0⟩⟨20⟩

⟨mdash⟩⟨1 1 1⟩

⟨mdash⟩⟨1 1 0⟩

⟨mdash⟩⟨1 1 0⟩

⟨0 12 0⟩⟨12⟩

⟨0 20 0⟩⟨20⟩

⟨mdash⟩⟨1 2 0⟩

I20

W11

R12

W13

W24

R15

T2

J12

T1

j1

i2

Figure 2 An example labeled POEG for an execution instance ofthe program in Algorithm 1

the label of the current thread segment accessing a sharedvariable and that 119901119871119887 represents any prior label stored in theaccess history

When a read access occurs to a shared variable 119909the detection protocol calls the ah read() function ofAlgorithm 3(a) that then reports data races by analyzing thehappens-before relation between the label of the accessingthread segment and the labels of prior accesses stored in119860119867(119909) sdot119882119903119894119905119890 Similarly upon a write access to a shared vari-able the monitoring function ah write() of Algorithm 3(b)reports data races by looking into both 119860119867(119909) sdot 119877119890119886119889 and119860119867(119909) sdot 119882119903119894119905119890 We assume the existence of a functionOrdered() that based on Theorem 7 decides whether athread segment 119905

119894happens before another thread segment 119905

119895

by comparing their labelsIn order to maintain a reduced size access history

accesses from critical sections that concern all interruptswithin a task are not inserted in the access history sincethey will never run into a data race with other accessesfrom interrupt handlers Also only the first read and writeaccesses to a shared variable from the same thread segmentare kept into the access history Subsequent similar accessesto the same shared variable from the same thread segmentare filtered outThis filtering operations are performed by theisFiltered() function of Algorithm 3(c) that is called by bothah read() and ah write() before any other action

After data races have been reported with the currentaccess the function maintain ah() of Algorithm 3(d) is usedfor keeping the size of the access history balanced Thisfunction replaces some previous accesses in the access historywith the current access Thus accesses from different tasksare considered from the same thread since tasks by defini-tion are all ordered by happens-before Similarly accesses

International Journal of Distributed Sensor Networks 7

(1) ah read(Label Lb Access History AH Variable x)(2) if (isFiltered(Lb AH(x)Read)) then return(3) for all labels pLb in AH(x)Write do(4) if (Not Ordered(Lb pLb)) then(5) report race(x pLb Lb)(6) end for(7) maintain ah(Lb AH(x)Read)

(a) Monitoring Protocol for Read Accesses(1) ah write(Label Lb Access History AHVariable x)(2) if (isFiltered(Lb AH(x)Write)) then return(3) for all labels pLb in(4) AH(x)Write cup AH(x)Read do(5) if (Not Ordered(Lb pLb)) then(6) report race(x pLb Lb)(7) end for(8) maintain ah(Lb AH(x)Write)

(b) Monitoring Protocol for Write Accesses(1) isFiltered(Label Lb Access History AH)(2) if (Lbcs = true) and (Lbtt = true)(3) then return true(4) if (any access with a label pLb exists in AH(5) such that pLb = Lb) then return true(6) else(7) return false

(c) Filtering Redundant Accesses(1) maintain ah(Label Lb Access History AH)(2) if (Lbtt = true) then(3) delete all entries in AH with label pLb(4) such that pLbtt = true(5) else(6) delete all entries in AH with label pLb such(7) that pLbtt = Lbtt and pLbtid = Lbtid(8) end if(9) add ah(Lb AH)

(d) Maintaining Access History

Algorithm 3 Labeling algorithms

from different instances of the same interrupt handler areconsidered from a single instance of the interrupt handlerThis property is already enforced by the labeling scheme thatreuses the same label for all instances of the same interrupthandler Again we assume the existence of another functionadd ah() which is able to insert the label of the current threadsegment into the appropriate set of the access history given ashared variable

Table 5 shows the different states of an access historyfor detecting data races with our protocol in the exampleprogram of Figure 2 where accesses to the shared variablemsg appear in the first column When the first access 119882

11

occurs the label ⟨0 20 0⟩⟨20⟩ of the thread segment 1198941

is simply added into the access history and no data racereported as there is no other access prior to 119882

11 The next

access 11987712 from the thread segment 119905

2is filtered out because

1199052that belongs to task 119879

1has disabled all interrupts The

following access 11988213

occurs on 1199053with interrupts enabled

Thus a data race11988213-11988211is reported by comparing the label

of 1199053 ⟨1 1 0⟩⟨minus⟩ with the label of 119894

1previously stored into the

access history Afterwards the label of 1199053is inserted into the

write set of the access history The thread segment 1199053is then

preempted by 1198942 but the access 119882

24is filtered out because

1198942which is a new instance of the interrupt 119868

20 also shares the

same label ⟨0 20 0⟩⟨20⟩ with 1198941which previously performed

a write and whose label is already in the access history Afterthe execution of 119894

2 1199053resumes to perform the read access

11987715 Another data race 119877

15-11988211

is reported and the label of1199053 ⟨1 1 0⟩⟨minus⟩ stored into the read set of the access history

Although the program received two instances of the ADCinterrupt our protocol precisely reported only the set of twodata races needed for effectively debug the program Thisshows that our detection protocol does not depend on thenumber of interrupts received at runtime which can be veryhigh

Although it keeps only a reduced set of labels in the accesshistory our precise detection protocol even with a minimalnumber of interrupts still guarantees to verify the existenceof data races in a program For this purpose the protocol isdesigned to report without false positives at least one datarace for each shared variable involved in a data race

8 International Journal of Distributed Sensor Networks

Theorem 8 Given an interrupt-driven program 119875 the detec-tion protocol reports a data race over a shared variable 119909 if andonly if there exists a data race over 119909 in an execution instanceof 119875

Proof We will first prove that if a data race is reported on ashared variable 119909 by the detection protocol then there alsoexists a data race in an execution instance of the program forthe same shared variable 119909 For this we assume a snapshotof the access history 119860119867(119909) of the shared variable 119909 at agiven time and the current access event 119886

119894to 119909 from a thread

segment 119905119894that owns a label we refer to as 119871119887

119894in an execution

instance of the program 119875 We can then distinguish twodifferent cases

Case A 119860119867(119909) is empty and there is obviously no data racefor 119909 In this case the current access 119886

119894is the only access to

119909 from the program 119875 and 119871119887119894will be added by the function

add ah() into119860119867(119909)sdot119877119890119886119889 if 119886119894is a read access or into119860119867(119909)sdot

119882119903119894119905119890 if it is a write access

Case B 119860119867(119909) is not empty Data races are reported usingthe ah read() function if 119886

119894is a read access or the ah write()

in case of a write access In both cases the happens-beforerelation is analyzed between the label 119871119887

119894of 119905119894and previous

labels stored in 119860119867(119909) according to Theorem 7 The outputsof these analyses can be classified into two categories

(1) There is no data race between 119886119894and119860119867(119909) therefore

all previous accesses stored in 119860119867(119909) happenedbefore 119886

119894meaning that no thread segment prior to

119905119894in 119875 is concurrent with 119905

119894 they are all ordered by

happens-before Thus for all 119886119895isin 119905119895and 119871119887119895isin 119860119867(119909)

119905119895

ℎ119887

997888997888rarr 119905119894 In which case 119875 also has no data race

(2) There are data races between 119886119894and prior accesses

in 119860119867(119909) this means that there are some labels in119860119867(119909) belonging to some prior thread segments thatare not ordered with 119905

119894 Concretely there are thread

segments in 119875 that are concurrent with 119905119894 Formally

exist119905119895isin 119875119871119887

119895isin 119860119867(119909) 119905

119894 119905119895Therefore the execution

instance of 119875 also contains data races over 119909

Now we show that if there exists a data race over a sharedvariable 119909 in an execution instance of the program 119875 thenour detection protocol also reports a data race over 119909 Usingthe contrapositive we will prove that if the detection protocoldoes not report any data race the program 119875 also does notcontain any for a shared variable 119909 We again consider asnapshot of the access history 119860119867(119909) for a shared variable119909 at a given time We also assume that the detection protocolreported no data races for the shared variable 119909 This meansthat 119860119867(119909) is either empty and there is clearly no data racesince there is no access to 119909 from 119875 yet or contains onlylabels of thread segments that are all ordered by the happens-before relation in which case there is no concurrent threadsegment in119875 which also implies that119875 has no data races over119909 Therefore we can formally write for all 119905

119894 119905119895isin 119875 and 119871119887

119894

119871119887119895isin 119860119867(119909) 119905

119894

ℎ119887

997888997888rarr 119905119895or 119905119895

ℎ119887

997888997888rarr 119905119894 Thus 119875 also contains no

data race for 119909

Table 1 Efficiency of the precise detection protocol

Space TimeNew label comparison Per access

O(1198811198792) O(1) O(LogT) O(TLogT)

5 Theoretical Analysis

On-the-fly data race detection techniques are known to havehigh runtime overhead that is generally provoked by the timespentmonitoring accesses to shared variables and to comparethread labels In this section we theoretically analyze theefficiency of our technique which depends on119881 the numberof shared variables and 119879 the total number of interruptsin the underlying system plus one the number of tasksIn general 119879 is very small since the number of interruptshandled by a system is also kept small For example thepopular Atmel ATmega128 [30] processor supports only amaximum of 34 interrupts Also because of the implicithappens-before relation that exists between all tasks they areviewed as a single task by the detection protocol

Table 1 presents the space and time complexities of ourlabeling and detection techniques The access history usedby our technique stores from each thread segment at mosttwo accesses for each shared variable Each label in the accesshistory has a worst-case size of 119874(119879) that represents thesize of its interrupt set Therefore the total labeling spacerequirement for each shared variable is 119874(1198792) Thus thetotal space complexity of our technique is 119874(1198811198792) Howeverbecause the number of interrupts is generally a reasonableinteger number the space complexity of our technique canfurther be reduced to 119874(119881)

A new label is always created in a constant amount of timeby our labeling algorithms Interrupts are generally givennumbers in a sequence order Therefore a binary searchtechnique is applied to locate values in interrupt sets whichcontain only interrupt numbers Thus the time complexityfor comparing any two labels is 119874(Log119879) in the worst-caseand 119874(1) in most cases where the interrupt sets are emptyAt each access to a shared variable the total time complexityto report data races with the previous accesses stored in theaccess history is then119874(119879Log119879)The labels and the detectionprotocol do not depend on the total number of interruptsreceived at runtimemaking our technique highly scalable andpractical

6 Experimental Results

We empirically evaluated the preciseness and efficiency ofour technique using a set of well-known programs providedwith TinyOS [10] This section details the implementationchallenges and presents the analysis results of the experimen-tation

61 Implementation We implemented a prototype of ourdata race detection tool for interrupt-driven programs callediRace based on the concurrency model of TinyOS [21] andits nesC language [25] We implemented iRace on top

International Journal of Distributed Sensor Networks 9

Table 2 Characteristics of the benchmark applications from TinyOS 210 [10]

Applications LOC Int SV Brief description6LowPanCli 19116 4 9 Implements a command line interface using the 6lowpan stackAnthiTheft 27421 5 4 Detects and reports a stolen mote by checking some sensorsBaseStation 14905 6 13 Sends and receives simple active messages on the radio and serial linksBlink 3688 1 0 Periodically blinks the three LEDs of a moteMultihopOscilloscope 24310 9 8 Periodically samples and broadcasts sensor data to a base stationOscilloscope 11686 5 5 Periodically samples sensor data and broadcasts a message every 10 timesPowerup 2352 0 0 Turns on the red LED when the mote powers upRadioCountToLeds 11165 4 3 Periodically samples sensors and displays the last 3 bits on its LEDsRadioSenseToLeds 11472 5 2 Periodically update a counter and displays the last 3 bits on its LEDsSense 5743 2 0 Periodically takes sensor readings and displays the values on the LEDs

of the Avrora [23] simulation and analysis framework thatprovides a time accurate simulator different analysis toolsknown asmonitors and a binary instrumentation engine [24]for programs written for AVR microcontrollers and sensornetworks All these features make Avrora just suitable forour tool Simulation is commonly used in the developmentcycle of embedded software because target systems generallylack conventional inputoutput interfaces With iRace thefunctionalities of Avrora are extendedwith a precise data racedetection tool that can easily be loaded as one of Avrorarsquosmonitors

As shown in Figure 3 iRace consists of two maincomponents the Instrumenter which exploits the Avrorainstrumentation engine [24] and the core Race Detectoritself The Instrumenter monitors task and interrupts activ-ities through its iMonitor component and provides entrypoints for the labeling routines Instrumentation points areinserted at the beginning of each task at the start andexit of each interrupt handler and also before interruptsare either enabled or disabled Similarly accesses to sharedmemory locations are monitored using the AccessMonitorcomponent that also calls routines of the detection proto-col The AccessMonitor instruments before read and writeaccesses to shared variables The Race Detector is composedof labeling routines that generate and manage labels for eachactive thread segment at runtime and detection routines formaintaining the access history and reporting data races on-the-fly as shared variables are manipulated

In TinyOS [21 25] interrupt handlers and tasks all runto completion unless preempted by other interrupt handlersThus only a single thread runs at any point of time givingus the opportunity to use only a single variable to maintainthe active thread label at detection time However whenpreemption happens the label of the preempted threadsegment must be saved and restored again when the activeinterrupt handler exits Otherwise there will be no way toknow about the concurrency information of the preemptedthread segment when it gains back the execution control Inorder tomanage the preemption we use a stack data structurewhere labels can be pushed and restored from since interruptscan be nested This provides a simple but powerful way tohandle preemption with negligible overhead

Our labeling scheme needs to uniquely identify eachinterrupt handler whenever it is invoked and each taskwhen it starts its execution Each interrupt handler is easilyidentified by the interrupt number of the interrupt it servesFor task we chose to generate a unique task identifier bysimply maintaining a counter that increases at each timea task starts its execution Thus each task is attributed aunique identifier thanks to the fact that once posted TinyOStasks run in a first-in first-out manner and cannot preempteach other By manipulating only binary files iRace doesnot require any source file nor source code annotation toeffectively detect data races However compiling the programwith debugging information may help iRace generate moredetailed debugging information on data races it reports

62 Environment We implemented and evaluated our tech-nique on a system that runs the kernel 30 of the Linuxoperating system using an Intel dual-core 320GHz and 4GBof mainmemory On this machine was installed TinyOS 210[10] along with the nesC compiler 131 the GNU C compiler46 the Java Runtime Environment 17 and the latest Avroraframework version 17115

Table 2 presents the characteristics and a brief descriptionof the benchmark applications selected for our evaluationfrom the TinyOS 210 repository [10] These applications arefreely distributed with each version of TinyOS The columnldquoLOCrdquo indicates the number of line of C codes generatedby the nesC compiler after compilation while the columnldquoIntrdquo contains the number of interrupts handled by eachapplicationThe number of shared variables for each programappears in column ldquoSVrdquo It is worthmentioning that this num-ber represents only the shared variables in the source code ofeach program The exact number of variables shared by eachapplicationmay finally depend on the number of componentsit shares with the operating system itself Applications writtenin nesC share TinyOS components by wiring themselvesto these components increasing therefore their number ofshared resources We compiled each application for theMica2 platform and measured the average runtime overheadover five executions The Mica2 motes are sensor networknodes based on the ATmega128 [30] microcontrollers Weconfigured Avrora to simulate a sensor network during 10

10 International Journal of Distributed Sensor Networks

iRace

and instrumentationframework

iMonitor

AccessMonitor

Race detector

Labeling routines

Detection protocolroutines

Binarycode Data

races

Avrora simulation

Figure 3 The Architecture of the Data Race Detection Tool

Table 3 Reported data races on the benchmark applications

Applications Data races Verif races Time (Sec) Memory (Kb)Original iRace Original iRace

6LowPanCli 0 0 269 357 1936960 2168160AnthiTheft 15 15 292 457 1944400 2320000BaseStation 0 0 279 376 1912240 2148240Blink 0 0 161 180 1887280 1949600MultihopOscilloscope 0 0 278 400 1939600 2196960Oscilloscope 13 13 179 328 1909360 2201760Powerup 0 0 151 186 1873600 1879920RadioCountToLeds 17 17 196 1255 1942880 2326320RadioSenseToLeds 15 15 181 1305 1944480 2298240Sense 0 0 158 192 1866400 2050560

seconds allowing each application to run exactly for 73728000instruction cycles We set to five the number of nodes in thesimulated sensor network

63 Evaluation For data race detectionwe instructedAvrorato simulate a sensor network with five nodes while thebenchmark applications are monitored by iRace Table 3represents the results of the evaluation of iRace on thebenchmark applications in its column ldquoData Racesrdquo whichcontains the number of shared variables involved in adata race for each application Our tool reported no datarace for all applications except for four AnthiTheft with15 shared variables Oscilloscope with 13 shared vari-ables RadioCountToLeds with 17 shared variables andRadioSenseToLeds with 15 shared variables Surprisinglymost of the reported data races reside inside some sharedsystem components of TinyOS itself The reported data racesare caused by two different interrupts namely the serialperipheral interface (SPI) interrupt and the timer interruptAlso we carefully analyzed the reported data races andinspected the source code of the applications for a manualverification process as shown in the column ldquoVerified racesrdquoof Table 3

In the AnthiTheft application for example which isused to detect and report a stolen mote by checking the levelof the light or the acceleration data races were reported over

15 shared variables Most of the reported data races are how-ever due to the fact that the SPI interrupt handler can preemptsome functions or tasks owned by the underlying radio stackThis implies that if a message is being received or sent whena SPI interrupt occurs shared variables like radioState orcount in the component CC1000SendReceiveP might becorrupted However all shared variables involved in a datarace do not directly belong to AnthiTheft but to functionsof the CC1000 radio stack in use in Mica2 motes Similardata races are also reported in RadioCountToLeds andRadioSenseToLeds since they all access the radio stack

Another data race reported both in RadioCountToLedsand RadioSenseToLeds and worthy to be commentedconcerns the shared variable reqResId inside thecomponent ArbiterP (Recently this data race hasalso been pointed out and a patch discussed in theTinyOS Forum) This data race occurs when the functionimmediateRequest() called within the function read ofthe component immediateRequest is preempted by a SPIinterrupt handler An excerpt of this function is presentedin Algorithm 4 At line 2 of Algorithm 4 the handle of aresource is requested and eventually locked out if grantedHowever if a SPI interrupt is received somewhere afterline 2 a data race will occur and the resource handlerstored in reqResId (line 6 Algorithm 4) may eventually beoverwritten by the SPI interrupt handler Also other shared

International Journal of Distributed Sensor Networks 11

(1) async command error t ResourceimmediateRequest [uint8 t id] ( ) (2) signal ResourceRequested immediateRequested [resId] ( )(3) atomic (4) if (state == RES CONTROLLED) (5) state = RES IMM GRANTING(6) reqResId = id(7)

(8) else return FAIL(9)

(10) signal ResourceDefaultOwner immediateRequested( )(11) sdot sdot sdot (12)

Algorithm 4 An excerpt of the request function from the component ArbiterP

(1) command error t Send send[uint8 t clientId] (message t lowastmsg(2) uint8 t len) (3) sdot sdot sdot (4) queue [clientId] msg = msg(5) call Packet setPayloadLength (msg len)(6) sdot sdot sdot (7)

Algorithm 5 An excerpt of the send function from the component AMSend

variables in ArbiterP like state can be corrupted as wellby the SPI interrupt handler

In the case of Oscilloscope which is a data collectionapplication that periodically samples the default sensor ofa mote and broadcasts a message over the radio every 10readings the reported data races are similar to those reportedin AnthiTheft Moreover our results suggest that if a SPIinterrupt arrives when a message is being transmitted viathe send() function of the AMSend interface the sharedbuffer sendBuf used by Oscilloscope to store incomingmessages will be updated and a corruptedmessage deliveredThe extract of code from the function Send presentedin Algorithm 5 shows that the variable msg which inthis case is a pointer to the shared variable sendBuf inOscilloscope can be updated at any time if the functionsend() is preempted because operations are not performedin a critical section This data race is somehow similar to theone we described in the example program of Algorithm 1Wealso find in Oscilloscope some of the data races reportedin the other applicationsThe results in Table 3 show that ourtool precisely detects data races on-the-fly in interrupt-drivenprograms without any false positives

64 Detection Overhead In order to evaluate the impactof our technique on the monitored programs we config-ured Avrora to simulate a single node sensor network Wethen measured the average runtime overhead and memoryspace required by each application of the benchmark ofTable 2 to execute 73728000 instruction cycles The totaloverhead of our technique mainly comes from actions of theinstrumentation engine the labeling routines and the race

Exec

utio

ns ti

me (

s)

14

12

10

8

6

4

2

0

TinyOS 210 programsOriginalWith iRace

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 4 Average runtime overhead

detection engine which also has the responsibility to createandmaintain the access historiesThe average execution timeand memory consumption collected for each application arepresented in Table 3 The columns ldquoOriginalrdquo and ldquoiRacerdquorepresent the execution time and memory consumption foreach application without and with iRace respectively

Figure 4 which plots the data from the column ldquoTimerdquoof Table 3 compares the average execution times of each

12 International Journal of Distributed Sensor Networks

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

TinyOS 210 programsOriginalWith iRace

Figure 5 Average memory consumption

application for five runs with and without iRace The captionldquoOriginalrdquo represents the native execution of the applica-tions without iRace The results of Figure 4 suggest thatthe average runtime overhead of our tool is only about1x for all applications except for RadioCountToLeds andRadioSenseToLeds that reaches 6x the original executiontime These two applications sample and display data at highfrequency rate (eg 4Hz in our case) and therefore arelikely to receive a higher number of interrupts at runtimeThis low runtime overhead is achieved by exploiting the factthat shared variables are stored by Avrora at program loadingtime Instead of monitoring all memory accesses to filterthose targeting shared variables iRace precisely instrumentsonly accesses to the well-known set of shared memory loca-tions obtained from the Avrora loader This technique allowsthe tool to drastically cuts down its monitoring overhead andto increase its scalability

Likewise wemeasured the averagememory consumptionfor each application during five runs The collected data arerepresented in Figure 5 The overall average memory con-sumption of the tool is very small at just about 20 above theoriginal memory requirement even for high frequency appli-cations like RadioCountToLeds and RadioSenseToLedsthanks to our labeling scheme that reuses the same labelfor different instances of the same interrupt handler and thedetection protocol that maintains only a reduced size accesshistoryThis result corroborates the theoretical analysis of ourtechnique as summarized in Table 1 The low runtime over-head and memory consumption of our on-the-fly techniquemake it very scalable and suitable even for interrupt-drivenprograms that handle interrupts at high frequency rates

65 Labeling Overhead Since the race detection subcompo-nent of iRace is in charge of implementing the labeling and

Table 4 Average labeling overhead

Applications Time (Sec) Memory (Kb)6LowPanCli 313 2101760AnthiTheft 308 2148880BaseStation 309 2077680Blink 174 1964640MultihopOscilloscope 313 2141920Oscilloscope 249 2092640Powerup 176 1894720RadioCountToLeds 218 2167440RadioSenseToLeds 224 2140080Sense 184 2011040

Table 5The states of an access history for detecting data races withour protocol considering the example of Figure 2

119886119894

Read Write Races11988211

⟨0 20 0⟩ ⟨20⟩

11987712

⟨0 20 0⟩ ⟨20⟩

11988213

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11988213-11988211

11988224

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩11987715

⟨1 1 0⟩ ⟨mdash⟩ ⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11987715-11988211

the detection routines it was legitimate to analyze the impactof the labeling scheme alone on the monitored programsat runtime For this purpose we deactivated the detectionengine of iRace before running it again on the benchmarkapplications We collected the average execution time andmemory space consumed by each application after five runsas shown in Table 4 The simulation was done using a singlenode network

Figure 6 compares the original execution time of eachprogram from Table 3 with the execution time of the labelingengine as presented in the column ldquoTimerdquo of Table 4 Thecomparison shows that the average runtime overhead ofiRace with just the labeling engine activated is only about04x in the worst-case This represents only a small fractionof the average runtime overhead of 6x imposed by iRace asdiscussed in Section 64Themain reason of this performancecan be found at source level When browsing the source codeof the benchmark applications we came to understand thatprogrammers generally disable and enable all the interruptsat once instead of focusing on particular interrupts In thiscase the size of each label generated by the labeling enginenever approaches its worst-case of 119874(119879) meaning that thecomparison overhead of labels is also reduced This is veryimportant because as discussed in Section 64 althoughthe size of labels does not depend on the total number ofinterrupts received at runtime this number still has impacton the overall detection overheadThus we can conclude thatthe labeling engine of iRace incurs a very low overhead atruntime making it a highly scalable tool

We also compared the memory consumption of thelabeling engine with the memory requirement of the original

International Journal of Distributed Sensor Networks 13

TinyOS 210 programs

4

35

3

25

2

15

1

05

0

Exec

utio

n tim

e (s)

With iRace labeling engineOriginal

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 6 Average runtime overhead of the labeling engine

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

With iRace labeling engineOriginal

TinyOS 210 programs

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 7 Average memory consumption by the labeling engine

program execution presented inTable 3The results plotted inFigure 7 show that when the labeling engine alone is runningiRace consumes only about 12 more memory on averageHowever during the detection phase the overall averagememory consumption of iRace is only about 20 Theseresults also tell us that the detection engine does not increasethat much the memory consumption of iRace This is alsoreasonable because we only disabled the detection engine ofiRace but did not remove the resources it used

7 Conclusions

Asynchronous interrupt handling introduces fine-grainedconcurrency into interrupt-driven programs making themprone to data races In this paper we presented an efficientand scalable on-the-fly technique to precisely detect dataraces without false positives in such programs Our techniqueintroduced a lightweight labeling scheme for maintainingconcurrency information for the main program and each ofits interrupt handlers and a precise detection protocol thatguarantees to verify the existence of data races in a programby reporting at least one data race for each shared variableinvolved in a data race

We implemented a prototype of our technique as a racedetection tool called iRace on top of the Avrora simulationframework An evaluation of iRace revealed the existence ofdata races in some programs distributed with TinyOS andin some TinyOS components while incurring a slowdownof only about 6 times on average in the worst-case with anaverage memory consumption of only about 20 above theoriginal memory requirements of the program Moreoverthis evaluation also proved that the labeling engine alonegenerates an average runtime overhead of only about 04xwith an average memory consumption of only about 12relatively to the original program execution These resultscoupled with a theoretical analysis showed that the proposedtechnique is highly scalable and practical even for largeinterrupt-driven programs

Detecting data races that occur first in a program isan important issue since it may make other data races todisappear It is therefore interesting to detect this class of dataraces in interrupt-driven programs to facilitate the debuggingprocess Also we may be interested in extending our work tosupport multithreaded programs that handle interrupts andalso to detect data races in low-level kernel codes

Acknowledgment

This research was financially supported by the Ministryof Education (MOE) and National Research Foundation ofKorea (NRF) through the Human Resource Training Projectfor Regional Innovation and BK21+ program

References

[1] U Banerjee B Bliss Z Ma and P Petersen ldquoA theory of datarace detectionrdquo in Proceedings of the Workshop on Parallel andDistributed Systems Testing and Debugging (PADTAD rsquo06) pp69ndash78 July 2006

[2] R H B Netzer and B P Miller ldquoWhat are race conditionsSome issues and formalizationsrdquo ACM letters on programminglanguages and systems vol 1 no 1 pp 74ndash88 1992

[3] N G Leveson and C S Turner ldquoInvestigation of theTherac-25accidentsrdquo Computer vol 26 no 7 pp 18ndash41 1993

[4] USA Canada Final Report on the August 14 2003 Blackout inthe United States and Canada Causes and RecommendationsUS-Canada Power System Outage Task Force 2004

[5] J Erickson M Musuvathi S Burckhardt and K OlynykldquoEffective data-race detection for the kernelrdquo in Proceedings of

14 International Journal of Distributed Sensor Networks

the 9th Conference on Operating Systems Design and Implemen-tation (USENIX rsquo10) 2010

[6] B-K Lee M-H Kang K C Park J S Yi S W Yang andY-K Jun ldquoProgram conversion for detecting data races inconcurrent interrupt handlersrdquo Communications in Computerand Information Science vol 257 pp 407ndash415 2011

[7] L Effinger-Dean B Lucia L Ceze D Grossman and H JBoehm ldquoIFRit interference-free regions for dynamic data-racedetectionrdquo in Proceedings of the ACM International Conferenceon Object Oriented Programming Systems Languages and Appli-cations (OOPSLA rsquo12) vol 47 pp 467ndash484 2012

[8] C Flanagan and S N Freund ldquoFastTrack efficient and precisedynamic race detectionrdquo in Proceedings of the ACM Interna-tional Conference on Object Oriented Programming SystemsLanguages and Applications (OOPSLA rsquo09) vol 44 pp 121ndash1332009

[9] X Xie and J Xue ldquoAcculock accurate and efficient detection ofdata racesrdquo in Proceedings of the 9th International Symposium onCode Generation and Optimization (CGO rsquo11) pp 201ndash212 April2011

[10] TinyOSCommunity ForumTinyOSAnopen source operatingsystem for low-power wireless devices 2008 httpwwwtinyosnet

[11] M Higashi T Yamamoto Y Hayase T Ishio and K InoueldquoAn effective method to control interrupt handler for datarace detectionrdquo in International Workshop on Automation ofSoftware Test (ACM rsquo10) pp 79ndash86 May 2010

[12] Z Lai S C Cheung and W K Chan ldquoInter-context control-flow and data-flow test adequacy criteria for nesC applicationsrdquoin Proceedings of the 16th ACM SIGSOFT International Sympo-sium on the Foundations of Software Engineering (SIGSOFT rsquo08)pp 94ndash104 November 2008

[13] J Regehr ldquoRandom testing of interrupt-driven softwarerdquo inPro-ceedings of the 5th ACM International Conference on EmbeddedSoftware (EMSOFT rsquo05) pp 290ndash298 September 2005

[14] Y Zhou X Chen M R Lyu and J Liu ldquoSentomist Unveil-ing transient sensor network bugs via symptom miningrdquoin Proceedings of the 30th IEEE International Conference onDistributed Computing Systems (ICDCS rsquo10) pp 784ndash794 June2010

[15] G Gracioli and S Fischmeister ldquoTracing interrupts in embed-ded softwarerdquo in Proceedings of the ACM SIGPLANSIGBEDConference on Languages Compilers and Tools for EmbeddedSystems (LCTES rsquo09) vol 44 pp 137ndash146 2009

[16] V Sundaram P Eugster and X Zhang ldquoLightweight tracingfor wireless sensor networks debuggingrdquo in Proceedings of the4th International Workshop on Middleware Tools Services andRun-Time Support for Sensor Networks (ACM rsquo09) pp 13ndash18December 2009

[17] O K Ha and Y K Jun ldquoMonitoring of programs with nestedparallelismusing efficient thread labelingrdquo International JournalFo U- and E- Service Science and Technology vol 4 pp 21ndash362011

[18] Y C Kim S S Jun and Y K Jun ldquoA tool for space-efficient on-the-fly race detectionrdquo International Journal of DatabaseTheoryand Application vol 4 pp 25ndash38 2011

[19] S Savage M Burrows G Nelson P Sobalvarro and T Ander-son ldquoEraser a dynamic data race detector for multithreadedprogramsrdquo ACM Transactions on Computer Systems vol 15 no4 pp 391ndash411 1997

[20] A Jannesari and W F Tichy ldquoOn-the-fly race detection inmulti-threaded programsrdquo in Proceedings of the 6th Workshop

on Parallel and Distributed Systems Testing Analysis andDebugging (PADTAD rsquo08) July 2007

[21] D Gay M Welsh P Levis E Brewer R Von Behren and DCuller ldquoThe nesC language a holistic approach to networkedembedded systemsrdquo inACMSIGPLANConference on Program-ming Language Design and Implementation pp 1ndash11 June 2003

[22] J Regehr and N Cooprider ldquoInterrupt verification via threadverificationrdquo Electronic Notes in Theoretical Computer Sciencevol 174 no 9 pp 139ndash150 2007

[23] B L Titzer D K Lee and J Palsberg ldquoAvrora scalable sensornetwork simulation with precise timingrdquo in Proceedings ofthe 4th International Symposium on Information Processing inSensor Networks (IPSN rsquo05) pp 477ndash482 April 2005

[24] B L Titzer and J Palsberg ldquoNonintrusive precision instru-mentation of microcontroller softwarerdquo in Proceedings of theConference on Languages Compilers and Tools for EmbeddedSystems (Lctes rsquo05) vol 40 pp 59ndash68 2005

[25] J Hil R Szewczyk A Woo S Hollar D Culler and K PisterldquoSystem architecture directions for networked sensorsrdquo inProceedings of the 9th International Conference on ArchitecturalSupport for Programming Languages and Operating Systems(ASPLOS rsquo00) pp 93ndash104 2000

[26] GMTchamgoue KHKim andYK Jun ldquoTesting anddebug-ging concurrency bugs in event-driven programsrdquo InternationalJournal of Advanced Science and Technology vol 40 pp 55ndash682012

[27] GM Tchamgoue O K Ha K H Kim and Y K Jun ldquoA frame-work for on-the-fly race healing in ARINC-653 applicationsrdquoInternational Journal of Hybrid Information Technology vol 4pp 1ndash12 2011

[28] A Dinning and E Schonberg ldquoDetecting access anomaliesin programs with critical sectionsrdquo in Proceedings of theACMONR Workshop on Parallel and Distributed Debugging(PADD rsquo91) vol 26 pp 85ndash96 1991

[29] L Lamport ldquoTime clocks and the ordering of events in adistributed systemrdquo Communications of the ACM vol 21 no7 pp 558ndash565 1978

[30] Atmel inc ATmega128 Datasheet 2011 httpwwwatmelcomImagesdoc2467pdf

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 3: Research Article Verification of Data Races in Concurrent ...downloads.hindawi.com/journals/ijdsn/2013/953593.pdf · photoTask I20 W 11 R 12 W 13 W 24 R 15 T 2 J12 T 1 F : A POEG

International Journal of Distributed Sensor Networks 3

(1) TOS Msg msg lowast shared variable lowast(2) uint8 t packet Index lowast shared variable lowast(3) async event result t ADCdataReady (uint16 t data) (4) struct OscopeMsg lowastpack(5) pack = (struct OscopeMsg lowast) msgdata(6) pack-gtdata [packetIndex++] = data(7) if (packetIndex == BUFFER SIZE) (8) post dataTask ( )(9)

(10)

(11) async event result t PhotoSensordataReady (uint16 t data) (12) post photoTask ( )(13)

(14) task void dataTask ( ) (15) struct OscopeMsg lowastpack(16) atomic (17) pack = (struct OscopeMsg lowast)msgdata(18) check (pack)(19) packetIndex = 0(20)

(21) pack-gtchannel = 1(22) pack-gtsourceMoteID = TOS LOCAL ADDRESS(23) send (ampmsg)(24)

(25) task void photoTask ( ) lowast do something lowast

Algorithm 1 An example nesC program

In order to minimize the impact of interrupt handlerson tasks all long-latency operations must run in split-phaseWith this mechanism interrupt handlers immediately returnafter servicing critical operations and post heavy compu-tations for later execution as new tasks The concurrencymodel of interrupt-driven programs allows at runtime anexponentially increasing number of execution paths makingthem difficult to reason about Since the only way to sharedata with interrupt handlers is through global variablesinterrupt-driven programs are prone to concurrency bugslike data races [1 2 13 26] To protect sensitive parts of theprogram interrupts must be disabled before and enabled onlyafter critical sections Interrupts can be either asynchronousor synchronous Asynchronous interrupts are signaled byexternal devices such as network interfaces and can fireat any time that the corresponding device is enabled [13]Synchronous interrupts on the other hand are those thatarrive in response to a specific action taken by the processorsuch as setting a timer

22 Data Races In shared-memory concurrent programsdata races [1 2 27] represent one of the most notorious classof concurrency bugs Data races occur when two threadsaccess a shared memory location without proper synchro-nization and at least one of the accesses is a write It is impor-tant to detect data races because they may lead programs tonondeterministic executions with unpredictable results

For illustration we consider the example program ofAlgorithm 1 that is adapted from theOscilloscope applicationdistributed with TinyOS [10] This application allows a node

of a sensor network to periodically sample cache and senddata from its sensors We assume that interrupt handlers inAlgorithm 1 are not reentrant and that they automaticallydisable themselves before they run We define a partial orderexecution graph (POEG) [28] to represent an executioninstance of an interrupt-driven program Unless explicitlymentioned wewill use the term thread to refer to either a taskor an interrupt handlerTherefore we define a thread segmentas a sequence of instructions executed by a single thread anddelimited by either a disabling or an enabling instructionThus a thread119879

119896is a finite set of 119899 sequentially ordered thread

segments that is 119879119896= 1199051 1199052 119905

119899 where 119899 represents the

number of disableenable instructions plus one and 119896 a taskidentifier or an interrupt number We therefore introduce thefollowing definitions

Definition 1 (POEG) A POEG is a directed acyclic graph of2-tuple 119866 = (119878 119864) where 119878 is a set of thread segments and119864 a set of synchronization edges A synchronization edge iseither a task posting edge for posting operations or a taskordering edge to represent the implicit order relation thatexists between tasks

Definition 2 (happens-before [29]) Given a program execu-tion if a thread segment 119905

119894runs earlier than another thread

segment 119905119895 then 119905

119894happens-before 119905

119895 denoted 119905

119894

ℎ119887

997888997888rarr 119905119895 If

the happens-before relation (ie 119905119894

ℎ119887

997888997888rarr 119905119895or 119905119895

ℎ119887

997888997888rarr 119905119894) is

not satisfied between 119905119894and 119905119895 the two thread segments are

concurrent and denoted by 119905119894 119905119895

4 International Journal of Distributed Sensor Networks

ADC20

20

20

20

12

12

Task posting edgeTask ordering edge

lowast

lowast

PhotoSensor

dataTask

photoTask

I20W11

R12

W13

W24

R15

T2

J12

T1

Figure 1 A POEG of an execution instance of the program inAlgorithm 1 Only accesses to msg are represented

In a POEG two accesses are concurrent if and onlyif there is no path between them Otherwise they areordered by the happens-before relation An example POEGfor the program in Algorithm 1 considering only accesses tothe shared variable msg is represented in Figure 1 In thisPOEG vertical lines symbolize threads while read and writeaccesses to shared variables are respectively represented bythe letters 119877

119894119895and119882

119894119895 where 119894 is the thread instance number

and 119895 an observed execution order We use the couple ofsymbols⊓119899 and⊔119899 to respectively represent the disabling andenabling of interrupts where 119899 is the corresponding interruptnumber that will be replaced with the star symbol (lowast) ifall interrupts are concerned The POEG of Figure 1 depictsan execution where a first instance of the ADC (Anolog-to-Digital Converter) interrupt handler collected new dataand posted an instance of dataTask Furthermore anotherinstance of the ADC interrupt arrives and preempts theexecution of dataTask

Although the first instance of the ADC interrupt handlerposted dataTask if the second ADC interrupt is receivedright before the message is sent at line 23 of Algorithm 1 dataraces will still occur between the write access in ADC andthe accesses 119882

13and 119877

15in dataTask making a corrupted

message to be delivered Those are the data races representedin Figure 1 by the pair of accesses 119882

13-11988224

and 11987715-11988224 If

a task is posted by an interrupt handler the task postingedge guarantees a happens-before relation only between theposted task and the actual instance of the interrupt handlerthat posted it but not with future instances of the sameinterrupt handler Consequently we ignore the task postingedge when traversing the graph for analyzing the happens-before relation because we use a single arrow to represent

all instances of the same interrupt Thus we can also reportthe data races 119882

13-11988211

and 11987715-11988211

between 1198791and 119868

20

Therefore in our POEG we assume that two thread segments119905119894and 119905119895 respectively from two different threads119879

119894and119879119895are

always concurrent unless they are ordered by a task orderingedge or one of them is disabled by the other one The nesClanguage [21] contains a static data race detector that howeveris still unsound since it cannot follow pointers [12 21 22]Thus the data races presented in Figure 1 cannot be reportedby the native nesC data race detector

3 Lightweight Labeling Scheme

For an effective dynamic race detection it is essentialto adequately determine the order of precedence betweenthread segments accessing shared memory locations Forthis purpose this section presents our lightweight labelingscheme used for capturing the logical concurrency betweenthread segments of interrupt-driven programs

31 GeneratingThread Labels Happens-before based on-the-fly race detection techniques generally rely on a variationof Lamportrsquos logical timestamp [29] in order to preciselyidentify concurrent thread segments of a program Theeffectiveness of on-the-fly techniques is highly dependenton the cost and accuracy of this identification process Ourlabeling scheme generates a unique identifier called label foreach thread segment 119905

119894during an execution instance of an

interrupt-driven program For the labeling scheme we needto introduce the following definitions

Definition 3 (critical section) A critical section with respectto a given interrupt 119896 is a portion of program comprisingbetween a disable and probably an enable instruction tointerrupt 119896

Definition 4 (interrupt set) The interrupt set or 119894119904119890119905 of athread segment 119905

119894represents the set of all interrupts 119896disabled

by 119905119894Thus this guarantees a happens-before relation between

119905119894and every interrupt in its 119894119904119890119905 Formally for all 119896 isin 119905

119894sdot 119894119904119890119905

for all 119905119895isin 119879119896 119905119894

ℎ119887

997888997888rarr 119905119895or 119905119895

ℎ119887

997888997888rarr 119905119894

Definition 5 (critical section violation) A thread segment 119905119894

of a thread 119879119896violates the critical section of another thread

segment 119905119895 denoted by 119905

119894

V999492999484 119905119895 when 119905

119895can be preempted by

119905119894

The label of each thread segment 119905119894consists of two compo-

nents an identifier referred to as segment descriptor 119878119863119894 and

an interrupt set 119894119904119890119905119894 Put together we use 119878119863

119894⟨119894119904119890119905119894⟩ to desig-

nate the label of a thread segment 119905119894 The segment descriptor

119878119863119894of a thread segment 119905

119894is defined as a triplet ⟨119905119905

119894 119905119894119889119894 119888119904119894⟩

where 119905119905119894specifies whether 119905119894119889

119894is a task or an interrupt

handler 119905119894119889119894contains the task identifier of a task or the

interrupt number of an interrupt and 119888119904119894indicates whether

the thread segment has disabled all interrupts in the systemCreating labels for thread segments of an interrupt-driven

program is straightforward with our labeling scheme as

International Journal of Distributed Sensor Networks 5

suggested by the algorithms in Algorithm 2 A new labeledthread 119879

119894is created when a new task starts its execution or

whenever an interrupt handler is invoked When a new taskstarts a new label is generated for the first thread segment ofthe task by simply initializing the component of the label asin Algorithm 2(a) A new label is similarly created when aninterrupt handler is invoked in Algorithm 2(b) Each threadthen creates additional thread segments whenever it disablesor enables interrupts

When a thread segment 119905119894of a thread 119879

119894disables an

interrupt 119896 the label of the new thread segment 119905119895is generated

by adding the interrupt number 119896 into the interrupt set ofthe current label owned by 119905

119894 However if all interrupts in

the system are disabled by 119905119894 the new label for 119905

119895is obtained

by setting 119888119904119894and by emptying 119894119904119890119905

119894from the current label

owned by 119905119894as shown in Algorithm 2(c) Thus as seen in

Algorithm 2(d) if 119905119894enables an interrupt 119896 a new label for a

new thread segment 119905119895is obtained by simply removing 119896 from

the interrupt set of the current label in 119905119894 If all interrupts are

enabled at the same time the flag 119888119904119894and the interrupt set

119894119904119890119905119894are both reset from the current label of 119905

119894to generate

the new label for the new thread segment 119905119895 This helps

maintain a reduced-size interrupt set for each thread segmentsince programmers generally disable and enable all interruptsrather than manipulating individual interrupts

Moreover the size of an interrupt set is bounded by thetotal number of interrupts supported by the hardware onwhich the program runs which is generally a reasonableinteger number By reusing the same label for differentinstances of the same interrupt handler our labeling schemedoes not depend on the total number of interrupts receivedat runtime which can be very high

Lemma 6 Given two labeled thread segments 119905119894and 119905119895 with

119905119894belonging to an interrupt handler 119879

119896 119905119894

V999492999484 119905119895rArr 119896 notin 119894119904119890119905

119895or

119888119904119895= 119891119886119897119904119890

Proof Let us consider two labeled thread segments 119905119894and 119905119895

and let us assume that 119905119894belongs to an interrupt handler 119879

119896

We also assume that 119905119894violates the critical section of 119905

119895 By

definition thread segments are delimited by either the disableor enable instructions Therefore if 119905

119895was created after a

the disable instruction targeting interrupt 119896 the interruptnumber 119896 will be in the interrupt set 119894119904119890119905

119895of 119905119895according

to the labeling algorithms of Algorithm 2 However if 119905119895was

created after the disabling of all interrupts 119888119904119895would then be

true by the labeling algorithms of Algorithm 2 to signify thatall interrupts including 119896 are disabledThus for all 119905

119894isin 119879119896 119905119894

cannot violate 119905119895 which is contradictory since we assumed

119905119894

V999492999484 119905119895 Therefore 119905

119895has necessarily enabled 119896 that is

for all 119905119894isin 119879119896 119905119894

V999492999484 119905119895rArr 119896 notin 119894119904119890119905

119895or 119888119904119895= 119891119886119897119904119890

32 Concurrency Relation Using our labeling scheme itbecomes easy to analyze the happens-before relation betweenany pair of thread segments 119905

119894and 119905119895 by comparing their

labels

[a Task Starting]a1 tsk start()a2 Label Lba3 Lbt = truea4 Lbtid = getTaskid()a5 Lbcs = falsea6 Lbiset =

a7 return Lb[b Interrupt Invocation]b1 int invoke()b2 Label Lbb3 Lbt = falseb4 Lbtid = getIntNumber()b5 Lbcs = falseb6 Lbiset =

b7 return Lb[c Interrupt Disabling]c1 int disable(Label Lb Interrupt n)c2 if(n gt 0) Lbiset cup 119899

c3 elsec4 Lbcs = truec5 Lbiset =

c6 endifc7 return Lb[d Interrupt Enabling]d1 int enable(Label Lb Interrupt n)d2 if(n gt 0) Lbiset-nd3 elsed4 Lbcs = falsed5 Lbiset =

d6 endifd7 return Lb

Algorithm 2 Labeling algorithms

Theorem7 A labeled thread segment 119905119894happens before anoth-

er labeled thread segment 119905119895 denoted by 119905

119894

ℎ119887

997888997888rarr 119905119895 if and only if

either that 119905119894and 119905119895belong to tasks or that 119905

119895does not violate

119905119894 that is not(119905

119895

V999492999484 119905119894)

Proof Let us assume two labeled thread segments 119905119894and

119905119895 such that 119905

119894

ℎ119887

997888997888rarr 119905119895 Thus according to the labeling

algorithms of Algorithm 2 we should have 119905119905119894= 119905119905119895= 119905119903119906119890

meaning that 119905119894and 119905

119895are either from the same or from

different tasks By definition tasks are all ordered and cannotpreempt each other However if 119905

119895is a thread segment of

an interrupt handler 119879119896 according to Definition 4 either

119888119904119894= 119905119903119906119890 because 119905

119894disabled all interrupts or the thread

identifier 119905119894119889119895of 119905119895that corresponds to 119896 the interrupt

number of 119879119896 is in 119894119904119890119905

119894 the interrupt set of 119905

119894 This means

that 119905119894is a critical section thread segment that disables either

all interrupts or particularly the interrupt number 119896 Thusaccording to Lemma 6 119905

119895does not violate the critical section

of 119905119894 Therefore we can say that 119905

119894

ℎ119887

997888997888rarr 119905119895implies either that 119905

119894

and 119905119895belong to tasks or that not(119905

119895

V999492999484 119905119894)

6 International Journal of Distributed Sensor Networks

Now let us assume that 119905119894and 119905

119895belong to tasks

Naturally 119905119894and 119905119895are ordered and the labeling algorithms of

Algorithm 2 will set 119905119905119894= 119905119905119895= 119905119903119906119890 meaning that 119905

119894

ℎ119887

997888997888rarr 119905119895

On the other hand if we assume that not(119905119895

V999492999484 119905119894) then 119905

119895

must be part of a given interrupt handler 119879119896and either 119905

119894has

disabled all interrupts to set 119888119904119894= 119905119903119906119890 or the thread identifier

119905119894119889119895= 119896 of 119905

119895is in the interrupt set 119894119904119890119905

119894of 119905119894 according to

Definition 4 and Lemma 6 Therefore we can conclude that119905119894

ℎ119887

997888997888rarr 119905119895

Any two labeled thread segments 119905119894and 119905119895are therefore

concurrent if they do not satisfy the happens-before relationas stated by Theorem 7 which can formally be rewritten asfollows

119905119894

ℎ119887

997888997888rarr 119905119895lArrrArr

119905119905119894= 119905119905119895= 119905119903119906119890

or(119905119905119895= 119891119886119897119904119890) and (119888119904

119894= 119905119903119906119890 or 119905119894119889

119895isin 119894119904119890119905119894)

(1)

By comparing based on Theorem 7 the labels of thethread segments in the POEG represented by Figure 2 wecan easily obtain that the thread segments of 119868

20 1198941 and 119894

2

are concurrent with other thread segments 1199051 1199053 1199054 and

1198951 Therefore the write accesses in 119894

1and 1198942conflict with

the other accesses in 1199053 leading to four data races 119882

11-11988213

11988211-1198771511988224-11988213 and119882

24-11987715 However since the accesses

11988211and119882

23 respectively from 119894

1and 1198942 represent the same

write access the set of data races to be reported for an effectivedebugging process should be reduced to only 119882

11-11988213

and11988211-11987715 By disabling all interrupts 119905

2enforces the happens-

before relation with other thread segments Although 1198791is

posted by 11986820 they are still concurrent Also the thread

segment 1198951is concurrent with 119894

1 1198942 1199051 1199053 and 119905

4

4 Precise Detection Protocol

The race detection protocol monitors memory accesses bymaintaining a special data structure called access history intowhich are stored the labels of prior accesses to each sharedvariable Our detection protocol stores in each entry of theaccess history of a shared variable a unique label representingthe thread segment that accessed the variable as describedin Section 3 At each access to a shared variable data racesare reported by analyzing based onTheorem 7 the happens-before relation between the current thread segment and thelabels of prior thread segments stored in the access historyTherefore the access history for each shared variable hasto be efficiently organized in order to reduce the overheadgenerated by the detection process

Our precise detection protocol maintains an access his-tory consisting of two sets for each shared variable a Readset and a Write set to respectively store labels of threadsegments that read and write the variable We use 119860119867(119909)

to designate the access history of a shared variable 119909 Thus119860119867(119909) sdot 119877119890119886119889 and 119860119867(119909) sdot 119882119903119894119905119890 refer to the read set andthe write set of 119860119867(119909) respectively The algorithms used bythe detection protocol are presented in Algorithm 3 In thesealgorithms we assume that the variable 119871119887 always contains

20

20

20

20

t1

t2

t3

t4

12

12

lowast

lowast

i1⟨0 20 0⟩⟨20⟩

⟨mdash⟩⟨1 1 1⟩

⟨mdash⟩⟨1 1 0⟩

⟨mdash⟩⟨1 1 0⟩

⟨0 12 0⟩⟨12⟩

⟨0 20 0⟩⟨20⟩

⟨mdash⟩⟨1 2 0⟩

I20

W11

R12

W13

W24

R15

T2

J12

T1

j1

i2

Figure 2 An example labeled POEG for an execution instance ofthe program in Algorithm 1

the label of the current thread segment accessing a sharedvariable and that 119901119871119887 represents any prior label stored in theaccess history

When a read access occurs to a shared variable 119909the detection protocol calls the ah read() function ofAlgorithm 3(a) that then reports data races by analyzing thehappens-before relation between the label of the accessingthread segment and the labels of prior accesses stored in119860119867(119909) sdot119882119903119894119905119890 Similarly upon a write access to a shared vari-able the monitoring function ah write() of Algorithm 3(b)reports data races by looking into both 119860119867(119909) sdot 119877119890119886119889 and119860119867(119909) sdot 119882119903119894119905119890 We assume the existence of a functionOrdered() that based on Theorem 7 decides whether athread segment 119905

119894happens before another thread segment 119905

119895

by comparing their labelsIn order to maintain a reduced size access history

accesses from critical sections that concern all interruptswithin a task are not inserted in the access history sincethey will never run into a data race with other accessesfrom interrupt handlers Also only the first read and writeaccesses to a shared variable from the same thread segmentare kept into the access history Subsequent similar accessesto the same shared variable from the same thread segmentare filtered outThis filtering operations are performed by theisFiltered() function of Algorithm 3(c) that is called by bothah read() and ah write() before any other action

After data races have been reported with the currentaccess the function maintain ah() of Algorithm 3(d) is usedfor keeping the size of the access history balanced Thisfunction replaces some previous accesses in the access historywith the current access Thus accesses from different tasksare considered from the same thread since tasks by defini-tion are all ordered by happens-before Similarly accesses

International Journal of Distributed Sensor Networks 7

(1) ah read(Label Lb Access History AH Variable x)(2) if (isFiltered(Lb AH(x)Read)) then return(3) for all labels pLb in AH(x)Write do(4) if (Not Ordered(Lb pLb)) then(5) report race(x pLb Lb)(6) end for(7) maintain ah(Lb AH(x)Read)

(a) Monitoring Protocol for Read Accesses(1) ah write(Label Lb Access History AHVariable x)(2) if (isFiltered(Lb AH(x)Write)) then return(3) for all labels pLb in(4) AH(x)Write cup AH(x)Read do(5) if (Not Ordered(Lb pLb)) then(6) report race(x pLb Lb)(7) end for(8) maintain ah(Lb AH(x)Write)

(b) Monitoring Protocol for Write Accesses(1) isFiltered(Label Lb Access History AH)(2) if (Lbcs = true) and (Lbtt = true)(3) then return true(4) if (any access with a label pLb exists in AH(5) such that pLb = Lb) then return true(6) else(7) return false

(c) Filtering Redundant Accesses(1) maintain ah(Label Lb Access History AH)(2) if (Lbtt = true) then(3) delete all entries in AH with label pLb(4) such that pLbtt = true(5) else(6) delete all entries in AH with label pLb such(7) that pLbtt = Lbtt and pLbtid = Lbtid(8) end if(9) add ah(Lb AH)

(d) Maintaining Access History

Algorithm 3 Labeling algorithms

from different instances of the same interrupt handler areconsidered from a single instance of the interrupt handlerThis property is already enforced by the labeling scheme thatreuses the same label for all instances of the same interrupthandler Again we assume the existence of another functionadd ah() which is able to insert the label of the current threadsegment into the appropriate set of the access history given ashared variable

Table 5 shows the different states of an access historyfor detecting data races with our protocol in the exampleprogram of Figure 2 where accesses to the shared variablemsg appear in the first column When the first access 119882

11

occurs the label ⟨0 20 0⟩⟨20⟩ of the thread segment 1198941

is simply added into the access history and no data racereported as there is no other access prior to 119882

11 The next

access 11987712 from the thread segment 119905

2is filtered out because

1199052that belongs to task 119879

1has disabled all interrupts The

following access 11988213

occurs on 1199053with interrupts enabled

Thus a data race11988213-11988211is reported by comparing the label

of 1199053 ⟨1 1 0⟩⟨minus⟩ with the label of 119894

1previously stored into the

access history Afterwards the label of 1199053is inserted into the

write set of the access history The thread segment 1199053is then

preempted by 1198942 but the access 119882

24is filtered out because

1198942which is a new instance of the interrupt 119868

20 also shares the

same label ⟨0 20 0⟩⟨20⟩ with 1198941which previously performed

a write and whose label is already in the access history Afterthe execution of 119894

2 1199053resumes to perform the read access

11987715 Another data race 119877

15-11988211

is reported and the label of1199053 ⟨1 1 0⟩⟨minus⟩ stored into the read set of the access history

Although the program received two instances of the ADCinterrupt our protocol precisely reported only the set of twodata races needed for effectively debug the program Thisshows that our detection protocol does not depend on thenumber of interrupts received at runtime which can be veryhigh

Although it keeps only a reduced set of labels in the accesshistory our precise detection protocol even with a minimalnumber of interrupts still guarantees to verify the existenceof data races in a program For this purpose the protocol isdesigned to report without false positives at least one datarace for each shared variable involved in a data race

8 International Journal of Distributed Sensor Networks

Theorem 8 Given an interrupt-driven program 119875 the detec-tion protocol reports a data race over a shared variable 119909 if andonly if there exists a data race over 119909 in an execution instanceof 119875

Proof We will first prove that if a data race is reported on ashared variable 119909 by the detection protocol then there alsoexists a data race in an execution instance of the program forthe same shared variable 119909 For this we assume a snapshotof the access history 119860119867(119909) of the shared variable 119909 at agiven time and the current access event 119886

119894to 119909 from a thread

segment 119905119894that owns a label we refer to as 119871119887

119894in an execution

instance of the program 119875 We can then distinguish twodifferent cases

Case A 119860119867(119909) is empty and there is obviously no data racefor 119909 In this case the current access 119886

119894is the only access to

119909 from the program 119875 and 119871119887119894will be added by the function

add ah() into119860119867(119909)sdot119877119890119886119889 if 119886119894is a read access or into119860119867(119909)sdot

119882119903119894119905119890 if it is a write access

Case B 119860119867(119909) is not empty Data races are reported usingthe ah read() function if 119886

119894is a read access or the ah write()

in case of a write access In both cases the happens-beforerelation is analyzed between the label 119871119887

119894of 119905119894and previous

labels stored in 119860119867(119909) according to Theorem 7 The outputsof these analyses can be classified into two categories

(1) There is no data race between 119886119894and119860119867(119909) therefore

all previous accesses stored in 119860119867(119909) happenedbefore 119886

119894meaning that no thread segment prior to

119905119894in 119875 is concurrent with 119905

119894 they are all ordered by

happens-before Thus for all 119886119895isin 119905119895and 119871119887119895isin 119860119867(119909)

119905119895

ℎ119887

997888997888rarr 119905119894 In which case 119875 also has no data race

(2) There are data races between 119886119894and prior accesses

in 119860119867(119909) this means that there are some labels in119860119867(119909) belonging to some prior thread segments thatare not ordered with 119905

119894 Concretely there are thread

segments in 119875 that are concurrent with 119905119894 Formally

exist119905119895isin 119875119871119887

119895isin 119860119867(119909) 119905

119894 119905119895Therefore the execution

instance of 119875 also contains data races over 119909

Now we show that if there exists a data race over a sharedvariable 119909 in an execution instance of the program 119875 thenour detection protocol also reports a data race over 119909 Usingthe contrapositive we will prove that if the detection protocoldoes not report any data race the program 119875 also does notcontain any for a shared variable 119909 We again consider asnapshot of the access history 119860119867(119909) for a shared variable119909 at a given time We also assume that the detection protocolreported no data races for the shared variable 119909 This meansthat 119860119867(119909) is either empty and there is clearly no data racesince there is no access to 119909 from 119875 yet or contains onlylabels of thread segments that are all ordered by the happens-before relation in which case there is no concurrent threadsegment in119875 which also implies that119875 has no data races over119909 Therefore we can formally write for all 119905

119894 119905119895isin 119875 and 119871119887

119894

119871119887119895isin 119860119867(119909) 119905

119894

ℎ119887

997888997888rarr 119905119895or 119905119895

ℎ119887

997888997888rarr 119905119894 Thus 119875 also contains no

data race for 119909

Table 1 Efficiency of the precise detection protocol

Space TimeNew label comparison Per access

O(1198811198792) O(1) O(LogT) O(TLogT)

5 Theoretical Analysis

On-the-fly data race detection techniques are known to havehigh runtime overhead that is generally provoked by the timespentmonitoring accesses to shared variables and to comparethread labels In this section we theoretically analyze theefficiency of our technique which depends on119881 the numberof shared variables and 119879 the total number of interruptsin the underlying system plus one the number of tasksIn general 119879 is very small since the number of interruptshandled by a system is also kept small For example thepopular Atmel ATmega128 [30] processor supports only amaximum of 34 interrupts Also because of the implicithappens-before relation that exists between all tasks they areviewed as a single task by the detection protocol

Table 1 presents the space and time complexities of ourlabeling and detection techniques The access history usedby our technique stores from each thread segment at mosttwo accesses for each shared variable Each label in the accesshistory has a worst-case size of 119874(119879) that represents thesize of its interrupt set Therefore the total labeling spacerequirement for each shared variable is 119874(1198792) Thus thetotal space complexity of our technique is 119874(1198811198792) Howeverbecause the number of interrupts is generally a reasonableinteger number the space complexity of our technique canfurther be reduced to 119874(119881)

A new label is always created in a constant amount of timeby our labeling algorithms Interrupts are generally givennumbers in a sequence order Therefore a binary searchtechnique is applied to locate values in interrupt sets whichcontain only interrupt numbers Thus the time complexityfor comparing any two labels is 119874(Log119879) in the worst-caseand 119874(1) in most cases where the interrupt sets are emptyAt each access to a shared variable the total time complexityto report data races with the previous accesses stored in theaccess history is then119874(119879Log119879)The labels and the detectionprotocol do not depend on the total number of interruptsreceived at runtimemaking our technique highly scalable andpractical

6 Experimental Results

We empirically evaluated the preciseness and efficiency ofour technique using a set of well-known programs providedwith TinyOS [10] This section details the implementationchallenges and presents the analysis results of the experimen-tation

61 Implementation We implemented a prototype of ourdata race detection tool for interrupt-driven programs callediRace based on the concurrency model of TinyOS [21] andits nesC language [25] We implemented iRace on top

International Journal of Distributed Sensor Networks 9

Table 2 Characteristics of the benchmark applications from TinyOS 210 [10]

Applications LOC Int SV Brief description6LowPanCli 19116 4 9 Implements a command line interface using the 6lowpan stackAnthiTheft 27421 5 4 Detects and reports a stolen mote by checking some sensorsBaseStation 14905 6 13 Sends and receives simple active messages on the radio and serial linksBlink 3688 1 0 Periodically blinks the three LEDs of a moteMultihopOscilloscope 24310 9 8 Periodically samples and broadcasts sensor data to a base stationOscilloscope 11686 5 5 Periodically samples sensor data and broadcasts a message every 10 timesPowerup 2352 0 0 Turns on the red LED when the mote powers upRadioCountToLeds 11165 4 3 Periodically samples sensors and displays the last 3 bits on its LEDsRadioSenseToLeds 11472 5 2 Periodically update a counter and displays the last 3 bits on its LEDsSense 5743 2 0 Periodically takes sensor readings and displays the values on the LEDs

of the Avrora [23] simulation and analysis framework thatprovides a time accurate simulator different analysis toolsknown asmonitors and a binary instrumentation engine [24]for programs written for AVR microcontrollers and sensornetworks All these features make Avrora just suitable forour tool Simulation is commonly used in the developmentcycle of embedded software because target systems generallylack conventional inputoutput interfaces With iRace thefunctionalities of Avrora are extendedwith a precise data racedetection tool that can easily be loaded as one of Avrorarsquosmonitors

As shown in Figure 3 iRace consists of two maincomponents the Instrumenter which exploits the Avrorainstrumentation engine [24] and the core Race Detectoritself The Instrumenter monitors task and interrupts activ-ities through its iMonitor component and provides entrypoints for the labeling routines Instrumentation points areinserted at the beginning of each task at the start andexit of each interrupt handler and also before interruptsare either enabled or disabled Similarly accesses to sharedmemory locations are monitored using the AccessMonitorcomponent that also calls routines of the detection proto-col The AccessMonitor instruments before read and writeaccesses to shared variables The Race Detector is composedof labeling routines that generate and manage labels for eachactive thread segment at runtime and detection routines formaintaining the access history and reporting data races on-the-fly as shared variables are manipulated

In TinyOS [21 25] interrupt handlers and tasks all runto completion unless preempted by other interrupt handlersThus only a single thread runs at any point of time givingus the opportunity to use only a single variable to maintainthe active thread label at detection time However whenpreemption happens the label of the preempted threadsegment must be saved and restored again when the activeinterrupt handler exits Otherwise there will be no way toknow about the concurrency information of the preemptedthread segment when it gains back the execution control Inorder tomanage the preemption we use a stack data structurewhere labels can be pushed and restored from since interruptscan be nested This provides a simple but powerful way tohandle preemption with negligible overhead

Our labeling scheme needs to uniquely identify eachinterrupt handler whenever it is invoked and each taskwhen it starts its execution Each interrupt handler is easilyidentified by the interrupt number of the interrupt it servesFor task we chose to generate a unique task identifier bysimply maintaining a counter that increases at each timea task starts its execution Thus each task is attributed aunique identifier thanks to the fact that once posted TinyOStasks run in a first-in first-out manner and cannot preempteach other By manipulating only binary files iRace doesnot require any source file nor source code annotation toeffectively detect data races However compiling the programwith debugging information may help iRace generate moredetailed debugging information on data races it reports

62 Environment We implemented and evaluated our tech-nique on a system that runs the kernel 30 of the Linuxoperating system using an Intel dual-core 320GHz and 4GBof mainmemory On this machine was installed TinyOS 210[10] along with the nesC compiler 131 the GNU C compiler46 the Java Runtime Environment 17 and the latest Avroraframework version 17115

Table 2 presents the characteristics and a brief descriptionof the benchmark applications selected for our evaluationfrom the TinyOS 210 repository [10] These applications arefreely distributed with each version of TinyOS The columnldquoLOCrdquo indicates the number of line of C codes generatedby the nesC compiler after compilation while the columnldquoIntrdquo contains the number of interrupts handled by eachapplicationThe number of shared variables for each programappears in column ldquoSVrdquo It is worthmentioning that this num-ber represents only the shared variables in the source code ofeach program The exact number of variables shared by eachapplicationmay finally depend on the number of componentsit shares with the operating system itself Applications writtenin nesC share TinyOS components by wiring themselvesto these components increasing therefore their number ofshared resources We compiled each application for theMica2 platform and measured the average runtime overheadover five executions The Mica2 motes are sensor networknodes based on the ATmega128 [30] microcontrollers Weconfigured Avrora to simulate a sensor network during 10

10 International Journal of Distributed Sensor Networks

iRace

and instrumentationframework

iMonitor

AccessMonitor

Race detector

Labeling routines

Detection protocolroutines

Binarycode Data

races

Avrora simulation

Figure 3 The Architecture of the Data Race Detection Tool

Table 3 Reported data races on the benchmark applications

Applications Data races Verif races Time (Sec) Memory (Kb)Original iRace Original iRace

6LowPanCli 0 0 269 357 1936960 2168160AnthiTheft 15 15 292 457 1944400 2320000BaseStation 0 0 279 376 1912240 2148240Blink 0 0 161 180 1887280 1949600MultihopOscilloscope 0 0 278 400 1939600 2196960Oscilloscope 13 13 179 328 1909360 2201760Powerup 0 0 151 186 1873600 1879920RadioCountToLeds 17 17 196 1255 1942880 2326320RadioSenseToLeds 15 15 181 1305 1944480 2298240Sense 0 0 158 192 1866400 2050560

seconds allowing each application to run exactly for 73728000instruction cycles We set to five the number of nodes in thesimulated sensor network

63 Evaluation For data race detectionwe instructedAvrorato simulate a sensor network with five nodes while thebenchmark applications are monitored by iRace Table 3represents the results of the evaluation of iRace on thebenchmark applications in its column ldquoData Racesrdquo whichcontains the number of shared variables involved in adata race for each application Our tool reported no datarace for all applications except for four AnthiTheft with15 shared variables Oscilloscope with 13 shared vari-ables RadioCountToLeds with 17 shared variables andRadioSenseToLeds with 15 shared variables Surprisinglymost of the reported data races reside inside some sharedsystem components of TinyOS itself The reported data racesare caused by two different interrupts namely the serialperipheral interface (SPI) interrupt and the timer interruptAlso we carefully analyzed the reported data races andinspected the source code of the applications for a manualverification process as shown in the column ldquoVerified racesrdquoof Table 3

In the AnthiTheft application for example which isused to detect and report a stolen mote by checking the levelof the light or the acceleration data races were reported over

15 shared variables Most of the reported data races are how-ever due to the fact that the SPI interrupt handler can preemptsome functions or tasks owned by the underlying radio stackThis implies that if a message is being received or sent whena SPI interrupt occurs shared variables like radioState orcount in the component CC1000SendReceiveP might becorrupted However all shared variables involved in a datarace do not directly belong to AnthiTheft but to functionsof the CC1000 radio stack in use in Mica2 motes Similardata races are also reported in RadioCountToLeds andRadioSenseToLeds since they all access the radio stack

Another data race reported both in RadioCountToLedsand RadioSenseToLeds and worthy to be commentedconcerns the shared variable reqResId inside thecomponent ArbiterP (Recently this data race hasalso been pointed out and a patch discussed in theTinyOS Forum) This data race occurs when the functionimmediateRequest() called within the function read ofthe component immediateRequest is preempted by a SPIinterrupt handler An excerpt of this function is presentedin Algorithm 4 At line 2 of Algorithm 4 the handle of aresource is requested and eventually locked out if grantedHowever if a SPI interrupt is received somewhere afterline 2 a data race will occur and the resource handlerstored in reqResId (line 6 Algorithm 4) may eventually beoverwritten by the SPI interrupt handler Also other shared

International Journal of Distributed Sensor Networks 11

(1) async command error t ResourceimmediateRequest [uint8 t id] ( ) (2) signal ResourceRequested immediateRequested [resId] ( )(3) atomic (4) if (state == RES CONTROLLED) (5) state = RES IMM GRANTING(6) reqResId = id(7)

(8) else return FAIL(9)

(10) signal ResourceDefaultOwner immediateRequested( )(11) sdot sdot sdot (12)

Algorithm 4 An excerpt of the request function from the component ArbiterP

(1) command error t Send send[uint8 t clientId] (message t lowastmsg(2) uint8 t len) (3) sdot sdot sdot (4) queue [clientId] msg = msg(5) call Packet setPayloadLength (msg len)(6) sdot sdot sdot (7)

Algorithm 5 An excerpt of the send function from the component AMSend

variables in ArbiterP like state can be corrupted as wellby the SPI interrupt handler

In the case of Oscilloscope which is a data collectionapplication that periodically samples the default sensor ofa mote and broadcasts a message over the radio every 10readings the reported data races are similar to those reportedin AnthiTheft Moreover our results suggest that if a SPIinterrupt arrives when a message is being transmitted viathe send() function of the AMSend interface the sharedbuffer sendBuf used by Oscilloscope to store incomingmessages will be updated and a corruptedmessage deliveredThe extract of code from the function Send presentedin Algorithm 5 shows that the variable msg which inthis case is a pointer to the shared variable sendBuf inOscilloscope can be updated at any time if the functionsend() is preempted because operations are not performedin a critical section This data race is somehow similar to theone we described in the example program of Algorithm 1Wealso find in Oscilloscope some of the data races reportedin the other applicationsThe results in Table 3 show that ourtool precisely detects data races on-the-fly in interrupt-drivenprograms without any false positives

64 Detection Overhead In order to evaluate the impactof our technique on the monitored programs we config-ured Avrora to simulate a single node sensor network Wethen measured the average runtime overhead and memoryspace required by each application of the benchmark ofTable 2 to execute 73728000 instruction cycles The totaloverhead of our technique mainly comes from actions of theinstrumentation engine the labeling routines and the race

Exec

utio

ns ti

me (

s)

14

12

10

8

6

4

2

0

TinyOS 210 programsOriginalWith iRace

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 4 Average runtime overhead

detection engine which also has the responsibility to createandmaintain the access historiesThe average execution timeand memory consumption collected for each application arepresented in Table 3 The columns ldquoOriginalrdquo and ldquoiRacerdquorepresent the execution time and memory consumption foreach application without and with iRace respectively

Figure 4 which plots the data from the column ldquoTimerdquoof Table 3 compares the average execution times of each

12 International Journal of Distributed Sensor Networks

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

TinyOS 210 programsOriginalWith iRace

Figure 5 Average memory consumption

application for five runs with and without iRace The captionldquoOriginalrdquo represents the native execution of the applica-tions without iRace The results of Figure 4 suggest thatthe average runtime overhead of our tool is only about1x for all applications except for RadioCountToLeds andRadioSenseToLeds that reaches 6x the original executiontime These two applications sample and display data at highfrequency rate (eg 4Hz in our case) and therefore arelikely to receive a higher number of interrupts at runtimeThis low runtime overhead is achieved by exploiting the factthat shared variables are stored by Avrora at program loadingtime Instead of monitoring all memory accesses to filterthose targeting shared variables iRace precisely instrumentsonly accesses to the well-known set of shared memory loca-tions obtained from the Avrora loader This technique allowsthe tool to drastically cuts down its monitoring overhead andto increase its scalability

Likewise wemeasured the averagememory consumptionfor each application during five runs The collected data arerepresented in Figure 5 The overall average memory con-sumption of the tool is very small at just about 20 above theoriginal memory requirement even for high frequency appli-cations like RadioCountToLeds and RadioSenseToLedsthanks to our labeling scheme that reuses the same labelfor different instances of the same interrupt handler and thedetection protocol that maintains only a reduced size accesshistoryThis result corroborates the theoretical analysis of ourtechnique as summarized in Table 1 The low runtime over-head and memory consumption of our on-the-fly techniquemake it very scalable and suitable even for interrupt-drivenprograms that handle interrupts at high frequency rates

65 Labeling Overhead Since the race detection subcompo-nent of iRace is in charge of implementing the labeling and

Table 4 Average labeling overhead

Applications Time (Sec) Memory (Kb)6LowPanCli 313 2101760AnthiTheft 308 2148880BaseStation 309 2077680Blink 174 1964640MultihopOscilloscope 313 2141920Oscilloscope 249 2092640Powerup 176 1894720RadioCountToLeds 218 2167440RadioSenseToLeds 224 2140080Sense 184 2011040

Table 5The states of an access history for detecting data races withour protocol considering the example of Figure 2

119886119894

Read Write Races11988211

⟨0 20 0⟩ ⟨20⟩

11987712

⟨0 20 0⟩ ⟨20⟩

11988213

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11988213-11988211

11988224

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩11987715

⟨1 1 0⟩ ⟨mdash⟩ ⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11987715-11988211

the detection routines it was legitimate to analyze the impactof the labeling scheme alone on the monitored programsat runtime For this purpose we deactivated the detectionengine of iRace before running it again on the benchmarkapplications We collected the average execution time andmemory space consumed by each application after five runsas shown in Table 4 The simulation was done using a singlenode network

Figure 6 compares the original execution time of eachprogram from Table 3 with the execution time of the labelingengine as presented in the column ldquoTimerdquo of Table 4 Thecomparison shows that the average runtime overhead ofiRace with just the labeling engine activated is only about04x in the worst-case This represents only a small fractionof the average runtime overhead of 6x imposed by iRace asdiscussed in Section 64Themain reason of this performancecan be found at source level When browsing the source codeof the benchmark applications we came to understand thatprogrammers generally disable and enable all the interruptsat once instead of focusing on particular interrupts In thiscase the size of each label generated by the labeling enginenever approaches its worst-case of 119874(119879) meaning that thecomparison overhead of labels is also reduced This is veryimportant because as discussed in Section 64 althoughthe size of labels does not depend on the total number ofinterrupts received at runtime this number still has impacton the overall detection overheadThus we can conclude thatthe labeling engine of iRace incurs a very low overhead atruntime making it a highly scalable tool

We also compared the memory consumption of thelabeling engine with the memory requirement of the original

International Journal of Distributed Sensor Networks 13

TinyOS 210 programs

4

35

3

25

2

15

1

05

0

Exec

utio

n tim

e (s)

With iRace labeling engineOriginal

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 6 Average runtime overhead of the labeling engine

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

With iRace labeling engineOriginal

TinyOS 210 programs

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 7 Average memory consumption by the labeling engine

program execution presented inTable 3The results plotted inFigure 7 show that when the labeling engine alone is runningiRace consumes only about 12 more memory on averageHowever during the detection phase the overall averagememory consumption of iRace is only about 20 Theseresults also tell us that the detection engine does not increasethat much the memory consumption of iRace This is alsoreasonable because we only disabled the detection engine ofiRace but did not remove the resources it used

7 Conclusions

Asynchronous interrupt handling introduces fine-grainedconcurrency into interrupt-driven programs making themprone to data races In this paper we presented an efficientand scalable on-the-fly technique to precisely detect dataraces without false positives in such programs Our techniqueintroduced a lightweight labeling scheme for maintainingconcurrency information for the main program and each ofits interrupt handlers and a precise detection protocol thatguarantees to verify the existence of data races in a programby reporting at least one data race for each shared variableinvolved in a data race

We implemented a prototype of our technique as a racedetection tool called iRace on top of the Avrora simulationframework An evaluation of iRace revealed the existence ofdata races in some programs distributed with TinyOS andin some TinyOS components while incurring a slowdownof only about 6 times on average in the worst-case with anaverage memory consumption of only about 20 above theoriginal memory requirements of the program Moreoverthis evaluation also proved that the labeling engine alonegenerates an average runtime overhead of only about 04xwith an average memory consumption of only about 12relatively to the original program execution These resultscoupled with a theoretical analysis showed that the proposedtechnique is highly scalable and practical even for largeinterrupt-driven programs

Detecting data races that occur first in a program isan important issue since it may make other data races todisappear It is therefore interesting to detect this class of dataraces in interrupt-driven programs to facilitate the debuggingprocess Also we may be interested in extending our work tosupport multithreaded programs that handle interrupts andalso to detect data races in low-level kernel codes

Acknowledgment

This research was financially supported by the Ministryof Education (MOE) and National Research Foundation ofKorea (NRF) through the Human Resource Training Projectfor Regional Innovation and BK21+ program

References

[1] U Banerjee B Bliss Z Ma and P Petersen ldquoA theory of datarace detectionrdquo in Proceedings of the Workshop on Parallel andDistributed Systems Testing and Debugging (PADTAD rsquo06) pp69ndash78 July 2006

[2] R H B Netzer and B P Miller ldquoWhat are race conditionsSome issues and formalizationsrdquo ACM letters on programminglanguages and systems vol 1 no 1 pp 74ndash88 1992

[3] N G Leveson and C S Turner ldquoInvestigation of theTherac-25accidentsrdquo Computer vol 26 no 7 pp 18ndash41 1993

[4] USA Canada Final Report on the August 14 2003 Blackout inthe United States and Canada Causes and RecommendationsUS-Canada Power System Outage Task Force 2004

[5] J Erickson M Musuvathi S Burckhardt and K OlynykldquoEffective data-race detection for the kernelrdquo in Proceedings of

14 International Journal of Distributed Sensor Networks

the 9th Conference on Operating Systems Design and Implemen-tation (USENIX rsquo10) 2010

[6] B-K Lee M-H Kang K C Park J S Yi S W Yang andY-K Jun ldquoProgram conversion for detecting data races inconcurrent interrupt handlersrdquo Communications in Computerand Information Science vol 257 pp 407ndash415 2011

[7] L Effinger-Dean B Lucia L Ceze D Grossman and H JBoehm ldquoIFRit interference-free regions for dynamic data-racedetectionrdquo in Proceedings of the ACM International Conferenceon Object Oriented Programming Systems Languages and Appli-cations (OOPSLA rsquo12) vol 47 pp 467ndash484 2012

[8] C Flanagan and S N Freund ldquoFastTrack efficient and precisedynamic race detectionrdquo in Proceedings of the ACM Interna-tional Conference on Object Oriented Programming SystemsLanguages and Applications (OOPSLA rsquo09) vol 44 pp 121ndash1332009

[9] X Xie and J Xue ldquoAcculock accurate and efficient detection ofdata racesrdquo in Proceedings of the 9th International Symposium onCode Generation and Optimization (CGO rsquo11) pp 201ndash212 April2011

[10] TinyOSCommunity ForumTinyOSAnopen source operatingsystem for low-power wireless devices 2008 httpwwwtinyosnet

[11] M Higashi T Yamamoto Y Hayase T Ishio and K InoueldquoAn effective method to control interrupt handler for datarace detectionrdquo in International Workshop on Automation ofSoftware Test (ACM rsquo10) pp 79ndash86 May 2010

[12] Z Lai S C Cheung and W K Chan ldquoInter-context control-flow and data-flow test adequacy criteria for nesC applicationsrdquoin Proceedings of the 16th ACM SIGSOFT International Sympo-sium on the Foundations of Software Engineering (SIGSOFT rsquo08)pp 94ndash104 November 2008

[13] J Regehr ldquoRandom testing of interrupt-driven softwarerdquo inPro-ceedings of the 5th ACM International Conference on EmbeddedSoftware (EMSOFT rsquo05) pp 290ndash298 September 2005

[14] Y Zhou X Chen M R Lyu and J Liu ldquoSentomist Unveil-ing transient sensor network bugs via symptom miningrdquoin Proceedings of the 30th IEEE International Conference onDistributed Computing Systems (ICDCS rsquo10) pp 784ndash794 June2010

[15] G Gracioli and S Fischmeister ldquoTracing interrupts in embed-ded softwarerdquo in Proceedings of the ACM SIGPLANSIGBEDConference on Languages Compilers and Tools for EmbeddedSystems (LCTES rsquo09) vol 44 pp 137ndash146 2009

[16] V Sundaram P Eugster and X Zhang ldquoLightweight tracingfor wireless sensor networks debuggingrdquo in Proceedings of the4th International Workshop on Middleware Tools Services andRun-Time Support for Sensor Networks (ACM rsquo09) pp 13ndash18December 2009

[17] O K Ha and Y K Jun ldquoMonitoring of programs with nestedparallelismusing efficient thread labelingrdquo International JournalFo U- and E- Service Science and Technology vol 4 pp 21ndash362011

[18] Y C Kim S S Jun and Y K Jun ldquoA tool for space-efficient on-the-fly race detectionrdquo International Journal of DatabaseTheoryand Application vol 4 pp 25ndash38 2011

[19] S Savage M Burrows G Nelson P Sobalvarro and T Ander-son ldquoEraser a dynamic data race detector for multithreadedprogramsrdquo ACM Transactions on Computer Systems vol 15 no4 pp 391ndash411 1997

[20] A Jannesari and W F Tichy ldquoOn-the-fly race detection inmulti-threaded programsrdquo in Proceedings of the 6th Workshop

on Parallel and Distributed Systems Testing Analysis andDebugging (PADTAD rsquo08) July 2007

[21] D Gay M Welsh P Levis E Brewer R Von Behren and DCuller ldquoThe nesC language a holistic approach to networkedembedded systemsrdquo inACMSIGPLANConference on Program-ming Language Design and Implementation pp 1ndash11 June 2003

[22] J Regehr and N Cooprider ldquoInterrupt verification via threadverificationrdquo Electronic Notes in Theoretical Computer Sciencevol 174 no 9 pp 139ndash150 2007

[23] B L Titzer D K Lee and J Palsberg ldquoAvrora scalable sensornetwork simulation with precise timingrdquo in Proceedings ofthe 4th International Symposium on Information Processing inSensor Networks (IPSN rsquo05) pp 477ndash482 April 2005

[24] B L Titzer and J Palsberg ldquoNonintrusive precision instru-mentation of microcontroller softwarerdquo in Proceedings of theConference on Languages Compilers and Tools for EmbeddedSystems (Lctes rsquo05) vol 40 pp 59ndash68 2005

[25] J Hil R Szewczyk A Woo S Hollar D Culler and K PisterldquoSystem architecture directions for networked sensorsrdquo inProceedings of the 9th International Conference on ArchitecturalSupport for Programming Languages and Operating Systems(ASPLOS rsquo00) pp 93ndash104 2000

[26] GMTchamgoue KHKim andYK Jun ldquoTesting anddebug-ging concurrency bugs in event-driven programsrdquo InternationalJournal of Advanced Science and Technology vol 40 pp 55ndash682012

[27] GM Tchamgoue O K Ha K H Kim and Y K Jun ldquoA frame-work for on-the-fly race healing in ARINC-653 applicationsrdquoInternational Journal of Hybrid Information Technology vol 4pp 1ndash12 2011

[28] A Dinning and E Schonberg ldquoDetecting access anomaliesin programs with critical sectionsrdquo in Proceedings of theACMONR Workshop on Parallel and Distributed Debugging(PADD rsquo91) vol 26 pp 85ndash96 1991

[29] L Lamport ldquoTime clocks and the ordering of events in adistributed systemrdquo Communications of the ACM vol 21 no7 pp 558ndash565 1978

[30] Atmel inc ATmega128 Datasheet 2011 httpwwwatmelcomImagesdoc2467pdf

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 4: Research Article Verification of Data Races in Concurrent ...downloads.hindawi.com/journals/ijdsn/2013/953593.pdf · photoTask I20 W 11 R 12 W 13 W 24 R 15 T 2 J12 T 1 F : A POEG

4 International Journal of Distributed Sensor Networks

ADC20

20

20

20

12

12

Task posting edgeTask ordering edge

lowast

lowast

PhotoSensor

dataTask

photoTask

I20W11

R12

W13

W24

R15

T2

J12

T1

Figure 1 A POEG of an execution instance of the program inAlgorithm 1 Only accesses to msg are represented

In a POEG two accesses are concurrent if and onlyif there is no path between them Otherwise they areordered by the happens-before relation An example POEGfor the program in Algorithm 1 considering only accesses tothe shared variable msg is represented in Figure 1 In thisPOEG vertical lines symbolize threads while read and writeaccesses to shared variables are respectively represented bythe letters 119877

119894119895and119882

119894119895 where 119894 is the thread instance number

and 119895 an observed execution order We use the couple ofsymbols⊓119899 and⊔119899 to respectively represent the disabling andenabling of interrupts where 119899 is the corresponding interruptnumber that will be replaced with the star symbol (lowast) ifall interrupts are concerned The POEG of Figure 1 depictsan execution where a first instance of the ADC (Anolog-to-Digital Converter) interrupt handler collected new dataand posted an instance of dataTask Furthermore anotherinstance of the ADC interrupt arrives and preempts theexecution of dataTask

Although the first instance of the ADC interrupt handlerposted dataTask if the second ADC interrupt is receivedright before the message is sent at line 23 of Algorithm 1 dataraces will still occur between the write access in ADC andthe accesses 119882

13and 119877

15in dataTask making a corrupted

message to be delivered Those are the data races representedin Figure 1 by the pair of accesses 119882

13-11988224

and 11987715-11988224 If

a task is posted by an interrupt handler the task postingedge guarantees a happens-before relation only between theposted task and the actual instance of the interrupt handlerthat posted it but not with future instances of the sameinterrupt handler Consequently we ignore the task postingedge when traversing the graph for analyzing the happens-before relation because we use a single arrow to represent

all instances of the same interrupt Thus we can also reportthe data races 119882

13-11988211

and 11987715-11988211

between 1198791and 119868

20

Therefore in our POEG we assume that two thread segments119905119894and 119905119895 respectively from two different threads119879

119894and119879119895are

always concurrent unless they are ordered by a task orderingedge or one of them is disabled by the other one The nesClanguage [21] contains a static data race detector that howeveris still unsound since it cannot follow pointers [12 21 22]Thus the data races presented in Figure 1 cannot be reportedby the native nesC data race detector

3 Lightweight Labeling Scheme

For an effective dynamic race detection it is essentialto adequately determine the order of precedence betweenthread segments accessing shared memory locations Forthis purpose this section presents our lightweight labelingscheme used for capturing the logical concurrency betweenthread segments of interrupt-driven programs

31 GeneratingThread Labels Happens-before based on-the-fly race detection techniques generally rely on a variationof Lamportrsquos logical timestamp [29] in order to preciselyidentify concurrent thread segments of a program Theeffectiveness of on-the-fly techniques is highly dependenton the cost and accuracy of this identification process Ourlabeling scheme generates a unique identifier called label foreach thread segment 119905

119894during an execution instance of an

interrupt-driven program For the labeling scheme we needto introduce the following definitions

Definition 3 (critical section) A critical section with respectto a given interrupt 119896 is a portion of program comprisingbetween a disable and probably an enable instruction tointerrupt 119896

Definition 4 (interrupt set) The interrupt set or 119894119904119890119905 of athread segment 119905

119894represents the set of all interrupts 119896disabled

by 119905119894Thus this guarantees a happens-before relation between

119905119894and every interrupt in its 119894119904119890119905 Formally for all 119896 isin 119905

119894sdot 119894119904119890119905

for all 119905119895isin 119879119896 119905119894

ℎ119887

997888997888rarr 119905119895or 119905119895

ℎ119887

997888997888rarr 119905119894

Definition 5 (critical section violation) A thread segment 119905119894

of a thread 119879119896violates the critical section of another thread

segment 119905119895 denoted by 119905

119894

V999492999484 119905119895 when 119905

119895can be preempted by

119905119894

The label of each thread segment 119905119894consists of two compo-

nents an identifier referred to as segment descriptor 119878119863119894 and

an interrupt set 119894119904119890119905119894 Put together we use 119878119863

119894⟨119894119904119890119905119894⟩ to desig-

nate the label of a thread segment 119905119894 The segment descriptor

119878119863119894of a thread segment 119905

119894is defined as a triplet ⟨119905119905

119894 119905119894119889119894 119888119904119894⟩

where 119905119905119894specifies whether 119905119894119889

119894is a task or an interrupt

handler 119905119894119889119894contains the task identifier of a task or the

interrupt number of an interrupt and 119888119904119894indicates whether

the thread segment has disabled all interrupts in the systemCreating labels for thread segments of an interrupt-driven

program is straightforward with our labeling scheme as

International Journal of Distributed Sensor Networks 5

suggested by the algorithms in Algorithm 2 A new labeledthread 119879

119894is created when a new task starts its execution or

whenever an interrupt handler is invoked When a new taskstarts a new label is generated for the first thread segment ofthe task by simply initializing the component of the label asin Algorithm 2(a) A new label is similarly created when aninterrupt handler is invoked in Algorithm 2(b) Each threadthen creates additional thread segments whenever it disablesor enables interrupts

When a thread segment 119905119894of a thread 119879

119894disables an

interrupt 119896 the label of the new thread segment 119905119895is generated

by adding the interrupt number 119896 into the interrupt set ofthe current label owned by 119905

119894 However if all interrupts in

the system are disabled by 119905119894 the new label for 119905

119895is obtained

by setting 119888119904119894and by emptying 119894119904119890119905

119894from the current label

owned by 119905119894as shown in Algorithm 2(c) Thus as seen in

Algorithm 2(d) if 119905119894enables an interrupt 119896 a new label for a

new thread segment 119905119895is obtained by simply removing 119896 from

the interrupt set of the current label in 119905119894 If all interrupts are

enabled at the same time the flag 119888119904119894and the interrupt set

119894119904119890119905119894are both reset from the current label of 119905

119894to generate

the new label for the new thread segment 119905119895 This helps

maintain a reduced-size interrupt set for each thread segmentsince programmers generally disable and enable all interruptsrather than manipulating individual interrupts

Moreover the size of an interrupt set is bounded by thetotal number of interrupts supported by the hardware onwhich the program runs which is generally a reasonableinteger number By reusing the same label for differentinstances of the same interrupt handler our labeling schemedoes not depend on the total number of interrupts receivedat runtime which can be very high

Lemma 6 Given two labeled thread segments 119905119894and 119905119895 with

119905119894belonging to an interrupt handler 119879

119896 119905119894

V999492999484 119905119895rArr 119896 notin 119894119904119890119905

119895or

119888119904119895= 119891119886119897119904119890

Proof Let us consider two labeled thread segments 119905119894and 119905119895

and let us assume that 119905119894belongs to an interrupt handler 119879

119896

We also assume that 119905119894violates the critical section of 119905

119895 By

definition thread segments are delimited by either the disableor enable instructions Therefore if 119905

119895was created after a

the disable instruction targeting interrupt 119896 the interruptnumber 119896 will be in the interrupt set 119894119904119890119905

119895of 119905119895according

to the labeling algorithms of Algorithm 2 However if 119905119895was

created after the disabling of all interrupts 119888119904119895would then be

true by the labeling algorithms of Algorithm 2 to signify thatall interrupts including 119896 are disabledThus for all 119905

119894isin 119879119896 119905119894

cannot violate 119905119895 which is contradictory since we assumed

119905119894

V999492999484 119905119895 Therefore 119905

119895has necessarily enabled 119896 that is

for all 119905119894isin 119879119896 119905119894

V999492999484 119905119895rArr 119896 notin 119894119904119890119905

119895or 119888119904119895= 119891119886119897119904119890

32 Concurrency Relation Using our labeling scheme itbecomes easy to analyze the happens-before relation betweenany pair of thread segments 119905

119894and 119905119895 by comparing their

labels

[a Task Starting]a1 tsk start()a2 Label Lba3 Lbt = truea4 Lbtid = getTaskid()a5 Lbcs = falsea6 Lbiset =

a7 return Lb[b Interrupt Invocation]b1 int invoke()b2 Label Lbb3 Lbt = falseb4 Lbtid = getIntNumber()b5 Lbcs = falseb6 Lbiset =

b7 return Lb[c Interrupt Disabling]c1 int disable(Label Lb Interrupt n)c2 if(n gt 0) Lbiset cup 119899

c3 elsec4 Lbcs = truec5 Lbiset =

c6 endifc7 return Lb[d Interrupt Enabling]d1 int enable(Label Lb Interrupt n)d2 if(n gt 0) Lbiset-nd3 elsed4 Lbcs = falsed5 Lbiset =

d6 endifd7 return Lb

Algorithm 2 Labeling algorithms

Theorem7 A labeled thread segment 119905119894happens before anoth-

er labeled thread segment 119905119895 denoted by 119905

119894

ℎ119887

997888997888rarr 119905119895 if and only if

either that 119905119894and 119905119895belong to tasks or that 119905

119895does not violate

119905119894 that is not(119905

119895

V999492999484 119905119894)

Proof Let us assume two labeled thread segments 119905119894and

119905119895 such that 119905

119894

ℎ119887

997888997888rarr 119905119895 Thus according to the labeling

algorithms of Algorithm 2 we should have 119905119905119894= 119905119905119895= 119905119903119906119890

meaning that 119905119894and 119905

119895are either from the same or from

different tasks By definition tasks are all ordered and cannotpreempt each other However if 119905

119895is a thread segment of

an interrupt handler 119879119896 according to Definition 4 either

119888119904119894= 119905119903119906119890 because 119905

119894disabled all interrupts or the thread

identifier 119905119894119889119895of 119905119895that corresponds to 119896 the interrupt

number of 119879119896 is in 119894119904119890119905

119894 the interrupt set of 119905

119894 This means

that 119905119894is a critical section thread segment that disables either

all interrupts or particularly the interrupt number 119896 Thusaccording to Lemma 6 119905

119895does not violate the critical section

of 119905119894 Therefore we can say that 119905

119894

ℎ119887

997888997888rarr 119905119895implies either that 119905

119894

and 119905119895belong to tasks or that not(119905

119895

V999492999484 119905119894)

6 International Journal of Distributed Sensor Networks

Now let us assume that 119905119894and 119905

119895belong to tasks

Naturally 119905119894and 119905119895are ordered and the labeling algorithms of

Algorithm 2 will set 119905119905119894= 119905119905119895= 119905119903119906119890 meaning that 119905

119894

ℎ119887

997888997888rarr 119905119895

On the other hand if we assume that not(119905119895

V999492999484 119905119894) then 119905

119895

must be part of a given interrupt handler 119879119896and either 119905

119894has

disabled all interrupts to set 119888119904119894= 119905119903119906119890 or the thread identifier

119905119894119889119895= 119896 of 119905

119895is in the interrupt set 119894119904119890119905

119894of 119905119894 according to

Definition 4 and Lemma 6 Therefore we can conclude that119905119894

ℎ119887

997888997888rarr 119905119895

Any two labeled thread segments 119905119894and 119905119895are therefore

concurrent if they do not satisfy the happens-before relationas stated by Theorem 7 which can formally be rewritten asfollows

119905119894

ℎ119887

997888997888rarr 119905119895lArrrArr

119905119905119894= 119905119905119895= 119905119903119906119890

or(119905119905119895= 119891119886119897119904119890) and (119888119904

119894= 119905119903119906119890 or 119905119894119889

119895isin 119894119904119890119905119894)

(1)

By comparing based on Theorem 7 the labels of thethread segments in the POEG represented by Figure 2 wecan easily obtain that the thread segments of 119868

20 1198941 and 119894

2

are concurrent with other thread segments 1199051 1199053 1199054 and

1198951 Therefore the write accesses in 119894

1and 1198942conflict with

the other accesses in 1199053 leading to four data races 119882

11-11988213

11988211-1198771511988224-11988213 and119882

24-11987715 However since the accesses

11988211and119882

23 respectively from 119894

1and 1198942 represent the same

write access the set of data races to be reported for an effectivedebugging process should be reduced to only 119882

11-11988213

and11988211-11987715 By disabling all interrupts 119905

2enforces the happens-

before relation with other thread segments Although 1198791is

posted by 11986820 they are still concurrent Also the thread

segment 1198951is concurrent with 119894

1 1198942 1199051 1199053 and 119905

4

4 Precise Detection Protocol

The race detection protocol monitors memory accesses bymaintaining a special data structure called access history intowhich are stored the labels of prior accesses to each sharedvariable Our detection protocol stores in each entry of theaccess history of a shared variable a unique label representingthe thread segment that accessed the variable as describedin Section 3 At each access to a shared variable data racesare reported by analyzing based onTheorem 7 the happens-before relation between the current thread segment and thelabels of prior thread segments stored in the access historyTherefore the access history for each shared variable hasto be efficiently organized in order to reduce the overheadgenerated by the detection process

Our precise detection protocol maintains an access his-tory consisting of two sets for each shared variable a Readset and a Write set to respectively store labels of threadsegments that read and write the variable We use 119860119867(119909)

to designate the access history of a shared variable 119909 Thus119860119867(119909) sdot 119877119890119886119889 and 119860119867(119909) sdot 119882119903119894119905119890 refer to the read set andthe write set of 119860119867(119909) respectively The algorithms used bythe detection protocol are presented in Algorithm 3 In thesealgorithms we assume that the variable 119871119887 always contains

20

20

20

20

t1

t2

t3

t4

12

12

lowast

lowast

i1⟨0 20 0⟩⟨20⟩

⟨mdash⟩⟨1 1 1⟩

⟨mdash⟩⟨1 1 0⟩

⟨mdash⟩⟨1 1 0⟩

⟨0 12 0⟩⟨12⟩

⟨0 20 0⟩⟨20⟩

⟨mdash⟩⟨1 2 0⟩

I20

W11

R12

W13

W24

R15

T2

J12

T1

j1

i2

Figure 2 An example labeled POEG for an execution instance ofthe program in Algorithm 1

the label of the current thread segment accessing a sharedvariable and that 119901119871119887 represents any prior label stored in theaccess history

When a read access occurs to a shared variable 119909the detection protocol calls the ah read() function ofAlgorithm 3(a) that then reports data races by analyzing thehappens-before relation between the label of the accessingthread segment and the labels of prior accesses stored in119860119867(119909) sdot119882119903119894119905119890 Similarly upon a write access to a shared vari-able the monitoring function ah write() of Algorithm 3(b)reports data races by looking into both 119860119867(119909) sdot 119877119890119886119889 and119860119867(119909) sdot 119882119903119894119905119890 We assume the existence of a functionOrdered() that based on Theorem 7 decides whether athread segment 119905

119894happens before another thread segment 119905

119895

by comparing their labelsIn order to maintain a reduced size access history

accesses from critical sections that concern all interruptswithin a task are not inserted in the access history sincethey will never run into a data race with other accessesfrom interrupt handlers Also only the first read and writeaccesses to a shared variable from the same thread segmentare kept into the access history Subsequent similar accessesto the same shared variable from the same thread segmentare filtered outThis filtering operations are performed by theisFiltered() function of Algorithm 3(c) that is called by bothah read() and ah write() before any other action

After data races have been reported with the currentaccess the function maintain ah() of Algorithm 3(d) is usedfor keeping the size of the access history balanced Thisfunction replaces some previous accesses in the access historywith the current access Thus accesses from different tasksare considered from the same thread since tasks by defini-tion are all ordered by happens-before Similarly accesses

International Journal of Distributed Sensor Networks 7

(1) ah read(Label Lb Access History AH Variable x)(2) if (isFiltered(Lb AH(x)Read)) then return(3) for all labels pLb in AH(x)Write do(4) if (Not Ordered(Lb pLb)) then(5) report race(x pLb Lb)(6) end for(7) maintain ah(Lb AH(x)Read)

(a) Monitoring Protocol for Read Accesses(1) ah write(Label Lb Access History AHVariable x)(2) if (isFiltered(Lb AH(x)Write)) then return(3) for all labels pLb in(4) AH(x)Write cup AH(x)Read do(5) if (Not Ordered(Lb pLb)) then(6) report race(x pLb Lb)(7) end for(8) maintain ah(Lb AH(x)Write)

(b) Monitoring Protocol for Write Accesses(1) isFiltered(Label Lb Access History AH)(2) if (Lbcs = true) and (Lbtt = true)(3) then return true(4) if (any access with a label pLb exists in AH(5) such that pLb = Lb) then return true(6) else(7) return false

(c) Filtering Redundant Accesses(1) maintain ah(Label Lb Access History AH)(2) if (Lbtt = true) then(3) delete all entries in AH with label pLb(4) such that pLbtt = true(5) else(6) delete all entries in AH with label pLb such(7) that pLbtt = Lbtt and pLbtid = Lbtid(8) end if(9) add ah(Lb AH)

(d) Maintaining Access History

Algorithm 3 Labeling algorithms

from different instances of the same interrupt handler areconsidered from a single instance of the interrupt handlerThis property is already enforced by the labeling scheme thatreuses the same label for all instances of the same interrupthandler Again we assume the existence of another functionadd ah() which is able to insert the label of the current threadsegment into the appropriate set of the access history given ashared variable

Table 5 shows the different states of an access historyfor detecting data races with our protocol in the exampleprogram of Figure 2 where accesses to the shared variablemsg appear in the first column When the first access 119882

11

occurs the label ⟨0 20 0⟩⟨20⟩ of the thread segment 1198941

is simply added into the access history and no data racereported as there is no other access prior to 119882

11 The next

access 11987712 from the thread segment 119905

2is filtered out because

1199052that belongs to task 119879

1has disabled all interrupts The

following access 11988213

occurs on 1199053with interrupts enabled

Thus a data race11988213-11988211is reported by comparing the label

of 1199053 ⟨1 1 0⟩⟨minus⟩ with the label of 119894

1previously stored into the

access history Afterwards the label of 1199053is inserted into the

write set of the access history The thread segment 1199053is then

preempted by 1198942 but the access 119882

24is filtered out because

1198942which is a new instance of the interrupt 119868

20 also shares the

same label ⟨0 20 0⟩⟨20⟩ with 1198941which previously performed

a write and whose label is already in the access history Afterthe execution of 119894

2 1199053resumes to perform the read access

11987715 Another data race 119877

15-11988211

is reported and the label of1199053 ⟨1 1 0⟩⟨minus⟩ stored into the read set of the access history

Although the program received two instances of the ADCinterrupt our protocol precisely reported only the set of twodata races needed for effectively debug the program Thisshows that our detection protocol does not depend on thenumber of interrupts received at runtime which can be veryhigh

Although it keeps only a reduced set of labels in the accesshistory our precise detection protocol even with a minimalnumber of interrupts still guarantees to verify the existenceof data races in a program For this purpose the protocol isdesigned to report without false positives at least one datarace for each shared variable involved in a data race

8 International Journal of Distributed Sensor Networks

Theorem 8 Given an interrupt-driven program 119875 the detec-tion protocol reports a data race over a shared variable 119909 if andonly if there exists a data race over 119909 in an execution instanceof 119875

Proof We will first prove that if a data race is reported on ashared variable 119909 by the detection protocol then there alsoexists a data race in an execution instance of the program forthe same shared variable 119909 For this we assume a snapshotof the access history 119860119867(119909) of the shared variable 119909 at agiven time and the current access event 119886

119894to 119909 from a thread

segment 119905119894that owns a label we refer to as 119871119887

119894in an execution

instance of the program 119875 We can then distinguish twodifferent cases

Case A 119860119867(119909) is empty and there is obviously no data racefor 119909 In this case the current access 119886

119894is the only access to

119909 from the program 119875 and 119871119887119894will be added by the function

add ah() into119860119867(119909)sdot119877119890119886119889 if 119886119894is a read access or into119860119867(119909)sdot

119882119903119894119905119890 if it is a write access

Case B 119860119867(119909) is not empty Data races are reported usingthe ah read() function if 119886

119894is a read access or the ah write()

in case of a write access In both cases the happens-beforerelation is analyzed between the label 119871119887

119894of 119905119894and previous

labels stored in 119860119867(119909) according to Theorem 7 The outputsof these analyses can be classified into two categories

(1) There is no data race between 119886119894and119860119867(119909) therefore

all previous accesses stored in 119860119867(119909) happenedbefore 119886

119894meaning that no thread segment prior to

119905119894in 119875 is concurrent with 119905

119894 they are all ordered by

happens-before Thus for all 119886119895isin 119905119895and 119871119887119895isin 119860119867(119909)

119905119895

ℎ119887

997888997888rarr 119905119894 In which case 119875 also has no data race

(2) There are data races between 119886119894and prior accesses

in 119860119867(119909) this means that there are some labels in119860119867(119909) belonging to some prior thread segments thatare not ordered with 119905

119894 Concretely there are thread

segments in 119875 that are concurrent with 119905119894 Formally

exist119905119895isin 119875119871119887

119895isin 119860119867(119909) 119905

119894 119905119895Therefore the execution

instance of 119875 also contains data races over 119909

Now we show that if there exists a data race over a sharedvariable 119909 in an execution instance of the program 119875 thenour detection protocol also reports a data race over 119909 Usingthe contrapositive we will prove that if the detection protocoldoes not report any data race the program 119875 also does notcontain any for a shared variable 119909 We again consider asnapshot of the access history 119860119867(119909) for a shared variable119909 at a given time We also assume that the detection protocolreported no data races for the shared variable 119909 This meansthat 119860119867(119909) is either empty and there is clearly no data racesince there is no access to 119909 from 119875 yet or contains onlylabels of thread segments that are all ordered by the happens-before relation in which case there is no concurrent threadsegment in119875 which also implies that119875 has no data races over119909 Therefore we can formally write for all 119905

119894 119905119895isin 119875 and 119871119887

119894

119871119887119895isin 119860119867(119909) 119905

119894

ℎ119887

997888997888rarr 119905119895or 119905119895

ℎ119887

997888997888rarr 119905119894 Thus 119875 also contains no

data race for 119909

Table 1 Efficiency of the precise detection protocol

Space TimeNew label comparison Per access

O(1198811198792) O(1) O(LogT) O(TLogT)

5 Theoretical Analysis

On-the-fly data race detection techniques are known to havehigh runtime overhead that is generally provoked by the timespentmonitoring accesses to shared variables and to comparethread labels In this section we theoretically analyze theefficiency of our technique which depends on119881 the numberof shared variables and 119879 the total number of interruptsin the underlying system plus one the number of tasksIn general 119879 is very small since the number of interruptshandled by a system is also kept small For example thepopular Atmel ATmega128 [30] processor supports only amaximum of 34 interrupts Also because of the implicithappens-before relation that exists between all tasks they areviewed as a single task by the detection protocol

Table 1 presents the space and time complexities of ourlabeling and detection techniques The access history usedby our technique stores from each thread segment at mosttwo accesses for each shared variable Each label in the accesshistory has a worst-case size of 119874(119879) that represents thesize of its interrupt set Therefore the total labeling spacerequirement for each shared variable is 119874(1198792) Thus thetotal space complexity of our technique is 119874(1198811198792) Howeverbecause the number of interrupts is generally a reasonableinteger number the space complexity of our technique canfurther be reduced to 119874(119881)

A new label is always created in a constant amount of timeby our labeling algorithms Interrupts are generally givennumbers in a sequence order Therefore a binary searchtechnique is applied to locate values in interrupt sets whichcontain only interrupt numbers Thus the time complexityfor comparing any two labels is 119874(Log119879) in the worst-caseand 119874(1) in most cases where the interrupt sets are emptyAt each access to a shared variable the total time complexityto report data races with the previous accesses stored in theaccess history is then119874(119879Log119879)The labels and the detectionprotocol do not depend on the total number of interruptsreceived at runtimemaking our technique highly scalable andpractical

6 Experimental Results

We empirically evaluated the preciseness and efficiency ofour technique using a set of well-known programs providedwith TinyOS [10] This section details the implementationchallenges and presents the analysis results of the experimen-tation

61 Implementation We implemented a prototype of ourdata race detection tool for interrupt-driven programs callediRace based on the concurrency model of TinyOS [21] andits nesC language [25] We implemented iRace on top

International Journal of Distributed Sensor Networks 9

Table 2 Characteristics of the benchmark applications from TinyOS 210 [10]

Applications LOC Int SV Brief description6LowPanCli 19116 4 9 Implements a command line interface using the 6lowpan stackAnthiTheft 27421 5 4 Detects and reports a stolen mote by checking some sensorsBaseStation 14905 6 13 Sends and receives simple active messages on the radio and serial linksBlink 3688 1 0 Periodically blinks the three LEDs of a moteMultihopOscilloscope 24310 9 8 Periodically samples and broadcasts sensor data to a base stationOscilloscope 11686 5 5 Periodically samples sensor data and broadcasts a message every 10 timesPowerup 2352 0 0 Turns on the red LED when the mote powers upRadioCountToLeds 11165 4 3 Periodically samples sensors and displays the last 3 bits on its LEDsRadioSenseToLeds 11472 5 2 Periodically update a counter and displays the last 3 bits on its LEDsSense 5743 2 0 Periodically takes sensor readings and displays the values on the LEDs

of the Avrora [23] simulation and analysis framework thatprovides a time accurate simulator different analysis toolsknown asmonitors and a binary instrumentation engine [24]for programs written for AVR microcontrollers and sensornetworks All these features make Avrora just suitable forour tool Simulation is commonly used in the developmentcycle of embedded software because target systems generallylack conventional inputoutput interfaces With iRace thefunctionalities of Avrora are extendedwith a precise data racedetection tool that can easily be loaded as one of Avrorarsquosmonitors

As shown in Figure 3 iRace consists of two maincomponents the Instrumenter which exploits the Avrorainstrumentation engine [24] and the core Race Detectoritself The Instrumenter monitors task and interrupts activ-ities through its iMonitor component and provides entrypoints for the labeling routines Instrumentation points areinserted at the beginning of each task at the start andexit of each interrupt handler and also before interruptsare either enabled or disabled Similarly accesses to sharedmemory locations are monitored using the AccessMonitorcomponent that also calls routines of the detection proto-col The AccessMonitor instruments before read and writeaccesses to shared variables The Race Detector is composedof labeling routines that generate and manage labels for eachactive thread segment at runtime and detection routines formaintaining the access history and reporting data races on-the-fly as shared variables are manipulated

In TinyOS [21 25] interrupt handlers and tasks all runto completion unless preempted by other interrupt handlersThus only a single thread runs at any point of time givingus the opportunity to use only a single variable to maintainthe active thread label at detection time However whenpreemption happens the label of the preempted threadsegment must be saved and restored again when the activeinterrupt handler exits Otherwise there will be no way toknow about the concurrency information of the preemptedthread segment when it gains back the execution control Inorder tomanage the preemption we use a stack data structurewhere labels can be pushed and restored from since interruptscan be nested This provides a simple but powerful way tohandle preemption with negligible overhead

Our labeling scheme needs to uniquely identify eachinterrupt handler whenever it is invoked and each taskwhen it starts its execution Each interrupt handler is easilyidentified by the interrupt number of the interrupt it servesFor task we chose to generate a unique task identifier bysimply maintaining a counter that increases at each timea task starts its execution Thus each task is attributed aunique identifier thanks to the fact that once posted TinyOStasks run in a first-in first-out manner and cannot preempteach other By manipulating only binary files iRace doesnot require any source file nor source code annotation toeffectively detect data races However compiling the programwith debugging information may help iRace generate moredetailed debugging information on data races it reports

62 Environment We implemented and evaluated our tech-nique on a system that runs the kernel 30 of the Linuxoperating system using an Intel dual-core 320GHz and 4GBof mainmemory On this machine was installed TinyOS 210[10] along with the nesC compiler 131 the GNU C compiler46 the Java Runtime Environment 17 and the latest Avroraframework version 17115

Table 2 presents the characteristics and a brief descriptionof the benchmark applications selected for our evaluationfrom the TinyOS 210 repository [10] These applications arefreely distributed with each version of TinyOS The columnldquoLOCrdquo indicates the number of line of C codes generatedby the nesC compiler after compilation while the columnldquoIntrdquo contains the number of interrupts handled by eachapplicationThe number of shared variables for each programappears in column ldquoSVrdquo It is worthmentioning that this num-ber represents only the shared variables in the source code ofeach program The exact number of variables shared by eachapplicationmay finally depend on the number of componentsit shares with the operating system itself Applications writtenin nesC share TinyOS components by wiring themselvesto these components increasing therefore their number ofshared resources We compiled each application for theMica2 platform and measured the average runtime overheadover five executions The Mica2 motes are sensor networknodes based on the ATmega128 [30] microcontrollers Weconfigured Avrora to simulate a sensor network during 10

10 International Journal of Distributed Sensor Networks

iRace

and instrumentationframework

iMonitor

AccessMonitor

Race detector

Labeling routines

Detection protocolroutines

Binarycode Data

races

Avrora simulation

Figure 3 The Architecture of the Data Race Detection Tool

Table 3 Reported data races on the benchmark applications

Applications Data races Verif races Time (Sec) Memory (Kb)Original iRace Original iRace

6LowPanCli 0 0 269 357 1936960 2168160AnthiTheft 15 15 292 457 1944400 2320000BaseStation 0 0 279 376 1912240 2148240Blink 0 0 161 180 1887280 1949600MultihopOscilloscope 0 0 278 400 1939600 2196960Oscilloscope 13 13 179 328 1909360 2201760Powerup 0 0 151 186 1873600 1879920RadioCountToLeds 17 17 196 1255 1942880 2326320RadioSenseToLeds 15 15 181 1305 1944480 2298240Sense 0 0 158 192 1866400 2050560

seconds allowing each application to run exactly for 73728000instruction cycles We set to five the number of nodes in thesimulated sensor network

63 Evaluation For data race detectionwe instructedAvrorato simulate a sensor network with five nodes while thebenchmark applications are monitored by iRace Table 3represents the results of the evaluation of iRace on thebenchmark applications in its column ldquoData Racesrdquo whichcontains the number of shared variables involved in adata race for each application Our tool reported no datarace for all applications except for four AnthiTheft with15 shared variables Oscilloscope with 13 shared vari-ables RadioCountToLeds with 17 shared variables andRadioSenseToLeds with 15 shared variables Surprisinglymost of the reported data races reside inside some sharedsystem components of TinyOS itself The reported data racesare caused by two different interrupts namely the serialperipheral interface (SPI) interrupt and the timer interruptAlso we carefully analyzed the reported data races andinspected the source code of the applications for a manualverification process as shown in the column ldquoVerified racesrdquoof Table 3

In the AnthiTheft application for example which isused to detect and report a stolen mote by checking the levelof the light or the acceleration data races were reported over

15 shared variables Most of the reported data races are how-ever due to the fact that the SPI interrupt handler can preemptsome functions or tasks owned by the underlying radio stackThis implies that if a message is being received or sent whena SPI interrupt occurs shared variables like radioState orcount in the component CC1000SendReceiveP might becorrupted However all shared variables involved in a datarace do not directly belong to AnthiTheft but to functionsof the CC1000 radio stack in use in Mica2 motes Similardata races are also reported in RadioCountToLeds andRadioSenseToLeds since they all access the radio stack

Another data race reported both in RadioCountToLedsand RadioSenseToLeds and worthy to be commentedconcerns the shared variable reqResId inside thecomponent ArbiterP (Recently this data race hasalso been pointed out and a patch discussed in theTinyOS Forum) This data race occurs when the functionimmediateRequest() called within the function read ofthe component immediateRequest is preempted by a SPIinterrupt handler An excerpt of this function is presentedin Algorithm 4 At line 2 of Algorithm 4 the handle of aresource is requested and eventually locked out if grantedHowever if a SPI interrupt is received somewhere afterline 2 a data race will occur and the resource handlerstored in reqResId (line 6 Algorithm 4) may eventually beoverwritten by the SPI interrupt handler Also other shared

International Journal of Distributed Sensor Networks 11

(1) async command error t ResourceimmediateRequest [uint8 t id] ( ) (2) signal ResourceRequested immediateRequested [resId] ( )(3) atomic (4) if (state == RES CONTROLLED) (5) state = RES IMM GRANTING(6) reqResId = id(7)

(8) else return FAIL(9)

(10) signal ResourceDefaultOwner immediateRequested( )(11) sdot sdot sdot (12)

Algorithm 4 An excerpt of the request function from the component ArbiterP

(1) command error t Send send[uint8 t clientId] (message t lowastmsg(2) uint8 t len) (3) sdot sdot sdot (4) queue [clientId] msg = msg(5) call Packet setPayloadLength (msg len)(6) sdot sdot sdot (7)

Algorithm 5 An excerpt of the send function from the component AMSend

variables in ArbiterP like state can be corrupted as wellby the SPI interrupt handler

In the case of Oscilloscope which is a data collectionapplication that periodically samples the default sensor ofa mote and broadcasts a message over the radio every 10readings the reported data races are similar to those reportedin AnthiTheft Moreover our results suggest that if a SPIinterrupt arrives when a message is being transmitted viathe send() function of the AMSend interface the sharedbuffer sendBuf used by Oscilloscope to store incomingmessages will be updated and a corruptedmessage deliveredThe extract of code from the function Send presentedin Algorithm 5 shows that the variable msg which inthis case is a pointer to the shared variable sendBuf inOscilloscope can be updated at any time if the functionsend() is preempted because operations are not performedin a critical section This data race is somehow similar to theone we described in the example program of Algorithm 1Wealso find in Oscilloscope some of the data races reportedin the other applicationsThe results in Table 3 show that ourtool precisely detects data races on-the-fly in interrupt-drivenprograms without any false positives

64 Detection Overhead In order to evaluate the impactof our technique on the monitored programs we config-ured Avrora to simulate a single node sensor network Wethen measured the average runtime overhead and memoryspace required by each application of the benchmark ofTable 2 to execute 73728000 instruction cycles The totaloverhead of our technique mainly comes from actions of theinstrumentation engine the labeling routines and the race

Exec

utio

ns ti

me (

s)

14

12

10

8

6

4

2

0

TinyOS 210 programsOriginalWith iRace

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 4 Average runtime overhead

detection engine which also has the responsibility to createandmaintain the access historiesThe average execution timeand memory consumption collected for each application arepresented in Table 3 The columns ldquoOriginalrdquo and ldquoiRacerdquorepresent the execution time and memory consumption foreach application without and with iRace respectively

Figure 4 which plots the data from the column ldquoTimerdquoof Table 3 compares the average execution times of each

12 International Journal of Distributed Sensor Networks

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

TinyOS 210 programsOriginalWith iRace

Figure 5 Average memory consumption

application for five runs with and without iRace The captionldquoOriginalrdquo represents the native execution of the applica-tions without iRace The results of Figure 4 suggest thatthe average runtime overhead of our tool is only about1x for all applications except for RadioCountToLeds andRadioSenseToLeds that reaches 6x the original executiontime These two applications sample and display data at highfrequency rate (eg 4Hz in our case) and therefore arelikely to receive a higher number of interrupts at runtimeThis low runtime overhead is achieved by exploiting the factthat shared variables are stored by Avrora at program loadingtime Instead of monitoring all memory accesses to filterthose targeting shared variables iRace precisely instrumentsonly accesses to the well-known set of shared memory loca-tions obtained from the Avrora loader This technique allowsthe tool to drastically cuts down its monitoring overhead andto increase its scalability

Likewise wemeasured the averagememory consumptionfor each application during five runs The collected data arerepresented in Figure 5 The overall average memory con-sumption of the tool is very small at just about 20 above theoriginal memory requirement even for high frequency appli-cations like RadioCountToLeds and RadioSenseToLedsthanks to our labeling scheme that reuses the same labelfor different instances of the same interrupt handler and thedetection protocol that maintains only a reduced size accesshistoryThis result corroborates the theoretical analysis of ourtechnique as summarized in Table 1 The low runtime over-head and memory consumption of our on-the-fly techniquemake it very scalable and suitable even for interrupt-drivenprograms that handle interrupts at high frequency rates

65 Labeling Overhead Since the race detection subcompo-nent of iRace is in charge of implementing the labeling and

Table 4 Average labeling overhead

Applications Time (Sec) Memory (Kb)6LowPanCli 313 2101760AnthiTheft 308 2148880BaseStation 309 2077680Blink 174 1964640MultihopOscilloscope 313 2141920Oscilloscope 249 2092640Powerup 176 1894720RadioCountToLeds 218 2167440RadioSenseToLeds 224 2140080Sense 184 2011040

Table 5The states of an access history for detecting data races withour protocol considering the example of Figure 2

119886119894

Read Write Races11988211

⟨0 20 0⟩ ⟨20⟩

11987712

⟨0 20 0⟩ ⟨20⟩

11988213

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11988213-11988211

11988224

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩11987715

⟨1 1 0⟩ ⟨mdash⟩ ⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11987715-11988211

the detection routines it was legitimate to analyze the impactof the labeling scheme alone on the monitored programsat runtime For this purpose we deactivated the detectionengine of iRace before running it again on the benchmarkapplications We collected the average execution time andmemory space consumed by each application after five runsas shown in Table 4 The simulation was done using a singlenode network

Figure 6 compares the original execution time of eachprogram from Table 3 with the execution time of the labelingengine as presented in the column ldquoTimerdquo of Table 4 Thecomparison shows that the average runtime overhead ofiRace with just the labeling engine activated is only about04x in the worst-case This represents only a small fractionof the average runtime overhead of 6x imposed by iRace asdiscussed in Section 64Themain reason of this performancecan be found at source level When browsing the source codeof the benchmark applications we came to understand thatprogrammers generally disable and enable all the interruptsat once instead of focusing on particular interrupts In thiscase the size of each label generated by the labeling enginenever approaches its worst-case of 119874(119879) meaning that thecomparison overhead of labels is also reduced This is veryimportant because as discussed in Section 64 althoughthe size of labels does not depend on the total number ofinterrupts received at runtime this number still has impacton the overall detection overheadThus we can conclude thatthe labeling engine of iRace incurs a very low overhead atruntime making it a highly scalable tool

We also compared the memory consumption of thelabeling engine with the memory requirement of the original

International Journal of Distributed Sensor Networks 13

TinyOS 210 programs

4

35

3

25

2

15

1

05

0

Exec

utio

n tim

e (s)

With iRace labeling engineOriginal

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 6 Average runtime overhead of the labeling engine

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

With iRace labeling engineOriginal

TinyOS 210 programs

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 7 Average memory consumption by the labeling engine

program execution presented inTable 3The results plotted inFigure 7 show that when the labeling engine alone is runningiRace consumes only about 12 more memory on averageHowever during the detection phase the overall averagememory consumption of iRace is only about 20 Theseresults also tell us that the detection engine does not increasethat much the memory consumption of iRace This is alsoreasonable because we only disabled the detection engine ofiRace but did not remove the resources it used

7 Conclusions

Asynchronous interrupt handling introduces fine-grainedconcurrency into interrupt-driven programs making themprone to data races In this paper we presented an efficientand scalable on-the-fly technique to precisely detect dataraces without false positives in such programs Our techniqueintroduced a lightweight labeling scheme for maintainingconcurrency information for the main program and each ofits interrupt handlers and a precise detection protocol thatguarantees to verify the existence of data races in a programby reporting at least one data race for each shared variableinvolved in a data race

We implemented a prototype of our technique as a racedetection tool called iRace on top of the Avrora simulationframework An evaluation of iRace revealed the existence ofdata races in some programs distributed with TinyOS andin some TinyOS components while incurring a slowdownof only about 6 times on average in the worst-case with anaverage memory consumption of only about 20 above theoriginal memory requirements of the program Moreoverthis evaluation also proved that the labeling engine alonegenerates an average runtime overhead of only about 04xwith an average memory consumption of only about 12relatively to the original program execution These resultscoupled with a theoretical analysis showed that the proposedtechnique is highly scalable and practical even for largeinterrupt-driven programs

Detecting data races that occur first in a program isan important issue since it may make other data races todisappear It is therefore interesting to detect this class of dataraces in interrupt-driven programs to facilitate the debuggingprocess Also we may be interested in extending our work tosupport multithreaded programs that handle interrupts andalso to detect data races in low-level kernel codes

Acknowledgment

This research was financially supported by the Ministryof Education (MOE) and National Research Foundation ofKorea (NRF) through the Human Resource Training Projectfor Regional Innovation and BK21+ program

References

[1] U Banerjee B Bliss Z Ma and P Petersen ldquoA theory of datarace detectionrdquo in Proceedings of the Workshop on Parallel andDistributed Systems Testing and Debugging (PADTAD rsquo06) pp69ndash78 July 2006

[2] R H B Netzer and B P Miller ldquoWhat are race conditionsSome issues and formalizationsrdquo ACM letters on programminglanguages and systems vol 1 no 1 pp 74ndash88 1992

[3] N G Leveson and C S Turner ldquoInvestigation of theTherac-25accidentsrdquo Computer vol 26 no 7 pp 18ndash41 1993

[4] USA Canada Final Report on the August 14 2003 Blackout inthe United States and Canada Causes and RecommendationsUS-Canada Power System Outage Task Force 2004

[5] J Erickson M Musuvathi S Burckhardt and K OlynykldquoEffective data-race detection for the kernelrdquo in Proceedings of

14 International Journal of Distributed Sensor Networks

the 9th Conference on Operating Systems Design and Implemen-tation (USENIX rsquo10) 2010

[6] B-K Lee M-H Kang K C Park J S Yi S W Yang andY-K Jun ldquoProgram conversion for detecting data races inconcurrent interrupt handlersrdquo Communications in Computerand Information Science vol 257 pp 407ndash415 2011

[7] L Effinger-Dean B Lucia L Ceze D Grossman and H JBoehm ldquoIFRit interference-free regions for dynamic data-racedetectionrdquo in Proceedings of the ACM International Conferenceon Object Oriented Programming Systems Languages and Appli-cations (OOPSLA rsquo12) vol 47 pp 467ndash484 2012

[8] C Flanagan and S N Freund ldquoFastTrack efficient and precisedynamic race detectionrdquo in Proceedings of the ACM Interna-tional Conference on Object Oriented Programming SystemsLanguages and Applications (OOPSLA rsquo09) vol 44 pp 121ndash1332009

[9] X Xie and J Xue ldquoAcculock accurate and efficient detection ofdata racesrdquo in Proceedings of the 9th International Symposium onCode Generation and Optimization (CGO rsquo11) pp 201ndash212 April2011

[10] TinyOSCommunity ForumTinyOSAnopen source operatingsystem for low-power wireless devices 2008 httpwwwtinyosnet

[11] M Higashi T Yamamoto Y Hayase T Ishio and K InoueldquoAn effective method to control interrupt handler for datarace detectionrdquo in International Workshop on Automation ofSoftware Test (ACM rsquo10) pp 79ndash86 May 2010

[12] Z Lai S C Cheung and W K Chan ldquoInter-context control-flow and data-flow test adequacy criteria for nesC applicationsrdquoin Proceedings of the 16th ACM SIGSOFT International Sympo-sium on the Foundations of Software Engineering (SIGSOFT rsquo08)pp 94ndash104 November 2008

[13] J Regehr ldquoRandom testing of interrupt-driven softwarerdquo inPro-ceedings of the 5th ACM International Conference on EmbeddedSoftware (EMSOFT rsquo05) pp 290ndash298 September 2005

[14] Y Zhou X Chen M R Lyu and J Liu ldquoSentomist Unveil-ing transient sensor network bugs via symptom miningrdquoin Proceedings of the 30th IEEE International Conference onDistributed Computing Systems (ICDCS rsquo10) pp 784ndash794 June2010

[15] G Gracioli and S Fischmeister ldquoTracing interrupts in embed-ded softwarerdquo in Proceedings of the ACM SIGPLANSIGBEDConference on Languages Compilers and Tools for EmbeddedSystems (LCTES rsquo09) vol 44 pp 137ndash146 2009

[16] V Sundaram P Eugster and X Zhang ldquoLightweight tracingfor wireless sensor networks debuggingrdquo in Proceedings of the4th International Workshop on Middleware Tools Services andRun-Time Support for Sensor Networks (ACM rsquo09) pp 13ndash18December 2009

[17] O K Ha and Y K Jun ldquoMonitoring of programs with nestedparallelismusing efficient thread labelingrdquo International JournalFo U- and E- Service Science and Technology vol 4 pp 21ndash362011

[18] Y C Kim S S Jun and Y K Jun ldquoA tool for space-efficient on-the-fly race detectionrdquo International Journal of DatabaseTheoryand Application vol 4 pp 25ndash38 2011

[19] S Savage M Burrows G Nelson P Sobalvarro and T Ander-son ldquoEraser a dynamic data race detector for multithreadedprogramsrdquo ACM Transactions on Computer Systems vol 15 no4 pp 391ndash411 1997

[20] A Jannesari and W F Tichy ldquoOn-the-fly race detection inmulti-threaded programsrdquo in Proceedings of the 6th Workshop

on Parallel and Distributed Systems Testing Analysis andDebugging (PADTAD rsquo08) July 2007

[21] D Gay M Welsh P Levis E Brewer R Von Behren and DCuller ldquoThe nesC language a holistic approach to networkedembedded systemsrdquo inACMSIGPLANConference on Program-ming Language Design and Implementation pp 1ndash11 June 2003

[22] J Regehr and N Cooprider ldquoInterrupt verification via threadverificationrdquo Electronic Notes in Theoretical Computer Sciencevol 174 no 9 pp 139ndash150 2007

[23] B L Titzer D K Lee and J Palsberg ldquoAvrora scalable sensornetwork simulation with precise timingrdquo in Proceedings ofthe 4th International Symposium on Information Processing inSensor Networks (IPSN rsquo05) pp 477ndash482 April 2005

[24] B L Titzer and J Palsberg ldquoNonintrusive precision instru-mentation of microcontroller softwarerdquo in Proceedings of theConference on Languages Compilers and Tools for EmbeddedSystems (Lctes rsquo05) vol 40 pp 59ndash68 2005

[25] J Hil R Szewczyk A Woo S Hollar D Culler and K PisterldquoSystem architecture directions for networked sensorsrdquo inProceedings of the 9th International Conference on ArchitecturalSupport for Programming Languages and Operating Systems(ASPLOS rsquo00) pp 93ndash104 2000

[26] GMTchamgoue KHKim andYK Jun ldquoTesting anddebug-ging concurrency bugs in event-driven programsrdquo InternationalJournal of Advanced Science and Technology vol 40 pp 55ndash682012

[27] GM Tchamgoue O K Ha K H Kim and Y K Jun ldquoA frame-work for on-the-fly race healing in ARINC-653 applicationsrdquoInternational Journal of Hybrid Information Technology vol 4pp 1ndash12 2011

[28] A Dinning and E Schonberg ldquoDetecting access anomaliesin programs with critical sectionsrdquo in Proceedings of theACMONR Workshop on Parallel and Distributed Debugging(PADD rsquo91) vol 26 pp 85ndash96 1991

[29] L Lamport ldquoTime clocks and the ordering of events in adistributed systemrdquo Communications of the ACM vol 21 no7 pp 558ndash565 1978

[30] Atmel inc ATmega128 Datasheet 2011 httpwwwatmelcomImagesdoc2467pdf

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 5: Research Article Verification of Data Races in Concurrent ...downloads.hindawi.com/journals/ijdsn/2013/953593.pdf · photoTask I20 W 11 R 12 W 13 W 24 R 15 T 2 J12 T 1 F : A POEG

International Journal of Distributed Sensor Networks 5

suggested by the algorithms in Algorithm 2 A new labeledthread 119879

119894is created when a new task starts its execution or

whenever an interrupt handler is invoked When a new taskstarts a new label is generated for the first thread segment ofthe task by simply initializing the component of the label asin Algorithm 2(a) A new label is similarly created when aninterrupt handler is invoked in Algorithm 2(b) Each threadthen creates additional thread segments whenever it disablesor enables interrupts

When a thread segment 119905119894of a thread 119879

119894disables an

interrupt 119896 the label of the new thread segment 119905119895is generated

by adding the interrupt number 119896 into the interrupt set ofthe current label owned by 119905

119894 However if all interrupts in

the system are disabled by 119905119894 the new label for 119905

119895is obtained

by setting 119888119904119894and by emptying 119894119904119890119905

119894from the current label

owned by 119905119894as shown in Algorithm 2(c) Thus as seen in

Algorithm 2(d) if 119905119894enables an interrupt 119896 a new label for a

new thread segment 119905119895is obtained by simply removing 119896 from

the interrupt set of the current label in 119905119894 If all interrupts are

enabled at the same time the flag 119888119904119894and the interrupt set

119894119904119890119905119894are both reset from the current label of 119905

119894to generate

the new label for the new thread segment 119905119895 This helps

maintain a reduced-size interrupt set for each thread segmentsince programmers generally disable and enable all interruptsrather than manipulating individual interrupts

Moreover the size of an interrupt set is bounded by thetotal number of interrupts supported by the hardware onwhich the program runs which is generally a reasonableinteger number By reusing the same label for differentinstances of the same interrupt handler our labeling schemedoes not depend on the total number of interrupts receivedat runtime which can be very high

Lemma 6 Given two labeled thread segments 119905119894and 119905119895 with

119905119894belonging to an interrupt handler 119879

119896 119905119894

V999492999484 119905119895rArr 119896 notin 119894119904119890119905

119895or

119888119904119895= 119891119886119897119904119890

Proof Let us consider two labeled thread segments 119905119894and 119905119895

and let us assume that 119905119894belongs to an interrupt handler 119879

119896

We also assume that 119905119894violates the critical section of 119905

119895 By

definition thread segments are delimited by either the disableor enable instructions Therefore if 119905

119895was created after a

the disable instruction targeting interrupt 119896 the interruptnumber 119896 will be in the interrupt set 119894119904119890119905

119895of 119905119895according

to the labeling algorithms of Algorithm 2 However if 119905119895was

created after the disabling of all interrupts 119888119904119895would then be

true by the labeling algorithms of Algorithm 2 to signify thatall interrupts including 119896 are disabledThus for all 119905

119894isin 119879119896 119905119894

cannot violate 119905119895 which is contradictory since we assumed

119905119894

V999492999484 119905119895 Therefore 119905

119895has necessarily enabled 119896 that is

for all 119905119894isin 119879119896 119905119894

V999492999484 119905119895rArr 119896 notin 119894119904119890119905

119895or 119888119904119895= 119891119886119897119904119890

32 Concurrency Relation Using our labeling scheme itbecomes easy to analyze the happens-before relation betweenany pair of thread segments 119905

119894and 119905119895 by comparing their

labels

[a Task Starting]a1 tsk start()a2 Label Lba3 Lbt = truea4 Lbtid = getTaskid()a5 Lbcs = falsea6 Lbiset =

a7 return Lb[b Interrupt Invocation]b1 int invoke()b2 Label Lbb3 Lbt = falseb4 Lbtid = getIntNumber()b5 Lbcs = falseb6 Lbiset =

b7 return Lb[c Interrupt Disabling]c1 int disable(Label Lb Interrupt n)c2 if(n gt 0) Lbiset cup 119899

c3 elsec4 Lbcs = truec5 Lbiset =

c6 endifc7 return Lb[d Interrupt Enabling]d1 int enable(Label Lb Interrupt n)d2 if(n gt 0) Lbiset-nd3 elsed4 Lbcs = falsed5 Lbiset =

d6 endifd7 return Lb

Algorithm 2 Labeling algorithms

Theorem7 A labeled thread segment 119905119894happens before anoth-

er labeled thread segment 119905119895 denoted by 119905

119894

ℎ119887

997888997888rarr 119905119895 if and only if

either that 119905119894and 119905119895belong to tasks or that 119905

119895does not violate

119905119894 that is not(119905

119895

V999492999484 119905119894)

Proof Let us assume two labeled thread segments 119905119894and

119905119895 such that 119905

119894

ℎ119887

997888997888rarr 119905119895 Thus according to the labeling

algorithms of Algorithm 2 we should have 119905119905119894= 119905119905119895= 119905119903119906119890

meaning that 119905119894and 119905

119895are either from the same or from

different tasks By definition tasks are all ordered and cannotpreempt each other However if 119905

119895is a thread segment of

an interrupt handler 119879119896 according to Definition 4 either

119888119904119894= 119905119903119906119890 because 119905

119894disabled all interrupts or the thread

identifier 119905119894119889119895of 119905119895that corresponds to 119896 the interrupt

number of 119879119896 is in 119894119904119890119905

119894 the interrupt set of 119905

119894 This means

that 119905119894is a critical section thread segment that disables either

all interrupts or particularly the interrupt number 119896 Thusaccording to Lemma 6 119905

119895does not violate the critical section

of 119905119894 Therefore we can say that 119905

119894

ℎ119887

997888997888rarr 119905119895implies either that 119905

119894

and 119905119895belong to tasks or that not(119905

119895

V999492999484 119905119894)

6 International Journal of Distributed Sensor Networks

Now let us assume that 119905119894and 119905

119895belong to tasks

Naturally 119905119894and 119905119895are ordered and the labeling algorithms of

Algorithm 2 will set 119905119905119894= 119905119905119895= 119905119903119906119890 meaning that 119905

119894

ℎ119887

997888997888rarr 119905119895

On the other hand if we assume that not(119905119895

V999492999484 119905119894) then 119905

119895

must be part of a given interrupt handler 119879119896and either 119905

119894has

disabled all interrupts to set 119888119904119894= 119905119903119906119890 or the thread identifier

119905119894119889119895= 119896 of 119905

119895is in the interrupt set 119894119904119890119905

119894of 119905119894 according to

Definition 4 and Lemma 6 Therefore we can conclude that119905119894

ℎ119887

997888997888rarr 119905119895

Any two labeled thread segments 119905119894and 119905119895are therefore

concurrent if they do not satisfy the happens-before relationas stated by Theorem 7 which can formally be rewritten asfollows

119905119894

ℎ119887

997888997888rarr 119905119895lArrrArr

119905119905119894= 119905119905119895= 119905119903119906119890

or(119905119905119895= 119891119886119897119904119890) and (119888119904

119894= 119905119903119906119890 or 119905119894119889

119895isin 119894119904119890119905119894)

(1)

By comparing based on Theorem 7 the labels of thethread segments in the POEG represented by Figure 2 wecan easily obtain that the thread segments of 119868

20 1198941 and 119894

2

are concurrent with other thread segments 1199051 1199053 1199054 and

1198951 Therefore the write accesses in 119894

1and 1198942conflict with

the other accesses in 1199053 leading to four data races 119882

11-11988213

11988211-1198771511988224-11988213 and119882

24-11987715 However since the accesses

11988211and119882

23 respectively from 119894

1and 1198942 represent the same

write access the set of data races to be reported for an effectivedebugging process should be reduced to only 119882

11-11988213

and11988211-11987715 By disabling all interrupts 119905

2enforces the happens-

before relation with other thread segments Although 1198791is

posted by 11986820 they are still concurrent Also the thread

segment 1198951is concurrent with 119894

1 1198942 1199051 1199053 and 119905

4

4 Precise Detection Protocol

The race detection protocol monitors memory accesses bymaintaining a special data structure called access history intowhich are stored the labels of prior accesses to each sharedvariable Our detection protocol stores in each entry of theaccess history of a shared variable a unique label representingthe thread segment that accessed the variable as describedin Section 3 At each access to a shared variable data racesare reported by analyzing based onTheorem 7 the happens-before relation between the current thread segment and thelabels of prior thread segments stored in the access historyTherefore the access history for each shared variable hasto be efficiently organized in order to reduce the overheadgenerated by the detection process

Our precise detection protocol maintains an access his-tory consisting of two sets for each shared variable a Readset and a Write set to respectively store labels of threadsegments that read and write the variable We use 119860119867(119909)

to designate the access history of a shared variable 119909 Thus119860119867(119909) sdot 119877119890119886119889 and 119860119867(119909) sdot 119882119903119894119905119890 refer to the read set andthe write set of 119860119867(119909) respectively The algorithms used bythe detection protocol are presented in Algorithm 3 In thesealgorithms we assume that the variable 119871119887 always contains

20

20

20

20

t1

t2

t3

t4

12

12

lowast

lowast

i1⟨0 20 0⟩⟨20⟩

⟨mdash⟩⟨1 1 1⟩

⟨mdash⟩⟨1 1 0⟩

⟨mdash⟩⟨1 1 0⟩

⟨0 12 0⟩⟨12⟩

⟨0 20 0⟩⟨20⟩

⟨mdash⟩⟨1 2 0⟩

I20

W11

R12

W13

W24

R15

T2

J12

T1

j1

i2

Figure 2 An example labeled POEG for an execution instance ofthe program in Algorithm 1

the label of the current thread segment accessing a sharedvariable and that 119901119871119887 represents any prior label stored in theaccess history

When a read access occurs to a shared variable 119909the detection protocol calls the ah read() function ofAlgorithm 3(a) that then reports data races by analyzing thehappens-before relation between the label of the accessingthread segment and the labels of prior accesses stored in119860119867(119909) sdot119882119903119894119905119890 Similarly upon a write access to a shared vari-able the monitoring function ah write() of Algorithm 3(b)reports data races by looking into both 119860119867(119909) sdot 119877119890119886119889 and119860119867(119909) sdot 119882119903119894119905119890 We assume the existence of a functionOrdered() that based on Theorem 7 decides whether athread segment 119905

119894happens before another thread segment 119905

119895

by comparing their labelsIn order to maintain a reduced size access history

accesses from critical sections that concern all interruptswithin a task are not inserted in the access history sincethey will never run into a data race with other accessesfrom interrupt handlers Also only the first read and writeaccesses to a shared variable from the same thread segmentare kept into the access history Subsequent similar accessesto the same shared variable from the same thread segmentare filtered outThis filtering operations are performed by theisFiltered() function of Algorithm 3(c) that is called by bothah read() and ah write() before any other action

After data races have been reported with the currentaccess the function maintain ah() of Algorithm 3(d) is usedfor keeping the size of the access history balanced Thisfunction replaces some previous accesses in the access historywith the current access Thus accesses from different tasksare considered from the same thread since tasks by defini-tion are all ordered by happens-before Similarly accesses

International Journal of Distributed Sensor Networks 7

(1) ah read(Label Lb Access History AH Variable x)(2) if (isFiltered(Lb AH(x)Read)) then return(3) for all labels pLb in AH(x)Write do(4) if (Not Ordered(Lb pLb)) then(5) report race(x pLb Lb)(6) end for(7) maintain ah(Lb AH(x)Read)

(a) Monitoring Protocol for Read Accesses(1) ah write(Label Lb Access History AHVariable x)(2) if (isFiltered(Lb AH(x)Write)) then return(3) for all labels pLb in(4) AH(x)Write cup AH(x)Read do(5) if (Not Ordered(Lb pLb)) then(6) report race(x pLb Lb)(7) end for(8) maintain ah(Lb AH(x)Write)

(b) Monitoring Protocol for Write Accesses(1) isFiltered(Label Lb Access History AH)(2) if (Lbcs = true) and (Lbtt = true)(3) then return true(4) if (any access with a label pLb exists in AH(5) such that pLb = Lb) then return true(6) else(7) return false

(c) Filtering Redundant Accesses(1) maintain ah(Label Lb Access History AH)(2) if (Lbtt = true) then(3) delete all entries in AH with label pLb(4) such that pLbtt = true(5) else(6) delete all entries in AH with label pLb such(7) that pLbtt = Lbtt and pLbtid = Lbtid(8) end if(9) add ah(Lb AH)

(d) Maintaining Access History

Algorithm 3 Labeling algorithms

from different instances of the same interrupt handler areconsidered from a single instance of the interrupt handlerThis property is already enforced by the labeling scheme thatreuses the same label for all instances of the same interrupthandler Again we assume the existence of another functionadd ah() which is able to insert the label of the current threadsegment into the appropriate set of the access history given ashared variable

Table 5 shows the different states of an access historyfor detecting data races with our protocol in the exampleprogram of Figure 2 where accesses to the shared variablemsg appear in the first column When the first access 119882

11

occurs the label ⟨0 20 0⟩⟨20⟩ of the thread segment 1198941

is simply added into the access history and no data racereported as there is no other access prior to 119882

11 The next

access 11987712 from the thread segment 119905

2is filtered out because

1199052that belongs to task 119879

1has disabled all interrupts The

following access 11988213

occurs on 1199053with interrupts enabled

Thus a data race11988213-11988211is reported by comparing the label

of 1199053 ⟨1 1 0⟩⟨minus⟩ with the label of 119894

1previously stored into the

access history Afterwards the label of 1199053is inserted into the

write set of the access history The thread segment 1199053is then

preempted by 1198942 but the access 119882

24is filtered out because

1198942which is a new instance of the interrupt 119868

20 also shares the

same label ⟨0 20 0⟩⟨20⟩ with 1198941which previously performed

a write and whose label is already in the access history Afterthe execution of 119894

2 1199053resumes to perform the read access

11987715 Another data race 119877

15-11988211

is reported and the label of1199053 ⟨1 1 0⟩⟨minus⟩ stored into the read set of the access history

Although the program received two instances of the ADCinterrupt our protocol precisely reported only the set of twodata races needed for effectively debug the program Thisshows that our detection protocol does not depend on thenumber of interrupts received at runtime which can be veryhigh

Although it keeps only a reduced set of labels in the accesshistory our precise detection protocol even with a minimalnumber of interrupts still guarantees to verify the existenceof data races in a program For this purpose the protocol isdesigned to report without false positives at least one datarace for each shared variable involved in a data race

8 International Journal of Distributed Sensor Networks

Theorem 8 Given an interrupt-driven program 119875 the detec-tion protocol reports a data race over a shared variable 119909 if andonly if there exists a data race over 119909 in an execution instanceof 119875

Proof We will first prove that if a data race is reported on ashared variable 119909 by the detection protocol then there alsoexists a data race in an execution instance of the program forthe same shared variable 119909 For this we assume a snapshotof the access history 119860119867(119909) of the shared variable 119909 at agiven time and the current access event 119886

119894to 119909 from a thread

segment 119905119894that owns a label we refer to as 119871119887

119894in an execution

instance of the program 119875 We can then distinguish twodifferent cases

Case A 119860119867(119909) is empty and there is obviously no data racefor 119909 In this case the current access 119886

119894is the only access to

119909 from the program 119875 and 119871119887119894will be added by the function

add ah() into119860119867(119909)sdot119877119890119886119889 if 119886119894is a read access or into119860119867(119909)sdot

119882119903119894119905119890 if it is a write access

Case B 119860119867(119909) is not empty Data races are reported usingthe ah read() function if 119886

119894is a read access or the ah write()

in case of a write access In both cases the happens-beforerelation is analyzed between the label 119871119887

119894of 119905119894and previous

labels stored in 119860119867(119909) according to Theorem 7 The outputsof these analyses can be classified into two categories

(1) There is no data race between 119886119894and119860119867(119909) therefore

all previous accesses stored in 119860119867(119909) happenedbefore 119886

119894meaning that no thread segment prior to

119905119894in 119875 is concurrent with 119905

119894 they are all ordered by

happens-before Thus for all 119886119895isin 119905119895and 119871119887119895isin 119860119867(119909)

119905119895

ℎ119887

997888997888rarr 119905119894 In which case 119875 also has no data race

(2) There are data races between 119886119894and prior accesses

in 119860119867(119909) this means that there are some labels in119860119867(119909) belonging to some prior thread segments thatare not ordered with 119905

119894 Concretely there are thread

segments in 119875 that are concurrent with 119905119894 Formally

exist119905119895isin 119875119871119887

119895isin 119860119867(119909) 119905

119894 119905119895Therefore the execution

instance of 119875 also contains data races over 119909

Now we show that if there exists a data race over a sharedvariable 119909 in an execution instance of the program 119875 thenour detection protocol also reports a data race over 119909 Usingthe contrapositive we will prove that if the detection protocoldoes not report any data race the program 119875 also does notcontain any for a shared variable 119909 We again consider asnapshot of the access history 119860119867(119909) for a shared variable119909 at a given time We also assume that the detection protocolreported no data races for the shared variable 119909 This meansthat 119860119867(119909) is either empty and there is clearly no data racesince there is no access to 119909 from 119875 yet or contains onlylabels of thread segments that are all ordered by the happens-before relation in which case there is no concurrent threadsegment in119875 which also implies that119875 has no data races over119909 Therefore we can formally write for all 119905

119894 119905119895isin 119875 and 119871119887

119894

119871119887119895isin 119860119867(119909) 119905

119894

ℎ119887

997888997888rarr 119905119895or 119905119895

ℎ119887

997888997888rarr 119905119894 Thus 119875 also contains no

data race for 119909

Table 1 Efficiency of the precise detection protocol

Space TimeNew label comparison Per access

O(1198811198792) O(1) O(LogT) O(TLogT)

5 Theoretical Analysis

On-the-fly data race detection techniques are known to havehigh runtime overhead that is generally provoked by the timespentmonitoring accesses to shared variables and to comparethread labels In this section we theoretically analyze theefficiency of our technique which depends on119881 the numberof shared variables and 119879 the total number of interruptsin the underlying system plus one the number of tasksIn general 119879 is very small since the number of interruptshandled by a system is also kept small For example thepopular Atmel ATmega128 [30] processor supports only amaximum of 34 interrupts Also because of the implicithappens-before relation that exists between all tasks they areviewed as a single task by the detection protocol

Table 1 presents the space and time complexities of ourlabeling and detection techniques The access history usedby our technique stores from each thread segment at mosttwo accesses for each shared variable Each label in the accesshistory has a worst-case size of 119874(119879) that represents thesize of its interrupt set Therefore the total labeling spacerequirement for each shared variable is 119874(1198792) Thus thetotal space complexity of our technique is 119874(1198811198792) Howeverbecause the number of interrupts is generally a reasonableinteger number the space complexity of our technique canfurther be reduced to 119874(119881)

A new label is always created in a constant amount of timeby our labeling algorithms Interrupts are generally givennumbers in a sequence order Therefore a binary searchtechnique is applied to locate values in interrupt sets whichcontain only interrupt numbers Thus the time complexityfor comparing any two labels is 119874(Log119879) in the worst-caseand 119874(1) in most cases where the interrupt sets are emptyAt each access to a shared variable the total time complexityto report data races with the previous accesses stored in theaccess history is then119874(119879Log119879)The labels and the detectionprotocol do not depend on the total number of interruptsreceived at runtimemaking our technique highly scalable andpractical

6 Experimental Results

We empirically evaluated the preciseness and efficiency ofour technique using a set of well-known programs providedwith TinyOS [10] This section details the implementationchallenges and presents the analysis results of the experimen-tation

61 Implementation We implemented a prototype of ourdata race detection tool for interrupt-driven programs callediRace based on the concurrency model of TinyOS [21] andits nesC language [25] We implemented iRace on top

International Journal of Distributed Sensor Networks 9

Table 2 Characteristics of the benchmark applications from TinyOS 210 [10]

Applications LOC Int SV Brief description6LowPanCli 19116 4 9 Implements a command line interface using the 6lowpan stackAnthiTheft 27421 5 4 Detects and reports a stolen mote by checking some sensorsBaseStation 14905 6 13 Sends and receives simple active messages on the radio and serial linksBlink 3688 1 0 Periodically blinks the three LEDs of a moteMultihopOscilloscope 24310 9 8 Periodically samples and broadcasts sensor data to a base stationOscilloscope 11686 5 5 Periodically samples sensor data and broadcasts a message every 10 timesPowerup 2352 0 0 Turns on the red LED when the mote powers upRadioCountToLeds 11165 4 3 Periodically samples sensors and displays the last 3 bits on its LEDsRadioSenseToLeds 11472 5 2 Periodically update a counter and displays the last 3 bits on its LEDsSense 5743 2 0 Periodically takes sensor readings and displays the values on the LEDs

of the Avrora [23] simulation and analysis framework thatprovides a time accurate simulator different analysis toolsknown asmonitors and a binary instrumentation engine [24]for programs written for AVR microcontrollers and sensornetworks All these features make Avrora just suitable forour tool Simulation is commonly used in the developmentcycle of embedded software because target systems generallylack conventional inputoutput interfaces With iRace thefunctionalities of Avrora are extendedwith a precise data racedetection tool that can easily be loaded as one of Avrorarsquosmonitors

As shown in Figure 3 iRace consists of two maincomponents the Instrumenter which exploits the Avrorainstrumentation engine [24] and the core Race Detectoritself The Instrumenter monitors task and interrupts activ-ities through its iMonitor component and provides entrypoints for the labeling routines Instrumentation points areinserted at the beginning of each task at the start andexit of each interrupt handler and also before interruptsare either enabled or disabled Similarly accesses to sharedmemory locations are monitored using the AccessMonitorcomponent that also calls routines of the detection proto-col The AccessMonitor instruments before read and writeaccesses to shared variables The Race Detector is composedof labeling routines that generate and manage labels for eachactive thread segment at runtime and detection routines formaintaining the access history and reporting data races on-the-fly as shared variables are manipulated

In TinyOS [21 25] interrupt handlers and tasks all runto completion unless preempted by other interrupt handlersThus only a single thread runs at any point of time givingus the opportunity to use only a single variable to maintainthe active thread label at detection time However whenpreemption happens the label of the preempted threadsegment must be saved and restored again when the activeinterrupt handler exits Otherwise there will be no way toknow about the concurrency information of the preemptedthread segment when it gains back the execution control Inorder tomanage the preemption we use a stack data structurewhere labels can be pushed and restored from since interruptscan be nested This provides a simple but powerful way tohandle preemption with negligible overhead

Our labeling scheme needs to uniquely identify eachinterrupt handler whenever it is invoked and each taskwhen it starts its execution Each interrupt handler is easilyidentified by the interrupt number of the interrupt it servesFor task we chose to generate a unique task identifier bysimply maintaining a counter that increases at each timea task starts its execution Thus each task is attributed aunique identifier thanks to the fact that once posted TinyOStasks run in a first-in first-out manner and cannot preempteach other By manipulating only binary files iRace doesnot require any source file nor source code annotation toeffectively detect data races However compiling the programwith debugging information may help iRace generate moredetailed debugging information on data races it reports

62 Environment We implemented and evaluated our tech-nique on a system that runs the kernel 30 of the Linuxoperating system using an Intel dual-core 320GHz and 4GBof mainmemory On this machine was installed TinyOS 210[10] along with the nesC compiler 131 the GNU C compiler46 the Java Runtime Environment 17 and the latest Avroraframework version 17115

Table 2 presents the characteristics and a brief descriptionof the benchmark applications selected for our evaluationfrom the TinyOS 210 repository [10] These applications arefreely distributed with each version of TinyOS The columnldquoLOCrdquo indicates the number of line of C codes generatedby the nesC compiler after compilation while the columnldquoIntrdquo contains the number of interrupts handled by eachapplicationThe number of shared variables for each programappears in column ldquoSVrdquo It is worthmentioning that this num-ber represents only the shared variables in the source code ofeach program The exact number of variables shared by eachapplicationmay finally depend on the number of componentsit shares with the operating system itself Applications writtenin nesC share TinyOS components by wiring themselvesto these components increasing therefore their number ofshared resources We compiled each application for theMica2 platform and measured the average runtime overheadover five executions The Mica2 motes are sensor networknodes based on the ATmega128 [30] microcontrollers Weconfigured Avrora to simulate a sensor network during 10

10 International Journal of Distributed Sensor Networks

iRace

and instrumentationframework

iMonitor

AccessMonitor

Race detector

Labeling routines

Detection protocolroutines

Binarycode Data

races

Avrora simulation

Figure 3 The Architecture of the Data Race Detection Tool

Table 3 Reported data races on the benchmark applications

Applications Data races Verif races Time (Sec) Memory (Kb)Original iRace Original iRace

6LowPanCli 0 0 269 357 1936960 2168160AnthiTheft 15 15 292 457 1944400 2320000BaseStation 0 0 279 376 1912240 2148240Blink 0 0 161 180 1887280 1949600MultihopOscilloscope 0 0 278 400 1939600 2196960Oscilloscope 13 13 179 328 1909360 2201760Powerup 0 0 151 186 1873600 1879920RadioCountToLeds 17 17 196 1255 1942880 2326320RadioSenseToLeds 15 15 181 1305 1944480 2298240Sense 0 0 158 192 1866400 2050560

seconds allowing each application to run exactly for 73728000instruction cycles We set to five the number of nodes in thesimulated sensor network

63 Evaluation For data race detectionwe instructedAvrorato simulate a sensor network with five nodes while thebenchmark applications are monitored by iRace Table 3represents the results of the evaluation of iRace on thebenchmark applications in its column ldquoData Racesrdquo whichcontains the number of shared variables involved in adata race for each application Our tool reported no datarace for all applications except for four AnthiTheft with15 shared variables Oscilloscope with 13 shared vari-ables RadioCountToLeds with 17 shared variables andRadioSenseToLeds with 15 shared variables Surprisinglymost of the reported data races reside inside some sharedsystem components of TinyOS itself The reported data racesare caused by two different interrupts namely the serialperipheral interface (SPI) interrupt and the timer interruptAlso we carefully analyzed the reported data races andinspected the source code of the applications for a manualverification process as shown in the column ldquoVerified racesrdquoof Table 3

In the AnthiTheft application for example which isused to detect and report a stolen mote by checking the levelof the light or the acceleration data races were reported over

15 shared variables Most of the reported data races are how-ever due to the fact that the SPI interrupt handler can preemptsome functions or tasks owned by the underlying radio stackThis implies that if a message is being received or sent whena SPI interrupt occurs shared variables like radioState orcount in the component CC1000SendReceiveP might becorrupted However all shared variables involved in a datarace do not directly belong to AnthiTheft but to functionsof the CC1000 radio stack in use in Mica2 motes Similardata races are also reported in RadioCountToLeds andRadioSenseToLeds since they all access the radio stack

Another data race reported both in RadioCountToLedsand RadioSenseToLeds and worthy to be commentedconcerns the shared variable reqResId inside thecomponent ArbiterP (Recently this data race hasalso been pointed out and a patch discussed in theTinyOS Forum) This data race occurs when the functionimmediateRequest() called within the function read ofthe component immediateRequest is preempted by a SPIinterrupt handler An excerpt of this function is presentedin Algorithm 4 At line 2 of Algorithm 4 the handle of aresource is requested and eventually locked out if grantedHowever if a SPI interrupt is received somewhere afterline 2 a data race will occur and the resource handlerstored in reqResId (line 6 Algorithm 4) may eventually beoverwritten by the SPI interrupt handler Also other shared

International Journal of Distributed Sensor Networks 11

(1) async command error t ResourceimmediateRequest [uint8 t id] ( ) (2) signal ResourceRequested immediateRequested [resId] ( )(3) atomic (4) if (state == RES CONTROLLED) (5) state = RES IMM GRANTING(6) reqResId = id(7)

(8) else return FAIL(9)

(10) signal ResourceDefaultOwner immediateRequested( )(11) sdot sdot sdot (12)

Algorithm 4 An excerpt of the request function from the component ArbiterP

(1) command error t Send send[uint8 t clientId] (message t lowastmsg(2) uint8 t len) (3) sdot sdot sdot (4) queue [clientId] msg = msg(5) call Packet setPayloadLength (msg len)(6) sdot sdot sdot (7)

Algorithm 5 An excerpt of the send function from the component AMSend

variables in ArbiterP like state can be corrupted as wellby the SPI interrupt handler

In the case of Oscilloscope which is a data collectionapplication that periodically samples the default sensor ofa mote and broadcasts a message over the radio every 10readings the reported data races are similar to those reportedin AnthiTheft Moreover our results suggest that if a SPIinterrupt arrives when a message is being transmitted viathe send() function of the AMSend interface the sharedbuffer sendBuf used by Oscilloscope to store incomingmessages will be updated and a corruptedmessage deliveredThe extract of code from the function Send presentedin Algorithm 5 shows that the variable msg which inthis case is a pointer to the shared variable sendBuf inOscilloscope can be updated at any time if the functionsend() is preempted because operations are not performedin a critical section This data race is somehow similar to theone we described in the example program of Algorithm 1Wealso find in Oscilloscope some of the data races reportedin the other applicationsThe results in Table 3 show that ourtool precisely detects data races on-the-fly in interrupt-drivenprograms without any false positives

64 Detection Overhead In order to evaluate the impactof our technique on the monitored programs we config-ured Avrora to simulate a single node sensor network Wethen measured the average runtime overhead and memoryspace required by each application of the benchmark ofTable 2 to execute 73728000 instruction cycles The totaloverhead of our technique mainly comes from actions of theinstrumentation engine the labeling routines and the race

Exec

utio

ns ti

me (

s)

14

12

10

8

6

4

2

0

TinyOS 210 programsOriginalWith iRace

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 4 Average runtime overhead

detection engine which also has the responsibility to createandmaintain the access historiesThe average execution timeand memory consumption collected for each application arepresented in Table 3 The columns ldquoOriginalrdquo and ldquoiRacerdquorepresent the execution time and memory consumption foreach application without and with iRace respectively

Figure 4 which plots the data from the column ldquoTimerdquoof Table 3 compares the average execution times of each

12 International Journal of Distributed Sensor Networks

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

TinyOS 210 programsOriginalWith iRace

Figure 5 Average memory consumption

application for five runs with and without iRace The captionldquoOriginalrdquo represents the native execution of the applica-tions without iRace The results of Figure 4 suggest thatthe average runtime overhead of our tool is only about1x for all applications except for RadioCountToLeds andRadioSenseToLeds that reaches 6x the original executiontime These two applications sample and display data at highfrequency rate (eg 4Hz in our case) and therefore arelikely to receive a higher number of interrupts at runtimeThis low runtime overhead is achieved by exploiting the factthat shared variables are stored by Avrora at program loadingtime Instead of monitoring all memory accesses to filterthose targeting shared variables iRace precisely instrumentsonly accesses to the well-known set of shared memory loca-tions obtained from the Avrora loader This technique allowsthe tool to drastically cuts down its monitoring overhead andto increase its scalability

Likewise wemeasured the averagememory consumptionfor each application during five runs The collected data arerepresented in Figure 5 The overall average memory con-sumption of the tool is very small at just about 20 above theoriginal memory requirement even for high frequency appli-cations like RadioCountToLeds and RadioSenseToLedsthanks to our labeling scheme that reuses the same labelfor different instances of the same interrupt handler and thedetection protocol that maintains only a reduced size accesshistoryThis result corroborates the theoretical analysis of ourtechnique as summarized in Table 1 The low runtime over-head and memory consumption of our on-the-fly techniquemake it very scalable and suitable even for interrupt-drivenprograms that handle interrupts at high frequency rates

65 Labeling Overhead Since the race detection subcompo-nent of iRace is in charge of implementing the labeling and

Table 4 Average labeling overhead

Applications Time (Sec) Memory (Kb)6LowPanCli 313 2101760AnthiTheft 308 2148880BaseStation 309 2077680Blink 174 1964640MultihopOscilloscope 313 2141920Oscilloscope 249 2092640Powerup 176 1894720RadioCountToLeds 218 2167440RadioSenseToLeds 224 2140080Sense 184 2011040

Table 5The states of an access history for detecting data races withour protocol considering the example of Figure 2

119886119894

Read Write Races11988211

⟨0 20 0⟩ ⟨20⟩

11987712

⟨0 20 0⟩ ⟨20⟩

11988213

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11988213-11988211

11988224

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩11987715

⟨1 1 0⟩ ⟨mdash⟩ ⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11987715-11988211

the detection routines it was legitimate to analyze the impactof the labeling scheme alone on the monitored programsat runtime For this purpose we deactivated the detectionengine of iRace before running it again on the benchmarkapplications We collected the average execution time andmemory space consumed by each application after five runsas shown in Table 4 The simulation was done using a singlenode network

Figure 6 compares the original execution time of eachprogram from Table 3 with the execution time of the labelingengine as presented in the column ldquoTimerdquo of Table 4 Thecomparison shows that the average runtime overhead ofiRace with just the labeling engine activated is only about04x in the worst-case This represents only a small fractionof the average runtime overhead of 6x imposed by iRace asdiscussed in Section 64Themain reason of this performancecan be found at source level When browsing the source codeof the benchmark applications we came to understand thatprogrammers generally disable and enable all the interruptsat once instead of focusing on particular interrupts In thiscase the size of each label generated by the labeling enginenever approaches its worst-case of 119874(119879) meaning that thecomparison overhead of labels is also reduced This is veryimportant because as discussed in Section 64 althoughthe size of labels does not depend on the total number ofinterrupts received at runtime this number still has impacton the overall detection overheadThus we can conclude thatthe labeling engine of iRace incurs a very low overhead atruntime making it a highly scalable tool

We also compared the memory consumption of thelabeling engine with the memory requirement of the original

International Journal of Distributed Sensor Networks 13

TinyOS 210 programs

4

35

3

25

2

15

1

05

0

Exec

utio

n tim

e (s)

With iRace labeling engineOriginal

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 6 Average runtime overhead of the labeling engine

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

With iRace labeling engineOriginal

TinyOS 210 programs

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 7 Average memory consumption by the labeling engine

program execution presented inTable 3The results plotted inFigure 7 show that when the labeling engine alone is runningiRace consumes only about 12 more memory on averageHowever during the detection phase the overall averagememory consumption of iRace is only about 20 Theseresults also tell us that the detection engine does not increasethat much the memory consumption of iRace This is alsoreasonable because we only disabled the detection engine ofiRace but did not remove the resources it used

7 Conclusions

Asynchronous interrupt handling introduces fine-grainedconcurrency into interrupt-driven programs making themprone to data races In this paper we presented an efficientand scalable on-the-fly technique to precisely detect dataraces without false positives in such programs Our techniqueintroduced a lightweight labeling scheme for maintainingconcurrency information for the main program and each ofits interrupt handlers and a precise detection protocol thatguarantees to verify the existence of data races in a programby reporting at least one data race for each shared variableinvolved in a data race

We implemented a prototype of our technique as a racedetection tool called iRace on top of the Avrora simulationframework An evaluation of iRace revealed the existence ofdata races in some programs distributed with TinyOS andin some TinyOS components while incurring a slowdownof only about 6 times on average in the worst-case with anaverage memory consumption of only about 20 above theoriginal memory requirements of the program Moreoverthis evaluation also proved that the labeling engine alonegenerates an average runtime overhead of only about 04xwith an average memory consumption of only about 12relatively to the original program execution These resultscoupled with a theoretical analysis showed that the proposedtechnique is highly scalable and practical even for largeinterrupt-driven programs

Detecting data races that occur first in a program isan important issue since it may make other data races todisappear It is therefore interesting to detect this class of dataraces in interrupt-driven programs to facilitate the debuggingprocess Also we may be interested in extending our work tosupport multithreaded programs that handle interrupts andalso to detect data races in low-level kernel codes

Acknowledgment

This research was financially supported by the Ministryof Education (MOE) and National Research Foundation ofKorea (NRF) through the Human Resource Training Projectfor Regional Innovation and BK21+ program

References

[1] U Banerjee B Bliss Z Ma and P Petersen ldquoA theory of datarace detectionrdquo in Proceedings of the Workshop on Parallel andDistributed Systems Testing and Debugging (PADTAD rsquo06) pp69ndash78 July 2006

[2] R H B Netzer and B P Miller ldquoWhat are race conditionsSome issues and formalizationsrdquo ACM letters on programminglanguages and systems vol 1 no 1 pp 74ndash88 1992

[3] N G Leveson and C S Turner ldquoInvestigation of theTherac-25accidentsrdquo Computer vol 26 no 7 pp 18ndash41 1993

[4] USA Canada Final Report on the August 14 2003 Blackout inthe United States and Canada Causes and RecommendationsUS-Canada Power System Outage Task Force 2004

[5] J Erickson M Musuvathi S Burckhardt and K OlynykldquoEffective data-race detection for the kernelrdquo in Proceedings of

14 International Journal of Distributed Sensor Networks

the 9th Conference on Operating Systems Design and Implemen-tation (USENIX rsquo10) 2010

[6] B-K Lee M-H Kang K C Park J S Yi S W Yang andY-K Jun ldquoProgram conversion for detecting data races inconcurrent interrupt handlersrdquo Communications in Computerand Information Science vol 257 pp 407ndash415 2011

[7] L Effinger-Dean B Lucia L Ceze D Grossman and H JBoehm ldquoIFRit interference-free regions for dynamic data-racedetectionrdquo in Proceedings of the ACM International Conferenceon Object Oriented Programming Systems Languages and Appli-cations (OOPSLA rsquo12) vol 47 pp 467ndash484 2012

[8] C Flanagan and S N Freund ldquoFastTrack efficient and precisedynamic race detectionrdquo in Proceedings of the ACM Interna-tional Conference on Object Oriented Programming SystemsLanguages and Applications (OOPSLA rsquo09) vol 44 pp 121ndash1332009

[9] X Xie and J Xue ldquoAcculock accurate and efficient detection ofdata racesrdquo in Proceedings of the 9th International Symposium onCode Generation and Optimization (CGO rsquo11) pp 201ndash212 April2011

[10] TinyOSCommunity ForumTinyOSAnopen source operatingsystem for low-power wireless devices 2008 httpwwwtinyosnet

[11] M Higashi T Yamamoto Y Hayase T Ishio and K InoueldquoAn effective method to control interrupt handler for datarace detectionrdquo in International Workshop on Automation ofSoftware Test (ACM rsquo10) pp 79ndash86 May 2010

[12] Z Lai S C Cheung and W K Chan ldquoInter-context control-flow and data-flow test adequacy criteria for nesC applicationsrdquoin Proceedings of the 16th ACM SIGSOFT International Sympo-sium on the Foundations of Software Engineering (SIGSOFT rsquo08)pp 94ndash104 November 2008

[13] J Regehr ldquoRandom testing of interrupt-driven softwarerdquo inPro-ceedings of the 5th ACM International Conference on EmbeddedSoftware (EMSOFT rsquo05) pp 290ndash298 September 2005

[14] Y Zhou X Chen M R Lyu and J Liu ldquoSentomist Unveil-ing transient sensor network bugs via symptom miningrdquoin Proceedings of the 30th IEEE International Conference onDistributed Computing Systems (ICDCS rsquo10) pp 784ndash794 June2010

[15] G Gracioli and S Fischmeister ldquoTracing interrupts in embed-ded softwarerdquo in Proceedings of the ACM SIGPLANSIGBEDConference on Languages Compilers and Tools for EmbeddedSystems (LCTES rsquo09) vol 44 pp 137ndash146 2009

[16] V Sundaram P Eugster and X Zhang ldquoLightweight tracingfor wireless sensor networks debuggingrdquo in Proceedings of the4th International Workshop on Middleware Tools Services andRun-Time Support for Sensor Networks (ACM rsquo09) pp 13ndash18December 2009

[17] O K Ha and Y K Jun ldquoMonitoring of programs with nestedparallelismusing efficient thread labelingrdquo International JournalFo U- and E- Service Science and Technology vol 4 pp 21ndash362011

[18] Y C Kim S S Jun and Y K Jun ldquoA tool for space-efficient on-the-fly race detectionrdquo International Journal of DatabaseTheoryand Application vol 4 pp 25ndash38 2011

[19] S Savage M Burrows G Nelson P Sobalvarro and T Ander-son ldquoEraser a dynamic data race detector for multithreadedprogramsrdquo ACM Transactions on Computer Systems vol 15 no4 pp 391ndash411 1997

[20] A Jannesari and W F Tichy ldquoOn-the-fly race detection inmulti-threaded programsrdquo in Proceedings of the 6th Workshop

on Parallel and Distributed Systems Testing Analysis andDebugging (PADTAD rsquo08) July 2007

[21] D Gay M Welsh P Levis E Brewer R Von Behren and DCuller ldquoThe nesC language a holistic approach to networkedembedded systemsrdquo inACMSIGPLANConference on Program-ming Language Design and Implementation pp 1ndash11 June 2003

[22] J Regehr and N Cooprider ldquoInterrupt verification via threadverificationrdquo Electronic Notes in Theoretical Computer Sciencevol 174 no 9 pp 139ndash150 2007

[23] B L Titzer D K Lee and J Palsberg ldquoAvrora scalable sensornetwork simulation with precise timingrdquo in Proceedings ofthe 4th International Symposium on Information Processing inSensor Networks (IPSN rsquo05) pp 477ndash482 April 2005

[24] B L Titzer and J Palsberg ldquoNonintrusive precision instru-mentation of microcontroller softwarerdquo in Proceedings of theConference on Languages Compilers and Tools for EmbeddedSystems (Lctes rsquo05) vol 40 pp 59ndash68 2005

[25] J Hil R Szewczyk A Woo S Hollar D Culler and K PisterldquoSystem architecture directions for networked sensorsrdquo inProceedings of the 9th International Conference on ArchitecturalSupport for Programming Languages and Operating Systems(ASPLOS rsquo00) pp 93ndash104 2000

[26] GMTchamgoue KHKim andYK Jun ldquoTesting anddebug-ging concurrency bugs in event-driven programsrdquo InternationalJournal of Advanced Science and Technology vol 40 pp 55ndash682012

[27] GM Tchamgoue O K Ha K H Kim and Y K Jun ldquoA frame-work for on-the-fly race healing in ARINC-653 applicationsrdquoInternational Journal of Hybrid Information Technology vol 4pp 1ndash12 2011

[28] A Dinning and E Schonberg ldquoDetecting access anomaliesin programs with critical sectionsrdquo in Proceedings of theACMONR Workshop on Parallel and Distributed Debugging(PADD rsquo91) vol 26 pp 85ndash96 1991

[29] L Lamport ldquoTime clocks and the ordering of events in adistributed systemrdquo Communications of the ACM vol 21 no7 pp 558ndash565 1978

[30] Atmel inc ATmega128 Datasheet 2011 httpwwwatmelcomImagesdoc2467pdf

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 6: Research Article Verification of Data Races in Concurrent ...downloads.hindawi.com/journals/ijdsn/2013/953593.pdf · photoTask I20 W 11 R 12 W 13 W 24 R 15 T 2 J12 T 1 F : A POEG

6 International Journal of Distributed Sensor Networks

Now let us assume that 119905119894and 119905

119895belong to tasks

Naturally 119905119894and 119905119895are ordered and the labeling algorithms of

Algorithm 2 will set 119905119905119894= 119905119905119895= 119905119903119906119890 meaning that 119905

119894

ℎ119887

997888997888rarr 119905119895

On the other hand if we assume that not(119905119895

V999492999484 119905119894) then 119905

119895

must be part of a given interrupt handler 119879119896and either 119905

119894has

disabled all interrupts to set 119888119904119894= 119905119903119906119890 or the thread identifier

119905119894119889119895= 119896 of 119905

119895is in the interrupt set 119894119904119890119905

119894of 119905119894 according to

Definition 4 and Lemma 6 Therefore we can conclude that119905119894

ℎ119887

997888997888rarr 119905119895

Any two labeled thread segments 119905119894and 119905119895are therefore

concurrent if they do not satisfy the happens-before relationas stated by Theorem 7 which can formally be rewritten asfollows

119905119894

ℎ119887

997888997888rarr 119905119895lArrrArr

119905119905119894= 119905119905119895= 119905119903119906119890

or(119905119905119895= 119891119886119897119904119890) and (119888119904

119894= 119905119903119906119890 or 119905119894119889

119895isin 119894119904119890119905119894)

(1)

By comparing based on Theorem 7 the labels of thethread segments in the POEG represented by Figure 2 wecan easily obtain that the thread segments of 119868

20 1198941 and 119894

2

are concurrent with other thread segments 1199051 1199053 1199054 and

1198951 Therefore the write accesses in 119894

1and 1198942conflict with

the other accesses in 1199053 leading to four data races 119882

11-11988213

11988211-1198771511988224-11988213 and119882

24-11987715 However since the accesses

11988211and119882

23 respectively from 119894

1and 1198942 represent the same

write access the set of data races to be reported for an effectivedebugging process should be reduced to only 119882

11-11988213

and11988211-11987715 By disabling all interrupts 119905

2enforces the happens-

before relation with other thread segments Although 1198791is

posted by 11986820 they are still concurrent Also the thread

segment 1198951is concurrent with 119894

1 1198942 1199051 1199053 and 119905

4

4 Precise Detection Protocol

The race detection protocol monitors memory accesses bymaintaining a special data structure called access history intowhich are stored the labels of prior accesses to each sharedvariable Our detection protocol stores in each entry of theaccess history of a shared variable a unique label representingthe thread segment that accessed the variable as describedin Section 3 At each access to a shared variable data racesare reported by analyzing based onTheorem 7 the happens-before relation between the current thread segment and thelabels of prior thread segments stored in the access historyTherefore the access history for each shared variable hasto be efficiently organized in order to reduce the overheadgenerated by the detection process

Our precise detection protocol maintains an access his-tory consisting of two sets for each shared variable a Readset and a Write set to respectively store labels of threadsegments that read and write the variable We use 119860119867(119909)

to designate the access history of a shared variable 119909 Thus119860119867(119909) sdot 119877119890119886119889 and 119860119867(119909) sdot 119882119903119894119905119890 refer to the read set andthe write set of 119860119867(119909) respectively The algorithms used bythe detection protocol are presented in Algorithm 3 In thesealgorithms we assume that the variable 119871119887 always contains

20

20

20

20

t1

t2

t3

t4

12

12

lowast

lowast

i1⟨0 20 0⟩⟨20⟩

⟨mdash⟩⟨1 1 1⟩

⟨mdash⟩⟨1 1 0⟩

⟨mdash⟩⟨1 1 0⟩

⟨0 12 0⟩⟨12⟩

⟨0 20 0⟩⟨20⟩

⟨mdash⟩⟨1 2 0⟩

I20

W11

R12

W13

W24

R15

T2

J12

T1

j1

i2

Figure 2 An example labeled POEG for an execution instance ofthe program in Algorithm 1

the label of the current thread segment accessing a sharedvariable and that 119901119871119887 represents any prior label stored in theaccess history

When a read access occurs to a shared variable 119909the detection protocol calls the ah read() function ofAlgorithm 3(a) that then reports data races by analyzing thehappens-before relation between the label of the accessingthread segment and the labels of prior accesses stored in119860119867(119909) sdot119882119903119894119905119890 Similarly upon a write access to a shared vari-able the monitoring function ah write() of Algorithm 3(b)reports data races by looking into both 119860119867(119909) sdot 119877119890119886119889 and119860119867(119909) sdot 119882119903119894119905119890 We assume the existence of a functionOrdered() that based on Theorem 7 decides whether athread segment 119905

119894happens before another thread segment 119905

119895

by comparing their labelsIn order to maintain a reduced size access history

accesses from critical sections that concern all interruptswithin a task are not inserted in the access history sincethey will never run into a data race with other accessesfrom interrupt handlers Also only the first read and writeaccesses to a shared variable from the same thread segmentare kept into the access history Subsequent similar accessesto the same shared variable from the same thread segmentare filtered outThis filtering operations are performed by theisFiltered() function of Algorithm 3(c) that is called by bothah read() and ah write() before any other action

After data races have been reported with the currentaccess the function maintain ah() of Algorithm 3(d) is usedfor keeping the size of the access history balanced Thisfunction replaces some previous accesses in the access historywith the current access Thus accesses from different tasksare considered from the same thread since tasks by defini-tion are all ordered by happens-before Similarly accesses

International Journal of Distributed Sensor Networks 7

(1) ah read(Label Lb Access History AH Variable x)(2) if (isFiltered(Lb AH(x)Read)) then return(3) for all labels pLb in AH(x)Write do(4) if (Not Ordered(Lb pLb)) then(5) report race(x pLb Lb)(6) end for(7) maintain ah(Lb AH(x)Read)

(a) Monitoring Protocol for Read Accesses(1) ah write(Label Lb Access History AHVariable x)(2) if (isFiltered(Lb AH(x)Write)) then return(3) for all labels pLb in(4) AH(x)Write cup AH(x)Read do(5) if (Not Ordered(Lb pLb)) then(6) report race(x pLb Lb)(7) end for(8) maintain ah(Lb AH(x)Write)

(b) Monitoring Protocol for Write Accesses(1) isFiltered(Label Lb Access History AH)(2) if (Lbcs = true) and (Lbtt = true)(3) then return true(4) if (any access with a label pLb exists in AH(5) such that pLb = Lb) then return true(6) else(7) return false

(c) Filtering Redundant Accesses(1) maintain ah(Label Lb Access History AH)(2) if (Lbtt = true) then(3) delete all entries in AH with label pLb(4) such that pLbtt = true(5) else(6) delete all entries in AH with label pLb such(7) that pLbtt = Lbtt and pLbtid = Lbtid(8) end if(9) add ah(Lb AH)

(d) Maintaining Access History

Algorithm 3 Labeling algorithms

from different instances of the same interrupt handler areconsidered from a single instance of the interrupt handlerThis property is already enforced by the labeling scheme thatreuses the same label for all instances of the same interrupthandler Again we assume the existence of another functionadd ah() which is able to insert the label of the current threadsegment into the appropriate set of the access history given ashared variable

Table 5 shows the different states of an access historyfor detecting data races with our protocol in the exampleprogram of Figure 2 where accesses to the shared variablemsg appear in the first column When the first access 119882

11

occurs the label ⟨0 20 0⟩⟨20⟩ of the thread segment 1198941

is simply added into the access history and no data racereported as there is no other access prior to 119882

11 The next

access 11987712 from the thread segment 119905

2is filtered out because

1199052that belongs to task 119879

1has disabled all interrupts The

following access 11988213

occurs on 1199053with interrupts enabled

Thus a data race11988213-11988211is reported by comparing the label

of 1199053 ⟨1 1 0⟩⟨minus⟩ with the label of 119894

1previously stored into the

access history Afterwards the label of 1199053is inserted into the

write set of the access history The thread segment 1199053is then

preempted by 1198942 but the access 119882

24is filtered out because

1198942which is a new instance of the interrupt 119868

20 also shares the

same label ⟨0 20 0⟩⟨20⟩ with 1198941which previously performed

a write and whose label is already in the access history Afterthe execution of 119894

2 1199053resumes to perform the read access

11987715 Another data race 119877

15-11988211

is reported and the label of1199053 ⟨1 1 0⟩⟨minus⟩ stored into the read set of the access history

Although the program received two instances of the ADCinterrupt our protocol precisely reported only the set of twodata races needed for effectively debug the program Thisshows that our detection protocol does not depend on thenumber of interrupts received at runtime which can be veryhigh

Although it keeps only a reduced set of labels in the accesshistory our precise detection protocol even with a minimalnumber of interrupts still guarantees to verify the existenceof data races in a program For this purpose the protocol isdesigned to report without false positives at least one datarace for each shared variable involved in a data race

8 International Journal of Distributed Sensor Networks

Theorem 8 Given an interrupt-driven program 119875 the detec-tion protocol reports a data race over a shared variable 119909 if andonly if there exists a data race over 119909 in an execution instanceof 119875

Proof We will first prove that if a data race is reported on ashared variable 119909 by the detection protocol then there alsoexists a data race in an execution instance of the program forthe same shared variable 119909 For this we assume a snapshotof the access history 119860119867(119909) of the shared variable 119909 at agiven time and the current access event 119886

119894to 119909 from a thread

segment 119905119894that owns a label we refer to as 119871119887

119894in an execution

instance of the program 119875 We can then distinguish twodifferent cases

Case A 119860119867(119909) is empty and there is obviously no data racefor 119909 In this case the current access 119886

119894is the only access to

119909 from the program 119875 and 119871119887119894will be added by the function

add ah() into119860119867(119909)sdot119877119890119886119889 if 119886119894is a read access or into119860119867(119909)sdot

119882119903119894119905119890 if it is a write access

Case B 119860119867(119909) is not empty Data races are reported usingthe ah read() function if 119886

119894is a read access or the ah write()

in case of a write access In both cases the happens-beforerelation is analyzed between the label 119871119887

119894of 119905119894and previous

labels stored in 119860119867(119909) according to Theorem 7 The outputsof these analyses can be classified into two categories

(1) There is no data race between 119886119894and119860119867(119909) therefore

all previous accesses stored in 119860119867(119909) happenedbefore 119886

119894meaning that no thread segment prior to

119905119894in 119875 is concurrent with 119905

119894 they are all ordered by

happens-before Thus for all 119886119895isin 119905119895and 119871119887119895isin 119860119867(119909)

119905119895

ℎ119887

997888997888rarr 119905119894 In which case 119875 also has no data race

(2) There are data races between 119886119894and prior accesses

in 119860119867(119909) this means that there are some labels in119860119867(119909) belonging to some prior thread segments thatare not ordered with 119905

119894 Concretely there are thread

segments in 119875 that are concurrent with 119905119894 Formally

exist119905119895isin 119875119871119887

119895isin 119860119867(119909) 119905

119894 119905119895Therefore the execution

instance of 119875 also contains data races over 119909

Now we show that if there exists a data race over a sharedvariable 119909 in an execution instance of the program 119875 thenour detection protocol also reports a data race over 119909 Usingthe contrapositive we will prove that if the detection protocoldoes not report any data race the program 119875 also does notcontain any for a shared variable 119909 We again consider asnapshot of the access history 119860119867(119909) for a shared variable119909 at a given time We also assume that the detection protocolreported no data races for the shared variable 119909 This meansthat 119860119867(119909) is either empty and there is clearly no data racesince there is no access to 119909 from 119875 yet or contains onlylabels of thread segments that are all ordered by the happens-before relation in which case there is no concurrent threadsegment in119875 which also implies that119875 has no data races over119909 Therefore we can formally write for all 119905

119894 119905119895isin 119875 and 119871119887

119894

119871119887119895isin 119860119867(119909) 119905

119894

ℎ119887

997888997888rarr 119905119895or 119905119895

ℎ119887

997888997888rarr 119905119894 Thus 119875 also contains no

data race for 119909

Table 1 Efficiency of the precise detection protocol

Space TimeNew label comparison Per access

O(1198811198792) O(1) O(LogT) O(TLogT)

5 Theoretical Analysis

On-the-fly data race detection techniques are known to havehigh runtime overhead that is generally provoked by the timespentmonitoring accesses to shared variables and to comparethread labels In this section we theoretically analyze theefficiency of our technique which depends on119881 the numberof shared variables and 119879 the total number of interruptsin the underlying system plus one the number of tasksIn general 119879 is very small since the number of interruptshandled by a system is also kept small For example thepopular Atmel ATmega128 [30] processor supports only amaximum of 34 interrupts Also because of the implicithappens-before relation that exists between all tasks they areviewed as a single task by the detection protocol

Table 1 presents the space and time complexities of ourlabeling and detection techniques The access history usedby our technique stores from each thread segment at mosttwo accesses for each shared variable Each label in the accesshistory has a worst-case size of 119874(119879) that represents thesize of its interrupt set Therefore the total labeling spacerequirement for each shared variable is 119874(1198792) Thus thetotal space complexity of our technique is 119874(1198811198792) Howeverbecause the number of interrupts is generally a reasonableinteger number the space complexity of our technique canfurther be reduced to 119874(119881)

A new label is always created in a constant amount of timeby our labeling algorithms Interrupts are generally givennumbers in a sequence order Therefore a binary searchtechnique is applied to locate values in interrupt sets whichcontain only interrupt numbers Thus the time complexityfor comparing any two labels is 119874(Log119879) in the worst-caseand 119874(1) in most cases where the interrupt sets are emptyAt each access to a shared variable the total time complexityto report data races with the previous accesses stored in theaccess history is then119874(119879Log119879)The labels and the detectionprotocol do not depend on the total number of interruptsreceived at runtimemaking our technique highly scalable andpractical

6 Experimental Results

We empirically evaluated the preciseness and efficiency ofour technique using a set of well-known programs providedwith TinyOS [10] This section details the implementationchallenges and presents the analysis results of the experimen-tation

61 Implementation We implemented a prototype of ourdata race detection tool for interrupt-driven programs callediRace based on the concurrency model of TinyOS [21] andits nesC language [25] We implemented iRace on top

International Journal of Distributed Sensor Networks 9

Table 2 Characteristics of the benchmark applications from TinyOS 210 [10]

Applications LOC Int SV Brief description6LowPanCli 19116 4 9 Implements a command line interface using the 6lowpan stackAnthiTheft 27421 5 4 Detects and reports a stolen mote by checking some sensorsBaseStation 14905 6 13 Sends and receives simple active messages on the radio and serial linksBlink 3688 1 0 Periodically blinks the three LEDs of a moteMultihopOscilloscope 24310 9 8 Periodically samples and broadcasts sensor data to a base stationOscilloscope 11686 5 5 Periodically samples sensor data and broadcasts a message every 10 timesPowerup 2352 0 0 Turns on the red LED when the mote powers upRadioCountToLeds 11165 4 3 Periodically samples sensors and displays the last 3 bits on its LEDsRadioSenseToLeds 11472 5 2 Periodically update a counter and displays the last 3 bits on its LEDsSense 5743 2 0 Periodically takes sensor readings and displays the values on the LEDs

of the Avrora [23] simulation and analysis framework thatprovides a time accurate simulator different analysis toolsknown asmonitors and a binary instrumentation engine [24]for programs written for AVR microcontrollers and sensornetworks All these features make Avrora just suitable forour tool Simulation is commonly used in the developmentcycle of embedded software because target systems generallylack conventional inputoutput interfaces With iRace thefunctionalities of Avrora are extendedwith a precise data racedetection tool that can easily be loaded as one of Avrorarsquosmonitors

As shown in Figure 3 iRace consists of two maincomponents the Instrumenter which exploits the Avrorainstrumentation engine [24] and the core Race Detectoritself The Instrumenter monitors task and interrupts activ-ities through its iMonitor component and provides entrypoints for the labeling routines Instrumentation points areinserted at the beginning of each task at the start andexit of each interrupt handler and also before interruptsare either enabled or disabled Similarly accesses to sharedmemory locations are monitored using the AccessMonitorcomponent that also calls routines of the detection proto-col The AccessMonitor instruments before read and writeaccesses to shared variables The Race Detector is composedof labeling routines that generate and manage labels for eachactive thread segment at runtime and detection routines formaintaining the access history and reporting data races on-the-fly as shared variables are manipulated

In TinyOS [21 25] interrupt handlers and tasks all runto completion unless preempted by other interrupt handlersThus only a single thread runs at any point of time givingus the opportunity to use only a single variable to maintainthe active thread label at detection time However whenpreemption happens the label of the preempted threadsegment must be saved and restored again when the activeinterrupt handler exits Otherwise there will be no way toknow about the concurrency information of the preemptedthread segment when it gains back the execution control Inorder tomanage the preemption we use a stack data structurewhere labels can be pushed and restored from since interruptscan be nested This provides a simple but powerful way tohandle preemption with negligible overhead

Our labeling scheme needs to uniquely identify eachinterrupt handler whenever it is invoked and each taskwhen it starts its execution Each interrupt handler is easilyidentified by the interrupt number of the interrupt it servesFor task we chose to generate a unique task identifier bysimply maintaining a counter that increases at each timea task starts its execution Thus each task is attributed aunique identifier thanks to the fact that once posted TinyOStasks run in a first-in first-out manner and cannot preempteach other By manipulating only binary files iRace doesnot require any source file nor source code annotation toeffectively detect data races However compiling the programwith debugging information may help iRace generate moredetailed debugging information on data races it reports

62 Environment We implemented and evaluated our tech-nique on a system that runs the kernel 30 of the Linuxoperating system using an Intel dual-core 320GHz and 4GBof mainmemory On this machine was installed TinyOS 210[10] along with the nesC compiler 131 the GNU C compiler46 the Java Runtime Environment 17 and the latest Avroraframework version 17115

Table 2 presents the characteristics and a brief descriptionof the benchmark applications selected for our evaluationfrom the TinyOS 210 repository [10] These applications arefreely distributed with each version of TinyOS The columnldquoLOCrdquo indicates the number of line of C codes generatedby the nesC compiler after compilation while the columnldquoIntrdquo contains the number of interrupts handled by eachapplicationThe number of shared variables for each programappears in column ldquoSVrdquo It is worthmentioning that this num-ber represents only the shared variables in the source code ofeach program The exact number of variables shared by eachapplicationmay finally depend on the number of componentsit shares with the operating system itself Applications writtenin nesC share TinyOS components by wiring themselvesto these components increasing therefore their number ofshared resources We compiled each application for theMica2 platform and measured the average runtime overheadover five executions The Mica2 motes are sensor networknodes based on the ATmega128 [30] microcontrollers Weconfigured Avrora to simulate a sensor network during 10

10 International Journal of Distributed Sensor Networks

iRace

and instrumentationframework

iMonitor

AccessMonitor

Race detector

Labeling routines

Detection protocolroutines

Binarycode Data

races

Avrora simulation

Figure 3 The Architecture of the Data Race Detection Tool

Table 3 Reported data races on the benchmark applications

Applications Data races Verif races Time (Sec) Memory (Kb)Original iRace Original iRace

6LowPanCli 0 0 269 357 1936960 2168160AnthiTheft 15 15 292 457 1944400 2320000BaseStation 0 0 279 376 1912240 2148240Blink 0 0 161 180 1887280 1949600MultihopOscilloscope 0 0 278 400 1939600 2196960Oscilloscope 13 13 179 328 1909360 2201760Powerup 0 0 151 186 1873600 1879920RadioCountToLeds 17 17 196 1255 1942880 2326320RadioSenseToLeds 15 15 181 1305 1944480 2298240Sense 0 0 158 192 1866400 2050560

seconds allowing each application to run exactly for 73728000instruction cycles We set to five the number of nodes in thesimulated sensor network

63 Evaluation For data race detectionwe instructedAvrorato simulate a sensor network with five nodes while thebenchmark applications are monitored by iRace Table 3represents the results of the evaluation of iRace on thebenchmark applications in its column ldquoData Racesrdquo whichcontains the number of shared variables involved in adata race for each application Our tool reported no datarace for all applications except for four AnthiTheft with15 shared variables Oscilloscope with 13 shared vari-ables RadioCountToLeds with 17 shared variables andRadioSenseToLeds with 15 shared variables Surprisinglymost of the reported data races reside inside some sharedsystem components of TinyOS itself The reported data racesare caused by two different interrupts namely the serialperipheral interface (SPI) interrupt and the timer interruptAlso we carefully analyzed the reported data races andinspected the source code of the applications for a manualverification process as shown in the column ldquoVerified racesrdquoof Table 3

In the AnthiTheft application for example which isused to detect and report a stolen mote by checking the levelof the light or the acceleration data races were reported over

15 shared variables Most of the reported data races are how-ever due to the fact that the SPI interrupt handler can preemptsome functions or tasks owned by the underlying radio stackThis implies that if a message is being received or sent whena SPI interrupt occurs shared variables like radioState orcount in the component CC1000SendReceiveP might becorrupted However all shared variables involved in a datarace do not directly belong to AnthiTheft but to functionsof the CC1000 radio stack in use in Mica2 motes Similardata races are also reported in RadioCountToLeds andRadioSenseToLeds since they all access the radio stack

Another data race reported both in RadioCountToLedsand RadioSenseToLeds and worthy to be commentedconcerns the shared variable reqResId inside thecomponent ArbiterP (Recently this data race hasalso been pointed out and a patch discussed in theTinyOS Forum) This data race occurs when the functionimmediateRequest() called within the function read ofthe component immediateRequest is preempted by a SPIinterrupt handler An excerpt of this function is presentedin Algorithm 4 At line 2 of Algorithm 4 the handle of aresource is requested and eventually locked out if grantedHowever if a SPI interrupt is received somewhere afterline 2 a data race will occur and the resource handlerstored in reqResId (line 6 Algorithm 4) may eventually beoverwritten by the SPI interrupt handler Also other shared

International Journal of Distributed Sensor Networks 11

(1) async command error t ResourceimmediateRequest [uint8 t id] ( ) (2) signal ResourceRequested immediateRequested [resId] ( )(3) atomic (4) if (state == RES CONTROLLED) (5) state = RES IMM GRANTING(6) reqResId = id(7)

(8) else return FAIL(9)

(10) signal ResourceDefaultOwner immediateRequested( )(11) sdot sdot sdot (12)

Algorithm 4 An excerpt of the request function from the component ArbiterP

(1) command error t Send send[uint8 t clientId] (message t lowastmsg(2) uint8 t len) (3) sdot sdot sdot (4) queue [clientId] msg = msg(5) call Packet setPayloadLength (msg len)(6) sdot sdot sdot (7)

Algorithm 5 An excerpt of the send function from the component AMSend

variables in ArbiterP like state can be corrupted as wellby the SPI interrupt handler

In the case of Oscilloscope which is a data collectionapplication that periodically samples the default sensor ofa mote and broadcasts a message over the radio every 10readings the reported data races are similar to those reportedin AnthiTheft Moreover our results suggest that if a SPIinterrupt arrives when a message is being transmitted viathe send() function of the AMSend interface the sharedbuffer sendBuf used by Oscilloscope to store incomingmessages will be updated and a corruptedmessage deliveredThe extract of code from the function Send presentedin Algorithm 5 shows that the variable msg which inthis case is a pointer to the shared variable sendBuf inOscilloscope can be updated at any time if the functionsend() is preempted because operations are not performedin a critical section This data race is somehow similar to theone we described in the example program of Algorithm 1Wealso find in Oscilloscope some of the data races reportedin the other applicationsThe results in Table 3 show that ourtool precisely detects data races on-the-fly in interrupt-drivenprograms without any false positives

64 Detection Overhead In order to evaluate the impactof our technique on the monitored programs we config-ured Avrora to simulate a single node sensor network Wethen measured the average runtime overhead and memoryspace required by each application of the benchmark ofTable 2 to execute 73728000 instruction cycles The totaloverhead of our technique mainly comes from actions of theinstrumentation engine the labeling routines and the race

Exec

utio

ns ti

me (

s)

14

12

10

8

6

4

2

0

TinyOS 210 programsOriginalWith iRace

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 4 Average runtime overhead

detection engine which also has the responsibility to createandmaintain the access historiesThe average execution timeand memory consumption collected for each application arepresented in Table 3 The columns ldquoOriginalrdquo and ldquoiRacerdquorepresent the execution time and memory consumption foreach application without and with iRace respectively

Figure 4 which plots the data from the column ldquoTimerdquoof Table 3 compares the average execution times of each

12 International Journal of Distributed Sensor Networks

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

TinyOS 210 programsOriginalWith iRace

Figure 5 Average memory consumption

application for five runs with and without iRace The captionldquoOriginalrdquo represents the native execution of the applica-tions without iRace The results of Figure 4 suggest thatthe average runtime overhead of our tool is only about1x for all applications except for RadioCountToLeds andRadioSenseToLeds that reaches 6x the original executiontime These two applications sample and display data at highfrequency rate (eg 4Hz in our case) and therefore arelikely to receive a higher number of interrupts at runtimeThis low runtime overhead is achieved by exploiting the factthat shared variables are stored by Avrora at program loadingtime Instead of monitoring all memory accesses to filterthose targeting shared variables iRace precisely instrumentsonly accesses to the well-known set of shared memory loca-tions obtained from the Avrora loader This technique allowsthe tool to drastically cuts down its monitoring overhead andto increase its scalability

Likewise wemeasured the averagememory consumptionfor each application during five runs The collected data arerepresented in Figure 5 The overall average memory con-sumption of the tool is very small at just about 20 above theoriginal memory requirement even for high frequency appli-cations like RadioCountToLeds and RadioSenseToLedsthanks to our labeling scheme that reuses the same labelfor different instances of the same interrupt handler and thedetection protocol that maintains only a reduced size accesshistoryThis result corroborates the theoretical analysis of ourtechnique as summarized in Table 1 The low runtime over-head and memory consumption of our on-the-fly techniquemake it very scalable and suitable even for interrupt-drivenprograms that handle interrupts at high frequency rates

65 Labeling Overhead Since the race detection subcompo-nent of iRace is in charge of implementing the labeling and

Table 4 Average labeling overhead

Applications Time (Sec) Memory (Kb)6LowPanCli 313 2101760AnthiTheft 308 2148880BaseStation 309 2077680Blink 174 1964640MultihopOscilloscope 313 2141920Oscilloscope 249 2092640Powerup 176 1894720RadioCountToLeds 218 2167440RadioSenseToLeds 224 2140080Sense 184 2011040

Table 5The states of an access history for detecting data races withour protocol considering the example of Figure 2

119886119894

Read Write Races11988211

⟨0 20 0⟩ ⟨20⟩

11987712

⟨0 20 0⟩ ⟨20⟩

11988213

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11988213-11988211

11988224

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩11987715

⟨1 1 0⟩ ⟨mdash⟩ ⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11987715-11988211

the detection routines it was legitimate to analyze the impactof the labeling scheme alone on the monitored programsat runtime For this purpose we deactivated the detectionengine of iRace before running it again on the benchmarkapplications We collected the average execution time andmemory space consumed by each application after five runsas shown in Table 4 The simulation was done using a singlenode network

Figure 6 compares the original execution time of eachprogram from Table 3 with the execution time of the labelingengine as presented in the column ldquoTimerdquo of Table 4 Thecomparison shows that the average runtime overhead ofiRace with just the labeling engine activated is only about04x in the worst-case This represents only a small fractionof the average runtime overhead of 6x imposed by iRace asdiscussed in Section 64Themain reason of this performancecan be found at source level When browsing the source codeof the benchmark applications we came to understand thatprogrammers generally disable and enable all the interruptsat once instead of focusing on particular interrupts In thiscase the size of each label generated by the labeling enginenever approaches its worst-case of 119874(119879) meaning that thecomparison overhead of labels is also reduced This is veryimportant because as discussed in Section 64 althoughthe size of labels does not depend on the total number ofinterrupts received at runtime this number still has impacton the overall detection overheadThus we can conclude thatthe labeling engine of iRace incurs a very low overhead atruntime making it a highly scalable tool

We also compared the memory consumption of thelabeling engine with the memory requirement of the original

International Journal of Distributed Sensor Networks 13

TinyOS 210 programs

4

35

3

25

2

15

1

05

0

Exec

utio

n tim

e (s)

With iRace labeling engineOriginal

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 6 Average runtime overhead of the labeling engine

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

With iRace labeling engineOriginal

TinyOS 210 programs

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 7 Average memory consumption by the labeling engine

program execution presented inTable 3The results plotted inFigure 7 show that when the labeling engine alone is runningiRace consumes only about 12 more memory on averageHowever during the detection phase the overall averagememory consumption of iRace is only about 20 Theseresults also tell us that the detection engine does not increasethat much the memory consumption of iRace This is alsoreasonable because we only disabled the detection engine ofiRace but did not remove the resources it used

7 Conclusions

Asynchronous interrupt handling introduces fine-grainedconcurrency into interrupt-driven programs making themprone to data races In this paper we presented an efficientand scalable on-the-fly technique to precisely detect dataraces without false positives in such programs Our techniqueintroduced a lightweight labeling scheme for maintainingconcurrency information for the main program and each ofits interrupt handlers and a precise detection protocol thatguarantees to verify the existence of data races in a programby reporting at least one data race for each shared variableinvolved in a data race

We implemented a prototype of our technique as a racedetection tool called iRace on top of the Avrora simulationframework An evaluation of iRace revealed the existence ofdata races in some programs distributed with TinyOS andin some TinyOS components while incurring a slowdownof only about 6 times on average in the worst-case with anaverage memory consumption of only about 20 above theoriginal memory requirements of the program Moreoverthis evaluation also proved that the labeling engine alonegenerates an average runtime overhead of only about 04xwith an average memory consumption of only about 12relatively to the original program execution These resultscoupled with a theoretical analysis showed that the proposedtechnique is highly scalable and practical even for largeinterrupt-driven programs

Detecting data races that occur first in a program isan important issue since it may make other data races todisappear It is therefore interesting to detect this class of dataraces in interrupt-driven programs to facilitate the debuggingprocess Also we may be interested in extending our work tosupport multithreaded programs that handle interrupts andalso to detect data races in low-level kernel codes

Acknowledgment

This research was financially supported by the Ministryof Education (MOE) and National Research Foundation ofKorea (NRF) through the Human Resource Training Projectfor Regional Innovation and BK21+ program

References

[1] U Banerjee B Bliss Z Ma and P Petersen ldquoA theory of datarace detectionrdquo in Proceedings of the Workshop on Parallel andDistributed Systems Testing and Debugging (PADTAD rsquo06) pp69ndash78 July 2006

[2] R H B Netzer and B P Miller ldquoWhat are race conditionsSome issues and formalizationsrdquo ACM letters on programminglanguages and systems vol 1 no 1 pp 74ndash88 1992

[3] N G Leveson and C S Turner ldquoInvestigation of theTherac-25accidentsrdquo Computer vol 26 no 7 pp 18ndash41 1993

[4] USA Canada Final Report on the August 14 2003 Blackout inthe United States and Canada Causes and RecommendationsUS-Canada Power System Outage Task Force 2004

[5] J Erickson M Musuvathi S Burckhardt and K OlynykldquoEffective data-race detection for the kernelrdquo in Proceedings of

14 International Journal of Distributed Sensor Networks

the 9th Conference on Operating Systems Design and Implemen-tation (USENIX rsquo10) 2010

[6] B-K Lee M-H Kang K C Park J S Yi S W Yang andY-K Jun ldquoProgram conversion for detecting data races inconcurrent interrupt handlersrdquo Communications in Computerand Information Science vol 257 pp 407ndash415 2011

[7] L Effinger-Dean B Lucia L Ceze D Grossman and H JBoehm ldquoIFRit interference-free regions for dynamic data-racedetectionrdquo in Proceedings of the ACM International Conferenceon Object Oriented Programming Systems Languages and Appli-cations (OOPSLA rsquo12) vol 47 pp 467ndash484 2012

[8] C Flanagan and S N Freund ldquoFastTrack efficient and precisedynamic race detectionrdquo in Proceedings of the ACM Interna-tional Conference on Object Oriented Programming SystemsLanguages and Applications (OOPSLA rsquo09) vol 44 pp 121ndash1332009

[9] X Xie and J Xue ldquoAcculock accurate and efficient detection ofdata racesrdquo in Proceedings of the 9th International Symposium onCode Generation and Optimization (CGO rsquo11) pp 201ndash212 April2011

[10] TinyOSCommunity ForumTinyOSAnopen source operatingsystem for low-power wireless devices 2008 httpwwwtinyosnet

[11] M Higashi T Yamamoto Y Hayase T Ishio and K InoueldquoAn effective method to control interrupt handler for datarace detectionrdquo in International Workshop on Automation ofSoftware Test (ACM rsquo10) pp 79ndash86 May 2010

[12] Z Lai S C Cheung and W K Chan ldquoInter-context control-flow and data-flow test adequacy criteria for nesC applicationsrdquoin Proceedings of the 16th ACM SIGSOFT International Sympo-sium on the Foundations of Software Engineering (SIGSOFT rsquo08)pp 94ndash104 November 2008

[13] J Regehr ldquoRandom testing of interrupt-driven softwarerdquo inPro-ceedings of the 5th ACM International Conference on EmbeddedSoftware (EMSOFT rsquo05) pp 290ndash298 September 2005

[14] Y Zhou X Chen M R Lyu and J Liu ldquoSentomist Unveil-ing transient sensor network bugs via symptom miningrdquoin Proceedings of the 30th IEEE International Conference onDistributed Computing Systems (ICDCS rsquo10) pp 784ndash794 June2010

[15] G Gracioli and S Fischmeister ldquoTracing interrupts in embed-ded softwarerdquo in Proceedings of the ACM SIGPLANSIGBEDConference on Languages Compilers and Tools for EmbeddedSystems (LCTES rsquo09) vol 44 pp 137ndash146 2009

[16] V Sundaram P Eugster and X Zhang ldquoLightweight tracingfor wireless sensor networks debuggingrdquo in Proceedings of the4th International Workshop on Middleware Tools Services andRun-Time Support for Sensor Networks (ACM rsquo09) pp 13ndash18December 2009

[17] O K Ha and Y K Jun ldquoMonitoring of programs with nestedparallelismusing efficient thread labelingrdquo International JournalFo U- and E- Service Science and Technology vol 4 pp 21ndash362011

[18] Y C Kim S S Jun and Y K Jun ldquoA tool for space-efficient on-the-fly race detectionrdquo International Journal of DatabaseTheoryand Application vol 4 pp 25ndash38 2011

[19] S Savage M Burrows G Nelson P Sobalvarro and T Ander-son ldquoEraser a dynamic data race detector for multithreadedprogramsrdquo ACM Transactions on Computer Systems vol 15 no4 pp 391ndash411 1997

[20] A Jannesari and W F Tichy ldquoOn-the-fly race detection inmulti-threaded programsrdquo in Proceedings of the 6th Workshop

on Parallel and Distributed Systems Testing Analysis andDebugging (PADTAD rsquo08) July 2007

[21] D Gay M Welsh P Levis E Brewer R Von Behren and DCuller ldquoThe nesC language a holistic approach to networkedembedded systemsrdquo inACMSIGPLANConference on Program-ming Language Design and Implementation pp 1ndash11 June 2003

[22] J Regehr and N Cooprider ldquoInterrupt verification via threadverificationrdquo Electronic Notes in Theoretical Computer Sciencevol 174 no 9 pp 139ndash150 2007

[23] B L Titzer D K Lee and J Palsberg ldquoAvrora scalable sensornetwork simulation with precise timingrdquo in Proceedings ofthe 4th International Symposium on Information Processing inSensor Networks (IPSN rsquo05) pp 477ndash482 April 2005

[24] B L Titzer and J Palsberg ldquoNonintrusive precision instru-mentation of microcontroller softwarerdquo in Proceedings of theConference on Languages Compilers and Tools for EmbeddedSystems (Lctes rsquo05) vol 40 pp 59ndash68 2005

[25] J Hil R Szewczyk A Woo S Hollar D Culler and K PisterldquoSystem architecture directions for networked sensorsrdquo inProceedings of the 9th International Conference on ArchitecturalSupport for Programming Languages and Operating Systems(ASPLOS rsquo00) pp 93ndash104 2000

[26] GMTchamgoue KHKim andYK Jun ldquoTesting anddebug-ging concurrency bugs in event-driven programsrdquo InternationalJournal of Advanced Science and Technology vol 40 pp 55ndash682012

[27] GM Tchamgoue O K Ha K H Kim and Y K Jun ldquoA frame-work for on-the-fly race healing in ARINC-653 applicationsrdquoInternational Journal of Hybrid Information Technology vol 4pp 1ndash12 2011

[28] A Dinning and E Schonberg ldquoDetecting access anomaliesin programs with critical sectionsrdquo in Proceedings of theACMONR Workshop on Parallel and Distributed Debugging(PADD rsquo91) vol 26 pp 85ndash96 1991

[29] L Lamport ldquoTime clocks and the ordering of events in adistributed systemrdquo Communications of the ACM vol 21 no7 pp 558ndash565 1978

[30] Atmel inc ATmega128 Datasheet 2011 httpwwwatmelcomImagesdoc2467pdf

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 7: Research Article Verification of Data Races in Concurrent ...downloads.hindawi.com/journals/ijdsn/2013/953593.pdf · photoTask I20 W 11 R 12 W 13 W 24 R 15 T 2 J12 T 1 F : A POEG

International Journal of Distributed Sensor Networks 7

(1) ah read(Label Lb Access History AH Variable x)(2) if (isFiltered(Lb AH(x)Read)) then return(3) for all labels pLb in AH(x)Write do(4) if (Not Ordered(Lb pLb)) then(5) report race(x pLb Lb)(6) end for(7) maintain ah(Lb AH(x)Read)

(a) Monitoring Protocol for Read Accesses(1) ah write(Label Lb Access History AHVariable x)(2) if (isFiltered(Lb AH(x)Write)) then return(3) for all labels pLb in(4) AH(x)Write cup AH(x)Read do(5) if (Not Ordered(Lb pLb)) then(6) report race(x pLb Lb)(7) end for(8) maintain ah(Lb AH(x)Write)

(b) Monitoring Protocol for Write Accesses(1) isFiltered(Label Lb Access History AH)(2) if (Lbcs = true) and (Lbtt = true)(3) then return true(4) if (any access with a label pLb exists in AH(5) such that pLb = Lb) then return true(6) else(7) return false

(c) Filtering Redundant Accesses(1) maintain ah(Label Lb Access History AH)(2) if (Lbtt = true) then(3) delete all entries in AH with label pLb(4) such that pLbtt = true(5) else(6) delete all entries in AH with label pLb such(7) that pLbtt = Lbtt and pLbtid = Lbtid(8) end if(9) add ah(Lb AH)

(d) Maintaining Access History

Algorithm 3 Labeling algorithms

from different instances of the same interrupt handler areconsidered from a single instance of the interrupt handlerThis property is already enforced by the labeling scheme thatreuses the same label for all instances of the same interrupthandler Again we assume the existence of another functionadd ah() which is able to insert the label of the current threadsegment into the appropriate set of the access history given ashared variable

Table 5 shows the different states of an access historyfor detecting data races with our protocol in the exampleprogram of Figure 2 where accesses to the shared variablemsg appear in the first column When the first access 119882

11

occurs the label ⟨0 20 0⟩⟨20⟩ of the thread segment 1198941

is simply added into the access history and no data racereported as there is no other access prior to 119882

11 The next

access 11987712 from the thread segment 119905

2is filtered out because

1199052that belongs to task 119879

1has disabled all interrupts The

following access 11988213

occurs on 1199053with interrupts enabled

Thus a data race11988213-11988211is reported by comparing the label

of 1199053 ⟨1 1 0⟩⟨minus⟩ with the label of 119894

1previously stored into the

access history Afterwards the label of 1199053is inserted into the

write set of the access history The thread segment 1199053is then

preempted by 1198942 but the access 119882

24is filtered out because

1198942which is a new instance of the interrupt 119868

20 also shares the

same label ⟨0 20 0⟩⟨20⟩ with 1198941which previously performed

a write and whose label is already in the access history Afterthe execution of 119894

2 1199053resumes to perform the read access

11987715 Another data race 119877

15-11988211

is reported and the label of1199053 ⟨1 1 0⟩⟨minus⟩ stored into the read set of the access history

Although the program received two instances of the ADCinterrupt our protocol precisely reported only the set of twodata races needed for effectively debug the program Thisshows that our detection protocol does not depend on thenumber of interrupts received at runtime which can be veryhigh

Although it keeps only a reduced set of labels in the accesshistory our precise detection protocol even with a minimalnumber of interrupts still guarantees to verify the existenceof data races in a program For this purpose the protocol isdesigned to report without false positives at least one datarace for each shared variable involved in a data race

8 International Journal of Distributed Sensor Networks

Theorem 8 Given an interrupt-driven program 119875 the detec-tion protocol reports a data race over a shared variable 119909 if andonly if there exists a data race over 119909 in an execution instanceof 119875

Proof We will first prove that if a data race is reported on ashared variable 119909 by the detection protocol then there alsoexists a data race in an execution instance of the program forthe same shared variable 119909 For this we assume a snapshotof the access history 119860119867(119909) of the shared variable 119909 at agiven time and the current access event 119886

119894to 119909 from a thread

segment 119905119894that owns a label we refer to as 119871119887

119894in an execution

instance of the program 119875 We can then distinguish twodifferent cases

Case A 119860119867(119909) is empty and there is obviously no data racefor 119909 In this case the current access 119886

119894is the only access to

119909 from the program 119875 and 119871119887119894will be added by the function

add ah() into119860119867(119909)sdot119877119890119886119889 if 119886119894is a read access or into119860119867(119909)sdot

119882119903119894119905119890 if it is a write access

Case B 119860119867(119909) is not empty Data races are reported usingthe ah read() function if 119886

119894is a read access or the ah write()

in case of a write access In both cases the happens-beforerelation is analyzed between the label 119871119887

119894of 119905119894and previous

labels stored in 119860119867(119909) according to Theorem 7 The outputsof these analyses can be classified into two categories

(1) There is no data race between 119886119894and119860119867(119909) therefore

all previous accesses stored in 119860119867(119909) happenedbefore 119886

119894meaning that no thread segment prior to

119905119894in 119875 is concurrent with 119905

119894 they are all ordered by

happens-before Thus for all 119886119895isin 119905119895and 119871119887119895isin 119860119867(119909)

119905119895

ℎ119887

997888997888rarr 119905119894 In which case 119875 also has no data race

(2) There are data races between 119886119894and prior accesses

in 119860119867(119909) this means that there are some labels in119860119867(119909) belonging to some prior thread segments thatare not ordered with 119905

119894 Concretely there are thread

segments in 119875 that are concurrent with 119905119894 Formally

exist119905119895isin 119875119871119887

119895isin 119860119867(119909) 119905

119894 119905119895Therefore the execution

instance of 119875 also contains data races over 119909

Now we show that if there exists a data race over a sharedvariable 119909 in an execution instance of the program 119875 thenour detection protocol also reports a data race over 119909 Usingthe contrapositive we will prove that if the detection protocoldoes not report any data race the program 119875 also does notcontain any for a shared variable 119909 We again consider asnapshot of the access history 119860119867(119909) for a shared variable119909 at a given time We also assume that the detection protocolreported no data races for the shared variable 119909 This meansthat 119860119867(119909) is either empty and there is clearly no data racesince there is no access to 119909 from 119875 yet or contains onlylabels of thread segments that are all ordered by the happens-before relation in which case there is no concurrent threadsegment in119875 which also implies that119875 has no data races over119909 Therefore we can formally write for all 119905

119894 119905119895isin 119875 and 119871119887

119894

119871119887119895isin 119860119867(119909) 119905

119894

ℎ119887

997888997888rarr 119905119895or 119905119895

ℎ119887

997888997888rarr 119905119894 Thus 119875 also contains no

data race for 119909

Table 1 Efficiency of the precise detection protocol

Space TimeNew label comparison Per access

O(1198811198792) O(1) O(LogT) O(TLogT)

5 Theoretical Analysis

On-the-fly data race detection techniques are known to havehigh runtime overhead that is generally provoked by the timespentmonitoring accesses to shared variables and to comparethread labels In this section we theoretically analyze theefficiency of our technique which depends on119881 the numberof shared variables and 119879 the total number of interruptsin the underlying system plus one the number of tasksIn general 119879 is very small since the number of interruptshandled by a system is also kept small For example thepopular Atmel ATmega128 [30] processor supports only amaximum of 34 interrupts Also because of the implicithappens-before relation that exists between all tasks they areviewed as a single task by the detection protocol

Table 1 presents the space and time complexities of ourlabeling and detection techniques The access history usedby our technique stores from each thread segment at mosttwo accesses for each shared variable Each label in the accesshistory has a worst-case size of 119874(119879) that represents thesize of its interrupt set Therefore the total labeling spacerequirement for each shared variable is 119874(1198792) Thus thetotal space complexity of our technique is 119874(1198811198792) Howeverbecause the number of interrupts is generally a reasonableinteger number the space complexity of our technique canfurther be reduced to 119874(119881)

A new label is always created in a constant amount of timeby our labeling algorithms Interrupts are generally givennumbers in a sequence order Therefore a binary searchtechnique is applied to locate values in interrupt sets whichcontain only interrupt numbers Thus the time complexityfor comparing any two labels is 119874(Log119879) in the worst-caseand 119874(1) in most cases where the interrupt sets are emptyAt each access to a shared variable the total time complexityto report data races with the previous accesses stored in theaccess history is then119874(119879Log119879)The labels and the detectionprotocol do not depend on the total number of interruptsreceived at runtimemaking our technique highly scalable andpractical

6 Experimental Results

We empirically evaluated the preciseness and efficiency ofour technique using a set of well-known programs providedwith TinyOS [10] This section details the implementationchallenges and presents the analysis results of the experimen-tation

61 Implementation We implemented a prototype of ourdata race detection tool for interrupt-driven programs callediRace based on the concurrency model of TinyOS [21] andits nesC language [25] We implemented iRace on top

International Journal of Distributed Sensor Networks 9

Table 2 Characteristics of the benchmark applications from TinyOS 210 [10]

Applications LOC Int SV Brief description6LowPanCli 19116 4 9 Implements a command line interface using the 6lowpan stackAnthiTheft 27421 5 4 Detects and reports a stolen mote by checking some sensorsBaseStation 14905 6 13 Sends and receives simple active messages on the radio and serial linksBlink 3688 1 0 Periodically blinks the three LEDs of a moteMultihopOscilloscope 24310 9 8 Periodically samples and broadcasts sensor data to a base stationOscilloscope 11686 5 5 Periodically samples sensor data and broadcasts a message every 10 timesPowerup 2352 0 0 Turns on the red LED when the mote powers upRadioCountToLeds 11165 4 3 Periodically samples sensors and displays the last 3 bits on its LEDsRadioSenseToLeds 11472 5 2 Periodically update a counter and displays the last 3 bits on its LEDsSense 5743 2 0 Periodically takes sensor readings and displays the values on the LEDs

of the Avrora [23] simulation and analysis framework thatprovides a time accurate simulator different analysis toolsknown asmonitors and a binary instrumentation engine [24]for programs written for AVR microcontrollers and sensornetworks All these features make Avrora just suitable forour tool Simulation is commonly used in the developmentcycle of embedded software because target systems generallylack conventional inputoutput interfaces With iRace thefunctionalities of Avrora are extendedwith a precise data racedetection tool that can easily be loaded as one of Avrorarsquosmonitors

As shown in Figure 3 iRace consists of two maincomponents the Instrumenter which exploits the Avrorainstrumentation engine [24] and the core Race Detectoritself The Instrumenter monitors task and interrupts activ-ities through its iMonitor component and provides entrypoints for the labeling routines Instrumentation points areinserted at the beginning of each task at the start andexit of each interrupt handler and also before interruptsare either enabled or disabled Similarly accesses to sharedmemory locations are monitored using the AccessMonitorcomponent that also calls routines of the detection proto-col The AccessMonitor instruments before read and writeaccesses to shared variables The Race Detector is composedof labeling routines that generate and manage labels for eachactive thread segment at runtime and detection routines formaintaining the access history and reporting data races on-the-fly as shared variables are manipulated

In TinyOS [21 25] interrupt handlers and tasks all runto completion unless preempted by other interrupt handlersThus only a single thread runs at any point of time givingus the opportunity to use only a single variable to maintainthe active thread label at detection time However whenpreemption happens the label of the preempted threadsegment must be saved and restored again when the activeinterrupt handler exits Otherwise there will be no way toknow about the concurrency information of the preemptedthread segment when it gains back the execution control Inorder tomanage the preemption we use a stack data structurewhere labels can be pushed and restored from since interruptscan be nested This provides a simple but powerful way tohandle preemption with negligible overhead

Our labeling scheme needs to uniquely identify eachinterrupt handler whenever it is invoked and each taskwhen it starts its execution Each interrupt handler is easilyidentified by the interrupt number of the interrupt it servesFor task we chose to generate a unique task identifier bysimply maintaining a counter that increases at each timea task starts its execution Thus each task is attributed aunique identifier thanks to the fact that once posted TinyOStasks run in a first-in first-out manner and cannot preempteach other By manipulating only binary files iRace doesnot require any source file nor source code annotation toeffectively detect data races However compiling the programwith debugging information may help iRace generate moredetailed debugging information on data races it reports

62 Environment We implemented and evaluated our tech-nique on a system that runs the kernel 30 of the Linuxoperating system using an Intel dual-core 320GHz and 4GBof mainmemory On this machine was installed TinyOS 210[10] along with the nesC compiler 131 the GNU C compiler46 the Java Runtime Environment 17 and the latest Avroraframework version 17115

Table 2 presents the characteristics and a brief descriptionof the benchmark applications selected for our evaluationfrom the TinyOS 210 repository [10] These applications arefreely distributed with each version of TinyOS The columnldquoLOCrdquo indicates the number of line of C codes generatedby the nesC compiler after compilation while the columnldquoIntrdquo contains the number of interrupts handled by eachapplicationThe number of shared variables for each programappears in column ldquoSVrdquo It is worthmentioning that this num-ber represents only the shared variables in the source code ofeach program The exact number of variables shared by eachapplicationmay finally depend on the number of componentsit shares with the operating system itself Applications writtenin nesC share TinyOS components by wiring themselvesto these components increasing therefore their number ofshared resources We compiled each application for theMica2 platform and measured the average runtime overheadover five executions The Mica2 motes are sensor networknodes based on the ATmega128 [30] microcontrollers Weconfigured Avrora to simulate a sensor network during 10

10 International Journal of Distributed Sensor Networks

iRace

and instrumentationframework

iMonitor

AccessMonitor

Race detector

Labeling routines

Detection protocolroutines

Binarycode Data

races

Avrora simulation

Figure 3 The Architecture of the Data Race Detection Tool

Table 3 Reported data races on the benchmark applications

Applications Data races Verif races Time (Sec) Memory (Kb)Original iRace Original iRace

6LowPanCli 0 0 269 357 1936960 2168160AnthiTheft 15 15 292 457 1944400 2320000BaseStation 0 0 279 376 1912240 2148240Blink 0 0 161 180 1887280 1949600MultihopOscilloscope 0 0 278 400 1939600 2196960Oscilloscope 13 13 179 328 1909360 2201760Powerup 0 0 151 186 1873600 1879920RadioCountToLeds 17 17 196 1255 1942880 2326320RadioSenseToLeds 15 15 181 1305 1944480 2298240Sense 0 0 158 192 1866400 2050560

seconds allowing each application to run exactly for 73728000instruction cycles We set to five the number of nodes in thesimulated sensor network

63 Evaluation For data race detectionwe instructedAvrorato simulate a sensor network with five nodes while thebenchmark applications are monitored by iRace Table 3represents the results of the evaluation of iRace on thebenchmark applications in its column ldquoData Racesrdquo whichcontains the number of shared variables involved in adata race for each application Our tool reported no datarace for all applications except for four AnthiTheft with15 shared variables Oscilloscope with 13 shared vari-ables RadioCountToLeds with 17 shared variables andRadioSenseToLeds with 15 shared variables Surprisinglymost of the reported data races reside inside some sharedsystem components of TinyOS itself The reported data racesare caused by two different interrupts namely the serialperipheral interface (SPI) interrupt and the timer interruptAlso we carefully analyzed the reported data races andinspected the source code of the applications for a manualverification process as shown in the column ldquoVerified racesrdquoof Table 3

In the AnthiTheft application for example which isused to detect and report a stolen mote by checking the levelof the light or the acceleration data races were reported over

15 shared variables Most of the reported data races are how-ever due to the fact that the SPI interrupt handler can preemptsome functions or tasks owned by the underlying radio stackThis implies that if a message is being received or sent whena SPI interrupt occurs shared variables like radioState orcount in the component CC1000SendReceiveP might becorrupted However all shared variables involved in a datarace do not directly belong to AnthiTheft but to functionsof the CC1000 radio stack in use in Mica2 motes Similardata races are also reported in RadioCountToLeds andRadioSenseToLeds since they all access the radio stack

Another data race reported both in RadioCountToLedsand RadioSenseToLeds and worthy to be commentedconcerns the shared variable reqResId inside thecomponent ArbiterP (Recently this data race hasalso been pointed out and a patch discussed in theTinyOS Forum) This data race occurs when the functionimmediateRequest() called within the function read ofthe component immediateRequest is preempted by a SPIinterrupt handler An excerpt of this function is presentedin Algorithm 4 At line 2 of Algorithm 4 the handle of aresource is requested and eventually locked out if grantedHowever if a SPI interrupt is received somewhere afterline 2 a data race will occur and the resource handlerstored in reqResId (line 6 Algorithm 4) may eventually beoverwritten by the SPI interrupt handler Also other shared

International Journal of Distributed Sensor Networks 11

(1) async command error t ResourceimmediateRequest [uint8 t id] ( ) (2) signal ResourceRequested immediateRequested [resId] ( )(3) atomic (4) if (state == RES CONTROLLED) (5) state = RES IMM GRANTING(6) reqResId = id(7)

(8) else return FAIL(9)

(10) signal ResourceDefaultOwner immediateRequested( )(11) sdot sdot sdot (12)

Algorithm 4 An excerpt of the request function from the component ArbiterP

(1) command error t Send send[uint8 t clientId] (message t lowastmsg(2) uint8 t len) (3) sdot sdot sdot (4) queue [clientId] msg = msg(5) call Packet setPayloadLength (msg len)(6) sdot sdot sdot (7)

Algorithm 5 An excerpt of the send function from the component AMSend

variables in ArbiterP like state can be corrupted as wellby the SPI interrupt handler

In the case of Oscilloscope which is a data collectionapplication that periodically samples the default sensor ofa mote and broadcasts a message over the radio every 10readings the reported data races are similar to those reportedin AnthiTheft Moreover our results suggest that if a SPIinterrupt arrives when a message is being transmitted viathe send() function of the AMSend interface the sharedbuffer sendBuf used by Oscilloscope to store incomingmessages will be updated and a corruptedmessage deliveredThe extract of code from the function Send presentedin Algorithm 5 shows that the variable msg which inthis case is a pointer to the shared variable sendBuf inOscilloscope can be updated at any time if the functionsend() is preempted because operations are not performedin a critical section This data race is somehow similar to theone we described in the example program of Algorithm 1Wealso find in Oscilloscope some of the data races reportedin the other applicationsThe results in Table 3 show that ourtool precisely detects data races on-the-fly in interrupt-drivenprograms without any false positives

64 Detection Overhead In order to evaluate the impactof our technique on the monitored programs we config-ured Avrora to simulate a single node sensor network Wethen measured the average runtime overhead and memoryspace required by each application of the benchmark ofTable 2 to execute 73728000 instruction cycles The totaloverhead of our technique mainly comes from actions of theinstrumentation engine the labeling routines and the race

Exec

utio

ns ti

me (

s)

14

12

10

8

6

4

2

0

TinyOS 210 programsOriginalWith iRace

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 4 Average runtime overhead

detection engine which also has the responsibility to createandmaintain the access historiesThe average execution timeand memory consumption collected for each application arepresented in Table 3 The columns ldquoOriginalrdquo and ldquoiRacerdquorepresent the execution time and memory consumption foreach application without and with iRace respectively

Figure 4 which plots the data from the column ldquoTimerdquoof Table 3 compares the average execution times of each

12 International Journal of Distributed Sensor Networks

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

TinyOS 210 programsOriginalWith iRace

Figure 5 Average memory consumption

application for five runs with and without iRace The captionldquoOriginalrdquo represents the native execution of the applica-tions without iRace The results of Figure 4 suggest thatthe average runtime overhead of our tool is only about1x for all applications except for RadioCountToLeds andRadioSenseToLeds that reaches 6x the original executiontime These two applications sample and display data at highfrequency rate (eg 4Hz in our case) and therefore arelikely to receive a higher number of interrupts at runtimeThis low runtime overhead is achieved by exploiting the factthat shared variables are stored by Avrora at program loadingtime Instead of monitoring all memory accesses to filterthose targeting shared variables iRace precisely instrumentsonly accesses to the well-known set of shared memory loca-tions obtained from the Avrora loader This technique allowsthe tool to drastically cuts down its monitoring overhead andto increase its scalability

Likewise wemeasured the averagememory consumptionfor each application during five runs The collected data arerepresented in Figure 5 The overall average memory con-sumption of the tool is very small at just about 20 above theoriginal memory requirement even for high frequency appli-cations like RadioCountToLeds and RadioSenseToLedsthanks to our labeling scheme that reuses the same labelfor different instances of the same interrupt handler and thedetection protocol that maintains only a reduced size accesshistoryThis result corroborates the theoretical analysis of ourtechnique as summarized in Table 1 The low runtime over-head and memory consumption of our on-the-fly techniquemake it very scalable and suitable even for interrupt-drivenprograms that handle interrupts at high frequency rates

65 Labeling Overhead Since the race detection subcompo-nent of iRace is in charge of implementing the labeling and

Table 4 Average labeling overhead

Applications Time (Sec) Memory (Kb)6LowPanCli 313 2101760AnthiTheft 308 2148880BaseStation 309 2077680Blink 174 1964640MultihopOscilloscope 313 2141920Oscilloscope 249 2092640Powerup 176 1894720RadioCountToLeds 218 2167440RadioSenseToLeds 224 2140080Sense 184 2011040

Table 5The states of an access history for detecting data races withour protocol considering the example of Figure 2

119886119894

Read Write Races11988211

⟨0 20 0⟩ ⟨20⟩

11987712

⟨0 20 0⟩ ⟨20⟩

11988213

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11988213-11988211

11988224

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩11987715

⟨1 1 0⟩ ⟨mdash⟩ ⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11987715-11988211

the detection routines it was legitimate to analyze the impactof the labeling scheme alone on the monitored programsat runtime For this purpose we deactivated the detectionengine of iRace before running it again on the benchmarkapplications We collected the average execution time andmemory space consumed by each application after five runsas shown in Table 4 The simulation was done using a singlenode network

Figure 6 compares the original execution time of eachprogram from Table 3 with the execution time of the labelingengine as presented in the column ldquoTimerdquo of Table 4 Thecomparison shows that the average runtime overhead ofiRace with just the labeling engine activated is only about04x in the worst-case This represents only a small fractionof the average runtime overhead of 6x imposed by iRace asdiscussed in Section 64Themain reason of this performancecan be found at source level When browsing the source codeof the benchmark applications we came to understand thatprogrammers generally disable and enable all the interruptsat once instead of focusing on particular interrupts In thiscase the size of each label generated by the labeling enginenever approaches its worst-case of 119874(119879) meaning that thecomparison overhead of labels is also reduced This is veryimportant because as discussed in Section 64 althoughthe size of labels does not depend on the total number ofinterrupts received at runtime this number still has impacton the overall detection overheadThus we can conclude thatthe labeling engine of iRace incurs a very low overhead atruntime making it a highly scalable tool

We also compared the memory consumption of thelabeling engine with the memory requirement of the original

International Journal of Distributed Sensor Networks 13

TinyOS 210 programs

4

35

3

25

2

15

1

05

0

Exec

utio

n tim

e (s)

With iRace labeling engineOriginal

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 6 Average runtime overhead of the labeling engine

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

With iRace labeling engineOriginal

TinyOS 210 programs

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 7 Average memory consumption by the labeling engine

program execution presented inTable 3The results plotted inFigure 7 show that when the labeling engine alone is runningiRace consumes only about 12 more memory on averageHowever during the detection phase the overall averagememory consumption of iRace is only about 20 Theseresults also tell us that the detection engine does not increasethat much the memory consumption of iRace This is alsoreasonable because we only disabled the detection engine ofiRace but did not remove the resources it used

7 Conclusions

Asynchronous interrupt handling introduces fine-grainedconcurrency into interrupt-driven programs making themprone to data races In this paper we presented an efficientand scalable on-the-fly technique to precisely detect dataraces without false positives in such programs Our techniqueintroduced a lightweight labeling scheme for maintainingconcurrency information for the main program and each ofits interrupt handlers and a precise detection protocol thatguarantees to verify the existence of data races in a programby reporting at least one data race for each shared variableinvolved in a data race

We implemented a prototype of our technique as a racedetection tool called iRace on top of the Avrora simulationframework An evaluation of iRace revealed the existence ofdata races in some programs distributed with TinyOS andin some TinyOS components while incurring a slowdownof only about 6 times on average in the worst-case with anaverage memory consumption of only about 20 above theoriginal memory requirements of the program Moreoverthis evaluation also proved that the labeling engine alonegenerates an average runtime overhead of only about 04xwith an average memory consumption of only about 12relatively to the original program execution These resultscoupled with a theoretical analysis showed that the proposedtechnique is highly scalable and practical even for largeinterrupt-driven programs

Detecting data races that occur first in a program isan important issue since it may make other data races todisappear It is therefore interesting to detect this class of dataraces in interrupt-driven programs to facilitate the debuggingprocess Also we may be interested in extending our work tosupport multithreaded programs that handle interrupts andalso to detect data races in low-level kernel codes

Acknowledgment

This research was financially supported by the Ministryof Education (MOE) and National Research Foundation ofKorea (NRF) through the Human Resource Training Projectfor Regional Innovation and BK21+ program

References

[1] U Banerjee B Bliss Z Ma and P Petersen ldquoA theory of datarace detectionrdquo in Proceedings of the Workshop on Parallel andDistributed Systems Testing and Debugging (PADTAD rsquo06) pp69ndash78 July 2006

[2] R H B Netzer and B P Miller ldquoWhat are race conditionsSome issues and formalizationsrdquo ACM letters on programminglanguages and systems vol 1 no 1 pp 74ndash88 1992

[3] N G Leveson and C S Turner ldquoInvestigation of theTherac-25accidentsrdquo Computer vol 26 no 7 pp 18ndash41 1993

[4] USA Canada Final Report on the August 14 2003 Blackout inthe United States and Canada Causes and RecommendationsUS-Canada Power System Outage Task Force 2004

[5] J Erickson M Musuvathi S Burckhardt and K OlynykldquoEffective data-race detection for the kernelrdquo in Proceedings of

14 International Journal of Distributed Sensor Networks

the 9th Conference on Operating Systems Design and Implemen-tation (USENIX rsquo10) 2010

[6] B-K Lee M-H Kang K C Park J S Yi S W Yang andY-K Jun ldquoProgram conversion for detecting data races inconcurrent interrupt handlersrdquo Communications in Computerand Information Science vol 257 pp 407ndash415 2011

[7] L Effinger-Dean B Lucia L Ceze D Grossman and H JBoehm ldquoIFRit interference-free regions for dynamic data-racedetectionrdquo in Proceedings of the ACM International Conferenceon Object Oriented Programming Systems Languages and Appli-cations (OOPSLA rsquo12) vol 47 pp 467ndash484 2012

[8] C Flanagan and S N Freund ldquoFastTrack efficient and precisedynamic race detectionrdquo in Proceedings of the ACM Interna-tional Conference on Object Oriented Programming SystemsLanguages and Applications (OOPSLA rsquo09) vol 44 pp 121ndash1332009

[9] X Xie and J Xue ldquoAcculock accurate and efficient detection ofdata racesrdquo in Proceedings of the 9th International Symposium onCode Generation and Optimization (CGO rsquo11) pp 201ndash212 April2011

[10] TinyOSCommunity ForumTinyOSAnopen source operatingsystem for low-power wireless devices 2008 httpwwwtinyosnet

[11] M Higashi T Yamamoto Y Hayase T Ishio and K InoueldquoAn effective method to control interrupt handler for datarace detectionrdquo in International Workshop on Automation ofSoftware Test (ACM rsquo10) pp 79ndash86 May 2010

[12] Z Lai S C Cheung and W K Chan ldquoInter-context control-flow and data-flow test adequacy criteria for nesC applicationsrdquoin Proceedings of the 16th ACM SIGSOFT International Sympo-sium on the Foundations of Software Engineering (SIGSOFT rsquo08)pp 94ndash104 November 2008

[13] J Regehr ldquoRandom testing of interrupt-driven softwarerdquo inPro-ceedings of the 5th ACM International Conference on EmbeddedSoftware (EMSOFT rsquo05) pp 290ndash298 September 2005

[14] Y Zhou X Chen M R Lyu and J Liu ldquoSentomist Unveil-ing transient sensor network bugs via symptom miningrdquoin Proceedings of the 30th IEEE International Conference onDistributed Computing Systems (ICDCS rsquo10) pp 784ndash794 June2010

[15] G Gracioli and S Fischmeister ldquoTracing interrupts in embed-ded softwarerdquo in Proceedings of the ACM SIGPLANSIGBEDConference on Languages Compilers and Tools for EmbeddedSystems (LCTES rsquo09) vol 44 pp 137ndash146 2009

[16] V Sundaram P Eugster and X Zhang ldquoLightweight tracingfor wireless sensor networks debuggingrdquo in Proceedings of the4th International Workshop on Middleware Tools Services andRun-Time Support for Sensor Networks (ACM rsquo09) pp 13ndash18December 2009

[17] O K Ha and Y K Jun ldquoMonitoring of programs with nestedparallelismusing efficient thread labelingrdquo International JournalFo U- and E- Service Science and Technology vol 4 pp 21ndash362011

[18] Y C Kim S S Jun and Y K Jun ldquoA tool for space-efficient on-the-fly race detectionrdquo International Journal of DatabaseTheoryand Application vol 4 pp 25ndash38 2011

[19] S Savage M Burrows G Nelson P Sobalvarro and T Ander-son ldquoEraser a dynamic data race detector for multithreadedprogramsrdquo ACM Transactions on Computer Systems vol 15 no4 pp 391ndash411 1997

[20] A Jannesari and W F Tichy ldquoOn-the-fly race detection inmulti-threaded programsrdquo in Proceedings of the 6th Workshop

on Parallel and Distributed Systems Testing Analysis andDebugging (PADTAD rsquo08) July 2007

[21] D Gay M Welsh P Levis E Brewer R Von Behren and DCuller ldquoThe nesC language a holistic approach to networkedembedded systemsrdquo inACMSIGPLANConference on Program-ming Language Design and Implementation pp 1ndash11 June 2003

[22] J Regehr and N Cooprider ldquoInterrupt verification via threadverificationrdquo Electronic Notes in Theoretical Computer Sciencevol 174 no 9 pp 139ndash150 2007

[23] B L Titzer D K Lee and J Palsberg ldquoAvrora scalable sensornetwork simulation with precise timingrdquo in Proceedings ofthe 4th International Symposium on Information Processing inSensor Networks (IPSN rsquo05) pp 477ndash482 April 2005

[24] B L Titzer and J Palsberg ldquoNonintrusive precision instru-mentation of microcontroller softwarerdquo in Proceedings of theConference on Languages Compilers and Tools for EmbeddedSystems (Lctes rsquo05) vol 40 pp 59ndash68 2005

[25] J Hil R Szewczyk A Woo S Hollar D Culler and K PisterldquoSystem architecture directions for networked sensorsrdquo inProceedings of the 9th International Conference on ArchitecturalSupport for Programming Languages and Operating Systems(ASPLOS rsquo00) pp 93ndash104 2000

[26] GMTchamgoue KHKim andYK Jun ldquoTesting anddebug-ging concurrency bugs in event-driven programsrdquo InternationalJournal of Advanced Science and Technology vol 40 pp 55ndash682012

[27] GM Tchamgoue O K Ha K H Kim and Y K Jun ldquoA frame-work for on-the-fly race healing in ARINC-653 applicationsrdquoInternational Journal of Hybrid Information Technology vol 4pp 1ndash12 2011

[28] A Dinning and E Schonberg ldquoDetecting access anomaliesin programs with critical sectionsrdquo in Proceedings of theACMONR Workshop on Parallel and Distributed Debugging(PADD rsquo91) vol 26 pp 85ndash96 1991

[29] L Lamport ldquoTime clocks and the ordering of events in adistributed systemrdquo Communications of the ACM vol 21 no7 pp 558ndash565 1978

[30] Atmel inc ATmega128 Datasheet 2011 httpwwwatmelcomImagesdoc2467pdf

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 8: Research Article Verification of Data Races in Concurrent ...downloads.hindawi.com/journals/ijdsn/2013/953593.pdf · photoTask I20 W 11 R 12 W 13 W 24 R 15 T 2 J12 T 1 F : A POEG

8 International Journal of Distributed Sensor Networks

Theorem 8 Given an interrupt-driven program 119875 the detec-tion protocol reports a data race over a shared variable 119909 if andonly if there exists a data race over 119909 in an execution instanceof 119875

Proof We will first prove that if a data race is reported on ashared variable 119909 by the detection protocol then there alsoexists a data race in an execution instance of the program forthe same shared variable 119909 For this we assume a snapshotof the access history 119860119867(119909) of the shared variable 119909 at agiven time and the current access event 119886

119894to 119909 from a thread

segment 119905119894that owns a label we refer to as 119871119887

119894in an execution

instance of the program 119875 We can then distinguish twodifferent cases

Case A 119860119867(119909) is empty and there is obviously no data racefor 119909 In this case the current access 119886

119894is the only access to

119909 from the program 119875 and 119871119887119894will be added by the function

add ah() into119860119867(119909)sdot119877119890119886119889 if 119886119894is a read access or into119860119867(119909)sdot

119882119903119894119905119890 if it is a write access

Case B 119860119867(119909) is not empty Data races are reported usingthe ah read() function if 119886

119894is a read access or the ah write()

in case of a write access In both cases the happens-beforerelation is analyzed between the label 119871119887

119894of 119905119894and previous

labels stored in 119860119867(119909) according to Theorem 7 The outputsof these analyses can be classified into two categories

(1) There is no data race between 119886119894and119860119867(119909) therefore

all previous accesses stored in 119860119867(119909) happenedbefore 119886

119894meaning that no thread segment prior to

119905119894in 119875 is concurrent with 119905

119894 they are all ordered by

happens-before Thus for all 119886119895isin 119905119895and 119871119887119895isin 119860119867(119909)

119905119895

ℎ119887

997888997888rarr 119905119894 In which case 119875 also has no data race

(2) There are data races between 119886119894and prior accesses

in 119860119867(119909) this means that there are some labels in119860119867(119909) belonging to some prior thread segments thatare not ordered with 119905

119894 Concretely there are thread

segments in 119875 that are concurrent with 119905119894 Formally

exist119905119895isin 119875119871119887

119895isin 119860119867(119909) 119905

119894 119905119895Therefore the execution

instance of 119875 also contains data races over 119909

Now we show that if there exists a data race over a sharedvariable 119909 in an execution instance of the program 119875 thenour detection protocol also reports a data race over 119909 Usingthe contrapositive we will prove that if the detection protocoldoes not report any data race the program 119875 also does notcontain any for a shared variable 119909 We again consider asnapshot of the access history 119860119867(119909) for a shared variable119909 at a given time We also assume that the detection protocolreported no data races for the shared variable 119909 This meansthat 119860119867(119909) is either empty and there is clearly no data racesince there is no access to 119909 from 119875 yet or contains onlylabels of thread segments that are all ordered by the happens-before relation in which case there is no concurrent threadsegment in119875 which also implies that119875 has no data races over119909 Therefore we can formally write for all 119905

119894 119905119895isin 119875 and 119871119887

119894

119871119887119895isin 119860119867(119909) 119905

119894

ℎ119887

997888997888rarr 119905119895or 119905119895

ℎ119887

997888997888rarr 119905119894 Thus 119875 also contains no

data race for 119909

Table 1 Efficiency of the precise detection protocol

Space TimeNew label comparison Per access

O(1198811198792) O(1) O(LogT) O(TLogT)

5 Theoretical Analysis

On-the-fly data race detection techniques are known to havehigh runtime overhead that is generally provoked by the timespentmonitoring accesses to shared variables and to comparethread labels In this section we theoretically analyze theefficiency of our technique which depends on119881 the numberof shared variables and 119879 the total number of interruptsin the underlying system plus one the number of tasksIn general 119879 is very small since the number of interruptshandled by a system is also kept small For example thepopular Atmel ATmega128 [30] processor supports only amaximum of 34 interrupts Also because of the implicithappens-before relation that exists between all tasks they areviewed as a single task by the detection protocol

Table 1 presents the space and time complexities of ourlabeling and detection techniques The access history usedby our technique stores from each thread segment at mosttwo accesses for each shared variable Each label in the accesshistory has a worst-case size of 119874(119879) that represents thesize of its interrupt set Therefore the total labeling spacerequirement for each shared variable is 119874(1198792) Thus thetotal space complexity of our technique is 119874(1198811198792) Howeverbecause the number of interrupts is generally a reasonableinteger number the space complexity of our technique canfurther be reduced to 119874(119881)

A new label is always created in a constant amount of timeby our labeling algorithms Interrupts are generally givennumbers in a sequence order Therefore a binary searchtechnique is applied to locate values in interrupt sets whichcontain only interrupt numbers Thus the time complexityfor comparing any two labels is 119874(Log119879) in the worst-caseand 119874(1) in most cases where the interrupt sets are emptyAt each access to a shared variable the total time complexityto report data races with the previous accesses stored in theaccess history is then119874(119879Log119879)The labels and the detectionprotocol do not depend on the total number of interruptsreceived at runtimemaking our technique highly scalable andpractical

6 Experimental Results

We empirically evaluated the preciseness and efficiency ofour technique using a set of well-known programs providedwith TinyOS [10] This section details the implementationchallenges and presents the analysis results of the experimen-tation

61 Implementation We implemented a prototype of ourdata race detection tool for interrupt-driven programs callediRace based on the concurrency model of TinyOS [21] andits nesC language [25] We implemented iRace on top

International Journal of Distributed Sensor Networks 9

Table 2 Characteristics of the benchmark applications from TinyOS 210 [10]

Applications LOC Int SV Brief description6LowPanCli 19116 4 9 Implements a command line interface using the 6lowpan stackAnthiTheft 27421 5 4 Detects and reports a stolen mote by checking some sensorsBaseStation 14905 6 13 Sends and receives simple active messages on the radio and serial linksBlink 3688 1 0 Periodically blinks the three LEDs of a moteMultihopOscilloscope 24310 9 8 Periodically samples and broadcasts sensor data to a base stationOscilloscope 11686 5 5 Periodically samples sensor data and broadcasts a message every 10 timesPowerup 2352 0 0 Turns on the red LED when the mote powers upRadioCountToLeds 11165 4 3 Periodically samples sensors and displays the last 3 bits on its LEDsRadioSenseToLeds 11472 5 2 Periodically update a counter and displays the last 3 bits on its LEDsSense 5743 2 0 Periodically takes sensor readings and displays the values on the LEDs

of the Avrora [23] simulation and analysis framework thatprovides a time accurate simulator different analysis toolsknown asmonitors and a binary instrumentation engine [24]for programs written for AVR microcontrollers and sensornetworks All these features make Avrora just suitable forour tool Simulation is commonly used in the developmentcycle of embedded software because target systems generallylack conventional inputoutput interfaces With iRace thefunctionalities of Avrora are extendedwith a precise data racedetection tool that can easily be loaded as one of Avrorarsquosmonitors

As shown in Figure 3 iRace consists of two maincomponents the Instrumenter which exploits the Avrorainstrumentation engine [24] and the core Race Detectoritself The Instrumenter monitors task and interrupts activ-ities through its iMonitor component and provides entrypoints for the labeling routines Instrumentation points areinserted at the beginning of each task at the start andexit of each interrupt handler and also before interruptsare either enabled or disabled Similarly accesses to sharedmemory locations are monitored using the AccessMonitorcomponent that also calls routines of the detection proto-col The AccessMonitor instruments before read and writeaccesses to shared variables The Race Detector is composedof labeling routines that generate and manage labels for eachactive thread segment at runtime and detection routines formaintaining the access history and reporting data races on-the-fly as shared variables are manipulated

In TinyOS [21 25] interrupt handlers and tasks all runto completion unless preempted by other interrupt handlersThus only a single thread runs at any point of time givingus the opportunity to use only a single variable to maintainthe active thread label at detection time However whenpreemption happens the label of the preempted threadsegment must be saved and restored again when the activeinterrupt handler exits Otherwise there will be no way toknow about the concurrency information of the preemptedthread segment when it gains back the execution control Inorder tomanage the preemption we use a stack data structurewhere labels can be pushed and restored from since interruptscan be nested This provides a simple but powerful way tohandle preemption with negligible overhead

Our labeling scheme needs to uniquely identify eachinterrupt handler whenever it is invoked and each taskwhen it starts its execution Each interrupt handler is easilyidentified by the interrupt number of the interrupt it servesFor task we chose to generate a unique task identifier bysimply maintaining a counter that increases at each timea task starts its execution Thus each task is attributed aunique identifier thanks to the fact that once posted TinyOStasks run in a first-in first-out manner and cannot preempteach other By manipulating only binary files iRace doesnot require any source file nor source code annotation toeffectively detect data races However compiling the programwith debugging information may help iRace generate moredetailed debugging information on data races it reports

62 Environment We implemented and evaluated our tech-nique on a system that runs the kernel 30 of the Linuxoperating system using an Intel dual-core 320GHz and 4GBof mainmemory On this machine was installed TinyOS 210[10] along with the nesC compiler 131 the GNU C compiler46 the Java Runtime Environment 17 and the latest Avroraframework version 17115

Table 2 presents the characteristics and a brief descriptionof the benchmark applications selected for our evaluationfrom the TinyOS 210 repository [10] These applications arefreely distributed with each version of TinyOS The columnldquoLOCrdquo indicates the number of line of C codes generatedby the nesC compiler after compilation while the columnldquoIntrdquo contains the number of interrupts handled by eachapplicationThe number of shared variables for each programappears in column ldquoSVrdquo It is worthmentioning that this num-ber represents only the shared variables in the source code ofeach program The exact number of variables shared by eachapplicationmay finally depend on the number of componentsit shares with the operating system itself Applications writtenin nesC share TinyOS components by wiring themselvesto these components increasing therefore their number ofshared resources We compiled each application for theMica2 platform and measured the average runtime overheadover five executions The Mica2 motes are sensor networknodes based on the ATmega128 [30] microcontrollers Weconfigured Avrora to simulate a sensor network during 10

10 International Journal of Distributed Sensor Networks

iRace

and instrumentationframework

iMonitor

AccessMonitor

Race detector

Labeling routines

Detection protocolroutines

Binarycode Data

races

Avrora simulation

Figure 3 The Architecture of the Data Race Detection Tool

Table 3 Reported data races on the benchmark applications

Applications Data races Verif races Time (Sec) Memory (Kb)Original iRace Original iRace

6LowPanCli 0 0 269 357 1936960 2168160AnthiTheft 15 15 292 457 1944400 2320000BaseStation 0 0 279 376 1912240 2148240Blink 0 0 161 180 1887280 1949600MultihopOscilloscope 0 0 278 400 1939600 2196960Oscilloscope 13 13 179 328 1909360 2201760Powerup 0 0 151 186 1873600 1879920RadioCountToLeds 17 17 196 1255 1942880 2326320RadioSenseToLeds 15 15 181 1305 1944480 2298240Sense 0 0 158 192 1866400 2050560

seconds allowing each application to run exactly for 73728000instruction cycles We set to five the number of nodes in thesimulated sensor network

63 Evaluation For data race detectionwe instructedAvrorato simulate a sensor network with five nodes while thebenchmark applications are monitored by iRace Table 3represents the results of the evaluation of iRace on thebenchmark applications in its column ldquoData Racesrdquo whichcontains the number of shared variables involved in adata race for each application Our tool reported no datarace for all applications except for four AnthiTheft with15 shared variables Oscilloscope with 13 shared vari-ables RadioCountToLeds with 17 shared variables andRadioSenseToLeds with 15 shared variables Surprisinglymost of the reported data races reside inside some sharedsystem components of TinyOS itself The reported data racesare caused by two different interrupts namely the serialperipheral interface (SPI) interrupt and the timer interruptAlso we carefully analyzed the reported data races andinspected the source code of the applications for a manualverification process as shown in the column ldquoVerified racesrdquoof Table 3

In the AnthiTheft application for example which isused to detect and report a stolen mote by checking the levelof the light or the acceleration data races were reported over

15 shared variables Most of the reported data races are how-ever due to the fact that the SPI interrupt handler can preemptsome functions or tasks owned by the underlying radio stackThis implies that if a message is being received or sent whena SPI interrupt occurs shared variables like radioState orcount in the component CC1000SendReceiveP might becorrupted However all shared variables involved in a datarace do not directly belong to AnthiTheft but to functionsof the CC1000 radio stack in use in Mica2 motes Similardata races are also reported in RadioCountToLeds andRadioSenseToLeds since they all access the radio stack

Another data race reported both in RadioCountToLedsand RadioSenseToLeds and worthy to be commentedconcerns the shared variable reqResId inside thecomponent ArbiterP (Recently this data race hasalso been pointed out and a patch discussed in theTinyOS Forum) This data race occurs when the functionimmediateRequest() called within the function read ofthe component immediateRequest is preempted by a SPIinterrupt handler An excerpt of this function is presentedin Algorithm 4 At line 2 of Algorithm 4 the handle of aresource is requested and eventually locked out if grantedHowever if a SPI interrupt is received somewhere afterline 2 a data race will occur and the resource handlerstored in reqResId (line 6 Algorithm 4) may eventually beoverwritten by the SPI interrupt handler Also other shared

International Journal of Distributed Sensor Networks 11

(1) async command error t ResourceimmediateRequest [uint8 t id] ( ) (2) signal ResourceRequested immediateRequested [resId] ( )(3) atomic (4) if (state == RES CONTROLLED) (5) state = RES IMM GRANTING(6) reqResId = id(7)

(8) else return FAIL(9)

(10) signal ResourceDefaultOwner immediateRequested( )(11) sdot sdot sdot (12)

Algorithm 4 An excerpt of the request function from the component ArbiterP

(1) command error t Send send[uint8 t clientId] (message t lowastmsg(2) uint8 t len) (3) sdot sdot sdot (4) queue [clientId] msg = msg(5) call Packet setPayloadLength (msg len)(6) sdot sdot sdot (7)

Algorithm 5 An excerpt of the send function from the component AMSend

variables in ArbiterP like state can be corrupted as wellby the SPI interrupt handler

In the case of Oscilloscope which is a data collectionapplication that periodically samples the default sensor ofa mote and broadcasts a message over the radio every 10readings the reported data races are similar to those reportedin AnthiTheft Moreover our results suggest that if a SPIinterrupt arrives when a message is being transmitted viathe send() function of the AMSend interface the sharedbuffer sendBuf used by Oscilloscope to store incomingmessages will be updated and a corruptedmessage deliveredThe extract of code from the function Send presentedin Algorithm 5 shows that the variable msg which inthis case is a pointer to the shared variable sendBuf inOscilloscope can be updated at any time if the functionsend() is preempted because operations are not performedin a critical section This data race is somehow similar to theone we described in the example program of Algorithm 1Wealso find in Oscilloscope some of the data races reportedin the other applicationsThe results in Table 3 show that ourtool precisely detects data races on-the-fly in interrupt-drivenprograms without any false positives

64 Detection Overhead In order to evaluate the impactof our technique on the monitored programs we config-ured Avrora to simulate a single node sensor network Wethen measured the average runtime overhead and memoryspace required by each application of the benchmark ofTable 2 to execute 73728000 instruction cycles The totaloverhead of our technique mainly comes from actions of theinstrumentation engine the labeling routines and the race

Exec

utio

ns ti

me (

s)

14

12

10

8

6

4

2

0

TinyOS 210 programsOriginalWith iRace

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 4 Average runtime overhead

detection engine which also has the responsibility to createandmaintain the access historiesThe average execution timeand memory consumption collected for each application arepresented in Table 3 The columns ldquoOriginalrdquo and ldquoiRacerdquorepresent the execution time and memory consumption foreach application without and with iRace respectively

Figure 4 which plots the data from the column ldquoTimerdquoof Table 3 compares the average execution times of each

12 International Journal of Distributed Sensor Networks

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

TinyOS 210 programsOriginalWith iRace

Figure 5 Average memory consumption

application for five runs with and without iRace The captionldquoOriginalrdquo represents the native execution of the applica-tions without iRace The results of Figure 4 suggest thatthe average runtime overhead of our tool is only about1x for all applications except for RadioCountToLeds andRadioSenseToLeds that reaches 6x the original executiontime These two applications sample and display data at highfrequency rate (eg 4Hz in our case) and therefore arelikely to receive a higher number of interrupts at runtimeThis low runtime overhead is achieved by exploiting the factthat shared variables are stored by Avrora at program loadingtime Instead of monitoring all memory accesses to filterthose targeting shared variables iRace precisely instrumentsonly accesses to the well-known set of shared memory loca-tions obtained from the Avrora loader This technique allowsthe tool to drastically cuts down its monitoring overhead andto increase its scalability

Likewise wemeasured the averagememory consumptionfor each application during five runs The collected data arerepresented in Figure 5 The overall average memory con-sumption of the tool is very small at just about 20 above theoriginal memory requirement even for high frequency appli-cations like RadioCountToLeds and RadioSenseToLedsthanks to our labeling scheme that reuses the same labelfor different instances of the same interrupt handler and thedetection protocol that maintains only a reduced size accesshistoryThis result corroborates the theoretical analysis of ourtechnique as summarized in Table 1 The low runtime over-head and memory consumption of our on-the-fly techniquemake it very scalable and suitable even for interrupt-drivenprograms that handle interrupts at high frequency rates

65 Labeling Overhead Since the race detection subcompo-nent of iRace is in charge of implementing the labeling and

Table 4 Average labeling overhead

Applications Time (Sec) Memory (Kb)6LowPanCli 313 2101760AnthiTheft 308 2148880BaseStation 309 2077680Blink 174 1964640MultihopOscilloscope 313 2141920Oscilloscope 249 2092640Powerup 176 1894720RadioCountToLeds 218 2167440RadioSenseToLeds 224 2140080Sense 184 2011040

Table 5The states of an access history for detecting data races withour protocol considering the example of Figure 2

119886119894

Read Write Races11988211

⟨0 20 0⟩ ⟨20⟩

11987712

⟨0 20 0⟩ ⟨20⟩

11988213

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11988213-11988211

11988224

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩11987715

⟨1 1 0⟩ ⟨mdash⟩ ⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11987715-11988211

the detection routines it was legitimate to analyze the impactof the labeling scheme alone on the monitored programsat runtime For this purpose we deactivated the detectionengine of iRace before running it again on the benchmarkapplications We collected the average execution time andmemory space consumed by each application after five runsas shown in Table 4 The simulation was done using a singlenode network

Figure 6 compares the original execution time of eachprogram from Table 3 with the execution time of the labelingengine as presented in the column ldquoTimerdquo of Table 4 Thecomparison shows that the average runtime overhead ofiRace with just the labeling engine activated is only about04x in the worst-case This represents only a small fractionof the average runtime overhead of 6x imposed by iRace asdiscussed in Section 64Themain reason of this performancecan be found at source level When browsing the source codeof the benchmark applications we came to understand thatprogrammers generally disable and enable all the interruptsat once instead of focusing on particular interrupts In thiscase the size of each label generated by the labeling enginenever approaches its worst-case of 119874(119879) meaning that thecomparison overhead of labels is also reduced This is veryimportant because as discussed in Section 64 althoughthe size of labels does not depend on the total number ofinterrupts received at runtime this number still has impacton the overall detection overheadThus we can conclude thatthe labeling engine of iRace incurs a very low overhead atruntime making it a highly scalable tool

We also compared the memory consumption of thelabeling engine with the memory requirement of the original

International Journal of Distributed Sensor Networks 13

TinyOS 210 programs

4

35

3

25

2

15

1

05

0

Exec

utio

n tim

e (s)

With iRace labeling engineOriginal

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 6 Average runtime overhead of the labeling engine

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

With iRace labeling engineOriginal

TinyOS 210 programs

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 7 Average memory consumption by the labeling engine

program execution presented inTable 3The results plotted inFigure 7 show that when the labeling engine alone is runningiRace consumes only about 12 more memory on averageHowever during the detection phase the overall averagememory consumption of iRace is only about 20 Theseresults also tell us that the detection engine does not increasethat much the memory consumption of iRace This is alsoreasonable because we only disabled the detection engine ofiRace but did not remove the resources it used

7 Conclusions

Asynchronous interrupt handling introduces fine-grainedconcurrency into interrupt-driven programs making themprone to data races In this paper we presented an efficientand scalable on-the-fly technique to precisely detect dataraces without false positives in such programs Our techniqueintroduced a lightweight labeling scheme for maintainingconcurrency information for the main program and each ofits interrupt handlers and a precise detection protocol thatguarantees to verify the existence of data races in a programby reporting at least one data race for each shared variableinvolved in a data race

We implemented a prototype of our technique as a racedetection tool called iRace on top of the Avrora simulationframework An evaluation of iRace revealed the existence ofdata races in some programs distributed with TinyOS andin some TinyOS components while incurring a slowdownof only about 6 times on average in the worst-case with anaverage memory consumption of only about 20 above theoriginal memory requirements of the program Moreoverthis evaluation also proved that the labeling engine alonegenerates an average runtime overhead of only about 04xwith an average memory consumption of only about 12relatively to the original program execution These resultscoupled with a theoretical analysis showed that the proposedtechnique is highly scalable and practical even for largeinterrupt-driven programs

Detecting data races that occur first in a program isan important issue since it may make other data races todisappear It is therefore interesting to detect this class of dataraces in interrupt-driven programs to facilitate the debuggingprocess Also we may be interested in extending our work tosupport multithreaded programs that handle interrupts andalso to detect data races in low-level kernel codes

Acknowledgment

This research was financially supported by the Ministryof Education (MOE) and National Research Foundation ofKorea (NRF) through the Human Resource Training Projectfor Regional Innovation and BK21+ program

References

[1] U Banerjee B Bliss Z Ma and P Petersen ldquoA theory of datarace detectionrdquo in Proceedings of the Workshop on Parallel andDistributed Systems Testing and Debugging (PADTAD rsquo06) pp69ndash78 July 2006

[2] R H B Netzer and B P Miller ldquoWhat are race conditionsSome issues and formalizationsrdquo ACM letters on programminglanguages and systems vol 1 no 1 pp 74ndash88 1992

[3] N G Leveson and C S Turner ldquoInvestigation of theTherac-25accidentsrdquo Computer vol 26 no 7 pp 18ndash41 1993

[4] USA Canada Final Report on the August 14 2003 Blackout inthe United States and Canada Causes and RecommendationsUS-Canada Power System Outage Task Force 2004

[5] J Erickson M Musuvathi S Burckhardt and K OlynykldquoEffective data-race detection for the kernelrdquo in Proceedings of

14 International Journal of Distributed Sensor Networks

the 9th Conference on Operating Systems Design and Implemen-tation (USENIX rsquo10) 2010

[6] B-K Lee M-H Kang K C Park J S Yi S W Yang andY-K Jun ldquoProgram conversion for detecting data races inconcurrent interrupt handlersrdquo Communications in Computerand Information Science vol 257 pp 407ndash415 2011

[7] L Effinger-Dean B Lucia L Ceze D Grossman and H JBoehm ldquoIFRit interference-free regions for dynamic data-racedetectionrdquo in Proceedings of the ACM International Conferenceon Object Oriented Programming Systems Languages and Appli-cations (OOPSLA rsquo12) vol 47 pp 467ndash484 2012

[8] C Flanagan and S N Freund ldquoFastTrack efficient and precisedynamic race detectionrdquo in Proceedings of the ACM Interna-tional Conference on Object Oriented Programming SystemsLanguages and Applications (OOPSLA rsquo09) vol 44 pp 121ndash1332009

[9] X Xie and J Xue ldquoAcculock accurate and efficient detection ofdata racesrdquo in Proceedings of the 9th International Symposium onCode Generation and Optimization (CGO rsquo11) pp 201ndash212 April2011

[10] TinyOSCommunity ForumTinyOSAnopen source operatingsystem for low-power wireless devices 2008 httpwwwtinyosnet

[11] M Higashi T Yamamoto Y Hayase T Ishio and K InoueldquoAn effective method to control interrupt handler for datarace detectionrdquo in International Workshop on Automation ofSoftware Test (ACM rsquo10) pp 79ndash86 May 2010

[12] Z Lai S C Cheung and W K Chan ldquoInter-context control-flow and data-flow test adequacy criteria for nesC applicationsrdquoin Proceedings of the 16th ACM SIGSOFT International Sympo-sium on the Foundations of Software Engineering (SIGSOFT rsquo08)pp 94ndash104 November 2008

[13] J Regehr ldquoRandom testing of interrupt-driven softwarerdquo inPro-ceedings of the 5th ACM International Conference on EmbeddedSoftware (EMSOFT rsquo05) pp 290ndash298 September 2005

[14] Y Zhou X Chen M R Lyu and J Liu ldquoSentomist Unveil-ing transient sensor network bugs via symptom miningrdquoin Proceedings of the 30th IEEE International Conference onDistributed Computing Systems (ICDCS rsquo10) pp 784ndash794 June2010

[15] G Gracioli and S Fischmeister ldquoTracing interrupts in embed-ded softwarerdquo in Proceedings of the ACM SIGPLANSIGBEDConference on Languages Compilers and Tools for EmbeddedSystems (LCTES rsquo09) vol 44 pp 137ndash146 2009

[16] V Sundaram P Eugster and X Zhang ldquoLightweight tracingfor wireless sensor networks debuggingrdquo in Proceedings of the4th International Workshop on Middleware Tools Services andRun-Time Support for Sensor Networks (ACM rsquo09) pp 13ndash18December 2009

[17] O K Ha and Y K Jun ldquoMonitoring of programs with nestedparallelismusing efficient thread labelingrdquo International JournalFo U- and E- Service Science and Technology vol 4 pp 21ndash362011

[18] Y C Kim S S Jun and Y K Jun ldquoA tool for space-efficient on-the-fly race detectionrdquo International Journal of DatabaseTheoryand Application vol 4 pp 25ndash38 2011

[19] S Savage M Burrows G Nelson P Sobalvarro and T Ander-son ldquoEraser a dynamic data race detector for multithreadedprogramsrdquo ACM Transactions on Computer Systems vol 15 no4 pp 391ndash411 1997

[20] A Jannesari and W F Tichy ldquoOn-the-fly race detection inmulti-threaded programsrdquo in Proceedings of the 6th Workshop

on Parallel and Distributed Systems Testing Analysis andDebugging (PADTAD rsquo08) July 2007

[21] D Gay M Welsh P Levis E Brewer R Von Behren and DCuller ldquoThe nesC language a holistic approach to networkedembedded systemsrdquo inACMSIGPLANConference on Program-ming Language Design and Implementation pp 1ndash11 June 2003

[22] J Regehr and N Cooprider ldquoInterrupt verification via threadverificationrdquo Electronic Notes in Theoretical Computer Sciencevol 174 no 9 pp 139ndash150 2007

[23] B L Titzer D K Lee and J Palsberg ldquoAvrora scalable sensornetwork simulation with precise timingrdquo in Proceedings ofthe 4th International Symposium on Information Processing inSensor Networks (IPSN rsquo05) pp 477ndash482 April 2005

[24] B L Titzer and J Palsberg ldquoNonintrusive precision instru-mentation of microcontroller softwarerdquo in Proceedings of theConference on Languages Compilers and Tools for EmbeddedSystems (Lctes rsquo05) vol 40 pp 59ndash68 2005

[25] J Hil R Szewczyk A Woo S Hollar D Culler and K PisterldquoSystem architecture directions for networked sensorsrdquo inProceedings of the 9th International Conference on ArchitecturalSupport for Programming Languages and Operating Systems(ASPLOS rsquo00) pp 93ndash104 2000

[26] GMTchamgoue KHKim andYK Jun ldquoTesting anddebug-ging concurrency bugs in event-driven programsrdquo InternationalJournal of Advanced Science and Technology vol 40 pp 55ndash682012

[27] GM Tchamgoue O K Ha K H Kim and Y K Jun ldquoA frame-work for on-the-fly race healing in ARINC-653 applicationsrdquoInternational Journal of Hybrid Information Technology vol 4pp 1ndash12 2011

[28] A Dinning and E Schonberg ldquoDetecting access anomaliesin programs with critical sectionsrdquo in Proceedings of theACMONR Workshop on Parallel and Distributed Debugging(PADD rsquo91) vol 26 pp 85ndash96 1991

[29] L Lamport ldquoTime clocks and the ordering of events in adistributed systemrdquo Communications of the ACM vol 21 no7 pp 558ndash565 1978

[30] Atmel inc ATmega128 Datasheet 2011 httpwwwatmelcomImagesdoc2467pdf

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 9: Research Article Verification of Data Races in Concurrent ...downloads.hindawi.com/journals/ijdsn/2013/953593.pdf · photoTask I20 W 11 R 12 W 13 W 24 R 15 T 2 J12 T 1 F : A POEG

International Journal of Distributed Sensor Networks 9

Table 2 Characteristics of the benchmark applications from TinyOS 210 [10]

Applications LOC Int SV Brief description6LowPanCli 19116 4 9 Implements a command line interface using the 6lowpan stackAnthiTheft 27421 5 4 Detects and reports a stolen mote by checking some sensorsBaseStation 14905 6 13 Sends and receives simple active messages on the radio and serial linksBlink 3688 1 0 Periodically blinks the three LEDs of a moteMultihopOscilloscope 24310 9 8 Periodically samples and broadcasts sensor data to a base stationOscilloscope 11686 5 5 Periodically samples sensor data and broadcasts a message every 10 timesPowerup 2352 0 0 Turns on the red LED when the mote powers upRadioCountToLeds 11165 4 3 Periodically samples sensors and displays the last 3 bits on its LEDsRadioSenseToLeds 11472 5 2 Periodically update a counter and displays the last 3 bits on its LEDsSense 5743 2 0 Periodically takes sensor readings and displays the values on the LEDs

of the Avrora [23] simulation and analysis framework thatprovides a time accurate simulator different analysis toolsknown asmonitors and a binary instrumentation engine [24]for programs written for AVR microcontrollers and sensornetworks All these features make Avrora just suitable forour tool Simulation is commonly used in the developmentcycle of embedded software because target systems generallylack conventional inputoutput interfaces With iRace thefunctionalities of Avrora are extendedwith a precise data racedetection tool that can easily be loaded as one of Avrorarsquosmonitors

As shown in Figure 3 iRace consists of two maincomponents the Instrumenter which exploits the Avrorainstrumentation engine [24] and the core Race Detectoritself The Instrumenter monitors task and interrupts activ-ities through its iMonitor component and provides entrypoints for the labeling routines Instrumentation points areinserted at the beginning of each task at the start andexit of each interrupt handler and also before interruptsare either enabled or disabled Similarly accesses to sharedmemory locations are monitored using the AccessMonitorcomponent that also calls routines of the detection proto-col The AccessMonitor instruments before read and writeaccesses to shared variables The Race Detector is composedof labeling routines that generate and manage labels for eachactive thread segment at runtime and detection routines formaintaining the access history and reporting data races on-the-fly as shared variables are manipulated

In TinyOS [21 25] interrupt handlers and tasks all runto completion unless preempted by other interrupt handlersThus only a single thread runs at any point of time givingus the opportunity to use only a single variable to maintainthe active thread label at detection time However whenpreemption happens the label of the preempted threadsegment must be saved and restored again when the activeinterrupt handler exits Otherwise there will be no way toknow about the concurrency information of the preemptedthread segment when it gains back the execution control Inorder tomanage the preemption we use a stack data structurewhere labels can be pushed and restored from since interruptscan be nested This provides a simple but powerful way tohandle preemption with negligible overhead

Our labeling scheme needs to uniquely identify eachinterrupt handler whenever it is invoked and each taskwhen it starts its execution Each interrupt handler is easilyidentified by the interrupt number of the interrupt it servesFor task we chose to generate a unique task identifier bysimply maintaining a counter that increases at each timea task starts its execution Thus each task is attributed aunique identifier thanks to the fact that once posted TinyOStasks run in a first-in first-out manner and cannot preempteach other By manipulating only binary files iRace doesnot require any source file nor source code annotation toeffectively detect data races However compiling the programwith debugging information may help iRace generate moredetailed debugging information on data races it reports

62 Environment We implemented and evaluated our tech-nique on a system that runs the kernel 30 of the Linuxoperating system using an Intel dual-core 320GHz and 4GBof mainmemory On this machine was installed TinyOS 210[10] along with the nesC compiler 131 the GNU C compiler46 the Java Runtime Environment 17 and the latest Avroraframework version 17115

Table 2 presents the characteristics and a brief descriptionof the benchmark applications selected for our evaluationfrom the TinyOS 210 repository [10] These applications arefreely distributed with each version of TinyOS The columnldquoLOCrdquo indicates the number of line of C codes generatedby the nesC compiler after compilation while the columnldquoIntrdquo contains the number of interrupts handled by eachapplicationThe number of shared variables for each programappears in column ldquoSVrdquo It is worthmentioning that this num-ber represents only the shared variables in the source code ofeach program The exact number of variables shared by eachapplicationmay finally depend on the number of componentsit shares with the operating system itself Applications writtenin nesC share TinyOS components by wiring themselvesto these components increasing therefore their number ofshared resources We compiled each application for theMica2 platform and measured the average runtime overheadover five executions The Mica2 motes are sensor networknodes based on the ATmega128 [30] microcontrollers Weconfigured Avrora to simulate a sensor network during 10

10 International Journal of Distributed Sensor Networks

iRace

and instrumentationframework

iMonitor

AccessMonitor

Race detector

Labeling routines

Detection protocolroutines

Binarycode Data

races

Avrora simulation

Figure 3 The Architecture of the Data Race Detection Tool

Table 3 Reported data races on the benchmark applications

Applications Data races Verif races Time (Sec) Memory (Kb)Original iRace Original iRace

6LowPanCli 0 0 269 357 1936960 2168160AnthiTheft 15 15 292 457 1944400 2320000BaseStation 0 0 279 376 1912240 2148240Blink 0 0 161 180 1887280 1949600MultihopOscilloscope 0 0 278 400 1939600 2196960Oscilloscope 13 13 179 328 1909360 2201760Powerup 0 0 151 186 1873600 1879920RadioCountToLeds 17 17 196 1255 1942880 2326320RadioSenseToLeds 15 15 181 1305 1944480 2298240Sense 0 0 158 192 1866400 2050560

seconds allowing each application to run exactly for 73728000instruction cycles We set to five the number of nodes in thesimulated sensor network

63 Evaluation For data race detectionwe instructedAvrorato simulate a sensor network with five nodes while thebenchmark applications are monitored by iRace Table 3represents the results of the evaluation of iRace on thebenchmark applications in its column ldquoData Racesrdquo whichcontains the number of shared variables involved in adata race for each application Our tool reported no datarace for all applications except for four AnthiTheft with15 shared variables Oscilloscope with 13 shared vari-ables RadioCountToLeds with 17 shared variables andRadioSenseToLeds with 15 shared variables Surprisinglymost of the reported data races reside inside some sharedsystem components of TinyOS itself The reported data racesare caused by two different interrupts namely the serialperipheral interface (SPI) interrupt and the timer interruptAlso we carefully analyzed the reported data races andinspected the source code of the applications for a manualverification process as shown in the column ldquoVerified racesrdquoof Table 3

In the AnthiTheft application for example which isused to detect and report a stolen mote by checking the levelof the light or the acceleration data races were reported over

15 shared variables Most of the reported data races are how-ever due to the fact that the SPI interrupt handler can preemptsome functions or tasks owned by the underlying radio stackThis implies that if a message is being received or sent whena SPI interrupt occurs shared variables like radioState orcount in the component CC1000SendReceiveP might becorrupted However all shared variables involved in a datarace do not directly belong to AnthiTheft but to functionsof the CC1000 radio stack in use in Mica2 motes Similardata races are also reported in RadioCountToLeds andRadioSenseToLeds since they all access the radio stack

Another data race reported both in RadioCountToLedsand RadioSenseToLeds and worthy to be commentedconcerns the shared variable reqResId inside thecomponent ArbiterP (Recently this data race hasalso been pointed out and a patch discussed in theTinyOS Forum) This data race occurs when the functionimmediateRequest() called within the function read ofthe component immediateRequest is preempted by a SPIinterrupt handler An excerpt of this function is presentedin Algorithm 4 At line 2 of Algorithm 4 the handle of aresource is requested and eventually locked out if grantedHowever if a SPI interrupt is received somewhere afterline 2 a data race will occur and the resource handlerstored in reqResId (line 6 Algorithm 4) may eventually beoverwritten by the SPI interrupt handler Also other shared

International Journal of Distributed Sensor Networks 11

(1) async command error t ResourceimmediateRequest [uint8 t id] ( ) (2) signal ResourceRequested immediateRequested [resId] ( )(3) atomic (4) if (state == RES CONTROLLED) (5) state = RES IMM GRANTING(6) reqResId = id(7)

(8) else return FAIL(9)

(10) signal ResourceDefaultOwner immediateRequested( )(11) sdot sdot sdot (12)

Algorithm 4 An excerpt of the request function from the component ArbiterP

(1) command error t Send send[uint8 t clientId] (message t lowastmsg(2) uint8 t len) (3) sdot sdot sdot (4) queue [clientId] msg = msg(5) call Packet setPayloadLength (msg len)(6) sdot sdot sdot (7)

Algorithm 5 An excerpt of the send function from the component AMSend

variables in ArbiterP like state can be corrupted as wellby the SPI interrupt handler

In the case of Oscilloscope which is a data collectionapplication that periodically samples the default sensor ofa mote and broadcasts a message over the radio every 10readings the reported data races are similar to those reportedin AnthiTheft Moreover our results suggest that if a SPIinterrupt arrives when a message is being transmitted viathe send() function of the AMSend interface the sharedbuffer sendBuf used by Oscilloscope to store incomingmessages will be updated and a corruptedmessage deliveredThe extract of code from the function Send presentedin Algorithm 5 shows that the variable msg which inthis case is a pointer to the shared variable sendBuf inOscilloscope can be updated at any time if the functionsend() is preempted because operations are not performedin a critical section This data race is somehow similar to theone we described in the example program of Algorithm 1Wealso find in Oscilloscope some of the data races reportedin the other applicationsThe results in Table 3 show that ourtool precisely detects data races on-the-fly in interrupt-drivenprograms without any false positives

64 Detection Overhead In order to evaluate the impactof our technique on the monitored programs we config-ured Avrora to simulate a single node sensor network Wethen measured the average runtime overhead and memoryspace required by each application of the benchmark ofTable 2 to execute 73728000 instruction cycles The totaloverhead of our technique mainly comes from actions of theinstrumentation engine the labeling routines and the race

Exec

utio

ns ti

me (

s)

14

12

10

8

6

4

2

0

TinyOS 210 programsOriginalWith iRace

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 4 Average runtime overhead

detection engine which also has the responsibility to createandmaintain the access historiesThe average execution timeand memory consumption collected for each application arepresented in Table 3 The columns ldquoOriginalrdquo and ldquoiRacerdquorepresent the execution time and memory consumption foreach application without and with iRace respectively

Figure 4 which plots the data from the column ldquoTimerdquoof Table 3 compares the average execution times of each

12 International Journal of Distributed Sensor Networks

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

TinyOS 210 programsOriginalWith iRace

Figure 5 Average memory consumption

application for five runs with and without iRace The captionldquoOriginalrdquo represents the native execution of the applica-tions without iRace The results of Figure 4 suggest thatthe average runtime overhead of our tool is only about1x for all applications except for RadioCountToLeds andRadioSenseToLeds that reaches 6x the original executiontime These two applications sample and display data at highfrequency rate (eg 4Hz in our case) and therefore arelikely to receive a higher number of interrupts at runtimeThis low runtime overhead is achieved by exploiting the factthat shared variables are stored by Avrora at program loadingtime Instead of monitoring all memory accesses to filterthose targeting shared variables iRace precisely instrumentsonly accesses to the well-known set of shared memory loca-tions obtained from the Avrora loader This technique allowsthe tool to drastically cuts down its monitoring overhead andto increase its scalability

Likewise wemeasured the averagememory consumptionfor each application during five runs The collected data arerepresented in Figure 5 The overall average memory con-sumption of the tool is very small at just about 20 above theoriginal memory requirement even for high frequency appli-cations like RadioCountToLeds and RadioSenseToLedsthanks to our labeling scheme that reuses the same labelfor different instances of the same interrupt handler and thedetection protocol that maintains only a reduced size accesshistoryThis result corroborates the theoretical analysis of ourtechnique as summarized in Table 1 The low runtime over-head and memory consumption of our on-the-fly techniquemake it very scalable and suitable even for interrupt-drivenprograms that handle interrupts at high frequency rates

65 Labeling Overhead Since the race detection subcompo-nent of iRace is in charge of implementing the labeling and

Table 4 Average labeling overhead

Applications Time (Sec) Memory (Kb)6LowPanCli 313 2101760AnthiTheft 308 2148880BaseStation 309 2077680Blink 174 1964640MultihopOscilloscope 313 2141920Oscilloscope 249 2092640Powerup 176 1894720RadioCountToLeds 218 2167440RadioSenseToLeds 224 2140080Sense 184 2011040

Table 5The states of an access history for detecting data races withour protocol considering the example of Figure 2

119886119894

Read Write Races11988211

⟨0 20 0⟩ ⟨20⟩

11987712

⟨0 20 0⟩ ⟨20⟩

11988213

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11988213-11988211

11988224

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩11987715

⟨1 1 0⟩ ⟨mdash⟩ ⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11987715-11988211

the detection routines it was legitimate to analyze the impactof the labeling scheme alone on the monitored programsat runtime For this purpose we deactivated the detectionengine of iRace before running it again on the benchmarkapplications We collected the average execution time andmemory space consumed by each application after five runsas shown in Table 4 The simulation was done using a singlenode network

Figure 6 compares the original execution time of eachprogram from Table 3 with the execution time of the labelingengine as presented in the column ldquoTimerdquo of Table 4 Thecomparison shows that the average runtime overhead ofiRace with just the labeling engine activated is only about04x in the worst-case This represents only a small fractionof the average runtime overhead of 6x imposed by iRace asdiscussed in Section 64Themain reason of this performancecan be found at source level When browsing the source codeof the benchmark applications we came to understand thatprogrammers generally disable and enable all the interruptsat once instead of focusing on particular interrupts In thiscase the size of each label generated by the labeling enginenever approaches its worst-case of 119874(119879) meaning that thecomparison overhead of labels is also reduced This is veryimportant because as discussed in Section 64 althoughthe size of labels does not depend on the total number ofinterrupts received at runtime this number still has impacton the overall detection overheadThus we can conclude thatthe labeling engine of iRace incurs a very low overhead atruntime making it a highly scalable tool

We also compared the memory consumption of thelabeling engine with the memory requirement of the original

International Journal of Distributed Sensor Networks 13

TinyOS 210 programs

4

35

3

25

2

15

1

05

0

Exec

utio

n tim

e (s)

With iRace labeling engineOriginal

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 6 Average runtime overhead of the labeling engine

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

With iRace labeling engineOriginal

TinyOS 210 programs

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 7 Average memory consumption by the labeling engine

program execution presented inTable 3The results plotted inFigure 7 show that when the labeling engine alone is runningiRace consumes only about 12 more memory on averageHowever during the detection phase the overall averagememory consumption of iRace is only about 20 Theseresults also tell us that the detection engine does not increasethat much the memory consumption of iRace This is alsoreasonable because we only disabled the detection engine ofiRace but did not remove the resources it used

7 Conclusions

Asynchronous interrupt handling introduces fine-grainedconcurrency into interrupt-driven programs making themprone to data races In this paper we presented an efficientand scalable on-the-fly technique to precisely detect dataraces without false positives in such programs Our techniqueintroduced a lightweight labeling scheme for maintainingconcurrency information for the main program and each ofits interrupt handlers and a precise detection protocol thatguarantees to verify the existence of data races in a programby reporting at least one data race for each shared variableinvolved in a data race

We implemented a prototype of our technique as a racedetection tool called iRace on top of the Avrora simulationframework An evaluation of iRace revealed the existence ofdata races in some programs distributed with TinyOS andin some TinyOS components while incurring a slowdownof only about 6 times on average in the worst-case with anaverage memory consumption of only about 20 above theoriginal memory requirements of the program Moreoverthis evaluation also proved that the labeling engine alonegenerates an average runtime overhead of only about 04xwith an average memory consumption of only about 12relatively to the original program execution These resultscoupled with a theoretical analysis showed that the proposedtechnique is highly scalable and practical even for largeinterrupt-driven programs

Detecting data races that occur first in a program isan important issue since it may make other data races todisappear It is therefore interesting to detect this class of dataraces in interrupt-driven programs to facilitate the debuggingprocess Also we may be interested in extending our work tosupport multithreaded programs that handle interrupts andalso to detect data races in low-level kernel codes

Acknowledgment

This research was financially supported by the Ministryof Education (MOE) and National Research Foundation ofKorea (NRF) through the Human Resource Training Projectfor Regional Innovation and BK21+ program

References

[1] U Banerjee B Bliss Z Ma and P Petersen ldquoA theory of datarace detectionrdquo in Proceedings of the Workshop on Parallel andDistributed Systems Testing and Debugging (PADTAD rsquo06) pp69ndash78 July 2006

[2] R H B Netzer and B P Miller ldquoWhat are race conditionsSome issues and formalizationsrdquo ACM letters on programminglanguages and systems vol 1 no 1 pp 74ndash88 1992

[3] N G Leveson and C S Turner ldquoInvestigation of theTherac-25accidentsrdquo Computer vol 26 no 7 pp 18ndash41 1993

[4] USA Canada Final Report on the August 14 2003 Blackout inthe United States and Canada Causes and RecommendationsUS-Canada Power System Outage Task Force 2004

[5] J Erickson M Musuvathi S Burckhardt and K OlynykldquoEffective data-race detection for the kernelrdquo in Proceedings of

14 International Journal of Distributed Sensor Networks

the 9th Conference on Operating Systems Design and Implemen-tation (USENIX rsquo10) 2010

[6] B-K Lee M-H Kang K C Park J S Yi S W Yang andY-K Jun ldquoProgram conversion for detecting data races inconcurrent interrupt handlersrdquo Communications in Computerand Information Science vol 257 pp 407ndash415 2011

[7] L Effinger-Dean B Lucia L Ceze D Grossman and H JBoehm ldquoIFRit interference-free regions for dynamic data-racedetectionrdquo in Proceedings of the ACM International Conferenceon Object Oriented Programming Systems Languages and Appli-cations (OOPSLA rsquo12) vol 47 pp 467ndash484 2012

[8] C Flanagan and S N Freund ldquoFastTrack efficient and precisedynamic race detectionrdquo in Proceedings of the ACM Interna-tional Conference on Object Oriented Programming SystemsLanguages and Applications (OOPSLA rsquo09) vol 44 pp 121ndash1332009

[9] X Xie and J Xue ldquoAcculock accurate and efficient detection ofdata racesrdquo in Proceedings of the 9th International Symposium onCode Generation and Optimization (CGO rsquo11) pp 201ndash212 April2011

[10] TinyOSCommunity ForumTinyOSAnopen source operatingsystem for low-power wireless devices 2008 httpwwwtinyosnet

[11] M Higashi T Yamamoto Y Hayase T Ishio and K InoueldquoAn effective method to control interrupt handler for datarace detectionrdquo in International Workshop on Automation ofSoftware Test (ACM rsquo10) pp 79ndash86 May 2010

[12] Z Lai S C Cheung and W K Chan ldquoInter-context control-flow and data-flow test adequacy criteria for nesC applicationsrdquoin Proceedings of the 16th ACM SIGSOFT International Sympo-sium on the Foundations of Software Engineering (SIGSOFT rsquo08)pp 94ndash104 November 2008

[13] J Regehr ldquoRandom testing of interrupt-driven softwarerdquo inPro-ceedings of the 5th ACM International Conference on EmbeddedSoftware (EMSOFT rsquo05) pp 290ndash298 September 2005

[14] Y Zhou X Chen M R Lyu and J Liu ldquoSentomist Unveil-ing transient sensor network bugs via symptom miningrdquoin Proceedings of the 30th IEEE International Conference onDistributed Computing Systems (ICDCS rsquo10) pp 784ndash794 June2010

[15] G Gracioli and S Fischmeister ldquoTracing interrupts in embed-ded softwarerdquo in Proceedings of the ACM SIGPLANSIGBEDConference on Languages Compilers and Tools for EmbeddedSystems (LCTES rsquo09) vol 44 pp 137ndash146 2009

[16] V Sundaram P Eugster and X Zhang ldquoLightweight tracingfor wireless sensor networks debuggingrdquo in Proceedings of the4th International Workshop on Middleware Tools Services andRun-Time Support for Sensor Networks (ACM rsquo09) pp 13ndash18December 2009

[17] O K Ha and Y K Jun ldquoMonitoring of programs with nestedparallelismusing efficient thread labelingrdquo International JournalFo U- and E- Service Science and Technology vol 4 pp 21ndash362011

[18] Y C Kim S S Jun and Y K Jun ldquoA tool for space-efficient on-the-fly race detectionrdquo International Journal of DatabaseTheoryand Application vol 4 pp 25ndash38 2011

[19] S Savage M Burrows G Nelson P Sobalvarro and T Ander-son ldquoEraser a dynamic data race detector for multithreadedprogramsrdquo ACM Transactions on Computer Systems vol 15 no4 pp 391ndash411 1997

[20] A Jannesari and W F Tichy ldquoOn-the-fly race detection inmulti-threaded programsrdquo in Proceedings of the 6th Workshop

on Parallel and Distributed Systems Testing Analysis andDebugging (PADTAD rsquo08) July 2007

[21] D Gay M Welsh P Levis E Brewer R Von Behren and DCuller ldquoThe nesC language a holistic approach to networkedembedded systemsrdquo inACMSIGPLANConference on Program-ming Language Design and Implementation pp 1ndash11 June 2003

[22] J Regehr and N Cooprider ldquoInterrupt verification via threadverificationrdquo Electronic Notes in Theoretical Computer Sciencevol 174 no 9 pp 139ndash150 2007

[23] B L Titzer D K Lee and J Palsberg ldquoAvrora scalable sensornetwork simulation with precise timingrdquo in Proceedings ofthe 4th International Symposium on Information Processing inSensor Networks (IPSN rsquo05) pp 477ndash482 April 2005

[24] B L Titzer and J Palsberg ldquoNonintrusive precision instru-mentation of microcontroller softwarerdquo in Proceedings of theConference on Languages Compilers and Tools for EmbeddedSystems (Lctes rsquo05) vol 40 pp 59ndash68 2005

[25] J Hil R Szewczyk A Woo S Hollar D Culler and K PisterldquoSystem architecture directions for networked sensorsrdquo inProceedings of the 9th International Conference on ArchitecturalSupport for Programming Languages and Operating Systems(ASPLOS rsquo00) pp 93ndash104 2000

[26] GMTchamgoue KHKim andYK Jun ldquoTesting anddebug-ging concurrency bugs in event-driven programsrdquo InternationalJournal of Advanced Science and Technology vol 40 pp 55ndash682012

[27] GM Tchamgoue O K Ha K H Kim and Y K Jun ldquoA frame-work for on-the-fly race healing in ARINC-653 applicationsrdquoInternational Journal of Hybrid Information Technology vol 4pp 1ndash12 2011

[28] A Dinning and E Schonberg ldquoDetecting access anomaliesin programs with critical sectionsrdquo in Proceedings of theACMONR Workshop on Parallel and Distributed Debugging(PADD rsquo91) vol 26 pp 85ndash96 1991

[29] L Lamport ldquoTime clocks and the ordering of events in adistributed systemrdquo Communications of the ACM vol 21 no7 pp 558ndash565 1978

[30] Atmel inc ATmega128 Datasheet 2011 httpwwwatmelcomImagesdoc2467pdf

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 10: Research Article Verification of Data Races in Concurrent ...downloads.hindawi.com/journals/ijdsn/2013/953593.pdf · photoTask I20 W 11 R 12 W 13 W 24 R 15 T 2 J12 T 1 F : A POEG

10 International Journal of Distributed Sensor Networks

iRace

and instrumentationframework

iMonitor

AccessMonitor

Race detector

Labeling routines

Detection protocolroutines

Binarycode Data

races

Avrora simulation

Figure 3 The Architecture of the Data Race Detection Tool

Table 3 Reported data races on the benchmark applications

Applications Data races Verif races Time (Sec) Memory (Kb)Original iRace Original iRace

6LowPanCli 0 0 269 357 1936960 2168160AnthiTheft 15 15 292 457 1944400 2320000BaseStation 0 0 279 376 1912240 2148240Blink 0 0 161 180 1887280 1949600MultihopOscilloscope 0 0 278 400 1939600 2196960Oscilloscope 13 13 179 328 1909360 2201760Powerup 0 0 151 186 1873600 1879920RadioCountToLeds 17 17 196 1255 1942880 2326320RadioSenseToLeds 15 15 181 1305 1944480 2298240Sense 0 0 158 192 1866400 2050560

seconds allowing each application to run exactly for 73728000instruction cycles We set to five the number of nodes in thesimulated sensor network

63 Evaluation For data race detectionwe instructedAvrorato simulate a sensor network with five nodes while thebenchmark applications are monitored by iRace Table 3represents the results of the evaluation of iRace on thebenchmark applications in its column ldquoData Racesrdquo whichcontains the number of shared variables involved in adata race for each application Our tool reported no datarace for all applications except for four AnthiTheft with15 shared variables Oscilloscope with 13 shared vari-ables RadioCountToLeds with 17 shared variables andRadioSenseToLeds with 15 shared variables Surprisinglymost of the reported data races reside inside some sharedsystem components of TinyOS itself The reported data racesare caused by two different interrupts namely the serialperipheral interface (SPI) interrupt and the timer interruptAlso we carefully analyzed the reported data races andinspected the source code of the applications for a manualverification process as shown in the column ldquoVerified racesrdquoof Table 3

In the AnthiTheft application for example which isused to detect and report a stolen mote by checking the levelof the light or the acceleration data races were reported over

15 shared variables Most of the reported data races are how-ever due to the fact that the SPI interrupt handler can preemptsome functions or tasks owned by the underlying radio stackThis implies that if a message is being received or sent whena SPI interrupt occurs shared variables like radioState orcount in the component CC1000SendReceiveP might becorrupted However all shared variables involved in a datarace do not directly belong to AnthiTheft but to functionsof the CC1000 radio stack in use in Mica2 motes Similardata races are also reported in RadioCountToLeds andRadioSenseToLeds since they all access the radio stack

Another data race reported both in RadioCountToLedsand RadioSenseToLeds and worthy to be commentedconcerns the shared variable reqResId inside thecomponent ArbiterP (Recently this data race hasalso been pointed out and a patch discussed in theTinyOS Forum) This data race occurs when the functionimmediateRequest() called within the function read ofthe component immediateRequest is preempted by a SPIinterrupt handler An excerpt of this function is presentedin Algorithm 4 At line 2 of Algorithm 4 the handle of aresource is requested and eventually locked out if grantedHowever if a SPI interrupt is received somewhere afterline 2 a data race will occur and the resource handlerstored in reqResId (line 6 Algorithm 4) may eventually beoverwritten by the SPI interrupt handler Also other shared

International Journal of Distributed Sensor Networks 11

(1) async command error t ResourceimmediateRequest [uint8 t id] ( ) (2) signal ResourceRequested immediateRequested [resId] ( )(3) atomic (4) if (state == RES CONTROLLED) (5) state = RES IMM GRANTING(6) reqResId = id(7)

(8) else return FAIL(9)

(10) signal ResourceDefaultOwner immediateRequested( )(11) sdot sdot sdot (12)

Algorithm 4 An excerpt of the request function from the component ArbiterP

(1) command error t Send send[uint8 t clientId] (message t lowastmsg(2) uint8 t len) (3) sdot sdot sdot (4) queue [clientId] msg = msg(5) call Packet setPayloadLength (msg len)(6) sdot sdot sdot (7)

Algorithm 5 An excerpt of the send function from the component AMSend

variables in ArbiterP like state can be corrupted as wellby the SPI interrupt handler

In the case of Oscilloscope which is a data collectionapplication that periodically samples the default sensor ofa mote and broadcasts a message over the radio every 10readings the reported data races are similar to those reportedin AnthiTheft Moreover our results suggest that if a SPIinterrupt arrives when a message is being transmitted viathe send() function of the AMSend interface the sharedbuffer sendBuf used by Oscilloscope to store incomingmessages will be updated and a corruptedmessage deliveredThe extract of code from the function Send presentedin Algorithm 5 shows that the variable msg which inthis case is a pointer to the shared variable sendBuf inOscilloscope can be updated at any time if the functionsend() is preempted because operations are not performedin a critical section This data race is somehow similar to theone we described in the example program of Algorithm 1Wealso find in Oscilloscope some of the data races reportedin the other applicationsThe results in Table 3 show that ourtool precisely detects data races on-the-fly in interrupt-drivenprograms without any false positives

64 Detection Overhead In order to evaluate the impactof our technique on the monitored programs we config-ured Avrora to simulate a single node sensor network Wethen measured the average runtime overhead and memoryspace required by each application of the benchmark ofTable 2 to execute 73728000 instruction cycles The totaloverhead of our technique mainly comes from actions of theinstrumentation engine the labeling routines and the race

Exec

utio

ns ti

me (

s)

14

12

10

8

6

4

2

0

TinyOS 210 programsOriginalWith iRace

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 4 Average runtime overhead

detection engine which also has the responsibility to createandmaintain the access historiesThe average execution timeand memory consumption collected for each application arepresented in Table 3 The columns ldquoOriginalrdquo and ldquoiRacerdquorepresent the execution time and memory consumption foreach application without and with iRace respectively

Figure 4 which plots the data from the column ldquoTimerdquoof Table 3 compares the average execution times of each

12 International Journal of Distributed Sensor Networks

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

TinyOS 210 programsOriginalWith iRace

Figure 5 Average memory consumption

application for five runs with and without iRace The captionldquoOriginalrdquo represents the native execution of the applica-tions without iRace The results of Figure 4 suggest thatthe average runtime overhead of our tool is only about1x for all applications except for RadioCountToLeds andRadioSenseToLeds that reaches 6x the original executiontime These two applications sample and display data at highfrequency rate (eg 4Hz in our case) and therefore arelikely to receive a higher number of interrupts at runtimeThis low runtime overhead is achieved by exploiting the factthat shared variables are stored by Avrora at program loadingtime Instead of monitoring all memory accesses to filterthose targeting shared variables iRace precisely instrumentsonly accesses to the well-known set of shared memory loca-tions obtained from the Avrora loader This technique allowsthe tool to drastically cuts down its monitoring overhead andto increase its scalability

Likewise wemeasured the averagememory consumptionfor each application during five runs The collected data arerepresented in Figure 5 The overall average memory con-sumption of the tool is very small at just about 20 above theoriginal memory requirement even for high frequency appli-cations like RadioCountToLeds and RadioSenseToLedsthanks to our labeling scheme that reuses the same labelfor different instances of the same interrupt handler and thedetection protocol that maintains only a reduced size accesshistoryThis result corroborates the theoretical analysis of ourtechnique as summarized in Table 1 The low runtime over-head and memory consumption of our on-the-fly techniquemake it very scalable and suitable even for interrupt-drivenprograms that handle interrupts at high frequency rates

65 Labeling Overhead Since the race detection subcompo-nent of iRace is in charge of implementing the labeling and

Table 4 Average labeling overhead

Applications Time (Sec) Memory (Kb)6LowPanCli 313 2101760AnthiTheft 308 2148880BaseStation 309 2077680Blink 174 1964640MultihopOscilloscope 313 2141920Oscilloscope 249 2092640Powerup 176 1894720RadioCountToLeds 218 2167440RadioSenseToLeds 224 2140080Sense 184 2011040

Table 5The states of an access history for detecting data races withour protocol considering the example of Figure 2

119886119894

Read Write Races11988211

⟨0 20 0⟩ ⟨20⟩

11987712

⟨0 20 0⟩ ⟨20⟩

11988213

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11988213-11988211

11988224

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩11987715

⟨1 1 0⟩ ⟨mdash⟩ ⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11987715-11988211

the detection routines it was legitimate to analyze the impactof the labeling scheme alone on the monitored programsat runtime For this purpose we deactivated the detectionengine of iRace before running it again on the benchmarkapplications We collected the average execution time andmemory space consumed by each application after five runsas shown in Table 4 The simulation was done using a singlenode network

Figure 6 compares the original execution time of eachprogram from Table 3 with the execution time of the labelingengine as presented in the column ldquoTimerdquo of Table 4 Thecomparison shows that the average runtime overhead ofiRace with just the labeling engine activated is only about04x in the worst-case This represents only a small fractionof the average runtime overhead of 6x imposed by iRace asdiscussed in Section 64Themain reason of this performancecan be found at source level When browsing the source codeof the benchmark applications we came to understand thatprogrammers generally disable and enable all the interruptsat once instead of focusing on particular interrupts In thiscase the size of each label generated by the labeling enginenever approaches its worst-case of 119874(119879) meaning that thecomparison overhead of labels is also reduced This is veryimportant because as discussed in Section 64 althoughthe size of labels does not depend on the total number ofinterrupts received at runtime this number still has impacton the overall detection overheadThus we can conclude thatthe labeling engine of iRace incurs a very low overhead atruntime making it a highly scalable tool

We also compared the memory consumption of thelabeling engine with the memory requirement of the original

International Journal of Distributed Sensor Networks 13

TinyOS 210 programs

4

35

3

25

2

15

1

05

0

Exec

utio

n tim

e (s)

With iRace labeling engineOriginal

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 6 Average runtime overhead of the labeling engine

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

With iRace labeling engineOriginal

TinyOS 210 programs

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 7 Average memory consumption by the labeling engine

program execution presented inTable 3The results plotted inFigure 7 show that when the labeling engine alone is runningiRace consumes only about 12 more memory on averageHowever during the detection phase the overall averagememory consumption of iRace is only about 20 Theseresults also tell us that the detection engine does not increasethat much the memory consumption of iRace This is alsoreasonable because we only disabled the detection engine ofiRace but did not remove the resources it used

7 Conclusions

Asynchronous interrupt handling introduces fine-grainedconcurrency into interrupt-driven programs making themprone to data races In this paper we presented an efficientand scalable on-the-fly technique to precisely detect dataraces without false positives in such programs Our techniqueintroduced a lightweight labeling scheme for maintainingconcurrency information for the main program and each ofits interrupt handlers and a precise detection protocol thatguarantees to verify the existence of data races in a programby reporting at least one data race for each shared variableinvolved in a data race

We implemented a prototype of our technique as a racedetection tool called iRace on top of the Avrora simulationframework An evaluation of iRace revealed the existence ofdata races in some programs distributed with TinyOS andin some TinyOS components while incurring a slowdownof only about 6 times on average in the worst-case with anaverage memory consumption of only about 20 above theoriginal memory requirements of the program Moreoverthis evaluation also proved that the labeling engine alonegenerates an average runtime overhead of only about 04xwith an average memory consumption of only about 12relatively to the original program execution These resultscoupled with a theoretical analysis showed that the proposedtechnique is highly scalable and practical even for largeinterrupt-driven programs

Detecting data races that occur first in a program isan important issue since it may make other data races todisappear It is therefore interesting to detect this class of dataraces in interrupt-driven programs to facilitate the debuggingprocess Also we may be interested in extending our work tosupport multithreaded programs that handle interrupts andalso to detect data races in low-level kernel codes

Acknowledgment

This research was financially supported by the Ministryof Education (MOE) and National Research Foundation ofKorea (NRF) through the Human Resource Training Projectfor Regional Innovation and BK21+ program

References

[1] U Banerjee B Bliss Z Ma and P Petersen ldquoA theory of datarace detectionrdquo in Proceedings of the Workshop on Parallel andDistributed Systems Testing and Debugging (PADTAD rsquo06) pp69ndash78 July 2006

[2] R H B Netzer and B P Miller ldquoWhat are race conditionsSome issues and formalizationsrdquo ACM letters on programminglanguages and systems vol 1 no 1 pp 74ndash88 1992

[3] N G Leveson and C S Turner ldquoInvestigation of theTherac-25accidentsrdquo Computer vol 26 no 7 pp 18ndash41 1993

[4] USA Canada Final Report on the August 14 2003 Blackout inthe United States and Canada Causes and RecommendationsUS-Canada Power System Outage Task Force 2004

[5] J Erickson M Musuvathi S Burckhardt and K OlynykldquoEffective data-race detection for the kernelrdquo in Proceedings of

14 International Journal of Distributed Sensor Networks

the 9th Conference on Operating Systems Design and Implemen-tation (USENIX rsquo10) 2010

[6] B-K Lee M-H Kang K C Park J S Yi S W Yang andY-K Jun ldquoProgram conversion for detecting data races inconcurrent interrupt handlersrdquo Communications in Computerand Information Science vol 257 pp 407ndash415 2011

[7] L Effinger-Dean B Lucia L Ceze D Grossman and H JBoehm ldquoIFRit interference-free regions for dynamic data-racedetectionrdquo in Proceedings of the ACM International Conferenceon Object Oriented Programming Systems Languages and Appli-cations (OOPSLA rsquo12) vol 47 pp 467ndash484 2012

[8] C Flanagan and S N Freund ldquoFastTrack efficient and precisedynamic race detectionrdquo in Proceedings of the ACM Interna-tional Conference on Object Oriented Programming SystemsLanguages and Applications (OOPSLA rsquo09) vol 44 pp 121ndash1332009

[9] X Xie and J Xue ldquoAcculock accurate and efficient detection ofdata racesrdquo in Proceedings of the 9th International Symposium onCode Generation and Optimization (CGO rsquo11) pp 201ndash212 April2011

[10] TinyOSCommunity ForumTinyOSAnopen source operatingsystem for low-power wireless devices 2008 httpwwwtinyosnet

[11] M Higashi T Yamamoto Y Hayase T Ishio and K InoueldquoAn effective method to control interrupt handler for datarace detectionrdquo in International Workshop on Automation ofSoftware Test (ACM rsquo10) pp 79ndash86 May 2010

[12] Z Lai S C Cheung and W K Chan ldquoInter-context control-flow and data-flow test adequacy criteria for nesC applicationsrdquoin Proceedings of the 16th ACM SIGSOFT International Sympo-sium on the Foundations of Software Engineering (SIGSOFT rsquo08)pp 94ndash104 November 2008

[13] J Regehr ldquoRandom testing of interrupt-driven softwarerdquo inPro-ceedings of the 5th ACM International Conference on EmbeddedSoftware (EMSOFT rsquo05) pp 290ndash298 September 2005

[14] Y Zhou X Chen M R Lyu and J Liu ldquoSentomist Unveil-ing transient sensor network bugs via symptom miningrdquoin Proceedings of the 30th IEEE International Conference onDistributed Computing Systems (ICDCS rsquo10) pp 784ndash794 June2010

[15] G Gracioli and S Fischmeister ldquoTracing interrupts in embed-ded softwarerdquo in Proceedings of the ACM SIGPLANSIGBEDConference on Languages Compilers and Tools for EmbeddedSystems (LCTES rsquo09) vol 44 pp 137ndash146 2009

[16] V Sundaram P Eugster and X Zhang ldquoLightweight tracingfor wireless sensor networks debuggingrdquo in Proceedings of the4th International Workshop on Middleware Tools Services andRun-Time Support for Sensor Networks (ACM rsquo09) pp 13ndash18December 2009

[17] O K Ha and Y K Jun ldquoMonitoring of programs with nestedparallelismusing efficient thread labelingrdquo International JournalFo U- and E- Service Science and Technology vol 4 pp 21ndash362011

[18] Y C Kim S S Jun and Y K Jun ldquoA tool for space-efficient on-the-fly race detectionrdquo International Journal of DatabaseTheoryand Application vol 4 pp 25ndash38 2011

[19] S Savage M Burrows G Nelson P Sobalvarro and T Ander-son ldquoEraser a dynamic data race detector for multithreadedprogramsrdquo ACM Transactions on Computer Systems vol 15 no4 pp 391ndash411 1997

[20] A Jannesari and W F Tichy ldquoOn-the-fly race detection inmulti-threaded programsrdquo in Proceedings of the 6th Workshop

on Parallel and Distributed Systems Testing Analysis andDebugging (PADTAD rsquo08) July 2007

[21] D Gay M Welsh P Levis E Brewer R Von Behren and DCuller ldquoThe nesC language a holistic approach to networkedembedded systemsrdquo inACMSIGPLANConference on Program-ming Language Design and Implementation pp 1ndash11 June 2003

[22] J Regehr and N Cooprider ldquoInterrupt verification via threadverificationrdquo Electronic Notes in Theoretical Computer Sciencevol 174 no 9 pp 139ndash150 2007

[23] B L Titzer D K Lee and J Palsberg ldquoAvrora scalable sensornetwork simulation with precise timingrdquo in Proceedings ofthe 4th International Symposium on Information Processing inSensor Networks (IPSN rsquo05) pp 477ndash482 April 2005

[24] B L Titzer and J Palsberg ldquoNonintrusive precision instru-mentation of microcontroller softwarerdquo in Proceedings of theConference on Languages Compilers and Tools for EmbeddedSystems (Lctes rsquo05) vol 40 pp 59ndash68 2005

[25] J Hil R Szewczyk A Woo S Hollar D Culler and K PisterldquoSystem architecture directions for networked sensorsrdquo inProceedings of the 9th International Conference on ArchitecturalSupport for Programming Languages and Operating Systems(ASPLOS rsquo00) pp 93ndash104 2000

[26] GMTchamgoue KHKim andYK Jun ldquoTesting anddebug-ging concurrency bugs in event-driven programsrdquo InternationalJournal of Advanced Science and Technology vol 40 pp 55ndash682012

[27] GM Tchamgoue O K Ha K H Kim and Y K Jun ldquoA frame-work for on-the-fly race healing in ARINC-653 applicationsrdquoInternational Journal of Hybrid Information Technology vol 4pp 1ndash12 2011

[28] A Dinning and E Schonberg ldquoDetecting access anomaliesin programs with critical sectionsrdquo in Proceedings of theACMONR Workshop on Parallel and Distributed Debugging(PADD rsquo91) vol 26 pp 85ndash96 1991

[29] L Lamport ldquoTime clocks and the ordering of events in adistributed systemrdquo Communications of the ACM vol 21 no7 pp 558ndash565 1978

[30] Atmel inc ATmega128 Datasheet 2011 httpwwwatmelcomImagesdoc2467pdf

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 11: Research Article Verification of Data Races in Concurrent ...downloads.hindawi.com/journals/ijdsn/2013/953593.pdf · photoTask I20 W 11 R 12 W 13 W 24 R 15 T 2 J12 T 1 F : A POEG

International Journal of Distributed Sensor Networks 11

(1) async command error t ResourceimmediateRequest [uint8 t id] ( ) (2) signal ResourceRequested immediateRequested [resId] ( )(3) atomic (4) if (state == RES CONTROLLED) (5) state = RES IMM GRANTING(6) reqResId = id(7)

(8) else return FAIL(9)

(10) signal ResourceDefaultOwner immediateRequested( )(11) sdot sdot sdot (12)

Algorithm 4 An excerpt of the request function from the component ArbiterP

(1) command error t Send send[uint8 t clientId] (message t lowastmsg(2) uint8 t len) (3) sdot sdot sdot (4) queue [clientId] msg = msg(5) call Packet setPayloadLength (msg len)(6) sdot sdot sdot (7)

Algorithm 5 An excerpt of the send function from the component AMSend

variables in ArbiterP like state can be corrupted as wellby the SPI interrupt handler

In the case of Oscilloscope which is a data collectionapplication that periodically samples the default sensor ofa mote and broadcasts a message over the radio every 10readings the reported data races are similar to those reportedin AnthiTheft Moreover our results suggest that if a SPIinterrupt arrives when a message is being transmitted viathe send() function of the AMSend interface the sharedbuffer sendBuf used by Oscilloscope to store incomingmessages will be updated and a corruptedmessage deliveredThe extract of code from the function Send presentedin Algorithm 5 shows that the variable msg which inthis case is a pointer to the shared variable sendBuf inOscilloscope can be updated at any time if the functionsend() is preempted because operations are not performedin a critical section This data race is somehow similar to theone we described in the example program of Algorithm 1Wealso find in Oscilloscope some of the data races reportedin the other applicationsThe results in Table 3 show that ourtool precisely detects data races on-the-fly in interrupt-drivenprograms without any false positives

64 Detection Overhead In order to evaluate the impactof our technique on the monitored programs we config-ured Avrora to simulate a single node sensor network Wethen measured the average runtime overhead and memoryspace required by each application of the benchmark ofTable 2 to execute 73728000 instruction cycles The totaloverhead of our technique mainly comes from actions of theinstrumentation engine the labeling routines and the race

Exec

utio

ns ti

me (

s)

14

12

10

8

6

4

2

0

TinyOS 210 programsOriginalWith iRace

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 4 Average runtime overhead

detection engine which also has the responsibility to createandmaintain the access historiesThe average execution timeand memory consumption collected for each application arepresented in Table 3 The columns ldquoOriginalrdquo and ldquoiRacerdquorepresent the execution time and memory consumption foreach application without and with iRace respectively

Figure 4 which plots the data from the column ldquoTimerdquoof Table 3 compares the average execution times of each

12 International Journal of Distributed Sensor Networks

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

TinyOS 210 programsOriginalWith iRace

Figure 5 Average memory consumption

application for five runs with and without iRace The captionldquoOriginalrdquo represents the native execution of the applica-tions without iRace The results of Figure 4 suggest thatthe average runtime overhead of our tool is only about1x for all applications except for RadioCountToLeds andRadioSenseToLeds that reaches 6x the original executiontime These two applications sample and display data at highfrequency rate (eg 4Hz in our case) and therefore arelikely to receive a higher number of interrupts at runtimeThis low runtime overhead is achieved by exploiting the factthat shared variables are stored by Avrora at program loadingtime Instead of monitoring all memory accesses to filterthose targeting shared variables iRace precisely instrumentsonly accesses to the well-known set of shared memory loca-tions obtained from the Avrora loader This technique allowsthe tool to drastically cuts down its monitoring overhead andto increase its scalability

Likewise wemeasured the averagememory consumptionfor each application during five runs The collected data arerepresented in Figure 5 The overall average memory con-sumption of the tool is very small at just about 20 above theoriginal memory requirement even for high frequency appli-cations like RadioCountToLeds and RadioSenseToLedsthanks to our labeling scheme that reuses the same labelfor different instances of the same interrupt handler and thedetection protocol that maintains only a reduced size accesshistoryThis result corroborates the theoretical analysis of ourtechnique as summarized in Table 1 The low runtime over-head and memory consumption of our on-the-fly techniquemake it very scalable and suitable even for interrupt-drivenprograms that handle interrupts at high frequency rates

65 Labeling Overhead Since the race detection subcompo-nent of iRace is in charge of implementing the labeling and

Table 4 Average labeling overhead

Applications Time (Sec) Memory (Kb)6LowPanCli 313 2101760AnthiTheft 308 2148880BaseStation 309 2077680Blink 174 1964640MultihopOscilloscope 313 2141920Oscilloscope 249 2092640Powerup 176 1894720RadioCountToLeds 218 2167440RadioSenseToLeds 224 2140080Sense 184 2011040

Table 5The states of an access history for detecting data races withour protocol considering the example of Figure 2

119886119894

Read Write Races11988211

⟨0 20 0⟩ ⟨20⟩

11987712

⟨0 20 0⟩ ⟨20⟩

11988213

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11988213-11988211

11988224

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩11987715

⟨1 1 0⟩ ⟨mdash⟩ ⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11987715-11988211

the detection routines it was legitimate to analyze the impactof the labeling scheme alone on the monitored programsat runtime For this purpose we deactivated the detectionengine of iRace before running it again on the benchmarkapplications We collected the average execution time andmemory space consumed by each application after five runsas shown in Table 4 The simulation was done using a singlenode network

Figure 6 compares the original execution time of eachprogram from Table 3 with the execution time of the labelingengine as presented in the column ldquoTimerdquo of Table 4 Thecomparison shows that the average runtime overhead ofiRace with just the labeling engine activated is only about04x in the worst-case This represents only a small fractionof the average runtime overhead of 6x imposed by iRace asdiscussed in Section 64Themain reason of this performancecan be found at source level When browsing the source codeof the benchmark applications we came to understand thatprogrammers generally disable and enable all the interruptsat once instead of focusing on particular interrupts In thiscase the size of each label generated by the labeling enginenever approaches its worst-case of 119874(119879) meaning that thecomparison overhead of labels is also reduced This is veryimportant because as discussed in Section 64 althoughthe size of labels does not depend on the total number ofinterrupts received at runtime this number still has impacton the overall detection overheadThus we can conclude thatthe labeling engine of iRace incurs a very low overhead atruntime making it a highly scalable tool

We also compared the memory consumption of thelabeling engine with the memory requirement of the original

International Journal of Distributed Sensor Networks 13

TinyOS 210 programs

4

35

3

25

2

15

1

05

0

Exec

utio

n tim

e (s)

With iRace labeling engineOriginal

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 6 Average runtime overhead of the labeling engine

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

With iRace labeling engineOriginal

TinyOS 210 programs

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 7 Average memory consumption by the labeling engine

program execution presented inTable 3The results plotted inFigure 7 show that when the labeling engine alone is runningiRace consumes only about 12 more memory on averageHowever during the detection phase the overall averagememory consumption of iRace is only about 20 Theseresults also tell us that the detection engine does not increasethat much the memory consumption of iRace This is alsoreasonable because we only disabled the detection engine ofiRace but did not remove the resources it used

7 Conclusions

Asynchronous interrupt handling introduces fine-grainedconcurrency into interrupt-driven programs making themprone to data races In this paper we presented an efficientand scalable on-the-fly technique to precisely detect dataraces without false positives in such programs Our techniqueintroduced a lightweight labeling scheme for maintainingconcurrency information for the main program and each ofits interrupt handlers and a precise detection protocol thatguarantees to verify the existence of data races in a programby reporting at least one data race for each shared variableinvolved in a data race

We implemented a prototype of our technique as a racedetection tool called iRace on top of the Avrora simulationframework An evaluation of iRace revealed the existence ofdata races in some programs distributed with TinyOS andin some TinyOS components while incurring a slowdownof only about 6 times on average in the worst-case with anaverage memory consumption of only about 20 above theoriginal memory requirements of the program Moreoverthis evaluation also proved that the labeling engine alonegenerates an average runtime overhead of only about 04xwith an average memory consumption of only about 12relatively to the original program execution These resultscoupled with a theoretical analysis showed that the proposedtechnique is highly scalable and practical even for largeinterrupt-driven programs

Detecting data races that occur first in a program isan important issue since it may make other data races todisappear It is therefore interesting to detect this class of dataraces in interrupt-driven programs to facilitate the debuggingprocess Also we may be interested in extending our work tosupport multithreaded programs that handle interrupts andalso to detect data races in low-level kernel codes

Acknowledgment

This research was financially supported by the Ministryof Education (MOE) and National Research Foundation ofKorea (NRF) through the Human Resource Training Projectfor Regional Innovation and BK21+ program

References

[1] U Banerjee B Bliss Z Ma and P Petersen ldquoA theory of datarace detectionrdquo in Proceedings of the Workshop on Parallel andDistributed Systems Testing and Debugging (PADTAD rsquo06) pp69ndash78 July 2006

[2] R H B Netzer and B P Miller ldquoWhat are race conditionsSome issues and formalizationsrdquo ACM letters on programminglanguages and systems vol 1 no 1 pp 74ndash88 1992

[3] N G Leveson and C S Turner ldquoInvestigation of theTherac-25accidentsrdquo Computer vol 26 no 7 pp 18ndash41 1993

[4] USA Canada Final Report on the August 14 2003 Blackout inthe United States and Canada Causes and RecommendationsUS-Canada Power System Outage Task Force 2004

[5] J Erickson M Musuvathi S Burckhardt and K OlynykldquoEffective data-race detection for the kernelrdquo in Proceedings of

14 International Journal of Distributed Sensor Networks

the 9th Conference on Operating Systems Design and Implemen-tation (USENIX rsquo10) 2010

[6] B-K Lee M-H Kang K C Park J S Yi S W Yang andY-K Jun ldquoProgram conversion for detecting data races inconcurrent interrupt handlersrdquo Communications in Computerand Information Science vol 257 pp 407ndash415 2011

[7] L Effinger-Dean B Lucia L Ceze D Grossman and H JBoehm ldquoIFRit interference-free regions for dynamic data-racedetectionrdquo in Proceedings of the ACM International Conferenceon Object Oriented Programming Systems Languages and Appli-cations (OOPSLA rsquo12) vol 47 pp 467ndash484 2012

[8] C Flanagan and S N Freund ldquoFastTrack efficient and precisedynamic race detectionrdquo in Proceedings of the ACM Interna-tional Conference on Object Oriented Programming SystemsLanguages and Applications (OOPSLA rsquo09) vol 44 pp 121ndash1332009

[9] X Xie and J Xue ldquoAcculock accurate and efficient detection ofdata racesrdquo in Proceedings of the 9th International Symposium onCode Generation and Optimization (CGO rsquo11) pp 201ndash212 April2011

[10] TinyOSCommunity ForumTinyOSAnopen source operatingsystem for low-power wireless devices 2008 httpwwwtinyosnet

[11] M Higashi T Yamamoto Y Hayase T Ishio and K InoueldquoAn effective method to control interrupt handler for datarace detectionrdquo in International Workshop on Automation ofSoftware Test (ACM rsquo10) pp 79ndash86 May 2010

[12] Z Lai S C Cheung and W K Chan ldquoInter-context control-flow and data-flow test adequacy criteria for nesC applicationsrdquoin Proceedings of the 16th ACM SIGSOFT International Sympo-sium on the Foundations of Software Engineering (SIGSOFT rsquo08)pp 94ndash104 November 2008

[13] J Regehr ldquoRandom testing of interrupt-driven softwarerdquo inPro-ceedings of the 5th ACM International Conference on EmbeddedSoftware (EMSOFT rsquo05) pp 290ndash298 September 2005

[14] Y Zhou X Chen M R Lyu and J Liu ldquoSentomist Unveil-ing transient sensor network bugs via symptom miningrdquoin Proceedings of the 30th IEEE International Conference onDistributed Computing Systems (ICDCS rsquo10) pp 784ndash794 June2010

[15] G Gracioli and S Fischmeister ldquoTracing interrupts in embed-ded softwarerdquo in Proceedings of the ACM SIGPLANSIGBEDConference on Languages Compilers and Tools for EmbeddedSystems (LCTES rsquo09) vol 44 pp 137ndash146 2009

[16] V Sundaram P Eugster and X Zhang ldquoLightweight tracingfor wireless sensor networks debuggingrdquo in Proceedings of the4th International Workshop on Middleware Tools Services andRun-Time Support for Sensor Networks (ACM rsquo09) pp 13ndash18December 2009

[17] O K Ha and Y K Jun ldquoMonitoring of programs with nestedparallelismusing efficient thread labelingrdquo International JournalFo U- and E- Service Science and Technology vol 4 pp 21ndash362011

[18] Y C Kim S S Jun and Y K Jun ldquoA tool for space-efficient on-the-fly race detectionrdquo International Journal of DatabaseTheoryand Application vol 4 pp 25ndash38 2011

[19] S Savage M Burrows G Nelson P Sobalvarro and T Ander-son ldquoEraser a dynamic data race detector for multithreadedprogramsrdquo ACM Transactions on Computer Systems vol 15 no4 pp 391ndash411 1997

[20] A Jannesari and W F Tichy ldquoOn-the-fly race detection inmulti-threaded programsrdquo in Proceedings of the 6th Workshop

on Parallel and Distributed Systems Testing Analysis andDebugging (PADTAD rsquo08) July 2007

[21] D Gay M Welsh P Levis E Brewer R Von Behren and DCuller ldquoThe nesC language a holistic approach to networkedembedded systemsrdquo inACMSIGPLANConference on Program-ming Language Design and Implementation pp 1ndash11 June 2003

[22] J Regehr and N Cooprider ldquoInterrupt verification via threadverificationrdquo Electronic Notes in Theoretical Computer Sciencevol 174 no 9 pp 139ndash150 2007

[23] B L Titzer D K Lee and J Palsberg ldquoAvrora scalable sensornetwork simulation with precise timingrdquo in Proceedings ofthe 4th International Symposium on Information Processing inSensor Networks (IPSN rsquo05) pp 477ndash482 April 2005

[24] B L Titzer and J Palsberg ldquoNonintrusive precision instru-mentation of microcontroller softwarerdquo in Proceedings of theConference on Languages Compilers and Tools for EmbeddedSystems (Lctes rsquo05) vol 40 pp 59ndash68 2005

[25] J Hil R Szewczyk A Woo S Hollar D Culler and K PisterldquoSystem architecture directions for networked sensorsrdquo inProceedings of the 9th International Conference on ArchitecturalSupport for Programming Languages and Operating Systems(ASPLOS rsquo00) pp 93ndash104 2000

[26] GMTchamgoue KHKim andYK Jun ldquoTesting anddebug-ging concurrency bugs in event-driven programsrdquo InternationalJournal of Advanced Science and Technology vol 40 pp 55ndash682012

[27] GM Tchamgoue O K Ha K H Kim and Y K Jun ldquoA frame-work for on-the-fly race healing in ARINC-653 applicationsrdquoInternational Journal of Hybrid Information Technology vol 4pp 1ndash12 2011

[28] A Dinning and E Schonberg ldquoDetecting access anomaliesin programs with critical sectionsrdquo in Proceedings of theACMONR Workshop on Parallel and Distributed Debugging(PADD rsquo91) vol 26 pp 85ndash96 1991

[29] L Lamport ldquoTime clocks and the ordering of events in adistributed systemrdquo Communications of the ACM vol 21 no7 pp 558ndash565 1978

[30] Atmel inc ATmega128 Datasheet 2011 httpwwwatmelcomImagesdoc2467pdf

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 12: Research Article Verification of Data Races in Concurrent ...downloads.hindawi.com/journals/ijdsn/2013/953593.pdf · photoTask I20 W 11 R 12 W 13 W 24 R 15 T 2 J12 T 1 F : A POEG

12 International Journal of Distributed Sensor Networks

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

TinyOS 210 programsOriginalWith iRace

Figure 5 Average memory consumption

application for five runs with and without iRace The captionldquoOriginalrdquo represents the native execution of the applica-tions without iRace The results of Figure 4 suggest thatthe average runtime overhead of our tool is only about1x for all applications except for RadioCountToLeds andRadioSenseToLeds that reaches 6x the original executiontime These two applications sample and display data at highfrequency rate (eg 4Hz in our case) and therefore arelikely to receive a higher number of interrupts at runtimeThis low runtime overhead is achieved by exploiting the factthat shared variables are stored by Avrora at program loadingtime Instead of monitoring all memory accesses to filterthose targeting shared variables iRace precisely instrumentsonly accesses to the well-known set of shared memory loca-tions obtained from the Avrora loader This technique allowsthe tool to drastically cuts down its monitoring overhead andto increase its scalability

Likewise wemeasured the averagememory consumptionfor each application during five runs The collected data arerepresented in Figure 5 The overall average memory con-sumption of the tool is very small at just about 20 above theoriginal memory requirement even for high frequency appli-cations like RadioCountToLeds and RadioSenseToLedsthanks to our labeling scheme that reuses the same labelfor different instances of the same interrupt handler and thedetection protocol that maintains only a reduced size accesshistoryThis result corroborates the theoretical analysis of ourtechnique as summarized in Table 1 The low runtime over-head and memory consumption of our on-the-fly techniquemake it very scalable and suitable even for interrupt-drivenprograms that handle interrupts at high frequency rates

65 Labeling Overhead Since the race detection subcompo-nent of iRace is in charge of implementing the labeling and

Table 4 Average labeling overhead

Applications Time (Sec) Memory (Kb)6LowPanCli 313 2101760AnthiTheft 308 2148880BaseStation 309 2077680Blink 174 1964640MultihopOscilloscope 313 2141920Oscilloscope 249 2092640Powerup 176 1894720RadioCountToLeds 218 2167440RadioSenseToLeds 224 2140080Sense 184 2011040

Table 5The states of an access history for detecting data races withour protocol considering the example of Figure 2

119886119894

Read Write Races11988211

⟨0 20 0⟩ ⟨20⟩

11987712

⟨0 20 0⟩ ⟨20⟩

11988213

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11988213-11988211

11988224

⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩11987715

⟨1 1 0⟩ ⟨mdash⟩ ⟨1 1 0⟩ ⟨mdash⟩ ⟨0 20 0⟩ ⟨20⟩ 11987715-11988211

the detection routines it was legitimate to analyze the impactof the labeling scheme alone on the monitored programsat runtime For this purpose we deactivated the detectionengine of iRace before running it again on the benchmarkapplications We collected the average execution time andmemory space consumed by each application after five runsas shown in Table 4 The simulation was done using a singlenode network

Figure 6 compares the original execution time of eachprogram from Table 3 with the execution time of the labelingengine as presented in the column ldquoTimerdquo of Table 4 Thecomparison shows that the average runtime overhead ofiRace with just the labeling engine activated is only about04x in the worst-case This represents only a small fractionof the average runtime overhead of 6x imposed by iRace asdiscussed in Section 64Themain reason of this performancecan be found at source level When browsing the source codeof the benchmark applications we came to understand thatprogrammers generally disable and enable all the interruptsat once instead of focusing on particular interrupts In thiscase the size of each label generated by the labeling enginenever approaches its worst-case of 119874(119879) meaning that thecomparison overhead of labels is also reduced This is veryimportant because as discussed in Section 64 althoughthe size of labels does not depend on the total number ofinterrupts received at runtime this number still has impacton the overall detection overheadThus we can conclude thatthe labeling engine of iRace incurs a very low overhead atruntime making it a highly scalable tool

We also compared the memory consumption of thelabeling engine with the memory requirement of the original

International Journal of Distributed Sensor Networks 13

TinyOS 210 programs

4

35

3

25

2

15

1

05

0

Exec

utio

n tim

e (s)

With iRace labeling engineOriginal

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 6 Average runtime overhead of the labeling engine

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

With iRace labeling engineOriginal

TinyOS 210 programs

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 7 Average memory consumption by the labeling engine

program execution presented inTable 3The results plotted inFigure 7 show that when the labeling engine alone is runningiRace consumes only about 12 more memory on averageHowever during the detection phase the overall averagememory consumption of iRace is only about 20 Theseresults also tell us that the detection engine does not increasethat much the memory consumption of iRace This is alsoreasonable because we only disabled the detection engine ofiRace but did not remove the resources it used

7 Conclusions

Asynchronous interrupt handling introduces fine-grainedconcurrency into interrupt-driven programs making themprone to data races In this paper we presented an efficientand scalable on-the-fly technique to precisely detect dataraces without false positives in such programs Our techniqueintroduced a lightweight labeling scheme for maintainingconcurrency information for the main program and each ofits interrupt handlers and a precise detection protocol thatguarantees to verify the existence of data races in a programby reporting at least one data race for each shared variableinvolved in a data race

We implemented a prototype of our technique as a racedetection tool called iRace on top of the Avrora simulationframework An evaluation of iRace revealed the existence ofdata races in some programs distributed with TinyOS andin some TinyOS components while incurring a slowdownof only about 6 times on average in the worst-case with anaverage memory consumption of only about 20 above theoriginal memory requirements of the program Moreoverthis evaluation also proved that the labeling engine alonegenerates an average runtime overhead of only about 04xwith an average memory consumption of only about 12relatively to the original program execution These resultscoupled with a theoretical analysis showed that the proposedtechnique is highly scalable and practical even for largeinterrupt-driven programs

Detecting data races that occur first in a program isan important issue since it may make other data races todisappear It is therefore interesting to detect this class of dataraces in interrupt-driven programs to facilitate the debuggingprocess Also we may be interested in extending our work tosupport multithreaded programs that handle interrupts andalso to detect data races in low-level kernel codes

Acknowledgment

This research was financially supported by the Ministryof Education (MOE) and National Research Foundation ofKorea (NRF) through the Human Resource Training Projectfor Regional Innovation and BK21+ program

References

[1] U Banerjee B Bliss Z Ma and P Petersen ldquoA theory of datarace detectionrdquo in Proceedings of the Workshop on Parallel andDistributed Systems Testing and Debugging (PADTAD rsquo06) pp69ndash78 July 2006

[2] R H B Netzer and B P Miller ldquoWhat are race conditionsSome issues and formalizationsrdquo ACM letters on programminglanguages and systems vol 1 no 1 pp 74ndash88 1992

[3] N G Leveson and C S Turner ldquoInvestigation of theTherac-25accidentsrdquo Computer vol 26 no 7 pp 18ndash41 1993

[4] USA Canada Final Report on the August 14 2003 Blackout inthe United States and Canada Causes and RecommendationsUS-Canada Power System Outage Task Force 2004

[5] J Erickson M Musuvathi S Burckhardt and K OlynykldquoEffective data-race detection for the kernelrdquo in Proceedings of

14 International Journal of Distributed Sensor Networks

the 9th Conference on Operating Systems Design and Implemen-tation (USENIX rsquo10) 2010

[6] B-K Lee M-H Kang K C Park J S Yi S W Yang andY-K Jun ldquoProgram conversion for detecting data races inconcurrent interrupt handlersrdquo Communications in Computerand Information Science vol 257 pp 407ndash415 2011

[7] L Effinger-Dean B Lucia L Ceze D Grossman and H JBoehm ldquoIFRit interference-free regions for dynamic data-racedetectionrdquo in Proceedings of the ACM International Conferenceon Object Oriented Programming Systems Languages and Appli-cations (OOPSLA rsquo12) vol 47 pp 467ndash484 2012

[8] C Flanagan and S N Freund ldquoFastTrack efficient and precisedynamic race detectionrdquo in Proceedings of the ACM Interna-tional Conference on Object Oriented Programming SystemsLanguages and Applications (OOPSLA rsquo09) vol 44 pp 121ndash1332009

[9] X Xie and J Xue ldquoAcculock accurate and efficient detection ofdata racesrdquo in Proceedings of the 9th International Symposium onCode Generation and Optimization (CGO rsquo11) pp 201ndash212 April2011

[10] TinyOSCommunity ForumTinyOSAnopen source operatingsystem for low-power wireless devices 2008 httpwwwtinyosnet

[11] M Higashi T Yamamoto Y Hayase T Ishio and K InoueldquoAn effective method to control interrupt handler for datarace detectionrdquo in International Workshop on Automation ofSoftware Test (ACM rsquo10) pp 79ndash86 May 2010

[12] Z Lai S C Cheung and W K Chan ldquoInter-context control-flow and data-flow test adequacy criteria for nesC applicationsrdquoin Proceedings of the 16th ACM SIGSOFT International Sympo-sium on the Foundations of Software Engineering (SIGSOFT rsquo08)pp 94ndash104 November 2008

[13] J Regehr ldquoRandom testing of interrupt-driven softwarerdquo inPro-ceedings of the 5th ACM International Conference on EmbeddedSoftware (EMSOFT rsquo05) pp 290ndash298 September 2005

[14] Y Zhou X Chen M R Lyu and J Liu ldquoSentomist Unveil-ing transient sensor network bugs via symptom miningrdquoin Proceedings of the 30th IEEE International Conference onDistributed Computing Systems (ICDCS rsquo10) pp 784ndash794 June2010

[15] G Gracioli and S Fischmeister ldquoTracing interrupts in embed-ded softwarerdquo in Proceedings of the ACM SIGPLANSIGBEDConference on Languages Compilers and Tools for EmbeddedSystems (LCTES rsquo09) vol 44 pp 137ndash146 2009

[16] V Sundaram P Eugster and X Zhang ldquoLightweight tracingfor wireless sensor networks debuggingrdquo in Proceedings of the4th International Workshop on Middleware Tools Services andRun-Time Support for Sensor Networks (ACM rsquo09) pp 13ndash18December 2009

[17] O K Ha and Y K Jun ldquoMonitoring of programs with nestedparallelismusing efficient thread labelingrdquo International JournalFo U- and E- Service Science and Technology vol 4 pp 21ndash362011

[18] Y C Kim S S Jun and Y K Jun ldquoA tool for space-efficient on-the-fly race detectionrdquo International Journal of DatabaseTheoryand Application vol 4 pp 25ndash38 2011

[19] S Savage M Burrows G Nelson P Sobalvarro and T Ander-son ldquoEraser a dynamic data race detector for multithreadedprogramsrdquo ACM Transactions on Computer Systems vol 15 no4 pp 391ndash411 1997

[20] A Jannesari and W F Tichy ldquoOn-the-fly race detection inmulti-threaded programsrdquo in Proceedings of the 6th Workshop

on Parallel and Distributed Systems Testing Analysis andDebugging (PADTAD rsquo08) July 2007

[21] D Gay M Welsh P Levis E Brewer R Von Behren and DCuller ldquoThe nesC language a holistic approach to networkedembedded systemsrdquo inACMSIGPLANConference on Program-ming Language Design and Implementation pp 1ndash11 June 2003

[22] J Regehr and N Cooprider ldquoInterrupt verification via threadverificationrdquo Electronic Notes in Theoretical Computer Sciencevol 174 no 9 pp 139ndash150 2007

[23] B L Titzer D K Lee and J Palsberg ldquoAvrora scalable sensornetwork simulation with precise timingrdquo in Proceedings ofthe 4th International Symposium on Information Processing inSensor Networks (IPSN rsquo05) pp 477ndash482 April 2005

[24] B L Titzer and J Palsberg ldquoNonintrusive precision instru-mentation of microcontroller softwarerdquo in Proceedings of theConference on Languages Compilers and Tools for EmbeddedSystems (Lctes rsquo05) vol 40 pp 59ndash68 2005

[25] J Hil R Szewczyk A Woo S Hollar D Culler and K PisterldquoSystem architecture directions for networked sensorsrdquo inProceedings of the 9th International Conference on ArchitecturalSupport for Programming Languages and Operating Systems(ASPLOS rsquo00) pp 93ndash104 2000

[26] GMTchamgoue KHKim andYK Jun ldquoTesting anddebug-ging concurrency bugs in event-driven programsrdquo InternationalJournal of Advanced Science and Technology vol 40 pp 55ndash682012

[27] GM Tchamgoue O K Ha K H Kim and Y K Jun ldquoA frame-work for on-the-fly race healing in ARINC-653 applicationsrdquoInternational Journal of Hybrid Information Technology vol 4pp 1ndash12 2011

[28] A Dinning and E Schonberg ldquoDetecting access anomaliesin programs with critical sectionsrdquo in Proceedings of theACMONR Workshop on Parallel and Distributed Debugging(PADD rsquo91) vol 26 pp 85ndash96 1991

[29] L Lamport ldquoTime clocks and the ordering of events in adistributed systemrdquo Communications of the ACM vol 21 no7 pp 558ndash565 1978

[30] Atmel inc ATmega128 Datasheet 2011 httpwwwatmelcomImagesdoc2467pdf

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 13: Research Article Verification of Data Races in Concurrent ...downloads.hindawi.com/journals/ijdsn/2013/953593.pdf · photoTask I20 W 11 R 12 W 13 W 24 R 15 T 2 J12 T 1 F : A POEG

International Journal of Distributed Sensor Networks 13

TinyOS 210 programs

4

35

3

25

2

15

1

05

0

Exec

utio

n tim

e (s)

With iRace labeling engineOriginal

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 6 Average runtime overhead of the labeling engine

Mem

ory

spac

e (kB

)

25000

20000

15000

10000

5000

0

With iRace labeling engineOriginal

TinyOS 210 programs

6Lo

wPa

nCli

Ant

hiTh

eft

Base

Stat

ion

Blin

k

Mul

tihop

Osc

illos

cope

Pow

erup

Radi

oCou

ntTo

Leds

Radi

oSen

seTo

Leds

Sens

e

Osc

illos

cope

Figure 7 Average memory consumption by the labeling engine

program execution presented inTable 3The results plotted inFigure 7 show that when the labeling engine alone is runningiRace consumes only about 12 more memory on averageHowever during the detection phase the overall averagememory consumption of iRace is only about 20 Theseresults also tell us that the detection engine does not increasethat much the memory consumption of iRace This is alsoreasonable because we only disabled the detection engine ofiRace but did not remove the resources it used

7 Conclusions

Asynchronous interrupt handling introduces fine-grainedconcurrency into interrupt-driven programs making themprone to data races In this paper we presented an efficientand scalable on-the-fly technique to precisely detect dataraces without false positives in such programs Our techniqueintroduced a lightweight labeling scheme for maintainingconcurrency information for the main program and each ofits interrupt handlers and a precise detection protocol thatguarantees to verify the existence of data races in a programby reporting at least one data race for each shared variableinvolved in a data race

We implemented a prototype of our technique as a racedetection tool called iRace on top of the Avrora simulationframework An evaluation of iRace revealed the existence ofdata races in some programs distributed with TinyOS andin some TinyOS components while incurring a slowdownof only about 6 times on average in the worst-case with anaverage memory consumption of only about 20 above theoriginal memory requirements of the program Moreoverthis evaluation also proved that the labeling engine alonegenerates an average runtime overhead of only about 04xwith an average memory consumption of only about 12relatively to the original program execution These resultscoupled with a theoretical analysis showed that the proposedtechnique is highly scalable and practical even for largeinterrupt-driven programs

Detecting data races that occur first in a program isan important issue since it may make other data races todisappear It is therefore interesting to detect this class of dataraces in interrupt-driven programs to facilitate the debuggingprocess Also we may be interested in extending our work tosupport multithreaded programs that handle interrupts andalso to detect data races in low-level kernel codes

Acknowledgment

This research was financially supported by the Ministryof Education (MOE) and National Research Foundation ofKorea (NRF) through the Human Resource Training Projectfor Regional Innovation and BK21+ program

References

[1] U Banerjee B Bliss Z Ma and P Petersen ldquoA theory of datarace detectionrdquo in Proceedings of the Workshop on Parallel andDistributed Systems Testing and Debugging (PADTAD rsquo06) pp69ndash78 July 2006

[2] R H B Netzer and B P Miller ldquoWhat are race conditionsSome issues and formalizationsrdquo ACM letters on programminglanguages and systems vol 1 no 1 pp 74ndash88 1992

[3] N G Leveson and C S Turner ldquoInvestigation of theTherac-25accidentsrdquo Computer vol 26 no 7 pp 18ndash41 1993

[4] USA Canada Final Report on the August 14 2003 Blackout inthe United States and Canada Causes and RecommendationsUS-Canada Power System Outage Task Force 2004

[5] J Erickson M Musuvathi S Burckhardt and K OlynykldquoEffective data-race detection for the kernelrdquo in Proceedings of

14 International Journal of Distributed Sensor Networks

the 9th Conference on Operating Systems Design and Implemen-tation (USENIX rsquo10) 2010

[6] B-K Lee M-H Kang K C Park J S Yi S W Yang andY-K Jun ldquoProgram conversion for detecting data races inconcurrent interrupt handlersrdquo Communications in Computerand Information Science vol 257 pp 407ndash415 2011

[7] L Effinger-Dean B Lucia L Ceze D Grossman and H JBoehm ldquoIFRit interference-free regions for dynamic data-racedetectionrdquo in Proceedings of the ACM International Conferenceon Object Oriented Programming Systems Languages and Appli-cations (OOPSLA rsquo12) vol 47 pp 467ndash484 2012

[8] C Flanagan and S N Freund ldquoFastTrack efficient and precisedynamic race detectionrdquo in Proceedings of the ACM Interna-tional Conference on Object Oriented Programming SystemsLanguages and Applications (OOPSLA rsquo09) vol 44 pp 121ndash1332009

[9] X Xie and J Xue ldquoAcculock accurate and efficient detection ofdata racesrdquo in Proceedings of the 9th International Symposium onCode Generation and Optimization (CGO rsquo11) pp 201ndash212 April2011

[10] TinyOSCommunity ForumTinyOSAnopen source operatingsystem for low-power wireless devices 2008 httpwwwtinyosnet

[11] M Higashi T Yamamoto Y Hayase T Ishio and K InoueldquoAn effective method to control interrupt handler for datarace detectionrdquo in International Workshop on Automation ofSoftware Test (ACM rsquo10) pp 79ndash86 May 2010

[12] Z Lai S C Cheung and W K Chan ldquoInter-context control-flow and data-flow test adequacy criteria for nesC applicationsrdquoin Proceedings of the 16th ACM SIGSOFT International Sympo-sium on the Foundations of Software Engineering (SIGSOFT rsquo08)pp 94ndash104 November 2008

[13] J Regehr ldquoRandom testing of interrupt-driven softwarerdquo inPro-ceedings of the 5th ACM International Conference on EmbeddedSoftware (EMSOFT rsquo05) pp 290ndash298 September 2005

[14] Y Zhou X Chen M R Lyu and J Liu ldquoSentomist Unveil-ing transient sensor network bugs via symptom miningrdquoin Proceedings of the 30th IEEE International Conference onDistributed Computing Systems (ICDCS rsquo10) pp 784ndash794 June2010

[15] G Gracioli and S Fischmeister ldquoTracing interrupts in embed-ded softwarerdquo in Proceedings of the ACM SIGPLANSIGBEDConference on Languages Compilers and Tools for EmbeddedSystems (LCTES rsquo09) vol 44 pp 137ndash146 2009

[16] V Sundaram P Eugster and X Zhang ldquoLightweight tracingfor wireless sensor networks debuggingrdquo in Proceedings of the4th International Workshop on Middleware Tools Services andRun-Time Support for Sensor Networks (ACM rsquo09) pp 13ndash18December 2009

[17] O K Ha and Y K Jun ldquoMonitoring of programs with nestedparallelismusing efficient thread labelingrdquo International JournalFo U- and E- Service Science and Technology vol 4 pp 21ndash362011

[18] Y C Kim S S Jun and Y K Jun ldquoA tool for space-efficient on-the-fly race detectionrdquo International Journal of DatabaseTheoryand Application vol 4 pp 25ndash38 2011

[19] S Savage M Burrows G Nelson P Sobalvarro and T Ander-son ldquoEraser a dynamic data race detector for multithreadedprogramsrdquo ACM Transactions on Computer Systems vol 15 no4 pp 391ndash411 1997

[20] A Jannesari and W F Tichy ldquoOn-the-fly race detection inmulti-threaded programsrdquo in Proceedings of the 6th Workshop

on Parallel and Distributed Systems Testing Analysis andDebugging (PADTAD rsquo08) July 2007

[21] D Gay M Welsh P Levis E Brewer R Von Behren and DCuller ldquoThe nesC language a holistic approach to networkedembedded systemsrdquo inACMSIGPLANConference on Program-ming Language Design and Implementation pp 1ndash11 June 2003

[22] J Regehr and N Cooprider ldquoInterrupt verification via threadverificationrdquo Electronic Notes in Theoretical Computer Sciencevol 174 no 9 pp 139ndash150 2007

[23] B L Titzer D K Lee and J Palsberg ldquoAvrora scalable sensornetwork simulation with precise timingrdquo in Proceedings ofthe 4th International Symposium on Information Processing inSensor Networks (IPSN rsquo05) pp 477ndash482 April 2005

[24] B L Titzer and J Palsberg ldquoNonintrusive precision instru-mentation of microcontroller softwarerdquo in Proceedings of theConference on Languages Compilers and Tools for EmbeddedSystems (Lctes rsquo05) vol 40 pp 59ndash68 2005

[25] J Hil R Szewczyk A Woo S Hollar D Culler and K PisterldquoSystem architecture directions for networked sensorsrdquo inProceedings of the 9th International Conference on ArchitecturalSupport for Programming Languages and Operating Systems(ASPLOS rsquo00) pp 93ndash104 2000

[26] GMTchamgoue KHKim andYK Jun ldquoTesting anddebug-ging concurrency bugs in event-driven programsrdquo InternationalJournal of Advanced Science and Technology vol 40 pp 55ndash682012

[27] GM Tchamgoue O K Ha K H Kim and Y K Jun ldquoA frame-work for on-the-fly race healing in ARINC-653 applicationsrdquoInternational Journal of Hybrid Information Technology vol 4pp 1ndash12 2011

[28] A Dinning and E Schonberg ldquoDetecting access anomaliesin programs with critical sectionsrdquo in Proceedings of theACMONR Workshop on Parallel and Distributed Debugging(PADD rsquo91) vol 26 pp 85ndash96 1991

[29] L Lamport ldquoTime clocks and the ordering of events in adistributed systemrdquo Communications of the ACM vol 21 no7 pp 558ndash565 1978

[30] Atmel inc ATmega128 Datasheet 2011 httpwwwatmelcomImagesdoc2467pdf

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 14: Research Article Verification of Data Races in Concurrent ...downloads.hindawi.com/journals/ijdsn/2013/953593.pdf · photoTask I20 W 11 R 12 W 13 W 24 R 15 T 2 J12 T 1 F : A POEG

14 International Journal of Distributed Sensor Networks

the 9th Conference on Operating Systems Design and Implemen-tation (USENIX rsquo10) 2010

[6] B-K Lee M-H Kang K C Park J S Yi S W Yang andY-K Jun ldquoProgram conversion for detecting data races inconcurrent interrupt handlersrdquo Communications in Computerand Information Science vol 257 pp 407ndash415 2011

[7] L Effinger-Dean B Lucia L Ceze D Grossman and H JBoehm ldquoIFRit interference-free regions for dynamic data-racedetectionrdquo in Proceedings of the ACM International Conferenceon Object Oriented Programming Systems Languages and Appli-cations (OOPSLA rsquo12) vol 47 pp 467ndash484 2012

[8] C Flanagan and S N Freund ldquoFastTrack efficient and precisedynamic race detectionrdquo in Proceedings of the ACM Interna-tional Conference on Object Oriented Programming SystemsLanguages and Applications (OOPSLA rsquo09) vol 44 pp 121ndash1332009

[9] X Xie and J Xue ldquoAcculock accurate and efficient detection ofdata racesrdquo in Proceedings of the 9th International Symposium onCode Generation and Optimization (CGO rsquo11) pp 201ndash212 April2011

[10] TinyOSCommunity ForumTinyOSAnopen source operatingsystem for low-power wireless devices 2008 httpwwwtinyosnet

[11] M Higashi T Yamamoto Y Hayase T Ishio and K InoueldquoAn effective method to control interrupt handler for datarace detectionrdquo in International Workshop on Automation ofSoftware Test (ACM rsquo10) pp 79ndash86 May 2010

[12] Z Lai S C Cheung and W K Chan ldquoInter-context control-flow and data-flow test adequacy criteria for nesC applicationsrdquoin Proceedings of the 16th ACM SIGSOFT International Sympo-sium on the Foundations of Software Engineering (SIGSOFT rsquo08)pp 94ndash104 November 2008

[13] J Regehr ldquoRandom testing of interrupt-driven softwarerdquo inPro-ceedings of the 5th ACM International Conference on EmbeddedSoftware (EMSOFT rsquo05) pp 290ndash298 September 2005

[14] Y Zhou X Chen M R Lyu and J Liu ldquoSentomist Unveil-ing transient sensor network bugs via symptom miningrdquoin Proceedings of the 30th IEEE International Conference onDistributed Computing Systems (ICDCS rsquo10) pp 784ndash794 June2010

[15] G Gracioli and S Fischmeister ldquoTracing interrupts in embed-ded softwarerdquo in Proceedings of the ACM SIGPLANSIGBEDConference on Languages Compilers and Tools for EmbeddedSystems (LCTES rsquo09) vol 44 pp 137ndash146 2009

[16] V Sundaram P Eugster and X Zhang ldquoLightweight tracingfor wireless sensor networks debuggingrdquo in Proceedings of the4th International Workshop on Middleware Tools Services andRun-Time Support for Sensor Networks (ACM rsquo09) pp 13ndash18December 2009

[17] O K Ha and Y K Jun ldquoMonitoring of programs with nestedparallelismusing efficient thread labelingrdquo International JournalFo U- and E- Service Science and Technology vol 4 pp 21ndash362011

[18] Y C Kim S S Jun and Y K Jun ldquoA tool for space-efficient on-the-fly race detectionrdquo International Journal of DatabaseTheoryand Application vol 4 pp 25ndash38 2011

[19] S Savage M Burrows G Nelson P Sobalvarro and T Ander-son ldquoEraser a dynamic data race detector for multithreadedprogramsrdquo ACM Transactions on Computer Systems vol 15 no4 pp 391ndash411 1997

[20] A Jannesari and W F Tichy ldquoOn-the-fly race detection inmulti-threaded programsrdquo in Proceedings of the 6th Workshop

on Parallel and Distributed Systems Testing Analysis andDebugging (PADTAD rsquo08) July 2007

[21] D Gay M Welsh P Levis E Brewer R Von Behren and DCuller ldquoThe nesC language a holistic approach to networkedembedded systemsrdquo inACMSIGPLANConference on Program-ming Language Design and Implementation pp 1ndash11 June 2003

[22] J Regehr and N Cooprider ldquoInterrupt verification via threadverificationrdquo Electronic Notes in Theoretical Computer Sciencevol 174 no 9 pp 139ndash150 2007

[23] B L Titzer D K Lee and J Palsberg ldquoAvrora scalable sensornetwork simulation with precise timingrdquo in Proceedings ofthe 4th International Symposium on Information Processing inSensor Networks (IPSN rsquo05) pp 477ndash482 April 2005

[24] B L Titzer and J Palsberg ldquoNonintrusive precision instru-mentation of microcontroller softwarerdquo in Proceedings of theConference on Languages Compilers and Tools for EmbeddedSystems (Lctes rsquo05) vol 40 pp 59ndash68 2005

[25] J Hil R Szewczyk A Woo S Hollar D Culler and K PisterldquoSystem architecture directions for networked sensorsrdquo inProceedings of the 9th International Conference on ArchitecturalSupport for Programming Languages and Operating Systems(ASPLOS rsquo00) pp 93ndash104 2000

[26] GMTchamgoue KHKim andYK Jun ldquoTesting anddebug-ging concurrency bugs in event-driven programsrdquo InternationalJournal of Advanced Science and Technology vol 40 pp 55ndash682012

[27] GM Tchamgoue O K Ha K H Kim and Y K Jun ldquoA frame-work for on-the-fly race healing in ARINC-653 applicationsrdquoInternational Journal of Hybrid Information Technology vol 4pp 1ndash12 2011

[28] A Dinning and E Schonberg ldquoDetecting access anomaliesin programs with critical sectionsrdquo in Proceedings of theACMONR Workshop on Parallel and Distributed Debugging(PADD rsquo91) vol 26 pp 85ndash96 1991

[29] L Lamport ldquoTime clocks and the ordering of events in adistributed systemrdquo Communications of the ACM vol 21 no7 pp 558ndash565 1978

[30] Atmel inc ATmega128 Datasheet 2011 httpwwwatmelcomImagesdoc2467pdf

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 15: Research Article Verification of Data Races in Concurrent ...downloads.hindawi.com/journals/ijdsn/2013/953593.pdf · photoTask I20 W 11 R 12 W 13 W 24 R 15 T 2 J12 T 1 F : A POEG

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of