Tracking Conflicting Accesses Efficiently for Software Record & Replay

Tracking Conflicting Accesses Efficiently for Software Record & ReplayMichael Bond (Ohio State)Milind Kulkarni (Purdue)

Concurrent software is nondeterministic

Record & replay: more important & harder

Record & Replay

Offline replayReproduce production bugs

Online replay• Replication-based fault tolerance• Offloading of security events

Why is record & replay hard?

Nondeterministic thread interleavings:

• Synchronization• Data races

Prior Work

Detects races high overhead[LeBlanc & Mellor-Crummey ’87]

Custom hardware support[FDR] [Rerun] [Strata] [DeLorean] [MRR]

Doesn’t support offline or online replay[Respec] [ODR] [PRES] [Weeratunge et al. ’10]

DoublePlay: extra cores; doesn’t scale well

Prior Work

Detects races high overhead[LeBlanc & Mellor-Crummey ’87]

Custom hardware support[FDR] [Rerun] [Strata] [DeLorean] [MRR]

Doesn’t support offline or online replay[Respec] [ODR] [PRES] [Weeratunge et al. ’10]

DoublePlay: extra cores; doesn’t scale well

Prior Work

Detects races high overhead[LeBlanc & Mellor-Crummey ’87]

Custom hardware support[FDR] [Rerun] [Strata] [DeLorean] [MRR]

Doesn’t support offline or online replay[Respec] [ODR] [PRES] [Weeratunge et al. ’10]

DoublePlay: extra cores; doesn’t scale well

Don’t track conflicting dependences

Tracking Conflicting Dependences

Every access might conflict

Synchronization conflicting access

T1if o.lastAccess != T1 …write o.f

T2if o.lastWrite != T2 …read o.f

Tracking Conflicting Dependences

Every access might conflict:Need instrumentation at every accessSynchronization conflicting accessT1if o.lastAccess != T1 … o.lastAccess = T1write o.f

T2if o.lastAccess != T2 … o.lastAccess = T2read o.f

Tracking Conflicting Dependences

Every access might conflict:Need synchronization at every accessSynchronization conflicting accessT1if o.lastAccess != T1 … o.lastAccess = T1write o.f

T2if o.lastAccess != T2 … o.lastAccess = T2read o.f

Tracking Conflicting Dependences

Every access might conflict:Need synchronization at every access

at conflicting accesses onlyT1if o.lastAccess != T1 … o.lastAccess = T1write o.f

T2if o.lastAccess != T2 … o.lastAccess = T2read o.f

Tracking Conflicting Dependences

Every access might conflict:Need synchronization at every access

at conflicting accesses onlyT2if o.lastAccess != T2 … … o.lastAccess = T2read o.f

Tracking Conflicting Dependences

Every access might conflict:Need synchronization at every access

at conflicting accesses only

T1……safe point:…

T2if o.lastAccess != T2 … … o.lastAccess = T2read o.f

Tracking Conflicting Dependences

Every access might conflict:Need synchronization at every access

at conflicting accesses only

T1if o.state != WrExT1

…write o.f

T2if o.state in { WrExT2 , RdEx T2 , RdSh } …read o.f

Tracking Conflicting Dependences

Every access might conflict:Need synchronization at every access

at conflicting accesses only

Related to locality & ownership tracking[Shasta] [Biased locking] [von Praun &

Gross ’01][CoreDet?] [IBM’s STM?]

Recording Happens-Before

…safe point

if o.state = … …read o.f

Record dynamic program location

Happens-before

Replaying Happens-Before

Increment counter Wait for counter

…safe point

if o.state = … …read o.f

Happens-before

Replaying Happens-Before

sync (o) { write o.f}

sync (o) { read o.f}

Happens-before

Replaying Happens-Before

sync (o) { write o.f}

sync (o) { read o.f}

Happens-before

Performance

eclip

se

hsql

db

xala

n

SPEC

jbb2

000

geom

ean

-10%

0%

10%

20%

30%

40%

50%

60%

70%

Track conf depsFast path

Ru

nti

me o

verh

ead

Performance

eclip

se

hsql

db

xala

n

SPEC

jbb2

000

geom

ean

-10%

0%

10%

20%

30%

40%

50%

60%

70%

Record & replayTrack conf depsFast path

Ru

nti

me o

verh

ead

Challenge: Performance

Non-conflicting accesses very fast• Static analysis

Conflicting accesses not too slow• Pessimistic concurrency?

Challenge: Replayability

Controlling other sources of nondeterminism:• I/O• Low-level VM concurrency• Timer-based sampling• Record vs. replay

Challenge: Replayability

Controlling other sources of nondeterminism:• I/O• Low-level VM concurrency• Timer-based sampling• Record vs. replay

Different heap layouts different hash codes

Challenge: Replayability

Controlling other sources of nondeterminism:• I/O• Low-level VM concurrency• Timer-based sampling• Record vs. replay

Different heap layouts different hash codes

Deterministic hash codes?

Summary

Software record & replay bytracking conflicting dependences• Optimistic concurrency control• Performance & replayability challenges

Apply concurrency control mechanismto other problems?