Upload
magdalene-simpson
View
219
Download
3
Embed Size (px)
Citation preview
Jinlin Yang and David Evans
[jinlin, evans]@cs.virginia.edu
Department of Computer Science
University of Virginia
PASTE 2004 June 7th 2004
http://www.cs.virginia.edu/ipa
Dynamically Inferring Temporal
Properties
PASTE 2004 Jinlin Yang 2
Temporal Properties
• Example: fopen/fclose
• Essential for program’s correctness
• Hard for humans to document correctly [Holzmann, FSE ’02 keynote]
• Can we infer them automatically?
Yes!
PASTE 2004 Jinlin Yang 3
Related Work
• Value-based invariants– Daikon [Ernst, TSE, Feb. ‘01]– We focus on invariants in sequence of events
• Temporal properties– Recovering thread models [Cook, JSS ‘04]– Mining specification [Ammons, POPL ‘02]– Extracting component interfaces [Whaley, ISSTA ‘02]– We focus on relationship between 2 or 3 events:
greater automation, scalability
PASTE 2004 Jinlin Yang 4
Our Approach
ProgramInstrumented
Program
Instrumentation
Test Suite
Execution Traces
Testing
Inferred Properties
CandidatePropertyPatterns
Infere
nc
e
PASTE 2004 Jinlin Yang 5
Program Verification
Inferred Properties
Model
Report
DerivedPropertiesSatisfaction or
Counter-examples
Re
port G
en
eratio
n
Va
lida
tion
Program
Dyn
amic In
feren
ce
PASTE 2004 Jinlin Yang 6
Program Evolution
Inferred Properties 1
DifferentProperties
Diffe
ren
ce
An
alyze
r
Inferred Properties 2
SharedPropertiesProgram
Version 1
Program Version 2
Dyn
amic In
feren
ce
PASTE 2004 Jinlin Yang 7
Property Patterns
• Response pattern [Dwyer, ICSE ‘99]
[-P]* (P [-S]* S [-P]*)*
SPPSPS SPSP • Too weak
• 3 primitive patterns
• 4 derived patterns
PASTE 2004 Jinlin Yang 8
Partial Order of Patterns
MultiEffectPSS
MultiCausePPS
EffectFirstSPS
AlternatingPSPS
OneCauseSPSS
OneEffectSPPS
CauseFirstPPSS
ResponseSPPSPS
Str
icte
r
PASTE 2004 Jinlin Yang 9
Logical Relation of Patterns
MultiEffectPSS
MultiCausePPS
EffectFirstSPS
AlternatingPSPS
OneCauseSPSS
OneEffectSPPS
CauseFirstPPSS
• For each combination of two events
• Decide if they satisfy CauseFirst, OneCause, or OneEffect
• Find the strictest pattern
ResponseSPPSPS
Str
icte
r
Λ Λ Λ
Λ
PASTE 2004 Jinlin Yang 10
Find Strictest Pattern
Trace 1: PSPS Trace 2: PPS All Traces
CauseFirst
OneCause
OneEffect
For any two events, determine the strictest pattern satisfied by all traces
PASTE 2004 Jinlin Yang 11
Find Strictest Pattern
Trace 1: PSPS Trace 2: PPS All Traces
CauseFirst
OneCause
OneEffect
For any two events, determine the strictest pattern satisfied by all traces
PASTE 2004 Jinlin Yang 12
Find Strictest Pattern
Trace 1: PSPS Trace 2: PPS All Traces
CauseFirst
OneCause
OneEffect
CauseFirstΛOneEffect MultiCause
For any two events, determine the strictest pattern satisfied by all traces
PASTE 2004 Jinlin Yang 13
Implementation
• Automated testing and inference• Instrumentation and properties diff by hand• 900 lines of Perl code• 93 different events, 100 traces, each has 222 events on average:
less than an hour
ProgramInstrumented
Program
Instrumentation
Test Suite
Execution Traces
Testing
Inferred Properties
CandidatePropertyPatterns
Inferen
ce
PASTE 2004 Jinlin Yang 14
Experiments• Hypotheses
– We can automatically extract interesting temporal properties.
– Differences of temporal properties among multiple versions can reveal interesting things.
• Target programs– Tour bus simulator (8 student submissions)– OpenSSL (0.9.6, 0.9.7-0.9.7d)– Simplified producer-consumer (in the paper)
PASTE 2004 Jinlin Yang 15
Tour Bus Simulator
• Bus and each passenger is a separate thread
• Assignment in Fall 2003 graduate-level course (before we started this project)
• 8 submissions from the instructor
PASTE 2004 Jinlin Yang 16
Testing
>cville_bus –N 2 –C 1 –T 2
Bus waiting for trip 1
Passenger 0 gets in
Bus drives around Charlottesville
Passenger 0 gets off
Bus waiting for trip 2
Passenger 1 gets in
Bus drives around Charlottesville
Passenger 1 gets off
Bus stops for the day
N, the number of people
C, the capacity of the bus
T, the number of trips
Executed each submission with 100 randomly generated inputs, where
20 < C ≤ 40
C+1 ≤ N ≤ 2C
1 ≤ T ≤ 10
PASTE 2004 Jinlin Yang 17
Results: Tour Bus SimulatorPattern 7 Correct Versions 1 Faulty Version
Alternating waitdrives
MultiEffectdrivesgets offwaitgets offwaitgets in
waitdrives
waitgets in
MultiCause gets indrives gets indrives
CauseFirstgets ingets off gets ingets off
drivesgets offwaitgets off
• Differences reveal problems– Bus drives around before all passengers get off– Other problems
PASTE 2004 Jinlin Yang 18
OpenSSL
• Widely used implementation of the Secure Socket Layer protocol
• 6 versions [0.9.6, 0.9.7, 0.9.7a-d]
• We focus on the handshake protocol
PASTE 2004 Jinlin Yang 19
SR_CLNT_HELLO SW_SRVR_HELLO
SW_CERT SW_KEY_EXCH SW_CERT_REQ
SW_SRVR_DONE
SR_CERT SR_KEY_EXCH SR_CERT_VRFY
SR_FINISHED
SW_CHANGE SW_FINISHED
BEFORE+ACCEPT
SW_FLUSH OK
SW_FLUSH
Client Server
PASTE 2004 Jinlin Yang 20
Testing
• Manually instrumented server
• Modified client: advance to a different state with 5% probability
• Executed each version of server with 1000 randomly generated clients
PASTE 2004 Jinlin Yang 21
Inferred Alternating Patterns
0.9.6 0.9.7 0.9.7a 0.9.7b 0.9.7c 0.9.7d
SR_KEY_EXCHSR_CERT_VRFY
SW_CERTSW_KEY_EXCH
SW_SRVR_DONESR_CERT
Documented improvement
Fixed bug Race condition
7 alternating patterns same for all versions
PASTE 2004 Jinlin Yang 22
Partitioning Traces
• Correct clients– Properties are exactly same as specified– All versions agree
• Faulty clients without server errors generated– Found one possible vulnerability of DOS attack
• Segmentation faults– A documented bug fixed since 0.9.7d
PASTE 2004 Jinlin Yang 23
Summary of Experiments
• Useful in program evolution– Reveal interesting changes in OpenSSL– Identify unexpected differences
• Revealed bugs– Tour bus: identified faulty implementation
• Multiple implementations are rare, but multiple representations are common (i.e. design, model, code)
– OpenSSL: partitioning traces
PASTE 2004 Jinlin Yang 24
Future Work/Research Questions• Automating instrumentation/validation
– Can we automatically identify interesting events?
• Identifying more useful patterns– Combine data-flow with events– More expressive, more events– Ed will talk more about this
• Prioritize results– How to identify interesting differences?
• Testing– How do the inferred properties vary with the test suite?
PASTE 2004 Jinlin Yang 25
Conclusion
• Automatically inferring temporal properties is feasible
• Even very simple property patterns reveal interesting program properties
PASTE 2004 Jinlin Yang 26
Thank You!http://www.cs.virginia.edu/ipa
This work is funded in part by the National Science Foundation.
We thank Marty Humphrey for providing the student submissions, Chengdu Huang for helping OpenSSL, Joel Winstead for discussion and comments on earlier version of this paper, anonymous PASTE reviewers for insightful comments, anonymous CS650 students who wrote the buggy code, and our research group members and CS graduate students at the UVa for helping improve the talk.