Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Towards Informed Swarm Verification
Anton Wijs SET meeting 13 april 2010
Table of Contents
• Model checking • Swarm Verification • Informed Swarm Verification • Conclusions / Future work
Model Driven Software Engineering (MDSE) PAGE 1 01-06-2011
Model Checking (1)
System satisfies p property S
Model M f Formal property
(Modelling Language)
(Temporal Logic)
State space explosion - reduction techniques needed
- E.g. Partial Order Reduction
Model Driven Software Engineering (MDSE) PAGE 2 01-06-2011
We will deal mainly with explicit state, action-based model checking (LTSs)
Model Checking (2)
• For large state spaces, we can focus on debugging instead of verification (guaranteeing bug-absence)
• Techniques such as swarm verification (SV) [Holzmann et al., ‘08] are very effective for debugging in parallel
• Contribution: extend SV for verification • Make SV robust for bug-free state spaces
PAGE 3 01-06-2011 Model Driven Software Engineering (MDSE)
Properties
- For safety properties, goal is to find “bad” states - Reachability analysis suffices - BFS / DFS are suitable - BFS high memory reqs. / short counter-examples - DFS low memory reqs. / long(er) counter-examples - BFS very suitable for distributed / multi-core computing
Model Driven Software Engineering (MDSE) PAGE 4 01-06-2011
Parallel Model Checking
• These days, main memory is a decreasingly important bottleneck
• Instead, a time explosion problem emerges • Parallel model checking techniques become
appealing: • Multi-core model checking: still a lot of work to do • Distributed model checking: often needs lots of
communication between workers (another bottleneck!)
PAGE 5 01-06-2011 Model Driven Software Engineering (MDSE)
Grid Computing
• SETI@Home
PAGE 6 01-06-2011
Embarrassingly Parallel! [Foster, ‘95]
Model Driven Software Engineering (MDSE)
Grid Computing [image: mcrl2.org]
• MC@Home?
PAGE 7 01-06-2011
Not obvious
Model Driven Software Engineering (MDSE)
Embarrassingly Parallel Verification
• Swarm Verification (SV) [Holzmann et al., ‘08] • Parallel Randomized DFS
[Sivaraj et al., ‘03, Dwyer et al., ‘07]
• Require no communication between workers • No need for synchronisation, workers can start
whenever they want • May perform duplicate work, but this is tolerated
PAGE 8 01-06-2011 Model Driven Software Engineering (MDSE)
Bug-hunting
PAGE 9 01-06-2011
If multiple bugs present, BFS finds the one with shortest trace first. Can be parallelised.
DFS can find “deep” bugs quicker, BUT it depends on the area. “Inherently sequential”.
Hard to detect!
Model Driven Software Engineering (MDSE)
Swarm Verification (1)
PAGE 10 01-06-2011
Main idea: To improve fast detection of deep bugs.
DFS inherently sequential, but we can launch multiple DFS based searches
Searches use unique ordering of successors (not just LIFO)!
Model Driven Software Engineering (MDSE)
Swarm Verification (2)
PAGE 11 01-06-2011
Very successful In bug-hunting! [Holzmann et al., ‘08]
- DFTP - DEOS - Fleet
But what about verification?! - No chance of early termination - All searches are exhaustive
Model Driven Software Engineering (MDSE)
Towards Informed SV (1)
• Can we bound workers to specific areas? • + No worker will exhaustively search • BUT: workers together must be exhaustive • Should remain embarrassingly parallel • Cumulatively Exhaustive Sets (CESs) of searches
PAGE 12 01-06-2011
{A,B} :R(A)∪ R(B) = SIdeally: R(A)⊂ S ∧ R(B)⊂ S
Iterative searches, but:
Random searches, but:
∃A :R(A) = S{...} →∞
Model Driven Software Engineering (MDSE)
Towards Informed SV (2)
• : Non-exhaustive searches! • Directed Model Checking [Edelkamp et al., ‘04]
− Informed search has guiding function − Can change state ordering − Exhaustive: e.g. Dijkstra’s Search, A* − Non-exhaustive: Nearest-Neighbour, Beam Search − + Very efficient, useful for bug-hunting − BUT cannot guarantuee bug-absence!
PAGE 13 01-06-2011
R(A)⊂ S
f :S→ N
What about e.g. sets of Beam Searches?!
Model Driven Software Engineering (MDSE)
System of Non-Communicating Processes (1)
PAGE 14 01-06-2011 Model Driven Software Engineering (MDSE)
System of Non-Communicating Processes (2)
PAGE 15 01-06-2011 Model Driven Software Engineering (MDSE)
System of Non-Communicating Processes (3)
PAGE 16 01-06-2011 Model Driven Software Engineering (MDSE)
System of Non-Communicating Processes (4)
PAGE 17 01-06-2011 Model Driven Software Engineering (MDSE)
System of Non-Communicating Processes (5)
PAGE 18 01-06-2011 Model Driven Software Engineering (MDSE)
System of Non-Communicating Processes (6)
PAGE 19 01-06-2011
Informed swarm of two workers: Each worker can ignore 1/3 of states
Model Driven Software Engineering (MDSE)
Informed SV Method (1)
• Analyse subsystem M of system N • Subset of set of parallel processes • Graph of M can still be very small
• Result is set of traces Σ through behaviour M • Each worker explores graph of N restricted to σ • Traces as guiding functions
PAGE 20 01-06-2011 Model Driven Software Engineering (MDSE)
Informed SV Worker Algorithm (1)
PAGE 21 01-06-2011 Model Driven Software Engineering (MDSE)
Informed SV (1)
PAGE 22 01-06-2011
Given: - Model N - Trace σ through M <a, b, c, …>
Model Driven Software Engineering (MDSE)
a b c
b b
Informed SV Method (2)
• Analyse graph subsystem M of system N • Subset of set of parallel processes
• Result is set of traces Σ through graph of M • Each worker explores graph M x N restricted to σ
• Trivial for non-communicating processes • What about communicating processes?
PAGE 23 01-06-2011
PROBLEM: if two traces σ, σ’ through graph M lead to same state s, this may not be the case in graph N !
Model Driven Software Engineering (MDSE)
System of Communicating Processes (1)
PAGE 24 01-06-2011 Model Driven Software Engineering (MDSE)
System of Communicating Processes (2)
PAGE 25 01-06-2011
{< push_button(2), get_tea, push_button(2), get_tea, walk_away > }
Model Driven Software Engineering (MDSE)
System of Communicating Processes (3)
PAGE 26 01-06-2011
{< push_button(2), get_tea, push_button(2), get_tea, walk_away >,
< push_button(2), get_tea, push_button(1), get_coffee > }
Model Driven Software Engineering (MDSE)
System of Communicating Processes (4)
PAGE 27 01-06-2011
{< push_button(2), get_tea, push_button(2), get_tea, walk_away >,
< push_button(2), get_tea, push_button(1), get_coffee >,
< push_button(1), get_coffee > }
Model Driven Software Engineering (MDSE)
System of Communicating Processes (5)
PAGE 28 01-06-2011
Never reached! Model Driven Software Engineering (MDSE)
System of Communicating Processes (6)
• Perform analysis graph M with DFS with only stack?
PAGE 29 01-06-2011
Very inefficient: every state with n incoming traces needs to be explored n times!
Alternative: construct weighted graph M
Model Driven Software Engineering (MDSE)
Identifying Traces graph M
PAGE 30 01-06-2011
Traces can be identified with IDs!
E.g. Trace 3: - 0 [0-4> - 2 [2-4> - 3 [2-4> - 5 [3-4> - 6 [3-4> - 7 [3-4>
Model Driven Software Engineering (MDSE)
Informed SV Method
PAGE 31 01-06-2011
- Weighted graph M - Set of unexplored traces Σ
?
Model N + unexplored trace
Feedback: What was seen?
Prune in Σ
Model Driven Software Engineering (MDSE)
Informed SV (2)
PAGE 32 01-06-2011
Given: - Model N - Trace σ through M <a, b, c, …>
Model Driven Software Engineering (MDSE)
a b c
c d
Feedback sets: 0: { a, b, c }
1: { c, d }
Informed SV Worker Algorithm (2)
PAGE 33 01-06-2011 Model Driven Software Engineering (MDSE)
Experimental Results
• Prototype worker in LTSmin toolset [Blom et al., ‘10] • Standalone prototype of manager • Experiments mostly “simulate” grid environment
PAGE 34 01-06-2011 Model Driven Software Engineering (MDSE)
- Searches are still diverse and reach great depths quickly - Each worker explores a fraction of the state space
- ½ % DRM, ⅙ % 1394
estimated time
125,139 s 19,477 s
17,325 s 105,020 s 60,784 s
7,294 s
Related Work
• [Lerda & Sista, ‘99] distribute work based on single process behaviour, not embarrassingly parallel
• [Groce & Joshi, ‘08] Restrict analysis of program based on trace of events (slicing on C program)
• [Staats & Păsăreanu, ‘10] Generate test cases for software testing based on “shallow” analysis of symbolic execution trees
PAGE 35 01-06-2011 Model Driven Software Engineering (MDSE)
Conclusions & Future Work
• Presented an informed swarm verification technique suitable for grid model checking
• Experimental results very promising
• Plans: • Perform more experiments (compare with SV) • Improve method − Support infinite subsystem behaviour − Reduce over-approximations − Automatic selection of suitable subsystem
• Investigate manager selection of σ • State-based method
PAGE 36 01-06-2011 Model Driven Software Engineering (MDSE)
PAGE 37 01-06-2011 Model Driven Software Engineering (MDSE)