Upload
vance
View
33
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Conformance Test Suites, Extensionally. Arend Rensink University of Twente Dutch Workshop on Formal Testing Techniques University of Twente 13 September 2007. Automaton construction. Non-deterministic algorithm. Approach. Specification. Finite if spec is regular On-the-fly construction - PowerPoint PPT Presentation
Citation preview
Conformance Test Suites, Extensionally
Arend RensinkUniversity of Twente
Dutch Workshop on Formal Testing Techniques
University of Twente13 September 2007
DWFTT, September 2007Conformance Test Suites,
Extensionally 2
Automaton construction
Approach
Specification
SuspensionAutomaton
Test Suite
Test Automaton
ImplementationUnder Test
Formal
Physical
Non-deterministicalgorithm
Closer to actual implementation
• Finite if spec is regular
• On-the-fly construction
• Completeness criteria
• Direct reasoning & manipulation
Current
New
DWFTT, September 2007Conformance Test Suites,
Extensionally 3
Background
• Specification s given by LTS– Distinguished inputs and outputs– Weakly convergent
• Stable, output-free states: quiescent– Made explicit by -labelled self-loop– Suspension language Susp(s): traces with
’s• Implementation i modelled by IOTS– Input enabled
• Conformance (ioco)i ioco s , 8 w 2 Susp(s):
Out(i after w) µ Out(s after w)
DWFTT, September 2007Conformance Test Suites,
Extensionally 4
Example
• Self-closing door
• Wrong implementation:
press
openpress press
close
input: pressoutputs: open close
press
openpress
press
closepress
Error: 2 Out(i after press¢open)
explicit quiescence
input enabled
DWFTT, September 2007Conformance Test Suites,
Extensionally 5
Example (continued)
• Alternative specs:
press
openpress press
close
press non-determinism
under-specification
for press
press
open
close
press
DWFTT, September 2007Conformance Test Suites,
Extensionally 6
IOCO analyzed
•The following are equivalent:– 8 w2Susp(s):
Out(i after w) µ Out(s after w)– 8 w2Susp(s), x2U:
w¢x 2 Susp(i) ) w¢x 2 Susp(s)– Susp(i) Å (Susp(s)¢U) µ Susp(s)
– Susp(i) Å (Susp(s)¢U n Susp(s)) = ;
•So the following are equivalent: – i conforms to s– i has no suspension trace accepted by
Susp(s)¢Un Susp(s)
DWFTT, September 2007Conformance Test Suites,
Extensionally 7
Intuition
• Susp(s): correct behaviour
• Susp(s)¢U: Just one more response…
• Susp(s)¢U n Susp(s): Forbidden paths
DWFTT, September 2007Conformance Test Suites,
Extensionally 8
Example
press
openpress
close
press
press
openpress
close
openclose
close
open
press
Spec s Construct: Susp(s)¢U n Susp(s)
press
open
press
close
openpress
close
openclose
close
open press
Example error trace: press¢open¢
final state
DWFTT, September 2007Conformance Test Suites,
Extensionally 9
reduction & -saturation
• Given a set of traces W µ L*
– W- removes all traces from W with multiple successive ’s (of the form v¢¢
¢w)– W+ adds v¢+¢w for all v¢¢w 2 W
• Facts: If W is a suspension language– W+ = W– (W-)+ = W
• Improvement on previous result:– i conforms to s iff – i has no suspension trace accepted by
Susp(s)-¢Un Susp(s)
remove -cycles
DWFTT, September 2007Conformance Test Suites,
Extensionally 10
Test automata
• Let T be an arbitrary automaton (with ’s)T accepts i , Susp(i) Å Lang(T) = ;
• Properties defined for arbitrary T– soundness: i ioco s implies T accepts i– exhaustiveness: T accepts i implies i ioco s
• Seen previously:– Susp(s)-¢U n Susp(s) sound & exhaustive for s
• General criteria (main result, part 1):– T is sound for s if and only if
Lang(T) µ (Susp(s)¢U n Susp(s)) ¢ L*
– T is exhaustive for s if Lang(T) ¶ (Susp(s)-¯ U n Susp(s))
No output after
DWFTT, September 2007Conformance Test Suites,
Extensionally 11
Main result (part 2)
• Define: T is full if– v · w 2 Lang(T) implies v 2 Lang(T)– v¢a¢w 2 Lang(T) implies v¢¢a¢w 2 Lang(T)– v¢a 2 Lang(T) implies v 2 Lang(T)– v¢U µ Lang(T) implies v 2 Lang(T)
• For all i: fill(T) accepts i iff T accepts i• Necessary criterion for exhaustiveness:– If T is sound for s, then
T is exhaustive for s only if Lang(fill(T)) ¶ (Susp(s)-¯ U n Susp(s))
fill(T) can be constructed
from T
DWFTT, September 2007Conformance Test Suites,
Extensionally 12
Example
press
openpress
close
press
Spec s
Susp(s)-¯U n Susp(s)
press
openpress
close
openclose
close
open
press
press
openpress
close
openclose
close
open
press
Susp(s)¢U n Susp(s)
Straightforward automata
constructions only
DWFTT, September 2007Conformance Test Suites,
Extensionally 13
Given:
Possible with extensional characterisation
Compositionality
Test Tsound/complete for
Spec s
Spec s’
Composition(Parallel composition,action refinement, …)
Test T’
CompositionDesired:
DWFTT, September 2007Conformance Test Suites,
Extensionally 14
Automaton construction
From test automata to test suites
Specification
SuspensionAutomaton
Test Suite
Test Automaton
ImplementationUnder Test
Formal
Physical
Non-deterministicalgorithm
DWFTT, September 2007Conformance Test Suites,
Extensionally 15
From test automata to test suites• Equivalent testing power:– Test automaton T– Set {Ti}i with i Lang(Ti) = Lang(T)
• Tretmans test cases t as test automata:– Cycle-free– One final accepting state (= fail) – One final non-accepting state (= pass) – Non-final states either stimulus or response
• Tretmans’ algorithm generates {ti}i s.t.:– Lang(ti) µ Susp(s)¢Un Susp(s) for all ti
– 9i: w2 Lang(ti) for all w2 Susp(s)¢Un Susp(s)
DWFTT, September 2007Conformance Test Suites,
Extensionally 16
Automaton construction
Direct testing with automata
Specification
SuspensionAutomaton
Test Suite
Test Automaton
ImplementationUnder Test
Formal
Physical
Non-deterministicalgorithm
DWFTT, September 2007Conformance Test Suites,
Extensionally 17
Direct testing with automata
• Given a test automaton T– Deterministic, with explicit quiescence– Corresponds to entire test suite
• Synchronisation (product) of IuT and T:– Stimulus specified by T sent to IuT– Response of IuT mimicked in T
• Test verdict– Final state of T reached: fail– Response of IuT absent in T: pass– No reachable final state in T: pass– Done testing: pass
DWFTT, September 2007Conformance Test Suites,
Extensionally 18
Conclusion
• Test automata fill gap in existing theory– Yield precise conditions for completeness– Enable constructions on test suite– Work modulo Tretmans’ “test criteria”
(not covered in this presentation)
• Re-opens discussion on– Interaction between test and IuT– Input enabledness of test cases