Passive Testing

Passive Testing

Wissam MallouliINT-LOR

[email protected]

Institut National des Télécommunications - Evry11/30/2007

11/30/2007 2

Table of contents

Introduction Active/Passive Testing Forward checking technique The backward checking technique Signature based technique Conclusion

11/30/2007 3

Table of contents


11/30/2007 4

Introduction

Fault detection : essential domain but fastidious. Need of automation.

Testing of a system. Active/Passive testing techniques

11/30/2007 5

Table of contents


11/30/2007 6

Active TestingActive Testing

IUTIUT Active Tester

Verdict:PASS,FAIL,INCONC.

Formal Specification

Formal Specification

Test SuitesTest

Suites

Automatic test generation based on formal descriptions

Generation of a : - reasonable test scenarios number (Execution)

- Complete (to cover all the system transitions)

Functional &Security

Functional &Security

11/30/2007 7

Conformance Testing(1/2)Conformance Testing(1/2)

• Check if the implementation of a system conforms to its specification

System(S)

System(I)

I

I

O

O1=O???

11/30/2007 8

Errors

1 2

3

3

1

1

21

i1 / o1

i1 / o2

i1 / o1

i1 / o2

SPECIFICATION

IMPLEMENTATION

output error

transfert error

mixte error

11/30/2007 9

Passive TestingPassive Testing

IUTIUT

Passive Tester

Verdict:PASS,FAIL,INCONC.

Security Properties

Specification

Security Properties

Specification

System UserSystem User

PO TraceCollection

11/30/2007 10

The two methods of conformity test (1/2) Active testing :

principe : send messages through the implementation, wait for answer, compare answer with the desired one

need two phases : test sequences generation tests application

pros and cons : + : possibility to focus on a particular area of the

specification - : automatic test generation is difficult - : can disturb (or crash) the IUT

11/30/2007 11

The two methods of conformity test (2/2) Passive testing :

principe : observation input/output sequences (trace), application of the trace on the spécification

pros and cons : + : no interference with the IUT + : no need of tests generation - : algorithms efficiency

11/30/2007 12

Table of contents

Introduction Active/Passive Testing Forward/backward checking

technique Signature based technique Conclusion

11/30/2007 13

Principles

Trace + Formal specification of the system

To map the trace in the specification

2 differents algorithms : forward and backward checking

11/30/2007 14

Example/Exercice

a/x b/y a/x b/y b/y a/y b/y a/y b/y (trace 1)a/x b/y a/x b/y b/y a/x b/x a/y b/y (trace 2)

S1 S2

S0 S3

a/y

a/y

b/y

b/y

b/y

a/x

b/y

a/x

11/30/2007 15

EFSM

EFSM (Extended Finite State Machine) : I/O events with or without parameters a predicate (to be satisfied) actions (to be done)

11/30/2007 16

EFSM : definitions

EFSM = heptuplet (S, s0, Sf, I, O, x, T) S : finite non empty set of states s0 : initial state Sf : finite set of final states I : finite set of input symbols ( parameters) O : finite set of output symbols ( parameters) x : finite set of variables T : finite set of transitions

11/30/2007 17

EFSM

Transition = sextuplet (st, ft, it, ot, Pt, At) st : initial state ft : final state it : input event ot : output event Pt : predicate At : set of actions

11/30/2007 1818

EFSM FormalismEFSM Formalism

S0 S2

S1 S3

a/y

a/y

b/yb/y

b/y

a/x

b/y

a/x

A(X0)

P(X0) true

S=(S0,S1,S2,S3) I=(a,b) O=(x,y)

11/30/2007 19

Variables values checking

Test by value determination Test by determination of interval of

variables

11/30/2007 20

Test by value determination

Principle value

determination according to an event trace (naive approach)

11/30/2007 21


Algorithm in two phases : phase 1 : homing. Trace exploration until

only one candidat state stays phase 2 : error detection. Trace exploration

until the end, or an error

11/30/2007 22


Variables determination : few possibles transitions giving different

values to one variable UNDEFINED variable

predicate with at least one UNDEFINED variable accepted predicate (doubt)

11/30/2007 23

State of art of passive testing

Test by value determination Test by determination of interval of

variables

11/30/2007 24

Test by determination of interval of variables

Principle Intervals to record values Assertions = boolean formula of predicates Uses Candidate Configuration Set (CCS)

11/30/2007 25

Le test par détermination d'intervalles de valeurs

Intervalles objectif : résoudre le problème de la perte

d'information (plusieurs valeurs possibles pour une variable)

notation : a v b R(v) = [a; b] v = a R(v) = [a; a]

11/30/2007 26


Intervals : operations sum of two intervals :

[a; b] + [c; d] = [a+c; b+d] Subtraction of two intervals :

[a; b] - [c; d] = [a-c; d-b] multiplication of an interval by a scalar :

w [a; b] = [wa; wb] si w 0 w [a; b] = [wb; wa] si w 0

11/30/2007 27


Consistency and refinement test refinement of intervals according to

constraints (predicats and actions) if an interval becomes empty then the

configuration is false

11/30/2007 28

Table of contents

Introduction Error types and testing methods State of art of passive testing The backward checking technique Conclusion

11/30/2007 29

The backward checking technique

Backward checking : principle Backtracking a transition Backtracking a trace Backtracking the past of a trace

11/30/2007 30

Backward checking : principle

Idea : « To understand present we must study the past »

11/30/2007 31


Backward checking : backtracking the event trace backtracking the past of the trace

trace

past of the trace

FAIL

FAIL

FAIL

FAIL

OK

sens of the algorithm

sens of the trace

11/30/2007 32


Advantages of this approach : start with correct information build artificially a longer trace

11/30/2007 33


To validate a trace in the past we must validate special variables : determinants

Definition : a variable v is a determinant for the trace t if v must necessarily be validated before we can validate t

11/30/2007 34



11/30/2007 35

Backtracking a transition

i f

P : u>=1A : x=1 y=y+1 z=v+w

R = < f ; u [0;3], x [1;1], y [2;8], z [1;2], a [7;9] ; Asrt = - ; D = {u, y, z, a} >

R = < i ; u [1;3], a [7;9], cste [1;2], y [1;7] ; Asrt = v+w=cste ; D = {u, a, y, v, w} >

4

11/30/2007 36



11/30/2007 37

Backtracking a trace

similar to intervals determination algorithm but backward

process the transition backtracking for each I/O couple

11/30/2007 38



11/30/2007 39

Table of contents

Introduction Active/Passive Testing Forward checking technique The backward checking technique Invariant based technique Conclusion

11/30/2007 40

Test by invariants : principle

Definition : invariant = property that is always true

Two step test : extraction of invariants from the specification application of invariants on event traces from

implementation Problem : temporal logic not in the model Solution : I/O invariants

11/30/2007 41

Test by invariants : I/O invariants

An invariant is composed of two parts : the test (an input or output) the preambule (I/O sequence)

3 kind of invariants : output invariants input invariants succession invariants

11/30/2007 42

Test by invariants : Output invariant

Definition : invariant in which the test is an output

Meaning : « immediatly after the sequence préambule there is always the output test

Example : (i1 / o1) (i2 / o2)

(preambule in blue, test in red)

11/30/2007 43

Test by invariants : Input invariant

Definition : invariant in which the test is an input

Meaning : « immediatly before the sequence préambule there is always the input test

Example : (i1 / o1) (i2 / o2) (i3/

(preambule in blue, test in red)

11/30/2007 44

Test by invariants : succession invariant

Definition : I/O invariant for complex properties(loops …)

Example : the 3 invariants below build the property :

« only the third i2 we meet is followed by o3 » (i1 / o1) (i2 / o2)(i1 / o1) (i2 / o2) (i2 / o2)(i1 / o1) (i2 / o2) (i2 / o2) (i2 / o3)

11/30/2007 45

Test by invariants

+ : powerful method (complex properties) + : fast when invariants are known

- : difficulties to extract interesting invariants (not automated)

- : sensible to non-determinism of specification

- : doesn't detect every errors

Documents

Passive Testing