Automated Testing of Classes

Automated Testing of Classes - 08/22/2000 - Alessandro Orso

Automated Testing of Classes

Alessandro OrsoPolitecnico di Milano - Georgia Institute of

Technology

http://www.cc.gatech.edu/~orsoUgo BuyUniversity of Illinois at Chicago

Mauro PezzèPolitecnico di Milano – Università di Milano Bicocca


Formal specs

Scripting

Source code

Formal specs

Scripting

Source code

Automated class testing

Problems with class testing– Object = data + operations– Behavior of objects highly depends on their state – Encapsulation and Information hiding the state is hidden

Breaking of the encapsulation

Generation ofmessage sequences

The overall goal is the automated, code-based generation of sequences of messages to exercise an object (i.e., an instance of a class)

Breaking of the encapsulation

Generation ofmessage sequences


Class and state:The CoinBox example

Class CoinBox {

int total;

int curQtrs;

int allowVend;

public:

CoinBox() {

total=0;

allowVend=0;

curQtrs=0;

}

void addQtr();

void returnQtrs();

void vend();

};

Fault:method returnQtrs

does not reset variable allowVend

Possible failure:CoinBox()CoinBox()addQtr()addQtr()addQtr()addQtr()returnQtr()returnQtr()vend()vend()

free drink!

void CoinBox::returnQtrs() { curQtrs=0;

}void CoinBox::addQtr() { curQtr=curQtr+1; if (curQtr>1)

allowVend=1;}void CoinBox::vend() { if (allowVend) { total=total+curQtrs; curQtrs=0; allowVend=0; }}

allowVend=0;


Steps of the technique

1. Data-flow analysisto identify “paths of interest” within the class

2. Symbolic executionto express the semantics of the class in terms of pre- and post-conditions

3. Automated deduction techniquesto generate sequences of messages for the class that exercise those paths


T

12 exit12 exit

11 curQtrs=0

20a t=total+curQtrs

14a t=curQtrs+119 if(allowVend)

18 vend()

17 exit

19 if(allowVend)

13 addQtr()

14b curQtrs=t

15 if(curQtrs>1)

16 allowVend=1

23 exit Vend()

10 returnQtrs()

18 vend()

20b total=tmp

22 allowVend=0

Virtual Node

23 exit

Step I: Data flow analysis

1 CoinBox()

2 total=0

3 allowVend=0

4 curQtrs=0

5 exit

20b total=t

21 curQtrs=0

Virtual Node

Class CoinBox

T

F

22 allowVend=0

F

Step I

21 curQtrs=0

11 curQtrs=0

DEF 20a t=total+curQtrs

USE


DU associations

DUA# variabile definition use1 curQtrs CoinBox (4) addQtr (14a)2 curQtrs CoinBox (4) vend (20a)3 allowVend CoinBox (3) vend (19)4 total CoinBox (2) vend (20a)5 curQtrs returnQtrs (11) addQtr (14a) 6 allowVend vend (22) vend (19)7 curQtrs returnQtrs (11) vend (20a)8 curQtrs addQtr (14b) addQtr (14a) 9 curQtrs addQtr (14b) addQtr (15)10 curQtrs addQtr (14b) vend (20a)11 allowVend addQtr (16) vend (19)12 total vend (20b) vend (20a)13 curQtrs vend (21) addQtr (14a)

Step I


Step II: Symbolic execution

For each path in each method of the class:– Execution conditions:

( curQtrs>0 )– Relation between input and output variables:

allowVend'=1, curQtrs'=curQtrs+1– Set of defined attributes

def={curQtrs,allowVend}

Limitations:– Approximations required for loops– Restrictions on the program constructs that symbolic

execution can handle

Step II


T

F

curQtrs=curQtrs+1

exit

addQtr()

allowVend=1

if(curQtrs>1)

(curQtrs>0)

allowVend'=1, curQtrs'=curQtrs+1

curQtrs=curQtrs+1

exit

addQtr()

allowVend=1

T

F

if(curQtrs>1)

(curQtrs == 0)

curQtrs'=1

Conditions for the CoinBox example

addQtr

(curQtrs>0) allowVend'=1, curQtrs'=curQtrs+1

def={curQtrs,allowVend}

(curQtrs==0) curQtrs’=1

def={curQtrs}

Step II

curQtrs=curQtrs+1

exit

addQtr()

allowVend=1

T

F

if(curQtrs>1)


Step III: Automated deduction

For each DU association <def,use, v>, identified with data flow analysis, build the method sequence:

Method that contains the definition of v

Method that contains the use of v

Def-clear path w.r.t. v

<<constructorconstructor,...,m,...,mii,..., ,..., mmDD,...,m,...,mjj,...,,...,mmUU>>

Step III


• node m = a path in m and the associated pre-condition cond(m)

• successors m’ (of m) = all paths pj whose post-conditions do not contradict cond(m)

cond(m’) is the conjunction of• cond(m), without clauses implied by the

post condition of pi

• precondition of pi

• node m = a path in m and the associated pre-condition cond(m)

The method invocation tree

Step III

m’ m’

m(a>0) &&(b>=3)

exit

m()

... F

if(a>0)

if(b<3)

...

TF

T

Fm’(c > d) (a’ = 1)(c == d) (b’ = 9)(c < d) (b’ = 2)

(b>=3) &&(c > d)

(a>0) &&(c==d)


Building the tree

1. root = a path in mu that traverses the use

2. If possible, add md to the tree and explore only the sub-tree rooted at md, else start exploring the paths that do not imply the condition of their predecessor

3. Once md has been added, the next goal is to add a constructor

• Stop when:– Both md and a constructor have been added: test case

– No more nodes can be added: infeasible DUA– Tree depth reaches a user-provided threshold : fail

Step III


addQtrs

(curQtrs>0) allowVend'=1

def={curQtrs,allowVend} curQtrs'=curQtrs+1

(curQtrs==0) total'=total

def={curQtrs} curQtrs'=1

returnQtrs

(true) curQtrs'=0

def={curQtrs}

Building the deduction tree for DUA #7

addQtraddQtr returnQtrs(curQtrs>0)

Test caseTest case:: CoinBox(),CoinBox(), addQtr(),addQtr(), addQtr(), addQtr(), returnQtr(),returnQtr(), vend()vend()

vend

(AllowVend!=0)

use

(AllowVend!=0)returnQtrs

def

CoinBox

(true) constructor

Step III

var: curQtrs

def: returnQtr node 11

use: vend node 20a

returnQtrs

(true) curQtrs'=0

def={curQtrs} (curQtrs == 0) &&(AllowVend!=0)

(AllowVend!=0)

vendaddQtr addQtr

(curQtrs = 0)(curQtrs>0) (curQtrs > 0) &&(AllowVend == 0)


Characteristics of the technique

• Focuses on problems related to the state(instance variables)

• Generates test cases from the source code(no specification required)

• Uses a stepwise automated approach(partial results can be used)


The experimental setting

C++CodeC++Code

C++ ParserC++ Parser

IL Analyzer

DF Tool

Abstract Graph

ILIL

DUAsDUAsSymbolic ExecutorSymbolic Executor

DUAs +Pre/Post Conditions

DUAs +Pre/Post Conditions

Automated DeductionAutomated Deduction MessageSequencesMessage

Sequences

DataFlow

Analyzer


Summary and future work

• Technique for automated generation of test data for classes• Combination of existing techniques in a new context: data-

flow analysis, symbolic execution, and automated deduction• Useful intermediate and partial results

– DUAs– Pre/Post– Subset of test cases

• Results of the preliminary experiments are encouraging• A tool is under development (first prototype due by the end

of the year)• Future work

– Experimentation– Extension to non-primitive attributes


Questions?

Documents

Automated Testing of Classes