10-stramaim-Estratégias de Teste Baseadas em Mutação

8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao

1/33

Unit and Integration Testing

Strategies for C Programs UsingMutation-Based Criteria

Auri Marcelo Rizzo Vincenzi


2/33

Mutation 2000 Symposium

San Jose CA October, 2000 2

Agenda

Introduction

Mutation testing

Experiment escription !esting Strategies

Score"Cost Incrementa# !esting Strategy

Su$$icient Incrementa# !esting Strategy Conc#usion and %uture &or'


3/33


San Jose CA October, 2000 (

Introduction

!est Acti)ities

*#anning

!est Case esign

!est Case Execution

+esu#t Ana#ysis

!esting Strategies

Incrementa# strategy


4/33


San Jose CA October, 2000

Introduction

!est esign !ec-ni.ues

%unctiona# /b#ac' box testing

Structura# /1-ite box testing

Error based

Comp#ementary tec-ni.ues


5/33


San Jose CA October, 2000

Introduction

!esting *-ases

3nit testing

Integration testing

System testing

How we should combine different testing criteria,

covering unit and integration testing phases, using

a low cost and effective strategy?


6/33



Mutation !esting

Mutation Ana#ysis /MA

3nit testing

Proteumtoo# $or C programs

56 mutant operators

Statement /6

Operator /4

7ariab#e /5

Constant /(


7/33



Mutation !esting

Inter$ace Mutation /IM

App#y t-e same concepts o$ mutation testing at t-e

interprocedura# #e)e#

Integration Errors

SI(B)

incorrect

A

Incorrect

outputB

SI(B)

incorreto

SO(B)incorrect

A Incorrect

output

B

SO(B)incorrect

f Incorrect

output

g


8/33



Mutation !esting

Inter$ace Mutation

PROTEUM/IMtoo# $or C programs

(( mutant operators

9roup"I /2

: Mutations inside t-e body o$ $unctionB /inter$ace

commands, g#oba# and inter$ace )ariab#es

9roup"II /;

: Mutations 1-ereAca##sB

B

A Group-II

Group-I


9/33


10/33



Mutation !esting

*rob#ems 1it- Mutation !esting


11/33



Experiment escription

Steps

*rogram Se#ection

!oo# Se#ection

!est Set 9eneration

Strategies

+esu#ts and ata Ana#ysis


12/33




*rogram Se#ection

!oo# Se#ectionProteum/Mutation Ana#ysis unit testing

PROTEUM/IM/Inter$ace Mutation integration testing


13/33


San Jose CA October, 2000 6(


!est Set 9eneration

!1o poo#s o$ test cases $or eac- program

3nit testing

Integration testing Eac- poo# is composed by

ad hoc test cases based on t-e program speci$ication

+andom test cases ta'e $rom &ong>s experiments

!est cases generated by -and

Ana#y=ing t-e #i)e mutants and determining t-e e.ui)a#ent ones

!-e entire poo# is an ade.uate test set 1?r?t? MA or IM


14/33




!est Set 9eneration

%rom eac- poo#, e#e)en ade.uate test set 1as

generated $or eac- program and eac- criteria

@o minimi=ation tec-ni.ue -as been app#ied


15/33




Strategies


16/33




Intra"p-ases !esting Strategies

Score"Cost !esting Strategy

App#ied $or eac- criterion indi)idua##y

integration #e)e#

unit #e)e#

!1o .uestions

How to select a subset of operators keeping the mutation score

as high as possible without reducing the overall effectiveness?How to achieve a mutation score of 1., with a minimal cost?


17/33




Su$$icient !esting Strategy

App#ied $or eac- criterion indi)idua##y

unit #e)e#

integration #e)e#

Inter"p-ase !esting Strategies

Combine bot- criteria, co)ering unit and

integration testing p-ases

3nit"Integration !esting Strategy

Su$$icient 3nit"Integration !esting Strategy


18/33





Estab#is- an incrementa# strategy o$ app#ication

E)a#uate t-e e$$ecti)eness o$ eac- operator 1?r?t? t-e $u##

set o$ operators

Ca#cu#ate t-e ratio bet1een t-e mutation score and t-e

cost o$ eac- operator

ata co##ected $or one program IM


19/33


San Jose CA October, 2000 6;



A)erage data $or t-e programs IM


20/33




Integration !esting

Score"Cost Inter$ace !esting Strategy /!"#!!"#!Strategy %irst Interaction

(( operators IM"ade.uate test set


21/33




!"#!!esting Strategy %ina# Step

2 operators

IM"ade.uate test set/54B o$ t-e e.ui)a#ent mutants

60 operatorsmutation score 0?;;

/6(B o$ t-e e.ui)a#ent mutants


22/33




3nit !estingScore"Cost 3nit !esting Strategy /!"$!

(6 operators MA"ade.uate test set

/85B o$ t-e e.ui)a#ent mutants

65 operatorsmutation score 0?;;/2?2B o$ t-e e.ui)a#ent mutants


23/33


San Jose CA October, 2000 2(



MA"Su$$icient Operator Set

SM!C, SS, OEDA, O++@, 7!& and 7!+

MA"Su$$icient MA


24/33




3nit !estingSu$$icient 3nit !esting Strategy /!$!

20 operatorsMA"ade.uate test set


operatorsmutation score o$ 0?;;


Comparing wit SCUSStrateg!

MA"ade.uate test C+ (2B and E. 85B Mutation Score o$ 0?;; C+ ;(B and E. 2?2B


25/33





IM"Su$$icient Operator Set

ArgAri@eg, Co)A##@od, ir7arDit@eg, Ind7arDit@eg,

Ind7ar+ep9#o, Ind7ar+epExt and Ind7ar+ep+e.

IM"Su$$icient IM


26/33




Integration !estingSu$$icient Inter$ace !esting Strategy /!#!

20 operatorsIM"ade.uate test set/5B o$ t-e e.ui)a#ent mutants

4 operators mutation score o$ 0?;;


Comparing wit SCISStrateg!

IM"ade.uate test C+ 2B and

E. 54B Mutation Score o$ 0?;; C+ 85B

and E. 6(B


27/33




3nit"Inter$ace !esting Strategy /$%#!

MA and IM are incomparab#e 1?r?t? subsume

re#ation


28/33




3nit"Inter$ace !esting Strategy /$%#!

65 operatorsIM"ade.uate test set/5B o$ t-e e.ui)a#ent mutants

operators mutation score o$ 0?;;

/?B o$ t-e e.ui)a#ent mutants

"#erall results

$%#!generates 65,552 /C+ (B 5,42 unit #e)e# 60,6(0 interation #e)e#

$%#!produces MA and IM"ade.uate

test sets


29/33


San Jose CA October, 2000 2;


Su$$icient 3nit"Inter$ace !esting Strategy /!$%#!

"#erall results

!$%#!generates 4,040

/C+ 55B 2,(0 unit #e)e# (,54 interation #e)e#

$%#!produces test cases sets

t-at detemines mutation scoreo)er 0?;; $or bot- MA and IM

criteria


30/33


San Jose CA October, 2000 (0


!esting Strategies

Summary o$ t-e +esu#ts


31/33



Conc#usion

MA and IM are comp#ementary criteria

!"#!, !#!, !"$!and !$!a##o1 t-e tester using t-e

mutant operators according 1it- t-e budget

restrictions and 1it- t-e mutation score t-at isexpected to be obtained

$%#!co)ers t-e unit and integration testing p-ases

and produces AM and IM"ade.uate test sets!$%#!co)ers a#so t-e unit and integration testing

p-ases and a mutation score o)er 0?;; is obtained


32/33



%uture &or'

%urt-er studies to in)estigate t-e sca#abi#ity o$ t-ese

resu#ts to #arge programs and ot-er app#ication

domains

In)estigate t-e e$$ecti)eness o$ t-ese strategies Impro)e t-e 1ay to se#ect t-e operators considering

t-e cost 1?r?t? t-e number o$ e.ui)a#ent mutants and

t-e e$$ecti)eness o$ eac- operator

Estab#is- -euristics to dea# 1it- e.ui)a#ent mutants


33/33


San Jose CA October 2000 ((

oubts, FuestionsGGG

Documents

10-stramaim-Estratégias de Teste Baseadas em Mutação