View
155
Download
2
Category
Preview:
Citation preview
Agenda
1) Mutation testing - definition2) Mutators3) Setup guide of Pitest in Maven4) Run mutation tests5) Pitest Coverage Report 6) Conclusions7) References
Definition
Mutation testing was originally proposed by Richard Lipton as a student in 1971 and first developed and published by DeMillo, Lipton and Sayward. The first implementation of a mutation testing tool has been created by Timothy Budd as part of his PhD work (titled Mutation Analysis) in 1980 from Yale University. [6]
Definition
Mutation testing is used to design new software tests and evaluate the quality of existing software tests. Basically the idea is about seeding artificial defects (mutations) into a source code and checks whether your test suite finds them.
If you change the source code▪ your test fails then mutation is killed - It’s good :)▪ your test passes then mutation survived - It’s bad :(
Mutators
Run by default by Pitest :
▪ INCREMENTS_MUTATOR▪ CONDITIONALS_BOUNDARY_MUTATOR (e.g.)▪ RETURN_VALS_MUTATOR▪ VOID_METHOD_CALL_MUTATOR (e.g.)▪ INVERT_NEGS_MUTATOR▪ MATH_MUTATOR▪ NEGATE_CONDITIONALS_MUTATOR (e.g.)
Conditionals Boundary Mutator
if (a < b) {//some code
}
mutated into
if (a <= b) {//some code
}
Original Mutated
< <=
<= <
> >=
>= >
Negate Conditionals Mutator
if (a == b) {//some code
}
mutated into
if (a != b) {//some code
}
Original Mutated
== !=
!= ==
<= >
>= <
< >=
> <=
Void method call mutator
public void myMethod(int a) {//some code
}public int rootMethod(int x) {
myMethod(x);return x;
}
mutated into
public void myMethod(int a) {//some code
}public int rootMethod(int x) {
return x;}
Mutators
Other mutators disabled by default :
▪ CONSTRUCTOR_CALLS - replaces constructor call with null▪ INLINE_CONSTS - assigns value to non-final variable▪ NON_VOID_METHOD_CALLS - removes calls to non void methods▪ REMOVE_CONDITIONALS - removes all conditionals statements▪ EXPERIMENTAL_MEMBER_VARIABLE - removes assignments to
member variables▪ EXPERIMENTAL_SWITCH - changes switch statement by replacing
the default label (wherever it is used) with first label. All the other labels are replaced by the default one
Conclusions
Mutation testing seems powerful and research indicates that mutation score is a better predictor of real fault detection rate than code coverage [2]. However, it has not yet received widespread popularity.
Creating mutations and executing tests against those mutations is not a lightweight process and can take quite a lot of time.
In addition to finding bugs, mutation testing is a great way to find redundant code thus making your code cleaner!
Conclusions
Should all mutants be killed ? No, but it depends.try { connection.close();} catch(SQLException e){ doNext("Call DevOps & create JIRA task for Dev :)");}
To get 100% line coverage here would mean adding a test scenario that throws an error on connection.close() and verify the call to doNext(..). If you haven’t got it covered, PIT will remove the call to ‘doNext’ and complain in turn that you didn’t kill that mutant. By all means do, if your time and budget are limitless.
Remember! Mutant is only an instance in which an artificial change to your code was not detected by your tests.
References1) www.pitest.org2) Are Mutants a Valid Substitute for Real Faults in Software
Testing? https://people.cs.umass.edu/~rjust/publ/mutants_real_faults_fse_2014.pdf
3) https://blog.codecentric.de/en/2016/02/sensible-mutation-testing-dont-go-killing-spree-2/
4) http://www.hascode.com/2015/05/mutation-testing-with-pitest-and-maven/
5) https://gofore.com/find-bugs-mutation-testing/6) https://en.wikipedia.org/wiki/Mutation_testing
THANKS!
Any questions ?
Thank you http://www.slidescarnival.com for layout
Recommended