Embed Size (px)
DESCRIPTION
An important challenge in creating automated tests is how to design test fixtures, i.e., the setup code that initializes the system under test before actual automated testing can start. Test designers have to choose between different approaches for the setup, trading off maintenance overhead with slow test execution. Over time, test code quality can erode and test smells can develop, such as the occurrence of overly general fixtures, obscure inline code and dead fields. In this paper, we investigate how fixture-related test smells evolve over time by analyzing several thousand revisions of five open source systems. Our findings indicate that setup management strategies strongly influence the types of test fixture smells that emerge in code, and that several types of fixture smells often emerge at the same time. Based on this information, we recommend important guidelines for setup strategies, and suggest how tool support can be improved to help in both avoiding the emergence of such smells as well as how to refactor code when test smells do appear.
Citation preview
Strategies for Avoiding Test Fixture Smells duringSoftware Evolution
Michaela Greiler, Andy Zaidman, Arie van Deursen, Margaret-Anne Storey
Test Fixture Smells
2
What is a test smell?
A symptom of a problem
• Can originate from several causes
• Refactoring needed
3
Structure of a Test
4
Test (method)
Test Structure
1. Statement
2. Statement
3. Statement
4. Statement
5. Statement
6. Statement
Setup of Test Fixture
Verify Outcome
Exercise SUT
Teardown Test Fixture
5
Test Fixture
The code that initializes and configures the system under test
6
How to structure the test fixture and where to place it?
Test – Inline Fixture
Setup within method
Test Fixture
8
Test – Implicit
Setup
Method
9
Test – Delegate
Method
createTestAirportOriginTest ()
createTestAirportOriginTest(); explicit invocation
deleteTestAirportOriginTest()
deleteTestAirportOriginTest(); }
10
Delegate
“Most of the complexity of writing tests involves how to write the Test Methods; what to include
inline and what to factor out into Test Utility Methods, and so on.”
(Meszaros)
11
Test Fixture Smells
• General Fixture
• Test Maverick
• Dead Fields
• Lack of Cohesion of Test Methods
• Vague Header
• Obscure In-line setupICST 2013: Greiler, van Deursen, Storey
Automatic Detection of Test Fixture Strategies and Smells
12
Developers recognize test fixture smells as a problem (ICST’13)
13
Developers recognize test fixture smells as a problem (ICST’13)
Resolving these smells after a long time can be problematic.
Immediately awareness of smells density important.
14
1. Investigate the evolution of test fixture smells and2. Understand which software changes lead to increased test smell densities
to develop strategies to avoid fixture smells and to determine the best time to alert developers.
15
Resolving these smells after a long time can be problematic.
Immediately awareness of smells density important.
Azure
39 KLOCs30 T-classes
358 T-methods300 revisions
1 year
JSoup
20 KLOCs 23 T-classes
372 T-methods 973 revisions
2.5 years
Checkstyle
66 KLOCs 156 T-classes
549 T-methods 2251 revisions
9.5 years
PMD
174 KLOCs 118 T-classes
739 T-methods 1900 revisions
5 years
Voldemort
130 KLOCs 132 T-classes
520 T-methods 2900 revisions
1.5 years
Case studies
16
TestEvoHound…is a static analysis tool that analyzes the test
fixture smell evolution.
TestEvoHound…is a static analysis tool that analyzes the test
fixture smell evolution.
1.Revision Checkout2.Build Process 3.Test Fixture Smell Analysis 4.Trend Analysis
Fixture smells among projects
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Azure Checkstyle JSoup PMD Voldemort
Test Mavericks General Fixture Vague Header Dead Fields LCOTM Inline
19
Does Test Fixture Smell Density Increase?
A general growth of test fixture smells over timedoes not occur.
20
Lehman’s law: when a system evolves, its complexity increases unless work is done to maintain or reduce it.
Increase in smell density
Checkstyle: dead fields
Refactoring and forgotten functionality: “Added a helper method to create a configuration for a check...”.
21
Decrease in smell density
Azure: general fixture
New, non-smelly tests added: “Table Client commit [...]”.
22
Smell Dispersion
Over time, smelly classes get smellier
Smells cluster in a few classes
23
Correlations
?Number of t-methods per test class correlates with test fixture smell density
Number of fields do not necessarily correlate with smell density
# T-methods
# Fields
24
Strategies and Recommendations
Keep test classes small
Keep inheritance structures flat
Limit the scope of super classes
Use composition instead of inheritance
Reconsider the “one test class per class” organization
25
Findings
0%
20%
40%
60%
80%
100%
Azure Checkstyle JSoup PMD VoldemortTest Mavericks General Fixture Vague Header
Dead Fields LCOTM Inline
# T-methods# Fields
26
0%
20%
40%
60%
80%
100%
Azure Checkstyle JSoup PMD VoldemortTest Mavericks General Fixture Vague Header
Dead Fields LCOTM Inline
# T-methods# Fields
Thanks! Questions?
Connect with meMichaela Greiler
mgreiler
27
