Upload
suvetigeorge
View
218
Download
0
Embed Size (px)
Citation preview
8/12/2019 Quality Assurance and Software Evolution - EmpiricalSE
1/26
Empirical Software Engineering with Examples
Dr. Cristina Marinescu
Empirical Software Engineering
a sub-domain of software engineering focusing onexperiments on software systems
devise experiments on software, in collecting data fromthese experiments, and elaborate laws and theories fromthis data (wikipedia)
2
Dr. Cristina Marinescu
Software Quality
?
Defects (Bugs)
3 Dr. Cristina Marinescu
Classication
4
Observe Defects
TRUE FALSE
PredictedTRUE TP FP
Defects FALSE FN TN
precision = TP/(TP + FP)
recall = TP/(TP + FN)
8/12/2019 Quality Assurance and Software Evolution - EmpiricalSE
2/26
Dr. Cristina Marinescu
Some tests...
arithmetic mean
mean(2, 3, 4) = 3mean(0, 3, 6) = 3
standard deviation
5 Dr. Cristina Marinescu
Rhttp://www.r-project.org/
6
Dr. Cristina Marinescu
T. Zimmermann, R. Premraj, and A. Zeller
Predicting defects for EclipseThird International Workshop on Predictor Models in SoftwareEngineering (PROMISE)
IEEE Computer Society, 2007
7 Dr. Cristina Marinescu
Predicting Defects for Eclipse
8
Num er of Files Packages
2.0 6729 377
2.1 7888 434
3.0 10593 661
8/12/2019 Quality Assurance and Software Evolution - EmpiricalSE
3/26
Dr. Cristina Marinescu
Used Metrics...
9
methods
FOUTMLOCNBDPARVG
Number of method calls (FAN OUT)Method Lines of CodeNested block depthNumber of parametersMcCabe Cyclomatic Complexity
classesNOFNOMNSFNSM
Number of eldsNumber of methodsNumber of static eldsNumber of static methods
lesACDNOINOTTLOC
Number of anonymous type declarationsNumber of interfacesNumber of classesTotal lines of code
packages NOCU Number of les
Dr. Cristina Marinescu
Spearman correlations
10
FOUTMLOCNBDPARVG
0.4000.4030.3920.3500.389
0.3190.3220.3200.2830.315
0.5370.5450.3920.5540.546
0.5230.5440.3200.5260.538
NOFNOMNSFNSM
0.2600.3190.1860.202
0.2040.2680.1700.179
0.5070.5020.4590.448
0.4800.4910.4140.371
ACDNOINOTTLOC
0.2580.160(-)0.1600.421
0.1800.129(-)0.1290.333
0.4420.1290.5180.581
0.4140.1100.4700.559
NOCU - - 0.514 0.461
Dr. Cristina Marinescu
Classication of les
11
Release Testing Precision Recall
2.02.02.13.0
0.6920.4780.613
0.2650.1910.171
2.12.02.13.0
0.6640.6680.717
0.2030.1600.139
3.02.02.13.0
0.5780.5280.675
0.2770.2200.224
Dr. Cristina Marinescu
Foutse Khomh, Massimiliano Di Penta, Yann-Gal Guhneuc,Giuliano Antoniol
An exploratory study of the impact of antipatterns on class change- andfault-proneness.
Empirical Software Engineering 17(3): 243-275 (2012)
12
8/12/2019 Quality Assurance and Software Evolution - EmpiricalSE
4/26
D r. C ri st in a M ar ine sc u 1 3
.
.
40
50
60
70
80
90
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18Releases
% o
f c
l a s s e s
p a r
t i c
i p a
t i n g
i n
a n t i p a t
t e r n s
ArgoUML Eclipse Mylyn Rhino
Fig. 1 Percentages of classes participating in antipatterns in the releases of the four systems
Dr. Cristina Marinescu
List of AntiPatterns...
14
AntiSingleton
Blob
ClassDataShouldBePrivate
ComplexClass
LargeClass
LazyClass
LongMethod
LongParameterList
MessageChain
RefusedParentBequest
SpaghettiCode
SpeculativeGenerality
SwissArmyKnife
Dr. Cristina Marinescu
Research Questions...
15 Dr. Cristina Marinescu
Research Questions...
16
8/12/2019 Quality Assurance and Software Evolution - EmpiricalSE
5/26
Dr. Cristina Marinescu
Research Questions...
17 Dr. Cristina Marinescu
Research Questions...
18
D r. C ri st in a M ar ine sc u 1 9
. ;
Dr. Cr is ti na Ma ri nes cu 20
.:
.
.
. :
.
..
.:
.
.
.
8/12/2019 Quality Assurance and Software Evolution - EmpiricalSE
6/26
D r. C ri st in a M ar ine sc u 2 1
.:
Dr. Cr is ti na Ma ri nes cu 22
Are the clients of awedclasses (also) defect prone?
Authors: Radu & CristinaMarinescu
IEEE International Working Conference on SourceCode Analysis and Manipulation, 2011
W o r k i n g i s h
a r d...
8/12/2019 Quality Assurance and Software Evolution - EmpiricalSE
7/26
... e s p e c i a l l y w
h e n r e s o u r c
e s a r e a w e
d
Clients of Flawed Classes
vs.Defects
D o e s a n y c o
r r e l a t i o n e x i
s t i n
E c l i p s e ?
D a t a
C l a s s
F e at u r e E n v y
G o d C l as s B r a
i n C l
a s s
has defects (pre and post release)
hasdesign aws
has provider classeswith design aws
?
8/12/2019 Quality Assurance and Software Evolution - EmpiricalSE
8/26
Chi-Square
Odds RatioMann-Whitney
Non-parametric statistical tests awed providers correlates with defects !
aws / awed providers
vs.defects
Odds Ratioon Defects
HASDesign Flaws
NODesign Flaws
HASFlawed Providers
NOFlawed Providers -
Odds Ratioon Defects
HASDesign Flaws
NODesign Flaws
HASFlawed Providers
NOFlawed Providers
0.5 - 0.7 -
8/12/2019 Quality Assurance and Software Evolution - EmpiricalSE
9/26
Odds Ratioon Defects
HASDesign Flaws
NODesign Flaws
HAS
Flawed Providers 2 - 3
NOFlawed Providers
0.5 - 0.7 -
Odds Ratioon Defects
HASDesign Flaws
NODesign Flaws
HAS
Flawed Providers 4 - 8 2 - 3NOFlawed Providers
0.5 - 0.7 -
Odds Ratioon Defects
HASDesign Flaws
NODesign Flaws
HASFlawed Providers 4 - 8 2 - 3
NOFlawed Providers
0.5 - 0.7 -
The biggest danger is using awed classes !
Percent ofclasses with... Design Flaws
NODesign Flaws
Flawed Providers
NOFlawed Providers
8/12/2019 Quality Assurance and Software Evolution - EmpiricalSE
10/26
Percent ofclasses with... Design Flaws
NODesign Flaws
Flawed Providers12
NOFlawed Providers
7
Percent ofclasses with... Design Flaws
NODesign Flaws
Flawed Providers12
42NOFlawed Providers
7 40
Are the Classes that useExceptions Defect Prone?
Author: Cristina Marinescu
12th International Workshop on Principles on Software Evolution7th ERCIM Workshop on Software Evolution
2011
E x c e p t i o n s
h i g h l i g h t e x c
e p t i o n a l
8/12/2019 Quality Assurance and Software Evolution - EmpiricalSE
11/26
S h o u l d W E C
A R E M O R E a
b o u t ?
Exceptions
&Defects
D o e s a n y c o
r r e l a t i o n e x i
s t i n
E c l i p s e ?
Class
Location,Name, Type
Class
8/12/2019 Quality Assurance and Software Evolution - EmpiricalSE
12/26
Location,Name, Type
Throws,Catches
Class
Location,Name, Type
Throws,Catches
Defects (Pre&Post Release)
Class
Chi-Square &
Odds Ratio
N o n - p a r a m e
t r i c S t a t i s t i c a l
T e s t s
No Exceptions
T h e S t r u c t u r
e o f t h e
C o n t i n g e n c y
T a b l e s.
8/12/2019 Quality Assurance and Software Evolution - EmpiricalSE
13/26
Defects NoDefects
No Exceptions
T h e S t r u c t u r
e o f t h e
C o n t i n g e n c y
T a b l e s.
Defects NoDefects
No Exceptions
Exceptions T h e S t r u c t u r
e o f t h e
C o n t i n g e n c y
T a b l e s.
p-value ? 0.05 p-value < 0.05
C o r r e l a t i o n
e x i s t s !
8/12/2019 Quality Assurance and Software Evolution - EmpiricalSE
14/26
Defects NoDefects
No Exceptions < >
Exceptions >
Exceptions >