Upload
hironori-washizaki
View
190
Download
0
Embed Size (px)
Citation preview
Generalized Software Reliability Model
(GSRM)Hironori Washizaki
Waseda University
National Institute of InformaticsTwitter: @Hiro_Washi [email protected]
http://www.washi.cs.waseda.ac.jp/
Kiyoshi Honda, Hironori Washizaki, YoshiakiFukazawa, “A Generalized Software Reliability Model Considering Uncertainty and Dynamics in Development,” PROFES 2013
Motivation
• When can we release software?• How many efforts are necessary for further
testing?
2Time
Software Reliability Model (SRM)
3
Counting the defects a day or a week
Approximate actual data to a curve and predict defects
At this time, 95% of all defects will be found
Prediction model
Types of SRM
• Statistic analysis model– From actual data approximate
to a curve.– Gompertz model– Logistic model
• Stochastic process model– The detection of defects
follows stochastic process – Non-homogeneous Poisson
process(NHPP) model [Goel]4[Goel] A.L. Goel and K. Okumoto A non-homogeneous poisson process model for
software reliability and other performance measures, 1979
5
#Defects
Actual
Predicted
Days
Case (Industry)
Further Challenges in SRM
• Uncertainty– Actual projects have many uncertain elements
which cause defects.– E.g. changes of specifications
• Dynamicity– Actual projects have some time dependency.– E.g. changes of developers
6
Idea: Generalized SRM• Conventional Logistic Model
• Assumptions– Number of defects that can be found is variable depending
on time.– Number of defects that can be found contains uncertainty,
which can be simulated with Gaussian white noise.
7
Dynamicity Uncertainty
A generalized software reliability model considering uncertainty and dynamics in development. PROFES ’13
Uncertainty
8
the uncertainty is greater at the start of the project than at the end.
The uncertainty is constant at any given time.
The uncertainty increases near the end.
Dynamicity
9
The number of developers is constant.
The number of developersper unit time changes at a certain time.
The number of developers per unit time increase near the end.
Combination of Uncertainty and Dynamicity
10
Similar to a logistic curve
Constant
Prediction with Probability
11
95% of defects are expected to be foundduring this term.
12
Visualization integrated with Continuous Integration Tool
Case (Industry)
13
abyss_data-api abyss_data-api abyss_data-api
Module Current PredictedTotal
PredictedCurrent
PredictedEnd Day
XYZ 147 144 134 156
Almost all defects seem to be detected.Now more debugging rather than testing.
#Defects #Predicted Total DefectsPredication Error
Case (OSS)
14
Open Research Questions
• How to Predict Uncertainty and Dynamicity?• How to dynamically adapt prediction?• Can we integrate testing and debugging
techniques with (G)SRM?• Any relations among Predicted Reliability and
other measures such as Testing Coverage and Mutation Testing Scores?
Background: Testing and Reliability• Empirical
– Bug Localization in Industry [ICST’14]
– Testing in OSS [PROFES’14]
• Web and Ajax– Gray Box Testing [FASE’13]– Mutation Testing [SEKE’13]– State-Machine Extraction
and Verification [ASE’13][AES’14]
• Software Reliability Model [PROFES’13]
16
0
0.2
0.4
0.6
0.8
1
1.2
0 0.5 1
The rate of tested
cases
Time
Model 1- 1
Model 2- 1
Model 3- 1
Semi-automatic Incompatibility Localization for Re-engineered Industrial Software, ICST’14Do open source software projects conduct tests enough?, PROFES’14Mutation Analysis for JavaScript Web Applications Testing, SEKE’13Automated Verification of Pattern-based Interaction Invariants in Ajax Applications, ASE’13Validating Ajax Applications Using a Delay-Based Mutation Technique, ASE’14A generalized software reliability model considering uncertainty and dynamics in development. PROFES’13
• Assoc. Prof., Head, Global Software Engineering Laboratory, Waseda University
• Visiting Assoc. Prof., National Institute of Informatics
• Chair, SEMAT Japan Chapter• Chair, ISO/IEC SC7/WG20 Japan• Secretary, IEEE CS Japan• Program Organizations: ASE,
SEKE, SPLC, APSEC, AsianPLoP• Many PCs 17
“Hiro”nori Washizaki
Waseda UniversityGlobal Software Engineering Lab.
• SE researches in global development era
• 6 professors• 17 partner members• 25 students (5 from
China, Norway, Mexico, Indonesia)
18
Recent PublicationsReuse and Development Environments
Unified Code Processing Framework (ICST’13)Evaluation of Separation of Concerns Paradigms (SEKE’13) Traceability Links between Requirements and Code (SPLC’13)Reusable Refactoring Scripts (SEKE’14)
Automated Testing and VerificationBug Localization for Re-engineered Industrial Software (ICST’14)Web Test Code Generator Based on Template (FASE’13)Validating Security Design Pattern Application (ARES’13-14) Mutation Analysis for JavaScript Testing (SEKE’13)Automated Verification of Ajax Applications (ASE’13, ASE’14)
Quality ManagementSoftware Reliability Model Considering Uncertainty (PROFES’13, APSEC’14) Organizational Changes and Quality (APSEC’13)Empirical Study of Testing in OSS (PROFES’14)
Agile, Patterns and EducationEffectiveness of workshops for learning agile (Agile’13)Pattern language for Agile Development and Robotics (PLoP’13-14, Agile’14)Team Characteristics for Educational Effectiveness (CSEET’13-14) 19