Models for Software Reliability

N. El KadriSEG3202

2

Definitions

• 1991 IEEE standard: the probability of failure-free software operation for a specified period of time in a specified environment

• The quality of the product improves over time, and we talk about reliability growth

• Software-reliability growth problem: estimating and predicting the reliability of a program as faults are identified and attempts made to fix them.

• Fault density - The number of faults per thousand lines of executable source code.

3

Using Reliability Models

4

Taxonomy of Software Reliability Models

5

Time-dependent Reliability Growth Models

6

Time-dependent Reliability Growth Models

7

Discussion

• One problem with models is an overwhelming number of models has been proposed to address the issue of software reliability assessment.

• We must be aware that no single model can be recommended universally to users under any circumstances.

• The best models may vary from time to time and differ form application to application.

8

Time-Independent Reliability Models: Fault Injection Estimates the number of faults NN in the

system in the case where we know the outcomes of the already detected faults.

1. Insert into a software module a certain number AA of faults

2. Proceed with its testing

3. Count the number of failures due to injected faults (ff) Count the number of failures due to inherent faults (i i )

4. Estimate the number of remaining faults

=>

9

Time-Independent Reliability Models: Fault Injection Example: 1. Insert into a software module A=25A=25 faults

2. Proceed with its testing (results: 32 failures)

3. Count the number of failures due to injected faults (f=17f=17)

4. Count the number of failures due to inherent faults (i=15i=15)

5. Estimate the number of remaining faults

10

Fault Injection: Confidence Levels about the Number of Faults1. If not all artificial faults have been found

(f<A), then:

If all artificial faults have been found (f=A), then:

11

Fault Injection: Example (cont.)

1. Confidence levels about the number of faultsLet us set E to 10. f < A =>

12

Fault Injection: Example 2 1. Assume all artificial faults have been

found (f=A)

2. Given E=10, certain confidence level λ and i<E,

how to determine the number of faults to be injected?

13

Time-Independent Reliability Models: Input Domain

1. Software reliability depends on how software operates on a certain input domain.

2. This point of view relates to selecting software test cases over the input domain according to how software is used

• Software usage information includes the environment information where software is used, as well as the information on the actual frequency of usage of different operations, functions, or features that the system offers.

• The usage information is quantified through operational profiles

14

Input Domain: Equivalence Classes

Software Module

16

Operational Profiles

• Methodology to develop operational profile:

1. Determine customer profile or usage context profile.

2. Determine user profile.3. Determine system modes and their profile.4. Determine the functional (requirements)

profile.5. Determine the operational (implementation)

profile.

17

Operational profile

• A set of relative frequencies (or probabilities) of occurrence of disjoint software operations during its operational use

• A software-based system may have one or more operational profiles.

• Operational profiles are used to select test cases and direct development, testing and maintenance efforts towards the most frequently used or most risky components.

18

Operational profile• Construction of an operational profile is

preceded by definition of a customer/user profile, a system mode profile, and a functional profile.

• Profiles are constructed by creating detailed hierarchical lists of customers, users, modes, functions and operations that the software needs to provide under each set of conditions.

• For each item estimate the probability of its occurrence and thus provide a quantitative description of the profile.

• If usage is available as a rate (e.g., transactions per hour) it needs to be converted into probability.  

19

User Profile - Example

20

System Mode

21

System Modes ExampleWe can identify five system modes:

1. Normal Traffic load2. High Traffic load3. Start/Restart4. Administration5. Troubleshooting

The first three are only relevant to the subscriber user type, the fourth to the operator user type and the fifth to the customer care system user type.

22

Functional Profile

23

Operational Profile

24

Operational Profile

25

Input Domain: Equivalence Classes

Software Module

26

Input Domain: Equivalence Classes & Operational Profiles1. Each Path through the Operational Profile

hierarchy defines an equivalent class:

• Path E1: subscriber, normal traffic, MO SMS, originating MO SMS

• Each path Ej has its associated probability Pj stating that the inputs will come from it under normal operation of the system

• Assuming the inputs are independent from each other, the probability of a path is a product of the corresponding operational probabilities, i.e.: for E1 the associate probability is

P1 = 0.9*0.6*0.05*0.95 = 0.35

27

Input Domain Model: Reliability

Estimation 1. Assumption: c equivalent classes Ei

2. With each class comes its operational profile3. Let Pi be the probability stating that the

inputs will come from Ei under normal operation of the system

4. nj is the number of test cases sampled from the jth input domain Ei, where fj out of them resulted in software failures

5. The estimated reliability is computed as:

28

System reliability• Knowing the reliability of individual

components Ri, one can easily compute the reliability of some architectures as follows:

(a) Series configuration: (b) Parallel configuration:

29

Classes of Reliability Models and their main Features

31

Availability

32

ANSI/IEEE 982.1-1988

Includes guidance for the following: • Applying product and project measures

throughout the software life cycle • Optimizing the development of reliable software

with respect to constraints • Maximizing the reliability in its actual use

environment • Developing the means to manage reliability in the

same manner that cost and schedule are managed

33

More Information