1

Software Testing and Quality Assurance

Lecture 31 – Testing Systems

2

Lecture Objectives Types of Testing Measuring Test Coverage Software Maintenance Activities

3

Types of testing Stress testing Life-cycle testing Performance testing

4

Types of testing: stress testing Stress testing is operating a system under conditions that

come close to exhausting the resources needed by the system:

Filling RAM with objects. Filling hard disk with records. Move the mouse back and forth. Natural growth of information that accumulates as the

system is operated. For OO systems:

Create large number of instances of classes

5

Types of testing: steps in stress testing

Identify the variable resources that increase the amount of work the system has to do

If there are relationships among these resources, develop a matrix that lists combinations of resource levels to use

Create test cases that use each combination

Execute and evaluate the results

6

Types of testing: life-cycle testing Two types of life cycles that do make sense

to test: Domain life cycles: these correspond to the

key processes in the domain (for accounting system run a series of test to cover a full fiscal year).

Computer application life cycles: it begins with the installation of the application and ends with its removal (test the installer and uninstaller programs).

7

Types of testing: performance testing Perception: object-oriented systems are

inherently slow. Testing for performance is like measuring

the reliability of a piece of software. Establishing the context within which the

performance will be measured: Context: description of the environment in

which the measurement will be made.

8

Types of testing: performance testing technique Define the context in which performance

measure applies: Describe the state of the application to be

tested. Describe the execution environment in

terms of the platform being used. Describe the other applications running at

the time of tests

9

Types of testing: performance testing technique

Identify the extremes within the context. Define, as expected result, what will

constitute acceptable performance. Execute the tests and evaluate the

results.

10

Types of testing: problems with real data in testing

Problems with real data in testing: Extensive past data available. Analysis of the data requires time.

11

Types of testing: problems with real data in testing

It might be quicker to manufacture data that has specific properties than to use real data: Analyze the existing data to identify patterns. Construct test data that follow these patterns but

for which the expected results are more easily determined

Design test cases that use the test data in the context of the of a complete life cycle.

Execute and evaluate the results.

12

Testing different types of systems

Reactive systems Embedded systems Multitiered systems Distributed systems

13

Measuring test coverage Coverage: is the metrics that indicates level

of confidence we have in our testing. What is to be covered?

To measure coverage two categories can be used:

Inputs: how many of the possible inputs have been used in test cases.

Outputs: how many of the possible outputs that a system can produce have been produced during testing.

14

Measuring test coverage As systems have become more

integrated into Life-critical and mission-critical systems, Expectations have increased to include

“does the system do any thing it is not suppose to do?”

15

Measuring test coverage (cont...) When is coverage measured?

Coverage data is collected continuously during testing

When is coverage used? “at release time”, coverage is part of the

decision criteria for release.

16

Software maintenance activities Corrective maintenance are activities

of handling problems resulting from faults.

Adaptive Maintenance involves adjusting the application to changes in the environment, or changes from other parts of the system.

17

Software maintenance activities

Perfective maintenance involves making changes to improve some aspect of the system, even when the changes are not suggested by faults.

Preventive maintenance involves changing some aspect of the system to prevent failures.

18

Software maintenance activities (cont...)

Corrective21%

Perfective50%

Adaptive25%

Preventive4%

19

Key points

Types of testing: stress testing, life-cycle testing, and performance testing

Coverage: is the metrics that indicates level of confidence we have in our testing.

20

Announcement SWE Revised Program Discussion

Monday December 29, 2008 at 12:10-1:00 in 24/141