1 / 38

CS 425/625 Software Engineering

Software Testing

Based on Chapter 20 of the textbook [Somm00] Ian Sommerville,Software Engineering, 6th Ed., Addison-Wesley, 2000 and on theCh20 PowerPoint presentation available at the book’s web-site:www.comp.lancs.ac.uk/computing/resources/IanS/SE6/Slides/index.html

November 12, 2002

2 / 38

Outline

IntroductionIntroduction Defect TestingDefect Testing

Black Box TestingBlack Box Testing Equivalence Partitioning

Structural (White Box) TestingStructural (White Box) Testing Path Testing

Integration Testing Integration Testing Top-DownTop-Down Bottom-Up Bottom-Up Interface TestingInterface Testing Stress TestingStress Testing

Object Oriented TestingObject Oriented Testing

3 / 38

Introduction..

A very high level view of testing phases [Fig. 20.1, Somm00]

Componenttesting

Integrationtesting

Software developer Independent testing team

4 / 38

.Introduction. Component testing is concerned with checking the

functionality of methods, classes, and modules Integration testing focuses on sub-systems and their

interaction as well as on the operation of the system as a whole

Component testing is individual programmers’ task while integration testing is assigned to an integration team

For critical systems independent testing teams are used during all testing

Integration testing is based on written requirements specifications

5 / 38

..Introduction

In object-oriented testing the boundaries between component and integration testing are blurred because Object-oriented systems do not have the clear

distinction between program units and program modules characteristic to structured (function-oriented) systems

The hierarchy of objects is not necessarily “nested”

6 / 38

Defect Testing

Defect testing is aimed at discovering errors in code

A successful defect test causes the system to perform incorrectly

During the testing process the following are created: Test cases Test data Test reports

7 / 38

Defect Testing

Comprehensive testing is not possible. Subsets of tests cases are designed based on criteria such as: All code statements should be executed at least once All functions accessed through menus should be

tested All combinations of functions accessed through the

same menu should be checked All user input situations should be checked with both

correct and incorrect data

8 / 38

Defect Testing: The Process

The defect testing process [Fig. 20.2, Somm00]

Design testcases

Prepare testdata

Run programwith test data

Compare resultsto test cases

Testcases

Testdata

Testresults

Testreports

9 / 38

Defect Testing: Black Box Testing.

In black box testing (or functional testing) the system is viewed from “outside,” only in terms of its functionality The software is tested against its specification Internal (implementation) details are ignored The behavior of the system is evaluated in terms

of inputs received, outputs expected, and outputs actually produced

A defect is discovered if the actual system output differs from the expected output

Applicable to both function-oriented and object-oriented systems

10 / 38

Defect Testing: .Black Box Testing

The black-box testing model [Fig. 20.3, Somm00]

Ie

Input test data

OeOutput test results

System

Inputs causinganomalousbehaviour

Outputs which revealthe presence ofdefects

11 / 38

Defect Testing: Equivalence Partitioning….

Equivalence partitioning makes use of classes of input test data (e.g., positive numbers, negative numbers, strings, etc.) Generally, systems tend to behave in a comparable way for

all members of a class Classes of input test data are called equivalence partitions or

domains Equivalence partitions are determined based on program

specification and user documentation; Test cases are derived for each equivalence partition Partition mid-point and boundary values provide useful test

data

12 / 38

.Defect Testing: .Equivalence Partitioning…

Equivalence partitioning [Fig. 20.4, Somm00]

System

Outputs

Invalid inputs Valid inputs

13 / 38

..Defect Testing: Equivalence Partitioning..

Equivalence partitions [Fig. 20.5, Somm00]

Between 10000 and 99999Less than 10000 More than 99999

999910000 50000

10000099999

Input values

Between 4 and 10Less than 4 More than 10

34 7

1110

Number of input values

14 / 38

…Defect Testing: Equivalence Partitioning.

Example: A search routine [Fig. 20.6, Somm00]

procedure Search (Key : ELEM ; T: ELEM_ARRAY;

Found : in out BOOLEAN; L: in out ELEM_INDEX) ;

Pre-condition -- the array has at least one element

T’FIRST <= T’LAST

Post-condition -- the element is found and is referenced by L

( Found and T (L) = Key) or

-- the element is not in the array

( not Found and not (exists i, T’FIRST <= i <= T’LAST, T (i) = Key ))

15 / 38

….Defect Testing: Equivalence Partitioning

Equivalence partitions for search routine [Fig. 20.7, Somm00]

16 / 38

Defect Testing: Structural Testing….

In contrast to black box testing structural testing (or white box, clear box, glass box testing) is based on the knowledge the testers have about the structure and the implementation of the software Applied generally to smaller program units The tester analyzes the code to determine test cases Types of structural testing:

Branch coverage: every branch of the unit is tested at least once

Statement coverage: every statement of the unit is executed at least once

Path coverage: every path through the unit is executed at least once

17 / 38

.Defect Testing: Structural Testing…

The structural testing process [Fig. 20.8, Somm00]

Componentcode

Testoutputs

Test data

DerivesTests

18 / 38

Structural Testing Example

Java

implementation

[Fig. 20.9, Somm00]

19 / 38

Defect Testing: …Structural Testing.

Equivalence classes for binary search [Fig. 20.10, Somm00]

Mid-point

Elements < Mid Elements > Mid

Equivalence class boundaries

20 / 38

….Defect Testing: Structural Testing

Test cases for binary search [Fig. 20.11, Somm00]

21 / 38

Defect Testing: Path Testing…

Path testing is a form of structural testing aimed at exercising each individual execution path of a program unit. It ensures that Each statement of the unit is executed at least once Each branch condition is tested with both true and

false conditions

The program flow graph is used for path testing. In a flow graph decisions are shown as nodes and the flow of control is represented by edges

22 / 38

Defect Testing: .Path Testing..

The cyclomatic complexity CC of a connected graph G is given by the formula:

CC(G) = Number of edges - Number of nodes + 2 If there are no go to statements and no

compound decisions (involving more than one test) then CC(G) = Number of predicate nodes + 1

(predicate nodes contain conditions that determine branching of the execution flow)

The cyclomatic complexity of a graph gives the minimum number of test cases needed to cover all paths

23 / 38

Defect Testing: ..Path TestingDefect Testing: ..Path Testing.

Flow graph for binary search [Fig. 20.12, Somm00]

1

2

3

4

65

7

while bottom <= top

if (elemArray [mid] == key

(if (elemArray [mid]< key8

9

bottom > top

24 / 38

Defect Testing: …Path Testing

Independent paths in previous case

1, 2, 3, 8, 9

1, 2, 3, 4, 6, 7, 2

1, 2, 3, 4, 5, 7, 2

1, 2, 3, 4, 6, 7, 2, 8, 9

In programs with complex branching structure the number of paths is high and it is often difficult to predict the program’s behaviour

A dynamic program analyzer can be used to identify all paths

25 / 38

Integration Testing.…..

Integration involves building sub-systems from program units and the system from subsystems

Integration testing uses software specifications as basis for checking the system

Errors are more difficult to locate than in unit testing The incremental approach is recommended for system

integration and testing Initially, a minimal system configuration can be used Tests need be repeated after each addition to the

system

26 / 38

.Integration Testing…...Integration Testing…..

Incremental integration testing [Fig. 20.13, Somm00]

T3

T2

T1

T4

T5

A

B

C

D

T2

T1

T3

T4

A

B

C

T1

T2

T3

A

B

Test sequence1

Test sequence2

Test sequence3

27 / 38

..Integration Testing….

Strategies for system integration and testing: Top-Down

Starts with high level components and continues at lower levels

Integral part of the top-down development process Stubs are needed to test the system

28 / 38

…Integration Testing…

Strategies for system integration and testing [cont’d]: Bottom-Up

Starts with lower level components and builds up the system by adding & testing higher level components

Can be used when components from previously developed systems are reused or when the system’s functionality relies on critical lower level modules

Driver modules are used to test the system

“Sandwich” Combines the above two strategies by using the Top-Down

approach for logic modules (higher level) and the Bottom-Up approach for operational (lower level) modules

29 / 38

….Integration Testing..

Top-down integration testing [Fig. 20.14, Somm00]

Level 2Level 2Level 2Level 2

Level 1 Level 1Testing

sequence

Level 2stubs

Level 3stubs

. . .

30 / 38

…..Integration Testing.

Bottom-up integration testing [Fig. 20.14, Somm00]

Level NLevel NLevel NLevel NLevel N

Level N–1 Level N–1Level N–1

Testingsequence

Testdrivers

Testdrivers

31 / 38

…...Integration Testing

Top-Down versus Bottom-Up: Architectural validation

Top-Down allows earlier discovery of high-level design errors

System demonstration Top-Down supports better an early demo; Bottom-Up can

support such demo if many reusable components are utilized

Test implementation Strict Top-Down testing is more difficult to implement given

that stubs are needed. It is easier to use drivers in a strict Bottom-Up testing

Test observation Equally challenging, since both stubs and drivers may not

allow complete observations

32 / 38

Interface Testing… Interface testing has the goal of detecting errors related to

interfacing modules and sub-systems Particularly important for object-oriented testing given objects’

collaboration relies on their interfaces Types of interfaces:

Parameter interfaces Shared memory interfaces Procedural interfaces Message passing interfaces

Classes of interface errors: Interface misuse Interface misunderstanding Timing errors

33 / 38

.Interface Testing..

Interface testing [Fig. 20.15, Somm00]: tests directed at the sub-system as a whole, not at particular components

Testcases

BA

C

34 / 38

..Interface Testing.

Guidelines for interface testingGuidelines for interface testing Test all external calls with parameters at extreme Test all external calls with parameters at extreme

rangesranges When pointers are used, test interfaces with null When pointers are used, test interfaces with null

pointerspointers For procedural interfaces try to design tests that would For procedural interfaces try to design tests that would

cause the components to failcause the components to fail Use stress testing for message passing interfacesUse stress testing for message passing interfaces Change activation order of components that share Change activation order of components that share

memorymemory

35 / 38

…Interface Testing

Stress testing exercise the systems with Stress testing exercise the systems with overloadingoverloading (e.g., number of transactions per minute, (e.g., number of transactions per minute, number of users in a distributed applications)number of users in a distributed applications)

The main purposes of stress testing:The main purposes of stress testing: Make sure the system does not lose or corrupt data Make sure the system does not lose or corrupt data

when overloadedwhen overloaded Reveal defects that otherwise would pass unnoticedReveal defects that otherwise would pass unnoticed

36 / 38

Object-Oriented Testing..

Differences from function-oriented systems: Objects are usually larger units, which

encompass several methods Objects are typically loosely coupled and there

may not be an obvious high top of the class hierarchy

Testers may not have access to the code of reused objects

37 / 38

.Object-Oriented Testing.

Levels of testing in object-oriented testing: Testing operations of object classes (sequences

of operations may be needed) Testing object classes (operations, attributes,

and states) Testing clusters of objects (use case or scenarios

based testing; thread testing, etc.); each operation in each class should be checked at least once

Testing the object-oriented system

38 / 38

..Object-Oriented Testing

Compared with testing function-oriented systems, testing of object-oriented system is facilitated by inheritance + reuse (previously tested objects) but specific challenges exist because: Information hiding is a complicating factor Writing additional methods (invoked for testing

purposes only) are often necessary New and overwritten methods must be tested Inherited methods must still be re-tested if they

interact with newly written methods