1

Chapter 17Chapter 17Software Testing Software Testing

TechniquesTechniques

2

Software Software TestingTesting

Testing is the process of exercising aTesting is the process of exercising aprogram with the specific intent of findingprogram with the specific intent of findingerrors prior to delivery to the end user.errors prior to delivery to the end user.

3

TestabilTestabilityity OperabilityOperability—it operates cleanly—it operates cleanly

ObservabilityObservability—the results of each test —the results of each test case are readily observedcase are readily observed

ControlabilityControlability—the degree to which —the degree to which testing can be automated and optimizedtesting can be automated and optimized

DecomposabilityDecomposability—testing can be targeted—testing can be targeted SimplicitySimplicity—reduce complex architecture —reduce complex architecture

and logic to simplify testsand logic to simplify tests StabilityStability—few changes are requested —few changes are requested

during testingduring testing UnderstandabilityUnderstandability—of the design—of the design

4

What Testing ShowsWhat Testing Shows

errorserrors

requirements conformancerequirements conformance

performanceperformance

an indicationan indicationof qualityof quality

5

Who Tests the Who Tests the Software?Software?

developerdeveloper independent testerindependent tester

Understands the system Understands the system

but, will test "gently"but, will test "gently"

and, is driven by "delivery"and, is driven by "delivery"

Must learn about the system,Must learn about the system,but, will attempt to break itbut, will attempt to break itand, is driven by qualityand, is driven by quality

6

Exhaustive TestingExhaustive Testing

loop < 20 Xloop < 20 X

There are 10 possible paths! If we execute oneThere are 10 possible paths! If we execute onetest per millisecond, it would take 3,170 years totest per millisecond, it would take 3,170 years totest this program!!test this program!!

1414

7

Selective TestingSelective Testing

loop < 20 Xloop < 20 X

Selected pathSelected path

8

Software TestingSoftware Testing

Methods

Strategies

white-boxmethods

black-box methods

9

Test Case Test Case DesignDesign

"Bugs lurk in corners "Bugs lurk in corners and congregate at and congregate at boundaries ..."boundaries ..."

Boris BeizerBoris Beizer

OBJECTIVEOBJECTIVE

CRITERIACRITERIA

CONSTRAINTCONSTRAINT

to uncover errorsto uncover errors

in a complete mannerin a complete manner

with a minimum of effort and timewith a minimum of effort and time

10

White-Box White-Box TestingTesting

... our goal is to ensure that all ... our goal is to ensure that all statements and conditions have statements and conditions have been executed at least once ...been executed at least once ...

11

Why Why Cover?Cover?

logic errors and incorrect assumptions logic errors and incorrect assumptions are inversely proportional to a path's are inversely proportional to a path's execution probabilityexecution probability

we often we often believebelieve that a path is not that a path is not likely to be executed; in fact, reality is likely to be executed; in fact, reality is often counter intuitiveoften counter intuitive

typographical errors are random; it's typographical errors are random; it's likely that untested paths will contain likely that untested paths will contain some some

12

Basis Path Basis Path TestingTesting

First, we compute the cyclomatic complexity:

number of simple decisions + 1

or

number of enclosed areas + 1

In this case, V(G) = 4

13

Cyclomatic Cyclomatic ComplexityComplexityA number of industry studies have indicated A number of industry studies have indicated

that the higher V(G), the higher the probability that the higher V(G), the higher the probability or errors.or errors.

V(G)V(G)

modulesmodules

modules in this range are modules in this range are more error pronemore error prone

14

Basis Path Basis Path TestingTestingNext, we derive the Next, we derive the

independent paths:independent paths:

Since V(G) = 4,Since V(G) = 4,there are four pathsthere are four paths

Path 1: 1,2,3,6,7,8Path 1: 1,2,3,6,7,8Path 2: 1,2,3,5,7,8Path 2: 1,2,3,5,7,8Path 3: 1,2,4,7,8Path 3: 1,2,4,7,8Path 4: 1,2,4,7,2,4,...7,8Path 4: 1,2,4,7,2,4,...7,8

Finally, we derive testFinally, we derive testcases to exercise these cases to exercise these paths.paths.

11

22

3344

55 66

77

88

15

Basis Path Testing Basis Path Testing NotesNotesyou don't need a flow chart, you don't need a flow chart, but the picture will help when but the picture will help when you trace program pathsyou trace program paths

count each simple logical test, count each simple logical test, compound tests count as 2 or compound tests count as 2 or moremore

basis path testing should be basis path testing should be applied to critical modulesapplied to critical modules

16

Loop TestingLoop Testing

Nested Nested LoopsLoops

Concatenated Concatenated LoopsLoops UnstructuredUnstructured

LoopsLoops

Simple Simple looploop

17

Loop Testing: Simple Loop Testing: Simple LoopsLoops

Minimum conditions—Simple LoopsMinimum conditions—Simple Loops

1. skip the loop entirely1. skip the loop entirely

2. only one pass through the loop2. only one pass through the loop3. two passes through the loop3. two passes through the loop4. m passes through the loop m < n4. m passes through the loop m < n5. (n-1), n, and (n+1) passes through 5. (n-1), n, and (n+1) passes through the loopthe loop

where n is the maximum number where n is the maximum number of allowable passesof allowable passes

18

Loop Testing: Nested Loop Testing: Nested LoopsLoops

Start at the innermost loop. Set all outer loops to their Start at the innermost loop. Set all outer loops to their minimum iteration parameter values.minimum iteration parameter values.

Test the min+1, typical, max-1 and max for the Test the min+1, typical, max-1 and max for the innermost loop, while holding the outer loops at their innermost loop, while holding the outer loops at their minimum values.minimum values.Move out one loop and set it up as in step 2, holding all Move out one loop and set it up as in step 2, holding all other loops at typical values. Continue this step until other loops at typical values. Continue this step until the outermost loop has been tested.the outermost loop has been tested.

If the loops are independent of one another If the loops are independent of one another then treat each as a simple loopthen treat each as a simple loop else* treat as nested loopselse* treat as nested loopsendif* endif*

for example, the final loop counter value of loop 1 is for example, the final loop counter value of loop 1 is used to initialize loop 2.used to initialize loop 2.

Nested LoopsNested Loops

Concatenated LoopsConcatenated Loops

19

Black-Box TestingBlack-Box Testing

requirementsrequirements

eventseventsinputinput

outputoutput

20

Equivalence Equivalence PartitioningPartitioning

useruserqueriesqueries mousemouse

pickspicks

outputoutputformatsformats

promptsprompts

FKFKinputinput

datadata

21

Sample Equivalence Sample Equivalence ClassesClasses

user supplied commandsuser supplied commandsresponses to system promptsresponses to system promptsfile namesfile namescomputational datacomputational data physical parameters physical parameters bounding valuesbounding values initiation valuesinitiation valuesoutput data formattingoutput data formattingresponses to error messagesresponses to error messagesgraphical data (e.g., mouse picks)graphical data (e.g., mouse picks)

data outside bounds of the program data outside bounds of the program physically impossible dataphysically impossible dataproper value supplied in wrong placeproper value supplied in wrong place

Valid dataValid data

Invalid dataInvalid data

22

Boundary Value Boundary Value AnalysisAnalysis

useruserqueriesqueries mousemouse

pickspicks

outputoutputformatsformats

promptsprompts

FKFKinputinput

datadata

outputoutputdomaindomaininput domaininput domain

23

Other Black Box TechniquesOther Black Box Techniques

error guessing methodserror guessing methods decision table techniquesdecision table techniques cause effect graphingcause effect graphing