CIS 4932Software Testing

White-Box Testing

2/2000 Unit Test Concepts 2

WHITE-BOX TESTING

SoftwareunderTest

Stimuli Response(s)

Testing to ensure that software does not do what is not supposed to do. Test ALL of it!

Tick -- Tick -- TickNo-Surprise Software!!

2/2000 Unit Test Concepts 3

WHITE-BOX TESTING

• Focus is thorough execution of program elements during the testing process.

• Warning: Tests only what is built, not what was intended!

SoftwareunderTest

Stimuli Response(s)

2/2000 Unit Test Concepts 4

WHITE-BOX TESTING

• Concept of coverage. Numeric measure of thoroughness of testing, relative to • Statements• Branches• Conditions• Paths

2/2000 Unit Test Concepts 5

CONTROL FLOW GRAPH

• Defines the flow of control through unit.

12 4

53

2/2000 Unit Test Concepts 6

CONTROL FLOW GRAPH TERMINOLOGY

• 5 NODES• sequential blocks of code

terminated by a branch• 3 PATHS:

• [1,2,3,5], [1,2,5], [1,4,5]• 2 BRANCHES

• 1 and 2 are decision nodes

12 4

53

2/2000 Unit Test Concepts 7

CONTROL FLOW GRAPH COVERAGE

• One test case forces execution of one path (red)

• Paths are determined by branches (decision nodes)

• A thorough test set forces execution of all paths (red, green, blue).

1

2 4

53

2/2000 Unit Test Concepts 8

COVERAGE LEVELS (%)

• Statement -- a statement has been executed at least once during testing

• Branch -- each outcome of a branch has been performed at least once during testing

• Path -- a path through the code has been executed at least once during during testing

• Condition -- a condition has evaluated to true and to false at least once during testing

2/2000 Unit Test Concepts 9

CONTROL FLOW GRAPH COVERAGE MEASUREMENT

• For 2 test cases (red, green)• Node (statement) cov = 4/5• Branch cov = 1/2 [2]• Path cov = 2/3

• Acceptable coverage levels• Statement cov = 90%• Branch cov = 80%• Path cov = 70%

1

2 4

53

2/2000 Unit Test Concepts 10

BRANCH vs CONDITION COVERAGE

• Code example 1

2 4

• 100% Branch coverage [1] • (x=0,y=2), (x=1,y=2) [TT,FT]

• But not 100% Condition coverage • Need case TF (x=0,y=1)

if (x<1 && y>1) x = x + y;else y = y - x;

2/2000 Unit Test Concepts 11

THE PROBLEM WITH COMPOUND CONDITIONS

• Makes complex logic appear simpler than it really is

• Test cases may be omitted• Logic results in 3 paths, not 2!!

12 3

if (x<1){if (y>1) x=x+y; else y=y-x;} else y=y-x;

12 5

3 4

2/2000 Unit Test Concepts 12

THE PROBLEM WITH PATH COVERAGE

• Not all paths are feasible• No test case can force path [1,2,3,4,5].

Consecutive decisions mutually exclusive.

if (x<1) y=2;if (x >= 1) y=3;z=y;

12

4

3

5

2/2000 Unit Test Concepts 13

Measuring Path Testing Difficulty

• McCabe metric -- logical code complexity • Formula: 1 + #decisions in control flow graph• Test Significance: #basis paths through code• Design use: complexity of code• Test use: min #test cases for 100% path coverage

• McCabe measures test (development) difficulty

2/2000 Unit Test Concepts 14

How To Design White Box Tests

• Test cases must execute different paths• Decision tables

• Rows -- elementary conditions in the code• Columns -- combinations of conditions in the code• Column based test case forces flow through

different logic paths in the code• Decision table built from code reflects what was

built versus intended (from spec)• Decision analysis for white-box testing.

2/2000 Unit Test Concepts 15

WHITE-BOX TESTING TOOLS

• Tools instrument source code and gathers coverage data when tests • Compiled languages• Script languages -- coming to market

• Some provide test case design assistance• List of paths not covered• Data conditions to force branch/path• Graphic depiction of graph coverage

2/2000 Unit Test Concepts 16

Problem P1Code (v1) & Decision Table

if (Age>80 || Weight>300) return 0; if (Age <= 12) return 1;if (Age > 65) return 2;if (Weight < 120) return 2else return 2+(Weight-120)/50;

Age>80 | Y N N N N N

Weight>300 | - Y N N N N

Age<=12 | Y N N N

Age>65 | Y N N

Weight<120 | Y N Pills = | 0 0 1 2 2 C

------------------------------

Note: C: 2+(Weight/120)/50

McCabe = 6

2/2000 Unit Test Concepts 17

Problem P1Code (v2) & Decision Table

if (Age>80 || Weight>300)return 0; if (Age <= 12) return 1;if (Age > 65 || (Age<=65 && Weight<120)) return 2;return 2+(Weight-120)/50;

Age>80 | Y N N N N N N

Weight>300 | - Y N N N N N

Age<=12 | Y N N N

Age>65 | Y N N

Weight<120 | - Y N Pills = | 0 0 0 1 2 2 C

McCabe = 7

2/2000 Unit Test Concepts 18

Your Turn -- White-Box Testing(1) Construct Decision Table

pills=0;if (Age < 80 && Weight <300){ pills=1; if (Age >= 65) pills=2; else if (Age > 12) pills=2+(Weight-120)/50;}return pills;

____________ |

____________ |

____________ |

____________ |

____________ |Pills = | - - - - - - -

---------------------------

Note: C: 2+(Weight/120)/50

2/2000 Unit Test Concepts 19

Your Turn -- P1(2) Derive White-Box Test Cases

Case 1 2 3 4 5 6 7 8 9

Age ___ ___ ___ ___ ___ ___ ___ ___ ___

Weight ___ ___ ___ ___ ___ ___ ___ ___ ___

Pills ___ ___ ___ ___ ___ ___ ___ ___ ___

2/2000 Unit Test Concepts 20

OBSERVATIONS -- WHITE-BOX TEST CASES

• Code may not be complete with respect to input combinations from the specification

• Decision table constructed from code is simpler -- subset of black-box table

• Claim: black-box test cases force coverage of logic• Unless the code implements the wrong (a

different) function

2/2000 Unit Test Concepts 21

MAIN POINTSWHITE-BOX TESTING

• White-box = logic testing• Limitation: can't tell what's missing• Don't forget exceptions -- throwing,

catching, propagating (debugger)• Perform decision analysis of code• Coverage tools help.• Use black-box test cases.