SWTest 04 - White Box Testing

7/28/2019 SWTest 04 - White Box Testing

1/56

Software Testing Slide 1

Software Testing(09CS 491)

Unit IIWhite Box Testing(Chapter 3)

Testing can prove presence of defects and not their absence!


2/56


Objectives

l To review the V Model and set the context for different types of

tests

l To focus on white box testing

l To distinguish between different types of white box testing

l To place special emphasis on

Code reviews / inspections

Methods of code coverage and relationship to quality

Code complexity


3/56


Bending the Waterfall

High-LevelDesign

Low-LevelDesign

Development

Testing

Unit Testing

ComponentTesting

IntegrationTesting

SystemTesting

AcceptanceTesting

SoftwareRequirements

OverallBusinessRequirements


4/56


OverallBusinessRequirements

SoftwareRequirements

High-LevelDesign

Low-Level

Design

Development Unit TestExecution

ComponentTestExecution

IntegrationTest Execution

SystemTest Execution

AcceptanceTest Execution

AcceptanceTest Design

SystemTest Design

IntegrationTest Design

Component

Test Design

Unit TestDesign

V Model


5/56


Principles of the V Model

l Do an early test design.

l Involve the right people in the design of the right type of tests.

l Make the tests reflect closely the steps on the left-hand side ofthe V.

l Unit testing

White box testing

Black box testing


6/56


White Box Testing

l Requires access to code

l Looks at the program code and performs testing by

mapping program code to functionality


7/56 Software Testing Slide 7

White-box testing is a method of testing software that tests internal

structures or workings of an application.

White box testing is also known as clear, open, structural, and glass

box testing.

White Box Testing method is applicable to the following levels of software

testing:

Unit Testing : For testing paths within a unit

Integration Testing : For testing paths between units

System Testing : For testing paths between subsystems

White Box Testing



Types of White Box Testing



Why White Box Testing?

l The program code truly represents what the program actually

does and not just what it is intended to do!

l It minimizes delay between defect injection and defect detection

(i.e., does not postpone detection to be done by someone else).

l Can catch obvious programming errors that do not necessarily

map to common user scenarios (e.g., divide by zero).



Static Testing

l Involves only the source code and not the executable or binaries

l Does not involve executing the program on a machine but rather

humans going through it or the usage of specialized tools

l Some of the things tested by static testing:

Whether the code works according to the functional requirement

Whether the code has been written in accordance with the design

developed earlier in the project life cycle

Whether the code follows all applicable standards

Whether the code for any functionality has been missed out

Whether the code handles errors properly



Static Testing by Humans

l Humans can find errors that computers cant.

l Multiple humans can provide multiple perspectives.

l A human evaluation can compare the code against thespecifications more thoroughly.

l It can detect multiple defects at one go.

l It reduces downstream, inline pressure.



Static Testing Types

l Different types

Desk checking of the code

Code walkthrough

Code review

Code inspection

l QA vs QC argument

l Increasing the involvement of people

l More variety of perspectives

l Increasing formalism

l Increasing the likelihood of identifying more complex defects



Desk Checking

l Author informally checks the code against the specifications and

corrects defects found.

l No structured method or formalism is required to ensure

completeness.

l No log or checklist is maintained.

l It relies only on the thoroughness of the author.

l It may suffice for obvious programming errors but may not be

effective for incomplete / misunderstood requirements.



Advantages and Disadvantages of

Desk Checking

l Advantages

The programmer knows the code and programming language well

and hence is best suited to read the program

Less scheduling and logistics overheads

Reduces delay in defect detection and correction

l Disadvantages

Person dependent, not scalable or reproducible

Tunnel visioning of developers

Developers prefer writing new code and dont like testing



Code Walkthrough

l Group oriented (as against desk checking)

l Brings in multiple perspectives

l Multiple specific roles (author, moderator, inspector, etc.),

as discussed in Fagan Inspection



Formal Inspection / Fagan Inspection

l Group-oriented activity

l Highly formal and structured

l Has specific roles

l Requires thorough preparation


17/56


Roles in a formal inspection

l Author Author of the work product

Makes available the required material to the reviewers

Fixes defects that are reported

l Moderator

Controls the meeting(s)

l Inspectors (reviewers)

Prepare by reading the required documents

Take part in the meeting(s) and report defects

l Scribe

Takes down notes during the meeting

Assigned in advance by turns

Can participate to review to the extent possible

Documents the minutes and circulates them to participants


18/56


Process in a Fagan Inspection

Typical documents circulated:

Program code

Design / program specifications

SRS (if needed)

Any applicable standards (e.g., coding standards)

Any necessary checklists (e.g., code review checklist)


19/56


Meetings in a Fagan Inspection

l Preliminary meeting (optional)

Author explains his / her perspective

Makes available the necessary documents

Highlights concern areas, if any, for which review comments are sought

l Defect Logging Meeting

All come prepared!

Moderator goes through the code sequentially

Each reviewer comes up with comments

Comments / defects categorized as defect / observation, major /

minor, systemic / mis-execution

Scribe documents all the findings and circulates them


20/56


Meetings in a Fagan Inspection

l Follow-up meeting (optional)

Author fixes defects

If required, a follow-up meeting is called to verify completeness of fixes


21/56


Advantages and Disadvantages of

Fagan Inspection

l Advantages

Thorough, when prepared well

Brings in multiple perspectives

Has been found to be very effective

l Disadvantages

Logistically difficult

Time consuming

May not be possible to exhaustively go through the entire code

White Box TestinSlide : 21


22/56


Combinations of various methods

l Prioritizing various programs

High priority / important codes subject to formal

inspection, medium ones for walkthrough and low ones

for desk checking

l Deciding between 100% coverage and partial coverage


23/56


Static Analysis Tools

Are used to identify errors such as:l Whether there are unreachable codes (usage of goto statements sometimes

creates this situation; there could be other reasons too)

l Variables declared but not used

l Mismatch in definition and assignment of values to variables

l Illegal or error-prone type-casting of variables

l Use of non-portable or architecture-dependent programming constructs

l Memory allocated but not having corresponding statements for freeing up

memory

l For calculation of cyclomatic complexity

l As an extension of compilers (lint, compiler flag driven checking)


24/56


Use of a Code Review Checklist

l The code review checklist aids in organizational learning

by indicating what to look for.

l It should be kept current as we learn.

l A discussion of a sample checklist is given on Page 54.


25/56


Structural Testing

l Done by running the executable on the machine

l Entails running the product against some predefined test cases and

comparing the results against the expected results

l Designing test cases and test data to exercise certain portions of

code

l Types of structural testing

Unit / code functional testing

Code coverage testing

Code complexity testing


26/56


Unit / Code Functional Testing

l Initial quick checks by developer

l Removes obvious errors

l Done before more expensive checks

l Building debug versions

l Running through an IDE

l Debugging or testing? (Who cares?!)


27/56


Code Coverage Testing

l Exercises different parts of code

l Maps parts of code to required functionality

l Finds out percentage of code covered by testing by adopting a

technique called instrumentation of code.

l Instrumentation rebuilds the product, linking it with libraries

provided by the tool vendors.

l Instrumented code can monitor what parts of code is covered

l Can help identify critical portions of code, executed often


28/56


Types of Code Coverage

l Statement coverage

l Path coverage

l Condition coverage

l Function coverage


29/56


Programming Constructs

l Sequential instructions

l Two-way decision statements (if then else)

l Multi-way decision statements (switch statements)

l Loops, like while/do, for, repeat/until, etc


30/56


Testing sequential instructions

l Generate test data to make the program enter the sequential

block, to make it go through the entire block (Is this valid?)

Asynchronous exceptions

Multiple entry points, in non-structured programming

l Statement coverage as a metric:

# of statements exercised / Total # of statements


31/56


Testing IF THEN ELSE

l Have data to test the THEN part

l Have data to test the ELSE part

Total = 0; /* set total to zero */

if (code == M) {

stmt1;

stmt2;

}

else

percent = value/Total*100; /* divide by zero */


32/56


Path Coverage

l Statement Coverage may not indicate true coverage

l Path Coverage provides better representation

Previous example: Having data go through the THEN part and

ELSE part gives only 50% coverage, irrespective of number of

statements in each path

l More detailed example of path coverage in next slide

l Path coverage = # of paths exercised / total # of paths in the

program


33/56


Path Coverage.. ExampleFig: flowchart for a date validation routine


34/56


Condition Coverage

l Further refinement of path coverage

l Makes sure each constituent condition in a Boolean expression

is covered

l Protects against compiler optimizations

l Condition coverage = # of conditions covered / # of conditions in the program


35/56


Function Coverage

l Finds how many functions are covered by test cases

l Higher level of abstraction; hence possible to achieve 100%

coverage

l More logical than the other types of coverage

l Can be derived from RTM

l Easy to prioritize as they are based on requirements

l Leads more naturally to black box tests

l Can also be a natural predecessor to performance testing


36/56


Other Uses of Code Coverage Methods

l Performance analysis and optimization

l Resource usage analysis

l Checking of critical sections

l Identifying memory leaks

l Dynamically generated code (e.g., dynamic generation of

URLs and security checking)


37/56


Code Complexity Testing

l Which of the paths are independent? (If paths are not

independent, we may be able to minimize the number of tests.)

l Is there an upper bound on the number of tests to be executed to

ensure that all the statements have been executed at least once?

l Cyclomatic complexity provides answers to some of these

questions.


38/56


Intuitive Meaning of Cyclomatic Complexity

l Provides an indication of how many independent paths

are there in a program

l What gives birth to new paths?

l How can one count the new paths that are born?

Steps in Determining


39/56



Cyclomatic Complexity

l Start with a conventional flow chart.

l Identify the decision points.

l Convert complex predicates to simple predicates (see

next slide).


40/56




41/56



Cyclomatic Complexity.. contd

l If there are loops, convert the loop conditions to simple predicates.

l Combine all the sequential statements into a single node in the flow

graph.

l When a set of sequential statements are followed by a simple predicate,

combine all the sequential statements and the predicate check into one

node and have two edges emanating out of this node. Nodes with two

emanating edges are called predicate nodes.

l Make sure that all edges terminate in some node, adding a node to

represent all the statements at the end of the program.


42/56


Conventional Flow ChartFlow Diagram for

Calculating Complexity

Formal and Intuitive Interpretation of Cyclomatic


43/56


Formal and Intuitive Interpretation of Cyclomatic

Complexity

l Formal definitions

CC = # of predicate nodes + 1

CC = E N + 2 (E = edges, N = nodes)

l Intuitive definitions

When a condition is introduced, adds a new path

See the next two slides


44/56


End

Case (i): A hypothetical program with no decision node

#of independent paths = 1 #of nodes, N = 2; #of edges, E = 1; Cyclomatic complexity = E N + 2 = 1 #of predicate nodes, P = 0; Cyclomatic complexity = P + 1 = 1


45/56


End

Case (ii): Adding one decision node

#of independent paths = 2 #of nodes, N = 4;

#of edges, E = 4; Cyclomatic complexity = E N + 2 = 2 #of predicate nodes, P = 1; Cyclomatic complexity = P + 1 = 2


46/56


Independent Paths and Basis Sets

l An independent path is a path in the flow graph that has

not been traversed before in other paths.

l A set of independent paths that cover all the edges iscalled a basis set.

l Find the basis set and write the test cases to execute all

the paths in the basis set.

Meaning and Interpretation of


47/56


Meaning and Interpretation of

Cyclomatic Complexity

Complexity What it means

1-10Well-written code, testability is high, cost / effort to

maintain is low

10-20Moderately complex, testability is medium, cost / effort

to maintain is medium

20-40 Very complex, testability is low, cost / effort to maintainis high

>40Not testable, any amount of money / effort to maintain

may not be enough


48/56


Challenges in White Box Testing

l Developers not finding time for testing

l They develop blind spots for their own defects

l Fully covered code may not correspond to realistic

scenarios


49/56


Objectives (Recap)

l To review the V Model and set the context for different types of

tests

l To focus on white box testing

l To distinguish between different types of white box testing

l To place special emphasis on

Code reviews / inspections

Methods of code coverage and relationship to quality

Code complexity


50/56


Appendix

EXAMPLES


51/56


52/56


53/56


Constructing the Logic Flow Diagram

Start

2

3

4 5

6

7

8 9

Exit

1

F

T F

T F

T


54/56


Finding the Test Cases

Start

2

3

4 5

6

7

8 9

Exit

1

b

d e

gf

i j

h

c

k l

a (Covered by any data)

(Data set must

(Data set must contain at least one value)

be empty)

(Total score > 0.0)(Total score < 0.0)

(Positive score) (Negative score)

(Reached if either f or

e is reached)

Self reading


55/56


Unit-testing Heuristics1. Create unit tests as soon as object

design is completed: Black-box test: Test the use

cases & functional model

White-box test: Test thedynamic model

Data-structure test: Test theobject model

2. Develop the test cases

Goal: Find the minimal numberof test cases to cover as manypaths as possible

3. Cross-check the test cases toeliminate duplicates

Don't waste your time!

4. Desk check your source code

Reduces testing time5. Create a test harness

Test drivers and test stubsare needed for integrationtesting

6. Describe the test oracle

Often the result of the firstsuccessfully executed test

7. Execute the test cases

Dont forget regressiontesting

Re-execute test cases everytime a change is made.

8. Compare the results of the testwith the test oracle

Automate as much as

possible

Big cost -> what should be done?


56/56

THANKS!