White Box Testing

Len Schroath

September 18, 2007

Agenda

• White-box vs Black-box

• Program Flow Controls

• White-box Test Methods

• Exercises

• Complexity

• Q&A

What is White-box Testing?

• Looking at the internal structure of a program and deriving test cases based on the logic or control flow.

• Test cases can be designed to reach every branch in the code and to exercise each condition

• Typically done during unit testing

• Also known as:– Structural Testing – Glass-Box Testing

What is Black-box Testing?

• Looking at the program from an external point of view and deriving test cases based on the specification.

• The only criteria upon which the program is judged is if it produces the correct output for a given input.

Why Do Both?

Black-box• Impossible to write a test

case for every possible set of inputs and outputs

• Some of the code may not be reachable without extraordinary measures

• Specifications are not always complete

White-box• Does not address the

question of whether or not the program matches the specification

• Does not tell you if all of the functionality has been implemented

• Does not discover missing program logic

Basic Program Flow Controls

• IF

• IF-Then-Else

• FOR

• While

• Do-While

• Case

IF Diagram

IF-THEN-ELSE Diagram

FOR or WHILE Diagram

DO-WHILE Diagram

CASE Diagram

Example Code Fragment

Example Control Flow Graph

Source: The Art of Software Testing – Glenford Myers

Exercise #1

• int main (int argc, char *argv[])• {• /* Process CTRL-C Interrupts */• signal(SIGINT,catcher);• if (validate_command_line(argc)) return(1);• if (job_initialize(argv)) return(1);• processTestList(); /* Process all testCases in

TestList */• displayResults();• fprintf(stdout,"Test Complete\n");• job_termination();• return(0);• } /* main */ Can you diagram

this code?

Did You Get Something Like This?

Exercise #2/* Attempt Statusreadback - log SRB data to logFile */int process_srb(void){

int srb_count = 0;do{

srb_count = read_printer(srb_in);} while (!srb_count); fprintf(logFile,"%s\n",srb_in); return(srb_count);

} /* process_srb */

/* Write String to Printer via Parallel or Serial port */void write_printer (char *outputline){

if (strstr(printertype,"PAR") != NULL){

bwrite_parST(printerport,outputline);}else if (strstr(printertype,"SER") != NULL)

{bwwrite_serial(printerport,outputline,strlen(outputline));

}else if (strstr(printertype, "FILE") !=NULL){

fprintf(printerFile,"%s",outputline);}

} /* write_printer */

Can you diagram this code?

Did You Get Something Like This?

White-box Test Methods

• Statement Coverage

• Decision/Branch Coverage

• Condition Coverage

• Decision/Condition Coverage

• Path Coverage

Example Code Fragment

• If ((A>1) & (B=0)) then Do;

• X=X/A;

• END;• If ((A==2) | (X>1))

then Do;• X=X+1;• END;• END;

Source: The Art of Software Testing – Glenford Myers

Statement Coverage

• Exercise all statements at least once

• How many test cases?A=2 and B=0 (ace)

Source: The Art of Software Testing – Glenford Myers

Decision/Branch Coverage

• Each decision has a true and a false outcome at least once

• How many test cases?A=2 and B=0 (ace)A=1 and X=1 (abd)

Source: The Art of Software Testing – Glenford Myers

Condition Coverage• Each condition in a

decision takes on all possible outcomes at least once

• Conditions: A>1, B=0, A=2, X>1

• How many test cases?A=2, B=0, and X=4 (ace)A=1, B=1, and X=1 (abd)

Source: The Art of Software Testing – Glenford Myers

Decision/Condition Coverage

• Each condition in a decision takes on all possible outcomes at least once, and each decision takes on all possible outcomes at least once

• How many test cases? A=2, B=0, and X=4 (ace) A=1, B=1, and X=1 (abd)

• What about these? A=1, B=0, and X=3 A=2, B=1, and X=1

(abe)

(abe)

Multiple Condition Coverage

• Exercise all possible combinations of condition outcomes in each decision

• Conditions:

Source: The Art of Software Testing – Glenford Myers

A>1, B=0A>1, B<>0A<=1, B=0A<=1, B<>0

A=2, X>1A=2, X<=1A<>2, X>1A<>2, X<=1

Multiple Condition Coverage

• How many test cases?A=2, B=0, X=4A=2, B=1, X=1A=1, B=0, X=2A=1, B=1, X=1

Source: The Art of Software Testing – Glenford Myers

(ace)

(abe)

(abe)(abd)

Path Coverage

• Every unique path through the program is executed at least once

• How many test cases?A=2, B=0, X=4 (ace)A=2, B=1, X=1 (abe)A=3, B=0, X=1 (acd)A=1, B=1, X=1 (abd)

Source: The Art of Software Testing – Glenford Myers

McCabe’s Cyclomatic Complexity

• Software metric

• Developed by Tom McCabe (circa 1976)

• Directly measures the number of linearly independent paths through a program’s source code, taking into account the various decision points

• Independent of implementation language

Source: Wikipedia

Calculating Complexity

Calculating Complexity

How Complex Should Code Be?

• <10: Simple module, not much risk

• 10-20: More Complex; moderate risk

• 20-50: Complex; high risk

• >50: Untestable; extremely high risk

Source: Carnegie Mellon Software Engineering Institute

Complexity Caveats

• As code is broken into smaller modules to decrease cyclomatic complexity, structural complexity increases

• Some modules may have high complexity but are very easy to comprehend and easy to test

• High complexity numbers are only an indicator of something to investigate

Questions?