Upload
thane-robbins
View
27
Download
4
Embed Size (px)
DESCRIPTION
“White box” or “glass box” tests. “White Box” (or “Glass Box”) Tests. developer. client. user. Test planning. White box testing--”lowest level”: Part of a comprehensive test plan—see fig. 11.1 in text. Basic goal of white box testing. Unit tests. User test activities. Client test - PowerPoint PPT Presentation
Citation preview
1
“White box” or “glass box” tests
“White Box” (or “Glass Box”) Tests
2
Basic goal of white box testing
White box testing--”lowest level”:Part of a comprehensive test plan—see fig. 11.1 in text
Test planning
developer userclient
Unit tests
Integration tests
Structure tests
Functional tests
Performance tests
User test
activities
Clienttest
activities
3
Path testing
General method 1: Path testing or basis path testing
basic goal: make sure all paths in the control structure of a unit are tested at least once
why?
1. "bad code" is more likely in rarely executed paths
2. what we think is a "rare" path may turn out to be taken a lot
3. syntax errors can occur anywhere
4
White box testing--example
example:integer a,b,count=0;input a,b;if (a == 0) while (b > 0) {b = b-1; count++;}else if (a > 0) while (b < 0) {b = b+1; count--;}else a = b;output a,b,count;
What (minimal) set of tests would make sure that all paths are tested in this code?
Are there other test cases that would be advisable?
5
White box testing requirements
what needs to be tested:
1. all independent paths must be followed at least once
2. all logical decisions must have both true and false input
3. all loops must be executed at boundaries and within their bounds
4. all internal data structures must be used
6
Basis path method
This method guarantees that every statement will be executed at least once
Flow graph: • node represents one or more statements• edges represent control flow, as in flowcharts
Example: flow graphs for control structures:
Sequential
If
Until
While
Case
7
Basis path method (continued)
Determine number of tests needed:
a.develop a "flow graph" G--make graph for basic control structure--sequential statements with no branches in or out can be
merged into one statement
b.compute the "cyclotomic complexity”: V(G) = E - N + 2, where E = #edges, N = #nodes
or V(G) = number of planar regions defined by the graph
V(G) is an upper bound on the number of "independent" paths through the code which must be followed through an appropriate choice of test cases in order to execute every statement at least once
NOTE: this gives the number of paths for the GRAPH; program structure may mean a smaller number of paths is actually needed
8
Basis path method--example
Example:integer a,b,count=0; //1input a,b; //2if (a == 0) //3 while (b > 0) //4 {b = b-1; //5 count++;} //6else if (a > 0) //7 while (b < 0) //8 {b = b+1; //9 count--;} //10else //11 a = b; //12output a,b,count; //13
Flow chart:1
2
3
4
5
6
7
8
9
10
11
12
13
11,12
1,2,3
5,6
4 7
9,10
8
13
Flow graph:E=11,N=8;E-N+2=55 paths
Tests:a b0 20 -11 -11 2-1 -1
* 1 edge,multiple paths
*
*
*
9
Basis path method—6 steps
1. Number lines of executable code:integer a,b,count=0; //1input a,b; //2if (a == 0) //3 while (b > 0) //4 {b = b-1; //5 count++;} //6else if (a > 0) //7 while (b < 0) //8 {b = b+1; //9 count--;} //10else //11 a = b; //12output a,b,count; //13
2. Construct flow chart:
1
2
3
4
5
6
7
8
9
10
11
12
137
9,10
8
3. Construct flow graph:
4. Compute upper bound on number of paths: E=11,N=8; E-N+2=5;5 paths
5. Determine tests:
a 0 0 1 1 -1 b 2 -1 -1 2 -1
11,12
1,2,3
5,6
4
136. Add additional tests (e.g., let b=0)
Note; treat a function call (including a recursive call) as ONE statement, don’t expand
10
Basis path method--augmentations
Basis path method can also be used with a “graph matrix” to automate the generation of test cases (Beizer, Software Testing Techniques, 1990):
Example: use the flow graph to define a graph matrix:
Entries can be, for example: --Edges (1 or 0) --Probability of execution --Time to traverse an edge --Memory needed to traverse edge --Resources needed to traverse edge
Values can be used to determine what
test cases should be run
1
4
3
25
1 2 3 4 5
1 1
2 1 1 1
3 1
4
5 1 1
11
Data flow testing
General method 2: based on definition / redefinition of variables:
data flow testing: focuses on program variables x---e.g., require that every definition use chain (DU) be covered at least once
DU chains:for each program statement S
def(S)={x|x defined in S} and use(S)={x|x is used in S}
example: if S is the statement a = b + c then
def(S) = {a}; use(S) = {b,c}
x at S is live at S':there is a path from S to S' in which x is not redefined
12
Def{S}, use{S}
Example: b = 3; //S
c = 4; //S'a = b + c; //S''d = a + b; //S'''
def(S) = {b}; def(S') = {c}; def(S'') = {a}; def (S''') = {d}
use(S) = ; use(S') = ; use(S'')= {b,c}; use(S''')={a,b}
a at S'' is live at S'''; b at S is live at S',S'', and S''';c at S' is live at S'';
13
DU chains
definition use chain (DU) of x is [x,S,S'] where x is in def(S) and use(S') and def. of x in S is live at S'
so in this example:b = 3; //Sc = 4; //S'a = b + c; //S''d = a + b; //S'''
DU chains are:a: [a, S'', S'''] b: [b, S,S''] and [b,S,S'''] c: [c, S', S'']
testing: must cover every DU chain at least once
14
DU chains--example
another simple example:0. input j,k;1. x = j; y = k;2. if (x > 0)3. y = y + 2;4. x = x + 5;5. endif6. z = x + y + 4;
DU chains:1-[j,0,1]; 2-[k,0,1]; 3-[x,1,2],4-[x,1,4],5-[x,1,6],6-[x,4,6]7-[y,1,3],8-[y,1,6],9-[y,3,6]
test cases:j = 4, k = 5;--covers 1,2,3,4,5,7,8 and j = -4, k = 5;--covers 6,9
15
State-based testing
General method 3: state-based testing (text, p. 461):
Works well for object-oriented designs
Derives tests from component’s associated state machine diagram
Each transition in the diagram must be traversed at least once
16
Example: Finite-state machine (FSM) model—what state transitions must be tested?
Idle
GoingUp
req > floor
req < floor
!(req > floor)
!(timer < 10)
req < floor
DoorOpen
GoingDn
req > floor
u,d,o, t = 1,0,0,0
u,d,o,t = 0,0,1,0
u,d,o,t = 0,1,0,0
u,d,o,t = 0,0,1,1
u is up, d is down, o is open
req == floor
!(req<floor)
timer < 10
t is timer_start
UnitControl process using a state machine
From Vahid/Grivargis, Embedded System Design, 2000
State-based testing (continued)
17
Polymorphism testing
Polymorphism testing: Must be sure to identify and test all possible bindings, static and dynamic
Example: fig. 11-15
Network interface
UMTSWaveLANEthernet
18
Additional white box testing
some methods to broaden the test coverage:
condition testing: add tests for each value of a given condition (e.g., if (a == c and b == d) )
domain testing: for each relational expression (a relational-operator b) generate tests for cases: a < b, a = b, a > b
loop testing: simple loops (max # of iterations = n):skip it one passtwo passesm passes, m < nn-1, n, n+1 passes
(8 <= # passes <= 4n+8)
19
Additional white box testing (continued)
nested loops: tests can grow geometrically ( (n2))some reductions may be used (see p. 458, Pressman)
concatenated loops:if loops are not independent, should treat as nested loops
unstructured loops: avoid these in your design and code
Nested loops Concatenated loops
Unstructured loops (to be avoided)