Coverage Criteria for Testing of Object Interactions in Sequence Diagrams

Atanas (Nasko) RountevScott KaganJason SawinOhio State University

UML Sequence Diagrams

Shows messages exchanged among objects

Contains control information relevant to message passing

a:A b:B c:C d:D

m1 m2[c1] m3

m4

[c3] m6 m2

[c2] m5

[c1] m3

m4

[c2] m5

Sequence Diagrams and Testing Object interaction testing Coverage criteria for sequence diagrams

all paths, all branches, etc. Most existing work utilizes the all paths

criterion

Sequence Diagrams and Testing Most existing work focuses on design time

diagrams Diagrams can also be reverse engineered

from code more paths due to less abstraction

Do existing sequence diagram based testing approaches generalize to reverse engineered diagrams? should the all paths criterion be considered?

Our Work

Investigated 4 different coverage criteria for reverse engineered sequence diagrams defined each criterion in terms of an RCFG tree developed a method for comparing criteria performed a comparison of the criteria on real

world Java code

RCFG Tree

Restricted Control Flow Graph (RCFG) similar to a traditional CFG for message passing only

RCFG tree represents a sequence diagram set of all start to end paths for both structures is the same

Root RCFG is top level method Edges connecting RCFGs represent method

invocations RCFG n is a child of RCFG m means method m may

invoke method n

m1

start

end

m2

m6

m5

end

m4

start m3

m5

end

m4

start m3

m6

start

end

m2

a:A d:Dc:Cb:B

[c1] m3

m2

m2

[c1] m3

m4

[c2] m5

m1

m4

[c2] m5

[c3] m6

m2m2

start

end

m4

RCFG Tree Details

Straightforward to construct when reverse-engineering sequence diagrams

Can be traversed at run time to collect coverage statistics

Coverage Criteria Considered All start to end paths All RCFG paths All RCFG branches All unique branches

m1

start

end

m2

m6

m5

end

m4

start m3

m5

end

m4

start m3

m6

start

end

m2

m2m2

Criterion Comparison Technique Each start to end path corresponds to one

test case For each criterion, calculate minimum

number of start to end paths required to achieve complete coverage indirect estimate of testing effort

Minimum Number of Paths

All start to end paths: 20 All RCFG paths: 5 All RCFG branches: 3 All unique

branches: 2

m1

start

end

m2

m6

m5

end

m4

start m3

m5

end

m4

start m3

m6

start

end

m2

m2m2

Calculating Minimum Number of Paths All start to end paths

calculate the number of paths from the start node of an RCFG to the end node of the RCFG must consider paths in children

start with the leaves, propagate up to the root All RCFG paths, all RCFG branches, all

unique branches integer linear programming

Valid Path Property If node n represents a method call and edge

(m, n) is part of a path p, p must include an inter-method edge (n, start) p must include an intra-method edge (n, r)

n

m

r

start

q

Valid Path Property

If node n represents a method call and edge (n, start) is part of a path p, p must include an intra-method edge (start, q)

n

m

r

start

q

RCFG Branches Property

Consider some set of paths S that satisfies the RCFG branch property every RCFG edge appears in at least one path

n

m

r

start

q

Integer Linear Programming Formulation ILP equations encode the valid path property

and the RCFG branches property variables are edge frequencies constraints

each frequency is at least 1 frequency sums are equal

(red, green, blue)n

m

r

start

q

Proof Obligation

Solution to the integer linear programming problem is a frequency count for each edge

Need to prove There exists a set of paths that meets the

coverage criterion and has the same edge frequencies

No smaller set of paths satisfies the criterion

Remaining Criteria

Integer linear programming formulation valid paths property a criterion specific property

All unique branches the sum of the edges frequencies for equivalent

edges is at least one All RCFG paths

each edge is covered at least as many times as its local edge frequency

Experimental Study

Implemented as part of the RED tool (Reverse Engineering of sequence Diagrams)

Implementation used lp_solve linear programming tool

Input code: code from the Java standard libraries 1104 RCFG trees average 36 RCFGs per tree

Minimum Number of Paths

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

1-5 6-10 11-100 101-1000 > 1000

Start to End Paths RCFG Paths RCFG Branches Unique Branches

Future Work

Implement dynamic analyses for obtaining coverage statistics

Measure coverage achieved by real world test suites