Research seminar 1. Dr. S. Vilkomir, Research directions in software testing

Research seminar

1. Dr. S. Vilkomir, Research directions in software testing.

2. Brandi Amstutz, Combinatorial Approach to Testing Mobile Applications: An Experimental Investigation.

Monday, October 28 2013

Room 232 S&T Bld. 1

Research directions

in software testing

Dr. S. Vilkomir

1. Effectiveness of combinatorial testing for logical expressions

2. Effectiveness of MC/DC

3. Testing mobile applications on different devices

1. Effectiveness of combinatorial testing for logical expressions

Combinatorial testing:

• Each choice

• Base Choice

• Pair-wise

• t-way

Pair-wise testing

–Dest: London, Paris, Chicago, NY

–M: 1000, 2000, 3000

–Direct: Y, N

Dest M Direct

t1 L 1 Y

t2 L 2 N

t3 L 3 -

t4 P 1 -

t5 P 2 Y

t6 P 3 N

t7 C 1 N

t8 C 2 -

t9 C 3 Y

t10 N 1 Y

t11 N 2 N

t12 N 3 -

Pair-WisePair-Wise: A value of each level for each factor must be combined : A value of each level for each factor must be combined with a value from every levels for each other factor.with a value from every levels for each other factor.

Our results

• Fault Evaluator tool

http://core.ecu.edu/STRG/tools.html

• Four research papers:

• PSAM 10, Seattle, Washington, USA

• PIC-2010, Shanghai, China

• ICST 2012, Montreal, Canada

• ICSTW 2013,Luxembourg, Luxembourg

http://core.ecu.edu/STRG/publications.html

New book

Introduction to Combinatorial Testing

by

D. Richard Kuhn,

Raghu N. Kacker, and

Yu Lei

2. Effectiveness of MC/DC

• “Independent affect” is the main idea of Modified Condition/Decision Coverage (MC/DC)

• Suggested in 1992-1994 for avionic software

• DO-178B. Software Considerations in Airborne Systems and Equipment Certification. USA, 1992.

• Chilenski, J. and Miller, S. Software Engineering Journal, 1994

Modified Condition/Decision Coverage

• Every point of entry and exit in the program has been invoked at least once, every condition in a decision in the program has taken on all possible outcomes at least once, every decision in the program has taken all possible outcomes at least once, and each condition in a decision has been shown to affect the decision's outcome independently.

A condition is shown to affect a decision's outcome independently by varying just that condition while holding fixed all other possible conditions.

From DO-178B.

9

3

1

2

• Do these test cases satisfy MC/DC?– t1: <1, 0, 1> d= 1

– t2: <0, 1, 1> d= 1

– t3: <0, 1, 0> d= 0

– t4: <0, 0, 1> d= 0

• Holding conditions – OK• Affect the decision's outcome – OK• Answer: YES

d = A (B C)

Plans – to continue this research direction

3. Testing mobile applications on different devices

Practical questions

• No scientific approaches

• No justification

• Optimal set of devices?

• Our approach – next presentation