1 1. Implementation & integration 2. Testing - module (unit) testing - system testing 3. Metrics 4. A test tool : Junit 5. Documentation (tool : javadoc)

1

1. Implementation & integration2. Testing

- module (unit) testing- system testing

3. Metrics4. A test tool : Junit5. Documentation (tool : javadoc)

2

Programming languages

Selection criteria

• client preference/requirement• hardware to be used for installation• compatibility with and maintenance of legacy software• amount of training required• company strategy for future products• investment in personnel and tools• application itself

• artificial intelligence (rule based systems) -> LISP• high performance numerical calculations -> C, FORTRAN• network management software -> C++, JAVA• administrative software -> C++, JAVA

3

Coding standards

examples

• methods no longer than 35 lines, if longer authorization required• nesting of if-statements : not more than 3 levels• don’t use “goto”, “break”, …• naming conventions• layout conventions (layout tools exist to format code)• policy for source code comments

• to be checked automatically• compromise between ease of development and efficient maintenance

4

A

B C D

E

F

G

H

I

J

K

Module dependence

Subsystem

• what order to code ?• what order to integrate

“naïve” strategy• code modules A … K separately• compile everything• “run” module A

Problem

No fault isolation !!!

5

Module dependence

Fault isolation

Module A :Stubs for modules B, C, D needed

Module E :Drivers for mimic B needed

Code, compile and test modulesseparately !

Stubs and drivers thrown away

A

B C D

E

F

G

H

I

J

K

6

Stubs

A

B C D

class A {String name;String passWd; boolean valid=false; A(String n,String p) {name=n;passWd=p;} public boolean authenticateUser { B userName=new B(); C userPassWd=new C(); name=userName.input(); passWd=userPassWd.input(); } public void checkValidity() { D checker=new D(); valid = checker.isValid(name,passWd); }}

class B { public B(){}; public String input() {return "a name";}}class C { public C(){}; public String input() {return "a password";}}class D { public D(){}; public boolean isValid(String n, String p) {

return false; }}

More intelligent stub• reports which method is called• gives some result, mimicking test cases

7

Drivers class F {

public static double squareRoot(double d) {// algorithm to compute square rootreturn Math.sqrt(d)

}}

class E {public static void main(String[] args) {

for(int i=-10;i<10;i++) System.out.println(F.squareRoot(i));

}}

Try to incorporate test cases !

E

F

8

Strategies

Top-down strategy

A

B C D

E

F

G

H

I

J

K

t+

t

• logical design errors show up early• no drivers required• BUT : “low-level” modules not

tested thoroughly !

9

Strategies

Bottom-up strategy

A

B C D

E

F

G

H

I

J

K

t+

t

• logical design errors show up late• no stubs required• “low-level” modules tested thoroughly !

10

Strategies

Sandwich

A

B C D

E

F

G

H

I

J

K

combine benefits of “top-down”and “bottom-up”

Operational modules• low-level, designed for specific task• likely to be reused

Logic modules• control flow of information, algorithmic• problem specific, not likely to be reused

Try to minimize “throw-away code” !

11

Sorts of testing

“informal” testing • Performed by programmer• Not systematic

methodical testing • By SQA• After informal testing by programmer

Non-execution based testing

Execution based testing

Review by team of experts

Run test-cases

How to select test cases ?

12

Black-box testing

= test to specifications

• ignore the code itself• based on specification document• = input/output testing• after informal testing by programmer• problem : combinatorial explosion of test cases !

ModuleINPUT OUTPUT

Specificationdocument

Consistency

13

Black-box testing

Techniques • add test cases to detect NEW possible error• keep track of test cases for regression testing

• Partition input space into equivalence classes• Take (at least) 1 test case for each class• Test “boundary” values

Equivalence and boundary analysis

14

Black-box testing

2. Character input belonging to predefined set

Equivalence class 1 Character belongs to set

Equivalence class 2 Character does not belong to set

Examples

1. Numerical input in interval [a,b]

a b

Equivalence class 1 Equivalence class 2Equivalence class 3

Boundaryvalue 1

Boundaryvalue 2

15

Black-box testing

A product reads in two positive real numbers a and b, where a in [0,100] and b in ]10,200[. A third real number c is read, with c>0.In addition a command is read. There are 10 possible commands.

How many test cases are necessary for black box testing ?

16

White-box testing(glass box testing)

• Test every possible path through the module• Combinatorial problems !• Does not reveal all possible problems !

= test to code

4 possible pathsLoop n times

4n possible paths

Module

17

White-box testing(glass box testing)

Test-case selection

Statement coverage

Branch coverage

Path coverage

Every statement at least executed once

Every branch at least executed once

Advanced techniques to reduce paths

Support from CASE-tools

18

Control graph

• nodes are statements or groups of statements executed in sequence• edges : control transfer between statement groups

public static int f(int n) {

1: int a=0;

2: if(n%2==0) {

3: a=1;

} else {

4: a=2;

}

5: while(a<n) {

6: if(a%2==0) {

7: a=2*a+1;

}

else {

8: a*=2;

}

}

9: return a;

}

1

2

3 4

5

6

7 8

9

19

Linearly independent paths

A B

C D

Each path can be associated with vector over arcs

[nA nB nC nD]

with nX : number of times arc X is present in the path

Paths are linearly independent iff associated vectors linearly

independent

Base B = maximum set of linearly independent paths

Cyclomatic complexity (G) = CV(G) = #B(G)

= V(G) + 1

= #edges - #nodes + #components + 1

Here : (e=4, n=3, c=1) -> CV(G) = 3

B = {AC, BC, AD}

AC = [1 0 1 0]

BC = [0 1 1 0]

AD = [1 0 0 1]

BD = [0 1 0 1] = BC + AD - AC

20

Coverage techniques

• statement coverage= node coverage in control graph

• branch coverage= edge coverage in control graph

• multiple condition coverage= extended version of branch coverage

• cyclomatic coverage= limited set (base set) of path coverage

• all paths coverage = cover all paths in control graph

21

System testing

After successful integration

Testing of the complete product

COTS

• SQA tests product against specifications of the complete product• check for FUNCTIONALITY and CONSTRAINTS• use of predefined test cases (scenario’s)

• ship preliminary versions to prospective customers• alpha and beta version of product

22

System testing

Custom software

Acceptance testing• done by

• client• client + SQA group• independent SQA group

• what ?• Correctness• Robustness• Performance• Documentation

• done on actual (instead of test) data• if accepted …

23

Complexity metrics

• number of code lines (KLOC = 1000 lines of code)• cyclomatic complexity (= number of module branches)• 4-tuple :

(#distinct operands, #distinct operators,#total operands, #total operators)

Experience : These metrics yield similar results !

24

Fault statistics

• number of test cases• number of failed test cases• if (#failed test cases/#total test cases) > limit

-> redesign/recode• how long to test a product ???

Assumption : fault probability decreases exponentially with fault free testing time

h

goaltotal

goal

goal

goal

t

ff

f

f

f

estTimefaultfreeT

5.0ln

5.0ln • fgoal= targeted number of faults

• ftotal = number of faults detected

• th = total amount of test time up to last failure

25

Fault statistics

Example

• product may contain 1 errors per 10 KLOC• size = 100 KLOC

fgoal= 10

30 errors100 hours

• ftotal = 30

• th = 100

120 hours

36410034.1

88.4100

1030

105.0ln

105.010

ln

estTimefaultfreeT 344 hours to test !

26

JUnit = test framework for unit testing“Never in the field of software development was so much owed by somany to so few lines of code”. [M. Fowler]“The jewel on the crown of XP” (philosophy : code a bit – test a bit)[freely downloadable from http://www.junit.org/ ]

“Testrunner” :• runs test suites (= set of test cases, can be composed of other suites)• reports results (textual or graphical UI)

• test case gives FAILURES == produces the wrong result • wrong return value• not throwing expected exception

• test case gives ERRORS == throws unexpected exceptions• each TestCase has at least 1 call to assert-method

Assert.assertTrue(<actual boolean result>);

Assert.assertEquals(<actual result>,<expected result>);

Steps(1) define TestCases(2) add TestCases to Suites (static – dynamic)(3) run TestRunner

http://www.junit.org/

27

Datesclass Date {

public static final int[] s={31,29,31,30,31,30,31,31,30,31,30,31};

private int d,m,y;

public Date(int dd,int mm,int yy) {d=dd;m=mm;y=yy;}

public boolean isValid() {

if((y>1583)&&(m<13)&&(m>0)) {

boolean res=true;

if((d<0)||(d>s[m-1])) {res=false;}

else if((m==2)&&(d==29)&&!isLeap(y)) res=false;

return res;

} else return false;

}

public static boolean isLeap(int year) {return (year%4==0);}

public boolean before(Date date) {

if(y!=date.y) {return y<date.y;}

else {

if(m!=date.m) {return m<date.m;}

else return d<date.d;

}

}

}

28

Specifying test suites statically

import junit.framework.*;

class DateValidTest extends TestCase {

public void runTest() {

Date d1=new Date(10,10,2004);Date d2=new Date(20,20,2004);

Assert.assertTrue(d1.isValid());

Assert.assertTrue(!(d2.isValid()));

Assert.assertTrue(!(new Date(-1,20,2004).isValid()));

}

}

class DateBeforeTest extends TestCase {


Date d1=new Date(10,10,2004); Date d2=new Date(20,20,2004);

Assert.assertTrue(d1.before(d2));

Assert.assertFalse(d2.before(d1));

}

}

public class DateTest extends TestCase {

public static Test suite() {

TestSuite s=new TestSuite();

s.addTest(new DateValidTest());

s.addTest(new DateBeforeTest());

return s;

}

}

Inherit from TestCase

Override runTest() (public void runTest)

Define : public static Test suite()

D:\>java junit.textui.TestRunner DateTest

..

Time: 0,01

OK (2 tests)

29

Fixtures

class DateTestFixture extends TestCase {

Date d1,d2;

public void setUp() {

d1=new Date(10,10,2004);

d2=new Date(20,20,2004);

}

public void tearDown() { System.out.println("TEAR DOWN.");}

}

class DateValidTest extends DateTestFixture {





}

}

class DateBeforeTest extends DateTestFixture {




}

}

Template pattern : test case running calls

setUp()

runTest()

tearDown()

D:\>java junit.textui.TestRunner DateFixTest

.TEAR DOWN.

.TEAR DOWN.

Time: 0,02

OK (2 tests)

Purpose : run collection of test cases against fixed set of objects (avoid test case setup code)

30

Anonymous inner classes

class DateTestFixture extends TestCase {

Date d1,d2;

public DateTestFixture(String s) {super(s);}


d1=new Date(10,10,2004);

d2=new Date(20,20,2004);

}

public void tearDown() { }

}

public class DateInnerTest extends TestCase {



s.addTest(new DateTestFixture("Validity testing.") {

public void runTest() {Assert.assertTrue(d1.isValid());Assert.assertTrue(!(d2.isValid())); }

});

s.addTest(new DateTestFixture("Ordering testing.") {

public void runTest() {Assert.assertTrue(d1.before(d2));Assert.assertFalse(d2.before(d1));}

});

return s;

}

}

Purpose : run collection of test cases against fixed set of objects (avoid test case setup code)

31

One Test classpublic class DateStaticTest extends TestCase {

Date d1,d2;

public DateStaticTest(String s) {super(s);}


d1=new Date(10,10,2004);

d2=new Date(20,20,2004);

}



s.addTest(new DateStaticTest("Validity testing.") {





}

});

s.addTest(new DateStaticTest("Ordering testing."){




}

});

return s;

}

}

32

Testrunner …

33

Specifying test suites dynamically

Hand testing method name to TestCase constructor ! (uses reflection !)

public class DateDynamicTest extends TestCase {

Date d1,d2;

public DateDynamicTest(String s) {super(s);}


d1=new Date(10,10,2004);

d2=new Date(20,20,2004);

}

public void testValidity() {




}

public void testOrdering() {



}



s.addTest(new DateDynamicTest("testValidity"));

s.addTest(new DateDynamicTest("testOrdering"));

return s;

}

}

34

Specifying test suites dynamically

Naming convention : start all testing methods with “test” ! (uses reflection !)

public class DateDynamicTest extends TestCase {

Date d1,d2;

public DateDynamicTest(String s) {super(s);}


d1=new Date(10,10,2004);

d2=new Date(20,20,2004);

}

public void testValidity() {




}

public void testOrdering() {



}


TestSuite s=new TestSuite(DateDynamicTest.class);

return s;

}

}

35

Why ?

• basis for contract (specification)• basis for testing (walkthrough, inspection)• basis for next phase• essential for maintenance

• no such thing as “self-explanatory code” “self-documenting code”• use of comments to clarify

- responsibility of class- purpose of method (pre- & post-conditions)- clarify obscure looking lines

• Java : “javadoc”-tool

Documents

Documenting code

36

javadoc

• include documentation in source code• easy to maintain !• tool extracts documentation automatically

syntax /** documentation*/

• embedded HTML• doc tagsAppears just before

• class definition• method definition• variable definition

37

javadoc/** MyClass serves as a demonstration* for the use of javadoc */public class MyClass {

/** The private variable x codes the number of iterations */private int x;/** The variable y is a public copy of x */public int y;/** Constructor to initialize x */public MyClass(int i) {x=i;y=x;}/** Prints a given message repeatedly */public void printMessage(String s) {

for(int i=0;i<x;i++)System.out.println(s);

}/** Gives the value of x */public int giveX() {

return x;}

}

javadoc MyClass.java

38

javadoc

Default :only public and protected shown- package shows also package members- private all members included

39

Embedded HTML/** Prints a given message repeatedly* making use of the following loop :* <pre> for(int i=0;i<x;i++) System.out.println(s) </pre>* with loop characteristics :* <ul><li> initialization : counter i = 0* <li> loop condition : counter i < x* <li> update : counter i is incremented* </ul> */public void printMessage(String s) {

for(int i=0;i<x;i++)System.out.println(s);

}

40

Doc tags

Hyperlink to other item @see

@see classname@see MyClass

@see fully-qualified-classname@see java.lang.String

@ see fully-qualified-classname#method-name@see java.lang.String#indexOf

Can occur “everywhere”

41

Class documentation

• embedded HTML• @see• @version• @author• @since

At start of line

/** MyClass serves as a demonstration* for the use of javadoc * @version 1.2* @author George Bush jr.* @see java.lang.String#indexOf */public class MyClass {

// ...}

javadoc -version - author MyClass.java

42

Variable documentation

• embedded HTML• @see

public class MyClass {private int x;/** Variable y holds public copy of x* @see java.lang.String#indexOf */public int y;

}

43

Method documentation• embedded HTML• @see• @param parameter-name description• @return description• @throws fully-qualified-class-name description• @deprecated

public class MyClass { // ... /** writes to System.out a given message * @param s the message to be printed * @return the number of iterations */ public int printMessage(String s) { for(int i=0;i<x;i++)

System.out.println(s);return x;

} // …}

Causes compiler warning if used !