Basic concepts and unit testing - 219343 Software Testing

Outline Basic concepts Testing principles Unit testing

Basic concepts and unit testing219343 Software Testing

Jittat Fakcharoenphol

Kasetsart University

November 7, 2008


Outline

1 Basic concepts

2 Testing principles

3 Unit testing


Testing1

Testing in not sexy.

When projects fail, testing is the reason.

In Europe in ancient times bearer of bad news got killedFollowing legends, even bearer of good news died ...

1Most of the materials on basic concepts are taken from slides developed forthe course in 2007 mostly by Uwe Guhl. (See the course homepage for thatsemester.) Thanks, Uwe!


Testing1






Testing1



In Europe in ancient times bearer of bad news got killed

Following legends, even bearer of good news died ...



Testing1






Definitions

”Testing is the process of establishing confidence that aprogram or system does what it is supposed to.” (Hetzel,1973)

”Testing is demonstrating that a system is fit for purpose.”(Evans et al. 1996)

”Testing is the process of executing a program or system withthe intent of finding errors.” (Myers, 1979)

”Testing is the process consisting of all life cycle activitiesconcerned with checking software and software-related workproducts.” (Gelperin and Hetzel, 1988)


Definitions






Definitions






Definitions






Statements

“Program testing can be used to show the presence of bugs,but never to show their absence!” (Dijkstra 1969)

“In most cases ‘what you test in a system is much moreimportant than ‘how much you test” (Craig 2002)

“Prioritise tests so that, when ever you stop testing, you havedone the best testing in the time available.” (ISEB testingfoundation course material 2003)


Statements





Statements





Goals

How many testers does it take to change a light bulb?

None.

Testers just noticed that the room was dark. Testers dont fixthe problems, they just find them.


Goals


None.



Goals


None.



Goals

Goals can vary. Define your mission.

Find important bugs fast.


Specific goals

Goal of Testing is to establish the ground for the acceptance of thesoftware by the customer based on the specification through

1 High test coverage

2 Low number of non critical defects There should be no criticaldefect

3 Statements concerning software quality


Specific goals



2 Low number of non critical defects

There should be no criticaldefect



Specific goals






Specific goals






Goals: high test coverage

Completeness. Ensure all requirements are implemented.

Total scope must be tested at least once (of course hopefullysuccessful)

Critical scope. Ensure that critical requirements areimplemented and work fine.All high prioritized requirements must be tested successfully



Completeness. Ensure all requirements are implemented.Total scope must be tested at least once (of course hopefullysuccessful)

Critical scope. Ensure that critical requirements areimplemented and work fine.

All high prioritized requirements must be tested successfully



Completeness. Ensure all requirements are implemented.Total scope must be tested at least once (of course hopefullysuccessful)

Critical scope. Ensure that critical requirements areimplemented and work fine.All high prioritized requirements must be tested successfully


Goals: how number of non critical defects

At the end the final version of the application:

should have no critical defects any moreshould have only a small number of tolerable defects

Demand on Testing is therefore to detect as much criticaldefects as soon as possible idea is to fix them during theTesting phase

The acceptance criteria should determine what the customerexpects. A contract should contain acceptance criteriaconcerning severity level and priority level of bugs.














Goals: Statements concerning software quality

Is it possible to install the software?

Is it possible to operate the software, is it compatible?

Fulfils the software the expected functionality?

Do the interfaces work?

Is it possible to use the software optimal(Software-ergonomics, usability, end user needs)

Does the software run steadily, with high performance, failproof?

Fulfils the software special cultural features (Multilingualism,English / metric system, weight units)?

Is the software safe / secure?


Tools/processes needed

Defect management (issue trackers)

Release management



Defect management

(issue trackers)

Release management




Release management




Release management


Testing principles2

Principles provide a general guideline.

2Materials from Graham, van Veenendall, Evans, and Black, Fundamentalsof Software Testing: ISTQB Certification, Thomson, 2007.


Testing shows presence of defect


Exhaustive testing is impossible


Early testing


Example: The Triangle Test3

Given the lengths of three sides of a triangle, determine thetype of that triangle.

EquilateralIsoscelesScalene

3Taken from Black, Pragmatic Software Testing, Wiley, 2007


The code that can be tested

Consider this code:

package lect02;

public class Triangle1 {

static void checkType(int a, int b, int c) {if((a==b) && (b==c))

System.out.println("Equilateral");

else if((a==b) ||(b==c) ||(a==c))System.out.println("Isosceles");

elseSystem.out.println("Scalene");

}

public static void main(String [] argv) {int a,b,c;

// .... read inputcheckType(a,b,c);

}}

How can we test thisprogram, in particularmethod checkType?

Automatically? Veryvery difficult.Because?

It is hard to check theoutput of the method.



Consider this code:

package lect02;






}



}}






Consider this code:

package lect02;






}



}}


Automatically?

Veryvery difficult.Because?




Consider this code:

package lect02;






}



}}






Consider this code:

package lect02;






}



}}





After some fix

package lect02;

public class Triangle {

enum Type { Equilateral, Isosceles, Scalene }

static Type checkType(int a, int b, int c) {if((a==b) && (b==c))

return Type.Equilateral;else if((a==b) ||(b==c) ||(a==c))return Type.Isosceles;

elsereturn Type.Scalene;

}}


Practice

Find as many interesting test cases for this method.Write in this form:

Tester action and data Expected result

We will use this example to talk about good test set again.


Practice

Find as many interesting test cases for this method.Write in this form:

Tester action and data Expected result

We will use this example to talk about good test set again.


Unit testing: classic example4

John: John works hard. He codes everyday. The project deadline is tomorrow.

He types in about two hundred new lines per hour, and thinks that after 6 hours

and roughly a thousand new lines added the program would work flawlessly.

Betty: Betty works hard. She codes everyday. The project deadline is

tomorrow. She types in about one hundred new lines per hour, and keeps

testing each method she adds. She does not proceed to write new codes unless

all previously written pieces work correctly.

Guess who will go to bed earlier?

4Materials regarding unit testing concepts are from Alberto Savoia’s slidesand George Necular’s software engineering course
























Developer Testing Revolution

Developer testing is a key component in a hot paradigm:Agile/eXtreme Programming.

The Developer Testing Trinity:

TestTest early and oftenTest well


Developer Testing Revolution

Developer testing is a key component in a hot paradigm:Agile/eXtreme Programming.

The Developer Testing Trinity:

TestTest early and oftenTest well


Good reasons for developer testing

Reduces unit-level bugs

Forces you to slow down and think

Improves design

Makes development faster

Tests are good documentation

Tests constrain features

Tests allows safe refactoring and reduce the cost of change

Tests defend against other programmers

Tests reduce fear





Improves design






Tests reduce fear





Improves design






Tests reduce fear





Improves design






Tests reduce fear





Improves design






Tests reduce fear





Improves design






Tests reduce fear





Improves design






Tests reduce fear





Improves design






Tests reduce fear





Improves design






Tests reduce fear


Goals

Does the code do what I want?

Does the code do what I want all the time?

Can I depend on it?

Also: get a document for the code.Plus: Always correct documentation for your intention.


Goals



Can I depend on it?



Goals



Can I depend on it?



Goals



Can I depend on it?

Also: get a document for the code.

Plus: Always correct documentation for your intention.


Goals



Can I depend on it?



Test your code

It is your code, and your responsibility

Do it for your current colleagues

Do it for future generation of colleagues

Do it for yourself


Test early and often


Test early and often


Heaven!

Every class has unit tests

The tests are executed many times each day

The tests are thorough, up to date, and easy to maintain andanalyze

In this class, we shall aim for that.


Heaven!






Heaven!






Heaven!






Heaven!






A unit

What’s a unit?

A single method/function/procedureA collection of related methods/functions/procedues

Ideal world: independent, self-sufficient, standalone

Real world: lots of dependence


A unit

What’s a unit?


Ideal world:

independent, self-sufficient, standalone



A unit

What’s a unit?





A unit

What’s a unit?



Real world:

lots of dependence


A unit

What’s a unit?





Basic structure

Setup

Create initial statesInitialize method parametersStore pre-execute values

Execute code

Compare results


Basic structure

Setup


Execute code

Compare results


Basic structure

Setup


Execute code

Compare results


Practice: Triangle

Write junit test cases for Triangle.


Discussion: test cases for Triangle

What are your test cases?


Test-driven development

Traditional steps:design, code, test, design, code, test, ...

TDD:test, code, refactor, test, code, refactor, ...

We’ll discuss more about TDD later. For now, let’s do it alittle.










We’ll discuss more about TDD later.

For now, let’s do it alittle.







TDD: Triangle

See demo.


Practice: Median

Documents

Basic concepts and unit testing - 219343 Software Testing