Object-Oriented Software Testing

Object-Oriented Software Testing

ICS 224AJoachim Feise

[email protected]

June 4, 1997 OO Software Testing 2

A Brief Meditation on the Essential Nature of Software Testing

Thus spoke the master: “Any program, no matter how small, contains bugs.”The novice did not believe the master’s words. “What if the program were so small that itperformed a single function?” he asked.

“Such a program would have no meaning,” said the master, “but if such a one existed, theoperating system would fail eventually, producing a bug.”But the novice was not satisfied. “What if the operating system did not fail?” he asked.

“There is no operating system that does not fail,” said the master, “but if such a one existed,the hardware would fail eventually, producing a bug.”The novice still was not satisfied. “What if the hardware did not fail?” he asked.

The master gave a great sigh. “There is no hardware that does not fail,” he said, “but if sucha one existed, the user would want the program to do something different, and this too is abug.A program without bugs would be an absurdity, a nonesuch. If there were a program withoutany bugs then the world would cease to exist.”

Geoffrey James, The Zen of Programming


Object-Oriented Testing Papers

• Object-Oriented Integration Testing– Jorgensen and Erickson– CACM Sept. 1994

• Object-Oriented Software Testing - A Hierarchical Approach– Shel Siegel– J. Wiley and Sons, 1996


Structure vs. Behavior

• most popular notations in software development portray software structure

• examples: E/R models, data flow diagrams, structure charts, etc.

– used to show relationships between components, interfaces, control and dataflow, etc.

– not very useful for testing

• SW testing is fundamentally concerned with behavior, i.e. what it does


Traditional Testing Notions

• Traditional software is– written in an imperative language– described by functional decomposition– developed in a waterfall life cycle– separated into three levels of testing

• unit level, integration level, system level

• these points do often not apply directly to object-oriented software


Imperative vs. Declarative

• Imperative languages lend themselves to a description as a directed graph

• declarative languages suppress sequentiality

• the event-driven nature of OO systems forces a “declarative spirit” on testing– not evident at unit level since most object-

oriented languages are imperative– but pronounced at integration and system levels


Unit Testing Overview

• Units can be defined in several ways• a single, cohesive function

• a function which, when coded fits on one page

• the smallest separately compilable segment of code

• the amount of code that can be written in 4 to 40 hours

• a task in a work breakdown structure

• code that is assigned to one person

• code that one person designs, codes and test in a three-month period


Unit Testing Overview (cont.)

• Many organizations have not chosen their definition of a software unit

• Units are always tested “by itself”– stubs and drivers are used to emulate inputs

and outputs– goal: verification that, taken by itself, the unit

functions correctly


Integration Testing Overview

• Once the units are separately tested, they are integrated together

• gradual replacement of stubs and drivers by separately tested units

• pairwise integration, i.e. each unit is integrated with its adjacent units

• bottom-up integration guided by functional decomposition tree

• “big bang” integration where all units are thrown together at once


System Testing Overview

• Conducted exclusively in terms of inputs and outputs that are visible at the port boundary of a system

• Tester only has access to those port events that are visible to the customer/user

• Can occur on the target platform only


System Testing Overview (cont.)

• Examples of system testing activities• sequences of machine instructions

• sequences of source instructions

• scenarios of normal usage

• system-level test cases

• stimulus/response pairs

• behavior that results from sequences of system-level inputs

• sequences of transitions in a state machine description of the system


Object-Oriented Software

– within an object, individual methods are imperative– declarative aspect because of event-driven nature– dynamic binding creates indefiniteness that

resembles declarative programs– it is impossible to ever know the full set of

“adjacent” objects with which a given object may be composed

• two object may be correct, but errors might result when they are composed


Composing Errors Illustrated


Levels of OO Testing

• Unit testing– method testing

• Integration testing– Message quiescence– Event quiescence

• System testing– Thread testing– Thread interaction testing


Integration Testing

• Method executions are linked by messages

• Quiescence provides natural endpoints

• Message quiescence– Method/Message path (MM-Path)

• sequence of message executions linked by messages• starts with a method• ends when method is reached that does not issue any

messages of its own– endpoint is called message quiescence


Integration Testing (cont.)

• Atomic System Function (ASF)• input port event,

• followed by a set of MM-Paths,

• terminated by output port event

– ASFs constitute the point at which integration and system testing meet

– elemental function visible at the system level


MM-Paths and ASFs Illustrated


ATM System Class Hierarchy


Example: ATM System

• Examples of– MM-Path: Digit entry– ASF: PIN Entry– Thread (sequence of ASFs):

• simple transaction: PIN entry, transaction type selection, account detail presentation, operation execution, result reporting

– Thread sequence:• a session consisting of multiple simple transactions


Digit Entry MM-Path Illustrated


PIN Entry ASF Illustrated


Observations

• Many-to-many relationships dominate (see E/R diagram)– an object may be involved in

many threads

– threads entail many objects

– an object may be involved with many ASFs

– ASFs may entail many objects


Observations (cont.)

• The five levels of OO testing result in distinct, useful testing goals

• Bottom-up testing order

• Seamless juncture between OO integration testing and system testing

• Constructs are directly usable as basis for test coverage metrics

• Works well with composition


The Hierarchical Approach

• Everything is a system

• Defines and applies testing standards for several levels of software components– objects– classes– foundation components– systems


Hierarchical Approach Illustrated


Safe Components

• Components that meet the testing standards for that kind of components

• only safe components can be integrated with other safe components to produce the next-level component

• Safe is a relative state– depends on risk management practices, attitude

toward risk, what is enforced, the application, etc.


Foundation Components

• Can be one complete class hierarchy

• or some cluster of classes– that performs a core function,– that represents a logical or physical

architectural component, etc.

• After testing to a safe level, can be integrated with other foundation components


Integration Testing of Foundation Components

• Needs to address the interconnections between the foundation components and any new composite functionality ONLY

• Eliminates the need to test all of the combinations of states during integration testing

• Improves productivity


Integration Testing Problem Illustrated


Test Suites

• Conditional Test Suite

• Hierarchical Incremental Test Suite

• Integration Test Suite

• System Test Suite

• Regression Test Suite


Conditional Test Suite

• Four types of test scripts– Assertions: abort if condition fails– Exceptions: raise exception if condition succeeds– Concurrent test operations: monitor condition– Message polling: evaluate condition on demand

• Conditions– class invariant, precondition, postcondition, loop

invariant


Pre/Postconditions Illustratedclass STACK2 [T] export

push, pop, top, empty, nb_elements, fullfeature

implementation: ARRAY [T]max_size: INTEGER;nb_elements: INTEGER;

Create (n: INTEGER) is-- Allocate stack for a maximum of n elements-- (or for no elements if n < 0)

doif n > 0 then max_size := n end;implementation.Create (1, max_size)

end; -- Create

empty: BOOLEAN is-- is stack empty?

doResult := (nb_elements = 0)

end; -- empty

full: BOOLEAN is-- Is stack full?

doResult := (nb_elements = max_size)

end; -- empty

pop is-- Remove top element

requirenot empty – i.e. nb_elements > 0

donb_elements := nb_elements – 1

ensurenot full;nb_elements = old nb_elements – 1

end; -- pop

top: T is-- Top element

requirenot empty – i.e. nb_elements > 0

doResult := implementation.entry (nb_elements)

end; -- top

push (x: T) is-- Add x on top

requirenot full

-- i.e. nb_elements < array_size in this-- representation

donb_elements := nb_elements + 1;implementation.enter (nb_elements, x)

ensurenot empty;top = x;nb_elements = old nb_elements + 1

end; -- pushend – class STACK2


Hierarchical Incremental Test Suite

• Structural models– control-flow model

• models class as series of control-flow graphs deriving from the method implementations

– data-flow model• models class as series of control-flow graphs with

data annotations showing the flow of data values through the method implementations


Hierarchical Incremental Test Suite (cont.)

• Functional models– state-transition model, aka OMT dynamic model

• models states and transitions for a class in the context of its position in the inheritance hierarchy

– transaction-flow model• models object life cycle as transaction from construction to

destruction

– exception model• specifies what methods raise what exceptions under what

conditions


Integration Test Suite

• Functional Test Suite– tests vertical integration of a cluster of

components, i.e. the way the cluster implements specific functions from the requirements

• Abstraction Test Suite– tests horizontal integration of a cluster of

components, i.e. the way the cluster abstracts a service contract into an API


Integration Test Suite (cont.)

• Possible test scripts– class-to-class test script

• interaction between classes based on message relationship

• protocol-interface, synchronization, exception handling tests

– UI test script• tests UI components of cluster and the components on

which they depend

• true-time, character-recognition, widget-playback tests


System Test Suite

• System Acceptance Test Suite– derived from the set of use cases covering the

system requirements

• User Options Test Suite– based on probable usage patterns of the features

and functions delivered with the system– designed to simulate the way the end users will

combine the features and functions


Regression Test Suite

• Series of tests run against a previous version of the CUT

• Almost always automated– performing manual tests is usually not cost

effective


Epilog

“Testing proves the presence, not the absence, of bugs.”

E.W. Dijkstra

“Absence of evidence is not evidence of absence.”

Source unknown

From: E.V. Berard, Issues in the Testing of OO Software

Slides available at http://www.ics.uci.edu/~jfeise/ICS224A

Documents

Object-Oriented Software Testing