Software Testing Chapter 11: Reusable Components Nagy Akos – Nick Baetens – Kevin Buyl – Matthias De Cock - Dieter De Hen

Software TestingChapter 11: Reusable Components

Nagy Akos – Nick Baetens – Kevin Buyl – Matthias De Cock - Dieter De Hen

2

Reusable Components

• Testing and Reuse The Role Of Testing in Reuse Reusing Test Suites

• Test Design Patterns Abstract Class Test Generic Class Test New Framework Test Popular Framework Test

3

Reusable Components



4

Testing and Reuse

• Reusable packages A single specification Multiple implementations

• Similar high-level behavior• Different low-level behavior

Example: Queue• Bounded vs Unbounded• Sequential vs Concurrent• …

5

Testing and Reuse

• Importance of reuse Solution to software development problems

• High development time• High development cost• Frequent failures• Low maintainability• Low adaptability

Error Reduction Ratios (C++)• Design errors – 49:1• Coding errors – 26:1• Rework and debug time – 100:1

6

Testing and Reuse

• Importance of testing Leads to more reliable components Often improves component design Shows inelegant or difficult-to-use components Tangible evidence of dependability

7

Testing and Reuse

• Testing and reuse Reuse is not limited to just the code

• Producer test suites Rerun by the consumer Establishes reusability

• Consumer test suites Specifically tailored to an individual user Can be incorporated by the producer

8

Reusable Components



9

Class Instances

• Abstract Class Test abstract class instance a subclass providing implementation

• Generic Class Test generic class instantiation class that results if type is provided in a

declaration/definition

10

Class Instances




11

Abstract Class Test

• Intent Develop a test suite for an abstract class

• Context Tested by developing fully implemented subclass Test suite for the entire hierarchy

• Fault model Fault hazards

• Design errors• Simple coding errors

12

Abstract Class Test

• Strategy Subclass must be developed Specification-based tests for flattened subclass Test cases for each implemented method

(by using appropriate method and class scope test design patterns)

• Entry criteria May be tested when it is compiled

• Exit criteria Method scope branch coverage is the minimum

13

Abstract Class Test

• Consequences The test subclass and its test suite may reveal

design errors The test suite may be used as documentation for

the consumers of the class

14

Class Instances




15

Generic Class Test

• Intent Develop a test suite for a class that requires type

parameter• Context

To produce an object we need• Generic class• Type argument

Class behaviour depends on• Implementation • Type argument

Testing• Which/how many type arguments

16

Generic Class Test

• Fault model We distinguish two kinds of faults

• Type-dependant faults• Type-specific faults

Compiler accepts/rejects a given type or class

17

Generic Class Test

• Type-dependant faults Unintended interaction occurs between the

implementation and some accepted class (not uniform behaviour for all types)

Combination of accepted type arguments may cause faults

Examples• Range and precision of scalar types in arguments and

returned values• Use of static and class variables• ...

18

Generic Class Test

• Type-specific faults Unintended interaction occurs between the

implementation and some accepted class Some generic methods may be inconsistent with

special-case type argument Combination of accepted type arguments may

cause faults Support only a subset of accepted classes

(may indicate poor design)

19

Generic Class Test

• Strategy Single type parameter

• Test suite has 3 parts Type-dependent tests Type-specific tests Type-overloading tests

Multiple type parameters• Exercise combination of types• As many as time permits

Automation• Test driver as a generic class• Rerun test suite for substituted type parameters

20

Generic Class Test

• Entry criteria Accepted classes should pass their test suite

• Exit criteria Attain 100% branch coverage on every tested

instantiation of the CUT• Consequences

Passing for several type parameters → no guarantee to pass for other type instantiation

Release the test suite with the generic class

21

Reusable Components



22

Framework Testing

• New Framework Test Applies to initial and early versions Design a test suite and the test instantiation from

the generic requirements for a new framework

• Popular Framework Test Applies to a framework that becomes widely used Design a test suite and the test instantiation from

a framework that becomes widely used. Cover new features and perform compatibility

testing

23

Framework Testing

• New Framework Test Applies to initial and early versions Design a test suite and the test instantiation from


• Popular Framework Test Applies to a framework that becomes widely used Design a test suite and the test instantiation from


testing

24

Framework Testing

• Don’t need to choose• Emphasis in testing evolves• Availability of an application instantiation of

the FW and the most probable kind of bugs dictate the differences

New Framework Test Popular Framework Test

Widespread useMany application-specific

instantiations

1st ReleaseFew, if any, instantiations

25

To Keep In Mind …

• Extended Use Case: page 281 Use case: describes interactions Not sufficient for system test design Extra info: relative frequency, sequential

dependencies, domain of participating variables

• Extended Use Case Test: page 722 Develop an application system test suite by

modeling essential capabilities as extended use cases.

26

To Keep In Mind …

• Class Association Test: page 569 Design a test suite to verify the implementation of

required associations among classes

• Covered in CRUD: page 732 Verify that all basic operations are exercised for

each problem domain object in the SUT

• Test Oracle Produces the results expected for a test case

27

New FW Test: Context

• FW Development: Requirements model without antecedent

application Generalized from existing system

• FW responsibilities and the Implementation UT will be new

28

New FW Test: Fault Model

• General promises of a FW:

Complete and consistent representation of the

salient aspects of the problem domain as classes

and associations

Provide basic capabilities to create, read, update,

delete objects of these classes and the

associations among them

Enforce sequential control necessary for required

behavior

29

New FW Test: Fault Model

• Each individual component is at risk for many kinds of bugs

Design Omissions: essential behavior or

representation is missing

Representational integrity bugs: allow association

constraints to be broken

Control bugs: FW typically use inverted control

strategy

Infrastructure bugs: typically complex code

30

New FW Test: Test Model

• Test suite: FW analysis and design models• DemoApp:

Testable instantiation Should provide minimal implementation of each

use case identified for the entire FW• Test design:

Extended Use Case Test Class Association Test Control strategy: model as state machine and

develop test suite with N+ Coverage

31

New FW Test: Test Model

• What if there is predecessor system?

Use it as an oracle, figure 11.2

DemoApp must implement features (nearly) the

same as predecessor system

Salvage some of the test cases of PS

Test design from PS will provide some insights for

test design of DemoApp

32

New FW Test: Entry Criteria

1. Testable system scope model

2. All component classes and subsystems should

have passed individually adequate test suites

3. Consider Risks if you skip/reduce 2

4. System scope test harness

5. Test environment

installed, tested and

stabilized

33

New FW Test: Exit Criteria

• At least one test for each row in the use case’s operational relation, meeting the Extended Use Case Test exit criteria

• At least one minimal test of every association in the class model, meeting the Class Association Test exit criteria

• CRUD Coverage• Control model coverage: transaction coverage

for a state machine or branch coverage on all sequence diagrams

34

New FW Test: Consequences

• Cost of testing is directly proportional to number of use cases and class associations.

• Producers should monitor and analyze the results of application instantiations

• Release FW after components and subsystems tests pass? Let the consumer test the FW Interleaved debugging Some features remain untested?

35

Framework Testing

• New Framework Test: Applies to initial and early versions Design a test suite and the test instantiation from


• Popular Framework Test: Applies to a framework that becomes widely used Design a test suite and the test instantiation from


testing

36

Popular FW Test: Context

• Test suite is needed to validate the new

features and to identify potential compatibility

problems

• FW itself is also at risk for faults of New FW

Test, fewer as FW becomes mature

• Failures of three new fault classes

37

Popular FW Test: Fault Model

• Feature Interaction: behavior or side effect

produced when several features are used

together. => Desirable/Undesirable??

• Compatibility bug: revision to the FW breaks

previously working feature

• Latent bug: bug that was dormant in the FW,

new instantiation reaches/triggers failures

38

Popular FW Test: Test Model

• Feature Interaction Tests FUT may exercise some new FI Additional test cases from anti-requirements

• Regression Tests• Compatibility Tests

Can use existing instantiations with test suites Next increment of FW is instantiated with an

application that was stabilized for previous version• Latent bugs => gain info about shift in usage• New Features: use EUCT and CAT

39

Popular FW Test: Entry Criteria

• New FW Test Entry Criteria +

• Optional: Build at least 1 actual consumer

instantiation of the current release

• Conduct an analysis of the new/changed

features

40

Popular FW Test: Exit Criteria

• New FW Test Exit Criteria

41

Popular FW Test: Consequences

• New FW Test Consequences

• Inform users about obsolete/unsupported features

• As the FW begins to stabilize: Rate of test decay will decrease What happens if the existing library of test cases

has revealed all bugs it can? => Pesticide Paradox

Documents

Software Testing Chapter 11: Reusable Components Nagy Akos – Nick Baetens – Kevin Buyl – Matthias De Cock - Dieter De Hen