28
October 26, 2022 1 Component Testing Ednaldo Dilorenzo de Souza Filho [email protected]

Component Testing

Embed Size (px)

Citation preview

Page 1: Component Testing

April 12, 2023 1

Component Testing

Ednaldo Dilorenzo de Souza [email protected]

Page 2: Component Testing

April 12, 2023 2

Summary

Introduction Fundaments of Testing Software Reuse Component Testing Motivation Test-Driven Development Third-Party Testing Regression Testing based on Version Changes of

Components Testability Analysis for Software Components White on Black

Page 3: Component Testing

April 12, 2023 3

Introduction

Testing is an important process to support assurance; As software becomes more pervasive and is used more often to

perform critical tasks, it will be required to be of higher quality; Because testing requires the execution of the software, it is

often called dynamic analysis [Harrold]; Testing consists in compare the outputs from the executions

with expected results to identify those test cases on which the software failed;

Testing cannot show the absence of faults, if can show only their presence.

Page 4: Component Testing

April 12, 2023 4

Fundaments of Testing

Testing goals: Find faults in a software program; A good test case have a high probability to find errors; A good test process should find errors;

Testing fundamentals: Testing should be connected with client requirements; Testing should be planned before testing execution; Isolation of suspected components; Testing should start in individual components; Complete testing is impossible; Testing shouldn’t be executed for developers;

Page 5: Component Testing

April 12, 2023 5

Software Reuse

Increased reliability: Reuse is supposed to have a positive effect on the quality of the software entity reused;

Reduce Process Risk: Software reuse can reduce risks inherent in software projects;

Effective use of specialists: Software reuse can also contribute to an effective use of specialists;

Standards compliance: Improve its standards compliance, which in turn can have other benefits;

Accelerated development: Software reuse can accelerate development;

[Beydeda]

Page 6: Component Testing

April 12, 2023 6

Component Testing Motivation

The Ariane 5 Lesson [Weyuker] In June 1996, during the maiden voyage of the Ariane 5 launch

vehicle, the launcher veered off course and exploded less than one minute after take-off;

The explosion resulted from insufficiently tested software reused from Ariane 4 launcher;

Context-Dependent Testing of a Component [Beydeda]; Insufficient Documentation of a Component; Component User’s Dependence on the Component Provider;

Page 7: Component Testing

April 12, 2023 7

Component Testing Motivation

Problems with systems built from reusable components: Performance problems; Fitting selected components together; Absence of low-level understanding;

Developing a component for a particular project with no expectation of reuse, testing proceed as usual;

Significant additional testing should be done for changing priority;

Modify or not the source code? Facilitating Reusable Component Testing

Associate a person or a team for maintaining each component; Add software specification; Add test suite; Add pointers between the specification and parts of the test suite;

Page 8: Component Testing

April 12, 2023 8

Test-Driven Development [Janzen]

Requires writing automated tests prior to developing functional code;

Used in agile methods; To promote testing to an analysis and design step the practice

of refactoring must be introduced; TDD in industry

18 percent to 50 percent more external test cases than code produced by corresponding control groups;

Less time debugging code; TDD in academia

Early detection of defects; Improvement in software quality and programmer productivity;

Page 9: Component Testing

April 12, 2023 9

Third-Party Testing and the Quality of Software Components [Councill]

10 contribuitors devoted 70 person-hour to developing a definition of component;

Ariane Project; Certification business introduced a Maturity Model focused on

large organizations; 99 percent of all US businesses are small businesses; Producers provide the developing and testing procedures,

purchasers or their agents conduct second-party testing and independent testing organizations perform third-party testing;

Page 10: Component Testing

April 12, 2023 10

Third-Party Testing and Stirrings of the New Software Engineering [Councill2]

Licensed software engineers must manage Intense requirements elicitation; Comprehensive requirements specifications that address all states

end endeavor to assure fail-safe and fault-tolerant operations; Designs that can be tested on paper; The anticipation of third-party testing;

Regulatory software component certification Software engineer licensing; Mandated standards for third-party testing of software

components;

Page 11: Component Testing

April 12, 2023 11

Regression Test Selection Based on Version Changes of Components [Sajeev]

Regression test strategy test components tested before to assure new components didn’t affected it;

Most regression testing techniques are based on white box strategies by analyzing code;

Use of UML (Unified Modeling Language) and its associated with OCL (Object Constraint Language) to formally specify the test selection process;

Page 12: Component Testing

April 12, 2023 12

Regression Test Selection Based on Version Changes of Components [Sajeev]

Regression Test Selection Let P be a program, and P’ be a new version of program created

by replacing some of the components in P with new versions; Let T be a set of test cases that is already run on P; The aim of regression testing is to show that P’ satisfies T; Base case

T = T’ Optimal case

T’ T

Page 13: Component Testing

April 12, 2023 13

Regression Test Selection Based on Version Changes of Components [Sajeev]

Page 14: Component Testing

April 12, 2023 14

Regression Test Selection Based on Version Changes of Components [Sajeev]

Test Selection Strategy Step 1: Select all test sequences which include a direct call to a

modified method; Step 2: Select all test sequences which include a call to any method

which in turn uses, directly or indirectly, a modified method of a revised component;

Union of two steps bring the test sequences for regression testing;

Page 15: Component Testing

April 12, 2023 15

Testability Analysis For Software Components [Nguyen]

Testability is a quality factor, which can be predicted as soon as the system is specified, to indicate the ease (or difficulty) for testing the system;

A component is said to be domain-observable, if distinct outputs are produced from distinct inputs;

A component is said to be a domain-controllable, if its output domain is covered from its input domain;

The testability of a system is based on controllability and observability of the components in the system;

Controllability is defined as the ease to generate the inputs of a component from the inputs of the system;

Observability is defined as the ease to propagate the outputs of a component to the final outputs of the system;

Page 16: Component Testing

April 12, 2023 16

Testability Analysis For Software Components [Nguyen]

Testability Model

Page 17: Component Testing

April 12, 2023 17

Testability Analysis For Software Components [Nguyen]

Testability measures Information channel Coefficient of information loss for a module Module capacity Edge capacity Information transfer Testability measures

Page 18: Component Testing

April 12, 2023 18

Testability Analysis For Software Components [Nguyen]

Static Single Assignment Form

Page 19: Component Testing

April 12, 2023 19

Testability Analysis For Software Components [Nguyen]

Case Study

Page 20: Component Testing

April 12, 2023 20

Testability Analysis For Software Components [Nguyen]

Case Study

Observability and Controlability values

Page 21: Component Testing

April 12, 2023 21

White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen]

It would be ideal to test a program with its entire input domain. A more practical work is to construct a test suite for testing. A test suite constructed should be as comprehensive as

possible, and it should be as small as possible. Approach for reduce the test case based on specification (Black

box) using white box information. In the black box approach, the category-partition method

(CPM) and the classification-tree method are particularly useful.

Page 22: Component Testing

April 12, 2023 22

White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen]

Background In partition testing approach input domain of a program is divided

into subsets, called subdomains; The tester identifies relevant aspects of the specification for

testing called classification; The subdomains associated with the classification are called

classes; Test cases are formed and every incompatible classes identified

said to be illegitimate; Only legitimate classes are considered reducing the number of

test cases in the test suite;

Page 23: Component Testing

April 12, 2023 23

White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen]

Background

Page 24: Component Testing

April 12, 2023 24

White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen]

Information of the paths executed by the test cases are used for helping test case reduction;

When two test cases are identical differing by only one class they are called a matching pair;

Classes differing from a matching pair are called differentiating classes, or a differentiating class pair;

Matching Pair Differentiating Classes

Page 25: Component Testing

April 12, 2023 25

White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen]

Based on the white box analysis we can verify which test cases can be omitted;

Here tc4 or tc5 can be ommited

Page 26: Component Testing

April 12, 2023 26

White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen]

Page 27: Component Testing

April 12, 2023 27

References

[Beydeda] Sami Beydeda and Volker Gruhn, Testing Commercial-of-the-Shelf Components and Systems, Springer, 2005.

[Councill] William T. Councill, Third-Part Testing and the Quality of Software Components, IEEE Software, Quality Time, July 1999.

[Councill2] William T. Councill, Third-Part Testing and Stirrings of the New Software Engineering, IEEE Software, Quality Time, July 1999.

[Chen] T.Y. Chen, P.L. Poon, S.F. Tang and Y.T. Yu, White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases, IEEE, 2000.

[Janzen] David Janzen and Hossein Saiedian, Test-Driven Development: Concepts, Taxonomy, and Future Direction, IEEE Computer Society, Cover Feature, Semtember 2005;

[Sajeev] A.S.M. Sajeev and Bugi Wibowo, Regression Test Selection Based on Version Changes of Components, IEEE, Proceedings of the Tenth Asia-Pacific Software Engineering Conference, 2003;

Page 28: Component Testing

April 12, 2023 28

References

[Nguyen] T.B. Nguyen, M. Delaunay and C. Robach, Testability Analysis For Software Components, IEEE, Proceedings of the International Conference on Software Maintnance, 2002.

[Harrold] M. J. Harrold, Testing a Roadmap, College of Computing, Georgia Institute Technology, 2000.