Software Engineering II e Engineering Testing · · 2015-03-25Using UML, Patterns, and Java e Engineering!! Software Engineering II Testing 5 May 2009 Bernd Bruegge Applied Software

Usi

ng U

ML

, P

atte

rns,

and J

ava!

Ob

ject

-Ori

ente

d S

oft

ware

En

gin

eeri

ng!

Software Engineering II Testing

5 May 2009

Bernd Bruegge Applied Software Engineering

Technische Universität München http://wwwbruegge.in.tum.de

2009 Bernd Bruegge! ! ! ! Software Engineering II, Lecture 3: Testing ! ! 2!

Outline of Testing

Today:

•! Terminology

•! Testing Activities

•! Unit testing

•! Integration testing

•! Testing strategy

•! Design patterns & testing

•! System testing

•! Function testing

•! Acceptance testing.

Next Lecture:

•! Model-based Testing

•! U2TP Testing Profile

Famous Problems •! A F-16 fighter plane crossed the equator using autopilot

•! Result: plane flipped over

•! Reason?

•! Reuse of autopilot software from a rocket

•! The Therac-25 was a machine used for radiation therapy

•! Result: Patients were given massive overdoses of radiation, approximately 100 times the intended dose (6 deaths)"

•! Reason: Failure only occurred when a particular sequence of keys was entered on the VT-100 controlling a PDP-11: X, !, E, Enter

•! The "X” erroneously selected 25MV photon mode. The “Cursor up” was an attempt to correct the mistake. The "E” followed by "Enter” then attempted to correctly select 25 MeV Electron mode.

•! The machine responded with an error message “MALFUNCTION 54”, which was not explained in the user manual. Machine shut down briefly and then showed under dosage of radiation.

•! Because of this the operator hit “P” to continue treatment.


Terminology

•! Failure: Any deviation of the observed behavior from the specified behavior

•! Erroneous state (error): The system is in a state such that further processing by the system can lead to a failure

•! Fault: The mechanical or algorithmic cause of an error (“bug”)

•! Validation: Activity of checking for deviations between the observed behavior of a system and its specification.


What is this?

A failure?

An error?

A fault?

We need to describe specified and desired behavior first!


Erroneous State (“Error”)


Algorithmic Fault


Mechanical Fault Superuser!


The F-16 Problem

•! Where is the failure?

•! Where is the error?

•! Where is the fault? •! Bad use of implementation

inheritance

•! A Plane is not a rocket.

Rocket!

Plane!


Examples of Faults and Errors

•! Faults in the Interface specification

•! Mismatch between what the client needs and what the server offers

•! Mismatch between requirements and implementation

•! Algorithmic Faults

•! Missing initialization

•! Incorrect branching condition

•! Missing test for null

•! Mechanical Faults (very hard to find)

•! Operating temperature outside of equipment specification

•! Errors •! Null reference errors

•! Concurrency errors

•! Exceptions.


How do we deal with Errors, Failures and Faults?


Modular Redundancy


Declaring the Bug as a Feature


Patching


Testing


Another View on How to Deal with Faults

•! Fault avoidance •! Use a good methodology to reduce complexity

•! Use configuration management to prevent inconsistency

•! Apply verification to prevent algorithmic faults

•! Use reviews

•! Fault detection •! Testing: Activity to provoke failures in a planned way

•! Debugging: Find and remove the cause (Faults) of an observed failure

•! Monitoring: Deliver information about state => Used during debugging

•! Fault tolerance •! Enable exception handling

•! Provide modular redundancy.


Taxonomy for Fault Handling Techniques 5 05 2009

Fault Handling!

Fault "

Avoidance!

Fault "

Detection!

Fault "

Tolerance!

Verification!

Configuration"

Management!Methodoloy!

Atomic"

Transactions!

Modular"

Redundancy!

System "

Testing!

Integration"

Testing!

Unit!

Testing!

Testing! Debugging!


Observations

•! It is impossible to completely test any nontrivial module or system

•! Practical limitations: Complete testing is prohibitive in time and cost

•! Theoretical limitations: e.g. Halting problem

•! “Testing can only show the presence of bugs, not their absence” (Dijkstra).

•! Testing is not for free

=> Define your goals and priorities

Edsger W. Dijkstra (1930-2002)"" " " "!

- First Algol 60 Compiler!

"- 1968:!

- T.H.E. !

- Go To considered Harmful, CACM!

"- Since 1970 Focus on Verification!

and Foundations of Computer Science !

"-"1972 A. M. Turing Award!


Testing takes creativity

•! To develop an effective test, one must have:

•! Detailed understanding of the system

•! Application and solution domain knowledge

•! Knowledge of the testing techniques

•! Skill to apply these techniques

•! Testing is done best by independent testers

•! We often develop a certain mental attitude that the program should in a certain way when in fact it does not

•! Programmers often stick to the data set that makes the program work

•! A program often does not work when tried by somebody else.

behave!


Testing Activities

Unit"

Testing!

Acceptance"

Testing!

Integration"

Testing!

System"

Testing!

Requirements"

Analysis!

Document!

Client!

Expectation!

System"

Design!

Document!

Object"

Design!

Document!

Developer! Client!

Unit"

Testing!

Acceptance"

Testing!

Integration"

Testing!

System"

Testing!


Types of Testing

•! Unit Testing

•! Individual component (class or subsystem)

•! Carried out by developers

•! Goal: Confirm that the component or subsystem is correctly coded and carries out the intended functionality

•! Integration Testing •! Groups of subsystems (collection of subsystems) and

eventually the entire system


•! Goal: Test the interfaces among the subsystems.

Unit"

Testing!

Acceptance"

Testing!

Integration"

Testing!

System"

Testing!


Types of Testing continued...

•! System Testing

•! The entire system


•! Goal: Determine if the system meets the requirements (functional and nonfunctional)

•! Acceptance Testing •! Evaluates the system delivered by developers

•! Carried out by the client. May involve executing typical transactions on site on a trial basis

•! Goal: Demonstrate that the system meets the requirements and is ready to use.

Unit"

Testing!

Acceptance"

Testing!

Integration"

Testing!

System"

Testing!


When should you write a test?

•! Traditionally after the source code to be tested

•! In XP before the source code to be tested

•! Test-Driven Development Cycle

•! Add a test

•! Run the automated tests

=> see the new one fail

•! Write some code

•! Run the automated tests

=> see them succeed

•! Refactor code.


Unit Testing

•! Static Testing (at compile time)

•! Static Analysis

•! Review

•! Walk-through (informal)

•! Code inspection (formal)

•! Dynamic Testing (at run time) •! Black-box testing

•! White-box testing.

Unit"

Testing!

Acceptance"

Testing!

Integration"

Testing!

System"

Testing!


Static Analysis with Eclipse

•! Compiler Warnings and Errors •! Possibly uninitialized Variable

•! Undocumented empty block

•! Assignment has no effect

•! Checkstyle •! Check for code guideline violations

•! http://checkstyle.sourceforge.net

•! FindBugs •! Check for code anomalies

•! http://findbugs.sourceforge.net

•! Metrics •! Check for structural anomalies

•! http://metrics.sourceforge.net

Unit"

Testing!

Acceptance"

Testing!

Integration"

Testing!

System"

Testing!


Black-box testing

•! Focus: I/O behavior

•! If for any given input, we can predict the output, then the component passes the test

•! Requires test oracle

•! Goal: Reduce number of test cases by equivalence partitioning:

•! Divide input conditions into equivalence classes

•! Choose test cases for each equivalence class.

Unit"

Testing!

Acceptance"

Testing!

Integration"

Testing!

System"

Testing!


Black-box testing: Test case selection

a) Input is valid across range of values •! Developer selects test cases from 3 equivalence classes:

•! Below the range

•! Within the range

•! Above the range

b) Input is only valid, if it is a member of a discrete set

•! Developer selects test cases from 2 equivalence classes:

•! Valid discrete values

•! Invalid discrete values

•! No rules, only guidelines.


Lecture Check: Where are we?

•! Terminology

•! Testing Activities

•! Unit testing

•! Static Testing

•! Dynamic Testing

•! Blackbox Testing

!!Example

Unit"

Testing!

Acceptance"

Testing!

Integration"

Testing!

System"

Testing!


Black box testing: An example

public class MyCalendar {

public int getNumDaysInMonth(int month, int year)

throws InvalidMonthException

{ … }

}

Representation for month:!1: January, 2: February, …., 12: December!

Representation for year:!1904, … 1999, 2000,…, 2006, …!

How many test cases do we need for the black box testing of

getNumDaysInMonth()?!


White-box testing overview

•! Code coverage

•! Branch coverage

•! Condition coverage

•! Path coverage

=> Details in the exercise session about testing

Unit"

Testing!

Acceptance"

Testing!

Integration"

Testing!

System"

Testing!


Unit Testing Heuristics

1. Create unit tests when object design is completed

•! Black-box test: Test the functional model

•! White-box test: Test the dynamic model

2. Develop the test cases

•! Goal: Find effective number of test cases

3. Cross-check the test cases to eliminate duplicates

•! Don't waste your time!

4. Desk check your source code

•! Sometimes reduces testing time

5. Create a test harness

•! Test drivers and test stubs are needed for integration testing

6. Describe the test oracle

•! Often the result of the first successfully executed test

7. Execute the test cases

•! Re-execute test whenever a change is made (“regression testing”)

8. Compare the results of the test with the test oracle

•! Automate this if possible.


Excursion J_Unit: A Java Framework for unit testing

•! A Java framework for writing and running unit tests

•! Test cases and fixtures

•! Test suites

•! Test runner

•! Written by Kent Beck and Erich Gamma

•! Written with “test first” and pattern-based development in mind

•! Tests written before code

•! Allows for regression testing

•! Facilitates refactoring

•! JUnit is Open Source

•! www.junit.org

•! JUnit Version 4.6, released April 2009

Unit"

Testing!

Acceptance"

Testing!

Integration"

Testing!

System"

Testing!


*

JUnit Classes

Test

run(TestResult)

ConcreteTestCase

setUp() tearDown() runTest()

TestResult

TestCase

run(TestResult) setUp() tearDown()

testName:String

runTest()

TestSuite

run(TestResult) addTest()

UnitToBeTested

Methods under Test!

Skip!

J_Unit!

Slides!


An example: Testing MyList

•! Unit to be tested

•! MyList

•! Methods under test •! add()

•! remove()

•! contains()

•! size()

•! Concrete Test case

•! MyListTestCase

Test

run(TestResult)

MyListTestCase

setUp() tearDown() runTest() testAdd() testRemove()!

TestResult

TestCase


testName:String

runTest()

TestSuite


MyList

add() remove() contains() size()

*


Writing TestCases in JUnit public class MyListTestCase extends TestCase {

public MyListTestCase(String name) {

super(name);

}

public void testAdd() {

// Set up the test

List aList = new MyList();

String anElement = “a string”;

// Perform the test

aList.add(anElement);

// Check if test succeeded

assertTrue(aList.size() == 1);

assertTrue(aList.contains(anElement));

}

protected void runTest() {

testAdd();

}

}

Test

run(TestResult)

MyListTestCase

setUp() tearDown() runTest() testAdd() testRemove()!

TestResult

TestCase


testName:String

runTest()

TestSuite


MyList

add() remove() contains() size()

*


Writing Fixtures and Test Cases

public class MyListTestCase extends TestCase {

// …

private MyList aList;

private String anElement;

public void setUp() {

aList = new MyList();

anElement = “a string”;

}

public void testAdd() {



assertTrue(aList.contains(anElement));

}

public void testRemove() {


aList.remove(anElement);


assertFalse(aList.contains(anElement));

}

Test Fixture

Test Case

Test Case


Collecting TestCases into TestSuites

public static Test suite() {

TestSuite suite = new TestSuite();

suite.addTest(new MyListTest(“testAdd”));

suite.addTest(new MyListTest(“testRemove”));

return suite;

}

Test

run(TestResult)

TestCase


testName:String

runTest()

TestSuite


Composite Pattern!

*


Design patterns in JUnit

Test

run(TestResult)

ConcreteTestCase


TestResult

TestCase


testName:String

runTest()

TestSuite


Command Pattern

Composite

Pattern

Adapter

Pattern

Template Method

Pattern

TestedUnit

*


Design patterns in JUnit

Test

run(TestResult)

ConcreteTestCase


TestResult

TestCase


testName:String

runTest()

TestSuite


Command Pattern

Composite

Pattern

Adapter

Pattern

Template Method

Pattern

TestedUnit

*


Other JUnit features

•! Textual and GUI interface •! Displays status of tests

•! Displays stack trace when tests fail

•! Integrated with Maven and Continuous Integration •! http://maven.apache.org

•! Build and Release Management Tool

•! http://Maven.apache.org/continuum

•! Continous integration server for Java programs

•! All tests are run before release (regression tests)

•! Test results are advertised as a project report

•! Many specialized variants •! Unit testing of web applications

•! J2EE applications


Additional Readings

•! [Herman Bruyninckx] Embedded Control Systems Design, WikiBook, Learning from Failure: http://en.wikibooks.org/wiki/Embedded_Control_Systems_Design/Learning_from_failure

•! [Dijkstra 68] •! E. W. Dijkstra, "Letters to the editor: go to statement

considered harmful“, Communications of the ACM 11 (3): 147–148, March 1968

•! [Meszaros 2007]

•! G. Meszaros, xUnit Test Patterns, Refactoring Test Code, Addison Wesley, 2007

•! http://www.junit.org/

•! Resources for Test Driven Development

Usi

ng U

ML

, P

atte

rns,

and J

ava!

Ob

ject

-Ori

ente

d S

oft

ware

En

gin

eeri

ng!

Testing Part 2: Integration & System Testing


Overview


•! Big bang

•! Bottom up

•! Top down

•! Sandwich

•! Continuous

•! System testing •! Functional

•! Performance

•! Acceptance testing

•! Summary


Integration Testing

•! The entire system is viewed as a collection of subsystems (sets of classes) determined during the system and object design

•! Goal: Test all interfaces between subsystems and the interaction of subsystems

•! The Integration testing strategy determines the order in which the subsystems are selected for testing and integration.


Why do we do integration testing?

•! Unit tests only test the unit in isolation

•! Many failures result from faults in the interaction of subsystems

•! Often many Off-the-shelf components are used that cannot be unit tested

•! Without integration testing the system test will be very time consuming

•! Failures that are not discovered in integration testing will be discovered after the system is deployed and can be very expensive.


Stubs and drivers

•! Driver: •! A component, that calls the TestedUnit

•! Controls the test cases

•! Stub: •! A component, the TestedUnit

depends on

•! Partial implementation

•! Returns fake values.

Driver!

Tested!

Unit!

Stub!


Example: A 3-Layer-Design

Layer I!

Layer II!

Layer III!

Spread"

SheetView!

A!

Calculator!

C!

BinaryFile "

Storage!

E!

XMLFile"

Storage!

F!

Currency"

DataBase!

G!

Currency!

Converter!

D!

Data"

Model!

B!

A!

C!

E! F! G!

D!B!

Spread"

SheetView!

BinaryFile "

Storage!

Entity!

Model!

A!

E! F!

Currency"

DataBase!

G!

Currency!

Converter!

D!B!

Calculator!

C!

XMLFile"

Storage!

(Spreadsheet)


A!

C!

E! F! G!

D!B!

Big-Bang Approach

Test A!

Test B!

Test G!

Test F!

Test E!

Test C!

Test D!Test !

A, B, C, D,!

E, F, G!


Bottom-up Testing Strategy

•! The subsystems in the lowest layer of the call hierarchy are tested individually

•! Then the next subsystems are tested that call the previously tested subsystems

•! This is repeated until all subsystems are included

•! Drivers are needed.


A!

C!

E! F! G!

D!B!

Bottom-up Integration A!

Test !

A, B, C, D,!

E, F, G!

E!Test E!

F!

Test F!

B!

Test B, E, F!

C!

Test C!

D!

Test D,G!

G!

Test G!


Pros and Cons of Bottom-Up Integration Testing

•! Con:

•! Tests the most important subsystem (user interface) last

•! Drivers needed

•! Pro

•! No stubs needed

•! Useful for integration testing of the following systems

•! Object-oriented systems

•! Real-time systems

•! Systems with strict performance requirements.


Top-down Testing Strategy

•! Test the top layer or the controlling subsystem first

•! Then combine all the subsystems that are called by the tested subsystems and test the resulting collection of subsystems

•! Do this until all subsystems are incorporated into the test

•! Stubs are needed to do the testing.


Top-down Integration

Test !

A, B, C, D,!

E, F, G!

All Layers!Layer I + II!

Test A, B, C, D!

Layer I!

Test A!

A!

E! F!

B! C! D!

G!


Pros and Cons of Top-down Integration Testing

Pro

•! Test cases can be defined in terms of the functionality of the system (functional requirements)

•! No drivers needed

Cons

•! Writing stubs is difficult: Stubs must allow all possible conditions to be tested.

•! Large number of stubs may be required, especially if the lowest level of the system contains many methods.

•! Some interfaces are not tested separately.


Sandwich Testing Strategy

•! Combines top-down strategy with bottom-up strategy

•! The system is viewed as having three layers •! A target layer in the middle

•! A layer above the target

•! A layer below the target

•! Testing converges at the target layer.


Sandwich Testing Strategy

Test !

A, B, C, D,!

E, F, G!

Test B, E, F!

Test D,G!

Test A!

Test E!

Test F!

Test G!

Test A,B,C, D!

A!

E! F!

B! C! D!

G!


Pros and Cons of Sandwich Testing

•! Top and Bottom Layer Tests can be done in parallel

•! Problem: Does not test the individual subsystems and their interfaces thoroughly before integration

•! Solution: Modified sandwich testing strategy


Modified Sandwich Testing Strategy

•! Test in parallel:

•! Middle layer with drivers and stubs

•! Top layer with stubs

•! Bottom layer with drivers

•! Test in parallel:

•! Top layer accessing middle layer (top layer replaces drivers)

•! Bottom accessed by middle layer (bottom layer replaces stubs).


Modified Sandwich Testing

Test F!

Test E!

Test B!

Test G!

Test D!

Test A!

Test C!

Test B, E, F!

Test D,G!

Test A,C!

Test !

A, B, C, D,!

E, F, G!

A!

E! F!

B! C! D!

G!


Continuous Testing

•! Continuous build:

•! Build from day one

•! Test from day one

•! Integrate from day one

!! System is always runnable

•! Requires integrated tool support: •! Continuous build server

•! Automated tests with high coverage

•! Tool supported refactoring

•! Software configuration management

•! Issue tracking.


Spread"

SheetView!

BinaryFile "

Storage!

Data"

Model!

Continuous Testing Strategy

Layer I!

Layer II!

Layer III!

A!

E! F!

Currency"

DataBase!

G!

Currency!

Converter!

D!B!

Calculator!

C!

XMLFile"

Storage!

Sheet View!+ Cells!

+ Addition!+ File Storage!


Steps in Integration Testing

.!

1. Based on the integration strategy, select a component to be tested. Unit test all the classes in the component.

2. Put selected component together; do any preliminary fix-up necessary to make the integration test operational (drivers, stubs)

3. Test functional requirements: Define test cases that exercise all uses cases with the selected component

4. Test subsystem decomposition: Define test cases that exercise all dependencies

5. Test non-functional requirements: Execute performance tests

6. Keep records of the test cases and testing activities.

7. Repeat steps 1 to 7 until the full system is tested.

The primary goal of integration testing is to identify failures with the (current) component configuration.


System Testing

•! Functional Testing •! Validates functional requirements

•! Performance Testing

•! Validates non-functional requirements

•! Acceptance Testing •! Validates clients expectations


. !

Functional Testing

Goal: Test functionality of system

•! Test cases are designed from the requirements analysis document (better: user manual) and centered around requirements and key functions (use cases)

•! The system is treated as black box

•! Unit test cases can be reused, but new test cases have to be developed as well.


Performance Testing

Goal: Try to violate non-functional requirements

•! Test how the system behaves when overloaded. •! Can bottlenecks be identified? (First candidates for

redesign in the next iteration)

•! Try unusual orders of execution •! Call a receive() before send()

•! Check the system’s response to large volumes of data

•! If the system is supposed to handle 1000 items, try it with 1001 items.

•! What is the amount of time spent in different use cases?

•! Are typical cases executed in a timely fashion?


Types of Performance Testing

•! Stress Testing

•! Stress limits of system

•! Volume testing

•! Test what happens if large amounts of data are handled

•! Configuration testing

•! Test the various software and hardware configurations

•! Compatibility test

•! Test backward compatibility with existing systems

•! Timing testing

•! Evaluate response times and time to perform a function

•! Security testing

•! Try to violate security requirements

•! Environmental test

•! Test tolerances for heat, humidity, motion

•! Quality testing

•! Test reliability, maintain- ability & availability

•! Recovery testing

•! Test system’s response to presence of errors or loss of data

•! Human factors testing

•! Test with end users.


Acceptance Testing

•! Goal: Demonstrate system is ready for operational use

•! Choice of tests is made by client

•! Many tests can be taken from integration testing

•! Acceptance test is performed by the client, not by the developer.

•! Alpha test:

•! Client uses the software at the developer’s environment.

•! Software used in a controlled setting, with the developer always ready to fix bugs.

•! Beta test:

•! Conducted at client’s environment (developer is not present)

•! Software gets a realistic workout in target environment


Testing has many activities

Establish the test objectives!

Design the test cases!

Write the test cases!

Test the test cases!

Execute the tests!

Evaluate the test results!

Change the system!

Do regression testing!


Test Team

Test!

Analyst!

Team!User!

Programmer!

too familiar!with code!

Professional!

Tester!

Configuration !

Management!

Specialist!

System !

Designer!


The 4 Testing Steps

1. Select what has to be tested

•! Analysis: Completeness of requirements

•! Design: Cohesion

•! Implementation: Source code

2. Decide how the testing is done

•! Review or code inspection

•! Proofs (Design by Contract)

•! Black-box, white box,

•! Select integration testing strategy (big bang, bottom up, top down, sandwich)

3. Develop test cases

•! A test case is a set of test data or situations that will be used to exercise the unit (class, subsystem, system) being tested or about the attribute being measured

4. Create the test oracle

•! An oracle contains the predicted results for a set of test cases

•! The test oracle has to be written down before the actual testing takes place.


Guidance for Test Case Selection

•! Use analysis knowledge about functional requirements (black-box testing):

•! Use cases

•! Expected input data

•! Invalid input data

•! Use design knowledge about system structure, algorithms, data structures (white-box testing):

•! Control structures

•! Test branches, loops, ...

•! Data structures

•! Test records fields, arrays, ...

•! Use implementation knowledge about algorithms and datastructures:

•! Force a division by zero

•! If the upper bound of an array is 10, then use 11 as index.


Summary

•! Testing is still a black art, but many rules and heuristics are available

•! Testing consists of •! Unit testing


•! System testing

•! Acceptance testing

•! Design patterns can be used for integration testing

•! Testing has its own lifecycle


Additional Reading

•! Lecture: Prozesse und Methoden beim Testen von Software, Dr. Stefan Kriebel

•! Andreas Spillner, Tilo Linz, Basiswissen Softwaretest, dpunkt.verlag, 2004

•! JUnit Website www.junit.org/index.htm

•! J. Thomas, M. Young, K. Brown, A. Glover, Java Testing Patterns, Wiley, 2004

•! D. Saff and M. D. Ernst, An experimental evaluation of continuous testing during development Int. Symposium on Software Testing and Analysis, Boston July 12-14, 2004, pp. 76-85"

•! A controlled experiment shows that developers using continuous testing were three times more likely to complete the task before the deadline than those without.

Usi

ng U

ML

, P

atte

rns,

and J

ava!

Ob

ject

-Ori

ente

d S

oft

ware

En

gin

eeri

ng!

Testing, Part 3 Model-based Testing


Outline of the Lectures on Testing

"!Terminology

"!Types of errors

"!Approaches for dealing with errors

"!Testing activities

"!Unit testing

"!Integration testing

"!System testing

•! Model-based Testing


Generating Tests from a System Model

•! There are many different ways to "derive" tests from a system model"

•! Manual generation"

•! Automatic generation"

•! Because testing is usually experimental and based on heuristics, there is no one best way to do this. "

•! It is common to consolidate all test related decisions into a package that is often known as "test requirements” or "test package”."

•! This test package can contain e.g. information about the part of the system model that should be the focus for testing, or about the conditions where it is correct to stop testing (test stopping criteria)."


Model Driven Architecture (MDA)

•! Recall: MDA focuses on forward engineering, i.e. producing code from a system model

•! Note: The term “architecture” in MDA does not mean the architecture of the system being modeled, but refers the architecture of the various standards and models that serve as the technology basis for MDA

•! UML: Unified Modeling Language

•! MOF: Meta-Object Facility

•! XMI: Metadata Interchange

•! EDOC: Enterprise Distributed Object Computing

•! SPEM: Software Process Engineering Metamodel

•! CWM: Common Warehouse Metamodel

An important objective of MDA is executable UML.


Executable UML

•! Executable UML means that given a UML system model the following steps can be performed automatically:

•! Code generation

•! Simulation Validation

•! Test generation (“Model based Testing”)


Model Based Testing

•! Definition Model Based Testing: All testing is based on a system model which is used to (manually) generate test cases and oracles

•! Advantage:

•! Increased effectiveness of testing

•! Decreased costs

•! Reuse of artifacts such as analysis and design models

•! Requirement:

•! The system model is formalized (specification model) and described in UML 2.0


Model Driven Testing

•! Model Driven Testing is a special case of model based testing:

•! All the test cases are automatically generated from the system model

•! Advantage:

•! Automatic generation of tests suites

•! Better test thoroughness

•! Reduces the cost of test execution.


Observations on Model Based Testing

•! A system model is an abstract representation of the system’s desired behavior

•! The test cases derived from this model are functional tests on the same level of abstraction as the model

•! These test cases are known as the abstract test suite

•! Because test suites are derived from models - not from source code - model-based testing is a form of black-box testing"

•! The abstract test suite cannot be executed

•! An executable test suite must be derived from the abstract test suite which communicates with the system under test (SUT)

•! This is done by mapping the abstract test suite to concrete test cases suitable for execution.

•! Mapping is not part of this lecture.


U2TP

•! UML-2 Testing Profile (U2TP):

•! A UML extension that makes UML applicable to software testing.

•! U2TP allows the collection of all information required for the model-based testing process

•! Deriving a test model from the system model.

•! Test Model: •! A set of test cases derived from the system model.


General Definitions used in U2TP

•! Test: An attempt to create a difference between the observed behavior and the specified behavior of a system a planned way

•! Test data: The structures and meaning of values to be processed in a test

•! System under test (SUT)

•! A system, subsystem or component

•! The SUT is treated as black-box. This means:

•! The SUT can be accessed only via a public interface

•! No information on the internal structure of the SUT is available for use in the specification of test cases.

•! Test case: Describes a specified behavior, that is, a behavior specified in the system model (normative behavior) of the SUT.


Test Objective

•! Test objective

•! Find any difference between the observed behavior and the specified behavior, that is, the behavior specified in the system model.

•! UML Example

•! The test objective in the diagram below specifies that the objective of the ValidWithdrawal test case is to validate the WithdrawMoney use case.

<<testcase>>!

ValidWithdrawal!<<usecase>>!

WithdrawMoney!<<objective>>!


Test Architecture Definitions

•! Test component: An object within a test system

•! Test architecture: The test components and their relationships in a test system

•! Testing configuration: The combination of a test system with a SUT

•! Test control: A ordering of the execution of the tests performed on the SUT

•! Test context: A description of

•! A set of test cases

•! A test configuration

•! A test control.


Example: Test System for a Vending Machine •! Vending Machine with 2 use cases:

•! The user inserts money

•! A bottle is returned, when the inserted money equals the price of the bottle.


Model-based Testing of MoneyBox

The test system MoneyUnitTest is developed with the purpose of unit testing the Moneybox, which is part of the system under test

MoneyBox must be imported by MoneyUnitTest to be accessible.

Test System! SUT!


UML Model of the MoneyBox

Now we need to test that the machine correctly counts the money

inserted by the user into the Moneybox -> MoneyUnitTest


Model-based Testing of MoneyBox

MoneyBox is the system under test

MoneyUnitTest is the test system

MoneyTest is the test context

addMoneyTest is the test case

Test Context!

Test

Case!

Test System!SUT!

Documents

Software Engineering II e Engineering Testing · · 2015-03-25Using UML, Patterns, and Java e Engineering!! Software Engineering II Testing 5 May 2009 Bernd Bruegge Applied Software