UML Testing Profile

© Telelogic AB

Graphical Testing With UMLThe UML Testing Profile

Eric Samuelssoneric.samuelsson@telelogic.com

UML Testing Profile

© Telelogic AB

Agenda

• Background

• Key concepts

• Advanced concepts

• Summary

UML Testing Profile

© Telelogic AB

Background

• Test technologies are not on par with modern design technologies such as model-driven architectures (MDA)

• There’s a need for a widespread graphical test language

• OMG realized this and issued and RFP for a graphical testing profile to UML

– Using the UML ensures widespread use across a wide range of application areas

– Numerous tool vendors support UML

– UML is a proven de facto standard

– But UML is focused on definition of system structure and behavior and has no built-in testing constructs...

UML Testing Profile

© Telelogic AB

UML Testing Profile RFP

• Issued by OMG in July 2001

• LOI’s submitted by

– Telelogic, Rational, SofTEAM, FOKUS, Ericsson, Motorola, IBM

• One joint submission

• Initial submission made in April 2002

• Revised initial submission made in June 2002

– Terminology changes

– Elaboration of Test Data

• Final submission in January 2003

UML Testing Profile

© Telelogic AB

Requirements and restrictions

• Define structural and behavioral test concepts

• Conformance testing only

– Black-box functional testing

• Based upon the UML meta-model

– UML 2.0 as proposed by the U2 Partners

– Inter-operation with existing testing technologies

– TTCN 3

– Junit

– ...

UML Testing Profile

© Telelogic AB

Current proposal

• A graphical testing language based the current UML 2 metamodel

• Reusing as many UML concepts as possible

– Recognition important for UML users

– No need to re-invent the wheel

• Designed with TTCN 3 in mind

– Semantics basically follows TTCN 3 semantics

– But caters for other technologies like JUnit as well

UML Testing Profile

© Telelogic AB

Current proposal

• Defined as a UML profile

– Will be recognized by UML users

• Basically a number of stereotypes

– Easy to implement for tool vendors

• Models will be interchangeable

– Any tool implementing XMI will be able to exchange models

«profile»Testing Profile«import»

UML

«stereotype»TestCase

«metaclass»BehavioralFeature

UML Testing Profile

© Telelogic AB

Key Concepts

• Test Architecture

– Test Context, Test Configuration

– Test Component, SUT

– Arbiter

• Test Behavior

– Test Case

– Verdict, Validation Action

– Defaults

• Test Data

– Logical partitions, Pattern matching

• Test Deployment

UML Testing Profile

© Telelogic AB

Test Objective

• An informal description of what to test in one test case

• Commonly expressed using natural language

• Can also be expressed using other diagrams, such asuse case diagrams and sequence diagrams

”Verify that if a user inserts and authorizes a valid card correctly, he is able to withdraw money if he has sufficient funds.”

Client ClientWithdrawal

InvalidPinTester

<<include>>

UML Testing Profile

© Telelogic AB

Test Case

• A specification of one test, including

– Test objective

– Test behavior

• Defined as a BehavioralFeature in UML

• It’s method is the behavior of the test case

«testCase» +validWithdrawal() : Verdict

«testContext»ATMContext

«testCase»+validWithdrawal() : Verdict+invalidPIN() : Verdict-authorizeCard() : Verdict

UML Testing Profile

© Telelogic AB

Test Behavior

• Describes the expected behavior of a test case

• Defined using any type of UML behavior

– Interactions, statemachines and activities

• Facilitates specification of arbitrarily complex behavior

sd ATMContext

invalidPIN()refref

[verdict == fail][verdict == pass]

validWithdrawal()refref

storeCardData(currentCard)

isPinCorrect(enteredPIN)

true

display(”Enter PIN”)

«sut»atm

hwe

Idle

WaitCode

UML Testing Profile

© Telelogic AB

Test Architecture

• A complete specification of all types used in the tests

– An ordinary UML package

– Contains Test Contexts, Test Components, Ports, Interfaces, etc.

«testArchitecture»

ATMTest

UML Testing Profile

© Telelogic AB

Test Architecture Example

«testArchitecture»

ATMTest

«testContext»ATMContext

«testCase»+validWithdrawal() : Verdict+invalidPIN() : Verdict-authorizeCard() : Verdict

Account

balance : Integernumber : String

credit(a : Integer)debit(a : Integer)

* accounts

ATM::CardData

pinCode : Integernumber : String

isPinCorrect(c: Integer) : Boolean

*

cards

«testComponent»BankEmulator

IAccount

bePort-pinOk : Boolean-enteredPIN : String-message : String

«testComponent»HWEmulator

hwCom

IScreen, ICardMachine,IMoneyBox

IATM

UML Testing Profile

© Telelogic AB

Test Component

• Implements (parts of ) the behavior of one or more test cases

• Each test case is normally executed by several components

• Ordinary classes or components

-pinOk : Boolean-enteredPIN : String-message : String

«testComponent»HWEmulator

hwCom

ATM::CardData

pinCode : Integernumber : String

isPinCorrect(c: Integer) : Boolean

0..1 -currentCard

IScreen, ICardMachine, IMoneyBox

IATM

UML Testing Profile

© Telelogic AB

Test Context

• A classifier that contains

– a number of test cases and their behavior and objectives

– default behavior used by the test cases

– configuration of test components

– test control behavior

• Can be a class, a component or a collaboration«testContext»ATMContext

«testCase»+validWithdrawal() : Verdict+invalidPIN() : Verdict-authorizeCard() : Verdict

UML Testing Profile

© Telelogic AB

Test Configuration

• A specification of the structural aspects of test cases

• Defines the parts participating in the test case and their interconnections

– Test Components, SUT, Ports, Interfaces and Connectors

• Defined as the internal structure of the Test Context

«sut»atm : BankATM

hwe : HWEmulator

be : BankEmulator

atmPort

hwCom

bePortnetCom

accnts : Account [0..*]

UML Testing Profile

© Telelogic AB

Test Control

• Test execution can be controlled is two ways

– By defining the classifier behavior of a test context

– Or by invoking the test cases from outside of the context

UML Testing Profile

© Telelogic AB

Test Control by ClassifierBehavior

sd ATMContext

invalidPIN()ref

[verdict == fail][verdict == pass]

validWithdrawal()ref

«testContext»ATMContext

«testCase»+validWithdrawal() : Verdict+invalidPIN() : Verdict-authorizeCard() : Verdict

UML Testing Profile

© Telelogic AB

Test Control by Invocation

*

-context

MyClass

-verdict : Verdict

«testContext»MyTestContext

«testCase»+TC1() : Verdict +TC2() : Verdict -TC3() : Verdict

ad myOp

verdict := context.TC1()

[else][verdict == pass]

verdict := context.TC2()

+myOp()

UML Testing Profile

© Telelogic AB

Test Data

• Parameters, default values, return values are reused from UML without changes

• Logical partitions

• Pattern matching

• TDB

UML Testing Profile

© Telelogic AB

Test Deployment

• Test Deployment is important, but there is probably no difference between deploying test systems compared to ordinary UML systems.

• We will try to reuse the existing UML deployment concepts without changes

• Under investigation

UML Testing Profile

© Telelogic AB

Advanced Concepts

• Test Evaluation

– Verdict

– Validation action

– Arbiter

• Default behavior

• Structural and behavioral reuse

UML Testing Profile

© Telelogic AB

Test Evaluation

• A number of concepts are used to evaluate test results

– Verdicts

– Validation actions

– Arbiter

UML Testing Profile

© Telelogic AB

Verdicts and Validation Actions

• Each test component is associated with a verdict

• Validation Actions are used to set the verdict

• A Validation Action sets the verdict of a test component and reports the verdict to the Arbiter

Client«sut»ATM

storeCardData(cardData)

isPinCorrect(pin)

true

display(”Enter PIN”)

alt

false

setverdict(fail)

UML Testing Profile

© Telelogic AB

Arbiter

• A test component that knows how to evaluate the test behavior

• Separates test evaluation from test case specification

– One behavior can yield different results

• Each test context has a default arbiter

– Implicit connections to all test components

– TTCN 3 semantics

• Advanced arbiters can be defined

– Caters for more advanced semantics

«arbiter»myArbiter

setVerdict(v : Verdict)

UML Testing Profile

© Telelogic AB

Default behavior

• Defines behavior for unexpected observations

• Can be expressed using any UML behavior

• Can be applied to several different levels:

– Test component

– Message reception

– Alternative statement

– ...Client

«sut»ATM

storeCardData(cardData)

isPinCorrect(pin)

true

display(”Enter PIN”)

sd authorizeCard

enterPinDefault

UML Testing Profile

© Telelogic AB

Default expressed as interaction...

sd enterPinDefault

self

alt getStatus

statusOk

ejectCard()

display(”Connection lost”)

setverdict(pass)

setverdict(inconc)

setverdict(fail)

UML Testing Profile

© Telelogic AB

...and as statemachine

The representations are semantically equivalent

enterPinDefault

*

getStatus / ^statusOk, setverdict(pass);

display(”Connection lost”) /setverdict(inconc)

ejectCard /setverdict(fail)

UML Testing Profile

© Telelogic AB

Meta Default

• Each test component has a meta default to cater for differences in semantics

• Can be redefined

MetaDefault

*

* / defer

UML Testing Profile

© Telelogic AB

Reuse of Test Structure and Behavior

UML Testing Profile

© Telelogic AB

«testContext»ATMContext

«sut»atm : BankATM

hwe : HWEmulator

be : BankEmulator

atmPort

hwCom

bePortnetComaccnts : Account [0..*]

ATM

Client

ac: AuthorizationContext

UML Testing Profile

© Telelogic AB

sd ValidWithdrawal

hwe«sut»atm be

refac.authorizeCard()

selectOperation(withdrawal)

true

withdraw(amount)

netCom

findAccount(cardData)

account

display(”Take cash”)

deliverMoney(amount)

true

debitAccount(account, amount)

true

atmPort

UML Testing Profile

© Telelogic AB

Summary

• The testing profile enables complete graphical specification of tests for conformance testing of UML models

– Defined as a UML profile

– Reuses existing UML concepts whenever possible

– Designed for inter-operation with TTCN-3 and other technologies

• Due date

– January 6th, 2003

• More information

– Testing Profile Consortium

– OMG’s Testing Profile page