Upload
lilka
View
31
Download
1
Tags:
Embed Size (px)
DESCRIPTION
CoWTeSt: A Cost Weigh t ed Test Strateg y. Test Planning Strategy. Objectives: Decide which and how many test cases should be executed. Give a practical help to managers to support test planning and evaluate the impact of testing the phase on the cost of the final product. Features. - PowerPoint PPT Presentation
Citation preview
CoWTeSt: A Cost
Weighted Test Strategy
Test Planning StrategyObjectives:Objectives:
Decide which and how many test cases Decide which and how many test cases should be executed.should be executed.
Give a practical help to managers to Give a practical help to managers to support test planning and evaluate the support test planning and evaluate the impact of testing the phase on the cost of impact of testing the phase on the cost of the final product.the final product.
Features (Exclusively) UML based: It uses the Use Cases (Exclusively) UML based: It uses the Use Cases
and Interaction diagrams developed during and Interaction diagrams developed during analysis and design phasesanalysis and design phases
Cowtest strategyCowtest strategy: : development of a systematic development of a systematic method for UML design-based integration test of method for UML design-based integration test of complex system.complex system.
Based exclusively on the use of UML diagrams: Based exclusively on the use of UML diagrams:
• immediately usable from industries already using UML immediately usable from industries already using UML
(no additional modeling or design costs); (no additional modeling or design costs); • analysis can be started soon, and done contemporarily analysis can be started soon, and done contemporarily
with project development; with project development;
DiDifferent strategies for test cases fferent strategies for test cases sselectionelection
Presentation Scheme
Test caseTest case
Test case
Test case
Test caseTest case
Test case
Test case
Test case
Test case
Test case
Test case
Test case
Test case
Test case
Test case
Obj 1 Obj 2 Obj 3
Method1()
Method3()
Method2()
UUII
TT
CoWTeStCoWTeSt
CoWTeSt: A Cost Weighted Test Strategy
Weighted Tree Derivation:Weighted Tree Derivation:1.1. Starting from the main UCD the UCs and Starting from the main UCD the UCs and
SDs are organized in a sort of hierarchical SDs are organized in a sort of hierarchical treetree
2.2. Deduce the Critical Profile: annotate Deduce the Critical Profile: annotate every arch with a value representing every arch with a value representing ““the the importanceimportance”” of the associate node (be i of the associate node (be itt a a UC or a SD scenarioUC or a SD scenario))
3.3. Test Case Derivation: applTest Case Derivation: apply y UIT for test UIT for test derivationderivation
SiteKeeper_SD
CallManagement_SD
ConnDisc_SD...
ConnDisc_SD...
NAM NRM
DiscofConn Conn
SK
REM C
CM
TcSOcS
S
OrigCall_SD...
OrigCa...
Connect_SD
Setup_SD
0.050.05 0.6
0.3
0.1 0.3 0.30.1
0.2
0.4 0.40.2 0.45
0.450.1
SiteKeeper_SD
CallManagement_SD
ConnDisc_SD...
ConnDisc_SD...
NAM NRM
DiscofConn Conn
SK
REM C
CM
TcSOcS
S
OrigCall_SD...
OrigCa...
Connect_SD
Setup_SD
0.050.05 0.6
0.3
0.1 0.3 0.30.1
0.2
0.4 0.40.2 0.45
0.450.1
Integration StageThe The first integration stagefirst integration stage is represented by the main is represented by the main
UC and the SDs (if any), which are children of this UC and the SDs (if any), which are children of this node (hence they are at level 2 of the tree).node (hence they are at level 2 of the tree).
TheThe i-th integration stagei-th integration stage is represented by the UCs is represented by the UCs positioned in at positioned in at i-thi-th level of the tree and every SDs, level of the tree and every SDs, children of these nodes, situated at children of these nodes, situated at i+1i+1-th level.-th level.
NOTE: In the NOTE: In the i-thi-th integration stage the SDs at level integration stage the SDs at level i+1 i+1 are are consideredconsidered,, because they represent the interaction between because they represent the interaction between the different components that realize the functionalities the different components that realize the functionalities described in the UCs at described in the UCs at i-th i-th level of the tree.level of the tree.
SiteKeeper_SD
CallManagement_SD
ConnDisc_SD...
ConnDisc_SD...
NAM NRM
DiscofConn Conn
SK
REM C
CM
TcSOcS
S
OrigCall_SD...
OrigCa...
Connect_SD
Setup_SD
0.050.05 0.6
0.3
0.1 0.3 0.30.1
0.2
0.4 0.40.2 0.45
0.450.1
3rd Integration Stage
Test Case SelectionFor every node, the product of all the nodes weights For every node, the product of all the nodes weights
on the complete path from the root to this node on the complete path from the root to this node represents its represents its final weight..
Int-Stage Leaves names Critical profile 2nd Stage/NTest 3rd Stage/NTest 4th stage/NTest
1st Stage SK 1 SK_SD 0.3 0.3 150 0.3 150 0.3 150SK_NDchild 0.7 - - - - - -
2nd Stage NAM 0.05 0.05 25 0.05 25 0.05 25NRM 0.05 0.05 25 0.05 25 0.05 25CM 0.6 - - - - - -CM_SD 0.2 0.12 60 0.12 60 0.12 60CM_NDchild 0.8 48 240 - - - -
3rd Stage EM 0.1 0.06 30 0.06 30S 0.3 - - - -S_SD 0.2 0.036 18 0.036 18S_NDchild 0.8 0.144 72 - -C 0.3 - - - -C_SD 0.10 0.018 9 0.018 9C_NDchild 0.90 0.162 81 - -R 0.1 0.06 30 0.06 30
4th Stage OcS 0.4 0.072 36TcS 0.4 0.072 36DiscofConn 0.45 0.081 41Conn 0.45 0.081 41
Test Case Selection
Leaves names Critical profile 2nd Stage/NTest 3rd Stage/NTestSK 1 SK_SD 0.3 0.3 150 0.3 150SK_NDchild 0.7 - - - -NAM 0.05 0.05 25 0.05 25NRM 0.05 0.05 25 0.05 25CM 0.6 - - - -CM_SD 0.2 0.12 60 0.12 60CM_NDchild 0.8 48 240 - -EM 0.1 0.06 30S 0.3 - -S_SD 0.2 0.036 18S_NDchild 0.8 0.144 72C 0.3 - -C_SD 0.10 0.018 9C_NDchild 0.90 0.162 81R 0.1 0.06 30OcS 0.4 TcS 0.4
SiteKeeper_SD
CallManagement_SD
ConnDisc_SD...
ConnDisc_SD...
NAM NRM
DiscofConn Conn
SK
REM C
CM
TcSOcS
S
OrigCall_SD...
OrigCa...
Connect_SD
Setup_SD
0.050.05 0.6
0.3
0.1 0.3 0.30.1
0.2
0.4 0.40.2 0.45
0.450.1
80% Functional Coverage
Leaves names 4th Stage weights
70%coveragenwf /MinNtest
80%coverage/nwf /MinNtest
90%coverage/nwf /MinNtest
100%coverage/nwf /MinNtest
SK_SD 0.3 0.413 5 0.354 5 0.137 6 0.3 17
CM_SD 0.12 0.165 2 0.141 2 0.126 3 0.12 7
DiscofConn 0.081 0.111 2 0.95 2 0.085 2 0.081 5
Conn 0.081 0.111 2 0.095 2 0.085 2 0.081 5
OcS 0.072 0.1 1 0.085 2 0.076 2 0.072 4
TcS 0.072 0.1 1 0.085 2 0.076 2 0.072 4
EM 0.06 0.071 1 0.063 2 0.06 4
R 0.06 0.071 1 0.063 2 0.06 4
NAM 0.05 0.052 1 0.05 3
NRM 0.05 0.052 1 0.05 3
S_SD 0.036 0.036 2
C_SD 0.018 0.018 1
Total 72.3%
13 84.6%
17 94.6%
23 100% 59
Selection of leaves for different coverage levels
Leaves names 4th Stage weights
70%coveragenwf /MinNtest
80%coverage/nwf /MinNtest
SK_SD 0.3 0.413 5 0.354 5
CM_SD 0.12 0.165 2 0.141 2
DiscofConn 0.081 0.111 2 0.95 2
Conn 0.081 0.111 2 0.095 2
OcS 0.072 0.1 1 0.085 2
TcS 0.072 0.1 1 0.085 2
EM 0.06 0.071 1
R 0.06 0.071 1
NAM 0.05
NRM 0.05
S_SD 0.036
C_SD 0.018
Total 72.3% 13 84.6% 17
Selection of leaves for different coverage levels
Presentation Scheme
Test caseTest case
Test case
Test case
Test caseTest case
Test case
Test case
Test case
Test case
Test case
Test case
Test case
Test case
Test case
Test case
Obj 1 Obj 2 Obj 3
Method1()
Method3()
Method2()
UUII
TT
CoWTeStCoWTeSt
A method to systematicallyA method to systematically derive derive Integration Test suites from UML diagrams:Integration Test suites from UML diagrams:• Use case diagrams• Sequence (Interaction) diagrams• Class diagrams
Incremental test strategy (over the Use Case Incremental test strategy (over the Use Case organizational structure)organizational structure)
Seven steps involvedSeven steps involved
Use Interaction Test methodology
Step 1: UML design analysis and search of relevant Use Cases
•Use Case diagram of uci.argo.KERNEL
Step 2/1: Analysis of Sequence and Class diagrams involved in the selected Use Case
•Sequence diagram of uci.argo.Kernel.USERMODEL
Step 2/2: Class diagram analysis
Class diagram of uci.argo.Kernel.USERMODEL
Designer
Decision Model
_decisions
DecisionModel()GetDecisions()
GoalModel
_goals
GoalModel()GetGoals()
Decision
Decision(name:String;priority:int)GetName()GetPriority()
Goal
Goal(name:String;priority:int)GetName()GetPriority()
1:1..* uses1:1..* uses
*
contains
*
contains
Step 3 Test Units definition
Test UnitsTest Units:: those system those system units separately units separately testable for exercising a possible use of testable for exercising a possible use of system:system:
Test UnitsTest Units: : DecisionModel, Decision, GoalModel, DecisionModel, Decision, GoalModel,
Goal.Goal.
Step 4: Research of Settings and Interactions Categories- - SettingsCategoriesSettingsCategories:: parameters and inputs relevant parameters and inputs relevant in the Test Unit.in the Test Unit.
- - InteractionsCategoriesInteractionsCategories:: objects interactions, objects interactions, exchanged messagesexchanged messages..
Categories:Categories:
DecisionModelDecisionModelSettings: Settings: __decisionsdecisionsInteractions:Interactions: DecisionModel() DecisionModel() getDecisions()getDecisions()DecisionDecisionInteractions: Interactions:
Decision(name:String,priority:int)Decision(name:String,priority:int)GetName()GetName()GetPriority()GetPriority() ........................
Step 5: Test Specification construction
For each identified category, we consider all For each identified category, we consider all its possible values and constraints.its possible values and constraints.
Test Specification:DecisionModel:
Settings: Interactions: _decisions getDecisions()
Naming Opening a new fileStorage Opening a saved fileModularity After a modification and before saving
Inheritance After a modification and after saving
Stereotypes DecisionModel()Relationship.. Class constructor
Step 6: Search of Message Sequences and Test Case definition Message Sequences: set of Messages (in temporal
order) , involved in a SD, used by objects to define and elaborate specific functionalities.
Step 6: Search of Message Sequences and Test Case definition Test Case: is constructed, from a Test
Specification, for each possible combination of choices of every category involved in a Message Sequence
Test Case getDecisions()getName()/getPriority()
Opening a saved file_decisions
NamingNote: Such a Test Case must be repeated for all possible values of _decisions
Step 7: Definition of UseCase Test Suite and Incremental Construction of TestFrame
UseCase Test Suite Design Critics
Use Case Test Suite Kernel
UseCase Test Suite UserModel
UseCase Test Suite ToDoList
Test Frame uci.argo.Kernel
Cow_Suite: Cow pluS UIT Environment
Tool for automatingTool for automatingTest cases derivation, from UITTest cases derivation, from UITTest management strategy, from Test management strategy, from
COWTeSTCOWTeST Uses UML diagrams already developed for Uses UML diagrams already developed for
the analysis and design phase the analysis and design phase no no additional effort or information.additional effort or information.
Rational Rose environment Rational Rose environment
Cow_Suite Architecture Uses REI, Uses REI, Rational Rose Extensibily Interface.Rational Rose Extensibily Interface. Composed by six different components:Composed by six different components:
1.1. MDL AnalyserMDL Analyser2.2. Test Tree GeneratorTest Tree Generator3.3. Weights ManagerWeights Manager4.4. Test Case GeneratorTest Case Generator5.5. Test Case BuilderTest Case Builder6.6. Test Case CheckerTest Case Checker
Graphics interface: organized in three Graphics interface: organized in three tabs:tabs: CowTreeCowTree UIT StructureUIT Structure Test SpecificationTest Specification
1.MDL Analyser Analysis of the .mdl file of RoseAnalysis of the .mdl file of Rose Search the information necessary to apply Search the information necessary to apply
test selection strategy:test selection strategy: Actors and Use Cases;Actors and Use Cases; Sequence diagrams and their messages;Sequence diagrams and their messages; Classes with attributes and methods;Classes with attributes and methods; Activity diagramsActivity diagrams
The information is passed as the input to the The information is passed as the input to the next component.next component.
2.Test Tree Generator All the UCDs and SDs from .mdl file are All the UCDs and SDs from .mdl file are
organized in a organized in a hierarchical treehierarchical treeusing the explicit diagrams links or using the UC using the explicit diagrams links or using the UC associations and classes relationsassociations and classes relations
To To each node each node we associate:we associate: Level identification number;Level identification number; Default weight Default weight
such that the sum of this weight plus the such that the sum of this weight plus the weightsweights
associated to all its brothers is equal to 1. associated to all its brothers is equal to 1.
3.Weights Manager Interacts with the user for:Interacts with the user for:
a)a) Assigning the real weight to each node:Assigning the real weight to each node:– Selecting directly a nodeSelecting directly a node– Selecting a level numberSelecting a level number Check Check that the sum of the user assigned weights of a that the sum of the user assigned weights of a
level nodes is equal to 1.level nodes is equal to 1.
b)b) Choosing the criterion for test case selection:Choosing the criterion for test case selection:– Fixing the maximum number of test cases to be Fixing the maximum number of test cases to be
executed executed – Fixing the percentage of functionalities to be covered.Fixing the percentage of functionalities to be covered.
4.Test Case Generator queries the user for the deepest integration level queries the user for the deepest integration level
he/she is interested in and calculate the he/she is interested in and calculate the final final weightweight of every leaf. of every leaf.
implements the implements the UITUIT method for test generation method for test generation associates to each SD its test cases organized in a associates to each SD its test cases organized in a
hierarchicalhierarchical manner. manner. calculates and visualizes the calculates and visualizes the number of test cases number of test cases
for each SDfor each SD depending on the chosen selection depending on the chosen selection criterion.criterion.
associates to each SD the associates to each SD the framesframes of the test cases of the test cases that should be instantiated. that should be instantiated.
Test Case FrameTEST CASE 8.2TEST CASE 8.2Description:Description:PreCondition:PreCondition:
Test Case 7Test Case 7 Flow of event:Flow of event:getEnterpriseForSourceAddress(caller)getEnterpriseForSourceAddress(caller)
[else][else]routingResult =doLRQ(caller, callee, Enterprise, routingResult =doLRQ(caller, callee, Enterprise, callcase,AccessAgent, BGAResult)callcase,AccessAgent, BGAResult)
CategoriesCategories::SettingsCategories:SettingsCategories:PEnterprise =PEnterprise =Callee =Callee =BGAResult ……BGAResult ……
InteractionsCategories:InteractionsCategories:getEnterpriseForSourceAddress =getEnterpriseForSourceAddress =doLRQ =doLRQ =PostCondition:PostCondition:Comment:Comment:
5.Test Case Builder Interaction with the user for test case Interaction with the user for test case
implementation:implementation:For the necessary parameters;For the necessary parameters;Adding and removing categories;Adding and removing categories;Changing operations value or even Changing operations value or even
test cases structure.test cases structure. The changes involving the UML design are The changes involving the UML design are
finally saved in a new file.finally saved in a new file.
6. Test Case Checker The tool maintains information about the test The tool maintains information about the test
cases generation. cases generation. This module will realize the This module will realize the comparison of comparison of
different versionsdifferent versions of the same .mdl file. of the same .mdl file.
The discovered differences, like, for example, the The discovered differences, like, for example, the existence of new test cases or changes in those existence of new test cases or changes in those
already already generated, are saved into a separate file.generated, are saved into a separate file.
The evaluation of the cost and impact required for The evaluation of the cost and impact required for the updates to the test plan with respect to the the updates to the test plan with respect to the “official version” is derived analyzing this file. “official version” is derived analyzing this file.
References:
F. Basanieri, A. Bertolino, “A Practical Approach F. Basanieri, A. Bertolino, “A Practical Approach to UML-based Derivation of Integration Tests”, to UML-based Derivation of Integration Tests”, QWE2000, Bruxelles, 20-24 November, 3TQWE2000, Bruxelles, 20-24 November, 3T..Basanieri, F., Bertolino, A., Marchetti, E., Basanieri, F., Bertolino, A., Marchetti, E., “C“CoWTeSt: A Cost Weighed Test StrategyoWTeSt: A Cost Weighed Test Strategy”, ”, accepted for: accepted for: ESCOM-SCOPE 2001, London, ESCOM-SCOPE 2001, London, EnglandEngland,, 2-4 April 2001.2-4 April 2001. Basanieri, F., Bertolino, A., Marchetti, E.Basanieri, F., Bertolino, A., Marchetti, E.,, RiboliniRibolini,, A., “An Automated Test Strategy Based A., “An Automated Test Strategy Based on UML Diagrams” submitted toon UML Diagrams” submitted to:: ICSE 2001 ICSE 2001 WAPATV WAPATV woworkshop.rkshop.