Upload
others
View
15
Download
0
Embed Size (px)
Citation preview
International Conference On
Software Testing, Analysis & ReviewNovember 19 - 23 Stockholm, Sweden
P r e s e n t a t i o n
T3Thursday 22nd November, 2001
System Testing Process inIterative Environment
Salia S Laitinen
Thursday 22 November 2001 T3
System Testing Process in an Iterative Development Environment Saila Laitinen •Education: M.Sc. (Comp. Eng.), University of Oulu, Finland Lic. Tech. student in Helsinki University of Technology •Working experience: –1994-1995 R&D trainee in a small medical technology company –1995-1998 Trainee / Software engineer in Nokia Telecommunications –1998-2000 Group Manager in Nokia Networks, –01/2001-06/2001 System testing manager in Nokia Ventures Organization –07/2001-> Testing Project manager in Nokia Mobile Phones.
1 © NOKIA EuroStarPresentation.PPT/SLa
���������������� ������������������������ ������������������������ ������������������������ ��������������������������������������������������������������������������������
� ����������
� ���
� �������
� �������
� �� ��� ������� �
� ���������
� �� ������ ��������
� �� ���������
� !�� ��� �"��� ���
� ����������
� ���
� �������
� �������
� �� ��� ������� �
� ���������
� �� ������ ��������
� �� ���������
� !�� ��� �"��� ���
2 © NOKIA EuroStarPresentation.PPT/SLa
Introduction• Speaker: Saila Laitinen• Education: M.Sc. (Comp. Eng.), University of Oulu, Finland
Lic. Tech. student in Helsinki University of Technology
• Working experience:– 1994-1995 R&D trainee in a small medical technology
company– 1995-1998Trainee / Software engineer in Nokia
Telecommunications– 1998-2000 Group Manager in Nokia Networks,– 01/2001-06/2001 System testing manager in Nokia Ventures
Organization– 07/2001-> Testing Project manager in Nokia Mobile Phones.
3 © NOKIA EuroStarPresentation.PPT/SLa
Facts Problems• BU was established in
order to ’create’ a totally new product (1999)
• Highly experienced management and software engineers
• Young and inexperienced (yet eager) integration & testing stuff
• We were on a fragile ice
• Management had some conflicting assumptions on how-to-do things.
• Low competence in product testing
4 © NOKIA EuroStarPresentation.PPT/SLa
Facts / 2 Problems / 2• Bad experience from the
previous project (prototype)• Bad quality
• No documentation
• No generally accepted rules for how to do things.
• Committed management
• People were critical about everything<-> they were open to any suggestions
• ’Jungle-rules’ applied
5 © NOKIA EuroStarPresentation.PPT/SLa
Problems Medication• We were on a fragile ice
• Management had some conflicting assumptions on how-to-do things.
• Low competence in product testing.
• Studying new technologies/strategy work (WAP, WLan, Bluetooth,…)
• Communication, communication, communication and once again communication!
• Arranged a private tailor-made software testing seminar for the personnel
6 © NOKIA EuroStarPresentation.PPT/SLa
Problems/ 2 Medication / 2• People were critical
about anything. <->They were open to any suggestions.
• Jungle-rules applied
• Talking with the people. Asking their opinions and solution suggestions
• Investigation of the tools available
7 © NOKIA EuroStarPresentation.PPT/SLa
Medication Results• Studying new
technologies/strategy work (WAP, WLan, Bluetooth,…)
• Communication, communication, communication and once again communication!
• Arranged a private tailor-made software testing seminar for the personnel
• General understanding of the product
• Less misunderstandings
• Software Testing competence increased, the role of testing was understood
8 © NOKIA EuroStarPresentation.PPT/SLa
Medication / 2 Results / 2• Talking with the people.
Asking their opinions and solution suggestions
• Investigation of the tools available
• Getting to know other people in the same organization
• ReqPro & ClearCase & ClearQuest from the Rational
9 © NOKIA EuroStarPresentation.PPT/SLa
ResultsHow long it took to get results
• General understanding of the product
• Less misunderstandings
• Software Testing competence increased, the role of testing was understood
• 1-2 months
• Continuous
• 1 week course and lots of reading
10 © NOKIA EuroStar Presentation.PPT/SLa
Results / 2How long it took to get results / 2
• Getting to know other people in the same organization
• ReqPro & ClearCase & ClearQuestfrom the Rational
• Continuous
• Basic knowledge already existed
11 © NOKIA EuroStar Presentation.PPT/SLa
The V-Model
R e q u ire m e n ts p e c if ic a t io n
(U s e C a s e l is tF u n c . s p e c .)
S y s te m d e s ig n(A rc h ite c tu a l
d e s ig nin te rn a l in te r fa c e
s p e c .)
D e ta ile d d e s ig n(M o d u le d e s c .)
C o d e
M o d u le T e s tE x e c u t io n
In te g ra t io n T e s tE x e c u t io n
S y s te m T e s tE x e c u t io n
R e q u ire m e n t'ste s tin g
S ys te nd e s ig nte s tin g
D e s ig nte s tin g
S ys te m T e s t C a se d e s ig n
In te g ra tio n T e s t C a se D e s ig n
M odule TestCase Design
12 © NOKIA EuroStar Presentation.PPT/SLa
Integration and system testing• Extreme coding
• Integration testing in a ‘medium’ (big)
• Exit criteria for the IT
• Entry criteria for the system testing
• A person called ’Integrator works with each two coders and is responsible for:
• integration between two modules
• System level integrators do integration testing and are responsible for:
• IT of all existing interfaces between modules
• Integration test manager provides a tested system, which fulfils the exit criteria for SYTE people
13 © NOKIA EuroStar Presentation.PPT/SLa
System Testing
14 © NOKIA EuroStar Presentation.PPT/SLa
System Testing /2
TO DOTO DO NOT TO DONOT TO DO
Find faults not only in the system but also in the specifications
Create confidence
Create a strong process
Save time and money but NOT in the expense of the quality
Have any useless bureaucracy
Create a strict process, which does not allow the use of a common sense
15 © NOKIA EuroStar Presentation.PPT/SLa
HOW?• By having a plain and clear instructions (general policy)
• By having willingness to change the instructions if needed
• By having willingness to change own habits, and attitudes if needed
• By gathering experience in how to design test cases, which really do find faults
16 © NOKIA EuroStar Presentation.PPT/SLa
How? / 2• By having strong enough management, who knows that ”no authorities-no
responsibilities”
• By continuous competence development
• By remembering what is our main goal: developing a fault poor product
All test cases are prioritised so that whenever the product isdecided to be released, we can be sure that we have done the besttesting in the time available. [ISEB SW Testing Certificate Training,Grove Consultant, UK]
17 © NOKIA EuroStar Presentation.PPT/SLa
Testing phases• Documentation testing
• The idea of documentation testing is to eliminate collisions within one product description or between two or more product descriptions. //[Chechik Marshal, Gannon John (1994) Automatic Verification of Requirement Implementation. In: International Symposium of Software Testing and Analysis, August 17-19, Seattle, Washington, USA, 2. 1-5, New York]
• Functional testing, based of use cases• Prioritised• No 1 priority cases = Smoke test• No 2 priority cases = Base functional testing• No 3 priority cases = Executed if schedule allows
• State transition testing• Based on state transition diagrams• Prioritised
• Usability testing• High level usability observation first
• Regression testing• Planning just before executing
• Acceptance testing• Product validating
18 © NOKIA EuroStar Presentation.PPT/SLa
Documentation testing in a V-ModelR e q u ire m e n ts p e c if ic a t io n
(U s e C a s e l is tF u n c . s p e c .)
S y s te m d e s ig n(A rc h ite c tu a l
d e s ig nin te rn a l in te r fa c e
s p e c .)
D e ta ile d d e s ig n(M o d u le d e s c .)
C o d e
M o d u le T e s tE x e c u t io n
In te g ra t io n T e s tE x e c u t io n
S y s te m T e s tE x e c u t io n
R e q u ire m e n t'ste s tin g
S ys te nd e s ig nte s tin g
D e s ig nte s tin g
S ys te m T e s t C a se d e s ig n
In te g ra tio n T e s t C a se D e s ig n
M odule TestCase Design
D ocum enta t ion tes ting
D ocum enta tion tes ting
In te g ra to rs
19 © NOKIA EuroStar Presentation.PPT/SLa
More problems• The process is quite confusing• Many parallel activities• No need for useless documentation
Unavoidable management
20 © NOKIA EuroStar Presentation.PPT/SLa
The process model
T im e scale
T est P lanning
Feature lis t
T est P lanning T est P lanningT est P lanningT est P lanning
T est scenariodes ign
H W /SW R eq.Spec.
T est casedes ign
Im plem enting
T est Execu tionand reporting
T est Execu tionand reporting
T est Execu tionand reporting
T est Execu tionand reporting
T est Execu tionand reporting
C orrec tiveim plem enting
T est scenariodes ign
T est scenariodes ign
T est scenariodes ign
T est scenariodes ign
T est casedes ign
T est casedes ign
T est casedes ign
T est casedes ign
C hange C ontro l B oard
Pro
ject
Man
agem
ent
Arc
hite
ctua
l des
ign
F ea ture lis t Fea ture lis tFea ture lis tFea ture lis t
F irs t R eleaseR eady
R elease T esting
P rogram S teering G roup
Second R eleaseR eady
R elease T esting
Im plem entingIm plem entingIm plem entingIm plem enting
C orrec tiveim plem enting
C orrec tiveim plem enting
C orrec tiveim plem enting
C orrec tiveim plem enting
H W /SW R eq.Spec.
H W /SW R eq.Spec.
H W /SW R eq.Spec.
H W /SW R eq.Spec.
Change Control Board reviews test reports, decides about changes in the productdecides when the product is released
Project Management does the documentation testing, manages resources
Program steering group makes the market analysis
21 © NOKIA EuroStar Presentation.PPT/SLa
Case Study
• The business program was closed right after the first experimental project was started:
• No real data available
22 © NOKIA EuroStar Presentation.PPT/SLa
Q & A