Upload
elias-nogueira
View
1.864
Download
2
Tags:
Embed Size (px)
DESCRIPTION
Palestra de Neli Duarte da IBM que teve sua apresentação dia 07/07/2011 no TDC 2011 - The Developer's Conference na Trilha de Teste
Citation preview
‹footer› ‹#›
Click to edit Master title style
� Click to edit Master text styles
�Second level
� Third level
� Fourth level
� Fifth level
‹date/time›
Exploratory Tests:
Is this really effective?Author:
Neli Duarte
‹footer› ‹#›
Click to edit Master title style
� Click to edit Master text styles
�Second level
� Third level
� Fourth level
� Fifth level
‹date/time›
Agenda
� Overview � Exploratory tests definitionS�What really are exploratory testing?� Exploratory X Ad Hoc?
� Apply the method: exploratory techniques� Positive & Negative points� Case study� Conclusion
TDC-SP/ Jul 2011
2
‹footer› ‹#›
Click to edit Master title style
� Click to edit Master text styles
�Second level
� Third level
� Fourth level
� Fifth level
‹date/time›
Overview
� Not strictly defined
� Suitable concept about exploratory testing
� Manager’s feeling� Don’t have formal documentation� “Poking around” on application is a exploratory technique
� Exploratory tests is just different way to executing tests but it doesn’t means that it is a messy!!!
� It is a pretty effective method to test software.
� “Exploratory” is very embracing word and it is also applied on technical side; See next…
TDC-SP/ Jul 2011
you can use “exploratory” way on performing tests like ad hoc or other one, you are
really “exploring” the software and you are able to call it as exploratory testing, but
you can’t say that exploratory testing are ad hoc tests because they are completely
different.
Not different than others aspect, exploratory testing is one more thing that is not
strictly defined yet. That’s why I am writing it, to explain the suitable concept about
exploratory testing and help you to understand; although this can also been
improved and changed further. =)
3
‹footer› ‹#›
Click to edit Master title style
� Click to edit Master text styles
�Second level
� Third level
� Fourth level
� Fifth level
‹date/time›
Exploratory tests definitionS
� “What’s the big deal? Exploratory testing is random pounding on the keys. Nothing to it. My toddler does it every day.”[1]
� … “Oh, Exploratory Testing,” said one of the developers, “that’s where the tester does a bunch of wacky, random stuff, right?” … [2]
� Exploratory, or ad hoc testing, can be an especially useful testing strategy… [3]
� "Exploratory testing is an interactive process of concurrent product exploration, test design and test execution.” James Bach, Satisfice (2001)
**Different literature defines exploratory tests in a different concepts
TDC-SP/ Jul 2011
It is just to show up that different literature defines exploratory tests in a different
concepts, people think different and defined their own perspective regarding topic. I
am not trying to dictate the right concept, I am just telling you the most suitable
concept to define exploratory test allowing for “exploratory” is pretty embracing
word.
4
‹footer› ‹#›
Click to edit Master title style
� Click to edit Master text styles
�Second level
� Third level
� Fourth level
� Fifth level
‹date/time›
What really are exploratory testing?
What is NOT exploratory tests?
� Test type (system…)
� Test phase (design, execution…)
� Pocking around
� Test Approach (white…)
� Test Strategy *
TDC-SP/ Jul 2011
It is another test execution method!!! It is another test execution method!!!
Test strategy: ET can be part of the strategy
Test Phase: Planning, analyses, design, execution, report and so on.
Test Type: It is the tests applied according to the development phase.
(component, system, integration, sanity, UAT and so on)
Test Approach: White, Black or Grey box testing.
Test Strategy: Besides it is a part of the test strategy, ET by himself is
not a complete test strategy.
“Pocking around”: Definitely poking around is not any testing
technique.
5
‹footer› ‹#›
Click to edit Master title style
� Click to edit Master text styles
�Second level
� Third level
� Fourth level
� Fifth level
‹date/time›
Exploratory X Ad Hoc?
� No, they aren’t against
TDC-SP/ Jul 2011
** Both� Independent of formal documentation to be executed� Are related to the tester’s creativity
So, Ad hoc testing can differ from Exploratory by the lack of
previous planning, but it does not mean that it is a worthless
technique of testing. If fact, we can think as ad hoc as a part of
the exploratory testing, once that you are creating your test as
long you are running it.
We dare to establish that: all ad hoc testing are exploratory
testing, but not all exploratory testing are ad hoc.
‹footer› ‹#›
Click to edit Master title style
� Click to edit Master text styles
�Second level
� Third level
� Fourth level
� Fifth level
‹date/time›
Apply the method: exploratory techniquesAd HocAd Hoc
� No planning neither structure� Bugs are found on by accidentally and it is hard to reproduce � Is kind of exploratory testing but Exploratory Testing is not Ad Hoc
� So, is there any scenario to apply Ad Hoc tests?� Yes, there are!!� Software’s stability � Get knowledge about new software: � Hands on by Test Lead / Test Manager� Tests blocked � Developers before delivery code
� Apply it, depends on the defined strategy. Testing software is a creative area and these examples are just to give you ideas to define the best strategy
� Bug’s traceability: well reported
TDC-SP/ Jul 2011
Ad Hoc tests are executed without any plan or structure. Tester just open the application and going through, browsing according to his creative. Bugs are found on by accidentally and it is hard to reproduce due to don’t follow any structure.
Software’s stability: After code freeze in the development phase or as
part of regression tests, Ad Hoc might be a good practice to help on
test software’s stability due to force and browsing in the software
without plan or pre-requirement.
Get knowledge about new software: When new resource is joined to
the team and if some bug is found, it is additional on learning process.
Hands on by test lead: Test leaders/Test Manager tend to don’t test
software in details as Testers does, in this case Test Lead can
performing Ad Hoc test to get feeling about the software that he is
responsible.
7
‹footer› ‹#›
Click to edit Master title style
� Click to edit Master text styles
�Second level
� Third level
� Fourth level
� Fifth level
‹date/time›
Apply the method: exploratory techniquesChecklistChecklist
� Checklist is created contained highlights regarding software to be tested
TDC-SP/ Jul 2011
when receive a call, when leave the
screen and so on
Save as draft
delete saved SMS, not saved, in
inbox, sent box and so on
Delete SMS
connected to phone, not connected,
during a calling or not and so on
Receive SMS
concatenated, not concatenated, in
roaming, not roaming and so on
Send SMS
Examples of scenarios to be
explored
Checkpoint
� Note: � Clean environment when move to next checkpoint due to bug’s reproduce� Strong skill on application� Bug’s traceability: well reported
In this case before starting tests a checklist is created contained
highlights regarding software to be tested. It means main function of
the features not in details or any steps.
Let’s imagine that it is software to use SMS function from your mobile
device. There are options to send, receive, delete, save as draft and
so on, then you create a checklist using these main function of the
software, just like that:
8
‹footer› ‹#›
Click to edit Master title style
� Click to edit Master text styles
�Second level
� Third level
� Fourth level
� Fifth level
‹date/time›
Apply the method: exploratory techniques
�� Session based testingSession based testing was created to track the testing they were performing. (James and Jonathan Bach)
� Sessions are time-boxes within which the testing occurs. Each session has a charter (a little mission) and results in a session report.
� Bug’s traceability: well reported
TDC-SP/ Jul 2011
Session based testing was created by James and his brother Jonathan Bach as a need to track the testing they were performing. Since Exploratory testing is a king of a ad hoc processing, not restricted to pre-defined test steps or test procedures with mission of finding bugs without previous noticed and find them fast sessions-based testing were created.
9
‹footer› ‹#›
Click to edit Master title style
� Click to edit Master text styles
�Second level
� Third level
� Fourth level
� Fifth level
‹date/time›
Positive & Negative points
TDC-SP/ Jul 2011
++� Has a planning and structure of testing � Easy to reproduce defects found� Do not depends of any formal documentation to be created� Projects with lack documentation or often changing� Easy to maintain and change� Minimize the time
--� Might to depend on the test experience � Easily misused� Not advised to be used as unique method to assure test coverage(happy path may not be coverage)
� KT: Knowledge transfer
‹footer› ‹#›
Click to edit Master title style
� Click to edit Master text styles
�Second level
� Third level
� Fourth level
� Fifth level
‹date/time›
Case study –Ad Hoc - Experienced tester & Error guessing
TDC-SP/ Jul 2011
� Project scenario: Validate promotions from telecom operator. (E.g. Mammy’s day), requirements were created from marketing area.
� Solution:� Getting input from Product Manager (from marketing area)� Testers used past experience and creativity� Error guessing based in both points above
� Key point: Testers experienced, even they hadn’t experience to the application the testers had a lot experience in software testing.
‹footer› ‹#›
Click to edit Master title style
� Click to edit Master text styles
�Second level
� Third level
� Fourth level
� Fifth level
‹date/time›
Case Study –Checklist / Sanity tests
V EBTS Apr 2011
� Project scenario:Weekly meeting and test team had to report sanity test result. (4 companies), new build at same day.
� Solution:� Checklist and Test points� Test type: sanity� Test objective: identify high priority bugs on new build
� Key point: Experienced testers in software testing and strong skill on application.
‹footer› ‹#›
Click to edit Master title style
� Click to edit Master text styles
�Second level
� Third level
� Fourth level
� Fifth level
‹date/time›
Case Study –Session Based Testing & Charter
V EBTS Apr 2011
� Project scenario: Agile (Scrum), 1 tester, 12 developers
� Solution:� Chart to save time (not detailed steps)� Focus on learning about product� Vulnerability areas�Working together to developers� Bugs were found, value was added and the client was satisfied with the software quality.
� Key point: Experienced tester in software testing and agile methodologies.
� **assumed risk on using only this method
‹footer› ‹#›
Click to edit Master title style
� Click to edit Master text styles
�Second level
� Third level
� Fourth level
� Fifth level
‹date/time›
Conclusion
� Get escaped bugs: avoid vicious scripted test
� Fits in several project scenarios: agile, not agile, test type
� Used according to the test strategy: save time…
� Improve test’s effective when used as additional method on test phase
�� Smarter Testing!!!Smarter Testing!!!
V EBTS Apr 2011
‹footer› ‹#›
Click to edit Master title style
� Click to edit Master text styles
�Second level
� Third level
� Fourth level
� Fifth level
‹date/time›TDC-SP/ Jul 2011
Questions
‹footer› ‹#›
Click to edit Master title style
� Click to edit Master text styles
�Second level
� Third level
� Fourth level
� Fifth level
‹date/time›
‹footer› ‹#›
Click to edit Master title style
� Click to edit Master text styles
�Second level
� Third level
� Fourth level
� Fifth level
‹date/time›
Backup
V EBTS Apr 2011
‹footer› ‹#›
Click to edit Master title style
� Click to edit Master text styles
�Second level
� Third level
� Fourth level
� Fifth level
‹date/time›
References
� [1].
http://www.quardev.com/articles/slides_breaking_down_and_building_up_explorat
ory_testing_skill
� [2]. http://testobsessed.com/2006/04/19/rigorousexploratory-
testing/RigorousExploratoryTesting
� [3]. http://blog.platinumsolutions.com/node/353
� [4] Oxford Advanced Learner's Dictionary of Current English, A S Hornby,
OXFORD – Oxford University Press 2000
� Images from:
� http://www.competence.co.in/
� http://bonniesbooks.blogspot.com/2011_05_01_archive.html
� http://www.ibm.com/us/en/
V EBTS Apr 2011