Embed Size (px)
Citation preview
Art, Science or Both? Selecting The Right Automated Testing
Tool HCMC STC MONTHLY MEETUP
NOV 2014
THAO VO
My Background
• QA Architect at KMS Technology Inc.
• Board member of HCMC Software Testing Club
• 12+ years in software development & software testing
• 6+ years in automated & performance testing
• 5+ years as a Test Lead
• Speaker and Facilitator at HCM STC Conference
• Trainer of S/W Testing at IITS, CSC, KMS and Digimate
2
Outline
3
• Test Automation Overview
• Steps to Evaluate a Tool
• Evaluation Criteria
• Automated Tool Landscape
Source: http://bit.ly/1EN7EPM
4
Test Automation Overview
Test Automation Process
5
Source: http://bit.ly/1wJ250x
1 • Assessment & Tool Evaluation
2 • Planning & Design
3 • Implementation
4 • Execution & Report
5 • Maintenance
6
Why is Tool Evaluation Important?
To be successful
7
Tool
People Approach
Source: http://bit.ly/1uttTt7
Source: http://bit.ly/1xEI5zd
Source: http://bit.ly/1utxkQq
8
Tool Evaluation Steps
Tool Evaluation Steps
9
Source: http://agilepainrelief.com/notesfromatooluser/2013/01/creating-acceptance-criteria-waiting-too-long.html
Define requirement & expectation
Define & prioritize criteria
Tool Evaluation Steps
10
Source: http://freedogpics.com/tag/german-shepherd-rescue/page/4/
Collect and list out all tools
on the market
Tool Evaluation Steps
11
Source: http://marketingdeviant.com/6-critical-factors-in-building-charisma/
Allocate suitable
resources
Tool Evaluation Steps
12
Source: http://www.careerealism.com/committed-new-years-resolutions/
Download and use
Do PoC with a sample of test cases
Tool Evaluation Steps
13
Source: http://www.returnofkings.com/38961/the-marriage-paradigm
Finalize the evaluation
14
Define Requirement & Criteria
15
Principle
Source: http://bit.ly/11fyOBJ
16
Common Criteria
1. Cost - License - Training & Support - Implementation - Maintenance
Source: http://bit.ly/1xK9qRt
Source: http://bit.ly/1sUHl4K
17
Common Criteria
2. Record & Playback
Source: http://bit.ly/1us1bbZ
18
Common Criteria
3. Cross environment (platform, browser, etc.) capability
19
Common Criteria
4. Scripting language
Source: http://bit.ly/1uzpnc6
20
Common Criteria
5. Ability to develop framework
Source: http://bit.ly/1sWrm5d
21
Common Criteria
6. Object Management Object Recognition Object Identity Tool Object Programming
Source: http://bit.ly/1fg0hoy
22
Common Criteria
7. Community & Support Document, online support Popularity Forum, question response
23
Other Criteria
• Integrated development environment & systems (version control, requirement, defect tracking)
• Scripting support: debugging, IntelliSense function and Code completion
• Error recovery
• Database support
• Speed of Execution
• Test Management capability
24
Prioritize Criteria
25
Score Ranking
Criteria Support Level Score
Outstanding 5
Good support 4
Support as normal 3
Support but need to have an expert or workaround
2
Be able to use but limitation 1
Don’t have 0
26
Expected Classification
Expectation Level Score
Must have 13
Should have 8
Nice to have 5
Rarely need 3
27
The Result
28
Automated Tool Landscape
• Unified Functional Testing (QTP), TestComplete, Telerik, SilkTest, Rational Functional Tester, Squish, eggPlant, Ranorex, TestArchitect, CodedUI, TestPartner, etc.
• Watir, Selenium, Cucumber, SpecFlow, Sahi, Robotium, Appium, MonkeyTalk, SoapUI, Sikuli, T-Plan Robot, AutoIT, etc.
29
Q&A
Profile
• Thao Vo – QA Architect at KMS Technology Inc. (http://www.kms-technology.com/); Board member of HCMC Software Testing Club (http://www.hcmc-stc.org/).
• Email: [email protected] or [email protected] or [email protected]
• LinkedIn: http://vn.linkedin.com/pub/thao-van-vo/24/31a/b35
30
© 2014 HCMC Software Testing Club
THANK YOU
