Selecting the Right Automated Testing tool

Art, Science or Both? Selecting The Right Automated Testing

Tool HCMC STC MONTHLY MEETUP

NOV 2014

THAO VO

My Background

• QA Architect at KMS Technology Inc.

• Board member of HCMC Software Testing Club

• 12+ years in software development & software testing

• 6+ years in automated & performance testing

• 5+ years as a Test Lead

• Speaker and Facilitator at HCM STC Conference

• Trainer of S/W Testing at IITS, CSC, KMS and Digimate

2

Outline

3

• Test Automation Overview

• Steps to Evaluate a Tool

• Evaluation Criteria

• Automated Tool Landscape

Source: http://bit.ly/1EN7EPM

4

Test Automation Overview

Test Automation Process

5

Source: http://bit.ly/1wJ250x

1 • Assessment & Tool Evaluation

2 • Planning & Design

3 • Implementation

4 • Execution & Report

5 • Maintenance

6

Why is Tool Evaluation Important?

To be successful

7

Tool

People Approach

Source: http://bit.ly/1uttTt7

Source: http://bit.ly/1xEI5zd

Source: http://bit.ly/1utxkQq

8

Tool Evaluation Steps

Tool Evaluation Steps

9

Source: http://agilepainrelief.com/notesfromatooluser/2013/01/creating-acceptance-criteria-waiting-too-long.html

Define requirement & expectation

Define & prioritize criteria

Tool Evaluation Steps

10

Source: http://freedogpics.com/tag/german-shepherd-rescue/page/4/

Collect and list out all tools

on the market

Tool Evaluation Steps

11

Source: http://marketingdeviant.com/6-critical-factors-in-building-charisma/

Allocate suitable

resources

Tool Evaluation Steps

12

Source: http://www.careerealism.com/committed-new-years-resolutions/

Download and use

Do PoC with a sample of test cases

Tool Evaluation Steps

13

Source: http://www.returnofkings.com/38961/the-marriage-paradigm

Finalize the evaluation

14

Define Requirement & Criteria

15

Principle

Source: http://bit.ly/11fyOBJ

16

Common Criteria

1. Cost - License - Training & Support - Implementation - Maintenance

Source: http://bit.ly/1xK9qRt

Source: http://bit.ly/1sUHl4K

17

Common Criteria

2. Record & Playback

Source: http://bit.ly/1us1bbZ

18

Common Criteria

3. Cross environment (platform, browser, etc.) capability

19

Common Criteria

4. Scripting language

Source: http://bit.ly/1uzpnc6

20

Common Criteria

5. Ability to develop framework

Source: http://bit.ly/1sWrm5d

21

Common Criteria

6. Object Management Object Recognition Object Identity Tool Object Programming

Source: http://bit.ly/1fg0hoy

22

Common Criteria

7. Community & Support Document, online support Popularity Forum, question response

23

Other Criteria

• Integrated development environment & systems (version control, requirement, defect tracking)

• Scripting support: debugging, IntelliSense function and Code completion

• Error recovery

• Database support

• Speed of Execution

• Test Management capability

24

Prioritize Criteria

25

Score Ranking

Criteria Support Level Score

Outstanding 5

Good support 4

Support as normal 3

Support but need to have an expert or workaround

2

Be able to use but limitation 1

Don’t have 0

26

Expected Classification

Expectation Level Score

Must have 13

Should have 8

Nice to have 5

Rarely need 3

27

The Result

28

Automated Tool Landscape

• Unified Functional Testing (QTP), TestComplete, Telerik, SilkTest, Rational Functional Tester, Squish, eggPlant, Ranorex, TestArchitect, CodedUI, TestPartner, etc.

• Watir, Selenium, Cucumber, SpecFlow, Sahi, Robotium, Appium, MonkeyTalk, SoapUI, Sikuli, T-Plan Robot, AutoIT, etc.

29

Q&A

Profile

• Thao Vo – QA Architect at KMS Technology Inc. (http://www.kms-technology.com/); Board member of HCMC Software Testing Club (http://www.hcmc-stc.org/).

• Email: thaovo@kms-technology.com or thaovo@hcmc-stc.org or vvthao@yahoo.com

• LinkedIn: http://vn.linkedin.com/pub/thao-van-vo/24/31a/b35

30

© 2014 HCMC Software Testing Club

THANK YOU