Automation testing

Prepared By:Mona Mohammad Abd El-Rahman.

(QC Engineer at Ibtikar Technologies Co.)

What is Automation testing ?

Why Automation testing is important ?

Levels of automation testing.

Automation testing Framework.

Framework approaches.

Automation lifecycle.

How to choose automation tool ?

Types of automation testing tools.

When to automate and by whom ?

Supporting practices.

Skills to be learned.

What is Automation Testing ?

The use of special software (separate from the software being tested) to control the execution of tests and the comparison of actual outcomes with predicted outcomes.

Why Automation Testing is important ?

Why Automation Testing is important ?

Automated Software Testing Saves Time and Money.

Testing Improves Accuracy.

Increase Test Coverage.

Automation Does What Manual Testing Cannot.

Automated QA Testing Helps Developers and Testers.

Team Morale Improves.

Levels of automation testing:

• Testing at source code level with variety of input arguments to observe the output.

Code Driven

• Testing at GUI level via keystrokes, mouse clicks to UI controls etc. and observes the changes that results in user interface.

GUI Driven

Automation testing framework:

An integrated solution that sets the rule of automation of specific product. It integrates the function libraries, test data sources, object details and various reusable models.

Data File

Driver Script

Actual OutputExpected OutputCompare

Framework approaches:

Record and Playback.

Linear scripting.

Modular scripting.

Data Driven approach.

Keyword Driven approach.

Record and Playback

Capture interactions with the system and replay it.

Popular approach among commercial tool.

Benefits Problems

- Very easy and fast to create initially.- No programming skills needed.

- Single change in UI breaks all the test.

- Hard to maintain.(Plenty of separate test scripts)

- System must be ready before automation starts.(Doesn’t accept acceptance test driven development).

Linear scripting

Non-structured scripts interacts directly with SUT.

Can use any programming language.

Also produced by capture and replay tools.

test Script

1

test Script

2

SUT

Benefits Problems

- Fast to create initially.- Flexible.- Can use common scripting

languages.- No license cost.

- Creating tests requiresprogramming skill.

- One change in the system may break all the test.

- Hard to maintain.(Plenty of test scripts).

- No reuse of code.

Modular scripting

Driver script “drive” test execution.

Interaction with SUT done by functions in test library.

SUT

Driver Script

1

Driver Script

2

Test Library

Function 1

Function 2

Function 3

Benefits Problems

- Reuse of code.- Creating new tests gets faster.- Easy to maintain.

(Changes in the system will cause changes in small areas in test code)

- Drivers scripts are simple.- Novice programmers can easily edit

in drivers scripts.

- Building test library requires initial efforts.

- Requires High programming skills (to create test library).

- Test data embedded into scripts.- New tests requires new driver

scripts.

Data driven approach

Test data taken out of test scripts.

One driver script can execute multiple similar tests.

New driver script still needed for different kind of tests.

SUT

Test Library

Function 1

Function 2

Function 3

Driver Script

1

Driver Script

2

TestData 1

TestData 2

Parser

Benefits Problems

- Test libraries provide modularity.- Creating and editing existing tests are

very easy.- Maintenance responsibilities can be

divided:a) Non programmers maintain test data.B) Programmers maintain automation test code.

- Test cases are similar.- New kinds of tests requires new

test scripts.- Creating driver script requires

programming skills.- Initial effort creating parsers and

other reusable components are big.

Keyword driven approach

Not only test data but also directives (keywords) telling how to use the data taken out of the test scripts.

Keywords and test data associated with them drive test execution.

SUT

Test Library

Keyword 1

Keyword 2

Keyword 3

TestFramewor

k

TestData 1

TestData 2

TestData 3

Benefits Problems

- All same benefits as data driven testing.

- Tests can be constructed freely from keywords.

- All tests can be executed by one framework.

- Initial efforts in really big.But, there is online open source tools available.

Automation testing lifecycle :

Decision to automate

Test tool acquisition

Automated tests

introduction Process

Test Planning, Design and

Development

Execution and management

of tests

Test program review and assessment


Imagine your self the tool decision – maker and purchaser …

How will you think ?


Do you have necessary skilled resource to allocate for automation tasks?

What is your budget?

Does the tool satisfy your testing needs? Is it suitable for the project environment and technology you are using? Does it support all tools and objects used in the code?


Does the tool provide you the free trial version so that you can evaluate it before making a decision? Also does the tool have all features available in trial version?

Is the current tool version stable? Is the vendor company established with good customer support as well as online help resources and user manual?

How is the tool learning curve? Is the learning time acceptable for your goals?


Do you want automation tool for only your project needs or you are looking for a common tool for all projects in your company?

Which testing types does it support? Tool which supports maximum testing types (Unit, functional, regression etc.) is always a better choice.

Does the tool support easy interface to create and maintain test scripts?


How easy it is to provide input test data for complex or load tests?

Does it provide the powerful reporting with graphical interface? Clear and concise reports will always help you to conclude the test results quickly.

Does it integrate well with your other testing tools like project planning and test management tools.

Is the vendor provide initial training.

Types of automation testing tools:

Commercial tools Good ones tends to be expensive.

Often hard to integrate with:- Other automation tools.- Version control and continuous integration tools.

Hard or impossible to customize.

Risk of product or company discontinuation.

Ex. Quick test professional (HP), Functional tester (IBM), SilkTest, TestComplete, Testdroid … etc.

Open source tools Large variety.

(Some are great – other are not).

Normally easy to integrate with other tools.

Free.

Can be customized freely.

Can never die.

Ex. Selenium, Robotium, Jmeter, SoapUI … etc.

When to automate and by whom ?

After Development

Often by different team.

Typical in waterfall projects.

Slow feedback loop.

Collaborative automation

Automation considered to be integral part of development.

Typical in Agile projects.

Supporting practices:

Version Control

Test data and code should be stored the same way as production code.

Recommended to store with production code.

Lots of great open source alternatives available (Subversion, Github, etc.)

Continuous integration

Key to full scale automation.

Tests are run automatically when:a) New tests are added.b) Code is changed.

Can also have scheduled test runs.

Skills to be learned:

Scripting languages.

Regular expressions.

(A must when parsing textual outputs)

Xpath and CSS Selectors.

(A must when doing web testing)

SQL.

(A must with databases)

Using version control.

Feel free to contact anytime …[email protected]