Understanding of Automation Testing and Automation Framework

What is Automation Testing?Automation Testing is a process where software or an Automation Tool is used to execute test scripts in a controlled and desired manner to compare the expected result with the actual result. In general terms Automation Testing involves automating a manual process which is already in place in the testing process.

Automation Fundamental Concepts

Prepared By: Priya Trivedi

Automation Fundamental Concepts

Automation life cycle is a subset of the entire test life cycle… Automation planning can be initiated in parallel to the test planning phase… Factors to be considered in automation planning,

Stability of AUT (Application under test) No of regression cycles to be performed Compatibility of App platform with testing tools Cost benefit analysis (ROI) Availability of skilled resources

When Automation is applicable? Regression Testing Cycles are long and iterative. If the application is planned to have multiple releases / builds If it’s a long running application where in small enhancements / Bug

Fixes keeps happening Test Repeatability is required

The Benefits/Purpose of Automation testing

Speed up Test Execution: In less span of time you can run more no of test scripts which not only reduces the time but also speeds up the execution task.

Achieved Improved Test Coverage: Your coverage is more than what you could do manually.

Ensures Consistency, Accuracy and Efficiency: The tests run are consistent, accurate and efficient.

Reliability and Transparency: The tests can perform the exact same steps every time without concern about human error.

Repeatability and Reusability: Once you automate your test cases i.e. you convert the test cases into test scripts they can be run any number of times since they are stored and they can be reused at a later point of time when needed.

Return on Investment (ROI) Quality

What Should Be Automated Good candidates

Tests executed for each build Business critical tests Tests that are difficult/tedious to perform manually

Bad candidates Tests without predictable results Test that require variable input/responses from the tester Tests that perform operations in multiple environments

Automation Tools?

Tool Name Company Name Latest Version

HP QTP HP 10.5

IBM Rational Functional Tester IBM Rational 8.1.0.0

Selenium Open Source Tool 1.0.1

SilkTest Micro Focus 2009

TestComplete Automated QA 7.52

TestPartner Micro Focus 6.3

WATIR Open Source Tool 1.6.5

Introduction to Test Automation Framework

What an Automation Framework is ? How many types of Automation Framework are there ? How to select & Deploy an Automation Framework ? What are the Constituents of Automation Framework ?

Areas covered Include:

What a Test Automation Framework is ?

It is a structured combination of :

Various Assumptions for Testing Testing Concepts Testing Practices

Aim is to provide support to Automated Software Testing

Utility of Test Automation Framework

Provides an Outline of overall Test Structure Ensures Consistency of Testing Minimizes the Amount of Code for Development - thereby Less Maintenance Maximizes Reusability Reduces Exposure of Non-Technical Testers to Code Enables Test Automation using Data

How Many Types of Automation Frameworks are there ?

Data Driven Automation Framework

Keyword Driven Automation Framework

Modular Automation Framework

Hybrid Automation Framework

Generally there are 4 Types :

Data Driven Automation Framework

Advantages: Repeated use of Test Scripts with Different Inputs and Response Data

coming out of Predefined Dataset Helps in Reducing Coding for Large Test Cases Ease of Testing of Time-Consuming & Complex Test Cases

Data Driven Automation Framework

Data-driven testing is a framework where test input and output values are read from data files (datapools, ODBC sources, csv files, Excel files, DAO objects, ADO objects, and such) and are loaded into variables in captured or manually coded scripts.

In this framework, variables are used for both input values and output verification values.

The test scripts contain all the coded information regarding reading of the data files, navigation through the application & test status logs etc.

This has a similarity with table-driven testing in a way that our test case is encapsulated in the data file as against in the test script. The test script just remains a "driver," or a means of delivery for the data. Here in data-driven testing, data files contain just the test data only.

Continue………..

Merits of data-driven testing Scripts may be developed while application development is still in progress Utilizing a modular design, and using files or records to both input and verify data, reduces

redundancy and duplication of effort in creating automated test scripts If functionality changes, only the specific "Business Function" script needs to be updated Data input/output and expected results are stored as easily maintainable text records. Functions return "TRUE" or "FALSE" values to the calling script, rather than aborting, allowing

for more effective error handling, and increasing the robustness of the test scripts. This, along with a well-designed "recovery" routine, enables "unattended" execution of test scripts.

Demerits of data-driven testing Requires proficiency in the Scripting language used by the tool (technical personnel) Multiple data-files are required for each Test Case. There may be any number of data-inputs

and verifications required, depending on how many different screens are accessed. This usually requires data-files to be kept in separate directories by Test Case

Tester must not only maintain the Detail Test Plan with specific data, but must also re-enter this data in the various required data-files

If a simple "text editor" such as Notepad is used to create and maintain the data-files, careful attention must be paid to the format required by the scripts/functions that process the files, or script-processing errors will occur due to data-file format and/or content being incorrect

Keyword Driven Automation Framework

Attributes: As the Name suggests, it enables Keyword Driven Testing or Table Driven

Testing Data & Keyword Tables being Independent of the Automation Tool Enables Documentation of the Functionality of the Application under Test

(AUT) in A Tabular Format

Keyword-driven testing is a Software testing technique that separates much of the programming work from the actual test steps so that the test steps can be developed earlier and can often be maintained with only minor updates, even when the application or testing needs change significantly.

The Keyword Driven framework consists of the basic components given below1. Control File2. Test Case File3. Startup Script4. Driver Script5. Utility Script

Keyword Driven Automation Framework

1. Control File a. Consists details of all the Test scenarios to be automated b. User will be able to select a specific scenario to execute based on turning on or off a

flag in the Control File c. Control File is in the form of an excel worksheet and contains columns for Scenario ID,

Execute (Y/N), Object Repository Path, Test Case File Path2. Test Case File

a. Contains the detailed steps to be carried out for the execution of a test case b. It is also in the form of an excel sheet and contains columns for Keyword, Object Name,

Parameter3. Startup Script

a. The Startup script is utilized for the initialization and reads the control files b. It then calls the driver script to execute all the scenarios marked for execution in the

control file4. Driver Script

a. It Reads the Test Case files. Checks the keywords and calls the appropriate utility script functions based on specific keyword

b. Error Handling is taken care of in the driver script.5. Utility Scripts

a. Perform generic tasks that can be used across applications. It should not be application dependent

Keyword Driven Automation Framework

Advantage of Framework. The main advantage of this framework is the low cost for maintenance. If

there is change to any test case then only the Test Case File needs to be updated and the Driver Script and Startup script will remain the same.

No need to update the scripts in case of changes to the applicationDisadvantages of keyword driven testing:

Development of "customized" (Application-Specific) Functions and Utilities requires proficiency in the tool’s Scripting language. (Note that this is also true for any method)

If application requires more than a few "customized" Utilities, this will require the tester to learn a number of "Key Words" and special formats. This can be time-consuming, and may have an initial impact on Test Plan Development. Once the testers get used to this, however, the time required to produce a test case is greatly improved.

Keyword Driven Automation Framework

Modular Automation Framework

Attributes: As the Name suggests, it enables Keyword Driven Testing or Table Driven

Testing Data & Keyword Tables being Independent of the Automation Tool Enables Documentation of the Functionality of the Application under Test

(AUT) in A Tabular Format

Test Script Modularity Framework:

Enables creation of Small, Independent Scripts representing Modules & Functions of the Application under Test (AUT). The use of this framework will yield a higher degree of modularization and add to the overall maintainability of the test scripts

Test Library Architecture Framework:

Enables creation of Library Files representing Modules & Functions of the Application under Test (AUT). This framework requires the creation of library files (SQABasic libraries, APIs, DLLs, and such) that represent modules, sections, and functions of the application-under-test. These library files are then called directly from the test case script. Much like script modularization this framework also yields a high degree of modularization and adds to the overall maintainability of the tests.

Types of Modular Frameworks :

Hybrid Automation Framework

Attributes: It is the Most Popularly Implemented Framework It is a Combination of the Three Types of Frameworks described before It has an Ability of Evolving Itself Over a Passage of Time and Over Many

Projects

This hybrid test automation framework is what most frameworks evolve into over time and multiple projects.

The most successful automation frameworks generally accommodate both Keyword-Driven testing as well as Data-Driven scripts. This allows data driven scripts to take advantage of the powerful libraries and utilities that usually accompany a keyword driven architecture.

The framework utilities can make the data driven scripts more compact and less prone to failure than they otherwise would have been.

The utilities can also facilitate the gradual and manageable conversion of existing scripts to keyword driven equivalents when and where that appears desirable.

On the other hand, the framework can use scripts to perform some tasks that might be too difficult to re-implement in a pure keyword driven approach, or where the keyword driven capabilities are not yet in place.

Hybrid Automation Framework

How to Implement Test Automation Framework Methodology?

10 Steps to Implement Automation Framework Approach

Steps to Implement Automation Framework Approach

1) Identification of the Scope of Testing: Company Oriented, Product Oriented, Project Oriented

2) Identification of the Needs of Testing: Identify Types of testing e.g. FT, Web Services etc. and application / modules to be tested

3) Identification of the Requirements of Testing: Find out the Nature of Requirements, Identification of type of actions for each requirement & identification of High Priority Requirements

4) Evaluation of the Test Automation Tool: Preparation of Evaluation Checklist, Identification of the Candidate Tools vailable, Sample Run, Rate & Select the Tool, Implementation & Training

5) Identification of the Actions to be automated: Actions, Validations & Requirements supported by the Tool

Steps to Implement Automation Framework Approach

6) Design of the Test Automation Framework: Framework Guidelines, Validations, Actions Involved, Systems Involved, Tool Extensibility Support, Customs Messages & UML Documentation

7) Design of the Input Data Bank: Identification of Types of Input file, Categorization & Design of File Prototypes

8) Development of the Automation Framework: Development of Script based upon Framework Design, Driver Scripts, Worker Scripts, Record / Playback, Screen / Window / Transaction, Action / Keyword & Data Driven

9) Population of Input Data Bank: Different Types of Data Input, Population of Data from Different Data Sources, Manual Input of Data and Parent – Child Data Hierarchy

10) Configuration of the Schedulers: Identify Scheduler Requirements & Configure the Schedulers

Benefits of Automation Framework Approach

Significant Reduction in Testing Cycle Time Comprehensive Coverage against Requirements Use of a "Common Standard" across the Organization / Product Team / Project

Team Generation of Reusable Test Scripts ( Utility Functions) Systematic Maintenance of Automation Scripts Data Pooling