IBM Confidential

Centralized Test Automation using Hybrid Model

1

Bharathi Easwaran

AgendaAgenda

2

Automation Decision Making Challenges of Automation Benefits of automation Components of Automation testing Reasons for Automation failure Successful Automation Testing Automation Decision making Process Automation Guidelines for Testers Identify Test Cases to automate Hybrid Framework Approach Elements of a Framework

Factors Decides Automation TestingFactors Decides Automation Testing

3

Test

Au

tom

ati

on

Test

Au

tom

ati

on

Challenges in Test AutomationChallenges in Test Automation

4

Benefits of AutomationBenefits of Automation

AS ALPS | Apr 10, 2023 | IBM Confidential 5

Components of Automation TestingComponents of Automation Testing

6

Automation Frame Work :

Automation Framework is a process of Structuring and

Organizing Scripts to increase the level of reusability

and readability of scripts.

Tool :

Is a mechanism used to simplify and speed the effort

required for testing.

Regression Suite :

A collection of Regression scenarios in the form of

automated build

7

While Talking about automation decision making, we have to list out the reasons for automation failure. This activity will help you to create a quality in automation testing.

Automation Failure Leads to..

Effective Test Execution can’t be continued Unable to achieve the defined ROI Decrease the defect finding Rate Dissatisfaction with Client

Reasons Why Automation Testing Might Fail ?

Successful Successful AutomationAutomation TestingTesting

8

9

Test Automation at ProjectTest Automation at Project

Requirement Analysis Tool Compatibility

• Products Functionality Study

• Record a Script with the available tool for core functional flow as per business

• Run the recorded scripts multiple times with different test base

• Validate the compatibility between tool and the application

• 80% - 90% of the test objects should be able to identified by the tool

Automation Suite

creation

Delivery

Select Framework • Setup the

Automation Environment

• Create automation suites

• Debug the suites

• Validate the suites

• Automation framework

• Regression Suites

• User Manual • Documents

for Process

FrameworkCompatibility

• Project duration• Project Size• Test data

Management• Business

requirement • Automation

Budget

Create a POC

Select Framework

Automation Guidelines for TestersAutomation Guidelines for Testers

10

Modularize test scripts for multiple execution combinations. Identify and abstract common functions used across multiple

test scripts. Structure scripts with minimal dependencies to ensure scripts

can run unattended even when multiple failures occur.

Decouple complex business function testing from navigation, limit-testing, and other simple verification and validation activities.

Abstract and decouple test data from the test scripts.

follow the consistency in Process. Maintain Logs and Documents. Analyze and evaluate the Failures.

Guidelines to Identify Test Cases to automatedGuidelines to Identify Test Cases to automated

11

Validation Yes/No

Accurate Expected Results

Clear Functional Requirement

Consistent Functional Behavior

Dependence on other systems

Core functional component

Test Data Availability and Connectivity

Each regression test case/scenario should be validated based on the Checklist.

IBM Confidential

Centralized Test Automation usingCentralized Test Automation usingHybrid Model Hybrid Model

12

13

Hybrid-Automation FrameworkAutomation Framework :A framework can be a wrapper around complex internal architecture which makes end user to interact with system easily. It also define guidelines and set standards for all phases of ADLC(automation development life cycle).

Hybrid-Framework :The hybrid Test automation framework is what most frameworks evolve into over time and multiple projects. The most successful automation frameworks generally accommodate both Keyword driven as well as Data driven This allows data driven scripts to take advantage of the powerful libraries and utilities that usually accompany a keyword driven architecture. The framework utilities can make the data driven scripts more compact and less prone to failure than they otherwise would have been.

Driver Script

Test Scripts 1

Test Scripts 2

Test Scripts 3

Function Library Data

Hybrid Automation Framework - Diagram

Centralized Automation FrameworkCentralized Automation Framework

14

Automation Script encapsulation :

Function Libraries, Scripts all are encapsulated components, Users will be able to access only the Scripts related to their testing user get prevented from accessing the entire framework scripts.

Systematic Approach: Defines a standard procedures, processes, best practices and guidelines for test automation implementation. A tester can’t create a script on this own style or way for the project test automation.

Quality of automation : The Framework will be evaluated in different levels to object quality in automation.

Framework Design: Hybrid model has been used to design the centralized automation framework.

15

Project 1

Technology -Web

Project 2

Technology Web

New Project 3

with

New Technology (SAP)

Framework need to be upgraded to the New SAP Technology.

Web Function Library

VB Function Library

Java Function Library

Recovery System

General Function Library

Environment

Centralized Framework

Start Up Scripts

Framework ready to implement test automation

Framework ready to implement test automation

Note: Assume that web technology is implemented in framework and “sap” is a new technology that’s need to be

implemented. So Project 1 and project2 can be implemented immediately and project 3 needs an up-gradation in the framework.

Example diagram – Extendable to New technology using Centralized Framework

Features of Centralized Automation FrameworkFeatures of Centralized Automation Framework

Design of the Framework is independent of “Functionality & Technology”

Minimum level of maintenance and framework change requirement

Framework should provide user friendly Regression Suite

Systematic Approach in automation implementation

On the shelf Product – Ready for the current market requirement.

16

Framework Benefits ComparisonFramework Benefits Comparison

Benefits Customized Automation Scripts

Automation Framework(Generic)

Centralized Automation Framework

Cost of Implementation

Low Medium High

Extendable to New technology

No No Yes

Functional Independence

No Yes Yes

Technology independence

No No Yes

Script Maintenance Required

High Medium Low

User Friendly No Yes Yes

Encapsulated Scripts

No Yes Yes

Reliable and Stable No Yes Yes

ROI Low Medium High

Reusability Low Limited High

17

18

Architecture of Centralized Automation Framework Architecture of Centralized Automation Framework

Application

Driver Sheet

Main DataSheet

ResultSheet

Sub Script

IF Flag Y/N

No

Yes

Perform Validation

Driver Script

IF Flag Y/N

Yes

No

Main Script

IF Flag Y/N

No

Yes

Perform Validation

SubSuite

Yes

No

Sub Suite Folder

Regression Suite

Automation Framework Start

Application

Driver Sheet

Main DataSheet

ResultSheet

Sub Script

IF Flag Y/N

No

Yes

Perform Validation

Driver Script

IF Flag Y/N

Yes

No

Main Script

IF Flag Y/N

No

Yes

Perform Validation

SubSuite

Yes

No

Sub Suite Folder

Regression Suite

Automation Framework Start

19

Framework - Work Flow DesignFramework - Work Flow Design

19

Web

App Type

SAP

Object Class

Web Button

Web Edit Box

Operation

Checkpoint

Function

Operation Type

Std.CheckPoint

TxtCheckPoint

User Defined Function Call

Framework design will be independent of functionality and technology

Framework need to be upgraded for new technology and Automation Suites to be upgraded for new functionally, but design remains constant

Application type, Object Class, Object Name, Operation and Operation Type are the deciding factor of the automation flow of execution

Your Data Drives the Flow not the framework

Framework will process the inputs and perform the operation in the application, update the result back to the suite,

If your data is incorrect then the suite will fail

20

High Level Framework Design High Level Framework Design

Driver Sheet :

The Starting Point of Framework and will contain all the information about Regression Suite( like (Path, Suite Name, Module Name and Flag, etc )

Driver Script :

Will check the Flag status of All the Regression Suite in the Driver sheet

Flag = Yes ( Suite need to be executed)

Flag = No ( Suite need not to execute)

Main Script :

It will locate the regression suite and get data from the Main datasheet

It will read all the rows in the datasheet, 1 to n rows,

before process a row, it will check the Flag Status( Yes/No)

Main Datasheet :

Each Row will contain technical data, test data and a flag

One line of Script will be converted into one row in Datasheet

Sub Script and Sub Data Sheet :

We can call Reusable Suites inside your Reg. Suites

Function Library : Set of Reusable Functions

ResultSheet : Customized results for user s better understanding

Driver Script Driver Sheet

Main Script Main Data Sheet

Sub Script Sub Data Sheet

n

n

Frame work Automation Suite

1. Driver Sheet2. Driver Script3. Main Script 4. Sub Script5. Function Library6. Recovery

scenarios

1. Main Data Sheet

2. Object Repository

3. Reusable Suite

4. Result Sheet

21

Steps to Create the Automation suiteSteps to Create the Automation suite

Step 1: Create a Folder for your Automation Suite (Ex. “SupportRequest”)

Step 2: Record the Flow which you need to automate and Save the Flow Script

Step 3: Save the Local Object Repository as Shared Obj. Repository in the Folder (Automation Suite Folder)

Step 4: Use the “Data Generator” to Convert your Flow into Datasheet and save the same In the automation Folder

Step 5: Save a Result Sheet Template

Create a Folder for Regression Suite

Record the scenarios and Save

Save as the Object Repository as “ObjRep.tsr” in Regression Suite Folder

Use “Data Generator” to Convert the Recorded

scripts into Datasheet. Save as the Datasheet as “DataSheet.xls” in

Regression Suite Folder

Insert a Result template in Regression Folder

“Results.xls”

Ready for Execution

22

Driver SheetDriver SheetApplication ModuleName Path Dependencies Flag

CRM Login C:\\Shell Automation\\CRM\\Login Yes

CRM Transactio C:\\Shell Automation\\CRM\\Transaction Yes

CRM Opportunity C:\\Shell Automation\\CRM\\Opportunity NO

SAP SupportRequest C:\\Shell Automation\\SAP\\SupportRequest Yes

Driver sheet is a Starting point of the framework architecture

Driver Sheet is an repository to store all the automation suite informations

Path and Flag are mandatory fields

We can free to add more fields for user reference. ( Like tester name, comments )

23

Flow ScriptFlow Script

23

Flow Script is the script which you recorded to automate

Later Data Generator will covert your flow script into data in a form which the framework can

understand.

24

Main Data SheetMain Data Sheet

Main Data Sheet will have a huge amount of data, they are Flow Data, technical and test data, Test case information's

Flow data : Windows Name, Object Name and Object Class etc.,

Test Data : Account Id, Customer Id, Login Details and URL etc.,

Technical Data : Operation, Operation Type, Parameters, Flag & Comments

Test case Info(Optional ) : Test case Id, Test Description etc.,

Your Flow Scripts will be converted into Main Data and the Same will be saved in Excel

Single Line of Flow Script will be converted into data structure in Main Data Sheet

Main Data Sheet is in a table form, so easy for the end user to understand the flow and execute/ maintain the same,

Main Data Sheet is like a “keyword view” of your flow script

25

Main Data SheetMain Data Sheet

IBM Confidential

Elements of a FrameworkElements of a Framework

26

Common ComponentsCommon Components

Below are the components of theAutomation Framework:

Data Tables Environment Libraries Object repository Results Runtime Scripts Recovery scenario Documents Tools

27

Tools

Data Tables EnvironmentObject

Repository

Scripts

LibrariesRecovery scenario

• Runtime• Documents

Framework

Results

Data TablesData Tables

Data table leads to Parameterization Repository for Test Data Designed for User Interaction Data table can be an Excel Document, Ms-Access, Database Framework will Design the Data table Structure

and mapping

28

Username Password

User1 pwd1

User2 pwd2

User3 pwd3

EnvironmentEnvironment

29

LibrariesLibraries

Time and resource can be saved using reusable functions.

Decomposing the scripts into functions, will increate the level of script Understandably.

Creating functions will reduce the dependencies within the scripts.

Scripts maintenance cost will considerably will come down.

Support Future requirements.

.30

ApplicationDriver Scripts

Function Library

Function library

Collection of Reusable operations in the form of Predefined Structure

Object Repositories / Object MapsObject Repositories / Object Maps

The Storage band for Application Objects

It Provides the interaction between scripts and

application

The object repository is shared and module-specific.

While Execution the Object repository will help the

script to Identify the Objects in the application

Repository will have list of objects along with

configured properties and their values

Object Repository will have enough no.of properties

to uniquely identify an Object

Shared object repository should have the extn :

ABC.tsr (Test Shared Repository)

Module-specific repository should have a extn :

ABV.mtr (Module Test Repository)

31

Object Repository

Object1

Property Name

Property Name

Property Value

Property Value

Object2

Property Name

Property Name

Property Value

Property Value

Structure of Object RepositoryStructure of Object Repository

Tools, Scripts, Recovery scenario & ResultsTools, Scripts, Recovery scenario & ResultsTools : This folder contains macro files. Using these files will reduce time for writing scripts, converting files, and shortcut to generate code. Scripts : This folder contains local test scripts; this folder will be used while development and maintenance stage. Once script gets delivered it will upload to centralize location Test Director or Quality Centre.Recovery scenario : This folder can be used as recovery scenario function library file. Also, through QTP particular scenario, converted file is kept under this folder and associates the test script.Results : The current execution status will be updated after every run. All the Script failures, test status, recovery actions will be logged in Result.

32

QTP Settings

Library Files

Recovery Scenario

s

Object Configurat

ion

Environment

Open Script

Test Script

Run Script

Test Script

Script Failure

Test ScriptResume Executio

n

Execution Completed

Update Result

Close QTP

Thank YouThank You

AS ALPS | Apr 10, 2023 | IBM Confidential 33